{"id":642985,"date":"2013-02-19T13:59:40","date_gmt":"2013-02-19T18:59:40","guid":{"rendered":"http:\/\/gigaom.com\/?p=611709"},"modified":"2013-02-19T13:59:40","modified_gmt":"2013-02-19T18:59:40","slug":"citusdb-today-sql-on-hadoop-tomorrow-the-world","status":"publish","type":"post","link":"https:\/\/mereja.media\/index\/642985","title":{"rendered":"CitusDB: Today, SQL on Hadoop. Tomorrow, the world!"},"content":{"rendered":"

Database startup Citus Data<\/a> on Tuesday joined those trying to enable fast SQL queries on Hadoop data, but it has much larger goals. It thinks it can be the only analytic database that anyone needs, able to query data wherever it\u2019s stored across a company\u2019s environment \u2014 in relational databases, Hadoop, MongoDB, Amazon S3 and elsewhere.<\/p>\n

Big data has opened companies\u2019 eyes to the importance of analytics and alternative data stores, but combining the two often means learning new languages, using multiple tools and probably sacrificing the performance they\u2019re used to from analytic platforms.<\/p>\n

Citus Data\u2019s flagship product, called CitusDB, is actually built atop PostgreSQL and its first iteration was designed for Google Dremel-like scale and speed<\/a> on relational data. Thanks to a feature called \u201cforeign data wrappers,\u201d though, it\u2019s able to run SQL on numerous data types (e.g., CSV, log and JSON files) that don\u2019t comport with how Postgres formats data natively. So, while CitusDB now officially supports the Hadoop Distributed File System in addition to Postgres, it is by no means limited to them.<\/p>\n

Matt Ocko, managing partner at Data Collective<\/a> and one of Citus Data\u2019s early investors, says the database can technically support any data source with an ODBC driver, and even could query something like log files straight from a data store.\u00a0In fact, Citus is working on extending its support to MongoDB \u2014 a capability that\u2019s in beta right now. Ocko is also particularly impressed with CitusDB\u2019s ability to act like a fabric connecting all these data sources rather than making users query each independently and then manually join the data. He cited a demonstration in which CitusDB carried out a query that required executing a join across Postgres and Hadoop.<\/p>\n

The other big thing about CitusDB is that it\u2019s not just flexible but fast, too. Ocko said CitusDB has outperformed Oracle\u2019s vaunted Exadata machine on a TPC-H benchmark test with data stored primarily on hard disk. That Postgres-Hadoop query he referenced completed in just a few seconds while running on the Amazon EC2 cloud.<\/p>\n

CitusDB is so fast, Citus Co-founder Umur Cubukcu told me, because of how it\u2019s architected. It moves the computation to where the data is rather than trying to move data across the network, and it has some impressive load-balancing the resource-management abilities baked in. If, for example, it needs data housed on a slow-running node in order to complete a task, the software will look for that data elsewhere rather than just wait for the congested resource to free up.<\/p>\n

In the case of Hadoop, MapReduce brings the computation to the data, too, but every job requires a scan over the entire dataset. This is why early SQL-on-Hadoop tools such as Hive are still relatively slow<\/a>. Citus software engineer Carl Steinbach, who came to the company from Cloudera, said CitusDB is between 3 and 20 times faster than Hive depending on the query type.<\/p>\n

It\u2019s actually much faster for short queries that might be typical in an interactive environment, but he acknowledged those aren\u2019t really what Hive was designed to do.<\/p>\n

\"Citus_Hadoop_Architecture\"<\/a><\/p>\n

Rather, CitusDB\u2019s real competition is the spate of SQL-on-Hadoop projects, products and startups of which it\u2019s now a part. We\u2019ll have a whole session dedicated to this topic at Structure: Data<\/a> next month, and there isn\u2019t enough room for everything on the market right now \u2014 Aster Data<\/a>, Platfora<\/a>, Cloudera (with Impala<\/a>), Apache Drill<\/a>, Drawn to Scale<\/a> and Hadapt<\/a>, to name several.<\/p>\n

These are impressive technologies (at least in theory where they\u2019re still under development), and Citus would be remiss to ignore them. But, aside from the ability to query multiple data sources, the company has something the others don\u2019t, Cubukcu said: It has the Postgres community and all the features they\u2019ve built into that database already. Things like connectors, authentication, full-text search and PostGIS for geospatial data that go beyond just running fast queries.<\/p>\n

\u201cWhen you\u2019re talking about an enterprise-class database,\u201d Steinbach said, \u201cyou\u2019re talking about more than a query execution engine.\u201d<\/p>\n

\"\"<\/p>\n

<\/a><\/p>\n

Related research and analysis from GigaOM Pro:<\/strong>
Subscriber content. Sign up for a free trial<\/a>.<\/p>\n