When Do We Get a Universal Streaming Connector?
Wouldn't it be nice to be able to drag and drop Big Data components and sources together? Shouldn't we have a "Universal Streaming Connector" by now -- similar to USBTM but for data streaming?
Kafka would be the leading candidate. In the world of streaming, Kafka is the first interface provided. But so many tools outside the streaming world lack Kafka connectors today: visualization tools, ETL tools, and many databases. VoltDB is the standout for providing a Kafka connector. There may be others. For HDFS, there is Camus, but it's far from drag-and-drop, and even in terms of writing code is challenging to set up.
Now, of course, drag-and-drop won't address the difficult 20% of problem (or maybe not even 50%), but it'll address a lot. And of course performance tuning is all but out of the question with drag and drop. But it's been three years since Kafka was open sourced. How many more years do we have to wait?
It was 20 years between when RDBMs went mainstream and JDBC was released. I hope we don't have to wait that long.