Apache Hadoop, the open source software framework at the heart of big data, is a batch computing engine. It is not well-suited to the online, interactive data processing required for truly real-time data insights. Or is it? Doug Cutting, creator of Hadoop and founder of the Apache Hadoop Project (and chief architect at Cloudera) says he believes Hadoop has a future beyond batch.
“I think batch has its place,” Cutting says. “If you’re moving bulk amounts of data and you need to really analyze everything, that’s not about interactive. But the combination of batch and online computation is what I think people will really appreciate.”
“I really see Hadoop becoming the kernel of the mainstream data processing system that businesses will be using,” he adds.