Technology vendors have backed a community-developed software platform as the critical piece to bring big data to the masses.
EMC and Intel have joined IBM and Red Hat in releasing their own flavours of an open-source software that manages how hardware in data centres accesses and processes information. The Hadoop open-source software by the Apache Software Foundation was released in October. It helps extract business insights from huge amounts of unstructured data, a trend commonly referred to as big data.
While it is free to download, its complexity has meant only large players, such as Google, Amazon, and Facebook, had the manpower, scale and incentive to customise it.
The new commercial releases are designed to make it easier, faster and safer to deploy. The vendors claim the cheaper technology will democratise big data.
This is Intel's first declaration of its software ambitions, according to Singapore-based Forrester analyst Dane Anderson, who said a standardised platform could unleash a new wave of data analytics innovation - like the iTunes App Store did for mobile apps.
''The risk for big data is that
it becomes over-hyped when tech vendors just use it as a means to sell more of their gear,'' Anderson said.
''That isn't what big data is all about, it's about the services that allow you to extract business or social value. This is the building block for that to happen.''
Famous for the chips inside personal computers, this time Intel has transcended the physical barrier to compete in the virtual world.
''The opportunity cost to not do this is that the evolution will be a lot slower, and the standards won't be established early enough,'' said Jason Fedder, Intel's general manager for Asia of data-centre products. ''Intel's whole business model depends on open standards - it needs to make hundreds of millions of anything to make money - so it needs to accelerate the take up of those standards.''
In 2004, EMC breached the software threshold when it acquired VMware, whose software generates additional server computing power, and became a preferred virtualisation platform.
The companies believe there is a similar opportunity in the big data market. For their troubles, they hope they will sell more of their Hadoop-friendly products.
Intel's announcement marks its first major foray into software. When asked whether it hoped to sell more Xeon processors from being the biggest contributor to the open-source project, Intel's mission critical computing director, Patrick Buddenbaum, said ''absolutely''.
The challenge for Intel, which failed to capitalise on the smartphone revolution, was to continue to drive demand for the hardware it made, he said.
Mahesh Sharma travelled to Singapore as a guest of Intel.
- FFX Aus