The 10 Coolest Big Data Products Of 2015

Big Data: Hot Market, Cool Products

The big data technology market remains one of the fastest growing segments of the IT industry. In November, market research firm IDC said the market for big data-related infrastructure, software and services will grow at a compound annual growth rate of 23.1 percent through 2019, with spending reaching $48.6 billion in 2019. Specifically, sales of big data software are expected to grow at a CAGR of 26.2 percent during that span.

So it's no wonder that new big data products continue to hit the market at a rapid pace. Here's a look at 10 that particularly caught our attention this year.

Apache Spark

Aside from the Hadoop platform, there's probably no single technology that's changing the big data game more than Apache Spark. The powerful in-memory, data processing engine turbo-charges big data systems like Hadoop and enables real-time business analytical applications using streaming data that weren't possible before.

Spark is open-source software and is also available commercially from Databricks, the company started by Spark's original developers. Spark 1.5, which debuted this fall, sports a number of changes that improve its performance, usability and operational stability.

BlueData EPIC 2.0

BlueData Software develops the BlueData EPIC platform that businesses use to deploy big data applications and infrastructure. EPIC utilizes virtualization technology to make it easier and more cost-effective for businesses to enable Hadoop-as-a-Service in an on-premise deployment model.

The 2.0 release, which debuted in September, leverages increasingly popular Docker containers to simplify Hadoop clusters. It also offers new Apache Spark functionality and provides an enhanced App Store with access to other big data software and analytical tools.

Also in September, BlueData allied itself with Intel in a project to develop BlueData's software to work with Intel's data center architecture based on the Xeon processor technology.

BlueTalon Policy Engine

BlueTalon's software provides data-centric security across Hadoop, SQL and other big data environments. The BlueTalon Policy Engine specifically addresses Hadoop's security shortcomings by helping IT manage data access when departments and even individuals require different levels of authorization to view the same data.

In July, BlueTalon struck a deal with Microsoft to integrate the Policy Engine with Microsoft's Azure HDInsight, Microsoft's Hadoop deployment on its Azure cloud platform. In August, Hadoop software developer Cloudera certified BlueTalon for its distribution of Hadoop.

Information Builders WebFocus Express

WebFocus is Information Builders' flagship business analytics software that's geared toward enterprise companies. This year, IBI debuted WebFocus Express, a version of the software that's specifically designed for small and mid-size businesses and is being sold exclusively through the channel.

IBI initially developed WebFocus Express with IBM, which has been selling it as DB2 Web Query. Offering the software as an SMB product is a key component of IBI's effort to expand its presence in the channel.

Looker's Powered by Looker

Looker markets a software-as-a-service business analytics platform the company says puts actionable intelligence in the hands of the employees who need it most.

This year, the company launched Powered by Looker, a service that allows businesses, ISVs and solution providers to embed the Looker analytics capabilities within any application, website or portal. That gives partners the ability to quickly deploy data analytics tools to their own customers.

MongoDB 3.2

MongoDB is one of a wave of next-generation NoSQL databases to hit the market in recent years that are positioned as alternatives to traditional relational database software and better able to handle huge volumes of data. Others include DataStax and Couchbase: We picked Couchbase N1QL as one of the coolest big data products in the first half of this year.

With the 3.2 edition, just unveiled last month, MongoDB took a major step forward in positioning itself for enterprise-class applications. The release incorporates a data storage engine that encrypts data both "in flight" and "at rest" for security – a necessity in some markets such as financial services, health care, retail and government. The release also includes an in-memory storage engine that makes real-time, high-throughput applications like fraud detection and online retail recommendation engines possible.

Pepperdata Chargeback Reporting Tools

Managing big data systems, from both technical and business perspectives, isn't easy. Pepperdata develops tools used for managing and optimizing Hadoop clusters in real time – a key requirement for deriving business value from the system

This year, Pepperdata added charge-back functionality to its software that makes it easier to measure and allocate costs associated with workloads across distributed Hadoop systems. That seemingly mundane capability should accelerate more widespread adoption of Hadoop for general business processes.

Qlik Sense Enterprise 2.0

Qlik is one of the leading vendors in data visualization with its QlikView "guided analytics" software, which provides users with prepared analytical applications. But late last year, the company debuted Qlik Sense, a more powerful tool that information workers can use to create their own interactive data visualization applications.

This year, the vendor debuted Qlik Sense Enterprise 2.0, a major update of the software with a raft of new capabilities. For example, the new Smart Data Load visual data-profiling function pulls data from disparate sources into a single analytical application. The new Qlik DataMarket tool provides a way to pull information from external syndicated data sources into Qlik Sense and combine it with internal operational data and files for analysis.

Snowflake Elastic Data Warehouse

Building an on-premise data warehouse remains a complex, costly endeavor. Startup Snowflake, founded in 2012 and officially launched in 2014, was founded to change all that by developing a data warehouse service based entirely in the cloud.

That plan became a reality in June with the general availability of the Snowflake Elastic Data Warehouse, a SQL-based data warehouse delivered as software-as-a-service in the Amazon Web Services cloud. The service offers native support for a diverse range of data types and works with structured and semi-structured data, the latter including increasingly analyzed machine data. It also boasts enterprise-class high-availability, disaster recovery and security.

Splice Machine 2.0

Getting information – and value — out of Hadoop is notoriously difficult. That has led to an explosion of startups offering different technologies and approaches for working with Hadoop.

High on that list is startup Splice Machine, which in May launched its SQL-on-Hadoop relational database software that's designed to scale real-time applications and operational analytics using commodity hardware without application rewrites.

The company just launched the beta of Splice Machine 2.0, which supports the Apache Spark in-memory processing engine (see slide No. 2). The added processing power provides the Splice Machine database with a significant power boost that allows it to simultaneously perform transaction processing and business analytics tasks. That's a major advantage as the line between OLTP and OLAP applications blurs.