Hortonworks Expands Streaming Data Capabilities With New DataFlow Release

Big data platform vendor Hortonworks launched a new release of its DataFlow software this week that the company said extends its analytics capabilities for Internet of Things and other streaming data applications.

Hortonworks, at this week's DataWorks Summit in San Jose, also struck a deal with IBM under which the companies will integrate the Hortonworks Data Platform (HDP) with IBM's Data Science Experience and Big SQL software to create big data management and analysis solutions.

IBM is adopting HDP as its distribution of Hadoop and will integrate it with the IBM Data Science Experience and IBM Machine Learning platforms. The company will also migrate users of the Hadoop-based IBM Big Insights product to HDP.

[Related: 2017 Big Data 100: 15 Coolest Big Data Platform Vendors]

Hortonworks, which already had a long-established alliance with IBM, will resell the IBM Data Science Experience with its Hadoop-based HDP system and adopt the IBM software as its strategic data science platform.

The two companies also will develop new system bundles that integrate HDP with IBM Big SQL, IBM's SQL engine for Hadoop.

"The combination of IBM's data science and Hortonworks' open and connected data platforms will benefit not only our respective clients, but also the Apache open source community because of our combined investment and collaboration," said Rob Bearden, Hortonworks CEO, in a statement.

IBM and Hortonworks are also founding members of the Open Data Platform Initiative, a consortium of companies that is working to define and promote open-source technologies to improve compatibility among big data platforms.

The new Hortonworks DataFlow (HDF) 3.0 provides new capabilities designed to make it easier for partners, ISVs and customers to develop streaming analytical applications – an increasingly important requirement for real-time analysis and Internet of Things applications.

"With HDF 3.0, we are improving our customers' experience by simplifying how they create and deploy streaming analytics applications to deliver real-time analytics," said Hortonworks CTO Scott Gnau in an interview with CRN.

Building streaming analytical applications is generally a complex and time-consuming task. New in HDF 3.0 is Streaming Analytics Manager, a toolset that allows application developers, business analysts and administrators to design, build, test and deploy streaming applications on HDF without coding.

The 3.0 edition, which was generally available starting this week, also includes a new repository of shared schemas that interact with data streaming engines including Apache Kafka, Apache Storm and Apache NiFi, providing improved data governance and operational efficiencies.

Hortonworks also announced a new support subscription service, Hortonworks Flex Support, which covers HDP deployments that are on-premise and/or in the cloud. Hortonworks is currently not selling the Flex Support through the channel, however.