
By Jnan Dash | Article Rating: |
|
October 24, 2015 04:00 AM EDT | Reads: |
3,323 |
Last June IBM made a serious commitment to the future of Apache Spark with a series of initiatives:
- It will offer Apache Spark as a service on Bluemix (Bluemix is an implementation of IBM's Open Cloud Architecture based on Cloud Foundry, an open source Platform as a Service (PaaS). Bluemix delivers enterprise-level services that can easily integrate with your cloud applications without you needing to know how to install or configure them.
- It committed to include 3500 researchers to work on Spark-related projects.
- It will donate IBM SystemML (its machine learning language and libraries) to Apache Spark open source
The question is why this move by IBM?
First let us look at what is Apache Spark? Developed at UC Berkeley's AMPLab, Spark gives us a comprehensive, unified framework to manage big data processing requirements with a variety of data sets that are diverse in nature (text data, graph data etc) as well as the source of data (batch v. real-time streaming data). Spark enables applications in Hadoop clusters to run up to 100 times faster in memory and 10 times faster even when running on disk. In addition to Map and Reduce operations, it supports SQL queries, streaming data, machine learning and graph data processing. Developers can use these capabilities stand-alone or combine them to run in a single data pipeline use case. In other words, Spark is the next-generation of Hadoop (came with its batch pedigree and high latency).
With other solutions for real-time analytics via in-memory processing such as RethinkDB, an ambitious Redis project or commercial in-memory SAP Hana, IBM needed a competitive offering. Other vendors betting on Spark range from Amazon to Zoomdata. IBM will run its own analytics software on top of Spark, including SystemML for machine learning, SPSS, and IBM Streams.
At this week's Strata conference, several companies like Uber described how they have deployed Spark all the way for speedy real-time analytics.
Published October 24, 2015 Reads 3,323
Copyright © 2015 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Jnan Dash
Jnan Dash is Senior Advisor at EZShield Inc., Advisor at ScaleDB and Board Member at Compassites Software Solutions. He has lived in Silicon Valley since 1979. Formerly he was the Chief Strategy Officer (Consulting) at Curl Inc., before which he spent ten years at Oracle Corporation and was the Group Vice President, Systems Architecture and Technology till 2002. He was responsible for setting Oracle's core database and application server product directions and interacted with customers worldwide in translating future needs to product plans. Before that he spent 16 years at IBM. He blogs at http://jnandash.ulitzer.com.
![]() Dec. 17, 2017 02:45 PM EST Reads: 2,037 |
By Elizabeth White ![]() Dec. 17, 2017 02:30 PM EST Reads: 1,977 |
By Pat Romanski ![]() Dec. 17, 2017 02:00 PM EST Reads: 1,095 |
By Liz McMillan ![]() Dec. 17, 2017 11:30 AM EST Reads: 743 |
By Elizabeth White ![]() Dec. 17, 2017 10:00 AM EST Reads: 1,098 |
By Elizabeth White ![]() Dec. 17, 2017 09:45 AM EST Reads: 720 |
By Elizabeth White ![]() Dec. 17, 2017 06:30 AM EST Reads: 636 |
By Elizabeth White ![]() Dec. 17, 2017 06:30 AM EST Reads: 880 |
By Liz McMillan ![]() Dec. 16, 2017 11:45 AM EST Reads: 988 |
By Liz McMillan ![]() Dec. 15, 2017 11:00 AM EST Reads: 2,097 |
By Elizabeth White ![]() Dec. 14, 2017 04:00 PM EST Reads: 1,307 |
By Liz McMillan ![]() Dec. 14, 2017 11:45 AM EST Reads: 1,355 |
By Elizabeth White ![]() Dec. 14, 2017 11:00 AM EST Reads: 1,365 |
By Pat Romanski ![]() Dec. 13, 2017 02:00 PM EST Reads: 1,152 |
By Elizabeth White ![]() Dec. 13, 2017 11:00 AM EST Reads: 1,258 |
By Liz McMillan ![]() Dec. 11, 2017 11:15 PM EST Reads: 10,109 |
By Liz McMillan ![]() Dec. 10, 2017 07:45 AM EST Reads: 3,961 |
By Elizabeth White ![]() Dec. 8, 2017 07:30 AM EST Reads: 1,246 |
By Pat Romanski ![]() Dec. 7, 2017 08:00 PM EST Reads: 2,778 |
By Liz McMillan ![]() Dec. 7, 2017 02:00 PM EST Reads: 2,093 |