
By Bill McColl | Article Rating: |
|
October 18, 2010 08:15 AM EDT | Reads: |
25,918 |

Over the past few years, Hadoop has become something of a poster child for the NoSQL movement. Whether it's interpreted as "No SQL" or "Not Only SQL", the message has been clear, if you have big data challenges, then your programming tool of choice should be Hadoop. Sure, continue to use SQL for your ancient legacy stuff, but when you need cutting edge performance and scalability, it's time to go Hadoop.
The only problem with this story is that the people who really do have cutting edge performance and scalability requirements today have already moved on from the Hadoop model. A few have moved back to SQL, but the much more significant trend is that, having come to realize the capabilities and limitations of MapReduce and Hadoop, a whole raft of new post-Hadoop architectures are now being developed that are, in most cases, orders of magnitude faster at scale than Hadoop.
The problem with simple batch processing tools like MapReduce and Hadoop is that they are just not powerful enough in any one of the dimensions of the big data space that really matters. If you need complex joins or ACID requirements, SQL beats Hadoop easily. If you have realtime requirements, Cloudscale beats Hadoop by three or four orders of magnitude. If you have supercomputing requirements, MPI or BSP
The one area where MapReduce/Hadoop wins today is that it's freely available to anyone, but for those that have reasonably challenging big data requirements, that simple type of architecture is nowhere near enough.
Published October 18, 2010 Reads 25,918
Copyright © 2010 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Bill McColl
Bill McColl left Oxford University to found Cloudscale. At Oxford he was Professor of Computer Science, Head of the Parallel Computing Research Center, and Chairman of the Computer Science Faculty. Along with Les Valiant of Harvard, he developed the BSP approach to parallel programming. He has led research, product, and business teams, in a number of areas: massively parallel algorithms and architectures, parallel programming languages and tools, datacenter virtualization, realtime stream processing, big data analytics, and cloud computing. He lives in Palo Alto, CA.
![]() Sep. 21, 2015 10:00 AM EDT Reads: 460 |
By Pat Romanski ![]() Sep. 21, 2015 08:45 AM EDT Reads: 279 |
By Liz McMillan Sep. 21, 2015 05:45 AM EDT Reads: 479 |
By Elizabeth White ![]() Sep. 20, 2015 06:45 PM EDT Reads: 535 |
By Elizabeth White ![]() Sep. 20, 2015 12:45 PM EDT Reads: 393 |
By Elizabeth White ![]() Sep. 20, 2015 12:00 PM EDT Reads: 417 |
By Pat Romanski ![]() Sep. 20, 2015 12:00 PM EDT Reads: 379 |
By Pat Romanski ![]() Sep. 20, 2015 12:00 PM EDT Reads: 429 |
By Elizabeth White ![]() Sep. 20, 2015 12:00 PM EDT Reads: 515 |
By Liz McMillan ![]() Sep. 20, 2015 10:30 AM EDT Reads: 266 |
By Carmen Gonzalez ![]() Sep. 19, 2015 09:00 AM EDT Reads: 2,310 |
By Liz McMillan ![]() Sep. 18, 2015 11:00 PM EDT Reads: 629 |
By Elizabeth White ![]() Sep. 18, 2015 06:45 PM EDT Reads: 652 |
By Pat Romanski ![]() Sep. 18, 2015 11:30 AM EDT Reads: 1,194 |
By Elizabeth White ![]() Sep. 17, 2015 11:30 PM EDT Reads: 692 |
By Pat Romanski ![]() Sep. 17, 2015 10:45 AM EDT Reads: 525 |
By Carmen Gonzalez Sep. 17, 2015 09:45 AM EDT Reads: 1,885 |
By Pat Romanski ![]() Sep. 17, 2015 09:45 AM EDT Reads: 586 |
By Carmen Gonzalez ![]() Sep. 17, 2015 09:00 AM EDT Reads: 1,988 |
By Pat Romanski ![]() Sep. 16, 2015 06:45 PM EDT Reads: 668 |