DevOps Articles

Optimizing Compute in the Post-Hadoop Era

In my recent CIO.com post, “Is There Life After Hadoop?”, I wrote about the post-Hadoop era and two key strategies that organizations can deploy to help them transition. These strategies are: Build a better lake, and Optimize the compute.

Hadoop has had a good run over the years, but for many organizations it’s time to move on, and Spark has emerged as the tool of choice to replace it.

This in turn has driven the migration from Hadoop and fueled the adoption of Spark and machine learning technologies. As organizations look to migrate existing Hadoop data and applications, they need an approach that will allow them to effectively manage their shrinking Hadoop investment, while at the same time increasing their investments in Spark and machine learning technologies.