Yesterday, I was reading the Docker newsletter from March 12, and I saw a lot of interesting links about how many people use Docker for several purposes. One of the writeups that picked my curiosity was a post written by Kenny Bastani called “Getting Started with Apache Spark and Neo4j using Docker Compose”. This article described in a simple manner how to use Apache Spark and Neo4j using a Docker-based image for it. Then, I began to search about Apache Spark in the Enterprise and in the process, I stopped in the Databricks site. This incredible team has developed an easy way to create an Apache Spark cluster with Databricks Cloud, which allows to deploy Spark As a Service in an unified Cloud-hosted data platform.
Apache Spark aims to become The Platform for Big Data for its incredible performance, ease of use and built for fast result delivery. Then, I spent almost a day reading everything I could about Apache Spark, and I found this great article at LinkedIn written by Kavitha Mariappan (VP of Marketing at Databricks), where she described why every company interested to extract value using from Apache Spark, could use Databricks Cloud for it.
Then, I began to think in the long term for Databricks Cloud making questions like: What about the new development trend to “Dockerize” everything? Docker containers is an amazing way to provide a fast, secure and clean way to deploy Enterprise applications; so what about if Databricks and Docker work together to create a Docker-based image to deploy Apache Spark quickly? But, beyond that, for all this, you need a strong base for this, and the recent release of Red Hat Enterprise Linux 7.1 is perfect for this; for its new Atomic Host offering, a version of the battle-tested Enterprise Linux distribution optimized to run Containers-based applications; combined with the Real-Time version, which could be a great complement for an Apache Spark cluster for its needs for low-latency response times for its distributed architecture. But, which benefits could bring to the world a collaboration among these great companies? Keep reading to find out.