CDH is normally on a six month release cycle and Spark is on 4 month release cycle. This often leads to an extra latency before a spark version gets integrated and supported in CDH. At times Cloudera might even choose to skip a couple of Spark versions. Reason for this latency seems to be the effort needed with integration testing and bugs to fix in Spark and other projects to get it all work together in time. But if you are like our teams, and can’t wait to get your hands on latest and greatest of both spark and CDH, you can always run the latest version of Spark on CDH. We encountered a similar scenario recently, one of our clients wanted to use the Spark Thrift Server on CDH5.5, but Spark Thrift JDBC/ODBC server is not included on CDH5.5. We figured this might be a common use case for many of you who…
Month: November 2015
Announcing Insight
Clairvoyant is proud to announce our new offering, Insight a managed service that meets all your big data needs. Why are we launching this offering? Over the course of the last three years, our work with various organizations and teams has showed us that there is no shortage of interesting problems; problems that can be solved by leveraging the data assets these organizations already have. There is a growing and widespread awareness of how all businesses are in some fashion or the other “DIGITAL BUSINESSES”. Data, lots of it, decisions and strategies powered by this data is the cornerstone of this transformation businesses are aiming for. Why is it then, that we still are not seeing so many successful applications that are truly data driven? What is hampering the true application of the data driven process at scale? From a technology perspective it all boils down to one thing – Infrastructure. Infrastructure needs to be…