Growing Data in the Desert

The Phoenix Data Conference on October 29th 2016 was the place to be for Arizona’s biggest big data event. Prominent companies in Big Data and Analytics assembled to spread their wisdom and experience to a mixed audience of experts and newbies hungry to explore this hot and dynamic field. Speakers from Cloudera, Hortonworks, MapR, Amazon, Microsoft to name some, demonstrated the case studies of their customers and valuable tips for setting up a high performing big-data environments. This was the third Data Conference in the valley organized by Clairvoyant along with their sponsors, the popularity and anticipation of this event skyrocketing through the years.

It was impressive to see the highly scalable and multitude of support on Azure HDInsight, Microsoft’s offered Hadoop distribution, demonstrated by Brig Lamoreaux during his presentation on Predictive Maintenance for Aerospace, where he went over the architecture and tools used for their use case. We were shown the phases leading up to the predictive analytics also going over the pipeline employing Kafka for stream processing, and analyzing with HDInsight powered by Cortona Intelligence, another Microsoft service, up to the final Web applications for the end users. Event hub, a stream ingestion solution was also utilized.

Another exciting session for me was the Hadoop Security Highlights, presented by Manish Ahluwalia and Scott Grzybowski from Cloudera, which brushed up my concepts on identity management, the triple A (Authentication, authorization and Auditing) and Transport layer security. The EDH (enterprise data hub) services, navigator for auditing and encryption are powerful tools to track, access and protect data with high performing key management. Further granular access control through ACLs and Sentry were also covered.

I have been attending all the conferences organized by Clairvoyant, and the PDC this year has been highly successful with a great turnout and reviews. The presentations have been valuable for me personally as a Hadoop administrator in aspects of security, service performance and products demonstrated. I would like to thank the organizers and sponsors for this great platform for networking and sharing with the community. Got to love the open source model in our industry!