We recently made several exciting announcements about Azure HDInsight, Microsoft’s fully managed, full spectrum, open-source analytics service for enterprises:
1. With effect from 5 January 2018, we have cut prices by up to 52 per cent on all workloads on HDInsight, and added an additional 80 per cent price reduction for the Microsoft R Server add-on. As part of this price reduction, we are renaming the “premium tier cluster” as “Enterprise Security Package”, which can be added optionally to your Azure HDInsight cluster.
2. General availability (GA) of Apache Kafka on HDInsight. The general availability of Apache Kafka will enable customers to build enterprise-grade, open-source, real-time analytics solutions such as IoT, fraud detection, clickstream analysis, social analytics and more, backed by Azure HDInsight’s SLA of 99.9 per cent.
3. Integration with Log Analytics – GA. Azure Log Analytics is now integrated with Azure HDInsight to enable enterprise-grade monitoring for mission-critical analytics workloads. Get alerts, monitor and debug all your Azure HDInsight workloads.
4. Integration with Power BI DirectQuery in preview. Create dynamic reports based on data and metrics you already have on your interactive query clusters. Build visualisations of your entire data set faster with the most recent data.
5. Development tools update for Spark. This release adds support in Visual Studio Code for Spark on Azure HDInsight. Developers can build robust production data pipelines and debug distributed Spark jobs. This also brings in the interactive experience for Python and Spark in Visual Studio Code.
Source: Azure Roadmap