Azure HDInsight announcements: Significant price reduction and amazing new capabilities

Today, we are really happy to announce that we are reducing the prices for Azure HDInsight service and making several awesome capabilities generally available.

Launched in 2013, Azure HDInsight is a fully-managed, full spectrum, open-source analytics cloud service by Microsoft that makes it easy, fast, and cost-effective to process massive amounts of data. You can use the most popular open-source engines such as Hadoop, Spark, Hive, LLAP, Kafka, Storm, HBase, R and install more open source frameworks from the OSS ecosystem.

Amazing value for our customers

Customers ranging from startups to enterprises are using Azure HDInsight for their mission-critical applications. The service enables a broad range of scenarios in manufacturing, retail education, nonprofit, government, healthcare, media, banking, telecommunication, insurance and many more industries ranging in use cases from ETL to Data Warehousing, from Machine Learning to IoT and many more. Many Fortune 500 customers are running their big data pipelines on Azure HDInsight:

AccuWeather is using this technology to gain real-time intelligence into weather and business patterns. Handling 17 billion requests for data each day, AccuWeather is helping 1.5 billion people safeguard and improve their lives and businesses.

Cornell Lab of Ornithology improved Machine Learning Workflow with Azure HDInsight. By moving their open-source workflow to Microsoft’s scalable Azure HDInsight service, the researchers reduced their analysis run times to 3 hours, generating results for more species and providing quicker results for conservation staff to use in planning.

Learn more about how other customers are using Azure HDInsight here.

50+% price reduction with even more value

Today we are excited to announce the following:

• Up to 52% price reduction in Azure HDInsight. We are happy to announce that we are lowering the prices in Azure HDInsight. Customers will get even more value from their batch processing, interactive querying, machine learning, streaming analytics, and real-time analytics workloads on Azure HDInsight at a much lower price. According to IDC, using Azure HDInsight, customers saw a 63% decrease in TCO as compared to on-premises solutions. The price reductions will substantially lower Azure HDInsight’s TCO even further. As a part of this price reduction, we will also be replacing the Premium cluster tier with the Enterprise Security Package that can be added to your Azure HDInsight cluster. Enterprise Security Package will still be in preview. The price reduction becomes effective on January 5, 2018. To learn more about the new pricing, read here.
• Additional 80% price reduction for R Server for Azure HDInsight. Accompanying the overall Azure HDInsight price reduction, the price of the Microsoft R Server for HDInsight is also being reduced by 80%. This reduces the cost to $0.016 per core hour, allowing Microsoft R Server on Azure HDInsight users to run distributed R analytics workloads at a significantly lower price.
• General availability of Apache Kafka. We are announcing the general availability of Apache Kafka on Azure HDInsight. Apache Kafka on Azure HDInsight enables customers to build enterprise-grade, open-source, real-time analytics solutions such as IoT, fraud detection, clickstream analysis, social analytics and more backed by Azure HDInsight 99.9% SLA.
• General availability of Azure Log Analytics integration. Azure Log Analytics with Azure HDInsight enables enterprise grade monitoring for mission critical analytics workloads. You can now get alerts, monitor and debug all your Azure HDInsight workloads.
• Public preview integration with Power BI direct query. We are excited to announce public preview integration with Power BI direct query, which allows you to create dynamic reports based on data and metrics you already have on your Interactive Query clusters in Azure BLOB store or Azure Data Lake Store. You can now also build visualizations on your entire data set much faster with the most recent data.
• Advanced development tools for Apache Spark. For Scala and Java developers building robust production data pipelines in Azure HDInsight, we now offer plug-ins for IntelliJ and Eclipse with unique ability to submit and debug Spark jobs on the HDInsight clusters and support for distributed debugging of Spark code running across multiple Spark executors. For PySpark developers, who value productivity of Python language, new Visual Studio Code plugin for HDInsight offers first class integration with this popular code editors. Developers can now edit their scripts locally on their machines and submit PySpark statements to Azure HDInsight cluster with interactive experience.

Azure HDInsight: Cloud-native full-spectrum open source analytics on Azure

Today’s announcements build upon the following core capabilities of Azure HDInsight service, which provides enterprises with a robust platform to run their production workloads:

• Cloud native. Azure HDInsight is the only service in the industry to provide an end-to-end SLA for your production workloads running Hadoop, Spark, Hive, LLAP, Kafka, Storm, HBase and R.

"At Johnson Controls, we use Azure HDInsight for performing real-time and batch analysis of sensor data that we collect from over 6,000 Connected Industrial Chillers deployed across the world. Azure HDInsight offers us the scalability, reliability, high-availability, performance, security and ease of deployment that we need in our production infrastructure to consistently deliver value to our customers".

– Vaidhyanathan Venkiteswaran, Platform Engineering Manager, Data Enabled Business, Johnson Controls

• Low cost. Azure HDInsight is a cost-effective service that is extremely powerful and reliable. You pay only for what you use. You can create clusters on demand and then scale them up or down as needed. The decoupled compute and storage architecture provides better performance and flexibility.

“PROS is a pioneer in using machine learning to give companies an accurate and profitable pricing. PROS Guidance product runs enormously complex pricing calculations based on variables that comprise multiple terabytes of data. In Azure HDInsight, a process that formerly took several days now takes just a few minutes.”’

• Secure and compliant. Azure HDInsight brings enterprise grade protection of your data with monitoring, virtual networks, encryption, Active Directory authentication, authorization, role-based access control and more. The service meets the most popular industry and government compliance standards.

“HDInsight as a Big Data Platform has enabled our data engineers and scientists to focus on developing data and analytics products rather than managing infrastructure and troubleshooting day-day issues related to very large clusters. The heavy lifting of installing and managing clusters, providing robust security with Apache Ranger, data at rest encryption, monitoring and scaling up/down is taken care by HDInsight. This platform is used for variety of use cases like real time streaming, machine learning, visualization, ETL. Overall a very positive experience with HDInsight engineering, product and support teams.”

– Navaljit Bhasin, Big Data Engineering Director, Honeywell

• Global. Azure HDInsight is available in more Azure regions than any other Big Data offering in any other cloud. Azure HDInsight is also available in Azure government clouds in the US and sovereign clouds including China and Germany to meet sovereign compliance requirements.
• Productive. With Azure HDInsight, you can use your preferred productivity tools such as Visual Studio, Eclipse, and IntelliJ for Scala, Python, R, Java and  .NET. Data scientists and machine learning professionals can collaborate on the most popular notebooks such as Jupyter and Zeppelin.
• Extensible. With Azure HDInsight, you can seamlessly integrate with the most popular big data solutions with a single-click and easily extend your analytics cluster capabilities with applications, edge nodes, and customize using script actions.

“Azure HDInsight Application Platform is the one of the most extensible platforms and has allowed us to make our product easily available to many customers using the Azure Platform. The one-click deploy experience is a game changer and takes away the most common pain point around discovering and installing applications from the Big Data ecosystem. It was by far the easiest way for our enterprise customers to deploy and experience AtScale running on the Azure Cloud."

–   Eddie White, VP of Business Development, AtScale

Get started now!

With Azure HDInsight, our mission is to provide a fully managed, full spectrum of open source technologies combined with the power of the cloud. Customers today are using these open source technologies to build a variety of different applications such as batch processing, ETL, Data Warehousing, Machine Learning, IoT and more. We hope you take full advantage of today’s announcements and we are excited to see what you will build with Azure HDInsight. Read this developer guide and follow the quickstart guide to learn more about implementing these pipelines and architectures on Azure HDInsight. Stay up-to-date on the latest Azure HDInsight news and features by following us on Twitter #HDInsight and @AzureHDInsight. For questions and feedback – please reach out to AskHDInsight@microsoft.com.

Source: Azure Blog Feed