Application consistent backup for Linux VMs using Azure Backup is generally available

13th February 2018 Anthony Mashford 0 Comments

We are excited to announce the general availability (GA) of application consistent backup for Linux VMs running in Azure using Azure Backup. We did a preview of this feature last year and customers are already using this framework for various applications like Oracle, MySQL, Mongo DB, SAP HANA, and PostGreSQL.

What is application consistent backup?

Application consistent backup ensures that the backed-up data is transactionally consistent, and that applications will boot up post VM restore. To ensure transactional consistency, applications need to be quiesced and there should be no unfinished transactions when taking a VM snapshot.

Windows has the Volume Snapshot Service (VSS) framework to ensure application consistent VM backup, but there is no such generic framework for Linux. With greater adoption of Azure Linux VMs, there is an increasing demand to backup critical enterprise applications running inside Azure VMs. We introduced a VSS-like generic framework for ensuring application consistent VM backup for Linux applications running on any Linux distribution. This framework gives you the flexibility to execute custom pre and post scripts as part of the VM backup process. These scripts can be used to quiesce application IOs while taking a VM snapshot that guarantees application consistency.

The framework, in fact, is more powerful as you can use it to take application native backup by invoking native backup APIs as part of the VM snapshot. Azure Backup will then move this data to an offsite recovery services vault making it secure against VM corruption, VM delete, or disaster recovery scenarios. All this without the need to shutdown the VM.

Key features

Effortlessly test and restore application data: You can now mount your applications directly from the recovery point using iSCSI. This will help you quickly validate your recovery points without copying any backup data. You can then recover specific application files without the need to restore the entire VM. For example, when there is a backup of an Azure Linux VM running MySQL, you can mount .sql data dumps from the cloud recovery point, directly attach MySQL database to this file, and quickly validate the backup or retrieve individual items such as tables without having to download the entire data dump.
Improved monitoring: With this release, any framework related error will surface in backup jobs to help you identify and easily fix the issues. For example, if you configured “Continue on failure” for scripts, the backup will proceed as a file system or crash consistent. However, the backup job will now show “Completed with warnings” and provide detailed error message and recommended action to fix the script error.
Application and distribution agnostic: The framework is agnostic of Linux distributions and versions, and works seamlessly for all supported Linux distributions as long as the guest application has APIs to pause and resume application IOs.
Sample scripts on GitHub: We are working with partners and ISVs to provide open source scripts on GitHub for popular Linux applications. As of this release, we have these in place for MySQL, Oracle, and Caché.
Automation at scale with Azure CL: You can now use Azure CLI to configure, manage, and automate Azure Backup tasks at scale. Take advantage of CLI features such as smart defaults for most common operations, tab completion, and pipe-able outputs to simply manage your VM backup and recovery operations. Apart from configuring backup and restore, you can also query on command outputs to automate other management tasks. For example, backup health information can be programmatically retrieved, and subsequent operations can be initiated based on the health status.

Getting started

The video below walks you through various steps on how to configure the framework for executing pre-script and post-script. For more details, please refer to the steps to configure application-consistent Linux VM backup.

You can now mount your applications directly from VM backup to validate your data, without the need to copy and backup data. Watch the below video for more details.

Customer success stories

The Open University (OU) leveraged the Azure Linux Consistent Backup solution to allow consistent image-based backups of Linux database servers for a set of Azure based virtual learning environments shortly going live in Azure.

The OU operates several virtual learning environments, such on-premises datacenters in Milton Keynes UK utilizing virtualization, and is now starting to deploy new virtual learning environments to Azure. By utilizing Azure Backup and Azure Linux Consistent Backup solution they are able to have a consistent approach to Linux database backups allowing them to simplify deployment, backup and recovery of new learning environments hosted in Azure.

“The pre-freeze and post-freeze process is almost identical to the process we use on Linux database servers with our on-premises virtual machine image level backup solution. This allows us to leverage our existing packaged pre-freeze and post-freeze scripts without modification. This has made deployment of an Azure virtual learning environment far simpler and allows us to use the Azure machine backup whilst ensuring database backup consistency”.

– Gareth Robins, Systems Programmer, IT Service & Support, The Open University

Intercept, a Microsoft partner, has implemented the application consistent backup for Oracle databases running in Azure Linux VM. Below is Intercept’s feedback on the feature.

“We have implemented the application consistent backup for Linux feature for one of our customer to make sure the Oracle Databases within the Azure VM’s will be backed up consistently. Our customer want their backup tooling to be simple and prefer an Azure native solution for ease of management and to have a uniform backup experience for all Azure resources , Azure application consistent backup for Linux makes that possible.”

– Remco Vrielink, Intercept

Impacting SA tried this solution for MySQL and MongoDB and have seen significant (up to 70%) improvement in their restore time as they need to only restore specific application files and not the entire VM. Below is the feedback from Impacting SA.

“It was easy to activate this functionality on, since we only had to follow the documentation and guidelines available. They were simple, explicit and easy to understand. A positive point of this feature was that it reduced the backup time, since the moment it starts until we are able to restore it. The backups recovers became a lot more faster, about 70% more, when compared to the previous process – it went from 4h to about 20 minutes.”

Sample scripts for Oracle DB

We have seen many customers use our solution for MySQL and Oracle, while MySQL open source scripts were already uploaded on our GitHub, we have now added sample Oracle scripts. You can use this as a reference to create your own scripts and leverage the power of this framework.

Seeking open-source community contribution

To make this framework generic and support a variety of applications like Windows VSS we are calling developers, ISVs, tech enthusiasts, and partners to contribute to open source scripts to the GitHub repository. The scripts will be available for everyone to use directly or customize based on their requirements. If you are interested in contributing please send an email to linuxazurebackupteam@service.microsoft.com and we will work with you to publish them on GitHub, explain usage guidelines, and ensure that you get acknowledgement for your contribution.

Mashford's Musings