Azure Blob Storage lifecycle management in public preview

12th June 2018 Anthony Mashford 0 Comments

Last year, we released Blob-Level Tiering which allows you to transition blobs between the Hot, Cool, and Archive tiers without moving data between accounts. Both Blob-Level Tiering and Archive Storage help you optimize storage performance and cost. You asked us to make it easier to manage and automate, so we did. Today we are excited to announce the public preview of Blob Storage lifecycle management so that you can automate blob tiering and retention with lifecycle management policies.

Lifecycle management

Data sets have unique lifecycles. Some data is accessed often early in the lifecycle, but the need for access drops drastically as the data ages. Some data remain idle in the cloud and is rarely accessed once stored. Some data expire days or months after creation while other data sets are actively read and modified throughout their lifetimes. Azure Blob Storage lifecycle management offers a rich, rule-based policy which you can use to transition your data to the best access tier and to expire data at the end of its lifecycle.

Lifecycle management policy helps you:

Transition blobs to a cooler storage tier (Hot to Cool, Hot to Archive, or Cool to Archive) to optimize for performance and cost
Delete blobs at the end of their lifecycles
Define rules to be executed once a day at the storage account level (it supports both GPv2 and Blob storage accounts)
Apply rules to containers or a subset of blobs (using prefixes as filters)

See Managing the Azure Storage Lifecycle to learn more.

Example

Consider a data set that is accessed frequently during the first month, is needed only occasionally for the next two months, is rarely accessed afterward, and is required to be expired after seven years. In this scenario, Hot storage is the best tier to use initially, Cool storage is appropriate for occasional access, and Archive storage is the best tier option after several months before it is deleted seven years later.

The following sample policy manages the lifecycle for such data. It applies to block blobs with prefix “foo”:

Tier blobs to Cool storage 30 days after last modification
Tier blobs to Archive storage 90 days after last modification
Delete blobs 2,555 days (seven years) after last modification
Delete blob snapshots 90 days after snapshot creation

{
   "version": "0.5",
   "rules": [
     {
       "name": "ruleFoo",
       "type": "Lifecycle",
       "definition": {
         "filters": {
           "blobTypes": [ "blockBlob" ],
           "prefixMatch": [ "foo" ]
         },
         "actions": {
           "baseBlob": {
             "tierToCool": { "daysAfterModificationGreaterThan": 30 },
             "tierToArchive": { "daysAfterModificationGreaterThan": 90 },
             "delete": { "daysAfterModificationGreaterThan": 2555 }
           },
           "snapshot": {
             "delete": { "daysAfterCreationGreaterThan": 90 }
           }
         }
       }
     }
   ]
}

Azure portal

How to get started

To enroll in public preview, you will need to submit a request to register this feature to your subscription. After your request is approved (within a few days), any existing and new GPv2 or Blob Storage account in West US 2 and West Central US will have the feature enabled. During preview, only block blob is supported. As with most previews, this feature should not be used for production workloads until it reaches GA.

To submit a request, run the following PowerShell or CLI commands.

PowerShell

Register-AzureRmProviderFeature -FeatureName DLM -ProviderNamespace Microsoft.Storage

CLI 2.0

az feature register –-namespace Microsoft.Storage –-name DLM

Cost

Lifecycle management feature is free of charge in preview. Customers are charged the regular operation cost for the List Blobs and Set Blob Tier API calls. See Block Blob pricing to learn more about pricing.

Get it, use it, and tell us about it

We are confident that Blob lifecycle management policy will simplify your cloud storage cost optimization strategy. We look forward to hearing your feedback on this feature through email at DLMFeedback@microsoft.com. As a reminder, we love hearing all of your ideas and suggestions about Azure Storage, which you can post at Azure Storage feedback forum.

Source: Azure Blog Feed

Mashford's Musings