Storage scenarios for Cray in Azure
When you get a dedicated Cray supercomputer on your Azure virtual network, you also get attached Cray® ClusterStor™ storage. This is a great solution for the high-performance storage you need while running jobs on the supercomputer. But what happens when the jobs are done? That depends on what you’re planning to do. Azure has a broad portfolio of storage products and solutions.
Post-processing
Many times, you’re using your Cray supercomputer as part of a multi-stage workflow. Using the weather forecasting scenario we wrote about, after the modeling is done, it’s time to generate products. The most familiar setup for most HPC administrators would be to attach Azure Disks to a virtual machine and run a central file server or a fleet of Lustre servers.
But if your post-processing workload can be updated to use object storage, you get another option. Azure Blob Storage our object storage solution. It provides secure, scalable storage for cloud-native workloads. This allows your jobs to run at large scale without having to manage file servers.
Our recent acquisition of Avere Systems will bring another option for high-performance file systems. Avere’s technology will also enable hybrid setups, allowing you to move your data between on-premises and Azure resources.
Archive
Sometimes when the work is done, the only thing left to do is store your results. You might need to keep the data for future re-analysis or for compliance purposes, but you don’t expect to re-use it very often. Azure Archive storage is a special tier of Blob storage. It provides reliable, low-cost storage for rarely accessed data. Because it’s part of our Blob storage, you don’t need separate tooling to use it in your workflow. In fact, you can even control the tier on a per-object basis within the same storage account.
Reference data
We’ve talked about what to do with data after your jobs are done, but what about before? Azure has many options for storing reference data and other files that you stage into your high-performance workload. Azure Files lets you deploy cloud file shares presented to clients with the SMB 3.0 protocol. Our partnership with NetApp will give you the ability to use the powerful NFS based data management capabilities that NetApp customers know and love as a first party Azure service. And of course, you can always use Azure Disks attached to a file server of your choice.
Data ingest and hybrid
Maybe what you really need is to just get your data into Azure in the first place. StorSimple is a hybrid solution that makes it easy to intelligently tier file share data into Azure. And if you have many terabytes or petabytes of data that you need to ingest into Azure, the Azure Data Box is a secure, ruggedized, appliance that easily fits into your network for large offline data transfers.
Conclusion
No matter what kind of storage you need around your Cray supercomputer, Azure has a product that works for you. To learn more about Azure’s storage offerings, see the Azure Storage page.
Source: Azure Blog Feed