Tuesday, October 17, 2017

Copy all Azure Blob containers from one storage account to another

If you're using Azure Storage, you'll probably want to be able to clone storage account for testing purposes, setting up multiple environments, etc. To my surprise, there is not a very straight forward way to just clone a storage account.

AzCopy

AzCopy is a utility that helps move data to and from storage account. It has a method to copy data between storage accounts, but can only do a single container at a time. Not sure why they couldn't just make it do all containers, but it doesn't, so what are our options. Well, if you're developing for Azure, then PowerShell is probably going to be your best bet. You could technically write the entire transfer script using PowerShell, but AzCopy does a good job copying per container so let's just use PowerShell to iterate all the containers and have AzCopy do the work.

The Script


cd 'C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy'

$sourceStorageAccountName = "SOURCE_STORAGE_ACCOUNT_NAME"
$sourceStorageAccountKey = "SOURCE_STORAGE_ACCOUNT_ACCESS_KEY"

$destStorageAccountName = "DESTINATION_STORAGE_ACCOUNT_NAME"
$destStorageAccountKey = "DESTINATION_STORAGE_ACCOUNT_ACCESS_KEY"

$sourceStorageAccount = New-AzureStorageContext -StorageAccountName $sourceStorageAccountName -StorageAccountKey $sourceStorageAccountKey
$destStorageAccount = New-AzureStorageContext -StorageAccountName $destStorageAccountName -StorageAccountKey $destStorageAccountKey

$containers = Get-AzureStorageContainer -Context $sourceStorageAccount
foreach($container in $containers) {
 Write-Host "Copying container $($conatiner.Name) from $($sourceStorageAccountName) to $($destStorageAccountName)"
 .\AzCopy.exe /Source:https://$sourceStorageAccountName.blob.core.windows.net/$($container.Name) /SourceKey:$sourceStorageAccountKey /Dest:https://$destStorageAccountName.blob.core.windows.net/$($container.Name) /DestKey:$destStorageAccountKey /S
}


Durability

If for some reason your script terminates, just run the same command that just terminated and you'll be prompted whether you'd like to resume or restart the transfer. Cool!

1 comment: