Using AzCopy with Batch Files and Task Scheduler

As regular readers of this blog will know, I’m a big fan of AzCopy, especially now that it has a sync option to keep local data synchronized with blob storage.

In a recent release of the tool, Microsoft introduced a new switch to ensure that old files are only flushed from the destination when using the sync option --delete-destination true.

This coincidentally has caused an issue if we are using Windows Task Scheduler to schedule the synchronization process, especially if we use a SAS (Shared Access Signature) token which can be quite long. What then happens is we have a command that is longer than Windows Task Scheduler allows, and the task will fail with a very unhelpful error message:

Task Scheduler failed to execute task "\AzureBlobStorageSync". Attempting to restart. Additional Data: Error Value: 2147942487.

The logical solution is to move the AzCopy sync command line into a batch file, and just run that from Task Scheduler instead.

Unfortunately, SAS tokens contain characters that need to be escaped when used inside a batch file. I’ve just spent the better part of an hour struggling with getting AzCopy working on two customer servers for this very reason, and while the solution is fairly straightforward, it did take some head-scratching and experimentation.

If you run into this problem yourself, you will need to perform a search and replace on the special characters in your AzCopy sync command to make sure it runs safely inside a batch file:

• firstly, all ampersand characters must be escaped with a caret, so replace & with ^&
• secondly, all percentage signs must be doubled up, so replace % with %%
• finally, all quotation marks must be escaped with a caret, so replace " with ^"

In my case, it was the quotation marks that had me stumped, because I used them around the source and destination values and I was not escaping them.

If you do the search and replace correctly, you’ll be able to run the batch file from the command line. Once you’ve tested that, you can call the batch file from your scheduled task.

For those of you interested in finding out which other special characters need escaping, I created the following table based on this page.

Share your thoughts in the comments below.

Photo by Fernanda kovacs on Unsplash.