How we manage and automate our deploys from AppVeyor to Azure

At Q42, we love AppVeyor as a build service for our .Net projects. It is super easy to setup, integrates great with GitHub and is able to deploy our applications to Azure. Because we also love Azure. It’s great for hosting our ASP.Net websites and it takes care of managing and updating the webservers, so we don’t have to do it.

Some of our projects are small, with just one or two web applications, so it’s no problem to manually setup Azure, set up deploy environments in AppVeyor and click the deploy button in AppVeyor to start a new deploy. But this becomes really cumbersome for larger projects. That’s why we created four open source tools to help us automate the deployment process.

Large projects with deployments to multiple regions

One of our projects for the Dutch National Lottery has 3 different CMS instances, 4 APIs and 3 frontend web applications and it’s still growing.

We usually have multiple environments on Azure: Test, Acceptance and Production. For this specific client, the production environment is duplicated in two regions. One main region and one fallback region.

This means there are 10 applications * 4 environments = 40 web applications in Azure to deploy to.

Doing a single deploy to the production environment in 2 regions means doing 20 deploys. We didn’t want to do that manually using the AppVeyor management website, so we needed to automate this.

Setting up a new environment on Azure

Let’s go through the steps needed to set up a new environment and the tools we created to automate some of the tasks. Let’s say for example we need to set up a second test environment in Azure to test a big feature. Because our complete project contains 10 websites, this means creating 10 websites in Azure. AppVeyor does our deploys, so we need to create 10 deployment environments in AppVeyor with the credentials from the Azure Website publish profiles.

A publish profile contains all the info to deploy to a website slot in Azure. It contains the URL, username, password etc. Downloading them all manually costs a lot of time. So we created a small tool to download all publish profiles with a single click.

DumpAzurePublishProfiles

michielpost/DumpAzurePublishProfiles
DumpAzurePublishProfiles - Mass download PublishSettings for Azure Websitesgithub.com

This tool downloads all the publish profiles from your Azure subscription. Using it is easy: just put your Azure Subscription Publish file in the same directory, run this tool and all publish files will be written to the current directory.

Setting up AppVeyor for deploys

Great, now we have 10 publish profiles! We need to use the information from the publish profile to create AppVeyor environments. Manually creating new environments and copy and pasting info from a publish file is not my favorite job, so let’s create another tool for that.

PublishSettings2AppVeyor

michielpost/PublishSettings2AppVeyor
PublishSettings2AppVeyor - Batch create AppVeyor environments based on .PublishSettings filesgithub.com

This tool is super easy. It does one job and does it well. It creates an AppVeyor environment for each publish profile. Put your publish profiles in the same directory as this tool and run it. It will ask for your AppVeyor API key and then does its work. You can run it multiple times and it won’t overwrite existing environments in AppVeyor.

Deploys using AppVeyor

AppVeyor now contains 10 new environments to deploy to. Our next task is actually deploying to the newly created environment. Remember, our application consists of 10 individual applications. AppVeyor can deploy applications, but you have to navigate on the AppVeyor website to your app, click the deploy tab, click the environment you want to deploy to, click the correct version and wait for it to deploy successfully. That’s OK for 1 website, but too much work for 10 websites. AppVeyor does not have great support for doing multiple deploys at once for a whole ecosystem of apps. There’s a long running issue about that here.

Luckily AppVeyor does have a great API, so we can create something that does just what we need.

AppVeyorDeployConsole

michielpost/AppVeyorDeployConsole
AppVeyorDeployConsole - Create deploy groups to deploy to multiple environments at oncegithub.com

The AppVeyorDeployConsole needs only one thing as input: your AppVeyor API key. It then lets you create a deploy environment that contains multiple AppVeyor environments.

So we create a Test environment that contains 10 AppVeyor environments and a production environment that contains 20 AppVeyor Environments (for both regions).

We only have to do that once, because your defined environments are getting saved to disk.

Deploying to production

We can now deploy, for example to the production environment. The AppVeyorDeployConsole will ask which version to deploy. We type in the version number, say we’re sure and it will send 20 deploy commands to AppVeyor. They all run in parallel and we get feedback when they succeed or fail.

This is going pretty well! We deployed to our production environment and it didn’t cost us a lot of time. But wait, for production, we’re deploying to staging slots on Azure. By using a staging slot you can deploy your website without downtime for the end user. A simple swap command switches the newly deployed version to the live production site. We don’t want to log into Azure and swap 20 websites, so we also created something to automatically swap our websites.

AzureWebsitesSuperSwapper

michielpost/AzureWebsitesSuperSwapper
AzureWebsitesSuperSwapper - Swap multilple website slots from a single commandgithub.com

The Azure Websites Super Swapper only does one thing: swap websites. It takes a CSV file as input which contains a list of websites to swap. When you run this tool it will ask for confirmation and swap all the websites. You can see when the swap is finished or if there were problems.

Now we’re really finished and our new version is live. By using these tools we saved a lot of time and clicks. I could have created one big tool to manage the whole process, but I like small tools that do a single job very well. By making the tools open source and available for everyone, I hope it might help others. Your deployment process might be different from ours, so only a single tool might be useful for you.

Check out our Engineering Blog for more in depth stories about pragmatic code for happy users!