Menu Close

Test ARM templates with What-If in a pipeline

When What-If was still in preview, I wrote a post about all the functionality it offers. It is still something I use daily to see what would be deployed before I actually deploy it. I also wrote a post before about how you can test ARM templates in your pipeline. In this post, I want to show you a bit of a combination of those two: I wrote a PowerShell script that you can use in a pipeline to Test ARM templates with What-If.


The script I made can help in a few ways to check on your ARM deployment in a pipeline:

Validate results

What-if can be used to see what an ARM template will deploy in your Azure tenant. It can give an overview of all resources that would be deployed. This can be useful in a pipeline where you have a manual approval, so the results can be viewed.

Check for errors

A template deployment can fail, which often has to do to a wrong syntax in the parameters, or if it tries to deploy a resource with a name that is not allowed. What-If will catch those errors for you and give you the reason for the failure. This can be helpful in an automated pipeline, especially if a lot of parameters are involved.

Multiple templates in one repository

In the previous blog posts, I already showed how you can use What-If in a pipeline. What I want to reach with this script, is one script to test all the templates. This will keep the pipeline clean and makes a manual check easier.

Working with pipeline variables

In almost all the pipelines I have seen in production, there is some use of variables. When you use these name resources, that has influence on what is deployed and whether that deployment is correct. The script keeps track of these variables and includes them in the test, as long as the variable has the same name as the parameter in the template.

Solution: PowerShell script

The script I created can be found HERE.


What the script does

The script can work in two ways: with a hash table of templates and parameterfiles, or by collecting the templates from the repository.
I will explain in the examples below how it does that.

After it has collected the templates, it will make a comparison between the available environment variables and the template parameters. If it finds matches, it will set the parameter value to the same value as the environment variable.

Note: this only works if the parameter name is the same as the variable name.

After this is done, it checks all resources with What-If. It returns the resourceIDs of the new resources. If there is an error in a template, the task will fail.


To show how you can make use of the script, I have created a repository with a few ARM templates and parameter files, which you can find on GitHub. There is a storage account with a correct parameterfile (azuredeploy.parameters.json) and a wrong parameterfile (azuredeploy.parameters2.json).
There is also a virtual machine template. This template will fail when no parameters are set through the pipeline, as the parameters in the parameterfile are all placeholders.

Use the script for all the templates

First, let’s run the task while we just define the resource group. It will look through the repository for Template files. When it finds them, it searches for a parameterfile within the default naming convention, so [filename].parameters.json.

In this example, it will pick up azuredeploy.parameters.json for the storage account, but not azuredeploy.parameters2.json. To take care of the parameters in the virtual machine deployment, I have created some variables

The script in PowerShell to reach this goal is the following:

.\Tests\whatif.ps1 -TestResourceGroupName ARMdeploymentTest

Note: The resource group to deploy to needs to exist in the Azure tenant. I use a dedicated resource group specifically for this task

Let’s put the script in an Azure DevOps pipeline task:

(this shows only the relevant part of pipeline)

Or a GitHub action:

(this shows only the relevant part of pipeline)


As you see, all the variable are picked up from the pipeline and the templates are correct for deployment.

Use it with defined templates

In your repository, it might not be fitting to have all the ARM templates be collected on its own. For example, what if you have multiple parameterfiles for one template? In this case, you can create a hashtable with the filepath and the parameterfilepath you need to test.

In our example, let’s test the storageaccount azuredeploy.json with the parameterfile azuredeploy.parameters2.json. This should fail as the storage account name is incorrect.

The PowerShell will look like this:

$Templates = @{
'StorageAccount\azuredeploy.json' = 'StorageAccount\azuredeploy.parameters2.json'
.\Tests\whatif.ps1 -TestResourcegroup ARMdeploymentTest -Templates $Templates

You can use this in Azure DevOps, but I have not found a way to pass a hashtable as a parameter to a filepath. Luckily, you can call the script as inline code.

(this shows only the relevant part of pipeline)

Or use it in a GitHub action:

(this shows only the relevant part of pipeline)


You can find all scripts, templates and pipelines that were used in this post, in my GitHub RepoThe templates are more or less copied from the Microsoft QuickStart templates.

So this is how you can test ARM templates with What-If in one step. This can take away some of the most basic errors that can happen during deployment. If you have any questions, let me know in the comments!

Leave a Reply

Your email address will not be published. Required fields are marked *