In previous blog posts, I showed how you can test and deploy your ARM templates through Azure DevOps and GitHub Actions. I mentioned one way to test ARM templates in a pipeline there (the WhatIf module). But there are other options available. And how about testing locally? In this post we will look at different tools that can help you with testing ARM templates.
Visual studio code ARM extension
If you are using Visual Studio Code for your development, the ARM Tools extension can really help you out. There has been some amazing development this last year and you can almost let the extension write the ARM templates for you. But when we focus on testing, the extension will tell you when there are errors in your deployment or when you are not following best practices. It will even connect parameter files to check if your parameters are valid.
Here is an example
Here I have left out a property from the template. There is a small yellow line that asks for my attention. If I hover my mouse over it, it shows the problem.
In this example, I have put a variable in the template that is not defined. The extension finds this issue and warns you.
For development of ARM templates, the extension is really a must in my opinion. You can download it here.
Validation check
If you deploy with PowerShell or AZ CLI, there is a build in validation check available. This will check if the ARM template is in a valid format. It is also able to catch some errors that will prevent your deployment from working.
In PowerShell, you would use the same syntax as New-AzResourceGroupDeployment, but replace New with Test.
In the example beneath, you see the error I get for a template with a storage account name that is not allowed.
With Az CLI, you would use az deployment group Validate
PowerShell: What if
A new option that is currently in Preview is the PowerShell What-if functionality. This enables you to check your ARM template against the live subscription. It will show you if resources will be created, modified or removed. This can save you some headaches if you made a typo somewhere.
To use WhatIf, you need to make sure you are running a preview version of the Azure Resources module.
The cmdlet is like this:
$Parameters = @{
ResourcegroupName = "ARMTesting"
Templatefile = ".\azuredeploy.json"
StorageAccountPrefix = "arm"
Mode = 'Incremental'
}
New-AzResourceGroupDeployment @Parameters -WhatIf
The result will be like this:
You are able to use this interactively or in a pipeline. You can read more about the functionality and how to use it in a pipeline in my previous blogpost.
ARM-TTK
The Azure Resource Manager Template Toolkit (arm-ttk) is a set of best practices that you can run against your ARM templates. So it is not a check if your template works, but more if it is written in a clean and efficient way. This tool is used to test the templates in the Azure Quickstart templates.
You can directly install the ARM toolkit from the GitHub repository, or clone the repository to your local computer. You can find the repository on GitHub.
After you have made the files available on your computer, you can import the module to your session by using import-module
Import-Module .\arm-ttk.psd1
After that you can test your templates with the following cmdlet
Test-AzTemplate -TemplatePath .\azuredeploy.json
With this tool, I have to admit I found some inconsistencies. It sometimes gives false positives. To get correct results for a single template, the best way is to have a folder with only one template file in them.
It is also good to know that Pester 5 is not supported yet, although I haven’t found it to be an issue yet.
I would recommend to use this tool with your own interpretation. Don’t panic if you get an error in the results, but check if it is valid.
Azure DevOps
When you’ve got everything working with green tests, it is possible to make use of this tool in an Azure DevOps pipeline. There is a custom task available to do this. You can find it here.
AZSK-ARM template checker
The Azure Secure DevOps Kit is a tool with a focus on security. It definitely is one I want to mention. The tool has a lot of different features, but not all of them seem to have active development as this point. A tool that is interesting for testing ARM templates, is use the ARM template checker in your Azure DevOps pipeline.
To install the pipeline task, see the manual here.
After you have installed it, you can use the task in a pipeline. If I deploy a very basic storage account, the deployment will fail:
Now in a multitask pipeline, it is a bit of a challenge to collect the logs and find out what is wrong. What you need to do, is open the pipeline screen and find Download Logs.
This will download a zipfile. In that file, you can find a csv with the failed tests in the folder \validateandtest\4_ArmTemplateChecker_Logs_20204407_134429.zip\20200807_134427
(the dates will be different).
In that CSV, you can find some security settings that you could improve..
So while it can feel like a bit of a hassle to use, it does provide some useful information. I think it is useful in a dev pipeline to consider if you wan to address the security issues.
Custom Pester
Another option that you can use, is create custom Pester testing. This can be helpful if you have ARM templates that should always have specific security measures (for example, maybe you want to have a resource lock on every resource) or if you want to specify in a pester test what resources should be deployed.
One of the ways to work with Pester at this point is to use Get-Content and after that ConvertFrom-Json. This will create a PowerShell object that you can work with for the tests.
Here are a few blogposts that describe how to create your own pester tests for ARM Templates.
Test ARM Templates using Pester & Azure DevOps
Pester Test Your ARM Template in Azure DevOps CI Pipelines
Building An Infrastructure Pipeline Part 2 – Testing
Conclusion
As you can see, there are multiple tools for testing ARM templates. Some will fit your situation better than others. This post expresses my opinion, I recommend trying them all and finding out what works best for you.
