Dynamic VM Lists in Azure DevOps Pipelines

Hardcoding VM names in pipelines breaks when VMs are added or removed. Here's how to dynamically discover VMs and run tasks against them.

The Challenge

You want to:

• Run a script against all VMs in a resource group
• Install updates on VMs matching a naming pattern
• Collect diagnostics from VMs with specific tags

But the VM list changes over time.

Discovering VMs with PowerShell

# Get all VMs in a resource group
$vms = Get-AzVM -ResourceGroupName "rg-production" |
  Select-Object Name, ResourceGroupName

# Filter by naming pattern
$webVms = Get-AzVM -ResourceGroupName "rg-production" |
  Where-Object { $_.Name -like "vm-web-*" }

# Filter by tag
$vms = Get-AzVM |
  Where-Object { $_.Tags["Environment"] -eq "Production" }

# Get running VMs only
$runningVms = Get-AzVM -ResourceGroupName "rg-production" -Status |
  Where-Object { $_.PowerState -eq "VM running" }

Passing VM List Between Stages

Option 1: Pipeline Variables

stages:
- stage: Discover
  jobs:
  - job: GetVMs
    steps:
    - task: AzurePowerShell@5
      name: DiscoverVMs
      inputs:
        azureSubscription: 'your-service-connection'
        ScriptType: 'InlineScript'
        Inline: |
          $vms = Get-AzVM -ResourceGroupName "rg-production" |
            Where-Object { $_.Tags["AutoUpdate"] -eq "true" }

          $vmJson = ($vms | Select-Object Name, ResourceGroupName | ConvertTo-Json -Compress)
          Write-Host "##vso[task.setvariable variable=vmList;isOutput=true]$vmJson"
        azurePowerShellVersion: 'LatestVersion'

- stage: Process
  dependsOn: Discover
  variables:
    vmList: $[ stageDependencies.Discover.GetVMs.outputs['DiscoverVMs.vmList'] ]
  jobs:
  - job: ProcessVMs
    steps:
    - task: PowerShell@2
      inputs:
        targetType: 'inline'
        script: |
          $vms = '$(vmList)' | ConvertFrom-Json
          foreach ($vm in $vms) {
            Write-Host "Processing: $($vm.Name)"
          }

Option 2: Pipeline Artifacts

For larger lists or complex data:

- stage: Discover
  jobs:
  - job: GetVMs
    steps:
    - task: AzurePowerShell@5
      inputs:
        azureSubscription: 'your-service-connection'
        ScriptType: 'InlineScript'
        Inline: |
          $vms = Get-AzVM -ResourceGroupName "rg-production" |
            Select-Object Name, ResourceGroupName, @{N='Tags';E={$_.Tags | ConvertTo-Json}}

          $vms | Export-Csv "$(Build.ArtifactStagingDirectory)/vms.csv" -NoTypeInformation
        azurePowerShellVersion: 'LatestVersion'

    - publish: $(Build.ArtifactStagingDirectory)/vms.csv
      artifact: vm-inventory

- stage: Process
  dependsOn: Discover
  jobs:
  - job: ProcessVMs
    steps:
    - download: current
      artifact: vm-inventory

    - task: PowerShell@2
      inputs:
        targetType: 'inline'
        script: |
          $vms = Import-Csv "$(Pipeline.Workspace)/vm-inventory/vms.csv"
          foreach ($vm in $vms) {
            Write-Host "Processing: $($vm.Name)"
          }

Running Commands on Each VM

Using Run Command

- task: AzurePowerShell@5
  inputs:
    azureSubscription: 'your-service-connection'
    ScriptType: 'InlineScript'
    Inline: |
      $vms = '$(vmList)' | ConvertFrom-Json

      foreach ($vm in $vms) {
        Write-Host "Running command on $($vm.Name)"

        Invoke-AzVMRunCommand `
          -ResourceGroupName $vm.ResourceGroupName `
          -VMName $vm.Name `
          -CommandId "RunPowerShellScript" `
          -ScriptString "Get-Service | Where-Object { $_.Status -eq 'Running' } | Measure-Object"
      }
    azurePowerShellVersion: 'LatestVersion'

Parallel Execution

For faster processing, use jobs matrix:

- stage: Process
  dependsOn: Discover
  jobs:
  - job: ProcessVM
    strategy:
      matrix:
        ${{ each vm in split(variables.vmList, ',') }}:
          ${{ vm }}:
            vmName: ${{ vm }}
    steps:
    - task: AzurePowerShell@5
      inputs:
        azureSubscription: 'your-service-connection'
        ScriptType: 'InlineScript'
        Inline: |
          $vmName = "$(vmName)"
          Write-Host "Processing: $vmName"
          # Your VM-specific task here

Scheduled VM Inventory

Create a scheduled pipeline that maintains an up-to-date inventory:

trigger: none

schedules:
- cron: "0 6 * * *"
  displayName: Daily VM Inventory
  branches:
    include:
    - main

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: AzurePowerShell@5
  displayName: 'Generate VM Inventory'
  inputs:
    azureSubscription: 'your-service-connection'
    ScriptType: 'InlineScript'
    Inline: |
      $inventory = @()

      $subscriptions = Get-AzSubscription
      foreach ($sub in $subscriptions) {
        Set-AzContext -SubscriptionId $sub.Id

        $vms = Get-AzVM -Status
        foreach ($vm in $vms) {
          $inventory += [PSCustomObject]@{
            Subscription = $sub.Name
            ResourceGroup = $vm.ResourceGroupName
            Name = $vm.Name
            Size = $vm.HardwareProfile.VmSize
            PowerState = $vm.PowerState
            OS = $vm.StorageProfile.OsDisk.OsType
            Tags = ($vm.Tags | ConvertTo-Json -Compress)
            LastUpdated = (Get-Date).ToString("o")
          }
        }
      }

      $inventory | Export-Csv "$(Build.ArtifactStagingDirectory)/inventory.csv" -NoTypeInformation
    azurePowerShellVersion: 'LatestVersion'

- publish: $(Build.ArtifactStagingDirectory)/inventory.csv
  artifact: vm-inventory

Using the Inventory in Other Pipelines

resources:
  pipelines:
  - pipeline: inventory
    source: 'VM Inventory Pipeline'
    trigger:
      branches:
        include:
        - main

steps:
- download: inventory
  artifact: vm-inventory

- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: |
      $vms = Import-Csv "$(Pipeline.Workspace)/inventory/vm-inventory/inventory.csv"
      $runningVms = $vms | Where-Object { $_.PowerState -eq "VM running" }

      Write-Host "Found $($runningVms.Count) running VMs"

Need help automating your Azure infrastructure? Get in touch - we help organisations build efficient DevOps processes.