Introduction

Configuration Data for DSC is somewhat analogous to configuration files in traditional applications; different environments can have their own Configuration Data file. The 2 main topics I will go over in this blog post is the use of Nodes in Configuration Data, and using multiple Configuration Data files.

Nodes in Configuration Data

A typical Configuration Data file looks something like this:

@{
    AllNodes =
    @(
        @{
            NodeName = "VM-1"
            Role     = "WebServer"
        },
        @{
            NodeName = "VM-2"
            Role     = "AppServer"
        },
        @{
            NodeName = "VM-3"
            Role     = "SQLServer"
        }
    );

    NonNodeData = @{
        LogFolder = "C:\Logs"
        InputFolder = "C:\Input"
        OutputFolder = "C:\Output"
        ApplicationUrl = "https://myapp.com"
        ServiceUrl = "https://myservice.com"
    }
}

AllNodes is the only required property; it is an array of Node objects. Each Node object must have a NodeName property. We can add as many additional properties as we would like to Node objects as well as the top top-level Configuration Data object itself. NonNodeData is generally used as a container to put all properties that do not apply to individual Nodes; however, we could rename it or split it into multiple objects. Taking the previous sample above, we can rearrange things to make it a bit easier to read:

@{
    AllNodes =
    @(
        @{
            NodeName = "VM-1"
            Role     = "WebServer"
        },
        @{
            NodeName = "VM-2"
            Role     = "AppServer"
        },
        @{
            NodeName = "VM-3"
            Role     = "SQLServer"
        }
    )
    Folders = @{
        LogFolder = "C:\Logs"
        InputFolder = "C:\Input"
        OutputFolder = "C:\Output"
    }
    Urls = @{
        ApplicationUrl = "https://myapp.com"
        ServiceUrl = "https://myservice.com"
    }
}

Regarding the NodeName property – you will often see this as the same as the VM Name. However, this isn’t required – NodeName can be anything. In fact, there is a good reason to not treat it as a VM Name, and instead use NodeName more like the Role of the VM. In the original example, what happens if we want to add a second WebServer? Since Configuration Data is set at compile time, we would need to recompile our DSC. Rewriting that Configuration Data file, we can eliminate the need to recompile if we want to add a new server:

@{
    AllNodes =
    @(
        @{
            NodeName = "WebServer"
        },
        @{
            NodeName = "AppServer"
        },
        @{
            NodeName = "SQLServer"
        }
    )
    Folders = @{
        LogFolder = "C:\Logs"
        InputFolder = "C:\Input"
        OutputFolder = "C:\Output"
    }
    Urls = @{
        ApplicationUrl = "https://myapp.com"
        ServiceUrl = "https://myservice.com"
    }
}

There is unfortunately one use case where the above does not work – if you have a DSC resource that needs to reference the actual VM name (such as the ComputerManagementDsc’s Computer resource which is used to join a computer to a domain). You would either have to revert back to specifying each VM name in your Configuration Data file, or rewrite the DSC resource to not need the actual VM name.

Dealing with multiple Configuration Data files

Let’s say we’ve properly split out our Configuration Data files so that we have one for each environment:

MySampleConfiguration.AzureCloud.psd1:

@{
    AllNodes =
    @(
        @{
            NodeName = "WebServer"
        },
        @{
            NodeName = "AppServer"
        },
        @{
            NodeName = "SQLServer"
        }
    )
    Environment = 
    @{
        Name = "AzureCloud"
    }
}
MySampleConfiguration.AzureUSGovernment.psd1:

@{
    AllNodes =
    @(
        @{
            NodeName = "WebServer"
        },
        @{
            NodeName = "AppServer"
        },
        @{
            NodeName = "SQLServer"
        }
    )
    Environment = 
    @{
        Name = "AzureUSGovernment"
    }
}

There is a good bit of duplicate content in both files. Since Configuration Data is just a hashtable object, and its value is set at compile time, we can split out our files, then merge them with powershell

MySampleConfiguration.shared.psd1:

@{
    AllNodes =
    @(
        @{
            NodeName = "WebServer"
        },
        @{
            NodeName = "AppServer"
        },
        @{
            NodeName = "SQLServer"
        }
    )
}
MySampleConfiguration.AzureCloud.psd1:

@{
    Environment = 
    @{
        Name = "AzureCloud"
    }
}
MySampleConfiguration.AzureUSGovernment.psd1:

@{
    Environment = 
    @{
        Name = "AzureUSGovernment"
    }
}

We need to write a function to merge the hashtable objects. In this case, any new properties are copied from the second object into a clone of the first, and any properties in the first that also exist in the second are overwritten by the second:

function Merge-Hashtables
{
    Param
    (
        [Parameter(Mandatory=$true)]
        [Hashtable]
        $First,

        [Parameter(Mandatory=$true)]
        [Hashtable]
        $Second
    )

    function ProcessKeys($first, $second)
    {
        foreach ($key in $second.Keys) {
            $firstValue = $first[$key]
            $secondValue = $second[$key]

            if ($firstValue -is [hashtable] -and $secondValue -is [hashtable])
            {
                ProcessKey($firstValue, $secondValue)
            }
            else
            {
                $first[$key] = $secondValue
            }
        }
    }

    $firstClone = $First.Clone()
    $secondClone = $Second.Clone()

    ProcessKeys -first $firstClone -second $secondClone

    return $firstClone
}

Then we load the shared and appropriate environment-specific Configuration Data file, merge them, and pass it in at compile time:

$sharedConfigData = Import-PowerShellDataFile MySampleConfiguration.shared.psd1
$azureCloudConfigData = Import-PowerShellDataFile MySampleConfiguration.AzureCloud.psd1
$mergedConfigurationData = Merge-Hashtables -First $sharedConfigData -Second $azureConfigData

$automationAccountName = "Your Azure Automation Account Name"
$resourceGroupName = "The resource group containing your automation account"
$configurationName = "Name of your configuration"
$pathToConfiguration = "Path to your DSC file"

Import-AzAutomationDscConfiguration -AutomationAccountName $automationAccountName -ResourceGroupName $resourceGroupName -SourcePath $pathToConfiguration -Force -Published

Start-AzAutomationDscCompilationJob -AutomationAccountName $automationAccountName -ResourceGroupName $resourceGroupName -ConfigurationName $configurationName -ConfigurationData $mergedConfigurationData

Introduction

Unfortunately, Azure DevOps does not have a SaaS offering running in Azure Government. The only options are to spin up Azure DevOps server in your Azure Government tenant or connect the Azure DevOps commercial PaaS offering (specifically Azure Pipelines) to Azure Government. Your customer may object to the latter approach; the purpose of this post is to provide you with additional ammunition in making a case that you can securely use commercial Azure DevOps with Azure Government.

Throughout this blog post, the biggest question you should always keep in mind is where is my code running?

Scenario

Take a simple example; a pipeline that calls a PowerShell script to create a key vault and randomly generates a secret to be used later (such as for a password during the creation of a VM)

add-type -AssemblyName System.Web
$rgName = "AisDwlBlog-rg"
$kvName = "AisDwlBlog-kv"
$pw = '"' + [System.Web.Security.Membership]::GeneratePassword(16, 4) + '"'
az group create --name $rgName --location eastus --output none
az keyvault create --name $kvName --resource-group $rgName --output none
az keyvault set-policy --name $kvName --secret-permissions list get --object-id 56155951-2544-4008-9c9a-e53c8e8a1ab2 --output none
az keyvault secret set --vault-name $kvName --name "VMPassword" --value $pw

The easiest way we can execute this script is to create a pipeline using a Microsoft-Hosted agent with an Azure PowerShell task that calls our deployment script:

pool:
  name: 'AisDwl'

steps:
- task: AzureCLI@2
  inputs:
    azureSubscription: 'DwlAzure'
    scriptType: 'ps'
    scriptLocation: 'scriptPath'
    scriptPath: '.\Deploy.ps1'

When we execute this, note that output from that PowerShell script flows back into the Pipeline:

Deployment Output

To circle back to that question that should still be in your mind….where is my code running? In this case, it is running in a Virtual Machine or container provided by Microsoft. If you have a customer that requires all interactions with potentially sensitive data to be executed in a more secure environment (such as IL-4 in Azure Government), you are out of luck as that VM/Container for the hosted build agent is not certified at any DoD Impact Level. Thus, we have to look at other options, where our deployment scripts can run in a more secure environment.

I’ll throw a second wrench into things…did you see the bug in my Deploy.ps1 script above? I forgot to add --output none to the last command (setting the password in the key vault). When I run the pipeline, this is shown in the output:

Secret Visible

Not good! In an ideal world, everyone writing these scripts would be properly handling output, but we need to code defensively to handle unintended situations. Also, think about any error messages that might bubble back to the output of the pipeline.

Option 1

Azure Pipelines provide the capability to run pipelines in self-hosted agents, which could be a VM or container-managed by you/your organization. If you set up this VM in a USGov or DoD region of Azure, your code is running in either an IL-4 or IL-5 compliant environment. However, we can’t simply spin up a build agent and call it a day. As with the Microsoft-hosted build agent, the default behavior of the pipeline still returns output to Azure DevOps. If there is ever an issue like I just demonstrated, or an inadvertent Write-Output or Write-Error, or an unhandled exception containing sensitive information, that will be displayed in the output of the pipeline. We need to prevent that information from flowing back to Azure Pipelines. Fortunately, there is a relatively simple fix for this: instead of having a task to execute your PowerShell scripts directly, create a wrapper/bootstrapper PowerShell script.

The key feature of the bootstrapper is that it executes the actual deployment script as a child process and captures the output from that child process, preventing any output or errors from flowing back into your Pipeline. In this case, I am simply writing output to a file on the build agent, but a more real-world scenario would be to upload that file to a storage account.

try
{
	& "$PSScriptRoot\Deploy.ps1" | Out-File "$PSScriptRoot\log.txt" -append
	Write-Output "Deployment complete"
}
catch
{
	Write-Error "there was an error"
}

The biggest disadvantage of this approach is the additional administrative burden of setting up and maintaining one (or more) VMs/containers to use as self-hosted build agents.

Option 2

If you would prefer to avoid managing infrastructure, another option is to run your deployment scripts in an Azure Automation Account. Your Pipeline (back to running in a Microsoft-hosted agent) starts an Azure Automation Runbook to kick off the deployment. The disadvantage of this approach is that all of your deployment scripts must either be staged to the Automation Account as modules or converted into “child” runbooks to be executed by the “bootstrapper” runbook. Also, keep in mind that the bootstrapper runbook must take the same preventative action of capturing output from any child scripts or runbooks to prevent potentially sensitive information from flowing back to the Pipeline.

Sample code of calling a runbook:

$resourceGroupName = "automation"
$automationAccountName = "dwl-aaa"
$runbookName = "Deployment-Bootstrapper"
                    
$job = Start-AzAutomationRunbook -AutomationAccountName $automationAccountName -ResourceGroupName $resourceGroupName -Name $runbookName -MaxWaitSeconds 120 -ErrorAction Stop
                    
$doLoop = $true
While ($doLoop) {
    Start-Sleep -s 5
    $job = Get-AzAutomationJob -ResourceGroupName $resourceGroupName –AutomationAccountName $automationAccountName -Id $job.JobId
    $status = $job.Status
    $doLoop = (($status -ne "Completed") -and ($status -ne "Failed") -and ($status -ne "Suspended") -and ($status -ne "Stopped"))
}
                    
if ($status -eq "Failed")
{
    Write-Error "Job Failed"
}

The Deployment script code running as an Azure Automation Runbook (Note that this has been converted to Azure PowerShell as the AzureCLI isn’t supported in an Automation Account Runbook):

$Conn = Get-AutomationConnection -Name AzureRunAsConnection
Connect-AzAccount -ServicePrincipal -Tenant $Conn.TenantID -ApplicationId $Conn.ApplicationID -CertificateThumbprint $Conn.CertificateThumbprint

add-type -AssemblyName System.Web
$rgName = "AisDwlBlog-rg"
$kvName = "AisDwlBlog-kv"
$pw = [System.Web.Security.Membership]::GeneratePassword(16, 4)

$rg = Get-AzResourceGroup -Name $rgName
if ($rg -eq $null)
{
	$rg = New-AzResourceGroup -Name $rgName -Location EastUs
}

$kv = Get-AzKeyVault -VaultName $kvName -ResourceGroupName $rgName
if ($kv -eq $null)
{
	$kv = New-AzKeyVault -Name $kvName -ResourceGroupName $rgName -location EastUs
}
Set-AzKeyVaultAccessPolicy -VaultName $kvName -PermissionsToSecrets list,get,set -ServicePrincipalName $Conn.ApplicationID

$securePw = ConvertTo-SecureString -String $pw -AsPlainText -Force
Set-AzKeyVaultSecret -VaultName $kvName -Name "VMPassword" -SecretValue $securePw
Azure Automation provides credential assets for securely passing credentials between the automation account and a Desired State Configuration (DSC). Credentials can be created directly through the portal or through PowerShell and easily accessed in the DSC configuration. However, there a few disadvantages with storing credentials in an automation account vs. storing credentials in a KeyVault:

  • More fine-grained permissions can be set on a KeyVault – for example, custom access policies can be set for different principals.
  • KeyVault secret values can be viewed in the portal (assuming you have access). Passwords in an automation account credential cannot.
  • In most scenarios, a KeyVault is the “single source of truth”, where all secure assets in a tenant are stored. If you need to access credentials in an ARM template and a DSC configuration, they must be in a KeyVault for use in the ARM template.

In this example, we will walk through a very simple workstation configuration that pulls a username and password from a KeyVault, then passes those parameters into a DSC resource to create a new local user on a target VM.

Prerequisites

  • A Key Vault already provisioned
  • An Automation Account already provisioned
  • The Az.Accounts and Az.KeyVault modules imported into the Automation Account

Permissions

The Automation Connection, or more specifically, the service principal of the automation connection, needs to have at least “Get” selected under Secret permissions in the KeyVault access policies.

Creating the DSC file

These are the minimum set of parameters necessary to extract a username and password from a KeyVault:

param
(
    [Parameter(Mandatory)]
    [string] $keyVaultName,

    [Parameter(Mandatory)]
    [string] $usernameSecretName,

    [Parameter(Mandatory)]
    [string] $passwordSecretName,

    [Parameter(Mandatory)]
    [string] $automationConnectionName
)

The first 3 parameters’ purpose should be self-explanatory. The final parameter, automationConnectionName, is used to establish a connection to Azure. Even though this code is executing in the context of an Automation Account, it is not connected to Azure in the same way as if we had connected using Login-AzAccount or Connect-AzAccount. There are special cmdlets available when running in an automation account that we can use to establish a “full” connection:

$automationConnection = Get-AutomationConnection -Name $connectionName

Note that we are calling Get-AutomationConnection, NOT Get-AzAutomationConnection. The latter command only works when you have already established a connection to Azure. Get-AutomationConnection is one of those special cmdlets available when running in an Automation Account. Conversely, Get-AutomationConnection will not work if the DSC is executing outside the context of an Automation Account. For more information on connections in Azure Automation, refer to https://docs.microsoft.com/en-us/azure/automation/automation-connections

Get-AutomationConnection returns an object containing all the necessary properties for us to establish a “full” connection to Azure using the Connect-AzAccount cmdlet:

Connect-AzAccount -Tenant $automationConnection.TenantID -ApplicationId $automationConnection.ApplicationID -CertificateThumbprint $automationConnection.CertificateThumbprint 

Note that for those of you that aren’t running this in the Azure public cloud (such as Azure Government or Azure Germany), you’ll also need to add an environment switch to point to the correct cloud environment (such as -Environment AzureUSGovernment)

At this point, we can run the az cmdlets to extract the secrets from the KeyVault:

$username = (Get-AzKeyVaultSecret -VaultName $keyVaultName -Name $usernameSecretName).SecretValueText

$password = (Get-AzKeyVaultSecret -VaultName $keyVaultName -Name $passwordSecretName).SecretValue

Full Example Configuration

configuration Workstation
{
	param
	(
		[Parameter(Mandatory)]
		[string] $keyVaultName,

		[Parameter(Mandatory)]
		[string] $usernameSecretName,

		[Parameter(Mandatory)]
		[string] $passwordSecretName,

		[Parameter(Mandatory)]
		[string] $automationConnectionName
	)

	Import-DscResource -ModuleName PSDesiredStateConfiguration

	$automationConnection = Get-AutomationConnection -Name $automationConnectionName
	Connect-AzAccount -Tenant $automationConnection.TenantID -ApplicationId $automationConnection.ApplicationID -CertificateThumbprint $automationConnection.CertificateThumbprint

	$username = (Get-AzKeyVaultSecret -VaultName $keyVaultName -Name $usernameSecretName).SecretValueText

	$password = (Get-AzKeyVaultSecret -VaultName $keyVaultName -Name $passwordSecretName).SecretValue

	$credentials = New-Object System.Management.Automation.PSCredential ($username, $password)

	Node SampleWorkstation
	{
		User NonAdminUser
		{
			UserName = $username
			Password = $credentials
		}
	}
}

Final Thoughts

Remember the nature of DSC compilation – all variables are resolved at compile-time and stored in the resulting MOF file that is stored in the automation account. The compiled MOF is what is actually downloaded and executed on the target Node/VM. This means that if you change one of the secret values in the KeyVault, the MOF will still contain the old values until you recompile the DSC.