Azure DevOps

Access Azure Key Vault secrets using a Service Principle with certificate within an Azure DevOps pipeline

Azure Key Vault is a cloud service that provides secure storage for secrets, keys, and certificates of your applications. Integrating Key Vault with Azure DevOps pipelines enhances security by managing sensitive information efficiently.

This article will guide you through the process of accessing secrets from an Azure Key Vault using a Service Principal with a certificate within an Azure pipeline. We will start for the very first steps of creating all the needed resources in Azure, before going back to Azure DevOps. For training reasons, we will perform all the configuration steps with CLI commands, avoiding the Azure Portal UI.

1. Create a new resource group in Azure with Azure CLI

az group create --name TestResourceGroup --location switzerlandnorth

2. Create a new Key Vault

Use the following command and replace the name and location with your preferred values. The name of the Key Vault must be globally unique, so it is possible that the name you chose is already taken:

az keyvault create --name ThisIsANewTestKeyVault --resource-group TestResourceGroup --location switzerlandnorth

3. Add a new role in the resource group

This is an important step, as without the new role permission you will receive the Caller is not authorized to perform action on resource or the The operation is not allowed by RBAC errors when you try to perform operations on the Key Vault, such as adding new secrets.

In the following command:

az role assignment create \
  --assignee "XXX#EXT#@XXX.onmicrosoft.com" \
  --role "Key Vault Administrator" \
  --scope /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/TestResourceGroup/providers/Microsoft.KeyVault/vaults/ThisIsNewATestKeyVault

4. Create a new Service Principal which uses a certificate

We will create a new identity for accessing our Key Vault from our Azure DevOps account. We can achieve that by creating a new Service Principal. We will use certificate authentication, as recommended by Microsoft, instead of a password.

Pay attention to the following points before running the command:

az ad sp create-for-rbac \
	--name TestKeyVaultServicePrincipal \
	--role "Key Vault Secrets User" \
	--scopes /subscriptions/3a916bd3-e232-4120-bb00-ea087ee791c0/resourceGroups/TestResourceGroup/providers/Microsoft.KeyVault/vaults/ThisIsANewTestKeyVault \
	--create-cert \
	--cert TestCertificate \
	--keyvault ThisIsANewTestKeyVault

After the command executes, keep the resulting JSON object somewhere safe, as we will need its values in the next steps.

5. Download the .pem certificate on your computer

We will use the content of the created certificate when we wire Azure DevOps to our Key Vault. To do this, we will create a .pem file containing the value of the certificate created in step 4.

First download the stored certificate from the Key Vault:

az keyvault secret download \
  --file cert.pfx \
  --vault-name ThisIsANewTestKeyVault \
  --name TestCertificate \
  --encoding base64

Then, create a .pem certificate from the downloaded .pfx file:

 openssl pkcs12 -in cert.pfx -passin pass: -passout pass: -out cert.pem -nodes

The certificate is now stored in the cert.pem You can view its contents with the cat cert.pem command.

6. Add a new secret into the Key Vault

Use the following command to add a new secret. We will retrieve this secret in our Azure pipeline shortly. Feel free to choose another dummy name and value:

az keyvault secret set --vault-name ThisIsNewATestKeyVault --name "TestSecret" --value "123456789"

7. Add a new Service Connection in Azure DevOps

8. Create a new Azure DevOps pipeline

We will now create a new pipeline and access the Key Vault from it. In your Azure DevOps project, navigate to Pipelines > Pipelines and click on New Pipeline. Add the following configuration in the newly opened YAML editor.

trigger:
- master

resources:
- repo: self

stages:
- stage: Prepare
  displayName: Prepare
  jobs:
  - job: Secrets
    displayName: Get Secrets
    steps:
      - task: AzureKeyVault@2
        inputs:
          azureSubscription: 'KeyVaultWithCertificate'
          KeyVaultName: 'ThisIsANewTestKeyVault'
          SecretsFilter: '*'
          RunAsPreJob: false
      - script: |
          echo "##vso[task.setvariable variable=TestSecret;isOutput=true]$(TestSecret)"
        name: SecretVariables

Save the pipeline and check for the first automated run. The connection to Azure should work.

9. Access the stored variable from another job

A common scenario after retrieving secrets and storing them in variables is to access their values from other jobs. To achieve this, edit your pipeline and add an extra dummy job that echoes the value of the TestSecret secret. Add the following YAML configuration at the end of the pipeline:

  - job: EchoTestSecret
    displayName: Echoes the value of TestSecret
    variables:
      varFromJobSecrets: $[ dependencies.Secrets.outputs['SecretVariables.TestSecret'] ]
    steps:
      - bash: echo $(varFromJobSecrets)

Conclusion

This guide has described all the necessary steps for creating a Key Vault, a new Service Principal with certificate and an Azure pipeline which uses all this information. Feel free to leave a comment if you liked my step-by-step guide.

comments powered by Disqus