Skip to content

Getting started Scenario 2 Onboard module library and CI environment

Jump to bottom
CARMLPipelinePrincipal edited this page Aug 29, 2022 · 28 revisions

The repository is built so that you can create your own 1:1 instance and thus reuse the same concepts and features in your own environment, such as GitHub.

This requires several steps:

  1. Configure your Azure environment
  2. Fork/clone the repository into your DevOps environment
  3. Configure the CI environment
  4. Deploy dependencies
  5. Update module test files
  6. (Optional) Convert library to ARM

Depending on the DevOps environment you choose (GitHub or Azure DevOps), make sure you also account for the specific requirements outlined below.

1. Configure your Azure environment

CARML tests the deployments and stores the module artifacts in an Azure subscription. To do so, it requires a service principal with access to it.

In this first step, make sure you

  • Have/create an Azure Active Directory Service Principal with at least Contributor & User Access Administrator permissions on the Management-Group/Subscription you want to test the modules in
  • Note down the following pieces of information
    • Application (Client) ID
    • Service Principal Object ID (not the object ID of the application)
    • Service Principal Secret (password)
    • Tenant ID
    • Subscription ID
    • Parent Management Group ID

Note: The Service Principal must be able to query its own details in the Azure Active Directory (AAD). To that end, ensure it has at least the (default) role 'Cloud application administrator'.

2. Fork/clone the repository into your DevOps environment

Next, you'll want to create your own copy of the code. Depending on the repository environment you want to use (GitHub or Azure DevOps), the setup will be slightly different.

Note: Whether you chose GitHub or Azure DevOps as your repository's environment, it does not affect your options when registering the pipelines.

Note: If you don't want to use all modules, you can remove those that should not be part of your library. However, when doing so, make sure you use the utility Get-CrossReferencedModuleList with the switch parameter PrintLocalReferencesOnly to check for any cross-module references. For example, you may find that when you'd remove the 'Microsoft.Network/privateEndpoints', that it is still referenced by some of the modules you may want to use (for example, 'Microsoft.KeyVault/vaults'). In those cases, make sure to not accidentally delete required references.

GitHub Repository

For GitHub, you have two choices depending on your planned repository visibility:

  • In case of a public repository, we recommend to create a simple fork into the target organization. As the CARML source repository is public, a fork must be public too.
  • If you need a private version instead, we recommend you create your target repository, download/clone the CARML repository (ref. 'how to' below) and upload the content to the created target repository

    Note: This disables the feature to 'fetch' from the upstream (CARML) repository. As a result, you have to port upstream updates manually.

Azure DevOps Repository

For a private Azure DevOps git, we recommend you create your target repository, download/clone the CARML repository (ref. 'how to' below) and upload the content to the created target repository.

Note: This disables the feature to 'fetch' from the upstream (CARML) repository. As a result, you have to port upstream updates manually.

How to: Clone/download the repository To save a local copy of the repository, you can either clone the repository or download it as a `.zip` file. A clone is a direct reference to the source repository which enables you to pull updates as they happen in the source repository. To achieve this, you have to have `Git` installed and run the following command: 
  git clone 'https://github.com/Azure/ResourceModules.git'

from a command-line of your choice (e.g., PowerShell).

If you just want to have a copy of the repository's content, you can instead download it in .zip format. You can do this by navigating to the repository folder of your choice (for example, root), then select the <> Code button on the top left and click on Download ZIP on the opening blade.

How to download repository

Alternatively, you can also do the same with a specific release by navigating to the releases page, scroll to the 'Assets' section at the bottom end of the release you'd like to get and download the packaged release (as it was when the release was created) with a simple click on the 'Source code' package (e.g., Source code (zip)) itself.

Download zip

3. Configure the CI environment

CARML uses a single (settings.yml) file for configuring the CI environment. To replicate the CI environment locally on your machine, and perform local modules tests and validations, you must install the (powershell-yaml) module from the PowerShell gallery by executing the following on your PowerShell CLI:

Install-Module -Name powershell-yaml

To configure the CI environment you have to perform several steps:

Note: While you can use the browser, we recommend that you clone all files to your local machine and update them using, for example, Visual Studio Code.

3.1 Update default namePrefix

To lower the barrier to entry and allow users to easily define their own naming conventions, we introduced a default 'name prefix' for all deployed resources.

Note: This prefix is only used by the CI environment you validate your modules in, and doesn't affect the naming of any resources you deploy as part of any multi-module solutions (applications/workloads) based on the modules.

Each pipeline in CARML deploying resources uses a logic that automatically replaces "tokens" (i.e., placeholders) in any module test file. Tokens are stored in only a few central locations to facilitate maintenance (e.g., local settings.yml, repository secrets or variable groups).

To update the namePrefix, perform the following steps:

  1. Open the settings.yml file in the repository's root directory.

  2. Replace the value of the localToken_namePrefix with a different value:

    localToken_namePrefix: 'cntso'

    Note: The value should be a 3-5 character long string like cntso. Longer strings are not recommended as they may conflict with Azure resource name length restrictions.

    Note: The CI pipelines automatically removes the localToken_ prefix from the name when processing the tokens replacement.

    Note: We highly encourage you to use the 'Check namePrefix availability' script (see the documentation here) to check if the intended resource name will be available, based on the provided prefix.

For further information on the token replacement logic, please refer to the corresponding Token replacement section.

If you do not prefer to set the namePrefix locally for your CI environment. You can optionally set it as a GitHub Secret or add it to the ADO variable group as the key TOKEN_NAMEPREFIX and its value your name prefix value, and this is only applied if the localToken_namePrefix in the settings.yml is left empty.

3.2 Set up CI-environment-specific configuration

While the concepts are the same, the configuration of the CI environment can differ drastically depending on the DevOps environment in which you want to register and run your pipelines. Following, you can find instructions on how to perform the remaining configuration in the corresponding DevOps environment:

GitHub

For GitHub, you have to perform the following environment-specific steps:

3.2.1 Set up secrets

To use the environment's pipelines you should use the information you gathered during the Azure setup to set up the following repository secrets:

Secret Name Example Description
ARM_MGMTGROUP_ID de33a0e7-64d9-4a94-8fe9-b018cedf1e05 The group ID of the management group to test-deploy modules in.
ARM_SUBSCRIPTION_ID d0312b25-9160-4550-914f-8738d9b5caf5 The ID of the subscription to test-deploy modules in.
ARM_TENANT_ID 9734cec9-4384-445b-bbb6-767e7be6e5ec The tenant ID of the Azure Active Directory tenant to test-deploy modules in.
AZURE_CREDENTIALS {"clientId": "4ce8ce4c-cac0-48eb-b815-65e5763e2929", "clientSecret": "<placeholder>", "subscriptionId": "d0312b25-9160-4550-914f-8738d9b5caf5", "tenantId": "9734cec9-4384-445b-bbb6-767e7be6e5ec" } The login credentials of the deployment principal used to log into the target Azure environment to test in. The format is described here.
PLATFORM_REPO_UPDATE_PAT <placeholder> A private access token (PAT) with enough permissions assigned to it to push into the main branch. This PAT is leveraged by pipelines that automatically generate ReadMe files to keep them up to date.
TOKEN_NAMEPREFIX cntso Optional. If you specify the name prefix token here, it is only applied if the localToken_namePrefix specified in the settings.yml is left empty.

How to: Add a repository secret to GitHub

  1. Navigate to the repository's Settings.

    Navigate to settings

  2. In the list of settings, expand Secrets and select Actions. You can create a new repository secret by selecting New repository secret on the top right.

    Navigate to secrets

  3. In the opening view, you can create a secret by providing a secret Name, a secret Value, followed by a click on the Add secret button.

    Add secret

Special case: AZURE_CREDENTIALS, This secret represent the service connection to Azure, and its value is a compressed JSON object that must match the following format:

{"clientId": "<client_id>", "clientSecret": "<client_secret>", "subscriptionId": "<subscriptionId>", "tenantId": "<tenant_id>" }

Make sure you create this object as one continuous string as shown above - using the information you collected during Step 1. Failing to format the secret as above, causes GitHub to consider each line of the json object as a separate secret string. If you're interested, you can find more information about this object here.

3.2.2 Set up settings file

The primary pipeline settings file (settings.yml) hosts the fundamental pipeline configuration. In the file you will find and can configure settings such as:

General
Variable Name Example Value Description
location "WestEurope" The default location to deploy resources to and store deployment metadata at. If no location is specified in the deploying module test file, this location is used.
resourceGroupName "validation-rg" The resource group to deploy all resources for validation to.
Template-specs specific (publishing)
Variable Name Example Value Description
templateSpecsRGName "artifacts-rg" The resource group to host the created template-specs.
templateSpecsRGLocation "WestEurope" The location of the resource group to host the template-specs. Is used to create a new resource group if not yet existing.
templateSpecsDescription "This is a module from the [Common Azure Resource Modules Library]" A description to add to the published template specs.
templateSpecsDoPublish "true" A central switch to enable/disable publishing to template-specs.
Private Bicep registry specific (publishing)
Variable Name Example Value Description
bicepRegistryName "adpsxxazacrx001" The container registry to publish Bicep templates to.

NOTE: Must be globally unique.
bicepRegistryRGName "artifacts-rg" The resource group of the container registry to publish Bicep templates into. It is used to create a new container registry if not yet existing.
bicepRegistryRgLocation 'West Europe' The location of the resource group of the container registry to publish Bicep templates into. Is used to create a new resource group if not yet existing.
bicepRegistryDoPublish "true" A central switch to enable/disable publishing to the private Bicep registry.

NOTE: If you plan to use the private container registry for Bicep, make sure to update its value in bicepRegistryName, as it must be globally unique.

NOTE: If you are uncertain which publishing option to choose, this section may be of help.

3.2.3 Enable actions

Finally, 'GitHub Actions' are disabled by default and hence, must be enabled first.

To do so, perform the following steps:

  1. Navigate to the Actions tab on the top of the repository page.

  2. Next, select 'I understand my workflows, go ahead and enable them'.

    Enable Actions

3.2.4 Set R/W Workflow permissions

To let the workflow engine publish their results into your repository, you have to enable the read / write access for the GitHub actions.

  1. Navigate to the Settings tab on the top of your repository page.

  2. Within the section Code and automation click on Actions and General

  3. Make sure to enable Read and write permissions

    Workflow Permissions

Azure DevOps

For Azure DevOps, you have to perform the following environment-specific steps:

3.2.1 Set up service connection

The service connection must be set up in the project's settings under Pipelines: Service connections (a step by step guide can be found here).

It's name must match the one configured as serviceConnection in the variable file's 'General' section.

3.2.2 Set up secrets in variable group

The variable group PLATFORM_VARIABLES must be set up in Azure DevOps as described here.

Based on the information you gathered in the Azure setup, you must configure the following secrets in the variable group:

Secret Name Example Description
ARM_MGMTGROUP_ID de33a0e7-64d9-4a94-8fe9-b018cedf1e05 The group ID of the management group to test-deploy modules in.
ARM_SUBSCRIPTION_ID d0312b25-9160-4550-914f-8738d9b5caf5 The ID of the subscription to test-deploy modules in.
ARM_TENANT_ID 9734cec9-4384-445b-bbb6-767e7be6e5ec The tenant ID of the Azure Active Directory tenant to test-deploy modules in.
TOKEN_NAMEPREFIX <cntso> Optional. If you specify the name prefix token here, it is only applied if the localToken_namePrefix specified in the settings.yml is left empty.

Make sure its name matches the group reference used in the module pipelines. For example

variables:
  - group: 'PLATFORM_VARIABLES'

Note: If you need to use different name than PLATFORM_VARIABLES, make sure to search & replace all references with the new name.

3.2.3 Set up settings file

The primary pipeline settings file (settings.yml) hosts the fundamental pipeline configuration. In the file, you will find and can configure information such as:

General
Variable Name Example Value Description
location 'WestEurope' The default location to deploy resources to. If no location is specified in the deploying module test file, this location is used.
resourceGroupName 'validation-rg' The resource group to deploy all resources for validation into.
serviceConnection 'Contoso-Connection' The service connection that points to the subscription to test in and publish to.
Template-specs specific (publishing)
Variable Name Example Value Description
templateSpecsRGName 'artifacts-rg' The resource group to host the created template-specs.
templateSpecsRGLocation 'WestEurope' The location of the resource group to host the template-specs. Is used to create a new resource group if not yet existing.
templateSpecsDescription 'This is a module from the [Common Azure Resource Modules Library]' A description to add to the published template specs.
templateSpecsDoPublish 'true' A central switch to enable/disable publishing to template-specs.
Private Bicep registry specific (publishing)
Variable Name Example Value Description
bicepRegistryName 'adpsxxazacrx001' The container registry to publish Bicep templates to.

NOTE: Must be globally unique.
bicepRegistryRGName 'artifacts-rg' The resource group of the container registry to publish Bicep templates to. Is used to create a new container registry if not yet existing.
bicepRegistryRgLocation 'West Europe' The location of the resource group of the container registry to publish Bicep templates to. Is used to create a new resource group if not yet existing.
bicepRegistryDoPublish 'true' A central switch to enable/disable publishing to the private Bicep registry.
Universal packages specific (publishing)
Variable Name Example Value Description
vstsFeedName 'ResourceModules' The name of the Azure DevOps universal packages feed to publish to.
vstsFeedProject '$(System.TeamProject)' The project that hosts the feed. The feed must be created in Azure DevOps ahead of time.
vstsFeedToken '$(System.AccessToken)' The token used to publish universal packages into the feed above.
artifactsFeedDoPublish 'true' A central switch to enable/disable publishing to Universal packages.

NOTE: If you plan to use the private container registry for Bicep, make sure to update its value in bicepRegistryName as it must be globally unique.

NOTE: If you are uncertain which publishing option to choose, this section may be of help.

3.2.4 Register pipelines

To use the pipelines that come with the environment in Azure DevOps, you need to register them first. You can either do this manually, or, run the utility Register-AzureDevOpsPipeline we provide in utilities/tools/AzureDevOps. For further information, please refer to the corresponding documentation.

3.2.5 Azure Artifacts Universal Packages

This section will explain what is required to publish the modules to Azure Artifacts Universal Packages. It will also assume you are publishing from Azure DevOps Pipelines.

The dependent components are

  1. An Azure DevOps organization and project
  2. An Azure DevOps artifacts feed

    Note: The default feed name is ResourceModules as configured in the settings.yml file's variable vstsFeedName. Update the value here if you want to use a different name, but make sure it matches the name of the artifact feed created in Azure DevOps.

  3. An Azure DevOps project to host the artifact feed

    Note: There are a couple options to consider when setting up an Azure Artifact feed. For example, organization-scoped feeds vs project-scoped feeds. Please see what option suits your needs by reviewing the feeds document first.

  4. If you chose the feed to be project-scoped, you will need the Project Build Service account to have Contributor access to publish to the Azure Artifacts feed. To set this, follow the Pipeline permission steps.

Implementation Guidance

Each ./azuredevops/modulePipelines YAML pipeline already calls /.azuredevops/pipelineTemplates/jobs.publishModule.yml. This YAML template contains a method to Publish module to artifacts feed via utilities\pipelines\resourcePublish\Publish-ModuleToUniversalArtifactFeed.ps1.

4. Deploy dependencies

In order to successfully deploy and test all modules in your desired environment, some modules require resources to be deployed beforehand.

The repository comes with a platform pipeline, i.e., the 'dependencies pipeline', that deploys a set of Azure services such as Virtual Networks and Key Vaults (along with dummy secrets) to be used by the module pipeline tests.

Run the dependencies pipeline by following instructions provided in the specific Dependencies pipeline usage section.

Note: For details about the dependencies pipeline design, please refer to the dedicated Dependencies pipeline design section.

4.1 Manual Dependencies

In special cases, manual actions may be required to provision certain resources that are not covered by the dependencies pipeline. In the following, you can find an overview of these resources, for which modules you need them and what you need to do:

Microsoft.Web/sites

To successfully deploy the sites module using the fa.parameters.json parameter file, you need to create an Azure Active Directory App with its API endpoint enabled (e.g., api://<app id>) and add a secret. The secret value needs then to be stored in a Key Vault secret.

5. Update module test files

Once the required dependencies are deployed, there is one more step left to get as many module pipelines running as possible.

Several module parameters reference resources with unique values. For example, if a module references a Key Vault key, its version identifier will only be available once the dependencies pipeline ran once.

For this reason, make sure to update the references in the following modules once the dependencies pipeline concluded:

File Parameter Notes
modules\Microsoft.Compute\virtualMachines\.test\linux.parameters.json extensionDiskEncryptionConfig.value.settings.KeyEncryptionKeyURL
modules\Microsoft.Compute\virtualMachines\.test\windows.parameters.json extensionDiskEncryptionConfig.value.settings.KeyEncryptionKeyURL
modules\Microsoft.Compute\virtualMachineScaleSets\.test\linux.parameters.json extensionDiskEncryptionConfig.value.settings.KeyEncryptionKeyURL
modules\Microsoft.Compute\virtualMachineScaleSets\.test\windows.parameters.json extensionDiskEncryptionConfig.value.settings.KeyEncryptionKeyURL
modules\Microsoft.Sql\managedInstances\.test\parameters.json keys.value.uri
modules\Microsoft.Network\applicationGateways\.test\parameters.json sslCertificates.value.properties.keyVaultSecretId
modules\Microsoft.Web\sites\.test\fa.parameters.json appSettingsKeyValuePairs.value.EASYAUTH_SECRET Key Vault secret URI without version
modules\Microsoft.Web\sites\.test\fa.parameters.json authSettingV2Configuration.value.identityProviders.azureActiveDirectory.registration.clientId App ID from the Azure Active Directory App
modules\Microsoft.Web\sites\.test\fa.parameters.json authSettingV2Configuration.value.identityProviders.azureActiveDirectory.validation.allowedAudiences API endpoint from the Azure Active Directory app

6. (Optional) Convert library to ARM

Note that in case you don't want to use Bicep, you always have the option to use the utility ConvertTo-ARMTemplate we provide in path utilities/tools to convert the repository to an ARM-only repository. Due to the way Bicep works and the CI environment is set up, you should be able to use it with ARM templates in the same way as you would when using Bicep. For further information on how to use the tool, please refer to the tool-specific documentation.

This wiki is being actively developed If you discover any documentation bugs or would like to request new content, please raise them as an issue.

Contributions to this wiki are done through the main repo under docs/wiki.

Uncyclo content

Clone this wiki locally