Structure of a YAML Pipeline
There are a lot of topics to cover for YAML-based Pipelines in Azure DevOps. So, I will be breaking this up into a multi-part series. This section will be updated over time with links to the whole series of articles once they are written.
Part 1 - The Basics
Part 2 - Structure of a YAML Pipeline (you're reading it now!)
Part 3 - Pipeline Triggers
Part 4 - Pipeline Resources
Part 5 - Pools and Agents
Part 6 - Parameters and Variables
Part 7 - Stages, Jobs, and Steps
Basic Sections of a YAML Pipeline
To over-simply things, an Azure DevOps YAML Pipeline has 2 "sections":
Pipeline-level information
This includes items such as triggers, parameters, agent pools, and more
The actual work being done by the Pipeline
This is where the stages, jobs, and steps of the Pipeline are found
Advanced Tip: If you want to view the full, raw YAML schema for Pipelines, then use the following link (make sure to replace with your Azure DevOps Organization name)
Section 1: Pipeline-level Information
Possible options:
There's no way to cover each one of these in detail in one blog post. Most of these will be covered, in detail, in their own separate, dedicated post. But, for now, I will give a quick overview of each one.
name:
This is a name that you can give to each 'run' of the pipeline. This is not to be confused with the actual name of the pipeline itself (which is defined in the Azure DevOps UI)
You can use expressions in order to make sure that each run name is unique
This is optional. By default, a name of YYYYMMDD.XX is used where:
YYYYMMDD = the current date
XX = an iterator that starts at 1 and increments with each run
appendCommitMessageToRunName:
This determines if the latest Git commit message is appended to the end of the run name
This is optional. By default, the value is true
trigger:
This defines CI Triggers for the Pipeline
It monitors the repo where the YAML file is stored, and if there are any updates to the repo then it will trigger a run of the Pipeline
This is optional. Important: By default, any push to any branch of the repo will cause the Pipeline to be triggered
pr:
This defines PR Triggers for the Pipeline
It monitors the repo where the YAML file is stored, and if there are any Pull Requests on the repo then it will trigger a run of the Pipeline
This is optional. Important: By default, a PR on any branch of the repo will cause the Pipeline to be triggered
schedules:
This defines Scheduled Triggers for the Pipeline
One or more schedules can be defined, and the Pipeline will automatically trigger at those times
This is optional. By default, no scheduled runs of the Pipeline will occur
parameters:
Parameters that are defined at the Pipeline-level are also known as 'Runtime Parameters'
These are good for storing values that may vary or change between different runs of the Pipeline
When you manually run a Pipeline in the Azure DevOps UI, you will be able to change the value of each Runtime Parameter before you run the Pipeline
Parameters must specify a type, such as string, number, etc.
This is optional. By default, your Pipeline simply won't use any Runtime Parameters
variables:
Pipeline-level variables can be used anywhere throughout your Pipeline
Variables do not have a type, they are all stored as strings
By default, variables are mutable, and the value can be overwritten from run to run or job to job. This default behavior can be changed, however
This is optional. By default, your Pipeline simply won't have any Pipeline-level Variables (although they could still be defined at other levels)
pool:
This is where you define what type of Agent will run your Pipeline
You can use your own private self-hosted Agents, or you can use Microsoft-hosted Agents that run in the cloud. You can even use your own private Virtual Machine Scale Set that Azure DevOps will automatically scale for you
This is optional. By default, YAML Pipelines use a Microsoft-Hosted Agent running the 'ubuntu-latest' image.
resources:
Here you can define various different types of resources that can be used by your Pipeline. The 6 options are: builds, containers, packages, pipelines, respositories, and webhooks.
You define a Resource for 2 possible reasons:
You want to consume and use a Resource in your Pipeline
You want to trigger your Pipeline based on updates to a Resource
This is optional. By default, your Pipeline simply won't use any Resources
lockBehavior:
This is an advanced feature and deals with Exclusive Lock checks on protected pipeline resources
This is optional. By default, the value is runLatest
Section 2: The Work Being Done
Azure Pipelines breaks up the work into one or more Stages. Each Stage is then broken up into one or more Jobs. Finally, each Job is broken up into one or more Steps.
Many times you will be defining all 3 layers (Stages, Jobs, Steps) in your YAML. However, Azure Pipelines offers 2 different shortcut syntaxes that you can potentially use:
If you have a single Stage with multiple Jobs, then you can omit the 'stages' layer. You will still need to define the 'jobs' and 'steps' layers.
If you have a single Stage with a single Job, then you can omit both the 'stages' and 'jobs' layers. You will only need to define the 'steps' layer.
Simple example of defining Stages, Jobs, and Steps:
Simple example of the first shortcut, defining just Jobs and Steps:
Simple example of the second shortcut, defining just Steps:
Finally, there is one more possible option that can be defined at the Pipeline-level and that is extends. This is an advanced subject dealing with Templates, and I will cover this in a later post.
To summarize, we briefly discussed every option that can be used at the root of your Pipeline. We also briefly discussed the 3 different ways that you can define the work being done by your Pipeline.
Next up, we'll discuss all the Trigger options in depth.