Pipeline options for Git repositories

TFS 2017 | TFS 2015

While editing a pipeline that uses a Git repo—in an Azure DevOps project, GitHub, GitHub Enterprise Server, Bitbucket Cloud, or another Git repo—you have the following options.

Branch

TFS 2017 RTM and TFS 2015: This field is called Default branch.

This is the branch that you want to be the default when you manually queue this build. If you set a scheduled trigger for the build, this is the branch from which your build will get the latest sources. The default branch has no bearing when the build is triggered through continuous integration (CI). Usually you'll set this to be the same as the default branch of the repository (for example, "master").

Clean the local repo on the agent

You can perform different forms of cleaning the working directory of your self-hosted agent before a build runs.

In general, for faster performance of your self-hosted agents, don't clean the repo. In this case, to get the best performance, make sure you're also building incrementally by disabling any Clean option of the task or tool you're using to build.

If you do need to clean the repo (for example to avoid problems caused by residual files from a previous build), your options are below.

Select one of the following options:

• Sources: The build pipeline performs an undo of any changes in $(Build.SourcesDirectory). More specifically, the following Git commands are executed prior to fetching the source.

  git clean -ffdx
  git reset --hard HEAD
  

• Sources and output directory: Same operation as Sources option above, plus: Deletes and recreates $(Build.BinariesDirectory). Note that the $(Build.ArtifactStagingDirectory) and $(Common.TestResultsDirectory) are always deleted and recreated prior to every build regardless of any of these settings.

• Sources directory: Deletes and recreates $(Build.SourcesDirectory). This results in initializing a new, local Git repository for every build.

• All build directories: Deletes and recreates $(Agent.BuildDirectory). This results in initializing a new, local Git repository for every build.

TFS 2017 RTM

If you select True then the build pipeline performs an undo of any changes. If errors occur, then it deletes the contents of $(Build.SourcesDirectory).

If you want the Clean switch described above to work differently, then on the Variables tab, define the Build.Clean variable and set its value to:

• all if you want to delete $(Agent.BuildDirectory), which is the entire working folder that contains the sources folder, binaries folder, artifacts folder, and so on.

• source if you want to delete $(Build.SourcesDirectory).

• binary If you want to delete $(Build.BinariesDirectory).

Label sources

You may want to label your source code files to enable your team to easily identify which version of each file is included in the completed build. You also have the option to specify whether the source code should be labeled for all builds or only for successful builds.

In the Label format you can use user-defined and predefined variables that have a scope of "All." For example:

$(Build.DefinitionName)_$(Build.DefinitionVersion)_$(Build.BuildId)_$(Build.BuildNumber)_$(My.Variable)

The first four variables are predefined. My.Variable can be defined by you on the variables tab.

The build pipeline labels your sources with a Git tag.

Some build variables might yield a value that is not a valid label. For example, variables such as $(Build.RequestedFor) and $(Build.DefinitionName) can contain white space. If the value contains white space, the tag is not created.

After the sources are tagged by your build pipeline, an artifact with the Git ref refs/tags/{tag} is automatically added to the completed build. This gives your team additional traceability and a more user-friendly way to navigate from the build to the code that was built. The tag is considered a build artifact since it is produced by the build. When the build is deleted either manually or through a retention policy, the tag is also deleted.

Report build status (Azure Pipelines, TFS 2017 and newer)

You've got the option to give your team a view of the build status from your remote source repository.

If your sources are in an Azure Repos Git repository in your project, then this option displays a badge on the Code page to indicate whether the build is passing or failing. The build status is displayed in the following tabs:

• Files: Indicates the status of the latest build for the selected branch.
• Commits: Indicates the build status of the each commit (this requires the continuous integration (CI) trigger to be enabled for your builds).
• Branches: Indicates the status of the latest build for each branch.

If you use multiple build pipelines for the same repository in your project, then you may choose to enable this option for one or more of the pipelines. In the case when this option is enabled on multiple pipelines, the badge on the Code page indicates the status of the latest build across all the pipelines. Your team members can click the build status badge to view the latest build status for each one of the build pipelines.

Other types of Git remote repositories

If your source is in any other type of remote repository, then you cannot use Azure Pipelines or TFS to automatically publish the build status to that repository. However, you can use a build badge as a way to integrate and show build status within your version control experiences.

Checkout submodules

Select if you want to download files from submodules. You can either choose to get the immediate submodules or all submodules nested to any depth of recursion. If you want to use LFS with submodules, be sure to see the note about using LFS with submodules.

The build pipeline will check out your Git submodules as long as they are:

• Unauthenticated: A public, unauthenticated repo with no credentials required to clone or fetch.

• Authenticated:

  • Contained in the same project, GitHub organization, or Bitbucket Cloud account as the Git repo specified above.

  • Added by using a URL relative to the main repository. For example, this one would be checked out: git submodule add /../../submodule.git mymodule This one would not be checked out: git submodule add https://dev.azure.com/fabrikamfiber/_git/ConsoleApp mymodule

Authenticated submodules

The same credentials that are used by the agent to get the sources from the main repository are also used to get the sources for submodules.

If your main repository and submodules are in an Azure Repos Git repository in your Azure DevOps project, then you can select the account used to access the sources. On the Options tab, on the Build job authorization scope menu, select either:

• Project collection to use the Project Collection Build service account

• Current project to use the Project Build Service account.

Make sure that whichever account you use has access to both the main repository as well as the submodules.

If your main repository and submodules are in the same GitHub organization, then the token stored in the GitHub service connection is used to access the sources.

Alternative to using the Checkout submodules option

In some cases you can't use the Checkout submodules option. You might have a scenario where a different set of credentials are needed to access the submodules. This can happen, for example, if your main repository and submodule repositories aren't stored in the same Azure DevOps organization or Git service.

If you can't use the Checkout submodules option, then you can instead use a custom script step to fetch submodules. First, get a personal access token (PAT) and prefix it with pat:. Next, base64-encode this prefixed string to create a basic auth token. Finally, add this script to your pipeline:

git -c http.https://<url of submodule repository>.extraheader="AUTHORIZATION: basic <BASE64_ENCODED_TOKEN_DESCRIBED_ABOVE>" submodule update --init --recursive

Be sure to replace "<BASIC_AUTH_TOKEN>" with your Base64-encoded token.

Use a secret variable in your project or build pipeline to store the basic auth token that you generated. Use that variable to populate the secret in the above Git command.

Checkout files from LFS

Select if you want to download files from large file storage (LFS).

In the classic editor, select the check box to enable this option.

In a YAML build, add a checkout step with lfs set to true:

steps:
- checkout: self
  lfs: true

• TFS 2017 RTM and TFS 2015 (macOS and Linux only): On the Variables tab, set Agent.Source.Git.Lfs to true.

If you're using TFS, or if you're using Azure Pipelines with a self-hosted agent, then you must install git-lfs on the agent for this option to work. If your hosted agents use Windows, consider using the System.PreferGitFromPath variable to ensure that pipelines use the versions of git and git-lfs you installed on the machine.

Using Git LFS with submodules

If a submodule contains LFS files, Git LFS must be configured prior to checking out submodules. The Microsoft-hosted macOS and Linux agents come preconfigured this way. Windows agents and self-hosted macOS / Linux agents may not.

Clone a second repo

By default, your pipeline is associated with one repo from Azure Repos or an external provider. This is the repo that can trigger builds on commits and pull requests.

You may want to include sources from a second repo in your pipeline. You can do this by writing a script.

git clone https://github.com/Microsoft/TypeScript.git

If the repo is not public, you will need to pass authentication to the Git command.

Azure Repos

Your pipeline will already have access to other repos in its project, and you can clone them in your pipeline using a script command, as shown in the following example.

- script: | 
    git clone -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" https://organization@dev.azure.com/project/FabrikamFiber/_git/reponame

If you need to clone a repo from another project that is not public, you will need to authenticate as a user who has access to that project.

For Azure Repos, you can use a personal access token with the Code (Read) permission. Send this as the password field in a "Basic" authorization header without a username. (In other words, base64-encode the value of :<PAT>, including the colon.)

AUTH=$(echo -n ":$REPO_PAT" | openssl base64 | tr -d '\n')
git -c http.<repo URL>.extraheader="AUTHORIZATION: basic $AUTH" clone <repo URL> --no-checkout --branch master

Don't sync sources (TFS 2017 and newer only)

Non-deployment jobs automatically fetch sources. Use this option if you want to skip that behavior. This option can be useful in cases when you want to:

• Git init, config, and fetch using your own custom options.

• Use a build pipeline to just run automation (for example some scripts) that do not depend on code in version control.

If you want to disable downloading sources:

• Azure Pipelines, TFS 2017.2, and newer: Click Advanced settings, and then select Don't sync sources.

• TFS 2017 RTM: Define Build.SyncSources on the Variables and set its value to false.

Shallow fetch

Select if you want to limit how far back in history to download. Effectively this results in git fetch --depth=n. If your repository is large, this option might make your build pipeline more efficient. Your repository might be large if it has been in use for a long time and has sizeable history. It also might be large if you added and later deleted large files.

In these cases this option can help you conserve network and storage resources. It might also save time. The reason it doesn't always save time is because in some situations the server might need to spend time calculating the commits to download for the depth you specify.

Azure Pipelines, TFS 2018, TFS 2017.2

After you select the check box to enable this option, in the Depth box specify the number of commits.

TFS 2017 RTM, TFS 2015 (macOS and Linux only)

On the Variables tab, define Agent.Source.Git.ShallowFetchDepth and set its value to the number of commits in history you want to download. Specify 0 to set no limit.

Prefer Git from path

The Windows agent comes with its own copy of Git. If you prefer to supply your own Git rather than use the included copy, set System.PreferGitFromPath to true. This setting is always true on non-Windows agents.

Trigger Options for Other Git

When using an Other/external Git repository, CI builds require that the repository is accessible from the internet. If the repository is behind a firewall or proxy, then only scheduled and manual builds will work.

FAQ

What protocols can the build agent use with Git?

The agent supports HTTPS.

The agent does not yet support SSH. See Allow build to use SSH authentication while checking out Git submodules.

I use TFS on-premises and I don't see some of these features. Why not?

Some of these features are available only on Azure Pipelines and not yet available on-premises. Some features are available on-premises if you have upgraded to the latest version of TFS.