In this article I will explain how to properly configure the permissions and secrets in a GitLab CI pipeline to pull other projects hosted in the same Gitlab instance. I will also explain how to activate and use the job token scope setting.

The problem

If you are using Terraform, you may have been stuck with a problem using modules hosted in different repositories than your core stack, in particular, with CI/CD deployment.

Let’s start with an example: when using the below syntax, you ask Terraform to pull the modules from another git repository before running the plan:

module "scoreboard" {
  source          = "git::https://gitlab.example.com/scoreboard.git"
  bucket_prefix   = "template"
  scoring_api_url = module.scoring.api_url
  admin_api_url   = module.admin.api_url
}

To have Gitlab pulling the sources, assuming you can only use HTTP and not SSH, you can put a .git-credentials in your runner with a static token, or generate a .git-credentials file with a global read token from the CI like this:

echo "https://my-user:my-token@gitlab.example.com" > ~/.git-credentials

However, this token could be stolen, require frequent rotations, and will give access to all your repositories, which is also not acceptable.

Ideally, you would like to use the dynamic CI_JOB_TOKEN, which is a protected Gitlab variable, and insert it in URLs like that:

module "scoreboard" {
  source          = "git::https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.example.com/scoreboard.git"
  bucket_prefix   = "template"
  scoring_api_url = module.scoring.api_url
  admin_api_url   = module.admin.api_url
}

However, this solution is also not ideal, because with this setup you can no longer perform a terraform plan locally, and it prevents testing your source code.

Patching the git URL

Fortunately, there is a very cool parameter in git configuration settings called insteadOf.

Using the setting insteadOf will allow you to transparently replace portions of the URL with something else, without changing your source code. It is really useful when dealing with migrated projects, but it can also tackle our authentication problem. In particular, I can ask git to replace the URL of my Gitlab instance with the prefix containing the job’s credentials. 

Here is an example:

.terraform_validate:
  image:
    name: hashicorp/terraform
    entrypoint: [""]
  stage: validate
  before_script:
    - git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.example.com".insteadOf "https://gitlab.example.com"
  script:
    - terraform init
    - terraform fmt -check -recursive
    - terraform validate

When running this step, git will add in the .gitconfig a line containing my ephemeral job token, and every git operation (including Terraform ones) will have the ephemeral CI_JOB_TOKEN injected for authentication.

Is that all ?

On legacy projects, this setup *may* work without any further configuration, however, I recommend you to continue reading because you may have a security problem here.

If this setup works without any additional configuration, you may be in one of the following cases:

• The target module is a public or internal project (it’s ok)
• The target module is private, and you have not activated the job token access setting. This particular case is a security concern.

Actually, the CI_JOB_TOKEN pushed in the job is tied to your user. Historically, in Gitlab, when you had access to a project, the CI assumes that any pipeline you run should have the same access level as you. As a consequence, a pipeline started by your user was able to checkout any project you had access to, at the time of the build.

Unfortunately, pipelines are not humans, and somebody hacking a pipeline by injecting malicious instructions into your .gitlab-ci (potentially from an external yaml you have included) will be able to have access to ALL the private repositories your user have access to, and it is not good. To prevent that, Gilab introduced the job token access setting.

What is the job token access setting ?

Gitlab job token access is a feature allowing to create a whitelist of projects allowed to use another gitlab’s project.

The documentation about job token access explains it as below: 

For example, project A can add project B to the allowlist. CI/CD jobs in project B (the “allowed project”) can now use their CI/CD job token to authenticate API calls to access project A. If project A is public or internal, the project can be accessed by project B without adding it to the allowlist.

Basically, you can now create trust relationships between projects, without the need to rely on the permissions of the user starting the pipeline. If you start a pipeline which tries to pull a private module having job token access enabled, it will not work if the source project is not explicitly allowed in the configuration, even if the user starting this pipeline has access to both projects.

How to take advantage of this setting ?

The configuration is really straightforward, you just need to activate this parameter in the project’s setting, and list the projects you would like to allow like in this screenshot:

There is also an API to do that programmatically: https://docs.gitlab.com/ee/api/project_job_token_scopes.html

Conclusion

You have learnt how to give access from a Gitlab pipeline to another project, with a Terraform example. Additionally, you currently know what the job token access setting is, how to use it, and why it is useful.