Introduction
On Saturday, April 26, 2025, Grafana Labs disclosed that an unauthorized user leveraged a vulnerability in a GitHub Actions workflow within a public Grafana Labs repository. This led to the exposure of a small number of secrets. Grafana Labs stated that their detections immediately triggered alerts, and the team responded by mitigating the vulnerability, rotating keys, and verifying that there was no access to production systems or data.
Grafana Labs plans to share more detailed information in a forthcoming blog post.
Independent security researcher Adnan Khan shared information about this incident publicly on Twitter.
Timeline of Key Updates
April 18, 2025 10:00 PM UTC – Grafana Labs officially disclosed the incident on LinkedIn.
Observed Details
Initial Access: Exploitation of Pwn Request
This is most likely how the initial credential was compromised. A GitHub Actions workflow named pr-patch-check-event.yml in the grafana/grafana repository was configured to trigger on pull_request_target events and contained a Pwn Request vulnerablity .The workflow contained the following behavior.
Trigger
pull_request_target
Steps
- Generate a GitHub App token using the tibdex/github-app-token action.
- Use the GitHub App token to dispatch a workflow in another repository (grafana/security-patch-actions).
Secrets
The secrets GRAFANA_DELIVERY_BOT_APP_ID and GRAFANA_DELIVERY_BOT_APP_PEM were used to generate the token. Due to the use of the pull_request_target trigger, these secrets were available to the workflow even when pull requests were opened from external forks.
This is most likely how the initial credential was compromised. The attacker exploited a script injection vulnerability by crafting a branch name that escaped out of the literal context and executed JavaScript to exfiltrate the credential.
What is Pwn Request Vulnerability?
A Pwn Request is a vulnerability in GitHub Actions where a workflow kicked off by an external pull request runs with repository-level permissions and injected secrets. If the workflow references attacker-controlled input, such as the branch name, without proper sanitization, the untrusted code can execute during the CI run and steal CI/CD credentials and backdoor production software builds.
In the Grafana incident, the pr-patch-check-event.yml workflow executed on the pull_request_target trigger, which automatically supplied two GitHub-App secrets to every run. An external contributor most likely crafted a branch name that escaped its expected context and ran a short script that exfiltrated those secrets. The compromise occurred during the very first workflow run launched by the untrusted pull request; no merge or further privilege escalation was required.
For a full primer on Pwn Request attacks and hardening guidance, see GitHub Actions "Pwn Request" Vulnerability.
Subsequent Actions: Pushing a Malicious Workflow
You can view an interactive demo of the exploit code below.
After obtaining access to the bot credentials, the attacker used the GitHub App token to push a malicious workflow into the grafana/grafana repository. The malicious commit added a workflow named hrgqavynjp with the following behavior: - Trigger: Pushes to a branch named hrgqavynjp.
Steps
- Serialize all available GitHub Actions secrets into a file.
- Encrypt the secrets using AES-256-CBC encryption.
- Encrypt the AES key using a hardcoded RSA public key.
- Upload both the encrypted secrets and the encrypted key as GitHub Actions artifacts.
Activity Evidence
The GitHub activity feed for the grafana/grafana repository shows that the grafana-delivery-bot account created a new branch (hrgqavynjp) and made a commit adding the malicious workflow, followed by deletion of the branch.
Mitigation strategies
The following section describes security best practices to avoid such GitHub Actions security incidents.
Avoid risky triggers wherever possible
Where feasible, replace high-risk triggers such as pull_request_target with safer alternatives such as pull_request and workflow_dispatch so that untrusted code from forks never runs with elevated privileges or inherited secrets. If the risky trigger is truly required (for example, to lint patches from forks), scope its permissions narrowly and run it on an isolated runner that has no access to production credentials.
Audit secrets – especially the ones used by risky-trigger workflows
Compile an inventory of every secret referenced in workflows that use, reusable workflows called from forks, or any job that runs on self-hosted runners. Validate that each secret is still needed, rotate it regularly, and monitor its usage. Treat any unexplained access as a potential compromise and investigate immediately.
Leverage Environment Secrets and Mandatory Reviews for Production Secrets
Environment secrets are stored at an environment level. They enable you to implement guardrails by enforcing mandatory reviews. This means they can only be accessed by an authorized reviewer who can approve the GitHub Actions workflow run. No job will have access to these secrets without the approval of the reviewer.
Environment secrets also allow you to implement other quality gates including static code analyzers that can flag issues and ensure the absence of vulnerable dependencies in the GitHub workflow.
Enable network and runtime monitoring (Harden-Runner)
Static checks can miss novel attacks, so enable continuous runtime monitoring on every runner. StepSecurity Harden-Runner inspects network calls, file writes, and process activity during the job and raises real-time detections if it sees suspicious behavior (for example, mass-export of secrets or outbound traffic to an unexpected host). These signals give responders precious minutes to revoke tokens and quarantine artifacts before damage spreads. Harden-Runner community tier is free for open-source projects.
Use least-privileged GitHub App permissions
Even if the compromised bot in this incident was not over-privileged, limiting a GitHub App to the minimum permissions and repository access it legitimately requires shrinks the blast radius of any credential leak. Review each permission (content, issues, workflows, administration, etc.) and set it to Read-only or remove it entirely unless it is demonstrably needed. Re-evaluate scopes whenever the app’s functionality changes.
Require approval for workflow runs from public fork
This is a must-have setting for public repositories that use GitHub Actions. In your public GitHub organization / repository, you can select when GitHub Action should wait for explicit approval before running workflows for forked pull request.
Current Status
As of April 28, 2025:
- GitHub Actions have been disabled across all public Grafana Labs repositories.
- Grafana Labs continues internal investigations and has stated that a full blog post with further information will be published.
.png)
