Detecting Anomalous PR Patterns in OSS Projects via GitHub Audit Logs

The Problem

The xz-utils supply chain attack — discovered in March 2024 — is the defining case study for long-game OSS compromise. The attacker operated under the alias “Jia Tan” for approximately two years before inserting a backdoor into the xz compression library that ships in nearly every Linux distribution. During those two years, the account built genuine contributor trust: it submitted real bug fixes, participated in issue discussions, and demonstrated patience that automated scanning tools are not built to detect.

What makes the xz attack instructive for detection engineering is that the behavioural pattern was, in retrospect, observable at the event level. Pull requests were submitted and merged at times outside the maintainer’s apparent timezone. Pressure was applied to accelerate commit access. Once commit access was granted, the attacker moved quickly to merge changes that altered the build system — the precise changes where a backdoor was concealed. None of this was detected in real time.

GitHub’s audit log captures all of these events. Every PR submission, every review approval, every merge, every team membership change, every CODEOWNERS modification is recorded with a precise timestamp and the actor who performed it. For GitHub Enterprise organizations the audit log is available as a streaming API endpoint, and for public repositories the REST API exposes a significant subset of events.

The structural problem is that almost no OSS project actually monitors this data. Audit log access requires organization-level permissions. Most projects run under individual accounts rather than GitHub organizations, making the audit log API inaccessible. Even projects that do run as organizations typically treat the audit log as a forensic resource — something to query after an incident — rather than a real-time detection feed.

The signal types that distinguish supply chain preparation from normal contributor activity are consistent across documented attacks:

• Outside-hours merges: the attacker’s local timezone differs from the project maintainer’s timezone, causing merge activity to cluster at unusual hours relative to historical patterns
• Self-merge without independent review: a contributor merges their own PR, or the review comes only from accounts that are themselves new contributors
• Accelerated privilege escalation: a contributor goes from first commit to CODEOWNERS or maintainer team membership in a compressed timeframe
• Bulk merge bursts: a large number of PRs merged in a short window, suggesting hurried activity or automated tooling
• Sudden contributor silence post-escalation: an active contributor who suddenly stops commenting on issues once commit access is obtained, consistent with a mission accomplished
• Review approvals from accounts with no prior engagement: sock-puppet reviewer accounts approving PRs from the target account

Building a detection pipeline over the GitHub audit log does not require GitHub Enterprise. The REST API endpoints for audit log data are available at the organization level on free and paid plans, though streaming to a SIEM requires the Enterprise plan. For community-run OSS security monitoring, the REST polling approach is sufficient for the detection rules covered here.

Threat Model

Adversary 1 — Long-game supply chain attacker. Access level: initially none; targets a maintainer-adjacent contributor role. Timeline: months to years. Objective: obtain merge rights and use them to insert a backdoor into a widely-distributed package that reaches end users via package manager or system distribution channel. Signature: patient contributor activity followed by a single high-value merge.

Adversary 2 — Compromised maintainer account. Access level: full commit and merge rights on one or more repositories. Objective: bulk-merge backdoored PRs while the legitimate maintainer is unaware their credentials are compromised. Signature: merge activity at unusual hours for the account, merge of PRs that were previously flagged or stalled, and merge without the typical review comments left by the legitimate maintainer.

Adversary 3 — Bot account posing as contributor. Access level: contributor-level; targets review approval on automated or low-scrutiny PRs. Objective: build a reviewer reputation to lend credibility to a target account’s PRs, or directly approve malicious PRs. Signature: account created recently, approves many PRs in a short timeframe, leaves no substantive review comments.

Adversary 4 — Insider maintainer. Access level: full; already has merge rights. Objective: insert a backdoor or credential-harvesting hook directly, potentially after a period of normal activity to obscure the intent. Signature: changes to security-sensitive paths (crypto, auth, network) that skip the usual review cycle, or modifications to build configuration that affect how binaries are produced.

Without monitoring: each of these adversaries operates undetected until the payload is discovered downstream, typically by a security researcher examining the distributed artifact. With monitoring: behavioural deviations from established patterns trigger alerts within hours of the anomalous activity, enabling human review of the specific PR or permission change before it propagates.

Hardening Configuration

Step 1 — Stream the GitHub Audit Log to S3 for Athena querying

GitHub Enterprise organizations can configure audit log streaming directly to S3. This is the recommended approach for organizations with more than a handful of repositories.

First, configure streaming from the GitHub organization settings or via API:

# Configure audit log streaming to S3 via GitHub API
# Requires: org admin token with admin:org scope

ORG="your-org"
BUCKET="your-audit-log-bucket"
REGION="us-east-1"
KEY_ID="your-kms-key-id"

curl -X PUT \
  -H "Authorization: Bearer $GITHUB_ADMIN_TOKEN" \
  -H "Accept: application/vnd.github+json" \
  "https://api.github.com/orgs/$ORG/audit-log/streaming" \
  -d "{
    \"enabled\": true,
    \"vendor\": \"s3\",
    \"bucket\": \"$BUCKET\",
    \"region\": \"$REGION\",
    \"key_id\": \"$KEY_ID\",
    \"authentication_type\": \"iam_role\",
    \"arn_role\": \"arn:aws:iam::123456789012:role/GitHubAuditLogStreaming\"
  }"

The S3 bucket IAM policy must allow GitHub’s streaming service to write objects:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowGitHubAuditLogStreaming",
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": [
        "s3:PutObject",
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::your-audit-log-bucket",
        "arn:aws:s3:::your-audit-log-bucket/*"
      ]
    }
  ]
}

Create an Athena table over the streamed JSON:

CREATE EXTERNAL TABLE github_audit_log (
  action STRING,
  actor STRING,
  actor_location STRUCT<country_code: STRING>,
  created_at BIGINT,
  org STRING,
  repo STRING,
  data MAP<STRING, STRING>
)
PARTITIONED BY (dt STRING)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://your-audit-log-bucket/github-audit-logs/'
TBLPROPERTIES ('has_encrypted_data'='false');

-- Refresh partitions daily via scheduled query or Lambda
MSCK REPAIR TABLE github_audit_log;

Step 2 — Python script to detect outside-hours merges and self-merges via the REST API

For projects not on GitHub Enterprise, the REST API allows polling the audit log. This script runs hourly and writes detections to a SIEM-compatible JSON stream:

#!/usr/bin/env python3
"""
github_audit_monitor.py — detect anomalous PR merge patterns
Run hourly via cron or GitHub Actions scheduled workflow.
"""

import json
import os
import sys
import datetime
from collections import defaultdict
import requests

GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
ORG = os.environ["GITHUB_ORG"]
# Maintainer working hours in UTC — adjust per project
WORKING_HOURS_START = 7   # 07:00 UTC
WORKING_HOURS_END = 20    # 20:00 UTC
BULK_MERGE_THRESHOLD = 8  # merges by single user within 24h
HEADERS = {
    "Authorization": f"Bearer {GITHUB_TOKEN}",
    "Accept": "application/vnd.github+json",
    "X-GitHub-Api-Version": "2022-11-28",
}

def fetch_audit_events(org: str, phrase: str, since_hours: int = 2) -> list[dict]:
    """Fetch audit log events matching phrase from the last N hours."""
    since = datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(hours=since_hours)
    since_ts = int(since.timestamp() * 1000)
    events = []
    url = f"https://api.github.com/orgs/{org}/audit-log"
    params = {"phrase": phrase, "include": "all", "per_page": 100}
    while url:
        resp = requests.get(url, headers=HEADERS, params=params)
        resp.raise_for_status()
        batch = resp.json()
        for event in batch:
            if event.get("@timestamp", 0) >= since_ts:
                events.append(event)
        # Pagination via Link header
        link = resp.headers.get("Link", "")
        url = None
        if 'rel="next"' in link:
            for part in link.split(","):
                if 'rel="next"' in part:
                    url = part.split(";")[0].strip().strip("<>")
                    params = {}
    return events

def detect_outside_hours_merges(events: list[dict]) -> list[dict]:
    """Flag merges that occur outside configured working hours."""
    findings = []
    for event in events:
        if event.get("action") != "pull_request.merge":
            continue
        ts_ms = event.get("@timestamp", 0)
        dt = datetime.datetime.fromtimestamp(ts_ms / 1000, tz=datetime.timezone.utc)
        hour = dt.hour
        if not (WORKING_HOURS_START <= hour < WORKING_HOURS_END):
            findings.append({
                "detection": "outside_hours_merge",
                "actor": event.get("actor"),
                "repo": event.get("repo"),
                "pull_request_id": event.get("pull_request_id"),
                "merged_at_utc": dt.isoformat(),
                "day_of_week": dt.strftime("%A"),
                "severity": "medium",
            })
    return findings

def detect_self_merges(events: list[dict]) -> list[dict]:
    """Flag PRs merged by the same account that opened them."""
    findings = []
    pr_authors: dict[str, str] = {}
    for event in events:
        if event.get("action") == "pull_request.create":
            pr_id = str(event.get("pull_request_id", ""))
            if pr_id:
                pr_authors[pr_id] = event.get("actor", "")
    for event in events:
        if event.get("action") == "pull_request.merge":
            pr_id = str(event.get("pull_request_id", ""))
            merger = event.get("actor", "")
            author = pr_authors.get(pr_id, "")
            if author and author == merger:
                findings.append({
                    "detection": "self_merge",
                    "actor": merger,
                    "repo": event.get("repo"),
                    "pull_request_id": pr_id,
                    "severity": "high",
                })
    return findings

def detect_bulk_merges(events: list[dict]) -> list[dict]:
    """Flag any user who merged more than BULK_MERGE_THRESHOLD PRs in a 24h window."""
    by_actor: dict[str, list] = defaultdict(list)
    for event in events:
        if event.get("action") == "pull_request.merge":
            by_actor[event.get("actor", "unknown")].append(event)
    findings = []
    for actor, actor_events in by_actor.items():
        if len(actor_events) >= BULK_MERGE_THRESHOLD:
            findings.append({
                "detection": "bulk_merge",
                "actor": actor,
                "merge_count": len(actor_events),
                "threshold": BULK_MERGE_THRESHOLD,
                "repos": list({e.get("repo") for e in actor_events}),
                "severity": "high",
            })
    return findings

def main():
    all_findings = []
    merge_events = fetch_audit_events(ORG, "action:pull_request.merge", since_hours=25)
    create_events = fetch_audit_events(ORG, "action:pull_request.create", since_hours=25)
    all_events = merge_events + create_events

    all_findings.extend(detect_outside_hours_merges(merge_events))
    all_findings.extend(detect_self_merges(all_events))
    all_findings.extend(detect_bulk_merges(merge_events))

    for finding in all_findings:
        print(json.dumps(finding))

    if all_findings:
        sys.exit(1)  # non-zero exit triggers alerting in CI

if __name__ == "__main__":
    main()

Step 3 — SIEM correlation rules for privilege escalation

These Sigma rules cover the most critical escalation patterns. Translate them to your SIEM’s native format (Elastic EQL, Splunk SPL, or Panther Python rules):

# sigma/github_new_contributor_escalation.yml
title: New GitHub contributor escalated to maintainer team within 7 days
status: experimental
description: >
  A GitHub account was added to an organization team with write or admin
  access within 7 days of making their first commit or pull request.
logsource:
  category: audit
  product: github
detection:
  selection_team_add:
    action: "team.add_member"
  selection_escalated_permissions:
    action: "team.change_parent_team"
  timeframe: 7d
  condition: |
    selection_team_add and
    first_contribution_within_7d(actor, org)
falsepositives:
  - Legitimate fast-tracked contributors from known organizations
  - Bot accounts that don't submit PRs before getting team access
level: high
tags:
  - supply-chain
  - privilege-escalation
  - github

# sigma/github_codeowners_modified_by_new_account.yml
title: CODEOWNERS file modified by account with fewer than 10 prior commits
status: experimental
description: >
  The CODEOWNERS file was modified by a contributor whose account has
  fewer than 10 commits in the repository. CODEOWNERS changes define
  who has review authority over security-sensitive paths.
logsource:
  category: audit
  product: github
detection:
  selection:
    action: "git.push"
    file_path|contains: "CODEOWNERS"
  filter_experienced:
    actor_commit_count|gte: 10
  condition: selection and not filter_experienced
level: critical
tags:
  - supply-chain
  - codeowners
  - github

For Elastic SIEM, the equivalent EQL query over streamed audit log events:

-- Detect: contributor added to org team within 7 days of first PR
sequence by actor with maxspan=7d
  [github_audit where action == "pull_request.create"]
  [github_audit where action in ("team.add_member", "org.add_member")
   and team_permission in ("write", "admin", "maintain")]

Step 4 — Detect audit log gaps with Prometheus alerting

If the audit log streaming pipeline goes silent, you want to know. This is the canary-in-the-coalmine check: if no audit events have been received in N minutes, either the pipeline is broken or someone has disabled streaming.

# prometheus/alerts/github_audit_log.yml
groups:
  - name: github_audit_log
    rules:
      - alert: GitHubAuditLogStreamGap
        expr: |
          (time() - github_audit_log_last_event_timestamp_seconds) > 1800
        for: 5m
        labels:
          severity: critical
          team: security
        annotations:
          summary: "GitHub audit log stream has been silent for >30 minutes"
          description: >
            No GitHub audit log events have been received in over 30 minutes.
            This may indicate that audit log streaming has been disabled,
            the S3 destination bucket policy has changed, or the streaming
            pipeline has failed. Investigate immediately.
          runbook: "https://wiki.example.com/runbooks/github-audit-log-gap"

      - alert: GitHubAuditLogHighAnomalyRate
        expr: |
          rate(github_audit_anomaly_detections_total[1h]) > 0.5
        for: 10m
        labels:
          severity: warning
          team: security
        annotations:
          summary: "Elevated GitHub audit anomaly detection rate"
          description: >
            More than 0.5 anomalous PR events per minute have been detected
            over the last hour. Review the detection feed for supply chain
            activity.

The metrics are emitted by the detection script deployed as a Prometheus exporter or by a Lambda function that parses the Athena query results and pushes metrics via the Pushgateway:

# Prometheus metric emission — add to detection script
from prometheus_client import CollectorRegistry, Gauge, Counter, push_to_gateway

registry = CollectorRegistry()
last_event_ts = Gauge(
    "github_audit_log_last_event_timestamp_seconds",
    "Unix timestamp of the most recent GitHub audit log event processed",
    registry=registry,
)
anomaly_counter = Counter(
    "github_audit_anomaly_detections_total",
    "Total anomalous PR pattern detections",
    ["detection_type", "severity"],
    registry=registry,
)

# After processing events:
last_event_ts.set(datetime.datetime.now().timestamp())
for finding in all_findings:
    anomaly_counter.labels(
        detection_type=finding["detection"],
        severity=finding["severity"],
    ).inc()

push_to_gateway(
    os.environ["PUSHGATEWAY_URL"],
    job="github_audit_monitor",
    registry=registry,
)

Step 5 — GitHub Actions workflow to run detection on a schedule

# .github/workflows/audit-log-monitor.yml
name: GitHub Audit Log Anomaly Detection

on:
  schedule:
    - cron: "0 * * * *"   # every hour
  workflow_dispatch:

permissions:
  contents: read

jobs:
  detect-anomalies:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: pip install requests prometheus-client

      - name: Run audit log anomaly detection
        env:
          GITHUB_TOKEN: ${{ secrets.AUDIT_LOG_PAT }}
          GITHUB_ORG: ${{ vars.GITHUB_ORG }}
          PUSHGATEWAY_URL: ${{ secrets.PUSHGATEWAY_URL }}
        run: python scripts/github_audit_monitor.py

      - name: Send findings to SIEM
        if: failure()
        uses: slackapi/slack-github-action@v1.26.0
        with:
          payload: |
            {
              "text": "GitHub audit anomaly detected in ${{ vars.GITHUB_ORG }}. Review detection output in workflow run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SECURITY_SLACK_WEBHOOK }}

Expected Behaviour After Hardening

Once the pipeline is operational, the detection coverage produces the following outcomes for each threat scenario:

• A contributor who merges their own PR without independent review generates a self_merge detection within one hour of the merge, triggering a Slack alert to the security team
• A merge performed at 03:00 UTC when historical maintainer activity clusters between 09:00 and 18:00 UTC generates an outside_hours_merge detection with medium severity; after two such detections within 72 hours, a human review is escalated
• A new account added to the maintainer team within 7 days of their first PR triggers the SIEM correlation rule with critical severity and fires a PagerDuty alert
• A user who merges more than 8 PRs within a 24-hour window generates a bulk_merge detection; this covers both compromised account scenarios and rushed end-of-release merges
• If GitHub audit log streaming is disabled or the S3 pipeline breaks, the Prometheus gap alert fires within 35 minutes (30-minute gap plus 5-minute for duration), before an attacker has time to complete a bulk operation

False positive rates depend heavily on project activity patterns. Busy release periods will generate outside-hours merges from maintainers in non-UTC timezones. Tune the working-hours window per project by analysing historical merge timestamp distributions before enabling alerting.

Trade-offs and Operational Considerations

Failure Modes

Related Articles

• Defending Against AI-Authored Malicious Pull Requests
• Public Repository Bot PR Attack Defence
• Building an Audit Log Pipeline
• Detecting Cloud Audit Log Tampering
• npm Supply Chain Runtime Detection