Copa in CI/CD: Automated Container Patch Pipelines with Trivy, cosign, and GitHub Actions

Copa in CI/CD: Automated Container Patch Pipelines with Trivy, cosign, and GitHub Actions

Problem

Container scanning in CI solves the wrong half of the problem. Trivy, Grype, and similar tools tell you which OS packages in your image are vulnerable. They do not fix them. The conventional response is to rebuild: update the base image, bump a FROM line, and let the pipeline run again. That works when your team owns the Dockerfile. It fails completely when the vulnerable packages live in a third-party base image you do not control — which is the case for most production workloads.

The traditional scan-build-push loop also has a timing problem. An image scanned clean at build time can accumulate CVEs between build and deploy. If your deployment cadence is weekly but new CVEs are disclosed daily, you are perpetually deploying images that were only clean at the moment of the scan. There is no in-pipeline mechanism to close that gap.

Copa (Copacetic) addresses both problems directly. It operates on the already-built image, using BuildKit to apply updated OS packages without rebuilding from a Dockerfile. The input is a Trivy JSON report; the output is a new image digest with the vulnerable packages replaced by patched versions. The process runs entirely within the pipeline, on images you did not build, sourced from registries you do not own.

The practical result is a scan-patch-verify loop that runs on any OCI-compliant image:

  1. Trivy scans the image and produces a structured vulnerability report.
  2. Copa reads the report and patches OS package CVEs in place via BuildKit.
  3. Trivy re-scans the patched image to confirm CVE elimination.
  4. cosign signs the verified patched digest with keyless OIDC attestation.
  5. The signed, verified image is promoted to the production registry.

This article covers the full implementation in GitHub Actions and Tekton Pipelines, including patch failure handling, promotion gating, and scheduled nightly re-patching of production images.

Target systems: Copa 0.6+; Trivy 0.50+; cosign 2.x; GitHub Actions; Tekton Pipelines v1 API; BuildKit 0.12+; any OCI-compliant registry; Debian, Alpine, and RHEL-based image OS packages.

Threat Model

Threat 1 — CVE disclosed between build and deploy. An image builds clean on Monday. A critical CVE in libssl is published Wednesday. The image deploys Friday with no additional scanning or patching. Without an in-pipeline patching mechanism, the build-time scan result is the only signal — and it is already stale.

Threat 2 — Critical CVE in a third-party base image. An application team uses FROM nginx:1.25 or a distroless base from a vendor. A new CVE affects a package in that base image. The application team has no upstream Dockerfile access. The conventional fix — modify the FROM line — is not available. The vulnerability persists until the upstream maintainer releases a new base image tag.

Threat 3 — Patched image pushed without re-verification. Copa runs and exits zero. The pipeline treats exit zero as success and promotes the image. However, Copa only patches packages for which a fix is available in the upstream package repository. If no fix exists, Copa may patch a subset of CVEs and leave others in place. Without a Trivy re-scan of the patched digest, the pipeline has no confirmation that the CVEs it intended to fix are actually gone.

Threat 4 — Patched image pushed unsigned. An admission controller in the production cluster requires cosign signature verification before allowing an image to run. A patched image pushed without cosign signing fails admission. Alternatively, without signing, there is no provenance record linking the patched digest to the pipeline run that produced it — any image with the right tag could be pulled.

Access level for all threats: No attacker action required for Threats 1–3; these are configuration gaps. Threat 4 requires a registry write access. The blast radius in each case is a production container running with exploitable OS packages, or a patched image that cannot pass admission control.

Configuration

GitHub Actions: Full Scan-Patch-Verify-Sign Workflow

The workflow is structured as four sequential jobs. Each job passes data forward using GitHub Actions artifacts and job outputs.

# .github/workflows/copa-patch.yml
name: Copa Container Patch Pipeline

on:
  push:
    branches: [main]
  workflow_dispatch:
    inputs:
      image:
        description: "Image to patch (e.g. ghcr.io/org/app:sha-abc123)"
        required: true
        type: string

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

permissions:
  contents: read
  packages: write
  id-token: write   # required for cosign OIDC keyless signing
  issues: write     # required for creating issues on patch failure

jobs:
  scan:
    name: Trivy Initial Scan
    runs-on: ubuntu-24.04
    outputs:
      image-ref: ${{ steps.set-image.outputs.image-ref }}
      critical-count: ${{ steps.parse.outputs.critical-count }}
    steps:
      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Set image reference
        id: set-image
        run: |
          if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
            echo "image-ref=${{ inputs.image }}" >> "$GITHUB_OUTPUT"
          else
            echo "image-ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}" >> "$GITHUB_OUTPUT"
          fi

      - name: Run Trivy scan
        uses: aquasecurity/trivy-action@0.28.0
        with:
          image-ref: ${{ steps.set-image.outputs.image-ref }}
          format: json
          output: trivy-report.json
          severity: CRITICAL,HIGH
          exit-code: "0"   # do not fail here; Copa will patch

      - name: Parse critical CVE count
        id: parse
        run: |
          COUNT=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity == "CRITICAL")] | length' trivy-report.json)
          echo "critical-count=${COUNT}" >> "$GITHUB_OUTPUT"
          echo "Found ${COUNT} CRITICAL CVEs before patching"

      - name: Upload Trivy report
        uses: actions/upload-artifact@v4
        with:
          name: trivy-report-pre-patch
          path: trivy-report.json
          retention-days: 7

  patch:
    name: Copa Patch
    runs-on: ubuntu-24.04
    needs: scan
    outputs:
      patched-image: ${{ steps.copa.outputs.patched-image }}
    steps:
      - name: Download Trivy report
        uses: actions/download-artifact@v4
        with:
          name: trivy-report-pre-patch

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        with:
          driver: docker-container
          use: true

      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Run Copa patch
        id: copa
        uses: project-copacetic/copa-action@v0.6.0
        with:
          image: ${{ needs.scan.outputs.image-ref }}
          image-report: trivy-report.json
          patched-tag: ${{ github.sha }}-patched
          output: patched-image
          buildkit-version: latest
          copa-version: latest

      - name: Verify Copa produced a patched image
        run: |
          PATCHED="${{ steps.copa.outputs.patched-image }}"
          if [ -z "${PATCHED}" ]; then
            echo "Copa did not produce a patched image. Checking for no-fix scenario."
            exit 1
          fi
          echo "Patched image: ${PATCHED}"

  verify:
    name: Trivy Post-Patch Verification
    runs-on: ubuntu-24.04
    needs: patch
    outputs:
      residual-critical: ${{ steps.parse.outputs.residual-critical }}
    steps:
      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Run Trivy scan on patched image
        uses: aquasecurity/trivy-action@0.28.0
        with:
          image-ref: ${{ needs.patch.outputs.patched-image }}
          format: json
          output: trivy-report-post.json
          severity: CRITICAL,HIGH
          exit-code: "0"

      - name: Check for residual critical CVEs
        id: parse
        run: |
          COUNT=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity == "CRITICAL")] | length' trivy-report-post.json)
          echo "residual-critical=${COUNT}" >> "$GITHUB_OUTPUT"
          if [ "${COUNT}" -gt "0" ]; then
            echo "ERROR: ${COUNT} CRITICAL CVEs remain after Copa patch."
            jq '.Results[]?.Vulnerabilities[]? | select(.Severity == "CRITICAL") | {id: .VulnerabilityID, pkg: .PkgName, fixed: .FixedVersion}' trivy-report-post.json
            exit 1
          fi
          echo "Verification passed: 0 CRITICAL CVEs in patched image."

      - name: Upload post-patch Trivy report
        uses: actions/upload-artifact@v4
        with:
          name: trivy-report-post-patch
          path: trivy-report-post.json
          retention-days: 30

  sign-and-push:
    name: cosign Sign and Push
    runs-on: ubuntu-24.04
    needs: [patch, verify]
    steps:
      - name: Install cosign
        uses: sigstore/cosign-installer@v3

      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Sign patched image with OIDC keyless signing
        env:
          COSIGN_EXPERIMENTAL: "1"
        run: |
          PATCHED="${{ needs.patch.outputs.patched-image }}"
          # Resolve the image digest before signing
          DIGEST=$(docker buildx imagetools inspect "${PATCHED}" --format '{{.Manifest.Digest}}')
          DIGEST_REF="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${DIGEST}"

          cosign sign --yes \
            --rekor-url https://rekor.sigstore.dev \
            --fulcio-url https://fulcio.sigstore.dev \
            "${DIGEST_REF}"

          echo "Signed: ${DIGEST_REF}"
          echo "PATCHED_DIGEST_REF=${DIGEST_REF}" >> "$GITHUB_ENV"

      - name: Promote to production registry
        run: |
          # Only runs after verify job passed — crane copy is the promotion gate
          crane copy \
            "${{ needs.patch.outputs.patched-image }}" \
            "${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest-patched"
        env:
          COSIGN_EXPERIMENTAL: "1"

Copa Action Inputs and Outputs

The project-copacetic/copa-action action wraps the copa patch CLI command. The key inputs:

- uses: project-copacetic/copa-action@v0.6.0
  with:
    image: ghcr.io/org/app:sha-abc123          # source image to patch
    image-report: trivy-report.json             # Trivy JSON report (--format json)
    patched-tag: sha-abc123-patched             # tag for the output image
    output: patched-image                       # name of the output variable
    buildkit-version: latest                    # BuildKit daemon version
    copa-version: latest                        # copa binary version
    timeout: "10m"                              # patch timeout (optional)

The action outputs a single value, patched-image, which is the full image reference including registry, repository, and the patched tag. This value is the input to both the verify and sign jobs. Passing it between jobs requires the outputs block on the patch job and needs.patch.outputs.patched-image in downstream jobs.

Copa requires BuildKit because it uses BuildKit’s image manipulation API to create a new layer with patched packages. The docker/setup-buildx-action with driver: docker-container starts a BuildKit daemon that Copa can connect to. Without this step, Copa exits with a buildkitd connection error.

Handling Copa Patch Failures

Copa can fail in two distinct ways: a hard failure (BuildKit error, registry auth failure, malformed Trivy report) and a soft failure (no fix available for one or more CVEs). Both require different handling.

The following workflow job handles Copa failures gracefully, creating a GitHub Issue when no fix is available:

  patch-with-fallback:
    name: Copa Patch with Failure Handling
    runs-on: ubuntu-24.04
    needs: scan
    outputs:
      patched-image: ${{ steps.copa.outputs.patched-image }}
      patch-status: ${{ steps.copa-status.outputs.status }}
    steps:
      - name: Download Trivy report
        uses: actions/download-artifact@v4
        with:
          name: trivy-report-pre-patch

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        with:
          driver: docker-container
          use: true

      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Run Copa patch
        id: copa
        continue-on-error: true
        uses: project-copacetic/copa-action@v0.6.0
        with:
          image: ${{ needs.scan.outputs.image-ref }}
          image-report: trivy-report.json
          patched-tag: ${{ github.sha }}-patched

      - name: Determine patch status
        id: copa-status
        run: |
          if [ "${{ steps.copa.outcome }}" = "success" ]; then
            echo "status=patched" >> "$GITHUB_OUTPUT"
          else
            # Check if failure is due to no-fix-available vs hard error
            # Copa exits with a non-zero code and "no updates available" message
            # when all CVEs lack a fix in the upstream package repo
            echo "status=no-fix" >> "$GITHUB_OUTPUT"
          fi

      - name: Create GitHub Issue for unfixable CVEs
        if: steps.copa-status.outputs.status == 'no-fix'
        uses: actions/github-script@v7
        with:
          script: |
            const report = require('fs').readFileSync('trivy-report.json', 'utf8');
            const parsed = JSON.parse(report);
            const criticals = parsed.Results?.flatMap(r => r.Vulnerabilities || [])
              .filter(v => v.Severity === 'CRITICAL' && !v.FixedVersion)
              .map(v => `- **${v.VulnerabilityID}** in \`${v.PkgName}\` (no fix available)`)
              .join('\n') || 'None';

            await github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: `[Copa] Unfixable CVEs in ${context.sha.slice(0, 8)} — manual remediation required`,
              labels: ['security', 'vulnerability', 'needs-triage'],
              body: `## Copa patch failed: no fix available\n\n**Image:** \`${{ needs.scan.outputs.image-ref }}\`\n**Workflow run:** ${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}\n\n### CVEs with no fix available\n${criticals}\n\n### Recommended actions\n1. Check if a new base image tag has been released.\n2. Consider switching to an alternative base image.\n3. Add affected CVE IDs to \`.copa-exceptions.txt\` if accepted risk.`
            });

      - name: Fail if no fix and not in exceptions list
        if: steps.copa-status.outputs.status == 'no-fix'
        run: |
          # Check known-exceptions list
          if [ -f .copa-exceptions.txt ]; then
            UNFIXED=$(jq -r '.Results[]?.Vulnerabilities[]? | select(.Severity == "CRITICAL" and (.FixedVersion == null or .FixedVersion == "")) | .VulnerabilityID' trivy-report.json)
            while IFS= read -r cve; do
              if ! grep -q "^${cve}$" .copa-exceptions.txt; then
                echo "CVE ${cve} is not in the exceptions list. Failing pipeline."
                exit 1
              fi
            done <<< "${UNFIXED}"
            echo "All unfixable CVEs are in the exceptions list. Continuing."
          else
            echo "No exceptions list found. Failing pipeline due to unfixable CVEs."
            exit 1
          fi

The .copa-exceptions.txt file is a plain-text list of CVE IDs (one per line) that have been reviewed and accepted as known exceptions, typically because no fix exists and the application code does not exercise the vulnerable code path. This file should be committed to the repository and reviewed in pull requests.

Promotion Gate

The promotion gate is enforced by job dependency ordering. The sign-and-push job only runs if the verify job passes. If Trivy finds residual CRITICAL CVEs after patching, the verify job exits non-zero, and the sign-and-push job is skipped.

For multi-environment promotion, use crane copy with explicit digest pinning:

  promote-to-production:
    name: Promote Patched Image to Production Registry
    runs-on: ubuntu-24.04
    needs: [verify, sign-and-push]
    if: needs.verify.outputs.residual-critical == '0'
    environment: production
    steps:
      - name: Install crane
        uses: imjasonh/setup-crane@v0.4

      - name: Log in to staging registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Log in to production registry
        uses: docker/login-action@v3
        with:
          registry: registry.prod.example.com
          username: ${{ secrets.PROD_REGISTRY_USER }}
          password: ${{ secrets.PROD_REGISTRY_TOKEN }}

      - name: Copy patched image to production registry
        run: |
          PATCHED="${{ needs.patch.outputs.patched-image }}"
          # Resolve digest to avoid tag mutability
          DIGEST=$(crane digest "${PATCHED}")
          SRC="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${DIGEST}"
          DST="registry.prod.example.com/${{ env.IMAGE_NAME }}:${{ github.sha }}-patched"

          crane copy "${SRC}" "${DST}"
          echo "Promoted ${SRC} to ${DST}"

The environment: production gate adds a required reviewer approval step if configured in repository settings — no patched image reaches production without both Copa verification passing and a human approval.

Scheduled Nightly Re-Patching Pipeline

This separate workflow re-scans and re-patches all production images on a schedule, closing the gap between build time and new CVE disclosure:

# .github/workflows/copa-scheduled-patch.yml
name: Nightly Copa Re-Patch

on:
  schedule:
    - cron: "0 2 * * *"   # 02:00 UTC nightly
  workflow_dispatch:

permissions:
  contents: read
  packages: write
  id-token: write
  issues: write

jobs:
  list-production-images:
    name: List Production Images
    runs-on: ubuntu-24.04
    outputs:
      matrix: ${{ steps.build-matrix.outputs.matrix }}
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Build image matrix from production manifest
        id: build-matrix
        run: |
          # Read production images from a manifest file
          # Format: one image reference per line
          IMAGES=$(jq -R -s 'split("\n") | map(select(length > 0))' production-images.txt)
          echo "matrix={\"image\": ${IMAGES}}" >> "$GITHUB_OUTPUT"

  nightly-patch:
    name: Patch ${{ matrix.image }}
    runs-on: ubuntu-24.04
    needs: list-production-images
    strategy:
      matrix: ${{ fromJson(needs.list-production-images.outputs.matrix) }}
      fail-fast: false   # patch all images even if one fails
    steps:
      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Run Trivy scan
        uses: aquasecurity/trivy-action@0.28.0
        with:
          image-ref: ${{ matrix.image }}
          format: json
          output: trivy-report.json
          severity: CRITICAL,HIGH
          exit-code: "0"

      - name: Check if patching is needed
        id: check
        run: |
          COUNT=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity == "CRITICAL")] | length' trivy-report.json)
          echo "needs-patch=$([ "${COUNT}" -gt 0 ] && echo true || echo false)" >> "$GITHUB_OUTPUT"
          echo "critical-count=${COUNT}" >> "$GITHUB_OUTPUT"

      - name: Set up Docker Buildx
        if: steps.check.outputs.needs-patch == 'true'
        uses: docker/setup-buildx-action@v3
        with:
          driver: docker-container
          use: true

      - name: Run Copa patch
        if: steps.check.outputs.needs-patch == 'true'
        id: copa
        continue-on-error: true
        uses: project-copacetic/copa-action@v0.6.0
        with:
          image: ${{ matrix.image }}
          image-report: trivy-report.json
          patched-tag: nightly-${{ github.run_id }}

      - name: Verify and sign if patched
        if: steps.check.outputs.needs-patch == 'true' && steps.copa.outcome == 'success'
        run: |
          PATCHED="${{ steps.copa.outputs.patched-image }}"
          # Re-scan
          trivy image --format json -o trivy-post.json --severity CRITICAL "${PATCHED}"
          COUNT=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity == "CRITICAL")] | length' trivy-post.json)
          if [ "${COUNT}" -gt "0" ]; then
            echo "Residual CVEs after nightly patch: ${COUNT}"
            exit 1
          fi
          # Sign
          cosign sign --yes "${PATCHED}"
          echo "Nightly patch complete for ${{ matrix.image }}"
        env:
          COSIGN_EXPERIMENTAL: "1"

Tekton Pipeline Implementation

For teams running Tekton Pipelines on Kubernetes, the same scan-patch-verify-sign loop maps to four Tasks connected by a Pipeline:

# tekton/tasks/trivy-scan-task.yaml
apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: trivy-scan
  namespace: cicd
spec:
  params:
    - name: image-ref
      type: string
      description: Full image reference to scan
    - name: severity
      type: string
      default: "CRITICAL,HIGH"
  results:
    - name: critical-count
      description: Number of CRITICAL CVEs found
  workspaces:
    - name: reports
      description: Shared workspace for scan reports
  steps:
    - name: scan
      image: aquasec/trivy:0.50.4
      script: |
        #!/usr/bin/env sh
        trivy image \
          --format json \
          --output "$(workspaces.reports.path)/trivy-report.json" \
          --severity "$(params.severity)" \
          --exit-code 0 \
          "$(params.image-ref)"

        COUNT=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity == "CRITICAL")] | length' \
          "$(workspaces.reports.path)/trivy-report.json")
        printf '%s' "${COUNT}" > "$(results.critical-count.path)"
# tekton/tasks/copa-patch-task.yaml
apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: copa-patch
  namespace: cicd
spec:
  params:
    - name: image-ref
      type: string
    - name: patched-tag
      type: string
  results:
    - name: patched-image
      description: Full reference of the patched image
  workspaces:
    - name: reports
      description: Workspace containing trivy-report.json
    - name: dockerconfig
      description: Docker config for registry auth
      mountPath: /kaniko/.docker
  sidecars:
    - name: buildkitd
      image: moby/buildkit:v0.12.5-rootless
      securityContext:
        seccompProfile:
          type: Unconfined
        runAsUser: 1000
        runAsGroup: 1000
      readinessProbe:
        exec:
          command: ["buildctl", "debug", "workers"]
        initialDelaySeconds: 5
        periodSeconds: 3
  steps:
    - name: patch
      image: ghcr.io/project-copacetic/copacetic:v0.6.0
      script: |
        #!/usr/bin/env sh
        copa patch \
          --image "$(params.image-ref)" \
          --report "$(workspaces.reports.path)/trivy-report.json" \
          --addr buildkit:///run/buildkit/buildkitd.sock \
          --tag "$(params.patched-tag)" \
          --output "$(results.patched-image.path)"
      env:
        - name: DOCKER_CONFIG
          value: /kaniko/.docker
# tekton/tasks/cosign-sign-task.yaml
apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: cosign-sign
  namespace: cicd
spec:
  params:
    - name: image-ref
      type: string
      description: Image reference to sign (should be digest-pinned)
  steps:
    - name: sign
      image: gcr.io/projectsigstore/cosign:v2.2.3
      script: |
        #!/usr/bin/env sh
        cosign sign --yes \
          --rekor-url https://rekor.sigstore.dev \
          --fulcio-url https://fulcio.sigstore.dev \
          "$(params.image-ref)"
      env:
        - name: COSIGN_EXPERIMENTAL
          value: "1"
# tekton/pipelines/copa-pipeline.yaml
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: copa-patch-pipeline
  namespace: cicd
spec:
  params:
    - name: image-ref
      type: string
    - name: patched-tag
      type: string
  workspaces:
    - name: reports
    - name: dockerconfig
  tasks:
    - name: scan
      taskRef:
        name: trivy-scan
      params:
        - name: image-ref
          value: $(params.image-ref)
      workspaces:
        - name: reports
          workspace: reports

    - name: patch
      taskRef:
        name: copa-patch
      runAfter: [scan]
      params:
        - name: image-ref
          value: $(params.image-ref)
        - name: patched-tag
          value: $(params.patched-tag)
      workspaces:
        - name: reports
          workspace: reports
        - name: dockerconfig
          workspace: dockerconfig

    - name: verify
      taskRef:
        name: trivy-scan
      runAfter: [patch]
      params:
        - name: image-ref
          value: $(tasks.patch.results.patched-image)
        - name: severity
          value: "CRITICAL"
      workspaces:
        - name: reports
          workspace: reports

    - name: sign
      taskRef:
        name: cosign-sign
      runAfter: [verify]
      params:
        - name: image-ref
          value: $(tasks.patch.results.patched-image)

  results:
    - name: patched-image
      value: $(tasks.patch.results.patched-image)
    - name: post-patch-critical-count
      value: $(tasks.verify.results.critical-count)

The Tekton Pipeline passes the patched image reference between tasks using tasks.patch.results.patched-image. The verify Task re-runs the TrivyScanTask against this output rather than the original image. The runAfter field enforces sequential execution.

Expected Behaviour

The following table describes pipeline outcomes for the primary scenarios:

Scenario Copa Action Pipeline Outcome Notification
No CVEs found in initial Trivy scan Copa is skipped; original image is promoted Pipeline passes; original image is signed and promoted None required
CVEs found; Copa patches all successfully; re-scan clean Copa patches; verify passes Pipeline passes; patched image is signed and promoted Slack/PR comment with CVE count patched
CVEs found; Copa patches some; re-scan shows residual CRITICAL CVEs Copa exits zero (partial patch) verify job fails; pipeline stops; no promotion GitHub Actions job failure notification; Trivy report uploaded as artifact
CVEs found; no fix available in upstream package repo Copa exits non-zero; patch job fails patch-with-fallback creates GitHub Issue; checks exceptions list; fails pipeline unless CVEs are in exceptions list GitHub Issue created with CVE details and remediation options
Copa exits non-zero due to BuildKit error Hard failure patch job fails; pipeline stops GitHub Actions job failure; runner logs contain BuildKit error
Cosign OIDC auth fails cosign exits non-zero sign-and-push job fails; patched image is in registry but unsigned GitHub Actions job failure; operator must investigate OIDC configuration

Trade-offs

Dimension Option A Option B Recommendation
Copa placement in pipeline Copa runs in the main build pipeline after every push Copa runs in a separate scheduled pipeline (nightly/weekly) Use both: main pipeline for new CVEs introduced by the PR; scheduled pipeline for CVEs disclosed after build
GitHub Actions vs. Tekton for Copa GitHub Actions: simpler setup, hosted runners, no BuildKit sidecar complexity Tekton: Kubernetes-native, integrates with Tekton Chains for supply chain attestation, more control over runner resources GitHub Actions for most teams; Tekton when you already run Tekton Pipelines and need Chains integration
copa-action vs. manual copa CLI steps copa-action abstracts BuildKit setup, simplifies version pinning, provides structured outputs Manual copa patch CLI gives full control over flags, easier to debug, no dependency on third-party action Use copa-action for standard cases; use manual CLI when you need custom BuildKit socket paths or non-standard registry configurations
Scan-then-patch vs. build-then-scan-patch Scan the base image before building; patch only the base if needed Build the full application image first; scan and patch the final image Scan-patch the final image: application layers can introduce OS packages that the base-only scan misses; Copa operates on the final image regardless
Keyless vs. key-based cosign signing Keyless OIDC: no key management, signatures tied to GitHub OIDC identity, ephemeral certificates Key-based: works outside GitHub Actions, portable, requires key management and rotation Keyless in GitHub Actions; key-based in Tekton or self-hosted runners where OIDC is not available

Failure Modes

Failure Symptom Root Cause Fix
BuildKit not available in runner Copa exits: failed to connect to buildkitd: context deadline exceeded docker/setup-buildx-action not called before copa-action, or driver: docker-container not set Add docker/setup-buildx-action@v3 with driver: docker-container before the Copa step; verify the action runs before Copa in the job
Trivy report format version mismatch with Copa Copa exits: unsupported report schema version or failed to parse vulnerability report Trivy JSON schema changes between Trivy versions; Copa expects a specific schema revision Pin Trivy to a version known to work with your Copa version; check Copa release notes for supported Trivy versions; use --format json not --format sarif
cosign OIDC fails in non-GitHub runner cosign exits: error getting Fulcio roots: failed to get certificate or OIDC token fetch error ACTIONS_ID_TOKEN_REQUEST_URL is not set outside GitHub-hosted runners; self-hosted runners behind a proxy may block Fulcio/Rekor endpoints On self-hosted runners, verify outbound access to fulcio.sigstore.dev and rekor.sigstore.dev; add id-token: write to workflow permissions; consider key-based signing as fallback
Patched image digest not passed correctly between jobs Downstream job uses original image reference; verify scans original image not patched image; false clean result outputs block on patch job missing or incorrectly named; steps.copa.outputs.patched-image referenced before Copa action completes Verify the patch job has outputs: patched-image: ${{ steps.copa.outputs.patched-image }}; verify downstream jobs reference needs.patch.outputs.patched-image not a hardcoded tag
Copa patches but does not update the manifest index crane copy pulls the wrong architecture Copa creates a single-platform patched image; multi-arch source manifests result in a single-arch patched image Set COPA_BUILDKIT_PLATFORM env var or pass --platform to Copa; run Copa once per platform and re-assemble a manifest list with crane index append
Registry rate limiting during nightly matrix Multiple parallel Copa jobs hit registry pull rate limits Nightly scheduled job runs fail-fast: false with many images simultaneously Add max-parallel: 3 to the matrix strategy; use a registry mirror or pull-through cache; add exponential backoff retry logic around the Trivy scan step