Kubernetes / Platform intermediate 12 min read

Container Image Signing Policy Enforcement: From cosign to Admission Control

container-signing cosign kyverno supply-chain admission-control sigstore

Container Image Signing Policy Enforcement: From cosign to Admission Control

The Verification Gap

Teams that sign container images in CI but do not enforce those signatures at admission control have accomplished nothing from a security standpoint. The signatures exist. Nobody checks them before the image runs. An attacker who pushes an unsigned image or a maliciously-modified image with no signature still gets code into production — the signing infrastructure becomes documentation, not defence.

The same gap appears more subtly when enforcement exists in one environment but not another. A staging cluster that enforces signatures and a production cluster that does not gives the appearance of a supply chain security posture while leaving the critical path unprotected.

Enforcement requires two things working together: a signing process in CI that produces verifiable artefacts, and an admission controller that refuses to schedule any workload whose image cannot be verified before the API server accepts the resource. This article covers both sides as a connected pipeline, not separate concerns.

Target systems: cosign 2.4+, Kyverno 1.12+, OPA Gatekeeper 3.16+ with Ratify 1.3+, Sigstore Policy Controller 0.9+, Kubernetes 1.28+, GitHub Actions.

Signing with cosign Keyless in GitHub Actions

Keyless signing in GitHub Actions uses the runner’s OIDC token as the signing identity. There are no private keys to store, rotate, or leak. The full signing flow is described in detail in Sigstore and Cosign: Keyless Container Image Signing and Verification. What matters here is the output that admission controllers need to verify.

A minimal GitHub Actions job that builds, pushes, and signs an image:

name: build-sign

on:
  push:
    branches: [main]

permissions:
  contents: read
  id-token: write
  packages: write

jobs:
  build-and-sign:
    runs-on: ubuntu-latest
    outputs:
      digest: ${{ steps.build.outputs.digest }}
    steps:
      - uses: actions/checkout@v4

      - name: Install cosign
        uses: sigstore/cosign-installer@v3

      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        id: build
        uses: docker/build-push-action@v5
        with:
          push: true
          tags: ghcr.io/org/app:${{ github.sha }}
          outputs: type=image,push=true

      - name: Sign image
        env:
          COSIGN_EXPERIMENTAL: "1"
        run: |
          cosign sign --yes \
            ghcr.io/org/app@${{ steps.build.outputs.digest }}

The id-token: write permission is mandatory — without it the runner cannot request an OIDC token from GitHub and Fulcio will reject the request. Always sign by digest, not by tag. Tags are mutable. The digest is the cryptographic identifier that the signature actually binds to; signing a tag name gives a false sense of security because a different image can replace the tag after signing.

The signed artefact lands in the registry at a tag derived from the image digest: sha256-<digest>.sig. Cosign also submits the signing event to Rekor. The resulting transparency log entry proves the signing identity and time without requiring the short-lived Fulcio certificate to still be valid at verification time.

SLSA Provenance Attestations

Signing the image digest confirms the image is unchanged since signing. It does not prove how the image was built. SLSA provenance attestations add the build context — what source commit was used, what workflow ran, what inputs were provided. Without attestations, you can verify the image was signed by your CI; with them, you can verify it was built from a specific commit in a specific workflow.

Generate a SLSA provenance attestation in the same job:

      - name: Attest build provenance
        uses: actions/attest-build-provenance@v1
        with:
          subject-name: ghcr.io/org/app
          subject-digest: ${{ steps.build.outputs.digest }}
          push-to-registry: true

Or with cosign directly:

cosign attest --yes \
  --predicate provenance.json \
  --type slsaprovenance \
  ghcr.io/org/app@sha256:<digest>

Kyverno can verify both the signature and the attestation contents in a single policy, blocking images that are signed but lack compliant provenance. The SLSA build provenance pipeline is covered in depth at SLSA Build Provenance.

Kyverno ClusterPolicy ImageVerify

Kyverno’s imageVerify rule verifies cosign signatures and attestations as part of admission control. The policy evaluates before the resource is persisted to etcd. If verification fails, the pod creation is rejected.

A ClusterPolicy enforcing keyless signature verification:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-image-signature
spec:
  validationFailureAction: Enforce
  background: false
  rules:
    - name: check-image-signature
      match:
        any:
          - resources:
              kinds: [Pod]
              namespaces: ["production", "staging"]
      verifyImages:
        - imageReferences:
            - "ghcr.io/org/*"
          attestors:
            - count: 1
              entries:
                - keyless:
                    subject: "https://github.com/org/app/.github/workflows/build-sign.yaml@refs/heads/main"
                    issuer: "https://token.actions.githubusercontent.com"
                    rekor:
                      url: https://rekor.sigstore.dev
          mutateDigest: true
          verifyDigest: true
          required: true

The subject field is the critical enforcement point. It pins the signature to a specific workflow file on a specific branch. A signature produced by any other workflow, repository, or branch — including a fork or a feature branch — does not match and is rejected. Without this constraint, any GitHub Actions workflow in any repository could produce a valid signature against the same Fulcio CA.

mutateDigest: true rewrites the image reference in the pod spec from a tag to the verified digest. This prevents the tag from being swapped between admission and scheduling. verifyDigest: true rejects image references that do not include a digest at all.

For SLSA provenance attestation verification alongside the signature:

      verifyImages:
        - imageReferences:
            - "ghcr.io/org/*"
          attestors:
            - entries:
                - keyless:
                    subject: "https://github.com/org/app/.github/workflows/build-sign.yaml@refs/heads/main"
                    issuer: "https://token.actions.githubusercontent.com"
          attestations:
            - type: https://slsa.dev/provenance/v1
              conditions:
                - all:
                    - key: "{{ buildDefinition.buildType }}"
                      operator: Equals
                      value: "https://slsa-framework.github.io/github-actions-buildtypes/workflow/v1"
                    - key: "{{ buildDefinition.externalParameters.workflow.ref }}"
                      operator: Equals
                      value: "refs/heads/main"

This verifies both that the image was signed by the CI workflow and that the provenance attestation confirms it was built from refs/heads/main. A build from a branch is rejected regardless of whether it is signed.

The Kyverno policy evaluation path and hardening considerations are covered in Kyverno Controller Security.

OPA Gatekeeper with Ratify

OPA Gatekeeper does not natively verify container image signatures. It evaluates Rego policies against Kubernetes resources but has no mechanism to call out to a registry to fetch and verify a cosign signature. Ratify fills that gap as an external data provider.

Ratify runs as a service alongside Gatekeeper. When Gatekeeper evaluates an image, it calls Ratify via the External Data framework. Ratify fetches the image’s signature and attestation artefacts from the registry, verifies them against configured verifier policies, and returns a structured result. Gatekeeper’s Rego policy then enforces based on the result.

Install Ratify with a cosign verifier configured:

apiVersion: config.ratify.deislabs.io/v1beta1
kind: Store
metadata:
  name: oras
spec:
  name: oras
  parameters:
    cacheEnabled: true
    ttl: 10
---
apiVersion: config.ratify.deislabs.io/v1beta1
kind: Verifier
metadata:
  name: cosign
spec:
  name: cosign
  artifactTypes: application/vnd.dev.cosign.artifact.sig.v1+json
  parameters:
    trustPolicies:
      - name: org-policy
        scopes:
          - "ghcr.io/org/*"
        keys:
          - provider: inline
            value: |
              -----BEGIN PUBLIC KEY-----
              <your-cosign-public-key>
              -----END PUBLIC KEY-----

Gatekeeper constraint template calling Ratify:

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: requiresignedimages
spec:
  crd:
    spec:
      names:
        kind: RequireSignedImages
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package requiresignedimages

        import future.keywords.in

        violation[{"msg": msg}] {
          image := input.review.object.spec.containers[_].image
          response := external_data({"provider": "ratify-provider", "keys": [image]})
          result := response.responses[_]
          result[0] == image
          not result[1].isSuccess
          msg := sprintf("image %v failed signature verification: %v", [image, result[1].errors])
        }

The Gatekeeper constraint that activates it:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: RequireSignedImages
metadata:
  name: require-signed-images
spec:
  enforcementAction: deny
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    namespaces: ["production", "staging"]

Sigstore Policy Controller

The Sigstore Policy Controller is a dedicated admission webhook built specifically for cosign signature enforcement. It is narrower in scope than Kyverno or Gatekeeper — it does one thing — which makes it simpler to reason about and audit.

Install via Helm:

helm repo add sigstore https://sigstore.github.io/helm-charts
helm install policy-controller sigstore/policy-controller \
  --namespace cosign-system \
  --create-namespace \
  --set "cosign.webhookName=policy.sigstore.dev"

Define verification policy with a ClusterImagePolicy:

apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
  name: org-image-policy
spec:
  images:
    - glob: "ghcr.io/org/**"
  authorities:
    - keyless:
        url: https://fulcio.sigstore.dev
        identities:
          - issuer: https://token.actions.githubusercontent.com
            subjectRegExp: "https://github.com/org/[^/]+/.github/workflows/build-sign\\.yaml@refs/heads/main"
      ctlog:
        url: https://rekor.sigstore.dev

The subjectRegExp applies to all repositories in the org GitHub organisation, removing the need for per-repository policy entries.

Namespaces opt in to enforcement with a label:

kubectl label namespace production policy.sigstore.dev/include=true

Namespaces without the label are not checked. This is a useful deployment pattern — roll out enforcement namespace by namespace rather than cluster-wide, reducing the blast radius of misconfigured policies during initial rollout.

Multi-Architecture Image Indexes

Multi-arch images add a layer of complexity. An OCI image index (manifest list) references multiple per-platform manifests. When you sign the index digest, you sign the pointer to the set of platform manifests. The individual platform manifests have their own digests, which are not separately signed.

Sign the index by digest:

IMAGE_DIGEST=$(docker buildx imagetools inspect ghcr.io/org/app:latest --format '{{.Manifest.Digest}}')
cosign sign --yes ghcr.io/org/app@${IMAGE_DIGEST}

When Kubernetes pulls an image, the kubelet resolves the image index to the per-platform manifest for the node’s architecture. The admission controller sees the pod spec with the original image reference. Kyverno’s imageVerify and the Sigstore Policy Controller both verify against the index digest in the pod spec rather than the resolved platform manifest digest — this works correctly as long as the pod spec references the index digest, not a platform-specific digest.

The problem occurs when tooling rewrites the image reference to the platform-specific digest before admission. Verify your build and deployment tooling preserves the index digest through to the pod spec. If the kubelet resolves to a per-platform digest before Kyverno evaluates, verification fails because only the index digest is signed.

To sign all platform manifests individually in addition to the index:

cosign sign --yes ghcr.io/org/app@sha256:<amd64-manifest-digest>
cosign sign --yes ghcr.io/org/app@sha256:<arm64-manifest-digest>
cosign sign --yes ghcr.io/org/app@sha256:<index-digest>

This is belt-and-suspenders and adds complexity to the signing pipeline, but it removes the dependency on the image reference being stable through admission.

Enterprise Key Management: KMS-Backed Signing

Keyless signing works when Fulcio and Rekor are reachable — either the public Sigstore instances or a private deployment. Air-gapped environments, regulatory constraints, or requirements for hardware-backed keys push teams toward KMS-backed cosign signing instead.

Cosign supports AWS KMS and GCP KMS as key providers natively:

# Generate a key in AWS KMS
cosign generate-key-pair \
  --kms awskms:///arn:aws:kms:us-east-1:123456789012:key/mrk-abc123

# Sign using the KMS key
cosign sign \
  --key awskms:///arn:aws:kms:us-east-1:123456789012:key/mrk-abc123 \
  ghcr.io/org/app@sha256:<digest>

# Verify using the public key
cosign verify \
  --key awskms:///arn:aws:kms:us-east-1:123456789012:key/mrk-abc123 \
  ghcr.io/org/app@sha256:<digest>

For GCP KMS:

cosign sign \
  --key gcpkms://projects/my-project/locations/global/keyRings/signing/cryptoKeys/cosign/cryptoKeyVersions/1 \
  ghcr.io/org/app@sha256:<digest>

In both cases, the private key never leaves the KMS. The CI runner needs IAM permissions to call kms:Sign (AWS) or cloudkms.cryptoKeyVersions.useToSign (GCP). The public key for verification is extracted once and distributed to admission controllers:

cosign public-key \
  --key awskms:///arn:aws:kms:us-east-1:123456789012:key/mrk-abc123 \
  > cosign.pub

Store cosign.pub in a Kubernetes Secret or ConfigMap and reference it in Kyverno policy:

      verifyImages:
        - imageReferences:
            - "ghcr.io/org/*"
          attestors:
            - entries:
                - keys:
                    secret:
                      name: cosign-pubkey
                      namespace: kyverno

Key rotation requires generating a new KMS key version, updating the admission controller configuration with the new public key, and maintaining the old public key temporarily to verify images signed before the rotation. Kyverno supports multiple key entries in a single attestor block for this transition window.

Exception Handling: Unsigned Legacy Images

A hard block on all unsigned images breaks existing workloads immediately. Practical rollout requires a time-bound exception mechanism for legacy images that have not yet been signed.

Kyverno supports exception policies that selectively bypass enforcement:

apiVersion: kyverno.io/v2alpha1
kind: PolicyException
metadata:
  name: legacy-images-exception
  namespace: kyverno
  annotations:
    owner: platform-team@org.com
    expires: "2026-08-01"
    ticket: JIRA-4821
spec:
  exceptions:
    - policyName: require-image-signature
      ruleNames:
        - check-image-signature
  match:
    any:
      - resources:
          kinds: [Pod]
          namespaces: ["legacy-apps"]
          selector:
            matchLabels:
              signing-exception: "approved"

The expiry annotation is informational — Kyverno does not enforce it. Enforce it with an external process: a scheduled job that queries PolicyException objects and alerts or deletes them when the expiry date passes. Without enforcement of the expiry, exception policies become permanent.

For Gatekeeper, use constraint parameters to maintain an allowlist:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: RequireSignedImages
metadata:
  name: require-signed-images
spec:
  enforcementAction: deny
  parameters:
    allowedUnsignedImages:
      - "ghcr.io/org/legacy-app:v1.2.3"
      - "ghcr.io/org/third-party-tool:2024.1"
    allowlistExpiry: "2026-08-01"

The allowlist should reference specific digests, not tags. An allowlisted tag is a mutable exemption — a compromised image pushed to the same tag bypasses signing enforcement indefinitely.

Debugging Signature Failures

When an image fails verification at admission, the rejection message is often terse. cosign verify --verbose provides the full verification trace:

cosign verify \
  --certificate-identity-regexp "https://github.com/org/.*" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  --rekor-url https://rekor.sigstore.dev \
  --verbose \
  ghcr.io/org/app@sha256:<digest>

The verbose output shows each step: fetching the signature artefact from the registry, verifying the Rekor inclusion proof, checking the Fulcio certificate chain, matching the subject and issuer against the constraints. Common failures:

  • No valid signatures found — the image was never signed, was signed at a tag that no longer resolves to this digest, or the signature artefact was not pushed to the registry.
  • Certificate subject does not match — the workflow that signed the image does not match the subject or subjectRegExp in the policy. This happens when a release workflow is different from the workflow named in the policy.
  • Rekor entry not found — the signing event was not logged (possible with --no-rekor-upload in the cosign invocation), or the Rekor instance configured in the policy does not match the one used during signing.
  • Error fetching Rekor entry — the admission controller cannot reach the Rekor URL. For keyless verification, Rekor is required. If Rekor is unreachable, verification fails. Private Sigstore deployments avoid this network dependency.

For Kyverno-specific failures, check the PolicyReport:

kubectl get policyreport -A
kubectl describe policyreport -n production <report-name>

Kyverno generates PolicyReport objects for background scan results. For admission failures, the rejection message appears in the API server response and in Kyverno controller logs:

kubectl logs -n kyverno -l app.kubernetes.io/name=kyverno -c kyverno \
  | grep "image verification failed"

Enable Kyverno’s image verification debug logging with the --imageVerifyMaxRetries and --log-level=4 flags on the controller deployment to get per-request verification traces including the exact cosign verification call and result.

For Ratify with Gatekeeper, query the Ratify verification results directly:

kubectl logs -n gatekeeper-system -l app=ratify \
  | grep "verification result"

Ratify’s REST API (when port-forwarded) accepts image digest queries and returns the full verification result including individual verifier outcomes, which is useful for diagnosing why a specific image fails when other images from the same registry succeed.

Enforcement Without Rekor: Private Sigstore

Public Rekor (rekor.sigstore.dev) introduces a network dependency and a privacy consideration — every image digest your cluster verifies is queryable in the public transparency log. Regulated environments or air-gapped clusters deploy private Sigstore infrastructure.

A minimal private deployment requires Fulcio (private CA), Rekor (private transparency log), and a TUF root that cosign clients trust:

cosign sign --yes \
  --fulcio-url https://fulcio.internal.org \
  --rekor-url https://rekor.internal.org \
  --oidc-issuer https://gitlab.internal.org \
  registry.internal.org/app@sha256:<digest>

Kyverno policy for a private Sigstore deployment:

      verifyImages:
        - imageReferences:
            - "registry.internal.org/*"
          attestors:
            - entries:
                - keyless:
                    subject: "https://gitlab.internal.org/org/app//build.yml@refs/heads/main"
                    issuer: "https://gitlab.internal.org"
                    rekor:
                      url: https://rekor.internal.org
                      pubKey: |
                        -----BEGIN PUBLIC KEY-----
                        <rekor-public-key>
                        -----END PUBLIC KEY-----

The pubKey field pins the Rekor instance’s signing key, preventing a policy from being satisfied by inclusion in a different Rekor instance.

Policy Scope and Rollout Strategy

Apply signature enforcement incrementally. Start with validationFailureAction: Audit in Kyverno, which logs violations without blocking. Review the PolicyReport to identify which images would fail enforcement, sign them, and only switch to Enforce once the violation count reaches zero for a namespace. Repeat per namespace.

The sequence:

  1. Deploy policy in Audit mode cluster-wide.
  2. Review PolicyReport violations by namespace.
  3. Sign images that fail, prioritising production namespaces.
  4. Switch production namespaces to Enforce.
  5. Address remaining namespaces with a deadline.
  6. Remove Audit-only exemptions.

Gate enforcement scope with namespace selectors to avoid blocking system namespaces:

  rules:
    - name: check-image-signature
      match:
        any:
          - resources:
              kinds: [Pod]
      exclude:
        any:
          - resources:
              namespaceSelector:
                matchExpressions:
                  - key: kubernetes.io/metadata.name
                    operator: In
                    values:
                      - kube-system
                      - kyverno
                      - cert-manager
                      - monitoring

Excluding the Kyverno namespace prevents the policy from blocking Kyverno’s own pods during upgrades, which would create a deadlock where Kyverno cannot start because its own images are not signed by your pipeline.

The full enforcement model — signing in CI, transparent logging in Rekor, and rejection at admission — gives the cluster a verifiable answer to “where did this image come from?” for every workload that runs. Signing without verification answers nothing.