The image was deployed, then the IDS lit up. Why we audit every image.

TL;DR — the 90-second summary

The pattern

Container image runs through the pipeline, the IDS lights up in the cluster — traffic nobody asked for

The cause

The build result is audited, third-party code (base images, sidecars, Helm charts) is taken for granted

Audit building blocks

Signature verification (Sigstore/cosign), SBOM (CycloneDX), provenance (SLSA), continuous re-audit

Hard stop

Admission controller in the cluster enforces "no signature, no deploy" — no exceptions

Regulatory anchor

NIS-2 requires supply-chain risk management; the CRA holds vendors liable for their software across the entire lifecycle

First iteration

almost always finds something — an old base, a forgotten sidecar, a deprecated dependency. That is not failure, that is the diagnosis

 

What is the problem?

It is one of the most expensive scenarios of modern IT organisations and at the same time one of the most common: a container image runs through the pipeline, is activated in the cluster, and a few minutes later the IDS lights up. Traffic appears inside the container that nobody expected and that has nothing to do with the team's own code. What follows is rarely the spectacular night-shift crisis. It is the quiet post-mortem in the meetings of the following days: where did this come from, how did it get in, why did the pipeline not flag it earlier?

We see this pattern regularly in intake conversations with Mittelstand IT teams. It is almost always the consequence of a blind spot that looks harmless in isolation but in aggregate makes every supply chain attackable: the own build result is thoroughly inspected. The code the pipeline pulls from outside (base images, dependencies, sidecars, Helm charts) is taken for granted. With that, the trust boundary shifts unnoticed from your repository to dozens of foreign registries, hundreds of foreign maintainers and everything that happens inside their build pipelines.

Impact: every stage is a hand-over point

Each of these stages — base image, package dependency, sidecar, Helm chart — is a hand-over point at which third-party code enters your infrastructure. The attacks on those hand-over points are becoming more professional, not more spectacular.

Where the attacks sit today

• Typosquatting in package managers — the same pattern as in the npm EVM/DeFi cluster of 6 May
• Compromised maintainer accounts — as in the Bitwarden CLI incident of 22 April
• Manipulated post-install scripts in npm and pip packages
• Covertly injected crypto miners in base images — a recurring topic on Docker Hub and Quay
• Manipulated distribution paths without source-code compromise — the dominant class for three years

None of these is a zero-day drama. All of them together are the reason why the IDS eventually reports something nobody asked for.

Who is affected?

At the core: every Mittelstand company running containers in production — whether on Kubernetes, Nomad, ECS or a Docker Swarm island. Three constellations are particularly exposed.

• Setups with a mixed base-image strategy — a build pulls Debian, Alpine, Wolfi and Ubuntu images from three different registries. Audit readiness close to zero.
• Setups with Helm charts from third-party sources — a helm install from a foreign repository silently adds dozens of container images to the cluster. Usually only the primary workload image is visible.
• Setups with an active sidecar strategy — Istio, Linkerd, Datadog agents, ELK sidecars. Every sidecar is its own distribution path. Anyone who does not actively verify it does not control it either.

Anyone running exclusively self-built base images from a single internal registry is significantly less exposed — but has a different obligation: to prove that registry is clean.

Mitigation and immediate actions — four building blocks

Quick start in the order in which we currently push pipelines through — each building block knocks out a specific class of incidents.

# Quick check of an image for signature and SBOM
cosign verify --certificate-identity-regexp=".*" \
  --certificate-oidc-issuer-regexp=".*" \
  registry.example.com/app:v1.2.3
syft registry.example.com/app:v1.2.3 -o cyclonedx-json > sbom.json
grype sbom:sbom.json

1. Signature verification via Sigstore and cosign

Every image that may run in production has a cryptographic signature from a named identity provider. Without a valid signature, no deploy. Period. The admission controller in the cluster (Kyverno, Connaisseur or Sigstore Policy Controller) enforces that hard. No exceptions, not even for "just for a moment" deploys.

2. SBOM, Software Bill of Materials

Every build artifact carries a machine-readable list of its components — ideally CycloneDX. When a new CVE surfaces tomorrow, you know within minutes which of your running workloads are affected. Without an SBOM, you know in days, or never. Dependency-Track as an open-source receiver is enough.

3. Provenance per SLSA

The origin of an artifact is documented machine-readably: which commit, which runner, which parameters, which environment. That is the difference between "we believe the image comes from us" and "we can prove the image comes from us". SLSA Level 2 as a realistic Mittelstand starting point, Level 3 as a maturity target.

4. Continuous re-audit

An image that is clean today may be a CVE candidate in six weeks. Our pipeline therefore re-audits running workloads daily against updated vulnerability databases (Trivy, Grype, Snyk), not only at deploy time. Visible on a dashboard non-technical staff can read.

Detection and verification

Five core questions — in half an hour you know where your supply chain has plain-text gaps today.

• Which base images run in production today — distribution, tag, digest? kubectl get pods -A -o jsonpath='{..image}' as a one-liner, then inventory.
• Which of these images carry a cosign signature you can verify today?
• Which of these images have an SBOM you can read today — or does the SBOM only exist in the pipeline's build-artifact storage?
• Which sidecars and init containers are running that nobody actively approved in a Helm chart or manifest?
• Who on the team can answer within 10 minutes whether a CVE published today even hits any of your workloads?

Quick check we run before every audit:

# All distinct running images in the cluster
kubectl get pods --all-namespaces \
  -o jsonpath='{range .items[*]}{range .spec.containers[*]}{.image}{"\n"}{end}{end}' \
  | sort -u

# Signature verification per image
for img in $(cat images.txt); do
  cosign verify "$img" > /dev/null 2>&1 \
    && echo "OK $img" || echo "FAIL $img"
done

Operator recommendation

What should be operationally in place for which setup — depending on today's maturity.

• If your cluster today admits images from more than one registry without signature verification — then a Kyverno or Connaisseur admission controller in audit mode is the next sprint task. Audit mode means: do not yet block, but make visible what runs without a signature.
• If you do not yet produce SBOMs — thensyft or Trivy SBOM output in the build pipeline is a 2-hour job. Dependency-Track as receiver runs on a small server.
• If you do not yet know SLSA — then GitHub Actions provenance or the GitLab CI component of Sigstore is the pragmatic entry point. Level 2 is realistic in two sprints, Level 3 is the target for the next half-year.
• If you do not yet do continuous scans — then the Trivy Operator in the cluster as a daily run is an entry point without new licence costs. Snyk or Aqua are the enterprise paths.
• If you fall under NIS-2 or the CRA — then audit readiness on which components run in which image is a regulatory obligation. Without SBOM and provenance this obligation cannot be met.

Cross-references: the npm EVM cluster post for mirror topologies, the Bitwarden post for pipeline discipline in detail, the CI/CD concentration post for the structural reasoning.

Conclusion

Supply-chain security is not an add-on you evaluate next quarter. It is the foundation for the fact that what runs at your site is also what you intended to run. Four building blocks — signature, SBOM, provenance, continuous audit — add up to a pipeline that experiences the next IDS event not as a surprise, but as a pre-inventoried finding.

The question is not whether the next manipulated image is coming. It is whether your pipeline would notice it before the IDS does — and whether after noticing you know in the same hour which workloads sit on which component in which version. With the four building blocks the answer moves from "hopefully" to "in two commands".

A longer write-up with concrete examples of our DevSecOps pipeline, the admission-controller configuration and the Dependency-Track integration is available (in German) at ole-hartwig.eu.