Image scan

Banzai Cloud Pipeline helps you secure your infrastructure by scanning the images used in your workloads for vulnerabilities. The Banzai Cloud Pipeline platform uses the Anchore-engine which is a third-party application that can scan container images.

The Security scan integrated service in Pipeline can use either an external Anchore-engine, or a locally deployed one.

When you enable the security scan integrated service, Pipeline deploys the resources necessary to enforce your active security policy, for example to reject creating vulnerable pods. You can choose a predefined policy or compose your own.

Architecture 🔗︎

The security scan integrated service provided by Pipeline relies on Anchore Image Validator which is an admission server that registers itself in the Kubernetes cluster as a validating webhook thus it will validate any Pod deployed to the cluster. The admission server inspects the images that are defined in PodSpec using the Anchore Engine endpoint running as part of Banzai Cloud Pipeline. Based on that response, the admission hook can decide whether to accept or reject that deployment.

Key aspects of container image vulnerability scans 🔗︎

• Every image should be scanned, no matter where it comes from (for example, deployment, operator, and so on)
• It should be possible to set up policies with certain rules to allow or reject a pod
• These policies should be associated with clusters
• If a policy rejects an image, pods using the image should not be created
• There should be an easy way to whitelist a Helm deployment

How it works 🔗︎

If the Anchore-engine is deployed as part of the Banzai Cloud Pipeline platform, Pipeline automates the following steps:

• Generates a technical user in Anchore for each cluster created.
• Saves the generated credentials to Vault — Pipeline’s main secret store
• Creates the predefined Policy bundles
• Deploys the Validating Admission Webhook configured with the previously created credentials

Pipeline provides a RESTful API for the management of policies and the inspection of scan results.

Predefined policy bundles 🔗︎

To simplify bootstrapping, we have predefined basic policy bundles for Anchore:

• Allow all and warn bundle: This policy is the most permissive. One can deploy anything, but it receives feedback about all the deployed images.
• Reject critical bundle: Reject images that contain vulnerabilities with Critical CVSS Severity Rating.
• Reject high bundle: Reject images that contain vulnerabilities with High CVSS Severity Rating.
• Block root bundle: Prevent deploying images that contain apps running with root privileges.
• Deny all images: Reject every image, except the ones explicitly permitted by a whitelist.

Example: creating a pod while the deny all policy is active 🔗︎

$ banzai cluster shell --cluster-name image-scan
[image-scan]$ kubectl run --generator=run-pod/v1 busybox1 --image=busybox -- sleep 3600
Error from server: admission webhook "anchore-anchore-policy-validator.admission.anchore.io" denied the request: Image failed policy check: busybox

Whitelisting deployments 🔗︎

In Banzai Cloud Pipeline (Pipeline) you can whitelist specific Helm deployments using custom Kubernetes resource definitions (CRDs). The Security scan admission hook will accept deployments that match any of the whitelist elements, regardless of its scan result. For details, see Whitelist images.

Scan Events (Audit logs) 🔗︎

Finding the result of an admission hook decision can be troublesome, so we introduced the Audit custom resource. With this custom resource it’s easy to track the result of each scan. Instead of searching in events, you can also easily filter these resources with kubectl. The CRD structure includes the following data:

• releaseName Scanned release
• resource Scanned resource (Pod)
• image Scanned images (in Pod)
• result Scan results (per image)
• action Admission action (allow, reject)

$ kubectl describe audits busybox1
Name:         busybox1
Namespace:
Labels:       fakerelease=true
Annotations:  <none>
API Version:  security.banzaicloud.com/v1alpha1
Kind:         Audit
Metadata:
  Creation Timestamp:  2019-09-16T08:25:39Z
  Generation:          3
  Resource Version:    5670
  Self Link:           /apis/security.banzaicloud.com/v1alpha1/audits/busybox1
  UID:                 9093b50f-d85b-11e9-8f41-42010a8a01a2
Spec:
  Action:  allowed
  Image:
    Image Digest:  sha256:dd97a3fe6d721c5cf03abac0f50e2848dc583f7c4e41bf39102ceb42edfd1808
    Image Name:    busybox
    Image Tag:     latest
    Last Updated:  2019-09-16T08:35:24Z
  Release Name:    busybox1
  Resource:        Pod
  Result:
    Image failed policy check: busybox
Status:
  State:
Events:   <none>

During image scans, Admission server logs result to audits.security.banzaicloud.com and set their ownerReferences to the scanned Pod’s parent. This provides us with a compact overview of the resources running on the cluster. Because these events are bound to Kubernetes resources, it allows for the cluster to clean them up when the original resource (pod) is no longer present.