Operator driven API security testing based on OpenAPI definition

Last autumn we open-sourced the dast-operator which helps checking web applications for security vulnerabilities. The first version was able to initiate a simple dynamic application security test based on custom resources and service annotations. To read more about the first version please check our Dynamic application security testing in Kubernetes blog post.

Today we are happy to announce that we are now extending the operator capabilities with a few new features to facilitate testing APIs as well.

To learn more about the different security aspects of the Pipeline platform, from our Vault operator and dynamic secret injection to pod security policies, network policies, Dex integration, CIS benchmarks, unpriviledged image builds, vulnerability scans, Istio CNI plugin and lots more, check our security-related blog posts.

Updates and new features 🔗︎

We updated the used packages and simplified the codebase. The raw webhook implementation was replaced with a controller-runtime based one, so we had to focus only on the business logic. After refactoring, we were prepared to implement new features:

• OWASP Zed Attack Proxy (ZAP) configuration can be set by dast.security.banzaicloud.io custom resource.
• OpenAPI based security testing

As we mentioned in our previous blog post, the dast-operator is running two reconcilers and one validating admission webhook to prevent vulnerable services becoming exposed.

The overall architecture looks like this:

Implementation of custom resource defined ZAP configuration 🔗︎

Implementing the configuration capabilities was straightforward. We had to add a new field to the ZaProxy struct:

type ZaProxy struct {
	Image     string   `json:"image,omitempty"`
	Name      string   `json:"name"`
	NameSpace string   `json:"namespace,omitempty"`
	APIKey    string   `json:"apikey,omitempty"`
	Config    []string `json:"config,omitempty"`
}

The config contains configurations as a string slice, and the dast reconciler creates the ZAP deployment using these configuration parameters as well. Using this feature we can set up authentication or replace some fields which can be useful for scanning APIs.

Implementation of OpenAPI based scan 🔗︎

While the feature above needed changes only in the dast-operator, initiating API based scans requires some changes in the dynamic-analyzer codebase as well. First of all, we had to implement a new subcommand responsible for the OpenAPI based scan. Before the scan, the OpenAPI definition must be imported and in this case, spidering the target isn’t necessary because the endpoints are properly defined.

Unfortunately, the zap-api-go doesn’t contain a solution for importing OpenAPI definitions, thus we had to fork it and replace it in the go.mod of the dynamic-analyzer until our pull request is merged.

func (o Openapi) ImportUrl(openapiURL, target string) (map[string]interface{}, error) {
	params := map[string]string{
		"url": openapiURL,
	}
	if target != "" {
		params["hostOverride"] = target
	}
	return o.c.Request("openapi/action/importUrl/", params)
}

Now we could import the OpenAPI URL and initiate active scan.

	_, err = client.Openapi().ImportUrl(openapiURL, target)
	if err != nil {
		log.Fatal(err)
	}
	urls, err := client.Core().Urls(target)
	if err != nil {
		log.Fatal(err)
	}

	if len(urls) == 0 {
		log.Print("Failed to import any URLs")
	}

	resp, err := client.Ascan().Scan(target, "True", "False", "", "", "", "")
	if err != nil {
		log.Fatal(err)
	}

A new implementation of the validation webhook 🔗︎

As it’s mentioned, we refactored the webhook implementation in order to simplify this part of the codebase. Now we let the controller-runtime do the heavy lifting. The two main rewards of the refactoring are:

• the capability that cert-manager can issue certificates for the webhook, and that
• the ValidatingWebhookConfiguration is generated based on code comments.

The line responsible for generating the ValidatingWebhookConfiguration:

// +kubebuilder:webhook:path=/ingress,mutating=false,failurePolicy=fail,groups="extensions",resources=ingresses,verbs=create,versions=v1beta1,name=dast.security.banzaicloud.io

Defining the interface for the webhook:

// IngressValidator implements Handle
type IngressValidator interface {
	Handle(context.Context, admission.Request) admission.Response
}

// NewIngressValidator creates new ingressValidator
func NewIngressValidator(client client.Client, log logr.Logger) IngressValidator {
	return &ingressValidator{
		Client: client,
		Log:    log,
	}
}

type ingressValidator struct {
	Client  client.Client
	decoder *admission.Decoder
	Log     logr.Logger
}

// ingressValidator validates ingress.
func (a *ingressValidator) Handle(ctx context.Context, req admission.Request) admission.Response {

The IngressValidator interface must implement the Handle function which contains the validation logic.

Registering the webhook to the webhook server:

		hookServer := mgr.GetWebhookServer()
		hookServer.Register("/ingress", &webhook.Admission{Handler: webhooks.NewIngressValidator(mgr.GetClient(), ctrl.Log.WithName("webhooks").WithName("Ingress"))})

Deploying the operator 🔗︎

Complete the following steps to deploy the operator.

1. First of all, you need to deploy the cert-manager

   kubectl create namespace cert-manager
   helm repo add jetstack https://charts.jetstack.io
   helm repo update
   kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.15.1/cert-manager.crds.yaml
   helm install cert-manager jetstack/cert-manager --namespace cert-manager --version v0.15.1
   

   To read more about the installation of cert-manager please check the official documentation.

2. Clone the dast-operator

   git clone https://github.com/banzaicloud/dast-operator.git
   cd dast-operator
   

3. Build the Docker images

   make docker-build
   make docker-analyzer
   

4. If you are using a Kind cluster for testing, load the images into.

   kind load docker-image banzaicloud/dast-operator:latest
   kind load docker-image banzaicloud/dast-analyzer:latest
   

5. Deploy the dast-operator

   make deploy
   

Examples 🔗︎

After deploying the operator, follow these steps to test the dast-operator. As a test application we use the open-source modern-go-application, a collection of best practices in Go development that contains OpenAPI definition as well.

Importing an OpenAPI definition 🔗︎

1. First, deploy ZAP to the zaproxy namespace by applying the dast.security.banzaicloud.io CustomResource.

   kubectl create ns zaproxy
   kubectl apply -f config/samples/security_v1alpha1_dast.yaml -n zaproxy
   

2. After you have installed the dast-operator and the dast.security.banzaicloud.io custom resource, you can initiate an internal scan using proper annotation in the Kubernetes service manifest. You have to define the OWASP ZAP name and namespace in the way described in the our earlier blog post. For using the API scan feature, set the following new annotations as well:

   • turn on the a API scan dast.security.banzaicloud.io/apiscan
   • define the OpenAPI URL dast.security.banzaicloud.io/openapi-url

3. Deploy the objects to the test namespace:

   kubectl create ns test
   kubectl apply -f config/samples/test-api.yaml -n test
   

   Use the following example service definition:

apiVersion: v1
kind: Service
metadata:
  name: test-api-service
  annotations:
    dast.security.banzaicloud.io/zaproxy: "dast-test"
    dast.security.banzaicloud.io/zaproxy-namespace: "zaproxy"
    dast.security.banzaicloud.io/apiscan: "true"
    dast.security.banzaicloud.io/openapi-url: "https://raw.githubusercontent.com/sagikazarmark/modern-go-application/master/api/openapi/todo/openapi.yaml"
spec:
  selector:
    app: mga
    secscan: dast
  ports:
  - port: 8000
    targetPort: 8000

After the test application is deployed, the following happens

1. The service reconciler of the dast-operator watches the service creations.
2. When dast.security.banzaicloud.io/apiscan is set true, the operator creates an analyzer job using the proper command and OpenAPI URL defined in dast.security.banzaicloud.io/openapi-url.
3. The analyzer job runs the apiscan command, which imports the OpenAPI definition and starts the active scan.

Authentication 🔗︎

In the dast.security.banzaicloud.io custom resource you can define some replacement rules which implement authentication through header manipulation during the scan.

apiVersion: security.banzaicloud.io/v1alpha1
kind: Dast
metadata:
  name: dast-sample
spec:
  zaproxy:
    name: dast-test
    apikey: abcd1234
    config:
      - "replacer.full_list(0).description=auth"
      - "replacer.full_list(0).enabled=true"
      - "replacer.full_list(0).matchtype=REQ_HEADER"
      - "replacer.full_list(0).matchstr=Authorization"
      - "replacer.full_list(0).regex=false"
      - "replacer.full_list(0).replacement=Bearer AbCdEf123456"

Formhandler 🔗︎

Similarly to the example of the authentication, in spec.zaproxy.config some rules can be defined to handle fields of OpenAPI.

    config:
      - "formhandler.fields.field(0).fieldId=id"
      - "formhandler.fields.field(0).value=example-todo-id"
      - "formhandler.fields.field(0).enabled=true"

Thanks to these additional configuration parameters, the API scanning capability becomes more useful.

The dast-operator roadmap 🔗︎

• API testing with JMeter and ZAP
• Parameterized security payload with fuzz
• Automated SQL injection testing using SQLmap

If you’d like to add your feature requests, PR, or just add a GitHub star, feel free to visit the dast-operator repository. Thank you!