One of the Banzai Cloud Pipeline platform’s key open-source projects is Bank-Vaults - the Vault swiss-army knife (and more) for Kubernetes. Feature requirements are part of the Pipeline platform, and the relatively large community around Bank-Vaults also has its own use cases and requirements. We’ve received lots of external contributions (thank you!), and we continue to find time to work on our community-driven features.
While there have been many besides, these are the most sought-after features of the last few weeks. Thankfully, we found the time between our two most recent Pipeline releases to implement them.
The Ingress resource gets configured 🔗︎
The most frequently requested feature for Bank-Vaults’ Kubernetes Vault operator was to automatically create an Ingress resource for the Services of provisioned Vault instances. This was implemented for and tested with NGiNX, Traefik and HAProxy proxy Ingress controllers (support, in this case, means that the relevant TLS marking and optional skip-verify annotations are added to these Ingress resources whenever Vault TLS is enabled, since they don’t all support mounting custom CA certificates).
The relevant part of the full CRD descriptor necessary to request an Ingress for the Vault service is:
1 # Request an Ingress controller with a default configuration
2 ingress:
3 # Specify Ingress object annotations here, if TLS is enabled (which is by default)
4 # the operator will add NGINX, Traefik and HAProxy Ingress compatible annotations
5 # to support TLS backends
6 annotations: {}
7 # Override the default Ingress specification here
8 # This follows the same format as the standard Kubernetes Ingress
9 # See: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#ingressspec-v1beta1-extensions
10 spec: {}Audit devices and outputs getting configured 🔗︎
Vault Audit Devices were another community requested feature that we found particularly useful from a DevOps point of view. From the official Vault documentation:
Audit devices are the components in Vault that keep a detailed log of all requests and responses to Vault. Because every operation with Vault is an API request/response, the audit log contains every authenticated interaction with Vault, including errors.
Creating an S3 fully functional example seemed feasible, but we had to touch a few components in the Bank-Vaults stack to make it work.
First, The Vault Operator had to be extended to configure the fluentd sidecar container through the CRD.
Second, we provided support for Audit Devices by extending bank-vaults configurer to configure Vault via the audit: block in the configurer YAML file (called externalConfig: in the Vault CRD).
In this case, the relevant part of the full CRD descriptor is:
1 fluentdEnabled: true
2
3 # Use a custom fluentd image which has the S3 plugin installed
4 fluentdImage: banzaicloud/fluentd-s3:latest
5
6 fluentdConfig: |
7 <source>
8 @type tail
9 format json
10 keep_time_key true
11 time_format %iso8601
12 path /vault/logs/vault.log
13 pos_file /vault/logs/vault.log.pos
14 tag s3.vault.audit
15 </source>
16 <match s3.*.*>
17 @type s3
18
19 aws_key_id "#{ENV['AWS_ACCESS_KEY_ID']}"
20 aws_sec_key "#{ENV['AWS_SECRET_ACCESS_KEY']}"
21 s3_bucket bank-vaults
22 s3_region eu-west-1
23 path logs/
24
25 time_slice_format %Y%m%d%H
26 time_slice_wait 10m
27 utc
28 </match>
29
30 # Pass a secret to bank-vaults' environment variables.
31 # kubectl create a secret generic aws-access-key \
32 # --from-literal=aws-access-key-id=$AWS_ACCESS_KEY_ID \
33 # --from-literal=aws-secret-access-key=$AWS_SECRET_ACCESS_KEY
34 envsConfig:
35 - name: AWS_ACCESS_KEY_ID
36 valueFrom:
37 secretKeyRef:
38 name: aws-access-key
39 key: aws-access-key-id
40 - name: AWS_SECRET_ACCESS_KEY
41 valueFrom:
42 secretKeyRef:
43 name: aws-access-key
44 key: aws-secret-access-key
45
46 # Tell the bank-vaults configurer to configure Vault to use a file based audit device
47 externalConfig:
48 audit:
49 - type: file
50 description: "File based audit logging device"
51 options:
52 file_path: /vault/logs/vault.log
53 mode: "0640"Inserting “Startup” secrets into Vault 🔗︎
In certain situations, it’s more practical to have a few secrets that aren’t really secrets, but which use the source of secrets as a singular source of truth for all of your configurations; we discussed this in the following community feature request.
The relevant part of the full CRD descriptor necessary to request that Bank-Vaults write “secrets” to Vault during the configuration phase (bootstrap) is:
1 externalConfig:
2 # Allows writing some secrets to Vault (useful for development purposes).
3 # See https://www.vaultproject.io/docs/secrets/kv/index.html for more information.
4 startupSecrets:
5 - type: kv
6 path: secret/data/config
7 data:
8 data:
9 my-config-url: https://localhost:1234
10 flags: "-d -z -c"Operator sync period configuration 🔗︎
One of our users created a new TLS secret with an existing name, and it was overwritten (you can see the issue, here). Scenarios like this can make a service that uses SSL unreachable, so it’s a valid reason to change and/or recycle old TLS certificates. We added a new feature into the operator that resyncs every sixty seconds (configurable), so if a resource is deleted (for example, a TLS certificate) it will be recreated relatively quickly. Also, creating a new resource with a pre-existing name will not overwrite the old resource by default.
In the making 🔗︎
Also based on ideas we received from the community, we’re planning to extend the Vault secrets webhook to source environment variables based on other sources and to add regional fallback support to Bank-Vaults’ Vault unsealing and configuration feature. If you are willing to collaborate, have additional feature requests or need help - as always - feel free to get in touch.
Thank you.
About Banzai Cloud Pipeline 🔗︎
Banzai Cloud’s Pipeline provides a platform for enterprises to develop, deploy, and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operations teams alike. Strong security measures — multiple authentication backends, fine-grained authorization, dynamic secret management, automated secure communications between components using TLS, vulnerability scans, static code analysis, CI/CD, and so on — are default features of the Pipeline platform.
