At Banzai Cloud we do a lot more than work on Pipeline, our container management platform, and PKE, our lightweight CNCF certified Kubernetes distribution. In fact, we’re currently developing a variety of services that run on Kubernetes. These range from operators (Istio, Vault, Kafka, Logging, HPA to name a few), webhooks, K8s and cloud controllers to more general applications that we develop and test each day. During the development phase for these projects, we usually need to experiment and rapidly iterate applications, but, when using Kubernetes, this isn’t as easy as running each application inside a container. Applications have to be built, container images produced (manually or by our CI), and pushed to a container registry then (re)deployed. Obviously that’s somewhat less than ideal, so we developed a small utility called kurun to hack this development cycle and shorten it significantly.
kurun 🔗︎
Kurun or ku-run (Kubernetes run if you’d prefer, after go run) began its life as a small shell script hack in May, 2018. It was capable of running Go applications on top of Kubernetes, using a single command (we work mostly in Golang). Back then, we wrote a small blog about kurun on it’s release. You might want to use that to catch up. Since our initial release, Google has done something very similar with ko, but kurun is much simpler. The original implementation was in Bash, but it was recently rewritten in Go to make it more extendable. Since the rewrite, besides seeing use as a development tool, it’s been employed as a helper application in the CI acceptance test of Bank-Vaults, in order to run Go examples inside Kubernetes.
We’ve also extended it with a new port-forward command, making it eaven easier to develop applications for Kubernetes (top-levels commands are now behind the run subcommand).
kurun port-forward 🔗︎
In the 0.2.0 release of kurun we’re introducing the ability to securely access external applications as Kubernetes Services, from your laptop or from any other host. This feature is called port-forward after the kubectl subcommand of the same name, since it does the same thing just the other way around.
kurun port-forward utilizes inlets and kubectl port-forward.
inlets is a pretty cool ngrok alternative by Alex Ellis
It also gives you the option of running ghostunnel by Square, which provides TLS termination in front of an inlets server in the same Pod. This inlets server Pod is connected to the inlets client on your machine with kubectl port-forward. The client then proxies application traffic to the server.
- inlets: combines a reverse proxy and websocket tunnel to expose your internal and development endpoints to the public internet via an exit node (here, we’ve replaced the internet with a Kubernetes cluster)
- ghostunnel: is a simple SSL/TLS proxy with mutual authentication for securing non-TLS services (inlets doesn’t support TLS by default)
Note: we use
kurunto test lots of Kubernetes webhooks, thus we need to add TLS support, as webhooks can be used only with TLS.
Note: Inlets doesn’t provide TLS termination, but its websocket transport can use TLS, see the docs.
The following command is a demonstration of how to port-forward:
$ kurun port-forward localhost:8443 --namespace vault-infra --servicename vault-secrets-webhook --tlssecret vault-secrets-webhook
This command proxies the application listening on your machine’s port 8443 into the vault-infra Kubernetes namespace under the Service name vault-secrets-webhook. Because we’ve referenced a Kubernetes TLS Secret named vault-secrets-webhook, a ghostunnel container will be started next to an inlets server which (ghostunnel) can terminate TLS for us, once we provide the certificate and private key found in the Secret (this kurun flag is optional).
Note: If the Service referenced by --servicename already exists kurun will not bail out, instead, it checks the selector field and labels its Pod accordingly, so traffic can find its way into the Pod. It basically replaces the application in the Kubernetes cluster with the local one for debugging purposes.
In the following screencast, you can see kurun port-forward in action. We install the Vault Kubernetes mutating webhook with Helm, setting the replicaCount to 0, so the Service won’t get any Endpoints by default. In the next step, we start the webhook on our laptop and port forward it with kurun with TLS termination (since Kubernetes webhooks can only be used with TLS). We exercise the webhook with a simple test deployment to make sure it will receive the Kubernetes AdmissionReview request:
A final point 🔗︎
We all like working on our own machines during the development of applications, and for debugging there’s usually no better place. However, kurun's potential doesn’t stop there: for example, during the Bank-Vaults acceptance test we’ve employed kurun in the Makefile, so that make -j webhook-up starts locally and wires into a Kubernetes Service. That way, we don’t have to go through the labor intensive steps of developing the code, building the Docker image, pushing it to a registry and redeploying after every line of code change. Combining kurun with something like go-watcher, which detects changes in your Go files and restarts applications automatically, makes the development flow a breeze, and makes it so we don’t have to repeat deployment and restart commands over and over again.
About Banzai Cloud Pipeline 🔗︎
Banzai Cloud’s Pipeline provides a platform for enterprises to develop, deploy, and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operations teams alike. Strong security measures — multiple authentication backends, fine-grained authorization, dynamic secret management, automated secure communications between components using TLS, vulnerability scans, static code analysis, CI/CD, and so on — are default features of the Pipeline platform.
