Client applications in the same mesh

This scenario covers using Kafka ACLs when your client applications are in the same Istio mesh as the Kafka cluster. In this scenario, the client application is identified as CN=<namespace>-<service-account-name>. For example, if you have a kafkacat application deployed into the kafka namespace with the default service account, it’s identified as CN=kafka-default.

Prerequisites 🔗︎

To use Kafka ACLs with Istio mTLS, you need:

• a Kubernetes cluster (version 1.17 and above), with
• at least 10 vCPU and 12 GB of memory, and
• with the capability to provision LoadBalancer Kubernetes services.
• A Kafka cluster.

Steps 🔗︎

1. Enable ACLs and configure an external listener using Supertubes. Complete the following steps.

   1. Verify that your deployed Kafka cluster is up and running:

      supertubes cluster get --namespace <namespace-of-your-cluster> --kafka-cluster <name-of-your-kafka-cluster> --kubeconfig <path-to-kubeconfig-file>
      

      Expected output:

      Namespace  Name   State           Image                               Alerts  Cruise Control Topic Status  Rolling Upgrade Errors  Rolling Upgrade Last Success
      kafka      kafka  ClusterRunning  banzaicloud/kafka:2.13-2.5.0-bzc.1  0       CruiseControlTopicReady      0
      

   2. Enable ACLs and configure an external listener. The deployed Kafka cluster has no ACLs, and external access is disabled by default. Enable them by applying the following changes:

      supertubes cluster update --namespace kafka --kafka-cluster kafka --kubeconfig <path-to-kubeconfig-file> -f -<<EOF
      apiVersion: kafka.banzaicloud.io/v1beta1
      kind: KafkaCluster
      spec:
        ingressController: "istioingress"
        istioIngressConfig:
          gatewayConfig:
            mode: PASSTHROUGH
      readOnlyConfig: |
          auto.create.topics.enable=false
          offsets.topic.replication.factor=2
          authorizer.class.name=kafka.security.authorizer.AclAuthorizer
          allow.everyone.if.no.acl.found=false
      listenersConfig:
          externalListeners:
          - type: "plaintext"
              name: "external"
              externalStartingPort: 19090
              containerPort: 9094
      EOF
      

   3. The update in the previous step reconfigures the Kafka cluster to receive rolling updates. Verify that this is reflected in the state of the cluster.

      supertubes cluster get --namespace kafka --kafka-cluster kafka --kubeconfig <path-to-kubeconfig-file>
      

      Expected output:

      Namespace  Name   State                    Image                               Alerts  Cruise Control Topic Status  Rolling Upgrade Errors  Rolling Upgrade Last Success
      kafka      kafka  ClusterRollingUpgrading  banzaicloud/kafka:2.13-2.5.0-bzc.1  0       CruiseControlTopicReady      0
      

   4. Wait until the reconfiguration is finished and the cluster is in the ClusterRunning state. This can take a while, as the rolling upgrade applies changes on a broker-by-broker basis.

2. (Optional) At this stage, every listener of the Kafka cluster is configured to the PLAINTEXT protocol. No SSL is configured, mTLS is provided by Istio. You can verify that mTLS is used by trying to connect with openssl (without a certificate) to see that a server certificate is returned and the SSL handshake fails:

   kubectl run openssl --image=frapsoft/openssl --restart=Never -it --rm -- s_client -connect kafka-all-broker.kafka.svc.cluster.local:29092
   

3. Create a default Kafka user called kafka-default, and set its access rights as needed for your environment. For example:

   kubectl create -f -<<EOF
   apiVersion: kafka.banzaicloud.io/v1alpha1
   kind: KafkaUser
   metadata:
   name: kafka-default
   namespace: kafka
   spec:
   clusterRef:
       name: kafka
   secretName: ""
   createCert: false
   topicGrants:
       - topicName: example-topic
         accessType: read
       - topicName: example-topic
         accessType: write
   EOF
   

4. From now on, Kafka applications are automatically authenticated and authorized by their service account.