Fluent Bit is an open source and multi-platform Log Processor and Forwarder which allows you to collect data/logs from different sources, unify and send them to multiple destinations.

Logging operator uses Fluent Bit as a log collector agent: Logging operator deploys Fluent Bit to your Kubernetes nodes where it collects and enriches the local logs and transfers them to a log forwarder instance.

You can configure the Fluent Bit deployment via the fluentbit section of the The Logging custom resource. This page shows some examples on configuring Fluent Bit. For the detailed list of available parameters, see FluentbitSpec.

Filters 🔗︎

Kubernetes (filterKubernetes) 🔗︎

Fluent Bit Kubernetes Filter allows you to enrich your log files with Kubernetes metadata. For example:

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: default-logging-simple
spec:
  fluentd: {}
  fluentbit:
    filterKubernetes:
       Kube_URL: "https://kubernetes.default.svc:443"
       Match: "kube.*"
  controlNamespace: logging

For the detailed list of available parameters for this plugin, see FilterKubernetes. More info

Tail input 🔗︎

The tail input plugin allows to monitor one or several text files. It has a similar behavior like tail -f shell command. For example:

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: default-logging-simple
spec:
  fluentd: {}
  fluentbit:
    inputTail:
       Refresh_Interval: "60"
       Rotate_Wait: "5"
  controlNamespace: logging

For the detailed list of available parameters for this plugin, see InputTail. More Info.

Buffering 🔗︎

Buffering in Fluent Bit places the processed data into a temporal location until is sent to Fluentd. By default, the Logging operator sets storage.path to /buffers and leaves fluent-bit defaults for the other options.

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: default-logging-simple
spec:
  fluentd: {}
  fluentbit:
    bufferStorage:
       storage.path: /buffers
  controlNamespace: logging

For the detailed list of available parameters for this plugin, see BufferStorage. More Info.

HostPath volumes for buffers and positions 🔗︎ 

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: default-logging-simple
spec:
  fluentd: {}
  fluentbit:
    bufferStorageVolume:
      hostPath:
        path: "" # leave it empty to automatically generate
    positiondb:
      hostPath:
        path: "" # leave it empty to automatically generate
  controlNamespace: logging

Custom Fluent Bit image 🔗︎

You can deploy custom images by overriding the default images using the following parameters in the fluentd or fluentbit sections of the logging resource.

The following example deploys a custom fluentd image:

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: default-logging-simple
spec:
  fluentd:
    image:
      repository: banzaicloud/fluentd
      tag: v1.10.4-alpine-1
      pullPolicy: IfNotPresent
  fluentbit: {}
  controlNamespace: logging

Volume Mount 🔗︎

Defines a pod volume mount. For example:

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: default-logging-tls
spec:
  fluentd: {}
  fluentbit:
    extraVolumeMounts:
    - source: /opt/docker
      destination: /opt/docker
      readOnly: true
  controlNamespace: logging

For the detailed list of available parameters for this plugin, see VolumeMount.

Custom Fluent Bit annotations 🔗︎ 

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: default-logging-simple
spec:
  fluentd: {}
  fluentbit:
    annotations:
      my-annotations/enable: true
  controlNamespace: logging

KubernetesStorage 🔗︎

Define Kubernetes storage.

Name Type Default Description
hostPath HostPathVolumeSource - Represents a host path mapped into a pod. If path is empty, it will automatically be set to “/opt/logging-operator//” 
emptyDir EmptyDirVolumeSource - Represents an empty directory for a pod. 

CPU and memory requirements 🔗︎

To adjust the CPU and memory limits and requests of the pods managed by Logging operator, see CPU and memory requirements.

Probe 🔗︎

A Probe is a diagnostic performed periodically by the kubelet on a Container. To perform a diagnostic, the kubelet calls a Handler implemented by the Container. You can configure a probe for Fluent Bit in the livenessProbe section of the The Logging custom resource. For example:

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: default-logging-simple
spec:
  fluentbit:
    livenessProbe:
      periodSeconds: 60
      initialDelaySeconds: 600
      exec:
        command:
        - "/bin/sh"
        - "-c"
        - >
          LIVENESS_THRESHOLD_SECONDS=${LIVENESS_THRESHOLD_SECONDS:-300};
          if [ ! -e /buffers ];
          then
            exit 1;
          fi;
          touch -d "${LIVENESS_THRESHOLD_SECONDS} seconds ago" /tmp/marker-liveness;
          if [ -z "$(find /buffers -type d -newer /tmp/marker-liveness -print -quit)" ];
          then
            exit 1;
          fi;
  controlNamespace: logging

You can use the following parameters:

Name Type Default Description
initialDelaySeconds int 10 Number of seconds after the container has started before liveness probes are initiated.
timeoutSeconds int 0 Number of seconds after which the probe times out.
periodSeconds int 10 How often (in seconds) to perform the probe.
successThreshold int 0 Minimum consecutive successes for the probe to be considered successful after having failed.
failureThreshold int 3 Minimum consecutive failures for the probe to be considered failed after having succeeded.
exec array {} Exec specifies the action to take. More info
httpGet array {} HTTPGet specifies the http request to perform. More info
tcpSocket array {} TCPSocket specifies an action involving a TCP port. More info

Note: To configure readiness probes, see Readiness probe.

Edit this page on GitHub