This guide describes how to collect application and container logs in Kubernetes using the Logging operator, and how to send them to CloudWatch.
The following figure gives you an overview about how the system works. The Logging operator collects the logs from the application, selects which logs to forward to the output, and sends the selected log messages to the output. For more details about the Logging operator, see the Logging operator overview.
Deploy the Logging operator and a demo Application 🔗︎
Install the Logging operator and a demo application using Helm.
Deploy the Logging operator with Helm 🔗︎
To install the Logging operator using Helm, complete these steps.
Note: For the Helm-based installation you need Helm v3.2.1 or later.
-
Add the chart repository of the Logging operator using the following commands:
helm repo add banzaicloud-stable https://kubernetes-charts.banzaicloud.com helm repo update
-
Install the Logging operator.
helm upgrade --install --wait --create-namespace --namespace logging logging-operator banzaicloud-stable/logging-operator
-
Install the demo application and its logging definition.
helm upgrade --install --create-namespace --namespace logging logging-demo banzaicloud-stable/logging-demo \ --set "cloudwatch.enabled=True" \ --set "cloudwatch.aws.secret_key=" \ --set "cloudwatch.aws.access_key=" \ --set "cloudwatch.aws.region=" \ --set "cloudwatch.aws.log_group_name=" \ --set "cloudwatch.aws.log_stream_name="
-
Create logging
Namespace
kubectl create ns logging
-
Create AWS
secret
If you have your
$AWS_ACCESS_KEY_ID
and$AWS_SECRET_ACCESS_KEY
set you can use the following snippet.kubectl -n logging create secret generic logging-cloudwatch --from-literal "awsAccessKeyId=$AWS_ACCESS_KEY_ID" --from-literal "awsSecretAccessKey=$AWS_SECRET_ACCESS_KEY"
Or set up the secret manually.
kubectl -n logging apply -f - <<"EOF" apiVersion: v1 kind: Secret metadata: name: logging-cloudwatch type: Opaque data: awsAccessKeyId: <base64encoded> awsSecretAccessKey: <base64encoded> EOF
-
Create the
logging
resource.kubectl -n logging apply -f - <<"EOF" apiVersion: logging.banzaicloud.io/v1beta1 kind: Logging metadata: name: default-logging-simple spec: fluentd: {} fluentbit: {} controlNamespace: logging EOF
Note: You can use the
ClusterOutput
andClusterFlow
resources only in thecontrolNamespace
. -
Create an CloudWatch
output
definition.kubectl -n logging apply -f - <<"EOF" apiVersion: logging.banzaicloud.io/v1beta1 kind: Output metadata: name: cloudwatch-output namespace: logging spec: cloudwatch: aws_key_id: valueFrom: secretKeyRef: name: logging-cloudwatch key: awsAccessKeyId aws_sec_key: valueFrom: secretKeyRef: name: logging-cloudwatch key: awsSecretAccessKey log_group_name: operator-log-group log_stream_name: operator-log-stream region: us-east-1 auto_create_stream: true buffer: timekey: 30s timekey_wait: 30s timekey_use_utc: true EOF
Note: In production environment, use a longer
timekey
interval to avoid generating too many objects. -
Create a
flow
resource.kubectl -n logging apply -f - <<"EOF" apiVersion: logging.banzaicloud.io/v1beta1 kind: Flow metadata: name: cloudwatch-flow spec: filters: - tag_normaliser: {} - parser: remove_key_name_field: true reserve_data: true parse: type: nginx match: - select: labels: app.kubernetes.io/name: log-generator localOutputRefs: - cloudwatch-output EOF
-
Install the demo application.
kubectl -n logging apply -f - <<"EOF" apiVersion: apps/v1 kind: Deployment metadata: name: log-generator spec: selector: matchLabels: app.kubernetes.io/name: log-generator replicas: 1 template: metadata: labels: app.kubernetes.io/name: log-generator spec: containers: - name: nginx image: banzaicloud/log-generator:0.3.2 EOF
Validate the deployment 🔗︎
If you don’t get the expected result you can find help in the troubleshooting section.