Amazon S3 plugin for Fluentd 🔗︎

Overview 🔗︎

s3 output plugin buffers event logs in local file and upload it to S3 periodically. This plugin splits files exactly by using the time of event logs (not the time when the logs are received). For example, a log ‘2011-01-02 message B’ is reached, and then another log ‘2011-01-03 message B’ is reached in this order, the former one is stored in “20110102.gz” file, and latter one in “20110103.gz” file.

Example: S3 Output Deployment

Example output configurations 🔗︎

spec:
 s3:
   aws_key_id:
     valueFrom:
       secretKeyRef:
         name: logging-s3
         key: awsAccessKeyId
   aws_sec_key:
     valueFrom:
       secretKeyRef:
         name: logging-s3
         key: awsSecretAccessKey
   s3_bucket: logging-amazon-s3
   s3_region: eu-central-1
   path: logs/${tag}/%Y/%m/%d/
   buffer:
     timekey: 10m
     timekey_wait: 30s
     timekey_use_utc: true*/

Configuration 🔗︎

Output Config 🔗︎

Assume Role Credentials 🔗︎

assume_role_credentials 🔗︎

Instance Profile Credentials 🔗︎

instance_profile_credentials 🔗︎

Shared Credentials 🔗︎

shared_credentials 🔗︎