Each Prometheus instance needs a persistent volume (PV) to store the recorded metrics.
To set up such persistent volumes, modify the Backyards control plane custom resource to allow for acquiring such volumes. Complete the following steps.
-
Find which storage classes does your Kubernetes cluster support. Log in to your cluster, then run the following command:
$ kubectl get storageclass (kubernetes-admin@turip-host/backyards-system) NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 7h17m
Select the appropriate storage class from the output list.
The previous example shows the default AWS settings for EKS or PKE clusters, in this case it is safe to use gp2 volumes.
-
Set up the Persistent Volume.
-
To enable Persistent Volumes for Prometheus, first customize this YAML patch file:
spec: backyards: prometheus: enabled: true storage: apiVersion: v1 kind: PersistentVolumeClaim spec: storageClassName: gp2 accessModes: - ReadWriteOnce resources: requests: storage: "50Gi"
-
Save this patch file as
persistent-storage.patch.yaml
. -
Enter the Storage Class name obtained in the previous step into the
storageClassName
property, and set the storage space required by updating the value ofspec.backyards.prometheus.storage.spec.resources.requests.storage
.
-
-
Backyards is controlled by a
ControlPlane
custom resource found in the Backyard’s namespace (default:backyards-system
) named backyards. To update the Custom Resource with the patch, run the following command.$ kubectl patch controlplane --type=merge --patch "$(cat persistent-storage.patch.yaml)" backyards controlplane.backyards.banzaicloud.io/backyards patched
If you are using Backyards in operator mode, skip this step.
Otherwise, execute a reconciliation so Backyards updates your Kubernetes cluster to the desired state described by the ControlPlane Custom Resource. Run the following command:
$ backyards operator reconcile
-
Verify that Persistent Volumes exist. To verify that Prometheus has the physical volumes attached, run the following command:
$ kubectl get pv -n backyards-system NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-79cc1a86-bcfa-46a9-a0e7-24e38d001d29 50Gi RWO Delete Bound backyards-system/prometheus-backyards-prometheus-db-prometheus-backyards-prometheus-0 gp2 7m10s
The list shows that a Physical Volume (named
backyards-system/prometheus-backyards-prometheus-db-prometheus-backyards-prometheus-0
in the example) has been created and attached (Bound
status).
Next steps 🔗︎
If you haven’t done yet, Set up High Availability for the Monitoring Stack.