Set up High Availability for the Monitoring Stack

To ensure that Prometheus is always available, enable the High Availability mode of Backyards. That way the monitoring stack uses multiple Prometheus instances and Thanos Query to ensure metric consistency and high availability. Complete the following steps.

Enable high availability. Backyards is controlled by a ControlPlane custom resource found in the Backyard’s namespace (default: backyards-system) named backyards. The following commands change the spec.backyards.highAvailability.enabled value to true, indicating that you want to have a Highly Available deployment.

Run the following commands:
```
$ cat > enable-ha.yaml <<EOF
spec:
  backyards:
    highAvailability:
      enabled: true
EOF

$ kubectl patch controlplane --type=merge --patch "$(cat enable-ha.yaml)" backyards
```
If you are using Backyards in operator mode, skip this step.

Otherwise, execute a reconciliation so Backyards updates your Kubernetes cluster to the desired state described by the ControlPlane Custom Resource. Run the following command:
```
$ backyards operator reconcile
```

Verify that High Availability is enabled. To verify that Prometheus is running in Highly Available mode, check the pods running in the backyards-system namespace for multiple Prometheus instances. Run the following command.
```
$ kubectl get pods -l app=prometheus -n backyards-system
NAME                                READY   STATUS    RESTARTS   AGE
prometheus-backyards-prometheus-0   5/5     Running   1          21m
prometheus-backyards-prometheus-1   5/5     Running   1          21m
```
If two running pods are present then the High Availability setup has been completed.

Next steps 🔗︎

Configure Backyards to use an existing or a new Alertmanager deployment.