Set up backups

A cluster’s snapshots go from the server straight to an S3 bucket you own; they never pass through the portal. This page points them at your bucket, sets how often they are taken and how many are kept, and checks that the bucket works. Without a bucket, the snapshots stay on the server’s disk and are lost with the server.

In progress: not bookable yet

Built in a lab: on 2026-10-10 a snapshot reached the bucket 25 seconds after it was asked for, a bucket added to a running cluster was used without a restart of k3s, and new keys replaced the old ones in 8 seconds. A self-hosted S3 server stood in for the bucket: Hetzner Object Storage itself, and keys you hold yourself, are In progress. Tell me when it opens.

What you need

• An S3 bucket and an access key and secret key for it. A bucket at Hetzner Object Storage in the cluster’s location costs least. Give the cluster a folder of its own, for example its name.
• To be a team admin.

Point the snapshots at your bucket

You can fill these fields when you create the cluster, or add the bucket later:

1. Open the cluster’s Backups tab and go to Schedule and target.

2. Choose Who holds the bucket’s keys (below). Enter the S3 endpoint as a host name, with a port if needed, without https:// and without a path, for example fsn1.your-objectstorage.com. Enter the Bucket, the Folder and the Region.

3. Keep the Schedule (cron, UTC) 0 */6 * * *, a snapshot every six hours, and Snapshots to keep at 28, which is seven days; or set your own (below). Choose Save. The node applies it at its next sync.

4. If the portal holds the keys: under Bucket keys, enter the Access key and the Secret key and choose Save new keys. The first keys are used at once.

A bucket added to a running cluster needs no restart of k3s: pbx-agent writes the Secret kube-system/pbx-etcd-s3, and the next snapshot goes to the bucket.

Who holds the bucket’s keys

You choose per cluster: The portal holds the S3 keys (the default) or The customer holds the S3 keys.

	The portal holds the keys	You hold the keys
How the keys reach the server	You paste them into the portal. It stores them encrypted and sends them to pbx-agent sealed to the node’s key; pbx-agent writes them into the Secret kube-system/pbx-etcd-s3, which k3s reads.	You create the Secret kube-system/pbx-etcd-s3 yourself. pbx-agent never creates, changes or deletes it.
Can the portal read your snapshots?	In principle, yes: with the cluster’s server token, which it also holds, the keys could open a snapshot.	No. The portal never sees the keys.
Restore in place	Works also while the Kubernetes API is down.	Works only while the Kubernetes API is up: pbx-agent reads your Secret before it stops k3s.

If you hold the keys, create the Secret with the type etcd.k3s.cattle.io/s3-config-secret and these keys:

apiVersion: v1
kind: Secret
metadata:
  name: pbx-etcd-s3
  namespace: kube-system
type: etcd.k3s.cattle.io/s3-config-secret
stringData:
  etcd-s3-endpoint: fsn1.your-objectstorage.com
  etcd-s3-bucket: my-snapshots
  etcd-s3-folder: upcheck-prod
  etcd-s3-region: fsn1
  etcd-s3-access-key: <access key>
  etcd-s3-secret-key: <secret key>

Switching a cluster from portal-held keys to your own deletes the keys the portal held.

Schedule and retention

• Schedule (cron, UTC): five cron fields of digits, *, /, - and commas. k3s takes the scheduled snapshots itself, so they continue when the portal cannot be reached.
• Snapshots to keep: from 1 to 500. k3s deletes the oldest beyond that number, in the bucket too. Snapshots are compressed.

A change of schedule or retention restarts k3s, so it is applied in the cluster’s next maintenance window. In progress Restarts that wait for the window have not run on a real cluster yet. A new target or new keys need no restart.

The bucket’s lifecycle rules, versioning and access policies are yours. A lifecycle rule that deletes objects can delete snapshots k3s still counts on.

Check that it works

1. Choose Snapshot now on the Backups tab or the Overview. pbx-agent confirms that the snapshot reached the bucket before it reports success.

2. The snapshot appears under Snapshots on the Backups tab, and Where says S3. A snapshot that says Local stayed on the server’s disk.

3. Look in your bucket: the file is in the folder you gave.

From then on, if two scheduled snapshots in a row do not arrive, the portal mails the team’s admins In progress.

New keys

When the portal holds the keys, you replace them under Bucket keys with Save new keys. New keys are tried first: pbx-agent lists the bucket with them, and only when that works do they replace the keys in use. The section shows each check as s3.verify with its result. A wrong pair fails the check, and the keys in use stay. Then revoke the old keys at your S3 provider. Rotate credentials covers this with the cluster’s other keys.

Take and restore snapshotsWhat a snapshot holds, and how to restore one in place.

5 · Practise a restoreLearn step 5: a snapshot, a change, a restore, and what came back.