Skip to content

Run staging and production as separate clusters

This is the fourth step of the Learn path. You put a staging cluster next to production, give each its own release channel and maintenance window, and deploy the same app to both. Each stage then fails, upgrades and restores on its own.

Two clusters for upcheck. upcheck-prod stays on the stable channel with its weekend window. upcheck-staging takes each release first, on the early channel, with a window on a weekday night. Each has its own Hetzner project, its own kubeconfigs, its own API ranges and its own snapshots, and both run the same upcheck.yaml under different names.

What that buys: an upgrade, a restore or a runaway test reaches one stage, and a release runs on staging before production sees it. What it costs: the price twice, €29 incl. VAT per cluster a month, plus two servers at Hetzner’s price.

  • upcheck-prod with upcheck from step 2, and the file upcheck.yaml.
  • A second Hetzner project for staging, with two Read & Write tokens, as in step 1. The token inside a cluster opens its whole project. Whoever can read staging’s token must not reach production’s servers.
  • A bucket for staging’s snapshots, with keys of its own, so that staging’s keys cannot read production’s snapshots.
  • A second host name, for example upcheck-staging.example.com, in your DNS zone.
  1. Open PaaSbox Clusters → Hetzner projects, choose Add a Hetzner project, name it upcheck-staging and paste the first token of the new project.

  2. Choose New cluster. Enter the Name upcheck-staging and choose the new Hetzner project, A new server, the Location of production and the Server type cpx22.

  3. Set the Release channel to early. A release reaches early first and spends seven days there before it reaches stable, so staging runs a release about a week before production can.

  4. Set PaaSbox Platform to saas-http01 and leave With observability (metrics, logs, traces, Grafana) unticked: with Postgres it needs a server with 8 GB, and the form would refuse the cpx22. ACME e-mail is filled in with your address; Let’s Encrypt registers the account under it. The cluster then starts with Flux and the platform switched on, as upcheck-prod has them since step 2.

  5. Under Maintenance window, set the Days to Tuesday only, From 02:00 To 05:00. An upgrade that runs in staging’s window in the early hours of Tuesday can be checked on Tuesday morning, days before production’s weekend window.

  6. Point the snapshots at staging’s bucket, with the Folder upcheck-staging. Open the API only to your address, give the cluster A separate token (recommended) with the second token, and choose Create cluster.

  7. When the cluster is ready, open its Upgrades tab. Under Channel, untick Install patch releases in the window without a click and choose Save. Every release then waits for your click on staging, and step 6 shows you the upgrade as it runs. Production keeps the setting: patch releases install in its window on their own.

  1. On staging’s Access tab, get an admin kubeconfig and download it. Keep both files and name each on every command, so that no command reaches the wrong stage:

    Terminal window
    PROD=~/Downloads/upcheck-prod-admin.kubeconfig
    STAGING=~/Downloads/upcheck-staging-admin.kubeconfig
    kubectl --kubeconfig $STAGING get platform

    Wait until the Platform is READY True; the Add-ons tab then says applied for Flux and PaaSbox Platform.

  2. On staging’s Overview, read the server’s Address under Nodes, and point upcheck-staging.example.com at it with an A record.

  3. Make staging’s copy of the file with its own name, and apply it:

    Terminal window
    sed 's/upcheck.example.com/upcheck-staging.example.com/' upcheck.yaml > upcheck-staging.yaml
    kubectl --kubeconfig $STAGING apply -f upcheck-staging.yaml
    kubectl --kubeconfig $STAGING -n upcheck get saasapp upcheck -w
  4. Compare the two stages:

    Terminal window
    kubectl --kubeconfig $PROD -n upcheck get saasapp
    kubectl --kubeconfig $STAGING -n upcheck get saasapp

    The same image, two URLs. Staging’s database is its own: nothing you do there reaches production’s data.

Both clusters run Flux, which came with the platform. Instead of kubectl apply from your computer, each cluster can pull its stage from one deploy repository: a directory per stage, with spec.image as the one line your CI changes, and production changed only through a pull request. You point Flux at the repository with a GitRepository and a Kustomization of your own. In progress In the lab, Flux ran the platform’s own parts; a deploy repository of your own has not run with PaaSbox yet.

Staging and production as two clusters, each with its own project, channel, window, keys and snapshots, running the same app. The next steps use staging for what you would never try first on production: a restore, then an upgrade.