Files
homelab-v/tofu/kubernetes
Vegard Hagen 67f160fea1 fix(tofu): volume-provisioning
A Proxmox update probably broke/changed the way API requests are handled, giving errors on UPDATE calls with parameters that aren't used

Error: unexpected response code '400': {"data":null,"errors":{"format":"property is not defined in schema and the schema does not allow additional properties","vmid":"property is not defined in schema and the schema does not allow additional properties","size":"property is not defined in schema and the schema does not allow additional properties","filename":"property is not defined in schema and the schema does not allow additional properties"}}

Also update Mastercard/restapi provider
2025-04-17 18:24:03 +02:00
..
2025-04-17 18:24:03 +02:00
2025-04-17 18:24:03 +02:00

Kubernetes Tofu

Install pre-requisites

  1. tofu
  2. talosctl
  3. kubectl

Initialize tofu

tofu init

Proxmox

Environment variable

export TF_VAR_proxmox_api_token="<YOUR_API_TOKEN>"

Optional External Secrets Manager / Other methods

Bitwarden Secrets Manager - Name your secret TF_VAR_proxmox_api_token in bws.

bws run -- tofu ...

Note: By default the shell is sh. Change with --shell if required.

Sealed-secrets

Generate certificate

openssl req -x509 -days 365 -nodes -newkey rsa:4096 -keyout sealed-secrets.key -out sealed-secrets.crt -subj "/CN=sealed-secret/O=sealed-secret"

Kubernetes

Output kubeconfig

tofu output -raw kube_config

Talos

Output talosconfig

tofu output -raw talos_config

Upgrading Talos and Kubernetes

Upgrade talos nodes one by one.

  1. Set talos_image.auto.tfvars -> image -> update_version to the required update version.
  2. Set talos_cluster.auto.tfvars -> talos_cluster_config -> kubernetes_version to the required kubernetes version.
  3. Set talos_nodes.auto.tfvars -> talos_nodes -> $node_1 -> update = true and run tofu apply.
  4. Set talos_nodes.auto.tfvars -> talos_nodes -> $node_2 -> update = true, leave the previous nodes update = true and run tofu apply.
  5. Set talos_nodes.auto.tfvars -> talos_nodes -> $node_3 -> update = true, leave the previous nodes update = true and run tofu apply.
  6. ...
  7. Set talos_nodes.auto.tfvars -> talos_nodes -> $node_n -> update = true, leave the previous nodes update = true and run tofu apply.
  8. After upgrading all nodes, Set talos_image.auto.tfvars -> image -> version to match the update version and set update = false for all nodes.

Upgrading Talos Schematic

  1. Create a new schematic file.
  2. Same process as above instead of image.version and image.update_version, change image.schematic and image.update_schematic, in talos_image.auto.tfvars.

Upgrading Kubernetes Only

Dry Run

sh upgrade-k8s.sh $CONTROLPLANE_NODE_IP --dry-run     # For testing

Upgrade

sh upgrade-k8s.sh $CONTROLPLANE_NODE_IP