Skip to content

Architecture: why this is shaped the way it is

Origin and actual goal

homelab-iac-common + homelab-platform + per-service repos (homelab-homepage, homelab-observability, caddy-iac, tailscale-iac) were originally built to prove out scalability patterns - Terraform + Ansible + CI, shared via a versioned collection and toolbox image. It's reasonable to look at that and wonder if it's overbuilt for a homelab.

The goal that actually matters is disaster recovery: rebuild the whole homelab on new hardware, assuming the source drives are gone, but TrueNAS (with its data) and the Bitwarden vault survive.

That distinction changes the calculus on any "is this too complicated" question. For "make it easy to add one more container," the pipeline can look oversized - as of the 2026-08-05 review, caddy-iac proxies 15 hostnames, of which only one (homepage) goes through the full Terraform → Ansible → CI → compose_stack pipeline; the rest are pre-existing appliances or TrueNAS SCALE apps that bypass it entirely, and apps01 is explicitly sized for "10-15 small services" but runs two (homepage, plus the host-level alloy-agent). A Portainer-based alternative looks attractive under that framing.

But for "rebuild from bare metal with nothing but git + Bitwarden," the Terraform/Ansible/versioned-artifact approach is the right shape, not overcomplicated. A Portainer-based alternative would leave Portainer's own state (endpoints, stacks, users) as exactly the kind of undeclared, non-git-backed state this design avoids. Don't re-litigate toward Portainer without re-litigating this tradeoff specifically - the DR goal is the reason the current shape is justified.

What survives a lost-all-local-drives event, by construction

  • Git history (GitHub)
  • Bitwarden vault (cloud service)
  • Container images (pulled from registries on deploy)
  • The toolbox dev image itself (GHCR)
  • VM/host definitions (config/hosts.yaml, committed)

What does not, as of 2026-08-06

  • Proxmox hypervisor config - network bridges, VLANs, storage pools - isn't captured as code anywhere. Tracked in proxmox-iac.
  • pfSense config - DHCP, and the Unbound DNS overrides that make *.flomico.com resolve internally - isn't captured as code or documented. Not yet tracked as its own issue.

Everything else in the fleet-of-repos rebuilds from git + Bitwarden alone; these two are the layer beneath that, and would need to be rebuilt by hand from memory today.

Status of the pattern itself

As of the 2026-08-06 apps01 rebuild drill, the core claim - "this fleet can be rebuilt from git + Bitwarden alone" - has been exercised end to end for the first time: apps01 was destroyed and recreated via Terraform, rebaselined via Ansible, and homelab-homepage was redeployed onto the new host via a clean run of its own CI, with no manual shortcuts. See that page for what was verified and what wasn't.

Known, already-tracked gaps (not new findings - don't re-flag these)

  • homelab-iac-common issue #1 - CI secrets are hand-copied per repo and recorded nowhere; rotation is O(repos) with no completeness check.
  • homelab-iac-common issue #2 - closed as of 2026-08-08. homelab-ci is authorized on apps01 and obs01 via docker_host_deploy_keys, and on the Caddy VM by caddy-iac's own playbook - it doesn't apply docker_host, and Terraform's user_account.keys can't touch a live VM, so it carries an ansible.posix.authorized_key task of its own. Every service pipeline now uses the dedicated key rather than the operator's personal one. The personal key remains authorized on every host - that is the operator's own access and the rollback path, not an oversight.
  • homelab-iac-common issue #5 - deploy-stack.yml's toolbox_tag default silently falls behind releases. Relevant to DR specifically: an unpinned default means a future rebuild could silently use a different toolbox than what was actually running when it mattered.
  • caddy-iac / tailscale-iac still carry their own copies of the Terraform and Bitwarden plumbing instead of using homelab-iac-common - migrating them is outstanding work, not the design.