Managing Proxmox LXC's with OpenTofu and Ansible

I have around ~25 applications running on my homelab at any one time, as well as development VMs and ephemeral development environments (Snaplets). Keeping all of these patched and up to date, whilst remembering how I configured certain resources was proving complex and problematic.

It also left me in a position where deploying new resources or services became a tedious ‘clickops’ journey. This blog will only scratch the surface of Ansible and OpenTofu's capabilities however they both work excellently in combination to both provision my infrastructure and then configure it.

Due to recent license changes to Terraform, I am using OpenTofu, this is a fork of Terraform with a more permissive license. For most users, they will be functionally equivalent.

Step 1: Provision the infrastructure

Both Ansible and Terraform have strengths and weaknesses, Terraform is generally seen as an infrastructure provisioning tool and Ansible is seen as a configuration tool. Whilst both tools overlap in their capabilities, using both of them combined leads to a much cleaner separation of concerns and allows each tool to perform most effectively.

Connecting to Proxmox

Firstly, OpenTofu needs a way to communicate with your Proxmox server, this can be done in the same way you might connect it to AWS, with a provider. My provider.tf looks something like this:

terraform {
  required_providers {
    proxmox = {
      source  = "telmate/proxmox"
      version = "3.0.2-rc01"
    }
  }
}

provider "proxmox" {
  pm_api_url      = var.pm_api_url
  pm_tls_insecure = var.pm_tls_insecure
  pm_user         = var.pm_user
  pm_password     = var.pm_password
}

This provider exposes a number of resources you can provision, these can be found here. I am currently only provisioning LXC's however you can also create VM's with this provider.

Provisioning containers

Once you have set up your OpenTofu project and have added the provider, you are ready to start provisioning containers and VMs. Here is an example snippet that I use to provision my Techtinium DNS server

resource "random_password" "technitium_password" {
  length           = 16
  special          = true
  override_special = "!#$%&*()-_=+[]{}<>:?"
}

resource "proxmox_lxc" "technitium_lxc" {
  target_node     = var.target_node
  hostname        = "technitium-lxc"
  ostemplate      = "local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst"
  password        = random_password.technitium_password.result
  unprivileged    = true
  onboot          = true
  start           = true
  memory          = var.technitium_lxc_memory
  ssh_public_keys = file("~/.ssh/id_ed25519.pub")
  cmode           = "shell"
  nameserver      = "1.1.1.1"

  features {
    nesting = true
  }

  // Terraform will crash without rootfs defined
  rootfs {
    storage = "local-lvm"
    size    = "50G"
  }

  network {
    name     = "eth0"
    bridge   = "vmbr0"
    ip       = "${var.technitium_lxc_ip}/24"
    gw       = var.gateway_ip
    firewall = true
  }

}

To ensure that Ansible can read the IPs allocated to the resources, an OpenTofu argument is also required.

output "technitium_lxc_ip" {
  value = proxmox_lxc.technitium_lxc.network[0].ip
}

Once you have configured all of these into their respective files, you can simply run

tofu init
tofu apply

After about 30 seconds, your LXC will have been provisioned. To destroy this, you can use

tofu destroy

Step 2: Configure the infrastructure

Whilst the above code provisions the infrastructure, we now need a way to configure it and make it all a single-stage process, this is where Ansible comes in. In my stack, Ansible triggers the infrastructure deployment by running the tofu commands, it then gets the IPs and adds them to the 'Ansible inventory' where they can be configured.

Continuing with the Techtinium DNS as a demo, this is a sample Ansible role that provisions it and then adds it to the inventory

---
- name: Initialize OpenTofu
  command: tofu init
  args:
    chdir: ../infrastructure

- name: Apply OpenTofu configuration
  command: >
    tofu apply -auto-approve
    -var pm_api_url={{ pm_api_url }}
    -var pm_user={{ pm_user }}
    -var pm_password={{ pm_password }}
    -var target_node={{ target_node }}
    -var technitium_lxc_memory={{ technitium_lxc_memory }}
    -var technitium_lxc_ip={{ technitium_lxc_ip }}
    -var gateway_ip={{ gateway_ip }}


  args:
    chdir: ../infrastructure

- name: Capture OpenTofu output
  command: tofu output -json
  args:
    chdir: ../infrastructure
  register: tf_output
  changed_when: false

- name: Parse OpenTofu JSON output
  set_fact:
    parsed_tf_output: "{{ tf_output.stdout | from_json }}"

- name: Extract IP without CIDR
  set_fact:
    technitium_ip: "{{ parsed_tf_output.technitium_lxc_ip.value.split('/')[0] }}"

- name: Add Technitium LXC to Ansible inventory
  add_host:
    name: "{{ technitium_ip }}"
    groups: 
      - technitium
      - lxc
    ansible_user: root

Once it has been added to the Ansible inventory, regular Ansible roles and tasks can be run against it. For Technitium, the task looks something like this

---    
- name: Install curl
  apt:
    name: curl
    state: present
    update_cache: yes
  become: yes

- name: Install Technitium
  ansible.builtin.shell: curl -sSL https://download.technitium.com/dns/install.sh | bash

Ansible is an extremely versatile tool that can handle very advanced configuration requirements, whilst this is a very simple demo, it can be used to configure an LXC to your exact specifications.

Step 3: Networking the infrastructure and beyond...

My next stage is setting up a reverse proxy for all the applications hosted on the stack. This will be done via a caddy LXC that is configured with Ansible and its powerful configuration templating system. Once this reverse proxy is configured, I am planning on using Ansible to configure custom firewall rules on my Unifi home network to completely protect the services behind the caddy server.

To configure the firewall, Ansible will use the Unifi gateway API to dynamically create the rules based on the IPs of the provisioned VMs. Proxmox also returns the MAC address, so this could also be used to configure the firewall rules.

Closing thoughts

Overall, leveraging infrastructure as code tooling has massively reduced the pain of updating and managing my homelabs services. It helps me keep it secure and online 24/7, whilst being maintainable and predictable, among all the other benefits of storing your infrastructure as code.

In the future, I would love to schedule a weekly run of the Ansible playbook which would update all of my services to the latest versions, currently, I just run my playbook manually however if I automate it, I may never have to worry about updating again!