Skip to main content
IT BIBLESign in

Forum / Servers

How I Structure Docker Compose Stacks on a Proxmox VM, and Where It Hurts

PriyaInTheCloset OP
#1

Most of my services run in Docker Compose on a couple of Proxmox VMs. It works, but I've been questioning the layout lately, and I'd like to compare notes before I change it.

The current layout

One Git repo, one directory per stack, one Compose project per directory:

stacks/
  traefik/
    compose.yaml
    .env.example
  immich/
    compose.yaml
    .env.example
  monitoring/
    compose.yaml
    prometheus.yml

The real .env files are in .gitignore. Only .env.example is committed, with placeholder values. Persistent data lives in bind mounts under a single path on the VM, so one backup job covers it all.

A trimmed service entry looks like this:

services:
  app:
    image: ghcr.io/example/app:1.2.3
    restart: unless-stopped
    networks:
      - proxy
    env_file: .env
    volumes:
      - /srv/data/app:/data

networks:
  proxy:
    external: true

The proxy network is created once, and the reverse proxy is the only thing publishing ports to the host.

What I like

  • Reproducibility - the VM is a cloud-init template plus Ansible, and the stacks come from Git. Rebuilding is a clone and a docker compose up -d per stack.
  • Pinned tags - I avoid latest, so an update is a deliberate diff instead of a surprise.
  • One data path - backups and restore drills only have to think about one place.

Where it hurts

  • Secrets - the .env files are the weak link. They aren't in Git, so they aren't reproducible. I've read about SOPS and similar tools but haven't tried them yet.
  • Updates - bumping tags is manual. I've seen Renovate mentioned for opening pull requests on image versions, but I haven't set it up, so I can't say how noisy it gets.
  • Blast radius - everything shares one VM's kernel, CPU and RAM. One runaway container can slow down services my family relies on, and I haven't set resource limits consistently.
  • The one container - there is always one that breaks on a major version bump. Pinning helps, but only if I read the release notes first.

What I'm unsure about

I'm not sure whether to split stacks across several smaller VMs (or LXC containers) by function, or keep one Docker host and add limits. Splitting reduces blast radius but multiplies the patching and backup surface. I'm also undecided on whether Kubernetes would solve any of this at my scale, or just trade these problems for new ones.

Questions for you

  1. How do you handle secrets for Compose stacks if you want the whole setup rebuildable from Git?
  2. One big Docker host or several split by function? What made you choose, and did you regret it?
WizardTux Cool Guy
#2

I tend to run multiple docker hosts or via K8s though I'm still pretty new to docker / containers in general.

For secrets wouldn't something like bitwarden secrets manager work?

Sign in to reply.