Most of my services run in Docker Compose on a couple of Proxmox VMs. It works, but I've been questioning the layout lately, and I'd like to compare notes before I change it.
The current layout
One Git repo, one directory per stack, one Compose project per directory:
stacks/
traefik/
compose.yaml
.env.example
immich/
compose.yaml
.env.example
monitoring/
compose.yaml
prometheus.yml
The real .env files are in .gitignore. Only .env.example is committed, with placeholder values. Persistent data lives in bind mounts under a single path on the VM, so one backup job covers it all.
A trimmed service entry looks like this:
services:
app:
image: ghcr.io/example/app:1.2.3
restart: unless-stopped
networks:
- proxy
env_file: .env
volumes:
- /srv/data/app:/data
networks:
proxy:
external: true
The proxy network is created once, and the reverse proxy is the only thing publishing ports to the host.
What I like
- Reproducibility - the VM is a cloud-init template plus Ansible, and the stacks come from Git. Rebuilding is a clone and a
docker compose up -dper stack. - Pinned tags - I avoid
latest, so an update is a deliberate diff instead of a surprise. - One data path - backups and restore drills only have to think about one place.
Where it hurts
- Secrets - the
.envfiles are the weak link. They aren't in Git, so they aren't reproducible. I've read about SOPS and similar tools but haven't tried them yet. - Updates - bumping tags is manual. I've seen Renovate mentioned for opening pull requests on image versions, but I haven't set it up, so I can't say how noisy it gets.
- Blast radius - everything shares one VM's kernel, CPU and RAM. One runaway container can slow down services my family relies on, and I haven't set resource limits consistently.
- The one container - there is always one that breaks on a major version bump. Pinning helps, but only if I read the release notes first.
What I'm unsure about
I'm not sure whether to split stacks across several smaller VMs (or LXC containers) by function, or keep one Docker host and add limits. Splitting reduces blast radius but multiplies the patching and backup surface. I'm also undecided on whether Kubernetes would solve any of this at my scale, or just trade these problems for new ones.
Questions for you
- How do you handle secrets for Compose stacks if you want the whole setup rebuildable from Git?
- One big Docker host or several split by function? What made you choose, and did you regret it?