Seven projects on one box: what is isolated and what is not

Start with why it happens. Seven projects on seven machines means seven sets of OS updates, seven monitoring setups, seven certificate rotations, seven disk alarms. At one-person scale that operational load costs far more than the server bill. So one box, nginx splitting by domain, one container per service is a sound decision.

The decision is not the problem. What follows it is: everyone starts assuming the projects cannot affect each other. For some things that holds. For others it does not, and those others are where the incidents come from.

Genuinely isolated

• nginx vhosts — one site config per domain; editing A's does not touch B's
• containers — their own processes, filesystems and ports
• application credentials and environment — one container's env is invisible to the others
• log groups — each project ships to its own, so queries never mix

That list is what creates the impression of independence. Here is the part that does not hold.

Not isolated

This is the list that matters.

1. Disk. One shared pool. Any single service's runaway logs, or accumulated Docker images nobody pruned, fills / — and then every service starts failing at once: cannot write logs, cannot write temp files, connection pools erroring.
2. The Docker daemon. If it misbehaves, or you restart it, every container is affected. The cleanup commands are global too, so one wrong flag crosses projects.
3. nginx reload. The vhost files are separate but reload is all-or-nothing. A syntax error in any one config fails the whole reload — your change does not take effect and the other six keep serving the old config (and if nginx cannot start at all, all seven go down together).
4. OS and kernel. Security updates, reboots, timezone settings — all at once. Which makes "find a maintenance window every client accepts" a real scheduling problem.
5. The public IP and its reputation. A shared egress address. One service getting blocklisted or rate-limited by an API provider is something the others inherit.
6. CPU and memory. Without limits, one runaway loop drags the whole box down. This is the easiest to overlook, because day-to-day load is low.

Sharing is not the problem. Treating the shared parts as isolated is.

The most common incident: disk

Of the cross-project incidents I have seen, disk accounts for the large majority, and the cause is always one of two things: logs with no rotation, and old images nobody cleared.

df -h /                      # overall usage
docker system df             # split across images / containers / volumes / build cache
du -sh /var/lib/docker/containers/* | sort -h | tail   # which container's log is fattest

Docker's default json-file log driver has no size limit. A chatty service can produce several GB in a few weeks. Cap it globally:

// /etc/docker/daemon.json
{
  "log-driver": "json-file",
  "log-opts": { "max-size": "50m", "max-file": "3" }
}

Note this only applies to containers created afterwards; existing ones need to be recreated to pick it up. So schedule a pass to replace them, or you will believe the problem is capped while the old containers keep growing.

Prune old images with a filter, never blindly:

docker image prune -a --filter "until=720h"   # only older than 30 days
# never run an unfiltered docker system prune -a on a shared box

Then alert on a df -h threshold. It earns its own dedicated alarm because it is the one cause that takes down every project simultaneously.

The order for editing nginx config

Because reload is global, changing any project's vhost follows the same sequence:

1. sudo cp site.conf /root/nginx-backup/site.conf.$(date +%s) — timestamped, and stored outside sites-enabled/
2. sudo nginx -t — the syntax check is global, so it also surfaces landmines someone else left
3. sudo systemctl reload nginx — reload, not restart

"Outside the directory" in step 1 is not fastidiousness. If that directory's include pattern is * rather than *.conf, the backup gets loaded as well and one server_name now has two configs — a failure that is hard to trace, because both files are individually correct.

When to split a project out

Sharing is my default, but a few situations get their own instance immediately:

• Compliance — the contract specifies data isolation or a region. Not a technical judgement
• A different SLA tier — if A can only be maintained overnight and B can stop any time, coupling them gives both the worse of the two terms
• A different traffic class — when one service's peak is ten times the other six combined, resource contention has stopped being theoretical
• An imminent handover — a project being handed to the client to run must be able to exist on its own first

Cost arguments usually push back on all four, so name the trade explicitly: another instance is a predictable fixed expense, and a whole-box incident is not.

One more thing: not everything is in Docker

On a long-lived shared box there is always a service or two running straight from systemd — it was urgent at the time, or that stack was awkward to containerise. The practical cost is an inconsistent debugging path: everything else answers to docker logs or CloudWatch, and this one needs journalctl -u.

The issue is not which is better. It is that during an incident you will reach for the wrong command first, see empty output, and start suspecting the service is dead. Writing down which service is inspected which way, somewhere everyone can find it, costs ten minutes and returns those five minutes of confusion on every incident.

If you remember one thing

Write the shared resources down as an explicit list, and assume each one will eventually hit every project at once. Disk, daemon, reload, kernel, egress IP — that is your blast radius. No amount of isolation elsewhere protects you from any item on it.