# Docker Deployment Manager → Coolify Migration This project started as a custom Docker deployment tool (CLI + Electron desktop app) for deploying from Windows to Linux servers via SSH/tar transfers. After evaluating the approach, we're migrating to **Coolify** — an open-source self-hosted PaaS that handles the entire deploy pipeline automatically. ## Why Coolify Instead of a Custom Tool? | Custom Tool | Coolify | |---|---| | Manual button clicks to deploy | Auto-deploys on `git push` | | SSH password stored in plaintext `.env` | SSH keys / deploy keys | | Tar-based image transfer (no registry) | Builds on server, no transfer needed | | Desktop app required on Windows | Web dashboard from any browser | | Custom file-diff comparison | Git is the source of truth | | One developer maintaining it | Thousands of contributors | ## Architecture ``` Windows (dev machine) 192.168.69.4 (box-stableapps) ┌──────────────────────┐ ┌──────────────────────────────┐ │ C:\.bucket\repos.gitea│ │ Coolify (port 2010) │ │ │ │ ├── Dashboard + DB │ │ edit code │ │ ├── Traefik proxy │ │ → git commit │ │ └── Manages deployments │ │ → git push │ │ │ │ │ │ │ Gitea (ports 2003/2004) │ │ ▼ │ │ ├── HTTP: 2003 │ │ Gitea (webhook) ────────────► │ └── SSH: 2004 │ │ gitea.clintmasden │ │ │ │ .duckdns.org │ │ Data: ~/containers/*/data/ │ └──────────────────────┘ └──────────────────────────────┘ │ │ SSH (root) ▼ 192.168.69.5 (box-repoapps) ┌──────────────────────────────┐ │ Deployed apps: │ │ ├── learnxiny │ │ ├── actualbudget.report │ │ └── ... │ │ │ │ Data: ~/containers/*/data/ │ └──────────────────────────────┘ Flow: edit → git push → Gitea webhook → Coolify auto-builds & deploys ``` ## Server Setup - **192.168.69.4** (box-stableapps): Coolify host + Gitea host + stable apps - **192.168.69.5** (box-repoapps): Working/repo apps, managed as remote server by Coolify --- ## Coolify Installation (on 192.168.69.4) Coolify runs as Docker containers at `~/containers/coolify/`, consistent with the existing container layout. ### 1. Create directories ```bash mkdir -p ~/containers/coolify/data/{source,ssh,applications,databases,backups,services,proxy,sentinel} mkdir -p ~/containers/coolify/data/ssh/{keys,mux} mkdir -p ~/containers/coolify/data/proxy/dynamic ``` ### 2. Create the .env file ```bash tee ~/containers/coolify/data/source/.env << 'ENVEOF' APP_ID=coolify APP_NAME=Coolify APP_ENV=production APP_KEY= DB_USERNAME=coolify DB_PASSWORD= REDIS_PASSWORD= PUSHER_APP_ID= PUSHER_APP_KEY= PUSHER_APP_SECRET= APP_PORT=2010 SOKETI_PORT=6001 REGISTRY_URL=ghcr.io ENVEOF ``` Generate secrets: ```bash sed -i "s|^APP_KEY=.*|APP_KEY=base64:$(openssl rand -base64 32)|" ~/containers/coolify/data/source/.env sed -i "s|^DB_PASSWORD=.*|DB_PASSWORD=$(openssl rand -base64 32)|" ~/containers/coolify/data/source/.env sed -i "s|^REDIS_PASSWORD=.*|REDIS_PASSWORD=$(openssl rand -base64 32)|" ~/containers/coolify/data/source/.env sed -i "s|^PUSHER_APP_ID=.*|PUSHER_APP_ID=$(openssl rand -hex 32)|" ~/containers/coolify/data/source/.env sed -i "s|^PUSHER_APP_KEY=.*|PUSHER_APP_KEY=$(openssl rand -hex 32)|" ~/containers/coolify/data/source/.env sed -i "s|^PUSHER_APP_SECRET=.*|PUSHER_APP_SECRET=$(openssl rand -hex 32)|" ~/containers/coolify/data/source/.env ``` ### 3. Create docker-compose.yml Use a single merged compose file at `~/containers/coolify/data/source/docker-compose.yml`. Key customizations from default: - All paths use `/home/clint/containers/coolify/data/` instead of `/data/coolify/` - Named volumes replaced with bind mounts (postgres, redis) - `APP_PORT=2010` instead of default 8000 - `host.docker.internal` enabled for accessing other containers on the host - Gitea known_hosts mounted for SSH git access ```yaml services: coolify: image: "${REGISTRY_URL:-ghcr.io}/coollabsio/coolify:${LATEST_IMAGE:-latest}" container_name: coolify restart: always working_dir: /var/www/html extra_hosts: - host.docker.internal:host-gateway volumes: - type: bind source: /home/clint/containers/coolify/data/source/.env target: /var/www/html/.env read_only: true - /home/clint/containers/coolify/data/ssh:/var/www/html/storage/app/ssh - /home/clint/containers/coolify/data/applications:/var/www/html/storage/app/applications - /home/clint/containers/coolify/data/databases:/var/www/html/storage/app/databases - /home/clint/containers/coolify/data/services:/var/www/html/storage/app/services - /home/clint/containers/coolify/data/backups:/var/www/html/storage/app/backups - /home/clint/containers/coolify/data/known_hosts:/root/.ssh/known_hosts:ro environment: - APP_ENV=${APP_ENV:-production} - PHP_MEMORY_LIMIT=${PHP_MEMORY_LIMIT:-256M} env_file: - /home/clint/containers/coolify/data/source/.env ports: - "${APP_PORT:-8000}:8080" expose: - "${APP_PORT:-8000}" healthcheck: test: curl --fail http://127.0.0.1:8080/api/health || exit 1 interval: 5s retries: 10 timeout: 2s depends_on: postgres: condition: service_healthy redis: condition: service_healthy soketi: condition: service_healthy networks: - coolify postgres: image: postgres:15-alpine container_name: coolify-db restart: always volumes: - /home/clint/containers/coolify/data/postgres:/var/lib/postgresql/data environment: POSTGRES_USER: "${DB_USERNAME}" POSTGRES_PASSWORD: "${DB_PASSWORD}" POSTGRES_DB: "${DB_DATABASE:-coolify}" healthcheck: test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME}", "-d", "${DB_DATABASE:-coolify}"] interval: 5s retries: 10 timeout: 2s networks: - coolify redis: image: redis:7-alpine container_name: coolify-redis restart: always command: redis-server --save 20 1 --loglevel warning --requirepass ${REDIS_PASSWORD} environment: REDIS_PASSWORD: "${REDIS_PASSWORD}" volumes: - /home/clint/containers/coolify/data/redis:/data healthcheck: test: redis-cli ping interval: 5s retries: 10 timeout: 2s networks: - coolify soketi: image: "${REGISTRY_URL:-ghcr.io}/coollabsio/coolify-realtime:1.0.10" container_name: coolify-realtime restart: always extra_hosts: - host.docker.internal:host-gateway ports: - "${SOKETI_PORT:-6001}:6001" - "6002:6002" volumes: - /home/clint/containers/coolify/data/ssh:/var/www/html/storage/app/ssh environment: APP_NAME: "${APP_NAME:-Coolify}" SOKETI_DEBUG: "${SOKETI_DEBUG:-false}" SOKETI_DEFAULT_APP_ID: "${PUSHER_APP_ID}" SOKETI_DEFAULT_APP_KEY: "${PUSHER_APP_KEY}" SOKETI_DEFAULT_APP_SECRET: "${PUSHER_APP_SECRET}" healthcheck: test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:6001/ready && wget -qO- http://127.0.0.1:6002/ready || exit 1"] interval: 5s retries: 10 timeout: 2s networks: - coolify networks: coolify: name: coolify external: true ``` ### 4. Set up Traefik proxy Coolify uses Traefik as its reverse proxy. The proxy has its own docker-compose at `~/containers/coolify/data/proxy/docker-compose.yml`: ```yaml name: coolify-proxy networks: coolify: external: true services: traefik: container_name: coolify-proxy image: 'traefik:v3.6' restart: unless-stopped extra_hosts: - 'host.docker.internal:host-gateway' networks: - coolify ports: - '80:80' - '443:443' - '443:443/udp' - '8080:8080' healthcheck: test: 'wget -qO- http://localhost:80/ping || exit 1' interval: 4s timeout: 2s retries: 5 volumes: - '/var/run/docker.sock:/var/run/docker.sock:ro' - '/home/clint/containers/coolify/data/proxy/:/traefik' command: - '--ping=true' - '--ping.entrypoint=http' - '--api.dashboard=true' - '--entrypoints.http.address=:80' - '--entrypoints.https.address=:443' - '--entrypoints.http.http.encodequerysemicolons=true' - '--entryPoints.http.http2.maxConcurrentStreams=250' - '--entrypoints.https.http.encodequerysemicolons=true' - '--entryPoints.https.http2.maxConcurrentStreams=250' - '--entrypoints.https.http3' - '--providers.file.directory=/traefik/dynamic/' - '--providers.file.watch=true' - '--certificatesresolvers.letsencrypt.acme.httpchallenge=true' - '--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=http' - '--certificatesresolvers.letsencrypt.acme.storage=/traefik/acme.json' - '--api.insecure=false' - '--providers.docker=true' - '--providers.docker.exposedbydefault=false' labels: - traefik.enable=true - traefik.http.routers.traefik.entrypoints=http - traefik.http.routers.traefik.service=api@internal - traefik.http.services.traefik.loadbalancer.server.port=8080 - coolify.managed=true - coolify.proxy=true ``` Key: the volume mount points to `/home/clint/containers/coolify/data/proxy/` (not `/data/coolify/proxy/`). ### 5. Create Docker network and start **Run these as separate commands:** ```bash sudo docker network create --attachable coolify ``` ```bash # Start the proxy first cd ~/containers/coolify/data/proxy sudo docker compose up -d # Then start Coolify cd ~/containers/coolify/data/source sudo docker compose up -d --pull always --remove-orphans --force-recreate ``` ### 6. Fix data directory permissions Each service needs specific ownership. **Do NOT blanket chown the entire data directory** — Postgres and Redis run as different UIDs inside their containers. ```bash # Postgres must be UID 70 (postgres user in Alpine) sudo chown -R 70:70 ~/containers/coolify/data/postgres # Redis must be UID 999 (redis user in Alpine) sudo chown -R 999:999 ~/containers/coolify/data/redis # SSH keys — Coolify runs as UID 9999 sudo chown -R 9999:root ~/containers/coolify/data/ssh sudo chmod -R 700 ~/containers/coolify/data/ssh # Proxy — root-owned, acme.json must be 600 sudo chown -R root:root ~/containers/coolify/data/proxy sudo chmod 600 ~/containers/coolify/data/proxy/acme.json # Everything else can be clint sudo chown -R clint:clint ~/containers/coolify/data/source sudo chown -R clint:clint ~/containers/coolify/data/applications sudo chown -R clint:clint ~/containers/coolify/data/databases sudo chown -R clint:clint ~/containers/coolify/data/services sudo chown -R clint:clint ~/containers/coolify/data/backups ``` If you accidentally `chown -R` the whole directory, fix it with the commands above and restart: `docker restart coolify-db coolify-redis coolify`. ### 7. Open dashboard Go to `http://192.168.69.4:2010` and create admin account (first visitor gets admin). --- ## Gitea SSH Fix (Required for Coolify) By default, Gitea's Docker image does NOT start its built-in SSH server. This means SSH keys added through Gitea's web UI won't work for git clone over SSH. Coolify needs SSH to clone private repos. ### The problem Gitea's Docker container has OpenSSH running on port 22 internally, but it's not connected to Gitea's user/key database. Keys added in the Gitea web UI are only used by Gitea's **built-in** SSH server, which is disabled by default. ### The fix Add these environment variables to your Gitea docker-compose (`~/containers/gitea/docker-compose.yml`): ```yaml gitea: environment: # ... existing env vars ... GITEA__server__START_SSH_SERVER: "true" GITEA__server__SSH_LISTEN_PORT: "2222" GITEA__server__ROOT_URL: "https://gitea.clintmasden.duckdns.org/" GITEA__server__SSH_DOMAIN: "192.168.69.4" GITEA__webhook__ALLOWED_HOST_LIST: "192.168.69.4,host.docker.internal" ports: - "2003:3000" - "2004:2222" # was 2004:22, now points to Gitea's built-in SSH ``` - `START_SSH_SERVER=true` — enables Gitea's built-in SSH server that reads keys from its database - `SSH_LISTEN_PORT=2222` — avoids conflict with OpenSSH already on port 22 inside the container - Port mapping changed from `2004:22` to `2004:2222` to match - `ROOT_URL` — must match how you access Gitea in the browser. Mismatch causes webhooks not to fire on real pushes - `SSH_DOMAIN` — keeps the IP for SSH clone URLs (Coolify connects via `ssh://git@192.168.69.4:2004/...`) - `ALLOWED_HOST_LIST` — Gitea blocks webhooks to local/private IPs by default. This allows webhooks to reach Coolify on the same machine Restart Gitea: ```bash cd ~/containers/gitea && docker compose up -d ``` Verify SSH works: ```bash ssh -p 2004 git@192.168.69.4 # Should respond with: "Hi ! You've successfully authenticated..." ``` --- ## Connecting Coolify to Gitea ### 1. Generate SSH key in Coolify In Coolify dashboard → **Keys & Tokens** → create a new private key (e.g., "gitea-key"). ### 2. Add public key to Gitea (one-time, all repos) Copy the public key from Coolify's key detail page. In Gitea → **User Settings → SSH / GPG Keys → Add Key** → paste it. This gives Coolify access to ALL your repos under that user. No need to add deploy keys per-repo. ### 3. Set up Gitea's known_hosts for Coolify The Coolify container needs to trust Gitea's SSH host key. Create a persistent known_hosts file: ```bash ssh-keyscan -p 2004 192.168.69.4 2>/dev/null | sudo tee /home/clint/containers/coolify/data/known_hosts > /dev/null ``` This file is mounted into the Coolify container via the docker-compose volume: ```yaml - /home/clint/containers/coolify/data/known_hosts:/root/.ssh/known_hosts:ro ``` ### 4. Add remote server (192.168.69.5) Coolify needs **root SSH access** to remote servers: On 192.168.69.5: 1. Add Coolify's SSH public key to `/root/.ssh/authorized_keys` 2. Ensure `/etc/ssh/sshd_config` has `PermitRootLogin prohibit-password` 3. `sudo systemctl restart sshd` In Coolify: **Servers → Add Server** → SSH user `root`, IP `192.168.69.5`. Do the same for localhost (192.168.69.4) if you want Coolify to manage apps on the same box. ### 5. Add an application 1. **Projects → Add Project** → name it 2. Inside the project → **Add Resource → Application** 3. Choose **Private Repository (with deploy key)** 4. Select the **gitea-key** private key 5. Repository URL (must include port via ssh:// format): ``` ssh://git@192.168.69.4:2004/clint/your-repo.git ``` 6. Choose **Docker Compose** as build pack 7. Click **Load Compose File** 8. Set environment variables in Coolify's UI ### SSH URL format — important Because Gitea SSH is on port 2004 (not 22), you **must** use the `ssh://` URL format: ``` ssh://git@192.168.69.4:2004/clint/repo-name.git ``` The standard `git@host:user/repo.git` format does NOT support custom ports and will fail. --- ## Deployment Repo Structure Each service gets a Gitea repo with Docker config: ``` dotrepo.learn.x.in.y/ ├── react/ │ ├── Dockerfile │ ├── docker-compose.yml ← absolute volume paths │ ├── server.js │ ├── package.json │ └── src/ ``` ### Volume paths — absolute, not relative ```yaml # WRONG — relative path breaks with Coolify volumes: - ./data:/app/data # CORRECT — absolute path, data stays where it already is volumes: - /home/clint/containers/learnxiny/data:/data ``` ### Environment variables `.env` files stay in `.gitignore` — never committed. Production env vars are managed in Coolify's dashboard per-app. Coolify auto-detects `${VARIABLE}` syntax in compose files and creates UI fields for them. --- ## Auto-deploy Webhook The webhook URL and secret are **global** — same for every repo. Coolify matches incoming webhooks to the right app based on the repo URL in the payload. ### One-time: get the URL and secret from Coolify 1. Open any app in Coolify → **Webhooks** tab 2. Under **Manual Git Webhooks → Gitea**, copy the URL (e.g., `http://192.168.69.4:2010/webhooks/source/gitea/events/manual`) 3. Set a **Gitea Webhook Secret** and save ### Per-repo: add webhook in Gitea For each Gitea repo you want to auto-deploy: 1. Repo → **Settings → Webhooks → Add Webhook → Gitea** 2. **Target URL:** the Coolify Gitea webhook URL from above 3. **HTTP Method:** POST 4. **Content Type:** application/json 5. **Secret:** the same secret you set in Coolify 6. **Trigger On:** Push Events 7. Save Test by clicking **Test Push Event** in the webhook, then check **Recent Deliveries** for a green 200 response. ### Gitea webhook allow-list Gitea blocks webhooks to private/local IPs by default. The `GITEA__webhook__ALLOWED_HOST_LIST` environment variable in Gitea's docker-compose must include the Coolify host IP — see [Gitea SSH Fix](#gitea-ssh-fix-required-for-coolify) section. Now every `git push` triggers: clone → build → deploy → health check — automatically. --- ## Traefik — Replacing Caddy Coolify manages Traefik as its built-in reverse proxy. Once all services are migrated, Traefik replaces Caddy entirely. ### Coolify dashboard domain In Coolify → **Settings** → set **Instance's Domain (FQDN)** to: ``` https://coolify.clintmasden.duckdns.org ``` Coolify auto-configures Traefik to route this domain to its dashboard with SSL. ### Coolify-managed apps For apps deployed through Coolify, set the domain in the app's settings. Coolify configures Traefik automatically — no manual config needed. ### Non-Coolify services (Gitea, Syncthing, etc.) For services NOT managed by Coolify, add custom Traefik config at: `~/containers/coolify/data/proxy/dynamic/custom.yml` ```yaml http: routers: gitea-web: rule: "Host(`gitea.clintmasden.duckdns.org`)" entryPoints: - https service: gitea-web tls: certResolver: letsencrypt syncthing: rule: "Host(`syncthing.clintmasden.duckdns.org`)" entryPoints: - https service: syncthing tls: certResolver: letsencrypt immich: rule: "Host(`immich.clintmasden.duckdns.org`)" entryPoints: - https service: immich tls: certResolver: letsencrypt affine: rule: "Host(`affine.clintmasden.duckdns.org`)" entryPoints: - https service: affine tls: certResolver: letsencrypt mattermost: rule: "Host(`mattermost.clintmasden.duckdns.org`)" entryPoints: - https service: mattermost tls: certResolver: letsencrypt vikunja: rule: "Host(`vikunja.clintmasden.duckdns.org`)" entryPoints: - https service: vikunja tls: certResolver: letsencrypt services: gitea-web: loadBalancer: servers: - url: "http://host.docker.internal:2003" syncthing: loadBalancer: servers: - url: "http://host.docker.internal:2000" immich: loadBalancer: servers: - url: "http://host.docker.internal:2001" affine: loadBalancer: servers: - url: "http://host.docker.internal:2005" mattermost: loadBalancer: servers: - url: "http://host.docker.internal:2006" vikunja: loadBalancer: servers: - url: "http://host.docker.internal:2009" ``` Traefik picks up changes automatically — no restart needed. ### Switching from Caddy to Traefik 1. Stop Caddy: `cd ~/containers/caddy && docker compose down` 2. Verify Traefik grabs ports 80/443: `docker ps | grep traefik` 3. If Traefik isn't binding, restart it: `docker restart coolify-proxy` 4. Test: `curl -I https://gitea.clintmasden.duckdns.org` 5. If something breaks, bring Caddy back: `cd ~/containers/caddy && docker compose up -d` ### Services that still need migration These Caddy static sites need to be containerized (simple nginx containers) before Caddy can be fully retired: - `dotrepo.com` + subdomains: fitnotes, racker, timer, travel, pianoxml (React/Angular SPAs served from `/sites/`) - `audiosphere.dotrepo.com`, `audiocapsule.dotrepo.com`, `audioevents.dotrepo.com`, `everynoise.dotrepo.com` (proxied to 192.168.69.5) - `clintmasden.duckdns.org` (simple "Hi, I'm Clint" response) --- ## Troubleshooting ### "Could not resolve hostname ssh/http" Coolify's custom git repo field **only supports SSH**. It strips protocol prefixes. Use the `ssh://` format with port for non-standard SSH ports. ### "Permission denied (publickey)" on git clone 1. Verify the right private key is selected in the application settings 2. Verify the matching public key is in Gitea (User Settings → SSH Keys, not repo deploy keys) 3. Test from inside Coolify: `docker exec coolify sh -c "ssh -p 2004 -i /var/www/html/storage/app/ssh/keys/ git@192.168.69.4"` 4. Debug with verbose SSH: `docker exec coolify sh -c "ssh -vvv -p 2004 -i /var/www/html/storage/app/ssh/keys/ -o StrictHostKeyChecking=no git@192.168.69.4 2>&1 | tail -30"` ### "Host key verification failed" Regenerate known_hosts: `ssh-keyscan -p 2004 192.168.69.4 2>/dev/null | sudo tee /home/clint/containers/coolify/data/known_hosts > /dev/null` ### SSH key storage not writable ```bash sudo chown -R 9999:root ~/containers/coolify/data/ssh sudo chmod -R 700 ~/containers/coolify/data/ssh ``` ### 500 error / Redis MISCONF / Postgres permission denied Usually caused by blanket `chown` on the data directory. Fix ownership per-service and restart: ```bash sudo chown -R 70:70 ~/containers/coolify/data/postgres sudo chown -R 999:999 ~/containers/coolify/data/redis sudo chown -R root:root ~/containers/coolify/data/proxy sudo chmod 600 ~/containers/coolify/data/proxy/acme.json docker restart coolify-db coolify-redis coolify ``` ### Gitea SSH rejects all keys Enable Gitea's built-in SSH server — see [Gitea SSH Fix](#gitea-ssh-fix-required-for-coolify) section above. ### Webhook "ALLOWED_HOST_LIST" error Gitea blocks webhooks to private IPs by default. Add `GITEA__webhook__ALLOWED_HOST_LIST: "192.168.69.4,host.docker.internal"` to Gitea's docker-compose environment and restart. ### Webhooks fire on test but not on real push Check Gitea's `ROOT_URL` setting. If it doesn't match the URL you use to access Gitea (e.g., set to `http://192.168.69.4:2003` but you access via `gitea.clintmasden.duckdns.org`), webhooks won't fire on actual push events. Fix by setting `GITEA__server__ROOT_URL` in the docker-compose. ### Useful debug commands List Coolify's SSH keys: ```bash docker exec coolify sh -c "ls /var/www/html/storage/app/ssh/keys/" ``` Show public key for each stored key: ```bash docker exec coolify sh -c "for key in /var/www/html/storage/app/ssh/keys/*; do echo \"--- \$key ---\"; ssh-keygen -y -f \$key 2>&1; done" ``` Note: Coolify's container is Alpine-based — use `sh` not `bash` for `docker exec`. --- ## Workflow Summary ``` Before (manual): edit → build tar → SSH upload → docker load → docker compose up → hope it works After (Coolify): edit → git commit → git push → done (Coolify handles the rest) ``` --- Built for managing 35+ Docker deployments across two servers via Gitea + Coolify.