Troubleshooting

A reference for the most common issues encountered when using Nanokit. Jump to the section that matches your problem.

Docker & Local Environment

Docker daemon is not running

Symptom: nkapp up fails immediately with Cannot connect to Docker daemon.

Fix:


# macOS / Windows — start Docker Desktop from the Applications menu, then:
nkapp doctor   # confirm Docker is reachable
 
# Linux — start the service
sudo systemctl start docker
 
# Confirm your user is in the docker group (avoid sudo)
sudo usermod -aG docker $USER
newgrp docker

Container stuck in a restart loop

Symptom: A container shows Restarting in nkapp hub or docker ps.

Diagnosis:


nkapp logs <service-name>           # tail recent crash output
nkapp shell <service-name>          # exec into the container if it stays up long enough

Common causes:

Missing environment variable (process.env.X is undefined)
Wrong start command (binary not found, wrong path)
Port already in use on the host

Port conflict — address already in use

Symptom: bind: address already in use on port 80 or 443.

Fix:


# Find what is holding the port
sudo lsof -i :80
sudo lsof -i :443
 
# Stop the conflicting process, or change the gateway ports in nanokit.yml:
gatewayPort: 8080
gatewayHttpsPort: 8443

HTTPS & Certificates

Browser shows “Your connection is not private” on `.localhost` domains

Symptom: After nkapp up -e local, Chrome/Firefox shows a certificate error for *.localhost.

Cause: The Nanokit Root CA has not been trusted on this machine. Nanokit’s Caddy gateway generates a local CA stored in .nanokit/gateway/.

Fix — install the Root CA into your system trust store with one command:


nkapp host trust

This handles macOS (System Keychain), Linux (update-ca-certificates plus NSS databases used by Chrome/Firefox), and WSL2 (also propagates to the Windows certificate store). See nkapp host trust.

Manual fallback — if you prefer to do it by hand, the root certificate lives at .nanokit/gateway/data/caddy/pki/authorities/local/root.crt; import it into your OS or browser trust store (for Firefox: about:preferences#privacy → Certificates → View Certificates → Authorities → Import).

[!TIP] After trusting the CA, you may need to restart your browser completely for the change to take effect.

Let’s Encrypt certificate not provisioning on VPS

Symptom: Public domain shows a self-signed or expired certificate after deploy.

Checklist:

Port 80 must be open on the VPS firewall (Let’s Encrypt HTTP-01 challenge)
The DNS A record must point to the VPS IP before nkapp up runs
The gateway.adminEmail must be set to a valid address in nanokit.yml
Check Caddy logs:
```
nkapp logs gateway -e stage
```

Force certificate reset:


# On the VPS, clear stale Caddy state and redeploy
nkapp deploy -e stage  # triggers SSL factory reset automatically

Custom service certificate (e.g. Stalwart mail)

Symptom: A mail client reports “Cannot verify identity of server”.

Cause: Services with their own TLS layer (Stalwart, custom nginx) need the Caddy-issued certificate mounted explicitly.

Fix — mount the Caddy cert into the container via nanokit.yml:


services:
  mail:
    volumes:
      - /opt/nanokit/projects/<name>/.nanokit/gateway/data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/<domain>/<domain>.crt:/etc/caddy-certs/tls.crt:ro
      - /opt/nanokit/projects/<name>/.nanokit/gateway/data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/<domain>/<domain>.key:/etc/caddy-certs/tls.key:ro

Then restart the service and configure it to use /etc/caddy-certs/tls.crt.

Remote / VPS Operations

SSH connection refused

Symptom: nkapp up -e stage fails with Connection refused or Timeout.

Checklist:


# Test SSH manually first
ssh -i ~/.ssh/id_rsa root@<VPS-IP> "echo OK"
 
# Check firewall on the VPS allows port 22
ufw status
 
# If you use a non-standard port, set it in nanokit.yml:
deploy:
  sshPort: 2222

`(HTTP code 404) no such container` on restart/stop

Symptom: nkapp restart <service> or nkapp stop <service> returns 404 no such container.

Cause: The local state file (.nanokit/services-state.json) contains a stale container ID from a previous run that no longer exists on the remote host.

Fix:


# Option 1: Force a full reconciliation
nkapp up -e <env>
 
# Option 2: Clear local state and rediscover
rm .nanokit/services-state.json
nkapp up -e <env>
 
# Option 3: For CI/CD — use teardown instead of destroy/stop
nkapp teardown -p <project> -e <env> -y

Deployment lock — “Shared modules are being initialized”

Symptom: nkapp up or nkapp deploy hangs or reports a stale lock.

Cause: A previous deploy was interrupted and left a lock file in place.

Fix:


nkapp infra unlock --env <env>

VPS out of disk space

Symptom: Deploy fails with no space left on device.

Fix:


# SSH into the VPS and clean Docker
ssh root@<VPS-IP>
docker system prune -af --volumes
 
# Back on your machine, rebuild fresh
nkapp up -e stage

Configuration & Parsing

`Environment 'X' not found in configuration`

Symptom: nkapp up -e myenv fails with environment not found.

Cause: The environment name does not match a key in the environments: block of nanokit.yml.

Fix — check spelling and add the environment if missing:


environments:
  myenv:           # must match exactly (case-sensitive)
    infra:
      provider: docker

YAML parse error

Symptom: nkapp up fails immediately with a YAML syntax error.

Fix:


# Validate your config with the built-in doctor
nkapp doctor
 
# Or use a YAML linter
npx js-yaml nanokit.yml

Common causes: tabs instead of spaces, missing quotes around values with special characters (:, #, @).

Secret not resolved — `env://VAR` returns empty

Symptom: A service starts but an env:// secret resolves to an empty string.

Cause: The referenced variable is not set in the host shell or .env files.

Fix:


# Check if the variable is exported in the current shell
echo $MY_SECRET_VAR
 
# Or add it to your .env.local (never commit this file)
echo 'MY_SECRET_VAR=value' >> .env.local
 
# Verify resolution
nkapp env list -e local

Hub & Bridge

Hub shows no services / blank dashboard

Symptom: nkapp hub opens but the dashboard is empty.

Checklist:

Make sure nkapp up -e local has been run first
The bridge container nk-<project>-local-bridge must be running:
```
docker ps | grep bridge
```
If the bridge is missing, restart it:
```
nkapp hub --port 3006
```

Hub shows services from wrong project

Symptom: The Hub dashboard shows containers from a different project.

Cause: The bridge uses global discovery and may attach to the first running bridge it finds.

Fix: Navigate in the Hub sidebar to select the correct project, or open the Hub URL with the project-specific bridge port.

Database Operations

`db fork` fails with “volume not found”

Symptom: nkapp db fork <name> fails because the source volume does not exist.

Cause: The base database container has never been started, so no Docker volume exists yet.

Fix:


# Start the project first to create the base volume
nkapp up -e local
 
# Then fork
nkapp db fork <name>

`db pull` blocked — “remote database is empty”

Symptom: nkapp db pull <name> --from stage is blocked with a safety warning.

Cause: The remote database has no data. Nanokit blocks pulls from empty remote DBs to prevent accidental local data loss.

Fix — only if you are sure the remote DB is intentionally empty:


nkapp db pull <name> --from stage --force

CLI & General

`nkapp: command not found`

Fix:


# Reinstall globally
npm install -g @nkapp/cli
 
# Or check your PATH
which nkapp
echo $PATH

`nkapp` is slow to start (>2s)

Symptom: Every CLI command takes several seconds before doing anything.

Cause: Usually a slow DNS lookup or unreachable SSH host in the config.

Fix:


# Run doctor to identify the bottleneck
nkapp doctor
 
# If you have an SSH tunnel configured for a remote env,
# run local commands explicitly with -e local to skip remote SSH
nkapp up -e local

Getting more diagnostic output

Most commands support a verbose flag for deeper inspection:


nkapp up -e local --verbose
nkapp deploy -e stage --verbose

You can also check the Caddy gateway logs at any time:


nkapp logs gateway -e <env>

Troubleshooting

Docker & Local Environment

Docker daemon is not running

Container stuck in a restart loop

Port conflict — address already in use

HTTPS & Certificates

Browser shows “Your connection is not private” on .localhost domains

Let’s Encrypt certificate not provisioning on VPS

Custom service certificate (e.g. Stalwart mail)

Remote / VPS Operations

SSH connection refused

(HTTP code 404) no such container on restart/stop

Deployment lock — “Shared modules are being initialized”

VPS out of disk space

Configuration & Parsing

Environment 'X' not found in configuration

YAML parse error

Secret not resolved — env://VAR returns empty

Hub & Bridge

Hub shows no services / blank dashboard

Hub shows services from wrong project

Database Operations

db fork fails with “volume not found”

db pull blocked — “remote database is empty”

CLI & General

nkapp: command not found

nkapp is slow to start (>2s)

Getting more diagnostic output

Browser shows “Your connection is not private” on `.localhost` domains

`(HTTP code 404) no such container` on restart/stop

`Environment 'X' not found in configuration`

Secret not resolved — `env://VAR` returns empty

`db fork` fails with “volume not found”

`db pull` blocked — “remote database is empty”

`nkapp: command not found`

`nkapp` is slow to start (>2s)