Skip to main content

Operations Runbook

This page contains day-2 operations guidance for the ScannerSky Docker stack.

Monitoring Commands​

# Check status of all containers
docker compose ps

# View live container logs
docker compose logs -f django
docker compose logs -f nginx
docker compose logs -f minio
docker compose logs -f postgres
docker compose logs -f mcp

# Inspect container health states
docker compose ps --format "table {{.Name}}\t{{.Service}}\t{{.Status}}"

Backup and Restore Operations​

1. Automated Backup Workflow (deploy/scripts/backup-docker.sh)​

Performs both PostgreSQL database dumping and MinIO media bucket exporting in a single execution.

# Execute containerized PostgreSQL & MinIO media backup:
bash ./deploy/scripts/backup-docker.sh
  • Database Output: ~/db_backups/postgres_dumps/scannersky_YYYYMMDD_HHMMSS.sql
  • Media Output: ~/media_backups/media/ (Single-instance overwrite)
  • Retention Policy: Automatically purges old SQL dumps older than 7 days.

Automated Nightly Backup Cron​

Add to host crontab (crontab -e):

0 2 * * * ~/scannersky-django/deploy/scripts/backup-docker.sh >> ~/db_backups/backup.log 2>&1

2. Database & Media Restoration (deploy/scripts/restore-docker.sh)​

The restore suite provides a unified entrypoint (restore-docker.sh) with three operating modes:

# 1. Restore BOTH database and media in one step (Recommended)
bash ./deploy/scripts/restore-docker.sh --all

# 2. Restore PostgreSQL database dump only
bash ./deploy/scripts/restore-docker.sh --db-only

# 3. Restore Media backup directory into MinIO bucket only
bash ./deploy/scripts/restore-docker.sh --media-only

3. Default Path Resolutions & Interactive Selection​

When no explicit paths are specified, the restore scripts resolve default locations automatically:

A. Database Dump Defaults (DUMP_FILE)​

  • Default Search Path: ~/db_backups/postgres_dumps/
  • Interactive Selection: Scans ~/db_backups/postgres_dumps/, lists available SQL dump files with file sizes and modification timestamps, and prompts the operator to choose a dump file number.
  • Non-Interactive Mode: Automatically selects the latest valid SQL dump (> 10 KB).

B. Media Backup Defaults (MEDIA_DIR)​

  • Default Search Path: ~/media_backups/media/
  • Confirmation Prompt: Displays the modification date and time of the media backup (~/media_backups/media/.backup_info) and prompts for operator confirmation (Continue restoring media backup from [TIMESTAMP]? [y/N]) before importing into MinIO.

4. Custom Path Overrides​

You can pass any custom dump file or media directory path on disk using environment variables or command arguments:

# Custom SQL Dump File Location
DUMP_FILE="/mnt/external_drive/backups/db_july.sql" bash ./deploy/scripts/restore-docker.sh --db-only

# Custom Media Backup Directory Location
MEDIA_DIR="/mnt/usb/media_backup_2026" bash ./deploy/scripts/restore-docker.sh --media-only

# Combined Custom Restore
DUMP_FILE="/tmp/custom_db.sql" MEDIA_DIR="/tmp/custom_media" bash ./deploy/scripts/restore-docker.sh --all

5. Internal Container Execution Details​

  • Database Restore:
    1. Executes psql -d postgres inside the postgres container.
    2. Runs DROP DATABASE IF EXISTS "scannersky" WITH (FORCE); to safely drop all active Django client connections.
    3. Executes CREATE DATABASE "scannersky";.
    4. Pipes the SQL dump directly into psql inside the container.
  • Media Restore (deploy/scripts/restore-media.sh):
    1. Copies backup files into the minio container using docker cp.
    2. Runs mc mirror --overwrite to push all media subdirectories (persons/, signatures/, profiles/, id_cards/, syllabi/) into MinIO bucket scannersky-media.
    3. Sets bucket policy mc anonymous set download local/scannersky-media so Nginx can proxy uploaded images directly.

Common Issues & Fixes​

1. Nginx 502 Bad Gateway​

  • Cause: django container is restarting or failing healthcheck.
  • Fix: Check docker compose logs django. Ensure DB_HOST=postgres and database migrations have run (docker compose exec django python manage.py migrate).

2. Unstyled Admin UI (Missing CSS/JS)​

  • Cause: MinIO static bucket empty or Nginx proxy rewrite issue.
  • Fix: Run docker compose exec django python manage.py collectstatic --noinput, verify deploy/scripts/setup-minio.sh, and reload Nginx (docker compose restart nginx).

3. CSRF Verification Failed (403 Forbidden)​

  • Cause: Missing Host header proxying or domain not in CSRF_TRUSTED_ORIGINS.
  • Fix: Ensure proxy_set_header Host $host; is set in Nginx location / block. scannersky/settings.py automatically populates CSRF_TRUSTED_ORIGINS for all ALLOWED_HOSTS.

4. MCP Server Import Error​

  • Cause: PyPI mcp 2.0.0 breaking change.
  • Fix: Pin mcp>=1.0.0,<2.0.0 in requirements.txt and rebuild images (docker compose build mcp).

Recovery Checklist​

  1. Verify Docker daemon is running (docker compose ps).
  2. Confirm postgres, minio, django, nginx, and mcp services report healthy.
  3. Execute bash ./deploy/scripts/validate-production.sh.
  4. Verify HTTP 200 response on /health/, /admin/, /api/, and /docs/.