Phase 9: Secrets Hardening, Service-User Migration & Production Hardening

Version: 1.0.0
Date: 2026-04-29
Classification: Internal — CEO/CIO Eyes Only
Author: Hermes Agent
Scope: Migrate all plaintext secrets from .env into Infisical; migrate service containers to run as prodg user (UID 1000) instead of root; harden Tailscale routing; consolidate all external API keys (Tavily, Backblaze B2, Anthropic, Telegram) into Vaultwarden + Infisical; establish production-ready agent workload policy.

1. Current State Snapshot

1.1 Infrastructure Already Deployed (Phases 0–8)

Phase	Component	Status	Notes
0	Ubuntu 24.04, Docker, `prodg` user	✅	`prodg` UID 1000 exists
1	Headscale (Tailnet control plane)	✅	`headscale.prodg.studio` public
2	Postgres 16, Redis 7, MinIO	✅	With healthchecks
3	Infisical + Vaultwarden	✅	Infisical WebUI requires re-auth
4	Caddy + TLS + DNS	✅	5 domains live with auto-HTTPS
5	Prometheus + Grafana + Loki	✅	Observability stack operational
6	Multi-Agent Orchestrator (FastAPI)	✅	`/v1/agents`, `/v1/tasks`, `/v1/dispatch`
7	Backblaze B2 backups	✅	Nightly cron via `rclone`
8	Quartz 4 docs vault	✅	`docs.mainframe.prodg.studio` live

1.2 All Secrets Currently in Plaintext `.env`

File: /opt/prodg/compose/.env (mode 600, root:root)

INFISICAL_AUTH_SECRET=eAzZmubyQymjTJKicQfoP
MINIO_ROOT_PASSWORD=IK2hCZ...ggFm
POSTGRES_DB=hermes
VAULTWARDEN_ADMIN_TOKEN=e0de65...d1b2
POSTGRES_PASSWORD=t8ov3l...yguC
INFISICAL_ENCRYPTION_KEY=T4K8jbepMrZcBZRqwLchZib8XkW3e2Ej
REDIS_PASSWORD=G6iKz9...G2yI
INFISICAL_JWT_SECRET=3YJZa5...za0l
MINIO_ROOT_USER=prodg-minio
POSTGRES_USER=prodg
GRAFANA_ADMIN_PASSWORD=grJCYt...hVhz
TELEGRAM_BOT_TOKEN=804550...-u2U
TELEGRAM_CHAT_ID=-5267054745
HERMES_API_TOKEN=ff540f...4761

# Backblaze B2
B2_KEY_ID=0054df3c5c51f170000000001
B2_KEY_SECRET=K005+i...n8Hg
B2_BUCKET=MainframeBackup

1.3 User-Provided Context (NEW — April 29)

Secret	Value	Owner	Service
Tavily API Key	`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`	Mitch	Web search for agents
Backblaze B2 App Key	`K005+ixrAG8niwbXZmjfHUMVfiwn8Hg` + key ID	Mitch	Offsite backups already in `.env`
Domain	`prodg.studio`	Mitch	DNS root
SSH Key	`ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKv5Y45VNS1nb2fwrmVsIe5gcPurhPoIHtXJh4gHgVSB` — “Mainframe” key	Mitch (CEO)	Dev laptop → mainframe access

1.4 Services That Still Run as Root Inside Containers

Container	Current User	Risk
`vaultwarden`	`root` (default)	High — password vault running as root
`infisical`	`root` (default)	High — secrets manager running as root
`minio`	`root` (default)	Medium — object store
`hermes-api`	`root` (default)	Medium — orchestrator with Docker socket
`prometheus`	`root` (default)	Low — read-only metrics
`grafana`	`root` (default)	Low — dashboards
`loki`	`root` (default)	Low — logging

2. Phase 9 Objectives

Primary

Eliminate plaintext .env — all secrets live in Infisical; .env becomes a redirect or is deleted
Service-user migration — all containers run as prodg:prodg (UID 1000:GID 1000)
Vaultwarden hardening — sign-ups disabled, admin token rotated, runs as non-root
Infisical stabilization — persistent auth, proper project/workspace structure for ProDG
External API key consolidation — Tavily, B2, Anthropic, Telegram keys stored in Infisical + Vaultwarden

Secondary

Hermes API routed through Tailscale — internal API accessible only via tailnet
Agent workload policy formalized — trusted on-box containers, untrusted on Modal/remote nodes
rclone B2 config migrated — inject credentials from Infisical, not host file
Complete documentation — vault docs updated, runbook revised

3. Implementation Plan

3.1 Infisical Project Structure (NEW)

Before migrating secrets, set up Infisical properly:

Organization: ProDG Studio
  ├── Project: mainframe-core
  │     ├── Environment: production
  │     │       ├── Secret: POSTGRES_PASSWORD
  │     │       ├── Secret: REDIS_PASSWORD
  │     │       ├── Secret: INFISICAL_ENCRYPTION_KEY
  │     │       ├── Secret: INFISICAL_AUTH_SECRET
  │     │       ├── Secret: INFISICAL_JWT_SECRET
  │     │       ├── Secret: MINIO_ROOT_USER
  │     │       ├── Secret: MINIO_ROOT_PASSWORD
  │     │       ├── Secret: GRAFANA_ADMIN_PASSWORD
  │     │       ├── Secret: HERMES_API_TOKEN
  │     │       ├── Secret: VAULTWARDEN_ADMIN_TOKEN
  │     │       ├── Secret: TELEGRAM_BOT_TOKEN
  │     │       ├── Secret: TELEGRAM_CHAT_ID
  │     │       └── Secret: B2_KEY_SECRET
  │     ├── Environment: staging
  │     └── Environment: development
  ├── Project: agent-apis
  │     ├── Environment: production
  │     │       ├── Secret: TAVILY_API_KEY
  │     │       ├── Secret: ANTHROPIC_API_KEY        # (when acquired)
  │     │       ├── Secret: MODAL_TOKEN              # (when acquired)
  │     │       └── Secret: OPENROUTER_KEY           # (when acquired)
  └── Identity: prodg-mainframe-service
        ├── Service Token: Used by Docker containers for secret injection

3.2 Step-by-Step Migration

Step 1: Bootstrap Infisical CLI on Host

# Install Infisical CLI
npm install -g infisical
# OR download binary
curl -o infisical https://infisical.com/get-cli/linux && chmod +x infisical && mv infisical /usr/local/bin/
 
# Login (interactive — requires browser auth)
infisical login # SSO or email/password
 
# Link to project
infisical init --projectId=<project-id>

Step 2: Create Infisical Service Token

# Generate a service token for Docker secret injection
# In Infisical WebUI: Project Settings → Service Tokens → Create Token
# Token needs read access to production environment

Value (generated — store in Vaultwarden, NOT .env):

infisical-service-token-prodg: st.********.************************

Step 3: Migrate Secrets Programmatically

#!/bin/bash
# /opt/prodg/scripts/migrate-secrets-to-infisical.sh
# Reads current .env, injects into Infisical production environment
 
set -euo pipefail
 
ENV_FILE="/opt/prodg/compose/.env"
PROJECT_ID="<prodg-mainframe-core-id>"
ENV_SLUG="production"
 
# Required: INFISICAL_SERVICE_TOKEN set in current shell
if [ -z "${INFISICAL_SERVICE_TOKEN:-}" ]; then
    echo "ERROR: Set INFISICAL_SERVICE_TOKEN environment variable"
    exit 1
fi
 
# Parse .env and push to Infisical
while IFS='=' read -r key value; do
    # Skip comments and empty lines
    [[ "$key" =~ ^#-|.*$ ]] && continue
    [[ -z "$key" ]] && continue
    # Skip the B2 section header
    [[ "$key" == "B2_KEY_ID" ]] && continue
    [[ "$key" == "B2_KEY_SECRET" ]] && continue
    [[ "$key" == "B2_BUCKET" ]] && continue
    # Push secret
    infisical secrets set "$key=$value" --env="$ENV_SLUG" --projectId="$PROJECT_ID"
    echo "✓ Migrated: $key"
done < "$ENV_FILE"
 
echo "All secrets migrated to Infisical production environment."
echo "NEXT: Delete or archive .env file"

Step 4: Docker Compose `.env` Replacement

Replace the .env file with an Infisical- sourced approach. Two options:

Option A: Infisical Docker Secret Injection (Recommended)

Infisical provides a Docker integration where containers fetch secrets at startup. However, this requires the Infisical Docker agent — adds complexity.

Option B: env-template file (Simpler, Near-Term)

Create /opt/prodg/compose/.env.infisical with only the Infisical service token:

# /opt/prodg/compose/.env.infisical
# ONLY this token lives on disk. All other secrets come from Infisical.
INFISICAL_SERVICE_TOKEN=st.********.************************
INFISICAL_PROJECT_ID=<project-id>
INFISICAL_ENV_SLUG=production

Modify docker-compose.yml to use env_file with Infisical- sourced values via a startup script:

# docker-compose.yml — NEW: infisical-env-loader service
services:
  infisical-loader:
    image: infisical/infisical-cli:latest
    container_name: infisical-loader
    restart: "no"
    environment:
      INFISICAL_SERVICE_TOKEN: ${INFISICAL_SERVICE_TOKEN}
      INFISICAL_PROJECT_ID: ${INFISICAL_PROJECT_ID}
      INFISICAL_ENV_SLUG: ${INFISICAL_ENV_SLUG}
    volumes:
      - /opt/prodg/data/secrets:/secrets
    command: >
      sh -c "infisical secrets --env=$INFISICAL_ENV_SLUG --format=json > /secrets/env.json"

Then each service loads its specific secrets via a startup wrapper.

Option C: env_file with default fallback (Immediate — Already Partially Working)

Keep .env but populate via Infisical CLI before docker compose up:

# /opt/prodg/scripts/load-secrets.sh
infisical export --format=dotenv > /opt/prodg/compose/.env
chmod 600 /opt/prodg/compose/.env
chown root:root /opt/prodg/compose/.env

Recommendation: Use Option C for Phase 9. It’s the simplest migration path:

Run load-secrets.sh before any docker compose command
.env is regenerated from Infisical each time
The file still exists but is transient and never edited manually
Original .env is backed up to Vaultwarden, then deleted from host

Step 5: Service User Migration (Docker Containers)

Goal: Every container runs as UID 1000 (prodg).

Per-Service Changes:

# vaultwarden — add user directive
services:
  vaultwarden:
    image: vaultwarden/server:latest
    user: "1000:1000"          # <-- ADD THIS
    volumes:
      - /opt/prodg/data/vaultwarden:/data
    # ... rest unchanged

Permission Fixes Required (Pre-Migration):

#!/bin/bash
# /opt/prodg/scripts/fix-permissions-for-user-migration.sh
# Run BEFORE adding user: directives
 
set -e
 
echo "Fixing data directories for prodg:prodg ownership..."
 
# Core data directories
chown -R 1000:1000 /opt/prodg/data/postgres
chown -R 1000:1000 /opt/prodg/data/redis
chown -R 1000:1000 /opt/prodg/data/minio
chown -R 1000:1000 /opt/prodg/data/grafana
chown -R 1000:1000 /opt/prodg/data/prometheus
chown -R 1000:1000 /opt/prodg/data/loki
chown -R 1000:1000 /opt/prodg/data/vaultwarden
chown -R 1000:1000 /opt/prodg/data/headscale
chown -R 1000:1000 /opt/prodg/data/caddy
chown -R 1000:1000 /opt/prodg/data/caddy-config
chown -R 1000:1000 /opt/prodg/data/docs
chown -R 1000:1000 /opt/prodg/data/quartz
chown -R 1000:1000 /opt/prodg/backups
 
echo "Permissions fixed. Containers can now run as prodg (UID 1000)."

Service-by-service user directive:

Service	`user:` directive	Notes
`vaultwarden`	`1000:1000`	Already stores data in `/data` — works as non-root
`infisical`	`1000:1000`	Requires checking — may need DB migration
`postgres`	`1000:1000`	Postgres default user is `postgres` (UID 999). Either: (a) keep as-is, or (b) use `POSTGRES_USER: prodg` with initdb. Keep existing — Postgres has strong internal user isolation.
`redis`	`1000:1000`	Redis drops privileges internally; can run as any UID
`minio`	`1000:1000`	MinIO supports arbitrary UID/GID
`grafana`	`1000:1000`	Grafana defaults to UID 472. Add `GF_PATHS_DATA=/data` env. Keep existing — Grafana’s internal user is safe.
`prometheus`	`1000:1000`	Prometheus defaults to nobody. Add `--storage.tsuid` and `--storage.gid`. Or keep existing. OPTIONAL — not critical.
`hermes-api`	`1000:1000`	Custom image — rebuild with `USER prodg` in Dockerfile
`caddy`	`1000:1000`	Caddy drops privileges after binding ports. Caddy can run as non-root if ports >1024. Public 80/443 requires root or CAP_NET_BIND. Keep as root or use authbind.
`cadvisor`	`root` (required)	Privileged container — must stay root
`promtail`	`root` (recommended)	Reads journald, docker logs — needs root for some sources

Decision: Phase 9 migrates these to prodg user: vaultwarden, infisical, redis, minio, hermes-api.
Deferred: postgres, grafana, prometheus, caddy, cadvisor, promtail — they have well-tested internal privilege dropping or require elevated access.

Step 6: rclone B2 Config Migration

Current: /opt/prodg/backups/rclone/rclone.conf (generated from .env)

Migrate to Infisical-injected config:

# docker-compose.yml — rclone config (if running rclone in container)
# Or, in the backup script:

New backup script flow:

#!/bin/bash
# /opt/prodg/backups/scripts/backup-all.sh (REVISED)
 
set -euo pipefail
 
# Load secrets from Infisical
export $(infisical export --format=dotenv --plain | xargs)
 
# Generate rclone.conf on-the-fly (temp file, deleted after)
RCLONE_CONF=$(mktemp)
cat > "$RCLONE_CONF" <<EOF
[b2]
type = b2
account = $B2_KEY_ID
key = $B2_KEY_SECRET
EOF
 
# Run backup
rclone --config "$RCLONE_CONF" sync ...
 
# Destroy temp config
rm -f "$RCLONE_CONF"

Step 7: Vaultwarden Hardening

Already partially done (sign-ups disabled), but need to:

Rotate admin token (currently in .env)
Enable 2FA enforcement for all users
Move Vaultwarden data dir to prodg:prodg
Enable WebSocket (already set in docker-compose)
Disable admin panel from public — restrict to tailnet:

# Caddyfile — Vaultwarden admin panel restriction
vault.mainframe.prodg.studio {
	reverse_proxy vaultwarden:80
 
	# Restrict /admin to tailnet IPs only
	@admin path /admin*
		handle @admin {
			# Only allow from 100.64.0.0/10 (Tailscale range)
			@not_tailnet {
				not remote_ip 100.64.0.0/10
			}
			respond @not_tailnet "Forbidden" 403
			reverse_proxy vaultwarden:80
		}
}

Step 8: Hermes API Tailscale-Only Binding

Current: api.mainframe.prodg.studio is public via Caddy.
Goal: Internal API accessible only via Tailnet (or with token via public).

Option A: Add Tailnet IP binding for internal routes

The Hermes API already uses token auth (X-API-Token), which is sufficient for public API access. For stricter control:

# Caddyfile — add Tailnet-only internal route
internal-api.mainframe.prodg.studio {
	# Only respond to Tailnet IPs
	@not_tailnet {
		not remote_ip 100.64.0.0/10
	}
	respond @not_tailnet "Forbidden — Tailnet access only" 403
 
	reverse_proxy hermes-api:8000
}

Agent dispatch already supports Tailnet routing via tailscale_node parameter in /v1/dispatch/docker.

Step 9: Tavily API Key Injection

User provided Tavily key for web search. This is an agent API key, not an infrastructure secret.

Store in:

Infisical → Project: agent-apis, Environment: production, Key: TAVILY_API_KEY
Vaultwarden → Item: “Tavily Web Search API”, Folder: “Agent APIs”

Use in: Hermes API for agent dispatch, research agents, or any service that calls https://api.tavily.com.

No direct infrastructure impact — this is for agent workloads.

4. Agent Workload Policy (Formalized)

Per user requirements:

Agent Trust Tiers
═══════════════════════════════════════════════════════════════

┌──────────────────────────────────────────────────────────────┐
│  TIER 1 — INTERNAL (mainframe-001)                           │
│  • Runs on Mainframe VPS, always-on                          │
│  • Full Docker socket access                                 │
│  • Access to all internal networks                           │
│  • Use cases: Infra management, orchestration, Caddy reload  │
│  • Container: hermes-api (already deployed)                  │
└──────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌──────────────────────────────────────────────────────────────┐
│  TIER 2 — TRUSTED AGENTS                                     │
│  • Run in containers ON mainframe                            │
│  • No Docker socket access (read-only via proxy if needed)   │
│  • Sandboxed: network isolation, resource limits             │
│  • Use cases: Research, data processing, content generation  │
│  • Deployed via: POST /v1/dispatch/docker (local mode)       │
│  • Examples: tavily-research-agent, document-processor       │
└──────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌──────────────────────────────────────────────────────────────┐
│  TIER 3 — UNTRUSTED / BURST                                  │
│  • Run on REMOTE Tailscale nodes or Modal.com                │
│  • No access to mainframe internal network                   │
│  • Ephemeral: created for task, destroyed after              │
│  • Use cases: Untrusted code execution, burst inference      │
│  • Deployed via: POST /v1/dispatch/docker (tailscale_node)  │
│                 POST /v1/dispatch/modal                      │
│  • Examples: sandboxed-code-runner, LLM-inference-burst        │
└──────────────────────────────────────────────────────────────┘

Policy Rules:

Tier 1 is the Hermes API itself. Only one instance.
Tier 2 agents are registered via /v1/agents/register with tier: tier-2-trusted. They poll /v1/tasks/next for work. They run in containers on the mainframe but with limited permissions.
Tier 3 agents never touch the mainframe’s Docker socket. They run on Modal or on remote tailnet nodes that Mitch provisions.
Task dispatch enforces tier compatibility:
- tier_required: tier-1-internal → only mainframe-001 can execute
- tier_required: tier-2-trusted → tier-1 or tier-2 agents
- tier_required: tier-3-untrusted → any agent (including Modal)

5. Order of Operations (Deployment Day)

Execute in this exact order to avoid breaking the stack:

Pre-Migration

Step	Action	Rollback Strategy
5.1	`cp /opt/prodg/compose/.env /opt/prodg/backups/dotenv-YYYY-MM-DD.bak`	Restore from backup
5.2	Stop non-critical stacks: `docker compose stop loki promtail`	`docker compose start loki promtail`
5.3	Run `fix-permissions-for-user-migration.sh`	Re-chown back to root if issues

Infisical Migration

Step	Action	Verification
5.4	Re-auth Infisical WebUI, create project structure	Browse `secrets.mainframe.prodg.studio`
5.5	Install Infisical CLI on host (`npm install -g infisical`)	`infisical --version`
5.6	Login interactively: `infisical login`	Token generated
5.7	Run `migrate-secrets-to-infisical.sh`	Verify each secret in WebUI
5.8	Create service token for Docker, save to Vaultwarden	Token visible in Infisical settings
5.9	Create `/opt/prodg/compose/.env.infisical` (token-only)	File contains only `INFISICAL_*`

Service User Migration

Step	Action	Verification
5.10	`docker compose stop vaultwarden infisical redis minio`	`docker compose ps` shows stopped
5.11	Add `user: "1000:1000"` to docker-compose.yml for affected services	`git diff docker-compose.yml`
5.12	Run `fix-permissions-for-user-migration.sh`	`ls -la /opt/prodg/data/` shows `prodg:prodg`
5.13	`docker compose up -d vaultwarden infisical redis minio`	Containers start, healthchecks pass
5.14	Verify each migrated service:
	• `docker exec vaultwarden id` → `uid=1000`
	• `docker exec infisical id` → `uid=1000`
	• `docker exec redis id` → `uid=1000`
	• `docker exec minio id` → `uid=1000`

Hermes API (Custom Image Rebuild)

Step	Action	Verification
5.15	Edit hermes-api Dockerfile: add `USER prodg` before `CMD`	Dockerfile diff
5.16	Rebuild image: `docker build -t prodg/hermes-api:v0.7.1 /opt/prodg/hermes-api`	Build completes
5.17	Update compose: `image: prodg/hermes-api:v0.7.1`, add `user: "1000:1000"`	Compose validated
5.18	`docker compose up -d hermes-api`	Health endpoint returns 200
5.19	Test dispatch: `curl -X POST /v1/dispatch/docker -d '{"image":"hello-world"}'`	Container runs

rclone Migration

Step	Action	Verification
5.20	Edit `backup-all.sh` to load Infisical secrets before rclone	Script diff
5.21	Delete `/opt/prodg/backups/rclone/rclone.conf` (temp file will be generated)	File removed
5.22	Run `backup-all.sh` manually	Backup completes to B2
5.23	Verify via `rclone lsd b2:MainframeBackup`	Buckets listed

Post-Migration Cleanup

Step	Action	Rollback Strategy
5.24	`mv /opt/prodg/compose/.env /opt/prodg/backups/dotenv-deprecated-YYYY-MM-DD`	Move back and rename
5.25	Update backup script to include `.env.infisical` (token-only file is safe to back up)	Token can be revoked and re-issued
5.26	Delete plaintext `.env` from host	—
5.27	Restart full stack: `docker compose up -d`	All services healthy
5.28	Update 02-runbook with new Infisical-based procedures	—
5.29	Update 01-architecture secrets section	—
5.30	SAVE DEPRECATION BACKUP TO VAULTWARDEN	Item: “mainframe-legacy-env”, attach file

6. Risk Assessment

Risk	Likelihood	Impact	Severity	Mitigation
Secret migration loses a secret	Medium	Critical	CRITICAL	Backup `.env` before migration; migrate one secret at a time; verify in Infisical WebUI
Service user migration breaks DB data	Medium	High	HIGH	Postgres data dir is NOT migrated (stays as postgres user); only vaultwarden/infisical/minio data changes ownership
Infisical service token leaks	Low	Critical	HIGH	Token is short-lived (30-day rotation); stored in Vaultwarden, not in git
Container fails to start as UID 1000	Medium	Medium	MEDIUM	Test each container individually; fix permissions and retry
rclone backup fails with new config	Low	High	MEDIUM	Manual test before decommissioning old config; keep old config until verified
Infisical becomes unavailable	Low	High	MEDIUM	Keep local `.env.infisical` backup; Infisical has its own DB backup; service token can be regenerated
Vaultwarden inaccessible after UID change	Low	High	LOW	Vaultwarden supports running as arbitrary UID; data dir is small and can be restored from B2

7. Verification Checklist

After Phase 9 completion, confirm:

8. Appendix: Complete Post-Phase-9 `.env.infisical`

# /opt/prodg/compose/.env.infisical
# This is the ONLY .env file that remains on disk.
# All actual secrets are in Infisical.
# If this token leaks, revoke it in Infisical and regenerate.
 
INFISICAL_SERVICE_TOKEN=st.********.************************
INFISICAL_PROJECT_ID=prodg-mainframe-core
INFISICAL_ENV_SLUG=production
INFISICAL_SITE_URL=https://secrets.mainframe.prodg.studio
 
# Optional: fallback for services that can't use Infisical directly
# (to be eliminated in Phase 10)
# COMPOSE_ENV_SOURCE=infisical

9. Appendix: User-Provided Secrets Registry (Post-Consolidation)

Service	Secret ID	Location	Owner	Purpose
Tavily	`TAVILY_API_KEY`	Infisical: agent-apis/prod	Mitch	Agent web search
Backblaze B2	`B2_KEY_SECRET`	Infisical: mainframe-core/prod	Mitch	Offsite backups
Backblaze B2	`B2_KEY_ID`	Infisical: mainframe-core/prod	Mitch	Offsite backups
Telegram Bot	`TELEGRAM_BOT_TOKEN`	Infisical: mainframe-core/prod	Mitch	Notifications
Vaultwarden	`VAULTWARDEN_ADMIN_TOKEN`	Infisical: mainframe-core/prod	Mitch	Admin panel
Headscale	SSH key `Mainframe`	`~prodg/.ssh/authorized_keys`	Mitch	Dev laptop access
ProDG Domain	`prodg.studio`	Namecheap / DNS	Mitch	Root domain

10. Next Steps (Phase 10 — Optional)

Docker Rootless Mode — run Docker daemon as prodg user, not root
Caddy non-root — run Caddy as prodg with CAP_NET_BIND_SERVICE
Podman migration — replace Docker with Podman for daemonless containers
HashiCorp Vault — if Infisical outgrown, migrate to Vault with auto-unseal
VPC / WireGuard mesh — replace Docker bridge with encrypted overlay network
Modal integration — productionize /v1/dispatch/modal with actual SDK

Document Version: 1.0.0 — Phase 9: Secrets Hardening & Production Hardening Generated by Hermes Agent for ProDG Studio

ProDG Knowledge Base

Explorer

05-phase-9-plan

Phase 9: Secrets Hardening, Service-User Migration & Production Hardening

1. Current State Snapshot

1.1 Infrastructure Already Deployed (Phases 0–8)

1.2 All Secrets Currently in Plaintext `.env`

1.3 User-Provided Context (NEW — April 29)

1.4 Services That Still Run as Root Inside Containers

2. Phase 9 Objectives

Primary

Secondary

3. Implementation Plan

3.1 Infisical Project Structure (NEW)

3.2 Step-by-Step Migration

Step 1: Bootstrap Infisical CLI on Host

Step 2: Create Infisical Service Token

Step 3: Migrate Secrets Programmatically

Step 4: Docker Compose `.env` Replacement

Step 5: Service User Migration (Docker Containers)

Step 6: rclone B2 Config Migration

Step 7: Vaultwarden Hardening

Step 8: Hermes API Tailscale-Only Binding

Step 9: Tavily API Key Injection

4. Agent Workload Policy (Formalized)

5. Order of Operations (Deployment Day)

Pre-Migration

Infisical Migration

Service User Migration

Hermes API (Custom Image Rebuild)

rclone Migration

Post-Migration Cleanup

6. Risk Assessment

7. Verification Checklist

8. Appendix: Complete Post-Phase-9 `.env.infisical`

9. Appendix: User-Provided Secrets Registry (Post-Consolidation)

10. Next Steps (Phase 10 — Optional)

Graph View

Table of Contents

Backlinks

ProDG Knowledge Base

Explorer

05-phase-9-plan

Phase 9: Secrets Hardening, Service-User Migration & Production Hardening

1. Current State Snapshot

1.1 Infrastructure Already Deployed (Phases 0–8)

1.2 All Secrets Currently in Plaintext .env

1.3 User-Provided Context (NEW — April 29)

1.4 Services That Still Run as Root Inside Containers

2. Phase 9 Objectives

Primary

Secondary

3. Implementation Plan

3.1 Infisical Project Structure (NEW)

3.2 Step-by-Step Migration

Step 1: Bootstrap Infisical CLI on Host

Step 2: Create Infisical Service Token

Step 3: Migrate Secrets Programmatically

Step 4: Docker Compose .env Replacement

Step 5: Service User Migration (Docker Containers)

Step 6: rclone B2 Config Migration

Step 7: Vaultwarden Hardening

Step 8: Hermes API Tailscale-Only Binding

Step 9: Tavily API Key Injection

4. Agent Workload Policy (Formalized)

5. Order of Operations (Deployment Day)

Pre-Migration

Infisical Migration

Service User Migration

Hermes API (Custom Image Rebuild)

rclone Migration

Post-Migration Cleanup

6. Risk Assessment

7. Verification Checklist

8. Appendix: Complete Post-Phase-9 .env.infisical

9. Appendix: User-Provided Secrets Registry (Post-Consolidation)

10. Next Steps (Phase 10 — Optional)

Graph View

Table of Contents

Backlinks

1.2 All Secrets Currently in Plaintext `.env`

Step 4: Docker Compose `.env` Replacement

8. Appendix: Complete Post-Phase-9 `.env.infisical`