# Production Runbook

## Deployment

Provision MySQL 8.4, authenticated Redis, private encrypted S3-compatible storage, SMTP, payment-provider credentials, TLS reverse proxy, API replicas, independent worker replicas, and exactly one cron scheduler. Inject secrets; never bake them into images.

1. Validate `.env` with `node server.js` in a sealed staging environment.
2. Take database/object-storage backups and run legacy prechecks.
3. Run `npx sequelize-cli db:migrate` against staging first; verify `SequelizeMeta`, constraints, indexes and counts.
4. Seed required roles/permissions, document types, shipping/configuration, SLA/help data using approved idempotent procedures.
5. Build the Node 22 image, scan it, deploy workers, API, and one `RUN_CRON=true` scheduler.
6. Configure Nginx/TLS/proxy trust; verify `/health/live` and `/health/ready`.
7. Run `npm run smoke`; run the staging-only commerce sequence with designated test identities/provider sandbox.
8. Monitor 5xx rate, readiness, DB/Redis, queue failures/backlog, webhook failures, cron failures, latency, memory and disk/log pressure.

Commands: API `npm start`; worker `node app/workers/index.js`; syntax `npm run check:syntax`; tests `npm test -- --runInBand`; dependencies `npm ls --depth=0`; audit `npm audit`; smoke `npm run smoke`; OpenAPI `npm run validate:openapi`.

Incident basics: stop hazardous writers, preserve logs/request IDs/provider event IDs, assess customer impact, rotate exposed credentials, prefer forward fixes, reconcile payments/inventory/ledgers, and communicate from verified database/provider truth. Roll back application images only when schema compatibility is proven; restore data only through the approved recovery procedure.