Sathira Sri Sathara
5158c52db5
- Introduced new endpoints for account overview, analytics, and metrics. - Implemented authorization matrix and backup/restore documentation. - Added smoke test script and updated package dependencies. - Created detailed production checklist and runbook for deployment. - Established cron operations and notification event matrix documentation. - Enhanced security and validation audits for analytics and metrics services. - Added unit tests for analytics and metrics functionalities.
1.7 KiB
Production Runbook
Deployment
Provision MySQL 8.4, authenticated Redis, private encrypted S3-compatible storage, SMTP, payment-provider credentials, TLS reverse proxy, API replicas, independent worker replicas, and exactly one cron scheduler. Inject secrets; never bake them into images.
- Validate
.envwithnode server.jsin a sealed staging environment. - Take database/object-storage backups and run legacy prechecks.
- Run
npx sequelize-cli db:migrateagainst staging first; verifySequelizeMeta, constraints, indexes and counts. - Seed required roles/permissions, document types, shipping/configuration, SLA/help data using approved idempotent procedures.
- Build the Node 22 image, scan it, deploy workers, API, and one
RUN_CRON=truescheduler. - Configure Nginx/TLS/proxy trust; verify
/health/liveand/health/ready. - Run
npm run smoke; run the staging-only commerce sequence with designated test identities/provider sandbox. - Monitor 5xx rate, readiness, DB/Redis, queue failures/backlog, webhook failures, cron failures, latency, memory and disk/log pressure.
Commands: API npm start; worker node app/workers/index.js; syntax npm run check:syntax; tests npm test -- --runInBand; dependencies npm ls --depth=0; audit npm audit; smoke npm run smoke; OpenAPI npm run validate:openapi.
Incident basics: stop hazardous writers, preserve logs/request IDs/provider event IDs, assess customer impact, rotate exposed credentials, prefer forward fixes, reconcile payments/inventory/ledgers, and communicate from verified database/provider truth. Roll back application images only when schema compatibility is proven; restore data only through the approved recovery procedure.