Skip to content
Meridian7 Wiki

Change Management Runbook

Overview

Section titled “Overview”

Operational companion to the Change Management Policy. Promotion chain: dev → staging → main (production). All changes are tracked via Git history, GitHub Issues, and PR comments.

1. Standard Change Process

Section titled “1. Standard Change Process”

Applies to all planned changes: application code, configuration, database migrations, CI/CD workflows, and policy documents.

StepAction
1. Branchgit checkout dev && git pull → git checkout -b feature/description
2. ImplementIncremental commits; imperative subject line; Fixes #123 where applicable
3. PR to devFill in: what changed, why, linked issue, testing notes
4. CIClaude Security Review, Policy Check run automatically
5. ReviewAt least one peer approval; PR author cannot self-approve; policies//procedures//guides/ require @Meridian7-io/security-reviewers
6. Merge to devAuto-deploys to dev-navi.meridian7.io; verify in dev environment
7. PR to stagingInclude dev verification results; validate at staging-navi.meridian7.io
8. PR to mainCTO reviews and approves; merges triggers production deploy
9. Post-deployMonitor GCP Cloud Monitoring 15 min; watch error rate, uptime, alert policies

If anything looks wrong post-deploy: initiate rollback (Section 4).

2. Emergency Change Process

Section titled “2. Emergency Change Process”

Applies only when an active SEV-1 or SEV-2 incident requires an immediate production fix.

Authorized for: active production incident, security vulnerability under active exploitation, data integrity issue requiring immediate correction.

  1. Post in #incidents before making any changes: description, authorizing role (CTO/CISO), UTC timestamp.
  2. CTO or CISO explicitly authorizes via Slack.
  3. Repo admin applies directly to main via admin override (tracked in GitHub audit log).
  4. CI lint, type-check, and build must pass; verbal review from one other engineer is sufficient.
  5. Create a GitHub incident issue immediately if one does not exist.
  6. Within 24 hours: open a retroactive PR with full description of what changed and why.
  7. Within 48 hours: post-implementation review documenting justification, commit SHA, testing performed, follow-up work. Attach to incident postmortem.

3. Infrastructure and Configuration Changes

Section titled “3. Infrastructure and Configuration Changes”

Applies to: Doppler secret updates, Cloudflare rules, GCP Cloud Run settings, Supabase configuration.

  1. Create a GitHub issue describing the change and expected post-change state.
  2. Apply the change in the target system.
  3. Run the automation verifier: 
    doppler run -p m7-security -c prd -- python3 automation/scripts/runner.py --mode verify --systems [system-name]
  4. If verify result changes, collect evidence: 
    doppler run -p m7-security -c prd -- python3 automation/scripts/runner.py --mode evidence --systems [system-name] --output-dir evidence
  5. Close issue with: what changed, verify output, evidence artifact paths.

Doppler Secret Rotation

Section titled “Doppler Secret Rotation”
  1. Generate new credential in the target system.
  2. Update in Doppler: doppler secrets set SECRET_NAME -p m7-security -c prd
  3. Verify dependent systems pick up new value (redeploy long-running services if needed).
  4. Revoke old credential in the target system.
  5. Run verify to confirm checks still pass.

4. Rollback Procedures

Section titled “4. Rollback Procedures”

4.1 GCP Cloud Run (web app)

Section titled “4.1 GCP Cloud Run (web app)”

GCP Console → Cloud Run → select web service → Revisions → select last known-good revision → set to 100% traffic.

4.2 GCP Cloud Run (agents)

Section titled “4.2 GCP Cloud Run (agents)”

GCP Console → Cloud Run → select agent service → Revisions → select previous known-good revision → set to 100% traffic.

4.3 Supabase (Database Migrations)

Section titled “4.3 Supabase (Database Migrations)”

Migrations are forward-only. For a migration that has run in production:

  1. Write a reverse migration SQL script.
  2. Put through the standard change process (branch → PR → review → merge).
  3. Never apply manually.

PITR (last resort — data loss risk): Supabase dashboard → Database → Backups. Creates a new instance; coordinate with CTO before initiating. Document the decision in a GitHub issue first.

4.4 Cloudflare

Section titled “4.4 Cloudflare”

Dashboard → select zone. Revert rules, DNS records, or proxy settings. Use Account-level Audit Log to identify what changed. Changes take effect within seconds.

4.5 Doppler

Section titled “4.5 Doppler”

doppler.com → project → config → secret → Version history. Revert via:

doppler secrets set SECRET_NAME -p m7-security -c prd

Ensure old credential is revoked in the target system before reverting.

4.6 Rollback Documentation

Section titled “4.6 Rollback Documentation”

All rollbacks must be documented in the GitHub issue with: reason, UTC timestamp, systems affected, person who initiated (role), follow-up root cause analysis issue.

5. Policy and Procedure Changes

Section titled “5. Policy and Procedure Changes”

Follow the standard change process. CODEOWNERS enforces @Meridian7-io/security-reviewers approval before merge. Full update process in Policy Update Procedure.

6. Change Tracking Reference

Section titled “6. Change Tracking Reference”
WhatTracked In
Application code changesGit commits + GitHub PRs
PR approvalsGitHub PR review history
Deployment eventsGCP Cloud Run deployment logs via Cloud Build (linked to commit SHA)
Infrastructure config changesGitHub Issues + verification runner output
Secret rotationsDoppler Activity log + GitHub issue
Emergency changesGitHub audit log + retroactive PR + incident postmortem
Policy changesGit commits + GitHub PR + GitHub Release
Section titled “7. Related Documents”

Meridian Seven — Confidential