Mailgun under pulumi management

[shaidar]

Description/Context

The Mailgun domains and Route53 DNS records across all environments (CI, QA/RC, Production — ~20 domains total) were created manually and are not under Pulumi management. This means changes are made ad-hoc outside of version control, there is no drift detection, and there is no consistent record of what DNS records or SMTP credentials exist per domain.

We need to:

1. Bring all existing Mailgun domains, SMTP credentials, and Route53 DNS records into Pulumi state via a one-time import
2. Refactor the mailgun Pulumi project (src/ol_infrastructure/applications/mailgun/) to manage multiple domains per stack from a structured config list
3. Update each stack's YAML config to declare all domains with their per-domain settings and encrypted credentials

Plan/Design

Phase 1 — One-time import script

A script src/ol_infrastructure/applications/mailgun/import_existing.py reads a per-environment JSON domain list, resolves Route53 zone IDs via boto3, and runs pulumi import for each existing resource per domain:

Resource	Pulumi type	Import ID format
mailgun.Domain	mailgun:index/domain:Domain	us:{domain}
mailgun.DomainCredential (one per login)	mailgun:index/domainCredential:DomainCredential	us:{login}@{domain}
SPF TXT record	aws:route53/record:Record	{zone_id}_{domain}_TXT
DKIM TXT record	aws:route53/record:Record	{zone_id}_mx._domainkey.{domain}_TXT
MX record	aws:route53/record:Record	{zone_id}_{domain}_MX
Tracking CNAME	aws:route53/record:Record	{zone_id}_email.{domain}_CNAME
DMARC TXT record	aws:route53/record:Record	{zone_id}__dmarc.{domain}_TXT

Usage (run from mailgun project dir with the target stack already selected):

pulumi stack select applications.mailgun.Production
uv run python import_existing.py --config domains-production.json --dry-run  # preview
uv run python import_existing.py --config domains-production.json             # execute

Repeat for QA and CI stacks. After each run, pulumi stack --show-urns should list all imported resources.

Phase 2 — Refactor main.py

Replace the current single hardcoded mitlearn domain with a loop over mailgun_config.require_object("domains"). Key points:

• Resource logical names must exactly match what the import script registered, e.g.:
  • ol-mailgun-domain-{name}
  • ol-mailgun-credential-{login}-{name}
  • ol-mailgun-spf-{name}, ol-mailgun-dkim-{name}, ol-mailgun-mx-{name}, ol-mailgun-tracking-{name}, ol-mailgun-dmarc-{name}
• mailgun.DomainCredential gets permanent ignore_changes=["password"] — the Mailgun API cannot return SMTP passwords so drift is undetectable; the config value is authoritative
• Per-domain settings that vary (e.g. use_automatic_sender_security, click_tracking, open_tracking) are read from config and passed through; inspect imported state via pulumi stack export before writing the code to confirm which fields differ across domains
• DNS zone ID is resolved per domain via dns_stack.get_output(d["zone_key"])["id"]

Phase 3 — Update stack YAML configs

Convert Pulumi.applications.mailgun.QA.yaml, Pulumi.applications.mailgun.Production.yaml, and a new CI stack YAML from the current single mitlearn:* key structure to a mailgun:domains list. Example entry:

mailgun:domains:
  - name: mail.learn.mit.edu
    zone_key: learn
    use_automatic_sender_security: true
    click_tracking: true
    open_tracking: true
    credentials:
      - login: no-reply
        smtp_password:
          secure: v1:...
      - login: postmaster
        smtp_password:
          secure: v1:...

Remove all mitlearn:* keys once migration is verified.

Verification

On each stack after import + code refactor:

pulumi preview    # must show zero pending changes

Send a test email per environment to confirm live DNS routing and SMTP credentials are intact post-migration.

Files

• main.py — full refactor (Phase 2)
• Pulumi.applications.mailgun.QA.yaml — update config (Phase 3)
• Pulumi.applications.mailgun.Production.yaml — update config (Phase 3)
• src/ol_infrastructure/applications/mailgun/Pulumi.applications.mailgun.CI.yaml — new (Phase 3)
• import_existing.py — one-time import script, can be deleted post-migration