Part of the SOA simplification & boundary-hardening remediation epic: https://github.com/mitodl/hq/issues/12151
Priority: p2 | Repo: mitodl/mit-learn
Description/Context
mit-learn re-normalizes 3 divergent catalog APIs via ~1150 lines of per-source ETL (etl/mitxonline.py 726 - mostly re-parsing the serialized req_tree back into relationships :411-575; xpro.py 259; micromasters.py 160) plus loaders.py 1996/utils.py 1333. Contracts differ on versioning (v2 vs v1+bare vs v0), pagination (only mitxonline paginates - xpro/micromasters return unpaginated lists, silent-truncation risk), published semantics (live+page__live vs coursepage-or-externalcoursepage vs live-only), department resolution by name string. RFC deliverable: one canonical "catalog feed" schema (aligned with the ol-data-platform webhook-push model and the existing learning_resources webhook contract from mit-learn PR #3557 - resources[] of canonical LearningResource dicts), standardized pagination/live semantics, and a canonical requirement-tree shape (mitxonline's operator tree is the only expressive one; xpro/micromasters emit degenerate trees). Decide per product line the system of record for catalog metadata vs edX Studio vs Wagtail (triple-entry today).
Note: Depends on the ETL ownership guard: #3565 (merged the guard rail; source cutover is separate follow-up work).
Plan/Design
Not yet started. See the linked epic for full cross-task context and evidence memories.
Part of the SOA simplification & boundary-hardening remediation epic: https://github.com/mitodl/hq/issues/12151
Priority: p2 | Repo: mitodl/mit-learn
Description/Context
mit-learn re-normalizes 3 divergent catalog APIs via ~1150 lines of per-source ETL (etl/mitxonline.py 726 - mostly re-parsing the serialized req_tree back into relationships :411-575; xpro.py 259; micromasters.py 160) plus loaders.py 1996/utils.py 1333. Contracts differ on versioning (v2 vs v1+bare vs v0), pagination (only mitxonline paginates - xpro/micromasters return unpaginated lists, silent-truncation risk), published semantics (live+page__live vs coursepage-or-externalcoursepage vs live-only), department resolution by name string. RFC deliverable: one canonical "catalog feed" schema (aligned with the ol-data-platform webhook-push model and the existing learning_resources webhook contract from mit-learn PR #3557 - resources[] of canonical LearningResource dicts), standardized pagination/live semantics, and a canonical requirement-tree shape (mitxonline's operator tree is the only expressive one; xpro/micromasters emit degenerate trees). Decide per product line the system of record for catalog metadata vs edX Studio vs Wagtail (triple-entry today).
Note: Depends on the ETL ownership guard: #3565 (merged the guard rail; source cutover is separate follow-up work).
Plan/Design
Not yet started. See the linked epic for full cross-task context and evidence memories.