diff --git a/docs/decisions/0014-newest-courses-sort-replica.rst b/docs/decisions/0014-newest-courses-sort-replica.rst new file mode 100644 index 00000000..b41ad885 --- /dev/null +++ b/docs/decisions/0014-newest-courses-sort-replica.rst @@ -0,0 +1,157 @@ +Newest-Courses-First Search Sort via a Recency-Sorted Algolia Replica +===================================================================== + +Status +------ + +Accepted + +Context +------- + +The enterprise Learner Portal search page sorts a single Algolia index by +relevance. Algolia does not re-sort an index at query time, so each alternate +sort order is a separate *replica* index with its own ``customRanking``; the +consumer switches sort by pointing its search at a different index name. + +The fundamental design of the enterprise-catalog logic assumes we will only ever +provision *one* replica: there is a single setting +(``ALGOLIA['REPLICA_INDEX_NAME']`` — the base "duration" replica the Learner +Portal points its video search at) and a single Python function that provisions +that one replica. + +Historically the whole ``ALGOLIA`` dict has been *replaced* (not merged) from the +deployment YAML, so anything code wants to keep in it — like a replica's ranking +definition — would be lost unless ops restated it in ``edx-internal``. + +We want to offer learners a "newest courses first" sort. The problem is how to +add a *second* replica — and roll it out across the repositories that together +own enterprise search — without a partially-configured replica degrading or +breaking the existing relevance search. + +Decision +-------- + +Add a recency-sorted replica (the ``enterprise_catalog_recently_released_desc`` +entry in ``ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS']``) that leads its ranking +with ``desc(recently_released_timestamp)`` — a per-course +Unix timestamp of the *earliest course-run start of any status* (the Discovery +course release date, the same signal as the ``is_new_content`` flag, via the +shared ``_earliest_course_run_start`` helper — ENT-11386). Courses with no run +start get ``0`` so they sort last under a descending ranking — deliberately not +the far-future ``ALGOLIA_DEFAULT_TIMESTAMP``, which would float undated courses to +the top. The primary index (``ALGOLIA['INDEX_NAME']``) keeps the relevance +ranking. + +The replica is declared as an Algolia *virtual* replica (``virtual(name)``), so it +mirrors the primary's records instead of duplicating them. This is a deliberate +cost/precision tradeoff versus a standard replica — see *Alternatives considered* +and *Consequences*. + +The sort is rolled out across three repositories: + +#. **enterprise-catalog** (this service) builds and configures the replica. +#. **edx-enterprise** exposes the ``enterprise.search_default_sort_newest`` + waffle flag via ``enterprise_features`` — the eligibility gate / kill-switch. +#. **frontend-app-learner-portal-enterprise** points the course ```` at + the replica when the flag is on *and* the Optimizely "newest" experiment + variant is active for the user. + +The design generalizes the old one-replica assumption to a settings-driven map, and +gates *user exposure* with the waffle flag rather than gating *declaration* on ops: + +* Backend: additional sort replicas are declared in + ``ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS']`` — an ``index_name -> index settings`` + map defined in ``settings/base.py`` as config-as-code (the ``customRanking`` is code, + since it sorts on a field the indexer computes). ``ALGOLIA`` is added to + ``DICT_UPDATE_KEYS`` so the deployment YAML is now *merged*, not replaced: ops can + override per-environment index names and credentials while these code-defined replica + settings are preserved. One map is the single source of truth — its entries are + declared on the primary index, configured during a reindex, and added to the secured-key + ``restrictIndices`` — so adding a future sort is one new entry plus the field its + ``customRanking`` sorts on. (The base ``REPLICA_INDEX_NAME`` keeps its own + per-environment key and its required-core-pair lifecycle in ``init_index`` / + ``index_exists``.) +* Backend (fail-safe): configuring each replica in ``configure_algolia_index`` is + wrapped so that any ``AlgoliaException`` is logged and skipped. One replica + failing to configure never aborts the reindex — the primary (relevance) index + and the other replicas are still configured. The failed replica keeps its prior + settings (or, when brand new, mirrors the primary's relevance ranking) until the + next successful run, so the degraded state is still the safe base sort. +* MFE: the course search uses the replica only when its index-name config var is + non-empty (and the flag + experiment gates pass); otherwise it falls back to + the primary (relevance) index. + +**The open question this ADR records:** how should we handle the case where the +replica *name is configured* but the Algolia index does **not yet exist** (e.g. +the MFE env var is set before this service has been deployed and reindexed)? + +The proposed answer is to **rely on operational guarantees rather than runtime +index-existence detection**: + +#. the documented rollout order — deploy + ``reindex_algolia`` here *before* + pointing the MFE at the replica; and +#. the ``enterprise.search_default_sort_newest`` waffle flag, which doubles as a + readiness gate and an instant kill-switch — it should not be enabled until the + replica is live, and flipping it off immediately reverts every learner to the + relevance index. + +We deliberately do **not** add code that detects a missing Algolia index at +search time and silently falls back to the primary index. + +Consequences +------------ + +* **Exposure is MFE/flag-gated, not declaration-gated:** the backend now declares the + replica in every environment (it ships in ``ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS``), so + "is it declared" is no longer the gate. The course search falls back to the primary + (relevance) index whenever the MFE's replica env var is unset or the flag/experiment is + off (the ``&& recentlyReleasedIndexName`` guard), so user exposure is controlled by the + MFE env var + waffle flag. Because the replica is *virtual* (no extra records), always + declaring it costs nothing. +* **No added record cost:** because the replica is *virtual*, it mirrors the + primary's records rather than duplicating them, so adding it does not grow our + Algolia record count. A standard (non-virtual) replica would roughly double the + indexed record count — and its cost — for each sort we add (see *Alternatives*). +* **Non-course records sort last, by design:** the replica is *virtual* over the + primary index, so it mirrors every record — programs, executive education, videos, + etc. — not just courses. ``recently_released_timestamp`` is only computed in the + ``content_type == COURSE`` branch, so non-course records have no such attribute and + Algolia ranks them last under ``desc(recently_released_timestamp)``. This is fine + because the consumer (the Learner Portal) points only its course ```` at the + replica and filters by content type; the "newest courses first" sort is, by + contract, a course sort. Were a future caller to query this replica for non-course + content, those records would all tie at the bottom — that caller would need its own + recency field. +* **The waffle flag is the readiness contract:** enabling it asserts "the replica + is live." This keeps the safe path a single, instantly reversible toggle + rather than per-request defensive logic in the search hot path. +* **Escape hatch is recorded:** if the operational mitigation proves too fragile + in practice, the documented next step is an ``onError``/try-primary fallback in + the MFE search path (see *Alternatives*). Capturing the question here lets us + revisit it without re-discovering the trade-off. + +Alternatives considered +------------------------ + +* **Runtime index-existence detection / fall back to base on Algolia error.** + Rejected for now: react-instantsearch would need an error path that re-renders + against the primary index, adding per-search complexity and an extra failure + mode, to protect a transient window the kill-switch flag already guards. It + remains the documented escape hatch if the operational approach proves + insufficient. +* **Always declare the replica (no config gate).** Rejected: would create a + ``virtual(None)`` replica on the primary index in environments where the name + is unset, and would couple every environment to the rollout. +* **A standard (non-virtual) replica instead of a virtual one.** A standard + replica is a full, independent copy of the index that sorts strictly by its own + ``ranking``/``customRanking`` — a fully deterministic newest-first order + regardless of the search query. We chose a *virtual* replica instead: a virtual + replica reuses the primary's records, so it adds no record count or cost (see + *Consequences*), whereas a standard replica roughly doubles our indexed record + count and its associated cost. The accepted tradeoff is that a virtual replica + always keeps textual relevance as the top-priority sort factor, so under a text + query "newest first" is relevance-biased rather than strictly chronological (it + *is* strictly chronological when browsing with no query). If a strictly + deterministic order under query later proves necessary, switching this one + replica to a standard replica is the documented escape hatch. diff --git a/docs/how_to/add_an_algolia_sort_replica.rst b/docs/how_to/add_an_algolia_sort_replica.rst new file mode 100644 index 00000000..7ec5165f --- /dev/null +++ b/docs/how_to/add_an_algolia_sort_replica.rst @@ -0,0 +1,144 @@ +Add a new Algolia sort replica +============================== + +The Learner Portal search page sorts a single Algolia index by relevance. Algolia does +not re-sort an index at query time, so every alternate sort order is a separate *replica* +index with its own ``customRanking``; the consumer (the MFE) switches sort by pointing its +search at a different index name. + +Beyond the base replica (``ALGOLIA['REPLICA_INDEX_NAME']``, used by the MFE video search), +every additional sort replica is declared in **one place** -- the +``ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS']`` map -- so adding a sort is mostly additive. +This guide walks through it end to end. See +``docs/decisions/0014-newest-courses-sort-replica.rst`` for the design rationale, and treat the +**recently-released ("newest first") replica** as the canonical example to copy. + +The mental model: one settings-driven map +------------------------------------------ + +``ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS']`` maps an **index name** to that replica's +**Algolia index settings** (its ``customRanking``). It is defined in ``settings/base.py`` as +config-as-code: a replica's ranking sorts on a field the indexing code must compute, so the +definition is intrinsically code, not deployment config. + +``ALGOLIA`` is *merged* (not replaced) from the deployment config -- it is listed in +``DICT_UPDATE_KEYS`` (``settings/production.py``) -- so an environment can override the per-env +index names / credentials while the code-defined ``ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS`` defaults are +preserved. + +The backend declares and configures every replica in this map on each reindex, and the secured API +key grants access to them. A sort is only *user-visible* once the MFE points a search at it, gated +by a waffle flag / Optimizely experiment -- so the flag, not the map, controls exposure (see +*Frontend* below and ADR 0014). + +Worked example +-------------- + +Suppose we want a **"price: low to high"** sort, using an index named +``enterprise_catalog_price_asc``. + +Backend steps (this service) +---------------------------- + +**1. Make sure the field you sort on is indexed.** A ``customRanking`` can only sort on a numeric +attribute that exists on the records. If your sort uses a field that is *already* indexed (the +price example reuses ``first_enrollable_paid_seat_price``), skip this step. If it needs a new +signal, in ``enterprise_catalog/apps/catalog/algolia_utils.py``: + +* write a ``get_course_(course)`` helper that returns the numeric value (see + ``get_course_recently_released_timestamp`` -- note it returns ``0`` for "no value" so those + records sort last under a ``desc`` ranking; pick a sentinel that sorts your missing values to the + *end* of *your* order); +* add the field name to ``ALGOLIA_FIELDS``; +* set it on the course object in ``_algolia_object_from_product`` (the ``content_type == COURSE`` + branch). + +**2. Define the replica's ranking and register it.** In ``enterprise_catalog/settings/base.py``, +add a settings constant and an entry in ``ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS']`` keyed by +the index name. Lead the ``customRanking`` with your sort criterion, then append the primary +index's shared tie-breakers so records that tie on your criterion (and any "missing value" bucket) +fall back to the relevance ordering and pagination stays deterministic: + +.. code-block:: python + + ALGOLIA_PRICE_ASC_REPLICA_INDEX_SETTINGS = { + 'customRanking': [ + 'asc(first_enrollable_paid_seat_price)', + # shared tie-breakers (same as the primary index) -- keep ties stable + 'asc(metadata_language)', + 'asc(visible_via_association)', + 'asc(created)', + 'desc(course_bayesian_average)', + 'desc(recent_enrollment_count)', + ], + } + + ALGOLIA = { + 'INDEX_NAME': '', + 'REPLICA_INDEX_NAME': '', + 'ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS': { + 'enterprise_catalog_recently_released_desc': ALGOLIA_RECENTLY_RELEASED_REPLICA_INDEX_SETTINGS, + 'enterprise_catalog_price_asc': ALGOLIA_PRICE_ASC_REPLICA_INDEX_SETTINGS, + }, + 'APPLICATION_ID': '', + 'API_KEY': '', + } + +**That is all the wiring.** You do **not** touch ``_get_algolia_replica_names``, +``_configured_replicas``, ``configure_algolia_index``, or the secured-key ``replica_index_names`` +-- they all read ``ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS``, so the new replica is automatically declared +on the primary index, has its settings applied during a reindex, and is added to the secured API +key's ``restrictIndices``. + +Tests +----- + +* Extend the configure / registry tests in + ``enterprise_catalog/apps/catalog/tests/test_algolia_utils.py`` (e.g. + ``test_get_algolia_replica_names_combines_base_and_additional_replicas`` and + ``test_configure_algolia_index_configures_additional_replica``) to cover the new replica, using + ``override_settings(ALGOLIA={...})``. +* If you added a field computation, unit-test the ``get_course_`` helper. +* The secured-key tests in ``api_client/tests/test_algolia.py`` exercise ``restrictIndices``; add + the new index name to the "all indices" expectation if you want explicit coverage. + +Deploy / ops +------------ + +Once the code is merged and deployed: + +#. The replica's name and settings ship in code (``ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS``), so no + edx-internal change is required to *declare* it -- and normally ops should not override it at all. + The ``ALGOLIA`` merge is *shallow*: the deployment YAML's top-level ``ALGOLIA`` keys override the + code defaults, but nested dicts are **not** deep-merged. So setting + ``ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS']`` in edx-internal **replaces the entire map** + for that environment (dropping every code-defined replica it does not restate) rather than + overriding individual entries. Prefer setting the index name in code; override the map only when + you intend to fully restate it. +#. Run ``./manage.py reindex_algolia``. A *virtual* replica exists as soon as it is declared on the + primary index's settings (it mirrors the primary's records), so the replica is live after one + reindex -- no separate population step. + +Frontend (only if the MFE will use the sort) +-------------------------------------------- + +The backend builds the replica regardless; a sort is only *user-visible* once the MFE points a +search at it. In ``frontend-app-learner-portal-enterprise``: + +* add an ``ALGOLIA__REPLICA_INDEX_NAME`` env var in ``src/index.tsx`` and + ``src/types/types.d.ts`` whose value matches the backend index name; +* point the relevant ```` at it (see ``SearchVideo.jsx``, which uses the base + replica, or ``SearchCourse.jsx`` for the recency replica); +* gate it behind a waffle flag and/or an Optimizely experiment, exactly as the recency sort does + (the flag doubles as a kill-switch -- see ADR 0014). + +Safety properties +----------------- + +* **Virtual replicas, no extra records.** Each replica is declared ``virtual(name)``, so it mirrors + the primary's records rather than duplicating them -- no added Algolia record count or cost. +* **Fail-safe configuration.** ``configure_algolia_index`` wraps each replica's settings call so an + ``AlgoliaException`` is logged and skipped -- one replica failing to configure never aborts the + reindex, and the primary index plus the other replicas stay configured. +* **Flag-gated exposure.** Declaring a replica does not make it user-visible; the MFE only queries + it when its waffle flag / experiment is on, so the flag is the kill-switch. diff --git a/enterprise_catalog/apps/api_client/algolia.py b/enterprise_catalog/apps/api_client/algolia.py index 65dfc422..8f209e30 100644 --- a/enterprise_catalog/apps/api_client/algolia.py +++ b/enterprise_catalog/apps/api_client/algolia.py @@ -46,6 +46,22 @@ def algolia_index_name(self): def algolia_replica_index_name(self): return settings.ALGOLIA.get('REPLICA_INDEX_NAME') + @property + def replica_index_names(self): + """ + Index names of every configured sort replica (base + additional), empty when none are set. + + The base replica is ``ALGOLIA['REPLICA_INDEX_NAME']`` (when set); the additional sort + replicas are the keys of ``ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS']``. Used to scope the + secured API key to exactly the replicas that exist. + """ + names = [] + base_replica = settings.ALGOLIA.get('REPLICA_INDEX_NAME') + if base_replica: + names.append(base_replica) + names.extend(settings.ALGOLIA.get('ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS', {}).keys()) + return names + def init_index(self): """ Initializes an index within Algolia. Initializing an index will create it if it doesn't exist. @@ -85,29 +101,37 @@ def init_index(self): ) raise exc - def set_index_settings(self, index_settings, primary_index=True): + def set_index_settings(self, index_settings, index_name=None): """ - Set default settings to use for the Algolia index. + Set settings on an Algolia index, defaulting to the primary index. + + Pass ``index_name`` to target a replica instead -- the base ``REPLICA_INDEX_NAME`` + replica, the "newest courses" sort replica, etc. A replica must already be + declared on the primary index's ``replicas`` setting (Algolia creates replica + indices when the primary index's settings are saved). Note: This will override manual updates to the index configuration on the Algolia dashboard but ensures consistent settings (configuration as code). Arguments: - settings (dict): A dictionary of Algolia settings. + index_settings (dict): A dictionary of Algolia settings. + index_name (str): Optional index to target; defaults to the primary index. """ if not self.algolia_index: logger.error('Algolia index does not exist. Did you initialize it?') return try: - if primary_index: - self.algolia_index.set_settings(index_settings) - else: - self.replica_index.set_settings(index_settings) + self._get_index(index_name).set_settings(index_settings) except AlgoliaException as exc: + # With no index_name, the target is the cached handle (self.algolia_index), which a + # caller may have wired to an alternate index (e.g. the incremental reindex command's + # --index-name path). Report that handle's actual name rather than the configured + # primary name (algolia_index_name), which could be stale or empty for that caller. + effective_index_name = index_name or getattr(self.algolia_index, 'name', None) or self.algolia_index_name logger.exception( 'Unable to set settings for Algolia\'s %s index due to an exception.', - self.algolia_index_name, + effective_index_name, ) raise exc @@ -422,8 +446,7 @@ def generate_secured_api_key(self, user_id, enterprise_catalog_query_uuids): indices = [] if self.algolia_index_name: indices.append(self.algolia_index_name) - if self.algolia_replica_index_name: - indices.append(self.algolia_replica_index_name) + indices.extend(self.replica_index_names) if indices: restrictions |= {'restrictIndices': indices} diff --git a/enterprise_catalog/apps/api_client/tests/test_algolia.py b/enterprise_catalog/apps/api_client/tests/test_algolia.py index 93f32d8d..f949bc59 100644 --- a/enterprise_catalog/apps/api_client/tests/test_algolia.py +++ b/enterprise_catalog/apps/api_client/tests/test_algolia.py @@ -6,7 +6,7 @@ import ddt from algoliasearch.exceptions import AlgoliaException from django.core.exceptions import ImproperlyConfigured -from django.test import TestCase +from django.test import TestCase, override_settings from enterprise_catalog.apps.api_client.algolia import AlgoliaSearchClient @@ -87,6 +87,116 @@ def test_get_index_raises_when_primary_uninitialized(self): with self.assertRaises(ImproperlyConfigured): client._get_index() + def test_set_index_settings_targets_primary_by_default(self): + """ + ``set_index_settings()`` with no name applies settings to the cached primary index. + """ + client = self._build_client() + settings_payload = {'customRanking': ['asc(created)']} + + client.set_index_settings(settings_payload) + + client.algolia_index.set_settings.assert_called_once_with(settings_payload) + client._client.init_index.assert_not_called() # primary is cached, not re-initialized + + def test_set_index_settings_targets_named_replica(self): + """ + ``set_index_settings(index_name=...)`` applies settings to the replica resolved by name. + """ + client = self._build_client() + replica = mock.MagicMock(name='recently_released_replica') + client._client.init_index.return_value = replica + settings_payload = {'customRanking': ['desc(recently_released_timestamp)']} + + client.set_index_settings(settings_payload, index_name='enterprise_catalog_recently_released_desc') + + client._client.init_index.assert_called_once_with('enterprise_catalog_recently_released_desc') + replica.set_settings.assert_called_once_with(settings_payload) + + def test_set_index_settings_noop_when_primary_uninitialized(self): + """ + With no primary index initialized, the call logs and returns without touching Algolia. + """ + client = AlgoliaSearchClient() # nothing initialized + client.set_index_settings({'customRanking': []}, index_name='some_replica') # must not raise + + def test_set_index_settings_reraises_algolia_exception(self): + """ + An AlgoliaException from set_settings propagates to the caller. + """ + client = self._build_client() + replica = mock.MagicMock(name='recently_released_replica') + replica.set_settings.side_effect = AlgoliaException('boom') + client._client.init_index.return_value = replica + with self.assertRaises(AlgoliaException): + client.set_index_settings({'customRanking': []}, index_name='some_replica') + + def test_set_index_settings_failure_logs_effective_index_name(self): + """ + When no index_name is given but the cached handle is wired to an alternate index, the + failure log reports that handle's actual name -- not the (possibly stale/empty) configured + primary name. Mirrors the incremental reindex command's --index-name path. + """ + client = self._build_client() + # The cached handle is wired to an alternate index, as the incremental command does. Use a + # name with no substring overlap with PRIMARY_INDEX_NAME so the assertions can't false-pass. + wired_index_name = 'some_alternate_index' + client.algolia_index.name = wired_index_name + client.algolia_index.set_settings.side_effect = AlgoliaException('boom') + + with self.assertLogs('enterprise_catalog.apps.api_client.algolia', level='ERROR') as logs: + with self.assertRaises(AlgoliaException): + client.set_index_settings({'customRanking': []}) # no index_name -> targets the handle + + # Assert on the formatted message only; the captured traceback contains the package path + # ("enterprise_catalog/...") which would otherwise collide with PRIMARY_INDEX_NAME. + message = logs.records[0].getMessage() + self.assertIn(wired_index_name, message) + self.assertNotIn(self.PRIMARY_INDEX_NAME, message) + + @override_settings(ALGOLIA={ + 'INDEX_NAME': 'enterprise_catalog', + 'REPLICA_INDEX_NAME': 'enterprise_catalog_duration_desc', + 'ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS': { + 'enterprise_catalog_recently_released_desc': {'customRanking': ['desc(recently_released_timestamp)']}, + }, + 'SEARCH_API_KEY': 'fake-search-key', + }) + @mock.patch('enterprise_catalog.apps.api_client.algolia.SearchClient.generate_secured_api_key') + def test_generate_secured_api_key_restricts_to_all_indices(self, mock_generate): + """The secured key's restrictIndices covers the primary, duration, and recency replicas.""" + mock_generate.return_value = 'secured-key' + client = AlgoliaSearchClient() + + result = client.generate_secured_api_key('user-1', ['query-uuid-1']) + + assert result['secured_api_key'] == 'secured-key' + _api_key, restrictions = mock_generate.call_args[0] + assert restrictions['restrictIndices'] == [ + 'enterprise_catalog', + 'enterprise_catalog_duration_desc', + 'enterprise_catalog_recently_released_desc', + ] + + @override_settings(ALGOLIA={ + 'INDEX_NAME': 'enterprise_catalog', + 'REPLICA_INDEX_NAME': 'enterprise_catalog_duration_desc', + 'SEARCH_API_KEY': 'fake-search-key', + }) + @mock.patch('enterprise_catalog.apps.api_client.algolia.SearchClient.generate_secured_api_key') + def test_generate_secured_api_key_omits_recency_replica_when_unset(self, mock_generate): + """When no recency replica is configured, it is excluded from restrictIndices.""" + mock_generate.return_value = 'secured-key' + client = AlgoliaSearchClient() + + client.generate_secured_api_key('user-1', ['query-uuid-1']) + + _api_key, restrictions = mock_generate.call_args[0] + assert restrictions['restrictIndices'] == [ + 'enterprise_catalog', + 'enterprise_catalog_duration_desc', + ] + def test_save_objects_batch_calls_save_on_primary(self): """ ``save_objects_batch`` delegates to ``save_objects`` on the primary index by default. diff --git a/enterprise_catalog/apps/catalog/algolia_utils.py b/enterprise_catalog/apps/catalog/algolia_utils.py index a18c1e42..9a071857 100644 --- a/enterprise_catalog/apps/catalog/algolia_utils.py +++ b/enterprise_catalog/apps/catalog/algolia_utils.py @@ -3,10 +3,12 @@ import logging import time +from algoliasearch.exceptions import AlgoliaException from algoliasearch.search_client import SearchClient from dateutil import parser from dateutil.relativedelta import relativedelta from django.conf import settings +from django.core.exceptions import ImproperlyConfigured from django.db.models import Q from django.utils.dateparse import parse_datetime from django.utils.translation import gettext as _ @@ -64,9 +66,25 @@ ALGOLIA_UUID_BATCH_SIZE = 100 ALGOLIA_JSON_METADATA_MAX_SIZE = 100000 -ALGOLIA_REPLICA_INDEX_NAME = settings.ALGOLIA.get('REPLICA_INDEX_NAME') -algolia_replica_index = f'virtual({ALGOLIA_REPLICA_INDEX_NAME})' + +def _get_algolia_replica_names() -> list[str]: + """ + Build the list of replica index names to declare on the primary index. + + Returns a ``virtual(name)`` entry for the base replica (``ALGOLIA['REPLICA_INDEX_NAME']``, when + set) and for each additional sort replica (the keys of + ``ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS']``). An unset base replica is omitted, so we never + declare a ``virtual(None)`` replica on the primary index. + """ + replica_names = [] + base_replica = settings.ALGOLIA.get('REPLICA_INDEX_NAME') + if base_replica: + replica_names.append(f'virtual({base_replica})') + for index_name in settings.ALGOLIA.get('ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS', {}): + replica_names.append(f'virtual({index_name})') + return replica_names + # keep attributes from content objects that we explicitly want in Algolia ALGOLIA_FIELDS = [ @@ -142,6 +160,7 @@ 'transcript_languages', 'translation_languages', 'is_new_content', + 'recently_released_timestamp', ] # default configuration for the index @@ -206,9 +225,6 @@ 'desc(course_bayesian_average)', 'desc(recent_enrollment_count)', ], - 'replicas': [ - algolia_replica_index - ], } ALGOLIA_REPLICA_INDEX_SETTINGS = { @@ -223,6 +239,25 @@ } +def _configured_replicas(): + """ + Return ``(index_name, index_settings)`` for each replica to configure. + + Covers the base replica (``ALGOLIA['REPLICA_INDEX_NAME']`` with ``ALGOLIA_REPLICA_INDEX_SETTINGS``, + when set) and every additional sort replica declared in + ``ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS']`` (an ``index_name -> settings`` map). Because each + additional replica carries its own settings, there is no separate settings lookup that can drift + out of sync; this is inert until at least one replica is configured. + """ + configured = [] + base_replica = settings.ALGOLIA.get('REPLICA_INDEX_NAME') + if base_replica: + configured.append((base_replica, ALGOLIA_REPLICA_INDEX_SETTINGS)) + for index_name, index_settings in settings.ALGOLIA.get('ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS', {}).items(): + configured.append((index_name, index_settings)) + return configured + + def _should_index_course(course_metadata): """ Replicates the B2C index check of whether a certain course should be indexed for search. @@ -413,10 +448,37 @@ def new_search_client_or_error(): def configure_algolia_index(algolia_client): """ - Configures the settings for an Algolia index. + Configures the settings for the primary Algolia index and its replicas. """ - algolia_client.set_index_settings(ALGOLIA_INDEX_SETTINGS) - algolia_client.set_index_settings(ALGOLIA_REPLICA_INDEX_SETTINGS, primary_index=False) + if not algolia_client.algolia_index: + # init_index() logs and returns without setting algolia_index when a required name + # (INDEX_NAME / REPLICA_INDEX_NAME) or credential is missing. set_index_settings() would + # then silently no-op for every index, making a misconfigured reindex look successful. + # Fail loudly here instead so a missing/empty primary index name can't pass as success. + raise ImproperlyConfigured( + 'Cannot configure Algolia index: the primary index is not initialized. Check the ' + 'ALGOLIA settings (INDEX_NAME, REPLICA_INDEX_NAME, APPLICATION_ID, API_KEY).' + ) + # Declare the replicas dynamically (from the current settings.ALGOLIA) rather than from an + # import-time global, so the set the primary declares stays consistent with the replicas + # _configured_replicas() actually configures below -- and so tests can vary it via + # override_settings. + primary_settings = {**ALGOLIA_INDEX_SETTINGS, 'replicas': _get_algolia_replica_names()} + algolia_client.set_index_settings(primary_settings) + for index_name, index_settings in _configured_replicas(): + # A replica's settings failing to apply must never abort the reindex: log and continue so + # the primary (relevance) index and the other replicas stay configured. The failed replica + # keeps its prior settings (or, if brand new, mirrors the primary's relevance ranking) + # until the next successful run, so the safe fallback is the base sort. See ADR 0014. + try: + algolia_client.set_index_settings(index_settings, index_name=index_name) + except AlgoliaException: + # set_index_settings() already logged the traceback before re-raising, so log a + # single-line warning here rather than a second stack trace. + LOGGER.warning( + 'Failed to configure replica index "%s"; continuing with the remaining indexes.', + index_name, + ) def get_algolia_object_id(content_type, uuid): @@ -632,18 +694,44 @@ def is_course_archived(course): return len(availability_list) == 0 or 'Archived' in availability_list -def is_course_new_content(course): - """True if the course's earliest run start (any status) is within the last 12 months (ENT-11386).""" +def _earliest_course_run_start(course): + """ + The earliest course-run start as a timezone-aware datetime, or None. + + Shared by the "new content" facet and the "newest courses" sort timestamp so both key off + the same signal: a course's earliest run start across runs of *any* status, matching the + Discovery course release date (ENT-11386). Earlier runs that were later unpublished/archived + still count, so a recent re-run can't make a long-standing course look new. + """ starts = [] for run in course.get('course_runs') or []: if run.get('start'): parsed = parse_datetime(run['start']) if parsed: starts.append(parsed) - if not starts: + return min(starts) if starts else None + + +def is_course_new_content(course): + """True if the course's earliest run start (any status) is within the last 12 months (ENT-11386).""" + earliest_start = _earliest_course_run_start(course) + if earliest_start is None: return False now = localized_utcnow() - return now - relativedelta(months=12) <= min(starts) <= now + return now - relativedelta(months=12) <= earliest_start <= now + + +def get_course_recently_released_timestamp(course) -> int: + """ + Unix timestamp (int) of the course's earliest course-run start (any status), used as the + custom-ranking attribute for the "newest courses first" Algolia replica. + + Sorting this descending surfaces courses released most recently. Courses with no run start + default to 0 so they sort last under a desc ranking -- ``ALGOLIA_DEFAULT_TIMESTAMP`` (a + far-future sentinel) is deliberately NOT used, since it would float undated courses to the top. + """ + earliest_start = _earliest_course_run_start(course) + return int(earliest_start.timestamp()) if earliest_start else 0 def get_course_partners(course): @@ -1674,6 +1762,7 @@ def _algolia_object_from_product(product, algolia_fields): 'translation_languages': get_course_translation_languages(searchable_product), 'metadata_language': searchable_product.get('metadata_language', 'en'), 'is_new_content': is_course_new_content(searchable_product), + 'recently_released_timestamp': get_course_recently_released_timestamp(searchable_product), }) elif searchable_product.get('content_type') == PROGRAM: # Build course metadata cache once for all program functions that need it diff --git a/enterprise_catalog/apps/catalog/tests/test_algolia_utils.py b/enterprise_catalog/apps/catalog/tests/test_algolia_utils.py index d76e5a29..69d4f099 100644 --- a/enterprise_catalog/apps/catalog/tests/test_algolia_utils.py +++ b/enterprise_catalog/apps/catalog/tests/test_algolia_utils.py @@ -3,8 +3,10 @@ from uuid import uuid4 import ddt +from algoliasearch.exceptions import AlgoliaException from dateutil.relativedelta import relativedelta -from django.test import TestCase +from django.core.exceptions import ImproperlyConfigured +from django.test import TestCase, override_settings from enterprise_catalog.apps.catalog import algolia_utils as utils from enterprise_catalog.apps.catalog.algolia_utils import _get_course_run @@ -247,6 +249,78 @@ def test_is_course_new_content(self): {'status': 'published', 'start': 'not-a-date'}, ]}) is False + def test_get_course_recently_released_timestamp(self): + """ + Verify the "newest courses" sort timestamp uses the earliest run start (any status). + """ + now = localized_utcnow() + recent = now - timedelta(days=30) + old = now - timedelta(days=500) + + # No run start -> 0 so the course sorts last under a desc ranking. + assert utils.get_course_recently_released_timestamp({'course_runs': []}) == 0 + # Malformed ISO strings are silently ignored (treated as no date). + assert utils.get_course_recently_released_timestamp( + {'course_runs': [{'status': 'published', 'start': 'not-a-date'}]} + ) == 0 + # Run status is irrelevant: an unpublished run's start still counts. + assert utils.get_course_recently_released_timestamp( + {'course_runs': [{'status': 'unpublished', 'start': recent.isoformat()}]} + ) == int(recent.timestamp()) + # A run start -> its Unix timestamp, regardless of age (unlike is_new_content's 12-month window). + assert utils.get_course_recently_released_timestamp( + {'course_runs': [{'status': 'published', 'start': recent.isoformat()}]} + ) == int(recent.timestamp()) + assert utils.get_course_recently_released_timestamp( + {'course_runs': [{'status': 'published', 'start': old.isoformat()}]} + ) == int(old.timestamp()) + # Earliest start wins across any status: an old unpublished run is the release date. + assert utils.get_course_recently_released_timestamp({'course_runs': [ + {'status': 'unpublished', 'start': old.isoformat()}, + {'status': 'published', 'start': recent.isoformat()}, + ]}) == int(old.timestamp()) + assert utils.get_course_recently_released_timestamp({'course_runs': [ + {'status': 'published', 'start': old.isoformat()}, + {'status': 'published', 'start': recent.isoformat()}, + ]}) == int(old.timestamp()) + + def test_get_algolia_replica_names_combines_base_and_additional_replicas(self): + """The base replica and every additional sort replica are declared as virtual replicas.""" + # pylint: disable=protected-access + with override_settings(ALGOLIA={ + 'REPLICA_INDEX_NAME': 'enterprise_catalog_duration_desc', + 'ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS': { + 'enterprise_catalog_recently_released_desc': {'customRanking': []}, + }, + }): + assert utils._get_algolia_replica_names() == [ + 'virtual(enterprise_catalog_duration_desc)', + 'virtual(enterprise_catalog_recently_released_desc)', + ] + # Only the base replica configured -> only it is declared. + with override_settings(ALGOLIA={'REPLICA_INDEX_NAME': 'enterprise_catalog_duration_desc'}): + assert utils._get_algolia_replica_names() == ['virtual(enterprise_catalog_duration_desc)'] + # An unset base replica is omitted (never virtual(None)); additional replicas still declare. + with override_settings(ALGOLIA={ + 'REPLICA_INDEX_NAME': '', + 'ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS': {'enterprise_catalog_recently_released_desc': {}}, + }): + assert utils._get_algolia_replica_names() == ['virtual(enterprise_catalog_recently_released_desc)'] + # Nothing configured -> no replicas at all. + with override_settings(ALGOLIA={}): + assert not utils._get_algolia_replica_names() + + def test_algolia_object_includes_recently_released_timestamp(self): + """The course Algolia object carries recently_released_timestamp (and is_new_content).""" + # pylint: disable=protected-access + course_metadata = ContentMetadataFactory(content_type=COURSE) + algolia_object = utils._algolia_object_from_product( + course_metadata.json_metadata, utils.ALGOLIA_FIELDS, + ) + expected = utils.get_course_recently_released_timestamp(course_metadata.json_metadata) + assert algolia_object['recently_released_timestamp'] == expected + assert 'is_new_content' in algolia_object + @ddt.data( ( { @@ -991,12 +1065,90 @@ def test_configure_algolia_index(self, mock_search_client): Verify that `configure_algolia_index_settings` makes call to configure index settings. """ algolia_client = utils.get_initialized_algolia_client() - utils.configure_algolia_index(algolia_client) - mock_search_client.return_value.set_index_settings.assert_any_call(utils.ALGOLIA_INDEX_SETTINGS) - mock_search_client.return_value.set_index_settings.assert_called_with( + with override_settings(ALGOLIA={'REPLICA_INDEX_NAME': 'enterprise_catalog_duration_desc'}): + utils.configure_algolia_index(algolia_client) + set_index_settings = mock_search_client.return_value.set_index_settings + # The primary index is declared with the dynamically-built replicas list (the single + # configured base replica here), not the bare ALGOLIA_INDEX_SETTINGS. + set_index_settings.assert_any_call({ + **utils.ALGOLIA_INDEX_SETTINGS, + 'replicas': ['virtual(enterprise_catalog_duration_desc)'], + }) + set_index_settings.assert_any_call( utils.ALGOLIA_REPLICA_INDEX_SETTINGS, - primary_index=False + index_name='enterprise_catalog_duration_desc', ) + # Only the base replica is configured here (no optional replica name set), so the primary + # plus the single base replica are the only two set_index_settings calls. + assert set_index_settings.call_count == 2 + + @mock.patch('enterprise_catalog.apps.catalog.algolia_utils.AlgoliaSearchClient') + def test_configure_algolia_index_configures_additional_replica(self, mock_search_client): + """ + Each entry in ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS has its settings applied, keyed by index name. + """ + algolia_client = utils.get_initialized_algolia_client() + replica_name = 'enterprise_catalog_recently_released_desc' + recency_settings = {'customRanking': ['desc(recently_released_timestamp)']} + with override_settings(ALGOLIA={ + 'ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS': {replica_name: recency_settings}, + }): + utils.configure_algolia_index(algolia_client) + mock_search_client.return_value.set_index_settings.assert_any_call( + recency_settings, + index_name=replica_name, + ) + + @mock.patch('enterprise_catalog.apps.catalog.algolia_utils.AlgoliaSearchClient') + def test_configure_algolia_index_replica_failure_is_safe(self, mock_search_client): + """ + One replica failing to configure must not abort the reindex: the primary and the other + replicas are still configured, the error is logged, and nothing propagates. + """ + algolia_client = utils.get_initialized_algolia_client() + base_name = 'enterprise_catalog_duration_desc' + recency_name = 'enterprise_catalog_recently_released_desc' + recency_settings = {'customRanking': ['desc(recently_released_timestamp)']} + + def fail_only_for_recency(index_settings, index_name=None): # pylint: disable=unused-argument + # Primary and base-replica calls succeed; only the recency replica raises. + if index_name == recency_name: + raise AlgoliaException('boom') + + mock_search_client.return_value.set_index_settings.side_effect = fail_only_for_recency + with override_settings(ALGOLIA={ + 'REPLICA_INDEX_NAME': base_name, + 'ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS': {recency_name: recency_settings}, + }): + # The per-replica handler logs a single-line WARNING (set_index_settings already logged + # the traceback before re-raising), so we don't double up stack traces. + with self.assertLogs(utils.LOGGER, level='WARNING') as warning_logs: + # Does not raise, even though the recency replica fails. + utils.configure_algolia_index(algolia_client) + set_index_settings = mock_search_client.return_value.set_index_settings + # The primary (relevance) index is declared with both replicas, and the base replica + # was still configured. + set_index_settings.assert_any_call({ + **utils.ALGOLIA_INDEX_SETTINGS, + 'replicas': [f'virtual({base_name})', f'virtual({recency_name})'], + }) + set_index_settings.assert_any_call(utils.ALGOLIA_REPLICA_INDEX_SETTINGS, index_name=base_name) + # The recency replica was attempted (and failed) -- logged, not swallowed silently. + set_index_settings.assert_any_call(recency_settings, index_name=recency_name) + assert any(recency_name in message for message in warning_logs.output) + + @mock.patch('enterprise_catalog.apps.catalog.algolia_utils.AlgoliaSearchClient') + def test_configure_algolia_index_raises_when_primary_uninitialized(self, mock_search_client): + """ + If the primary index was never initialized (e.g. init_index() bailed on a missing + INDEX_NAME), configuring fails fast instead of silently applying no settings. + """ + algolia_client = utils.get_initialized_algolia_client() + algolia_client.algolia_index = None # simulate init_index() bailing on missing config + with self.assertRaises(ImproperlyConfigured): + utils.configure_algolia_index(algolia_client) + # Nothing was configured -- we refuse rather than report a no-op reindex as successful. + mock_search_client.return_value.set_index_settings.assert_not_called() @ddt.data( ( diff --git a/enterprise_catalog/apps/search/management/commands/incremental_reindex_algolia.py b/enterprise_catalog/apps/search/management/commands/incremental_reindex_algolia.py index 5692039c..9308a69b 100644 --- a/enterprise_catalog/apps/search/management/commands/incremental_reindex_algolia.py +++ b/enterprise_catalog/apps/search/management/commands/incremental_reindex_algolia.py @@ -117,13 +117,29 @@ def handle(self, *args, **options): else: self.stdout.write('Configuring Algolia index settings...') replica_name = replica_index_name or f'{index_name}_repl' + # TODO (ENT-11982): this block is a temporary workaround and should be refactored before + # this command runs in production (the incremental-reindexing cutover) on two fronts: + # 1. It configures only the single base ("by duration") replica -- it does NOT loop the + # unified sort-replica registry the way configure_algolia_index() does (via + # _get_algolia_replica_names() / _configured_replicas()), so the "newest courses + # first" sort and any future ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS'] + # replicas are not configured by this path. + # 2. It re-plumbs AlgoliaSearchClient internals (assigning _client / algolia_index + # directly below) and spins up a second client via new_search_client_or_error() -- + # both are hacks that work around AlgoliaSearchClient.init_index() not accepting an + # arbitrary index name. The real fix is to parameterize init_index() so this command + # can target a non-primary index without reaching into the client's internals. sdk_client = new_search_client_or_error() algolia_client = AlgoliaSearchClient() + # Target a non-primary index: wire the SDK client and the primary handle directly so + # set_index_settings() resolves to this alternate index. The replica is configured by + # name, which goes through _get_index() -> self._client.init_index(replica_name), so + # the underlying client must be set (init_index() would target the production index). + algolia_client._client = sdk_client # pylint: disable=protected-access algolia_client.algolia_index = sdk_client.init_index(index_name) - algolia_client.replica_index = sdk_client.init_index(replica_name) primary_settings = {**ALGOLIA_INDEX_SETTINGS, 'replicas': [f'virtual({replica_name})']} algolia_client.set_index_settings(primary_settings) - algolia_client.set_index_settings(ALGOLIA_REPLICA_INDEX_SETTINGS, primary_index=False) + algolia_client.set_index_settings(ALGOLIA_REPLICA_INDEX_SETTINGS, index_name=replica_name) self.stdout.write(f'Content types: {", ".join(content_types) if content_types else "all"}') self.stdout.write(f'Target index: {resolved_index or "(not configured)"}') diff --git a/enterprise_catalog/apps/search/management/commands/tests/test_incremental_reindex_algolia.py b/enterprise_catalog/apps/search/management/commands/tests/test_incremental_reindex_algolia.py index 12ae9043..995dcad1 100644 --- a/enterprise_catalog/apps/search/management/commands/tests/test_incremental_reindex_algolia.py +++ b/enterprise_catalog/apps/search/management/commands/tests/test_incremental_reindex_algolia.py @@ -193,16 +193,16 @@ def test_configure_index_called_before_dispatch(self, mock_task): self._call('--index-name', 'enterprise_catalog_v2') sdk = self.mock_new_sdk_client.return_value algolia_instance = self.mock_algolia_cls.return_value - # Both primary and replica indices are initialized + # The primary handle is initialized directly; the replica is configured by name via + # set_index_settings(index_name=...), which initializes it lazily inside the client. sdk.init_index.assert_any_call('enterprise_catalog_v2') - sdk.init_index.assert_any_call('enterprise_catalog_v2_repl') # set_index_settings called twice: primary (with replicas overridden) then replica assert algolia_instance.set_index_settings.call_count == 2 primary_call_kwargs = algolia_instance.set_index_settings.call_args_list[0] primary_settings = primary_call_kwargs[0][0] assert primary_settings['replicas'] == ['virtual(enterprise_catalog_v2_repl)'] replica_call = algolia_instance.set_index_settings.call_args_list[1] - assert replica_call[1].get('primary_index') is False + assert replica_call[1].get('index_name') == 'enterprise_catalog_v2_repl' @mock.patch(TASK_PATH) def test_explicit_replica_name_used(self, mock_task): @@ -210,10 +210,12 @@ def test_explicit_replica_name_used(self, mock_task): self._call('--index-name', 'enterprise_catalog_v2', '--replica-name', 'my_replica') sdk = self.mock_new_sdk_client.return_value sdk.init_index.assert_any_call('enterprise_catalog_v2') - sdk.init_index.assert_any_call('my_replica') algolia_instance = self.mock_algolia_cls.return_value primary_settings = algolia_instance.set_index_settings.call_args_list[0][0][0] assert primary_settings['replicas'] == ['virtual(my_replica)'] + # The replica is configured by name (which initializes it lazily inside the client). + replica_call = algolia_instance.set_index_settings.call_args_list[1] + assert replica_call[1].get('index_name') == 'my_replica' @mock.patch(TASK_PATH) def test_configure_index_skipped_on_dry_run(self, mock_task): diff --git a/enterprise_catalog/settings/base.py b/enterprise_catalog/settings/base.py index 02aba971..6bafcf3b 100644 --- a/enterprise_catalog/settings/base.py +++ b/enterprise_catalog/settings/base.py @@ -432,9 +432,43 @@ STUDIO_BASE_URL = os.environ.get('STUDIO_BASE_URL', '') # Algolia +# The per-environment index names and credentials (INDEX_NAME, REPLICA_INDEX_NAME, APPLICATION_ID, +# API_KEY) are populated from the deployment config (edx-internal). ALGOLIA is listed in +# DICT_UPDATE_KEYS (see settings/production.py), so the deployment YAML is *merged* into this dict +# rather than replacing it -- per-environment keys override the defaults below while code-defined +# defaults like ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS are preserved. + +# customRanking for the "newest courses first" sort replica. Defined here as config-as-code and +# referenced from ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS'] below. Leads with the precomputed +# recently_released_timestamp (a course's earliest course-run start, any status) descending, then +# the primary index's relevance tie-breakers. +ALGOLIA_RECENTLY_RELEASED_REPLICA_INDEX_SETTINGS = { + 'customRanking': [ + 'desc(recently_released_timestamp)', + 'asc(metadata_language)', + 'asc(visible_via_association)', + 'asc(created)', + 'desc(course_bayesian_average)', + 'desc(recent_enrollment_count)', + ], +} + ALGOLIA = { 'INDEX_NAME': '', + # Base replica: a long-standing "by duration" sort (its customRanking leads with desc(duration), + # defined in ALGOLIA_REPLICA_INDEX_SETTINGS in apps/catalog/algolia_utils.py) that the + # learner-portal MFE points its video search (SEARCH_INDEX_IDS.VIDEOS) at, via the MFE's + # ALGOLIA_REPLICA_INDEX_NAME env var. 'REPLICA_INDEX_NAME': '', + # Additional sort replicas beyond the base one, mapped index_name -> Algolia index settings. + # This is the single source of truth for which extra replicas exist: each entry is declared on + # the primary index, has its settings applied during a reindex, and is added to the secured API + # key. The learner-portal only points its course at the "newest first" replica when the + # enterprise.search_default_sort_newest waffle flag + Optimizely experiment are on, so the flag + # -- not the mere presence of an entry here -- gates user-facing exposure. + 'ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS': { + 'enterprise_catalog_recently_released_desc': ALGOLIA_RECENTLY_RELEASED_REPLICA_INDEX_SETTINGS, + }, 'APPLICATION_ID': '', 'API_KEY': '', } diff --git a/enterprise_catalog/settings/production.py b/enterprise_catalog/settings/production.py index d5067751..f182bd56 100644 --- a/enterprise_catalog/settings/production.py +++ b/enterprise_catalog/settings/production.py @@ -21,7 +21,10 @@ # Keep track of the names of settings that represent dicts. Instead of overriding the values in base.py, # the values read from disk should UPDATE the pre-configured dicts. -DICT_UPDATE_KEYS = ('JWT_AUTH', 'REST_FRAMEWORK') +# ALGOLIA is merged (not replaced) so the deployment YAML can set per-environment index names and +# credentials while code-defined defaults such as ALGOLIA['ADDITIONAL_VIRTUAL_REPLICA_INDEX_SETTINGS'] +# (the sort-replica customRanking definitions) are preserved. +DICT_UPDATE_KEYS = ('JWT_AUTH', 'REST_FRAMEWORK', 'ALGOLIA') # This may be overridden by the YAML in catalog_CFG, # but it should be here as a default.