Skip to content

feat(observer): per-RPC observer groups with global fallback#62

Open
qk-santi wants to merge 2 commits into
0xPolygon:mainfrom
qk-santi:feat/observer-groups
Open

feat(observer): per-RPC observer groups with global fallback#62
qk-santi wants to merge 2 commits into
0xPolygon:mainfrom
qk-santi:feat/observer-groups

Conversation

@qk-santi
Copy link
Copy Markdown
Contributor

@qk-santi qk-santi commented Apr 14, 2026

Summary

Add support for overriding the observer set on a per-RPC-provider basis.
Each provider can now define its own observers group in config; providers without one fall back to the global observers.rpc set.
Non-RPC providers (exchange_rates, system, hash_divergence, heimdall, sensor, etc.) use a separate observers.system set on their own EventBus.

Changes

  • Add GlobalObservers{RPC, System} config struct; observers.rpc becomes the fallback for all RPC providers, observers.system for non-RPC
  • Add optional observers field on RPC provider config; when set, a dedicated EventBus is created for that provider
  • Rename GetEnabledObserverSetGetObserverSetFrom(cfg Observers) to accept an explicit config instead of reading global state
  • Replace promauto with manual prometheus.Register + registerOrExisting helper to prevent duplicate-registration panics on repeated Init calls
  • Fix Publish to only warn on empty subscriber sets for topics that were explicitly subscribed (avoids false warnings for unregistered topics)

Config example

observers:
  rpc:
    enabled:
      - "block"
      - "transaction_count"
  system:
    enabled:
      - "exchange_rates"

x-observer-groups:
  light: &light
    enabled:
      - "block"
      - "finalized_height"

providers:
  rpc:
    - name: "Ethereum"
      url: "https://..."
      # Uses observers.rpc fallback

    - name: "Ethereum"
      url: "https://..."
      observers:
        enabled:
          - "block"  # lightweight — only block tracking

    - name: "Ethereum"
      url: "https://..."
      observers: &light  # lightweight — only block tracking (using yaml anchor)

  Add support for overriding the observer set on a per-RPC-provider basis.
  Each provider can now define its own observers group in config; providers
  without one fall back to the global observers.rpc set. Non-RPC providers
  (exchange_rates, system, hash_divergence, etc.) use a separate
  observers.system set on their own EventBus.

  - Add GlobalObservers{RPC, System} config struct; observers.rpc becomes
    the fallback for all RPC providers, observers.system for non-RPC
  - Add optional observers field on RPC provider config; when set, a
    dedicated EventBus is created for that provider
  - Rename GetEnabledObserverSet → GetObserverSetFrom(cfg Observers) to
    accept an explicit config instead of reading global state
  - Replace promauto with manual Register + registerOrExisting helper to
    prevent duplicate-registration panics on repeated Init calls
  - Fix Publish to only warn on empty subscriber sets for topics that were
    explicitly subscribed (avoids false warnings for unregistered topics)
Comment thread runner/runner.go
}

p := provider.NewHeimdallProvider(n, eb, h)
p := provider.NewHeimdallProvider(n, systemEB, h)
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're leveraging here the systemEB as it's a singleton, even if it's chain related
We could create another event bus for chain-related stuff, but might be too much?

This affects:

  • Heimdall
  • SensorNetwork
  • ProverNetwork
  • Aggchain

Comment thread config.yml
## @param rpc.disabled - list of strings - optional
## @env PANOPTICHAIN_OBSERVERS_RPC_DISABLED - list of strings - optional
#
# rpc:
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new nesting level forces this to be a breaking change

@minhd-vu
Copy link
Copy Markdown
Collaborator

Hey @qk-santi 👋

I've been looking at this PR and wanted to suggest an alternative approach that might avoid the breaking config change. The core idea:

Keep the existing observers config unchanged, only add per-provider override capability.

Instead of changing observers.enabledobservers.rpc.enabled, we could:

  1. Keep global observers config as-is (backward compatible)
  2. Add optional observers field to provider config (which you already have)
  3. Provider without observers → uses global GetEnabledObserverSet()
  4. Provider with observers → gets its own EventBus with custom observer set

For observer groups, YAML anchors work out of the box with Viper:

# Global default (unchanged from current schema)
observers:
  enabled:
    - block
    - transaction_count
    - finalized_height

# Define reusable groups with YAML anchors
x-observer-groups:
  light: &light
    enabled:
      - block
      - finalized_height

providers:
  rpc:
    - name: "Polygon Mainnet"
      url: "https://internal-rpc"
      label: "internal"
      # No override = uses global observers

    - name: "Polygon Mainnet"
      url: "https://external-rpc"
      label: "external"
      observers: *light  # Uses the 'light' group

I verified that Viper correctly resolves YAML anchors - the *light reference becomes {enabled: [block, finalized_height]} at parse time.

This achieves the main goal (per-provider observers) without changing the config schema for existing users. The RPC/system split could be added later as an opt-in feature if needed.

What do you think?

@sonarqubecloud
Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants