refactor: avoid rebuilding script_pubkeys for BIP158 filter matching#781
Open
xdustinface wants to merge 1 commit into
Open
refactor: avoid rebuilding script_pubkeys for BIP158 filter matching#781xdustinface wants to merge 1 commit into
script_pubkeys for BIP158 filter matching#781xdustinface wants to merge 1 commit into
Conversation
`check_compact_filters_for_addresses` ran `address.script_pubkey()` per address per filter batch, pushing OP_DUP / OP_HASH160 / hash / OP_EQUALVERIFY / OP_CHECKSIG into a fresh `ScriptBuf` every call. A dhat profile of a full testnet sync showed `ScriptBuf::new_p2pkh` was hit ~7 million times for ~172 MB of lifetime bytes, dominated by this path. `AddressInfo` already stores `script_pubkey: ScriptBuf` from address generation. Read from that cache instead of reconstructing: - `AddressPool`, `ManagedAccountType`, `ManagedAccountRef`, and `ManagedAccountTrait` expose `all_script_pubkeys` alongside `all_addresses`. - `WalletInfoInterface` and `WalletInterface` expose `monitored_script_pubkeys` and `monitored_script_pubkeys_for`. The parallel `Vec<Address>` accessor is dropped, no remaining caller. - `BlockProcessingResult.new_scripts` and `SyncEvent::BlockProcessed.new_scripts` carry `Vec<ScriptBuf>` so the rescan path on `FiltersBatch` consumes cached scripts directly. - `check_compact_filters_for_addresses` becomes `check_compact_filters_for_script_pubkeys`, taking `&[ScriptBuf]` and handing the inner bytes to `BlockFilter::match_any`.
Contributor
📝 WalkthroughWalkthroughThis PR refactors wallet monitoring across the SPV sync pipeline from address-based to script-pubkey-based tracking. The ChangesScript-Based Wallet Monitoring Pipeline
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
key-wallet-manager/src/matching.rs (1)
34-38: ⚡ Quick winShort-circuit the empty-script case.
When
script_pubkeysis empty, this still scans every filter and callsmatch_anyon each one. An early return keeps the no-op path O(1), which fits the perf goal of this refactor.Suggested change
pub fn check_compact_filters_for_script_pubkeys( input: &HashMap<FilterMatchKey, BlockFilter>, script_pubkeys: &[ScriptBuf], min_height: CoreBlockHeight, ) -> BTreeSet<FilterMatchKey> { + if script_pubkeys.is_empty() { + return BTreeSet::new(); + } + let match_filter = |(key, filter): (&FilterMatchKey, &BlockFilter)| { if key.height() <= min_height { return None; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@key-wallet-manager/src/matching.rs` around lines 34 - 38, The function check_compact_filters_for_script_pubkeys currently iterates all filters even when script_pubkeys is empty; add an early return at the start of check_compact_filters_for_script_pubkeys that returns an empty BTreeSet when script_pubkeys.is_empty() to avoid scanning input and calling BlockFilter::match_any unnecessarily (references: check_compact_filters_for_script_pubkeys, FilterMatchKey, BlockFilter, match_any).key-wallet-manager/src/lib.rs (1)
24-24: ⚡ Quick winConsider keeping a deprecated alias for the old public helper.
This re-export drops a previously public symbol, so downstream callers will stop compiling even though the underlying change is mostly representational. If this is not meant to be a semver-major break, keep
check_compact_filters_for_addressesas a deprecated shim for one release and forward it to the new script-based helper.Compatibility-oriented shape
-pub use matching::{check_compact_filters_for_script_pubkeys, FilterMatchKey}; +pub use matching::{ + check_compact_filters_for_addresses, + check_compact_filters_for_script_pubkeys, + FilterMatchKey, +};#[deprecated(note = "use check_compact_filters_for_script_pubkeys instead")] pub fn check_compact_filters_for_addresses( input: &HashMap<FilterMatchKey, BlockFilter>, addresses: Vec<Address>, min_height: CoreBlockHeight, ) -> BTreeSet<FilterMatchKey> { let script_pubkeys: Vec<ScriptBuf> = addresses.into_iter().map(|address| address.script_pubkey()).collect(); check_compact_filters_for_script_pubkeys(input, &script_pubkeys, min_height) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@key-wallet-manager/src/lib.rs` at line 24, The pub re-export removed the old helper check_compact_filters_for_addresses which will break downstream code; add a deprecated, publicly exported shim named check_compact_filters_for_addresses that keeps the original signature (taking &HashMap<FilterMatchKey, BlockFilter>, Vec<Address>, CoreBlockHeight) and simply maps the addresses to their script_pubkey() values (collecting into Vec<ScriptBuf>) then calls and returns check_compact_filters_for_script_pubkeys(input, &script_pubkeys, min_height); annotate the shim with #[deprecated(note = "use check_compact_filters_for_script_pubkeys instead")] so callers get a deprecation warning but remain compatible.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@key-wallet-manager/src/lib.rs`:
- Line 24: The pub re-export removed the old helper
check_compact_filters_for_addresses which will break downstream code; add a
deprecated, publicly exported shim named check_compact_filters_for_addresses
that keeps the original signature (taking &HashMap<FilterMatchKey, BlockFilter>,
Vec<Address>, CoreBlockHeight) and simply maps the addresses to their
script_pubkey() values (collecting into Vec<ScriptBuf>) then calls and returns
check_compact_filters_for_script_pubkeys(input, &script_pubkeys, min_height);
annotate the shim with #[deprecated(note = "use
check_compact_filters_for_script_pubkeys instead")] so callers get a deprecation
warning but remain compatible.
In `@key-wallet-manager/src/matching.rs`:
- Around line 34-38: The function check_compact_filters_for_script_pubkeys
currently iterates all filters even when script_pubkeys is empty; add an early
return at the start of check_compact_filters_for_script_pubkeys that returns an
empty BTreeSet when script_pubkeys.is_empty() to avoid scanning input and
calling BlockFilter::match_any unnecessarily (references:
check_compact_filters_for_script_pubkeys, FilterMatchKey, BlockFilter,
match_any).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 4fb404b0-e099-4395-8887-3973d192ff5c
📒 Files selected for processing (19)
dash-spv-ffi/src/callbacks.rsdash-spv/src/sync/blocks/manager.rsdash-spv/src/sync/events.rsdash-spv/src/sync/filters/batch.rsdash-spv/src/sync/filters/manager.rsdash-spv/src/sync/filters/sync_manager.rsdash-spv/src/sync/mempool/sync_manager.rsdash-spv/tests/dashd_sync/helpers.rskey-wallet-manager/src/lib.rskey-wallet-manager/src/matching.rskey-wallet-manager/src/process_block.rskey-wallet-manager/src/test_utils/mock_wallet.rskey-wallet-manager/src/wallet_interface.rskey-wallet-manager/tests/spv_integration_tests.rskey-wallet/src/managed_account/address_pool.rskey-wallet/src/managed_account/managed_account_ref.rskey-wallet/src/managed_account/managed_account_trait.rskey-wallet/src/managed_account/managed_account_type.rskey-wallet/src/wallet/managed_wallet_info/wallet_info_interface.rs
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #781 +/- ##
==========================================
- Coverage 72.69% 72.19% -0.50%
==========================================
Files 321 321
Lines 70350 70396 +46
==========================================
- Hits 51140 50822 -318
- Misses 19210 19574 +364
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
check_compact_filters_for_addressesranaddress.script_pubkey()per address per filter batch, pushing OP_DUP / OP_HASH160 / hash / OP_EQUALVERIFY / OP_CHECKSIG into a freshScriptBufevery call. A dhat profile of a full testnet sync showedScriptBuf::new_p2pkhwas hit ~7 million times for ~172 MB of lifetime bytes, dominated by this path.AddressInfoalready storesscript_pubkey: ScriptBuffrom address generation. Read from that cache instead of reconstructing:AddressPool,ManagedAccountType,ManagedAccountRef, andManagedAccountTraitexposeall_script_pubkeysalongsideall_addresses.WalletInfoInterfaceandWalletInterfaceexposemonitored_script_pubkeysandmonitored_script_pubkeys_for. The parallelVec<Address>accessor is dropped, no remaining caller.BlockProcessingResult.new_scriptsandSyncEvent::BlockProcessed.new_scriptscarryVec<ScriptBuf>so the rescan path onFiltersBatchconsumes cached scripts directly.check_compact_filters_for_addressesbecomescheck_compact_filters_for_script_pubkeys, taking&[ScriptBuf]and handing the inner bytes toBlockFilter::match_any.Summary by CodeRabbit
Release Notes