fix: partition update#89
Draft
ccm32004 wants to merge 6 commits into
Draft
Conversation
Signed-off-by: Cece Ma <mayuqing131@gmail.com>
Signed-off-by: Cece Ma <mayuqing131@gmail.com>
483b862 to
34032c8
Compare
Continuous benchmarkConfig: batch size = Compared to the latest result on
Result: No regression beyond the threshold vs |
Signed-off-by: Cece Ma <mayuqing131@gmail.com>
Code CoveragePercentage of source code lines executed by unit tests.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
updates numaflow-java, due to changes made in numaflow
Found issues outlined in the current getPartition implementation, thus some changes were made in how the getActivePartition method is implemented.
Why watermarks weren't updating properly, causing the window to not close properly
getActivePartitions()(or deprecated methodgetPartitions) returned all Pulsar topic partitions ([0..4]), making Numaflow spin up 5 watermark processors per replica.Why this fix works
defaultPartitions()— a singleton with this replica's NUMAFLOW_REPLICA index, so the watermark partition is the replica, not a Pulsar topic partition.Matches Numaflow's upstream Rust Pulsar source, which has always modeled things this way.
fixes #87