perf: "two-pass" seurat hvg via scanpy.get.aggregate
#4013
+114
−35
scverse-benchmark / benchmark
failed
May 5, 2026 in 1h 6m 10s
Benchmark
Benchmark run successful
Details
All benchmarks:
| Change | Before [45325b1] | After [cf65665] | Ratio | Benchmark (Parameter) |
|---|---|---|---|---|
| 5.2G | 5.2G | 1.00 | preprocessing_counts.Agg.peakmem_agg('count_nonzero') | |
| 4.11G | 4.11G | 1.00 | preprocessing_counts.Agg.peakmem_agg('mean') | |
| 4.47G | 4.47G | 1.00 | preprocessing_counts.Agg.peakmem_agg('median') | |
| 4.11G | 4.11G | 1.00 | preprocessing_counts.Agg.peakmem_agg('sum') | |
| 4.12G | 4.12G | 1.00 | preprocessing_counts.Agg.peakmem_agg('var') | |
| 643±4ms | 647±3ms | 1.01 | preprocessing_counts.Agg.time_agg('count_nonzero') | |
| 90.6±1ms | 91.1±0.6ms | 1.01 | preprocessing_counts.Agg.time_agg('mean') | |
| 3.27±0.01s | 3.25±0.01s | 0.99 | preprocessing_counts.Agg.time_agg('median') | |
| 88.3±0.3ms | 89.0±0.5ms | 1.01 | preprocessing_counts.Agg.time_agg('sum') | |
| 134±2ms | 130±3ms | 0.98 | preprocessing_counts.Agg.time_agg('var') | |
| 374M | 373M | 1.00 | preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('bmmc', 'counts') | |
| 373M | 374M | 1.00 | preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('bmmc', 'counts-off-axis') | |
| 4.17G | 4.17G | 1.00 | preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('lung93k', 'counts') | |
| 4.17G | 4.17G | 1.00 | preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('lung93k', 'counts-off-axis') | |
| 440M | 441M | 1.00 | preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc3k', 'counts') | |
| 439M | 440M | 1.00 | preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc3k', 'counts-off-axis') | |
| 351M | 351M | 1.00 | preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc68k_reduced', 'counts') | |
| 351M | 351M | 1.00 | preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc68k_reduced', 'counts-off-axis') | |
| 376M | 376M | 1.00 | preprocessing_counts.FastSuite.peakmem_log1p('bmmc', 'counts') | |
| 376M | 375M | 1.00 | preprocessing_counts.FastSuite.peakmem_log1p('bmmc', 'counts-off-axis') | |
| 4.51G | 4.51G | 1.00 | preprocessing_counts.FastSuite.peakmem_log1p('lung93k', 'counts') | |
| 4.51G | 4.51G | 1.00 | preprocessing_counts.FastSuite.peakmem_log1p('lung93k', 'counts-off-axis') | |
| 446M | 446M | 1.00 | preprocessing_counts.FastSuite.peakmem_log1p('pbmc3k', 'counts') | |
| 447M | 447M | 1.00 | preprocessing_counts.FastSuite.peakmem_log1p('pbmc3k', 'counts-off-axis') | |
| 350M | 350M | 1.00 | preprocessing_counts.FastSuite.peakmem_log1p('pbmc68k_reduced', 'counts') | |
| 351M | 350M | 1.00 | preprocessing_counts.FastSuite.peakmem_log1p('pbmc68k_reduced', 'counts-off-axis') | |
| 461M | 462M | 1.00 | preprocessing_counts.FastSuite.peakmem_normalize_total('bmmc', 'counts') | |
| 460M | 460M | 1.00 | preprocessing_counts.FastSuite.peakmem_normalize_total('bmmc', 'counts-off-axis') | |
| 5.03G | 5.02G | 1.00 | preprocessing_counts.FastSuite.peakmem_normalize_total('lung93k', 'counts') | |
| 5.03G | 5.02G | 1.00 | preprocessing_counts.FastSuite.peakmem_normalize_total('lung93k', 'counts-off-axis') | |
| 534M | 531M | 1.00 | preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc3k', 'counts') | |
| 531M | 533M | 1.00 | preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc3k', 'counts-off-axis') | |
| 350M | 350M | 1.00 | preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc68k_reduced', 'counts') | |
| 350M | 350M | 1.00 | preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc68k_reduced', 'counts-off-axis') | |
| 12.8±0.2ms | 12.6±0.3ms | 0.99 | preprocessing_counts.FastSuite.time_calculate_qc_metrics('bmmc', 'counts') | |
| 12.5±0.1ms | 12.6±0.2ms | 1.01 | preprocessing_counts.FastSuite.time_calculate_qc_metrics('bmmc', 'counts-off-axis') | |
| 2.07±0s | 2.05±0.01s | 0.99 | preprocessing_counts.FastSuite.time_calculate_qc_metrics('lung93k', 'counts') | |
| 1.64±0.01s | 1.63±0.01s | 1.00 | preprocessing_counts.FastSuite.time_calculate_qc_metrics('lung93k', 'counts-off-axis') | |
| 38.6±0.8ms | 38.3±0.2ms | 0.99 | preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc3k', 'counts') | |
| 28.6±1ms | 27.8±0.6ms | 0.97 | preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc3k', 'counts-off-axis') | |
| 4.79±0.04ms | 4.64±0.02ms | 0.97 | preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc68k_reduced', 'counts') | |
| 4.73±0.06ms | 4.70±0.07ms | 1.00 | preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc68k_reduced', 'counts-off-axis') | |
| 1.81±0.2ms | 1.54±0.03ms | ~0.85 | preprocessing_counts.FastSuite.time_log1p('bmmc', 'counts') | |
| 1.61±0.03ms | 1.59±0.02ms | 0.99 | preprocessing_counts.FastSuite.time_log1p('bmmc', 'counts-off-axis') | |
| 641±2ms | 637±2ms | 0.99 | preprocessing_counts.FastSuite.time_log1p('lung93k', 'counts') | |
| 641±1ms | 634±3ms | 0.99 | preprocessing_counts.FastSuite.time_log1p('lung93k', 'counts-off-axis') | |
| 7.14±0.06ms | 7.10±0.03ms | 0.99 | preprocessing_counts.FastSuite.time_log1p('pbmc3k', 'counts') | |
| 7.27±0.03ms | 7.28±0.1ms | 1.00 | preprocessing_counts.FastSuite.time_log1p('pbmc3k', 'counts-off-axis') | |
| 447±10μs | 458±60μs | 1.03 | preprocessing_counts.FastSuite.time_log1p('pbmc68k_reduced', 'counts') | |
| 394±10μs | 387±0.6μs | 0.98 | preprocessing_counts.FastSuite.time_log1p('pbmc68k_reduced', 'counts-off-axis') | |
| 2.63±0.06ms | 2.76±0.6ms | 1.05 | preprocessing_counts.FastSuite.time_normalize_total('bmmc', 'counts') | |
| 10.8±2ms | 9.31±1ms | ~0.86 | preprocessing_counts.FastSuite.time_normalize_total('bmmc', 'counts-off-axis') | |
| 545±10ms | 552±7ms | 1.01 | preprocessing_counts.FastSuite.time_normalize_total('lung93k', 'counts') | |
| 2.76±0.04s | 2.63±0.04s | 0.95 | preprocessing_counts.FastSuite.time_normalize_total('lung93k', 'counts-off-axis') | |
| 8.71±0.1ms | 8.87±2ms | 1.02 | preprocessing_counts.FastSuite.time_normalize_total('pbmc3k', 'counts') | |
| 49.1±3ms | 41.3±10ms | ~0.84 | preprocessing_counts.FastSuite.time_normalize_total('pbmc3k', 'counts-off-axis') | |
| 548±0.8μs | 545±2μs | 0.99 | preprocessing_counts.FastSuite.time_normalize_total('pbmc68k_reduced', 'counts') | |
| 548±6μs | 547±0.8μs | 1.00 | preprocessing_counts.FastSuite.time_normalize_total('pbmc68k_reduced', 'counts-off-axis') | |
| 458M | 458M | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.peakmem_downsample_per_cell('pbmc3k', 'random_state') | |
| 457M | 458M | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.peakmem_downsample_per_cell('pbmc3k', 'rng') | |
| 394M | 395M | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.peakmem_downsample_per_cell('pbmc68k_reduced', 'random_state') | |
| 395M | 395M | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.peakmem_downsample_per_cell('pbmc68k_reduced', 'rng') | |
| 527M | 527M | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.peakmem_downsample_total('pbmc3k', 'random_state') | |
| 493M | 493M | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.peakmem_downsample_total('pbmc3k', 'rng') | |
| 399M | 400M | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.peakmem_downsample_total('pbmc68k_reduced', 'random_state') | |
| 397M | 397M | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.peakmem_downsample_total('pbmc68k_reduced', 'rng') | |
| 207±0.9ms | 203±0.9ms | 0.98 | preprocessing_counts.PreprocessingCountsRngSuite.time_downsample_per_cell('pbmc3k', 'random_state') | |
| 72.3±0.8ms | 73.0±1ms | 1.01 | preprocessing_counts.PreprocessingCountsRngSuite.time_downsample_per_cell('pbmc3k', 'rng') | |
| 25.0±0.09ms | 25.5±0.08ms | 1.02 | preprocessing_counts.PreprocessingCountsRngSuite.time_downsample_per_cell('pbmc68k_reduced', 'random_state') | |
| 16.1±0.04ms | 16.2±0.1ms | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.time_downsample_per_cell('pbmc68k_reduced', 'rng') | |
| 189±2ms | 191±3ms | 1.01 | preprocessing_counts.PreprocessingCountsRngSuite.time_downsample_total('pbmc3k', 'random_state') | |
| 62.8±0.2ms | 62.9±0.9ms | 1.00 | preprocessing_counts.PreprocessingCountsRngSuite.time_downsample_total('pbmc3k', 'rng') | |
| 13.2±0.2ms | 13.6±1ms | 1.03 | preprocessing_counts.PreprocessingCountsRngSuite.time_downsample_total('pbmc68k_reduced', 'random_state') | |
| 6.52±0.4ms | 7.05±0.3ms | 1.08 | preprocessing_counts.PreprocessingCountsRngSuite.time_downsample_total('pbmc68k_reduced', 'rng') | |
| 493M | 493M | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_cells('pbmc3k', 'counts') | |
| 493M | 493M | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_cells('pbmc3k', 'counts-off-axis') | |
| 361M | 362M | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_cells('pbmc68k_reduced', 'counts') | |
| 362M | 361M | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_cells('pbmc68k_reduced', 'counts-off-axis') | |
| 493M | 493M | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_genes('pbmc3k', 'counts') | |
| 493M | 493M | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_genes('pbmc3k', 'counts-off-axis') | |
| 361M | 361M | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_genes('pbmc68k_reduced', 'counts') | |
| 361M | 361M | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_genes('pbmc68k_reduced', 'counts-off-axis') | |
| 1.17G | 1.16G | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_scrublet('pbmc3k', 'counts') | |
| 1.17G | 1.17G | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_scrublet('pbmc3k', 'counts-off-axis') | |
| 578M | 584M | 1.01 | preprocessing_counts.PreprocessingCountsSuite.peakmem_scrublet('pbmc68k_reduced', 'counts') | |
| 586M | 586M | 1.00 | preprocessing_counts.PreprocessingCountsSuite.peakmem_scrublet('pbmc68k_reduced', 'counts-off-axis') | |
| 56.1±0.9ms | 56.2±1ms | 1.00 | preprocessing_counts.PreprocessingCountsSuite.time_filter_cells('pbmc3k', 'counts') | |
| 58.9±0.7ms | 59.9±0.8ms | 1.02 | preprocessing_counts.PreprocessingCountsSuite.time_filter_cells('pbmc3k', 'counts-off-axis') | |
| 10.3±0.9ms | 10.2±0.9ms | 0.98 | preprocessing_counts.PreprocessingCountsSuite.time_filter_cells('pbmc68k_reduced', 'counts') | |
| 10.1±0.7ms | 9.76±0.7ms | 0.97 | preprocessing_counts.PreprocessingCountsSuite.time_filter_cells('pbmc68k_reduced', 'counts-off-axis') | |
| 52.5±0.6ms | 52.1±0.3ms | 0.99 | preprocessing_counts.PreprocessingCountsSuite.time_filter_genes('pbmc3k', 'counts') | |
| 52.8±2ms | 51.2±0.9ms | 0.97 | preprocessing_counts.PreprocessingCountsSuite.time_filter_genes('pbmc3k', 'counts-off-axis') | |
| 10.8±0.8ms | 10.5±1ms | 0.97 | preprocessing_counts.PreprocessingCountsSuite.time_filter_genes('pbmc68k_reduced', 'counts') | |
| 10.6±0.9ms | 10.6±0.8ms | 1.00 | preprocessing_counts.PreprocessingCountsSuite.time_filter_genes('pbmc68k_reduced', 'counts-off-axis') | |
| 2.54±0.03s | 2.44±0.09s | 0.96 | preprocessing_counts.PreprocessingCountsSuite.time_scrublet('pbmc3k', 'counts') | |
| 5.75±2s | 2.24±0.02s | ~0.39 | preprocessing_counts.PreprocessingCountsSuite.time_scrublet('pbmc3k', 'counts-off-axis') | |
| 550±10ms | 556±10ms | 1.01 | preprocessing_counts.PreprocessingCountsSuite.time_scrublet('pbmc68k_reduced', 'counts') | |
| 562±9ms | 558±7ms | 0.99 | preprocessing_counts.PreprocessingCountsSuite.time_scrublet('pbmc68k_reduced', 'counts-off-axis') | |
| 8.89G | 8.89G | 1.00 | preprocessing_log.HVGSuite.peakmem_highly_variable_genes('cell_ranger', False) | |
| failed | failed | n/a | preprocessing_log.HVGSuite.peakmem_highly_variable_genes('cell_ranger', True) | |
| 8.89G | 8.89G | 1.00 | preprocessing_log.HVGSuite.peakmem_highly_variable_genes('seurat', False) | |
| 3.33G | 3.3G | 0.99 | preprocessing_log.HVGSuite.peakmem_highly_variable_genes('seurat', True) | |
| 8.89G | 8.89G | 1.00 | preprocessing_log.HVGSuite.peakmem_highly_variable_genes('seurat_v3', False) | |
| 3.04G | 3.26G | 1.07 | preprocessing_log.HVGSuite.peakmem_highly_variable_genes('seurat_v3', True) | |
| 8.29±0.05s | 8.15±0s | 0.98 | preprocessing_log.HVGSuite.time_highly_variable_genes('cell_ranger', False) | |
| failed | failed | n/a | preprocessing_log.HVGSuite.time_highly_variable_genes('cell_ranger', True) | |
| 8.77±0.01s | 8.72±0.01s | 0.99 | preprocessing_log.HVGSuite.time_highly_variable_genes('seurat', False) | |
| ! | 22.7±0s | failed | n/a | preprocessing_log.HVGSuite.time_highly_variable_genes('seurat', True) |
| 1.09±0s | 1.26±0.06s | ~1.16 | preprocessing_log.HVGSuite.time_highly_variable_genes('seurat_v3', False) | |
| * | failed | 15.7±0.03s | n/a | preprocessing_log.HVGSuite.time_highly_variable_genes('seurat_v3', True) |
| 629M | 629M | 1.00 | preprocessing_log.PreprocessingSuite.peakmem_pca('pbmc3k', 'off-axis') | |
| 648M | 648M | 1.00 | preprocessing_log.PreprocessingSuite.peakmem_pca('pbmc3k', None) | |
| 556M | 552M | 0.99 | preprocessing_log.PreprocessingSuite.peakmem_pca('pbmc68k_reduced', 'off-axis') | |
| 556M | 554M | 1.00 | preprocessing_log.PreprocessingSuite.peakmem_pca('pbmc68k_reduced', None) | |
| n/a | n/a | n/a | preprocessing_log.PreprocessingSuite.peakmem_regress_out('pbmc3k', 'off-axis') | |
| n/a | n/a | n/a | preprocessing_log.PreprocessingSuite.peakmem_regress_out('pbmc3k', None) | |
| 407M | 408M | 1.00 | preprocessing_log.PreprocessingSuite.peakmem_regress_out('pbmc68k_reduced', 'off-axis') | |
| 411M | 414M | 1.01 | preprocessing_log.PreprocessingSuite.peakmem_regress_out('pbmc68k_reduced', None) | |
| 1.36G | 1.36G | 1.00 | preprocessing_log.PreprocessingSuite.peakmem_scale('pbmc3k', 'off-axis') | |
| 1.56G | 1.56G | 1.00 | preprocessing_log.PreprocessingSuite.peakmem_scale('pbmc3k', None) | |
| 396M | 395M | 1.00 | preprocessing_log.PreprocessingSuite.peakmem_scale('pbmc68k_reduced', 'off-axis') | |
| 398M | 398M | 1.00 | preprocessing_log.PreprocessingSuite.peakmem_scale('pbmc68k_reduced', None) | |
| 1.94±0.01s | 1.96±0.01s | 1.01 | preprocessing_log.PreprocessingSuite.time_pca('pbmc3k', 'off-axis') | |
| 2.15±0.01s | 2.15±0.01s | 1.00 | preprocessing_log.PreprocessingSuite.time_pca('pbmc3k', None) | |
| 108±50ms | 71.9±50ms | ~0.67 | preprocessing_log.PreprocessingSuite.time_pca('pbmc68k_reduced', 'off-axis') | |
| 70.8±60ms | 169±4ms | ~2.38 | preprocessing_log.PreprocessingSuite.time_pca('pbmc68k_reduced', None) | |
| n/a | n/a | n/a | preprocessing_log.PreprocessingSuite.time_regress_out('pbmc3k', 'off-axis') | |
| n/a | n/a | n/a | preprocessing_log.PreprocessingSuite.time_regress_out('pbmc3k', None) | |
| 17.1±0.4ms | 16.7±0.5ms | 0.98 | preprocessing_log.PreprocessingSuite.time_regress_out('pbmc68k_reduced', 'off-axis') | |
| 17.9±0.8ms | 16.8±0.5ms | 0.94 | preprocessing_log.PreprocessingSuite.time_regress_out('pbmc68k_reduced', None) | |
| 501±4ms | 504±4ms | 1.01 | preprocessing_log.PreprocessingSuite.time_scale('pbmc3k', 'off-axis') | |
| 545±2ms | 543±10ms | 1.00 | preprocessing_log.PreprocessingSuite.time_scale('pbmc3k', None) | |
| 4.72±0.08ms | 4.74±0.1ms | 1.00 | preprocessing_log.PreprocessingSuite.time_scale('pbmc68k_reduced', 'off-axis') | |
| 4.77±0.1ms | 4.76±0.09ms | 1.00 | preprocessing_log.PreprocessingSuite.time_scale('pbmc68k_reduced', None) | |
| 412M | 411M | 1.00 | tools.ToolsSuite.peakmem_combat | |
| 349M | 349M | 1.00 | tools.ToolsSuite.peakmem_diffmap | |
| 354M | 355M | 1.00 | tools.ToolsSuite.peakmem_leiden | |
| 437M | 437M | 1.00 | tools.ToolsSuite.peakmem_rank_genes_groups | |
| 516M | 519M | 1.01 | tools.ToolsSuite.peakmem_umap | |
| 49.5±0.7ms | 49.7±1ms | 1.00 | tools.ToolsSuite.time_combat | |
| 16.2±0.4ms | 16.7±0.4ms | 1.03 | tools.ToolsSuite.time_diffmap | |
| 20.0±0.09ms | 19.9±0.09ms | 1.00 | tools.ToolsSuite.time_leiden | |
| 79.0±0.9ms | 80.2±1ms | 1.02 | tools.ToolsSuite.time_rank_genes_groups | |
| 335±0.7ms | 331±0.4ms | 0.99 | tools.ToolsSuite.time_umap |
Loading