Skip to content

Apply @r-pkg-optimizer skill to stretch out some performance improvements - #3111

Open
MichaelChirico wants to merge 1 commit into
mainfrom
r-pkg-optimizer
Open

Apply @r-pkg-optimizer skill to stretch out some performance improvements#3111
MichaelChirico wants to merge 1 commit into
mainfrom
r-pkg-optimizer

Conversation

@MichaelChirico

Copy link
Copy Markdown
Collaborator

cc @dshkol - https://github.com/dshkol/rpgkoptskill

Optimizer output:


This PR optimizes performance and memory usage across lintr's core execution pipeline without modifying public APIs, S3 classes, or linter semantics:

  1. Lazy Function Call Caching in build_xml_find_function_calls() (R/source_utils.R):

    • Defers XPath evaluation of //SYMBOL_FUNCTION_CALL/parent::* and //SLOT/parent::expr[...] so it only runs on demand when xml_find_function_calls() is queried, avoiding eager evaluations on AST subtrees that never use function-call indexing.
    • Replaces <<- with an encapsulated environment (cache_env) to adhere to package style guidelines.
  2. Pre-partitioned Parse Data Subsetting in get_source_expressions() (R/get_source_expressions.R):

    • Replaces repeated $O(N \times M)$ linear scans (parsed_content[which(top_level_map == id), ]) with a single-pass index grouping via split(seq_len(nrow(parsed_content)), factor(top_level_map, levels = tl_ids)).
  3. Package Namespace & S3 Generic Memoization (R/namespace.R):

    • Adds .namespace_cache to memoize results for namespace_imports(), imported_s3_generics(), and exported_s3_generics().
    • Keys cache entries by package path and NAMESPACE file modification timestamp (file.mtime()), eliminating hundreds of redundant file re-reads and namespace scans during multi-file linting while preserving real-time invalidation when files change.
  4. Short-Circuit Empty Nodesets in strip_comments_from_subtree() (R/xml_utils.R):

    • Adds if (length(expr) == 0L) return(expr) to avoid executing xml_find_first_(expr, ".//COMMENT") on empty nodesets across 12 linters.
  5. Streamlined Linter Orchestration in lint_impl_() (R/lint.R):

    • Pre-partitions active linters into expression-level and file-level subsets once per file to avoid redundant per-expression necessary_linters() overhead, and avoids accumulating 0-length lint lists.

Benchmarks & Performance Results

1. Microbenchmarks & Core Pipeline (bench::mark)

Workload Metric Baseline Optimized Relative Change
get_source_expressions("R/lint.R") Median Latency 147.0 ms 112.0 ms -23.8% (-35 ms)
Memory Allocations 26.6 MB 20.3 MB -23.7% (-6.3 MB)
Throughput 6.17 itr/s 7.67 itr/s +24.3%
Garbage Collections 31 24 -22.6%
lint("R/lint.R") Median Latency 1.05 s 0.965 s -8.1% (-85 ms)
Memory Allocations 33.1 MB 26.5 MB -20.0% (-6.6 MB)
Throughput 0.914 itr/s 0.976 itr/s +6.8%
Garbage Collections 200 178 -11.0%
S3 Generics Resolution (152 files) Wall-Clock Time 5.66 s 1.77 s -68.7% (3.2x faster)

2. End-to-End Codebase Linting: lint_dir("R") (152 files)

Metric Baseline Optimized Difference
Elapsed Wall-Clock Time 34.55 s 27.74 s -19.7% (-6.81 s)
Lints Found 1,348 1,348 Exact Match

3. Evaluation on Heavy & Massive R Source Files

Workload Lines / Size Expressions get_source_expressions() lint() Total Time Lints Found
QC.R (tools) 10,654 lines / 421 KB 411 1.63 s 13.62 s 5,551
tests.Rraw (data.table) 21,987 lines / 1.26 MB 15,344 16.75 s (1.09 ms/expr) 444.10 s 47,353

@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.73%. Comparing base (c3c9805) to head (024974a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##              main    #3111      +/-   ##
===========================================
- Coverage   100.00%   99.73%   -0.27%     
===========================================
  Files          129      129              
  Lines         7468     7533      +65     
===========================================
+ Hits          7468     7513      +45     
- Misses           0       20      +20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant