Skip to content

Support disabling code probes in test harness#13152

Open
tclinkenbeard-oai wants to merge 1 commit intoapple:mainfrom
tclinkenbeard-oai:dev/tclinkenbeard/disable-testharness2-code-probes
Open

Support disabling code probes in test harness#13152
tclinkenbeard-oai wants to merge 1 commit intoapple:mainfrom
tclinkenbeard-oai:dev/tclinkenbeard/disable-testharness2-code-probes

Conversation

@tclinkenbeard-oai
Copy link
Copy Markdown
Collaborator

For the FDB cluster backing Joshua, persisting code probe data is the most expensive part of running a Joshua ensemble. In some cases, this is unnecessary, so this PR supports disabling code probe tracking via a TH_DISABLE_CODE_PROBES environment variable on Joshua agents. Tested by running a Joshua ensemble with the environment variable set.

Copy link
Copy Markdown
Collaborator Author

@tclinkenbeard-oai tclinkenbeard-oai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generated by Codex.

What is it trying to do?

This adds a TH_DISABLE_CODE_PROBES / --disable-code-probes switch for TestHarness2 so Joshua agents can skip collecting per-test code coverage, avoid persisting coverage rows to FDB, and treat ensemble coverage checks as intentionally disabled when summarizing results.

Is it correct?

Yes, based on code inspection. The flag is wired through the relevant paths consistently:

  • summarize.py stops collecting CodeCoverage events and skips write_coverage(...)
  • results.py skips read_coverage(...), marks coverage_ok = True, and emits CodeProbeTrackingEnabled="0"
  • the README and Joshua wrapper comments document the new knob

The _read_env() change is also directionally correct: the old bool("false") == True behavior meant existing boolean environment variables could not reliably be set to false. The new parser fixes that for all bool-backed TestHarness2 env vars, not just this new one.

I did not run build or test validation for this review. GitHub currently reports no status checks on the PR branch.

Are there bugs?

I did not find any correctness bugs.

Are there omissions?

There are no automated tests for either the new disabled-code-probes path or the new shared boolean environment parsing behavior. Because _parse_env_value() changes semantics for every bool-backed env var, a small config-focused test covering accepted true/false spellings and invalid values would be useful, though I do not think its absence blocks this PR.

Are there better ways of doing things?

The shared boolean parser is better than adding one-off handling for only TH_DISABLE_CODE_PROBES; it fixes an existing class of env parsing mistakes at the right layer. I do not see a materially better design needed for this change.

Should this CL be LGTMd?

Yes, LGTM. I inspected the changed config, summarization, results, and documentation paths, and the disabled path appears internally consistent. The main remaining risk is the lack of automated coverage for the new boolean parsing behavior and disabled-code-probes flow.

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-ide on Linux RHEL 9

  • Commit ID: 3d13332
  • Duration 0:23:25
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

  • Commit ID: 3d13332
  • Duration 0:35:18
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang on Linux RHEL 9

  • Commit ID: 3d13332
  • Duration 0:40:43
  • Result: ❌ FAILED
  • Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-arm on Linux CentOS 7

  • Commit ID: 3d13332
  • Duration 0:45:48
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr on Linux RHEL 9

  • Commit ID: 3d13332
  • Duration 1:00:31
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

  • Commit ID: 3d13332
  • Duration 1:07:54
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@foundationdb-ci
Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos on macOS Ventura 13.x

  • Commit ID: 3d13332
  • Duration 2:59:08
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@gxglass
Copy link
Copy Markdown
Collaborator

gxglass commented May 6, 2026

@ploxiln could you have a look at this one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants