Skip to content

Fix uncaught StackOverflowError in StackTraces.getStackTrace - #12233

Open
dougqh wants to merge 3 commits into
masterfrom
dougqh/fix-stacktraces-soe-in-error-capture
Open

Fix uncaught StackOverflowError in StackTraces.getStackTrace#12233
dougqh wants to merge 3 commits into
masterfrom
dougqh/fix-stacktraces-soe-in-error-capture

Conversation

@dougqh

@dougqh dougqh commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Widens exception handling in StackTraces.getStackTrace to catch Error

Motivation

Intended to improve the case where a StackOverflowError is being reported
In that situation, StackTraces.getStackTrace is likely to trigger a fresh StackOverflowError

Additional Notes

  • StackTraces.getStackTrace caught printStackTrace() failures with catch (Exception ...), so a second StackOverflowError thrown while formatting an already stack-constrained throwable escaped uncaught instead of falling back to the manual getStackTrace()-based reconstruction.
  • Widened both catch clauses to Throwable so any Error (not just Exception) is handled the same way.
  • Found via Error Tracking issue 03773be6-9b34-11f1-930e-da7ad0900002 (service ci360-engage-optimize): a StackOverflowError in the customer's app was caught by our instrumentation with almost no stack margin left, and capturing its trace triggered a second overflow that propagated out of our error-handling code.

Jira: APMLP-1767

Test plan

  • Added TestThrowables.throwingStackOverflowOnPrintStackTrace() and a unit test asserting getStackTrace falls back gracefully instead of throwing
  • ./gradlew :dd-trace-core:test --tests "datadog.trace.core.util.StackTracesTest" passes
  • ./gradlew :dd-trace-core:spotlessApply

🤖 Generated with Claude Code

dougqh and others added 2 commits August 18, 2026 15:42
printStackTrace() can itself throw a second StackOverflowError when
formatting a throwable caught with little remaining stack margin.
The existing fallback only caught Exception, so the second
StackOverflowError (an Error) escaped uncaught. Widen the catches to
Throwable so it falls through to the manual getStackTrace()-based
reconstruction instead.

APMLP-1767

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Widen safeGetMessage's catch to Throwable, and try t.getMessage()
in the innermost fallback of StackTraces.getStackTrace before giving
up on just the class name, so a bit more diagnostic context survives
even when both printStackTrace() and getStackTrace() throw.

APMLP-1767

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dougqh
dougqh marked this pull request as ready for review August 18, 2026 19:55
@dougqh
dougqh requested a review from a team as a code owner August 18, 2026 19:55
@dougqh
dougqh requested a review from mhlidd August 18, 2026 19:55
@dd-octo-sts dd-octo-sts Bot added the tag: ai generated Largely based on code generated by an AI or LLM label Aug 18, 2026
@dd-octo-sts

dd-octo-sts Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4cf9c0427c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

try {
return t.getMessage();
} catch (Exception e) {
} catch (Throwable e) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restrict recovery to StackOverflowError

If ThreadDeath is delivered while safeGetMessage is executing—or an overridden getMessage() throws it—this broad catch converts the termination signal into a diagnostic string and allows the thread to continue. The new catch (Throwable) clauses in getStackTrace have the same problem for ThreadDeath and unrelated fatal VM errors. Since the intended recovery is specifically for stack exhaustion while retaining the existing handling of ordinary exceptions, catch Exception | StackOverflowError instead.

Useful? React with 👍 / 👎.

trace = sw.toString();
} catch (Exception ignored) {
// printStackTrace() failed (e.g. getMessage() throws inside toString()).
} catch (Throwable ignored) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Cover truncation in the StackOverflowError fallback

When the application catches the original StackOverflowError without first unwinding its recursive frames—the production scenario described by this change—this catch can recover from printStackTrace, but the method then calls truncate outside the fallback and its catch handles only Exception. With the stack still constrained, even trace.length() in truncate can raise another StackOverflowError; reproducing this with a recursive caller and a constrained -Xss causes getStackTrace to exit with that second error despite these new catches. Extend the specific StackOverflowError recovery through the truncation step and return the reconstructed trace if truncation overflows.

Useful? React with 👍 / 👎.

@datadog-prod-us1-4

This comment has been minimized.

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: FAIL

A recovered long stack trace still enters a final truncation block that catches only Exception, allowing another StackOverflowError to escape under the same low-stack condition this change targets.

Open Bits AI session

🤖 Datadog Autotest · Commit 4cf9c04 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

trace = sw.toString();
} catch (Exception ignored) {
// printStackTrace() failed (e.g. getMessage() throws inside toString()).
} catch (Throwable ignored) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Final truncation can still leak StackOverflowError

Error-reporting instrumentation can still propagate StackOverflowError into application code for configured or CI Visibility stack-trace limits.

Assertion details
  • Input: A throwable whose printStackTrace overflows, whose manual fallback produces a trace longer than the finite maxChars limit, and whose low remaining stack causes truncate to overflow.
  • Expected: Stack-trace capture should return the recovered trace rather than propagate another StackOverflowError from final formatting.
  • Actual: After recovering from the initial StackOverflowError, getStackTrace calls truncate; if truncation also throws StackOverflowError, its catch (Exception) does not intercept it and the error escapes.

Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

@dd-octo-sts

dd-octo-sts Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.98 s 13.93 s [-0.5%; +1.2%] (no difference)
startup:insecure-bank:tracing:Agent 12.91 s 13.05 s [-2.0%; -0.3%] (maybe better)
startup:petclinic:appsec:Agent 17.39 s 16.61 s [+0.3%; +9.1%] (maybe worse)
startup:petclinic:iast:Agent 17.44 s 17.48 s [-1.1%; +0.5%] (no difference)
startup:petclinic:profiling:Agent 17.26 s 17.25 s [-1.3%; +1.4%] (no difference)
startup:petclinic:sca:Agent 17.36 s 16.56 s [+0.4%; +9.3%] (maybe worse)
startup:petclinic:tracing:Agent 16.42 s 16.27 s [-3.2%; +5.1%] (no difference)

Commit: 6952989f · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

catch (Throwable) also swallowed ThreadDeath and unrelated fatal
Errors; narrow to the two cases this fix actually needs to recover
from. Also extend the same recovery to the truncate() call so a
second StackOverflowError while the stack is still constrained
during truncation doesn't escape uncaught.

Addresses codex review comments on #12233.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tag: ai generated Largely based on code generated by an AI or LLM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants