Skip to content

WIP: cDAC shared calling-convention port + ByRefLike walker + Win-x64 CallSiteLayout dump tests [NO-REVIEW]#128487

Closed
max-charlamb wants to merge 3 commits into
dotnet:mainfrom
max-charlamb:cdac-shared-calling-convention-v3
Closed

WIP: cDAC shared calling-convention port + ByRefLike walker + Win-x64 CallSiteLayout dump tests [NO-REVIEW]#128487
max-charlamb wants to merge 3 commits into
dotnet:mainfrom
max-charlamb:cdac-shared-calling-convention-v3

Conversation

@max-charlamb
Copy link
Copy Markdown
Member

Note

This PR was prepared with assistance from GitHub Copilot.

Status: WIP / not for review (NO-REVIEW).

This branch is a long-running working branch for porting the native
ArgIterator and calling-convention machinery into the cDAC, plus
follow-on infrastructure (ByRefLike GC ref enumeration walker,
CallSiteLayout dump tests).

The branch carries three commits today:

  1. WIP: cDAC shared calling-convention port (recovered) — per-arch
    ArgIterator port (X86, AMD64-Unix, AMD64-Windows, Arm32, Arm64,
    RiscV64 + LoongArch64) atop ArgIteratorBase / ArgIteratorData /
    ArgIteratorFactory / TransitionBlockLayout. New
    ICallingConvention contract + CallingConvention_1 impl driving
    the iterators from a decoded signature. New data-descriptor entries
    for TransitionBlock + ArgumentRegisters. GcScanner refactored
    to consume CallSiteLayout for caller-stack promotion. Unit-test
    suite under tests/CallingConvention/ covering 214 per-arch cases.
  2. WIP: cDAC ByRefLike GC ref enumeration walker and tests — adds
    IRuntimeTypeSystem.EnumerateInstanceFieldDescs and
    LookupApproxFieldTypeHandle; implements
    GcRefEnumeration.EnumerateByRefLikeRoots (recursive field scan);
    wires up GcScanner.ReportArgument to dispatch to the BRL walker
    via rts.IsByRefLike.
  3. WIP: cDAC CallSiteLayout comprehensive Win-x64 dump tests
    rebuilds the CallSiteLayout debuggee as a 54-frame deep chain
    covering categories A-H of AMD64 calling-convention shapes, and
    adds 46 Win-x64 [ConditionalTheory] tests asserting the explicit
    expected layout for each frame.

Validation so far

  • cDAC unit suite: 2558 passed, 0 failed (16 pre-existing skips).
  • CallSiteLayoutDumpTests_WinX64: 46/46 pass on local dump;
    net10.0 dump skipped via [SkipOnVersion] (pre-existing
    CodeVersions descriptor shape mismatch on net10.0 dumps).
  • SOS !clrstack -gc legacy DAC vs cDAC diff on
    CallSiteLayout.dmp: byte-identical except for one SOS header line.
    Confirms the GCInfo-based managed-frame stack walk matches; a
    follow-up debuggee using P/Invoke bouncers is planned to extend SOS
    parity coverage to ComputeCallSiteLayout itself (which is only
    exercised by GcScanner.PromoteCallerStack on transition frames).

CI ask

Looking to get a CdacDumps_linux_x64.tar.gz artifact from CI so the
CallSiteLayout debuggee can be used to author a sibling
CallSiteLayoutDumpTests_SysVX64 class in a follow-up commit.

Not for review

This PR is marked NO-REVIEW. Please do not invest review cycles —
it exists to capture CI artifacts and as a working branch.

Max Charlamb and others added 3 commits May 21, 2026 17:01
Recovered from a dangling git stash that was previously lost. Includes:

- Per-arch ArgIterator port under
  src/native/managed/cdac/.../Contracts/CallingConvention/ (X86, AMD64-Unix,
  AMD64-Windows, Arm32, Arm64, RiscV64+LoongArch64) atop ArgIteratorBase /
  ArgIteratorData / ArgIteratorFactory / TransitionBlockLayout.
- New ICallingConvention contract and CallingConvention_1 implementation that
  drives the iterators from a decoded signature.
- New data descriptor entries for TransitionBlock + ArgumentRegisters
  (src/coreclr/vm/datadescriptor/datadescriptor.inc, src/coreclr/vm/class.h).
- GcScanner refactored to consume CallSiteLayout for caller-stack promotion;
  CallingConvention contract fetched lazily so stack-walk tests that do not
  register it still work.
- Test suite under src/native/managed/cdac/tests/CallingConvention/ covering
  214 per-arch cases, plus MockDescriptors.CallingConvention support and
  RuntimeTypeSystemGetVectorSizeTests.

Post-recovery cleanup applied on top of the original stash:
- Files relocated from Contracts/StackWalk/CallingConvention/ to
  Contracts/CallingConvention/; helper namespace renamed to
  Contracts.CallingConventionHelpers (matches Contracts/GCInfo/ pattern).
- Collapsed the abstract AMD64UnixArgIterator + CdacAMD64UnixArgIterator
  pair into a single concrete AMD64UnixArgIterator.
- Dropped vestigial extraObjectFirstArg / extraFunctionPointerArg ctor
  parameters that were never set to true in any caller.
- Fixed API drift vs. current origin/main (GenericContextLoc-based check,
  non-generic ArgIterator types, Target.FieldInfo.TypeName).

Builds cdac.slnx clean (0 errors, 0 warnings).
Test result: 2469 passed, 0 failed, 16 skipped.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add IRuntimeTypeSystem.EnumerateInstanceFieldDescs and LookupApproxFieldTypeHandle primitives

- Implement GcRefEnumeration.EnumerateByRefLikeRoots walker (recursive field scan)

- Update CallingConvention_1.ComputeValueTypeHandle to populate handle for ByRefLike args

- GcScanner.ReportArgument dispatches to ByRefLike walker via rts.IsByRefLike

- Consolidate duplicate LookupApproxFieldTypeHandle/ResolveFieldTypeHandle helpers

- Add unit tests for ByRefLike dispatch in GcScannerReportArgumentTests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rebuild the CallSiteLayout debuggee as a 54-frame deep chain that
exercises every interesting AMD64 calling-convention shape (categories
A-H: empty/int/double argbanks, mixed reg banks, managed ref args,
small/large value types, ByRefLike + Span<T>, instance/retbuf/generic/
vararg specials, Vector64/128/256/512, composite kitchen-sink frames).
All 54 frames stay live to Environment.FailFast so a single dump
captures the whole matrix on the FailFast thread.

Replace CallSiteLayoutDumpTests with:

- CallSiteLayoutDumpTestsBase: shared DumpTestBase subclass providing
  CollectChainMethods / LayoutFor / AssertSingle{ByRef,ByValueVT,
  ManagedRef} helpers.
- CallSiteLayoutDumpTests_WinX64: 46 [ConditionalTheory] tests gated
  by [SkipOnOS=windows], [SkipOnArch=x64], [SkipOnVersion=net10.0]
  asserting the explicit Win-x64 layout (IsPassedByRef, slot offsets,
  ValueTypeHandle populated/null) for every frame.

The tests are expectation-snapshots derived from the AMD64 Windows ABI.
They pin cDAC ICallingConvention.ComputeCallSiteLayout output to the
hand-derived spec. Independent ground-truth validation via SOS
!clrstack -gc + transition frames is a tracked follow-up; the current
SOS path exercises ComputeCallSiteLayout only on InlinedCallFrames.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 22, 2026 14:52
@max-charlamb max-charlamb added the NO-REVIEW Experimental/testing PR, do NOT review it label May 22, 2026
@dotnet-policy-service
Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @steveisok, @tommcdon, @dotnet/dotnet-diag
See info in area-owners.md if you want to be subscribed.

@max-charlamb
Copy link
Copy Markdown
Member Author

Superseded by #128488 (AMD64-only slim). Closing in favor of that surface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-Diagnostics-coreclr NO-REVIEW Experimental/testing PR, do NOT review it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant