Skip to content

[AIE2P][AIE2PS] Legalize the 64-bit bfp16 block-exponent G_BUILD_VECTOR (fixes #995)#1184

Open
atassis wants to merge 1 commit into
Xilinx:aie-publicfrom
atassis:aie2p-bfp16-exp-buildvec-995
Open

[AIE2P][AIE2PS] Legalize the 64-bit bfp16 block-exponent G_BUILD_VECTOR (fixes #995)#1184
atassis wants to merge 1 commit into
Xilinx:aie-publicfrom
atassis:aie2p-bfp16-exp-buildvec-995

Conversation

@atassis

@atassis atassis commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Fixes #995.

Compiling the block_datatypes bfp16 matmul kernel (matmul_vectorized_bfp16) for AIE2P aborts the legalizer:

unable to legalize instruction: %..:_(<8 x s8>) = G_BUILD_VECTOR ...

The <8 x s8> is the bfp16 block-exponent vector (a splat feeding the BFP...mul.conf MAC). It is a valid register type (EXPVEC64, produced by the bfp16 intrinsics), but there was no way to build one: isValidVectorAIEP treats a vector as valid only if its size is 32 or greater than 64, so 64-bit vectors hit .unsupportedIf, and legalizeG_BUILD_VECTOR also asserts on 64-bit.

This routes 64-bit vectors to custom legalization and packs the elements into a same-sized scalar (zext/shl/or), then bitcasts to the vector. The scalar-to-vector move (for the exponent, GPR to EXPVEC64) is already a legal register copy, so this selects without a native build. On the failing kernel the exponent splat of 127 now materializes as 0x7F7F7F7F7F7F7F7F moved into the exponent register.

Applied to both AIE2P and AIE2PS (identical rule; shared helper).

Testing

  • The failing kernel now compiles. aie_kernels/aie2p/mm_bfp.cc (matmul_vectorized_bfp16) compiled for aie2p-none-unknown-elf -O2, both to assembly and to an object (-filetype=obj); it aborted the legalizer before this change.
  • Correct codegen. For that kernel the constant block-exponent splat of 127 materializes as movxm r6, #0x7F7F7F7F + mov el0, r6 / mov eh0, r7, i.e. the exponent register holds 0x7F7F7F7F7F7F7F7F = 127 in all 8 blocks, which is the intended block exponent.
  • New legalizer test (legalize-build-vector.mir, -run-pass=legalizer, runs on both aie2p and aie2ps) for the <8 x s8> build. Existing legalize-build-vector.mir and inst-select-scl2vec-bfp16.mir still pass on both triples.
  • Correctness reasoning. The datalayout is little-endian, so element i is packed at bits [i*EltSize, (i+1)*EltSize) and bitcasts back to lane i. A 64-bit element (a 1-element vector) is used as-is rather than zero-extended to its own size (which would be invalid).

Not covered: an end-to-end on-device numerical run (the only stock design that reaches this kernel is Chess-tuned), so verification here is at the compile/codegen level, which is what the crash fix requires.

@atassis

atassis commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@hunhoffe one 1.4-relevant note on this. It fixes the <8 x s8> G_BUILD_VECTOR crash that the block_datatypes/gemm_asymmetric_tile_buffering README cites as why its bfp16 kernels are pinned to use_chess=True. With this in, matmul_vectorized_bfp16 compiles on Peano, so those chess-only lit tests can move to the Peano backend. That unblocks the example for the Peano lit-enablement effort (#3075). Happy to put up the follow-up that flips the configs once this lands.

@atassis
atassis force-pushed the aie2p-bfp16-exp-buildvec-995 branch from d2e41cb to 80e24e7 Compare July 22, 2026 14:46
Fixes Xilinx#995.

Compiling the block_datatypes bfp16 matmul kernel (matmul_vectorized_bfp16)
for AIE2P aborts the legalizer:

  unable to legalize instruction: %..:_(<8 x s8>) = G_BUILD_VECTOR ...

The <8 x s8> is the bfp16 block-exponent vector (a splat feeding the
BFP...mul.conf MAC). It is a valid register type (EXPVEC64, produced by the
bfp16 intrinsics), but there was no way to build one: isValidVectorAIEP treats
a vector as valid only if its size is 32 or greater than 64, so 64-bit vectors
hit unsupportedIf, and legalizeG_BUILD_VECTOR also asserts on 64-bit.

Route 64-bit vectors to custom legalization and pack the elements into a
same-sized scalar (zext/shl/or), then bitcast to the vector. The
scalar-to-vector move (for the exponent, GPR to EXPVEC64) is already a legal
register copy, so this selects without a native build. On the failing kernel
the exponent splat of 127 now materializes as 0x7F7F7F7F7F7F7F7F moved into the
exponent register.

Applied to both AIE2P and AIE2PS (identical rule; shared helper). Adds a
legalizer test that runs on both.
@atassis
atassis force-pushed the aie2p-bfp16-exp-buildvec-995 branch from 80e24e7 to ab46e23 Compare July 24, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ICE in mm_bfp.cc: unable to legalize instruction: G_BUILD_VECTOR <8 x s8>

1 participant