Skip to content

Add mosaic normalization kernel - #1101

Open
samanklesaria wants to merge 9 commits into
openxla:mainfrom
samanklesaria:normalization_mosaic
Open

Add mosaic normalization kernel#1101
samanklesaria wants to merge 9 commits into
openxla:mainfrom
samanklesaria:normalization_mosaic

Conversation

@samanklesaria

Copy link
Copy Markdown

Adds a Mosaic version of the normalization kernel. Importantly, this implementation does not use any modern GPU features, and works with Ampere GPUs (e.g. A100) too!

@google-cla

google-cla Bot commented Jul 27, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@samanklesaria

samanklesaria commented Jul 28, 2026

Copy link
Copy Markdown
Author

Normalization benchmarks: Mosaic GPU vs Pallas-Triton

NVIDIA A100-SXM4-40GB, bf16, from _results/bench_output.log (2026-08-11).
Times in µs. Ratio = mosaic / pallas; <1 means mosaic is faster.

All lines logged an autotuning cache miss, so both backends ran heuristics
configs rather than tuned ones.

case pallas mosaic xla ratio
forward
384res_64chan 29.8 30.7 30.8 1.03
384res_128chan 59.7 58.3 59.7 0.98
768res_128chan 222 221 223 1.00
384res_128chan_axis0 67.5 63.3 144 0.94
768res_128chan_axis0 240 230 495 0.96
384res_64chan_vmap 221 220 227 1.00
384res_128chan_vmap 438 438 439 1.00
768res_128chan_vmap 1733 1739 1738 1.00
384res_128chan_axis0_vmap 473 455 1086 0.96
768res_128chan_axis0_vmap 1875 1815 4222 0.97
forward + vjp
384res_64chan 84.7 95.1 188 1.12
384res_128chan 175 167 367 0.95
768res_128chan 617 602 1348 0.98
384res_128chan_axis0 185 197 411 1.06
768res_128chan_axis0 671 674 1477 1.00
384res_64chan_vmap 623 624 1349 1.00
384res_128chan_vmap 1205 1181 2637 0.98
768res_128chan_vmap 4680 4635 10397 0.99
384res_128chan_axis0_vmap 1300 1356 3519 1.04
768res_128chan_axis0_vmap 5138 5069 17301 0.99

@samanklesaria
samanklesaria force-pushed the normalization_mosaic branch from f98e41d to 58f2598 Compare July 28, 2026 18:07
@samanklesaria
samanklesaria force-pushed the normalization_mosaic branch 2 times, most recently from 5b87df9 to 5e69b56 Compare August 10, 2026 16:53
@samanklesaria
samanklesaria marked this pull request as ready for review August 11, 2026 18:15
@samanklesaria
samanklesaria force-pushed the normalization_mosaic branch 5 times, most recently from 163babc to a42e11e Compare August 11, 2026 19:27
@samanklesaria

Copy link
Copy Markdown
Author

If the input is of shape (1, A, 1) and we write back the residuals, this kernel will currently fail. That's because we try to write the result directly to GMEM, which currently fails for scalars: store_untiled always use the optimized=True flag, which isn't available for writing to GMEM. This is ideally fixed upstream: the current branch used for scalars should actually be conditional on the layout rather than the size: keep the scalar path for WGSplatFragLayout (and rank-0), and let a tiled value of any rank ≥ 1 fall through to the general case which is correct for it already.

@samanklesaria
samanklesaria force-pushed the normalization_mosaic branch 5 times, most recently from d6d5776 to 3599af0 Compare August 12, 2026 17:50
@samanklesaria

samanklesaria commented Aug 14, 2026

Copy link
Copy Markdown
Author

I've edited the code to make use of the warpgroup semantics and emit_pipeline functionality now available for Ampere.

Comment thread .gitignore
@@ -214,3 +214,8 @@ __marimo__/

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll remove these eventually. Currently trying out jj, which means explicitly accounting for all untracked files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant