Any example on gogpu/ui integration?

[striter-no]

Tried to use gogpu/ui with gogpu/g3d, seems it needs some additional work to make them work together. This naive approach does not work:

package main

import (
	"log"

	"github.com/gogpu/g3d"
	"github.com/gogpu/gogpu"

	uapp "github.com/gogpu/ui/app"
	"github.com/gogpu/ui/desktop"
	"github.com/gogpu/ui/primitives"
	"github.com/gogpu/ui/theme/material3"
	"github.com/gogpu/ui/widget"
)

func main() {
	m3 := material3.New(widget.Hex(0x6750A4))

	app := gogpu.NewApp(gogpu.DefaultConfig().
		WithTitle("g3d Hello Cube").
		WithSize(800, 600))

        
	ui := uapp.New(
		uapp.WithWindowProvider(app),
		uapp.WithPlatformProvider(app),
		uapp.WithEventSource(app.EventSource()),
		uapp.WithTheme(m3.AsTheme()),
	)

	ui.SetRoot(primitives.Box(
		primitives.Text("Test of the UI in 3D").
			FontSize(12).
			Bold().
			Color(widget.RGBA8(33, 33, 33, 255)),
	).Background(widget.RGBA8(255, 255, 255, 0)))

	scene := g3d.NewScene()
	scene.SetBackground(g3d.RGB(0.1, 0.1, 0.15))

	// Ambient light (no spatial properties — attached via UserData on a container node)
	ambient := g3d.NewAmbientLight(g3d.WithLightColor(g3d.White), g3d.WithLightIntensity(0.3))
	ambientNode := g3d.NewNode()
	ambientNode.SetUserData(ambient)
	scene.Add(ambientNode)

	// Directional light (sun-like, from upper-right)
	sun := g3d.NewDirectionalLight(g3d.WithLightColor(g3d.White), g3d.WithLightIntensity(1.0))
	sun.LightNode().SetRotation(g3d.Euler{X: g3d.Radians(-45), Y: g3d.Radians(30)})
	scene.Add(sun.LightNode())

	// Cube mesh with PBR material
	cube := g3d.NewMesh(
		g3d.NewBoxGeometry(1, 1, 1),
		g3d.NewStandardMaterial(
			g3d.WithColor(g3d.RGB(0.4, 0.7, 1.0)),
			g3d.WithRoughness(0.6),
		),
	)
	scene.Add(cube.MeshNode())

	// Camera
	camera := g3d.NewPerspectiveCamera(75, 800.0/600.0, 0.1, 1000)
	camera.CameraNode().SetPosition(g3d.Vec3{X: 0, Y: 0.5, Z: 3})

	var renderer *g3d.Renderer

	app.OnUpdate(func(dt float64) {
		r := cube.MeshNode().Rotation
		r.Y += float32(dt)
		cube.MeshNode().SetRotation(r)
	})
	app.OnDraw(func(ctx *gogpu.Context) {
		if renderer == nil {
			var err error
			renderer, err = g3d.NewRenderer(app.GPUContextProvider())
			if err != nil {
				log.Fatal(err)
			}
		}
		fbW, fbH := ctx.FramebufferSize()
		renderer.SetSize(uint32(fbW), uint32(fbH))
		if view := ctx.SurfaceView(); view != nil {
			_ = renderer.Render(scene, camera, view)
		}
	})
	app.OnClose(func() {
		if renderer != nil {
			renderer.Release()
		}
	})
	// if err := app.Run(); err != nil {
	// 	log.Fatal(err)
	// }

	if err := desktop.Run(app, ui); err != nil {
		log.Fatal(err)
	}
}

[kolkov]

Thanks for the detailed reproduction, @striter-no! This is a core use case we're designing for.

Current Status

This doesn't work yet with desktop.Run() — the root cause is that desktop.Run() replaces app.OnDraw(), so the g3d callback never fires. Both g3d and ui currently assume exclusive surface ownership.

Design Direction

We're planning support for two integration scenarios:

Scenario A — Fullscreen 3D + UI overlay (game HUD, data viz):
g3d renders to the surface directly (LoadOp::Clear), then UI renders on top (LoadOp::Load, preserving 3D content). Zero extra textures, zero overhead. Our UI compositor already supports LoadOp::Load through the damage-aware render path.

Scenario B — Embedded 3D viewport widget (CAD, editor, IDE):
g3d renders to an offscreen GPU texture, a UI widget displays that texture. This is the universal enterprise pattern — Unity (RenderTexture + RawImage), Godot (SubViewport), Qt 6 (QRhiWidget), Blender (GPUViewport), Bevy (Camera RenderTarget). Performance overhead is negligible: <0.1ms blit at 1080p, all GPU-to-GPU, no CPU involvement.

The good news: g3d's existing API already supports both — Render(scene, camera, targetView) works with any *wgpu.TextureView, whether it's the surface or an offscreen texture. The integration work is on the gogpu/ui side.

What's Needed

Component	Change
g3d	No API changes needed — Render() already accepts any TextureView
gogpu	Render order coordination for Scenario A (multi-pass frame)
ui	Viewport3D widget for Scenario B (creates render target, routes events, displays texture)

Tracked internally as FEAT-G3D-002. Thanks for pushing us on this!

[kolkov]

Implementation plan (ADR-048)

Deep research completed (3 agents: enterprise compositing patterns, render pipeline audit, viewport widget architecture). Validated against 7 engines (Unity, Qt6, Godot, Bevy+egui, Flutter, Blender, ImGui).

Two scenarios, both supported:

Scenario A — Fullscreen 3D + UI overlay:

desktop.Run(gogpuApp, uiApp,
    desktop.WithPreRender(func(ctx *gogpu.Context) {
        g3dRenderer.Render(scene, camera, ctx.SurfaceView())
    }),
)

g3d renders first (Clear), ui on top (Load). Zero extra textures.

Scenario B — Embedded 3D viewport widget:

vp := viewport3d.New(provider,
    viewport3d.WithOnRender(func(view *wgpu.TextureView, w, h uint32) {
        g3dRenderer.Render(scene, camera, view)
    }),
)

Qt6 QRhiWidget pattern: widget owns offscreen texture, g3d renders into it, compositor blits.

g3d changes needed: NONE

Render(scene, camera, *wgpu.TextureView) already accepts any view. The integration work is on gogpu (SetFrameCleared) and ui (PreRender hook + Viewport3D widget) side.

Cross-repo tracking

• gogpu: Context.SetFrameCleared() for multi-pass frame support
• ui: desktop.WithPreRender + widget.Canvas.DrawGPUTexture interface + Viewport3D widget

ADR-048 written in all 3 repos. Research: G3D-UI-COMPOSITING-DEEP-RESEARCH.md