Formula-backed asset proof with Dagster Pipes and Node #33897
gregkonush
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I built a small example for a pattern I keep running into: Dagster should own the asset graph and materialization metadata, while a Node step owns spreadsheet-style formulas and returns a proof artifact.
The example uses
PipesSubprocessClientto run a TypeScript process with@bilig/workpaper. The TypeScript side edits one workbook input, recalculates dependent formulas, serializes the WorkPaper JSON, restores it, and emits compact Dagster Pipes metadata plus a full proof file.Runnable example:
https://github.com/proompteng/bilig/tree/main/examples/dagster-workpaper-asset
Writeup:
https://proompteng.github.io/bilig/dagster-workpaper-asset.html
Local validation I ran:
The smoke path verifies:
Inputs!B21458to21872187verified: trueThis is not meant to replace Excel for macros, pivots, external links, or exact desktop behavior. The point is narrower: when a Dagster asset needs reviewable business formulas, the formula step can produce calculated values and readback proof without driving Excel or trusting stale cached XLSX formula values.
Curious if this asset/Pipes boundary is useful to other Dagster users, or if there is a better Dagster-native shape for this kind of formula-backed materialization proof.
All reactions