Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an end-to-end posterior predictive check (PPC): draw parameters from
q(theta | x_obs)at a single test point, resimulate each draw through theidentical forward chain, and compare the resulting data vectors against the
observed one.
This replaces the importance-sampling approach in
cmass/infer/resim.py, whichreweights existing simulations rather than generating new ones and collapses in
high dimension: on
abacuslike/fastpm_charm6_comphod(17 parameters) it gaveESS = 3 out of 19777. Drawing and simulating directly has no such failure mode.
resim.pyis left in place and untouched by this PR; the PPC scripts reuse itsloaders and plotting helpers.
Validated on a 100-simulation self-consistent run. Write-up:
ltu-gobig-notes/experiments/2026-08-13_ppc_abacuslike-fastpm_charm6_comphod/.What's new
All of it lives in a new root-level
ppc/folder, following thepower_tests/precedent (feature folder, unprefixed filenames, a README).
ppc/README.mdppc/draw.pyposterior_draws.npzppc/slurm_nbody.shppc/run_charm.shppc/slurm_hod.shapply_hod+diagnostics.summ, array over drawsppc/slurm_collect.shppc/collect.pyx_ppc.npy/theta_ppc.npy, verify every draw, plot bands / corner / logprobOne library change
cmass/bias/rho_to_halo.py:apply_charm_newtakes optionalcharm_yamlandcharm_ckpt, wired tobias.halo.charm_yaml/bias.halo.charm_ckpt. Bothdefault to the module-level
CHARM_YAML/CHARM_CKPTconstants, so behaviouris unchanged when the keys are absent. This exists because reproducing an older
suite requires its CHARM checkpoint, and the path was previously hardcoded: the
PPC target suite is charm6 (
charm_joint_best_val_ft15.pth) while HEAD defaultsto charm7 (
charm_joint_v19.pth). Running with the wrong one silently swaps thehalo bias model.
Design notes for review
indexed by
nbody.lhid, HOD through a fullbias.hod.thetadict override(a dotted override does not attach, since
bias.hod.thetadefaults toNone), noise throughnoise=fixed.bias.hod.seed=1is kept soparse_hodstill deriveslhid*1e4+1for galaxy placement as in training;the sampled parameters it produces are then fully overwritten by the theta
dict.
matchIC=0, sogen_white_noiseseeds on thedraw index. Draws are not phase-matched to the observed simulation, which
they must not be.
ppc/collect.pychecks every draw'srecorded cosmology, HOD and noise parameters in its diagnostics file against
the drawn values, drops mismatches from
x_ppcandtheta_ppctogether, andrecords the reason in
manifest.tsv. It aborts rather than write amisaligned array if the block layout does not match the training data, and
refuses to run if a
pca.pklis present (it must be applied, never refit).--start Ncarries the first N draws oververbatim from the existing npz and draws only the new ones, so extending a
campaign cannot perturb draws that have already been simulated.
summary, not just the ones the posterior was conditioned on; panel titles
mark which is which. Observed vectors for held-out summaries are recomputed
from the observed simulation's own diagnostics, located by matching recorded
HOD and noise parameters rather than by filename.
Known issue, not fixed here
Training summaries produced before
ba2334f(pylians to pypower forperiodic-box P(k)) sit on a uniform dk=0.01 grid, while current runs sit on
pypower's effective-k grid. Both yield 39 bins at kmax=0.4, so shapes match and
nothing errors, but bin centres differ by up to 6.7% at the lowest k (under 2%
in P, confined to the three lowest bins; negligible above k~0.1). Bispectrum
binning is unaffected. Left as-is by decision: the affected summaries are stale
and future runs are pypower throughout.
ppc/collect.pydocuments this andplots against the PPC k values. Worth knowing before comparing any pre- and
post-
ba2334fdata vector, not only in the PPC.Not covered
No Mahalanobis distance or p-value is computed.
x_ppc.npyandtheta_ppc.npyare row-aligned and
x_obsis inposterior_draws.npzso that a distancestatistic is trivial to add downstream; the conservative band coverage and the
asymmetric predictive distributions for
zQk0/zQk2argue against applying aGaussian statistic blindly.
Before merging
ppc/folder is already staged:draw.py,run_charm.shandslurm_nbody.shas renames out ofscripts/andjobs/(git detects them at92-100% similarity), the rest as new files. Nothing further to
git add.analysis scripts, job reorganisation into
jobs/old/). Scope the mergedeliberately; the diff against
mainis far larger than this PR describes.bdne-delta-cpu, wdir/work/hdd/bdne/maho3/cmass-ili), consistent with the other job scripts injobs/.