S5 Regression Tests

[arnav-agrawal]

Populating a list of desired end-to-end regression tests for S5:

For each instrument and mode, have a set of reference outputs, that is something to baseline outputs against. Outputs to compare should include:

• Spectroscopy:

• Check the final values for each Parameter per channel (Mean for lsq fit, Mean, -1sigma, +1sigma1, 16th, 50th, 84th for emcee, potentially other values for other fit types)

• Parameters:
    - [ ] Planet radius rp
    - [ ] Orbital period per
    - [ ] Transit midpoint t0
    - [ ] Orbital inclination inc
    - [ ] Semi-major axis ars
    - [ ] Limb-darkening coefficients u1 and u2
    - [ ] Polynomial baseline c0
    - [ ] GP amplitude/length on a log scale A and m
    - [ ] Noise multiplier scatter_mult

• If desired, could check the whole posterior array for each of the free parameters above with the .h5 files

• Light curve data (stored in Table_Save):
    - [ ] Time time
    - [ ] Channel center wavelength wavelength
    - [ ] Channel half-width bin_width
    - [ ] Normalized flux lcdata
    - [ ] Flux uncertainty lcerr
    - [ ] Polynomial systematics model polynomial
    - [ ] Gaussian-process component GP
    - [ ] Combined astrophysical model astroModel
    - [ ] Full composite model model
    - [ ] Light curve data minus model residuals

• Photometry:

• Mostly same as above (would be only one channel) but add any other parameters in fitparams that are fitted for

Note: *_Table_Save.txt is rewritten after every fitter, so will have to use only one fit type for each ECF/EPF for comparison.
Note: Bug in the naming of the fitparams file? There are emcee_fitparams files and emcee_lsq_fitparams files, should the latter just be lsq_fitparams?

Note: Unit testing each model might be best, and then regression testing one instrument/datasets all the way thru.