HI I try to FT this model on a bit different data distribution, I keep encoder frozen and the moment I unlock decoder even a bit inference/eval mode will just pred a=1 pretty much everywhere, presigmoid logits become poisoned for some reason and saturate it completely. Using train() mode sidesteps this problem.
Morever, if I introduce 0.005 gaus noise to the input image in eval mode the problem is resolved 100% and I get world class preds. Then as I shift towards 0.0 sigma I can observe 'gain knob' effect where the bg area gradually begins to creep into the pred alpha. Tried BN affines + running stats and the ipt_blk* patch-injection blocks to no avail.
Another note: 'recalibrating' BN stats on the altered (and frozen) weights by doing forward passes in train mode over the FT dataset to refresh the running statistics this issue is mitigated, however turns out to be a fragile practice and doesn't really address the real issue.
So what am I missing in the science? Thanks!
Edit: referring to matting mode
HI I try to FT this model on a bit different data distribution, I keep encoder frozen and the moment I unlock decoder even a bit inference/eval mode will just pred a=1 pretty much everywhere, presigmoid logits become poisoned for some reason and saturate it completely. Using train() mode sidesteps this problem.
Morever, if I introduce 0.005 gaus noise to the input image in eval mode the problem is resolved 100% and I get world class preds. Then as I shift towards 0.0 sigma I can observe 'gain knob' effect where the bg area gradually begins to creep into the pred alpha. Tried BN affines + running stats and the ipt_blk* patch-injection blocks to no avail.
Another note: 'recalibrating' BN stats on the altered (and frozen) weights by doing forward passes in train mode over the FT dataset to refresh the running statistics this issue is mitigated, however turns out to be a fragile practice and doesn't really address the real issue.
So what am I missing in the science? Thanks!
Edit: referring to matting mode