stdbuf -oL tee

[Gldkslfmsd]

Hello,

I noticed that tee is missing stdbuf in the output. Then, the pipeline stucks with no error after some time. Sometimes shortly after beginning, sometimes later.

I'm attaching 2 pipeline.sh scripts:
FIXED.sh.txt -- this one is fixed. I edited it to insert stdbufs. It never stucks.

This is a wrong, current piplener output. Stucks after some time.
WRONG.pipeline.sh.txt

Also, there there has to be stdbuf -o0 if the commands input and output is binary (it means 0 bytes buffer), and stdbuf -oL if it is text (1 full line buffer).
Maybe it would work with -o0 anytime, but I'm not sure.

[Gldkslfmsd]

./auto_switcher.py is an empty py script for now, I'm going to implement it.

[Gldkslfmsd]

my.py.txt
This script is generating it.

[srdecny]

@Gldkslfmsd Hi, sorry for the late response. I have pushed a fix to the main branch, can you please confirm it works for you now?

[Gldkslfmsd]

Hi,
it's better, but there are still some tees without stdbuf:

stdbuf -oL nc -lk 5001 2> >(ts '[%Y-%m-%d %H:%M:%S]' > /pwd/logs/$DATE/00-cs_audio_rec.err) | tee >(while ! ss -ltn |

[Gldkslfmsd]

I can't re-run the pipeline, it takes too much time to pull, restart the workers etc. I only looked at the code.
You can check cruise-control/pipeliner-setups/202109-dominik whether it runs for couple of minutes without stuck. If yes, then stdbuf might be the problem.
(Don't forget to remove the committed cruise-control/pipeliner-setups/202109-dominik/pipeliner.py with my simple temporary fix.)

[srdecny]

Pushed another commit