Slight gap when reproducing LIBERO-Long with the released checkpoint (97.2% vs. 98.5%)

[daizzzR]

Hi authors,

Thanks for open-sourcing LingBot-VA and the LIBERO checkpoint!

We recently finished a full evaluation of the released
robbyant/lingbot-va-posttrain-libero-long checkpoint on LIBERO-Long
(libero_10). We followed the evaluation pipeline and the latest LIBERO
configuration in the repository.

We evaluated all 10 tasks with 50 trials per task (500 trials in total) and
obtained:

Task	Success
0	50/50
1	50/50
2	47/50
3	49/50
4	49/50
5	50/50
6	46/50
7	50/50
8	48/50
9	47/50
Overall	486/500 = 97.2%

The result is already quite close to the 98.5% reported in the paper, but we
would like to understand the remaining gap and make sure that our evaluation
setup is fully aligned with yours.

Our main setup was:

• Repository commit: 7c6ffa9
• Released LIBERO-Long checkpoint
• 50 trials per task
• frame_chunk_size=4
• action_per_frame=4
• attn_window=30
• 20 video denoising steps
• 50 action denoising steps
• action_snr_shift=0.05
• Evaluation attention mode: torch
• transformers==4.55.2
• torch==2.9.0+cu128
• robosuite==1.4.0
• mujoco==3.3.2

We used 8 GPUs only to evaluate different tasks in parallel. Each task still
ran with an independent model server and environment, so the rollout behavior
was not changed.

We also noticed the evaluation variance mentioned in #104. Is a 1.3 percentage
point difference within the expected variance for a single 500-trial run?

Could you also share:

1. The random seed or deterministic settings used for the reported result;
2. The exact LIBERO, robosuite, and MuJoCo versions;
3. Whether the reported 98.5% was obtained from the released checkpoint;
4. Whether it was a single 500-trial evaluation or an average over multiple runs.

Thanks again for your help and for releasing the model!