Commit d410a56
authored
fix(train): align DDP no_sync with gradient accumulation (#3472)
Synchronize DDP/FSDP gradients on the final microbatch of each accumulation window.
Verified with a two-rank CPU/Gloo reproduction, py_compile, diff checks, and a clean synthetic merge against current main.1 parent 6829ba7 commit d410a56
1 file changed
Lines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
648 | 648 | | |
649 | 649 | | |
650 | 650 | | |
651 | | - | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
652 | 654 | | |
653 | 655 | | |
654 | 656 | | |
| |||
0 commit comments