Skip to content

Commit ff567e2

Browse files
committed
Prevent permanent worker deadlock when cutover times out waiting for binlog sentinel
Buffer allEventsUpToLockProcessed to MaxRetries() so the applier's send always completes immediately even after waitForEventsUpToLock has timed out and exited.
1 parent c6f95cc commit ff567e2

1 file changed

Lines changed: 9 additions & 1 deletion

File tree

go/logic/migrator.go

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,15 @@ func NewMigrator(context *base.MigrationContext, appVersion string) *Migrator {
111111
ghostTableMigrated: make(chan bool),
112112
firstThrottlingCollected: make(chan bool, 3),
113113
rowCopyComplete: make(chan error),
114-
allEventsUpToLockProcessed: make(chan *lockProcessedStruct),
114+
// Buffered to MaxRetries() to prevent a deadlock when waitForEventsUpToLock times
115+
// out. The sentinel applyEventFunc may still be queued in applyEventsQueue when the
116+
// timeout fires; when the worker eventually executes it, it sends on this channel
117+
// with no active receiver. An unbuffered channel would block the worker permanently:
118+
// the queue fills, the listener goroutine stalls, heartbeat lag grows unboundedly,
119+
// and no further cutover attempts are made. With a buffer sized to the retry limit
120+
// the send always completes immediately. Stale sentinels accumulate in the buffer
121+
// and are discarded by the stale-skip loop in waitForEventsUpToLock.
122+
allEventsUpToLockProcessed: make(chan *lockProcessedStruct, context.MaxRetries()),
115123

116124
copyRowsQueue: make(chan tableWriteFunc),
117125
applyEventsQueue: make(chan *applyEventStruct, base.MaxEventsBatchSize),

0 commit comments

Comments
 (0)