Refill freed concurrency slots continuously instead of per batch

[kevinhermawan]

Summary

startInBackground processed jobs in fixed batches behind a barrier. It claimed a batch, then waited for every job in that batch to settle before claiming again. A single slow job left the other concurrency slots idle until it finished. With groupConcurrency set it stalled the whole group, because each claim is capped below the number of ready jobs, so the rest were never claimed until the in-flight batch fully drained.

This replaces the batch barrier with a continuous worker pool.

The problem

With concurrency: 3 and groupConcurrency: 3, a queue of same-group jobs is claimed 3 at a time. The old loop awaited all 3 before polling again:

const loop = async () => {
  currentBatchPromise = processJobs();        // claims a batch, runs it
  const processed = await currentBatchPromise; // waits for the WHOLE batch
  scheduleNext(processed === batchSize);       // only now claim again
};

So if 2 of the 3 finish quickly but the third runs for minutes, the two freed slots sit idle and the remaining queued jobs are never picked up until the slow one completes or times out. The claim SQL already supports incremental per-group refill (it subtracts the in-flight count per group), but the loop never re-polled mid-batch, so that capability went unused.

The fix

startInBackground now runs a continuous pool:

• Keep up to concurrency jobs in flight and refill each slot the instant it frees (from the job's finally), instead of waiting for the batch.
• Each claim asks only for the number of free slots, so it never exceeds concurrency or groupConcurrency.
• A periodic tick on pollInterval still runs the cron-enqueue hook and picks up newly added jobs.
• stopAndDrain waits on the set of in-flight job promises.

Compatibility

• No public API change. Same options, methods, and signatures.
• The one-shot start() (serverless) path is untouched.
• concurrency now behaves as a steady in-flight cap in background mode, which is what the docs implied. The doc comments were updated to match.

Testing

Added a regression test in processor.test.ts: a slow job plus several fast jobs in one group under groupConcurrency. It times out on the old barrier (the group stalls behind the slow job) and passes with the pool, which also confirms groupConcurrency stays honored. The full package test suite passes locally against Postgres and Redis.

[netlify]

👷 Deploy request for dataqueue-demo pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	289f263

[netlify]

✅ Deploy Preview for docsdataqueuedev ready!

Name	Link
🔨 Latest commit	289f263
🔍 Latest deploy log	https://app.netlify.com/projects/docsdataqueuedev/deploys/6a2b5755bac81100080a2546
😎 Deploy Preview	https://deploy-preview-41--docsdataqueuedev.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[codecov]

Codecov Report

❌ Patch coverage is 67.94872% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.22%. Comparing base (b284237) to head (289f263).

Files with missing lines	Patch %	Lines
packages/dataqueue/src/processor.ts	67.94%	24 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #41      +/-   ##
==========================================
- Coverage   80.18%   75.22%   -4.97%     
==========================================
  Files          39       22      -17     
  Lines        7515     5982    -1533     
  Branches      987      896      -91     
==========================================
- Hits         6026     4500    -1526     
+ Misses       1463     1455       -8     
- Partials       26       27       +1     

Flag	Coverage Δ
e2e	?
unit	75.22% <67.94%> (+0.06%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.