Skip to content

[13.x] Queue workers crash-loop on Redis Cluster / ElastiCache Serverless: cross-slot MGET in getPausedQueues() → TypeError in PhpRedisConnection::mget() #61138

Description

@dan-zonalusa

Laravel Version

13.25.0

PHP Version

8.4

Database Driver & Version

No response

Description

After upgrading to v13.25.0, every queue:work poll iteration throws:

TypeError: array_map(): Argument #2 ($array) must be of type array, false given
at vendor/laravel/framework/src/Illuminate/Redis/Connections/PhpRedisConnection.php:67

The exception is thrown inside Worker::getNextJob() before a job is popped, is caught by its
catch (Throwable) block, the worker sleeps 1s and retries — and fails again, forever. The net
effect is that queue workers stop consuming jobs almost entirely (we saw this take down all
production queues; cache/queue backend is AWS ElastiCache Serverless Valkey 7.2, which
enforces Redis Cluster semantics).

The chain:

  1. [13.x] Add a global pause switch for queues #61126 (shipped in v13.25.0, "Add a global pause switch for queues") changed
    QueueManager::getPausedQueues() to fetch a global pause key alongside the per-queue keys:

    $keys = array_map(fn ($queue) => "illuminate:queue:paused:{$connection}:{$queue}", $queues);
    
    $states = $this->app['cache']
        ->store()
        ->many(
            array_merge(['illuminate:queues:paused'], $keys)
        );
  2. On a Redis cache store this becomes a multi-key MGET. On any cluster-semantics Redis
    (Redis Cluster, ElastiCache Serverless, Valkey serverless), illuminate:queues:paused and
    illuminate:queue:paused:{connection}:{queue} hash to different slots, so the server replies
    CROSSSLOT Keys in request don't hash to the same slot — on every single worker loop.

  3. phpredis signals command error replies by returning false (no exception), so
    PhpRedisConnection::mget() does array_map($closure, false)TypeError. The real
    CROSSSLOT error is swallowed (getLastError() is never surfaced), which made this
    painful to diagnose.

Notes:

  • [13.x] Optimize Worker queue pause check #60109 (v13.24.0) introduced the many() batching for the pause check. That could already
    cross-slot for a worker polling multiple queues (--queue=a,b), but the unconditional global
    key in v13.25.0 makes even single-queue workers fail 100% of the time.
  • RedisStore::many() special-cases PredisClusterConnection and falls back to manyAlias()
    (per-key gets), but there is no equivalent handling for phpredis — and none is possible via
    connection type when the endpoint enforces cluster semantics behind a single proxy endpoint
    (ElastiCache Serverless is used with the regular non-cluster phpredis client).

Steps To Reproduce

  1. Point CACHE_DRIVER=redis and QUEUE_CONNECTION=redis at any cluster-semantics Redis
    (Redis Cluster, or AWS ElastiCache Serverless Redis/Valkey) using phpredis.
  2. Run php artisan queue:work.
  3. Every poll iteration logs the TypeError above and no jobs are consumed.

Minimal reproduction without a worker:

Cache::many(['illuminate:queues:paused', 'illuminate:queue:paused:redis:{default}']);
// TypeError: array_map(): Argument #2 ($array) must be of type array, false given

Truncated trace:

#0 .../Redis/Connections/PhpRedisConnection.php(67): array_map(Object(Closure), false)
#1 .../Cache/RedisStore.php(112): PhpRedisConnection->mget(Array)
#2 .../Cache/Repository.php(167): RedisStore->many(Array)
#3 .../Queue/QueueManager.php(335): Repository->many(Array)
#4 .../Queue/Worker.php(489): QueueManager->getPausedQueues('redis', Array)
#5 .../Queue/Worker.php(450): Worker->getPausedQueues('redis', Array)
#6 .../Queue/Worker.php(243): Worker->getNextJob(Object(RedisQueue), '{default}')

Expected behavior / suggested fixes

Any of these would have prevented the outage, and they're complementary:

  1. Fail open: a failure while checking pause state should not prevent job consumption —
    wrap the pause lookup so workers keep processing when the cache read fails.
  2. Cross-slot-safe lookup: fetch the global key separately from the per-queue keys (or fall
    back to per-key gets à la manyAlias() when MGET fails), so cluster deployments work.
  3. Don't swallow the Redis error: PhpRedisConnection::mget() should detect false and
    throw a RedisException including getLastError() instead of a bare TypeError from
    array_map().

Workarounds

  • Pin laravel/framework to ~13.24.0 (sufficient for single-queue workers), or
  • Queue::withoutInterruptionPolling() in a service provider (disables pause and restart polling).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions