ProcessPool with fork context triggers unexpected parent process shutdown on task timeout (v5.2.0)

[aircloud]

Description:
I am experiencing an issue where the parent process (running a Quart/Hypercorn web server) unexpectedly enters a shutdown sequence whenever a task in the ProcessPool reaches its timeout. This behavior started after upgrading to v5.2.0 and specifically occurs when using the fork multiprocessing context.

Environment:

• Pebble Version: 5.2.0
• Python Version: 3.8+
• OS: Linux

Multiprocessing Context: fork

Steps to Reproduce:

1. Create a ProcessPool with context=multiprocessing.get_context("fork").

2. Schedule a long-running task with a short timeout (e.g., 15s).

3. Use pebble.asynchronous.process or a manual bridge to asyncio.

4. Observe that when the TimeoutError is raised, the parent process receives a signal (likely SIGTERM) and triggers its own shutdown hooks (@app.after_serving in Quart).

Analysis:
Based on the logs and call stacks, when a timeout occurs, PoolManager.update_tasks calls self.worker_manager.maybe_stop_worker(task.worker_id), which eventually invokes stop_process(worker).

In a fork environment, the worker process belongs to the same Process Group as the parent. It appears that the signal sent to terminate the worker is either leaking to the parent process or being treated as a group-wide signal, causing the parent (web server) to initiate its graceful shutdown sequence.

Proposed Fix/Question:
Should pebble explicitly ensure that signals sent during maybe_stop_worker are scoped strictly to the child PID, or should it automatically call os.setpgrp() in forked workers to isolate them from the parent's signal group?

[noxdafox]

Hello,

can you reproduce the issue with older versions of Pebble or was this introduced in the last version?

Could you provide a minimum piece of code which reproduces the problem? It is not clear from your description what is the sequence of actions. For example: are you using asynchronous.process function/decorator within a worker of a ProcessPool? If so, which "parent process" is getting killed (there are at least "two parent processes" and "two child processes here")?

[aircloud]

I've done a deeper investigation and found the root cause. The issue is not about signal group leakage or os.setpgrp() — the actual problem is more subtle.

python：3.8
usage: with quart
pebble 5.1.3 is ok, 5.2.0 is not ok

Root Cause

In 5.2.0, the worker's SIGTERM handler was changed from signal.SIG_DFL to process_exit():

# 5.1.3
signal.signal(signal.SIGTERM, signal.SIG_DFL)

# 5.2.0
signal.signal(signal.SIGTERM, process_exit)

# where:
def process_exit(exitcode, *_):
    multiprocessing.util._exit_function()  # ← this is the problem
    os._exit(exitcode)

When using fork context, the worker process inherits all file descriptors from the parent, including the asyncio event loop's self-pipe fd (registered via signal.set_wakeup_fd()).
When a task times out and the worker is killed via SIGTERM:

1. process_exit() → _exit_function() → runs finalizers → closes the inherited self-pipe fd
2. Parent process receives SIGCHLD (child died)
3. Python tries to write the signal number to the wakeup fd
4. OSError: [Errno 9] Bad file descriptor — because the fd was closed by the dying worker
5. asyncio event loop detects the broken pipe → triggers shutdown
6. @app.after_serving fires → server exits
   With 5.1.3, SIG_DFL + os._exit() never calls _exit_function(), so the inherited fd is never explicitly closed — the kernel just cleans it up without affecting the parent.
   Evidence
   Logs from the parent process clearly show the fd corruption:

Exception ignored when trying to write to the signal wakeup fd:
  File "server.py", line 396, in handler
OSError: [Errno 9] Bad file descriptor

And the sequence confirms it:

[STOP_PROCESS] thread=Thread-2 killing PID=112702   # pebble kills timed-out worker
[SIGNAL] SIGCHLD received! PID=112595                # parent gets SIGCHLD
...
finalize_pool: Closing pebble pool...                # shutdown triggered

Minimal Reproduction

import asyncio, multiprocessing, os, time
from pebble import ProcessPool
from quart import Quart
app = Quart(__name__)
pool = None

def slow_task(n):
    time.sleep(60)
    return n

@app.before_serving
async def startup():
    global pool
    pool = ProcessPool(max_workers=2, context=multiprocessing.get_context("fork"))

@app.after_serving
async def shutdown():
    print("BUG: after_serving triggered unexpectedly!", flush=True)
    if pool:
        pool.close()
        pool.join(timeout=3)
        pool.stop()

@app.route("/slow")
async def slow():
    future = pool.schedule(slow_task, args=(1,), timeout=5)
    loop = asyncio.get_running_loop()
    aio_future = loop.create_future()
    def on_done(f):
        try:
            loop.call_soon_threadsafe(aio_future.set_result, f.result())
        except Exception as e:
            loop.call_soon_threadsafe(aio_future.set_exception, e)
    future.add_done_callback(on_done)
    try:
        return f"Result: {await aio_future}"
    except Exception as e:
        return f"Timed out (expected): {e}", 200

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=9999, use_reloader=False)

pip install pebble==5.2.0 quart
python repro.py &
sleep 2 && curl http://localhost:9999/slow
# Server shuts down after timeout. Replace with pebble==5.1.3 → works fine.

Suggested Fix

# In worker_process(), before processing tasks:
try:
    signal.set_wakeup_fd(-1)
except (ValueError, OSError):
    pass

preserves the intent of _exit_function() cleanup while preventing the fd inheritance issue specific to fork context.

[noxdafox]

I tried your logic above and it works on my workstation on Linux with Python 3.13.

Have you tried with a more recent version of Python? There were several issues in the past with os.fork and asyncio which have been fixed over the years. Python 3.8 went EOL in 2024 and it's not supported anymore.