BUG: Thread-based timeout cannot interrupt single big-integer operations

[Nkluge-correa]

Problem

When a CodeAgent model generates a single explosive integer operation — most critically a ** b where b is astronomically large (10 ** 10**8, etc.), but also a << b or repeated a * b * ... — CPython computes the gigantic integer entirely in C while holding the GIL and never crossing a bytecode boundary.

The timeout() decorator (from smolagents.local_python_executor, v1.26.0) is a thread-based timeout. It cannot interrupt a single CPU-bound C operation — future.result(timeout=timeout_seconds) will never raise FuturesTimeoutError because the worker thread never reaches a Python bytecode boundary for the timeout signal to be delivered.

Even with the shutdown(wait=False) fix from the first deadlock (BUG #1), the leaked worker thread keeps churning, holding the GIL indefinitely. On the next agent step, the main thread calls ThreadPoolExecutor.submit() → Thread.start(), which must wait for the new worker to signal startup — impossible while the leaked thread monopolizes the GIL. The entire process freezes silently with zero further output.

smolagents' own MAX_OPERATIONS = 10_000_000 (per-AST-node) guard does not help: a single a ** b is one operation, regardless of result size. Only sprawling pure-Python infinite loops get caught by the operation counter.

Steps to reproduce

import time
import threading
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeoutError
from smolagents.local_python_executor import evaluate_python_code

# Simulate what the agent pipeline does: a thread-based timeout around
# sandboxed execution, two steps back-to-back.

def execute_with_timeout(code_str, timeout_seconds):
    executor = ThreadPoolExecutor(max_workers=1)
    future = executor.submit(
        evaluate_python_code,
        code_str, {}, {},
        authorized_imports=["math"],
    )
    try:
        return future.result(timeout=timeout_seconds)
    except FuturesTimeoutError:
        executor.shutdown(wait=False)  # BUG #1 fix applied
        raise

# Step 1: an explosive computation that the timeout cannot interrupt
try:
    execute_with_timeout("10 ** 10 ** 8", 10)
except Exception:
    pass

# At this point a stuck worker thread is holding the GIL forever.
# Step 2: the main thread blocks in submit() because the leaked thread
#         won't release the GIL for the new worker to start.
print("Attempting Step 2...")
execute_with_timeout("2 + 2", 10)   # <-- hangs forever
print("NEVER REACHED")

Expected behavior

Step 2 completes normally (or the first step is quickly rejected), and the process proceeds to the next trace.

Actual behavior

Step 1 never raises FuturesTimeoutError — the worker thread computes a multi-terabyte integer in C while holding the GIL. Step 2 blocks forever in ThreadPoolExecutor.submit() → Thread.start(). The entire process freezes with zero CPU activity on the main thread and zero log output, but the leaked worker thread still shows CPU usage.

Confirmed in production on 2026-07-06: two 1-node 8×A40 jobs (smolagents 1.26.0 + vLLM + Qwen3-235B-A22B) both froze at the identical evaluate_binop / ast.Pow (left_val ** right_val) line for 19.5+ hours. Live py-spy traces confirmed the MainThread in Thread.start() and a leaked ThreadPoolExecutor worker in evaluate_binop.

Proposed fix

Patch smolagents.local_python_executor.evaluate_binop to estimate result bit-length before computing the integer for the three explosive operators (**, <<, *), and raise InterpreterError if the result would exceed a sane bound (e.g., 10 Mbit ≈ 1.25 MB integer):

est_bits(**):   right_val × left_val.bit_length()   (when |left| ≥ 2, right ≥ 2)
est_bits(<<):   left_val.bit_length() + right_val   (when left ≠ 0, right > 0)
est_bits(*):    left_val.bit_length() + right_val.bit_length()

Environment

• smolagents 1.26.0
• Python 3.12.3

[rkfshakti]

Opened PR #2564 with the fix. The change adds _check_safe_operation() guards in evaluate_binop, evaluate_augassign, and BASE_PYTHON_TOOLS (via _safe_pow) that estimate the bit-length of the result without computing it and raise InterpreterError before the C-level freeze can occur.