Input-token cost grows ~O(n²) with step count — full memory is replayed every action step (docs/mitigation)

[wartzar-bee]

Hi — thanks for keeping smolagents small and readable; it made this easy to trace.

I want to flag a cost characteristic that isn't spelled out in the docs and that surprises people
on longer tool-loop runs: input tokens grow quadratically with the number of steps, because the
full accumulated memory is re-sent to the model on every action step.

The mechanism, in v1.26.0:

• write_memory_to_messages walks all of self.memory.steps and appends each step's messages
  (agents.py L758-770).
• memory.steps only ever grows within a run — it's a list initialised empty and appended to,
  cleared only by reset()
  (memory.py L230,
  L232-234).
• The action loop calls it with no summary_mode, so the full history replays each step
  (agents.py L1284);
  summary_mode=True is used only for planning
  (agents.py L684).
• The default step ceiling is max_steps: int = 20
  (agents.py L300).

So if P = system-prompt tokens and s = tokens each completed step adds to memory (thought +
code + tool observation), cumulative input tokens over an n-step run are:

Σ (k=1..n) [ P + (k-1)·s ]  =  n·P  +  s · n(n-1)/2

The n(n-1)/2 term is quadratic. Priced naively as "n steps ≈ n × one step", the history you pay
for is n · s; the actual history replayed is n(n-1)/2 · s — e.g. 9.5× more replay at the
default max_steps=20 (190 vs 20). It bites hardest when a tool returns a large observation (a web
page chunk, a file, a dataframe), since that payload is then re-sent on every later step.

To be clear, this is a reasonable default — the model needs context — and smolagents is more honest
about it than most: every ActionStep already carries token_usage
(memory.py L63),
so summing step.token_usage.input_tokens across agent.memory.steps after a run shows the curve
directly. The gap is just that it isn't documented, and there's no in-tree example of bounding it.

What I'd suggest, using the hooks you already ship: step_callbacks is first-class
(agents.py L282/L304,
wired at L416),
and because memory.steps is a plain list the caller owns, a callback can cap how much history
survives into the next write_memory_to_messages — e.g. keep the last N ActionSteps and collapse
older observations to a short marker. That turns the input curve from quadratic back toward linear
for a bounded recall trade-off.

Happy to open a small PR if you're open to it — either:

1. a short docs note ("Controlling context cost on long runs") showing the token_usage
   summation and a step_callbacks trimming example, and/or
2. an optional helper (e.g. a keep_last_n_action_steps callback factory) so the mitigation is
   one import rather than hand-rolled.

Would a docs-only PR, a helper, or neither fit best with where you want the memory API to go?
Thanks!