Skip to content

docs: explain context growth in long agent runs - #2590

Open
cupkk wants to merge 1 commit into
huggingface:mainfrom
cupkk:docs-context-cost-long-runs
Open

docs: explain context growth in long agent runs#2590
cupkk wants to merge 1 commit into
huggingface:mainfrom
cupkk:docs-context-cost-long-runs

Conversation

@cupkk

@cupkk cupkk commented Jul 30, 2026

Copy link
Copy Markdown

Closes #2566.

This adds a short section to the memory tutorial that:

  • shows why replaying accumulated memory makes total input-token usage grow quadratically with the step count;
  • demonstrates how to inspect reported input-token usage; and
  • uses an existing step callback to compact older tool observations.

The callback example also calls out its tradeoff: observation compaction is lossy and does not remove the rest of each historical step.

Validation:

  • parsed the new Python example with ast;
  • executed it against the repository's ActionStep and CallbackRegistry classes;
  • checked Markdown fence balance and the numeric example in the formula.

Copilot AI review requested due to automatic review settings July 30, 2026 01:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Input-token cost grows ~O(n²) with step count — full memory is replayed every action step (docs/mitigation)

2 participants