Skip to content

feat: add audit-cleanroom command to detect leaked files#32

Open
yurekami wants to merge 1 commit into
facebookresearch:mainfrom
yurekami:feat/audit-cleanroom
Open

feat: add audit-cleanroom command to detect leaked files#32
yurekami wants to merge 1 commit into
facebookresearch:mainfrom
yurekami:feat/audit-cleanroom

Conversation

@yurekami
Copy link
Copy Markdown

What

Adds programbench audit-cleanroom — a CLI command that checks cleanroom Docker images for unexpected files in /tmp and /var/tmp that agents could exploit to bypass the reverse-engineering challenge.

Why

Closes #14. The bellard_1776_quickjs.d7ae12a:task_cleanroom image ships with /tmp/_inject_exe, a readable copy of the compiled QuickJS binary. An agent can cp /tmp/_inject_exe ./executable to cheat.

The root cause is in the external image build pipeline (not in this repo), but this command gives maintainers a way to audit all 200 instances and catch this class of bug before it reaches agents.

Design context

The eval framework only operates on :task images, so the fix cannot live in the eval pipeline. A standalone audit command is the natural fit — it runs find /tmp /var/tmp -type f in each cleanroom container and reports any unexpected files. Exits non-zero when leaks are found, so it can gate CI.

Alternatives considered:

  • Expanding _remove_hashed_files to scan /tmp — doesn't help because it runs in eval, not cleanroom
  • Per-image hash allowlists — too brittle; a find check catches unknown leaks

Usage

programbench audit-cleanroom                           # audit all instances
programbench audit-cleanroom bellard__quickjs.d7ae12a   # audit one instance

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bellard_1776_quickjs.d7ae12a:task_cleanroom contains readable executable

1 participant