More
Evidence and limits
Every claim on this site points to a real, dated run on the machine where PyBox was built. Here they are, along with what they don't prove.
Recorded runs#
These were recorded on 18–19 October 2026. The raw evidence stays private on the build machine; the cleaned copies used on this site are in data/replay/, each naming its source file and SHA-256 fingerprint.
| Program | What happened | What the receipt said |
|---|---|---|
print('hello inspection') | Exited by itself in 0.075 s | Exit 0 · TARGET_EXIT_OBSERVED · removed |
Raises ValueError | Crashed with a traceback | Exit 1 · traceback kept as untrusted output · not a PyBox error |
| Sleeps 8 s, 0.4 s limit | Stopped by PyBox's own deadline at 0.447 s | Exit 137 · SUPERVISOR_KILL_OBSERVED · exit code 124 |
| A child allocates 160 MiB under a 128 MiB limit | The child was killed; the parent printed −9 and exited normally | Exit 0 · engine: out of memory · kernel: 1 out-of-memory kill |
| Tries to start 36 processes under a limit of 32 | 30 started, 6 refused (PyBox's timer and Python use 2) | Kernel: peak 32 processes, 6 refusals |
| Reads a file on the Mac and a private setting | Got "no such file" and nothing | Printed [2, null] · no mounts, no network (verified) |
| Prints a fake protocol message (MCP) | The message stayed inside the output | 0 protocol messages with the forged id |
| Sleeps 8 s, then PyBox is force-killed | The box timer stopped it at 3.03 s | Exit 137 · 0 stop commands sent by anyone |
Images, files and longer runs#
Recorded on 21 October 2026 in a fresh, throwaway installation on the build machine. It was set up from nothing (3 min 17 s), used, then deleted along with its VM. The original installation was never touched.
| Check | What happened |
|---|---|
Default box through pybox exec | Exit 0 in 0.166 s; settings verified before start; box removed |
Files in and out (stdlib) | A CSV went in; 2 files came back; a symbolic link to /etc/passwd was not brought back; writing to the input copy failed with PermissionError |
pandas in the data image | Grouped the CSV and wrote a result file; memory peak 42.4 MiB of 512 |
Sleeps 12 s with --timeout 15 | Exited by itself after 12.13 s; 50 kernel samples |
Sleeps 60 s with --timeout 30 | Stopped by PyBox at 30.01 s (exit 137, SUPERVISOR_KILL_OBSERVED); removed |
300 MiB array in data | Fits under 512 MiB; exit 0 |
700 MiB array in data | Exit 137; engine reported out of memory; cause marked not established |
Network from data | Name lookup failed: no network |
| 9 MiB output file plus 70 small files | 64 files brought back; the big file and the rest listed as not brought back |
Program exits with os._exit | Exit code kept; no files; "no transfer received" |
| 100 KB of stdout plus an output file | stdout cut at 65,536 bytes and flagged; the file still came back intact |
| 9 MiB input; non-empty output folder | Both refused on the Mac before any box was built |
MCP with --images stdlib,data --max-timeout 30 | pandas call with a file in and a file back succeeded; 31 s and an unknown image were refused "(set by the server)" |
Claude Code (claude -p) over MCP | Used run_python with the data image and returned files; 3 turns |
| Performance (M9) | Median 0.316 s per complete stdlib run (20 runs), 0.363 s in data (10 runs). Superseded below |
Custom images, final readings, speed and an independent review#
Recorded on 21 October 2026, in a second throwaway installation set up from nothing (3 min 46 s, including the data image), then deleted.
| Check | What happened |
|---|---|
Your own image from a uv lock (tabulate, rich, pyyaml) | Built in 4.5 s; used by pybox exec and over MCP; the receipt notes that its packages can run start-up code |
matplotlib in data | A PNG chart came back with --output |
300 MiB array in data | Peak 327.6 MiB with the final-reading pause; it showed 146.8 MiB before |
| Exits with code 3 just before a 10 s deadline | Exit 3 kept; no deadline fired (the pause stops short of the deadline) |
Hostile output names (terminal escapes, a newline, invalid UTF-8, a link to /etc) | Receipt intact; names shown escaped; the linked folder listed as not brought back |
A.txt and a.txt both written | Transfer refused; no output folder was created |
| Returned files | Carry com.apple.quarantine |
| Cursor's command-line agent over MCP | Two sessions; charts and a custom-image table came back; all boxes removed |
| Performance | Median 0.204 s per complete stdlib run (20 runs), 0.321 s in data; arming the watcher 9 ms. Details |
An independent review session read the new code adversarially before release. It found no way for a program or agent to reach the Mac's files, the network or past the server's ceilings. It did find 2 medium issues (a program could forge the terminal receipt, or blank the MCP receipt with crafted file names), 6 low and 6 informational ones, and 11 places where the docs said more than the code did. All are fixed or documented, with regression tests; see docs/M10_SECURITY_REVIEW.md in the repository.
The test campaign#
| Area | What was checked | Result |
|---|---|---|
| Restrictions | 13 checks, most run twice: with the protection and with only that protection removed | All passed |
| Resources | 9 checks of memory, processes and CPU, read straight from the kernel | All passed |
| Crash safety | PyBox force-killed while running, and before start; the program signalling the timer | Timer held (3.03 s); leftovers removed after expiry |
| Receipts | 8 kinds of run, three passes each | All passed |
| MCP | 13 protocol checks, including cancellation and oversized input | All passed |
| AI repair | 4 broken programs × 2 feedback styles, with a real model | 8 of 8 repaired |
| Automated tests | Unit and protocol tests (no VM needed) | 209 passing |
Failed attempts during testing were kept, not overwritten. An independent review of the whole codebase found one high-severity issue and two medium ones; all three were fixed and re-checked.
What is assumed#
- The virtual machine is healthy and keeps getting CPU time. If your Mac sleeps, time in the box pauses too.
- Nothing outside PyBox tampers with the box's timer or the VM.
- The Docker command-line tool reports errors in its usual English wording. PyBox relies on that wording in one narrow case.
What is not claimed#
- That the box resists every possible attack. Thirteen checks prove thirteen cases.
- Exact real-time limits, or exact final resource counts.
- That PyBox works with every MCP app. It was tested with its own client, Claude Code and Cursor's command-line agent, not Claude Desktop or the official SDKs.
- That returned files are trustworthy. They are the program's work, carried by a helper running as the program's own user.
- That the AI repair results generalize. Four problems is an example.
- That setup has been proven on a second Mac. It's unit-tested, and an end-to-end run on another machine is the next milestone.
- Anything about cost. Unknown costs are recorded as unknown.