compression

Four hours, 8.4 million tokens in, 2.9 million compressed away. Nothing lost.

2026-07-10 · PromptForce team

Tokens in
8.4M
Compressed away
2.9M
Effective compaction
34.3%

Coding agents haul enormous backpacks: file dumps, logs, test output, the same page re-read five times. Most of it gets re-billed every single turn. prefex runs a crew of specialized compactors on that backpack before it reaches the model.

We tabulated one ordinary 4-hour window straight from the compression_analysis table. No cherry-picking. Every event landed in one of the four lanes below, so the totals reconcile exactly.

Four compactors, each doing its job

Lane Tokens in Trimmed Trim rate
ML text compactor 7,104,151 2,054,021 28.9%
Native prose crusher 476,107 190,154 39.9%
Log squeezer 642,010 478,074 74.5%
Stale & duplicate reads 161,623 155,902 96.5%
Aggregate 8,383,891 2,878,151 34.3%

Of every 100 tokens the agent produced, 34 never had to be paid for as raw text.

Two details matter more than the totals

First, each scheme performed to spec. The ML compactor's ~29% trim, the crusher's ~40%, the log squeezer's ~75%, and stale-read stubbing at ~97% are all inside each compactor's designed band. Predictable machines, not magic.

Why does the aggregate (34.3%) land low? It's volume-weighted. The ML compactor alone is 7.1M of the 8.4M input (85% of the pipe) and only trims 28.9%. The high-rate lanes (log 74.5%, stale 96.5%) squeeze hard but are small in volume, so they barely lift the blend. The headline is set by the biggest, gentlest lane.

Second, everything lossy leaves a claim ticket. Any lossy transform stores the original locally and drops a <<ccr:HASH>> marker in its place. The model itself can pull the full text back mid-conversation if it ever needs it. In four hours of trimming, not one answer went without something it needed.

How to read "trim rate"

Trim rate is (tokens_in − tokens_out) / tokens_in: how hard a lane squeezes. Aggregated across all four lanes it becomes the effective compaction rate: the fraction of the agent's raw token output that never reached the model as-is. It's window-dependent because content mix changes hour to hour, so we always quote it from the DB for the exact window shown.

Proof. Per-lane event counts and token deltas live in the compression_analysis table. This window is 2026-07-10 14:26–18:26 UTC, 5,906 events, reproducible with a single GROUP BY filter_type across kompress-text, textcrusher/caveman, router-log, and stale-read/superseded-read/dedup. Trim rates independently match each scheme's published spec. Lossy events carry CCR markers; originals are retained in largeout.