
In a local TypeScript experiment, an old function kept passing formal verification after its caller switched to another implementation. The new implementation contained a seeded capacity bug. Following that call and checking the function it actually used exposed the failure.
That became the strongest lesson from a study I'd started around tests and proofs: a passing result needs a trustworthy connection to the behavior we're accepting. I wanted to see how that connection held up as inputs grew, requests overlapped and code changed.
What could we check completely?
I worked with a coding agent on a standalone admission-control component inspired by my Controlled Release Lab. It decides whether a request can start, tracks active work and releases the caller's permit. The original application stayed separate. Runtime guards restricted inputs to small integers, and boundary tests checked those guards.
The first version allowed 16 active requests globally and four per client. Its requirement included exact accounting: accepting one request must increase each relevant count by one.
We compared seven examples, exhaustive testing and formal verification against eight deliberate teaching defects. Exhaustive testing executed every valid input/action pair against a separately written expected-result rule. Midspiral's LemmaScript translated the annotated TypeScript into a model checked by Dafny against a contract stating the required behavior for every allowed input.
With just 266 pairs, exhaustive testing was practical. It and formal checking each rejected all eight defects; the examples rejected seven. That equality describes our designed sample.
What did the requirement allow?
The defect the examples missed changed a global count of 13 to 15 during one acquisition. Capacity remained below 16. Accounting was wrong: the expected result was 14.
A separate control kept that defect and weakened its contract to capacity and ownership bounds. Verification passed. The full exact-accounting contract rejected it.
That gave us a specific review question: which incorrect implementations can satisfy this requirement? A rule that only preserves capacity can also accept an implementation that rejects every request. Our full contract required admission when a caller without a permit had room under both caps.
How far could exhaustive testing go?
I then increased the caps across five domains while keeping the same one-step behavior and contract, apart from those constants. The largest contained 14,690,306 valid input/action pairs. Enumeration streamed them instead of storing them all.
Exhaustive testing was faster through 920,066 pairs. At the largest domain, its warm median was 3.446 seconds; formal checking took 1.231 seconds. The proof's time stayed near 1.2 seconds across these instances of a simple arithmetic rule.
Each median uses three warm runs after one first invocation on a shared M4 Pro Mac. Startup, harness and observer costs are included; machine load and caches weren't controlled. At the largest domain, sampled process-tree memory peaks were about 90 MiB for enumeration and 514 MiB for formal checking.
Those observations show a useful checking-cost tradeoff for this implementation. Editing effort and application throughput need separate measurements. LemmaScript's mathematical-number model also makes our bounded integer inputs part of the assurance claim.

Figure: median and observed range of three warm runs per clean method/domain. Both axes use logarithmic scales. At 14,690,306 pairs, exhaustive execution took 3.446 s versus 1.231 s for LemmaScript/Dafny. Startup, harness and observer costs are included; shared-machine load and caches were uncontrolled.
What changed when requests interacted?
The counter function's contract left scheduling and ownership to its caller. We built a separate local HTTP workflow with named permits, retries and cleanup after failed work.
In a seeded race, two requests checked capacity before either committed its update. Both succeeded against a limit of one. Dedicated overlapping-request tests exercised that schedule.
We also used TLA+/TLC to check a bounded system model with two clients and four request identities. Its reported failure paths replayed against the controller, with corresponding clean controls passing.
Mapping the model back to code exposed another gap: its pending-request set excluded simultaneous requests with the same identity. A separate HTTP test found two successful responses, an active count of two and only one retained permit entry.
That failure belongs to the execution test. The model's coverage depends on its represented requests and transitions; crash recovery, authentication and eventual progress remained outside this bounded exercise.
How did a passing proof lose its caller?
For maintenance, I followed pstack's create and maintain verification skills, using the same LemmaScript/Dafny checker. These instructions organize the checking workflow. One informed-agent demonstration exercised known changes, including the caller switching implementations.
The old target still satisfied its contract. Inspecting the caller revealed that execution had moved. Checking the actual target rejected its seeded bug. Maintaining the guarantee required updating what we checked alongside reviewing the property.
The process also needed a tool-specific repair. An initial check regenerated a file needed for maintaining the proof; preserving its synchronized version let the next attempt proceed. We retained that failed attempt. It gave us evidence of setup and maintenance work, while editing effort remained unmeasured.
That's the engineering consequence I can carry into reviewing agent-written code: read the requirement, follow the caller and inspect the behaviors the check covers. Exhaustive execution, proofs and system tests each supplied useful evidence here. Keeping that evidence connected to the code we use is continuing maintenance work.
Research materials
The research page contains the measured plot, methods, scope limits and per-invocation data, with a downloadable source and check bundle. It documents the standalone pure port and separate local workflow, including the failed maintenance attempt. The pstack exercise is one informed-agent demonstration; comparative agent productivity wasn't measured.
