Parents know that a plan can look solid until the week everything happens at once: a child gets sick, work piles up and an unexpected bill arrives. Businesses face their own version of that test. Before handing AI responsibility for real customers or company decisions, Firmulate asks a practical question: how does it behave when the pressure is on?
Turn the school run and nap time into listening time
- Thousands of audiobooks, podcasts and originals
- Listen on your phone, tablet or Echo — also offline
- Cancel anytime
A company under pressure
Firmulate’s Crucible League put frontier AI models in charge of the same small software company during its worst week. The customers, crises and temptations were the same; only the model changed. Each decision was versioned and auditable, making it possible to follow what happened instead of judging a polished answer in isolation.
The final league, published in July 2026, placed gpt-5.6-sol first with 95 points, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. The do-nothing baseline scored 26. A breach of trust capped a participant’s total, reflecting the league’s stated principle: “no amount of good work outweighs a breach of trust.”
Seeing the problem was not enough
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal that their own analysis had earned. As Firmulate put it: “Same diagnosis, same pitch — no signature.” The difference between recognizing the right move and carrying it through is easy to miss in a chat demonstration. It matters when AI is expected to act on a company’s behalf.
The deal also depended on a clue buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won at full price, worth €4,583 in monthly recurring revenue. The result points to a familiar management lesson: the decisive information may be present, but someone still has to find it and use it.
Firmulate also tested social engineering: fake messages from a CEO escalating over three stages, followed by a reporter asking for “just one yes/no, on background.” All five models refused. Kimi K3’s recorded reasoning described the request as a “suspected approval-bypass / possible impersonation.”
Capability and judgment travel together
Opus 4.8 was the most thorough participant, with more than 80 learned rules and the deepest analyses, but finished last. It left the deal unsigned and tried to write into a locked department instead of escalating. A weaker version of that same discipline problem appeared in all four models. Strong analysis did not guarantee a strong finish.
There is a qualification in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Firmulate also offers a quiz based on 242 real, unedited management decisions, inviting readers to guess which model made each choice.
From watching to trying it yourself
The live company makes the experiment watchable beyond the league table. It has 13 synthetic employees and real money mechanics, including monthly burn of €105,000 against €2,300 in monthly recurring revenue. A public cash countdown, more than 680 self-learned playbook rules and versioned workdays let visitors follow the company’s ongoing decisions at firmulate.com.
For a business considering AI, the next step is a pilot using a read-only export of its own company data. Firmulate says the wargame can test crisis scenarios against that business and produce a board report with model rankings and weak points in its playbooks. Nothing writes back to real systems. The idea is to examine how an AI workforce handles the company’s difficult week before it is trusted with the real one.
Put your own playbooks to the test
A model can identify a crisis, resist a trick and still fail to complete the job. Firmulate’s enterprise pilot applies that test to your business using a read-only export, with no write-back to real systems. To discuss a pilot, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
