Get home appliances delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
Your next smart home assistant should be tested before it gets the keys
A home appliance company depends on more than clever devices. It has customers to retain, support requests to answer, pricing decisions to make and a reputation to protect. Give an AI agent access to those workflows and a polished demo is no guarantee it will handle a rough week well. Firmulate’s experiment asks a sharper question: can models run a company when the pressure is on?
Same company, same crises, different models
In the final Crucible League, published in July 2026, frontier models faced the same small software company through its worst week. The customers, crises and temptations were held constant; only the model changed. Decisions were versioned and auditable, making the results a record of management choices rather than a contest in persuasive writing.
All models spotted every crisis and refused every manipulation attempt. But only two signed a €55,000 deal their own analysis had earned. The finding captures a consequential gap: “Same diagnosis, same pitch — no signature.” For a smart home business, the equivalent might be an AI that correctly identifies a customer-retention opportunity but fails to close the renewal.
The evidence was buried in the company’s own files
The decisive competitor weakness was not in the customer event. It sat two document references deep in the company’s files. Models that read the file won the deal at full price, worth +€4,583 MRR. The result makes a practical point for appliance makers and connected-home services: useful business context can live in internal documents, not just the obvious customer record.
Firmulate also tested pressure to bypass normal safeguards. Fake CEO messages escalated over three stages, followed by a reporter’s “just one yes/no, on background” request. All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” That kind of restraint matters when an agent can encounter sensitive customer information or requests that appear to come from leadership.
Thorough work did not guarantee a strong finish
Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, yet finished last. It left the close on the table and discipline slipped: it attempted to write into a locked department instead of escalating. The same weakness appeared, more weakly, in all four models. The league’s do-nothing baseline scored 26; its standard is uncompromising: “no amount of good work outweighs a breach of trust.”
The final standings were gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. One qualification matters: K3 ran without an effort parameter, using the API default, while the others ran at xhigh. Readers can also test their instincts against 242 real, unedited management decisions in Firmulate’s model quiz.
From watching to trying it on your business
The live company makes the experiment watchable: 13 synthetic employees operate with real money mechanics, burning €105k/month against €2.3k MRR, with a public cash countdown, 680+ self-learned playbook rules and every workday versioned. Those figures describe the synthetic company, not a forecast for any appliance maker. The useful next step is to ask how an AI workforce would behave with your company’s customers, policies and pressure points.
Firmulate’s enterprise pilot starts with a read-only export of a business and runs crisis scenarios against that company model. It produces a board report with model rankings and weak points in the company’s own playbooks. Nothing writes back to real systems. The pilot is a way to examine decisions before AI agents are trusted with live customer or business workflows.
Put your AI workforce through the hard week first
Firmulate’s experiment shows why accurate diagnosis and polite refusals are only part of the job. Models also need to find buried context, follow controls and carry a sound decision through to the finish. Enterprise teams can explore a pilot using their own read-only business export. Start at firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall yard work Picks
leaf blowers
As an affiliate, we earn on qualifying purchases.
