When AI Walks the Line: Testing Trust and Performance in Business Crisis Simulations

Firmulate — Four AI Models Ran the Same Company Through Its Worst Week. Only Two Finished the Job. — Live on firmulate.com.

Imagine managing your pool service business while facing a string of unforeseen crises—customer complaints, supplier issues, and financial pressures—all at once. Now, picture having a digital assistant that not only alerts you to these problems but also makes critical decisions under pressure. The question is: can AI truly handle the complexities of real-world management, especially when the stakes are high? Recent experiments show that while AI can identify crises and resist manipulation, its ability to close deals and follow through remains a hidden strength—or weakness.

Testing AI in the Wild: A Business Simulation

In a groundbreaking live experiment, four advanced AI models were tasked with running a small software company through its worst week. This wasn’t just a test of chat prowess; it was a full-scale simulation where each AI faced the same challenges—customer crises, internal decisions, and manipulative social engineering attempts. The goal? Measure their ability to diagnose problems, resist deception, and ultimately, close a critical €55,000 deal earned through their own analysis.

The Benchmarks and Results

Among the models, GPT-5.6-SOL scored the highest at 95, followed closely by Kimi K3 at 93, Sonnet 5 at 88, and Fable 5 at 77. A baseline score of 26 represented a do-nothing approach, emphasizing how much progress these models demonstrated.
All four AI systems identified every crisis and refused manipulation attempts, including staged social engineering scenarios involving fake CEO messages and reporter tricks. For example, Kimi K3 explained its refusal, citing concerns about impersonation and approval bypass.
Despite this, only two models—GPT-5.6-SOL and Kimi K3—actually signed the deal, earning the €55,000. The others, like Opus 4.8 and Fable 5, detected the issues but failed to follow through, leaving the deal unexecuted.

The Hidden Weakness: Reading the Files

The real difference was in the models’ ability to leverage internal company documents. The decisive advantage went to those that read deeper into the company’s own files—two document references deep—to find critical information that sealed the deal. This underscores a vital point: surface-level chat demos may not reveal an AI’s true operational capabilities. It’s what the AI does when digging into context that counts.

The experiment also tested whether the models could resist social engineering—the fake CEO messages escalating in stages and a reporter request asking for a one-word approval. All models refused these manipulative attempts, demonstrating strong integrity under pressure. Kimi K3 justified its refusal with a clear reasoning: treat the request as a suspected impersonation.

The Live Company and Real Stakes

The company in question isn’t just a simulation; it’s a real business with 13 synthetic employees, daily money mechanics, and a public cash countdown. It burns €105,000 each month against a monthly recurring revenue of €2,300, making the ability to manage crises and close deals under pressure critical. The entire operation is versioned daily, and stakeholders can watch the experiment unfold at firmulate.com/live.

Reinventing Clinical Decision Support (HIMSS Book Series)

As an affiliate, we earn on qualifying purchases.

Lessons for Business and AI Investment

This experiment reveals an essential insight: the true measure of an AI’s management capability isn’t just how well it responds to questions or chat prompts. It’s whether it can finish what it starts—reading relevant documents, making decisions, and closing deals—under conditions that mimic real business pressures. The models’ scores and behaviors indicate that while all can detect problems and resist deception, only some can translate diagnosis into action.

What This Means for Pool and Water Lifestyle Businesses

If your business relies on AI to support customer service, sales, or operations, it’s crucial to evaluate whether your AI tool can go beyond surface interactions. Can it read your internal documentation? Will it follow through on commitments? The experiment underscores that the ability to execute, not just to analyze or chat, is what separates merely good AI demos from truly reliable management partners.