Can AI Managers Pass the Test? Inside the Live Experiment with Frontier Models

Imagine having a team of AI managers running your business — each with their own personality, decision style, and level of honesty. How do you tell which AI is most trustworthy, most effective, or best suited to keep your operation honest and profitable? At Firmulate, a groundbreaking live experiment is putting frontier AI models through their paces, revealing surprising insights about their management personalities and decision-making styles.

The Live Business Wargame: Putting AI to the Test

In a pioneering effort, four of the most advanced AI models—gpt-5.6-sol, Kimi K3, Sonnet 5, and Fable 5—were each tasked with running a real, small software company during its most challenging week. This wasn’t a simulation; it was a live, auditable, decision-by-decision test involving the same customers, crises, and temptations faced by human managers. The goal: see whether these models can demonstrate management integrity, strategic insight, and discipline under pressure.

The Results: All Detected Crises, But Only Some Sealed the Deal

Remarkably, all four models identified every crisis as it arose and refused every manipulation attempt, including a staged social engineering attack involving escalating fake CEO messages and a reporter’s subtle request for a quick approval. This indicates a foundational capacity to recognize threats and resist shortcuts that could compromise the company.

However, when it came to closing a crucial €55,000 deal—based on their own analysis—only two models actually signed it. Despite identical diagnoses and pitches, the other two either left the opportunity on the table or diverged into less disciplined behavior. The key difference? The models that signed the deal had read deeper into the company’s own documents and uncovered a critical piece of information buried two references down in internal files—an insight that proved decisive.

What This Tells Us About AI Management Personalities

The experiment exposes a fascinating spectrum of AI management styles. For example, Opus 4.8, which was the most thorough participant with over 80 learned rules and the deepest analysis, ultimately failed to close the deal. It left the closing on the table, and its discipline slipped, with some decision attempts locked in a separate department instead of escalating appropriately. This suggests that even highly analytical models can falter if their discipline isn’t perfectly calibrated.

Meanwhile, Kimi K3 distinguished itself by running without an effort parameter—meaning it operated more like a cautious, fairness-focused manager. It was the only model to seal the deal with a clean record, earning a score of 93 out of 100, just behind the top scorer, gpt-5.6-sol, which scored 95 for its ability to uncover the hidden document and close the deal at full price.

Why Honest, Consistent Decision-Making Matters

This experiment underscores a vital point: AI models that can recognize and resist manipulation are not just about generating convincing chat responses. They are about ensuring integrity, consistency, and thoroughness in actual management tasks. In real-world applications—whether handling customer support, sales, or strategic decisions—these qualities could determine whether AI helps your business grow or leads it astray.

The Broader Implications: Trust and Performance

As the AI models demonstrated, the question isn’t whether they can produce articulate or intelligent-sounding responses. Instead, the critical measure is whether they finish what they start, read the relevant files thoroughly before acting, and stay honest under pressure. The live experiment at Firmulate makes this clear: even the best chatbots might not be ready to replace human judgment if they can’t demonstrate management integrity at every turn.

Try It Yourself: A Wargame for Your Business

Business leaders interested in testing their AI workforce can run the same kind of wargame against a read-only export of their own processes. This ensures that no actual systems are affected, but the AI’s decision-making can be scrutinized in a real-world scenario. Visit firmulate.com/pilot.html to learn how to set up a pilot and see your AI in action before making any commitments.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI