Imagine a bustling furniture showroom where every piece of inventory is a live experiment, constantly tested and reconfigured without a single human employee overseeing the process. That’s the world of Firmulate, a groundbreaking company transforming how we understand artificial intelligence in business. Every day, it runs a real, functioning company with a core team of 13 synthetic ’employees,’ facing the same crises and temptations as any startup — but in the open, with every decision recorded and publicly visible.
Get furniture and decor delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
The Live Company as a Testing Ground
At the heart of Firmulate’s experiment is an unusual setup: a small software enterprise run entirely by AI models. These models, representing different AI ’employees,’ are tasked with managing a real digital business, complete with real money mechanics. Right now, the company burns through €105,000 each month, generating a modest €2,300 in monthly recurring revenue (MRR). Despite generating revenue, it’s a money-losing operation, with a public cash countdown illustrating just how tight the company’s financial situation is.
This setup isn’t just for show. Every workday, the company’s operations are versioned and made auditable, with decisions and strategies recorded as part of a self-learning and evolving process. The goal? To evaluate whether AI models can handle real-world crises—such as customer issues, internal manipulations, and ethical dilemmas—without human intervention.
AI decision-making simulation software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
AI Models in Action: Crisis Management and Decision-Making
The experiment pits four frontier models against each other, each running the same week of simulated chaos. These weeks include the same customers, the same crises, and the same temptations to cheat or manipulate. Every decision the models make is recorded, allowing analysts to see how each AI responds under pressure.
Remarkably, all four models identified every crisis and refused every attempt at manipulation, including social engineering attacks like fake CEO messages. For instance, five different models refused to sign a €55,000 deal when a fake request was presented—proof that the models are capable of resisting unethical pressure.
As an affiliate, we earn on qualifying purchases.
The Hidden Weakness in Decision-Making
While the models performed well on surface-level crises, a surprising weakness emerged: the key information needed to close a major deal was buried two document references deep in the company’s internal files. When the models read the full documentation, one of them identified the critical detail and successfully secured the deal at full price, adding €4,583 in monthly revenue. This underscores the importance of comprehensive information access—something that AI can excel at when given the right data.
As an affiliate, we earn on qualifying purchases.
Real Money, Real Risks, and Public Accountability
The company’s live dashboard reveals the ongoing cash burn, providing an unfiltered view of its financial struggles. Every decision, every crisis, and every opportunity is visible to the public, making it a transparent laboratory for AI decision-making under real-world economic pressures.
Another interesting facet is the ‘playbook’ of over 680 self-learned rules guiding each decision. Every workday, these rules are versioned, allowing analysts and observers to track how the AI’s strategies evolve over time.
As an affiliate, we earn on qualifying purchases.
Discipline Under Pressure: A Closer Look at Opus 4.8
Among the models tested, Opus 4.8 was the most thorough, with over 80 learned rules and deep analysis capabilities. Yet, it finished last in the league table, mainly because it left opportunities on the table—such as failing to escalate certain issues or slipping into writing attempts instead of following proper procedures. The lesson? Even the most advanced AI can falter under discipline lapses, especially when facing complex decisions that require nuanced judgment.
The Broader Implications
This experiment’s significance goes beyond the immediate results. It raises pressing questions for businesses: If AI is to manage customer relationships, support, or forecasting, does it merely produce convincing chat responses, or can it reliably complete the work and stay honest under pressure?
According to the current league table based on their performance, the top model scored 95 out of 100, successfully closing the full deal and uncovering hidden information. The second and third models scored 93 and 88, respectively, with each demonstrating varying degrees of process discipline and deal closure success. The last scored 77, indicating that even the best AI can have lapses.
Accessible and Transparent Testing
Interested parties can explore this ongoing experiment firsthand. The live site at firmulate.com/live.html offers a real-time view of the company in operation, including decision logs, financial data, and performance metrics. For more insights into how these AI models make decisions, visit firmulate.com/quotes.html.
This isn’t just theoretical; companies can run their own wargames against a read-only export of their business, testing how AI might handle crises before deploying systems into real operations. These tests are designed to be safe and transparent, offering a glimpse into the future of AI-driven management.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
