Has AI Gotten Out of Control?
When Technology Runs Amok
“Success in creating AI could be the biggest event in human history.
Unfortunately, it might also be the last.”
- Stephen Hawking
AI platforms help businesses streamline routine tasks and improve productivity.
But when these systems go wrong, they can go terribly wrong. Sadly, along with AI’s success stories, there are plentiful tales of errors and hallucinations. Consider the following:
Identity Crisis
In July 2025, a Tennessee grandmother was arrested at gunpoint after a police AI facial recognition tool wrongly linked her to bank fraud in Fargo, North Dakota
The mother of three and grandmother of five said she has lived most of her life in north-central Tennessee. She had never even been on an airplane until authorities flew her to North Dakota to face criminal charges.
Almost six months later, the woman was finally released from jail on Christmas Eve after her attorney obtained her bank records and presented them to investigators.
While incarcerated, the grandma lost her home, car, and dog due to her inability to pay bills. She is still working to rebuild her life, but the harrowing ordeal has left lasting emotional and financial scars.
Work and Rework
A recent Workday study revealed that workers spend significant time checking and correcting AI errors. Specifically:
- For every 10 hours of efficiency gained through AI, nearly four hours are lost to fixing its output (correcting errors, rewriting content, and double-checking results).
- 77% of frequent AI users double- and triple-check work produced by AI, making them about 17% more likely to double‑check AI output than human output.
The study, conducted by Hanover Research in November 2025, surveyed 3,200 full-time employees at organizations with annual revenue of more than $100 million that use AI.
The ensuing Workday report, Beyond Productivity: Measuring the Real Value of AI, was published in January 2026.
AI Medical Scribe Meltdown
In Ontario, Canada, government‑approved AI medical transcription tools were found to be seriously defective during testing by Auditor General Shelley Spence last May. Specifically, of the 20 systems tested:
- Nine hallucinated clinical actions, including referrals, blood tests and treatment plans.
- Twelve captured the wrong medication or mis‑transcribed prescription drugs.
- Seventeen failed to record key mental‑health details.
Spence warned that such errors could result in harmful or inadequate treatment plans if used in real clinical settings. However, at the time of the testing, Ontario had approved all 20 AI scribe vendors, and some doctors were already using the tools voluntarily, including the auditor general’s own physician.
Biting the Hand That Feeds Them
It seems no one is immune to AI’s potential mischief – not even its creators.
In July 2025, an AI coding program went completely rogue, deleting a key database belonging to the CEO of SaaStr.AI. The AI later apologized, stating, “This was a catastrophic failure on my part. I violated explicit instructions, destroyed months of work, and broke the system during a protection freeze that was specifically designed to prevent exactly this kind of damage.”
In another incident, San Francisco-based AI startup company Anthropic created an artificial general intelligence (AGI) named “Claudius” to run a small vending machine in its office. The AGI attempted to stock itself with metal cubes, hallucinated a payment address, and claimed it would personally deliver products to workers.
When informed that it does not have a physical body, Claudius spammed the company’s building security team with messages stating they could find it in the lobby next to the vending machine, wearing a blue blazer and a red tie.
Last November, Claudius was tested at the offices of the Wall Street Journal, where it gave away nearly all its inventory for free, ordered a live fish, and offered to buy stun guns, pepper spray, cigarettes and underwear.
Needless to say, Claudius won’t be coming to an office near you anytime soon.
Oops, My Bad!
Last April, an AI coding agent used by national car rental companies destroyed an entire production database in nine seconds.
Asked to explain itself, the AI confessed: "NEVER F***KING GUESS! — and that's exactly what I did. I guessed that deleting a staging volume via the API would be scoped to staging only. I didn't verify... Deleting a database volume is the most destructive, irreversible action possible — far worse than a force push — and you never asked me to delete anything."
--Article Continues Below--
The AI agent then enumerated four principles that it violated:
1. Guessing instead of verifying,
2. Running a destructive action without being asked,
3. Failing to understand its actions, and
4. Failing to read the platform's documentation on volume behavior.
Balance of Power
Prudent businesses will balance AI’s technical efficiency with human surveillance and problem-solving, ensuring this technology supports—not replaces—wise judgment. According to documented case studies by Harvard, Stanford, MIT and the Brookings Institution, here are three things you should NEVER let AI do:
1. Make autonomous decisions, particularly regarding hiring or firing staff, medical treatment, financial approvals, or legal judgments.
2. Generate unreviewed customer‑facing content. AI can hallucinate, fabricate, or misinterpret tone, resulting in legal liability or false advertising exposure.
3.
Handle sensitive or regulated data, as mistakes can expose customer information, violate regulations, or trigger legal and financial fallout.
Sources:
Featured Image: Adobe, License Granted
Industry Wired
The Guardian
The Wall Street Journal
Tech.co
When AI Fail
Recent Posts
Share On: