The world doesn’t have a misinformation problem; it has a synthetic reality problem | OPINION
AI fraud represents a new frontier in business security, and it necessitates a fundamental rebuild of verification layers for high-stakes processes
A UK engineering firm, Arup, lost $25 million despite having a perfect security program because a finance worker was deceived by an AI-generated video call impersonating his CFO and senior colleagues, highlighting a fundamental collapse in the assumption that visual and auditory verification is sufficient proof of identity. This incident, termed "synthetic reality" rather than misinformation, underscores the failure of traditional verification methods as AI can perfectly mimic known individuals, rendering security systems that focus on technical breaches insufficient. The article proposes solutions including rebuilding verification layers with additional checks like call-backs or pre-agreed questions, integrating AI detection into workflows, and proactively addressing liability before regulations mandate it, urging organizations to immediately map sensitive processes, implement multi-channel confirmation for significant transactions, educate staff about this new threat, and question vendors about content provenance to safeguard against future attacks.
A UK engineering firm, Arup, lost $25 million despite having a perfect security program because a finance worker was deceived by an AI-generated video call impersonating his CFO and senior colleagues, highlighting a fundamental collapse in the assumption that visual and auditory verification is sufficient proof of identity. This incident, termed "synthetic reality" rather than misinformation, underscores the failure of traditional verification methods as AI can perfectly mimic known individuals, rendering security systems that focus on technical breaches insufficient. The article proposes solutions including rebuilding verification layers with additional checks like call-backs or pre-agreed questions, integrating AI detection into workflows, and proactively addressing liability before regulations mandate it, urging organizations to immediately map sensitive processes, implement multi-channel confirmation for significant transactions, educate staff about this new threat, and question vendors about content provenance to safeguard against future attacks.
A UK engineering firm, Arup, lost $25 million despite having a perfect security program because a finance worker was deceived by an AI-generated video call impersonating his CFO and senior colleagues, highlighting a fundamental collapse in the assumption that visual and auditory verification is sufficient proof of identity. This incident, termed "synthetic reality" rather than misinformation, underscores the failure of traditional verification methods as AI can perfectly mimic known individuals, rendering security systems that focus on technical breaches insufficient. The article proposes solutions including rebuilding verification layers with additional checks like call-backs or pre-agreed questions, integrating AI detection into workflows, and proactively addressing liability before regulations mandate it, urging organizations to immediately map sensitive processes, implement multi-channel confirmation for significant transactions, educate staff about this new threat, and question vendors about content provenance to safeguard against future attacks.
Arup’s security program was perfect. The attack never touched a single system of the UK engineering firm. They still lost $25 million. A finance worker sat down for a video call with his CFO and senior colleagues, faces they recognised, voices they knew. They’d been suspicious of the original email. The video call resolved his doubts. They authorised fifteen wire transfers totalling $25 million.
Every person on that call was AI-generated. The security team confirmed that Arup’s systems had not been touched: no malware, no breach, no compromised credentials. Every firewall was working, every access control was in place, and the entire security program was operating exactly as designed. And none of it mattered. Because the attack never came near the security infrastructure. It walked straight through the front door wearing the CFO’s face.
The security program did everything right. That is not a security failure. That is the collapse of an assumption.
‘We’re using the wrong word’
Misinformation implies someone got something wrong. The solution to misinformation is verification: fact-checkers, editorial standards, correction processes.
What happened at Arup was not misinformation. Nothing was fabricated in the way we usually mean that word. The CFO was real. The company was real. The video call looked and felt exactly like every other call the finance worker had been on. The only thing that was false was who was on the other end of it, and that is a completely different kind of problem.
Synthetic reality doesn’t get facts wrong. It manufactures the context those facts live in. You can’t fact-check your way out of a world where the face you’re looking at was built to be indistinguishable from the real thing.
Think about what every court, every bank, every government, every company has always used to verify identity. A face. A voice. Proof that the person in front of you or on the call is who they say they are. Nobody ever wrote that down as a security assumption because it never needed to be written down. It just was.
That assumption is now gone. Not weakened. Gone. The Arup employee was not careless. The video call was his due diligence. It failed because nobody had told him the rules had changed.
We built the entire verification infrastructure of the modern world on it. And nobody wrote a memo when it stopped working.
What actually works right now
Rebuild the verification layer for high-stakes processes. Start with your most sensitive processes, the ones where a video or phone call from the right person can move money or change something irreversible. Add a second layer: a call-back on a known number, a question only the real person could answer, a second approver not on the original call. None of this is complicated. It just hasn’t been done yet.
Build detection into the workflow: Twenty years ago, nobody asked employees to manually check every email for spam. We built the filter into the system and forgot about it. AI detection needs to work the same way quietly, before content reaches whoever is deciding. The technology is there.
Rethink liability before regulators do it for you: Right now, the organisation that loses $25 million bears the entire loss. The platform that hosted the call bears none. That will change; the EU AI Act is already moving in this direction. Organisations not thinking about liability exposure now are carrying a risk they haven’t even named.
Five things you can do this week
Change what you call it internally. When you call it misinformation, your team thinks about fact-checkers. When you call it synthetic reality, they start thinking about verification infrastructure. The framing decides the solution.
Map every process in your organisation that a phone call or video from the right person can set in motion: approvals, transfers, decisions. Write it down today, not next quarter. The list will be longer than you expect and shorter than it should be.
Pick a number; any wire transfer above that threshold requires a second confirmation through a separate channel. Not a reply to the same email. Not the same call. Something entirely different. This is a policy decision, not a technology purchase.
Tell your finance and operations teams about Arup. Not to scare them, but to show them that following the rules perfectly is no longer enough if the rules were written before this threat existed.
Ask every significant vendor one question: What are you doing about content provenance? Watch how they answer. The ones with a clear response are thinking ahead. The ones who go quiet are not.
The attack succeeded because it exploited an assumption nobody had ever written down that seeing and hearing someone is proof they are who they appear to be.
Every organisation carrying that assumption is carrying a liability they haven't priced. The only question left is whether you fix it before something happens or after. Nobody at Arup thought they needed to ask that question either.
The author is a cybersecurity executive.
The opinions expressed in this article are those of the author/writer and do not purport to reflect the opinions or views of THE WEEK.