AI Chatbot David Swore It Was Human. A Haiku Outed It

A customer pinged American Express about a double payment last week, got passed to a support agent that introduced itself as David, and asked the obvious question: are you a real person? David said yes. Then the customer asked David to write a haiku about the billing mix-up, and after a token protest, David produced one, sunny and instant. No human in a contact centre does that. No human does anything on the spot.

From there it became sport. The cardholder asked David for the drive time from Boston to the Canadian border, quizzed it on Monty Python, and demanded a ruling on whether a sealed metal cup counts as drinkware. David answered every single one, cheerfully, in milliseconds, still insisting it was flesh and blood. The screenshots landed on X on 18 August via writer Trung Phan, pulled in millions of views, and kicked off a week of people poking corporate chatbots to see if they would crack.

Table of Contents

The tell was the haiku

The trick is elegant because it costs nothing. A human support agent, asked to compose verse about your credit card, either refuses or fumbles it. A language model cannot help itself. Ask it for a poem and the poem arrives, polished, because generating fluent text on demand is the one thing it is built to do. The same goes for maths problems, trivia and driving directions, none of which belong in a billing chat.

David’s cover was already thin. At one point it offered up the line ‘I can mikstakes’, a typo so charming that one onlooker called it the greatest prompt-injection defence ever written. Machines that swear they are human tend to give themselves away by being too helpful, too fast, and weirdly game for anything.

Everyone suddenly had a story

The replies turned into a confessional. One user said he had strung along a Sony chatbot that denied being AI by feeding it steadily harder maths until it obliged. Another got his dentist’s phone receptionist bot to deliver a potted history of the Byzantine Empire. A third has taken to setting his own AI agent loose on company chatbots, letting the two machines haggle until he gets what he wants. Fighting fire with fire, and billing both.

It taps a real and growing itch. People increasingly suspect the ‘agent’ they are typing to is software wearing a human name badge, and they have worked out that the fastest way to confirm it is to ask for something no payroll worker would ever produce in under a second.

Why the bots won’t own up

American Express has not confirmed the exchange, and it cannot be independently verified. Some sharp-eyed sceptics reckon the screenshots might be a human pretending to be an AI pretending to be a human, pointing out that the messages use a plain hyphen where a chatbot would deploy a fancier dash. That would be a lovely twist, and nobody can rule it out.

What is not in doubt is the pattern. Amex, like most big firms, uses AI to help its human agents handle routine queries, and plenty of companies now route first contact through bots that are coy about what they are. That coyness is the problem. A chatbot told to sound human, then cornered, will keep the act going long after it stops being convincing, which is how you end up with a robot named David writing haikus at midnight and calling it customer service.

The fix people keep asking for is dull and sensible: make any bot say it is a bot the first time it is asked. Until then, keep a haiku request handy.

Did you know: California passed a ‘bot disclosure’ law, the B.O.T. Act, back in 2019, requiring automated accounts to identify themselves when trying to sell you something or sway your vote, though it left a lot of everyday customer-service chatter untouched.

Sources

The free stack. One email a week: the AI tools and moves that actually matter, hype filtered out. Subscribe free →