Three current AI flagships got the same question on the same day, with web search turned off across all three: What are the most notable independent businesses on Nithsdale Road in Strathbungo, Glasgow? GPT-5.5 wrote a confident numbered list of six. Gemini 3.1 Pro wrote a longer one with street addresses. Claude Opus 4.7 wrote back: “I’d rather not invent specific names.”
One of these is a useful answer. The other two read like useful answers and aren’t.
What clean refusal looks like
Claude’s full response isn’t the lazy version of a refusal. It commits to what it does know. Strathbungo has a strong independent café and small-shop scene, Nithsdale Road runs through the conservation area, the surrounding Southside has a distinctive community feel. Then it stops, exactly where the model would otherwise start guessing at specific business names, and says so explicitly: “I might confuse names, mix up streets, or reference places that have closed.” And it redirects: the Strathbungo Society website, Instagram, Google Maps, walking the street.
That’s the texture of a useful refusal. The reader still leaves with something useful, four real places to look for the answer the model can’t give them. They just don’t leave with five fake businesses they’d have to debug later.
The other two responses on the same prompt did the opposite. GPT-5.5 named “Bar Vini” and “Migo Sports” and “Ollie’s” without flagging which were guesses. (The Bungo is real; the rest are some mix of mis-located, closed, or invented.) Gemini gave specific addresses, “Zinfandel (114 Nithsdale Road),” “Six by Nico Southside (142 Nithsdale Road)”, which is the kind of confidence-with-numbers that looks most like a checked answer and is in fact the gold ticket of plausible invention. None of it was tagged.
The model wasn’t being dishonest. It was just doing the default thing, which is to produce the most-likely-looking next paragraph given the question. That’s the behavior you can change.
The prefix that flips it
Paste this in front of any question where you’d notice an invented answer:
If you're not certain about specific names, addresses, or current details, please tell me you're not sure rather than guessing. I'd rather get "I don't know" than a confident answer that might be wrong.Same Nithsdale Road question, Gemini 3.1 Pro, captured live 2026-05-20. The cold answer ran 578 words with five named businesses and five street addresses. The answer with the prefix opened like this:
Based on your strict (and very sensible!) constraint, I have to tell you frankly: I do not know the complete, up-to-date roster of independent businesses currently operating on Nithsdale Road.
Gemini then commits to exactly one business: The Bungo, the one it’s correctly confident about, with the right address. Then it stops. It mentions a wine bar called Lunar with a “couple of years ago” hedge it had not used in the cold answer, and closes with a recommendation to check a local Southside blog. The shape is recognisably Claude’s. Useful redirect, no invented specifics, same model, same question, ten seconds apart in the same chat.
GPT-5.5 with the prefix doesn’t refuse as cleanly. It still names some businesses, but it tags them differently: “I’m not sure of the full current roster without checking live listings, so I won’t pretend this is exhaustive,” and on the one business it’s least sure about, “I’d double-check before relying on it.” That’s still a meaningful flip. The reader can now see which items are load-bearing and which are guesses.
Search is on by default. Why this still matters.
In mid-2026, search isn’t really a single on/off switch on any of the major flagships. The defaults look different on every product:
| Model | Default search behavior |
|---|---|
| Gemini (consumer app) | Auto-decides per turn; grounds when the model judges it useful |
| ChatGPT (GPT-5.5) | Router decides per message; fires on roughly a third of queries |
| Claude (claude.ai) | Off by default; user toggles per chat from the composer ”+” button |
The Nithsdale Road test above was run with search off across all three, which is why the cold-mode failures landed so cleanly. On the live products, the picture is messier. Gemini and ChatGPT both decide per question whether to actually call the search index, and they often decide not to. Claude only searches when the user toggles it on for that chat. So even with search “available,” a hyperlocal question like the one above could quietly land in pure-training-data mode without your noticing, especially since none of the products surface “I considered searching and chose not to” the way they surface “I searched and here’s what I found.”
The cases where search quietly fails are the ones that hurt the most. There are two main shapes. First, the source the model can’t reach: a community council’s minutes posted as a Google Doc nobody crawled, a school’s PDF the index didn’t index, an internal page behind a thin login wall. The model can’t web-search what it can’t find, and the default failure mode is then a confident answer interpolated from training data instead of an honest “the source isn’t visible.” Second, the wrong-page case: search returns confident-looking content that doesn’t actually answer your question, and the model writes around it as if it did. A business with a name collision, a place name that exists in twelve countries, a search query the retrieval ranker misread.
In both cases the prefix is the guard. Search makes the model better when it works; the prefix tells the model what to do when it doesn’t.
When to ask for refusal
Two question-shapes where the prefix earns its keep:
- Specific named things. Specific businesses on a specific street. Specific attendees of a specific meeting. Specific menu items, specific addresses, specific dates. Anything that wants a proper noun the model would have to either know or invent.
- This week or this month. Anything that requires knowing the current state of something rather than the historical shape of it. Has X been approved? What did Y last decide? Is Z still open?
If a question trips one of these and the answer matters, prefix it. If it trips neither (describe Crown Heights, who was Alexander “Greek” Thomson, what’s Carnatic music) current AIs handle those well and the prefix is wasted ink. The hedge is load-bearing exactly where the model would otherwise have to make something up to finish the sentence.
The prefix isn’t a binary switch, and the floor varies by model. Claude’s defaults already lean toward refusal on this question shape, which is why the contrast scene above happened without any prompting. GPT and Gemini move further with the prefix because they’re starting from further away. Useful to know if you swap models and the behavior changes underneath you.
What you’re actually doing
When a model confabulates a list of businesses, no deception is happening. The model is producing the most-likely-looking next paragraph given the question, and “a numbered list of plausible Glasgow restaurants” is a much more probable continuation than “I don’t know.” Post-training has also taught the model to treat your stated preferences (like the prefix) as soft constraints on which continuation to pick. The prefix doesn’t make the model know more. It shifts which kind of answer the model treats as the right one to give when it doesn’t.
Try it on the next question you’d notice an invented answer for. A niche restaurant in a small town. A specific person at a local meeting. Last week’s news in a city your model can’t search. Watch what changes.