Users who get answers from AI search
Some users get an AI-generated answer from a search engine or chatbot before, or instead of, visiting your service.
Who these users are
These are people who ask a search engine or AI chatbot a question and get a direct answer, such as an AI summary with citations. They may never click through to your service.
Users may see AI answers in different places, including:
| Term | What it is |
|---|---|
| AI summaries in search results | A short AI-generated summary at the top of some search results, with links to sources. Examples include AI Overviews in Google and Ecosia, and Copilot answers in Bing. The search engine decides when to show one. |
| AI search modes | A separate, conversational mode in a search engine for questions that need more reasoning or comparison. It gives a longer answer with links, and users can ask follow-up questions. Examples include Google AI Mode, Copilot Search in Bing and Ecosia AI Chat. |
| AI chatbots with search | Tools such as ChatGPT, Copilot, Claude, Gemini and Perplexity that search the web to answer a question. |
For example, a user might ask:
- “What are the rules for [activity] at [location]?”
- “How do I apply for [permit or data] from [organisation]?”
What changes for these users
Search is shifting from ranked links to cited answers. Before, users saw a list of links and chose which source to open. Now they may get a direct answer that feels complete, even when it is based on crawlable web pages rather than your live data or journey steps.
AI search answers are generated by large language models (LLMs). They work from content that search crawlers have already indexed, such as public web pages, reports and PDFs, and from what the model learned in training.
Search crawlers and LLMs do not move through interactive journeys. They cannot answer a question, submit it, see the next question and then view tailored results. Anything that only appears partway through your service, such as a call to action after triage questions or results for a specific location, is invisible to them.
When your authoritative route or data is only available inside an interactive journey, the LLM fills the gap from nearby evidence it can see. That might be historic guidance, third-party PDFs, older reports or steps from a related process.
Some AI tools use individual sections of a page rather than the whole page. If a section only makes sense alongside the content around it, the answer built from it may be incomplete or wrong.
GDS has reported a fall in user traffic to GOV.UK as more answers are given within search results. When an answer appears before the click, the missing visit may never show up in your analytics.
Risks to users and your service
- False confidence: an answer looks authoritative but is not based on your live data or official result.
- Wrong route: users are sent to historic or slower routes, creating manual work.
- Invented steps: users expect questions or requirements that are not in your service.
- Hidden demand: users get an answer without visiting, so you cannot see or support them.
- Support demand: users contact you to query or correct information they were given elsewhere.
What you should do
Monitor what AI tools are telling users
- Run prompt checks on your highest-risk user questions.
- Ask users in feedback and research whether they arrived via an AI answer.
Work with your performance analyst
- Use analytics alongside search console data, so you can compare impressions with clicks and spot answers being given before the click.
- Use the Generative AI performance report in Google Search Console. It shows how often your pages appear in AI Overviews and AI Mode, by page, country, device and date. It does not show clicks or the questions people asked.
- Use SEO and AI visibility tools to monitor citations and prompt visibility at scale.
Measure your visibility in AI answers
Traffic still matters, but it no longer tells the whole story. When users get an answer without clicking, measure whether your content is present and trusted in that answer.
| Measure | What it tells you |
|---|---|
| AI impressions | How often your content appears in AI-generated answers. Were you present in the answer? |
| AI citations | How often AI tools cite your content as a source. Did the AI trust your content enough to use it? |
| AI share of voice | Your share of citations compared with other sources on the same topic. Are users being given your answer, or someone else's? |
| AI referrals | Visits that arrive at your service from AI tools. How much of your user traffic comes from AI answers? |
Google Analytics now groups visits from recognised AI assistants, such as ChatGPT, Gemini and Claude, into an ‘AI Assistant’ channel. You do not need to set it up. Google has not published the full list of AI tools it recognises, and visits that arrive without referral information still appear as direct traffic. If you need to track AI tools the channel does not cover, create a custom channel group.
Make the right route visible
- Make sure your start page describes all main routes, including ones that sit behind triage questions or later steps.
- Say plainly which service is the authoritative source for the data or decision.
- Work with the BAU content team on your GOV.UK start page. They manage GOV.UK start pages, as Designing for different content types explains.
- Build pages with progressive enhancement, so routes and key content are in the HTML and do not rely on JavaScript to render. Google renders JavaScript, but most other AI crawlers do not.
- Review other crawl blockers, such as content that only appears after a form is submitted.
- Consider whether important calls to action are hidden too far into the journey for people or crawlers to find.
Write content that AI search can quote accurately
Work with your content designer to apply these practices, using the Defra style guide. Good content design works for people and AI tools. These practices make it more likely that an AI answer quotes your content correctly. Google says you do not need to rewrite or ‘chunk’ content specifically for its AI features, so apply them when you write or review content, rather than restructuring pages just for AI.
- Make each section self-contained. Repeat the name of the service or subject instead of relying on ‘it’ or ‘this’, avoid ‘see above’, and keep everything about one topic under one heading.
- Put the answer first. Give the key answer in the first sentence after each heading, then add detail and evidence.
- Use descriptive headings in your users' words. Each heading should say what its section is about, using the terms users search with.
- Write processes as numbered steps. Steps are easy for people to follow and for AI tools to quote, which reduces the chance of an invented journey.
- Answer the questions users ask. People now ask longer, conversational questions. Structure content so each common question has a clear, direct answer.
- Write clear page titles and meta descriptions that describe what the page is for.
- Show when content was last updated. On your own service domain, also say which organisation runs the service.
- Consider structured data (schema markup) on pages you control, to tell search engines what the page is and who provides it. Google says it is not needed for its AI features, but it still helps with rich results, and other tools may use it.
Reduce ambiguity in older content
- Find guidance pages, PDFs and reports that describe previous routes or give outdated answers.
- Update, archive or add clear warnings to them.
- Use content audit tools, such as CART, to find pages that could mislead AI tools.
Check AI tools can find your official content
If AI crawlers are blocked, AI tools cannot cite your official content. They will use other sources instead. Ask your developers to check that crawlers are not all blocked by default, and that your site has not opted out of Google's AI features.
Guide AI tools to the right source
- Where you publish reliable APIs, feeds or structured data, say what they can and cannot be used for.
- Decide whether AI tools should be discouraged from summarising your answer, or guided to the right structured source with clear caveats and onward routes. People already accept AI answers for things like weather and bus times because they draw on structured data.
How to test
Run a prompt check
- List the questions users most often ask about your service, and those where a wrong answer would cause the most harm.
- Ask each question in the AI tools your users are likely to use, such as Google AI Overviews and AI Mode, ChatGPT, Copilot, Gemini and Ecosia.
- Compare each answer with your live service.
- Record what you find in a shared log.
- Repeat regularly, and after you change content or journeys.
A manual check shows you what a few answers look like. To see how often AI tools get your service wrong, and whether that is improving, also test at scale.
For each answer, record:
| What to record | Example |
|---|---|
| Accuracy | Was the information or result correct for the user's situation? |
| Route recommended | Did it point to the online service, an old route or an invented one? |
| Caveats | Did it say the answer should be checked against the official service? |
| Citations | Which sources did it use, and were they current and official? |
Test prompts at scale
AI answers vary between tools, wording, locations and days. A handful of manual checks cannot show how often users get a wrong answer. Testing at scale can.
- Choose a small set of questions where a wrong answer would have the highest risk or impact. Use search console data, support contacts and research findings to see how users word them.
- Use SEO and AI visibility tools to run your prompt set across several AI tools on a regular schedule.
- Track how often your service is cited, which of your pages are cited, and which other sources appear instead.
- Track trends over time, such as your AI share of voice, the share of answers that recommend the right route, and the outdated pages cited most often.
- Investigate the worst results with manual prompt checks, then fix the content or journey behind them.
- Compare trends with your user traffic, AI referrals and support contacts, to see whether your changes are reducing failure demand.
Check that any tool you use is approved for use in Defra, and do not include personal data in prompts. Read Choosing a tool and Using data with AI in the AI digital toolkit.