Learning how to get cited by AI is an operator job: open the crawler gates, rebuild the pages worth quoting, then track the links that appear.
Book a free SEO strategy session →
This runbook gives you the order I'd work in, the exact robots.txt lines, and the traps that cost teams a week.
If you'd rather hand the whole engagement to a team, book a free SEO strategy session with Goldie Agency and bring this list with you.
Nobody can promise you a citation, and Google says plainly on its hiring-an-SEO page that no one can guarantee a #1 ranking.
What you can control is access, quality and measurement.
Step 0: load the official rules before you touch a file
Start with the platform docs, not with forum threads.
Google's AI features page says AI Overviews and AI Mode need nothing beyond normal SEO best practices.
Its generative AI optimization guide adds that AEO and GEO work is still SEO from Google's side.
The same guide says Google's systems use retrieval-augmented generation, pulling fresh pages from the Search index and linking the ones that support the answer.
So for Google, a citation starts with a page that is indexed, crawlable and allowed a snippet.
For ChatGPT search, Perplexity and Claude, it starts with a separate search crawler that has to reach your page.
Save the four crawler docs in one shared folder: OpenAI's crawler overview, Perplexity's bot page, Anthropic's crawler article and Google's common crawlers list.
Step 1: confirm Google can index and snippet the page
- Run URL Inspection in Search Console on your ten most important pages.
- Check that each page reports "URL is on Google" with the canonical you expect.
- Search the page source for
noindex,nosnippetandmax-snippet:0. - Look for
data-nosnippetwrapped around your main answer paragraph. - Repeat the check for the X-Robots-Tag header with a header viewer, because PDFs and some CDNs set it there.
Google documents every one of those controls on its robots meta tag page.
A page that can't show a snippet can't be a supporting link either, so this step comes first.
Step 2: audit robots.txt for the three AI search bots
Open yourdomain.com/robots.txt and read every group, top to bottom.
You are looking for any rule that blocks OAI-SearchBot, PerplexityBot or Claude-SearchBot, including a blanket User-agent: * with Disallow: /.
Book a free SEO strategy session →
A clean starting block for the search crawlers looks like this:
User-agent: OAI-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
OpenAI says sites that block OAI-SearchBot won't be shown in ChatGPT search answers, though they can still appear as navigational links.
Perplexity says PerplexityBot is the crawler that surfaces and links websites in its results, and it recommends allowing it.
Anthropic says blocking Claude-SearchBot stops it indexing your content for search and may reduce your visibility in user search results.
Keep private paths such as /cart/ or /account/ disallowed inside each group if you need to.
🔥 Want to run this runbook yourself, with someone checking your work? Inside the AI Profit Boardroom you learn AI SEO yourself: step-by-step video trainings, the installable Agent OS and four coaching calls a week where you can share your robots.txt and page rewrites. → Join the AI Profit Boardroom
Step 3: check the firewall and CDN, not just robots.txt
A perfect robots.txt means nothing if a bot-fight setting returns a challenge page.
- Open your CDN or host dashboard and list every rule that touches automated traffic.
- Find any "block AI bots" or "challenge all bots" toggle and note what it covers.
- Allow the three search crawlers by their documented user agents, and verify them the way each vendor's docs describe.
- Check your server logs a few days later for requests from each bot and the status code it received.
Anthropic advises against IP-blocking its bots, because a blocked bot can't even read your robots.txt rules.
Step 4: treat training crawlers as a separate decision
Search inclusion and model training are different switches on every platform.
| Platform | Search crawler | Training crawler | User-triggered fetcher |
|---|---|---|---|
| OpenAI | OAI-SearchBot | GPTBot | ChatGPT-User |
| Perplexity | PerplexityBot | Not used for foundation model training (per Perplexity) | Perplexity-User |
| Anthropic | Claude-SearchBot | ClaudeBot | Claude-User |
| Googlebot | Google-Extended (separate control) | Not covered here |
OpenAI states that GPTBot is for training, so blocking it does not take you out of ChatGPT search.
If the business doesn't want its pages used for training, block GPTBot and ClaudeBot in their own groups and leave the search crawlers open.
Book a free SEO strategy session →
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-triggered fetchers behave differently: OpenAI says robots.txt may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores it because a person asked for the page.
📞 Not sure which bots your firewall is blocking right now? Book a free SEO strategy session and the Goldie Agency team will read your robots.txt, CDN rules and top pages with you on the call. → Get my free strategy session
Step 5: time the recheck so you don't chase ghosts
OpenAI says its search systems take about 24 hours to reflect a robots.txt change.
Log the time of every robots.txt edit in your change sheet.
Don't test ChatGPT search for that page until a full day has passed.
For Perplexity and Claude no timing is published, so recheck weekly rather than hourly.
Step 6: rebuild the pages worth citing around first-hand evidence
Once the bots can read you, the page still has to beat everyone else's version of the same answer.
Google advises a unique point of view, such as first-hand experience, and says summaries of existing content add little.
It calls generic tip lists "commodity content" and asks for people-first pages instead.
Here's the rewrite order I'd use on a small service business site:
- Pick the five pages that answer the questions customers ask before they buy.
- Put the direct answer in the first two sentences of each page.
- Add something only this business has: job photos, a process the team follows, a real mistake and the fix.
- Name the person who wrote or reviewed it and link to their profile page.
- Delete any paragraph that restates what every competitor already says.
Google's helpful content guidance is the checklist to score each rewrite against.
Step 7: format each page so an answer engine can lift a clean fact
Bing's AI Performance announcement advises clear headings, tables and FAQ sections so AI systems can reference information accurately.
It also says to support claims with examples, data and cited sources.
- One H2 per question a buyer asks.
- One table that compares options, specs or steps.
- A four-to-six question FAQ written from real customer emails.
- Outbound links to the primary source behind each claim.
Book a free SEO strategy session →
No single schema type or file causes a citation, so don't burn a sprint on markup alone.
Step 8: notify engines with IndexNow after every real change
Bing recommends keeping content fresh and using IndexNow to tell participating search engines about added, updated or removed URLs.
The IndexNow documentation covers the key file and the submit call.
Wire the ping into your publishing step so nobody has to remember it.
Only change the visible "updated" date when the content actually changed.
Step 9: set up the tracking sheet before you report anything
Every engagement needs one sheet with one row per page and one column per data source.
- Search Console: the Generative AI performance report shows AI Overviews and AI Mode impressions by page, country, date and device. It counts impressions, not clicks.
- Bing Webmaster Tools: the AI Performance report lists total citations, average cited pages and sampled grounding queries for Copilot and AI summaries in Bing.
- Analytics: OpenAI's help centre says ChatGPT tags referral links with a chatgpt.com source parameter, so filter for it.
Bing says its citation counts don't indicate placement, ranking or authority, so report them as activity, not as rank.
No source documents a Perplexity analytics report or a Claude referral tag, so don't promise either in a client report.
The gotchas that waste a week
- Blocking GPTBot and calling it done. It is the training bot; OAI-SearchBot decides ChatGPT search.
- Testing ChatGPT an hour after the edit. Wait the full day OpenAI documents.
- Adding llms.txt for Google. Google's guide says Search doesn't use it.
- Buying mentions. Google lists inauthentic mentions as a myth, and its spam policies cover attempts to manipulate generative AI responses.
- Looking for a submission form. No source documents one for ChatGPT, Claude or Perplexity.
- Promising a ranking order. None of the AI engines publishes how it picks sources.
Where AI agents speed up the runbook
An agent can take the repeat work off your plate while a person signs off each change.
- Fetch robots.txt for a list of domains and flag any rule that blocks the three search bots.
- Read server logs and count hits per bot per status code.
- Draft FAQ questions from a folder of customer emails.
- Compare a rewrite against the old page and list what is new first-hand detail.
Book a free SEO strategy session →
Keep the agent on draft-only: it proposes, a human edits robots.txt and publishes.
Log every agent finding next to the change it led to, so the next audit starts from a known baseline.
For a wider pre-check across Google and the assistants, the AI visibility audit process on this site pairs well with this runbook.
When to hand the engagement to a specialist
Hand it over when the firewall belongs to a vendor you can't reach, when the rewrite list runs past twenty pages, or when you need real coverage that earns genuine mentions.
Goldie Agency works across SEO, GEO, AEO and link building, and it describes its link work as "manual outreach, no PBNs."
On his about page, Julian's own published numbers list 1,000+ SEO and link building campaigns, 240+ clients and a 100% Job Success Score on Upwork.
Paid placements should carry rel="sponsored" under Google's link spam rules, so the coverage work stays inside policy.
📞 Hand over the runbook and keep the results Book a free SEO strategy session and get your crawler access, page rewrites and tracking sheet planned with the team. → Book my free strategy session
Want to learn AI SEO yourself before the call? The AI Profit Boardroom has the trainings and the agent workflows.
Two free downloads help with prep: 200+ AI SEO Prompts for the rewrites and the free link building book for the coverage side.
Or grab a slot directly: Book a free SEO strategy session.
Book a free SEO strategy session →
Also On Our Network
- 🌐 What getting cited by AI involves for a small business, and when to hire
- 🌐 A scorecard rating five routes to getting cited by AI
- 🌐 Five Agent OS prompts that do the groundwork for AI citations
- 🌐 Which AI models handle each AI citation task, by GoldieBench score
FAQ: How To Get Cited By AI
Which robots.txt line matters most for ChatGPT citations?
The OAI-SearchBot group. OpenAI says sites that block OAI-SearchBot will not be shown in ChatGPT search answers, while GPTBot is a separate training control.
Can a firewall stop AI engines citing me even with a clean robots.txt?
Yes. If a CDN or bot setting challenges the crawler, it never reads the page. Anthropic also advises against IP-blocking its bots because they then cannot read robots.txt.
How long should I wait before rechecking ChatGPT search?
OpenAI says its search systems take about 24 hours to reflect a robots.txt change, so wait a full day.
Does Perplexity use PerplexityBot to train models?
Perplexity says PerplexityBot surfaces and links websites in its results and is not used to crawl content for AI foundation models.
What should an operator report to a client?
Search Console generative AI impressions, Bing AI Performance citations and ChatGPT referral visits. Bing says its citation counts do not show placement, ranking or authority.
📺 Video notes + links to the tools 👉
🎥 Learn how I make these videos 👉
🆓 Get a FREE AI Course + Community + 1,000 AI Agents 👉
About Julian
I'm Julian Goldie, an SEO entrepreneur, author and founder of the AI Profit Boardroom, which has 3,400+ members.
I help business owners scale with AI agents, automation and SEO.
- I've built a 7-figure SEO and link building agency, Goldie Agency, with a team of 70+ SEO experts.
- I've grown a YouTube channel to 400,000+ subscribers.
- I wrote "Link Building Mastery".
- My Udemy courses have taught over 29,000 students.
→ Get my best AI training inside the AI Profit Boardroom
That is how to get cited by AI: work the nine steps in order, keep the agent on draft-only, and book a free strategy session when you want a team to run it.











