AI Search Optimization Services: 5-Part Playbook (2026)

AI search optimization services come down to five jobs I run in a fixed order: crawler access, content, entity and brand signals, links and mentions, then measurement.

Skip the order and you end up polishing pages that OAI-SearchBot was never allowed to fetch.

Every step below rests on a platform's own documentation, so you can check the work against Google, OpenAI, Microsoft, Perplexity and Anthropic.

If you'd rather have my team run it with you, grab a free SEO strategy session, or go straight to the calendar: Book a free SEO strategy session.

The ground rule before any work starts

Google's guide to optimizing for generative AI features says that, from Google Search's point of view, optimizing for generative AI search is "still SEO".

Its AI features page adds that AI Overviews and AI Mode have no additional requirements beyond normal Search.

A page only has to be indexed and eligible to show with a snippet.

So the operator's job isn't inventing new tricks.

It's making sure the basics hold on every surface where AI answers get assembled: Google, Bing and Copilot, ChatGPT search, Perplexity and Claude.

Nobody can promise a #1 spot or a ChatGPT mention, and Google's page on hiring an SEO tells owners to find someone else if a provider guarantees first place.

Part 1: write robots.txt so each AI search bot gets in

Each AI platform runs a search crawler and a separate training crawler, and you set them one at a time.

OpenAI's crawler overview says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers.

The same page calls GPTBot the training bot and says the two settings are independent.

Perplexity's crawler docs say PerplexityBot surfaces sites in its results and isn't used to crawl content for foundation models.

Anthropic's crawler help article warns that disabling Claude-SearchBot may reduce your visibility in Claude's search results.

Here's the block I start from on a client site:

# AI search crawlers: let them in
User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Bingbot
Allow: /

User-agent: Googlebot
Allow: /

# Training crawlers: a separate business decision
# User-agent: GPTBot
# Disallow: /
# User-agent: ClaudeBot
# Disallow: /
# User-agent: Google-Extended
# Disallow: /

Sitemap: https://www.example.com/sitemap.xml

Blocking the training bots is a business call, not a visibility one.

Google-Extended is the same story: Google's common crawlers page says it "does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search."

OpenAI says its search systems take roughly 24 hours to pick up a robots.txt change, and Perplexity says up to 24 hours.

Part 1b: clear the CDN and firewall, not just the file

A clean robots.txt does nothing if your CDN answers the bot with a challenge page.

OpenAI's ChatGPT search help article asks owners to confirm their host or CDN lets through traffic from OpenAI's published searchbot IP addresses.

Perplexity's docs walk through a Cloudflare WAF rule that matches each user agent to its own IP list.

My check: pull a week of server logs, filter for OAI-SearchBot, PerplexityBot, Claude-SearchBot and Bingbot, and read the status codes.

A wall of 403 responses means the firewall is the problem, not the content.

Gotcha: ChatGPT-User and Perplexity-User are fetchers that users trigger, and both vendors say robots.txt rules may not apply to them, so never judge search access by their traffic.

Part 2: audit the preview controls that strip you out of answers

Plenty of sites block themselves from AI answers by accident through snippet tags.

Google's robots meta tag reference says nosnippet also stops a page being used as a direct input for AI Overviews and AI Mode.

max-snippet caps how much text those features can draw on, and max-snippet:0 behaves like nosnippet.

On Bing, the Webmaster Guidelines say NOARCHIVE keeps content out of Copilot responses and NOCACHE limits Copilot to the URL, title and snippet.

Microsoft is replacing both with new AI Training and AI Grounding preferences that it starts honoring on February 1, 2027.

My checklist for this pass:

Crawl the site and list every URL carrying nosnippet, max-snippet, data-nosnippet or noindex.
Confirm with the client that each one was set on purpose.
List any NOARCHIVE or NOCACHE tags and plan the switch to Bing's new preferences before March 1, 2027.
Run URL Inspection in Search Console to see the HTML Googlebot actually received.
If key copy is rendered by JavaScript, check the rendered HTML against Google's JavaScript SEO basics, since Bing also warns against hiding critical content behind client-side rendering.

📞 Want the five-part audit run on your own site? Book a free SEO strategy session and my team will walk through your crawler access, snippet tags and entity setup with you on the call. → Book a free SEO strategy session

Part 3: content that gives AI engines a reason to cite you

Google's AI guide says unique, non-commodity content will likely shape your presence in generative AI search more than anything else it suggests.

Its own example contrasts a generic list of first-time homebuyer tips with a first-hand account of one inspection decision.

For a service business, that means pages built on what only you know: real job photos, the process you follow, the questions customers ask on sales calls.

Bing's guidelines push the same direction: state facts and definitions outright, keep each URL on one main topic, and put key information on the page itself.

My brief template has four lines: the question, the answer in two sentences, the proof only this business has, and the next step for the reader.

Don't spin up a page for every phrasing of a query.

Google says doing that mainly to manipulate AI responses falls under its scaled content abuse policy.

Part 4: make the business an entity machines can't confuse

AI answers name businesses, so the business has to be easy to identify.

Google's Organization structured data docs say the markup helps Google disambiguate your organization, and sameAs lets you list your profiles on other sites.

Add Organization, or its subtype LocalBusiness, to the home page with name, url, logo, address, telephone and sameAs filled in.

Keep the business name identical everywhere: the site, Google Business Profile, Bing Places and every social profile.

Bing's guidelines ask for clear, consistent naming of people, organizations, products and locations.

For ChatGPT, OpenAI's Business Hub article (updated November 2026) lets an owner claim a business, verify it and keep its details current.

OpenAI also says claiming doesn't guarantee increased visibility, so file it under accuracy work, not a ranking lever.

Gotcha: markup must match the visible text, and Google says AI features need no special schema.org type.

Part 5: earn links and mentions inside the spam rules

Links still help pages get discovered, and Bing lists external links from relevant websites as a discovery signal.

Google's spam policies define link spam as links created mainly to manipulate search rankings.

Paid placements are fine when the link carries rel="sponsored" or rel="nofollow", per Google's guide to qualifying outbound links.

The engine I trust is outreach: pitch sites your buyers already read, with something genuinely worth linking to.

Goldie Agency describes its own link building as "Manual outreach, no PBNs".

Mentions follow the same logic, and Google's AI guide says chasing inauthentic mentions isn't as helpful as it might seem.

Real reviews, interviews, podcast spots and forum answers from your actual team are the mentions worth having.

Part 6: measure what each AI surface actually reports

You can't manage AI visibility from a screenshot of one prompt.

Google: the Generative AI performance report in Search Console, rolled out to all sites on August 31, 2026, shows impressions, pages, countries, devices and dates.
Bing: the AI Performance report in Bing Webmaster Tools shows total citations, average cited pages, grounding queries and citations per URL.
ChatGPT: Business Hub insights cover mentions, website clicks and discovery topics across the past 28 days for claimed businesses.
Analytics: in our own accounts, ChatGPT referrals usually arrive with chatgpt.com as the source, so build a segment for it.

Report the trend each month and tie it to leads and calls, not impressions alone.

What I cut from every proposal

Three tactics get struck out before a client signs.

llms.txt sold as a Google win: Google's AI guide says Google Search ignores those files, so they neither help nor harm rankings there.
Chopping pages into tiny chunks: Google says its systems can understand several topics on one page.
Rewriting copy "for AI": Google says its systems understand synonyms, so you don't need every long-tail variation.

If some other service reads llms.txt, fine, but sell it as exactly that and nothing more.

Where AI agents speed up each part

Agents are great at the repetitive half of this work.

Access: an agent can parse server logs and flag AI bot requests that got blocked.
Preview controls: it can crawl a site export and list every snippet directive by template.
Content: it can turn call notes into first-draft briefs, which a human then fills with proof.
Entities: it can draft Organization JSON-LD from the About page for a developer to check.
Links: it can build a research list of relevant sites in draft-only mode, with a person sending every pitch.

Keep a human on publishing, outreach and anything that touches the live site.

If you want to learn AI SEO yourself, the AI Profit Boardroom has the trainings and the Agent OS I use for this.

🔥 Want to run this AI search playbook yourself, with agents doing the grunt work? Inside the AI Profit Boardroom you get step-by-step video trainings, my installable Agent OS and four coaching calls a week with 3,600+ members learning AI SEO. → Join the AI Profit Boardroom

Two small-business runs, step by step

A roofing contractor

Check server logs for OAI-SearchBot and PerplexityBot hits, and fix any firewall blocks.
Remove a leftover nosnippet tag from the service-area template.
Write one page per service with real job photos, materials used and warranty terms.
Add LocalBusiness markup with sameAs links to the Business Profile and social pages.
Ask recent customers for genuine reviews with no incentive attached, in line with Google's review tips.
Track the Generative AI report and inbound calls every month.

An online tea shop

Allow OAI-SearchBot and Claude-SearchBot, then decide separately on GPTBot and ClaudeBot.
Keep Merchant Center data current, which Google's AI features page lists as a best practice.
Claim the brand in ChatGPT Business Hub and connect a product feed if the shop qualifies.
Publish brewing guides based on the shop's own tastings, not rewrites of other blogs.
Pitch tea and food writers for editorial reviews, and mark any paid placement sponsored.

How I run the engagement week to week

Week one is access: read-only Search Console and Bing Webmaster Tools, plus server logs.

Google's hiring page says to grant only read access at the audit stage, and I hold to that.

Week two is the technical fix list, ranked by how many pages each issue touches.

From week three, content and entity work alternate, with link outreach running in the background.

Every change goes in a shared log with the reason and the doc it rests on.

Google suggests asking whether an SEO cites official documentation as evidence, so the log answers that before the client asks.

For context on my team, Julian's own published numbers list 1,000+ SEO and link building campaigns, 240+ clients, a 100% Job Success Score on Upwork and 425K+ YouTube subscribers.

Want those weeks mapped to your site? Book a free SEO strategy session.

📞 Rather hand the whole engagement to a team? Book a free SEO strategy session to see how Goldie Agency would sequence access, content, entity, link and reporting work for your business. → Book a free SEO strategy session

Operator gotchas checklist

Allowing a bot in robots.txt while the CDN still blocks it.
Judging access by ChatGPT-User traffic, which OpenAI says isn't used for search inclusion.
Leaving max-snippet:0 on a template and wondering why AI Overviews skip it.
Schema that says one thing while the page says another.
Paid links shipped without rel="sponsored".
Reporting a single prompt screenshot as proof of visibility.
Forgetting the Bing preference deadline of March 1, 2027 on sites that use NOARCHIVE.

🎁 Two free tools for the content and link parts Grab the 200+ AI SEO Prompts for briefs, schema drafts and audits, and the free link building book for the outreach side.

Also On Our Network

FAQ: AI Search Optimization Services

Which robots.txt user agents matter for AI search?

OAI-SearchBot for ChatGPT search, PerplexityBot for Perplexity, Claude-SearchBot for Claude, plus Googlebot and Bingbot. The training bots (GPTBot, ClaudeBot, Google-Extended) are separate settings.

Does blocking Google-Extended hurt my Google rankings?

No. Google says Google-Extended does not affect inclusion in Google Search and is not a ranking signal. AI Overviews and AI Mode are governed by Googlebot and snippet controls.

Why is my site missing from AI Overviews even though it ranks?

Check for nosnippet or max-snippet:0 on the page. Google says nosnippet also stops content being used as a direct input for AI Overviews and AI Mode.

Should I build an llms.txt file?

Not for Google. Google says Google Search ignores llms.txt files, so they neither help nor harm rankings there. Build one only if another service you care about reads it.

How long until a robots.txt change shows up in ChatGPT search?

OpenAI says it can take about 24 hours for its search systems to adjust after a robots.txt update.

📺 Video notes + links to the tools 👉

🎥 Learn how I make these videos 👉

🆓 Get a FREE AI Course + Community + 1,000 AI Agents 👉

About Julian

I'm Julian Goldie, an SEO entrepreneur, author and founder of the AI Profit Boardroom, which has 3,400+ members.

I help business owners scale with AI agents, automation and SEO.

I've built a 7-figure SEO and link building agency, Goldie Agency, with a team of 70+ SEO experts.
I've grown a YouTube channel to 400,000+ subscribers.
I wrote "Link Building Mastery".
My Udemy courses have taught over 29,000 students.

→ Get my best AI training inside the AI Profit Boardroom

Run the five parts in order, log every change against the docs, and book a free SEO strategy session if you want my team to deliver AI search optimization services with you.