AI Search Visibility: Relevance Engineering | Victor Pan

With Victor Pan, principal SEO at HubSpot · Streamed live · 52 min

What this episode covers

Victor Pan joins the SEO Video Show to explain how LLM answers about brands are shaped by training data, Wikipedia, and third-party sources, and why rebrands take time to appear in AI answers. He describes testing methods using prompts that avoid web search and stresses relevance engineering, human review, and survey-based validation of AI visibility tools.

Key takeaways

  • ✦Keep Wikipedia and third-party profiles about your brand accurate, since LLMs often cite them as grounding sources.
  • ✦Expect a lag after a rebrand, because AI assistants only reflect changes once training data reaches a newer knowledge cutoff date.
  • ✦Validate AI visibility tracking tools directionally and use customer surveys asking how people found you.
  • ✦Keep a human in the loop to catch hallucinations and legal risks in AI-assisted work.

Key moments

  1. 19:51Brand Updates in AI Answers
    “does it know me as me two years ago when I was a single product company”
  2. 27:47Rebrands and Knowledge Cutoffs
    “you will have to wait quite a while until the knowledge cut off date hits”
  3. 30:54Wikipedia as a Citation Source
    “across most studies, you're seeing Wikipedia get cited over and over as a very dominant citation source.”
  4. 46:36Human in the Loop Review
    “they really need to understand hallucinations. Okay, they really need to not put things out there that would put the company at risk”
  5. 47:18Visibility Tools as Directional
    “they're directional just like competitive tools that you can use to estimate how much traffic your competitor”

Quotes are from the episode transcript, lightly cleaned of auto-caption errors.

Questions answered in this episode

Why do AI answers about my brand lag behind a rebrand?
Victor explains that AI assistants rely on their knowledge cutoff date, so changes made after that date will not appear in default answers until training data updates. Until then, they may pull outdated information.
Why does Wikipedia show up so often in AI answers?
Victor suggests LLMs use Wikipedia as a grounding knowledge source because it is closer to facts than probability, and studies show it being cited frequently. He notes that outdated Wikipedia pages can hurt single-product or changed companies.
How can you validate AI visibility tracking results?
Victor says AI visibility tools are directional and can be off significantly, so they should be paired with qualitative surveys asking real customers how they found you.

More episodes on AI Search & LLMs