Meta · AI Infrastructure · Trust UX
Knowledge Search 2.0
New models were shipping faster than trust could keep up. Meta’s assistant could answer, but people didn’t trust it, so they re-checked everything. I rearchitected search on grounded RAG so every answer carried inline citations, source indicators, and explainability, with Agentic Rich Cards to act in place. Shipped as product and platform, it raised trust company-wide.
Situation
The business and people problem
Meta had AI that answered but couldn’t be trusted, right when LLMs were new and a reliable source of truth mattered most. Answers came without sources, so people re-verified even correct ones, and getting anything done meant chasing tools across four search surfaces. The major pain points, reframed:
Retrieval
Ranking matched keywords, not what the person was trying to get done.
Search could not distinguish navigating, fact-finding, and investigating.
We didn't understand our own available content, so misses looked like ranking failures.
Experience
Four search surfaces, each with its own behavior, index, and permissions story.
People used the AI widget and still re-verified everything it said before acting.
The same question returned different results depending on the surface, with no shared source of truth.
Where the volume actually was, by intent class:
Mental models
Key types for searching @work
- Navigation Search: get to a known tool and act.
- Fact Finding: look up or re-find one specific thing, fast.
- Landscape Searches: scope a new but defined space to plan work.
- Investigation Searches: iterative and analytic; define the need and find the answer at once.
Task
The design hypothesis
With AI, people stop telling search what to do. They tell it the outcome they want.
“Future AI systems will likely have a hybrid user interface that combines intent-based and command-based interfaces while still retaining many GUI elements.”Jakob Nielsen
That hybrid drove the hypothesis. The data ruled out a chat-only bet: most queries are two-word keywords, only about 19% better handled conversationally. So I bet on a blended experience, a grounded AI answer first, traditional results beneath, capturing both without forcing a choice.
Traditional search experience
We keep returning search results (i.e. blue links).
- 81% of queries are not serviceable by Meta’s AI assistant today (fewer than 4 words or Q&A-type). The median query on Intern Search is only 2 words.
- Query length and use of wh-words (except “how”) show no shift toward natural language.
Blended · traditional + AI-generated answer
An AI-generated answer first, with traditional results (i.e. blue links) below it.
- The vast majority of queries are still primarily keyword-based.
- Supports both traditional queries and those better handled by conversational AI (~19% of search queries).
Pure conversational AI
All queries handled by a conversational AI, in the form of an answer.
- Better handles long-form and conversational queries.
- Supports multi-turn conversations (follow-ups) without reformulating.
- Does not handle navigational queries well (finding a link to a tool or document, or taking an action).
Action
Five initiatives, one blended experience
Broaden how people search, from error messages and tool navigation toward semantic queries like “summarize my team’s H1 priorities,” while teaching conversational AI inside the surfaces people already used.
- Build + test: pure conversational. Every surface below went through concept testing in H1. Qual and quant in parallel, with A/B quality experiments gating every step toward the 2024 experience.
Initiative 1
Blended SERP with action
People trusted a button more than a paragraph. So I kept both on one page: a human-like AI answer up top, ranked links beneath, and any result you can act on or summarize right there, without leaving search.
“It is easier when you just have a button.” (P6)
Initiative 2
Intent understanding
Keyword matching couldn’t tell navigating from investigating. So retrieval moved to the LLM: search reads the outcome you want, then resolves into a blended SERP or an assistant thread.
Multi-turn with AI prompts
People read the AI answer first, and trust it only when they can see where it came from. So an AI prompt opens a thread with follow-ups, each refreshing its links, and every result carries its provenance: freshness, author, and who else relies on it.
“The assistant’s answer is the one I care about. I don’t really read past that.” (P2)
Initiative 3
Typeahead
People kept losing things they’d already found. So before you type, your history and suggested prompts surface, and re-finding costs one glance instead of a reconstructed query.
Search wasn’t the one place people started, so they asked for the assistant inside the bar. So the bar makes two offers at once, keywords or AI prompts, and the choice disappears into autocomplete.
Initiative 4
Build once, search everywhere
The same question gave different answers depending on where you asked it. So I designed each card once, partnered with ML on investigational intent, and rendered it everywhere people search: the SERP, PeopleHub, Help Hub, Recruiting, and assistant replies. One component, one answer, every surface.
Agentic widgets and shared components, shipped once with EPS and rendered on every surface.
From the field
"The AI concepts are much more useful than a plain chatbot. There are UI elements that give more confidence… it is easier when you just have a button."
Concept-test participant
Reflection
Where I held the line
The biggest call was the paradigm itself: how AI should enter search.
Blended over pure conversational. For getting work done, people wanted blended search, not a chatbot. The data killed the fashionable answer: 81% of queries couldn’t be served by chat, median query two words. Two systems on one surface, but it kept navigation working and opened investigation queries we’d never served.
Trust as a contract, not a feature. Inline citations, source indicators, confidence, and freshness on every answer, explainability on demand, and permissions inherited at the platform, so people act in place instead of re-verifying. These trust patterns became the template AI search now follows.
De-risked before build. Every surface went through H1 concept testing and third-party evaluation; the vision held while scope moved.
| Criteria | Traditional search | Blended (chosen) | Pure conversational |
|---|---|---|---|
| Query coverage | Serves the 81% of short, keyword queries; fails broad landscape questions. | Serves keyword queries and the ~19% better handled conversationally. | Handles long-form well; the two-word median query fails it. |
| Intent understanding | Lexical matching only; can't separate navigate, find, investigate. | LLM retrieval on the RAG backbone, with ranked results as recovery. | LLM-driven, but a missed answer leaves nothing to fall back on. |
| Navigation & action | Strong; a third of all volume is navigational. | Strong; links, tools, and actions stay one click away. | Weak; poor fit for reaching a tool or taking an action. |
| Multi-turn | None; every refinement is a new query. | Follow-ups escalate into a Meta's AI assistant thread without reformulating. | Native multi-turn conversation. |
| Trust & verification | Familiar, but relevance is unverifiable. | Every answer carries sources, confidence, and freshness. | Text answers people re-verify by hand. |
What I got wrong
I built a pure conversational prototype first. It broke against how people actually search, navigation and fact-finding, so I killed the chat-only bet. The blue links stayed and became the grounded source of truth for all enterprise information.
On the model. It is fluent enough to sound right when it is wrong. So no answer ships without its sources: confidence and freshness travel with every result, and the blue links stay so people can check.
Results
