Case Study: Enterprise AI knowledge

How do you build an AI that can answer questions about 100,000 internal confluence pages, 50,000 PDFs, and 10 million SQL rows across 20 global regions?

1. Architecture: The Router Pattern

Don't send every question to the Vector DB. Use a **Semantic Router**.

• If user asks about 'Salary', route to the HR SQL Database.
• If user asks about 'Deployment', route to the Confluence Vector Index.
• If user asks 'Who are you?', route to a fixed "About" prompt.

This ensures the AI is always looking in the right place, reducing noise and cost.

2. Security: Document Level RBAC

The AI must not leak the CEO's salary to a junior developer. We store the **ACL (Access Control List)** in our Vector Database metadata. When a user searches, we add a filter: `where access_groups contains 'Developer'`. The AI only 'Retrieves' what the user is legally allowed to see.

4. Interview Mastery

Q: "How do you measure 'Success' for an Enterprise AI assistant?"

Architect Answer: "We use the **RAG Triad**: 1) **Context Relevance:** Did we retrieve the right docs? 2) **Groundedness:** Is the AI's answer backed by the docs? 3) **Answer Relevance:** Does the answer actually help the user? We use a secondary 'Judge' LLM to score these three metrics for every session. If Groundedness falls below 0.8, we automatically flag the session for human review to prevent hallucinations."