You are building an enterprise knowledge assistant for a company of 100,000 employees. The company has 10 million documents spread across Confluence, Google Drive, Slack, Jira and an internal wiki. Employees should be able to ask questions in natural language and receive answers with citations.
Two things make this harder than a demo RAG pipeline. First, document-level permissions: an employee must never see content from a document they cannot open, and permissions change constantly as people join teams and projects. Second, freshness: documents are edited thousands of times per day, and an answer citing a deleted or outdated page destroys trust immediately.
The company has tried an off-the-shelf tool. Employees stopped using it after two weeks because it confidently produced wrong answers and could not say "I do not know".
Design the system, and be specific about how you would know whether it is actually working.
Build the architecture on a canvas: place the components, configure them, connect them into a data flow, and write a short reason for each one. The AI reviewer grades your design against a rubric written specifically for this problem.
Why would you choose pre-filtering over post-filtering for permissions? Show me the case where post-filtering silently fails.
A user asks a follow-up question: 'what about the timeout?'. What does your system do before it retrieves anything?
Your answers are wrong 15% of the time. How do you find out whether that is a retrieval problem or a generation problem?
Minimum 8 components · needs a wide desktop screen