You are building a consumer chat application on top of a hosted LLM API. Target: 2 million monthly active users, 500,000 daily, averaging 12 messages per conversation and 4 conversations per week.
Users expect the experience they know: instant token streaming, conversations that persist and can be resumed months later, editing an earlier message and branching from it, file upload, and stop-generation.
You are paying per token. At scale, the difference between a naive implementation and a careful one is the difference between a viable business and one that loses money on every user.
The free tier must be sustainable and the paid tier must feel unlimited.
Build the architecture on a canvas: place the components, configure them, connect them into a data flow, and write a short reason for each one. The AI reviewer grades your design against a rubric written specifically for this problem.
A user edits their third message in a 20-message conversation. What does your data model do?
A user hits stop after 40 tokens of a 900-token response. What has to happen for you not to pay for the rest?
A conversation reaches 200 messages. Walk me through what you send to the model on message 201.
Minimum 6 components · needs a wide desktop screen