Your company runs 80 models in production. Monitoring today is limited to infrastructure metrics: latency, error rate, CPU. Nobody is watching whether the models are still correct.
Two failures last quarter make the case. A credit risk model degraded over four months as the customer mix shifted; it was noticed only when the finance team queried default rates. A recommendation model broke within an hour of a schema change upstream, which silently sent nulls for a key feature; a downstream metric caught it eight days later.
These two failures have very different signatures: one slow and real, one instant and mechanical, and the monitoring system must catch both.
Labels arrive with wildly different delays: minutes for click models, months for credit models.
Build the architecture on a canvas: place the components, configure them, connect them into a data flow, and write a short reason for each one. The AI reviewer grades your design against a rubric written specifically for this problem.
Your two incidents had very different signatures. Which layer catches each, and how fast?
A feature drifts significantly but accuracy is unchanged. Do you alert? Why?
Labels for the credit model arrive four months late. What do you monitor in the meantime?
Minimum 6 components · needs a wide desktop screen