The ML System Design Interview

A typical round: 45 to 60 minutes, one open prompt ('design a system to recommend products', 'detect fraudulent payments'), a whiteboard or shared document, and an interviewer who asks follow-ups. Interviewers usually score:

• problem understanding: clarifying questions, requirements, a precise ML framing;
• data: sources, labels, biases, features and how they are served;
• modelling: a baseline, a justified model choice, awareness of alternatives;
• evaluation: offline measures that fit the task, online experiments, guardrails;
• production: serving, scale, latency, monitoring, retraining, failure handling;
• communication: structure, clarity, trade-offs, responding to hints.

Knowing the rubric tells you where to spend time.

• Modelling is one row of six; candidates who spend thirty minutes on it fail the others.
• Communication runs through all of them: a good design poorly explained scores like a weak one.

Follow the eight steps of module 1 with a rough clock in mind:

• spend the first five minutes on questions and requirements;
• draw the end-to-end system by minute twenty;
• use the middle for one or two deep dives where the problem is hardest;
• keep the last minutes for monitoring, a summary and questions.

Adjust to the interviewer: if they steer you into a deep dive, follow, then return to the plan.

A plan protects you from the most common failure: running out of time before serving, evaluation or monitoring.

• Interviewers notice when a candidate manages the time themselves.

The first five minutes set the tone:

• restate the problem in one sentence;
• ask the few questions that change the design: goal and metric, scale, latency, labels, constraints (module 3);
• state assumptions for anything unknown ('I'll assume 10 million daily users and a 100 ms budget');
• write them down where both of you can see them, and refer back to them.

It shows you design from requirements, not from a memorised answer.

• Written assumptions become the justification for every later decision.
• It also gives the interviewer a chance to steer you towards what they want to discuss.

Before any architecture, say:

• the prediction target, precisely (event and window);
• the task type and why (classification, regression, ranking);
• the offline metric and the online metric, and one guardrail;
• the baseline you would beat.

This is the step most candidates skip and most interviewers weigh heavily.

• It turns a vague product question into an engineering problem.
• Everything later (labels, features, evaluation) follows from it.

Draw the end-to-end picture early, at a high level:

• data sources and logging → labels → feature pipelines and store → training pipeline and registry → serving path → monitoring, with the loop back;
• then say where you want to go deep, or ask the interviewer: 'I could go deeper on candidate retrieval or on labels; which is more useful?'

The full picture shows you see the whole system before optimising any part.

• It gives the interviewer a map to point at.
• Deep dives then have context, and nothing major is left out if time runs short.

Spend deep-dive time where this problem is hard:

• recommendations and search: candidate generation, ranking features, position bias, cold start;
• fraud and moderation: labels and delay, imbalance, thresholds and review capacity, adversaries;
• ETA and forecasting: the target and its asymmetry, live features, evaluation over time;
• ads: calibration, the auction, sparse features;
• imagery and video: detection recall on small objects, making video searchable, the domain gap;
• feeds and social graphs: the value score beyond engagement, two-hop scale, privacy of suggestions;
• events and listings: items that are always new; similarity learned from sessions;
• any problem: point-in-time correctness and training-serving skew.

Deep dives are where seniority shows.

• Choosing the right one shows judgement.
• Going deep on the generic parts (load balancers, databases) wastes the time that ML-specific depth would earn.

For each significant choice, say briefly:

• what you chose;
• the alternative;
• why, tied to a requirement;
• what it costs, and when you would change your mind.

Trade-offs are the core of design, and the clearest signal of experience.

• 'Batch would be cheaper, but the row must react to the current basket, so I'll precompute candidates in batch and rank online' is a complete trade-off in one sentence.

How to handle the moments that go off-script:

• a hint ('what about new users?') is the interviewer telling you what they want to discuss: take it, address it, return to your plan;
• pushback ('why not just use X?'): consider it seriously; agree if it is better; if not, explain the trade-off calmly;
• not knowing: say so, then reason from principles ('I haven't used that, but if it works like Y, I'd expect...');
• changing your mind when given new information is a strength, not a weakness.

Interviews test how you work with people, not just what you know.

• Defensiveness under pushback is marked down more than an imperfect design.
• Reasoning aloud from first principles is often scored higher than a recalled fact.

In the last minutes:

• cover monitoring: what you would watch, at four layers, and the alerts;
• cover iteration: retraining, the next experiments, how you would improve version 2;
• summarise the design in a few sentences, naming the two or three key trade-offs;
• mention risks: fairness, privacy or abuse if relevant.

Monitoring and iteration show you think beyond launch, which is what distinguishes production experience.

• A crisp summary leaves the interviewer with a clear memory of a coherent design.

The same question is answered differently at different levels:

• junior: a correct, complete pipeline; a sensible baseline and model; standard metrics; needs prompting on production concerns;
• mid-level: drives the structure; names labels, skew and monitoring unprompted; justifies choices with requirements;
• senior: identifies the hardest part of the problem early; deep, accurate dives; quantifies trade-offs; designs for failure, iteration and team ownership;
• staff and above: frames the business problem itself; considers several systems and teams; anticipates long-term costs, risks and organisational impact.

Knowing the bar helps you aim.

• Senior candidates are expected to lead without prompting and to go beyond a standard answer in at least one area.

The most common, in rough order of frequency:

• naming a model before the goal, target and metric;
• no baseline;
• accuracy as the metric for imbalanced or ranking problems;
• features that would not exist at prediction time (leakage);
• no word on where labels come from or when they arrive;
• a single model for everything at scale, with no retrieval stage;
• no fallback, no monitoring, no retraining;
• running out of time before evaluation and serving.

Each is easy to avoid once named.

• Most of them come from jumping to the model; the framework in module 1 prevents nearly all of them.

Memorise the shape, not an answer. Write these headings on the board at the start and fill them in as you go.

A visible structure keeps you and the interviewer oriented.

• It also makes it obvious if a section is still empty with ten minutes left.