Framing the Problem

Every ML project starts with a goal stated in business terms, not model terms:

• 'Increase how much people buy per visit' (not 'build a recommender').
• 'Stop losing money to fraud without annoying honest customers' (not 'train a fraud classifier').
• 'Waste less fresh produce' (not 'forecast demand').

The model is one possible way to reach the goal. Write the goal down first, and come back to it at every decision.

The goal decides what 'good' means.

• 'Increase basket size' and 'increase repeat visits' lead to different recommendation models.
• The goal also tells you when to stop: a model that is more accurate but does not move the goal is not better.

Before designing anything, ask about five things:

• Goal: what business result should change, and how is it measured today?
• Users and scale: who uses it, how many, how many requests a second at peak?
• Time limit: does the prediction have to be ready in milliseconds, or can it be computed overnight?
• Data: what is logged today? Are there right answers (labels), and how long until they arrive?
• Constraints: privacy rules, explanations required, devices it must run on, budget.

Each answer removes options.

• 'Overnight' allows batch prediction; '50 ms' rules out large models at request time.
• 'Labels arrive after 60 days' means you cannot judge a new model quickly.
• In an interview, these questions are scored: they show you know what matters before you know the answer.

The prediction target is the exact thing the model outputs. It must be:

• connected to the business goal;
• observable: you can later see whether it came true, so you have labels;
• available soon enough to train and evaluate on;
• specific: a precise event in a precise window, such as 'the user adds this item to the basket within this session'.

A vague target gives a vague model.

• 'Will the user like this product?' cannot be observed.
• 'Will the user add it to the basket in this session?' can, within minutes, millions of times a day.
• The choice of window matters too: 'buys within 7 days' and 'buys within the session' train different models.

The same goal can usually be framed as more than one ML task. For 'show each customer good products':

• Classification: for each product, the probability this user adds it to the basket. Then sort.
• Ranking: directly learn the order of products for this user.
• Regression: predict how much the user would spend on each product.
• Retrieval: find the products whose embedding is closest to the user's.

Large systems often combine framings: retrieval to find candidates, classification or ranking to order them.

The framing decides the labels, the model, the metrics and the cost.

• Pointwise classification is simple and well understood.
• Ranking matches what the user sees.
• Retrieval scales to millions of items where scoring each one is impossible.
• Naming the alternatives and choosing one with a reason is a strong interview move.

Three levels of metric connect a model to the business:

• Business metric (the north star): revenue, orders, cost of fraud. What leadership cares about. Slow and noisy.
• Product metric: basket additions per session, complaints about late delivery. Moves faster, measured in experiments.
• Model metric: precision, recall, NDCG, error in minutes. Measured offline, in minutes, before launch.

Plus guardrail metrics: things that must not get worse, such as app speed, cancellations or complaints.

Each level answers a different question.

• Model metrics let you iterate quickly offline.
• Product metrics, from an A/B test, tell you whether users responded.
• Business metrics tell you whether it mattered.
• A model that improves offline but not online is common; knowing all three levels is how you catch it.

Early on, decide when predictions are made:

• Batch: compute predictions for everyone in advance, on a schedule, and store them. Tomorrow's forecast for every store.
• Online: compute when the request arrives, using the latest information. The fraud score for this payment, now.

Many systems mix the two: batch for what changes slowly, online for what depends on the moment.

It is the decision that most shapes the architecture.

• Batch is simpler, cheaper and easier to check before anyone sees it, but can only use what was known when the batch ran.
• Online can use the current session and context, but needs fast features and a service that never goes down.
• Module 18 covers the patterns in depth.

Constraints remove options before any modelling starts:

• Latency: the time budget for one prediction, inside the product's total budget.
• Throughput: predictions a second at peak.
• Cost: what the business will pay per thousand predictions, and for training.
• Privacy and law: what data may be used, kept, or moved.
• Explainability: whether decisions must be explained to customers or regulators.
• Where it runs: a data centre, a phone, a device with no network.

A design that ignores a hard constraint is not a design.

• A lender that must explain refusals cannot use an unexplainable model for the decision.
• A phone keyboard's next-word model must fit in a few megabytes and run without the network.
• Stating constraints out loud also protects you from over-building.

A baseline is the simplest thing that could work. Build it first:

• Rules: 'show the most popular products in the customer's city'.
• Heuristics: 'ETA = travel time by distance + the store's average packing time'.
• A simple model: logistic regression on five obvious features.

Every later model is judged by how much it beats the baseline.

Baselines are underrated.

• They often deliver most of the value: 'popular in your area' is a strong recommender.
• They tell you whether ML is worth it: if a complex model beats the rule by 1%, the upkeep may not pay.
• They are the fallback when the model fails.
• They make the whole pipeline work end to end before the hard part starts.

The model outputs a prediction; the product takes an action. Between them sits a decision layer:

• thresholds: 'block above 0.9, review between 0.6 and 0.9';
• business rules: 'never recommend alcohol to under-21 accounts';
• limits: 'at most two promoted items in the top ten';
• costs: 'review only as many as the team can handle today'.

Keeping the two separate makes the system easier to change.

• The business can change a threshold or a rule today, without retraining.
• The model stays focused on predicting well; policy lives in code people can read.
• It is also where legal and safety requirements are enforced reliably.

The questions interviewers ask about framing an ML problem, from the first screening call to the senior round. Cover the answer, say yours aloud, then compare.

Reading an idea and explaining it are different skills.

• Each answer below is the two or three sentences an interviewer is listening for.
• The reasoning behind each one is in the cards above. If an answer surprises you, go back to that card.

A short build that uses this module's ideas. Write it yourself before looking anything up; the checks at the end tell you whether it works.

An idea you have only read fades in a week.

• Building it once, even in fifty lines, is what makes it yours.
• Interviewers can tell the difference between someone who has read about a loop and someone who has debugged one.