The ML Platform (MLOps)

An ML platform is the set of shared services every model needs, built once and used by all teams:

• data and features: pipelines, a feature store, validation;
• training: compute, orchestration, experiment tracking;
• model management: a registry, the promotion gate, approvals;
• serving: model servers, scaling, release controls;
• monitoring: dashboards, drift detection, alerts.

Without a platform, each team builds its own version of every piece, differently and incompletely.

• The same bugs (skew, no rollback, no monitoring) recur in every team.
• A platform makes the right way the easy way.
• Well-known examples: Uber's Michelangelo, Meta's FBLearner, Airbnb's Bighead, Netflix's Metaflow.

A common way to describe progress (adapted from Google's MLOps guidance):

• Level 0, manual: notebooks; models handed over as files; deployed by hand; rarely retrained; little monitoring.
• Level 1, automated training: training pipelines run on schedules or triggers; models validated and registered automatically; continuous training.
• Level 2, automated delivery: the pipelines themselves are tested and released through CI/CD; new model designs move to production quickly and safely.

Levels help a team decide what to build next.

• Moving from 0 to 1 removes most incidents caused by manual steps.
• Level 2 matters when many people change pipelines often.
• Not every model needs level 2; a quarterly batch model may be fine at level 1.

Each platform component has well-known open-source and managed options:

• orchestration: Airflow, Dagster, Prefect, Kubeflow Pipelines, Argo Workflows;
• data processing: Spark, the warehouse (dbt), Ray, Flink for streams;
• features: Feast, Tecton, Hopsworks, cloud feature stores;
• tracking and registry: MLflow, Weights & Biases;
• serving: Triton, KServe, Seldon, BentoML, Ray Serve;
• monitoring: Evidently, WhyLabs, Arize, plus Prometheus and Grafana for system metrics.

Cloud suites bundle most of these: SageMaker, Vertex AI, Azure ML, Databricks.

Knowing the landscape lets you choose quickly and talk precisely.

• The tools change; the components (and the problems they solve) do not.
• In interviews, name the component first and the tool second.

For each component, choose between:

• managed services (a cloud ML suite or a vendor): fast to start, less operations work, less control, ongoing fees, some lock-in;
• open source run yourself: flexible, no licence fees, but your team runs and upgrades it;
• build your own: only for needs nothing else meets, usually at very large scale.

Engineers' time is usually the scarcest resource.

• A small team should buy or use managed open source for almost everything and spend its effort on models.
• Large companies build when scale, cost or unusual needs make off-the-shelf tools inadequate.

Most ML platforms run on Kubernetes, which schedules containers across a cluster of machines:

• training jobs and model servers run as containers;
• operators add ML-specific resources: distributed training jobs (Kubeflow Training Operator), inference services (KServe), Ray clusters;
• GPU scheduling assigns GPUs to jobs, with quotas per team;
• autoscaling adds and removes machines as jobs and traffic demand.

One cluster can serve every team's training and serving, with isolation and fair sharing.

• Containers make jobs reproducible and portable.
• But Kubernetes is complex; managed versions (EKS, GKE, AKS) and higher-level tools hide much of it from data scientists.

GPUs and large clusters are expensive. Platforms manage them with:

• quotas per team, and priorities (live serving before training, training before experiments);
• spot or pre-emptible machines for checkpointed training (module 15);
• scale to zero for idle services and notebooks;
• cost attribution: every job and service tagged with its team and model, with monthly reports.

Without attribution nobody knows what anything costs, and nothing gets cheaper.

• Showing each team its bill per model changes behaviour quickly: idle GPUs get released, oversized models get trimmed.

A paved road (or golden path) is the easy, supported way to do common things:

• a project template with training pipeline, tests, serving and monitoring already wired up;
• one command to train, register, deploy and roll back;
• sensible defaults: logging, gates, canaries and alerts on by default;
• freedom to leave the road for unusual needs, with the extra responsibility that brings.

The best way to get good practice is to make it the default.

• A template that includes monitoring means every new model is monitored.
• New teams ship their first model in days instead of months.

Notebooks are excellent for exploration and poor for production. The usual path:

• explore and prototype in a notebook;
• move the code into modules with tests (feature functions, training, evaluation);
• wire them into the pipeline template;
• keep notebooks for analysis, reading from the same modules.

Notebook code is run out of order, hard to test and hard to review.

• Moving it into modules early avoids the 'rewrite everything for production' phase.
• Tools such as Metaflow, Kedro or ZenML make the move smoother.

Every production model should have short, standard documentation:

• a model card: what it predicts, intended use and misuse, training data, metrics overall and per group, limitations;
• a datasheet for important datasets: how collected, what is in it, known gaps;
• ownership and runbooks (module 21).

Documentation is how people who did not build the model use it correctly.

• Model cards stop a model built for one purpose being reused for another it was not tested on.
• Regulators and auditors increasingly expect them.

Platform teams measure their success by their users' outcomes:

• time to production: from idea to a model serving real traffic;
• time to ship a change: from a retrain or new feature to production;
• adoption: share of models on the paved road;
• reliability: ML incidents and their duration;
• cost: compute per model and per prediction.

A platform exists to make product teams faster and safer.

• Measuring adoption and speed keeps it focused on users rather than on technology.
• Falling incident rates show the defaults are working.

The questions interviewers ask about the ML platform, from the first screening call to the senior round. Cover the answer, say yours aloud, then compare.

Reading an idea and explaining it are different skills.

• Each answer below is the two or three sentences an interviewer is listening for.
• The reasoning behind each one is in the cards above. If an answer surprises you, go back to that card.

A short build that uses this module's ideas. Write it yourself before looking anything up; the checks at the end tell you whether it works.

An idea you have only read fades in a week.

• Building it once, even in fifty lines, is what makes it yours.
• Interviewers can tell the difference between someone who has read about a loop and someone who has debugged one.