Practice Problems

Realistic AI engineering scenarios with reference solutions and problem-specific grading rubrics.

96 problems

ProblemCategoryDifficultyTimeBest

A Confusion Matrix in One Pass
Integer indexing · bincount · Reshaping · Evaluation
NumPyEasy15 minEasy15m–Solve
Cosine Similarity Without Loops
Broadcasting · Vectorization · Linear algebra · Numerical stability

One-Hot Encode a Label Vector

Fancy indexing · Broadcasting · NumPy · Encoding

NumPyEasy15 minEasy15m–Solve

Group Sums Without a GroupBy

bincount · Scatter-add · Safe division · Vectorization

NumPyMedium25 minMedium25m–Solve

Intersection Over Union for Every Box Pair

Broadcasting · Clipping · Geometry · Vectorization

NumPyMedium25 minMedium25m–Solve

Moving Average Without a Loop

Prefix sums · Vectorization · Off-by-one reasoning · Dtype handling

NumPyMedium20 minMedium20m–Solve

Softmax That Does Not Overflow

Numerical stability · Broadcasting · Axis handling · Vectorization

NumPyMedium25 minMedium25m–Solve

Standardise Features That Have Holes In Them

Missing data · Axis handling · Numerical guards · Feature scaling

NumPyMedium25 minMedium25m–Solve

Top-K Retrieval Without Sorting Everything

argpartition · Vectorization · Ranking · Complexity

NumPyMedium25 minMedium25m–Solve

Fill the Gaps in Every Row

Accumulate tricks · Fancy indexing · Missing data · Vectorization

NumPyHard40 minHard40m–Solve

Monthly Active Users

groupby · Datetime handling · Deduplication · pandas

PandasEasy15 minEasy15m–Solve

Join Without Letting a Dirty Dimension Table Multiply Your Rows

merge · Deduplication · Data quality · Join semantics

PandasMedium25 minMedium25m–Solve

Pad Each Store's Own Calendar, Not the Global One

Time series · GroupBy · Reindexing · Data leakage

PandasMedium30 minMedium30m–Solve

Pivot Event Counts, With the Zeros Included

pivot / unstack · Reshaping · Categorical data · GroupBy

PandasMedium25 minMedium25m–Solve

Pull the SKU Out of a Free-Text Note

str.extract · Regular expressions · Data cleaning · Missing data

PandasMedium20 minMedium20m–Solve

Revenue Per Customer From a Messy CSV

read_csv · Filtering · GroupBy · Data cleaning

PandasMedium25 minMedium25m–Solve

Rolling 7-Day Spend Per Customer

GroupBy · Time series · Rolling windows · Feature engineering

PandasMedium30 minMedium30m–Solve

Sessionise an Event Stream

GroupBy · Time series · cumsum · Feature engineering

PandasMedium30 minMedium30m–Solve

The Top Three Per Group, Deterministically

sort_values · GroupBy · Ranking · Tie-breaking

PandasMedium25 minMedium25m–Solve

A Monthly Retention Cohort Grid

Cohort analysis · Period arithmetic · GroupBy · Dense reindexing

PandasHard45 minHard45m–Solve

Point-in-Time Feature Join

merge_asof · Data leakage · Feature stores · Time series

PandasHard35 minHard35m–Solve

Macro and Micro F1, From the Confusion Matrix Alone

Model evaluation · Multiclass metrics · Precision/recall · F1

Machine LearningEasy20 minEasy20m–Solve

Precision, Recall and F1 by Hand

Model evaluation · Precision/recall · Confusion matrix · NumPy

Machine LearningEasy15 minEasy15m–Solve

A Bootstrap Confidence Interval for a Metric

Bootstrap · Confidence intervals · Resampling · Statistics

Machine LearningMedium25 minMedium25m–Solve

A Pipeline That Does Not Leak

Pipelines · Data leakage · Cross-validation · Preprocessing

Machine LearningMedium30 minMedium30m–Solve

A Split That Does Not Leak Across Groups

Data leakage · Cross-validation · Grouped data · Evaluation

Machine LearningMedium30 minMedium30m–Solve

How Wrong Are Your Probabilities?

Calibration · Model evaluation · Probability · Binning

Machine LearningMedium25 minMedium25m–Solve

k-Nearest Neighbours, With a Tie-Break That Never Changes

k-NN · Distance metrics · Tie-breaking · Vectorization

Machine LearningMedium25 minMedium25m–Solve

Permutation Importance, Averaged Honestly

Feature importance · Permutation testing · Reproducibility · Model evaluation

Machine LearningMedium25 minMedium25m–Solve

Pick a Threshold for an Imbalanced Classifier

Imbalanced data · Model evaluation · Precision/recall · Thresholding

Machine LearningMedium25 minMedium25m–Solve

The Best Single-Feature Split, by Gini

Decision trees · Gini impurity · Vectorization · Cumulative counts

Machine LearningMedium30 minMedium30m–Solve

Would You Notice the Input Drifting?

Drift detection · Monitoring · Quantile binning · Model maintenance

Machine LearningMedium30 minMedium30m–Solve

k-Means, and the Cluster That Loses All Its Points

Clustering · k-means · Convergence · Vectorization

Machine LearningHard45 minHard45m–Solve

ReLU, Forward and Backward

Backpropagation · Activations · NumPy · Gradients

Deep LearningEasy15 minEasy15m–Solve

Adam, One Step at a Time

Optimisers · Adam · Bias correction · Training dynamics

Deep LearningMedium25 minMedium25m–Solve

Batch Norm Behaves Differently at Inference

Batch normalisation · Running statistics · Train/eval mode · NumPy

Deep LearningMedium30 minMedium30m–Solve

Linear Layer Forward and Backward

Backpropagation · Chain rule · Matrix calculus · Gradient checking

Deep LearningMedium35 minMedium35m–Solve

Layer Norm, Backwards

Backpropagation · Layer normalisation · Chain rule · NumPy

Deep LearningHard40 minHard40m–Solve

Scaled Dot-Product Attention

Attention · Transformers · Masking · Numerical stability

Deep LearningHard40 minHard40m–Solve

Count the Common N-grams

Tokenisation · N-grams · Counting · Deterministic ordering

NLPEasy15 minEasy15m–Solve

Chunk a Document for Retrieval

Chunking · RAG · Text processing · Sliding window

NLPMedium25 minMedium25m–Solve

Chunk-Level Named Entity Merging

Named entity recognition · BIO tagging · Span extraction · Sequence labelling

NLPMedium25 minMedium25m–Solve

TF-IDF From Scratch

TF-IDF · Sparse representations · Lexical retrieval · Vectorization

NLPMedium30 minMedium30m–Solve

Byte-Pair Encoding Tokenizer

Tokenization · BPE · Subword models · Vocabulary construction

NLPHard40 minHard40m–Solve

Chunk a Document by Size

RAG · Chunking · Overlap · Retrieval

LLM / GenAIEasy15 minEasy15m–Solve

Reciprocal Rank Fusion

Hybrid search · Rank fusion · Retrieval · RAG

LLM / GenAIEasy20 minEasy20m–Solve

A Semantic Cache That Forgets Properly

Caching · Cosine similarity · LRU eviction · Cost control

LLM / GenAIMedium30 minMedium30m–Solve

Fit the Conversation in the Window

Context window · Token budgeting · Chat applications · Prompt construction

LLM / GenAIMedium25 minMedium25m–Solve

Retrieve Five Passages, Not One Passage Five Times

RAG · Maximal marginal relevance · Retrieval · Cosine similarity

LLM / GenAIMedium30 minMedium30m–Solve

Structure-Aware Chunker With Overlap

Chunking · RAG · Context construction · Retrieval quality

LLM / GenAIMedium30 minMedium30m–Solve

Reassemble a Streamed Response

Streaming · Server-sent events · Parsing · LLM plumbing

AI EngineeringEasy15 minEasy15m–Solve

A Rate Limiter That Allows a Burst

Rate limiting · Token bucket · Backpressure · API design

AI EngineeringMedium25 minMedium25m–Solve

Batch Requests Without Blowing the Latency Budget

Dynamic batching · Inference serving · Latency budgets · Throughput

AI EngineeringMedium30 minMedium30m–Solve

Retry With Exponential Backoff and Jitter

Retries · Backoff · Thundering herd · Error handling

AI EngineeringMedium25 minMedium25m–Solve

Stop Calling the Provider That Is Down

Circuit breaker · Resilience · State machines · Failure handling

AI EngineeringMedium30 minMedium30m–Solve

Token Cost From a Raw Usage Log

Cost tracking · Data cleaning · CSV parsing · Normalization

AI EngineeringMedium25 minMedium25m–Solve

Token-Budgeted Context Packer

Context management · Token budgets · RAG · Prompt construction

AI EngineeringMedium25 minMedium25m–Solve

Design a URL Shortener

Capacity estimation · Key generation · Caching · Read-heavy systems

ML System DesignEasy35 minEasy35m–Design

Diagnose Model Drift

Drift Analysis · Root Cause Analysis · Covariate Shift · Concept Drift

ML System DesignEasy25 minEasy25m–Design

Design a Demand Forecasting Platform

Time Series · Hierarchical Forecasting · Backtesting · Intermittent Demand

ML System DesignMedium40 minMedium40m–Design

Design a Large-Scale Classification Platform

Extreme Multi-Label · Hierarchical Classification · Active Learning · Human-in-the-Loop

ML System DesignMedium40 minMedium40m–Design

Design a Predictive Maintenance System

Sensor Data · Survival Analysis · Rare Events · Edge Inference · Time Windows

ML System DesignMedium40 minMedium40m–Design

Design a Real-Time Anomaly Detection System

Time Series · Unsupervised Learning · Streaming · Alerting · Seasonality

ML System DesignMedium40 minMedium40m–Design

Design a Recommendation System

Candidate Generation · Ranking · Embeddings · Two-Tower Models · Cold Start

ML System DesignMedium45 minMedium45m–Design

Design an Image Search System

Vision Embeddings · ANN Search · Multimodal · Index Sharding · CLIP

ML System DesignMedium40 minMedium40m–Design

Design an ML Training Pipeline

Pipelines · Reproducibility · Experiment Tracking · Data Versioning · Orchestration

ML System DesignMedium35 minMedium35m–Design

Design Feature Monitoring

Data Quality · Skew Detection · Lineage · Freshness SLAs · Root Cause

ML System DesignMedium30 minMedium30m–Design

Design Model Monitoring

Drift Detection · Delayed Labels · Alerting · Segment Analysis · Observability

ML System DesignMedium35 minMedium35m–Design

Diagnose Training-Serving Skew

Skew Debugging · Feature Pipelines · Data Leakage · Root Cause Analysis

ML System DesignMedium30 minMedium30m–Design

Improve a Low-Recall Model

Imbalanced Data · Threshold Tuning · Error Analysis · Precision-Recall Tradeoff

ML System DesignMedium30 minMedium30m–Design

Debug a Production ML Model

Incident Response · Root Cause Analysis · Debugging · Systems Thinking

ML System DesignHard35 minHard35m–Design

Design a Model Serving Platform

Model Serving · GPU Scheduling · Autoscaling · Canary Deploys · Multi-tenancy

ML System DesignHard45 minHard45m–Design

Design a Personalized News Feed

Ranking · Fan-out · Multi-Objective Optimization · Freshness · Feedback Loops

ML System DesignHard45 minHard45m–Design

Design a Real-Time Fraud Detection System

Streaming · Imbalanced Data · Feature Store · Low Latency · Adversarial ML

ML System DesignHard50 minHard50m–Design

Design a Search Ranking System

Learning to Rank · BM25 · Query Understanding · Click Models · Position Bias

ML System DesignHard45 minHard45m–Design

Design an Ad Ranking System

CTR Prediction · Auctions · Calibration · Delayed Feedback · Budget Pacing

ML System DesignHard50 minHard50m–Design

Design an ML Feature Store

Feature Store · Point-in-Time Correctness · Streaming · Data Engineering

ML System DesignHard45 minHard45m–Design

Design Continuous ML Deployment

CI/CD · Canary Analysis · Shadow Deploys · Rollback · Automated Promotion

ML System DesignHard40 minHard40m–Design

Design a ChatGPT-like Application

Streaming · Context Management · Conversation State · Safety · Cost Control

LLM System DesignMedium40 minMedium40m–Design

Design a Self-Serve LLM Fine-Tuning Platform

GPU scheduling · Model registry · LoRA · Eval gating · Multi-tenancy

LLM System DesignMedium40 minMedium40m–Design

Design an LLM Gateway

API Gateway · Rate Limiting · Caching · Observability · Failover

LLM System DesignMedium40 minMedium40m–Design

Design an LLM Evaluation Platform

LLM-as-Judge · Eval Design · Regression Testing · Human Labeling · Statistics

LLM System DesignHard45 minHard45m–Design

Design an LLM Inference Serving System

Continuous batching · KV cache · GPU autoscaling · Quantization · Latency budgets

LLM System DesignHard45 minHard45m–Design

Design a Document Intelligence Platform

OCR · Structured Extraction · Confidence Calibration · Human Review · Multimodal

RAGMedium40 minMedium40m–Design

Design a Semantic Search System

Embeddings · Hybrid Search · Chunking · Reranking · Index Design

RAGMedium35 minMedium35m–Design

Design a Production RAG System

RAG · Embeddings · Vector DB · Reranking · LLM Evaluation · Access Control

RAGHard45 minHard45m–Design

Design an Enterprise Knowledge Assistant

RAG · Tool Use · Personalization · Enterprise Integration · Guardrails

RAGHard45 minHard45m–Design

Design an AI Agent for Customer Refunds

Tool calling · Irreversible actions · Policy enforcement · Escalation

AI AgentsMedium40 minMedium40m–Design

Design a Tool-Calling Agent Over 500 APIs

Tool selection · Retrieval · Permissions · Failure handling · Observability

AI AgentsHard45 minHard45m–Design

Design an AI Coding Assistant

Code Context · Low Latency · Fill-in-the-Middle · Repo Indexing · Acceptance Metrics

AI AgentsHard45 minHard45m–Design

Design an AI Customer Support Agent

Agents · Tool Use · Escalation · Safety · Containment Rate

AI AgentsHard45 minHard45m–Design