Bezen
ProblemsDailyPricingLog inSign up

Practice Problems

Realistic AI engineering scenarios with reference solutions and problem-specific grading rubrics.

118 problems
ProblemCategoryDifficultyTimeBest
  • A Confusion Matrix in One Pass

    Integer indexing · bincount · Reshaping · Evaluation

    NumPy·Easy·15 minEasy15m–Solve
  • Accuracy From Model Scores

    argmax · Axis reduction · Boolean mean · Evaluation

NumPy·Easy·15 min
Easy
15m
–
Solve
  • Class Weights From Label Counts

    bincount · Class imbalance · Vectorization · NumPy

    NumPy·Easy·15 minEasy15m–Solve
  • Clip Outliers to a Percentile Range

    Percentiles · Clipping · Broadcasting · Outliers

    NumPy·Easy·15 minEasy15m–Solve
  • Cosine Similarity Without Loops

    Broadcasting · Vectorization · Linear algebra · Numerical stability

    NumPy·Easy·20 minEasy20m–Solve
  • Flatten a Batch of Images

    reshape · Shape manipulation · Views · NumPy

    NumPy·Easy·15 minEasy15m–Solve
  • Normalise Rows to Unit Length

    Norms · keepdims · Broadcasting · Embeddings

    NumPy·Easy·15 minEasy15m–Solve
  • One-Hot Encode a Label Vector

    Fancy indexing · Broadcasting · NumPy · Encoding

    NumPy·Easy·15 minEasy15m–Solve
  • Pad Sequences to a Fixed Length

    Array construction · Slicing · Padding · NLP preprocessing

    NumPy·Easy·20 minEasy20m–Solve
  • Pairwise Euclidean Distances

    Broadcasting · Axis reduction · Distance metrics · NumPy

    NumPy·Easy·20 minEasy20m–Solve
  • Pick the Rows That Pass a Threshold

    Boolean masking · Column indexing · Filtering · NumPy

    NumPy·Easy·15 minEasy15m–Solve
  • Reproducible Train and Test Indices

    Random generators · Permutation · Reproducibility · NumPy

    NumPy·Easy·15 minEasy15m–Solve
  • Running Share of a Total

    cumsum · Normalisation · Vectorization · NumPy

    NumPy·Easy·15 minEasy15m–Solve
  • Scale Every Column to 0 and 1

    Broadcasting · Axis reduction · Feature scaling · NumPy

    NumPy·Easy·15 minEasy15m–Solve
  • Group Sums Without a GroupByPro

    bincount · Scatter-add · Safe division · Vectorization

    NumPy·Medium·25 minMedium25m–Preview
  • Intersection Over Union for Every Box PairPro

    Broadcasting · Clipping · Geometry · Vectorization

    NumPy·Medium·25 minMedium25m–Preview
  • Moving Average Without a LoopPro

    Prefix sums · Vectorization · Off-by-one reasoning · Dtype handling

    NumPy·Medium·20 minMedium20m–Preview
  • Softmax That Does Not Overflow

    Numerical stability · Broadcasting · Axis handling · Vectorization

    NumPy·Medium·25 minMedium25m–Solve
  • Standardise Features That Have Holes In ThemPro

    Missing data · Axis handling · Numerical guards · Feature scaling

    NumPy·Medium·25 minMedium25m–Preview
  • Top-K Retrieval Without Sorting EverythingPro

    argpartition · Vectorization · Ranking · Complexity

    NumPy·Medium·25 minMedium25m–Preview
  • Fill the Gaps in Every RowPro

    Accumulate tricks · Fancy indexing · Missing data · Vectorization

    NumPy·Hard·40 minHard40m–Preview
  • Add a Line Total Column

    Vectorised arithmetic · assign · Immutability · Pandas

    Pandas·Easy·15 minEasy15m–Solve
  • Attach Customer Names to Orders

    merge · Joins · Referential integrity · Pandas

    Pandas·Easy·20 minEasy20m–Solve
  • Average Order Value Per Customer

    groupby · Aggregation · reset_index · Pandas

    Pandas·Easy·15 minEasy15m–Solve
  • Count Orders Per Customer

    value_counts · groupby size · Pandas

    Pandas·Easy·15 minEasy15m–Solve
  • Each Product's Share of Its Category

    groupby transform · Broadcasting · Normalisation · Pandas

    Pandas·Easy·20 minEasy20m–Solve
  • Fill Missing Prices With the Median

    fillna · Missing data · Imputation · Pandas

    Pandas·Easy·15 minEasy15m–Solve
  • Filter Rows and Keep Two Columns

    Boolean indexing · Column selection · Pandas

    Pandas·Easy·15 minEasy15m–Solve
  • Keep Only Each User's Latest Row

    sort_values · drop_duplicates · Deduplication · Pandas

    Pandas·Easy·20 minEasy20m–Solve
  • Monthly Active Users

    groupby · Datetime handling · Deduplication · pandas

    Pandas·Easy·15 minEasy15m–Solve
  • Pull the Month Out of a Timestamp

    to_datetime · dt accessor · Dates · Pandas

    Pandas·Easy·15 minEasy15m–Solve
  • Tidy Up Column Names

    String methods · Column renaming · Data cleaning · Pandas

    Pandas·Easy·15 minEasy15m–Solve
  • Top N Products by Revenue

    groupby · sort_values · Ranking · Pandas

    Pandas·Easy·15 minEasy15m–Solve
  • Join Without Letting a Dirty Dimension Table Multiply Your RowsPro

    merge · Deduplication · Data quality · Join semantics

    Pandas·Medium·25 minMedium25m–Preview
  • Pad Each Store's Own Calendar, Not the Global OnePro

    Time series · GroupBy · Reindexing · Data leakage

    Pandas·Medium·30 minMedium30m–Preview
  • Pivot Event Counts, With the Zeros IncludedPro

    pivot / unstack · Reshaping · Categorical data · GroupBy

    Pandas·Medium·25 minMedium25m–Preview
  • Pull the SKU Out of a Free-Text NotePro

    str.extract · Regular expressions · Data cleaning · Missing data

    Pandas·Medium·20 minMedium20m–Preview
  • Revenue Per Customer From a Messy CSVPro

    read_csv · Filtering · GroupBy · Data cleaning

    Pandas·Medium·25 minMedium25m–Preview
  • Rolling 7-Day Spend Per CustomerPro

    GroupBy · Time series · Rolling windows · Feature engineering

    Pandas·Medium·30 minMedium30m–Preview
  • Sessionise an Event StreamPro

    GroupBy · Time series · cumsum · Feature engineering

    Pandas·Medium·30 minMedium30m–Preview
  • The Top Three Per Group, Deterministically

    sort_values · GroupBy · Ranking · Tie-breaking

    Pandas·Medium·25 minMedium25m–Solve
  • A Monthly Retention Cohort GridPro

    Cohort analysis · Period arithmetic · GroupBy · Dense reindexing

    Pandas·Hard·45 minHard45m–Preview
  • Point-in-Time Feature JoinPro

    merge_asof · Data leakage · Feature stores · Time series

    Pandas·Hard·35 minHard35m–Preview
  • Macro and Micro F1, From the Confusion Matrix Alone

    Model evaluation · Multiclass metrics · Precision/recall · F1

    Machine Learning·Easy·20 minEasy20m–Solve
  • Precision, Recall and F1 by Hand

    Model evaluation · Precision/recall · Confusion matrix · NumPy

    Machine Learning·Easy·15 minEasy15m–Solve
  • A Bootstrap Confidence Interval for a MetricPro

    Bootstrap · Confidence intervals · Resampling · Statistics

    Machine Learning·Medium·25 minMedium25m–Preview
  • A Pipeline That Does Not LeakPro

    Pipelines · Data leakage · Cross-validation · Preprocessing

    Machine Learning·Medium·30 minMedium30m–Preview
  • A Split That Does Not Leak Across GroupsPro

    Data leakage · Cross-validation · Grouped data · Evaluation

    Machine Learning·Medium·30 minMedium30m–Preview
  • How Wrong Are Your Probabilities?Pro

    Calibration · Model evaluation · Probability · Binning

    Machine Learning·Medium·25 minMedium25m–Preview
  • k-Nearest Neighbours, With a Tie-Break That Never ChangesPro

    k-NN · Distance metrics · Tie-breaking · Vectorization

    Machine Learning·Medium·25 minMedium25m–Preview
  • Permutation Importance, Averaged HonestlyPro

    Feature importance · Permutation testing · Reproducibility · Model evaluation

    Machine Learning·Medium·25 minMedium25m–Preview
  • Pick a Threshold for an Imbalanced Classifier

    Imbalanced data · Model evaluation · Precision/recall · Thresholding

    Machine Learning·Medium·25 minMedium25m–Solve
  • The Best Single-Feature Split, by GiniPro

    Decision trees · Gini impurity · Vectorization · Cumulative counts

    Machine Learning·Medium·30 minMedium30m–Preview
  • Would You Notice the Input Drifting?Pro

    Drift detection · Monitoring · Quantile binning · Model maintenance

    Machine Learning·Medium·30 minMedium30m–Preview
  • k-Means, and the Cluster That Loses All Its PointsPro

    Clustering · k-means · Convergence · Vectorization

    Machine Learning·Hard·45 minHard45m–Preview
  • ReLU, Forward and Backward

    Backpropagation · Activations · NumPy · Gradients

    Deep Learning·Easy·15 minEasy15m–Solve
  • Adam, One Step at a Time

    Optimisers · Adam · Bias correction · Training dynamics

    Deep Learning·Medium·25 minMedium25m–Solve
  • Batch Norm Behaves Differently at InferencePro

    Batch normalisation · Running statistics · Train/eval mode · NumPy

    Deep Learning·Medium·30 minMedium30m–Preview
  • Linear Layer Forward and BackwardPro

    Backpropagation · Chain rule · Matrix calculus · Gradient checking

    Deep Learning·Medium·35 minMedium35m–Preview
  • Layer Norm, BackwardsPro

    Backpropagation · Layer normalisation · Chain rule · NumPy

    Deep Learning·Hard·40 minHard40m–Preview
  • Scaled Dot-Product AttentionPro

    Attention · Transformers · Masking · Numerical stability

    Deep Learning·Hard·40 minHard40m–Preview
  • Count the Common N-grams

    Tokenisation · N-grams · Counting · Deterministic ordering

    NLP·Easy·15 minEasy15m–Solve
  • Chunk a Document for RetrievalPro

    Chunking · RAG · Text processing · Sliding window

    NLP·Medium·25 minMedium25m–Preview
  • Chunk-Level Named Entity MergingPro

    Named entity recognition · BIO tagging · Span extraction · Sequence labelling

    NLP·Medium·25 minMedium25m–Preview
  • TF-IDF From Scratch

    TF-IDF · Sparse representations · Lexical retrieval · Vectorization

    NLP·Medium·30 minMedium30m–Solve
  • Byte-Pair Encoding TokenizerPro

    Tokenization · BPE · Subword models · Vocabulary construction

    NLP·Hard·40 minHard40m–Preview
  • Chunk a Document by Size

    RAG · Chunking · Overlap · Retrieval

    LLM / GenAI·Easy·15 minEasy15m–Solve
  • Reciprocal Rank Fusion

    Hybrid search · Rank fusion · Retrieval · RAG

    LLM / GenAI·Easy·20 minEasy20m–Solve
  • A Semantic Cache That Forgets ProperlyPro

    Caching · Cosine similarity · LRU eviction · Cost control

    LLM / GenAI·Medium·30 minMedium30m–Preview
  • Fit the Conversation in the Window

    Context window · Token budgeting · Chat applications · Prompt construction

    LLM / GenAI·Medium·25 minMedium25m–Solve
  • Retrieve Five Passages, Not One Passage Five TimesPro

    RAG · Maximal marginal relevance · Retrieval · Cosine similarity

    LLM / GenAI·Medium·30 minMedium30m–Preview
  • Structure-Aware Chunker With OverlapPro

    Chunking · RAG · Context construction · Retrieval quality

    LLM / GenAI·Medium·30 minMedium30m–Preview
  • Reassemble a Streamed Response

    Streaming · Server-sent events · Parsing · LLM plumbing

    AI Engineering·Easy·15 minEasy15m–Solve
  • A Rate Limiter That Allows a BurstPro

    Rate limiting · Token bucket · Backpressure · API design

    AI Engineering·Medium·25 minMedium25m–Preview
  • Batch Requests Without Blowing the Latency BudgetPro

    Dynamic batching · Inference serving · Latency budgets · Throughput

    AI Engineering·Medium·30 minMedium30m–Preview
  • Retry With Exponential Backoff and Jitter

    Retries · Backoff · Thundering herd · Error handling

    AI Engineering·Medium·25 minMedium25m–Solve
  • Stop Calling the Provider That Is DownPro

    Circuit breaker · Resilience · State machines · Failure handling

    AI Engineering·Medium·30 minMedium30m–Preview
  • Token Cost From a Raw Usage LogPro

    Cost tracking · Data cleaning · CSV parsing · Normalization

    AI Engineering·Medium·25 minMedium25m–Preview
  • Token-Budgeted Context PackerPro

    Context management · Token budgets · RAG · Prompt construction

    AI Engineering·Medium·25 minMedium25m–Preview
  • Design a URL Shortener

    Capacity estimation · Key generation · Caching · Read-heavy systems

    ML System Design·Easy·35 minEasy35m–Design
  • Diagnose Model Drift

    Drift Analysis · Root Cause Analysis · Covariate Shift · Concept Drift

    ML System Design·Easy·25 minEasy25m–Design
  • Design a Demand Forecasting PlatformPro

    Time Series · Hierarchical Forecasting · Backtesting · Intermittent Demand

    ML System Design·Medium·40 minMedium40m–Preview
  • Design a Large-Scale Classification PlatformPro

    Extreme Multi-Label · Hierarchical Classification · Active Learning · Human-in-the-Loop

    ML System Design·Medium·40 minMedium40m–Preview
  • Design a Predictive Maintenance SystemPro

    Sensor Data · Survival Analysis · Rare Events · Edge Inference · Time Windows

    ML System Design·Medium·40 minMedium40m–Preview
  • Design a Real-Time Anomaly Detection SystemPro

    Time Series · Unsupervised Learning · Streaming · Alerting · Seasonality

    ML System Design·Medium·40 minMedium40m–Preview
  • Design a Recommendation SystemPro

    Candidate Generation · Ranking · Embeddings · Two-Tower Models · Cold Start

    ML System Design·Medium·45 minMedium45m–Preview
  • Design an Image Search SystemPro

    Vision Embeddings · ANN Search · Multimodal · Index Sharding · CLIP

    ML System Design·Medium·40 minMedium40m–Preview
  • Design an ML Training Pipeline

    Pipelines · Reproducibility · Experiment Tracking · Data Versioning · Orchestration

    ML System Design·Medium·35 minMedium35m–Design
  • Design Feature MonitoringPro

    Data Quality · Skew Detection · Lineage · Freshness SLAs · Root Cause

    ML System Design·Medium·30 minMedium30m–Preview
  • Design Model MonitoringPro

    Drift Detection · Delayed Labels · Alerting · Segment Analysis · Observability

    ML System Design·Medium·35 minMedium35m–Preview
  • Diagnose Training-Serving SkewPro

    Skew Debugging · Feature Pipelines · Data Leakage · Root Cause Analysis

    ML System Design·Medium·30 minMedium30m–Preview
  • Improve a Low-Recall ModelPro

    Imbalanced Data · Threshold Tuning · Error Analysis · Precision-Recall Tradeoff

    ML System Design·Medium·30 minMedium30m–Preview
  • Debug a Production ML ModelPro

    Incident Response · Root Cause Analysis · Debugging · Systems Thinking

    ML System Design·Hard·35 minHard35m–Preview
  • Design a Model Serving PlatformPro

    Model Serving · GPU Scheduling · Autoscaling · Canary Deploys · Multi-tenancy

    ML System Design·Hard·45 minHard45m–Preview
  • Design a Personalized News FeedPro

    Ranking · Fan-out · Multi-Objective Optimization · Freshness · Feedback Loops

    ML System Design·Hard·45 minHard45m–Preview
  • Design a Real-Time Fraud Detection SystemPro

    Streaming · Imbalanced Data · Feature Store · Low Latency · Adversarial ML

    ML System Design·Hard·50 minHard50m–Preview
  • Design a Search Ranking SystemPro

    Learning to Rank · BM25 · Query Understanding · Click Models · Position Bias

    ML System Design·Hard·45 minHard45m–Preview
  • Design an Ad Ranking SystemPro

    CTR Prediction · Auctions · Calibration · Delayed Feedback · Budget Pacing

    ML System Design·Hard·50 minHard50m–Preview
  • Design an ML Feature StorePro

    Feature Store · Point-in-Time Correctness · Streaming · Data Engineering

    ML System Design·Hard·45 minHard45m–Preview
  • Design Continuous ML DeploymentPro

    CI/CD · Canary Analysis · Shadow Deploys · Rollback · Automated Promotion

    ML System Design·Hard·40 minHard40m–Preview
  • Design a ChatGPT-like ApplicationPro

    Streaming · Context Management · Conversation State · Safety · Cost Control

    LLM System Design·Medium·40 minMedium40m–Preview
  • Design a Self-Serve LLM Fine-Tuning PlatformPro

    GPU scheduling · Model registry · LoRA · Eval gating · Multi-tenancy

    LLM System Design·Medium·40 minMedium40m–Preview
  • Design an LLM Gateway

    API Gateway · Rate Limiting · Caching · Observability · Failover

    LLM System Design·Medium·40 minMedium40m–Design
  • Design an LLM Evaluation PlatformPro

    LLM-as-Judge · Eval Design · Regression Testing · Human Labeling · Statistics

    LLM System Design·Hard·45 minHard45m–Preview
  • Design an LLM Inference Serving SystemPro

    Continuous batching · KV cache · GPU autoscaling · Quantization · Latency budgets

    LLM System Design·Hard·45 minHard45m–Preview
  • Design a Document Intelligence PlatformPro

    OCR · Structured Extraction · Confidence Calibration · Human Review · Multimodal

    RAG·Medium·40 minMedium40m–Preview
  • Design a Semantic Search System

    Embeddings · Hybrid Search · Chunking · Reranking · Index Design

    RAG·Medium·35 minMedium35m–Design
  • Design a Production RAG SystemPro

    RAG · Embeddings · Vector DB · Reranking · LLM Evaluation · Access Control

    RAG·Hard·45 minHard45m–Preview
  • Design an Enterprise Knowledge AssistantPro

    RAG · Tool Use · Personalization · Enterprise Integration · Guardrails

    RAG·Hard·45 minHard45m–Preview
  • Design an AI Agent for Customer Refunds

    Tool calling · Irreversible actions · Policy enforcement · Escalation

    Agentic AI·Medium·40 minMedium40m–Design
  • Design a Multi-Agent AI SystemPro

    Agent Orchestration · Planning · State Management · Error Recovery · Cost Control

    Agentic AI·Hard·50 minHard50m–Preview
  • Design a Multi-Agent Data AnalystPro

    Multi-agent · Text-to-SQL · Sandboxing · Verification · Human-in-the-loop

    Agentic AI·Hard·50 minHard50m–Preview
  • Design a Multi-Agent Incident Response SystemPro

    Multi-agent orchestration · Observability · Irreversible actions · Coordination

    Agentic AI·Hard·50 minHard50m–Preview
  • Design a Self-Correcting Code AgentPro

    Coding agents · Repository context · Verification loops · Sandboxing

    Agentic AI·Hard·50 minHard50m–Preview
  • Design a Tool-Calling Agent Over 500 APIsPro

    Tool selection · Retrieval · Permissions · Failure handling · Observability

    Agentic AI·Hard·45 minHard45m–Preview
  • Design an AI Coding AssistantPro

    Code Context · Low Latency · Fill-in-the-Middle · Repo Indexing · Acceptance Metrics

    Agentic AI·Hard·45 minHard45m–Preview
  • Design an AI Customer Support AgentPro

    Agents · Tool Use · Escalation · Safety · Containment Rate

    Agentic AI·Hard·45 minHard45m–Preview
  • Design an Autonomous Research AgentPro

    Agent planning · Tool use · Verification · Termination · Cost control

    Agentic AI·Hard·50 minHard50m–Preview