Bezen
Log inSign up

Practice Problems

Realistic AI engineering scenarios with reference solutions and problem-specific grading rubrics.

12 problems
ProblemCategoryDifficultyTimeBest
  • Macro and Micro F1, From the Confusion Matrix Alone

    Model evaluation · Multiclass metrics · Precision/recall · F1

    Machine Learning·Easy·20 minEasy20m–Solve
  • Precision, Recall and F1 by Hand

    Model evaluation · Precision/recall · Confusion matrix · NumPy

Machine Learning
·
Easy
·
15 min
Easy
15m
–
Solve
  • A Bootstrap Confidence Interval for a Metric

    Bootstrap · Confidence intervals · Resampling · Statistics

    Machine Learning·Medium·25 minMedium25m–Solve
  • A Pipeline That Does Not Leak

    Pipelines · Data leakage · Cross-validation · Preprocessing

    Machine Learning·Medium·30 minMedium30m–Solve
  • A Split That Does Not Leak Across Groups

    Data leakage · Cross-validation · Grouped data · Evaluation

    Machine Learning·Medium·30 minMedium30m–Solve
  • How Wrong Are Your Probabilities?

    Calibration · Model evaluation · Probability · Binning

    Machine Learning·Medium·25 minMedium25m–Solve
  • k-Nearest Neighbours, With a Tie-Break That Never Changes

    k-NN · Distance metrics · Tie-breaking · Vectorization

    Machine Learning·Medium·25 minMedium25m–Solve
  • Permutation Importance, Averaged Honestly

    Feature importance · Permutation testing · Reproducibility · Model evaluation

    Machine Learning·Medium·25 minMedium25m–Solve
  • Pick a Threshold for an Imbalanced Classifier

    Imbalanced data · Model evaluation · Precision/recall · Thresholding

    Machine Learning·Medium·25 minMedium25m–Solve
  • The Best Single-Feature Split, by Gini

    Decision trees · Gini impurity · Vectorization · Cumulative counts

    Machine Learning·Medium·30 minMedium30m–Solve
  • Would You Notice the Input Drifting?

    Drift detection · Monitoring · Quantile binning · Model maintenance

    Machine Learning·Medium·30 minMedium30m–Solve
  • k-Means, and the Cluster That Loses All Its Points

    Clustering · k-means · Convergence · Vectorization

    Machine Learning·Hard·45 minHard45m–Solve