A single number — 0.83 AUC — says nothing about how much you'd trust it on a different sample of the same size. The bootstrap answers that without assuming any particular distribution, and it has exactly one step that is easy to get backwards.
Write bootstrap_ci(y_true, y_pred, metric_fn, n_boot, seed, alpha).
metric_fn(y_true, y_pred) on the original, unresampled data.rng = np.random.default_rng(seed).n_boot times: draw n row indices with replacement using rng, and apply the same drawn indices to both y_true and y_pred together — never resample them independently. Compute metric_fn on that resampled pair.(point_estimate, lower, upper), where lower and upper are the 100 · alpha / 2 and 100 · (1 − alpha / 2) percentiles of the n_boot bootstrap scores — the percentile bootstrap. For alpha = 0.05 that is a 95% interval: the 2.5th and 97.5th percentiles, not the 5th and 95th.You have read the opening. The rest is free with a Bezen account, and no card is needed.
Behind the sign-in