Pick a Threshold for an Imbalanced Classifier

A fraud model outputs a probability. Somebody has to decide where to cut, and 0.5 is almost never the right answer.

Given y_true (0/1) and y_score (predicted probabilities), return the threshold that maximises F1.

Rules:

• The threshold is chosen from the candidate scores themselves: a threshold between two observed scores classifies identically to one at the lower of them
• A prediction is positive when score >= threshold
• If several thresholds tie on F1, return the largest of them (fewer alerts for the same F1 is strictly better operationally)
• Return a plain float

At 0.3% positives, accuracy is useless: a model that predicts "never fraud" scores 99.7%. F1 is what forces the trade between missing fraud and drowning the review team.