A churn prediction model has been in production for 14 months. It was deployed at 0.84 AUC on the holdout set and held around 0.82 in production for the first year.
Over the last 10 weeks, production AUC has declined steadily to 0.71. There was no deploy, no code change, and no model retrain in that period. The prediction volume is unchanged. Infrastructure metrics are all normal.
The business is asking whether to retrain, and you have been asked for a diagnosis first.
You have read the opening. The rest is free with a Bezen account, and no card is needed.
Behind the sign-in