A delivery-time prediction model scored a mean absolute error of 3.2 minutes on its offline test set. In production it is running at 7.8 minutes: more than double.
The model has been deployed twice with the same result. The offline evaluation has been re-run and reproduces 3.2 minutes. The serving code has been reviewed by two engineers and looks correct. Traffic is normal and the input volume matches expectations.
The team is convinced the model is fine and something in production is broken, but they cannot find it.
You have read the opening. The rest is free with a Bezen account, and no card is needed.
Behind the sign-in