Tuesday, 20 October 2026 · 6 min read
LLM agents approach but do not yet match expert-level operations research solutions
OR-License benchmark tests 11 large language models on 31 real-world OR tasks with client interaction; best model achieves 88.39% feasibility and 0.87 deployment-adjusted quality versus expert reference of 1, revealing gaps in sustained information acquisition and problem formulation.
Based on the paper OR-License: Evaluating LLM Agents as Real-World Operations Research Practitioners by Peisong Wang, Feixue Liu, Yize Meng, Bowen Liu and 15 others
Challenges in automating real-world operations research with LLMs
Operations research (OR) involves transforming incomplete, ambiguous business requirements into executable optimization models and solutions, a process requiring expert knowledge in problem formulation, algorithm selection, and implementation. This complexity makes automating OR challenging.
Existing benchmarks for LLMs in OR often use synthetic or simplified tasks with fully specified problem descriptions, ignoring the critical step of translating original client materials into formal models.
Moreover, real OR projects require iterative information acquisition from clients to clarify requirements and validate solutions, a process not captured in prior evaluations, limiting assessment of LLMs as practical OR practitioners.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- Three key formulas from the paper, explained symbol by symbol
- Three figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short