BizBench evaluates long-horizon agents on valid, constraint-compliant business task execution

Real-world business operations, such as payments, require agents to execute long, multi-step workflows governed by extensive, sometimes outdated or conflicting policies. Unlike simple task completion, agents must maintain valid execution at every step, as intermediate violations (e.g., unauthorized payments) can cause financial or regulatory harm regardless of the final outcome.

Existing benchmarks focus mainly on final task success and do not assess whether agents respect intermediate constraints or correctly interpret which business rules apply. This gap is critical because business rules are scattered across heterogeneous documents with varying scopes, versions, and effective periods, making it challenging for agents to identify authoritative guidance.

Furthermore, business workflows involve shared resources, temporal dependencies, and prior commitments, requiring agents to plan and adapt actions carefully. Violations at intermediate steps, such as breaching minimum reserve requirements or acting without approval, constitute task failure even if the final goal is met. Reliable deployment demands evaluation beyond outcome-based metrics.