A global consortium has released the Quantum Optimization Benchmarking Library, providing a model-independent framework to compare quantum, classical, and hybrid algorithms on hard combinatorial problems. The open-source platform aims to clarify claims of quantum advantage.
The Quantum Optimization Benchmarking Library (QOBLIB), developed by an international team led by IBM Quantum, Zuse Institute Berlin, Technische Universität Berlin, and Purdue University, introduces a new open-source standard for evaluating quantum, classical, and hybrid optimization algorithms. Published in Nature Computational Science, QOBLIB is designed to address a persistent challenge in quantum computing: the lack of rigorous, reproducible benchmarks for comparing quantum algorithms with the best available classical methods on genuinely hard problems.
Benchmarking Across Problem Classes
QOBLIB focuses on ten classes of NP-hard combinatorial optimization problems, including market split, low-autocorrelation binary sequences, minimum Birkhoff decomposition, Steiner tree packing, sports tournament scheduling, portfolio optimization, maximum independent set, network design, capacitated vehicle routing, and topology design. The library curates 1,264 specific problem instances, with variable counts ranging from 20 to over 3 million, selected to become computationally challenging for classical solvers at realistic scales. Each instance is available in multiple mathematical formulations-such as mixed-integer programming (MIP), integer linear programming (ILP), and quadratic unconstrained binary optimization (QUBO)-to enable fair cross-platform comparison.
Model-Independent Evaluation and Public Tracking
Unlike hardware-specific benchmarks, QOBLIB employs a model-independent architecture. Researchers can submit solutions using any algorithm or hardware, provided the problem instance and formulation are preserved. The public web portal features an interactive visualization of the complexity landscape, mapping instances by size and matrix density. A live registry tracks the best-known solutions from both classical and quantum approaches, with automated validation to ensure reproducibility and fair reporting. At launch, the platform had already received over 2,600 benchmark submissions from 24 institutions, including industrial users, national supercomputing centers, academic groups, and commercial quantum software providers.
Classical Baselines and Quantum Claims
One of QOBLIB's central aims is to clarify the conditions under which quantum algorithms can claim a practical advantage. While heuristic quantum algorithms such as the Quantum Approximate Optimization Algorithm (QAOA) and quantum annealing have shown promise, their performance must be measured against the strongest classical solvers. The library's curated instances are specifically chosen to be hard for classical methods, and the evolving nature of classical baselines is evident: since QOBLIB's initial preprint, classical solvers have nearly doubled the size of the largest solved market split instances, raising the bar for quantum hardware to demonstrate a clear advantage. This dynamic is reminiscent of other recent efforts to benchmark quantum and hybrid algorithms in operational settings, such as the collaboration described in a recent Science Report article on hybrid quantum-classical optimization for defense logistics.
Open Standards and Community Adoption
QOBLIB's open-source approach and transparent submission process are intended to foster broad adoption and continuous improvement. The library is co-authored by researchers from the founding institutions, with lead contributions from Thorsten Koch and Stefan Woerner. Early participation from a diverse set of contributors-including energy companies, supercomputing centers, and quantum software startups-demonstrates the demand for standardized, reproducible benchmarks in the field. The platform's live tracking of best-known solutions and automated validation tools are designed to reduce ambiguity and ensure that claims of quantum advantage are grounded in fair, up-to-date comparisons.
Benchmarking in quantum optimization requires careful attention to the definition of advantage, the selection of problem instances, and the evolving capabilities of both quantum and classical hardware. QOBLIB's model-independent framework and public registry represent a significant step toward transparent, reproducible evaluation of quantum algorithms as the field moves beyond proof-of-principle demonstrations.
Understanding quantum advantage in optimization depends on more than raw processor speed or qubit count. A fair comparison requires that quantum and classical algorithms tackle the same well-defined problem, using the same input data and constraints, and that results are verified using reproducible methods. As classical algorithms and hardware continue to improve, the threshold for demonstrating a genuine quantum advantage shifts. Transparent, model-independent benchmarks like those in QOBLIB are essential for distinguishing between incremental progress and transformative capability in quantum computing.
