An arXiv benchmark on IBM's 156-qubit Heron processor finds Q-CTRL led exact-output sampling while Qedma delivered the lowest estimator error at a much higher QPU time cost
On IBM's 156-qubit Heron processor, Q-CTRL produced the strongest exact-output results in every sampling test examined by a new independent benchmark, while Qedma QESEM achieved the lowest observable-estimation error at a far greater runtime cost.
Three software stacks were tested. The study is an arXiv preprint from researchers at The Catholic University of America, the University of Deusto, and Universidad de los Andes. It compares IBM Qiskit Runtime's native error-management primitives with two managed pipelines available through the Qiskit Functions Catalog: Q-CTRL Performance Management and Qedma QESEM.
All experiments ran on ibm_pittsburgh, a 156-qubit IBM Quantum Heron R3 superconducting processor. IBM's Heron line was introduced in 2023 and has appeared in both 133-qubit and 156-qubit configurations; the larger device has become a recurring platform for heavy-hex benchmarking and studies of error suppression. Plans announced by IBM and the University of Tokyo to deploy a 156-qubit Heron in IBM Quantum System One for Japan's Quantum Innovation Initiative Consortium also indicate that this hardware class is being positioned for continuing research and commercial access.
The researchers tested two different interfaces because quantum workloads do not all produce the same kind of output. Sampler jobs return bitstring counts from circuit measurements, whereas Estimator jobs return expectation values for selected observables. This distinction is comparable to separating a classification-like success probability from a numerical measurement error: a pipeline can be excellent at recovering one particular bitstring yet less effective at estimating an average physical quantity.
IBM and Q-CTRL used matched budgets of 215 shots per instance, or 32,768 repetitions. Qedma QESEM instead used its native precision-and-time configuration with a 0.1 precision target and a 600-second QPU time cap. Abstract workloads were submitted during identical hardware-calibration windows so that changing device conditions would not be an obvious explanation for the comparison.
The article reports the laboratory affiliations and workload-level results, but it does not provide p-values or confidence intervals for the stack-to-stack differences. The reported figures should therefore be read as benchmark measurements under specified execution conditions, not as a formal population-level significance analysis. The sample structure is explicit for the Sampler track-12 test cases-and includes three system sizes for the Estimator track, but it is not a randomized clinical-style trial of all possible circuits or hardware states.
Sampling favored Q-CTRL. Q-CTRL led all 12 Sampler test cases across Bernstein-Vazirani circuits from 25 to 75 qubits, Quantum Phase Estimation from 10 to 30 counting qubits, GHZ-state preparation from 25 to 50 qubits, and Randomized Mirror Circuits from 25 to 100 qubits.
The sharpest separation appeared in 30-qubit Quantum Phase Estimation. IBM's raw and measurement-twirled runs produced no exact-match shots in 32,768 attempts, corresponding to a measured success probability of 0.00%. Q-CTRL reached 12.69%. In the 100-qubit Randomized Mirror Circuit test, Q-CTRL produced the exact 100-bit target string in 76.45% of trials, compared with 9.14% for IBM raw execution.
A separate 2026 demonstration used Q-CTRL software for a 100-qubit Quantum Fourier Transform on IBM's ibm_boston Heron R3 processor, where the correct frequency bitstring remained the highest-probability outcome. Taken together, these results suggest that software-level compilation, calibration, and control strategies can materially alter the distribution of measured outputs on the same physical-qubit technology, although they do not establish performance for every algorithmic family.
These figures describe error suppression and improved sampling outcomes rather than error correction. The processor still used physical qubits, and the benchmark does not report logical qubits, fault-tolerant operations, or elimination of hardware errors. Its result is narrower but useful: under the stated workloads, one managed control pipeline preserved target outputs more effectively than the tested native configurations.
The result also belongs to a different layer of quantum engineering than a network demonstration such as an earlier network study. That work concerned communication over a quantum link, whereas this benchmark measures how software-level execution choices affect computation on one superconducting processor.
Estimator accuracy cost time. The Estimator track used an eight-layer Transverse-Field Ising Model circuit at 25, 50, and 75 qubits. Results were compared with an exact Matrix Product State reference and reported as mean absolute error for magnetization mX, correlators cZZ, and their overall average.
IBM raw execution produced an overall MAE of 0.0883 with about 17.8 seconds of QPU time per job. IBM TREX with twirling lowered overall MAE to 0.0807 but required roughly 28.0 to 37.8 seconds. Q-CTRL reached 0.0285 overall MAE with about 28.0 seconds per job, while Qedma QESEM reached 0.0188 with 211 to 311 seconds per job.
On the aggregate measure, Q-CTRL reduced error by a factor of 3.10 compared with raw execution and Qedma reduced it by 4.70. The component values show the same tradeoff: Q-CTRL recorded magnetization and correlator MAEs of 0.0376 and 0.0194, while Qedma recorded 0.0278 and 0.0097. Qedma therefore delivered the best numerical accuracy in this track but consumed 7.5 to 11.1 times as much QPU time as Q-CTRL.
That tradeoff matters because QPU time is a constrained resource in cloud quantum computing. A lower estimator error may be preferable for a precision-sensitive simulation, while a slightly less accurate result obtained with much shorter hardware occupation may be more useful for iterative algorithm development. The benchmark consequently evaluates at least two distinct objective functions: output fidelity for Sampler workloads and numerical accuracy per unit of QPU time for Estimator workloads.
The wider Heron ecosystem reinforces this hybrid view of quantum computing. In February 2026, IBM and RIKEN reported a closed-loop workflow linking an IBM Heron processor with the Fugaku supercomputer for a quantum-chemistry computation. Such workflows place the quantum processor inside a classical optimization and analysis loop rather than treating it as a standalone replacement for a supercomputer. The role of calibration, compilation, and post-processing is therefore likely to remain central even as hardware generations change.
IBM researchers have also reported a Heron-based error-suppression method that reduced inferred sampling overhead by as much as 63 times in one workload, from 85,545 samples at depth six to 1,359. That result was described as an IBM-authored preprint rather than independently reproduced evidence. Its headline reduction, like the present benchmark, is workload-specific: sampling overhead is an operational metric and should not be interpreted as a universal reduction in physical error rates.
What the benchmark establishes. The comparison makes a practical distinction that is often lost in processor-level discussions. A suppression pipeline can improve the reliability of measured outputs without changing the processor into a fault-tolerant machine. Error mitigation uses calibration, characterization, extrapolation, or related processing to reduce the effect of noise after or around an execution; quantum error correction encodes information across physical qubits and actively detects and corrects errors.
Here the strongest evidence is workload-specific performance under controlled execution conditions. Q-CTRL was dominant for exact bitstring recovery, while Qedma was more accurate for the TFIM observables. IBM's native settings were not uniformly inferior in every metric either: TREX and twirling improved the overall MAE over raw execution but did not match either third-party pipeline on that aggregate result.
The benchmark does not establish a general computational advantage over classical computing. The TFIM reference was an exact Matrix Product State calculation, and the study reports agreement with that reference rather than a useful application completed faster than a classical alternative. Nor does the paper show that either third-party stack scales economically to larger processors or longer algorithms.
This limitation is central to the commercial question. Q-CTRL's result is attractive because it paired a substantial reduction in estimator error with QPU time close to the IBM baseline. Qedma's higher precision came with a large sampling and characterization burden. In cloud quantum computing, the best numerical answer is not automatically the best operational choice when access time is scarce.
The distinction resembles the methodological caution used in fields such as Nature research, where an effect observed under a defined experimental protocol is not automatically generalized beyond the tested population or mechanism. For quantum benchmarking, that means reporting circuit families, shot budgets, calibration windows, estimator definitions, runtime, and uncertainty is as important as reporting a single headline percentage.
This preprint therefore offers a more valuable conclusion than a simple ranking. It shows that error-management software can change the measured performance of the same hardware and that different pipelines optimize different objectives. The evidence supports treating suppression quality and resource overhead as separate benchmark dimensions; it does not support claims of fault tolerance or broad quantum utility. For users deciding between cloud workflows, Q-CTRL appears strongest on the tested sampling and time-efficiency profile, while Qedma leads on estimator accuracy at a steep QPU cost.
Q-CTRL has indicated that it will continue releasing technical updates, larger datasets, and benchmarking insights, suggesting that its calibration and performance-management stack remains under active development. Future comparisons would be stronger if they included repeated calibration windows, confidence intervals, preregistered workloads, independent replication, and a common accounting method for classical preprocessing and post-processing time.
Error mitigation is best understood as a correction to the interpretation of noisy measurements rather than a replacement for logical qubits. A logical qubit requires an encoding and repeated syndrome information that are absent from this comparison. That distinction makes the benchmark technically meaningful but also sets its boundary: it measures better use of noisy physical hardware, not the arrival of a fault-tolerant quantum computer.
