Recent experiments have demonstrated groundbreaking quantum computational power in the laboratory, showing quantum computational advantage [AAB+19, ZWD+20, WBC+21, ZCC+22]. In the past decade, much theoretical work has gone into designing experimental protocols expressly for this purpose, and providing evidence for the classical hardness of reproducing the experimental results [AA11, FH19, LBR17, HM17, BIS+18, TER18, NRK+18, BGK18, BGK+20, BFN+19, AG20, BCM+21, BKV+20, KCV+22]. A difficulty with many of them, however, is that the quantum machine’s output is hard to verify. In many cases, the best known algorithm for directly checking the solution is equivalent to classically performing the computational task itself. This presents challenges for validation of the test’s results, because an ideal demonstration of quantum advantage occurs in the regime where a classical solution is not just difficult, but impossible with current technology. In that regime, experiments have had to resort to indirect methods to demonstrate that their devices are producing correct results [AAB+19, ZWD+20, WBC+21, ZCC+22].
In 2008, an efficiently-verifiable test of quantum computational advantage was proposed based on “instantaneous quantum polynomial-time” (IQP) circuits—quantum circuits in which all operations commute [SB09]. The protocol places only moderate requirements on the quantum device, making it potentially a good candidate for near-term hardware. Furthermore, later papers showed based on reasonable assumptions that classically sampling from the resulting distribution should be hard [BJS11, BMS16]. This suggests that a “black-box” approach to cheating classically (by simply simulating the quantum device) is indeed computationally hard, and only a couple hundred qubits would be required to make a classical solution intractable.
Importantly, however, the classical verifier of the efficiently-verifiable protocol does not explicitly check whether the prover’s samples come from the correct distribution (in fact, doing such a check efficiently is probably not possible [BJS11]). Instead, the sampling task is designed such that bitstrings from its distribution will be orthogonal to some secret binary vector s with high probability, and it is this property that is checked by the verifier. A question that has remained open is whether a classical machine can efficiently generate samples satisfying the orthogonality check, without necessarily approximating the actual circuit’s distribution. In this work we show that the answer to this question is yes. We give an explicit algorithm that can extract the secret bistring s underlying any instance of the protocol, thus making it trivial to generate orthogonal samples that pass the verifier’s test. The main results described here are a statement of the algorithm, a proof that a single iteration of it will extract the secret vector s with probability 12 (which can be made arbitrarily close to 1 by repetition), and empirical results demonstrating that the algorithm is efficient in practice (summarized in Figure 4.1).
The following is a summary of the paper’s structure. In Section 4.2, we review the protocol’s construction and some relevant analysis from the original paper. In Section 4.3 we describe the algorithm to extract the secret key, and therefore break the protocol’s security against classical provers. There we also discuss briefly our implementation of the algorithm. In Section 4.4 we discuss the results, and provide the secret key underlying the “$25 challenge” that accompanied the publication of the protocol.
Here we summarize the IQP-based protocol for quantum advantage, in the standard cryptographic terms of an ostensibly quantum prover attempting to prove its quantum capability to a classical verifier. We refer the reader to the work that proposed the protocol for any details not covered here [SB09]. The core of the protocol is a sampling problem. The verifier generates a Hamiltonian HP consisting of a sum of products of Pauli X operators, and asks the quantum prover to generate samples by measuring the state eiHPθ∣∣0⊗n⟩ for some value of the “action” θ. The Hamiltonian HP is designed such that the measured bitstrings {xi} are biased with respect to a secret binary vector s, so that xi⋅s=0 with high probability (where (⋅) represents the binary inner product, modulo 2). The classical verifier, with knowledge of s, can quickly check that the samples have such a bias. Since s should be only known to the verifier, it was conjectured that the only efficient way to generate such samples is by actually computing and measuring the quantum state [SB09]. However, in Section 4.3 we show that it is possible to extract s classically from just the description of the Hamiltonian.
A Hamiltonian of the type used in this protocol can be described by a rectangular matrix of binary values, for which each row corresponds to a term of the Hamiltonian. Given such a binary matrix P (called an “X-program”), the Hamiltonian is
| HP=∑i∏jXPij | (4.1) |
In words, a 1 in P at row i and column j corresponds to the inclusion of a Pauli X operator on the jth site in the ith term of the Hamiltonian. The X-program also has one additional parameter θ, which is the “action”—the integrated energy over time for which the Hamiltonian will be applied. For the protocol relevant to this work, the action is set to θ=π/8 (see below).
In order to bias the output distribution along s, a submatrix with special properties is embedded within the matrix P. Notationally, for a vector s and matrix P, let the submatrix Ps be that which is generated by deleting all rows of P that are orthogonal to s. Letting X represent the distribution of measurement results for a given X-program, it can be shown that the probability that a measurement outcome is orthogonal to the vector s, Pr[X⋅s=0], depends only on the submatrix Ps. The rows of P that are orthogonal to s are irrelevant. The protocol uses that fact to attempt to hide Ps (and thus s): starting with a matrix Ps that produces a bias, we may attempt to hide it in a larger matrix P by appending rows that are random aside from having p⋅s=0, and then scrambling the new, larger matrix in a way that preserves the bias.
But what matrix Ps should one start with? In the protocol, the verifier sets Ps to the generator matrix for a binary code of block length q≡7(mod8) whose codewords c have wt(c)∈{−1,0}(mod4) (and both those weights are represented, that is, the codewords do not all have weight 0(mod4)). In [SB09], the authors suggest specifically using a binary quadratic residue (QR) code because it has the desired codeword weights. The action θ is set to π/8. As described in Facts 1 and 2 below, this configuration leads to a gap between the quantum and classical probabilities of generating samples orthogonal to s (for the best known classical strategy before this work). The verifier’s check is then simply to request a large number of samples, and determine if the fraction orthogonal to s is too large to have likely been generated by any classical strategy.
In the two Facts below, we recall the probabilities corresponding to the quantum strategy and previously best-known classical strategy [SB09]. The reasoning behind the classical strategy (Fact 2) forms the setup for the new algorithm described in this paper; it is worth understanding its proof before moving on to the algorithm in Section 4.3.
Quantum strategy
Let P be an X-program constructed by embedding a submatrix with the properties described above. Let X be a random variable representing the distribution of bitstrings from an n-qubit quantum state eiHPπ/8|0⟩ measured in the Z basis, where HP is defined as in Equation 4.1. Then,
| Pr[X⋅s=0]=cos2(π8)≈0.85⋯ | (4.2) |
The entire proof is contained in [SB09]. To summarize, it is shown that for any string z and corresponding submatrix Pz, the probability is
| Pr[X⋅z=0]=Ec[cos2(θ⋅(q−2wt(c))] | (4.3) |
where θ is the action, q is the number of rows in Pz and the expectation is taken over the codewords c of the code generated by the submatrix Pz. When the values of θ=π/8, q≡7(mod8) and wt(c)∈{−1,0}(mod4) corresponding to the specific submatrix Ps are substituted into this expression, the result is Equation 4.2.
∎
Classical strategy of [SB09]
Again let P be an X-program constructed by embedding a submatrix with the properties described above. Let d,e be two bitstrings of length n (the length of a row of P). Define Pd,e as the matrix generated by deleting the rows of P orthogonal to d or e. 11 1 In [SB09], Pd,e is written as Pd∩Pe. Let y=∑pi∈Pd,epi be the vector sum of the rows of Pd,e. Letting Y be the random variable representing the distribution of y when d and e are chosen uniformly at random, then
| Pr[Y⋅s=0]=34 | (4.4) |
(From [SB09]) With y defined as above, we have
| y⋅s=∑pi∈Pd,epi⋅s | (4.5) |
By defintion, pi⋅s=1 if pi∈Ps. Therefore y⋅s is equivalent to simply counting the number of rows common to Ps and Pd,e, or equivalently, counting the rows in Ps for which p⋅d and p⋅e are both 1. We can express this using the matrix-vector products of Ps with d and e:
| y⋅s | =∑pi∈Ps(p⋅d)(p⋅e) | (4.6) | ||
| =(Ps d)⋅(Ps e) | (4.7) |
Considering that Ps is the generator matrix for an error correcting code, denote cd=Ps d as the encoding of d under Ps. Then we have
| y⋅s | =cd⋅ce | (4.8) |
Now, note that if a code has wt(c)∈{−1,0}(mod4) for all codewords c, the extended version of that code (created by adding a single parity bit) is doubly even, that is, has all codeword weights exactly 0(mod4). A doubly even binary code is necessarily self-dual, meaning all its codewords are orthogonal. This implies that any two codewords cd and ce of the original (non-extended) code have cd⋅ce=0 iff either cd or ce has even parity. Half of our code’s words have even parity and cd and ce are random codewords, so the probability that either of them has even parity is 34. Thus, the probability that y⋅s=0 is 34, proving the fact. ∎
In the next section, we show that the classical strategy just described can be improved.
The classical strategy described in Fact 2 above generates vectors that are orthogonal to s with probability 34. The key to classically defeating the protocol is that it is possible to correlate the vectors generated by that strategy, such that there is a non-negligible probability of generating a large set of vectors that all are orthogonal to s. These vectors form a system of linear equations that can be solved to yield s. Finally, with knowledge of s it is trivial to generate samples that pass the verifier’s test.
We follow a modified version of the classical strategy of Fact 2 to generate each vector in the correlated set. Crucially, instead of choosing random bitstrings for both d and e each time, we generate a single random bitstring d and hold it constant, only choosing new random values for e with each iteration. If the encoding cd of d under Ps has even parity, all of the generated vectors mi will have mi⋅s=0 (see Theorem 1 below). This occurs with probability 12 over our choice of d.
In practice, it is more convenient to do the linear solve if all mi⋅s=1 instead of 0. This can be easily accomplished by adding to each mi a vector m∗ with m∗⋅s=1. It turns out that m∗=∑p∈rows(P)p has this property; see proof of Theorem 1.
The explicit algorithm for extracting the vector s is given in Algorithm 4.1.
In this section we present a theorem and an empirical claim which demonstrate together that Algorithm 4.1 can be used to efficiently extract the key from any X-program constructed according to the protocol described in Section 4.2. The theorem shows that with probability 1/2 a single iteration of the algorithm finds the vector s. The empirical claim is that Algorithm 4.1 is efficient.
If s is contained in the set {si} generated in step 4 of the algorithm, the correct vector s will be output via the check in step 5 because there is a unique submatrix Ps with codewords having wt(c)∈{−1,0}(mod4). s will be contained in {si} as long as M satisfies the equation Ms=1. Thus the proof reduces to showing that Ms=1 with probability 12.
Each row of M is
| mi=m∗+¯mi | (4.9) |
for a vector ¯mi defined as
| ¯mi=∑p∈rows(P)p⋅d=p⋅e=1p | (4.10) |
Here we will show that m∗⋅s=1 always and ¯mi⋅s=0 for all i with probability 12, implying that Ms=1 with probability 12.
First we show that m∗⋅s=1. m∗ is the sum of all rows of P, so we have
| m∗⋅s=∑p∈rows(P)p⋅s=∑p∈rows(Ps)1 | (4.11) |
We see that the inner product is equal to the number of rows in the submatrix Ps (mod 2). This submatrix is a generator matrix for a code of block size 7(mod8); thus the number of rows is odd and
| m∗⋅s=1 | (4.12) |
Now we turn to showing that ¯mi⋅s=0 for all i with probability 12. In the proof of Fact 2, it was shown that for any two vectors d and e, vectors ¯mi generated by summing rows pi of P for which d⋅pi=e⋅pi=1 have
| ¯mi⋅s=0 iff cd or ce has even parity | (4.13) |
where cd and ce are the encodings under Ps of d and e respectively. If d is held constant for all i, and d happened to be chosen such that cd=Ps d has even parity, then ¯mi⋅s=0 for all i by Equation 4.13. Because half of the codewords have even parity, for d selected uniformly at random we have ¯mi⋅s=0 for all i with probability 12.
We have shown that m∗⋅s=1 always and ¯mi⋅s=0 for all i with probability 12. Therefore we have
| Prd[mi⋅s=1 ∀ i]=12 |
Thus Ms=1 with probability 12. The algorithm will output s whenever Ms=1, proving the theorem. ∎
Before we move on, we remark that while Theorem 1 treats X-programs containing a single unique submatrix with the relevant properties, the algorithm can easily be modified to return the vectors s corresponding to all such submatrices, if more exist, by simply accumulating all vectors s for which the check in Step 5(b) succeeds. We do note, however, that for the protocol described in Section 4.2, the probability of “extraneous” submatrices other than the one intentionally built into the matrix arising by chance is vanishingly small—corresponding to the probability that a random binary linear code happens to be doubly even and self-dual, which is bounded from above by 1/4n.
Now, having established that each iteration of the algorithm outputs s with probability 1/2, we now turn to analyzing its runtime.
(empirical) Algorithm 4.1 halts in O(n3) time on average.
All steps of the algorithm except for step 5 have O(n3) scaling by inspection. The obstacle preventing Claim 1 from trivially holding is that it is hard to make a rigorous statement about how large the set of candidate vectors {si} is. Because |{si}|=2n−rank(M), we’d like to show that on average, the rank of M is close to or equal to n. It seems reasonable that this would be the case: we are generating the rows of M by summing rows from P, and P must have full rank because it contains a rank-n error correcting code. But the rows of P summed into each mi are not selected independently—they are always related via their connection to the vectors d and e, and it’s not clear how these correlations affect the linear independence of the resulting mi.
Despite the lack of a proof, empirical evidence supports Claim 1 when the algorithm is applied to X-programs generated in the manner described in Section 4.2. Figure 4.2(a) shows the average number of candidate keys checked by the algorithm before s is found, as a function of problem size. The value is constant, demonstrating that the average size of the set {si} does not scale with n. Furthermore, the value is small—only about 4. This implies that M usually has high rank. In Figure 4.2(b) we plot explicitly the distribution of the rank of the matrix M over 1000 runs of the algorithm on unique X-programs of size n=245. The blue bars (on the left of each pair) show the distribution over all X-programs tested, and the sharply decaying tail supports the claim that low-rank M almost never occur.
A natural next question is whether there is some feature of the X-programs in that tail that causes M to be low rank. To investigate that question, the algorithm was re-run 100 times on each of the X-programs that had n−rank(M)>4 in the blue distribution. The orange bars of Figure 4.2(b) (on the right of each pair) plot the distribution of n−rank(M) for that second run. The similarity of the blue and orange distributions suggests that the rank of M is not correlated between runs; that is, the low rank of M in the first run was not due to any feature of the input X-programs. From a practical perspective, this data suggests that if the rank of M is found to be unacceptably low, the algorithm can simply be re-run with new randomness and the rank of M is likely to be higher the second time.
An implementation of Algorithm 4.1 in the programming language Julia (along with the code to generate the figures in this manuscript) is available online [KAH23]. Figure 4.1 shows the runtime of this implementation for various problem sizes. Experiments were completed using one thread on an Intel 8268 “Cascade Lake” processor.
Note that Figure 4.1 shows O(n2) scaling, rather than O(n3) from Claim 1. This is due to data-level parallelism in the implementation. Zn2 vectors are stored as the bits of 64-bit integers, so operations like vector addition can be performed on 64 elements at once via bitwise operations. Furthermore, with AVX SIMD CPU instructions, those operations can be applied to multiple 64-bit integers in one CPU cycle. Thus, for n of order 100, the ostensibly O(n) vector inner products and vector sums are performed in constant time, removing one factor of n from the runtime. The tests in Figure 4.1 were performed on a CPU with 512 bit vector units.
A natural question is whether it is possible to modify the original protocol such that this attack is not successful. Perhaps P can be engineered such that either 1) it is not possible to generate a large number of vectors that all have a known inner product with s, or 2) the rank of the matrix M formed by these generated vectors will never be sufficiently high to allow solution of the linear system.
For 1), our ability to generate many vectors orthogonal to s relies on the fact that the code generated by the hidden submatrix Ps has codewords c with wt(c)∈{−1,0}(mod4), as shown in the proof of Theorem 1. Unfortunately, this property regarding the weights of the codewords is precisely what gives the quantum sampling algorithm its bias toward generating vectors with x⋅s=0 (see Fact 1). This fact seems to preclude the possibility of simply removing the special property of the submatrix Ps to prevent the attack.
For 2), the main obstacle is that the matrix P must have rank n because embedded in it is a code of rank n. The only hope is to somehow engineer the matrix such that linear combinations generated in the specific way described above will not themselves be linearly independent. It is not at all clear how one would do that, and furthermore, adding structure to the previously-random extra rows of P runs the risk of providing even more information about the secret vector s. Perhaps one could prove that the rank of M will be large even for worst-case inputs P—this could be an interesting future direction.
The attack described in this paper reiterates the value of building protocols for which passing the test itself, rather than just simulating the quantum device, can be shown to be hard under well-established cryptographic assumptions. In the past few years, a number of new trapdoor claw-free function based constructions have been proposed for demonstrating quantum computational advantage [BCM+21, BKV+20, KCV+22, AMM+22], as well as some based on other types of cryptography [YZ22, KLV+22]. Unfortunately, such rigorous results come with a downside, which is an increase in the size and complexity of circuits that must be run on the quantum device. Exploring simplified protocols that are provably secure is an exciting area for further research.
When the protocol was first proposed in [SB09], it was accompanied by an internet challenge. The authors posted a specific instance of the matrix P, and offered $25 to anyone who could send them samples passing the verifier’s check. The secret vector s corresponding to their challenge matrix P is (encoded as a base-64 string):
BilbHzjYxrOHYH4OlEJFBoXZbps4a54kH8flrRgo/g==
The key was extracted using the implementation of Algorithm 4.1 described in Section 4.3.2.
Shepherd and Bremner, the authors of the challenge, have graciously confirmed that this indeed is the correct key.
Here we have described a classical algorithm that passes the interactive quantum test described in [SB09]. We have proven that a single iteration of the algorithm will return the underlying secret vector with probability 12, and empirically shown that it is efficient. The immediate implication of this result is that the protocol in its original form is no longer effective as a test of quantum computational power. While it may be possible to reengineer that protocol to thwart this attack, this paper reiterates the value of proving the security of the verification step. Furthermore, while protocols for quantum advantage with provable classical hardness are valuable in their own right, they can also be used as building blocks for achieving new, more complex cryptographic tasks, like certifiable random number generation, secure remote state preparation, and even the verification of arbitrary quantum computations [BCM+21, GV19, MAH18]. As quantum hardware continues to improve and to surpass the abilities of classical machines, quantum cryptographic tools will play an important role in making quantum computation available as a service. Establishing the security of these protocols is an important first step.