Rabin cryptosystem

The keys for the Rabin cryptosystem are generated as follows:

1. Choose two large distinct prime numbers ${\displaystyle p}$ and ${\displaystyle q}$ such that ${\displaystyle p\equiv 3{\bmod {4}}}$ and ${\displaystyle q\equiv 3{\bmod {4}}}$.
2. Compute ${\displaystyle n=pq}$.

Then ${\displaystyle n}$ is the public key and the pair ${\displaystyle (p,q)}$ is the private key.

A message ${\displaystyle M}$ can be encrypted by first converting it to a number ${\displaystyle m<n}$ using a reversible mapping, then computing ${\displaystyle c=m^{2}{\bmod {n}}}$. The ciphertext is ${\displaystyle c}$.

The message ${\displaystyle m}$ can be recovered from the ciphertext ${\displaystyle c}$ by taking its square root modulo ${\displaystyle n}$ as follows.

1. Compute the square root of ${\displaystyle c}$ modulo ${\displaystyle p}$ and ${\displaystyle q}$ using these formulas:

   ${\displaystyle {\begin{aligned}m_{p}&=c^{{\frac {1}{4}}(p+1)}{\bmod {p}}\\m_{q}&=c^{{\frac {1}{4}}(q+1)}{\bmod {q}}\end{aligned}}}$

2. Use the extended Euclidean algorithm to find ${\displaystyle y_{p}}$ and ${\displaystyle y_{q}}$ such that ${\displaystyle y_{p}\cdot p+y_{q}\cdot q=1}$.
3. Use the Chinese remainder theorem to find the four square roots of ${\displaystyle c}$ modulo ${\displaystyle n}$:

   ${\displaystyle {\begin{aligned}r_{1}&=\left(y_{p}\cdot p\cdot m_{q}+y_{q}\cdot q\cdot m_{p}\right){\bmod {n}}\\r_{2}&=n-r_{1}\\r_{3}&=\left(y_{p}\cdot p\cdot m_{q}-y_{q}\cdot q\cdot m_{p}\right){\bmod {n}}\\r_{4}&=n-r_{3}\end{aligned}}}$

One of these four values is the original plaintext ${\displaystyle m}$, although which of the four is the correct one cannot be determined without additional information.

We can show that the formulas in step 1 above actually produce the square roots of ${\displaystyle c}$ as follows. For the first formula, we want to prove that ${\displaystyle m_{p}^{2}\equiv c{\bmod {p}}}$. Since ${\displaystyle p\equiv 3{\bmod {4}},}$ the exponent ${\textstyle {\frac {1}{4}}(p+1)}$ is an integer. The proof is trivial if ${\displaystyle c\equiv 0{\bmod {p}}}$, so we may assume that ${\displaystyle p}$ does not divide ${\displaystyle c}$. Note that ${\displaystyle c\equiv m^{2}{\bmod {pq}}}$ implies that ${\displaystyle c\equiv m^{2}{\bmod {p}}}$, so c is a quadratic residue modulo ${\displaystyle p}$. Then

${\displaystyle m_{p}^{2}\equiv c^{{\frac {1}{2}}(p+1)}\equiv c\cdot c^{{\frac {1}{2}}(p-1)}\equiv c\cdot 1\mod p}$

The last step is justified by Euler's criterion.

As an example, take ${\displaystyle p=7}$ and ${\displaystyle q=11}$, then ${\displaystyle n=77}$. Take ${\displaystyle m=20}$ as our plaintext. The ciphertext is thus ${\displaystyle c=m^{2}{\bmod {n}}=400{\bmod {77}}=15}$.

Decryption proceeds as follows:

1. Compute ${\displaystyle m_{p}=c^{{\frac {1}{4}}(p+1)}{\bmod {p}}=15^{2}{\bmod {7}}=1}$ and ${\displaystyle m_{q}=c^{{\frac {1}{4}}(q+1)}{\bmod {q}}=15^{3}{\bmod {11}}=9}$.
2. Use the extended Euclidean algorithm to compute ${\displaystyle y_{p}=-3}$ and ${\displaystyle y_{q}=2}$. We can confirm that ${\displaystyle y_{p}\cdot p+y_{q}\cdot q=(-3\cdot 7)+(2\cdot 11)=1}$.
3. Compute the four plaintext candidates:

   ${\displaystyle {\begin{aligned}r_{1}&=(-3\cdot 7\cdot 9+2\cdot 11\cdot 1){\bmod {77}}=64\\r_{2}&=77-64=13\\r_{3}&=(-3\cdot 7\cdot 9-2\cdot 11\cdot 1){\bmod {77}}=\mathbf {20} \\r_{4}&=77-20=57\end{aligned}}}$

and we see that ${\displaystyle r_{3}}$ is the desired plaintext. Note that all four candidates are square roots of 15 mod 77. That is, for each candidate, ${\displaystyle r_{i}^{2}{\bmod {77}}=15}$, so each ${\displaystyle r_{i}}$ encrypts to the same value, 15.

A message ${\displaystyle m<n}$ can be signed with a private key ${\displaystyle (p,q)}$ as follows.

1. Generate a random value ${\displaystyle u}$.
2. Use a cryptographic hash function ${\displaystyle H}$ to compute ${\displaystyle c=H(m|u)}$, where the bar denotes concatenation. ${\displaystyle c}$ should be an integer less than ${\displaystyle n}$.
3. Treat ${\displaystyle c}$ as a Rabin-encrypted value and attempt to decrypt it, using the private key ${\displaystyle (p,q)}$. This will produce the usual four results, ${\displaystyle r_{1},r_{2},r_{3},r_{4}}$.
4. One might expect that encrypting each ${\displaystyle r_{i}}$ would produce ${\displaystyle c}$. However, this will be true only if ${\displaystyle c}$ happens to be a quadratic residue modulo ${\displaystyle p}$ and ${\displaystyle q}$. To determine if this is the case, encrypt the first decryption result ${\displaystyle r_{1}}$. If it does not encrypt to ${\displaystyle c}$, repeat this algorithm with a new random ${\displaystyle u}$. The expected number of times this algorithm needs to be repeated before finding a suitable ${\displaystyle c}$ is 4.
5. Having found an ${\displaystyle r_{1}}$ which encrypts to ${\displaystyle c}$, the signature is ${\displaystyle (r_{1},u)}$.

A signature ${\displaystyle (r,u)}$ for a message ${\displaystyle m}$ can be verified using the public key ${\displaystyle n}$ as follows.

1. Compute ${\displaystyle c=H(m|u)}$.
2. Encrypt ${\displaystyle r}$ using the public key ${\displaystyle n}$.
3. The signature is valid if and only if the encryption of ${\displaystyle r}$ equals ${\displaystyle c}$.

Decrypting produces three false results in addition to the correct one, so that the correct result must be guessed. This is the major disadvantage of the Rabin cryptosystem and one of the factors which have prevented it from finding widespread practical use.

If the plaintext is intended to represent a text message, guessing is not difficult; however, if the plaintext is intended to represent a numerical value, this issue becomes a problem that must be resolved by some kind of disambiguation scheme. It is possible to choose plaintexts with special structures, or to add padding, to eliminate this problem. A way of removing the ambiguity of inversion was suggested by Blum and Williams: the two primes used are restricted to primes congruent to 3 modulo 4 and the domain of the squaring is restricted to the set of quadratic residues. These restrictions make the squaring function into a trapdoor permutation, eliminating the ambiguity.[3]

For encryption, a square modulo n must be calculated. This is more efficient than RSA, which requires the calculation of at least a cube.

For decryption, the Chinese remainder theorem is applied, along with two modular exponentiations. Here the efficiency is comparable to RSA.

Disambiguation introduces additional computational costs, and is what has prevented the Rabin cryptosystem from finding widespread practical use.

It has been proven that any algorithm which decrypts a Rabin-encrypted value can be used to factor the modulus ${\displaystyle n}$. Thus, Rabin decryption is at least as hard as the integer factorization problem, something that has not been proven for RSA. It is generally believed that there is no polynomial-time algorithm for factoring, which implies that there is no efficient algorithm for decrypting a Rabin-encrypted value without the private key ${\displaystyle (p,q)}$.

The Rabin cryptosystem does not provide indistinguishability against chosen plaintext attacks since the process of encryption is deterministic. An adversary, given a ciphertext and a candidate message, can easily determine whether or not the ciphertext encodes the candidate message (by simply checking whether encrypting the candidate message yields the given ciphertext).

The Rabin cryptosystem is insecure against a chosen ciphertext attack (even when challenge messages are chosen uniformly at random from the message space).[1]^:150 By adding redundancies, for example, the repetition of the last 64 bits, the system can be made to produce a single root. This thwarts this specific chosen-ciphertext attack, since the decryption algorithm then only produces the root that the attacker already knows. If this technique is applied, the proof of the equivalence with the factorization problem fails, so it is uncertain as of 2004 if this variant is secure. The Handbook of Applied Cryptography by Menezes, Oorschot and Vanstone considers this equivalence probable, however, as long as the finding of the roots remains a two-part process (1. roots ${\displaystyle {\bmod {p}}}$ and ${\displaystyle {\bmod {q}}}$ and 2. application of the Chinese remainder theorem).