Fuzzy Logic, Neural Netvork, Expert Systems...

Selections from Cryptologia: History, People, and Technology
-Cipher A. Deavours et al-

The Twofish Encryption Algorithm-A 128-Bit Block Cipher
-Bruce Schneier et al-

Cryptography-Theory and Practice
-Douglas Stinson-


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson
Table of Contents

Preface

My objective in writing this book was to produce a general, comprehensive textbook that treats all the essential core areas of cryptography. Although many books and monographs on cryptography have been written in recent years, the majority of them tend to address specialized areas of cryptography. On the other hand, many of the existing general textbooks have become out-of-date due to the rapid expansion of research in cryptography in the past 15 years.

I have taught a graduate level cryptography course at the University of Nebraska-Lincoln to computer science students, but I am aware that cryptography courses are offered at both the undergraduate and graduate levels in mathematics, computer science and electrical engineering departments. Thus, I tried to design the book to be flexible enough to be useful in a wide variety of approaches to the subject.

Of course there are difficulties in trying to appeal to such a wide audience. But basically, I tried to do things in moderation. I have provided a reasonable amount of mathematical background where it is needed. I have attempted to give informal descriptions of the various cryptosystems, along with more precise pseudo-code descriptions, since I feel that the two approaches reinforce each other. As well, there are many examples to illustrate the workings of the algorithms. And in every case I try to explain the mathematical underpinnings; I believe that it is impossible to really understand how a cryptosystem works without understanding the underlying mathematical theory.

The book is organized into three parts. The first part, Chapters 1-3, covers private-key cryptography. Chapters 4–9 concern the main topics in public-key cryptography. The remaining four chapters provide introductions to four active research areas in cryptography.

The first part consists of the following material: Chapter 1 is a fairly elementary introduction to simple "classical" cryptosystems. Chapter 2 covers the main elements of Shannon's approach to cryptography, including the concept of perfect secrecy and the use of information theory in cryptography. Chapter 3 is a lengthy discussion of the Data Encryption Standard; it includes a treatment of differential cryptanalysis.

The second part contains the following material: Chapter 4 concerns the RSA Public-key Cryptosystem, together with a considerable amount of background on number-theoretic topics such as primality testing and factoring. Chapter 5 discusses some other public-key systems, the most important being the ElGamal System based on discrete logarithms. Chapter 6 deals with signature schemes, such as the Digital Signature Standard, and includes treatment of special types of signature schemes such as undeniable and fail-stop signature schemes. The subject of Chapter 7 is hash functions. Chapter 8 provides an overview of the numerous approaches to key distribution and key agreement protocols. Finally, Chapter 9 describes identification schemes.

The third part contains chapters on selected research-oriented topics, namely, authentication codes, secret sharing schemes, pseudo-random number generation, and zero-knowledge proofs.

Thus, I have attempted to be quite comprehensive in the "core" areas of cryptography, as well as to provide some more advanced chapters on specific research areas. Within any given area, however, I try to pick a few representative systems and discuss them in a reasonable amount of depth. Thus my coverage of cryptography is in no way encyclopedic.

Certainly there is much more material in this book than can be covered in one (or even two) semesters. But I hope that it should be possible to base several different types of courses on this book. An introductory course could cover Chapter 1, together with selected sections of Chapters 2–5. A second or graduate course could cover these chapters in a more complete fashion, as well as material from Chapters 6–9. Further, I think that any of the chapters would be a suitable basis for a "topics" course that might delve into specific areas more deeply.

But aside from its primary purpose as a textbook, I hope that researchers and practitioners in cryptography will find it useful in providing an introduction to specific areas with which they might not be familiar. With this in mind, I have tried to provide references to the literature for further reading on many of the topics discussed.

One of the most difficult things about writing this book was deciding how much mathematical background to include. Cryptography is a broad subject, and it requires knowledge of several areas of mathematics, including number theory, groups, rings and fields, linear algebra, probability and information theory. As well, some familiarity with computational complexity, algorithms and NP-completeness theory is useful. I have tried not to assume too much mathematical background, and thus I develop mathematical tools as they are needed, for the most part. But it would certainly be helpful for the reader to have some familiarity with basic linear algebra and modular arithmetic. On the other hand, a more specialized topic, such as the concept of entropy from information theory, is introduced from scratch.

I should also apologize to anyone who does not agree with the phrase "Theory and Practice" in the title. I admit that the book is more theory than practice. What I mean by this phrase is that I have tried to select the material to be included in the book both on the basis of theoretical interest and practical importance. So, I may include systems that are not of practical use if they are mathematically elegant or illustrate an important concept or technique. But, on the other hand, I do describe the most important systems that are used in practice, e.g., DES and other U. S. cryptographic standards.

I would like to thank the many people who provided encouragement while I wrote this book, pointed out typos and errors, and gave me useful suggestions on material to include and how various topics should be treated. In particular, I would like to convey my thanks to Mustafa Atici, Mihir Bellare, Bob Blakley, Carlo Blundo, Gilles Brassard, Daniel Ducharme, Mike Dvorsky, Luiz Frota-Mattos, David Klarner, Don Kreher, Keith Martin, Vaclav Matyas, Alfred Menezes, Luke O'Connor, William Read, Phil Rogaway, Paul Van Oorschot, Scott Vanstone, Johan van Tilburg, Marc Vauclair and Mike Wiener. Thanks also to Mike Dvorsky for helping me prepare the index.

Douglas R. Stinson

The CRC Press Series on Discrete Mathematics and Its Applications

Discrete mathematics is becoming increasingly applied to computer science, engineering, the physical sciences, the natural sciences, and the social sciences. Moreover, there has also been an explosion of research in discrete mathematics in the past two decades. Both trends have produced a need for many types of information for people who use or study this part of the mathematical sciences. The CRC Press Series on Discrete Mathematics and Its Applications is designed to meet the needs of practitioners, students, and researchers for information in discrete mathematics. The series includes handbooks and other reference books, advanced textbooks, and selected monographs. Among the areas of discrete mathematics addressed by the series are logic, set theory, number theory, combinatorics, discrete probability theory, graph theory, algebra, linear algebra, coding theory, cryptology, discrete optimization, theoretical computer science, algorithmics, and computational geometry.

Kenneth H. Rosen, Series Editor
Distinguished Member of Technical Staff
AT&T Bell Laboratories
Holmdel, New Jersey
e-mail:krosen@arch4.ho.att.com

Advisory Board

Charles Colbourn
Department of Combinatorics and Optimization, University of Waterloo
Jonathan Gross
Department of Computer Science, Columbia University
Andrew Odlyzko
AT&T Bell Laboratories

Table of Contents


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson
Table of Contents

Dedication

To my children, Michela and Aiden

Table of Contents


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson
Table of Contents

Further Reading

Other recommended textbooks and monographs on cryptography include the following:

Beker and Piper [BP82] | Beutelspacher [Be94]
Brassard [Br88] | Biham and Shamir [BS93]
Denning [De82] | Kahn [Ka67]
Kaufman, Perlman and Speciner [KPS95] | Koblitz [Ko94]
Konheim [Ko81] | Kranakis [Kr86]
Menezes [Me93] | Meyer and Matyas [MM82]
Patterson [Pa87] | Pomerance [Po90a]
Rhee [Rh94] | Rueppel [Ru86]
Salomaa [Sa90] | Schneier [Sc95]
Seberry and Pieprzyk [SP89] | Simmons [Si92b]
Stallings [St95] | van Tilborg [vT88]
Wayner [Wa96] | Welsh [We88]

For a thorough and highly recommended reference on all aspects of practical cryptogrpahy, see Menezes, Van Oorschot and Vanstone [MVV96].

The main research journals in cryptography are the Journal of Cryptology, Designs, Codes and Cryptography and Cryptologia. The Journal of Cryptology is the journal of the International Association for Cryptologic Research (or IACR) which also sponsors the two main annual cryptology conferences, CRYPTO and EUROCRYPT.

CRYPTO has been held since 1981 in Santa Barabara. The proceedings of CRYPTO have been published annually since 1982:

CRYPTO '82 [CRS83] | CRYPTO '83 [Ch84]
CRYPTO '84 [BC85] | CRYPTO '85 [Wi86]
CRYPTO '86 [Od87] | CRYPTO '87 [Po88]
CRYPTO '88 [Go90] | CRYPTO '89 [Br90]
CRYPTO '90 [MV91] | CRYPTO '91 [Fe92]
CRYPTO '92 [Br93] | CRYPTO '93 [St94]
CRYPTO '94 [De94] | CRYPTO '95 [Co95]
CRYPTO '96 [Ko96] |

EUROCRYPT has been held annually since 1982, and except for 1983 and 1986, its proceedings have been published, as follows:

EUROCRYPT '82 [Be83] | EUROCRYPT '84 [BCI85]
EUROCRYPT '85 [Pi86] | EUROCRYPT '87 [CP88]
EUROCRYPT '88 [Gu88a] | EUROCRYPT '89 [QV90]
EUROCRYPT '90 [Da91] | EUROCRYPT '91 [Da91a]
EUROCRYPT '92 [Ru93] | EUROCRYPT '93 [He94]
EUROCRYPT '94 [De95] | EUROCRYPT '95 [GQ95]
EUROCRYPT '96 [Ma96] |

A third conference series, AUSCRYPT/ASIACRYPT, has been held "in association with" the IACR. Its conference proceedings have also been published:

AUSCRYPT '90 [SP90] | ASIACRYPT '91 [IRM93]
AUSCRYPT '92 [SZ92] | ASIACRYPT '94 [PS95]

Bibliography

[ACGS88] | W. Alexi, B. Chor, O. Goldreich and C. P. Schnorr. RSA and Rabin functions: certain parts are as hard as the whole. SIAM Jounal on Computing, 17 (1988), 194-209.
[An91] | H. Anton. Elementary Linear Algebra (Sixth Edition). John Wiley and Sons, 1991.
[BHS93] | D. Bayer, S. Haber and W. S. Stornetta. Improving the efficiency and reliability of digital time-stamping. In Sequences II, Methods in Communication, Security, and Computer Science, pages 329-334. Springer-Verlag, 1993.
[BB88] | P. Beauchemin and G. Brassard. A generalization of Hellman's extension to Shannon's approach to cryptography. Journal of Cryptology, 1 (1988), 129-131.
[BBCGP88] | P. Beauchemin, G. Brassard, C. Crépeau, C. Goutier and C. Pomerance. The generation of random numbers that are probably prime. Journal of Cryptology, 1 (1988), 53-64.
[BC94] | A. Beimel and B. Chor. Interaction in key distribution schemes. Lecture Notes in Computer Science, 773 (1994), 444-455. (Advances in Cryptology - CRYPTO '93.)
[BP82] | H. Beker and F. Piper. Cipher Systems, The Protection of Communications. John Wiley and Sons, 1982.
[BL90] | J. Benaloh and J. Leichter. Generalized secret sharing and monotone functions. Lecture Notes in Computer Science, 403 (1990), 27-35. (Advances in Cryptology - CRYPTO '88.)
[Be83] | T. Beth (Ed.) Cryptography Proceedings, 1982. Lecture Notes in Computer Science, vol. 149, Springer-Verlag, 1983.
[BCI85] | T. Beth, N. Cot and I. Ingemarsson (Eds.) Advances in Cryptology: Proceedings of EUROCRYPT '84. Lecture Notes in Computer Science, vol. 209, Springer-Verlag, 1985.
[BJL85] | T. Beth, D. Jungnickel, and H. Lenz. Design Theory. Bibliographisches Institut, Zurich, 1985.
[Be94] | A. Beutelspacher. Cryptology. Mathematical Association of America, 1994.
[BS91] | E. Biham and A. Shamir. Differential cryptanalysis of DES-like cryptosystems. Journal of Cryptology, 4 (1991), 3-72.
[BS93] | E. Biham and A. Shamir. Differential Cryptanalysis of the Data Encryption Standard. Springer-Verlag, 1993.
[BS93a] | E. Biham and A. Shamir. Differential cryptanalysis of the full 16-round DES. Lecture Notes in Computer Science, 740 (1993), 494-502. (Advances in Cryptology - CRYPTO '92.)
[Bl79] | G. R. Blakley. Safeguarding cryptographic keys. AFIPS Conference Proceedings, 48 (1979), 313-317.
[BC85] | G. R. Blakley and D. Chaum (Eds.) Advances in Cryptology: Proceedings of CRYPTO '84. Lecture Notes in Computer Science, vol. 196, Springer-Verlag, 1985.
[Bl85] | R. Blom An optimal class of symmetric key generation schemes. Lecture Notes in Computer Science, 209 (1985), 335-338. (Advances in Cryptology - EUROCRYPT '84.)
[BBS86] | L. Blum, M. Blum and M. Shub. A simple unpredictable random number generator. SIAM Jounal on Computing, 15 (1986), 364-383.
[Bl82] | M. Blum. Coin flipping by telephone: a protocol for solving impossible problems In 24th IEEE Spring Computer Conference, pages 133-137. IEEE Press, 1982.
[BG85] | M. Blum and S. Goldwasser. An efficient probabilistic public-key cryptosystem that hides all partial information. Lecture Notes in Computer Science, 196 (1985), 289-302. (Advances in Cryptology - CRYPTO '84.)
[BM84] | M. Blum and S. Micali. How to generate cryptographically strong sequences of pseudo-random bits. SIAM Journal on Computing, 13 (1984), 850-864.
[Bo89] | J. Boyar. Inferring sequences produced by pseudo-random number generators. Journal of Association for Computing Machinery, 36 (1989), 129-141.
[BDSV93] | C. Blundo, A. De Santis, D. R. Stinson, and U. Vaccaro. Graph decompositions and secret sharing schemes. Lecture Notes in Computer Science, 658 (1993), 1-24. (Advances in Cryptology - EUROCRYPT '92.)
[BDSHKVY93] | C. Blundo, A. De Santis,A. Herzberg, S. Kutten, U. Vaccaro and M. Yung. Perfectly-secure key distribution for dynamic conferences. Lecture Notes in Computer Science, 740 (1993), 471-486. (Advances in Cryptology - CRYPTO '92.)
[BC93] | J. N. E. Bos and D. Chaum. Provably unforgeable signatures. Lecture Notes in Computer Science, 740 (1993), 1-14. (Advances in Cryptology - CRYPTO '92.)
[Br88] | G. Brassard. Modern Cryptology - A Tutorial. Lecture Notes in Computer Science, vol. 325, Springer-Verlag, 1988.
[Br90] | G. Brassard (Ed.) Advances in Cryptology - CRYPTO '89 Proceedings. Lecture Notes in Computer Science, vol. 435, Springer-Verlag, 1990.
[BB88a] | G. Brassard and P. Bratley. Algorithmics, Theory and Practice. Prentice Hall, 1988.
[BCC88] | G. Brassard, D. Chaum and C. Crépeau. Minimum disclosure proofs of knowledge. Journal of Computer and Systems Science, 37 (1988), 156-189.
[BC90] | G. Brassard and C. Crépeau. Sorting out zero-knowledge. Lecture Notes in Computer Science, 434 (1990), 181-191. (Advances in Cryptology - EUROCRYPT '89.)
[Br89] | D. M. Bressoud. Factorization and Primality Testing. Springer-Verlag, 1989.
[Br85] | E. F. Brickell. Breaking iterated knapsacks. Lecture Notes in Computer Science, 218 (1986), 342-358. (Advances in Cryptology - CRYPTO '85.)
[Br89a] | E. F. Brickell. Some ideal secret sharing schemes. Journal of Combinatorial Mathematics and Combinatorial Computing, 9 (1989), 105-113.
[Br93] | E. F. Brickell (Ed.) Advances in Cryptology - CRYPTO '92 Proceedings. Lecture Notes in Computer Science, vol. 740, Springer-Verlag, 1993.
[BD91] | E. F. Brickell and D. M. Davenport. On the classification of ideal secret sharing schemes. Journal of Cryptology, 4 (1991), 123-134.
[BM92] | E. F. Brickell and K. S. McCurley. An interactive identification scheme based on discrete logarithms and factoring. Journal of Cryptology, 5 (1992), 29-39.
[BMP87] | E. F. Brickell, J. H. Moore and M. R. Purtill. Structure in the S-boxes of DES. Lecture Notes in Computer Science, 263 (1987), 3-8. (Advances in Cryptology - CRYPTO '86.)
[BO92] | E. F. Brickell and A. M. Odlyzko. Cryptanalysis, a survey of recent results. In Contemporary Cryptology, The Science of Information Integrity, pages 501-540. IEEE Press, 1992.
[BS92] | E. F. Brickell and D. R. Stinson. Some improved bounds on the information rate of perfect secret sharing schemes. Journal of Cryptology, 5 (1992), 153-166.
[BKPS90] | L. Brown, M. Kwan, J. Pieprzyk and J. Seberry. LOKI - A cryptographic primitive for authentication and secrecy applications. Lecture Notes in Computer Science, 453 (1990), 229-236. (Advances in Cryptology - AUSCRYPT '90.)
[BDB92] | M. Burmester, Y. Desmedt and T. Beth. Efficient zero-knowledge identification schemes for smart cards. The Computer Journal, 35 (1992), 21-29.
[CDGV93] | R. M. Capocelli, A. De Santis, L. Gargano, and U. Vaccaro. On the size of shares for secret sharing schemes. Journal of Cryptology, 6 (1993), 157-167.
[Ch95] | F. Chabaud. On the security of some cryptosystems based on error-correcting codes. Lecture Notes in Computer Science, to appear. (Advances in Cryptology - EUROCRYPT '94.)
[Ch84] | D. Chaum (Ed.) Advances in Cryptology: Proceedings of CRYPTO '83. Plenum Press, 1984.
[CP88] | D. Chaum and W. L. Price (Eds.) Advances in Cryptology - EUROCRYPT '87 Proceedings. Lecture Notes in Computer Science, vol. 304, Springer-Verlag, 1988.
[CRS83] | D. Chaum, R. L. Rivest and A. T. Sherman (Eds.) Advances in Cryptology: Proceedings of CRYPTO '82. Plenum Press, 1983.
[CvA90] | D. Chaum and H. van Antwerpen. Undeniable signatures. Lecture Notes in Computer Science, 435 (1990), 212-216. (Advances in Cryptology - CRYPTO '89.)
[CvHP92] | D. Chaum, E. van Heijst and B. Pfitzmann. Cryptographically strong undeniable signatures, unconditionally secure for the signer. Lecture Notes in Computer Science, 576 (1992), 470-484. (Advances in Cryptology - CRYPTO '91.)
[CR88] | B. Chor and R. L. Rivest. A knapsack-type public key cryptosystem based on arithmetic in finite fields. IEEE Transactions on Information Theory, 45 (1988), 901-909.
[Co95] | D. Coppersmith (Ed.) Advances in Cryptology - CRYPTO '95 Proceedings. Lecture Notes in Computer Science, vol. 963, Springer-Verlag, 1995.
[CKM94] | D. Coppersmith, H. Krawczyz and Y. Mansour. The shrinking generator. Lecture Notes in Computer Science, 773 (1994), 22-39. (Advances in Cryptology - CRYPTO '93.)
[CSV94] | D. Coppersmith, J. Stern and S. Vaudenay. Attacks on the birational permutation signature schemes. Lecture Notes in Computer Science, 773 (1994), 435-443. (Advances in Cryptology - CRYPTO '93.)
[CW91] | T. W. Cusick and M. C. Wood. The REDOC-II cryptosystem. Lecture Notes in Computer Science, 537 (1991), 545-563. (Advances in Cryptology - CRYPTO '90.)
[Da90] | I. B. Damgård. A design principle for hash functions. Lecture Notes in Computer Science, 435 (1990), 416-427. (Advances in Cryptology - CRYPTO '89.)
[Da91] | I. B. Damgård (Ed.) Advances in Cryptology - EUROCRYPT '90 Proceedings. Lecture Notes in Computer Science, vol. 473, Springer-Verlag, 1991.
[DLP93] | I. Damgård, P. Landrock and C. Pomerance. Average case error estimates for the strong probable prime test. Mathematics of Computation, 61 (1993), 177-194.
[Da91a] | D. W. Davies (Ed.) Advances in Cryptology - EUROCRYPT '91 Proceedings. Lecture Notes in Computer Science, vol. 547, Springer-Verlag, 1991.
[De84] | J. M. DeLaurentis. A further weakness in the common modulus protocol for the RSA cryptosystem. Cryptologia, 8 (1984), 253-259.
[dBB92] | B. den Boer and A. Bossalaers. An attack on the last two rounds of MD4. Lecture Notes in Computer Science, 576 (1992), 194-203. (Advances in Cryptology - CRYPTO '91.)
[De82] | D. E. R. Denning. Cryptography and Data Security. Addison-Wesley, 1982.
[De95] | A. De Santis (Ed.) Advances in Cryptology - EUROCRYPT '94 Proceedings. Lecture Notes in Computer Science, vol. 950, Springer-Verlag, 1995.
[De94] | Y. G. Desmedt (Ed.) Advances in Cryptology - CRYPTO '94 Proceedings. Lecture Notes in Computer Science, vol. 839, Springer-Verlag, 1994.
[dWQ93] | D. de Waleffe and J.-J. Quisquater. Better login protocols for computer networks. Lecture Notes in Computer Science, 741 (1993), 50-70. (Computer Security and Industrial Cryptography, State of the Art and Evolution, ESAT Course, May 1991.)
[Di92] | W. Diffie. The first ten years of public-key cryptography. In Contemporary Cryptology, The Science of Information Integrity, pages 135-175. IEEE Press, 1992.
[DH76] | W. Diffie and M. E. Hellman. Multiuser cryptographic techniques. AFIPS Conference Proceedings, 45 (1976), 109-112.
[DH76a] | W. Diffie and M. E. Hellman. New directions in cryptography. IEEE Transactions on Information Theory, 22 (1976), 644-654.
[DVW92] | W. Diffie, P. C. Van Oorschot and M. J. Wiener. Authentication and authenticated key exchanges. Designs, Codes and Cryptography, 2 (1992), 107-125.
[Eb93] | H. Eberle. A high-speed DES implementation for network applications. Lecture Notes in Computer Science, 740 (1993), 527-545. (Advances in Cryptology - CRYPTO '92.)
[El85] | T. ElGamal. A public key cryptosystem and a signature scheme based on discrete logarithms. IEEE Transactions on Information Theory, 31 (1985), 469-472.
[EAKMM86] | D. Estes, L. M. Adleman, K. Kompella, K. S. McCurley and G. L. Miller. Breaking the Ong-Schnorr-Shamir signature schemes for quadratic number fields. Lecture Notes in Computer Science, 218 (1986), 3-13. (Advances in Cryptology - CRYPTO '85.)
[FFS88] | U. Feige, A. Fiat and A. Shamir. Zero-knowledge proofs of identity. Journal of Cryptology, 1 (1988), 77-94.
[Fe92] | J. Feigenbaum (Ed.) Advances in Cryptology - CRYPTO '91 Proceedings. Lecture Notes in Computer Science, vol. 576, Springer-Verlag, 1992.
[Fe73] | H. Feistel. Cryptography and computer privacy. Scientific American, 228(5) (1973), 15-23.
[FN91] | A. Fiat and M. Naor. Rigorous time/space trade-offs for inverting functions. In Proceedings of the 23rd Symposium on the Theory of Computing, pages 534-541. ACM Press, 1991.
[FS87] | A. Fiat and A. Shamir. How to prove yourself: practical solutions to identification and signature problems. Lecture Notes in Computer Science, 263 (1987), 186-194. (Advances in Cryptology - CRYPTO '86.)
[FOM91] | A. Fujioka, T. Okamoto and S. Miyaguchi. ESIGN: an efficient digital signature implementation for smart cards. Lecture Notes in Computer Science, 547 (1991), 446-457. (Advances in Cryptology - EUROCRYPT '91.)
[Gib91] | J. K. Gibson. Discrete logarithm hash function that is collision free and one way. IEE Proceedings-E, 138 (1991), 407-410.
[GMS74] | E. N. Gilbert, F. J. MacWilliams and N. J. A. Sloane. Codes which detect deception. Bell Systems Technical Journal, 53 (1974), 405-424.
[Gir91] | M. Girault. Self-certified public keys. Lecture Notes in Computer Science, 547 (1991), 490-497. (Advances in Cryptology - EUROCRYPT '91.)
[GP91] | C. M. Goldie and R. G. E. Pinch. Communication Theory. Cambridge University Press, 1991.
[GMW91] | O. Goldreich, S. Micali and A. Wigderson. Proofs that yield nothing but their validity or all languages in NP have zero-knowledge proof systems. Journal of the ACM, 38 (1991), 691-729.
[Go90] | S. Goldwasser (Ed.) Advances in Cryptology - CRYPTO '88 Proceedings. Lecture Notes in Computer Science, vol. 403, Springer-Verlag, 1990.
[GM84] | S. Goldwasser and S. Micali. Probabilistic encryption. Journal of Computer and Systems Science, 28 (1984), 270-299.
[GMR89] | S. Goldwasser, S. Micali and C. Rackoff. The knowledge complexity of interactive proof systems. SIAM Journal on Computing, 18 (1989), 186-208.
[GMT82] | S. Goldwasser, S. Micali and P. Tong. Why and how to establish a common code on a public network. In 23rd Annual Symposium on the Foundations of Computer Science, pages 134-144. IEEE Press, 1982.
[GM93] | D. M. Gordon and K. S. McCurley. Massively parallel computation of discrete logarithms. Lecture Notes in Computer Science, 740 (1993), 312-323. (Advances in Cryptology - CRYPTO '92.)
[GQ88] | L. C. Guillou and J.-J. Quisquater. A practical zero-knowledge protocol fitted to security microprocessor minimizing both transmission and memory. Lecture Notes in Computer Science, 330 (1988), 123-128. (Advances in Cryptology - EUROCRYPT '88.)
[GQ95] | L. C. Guillou and J.-J. Quisquater (Eds.) Advances in Cryptology - EUROCRYPT '95 Proceedings. Lecture Notes in Computer Science, vol. 921, Springer-Verlag, 1995.
[Gu88] | C. G. Gunther Alternating step generators controlled by de Bruijn sequences. Lecture Notes in Computer Science, 304 (1988), 88-92. (Advances in Cryptology - EUROCRYPT '87.)
[Gu88a] | C. G. Gunther (Ed.) Advances in Cryptology - EUROCRYPT '88 Proceedings. Lecture Notes in Computer Science, vol. 330, Springer-Verlag, 1988.
[HS91] | S. Haber and W. S. Stornetta. How to timestamp a digital document. Journal of Cryptology, 3 (1991), 99-111.
[HSS93] | J. Håstad, A. W. Schrift and A. Shamir. The discrete logarithm modulo a composite hides O(n) bits. Journal of Computer and Systems Science, 47 (1993), 376-404.
[He80] | M. E. Hellman. A cryptanalytic time-memory trade-off. IEEE Transactions on Information Theory, 26 (1980), 401-406.
[Hi29] | L. S. Hill. Cryptogaphy in an algebraic alphabet. American Mathematical Monthly, 36 (1929), 306-312.
[He94] | T. Helleseth (Ed.) Advances in Cryptology - EUROCRYPT '93 Proceedings. Lecture Notes in Computer Science, vol. 765, Springer-Verlag, 1994.
[HLLPRW91] | D. G. Hoffman, D. A. Leonard, C. C. Lindner, K. T. Phelps, C. A. Rodger and J. R. Wall. Coding Theory, The Essentials. Marcel Dekker, 1991.
[IRM93] | H. Imai, R. L. Rivest and T. Matsumoto (Eds.) Advances in Cryptology - ASIACRYPT '91 Proceedings. Lecture Notes in Computer Science, vol. 739, Springer-Verlag, 1993.
[ISN87] | M. Ito, A. Saito, and T. Nishizeki. Secret sharing scheme realizing general access structure. Proceedings IEEE Globecom '87, pages 99-102, 1987.
[Jo88] | D. S. Johnson. The NP-completeness column: an ongoing guide. Journal of Algorithms, 9 (1988), 426-444.
[Ka67] | D. Kahn. The Codebreakers. The Story of Secret Writing. Macmillan, 1967.
[KPS95] | C. Kaufman, R. Perlman and M. Speciner. Network Security. Private Communication in a Public World. Prentice Hall, 1995.
[Ko87] | N. Koblitz. Elliptic curve cryptosystems. Mathematics of Computation, 48 (1987), 203-209.
[Ko94] | N. Koblitz. A Course in Number Theory and Cryptography (Second Edition). Springer-Verlag, 1994.
[Ko96] | N. Koblitz (Ed.) Advances in Cryptology - CRYPTO '96 Proceedings. Lecture Notes in Computer Science, vol. 1109, Springer-Verlag, 1996.
[KN93] | J. Kohl and C. Neuman. The Kerboros Network Authentication Service. Network Working Group Request for Comments: 1510, September 1993.
[Ko81] | A. G. Konheim. Cryptography, A Primer. John Wiley and Sons, 1981.
[Kr86] | E. Kranakis. Primality and Cryptography. John Wiley and Sons, 1986.
[La90] | J. C. Lagarias Pseudo-random number generators in cryptography and number theory. In Cryptology and Computational Number Theory, pages 115-143. American Mathematical Society, 1990.
[LO91] | B. A. Lamacchia and A. M. Odlyzko. Computation of discrete logarithms in prime fields. Designs, Codes and Cryptography, 1 (1991), 47-62.
[LL93] | A. K. Lenstra and H. W. Lenstra, Jr. (Eds.) The Development of the Number Field Sieve. Lecture Notes in Mathematics, vol. 1554. Springer-Verlag, 1993.
[LL90] | A. K. Lenstra and H. W. Lenstra, Jr. Algorithms in number theory. In Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity, pages 673-715. Elsevier Science Publishers, 1990.
[LN83] | R. Lidl and H. Niederreiter. Finite Fields. Addison-Wesley, 1983.
[LW88] | D. L. Long and A. Wigderson. The discrete log hides O(log n) bits. SIAM Jounal on Computing, 17 (1988), 363-372.
[MS77] | F. J. MacWilliams and N. J. A. Sloane. The Theory of Error-Correcting Codes. North-Holland, 1977.
[Ma86] | J. L. Massey. Cryptography - a selective survey. In Digital Communications, pages 3-21. North-Holland, 1986.
[Ma94] | M. Matsui. Linear cryptanalysis method for DES cipher. Lecture Notes in Computer Science, 765 (1994), 386-397. (Advances in Cryptology - EUROCRYPT '93.)
[Ma94a] | M. Matsui. The first experimental cryptanalysis of the data encryption standard. Lecture Notes in Computer Science, 839 (1994), 1-11. (Advances in Cryptology - CRYPTO '94.)
[MTI86] | T. Matsumoto, Y. Takashima and H. Imai. On seeking smart public-key distribution systems. Transactions of the IECE (Japan), 69 (1986), 99-106.
[Ma96] | U. Maurer (Ed.) Advances in Cryptology - EUROCRYPT '96 Proceedings. Lecture Notes in Computer Science, vol. 1070, Springer-Verlag, 1996.
[Mc90] | K. McCurley The discrete logarithm problem. In Cryptology and Computational Number Theory, pages 49-74. American Mathematical Society, 1990.
[Mc78] | R. McEliece. A public-key cryptosystem based on algebraic coding theory. DSN Progress Report, 42-44 (1978), 114-116.
[Mc87] | R. McEliece. Finite Fields for Computer Scientists and Engineers. Kluwer Academic Publishers, 1987.
[Me93] | A. J. Menezes. Elliptic Curve Public Key Cryptosystems. Kluwer Academic Publishers, 1993.
[MBGMVY93] | A. J. Menezes, I. F. Blake, X. Gao, R. C. Mullin, S. A. Vanstone and T. Yaghoobian. Applications of Finite Fields. Kluwer Academic Publishers, 1993.
[MOV94] | A. J. Menezes, T. Okamoto and S. A. Vanstone. Reducing elliptic curve logarithms to logarithms in a finite field. IEEE Transactions on Information Theory, 39 (1993), 1639-1646.
[MV91] | A. J. Menezes and S. A. Vanstone (Eds.) Advances in Cryptology - CRYPTO '90 Proceedings. Lecture Notes in Computer Science, vol. 537, Springer-Verlag, 1991.
[MV93] | A. J. Menezes and S. A. Vanstone. Elliptic curve cryptosystems and their implementation. Journal of Cryptology, 6 (1993), 209-224.
[MVV96] | A. J. Menezes, P. C. Van Oorschot and S. A. Vanstone. Handbook of Applied Cryptography. CRC Press, 1996.
[Me78] | R. C. Merkle. Secure communications over insecure channels. Communications of the ACM, 21 (1978), 294-299.
[Me90] | R. C. Merkle. One way hash functions and DES. Lecture Notes in Computer Science, 435 (1990), 428-446. (Advances in Cryptology - CRYPTO '89.)
[Me90a] | R. C. Merkle. A fast software one-way hash function. Journal of Cryptology, 3 (1990), 43-58.
[MH78] | R. C. Merkle and M. E. Hellman. Hiding information and signatures in trapdoor knapsacks. IEEE Transactions on Information Theory, 24 (1978), 525-530.
[MM82] | C. Meyer and S. Matyas. Cryptography: A New Dimension in Computer Security. John Wiley and Sons, 1982.
[Mi76] | G. L. Miller. Riemann's hypothesis and tests for primality. Journal of Computer and Systems Science, 13 (1976), 300-317.
[Mi86] | V. Miller. Uses of elliptic curves in cryptography. Lecture Notes in Computer Science, 218 (1986), 417-426. (Advances in Cryptology - CRYPTO '85.)
[MPW92] | C. J. Mitchell, F. Piper and P. Wild. Digital signatures. In Contemporary Cryptology, The Science of Information Integrity, pages 325-378. IEEE Press, 1992.
[Mi91] | S. Miyaguchi. The FEAL cipher family. Lecture Notes in Computer Science, 537 (1991), 627-638. (Advances in Cryptology - CRYPTO '90.)
[MOI90] | S. Miyaguchi, K. Ohta and M. Iwata. 128-bit hash function (N-hash). Proceedings of SECURICOM 1990, 127-137.
[Mo92] | J. H. Moore. Protocol failures in cryptosystems. In Contemporary Cryptology, The Science of Information Integrity, pages 541-558. IEEE Press, 1992.
[NBS77] | Data Encryption Standard (DES). National Bureau of Standards FIPS Publication 46, 1977.
[NBS80] | DES modes of operation. National Bureau of Standards FIPS Publication 81, 1980.
[NBS81] | Guidelines for implementing and using the NBS data encryption standard. National Bureau of Standards FIPS Publication 74, 1981.
[NBS85] | Computer data authentication. National Bureau of Standards FIPS Publication 113, 1985.
[NBS93] | Secure hash standard. National Bureau of Standards FIPS Publication 180, 1993.
[NBS94] | Digital signature standard. National Bureau of Standards FIPS Publication 186, 1994.
[Od87] | A. M. Odlyzko (Ed.) Advances in Cryptology - CRYPTO '86 Proceedings. Lecture Notes in Computer Science, vol. 263, Springer-Verlag, 1987.
[Ok93] | T. Okamoto. Provably secure and practical identification schemes and corresponding signature schemes. Lecture Notes in Computer Science, 740 (1993), 31-53. (Advances in Cryptology - CRYPTO '92.)
[OSS85] | H. Ong, C. P. Schnorr and A. Shamir. Efficient signature schemes based on polynomial equations. Lecture Notes in Computer Science, 196 (1985), 37-46. (Advances in Cryptology - CRYPTO '84.)
[Pa87] | W. Patterson. Mathematical Cryptology for Computer Scientists and Mathematicians. Rowman and Littlefield, 1987.
[Pe86] | R. Peralta. Simultaneous security of bits in the discrete log. Lecture Notes in Computer Science, 219 (1986), 62-72. (Advances in Cryptology - EUROCRYPT '85.)
[Pi86] | F. Pichler (Ed.) Advances in Cryptology - EUROCRYPT '85 Proceedings. Lecture Notes in Computer Science, vol. 219, Springer-Verlag, 1986.
[PS95] | J. Piepryzk and R. Safavi-Naini (Eds.) Advances in Cryptology - ASIACRYPT '94 Proceedings. Lecture Notes in Computer Science, vol. 917, Springer-Verlag, 1995.
[PB45] | R. L. Plackett and J. P. Burman. The design of optimum multi-factorial experiments. Biometrika, 33 (1945), 305-325.
[PH78] | S. C. Pohlig and M. E. Hellman. An improved algorithm for computing logarithms over GF(p) and its cryptographic significance. IEEE Transactions on Information Theory, 24 (1978), 106-110.
[Po88] | C. Pomerance (Ed.) Advances in Cryptology - CRYPTO '87 Proceedings. Lecture Notes in Computer Science, vol. 293, Springer-Verlag, 1988.
[Po90] | C. Pomerance. Factoring. In Cryptology and Computational Number Theory, pages 27-47. American Mathematical Society, 1990.
[Po90a] | C. Pomerance (Ed.) Cryptology and Computational Number Theory, American Mathematical Society, 1990.
[PGV93] | B. Preneel, R. Govaerts and J. Vandewalle. Information authentication: hash functions and digital signatures. Lecture Notes in Computer Science, 741 (1993), 87-131. (Computer Security and Industrial Cryptography, State of the Art and Evolution, ESAT Course, May 1991.)
[PGV94] | B. Preneel, R. Govaerts and J. Vandewalle. Hash functions based on block ciphers: a synthetic approach. Lecture Notes in Computer Science, 773 (1994), 368-378. (Advances in Cryptology - CRYPTO '93.)
[QG90] | J.-J. Quisquater and L. Guillou. How to explain zero-knowledge protocols to your children. Lecture Notes in Computer Science, 435 (1990), 628-631. (Advances in Cryptology - CRYPTO '89.)
[QV90] | J.-J. Quisquater and J. Vandewalle (Eds.) Advances in Cryptology - EUROCRYPT '89 Proceedings. Lecture Notes in Computer Science, vol. 434, Springer-Verlag, 1990.
[Ra79] | M. O. Rabin. Digitized signatures and public-key functions as intractible as factorization. MIT Laboratory for Computer Science Technical Report, LCS/TR-212, 1979.
[Ra80] | M. O. Rabin. Probabilistic algorithms for testing primality. Journal of Number Theory, 12 (1980), 128-138.
[Rh94] | M. Y. Rhee. Cryptography and Secure Communications. McGraw-Hill, 1994.
[Ri91] | R. L. Rivest. The MD4 message digest algorithm. Lecture Notes in Computer Science, 537 (1991), 303-311. (Advances in Cryptology - CRYPTO '90.)
[RSA78] | R. L. Rivest, A. Shamir, and L. Adleman. A method for obtaining digital signatures and public key cryptosystems. Commununications of the ACM, 21 (1978), 120-126.
[Ro93] | K. H. Rosen. Elementary Number Theory and its Applications (Third Edition). Addison Wesley, 1993.
[Ru86] | R. A. Rueppel. Analysis and Design of Stream Ciphers. Springer-Verlag, 1986.
[Ru93] | R. A. Rueppel (Ed.) Advances in Cryptology - EUROCRYPT '92 Proceedings. Lecture Notes in Computer Science, vol. 658, Springer-Verlag, 1993.
[RV94] | R. A. Rueppel and P. C. Van Oorschot Modern key agreement techniques. To appear in Computer Communications, 1994.
[Sa90] | A. Salomaa. Public-Key Cryptography. Springer-Verlag, 1990.
[Sc94] | J. I. Schiller. Secure distributed computing. Scientific American, 271(5) (1994), 72-76.
[Sc95] | B. Schneier. Applied Cryptography, Protocols, Algorithms and Source Code in C (Second Edition). John Wiley and Sons, 1995.
[Sc91] | C. P. Schnorr. Efficient signature generation by smart cards. Journal of Cryptology, 4 (1991), 161-174.
[SP89] | J. Seberry and J. Pieprzyk Cryptography: An Introduction to Computer Security. Prentice-Hall, 1989.
[SP90] | J. Seberry and J. Pieprzyk (Eds.) Advances in Cryptology - AUSCRYPT '90 Proceedings. Lecture Notes in Computer Science, vol. 453, Springer-Verlag, 1990.
[SZ92] | J. Seberry and Y. Zheng (Eds.) Advances in Cryptology - AUSCRYPT '92 Proceedings. Lecture Notes in Computer Science, vol. 718, Springer-Verlag, 1993.
[Sh79] | A. Shamir. How to share a secret. Communications of the ACM, 22 (1979), 612-613.
[Sh84] | A. Shamir. A polynomial-time algorithm for breaking the basic Merkle-Hellman cryptosystem. IEEE Transactions on Information Theory, 30 (1984), 699-704.
[Sh90] | A. Shamir. An efficient identification scheme based on permuted kernels. Lecture Notes in Computer Science, 435 (1990), 606-609. (Advances in Cryptology - CRYPTO '89.)
[Sh94] | A. Shamir. Efficient signature schemes based on birational permutations. Lecture Notes in Computer Science, 773 (1994), 1-12. (Advances in Cryptology - CRYPTO '93.)
[Sh48] | C. E. Shannon. A mathematical theory of communication. Bell Systems Technical Journal, 27 (1948), 379-423, 623-656.
[Sh49] | C. E. Shannon. Communication theory of secrecy systems. Bell Systems Technical Journal, 28 (1949), 656-715.
[ST92] | J. H. Silverman and J. Tate. Rational Points on Elliptic Curves. Springer-Verlag, 1992.
[Si85] | G. J. Simmons. Authentication theory / coding theory. Lecture Notes in Computer Science, 196 (1985), 411-432. (Advances in Cryptology - CRYPTO '84.)
[Si88] | G. J. Simmons. A natural taxonomy for digital information authentication schemes. Lecture Notes in Computer Science, 293 (1988), 269-288. (Advances in Cryptology - CRYPTO '87.)
[Si92] | G. J. Simmons. A survey of information authentication. In Contemporary Cryptology, The Science of Information Integrity, pages 379-419. IEEE Press, 1992.
[Si92a] | G. J. Simmons. An introduction to shared secret and/or shared control schemes and their application. In Contemporary Cryptology, The Science of Information Integrity, pages 441-497. IEEE Press, 1992.
[Si92b] | G. J. Simmons (Ed.) Contemporary Cryptology, The Science of Information Integrity. IEEE Press, 1992.
[SB92] | M. E. Smid and D. K. Branstad. The data encryption standard: past and future. In Contemporary Cryptology, The Science of Information Integrity, pages 43-64. IEEE Press, 1992.
[SB93] | M. E. Smid and D. K. Branstad. Response to comments on the NIST proposed digital signature standard. Lecture Notes in Computer Science, 740 (1993), 76-88. (Advances in Cryptology - CRYPTO '92.)
[SS77] | R. Solovay and V. Strassen. A fast Monte Carlo test for primality. SIAM Journal on Computing, 6 (1977), 84-85.
[St95] | W. STallings. Network and Internetwork Security. Principles and Practice. Prentice Hall, 1995.
[St88] | D. R. Stinson. Some constructions and bounds for authentication codes. Journal of Cryptology, 1 (1988), 37-51.
[St90] | D. R. Stinson. The combinatorics of authentication and secrecy codes. Journal of Cryptology, 2 (1990), 23-49.
[St92] | D. R. Stinson. Combinatorial characterizations of authentication codes. Designs, Codes and Cryptography, 2 (1992), 175-187.
[St92a] | D. R. Stinson. An explication of secret sharing schemes. Designs, Codes and Cryptography, 2 (1992), 357-390.
[St94] | D. R. Stinson (Ed.) Advances in Cryptology - CRYPTO '93 Proceedings. Lecture Notes in Computer Science, vol. 773, Springer-Verlag, 1994.
[vHP93] | E. van Heyst and T. P. Pedersen. How to make efficient fail-stop signatures. Lecture Notes in Computer Science, 658 (1993), 366-377. (Advances in Cryptology - EUROCRYPT '92.)
[VV89] | S. A. Vanstone and P. C. Van Oorschot. An Introduction to Error Correcting Codes with Applications. Kluwer Academic Publishers, 1989.
[vT88] | H. C. A. van Tilborg. An Introduction to Cryptology. Kluwer Academic Publishers, 1988.
[vT93] | J. van Tilburg. Secret-key exchange with authentication. Lecture Notes in Computer Science, 741 (1993), 71-86. (Computer Security and Industrial Cryptography, State of the Art and Evolution, ESAT Course, May 1991.)
[VV84] | U. Vazirani and V. Vazirani. Efficient and secure pseudorandom number generation. In Proceedings of the 25th Annual Symposium on the Foundations of Computer Science, pages 458-463. IEEE Press, 1984.
[Wa90] | M. Walker. Information-theoretic bounds for authentication systems. Journal of Cryptology, 2 (1990), 131-143.
[Wa96] | P. Wayner. Disappearing Cryptography. Academic Press, 1996.
[We88] | D. Welsh. Codes and Cryptography. Oxford Science Publications, 1988.
[Wi94] | M. J. Wiener. Efficient DES key search. Technical report TR-244, School of Computer Science, Carleton University, Ottawa, Canada, May 1994 (also presented at CRYPTO '93 Rump Session).
[Wi80] | H. C. Williams. A modification of the RSA public-key encryption procedure. IEEE Transactions on Information Theory, 26 (1980), 726-729.
[Wi86] | H. C. Williams (Ed.) Advances in Cryptology - CRYPTO '85 Proceedings. Lecture Notes in Computer Science, vol. 218, Springer-Verlag, 1986.
[Ya82] | A. Yao. Theory and applications of trapdoor functions. In Proceedings of the 23rd Annual Symposium on the Foundations of Computer Science, pages 80-91. IEEE Press, 1982.

Table of Contents


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson
Table of Contents

Index

abelian group, 4, 116, 184
accept, 385
access structure, 331

threshold, 332, 333

active adversary, 258
additive identity, 3
additive inverse, 3
adjoint matrix, 16
adversary

active, 258
passive, 258

Affine Cipher, 8, 8-12

cryptanalysis of, 26-27

affine function, 8
Affine-Hill Cipher, 41
algorithm

deterministic, 129
Las Vegas, 139, 171, 234
Monte Carlo, 129, 129
probabilistic, 129

associative property, 3

of cryptosystems, 66

authentication code, 304, 304-323

combinatorial bounds, 311-313
deception probability, 305, 306-313, 319-323
entropy bounds, 321-323
impersonation attack, 305, 306-308
orthogonal array characterization, 319-320
substitution attack, 305, 307-309

authentication matrix, 306
authentication rule, 305
authentication tag, 305
authorized subset, 331
minimal, 332
Autokey Cipher, 23, 23
basis, 332
Bayes' Theorem, 45, 60, 135, 340, 341
binding, 399
binomial coefficient, 31
birthday paradox, 236
bit commitment scheme, 399, 398-401, 405-407
blob, 399
block cipher, 20
Blom Key Predistribution Scheme, 261, 260-263
Blum-Blum-Shub Generator, 371, 370-377, 379
Blum-Goldwasser Cryptosystem, 380, 379-382
boolean circuit, 333

fan-in, 333
fan-out, 333
monotone, 333

boolean formula, 333

conjunctive normal form, 337
disjunctive normal form, 334

Bos-Chaum Signature Scheme, 216, 215-217
Brickell Secret Sharing Scheme, 344, 343-348
Caesar Cipher, 4
certificate, 264
challenge, 385
challenge-and-response protocol, 217, 283, 385
Chaum-van Antwerpen Signature Scheme, 218, 217-223
Chaum-van Heijst-Pfitzmann hash function, 238, 238-241
Chinese remainder theorem, 122, 119-122, 142, 166, 380
Chor-Rivest Cryptosystem, 115
chosen ciphertext cryptanalysis, 25
chosen plaintext cryptanalysis, 25
cipher

block, 20
stream, 20, 20-24, 360

cipher block chaining mode, 83, 83, 267
cipher feedback mode, 83, 85
ciphertext, 1, 20, 378
ciphertext-only cryptanalysis, 25
closure, 332
closure property, 3
code, 194

distance of, 194
dual code, 194
generating matrix, 194
Goppa code, 195
Hamming code, 196
nearest neighbor decoding, 194
parity-check matrix, 194
syndrome, 194
syndrome decoding, 195

coin-flipping by telephone, 400
commutative cryptosystems, 66
commutative property, 3
complete graph, 346
complete multipartite graph, 346, 352, 353
completeness, 286, 386
Composites, 129, 130
computational security, 44
concave function, 56

strictly, 56

concealing, 399
conditional entropy, 59
conditional probability, 45
congruence, 3
conjunctive normal form boolean formula, 337
cryptanalysis, 6

chosen ciphertext, 25
chosen plaintext, 25
ciphertext-only, 25
known-plaintext, 25

cryptogram, 7
cryptosystem, 1

endomorphic, 64
idempotent, 66
iterated, 66
monoalphabetic, 12
polyalphabetic, 13
private-key, 114
probabilistic public-key, 378
product, 64, 64-67
public-key, 114

cyclic group, 123, 183, 187
Data Encryption Standard, 51, 70

description of, 70-78
differential cryptanalysis of, 89, 89-104
dual keys, 110
exhaustive key search, 82
expansion function, 71, 73
initial permutation, 70, 73
key schedule, 71, 75-78
modes of operation, 83, 83-86
S-boxes, 72, 73-75, 82
time-memory tradeoff, 86, 86-89

dealer, 326
deception probability, 305
decision problem, 129, 190
decomposition construction, 354, 355, 353-357
decryption rule, 1, 21, 378
determinant, 16
deterministic algorithm, 129
differential cryptanalysis, 89

characteristic, 98
filtering operation, 101
input x-or, 89
output x-or, 89
right pair, 100
wrong pair, 100

Diffie-Hellman Key Exchange, 270, 270-271
Diffie-Hellman Key Predistribution Scheme, 265, 263-267
Diffie-Hellman problem, 266, 265-267, 275
Digital Signature Standard, 205, 211, 209-213
digram, 25
disavowal protocol, 217
Discrete Logarithm Generator, 383
Discrete Logarithm problem, 162, 163, 164-177, 206, 207, 210, 238, 263, 266, 276, 287, 290, 362, 397, 400, 406

bit security of, 172-177, 400
elliptic curve, 187
generalized, 177, 177-180
in Galois fields, 183
index calculus method, 170-172
ith Bit problem, 173
Pohlig-Hellman algorithm, 169, 166-170
Shanks' algorithm, 165, 165-166

disjunctive normal form boolean formula, 334
distinguishable probability distributions, 364
distinguisher, 364
distribution rule, 338
distributive property, 4
electronic codebook mode, 83, 83
ElGamal Cryptosystem, 115, 163, 162-164, 266-267

elliptic curve, 187-190
generalized, 178, 177-178

ElGamal Signature Scheme, 205, 205-209
elliptic curve, 183, 183-187

point at infinity, 183

Elliptic Curve Cryptosystem, 115, 187-190
encryption matrix, 47
encryption rule, 1, 21, 378
endomorphic cryptosystem, 64
entropy, 52, 51-52

conditional, 59
of a natural language, 61
of a secret sharing scheme, 349-352
of authentication code, 321-323
properties of, 56-59, 349

Euclidean algorithm, 116-120, 140, 179, 181

extended, 117, 119
running time of, 128

Euler phi-function, 9
Euler pseudo-prime, 132
Euler's criterion, 130, 131, 173
exclusive-or, 21
exhaustive key search, 6, 13

of DES, 82

factor base, 171
factoring, 150-156

factor base, 153
number field sieve, 155
p - 1 algorithm, 151, 151-152
quadratic sieve, 154
trial division, 150

fan-in, 333
fan-out, 333
Fermat's theorem, 122, 137
Fibonacci number, 128
field, 10, 181
forging algorithm, 390

for Graph 3-colorability, 405
for Graph Isomorphism, 391, 394

Galois field, 180-183
Girault Key Agreement Scheme, 278, 276-279
Goldwasser-Micali Cryptosystem, 379, 378-379, 399
graph, 346

complete, 346
complete multipartite, 346, 352, 353
induced subgraph, 352
isomorphic, 386
proper 3-coloring, 401

Graph 3-colorability, 401
Graph 3-colorability Interactive Proof System, 402, 400-404, 406-407
Graph Isomorphism, 386
Graph Isomorphism Interactive Proof System, 389, 388-395
Graph Non-isomorphism, 386
Graph Non-isomorphism Interactive Proof System, 387, 386-388, 395-396
group, 4

abelian, 4, 116, 184
cyclic, 123, 183, 187
order of element in, 122

Guillou-Quisquater Identification Scheme, 296, 295-299

identity-based, 300

Hamming distance, 194
hash function, 203, 232, 232-254

birthday attack, 236-237
collision-free, 233-236
constructed from a cryptosystem, 246
extending, 241-246
one-way, 234
strongly collision-free, 233
weakly collision-free, 233

Hill Cipher, 13-17, 18

cryptanalysis of, 36-37

Huffman encoding, 53-56
Huffman's algorithm, 55
ideal decomposition, 353
ideal secret sharing scheme, 343, 344, 346-348
idempotent cryptosystem, 66
identification scheme, 282-300

converted to signature scheme, 300
identity-based, 299, 299

identity matrix, 14
impersonation, 305
implicit key authentication, 276, 278
independent random variables, 45
index of coincidence, 31

mutual, 33

indistinguishable probability distributions, 363-370, 378, 404
induced subgraph, 352
information rate, 342

monotone circuit construction, 343

injective function, 2
interactive argument

perfect zero-knowledge, 407
zero-knowledge, 406, 405-407

interactive proof, 385, 385-397

computational zero-knowledge, 398, 404, 400-404
perfect zero-knowledge, 393, 388-397
perfect zero-knowledge for Vic, 391
zero-knowledge, 385

intruder-in-the-middle attack, 271, 305
inverse matrix, 15
inverse permutation, 7
isomorphic graphs, 386
iterated cryptosystem, 66
Jacobi symbol, 132, 132-134, 370, 379
Jensen's Inequality, 56, 63, 316
joint probability, 45
Kasiski test, 31
Kerberos, 268, 267-270

key lifetime, 268
session key, 267
timestamp, 268

Kerckhoff's principle, 24
key, 1, 20, 203, 305, 326, 378
key agreement, 258

authenticated, 271

key confirmation, 269
key distribution, 258

on-line, 259

key equivocation, 59
key freshness, 267
key predistribution, 259, 260-267
key server, 259
keystream, 20
keystream alphabet, 21
keystream generator, 21
keyword, 12
known-plaintext cryptanalysis, 25
Lagrange interpolation formula, 329, 329-330
Lagrange's theorem, 122
Lamé's theorem, 128
Lamport Signature Scheme, 213, 213-215
Las Vegas algorithm, 139, 171, 234
Legendre symbol, 131, 131-132
Linear Congruential Generator, 360, 360
linear feedback shift register, 22, 360, 362
linear recurrence, 21
linear transformation, 14
m-gram Substitution Cipher, 68
matrix product, 14
McEliece Cryptosystem, 115, 196, 193-198
MD4 Hash Function, 248, 247-250
MD5 Hash Function, 247, 250
memoryless source, 53
Menezes-Vanstone Cryptosystem, 189, 188-190
Merkle-Hellman Cryptosystem, 115, 193, 190-193
message, 203, 305
message authentication code, 86, 304
message digest, 232
Miller-Rabin algorithm, 129, 130, 137, 136-138

error probability of, 138

mod operator, 3
modular exponentiation, 127

square-and-multiply algorithm, 127, 127, 131

modular multiplication, 126
modular reduction, 3
modulus, 3
monoalphabetic cryptosystem, 12
monotone circuit, 333
monotone circuit construction, 333, 335

information rate, 343

monotone property, 332
Monte Carlo algorithm, 129, 129, 374

error probability of, 129
no-biased, 129
unbiased, 374, 374-377
yes-biased, 129

MTI Key Agreement Protocol, 274, 273-276
Multiplicative Cipher, 65, 65
multiplicative identity, 4
multiplicative inverse, 10
mutual index of coincidence, 33
next bit predictor, 365-370
NP-complete problem, 44, 191, 193, 400, 404
Okamoto Identification Scheme, 291, 290-295
One-time Pad, 50, 50
one-way function, 116, 213, 234

trapdoor, 116

oracle, 139
orthogonal array, 314, 313-320

bounds, 315-318
constructions, 318-319

output feedback mode, 83, 85, 362
passive adversary, 258
perfect secrecy, 48, 44-51
perfect secret sharing scheme, 332, 339, 349
periodic stream cipher, 21
permutation, 2
Permutation Cipher, 18, 17-20
permutation matrix, 19
plaintext, 1, 20, 378
polyalphabetic cryptosystem, 13
polynomial

congruence of, 180
degree of, 180
division, 180
irreducible, 181
modular reduction of, 181

polynomial equivalence, 126
prefix-free encoding, 54
previous bit predictor, 373
primality testing, 129-138
prime, 9
Prime number theorem, 129, 135
primitive element, 123
principal square root, 373, 379
private-key cryptosystem, 114
probabilistic algorithm, 129
probabilistic encryption, 377-382
probabilistic public-key cryptosystem, 378
probability, 45

conditional, 45
joint, 45

product cryptosystem, 64, 64-67
proof of forgery algorithm, 224
proof of knowledge, 285
proper 3-coloring, 401
protocol failure, 156, 158, 208
prover, 385
pseudo-random bit generator, 359, 359-377
pseudo-square, 370
public-key cryptosystem, 114

probabilistic, 378

quadratic non-residue, 130
Quadratic Non-residues Interactive Proof System, 408
quadratic reciprocity, 132
quadratic residue, 130
Quadratic Residues, 130, 130, 371, 370-371, 374, 375, 377, 396, 399, 406
Quadratic Residues Interactive Proof System, 396, 396-397
Rabin Cryptosystem, 147, 145-150

security of, 149-150

rank, 226
redundancy of a natural language, 61
reject, 385
relative shift, 33
relatively prime, 9
replay attack, 269
response, 385
ring, 4, 180
round, 385
RSA Cryptosystem, 114, 124, 124

attacks on, 138-145
bit security of, 144-145
implementation of, 125-128

RSA Generator, 362, 362-363
RSA Signature Scheme, 203, 204
Schnorr Identification Scheme, 286, 284-289, 295
Schnorr Signature Scheme, 301
search problem, 190
secret sharing scheme, 326-357

decomposition construction, 353-357
ideal, 343, 344, 346-348
information rate, 342, 341-343, 349-355
monotone circuit construction, 333-338
threshold scheme, 326-331

Secure Hash Standard, 247, 250-252
security parameter, 284, 378
seed, 359
self-certifying public key, 276
session key, 259
Shamir Threshold Scheme, 327, 327-330, 343, 346
share, 326
Shift Cipher, 4, 3-7
Shrinking Generator, 362
signature, 203
signature scheme, 203, 202-229

constructed from identification
scheme, 300
fail-stop, 224-229
one-time, 213-217, 228
undeniable, 217-223

signing algorithm, 203
simulator, 390
Solovay-Strassen algorithm, 133, 129-136

error probability, 136, 134-136

soundness, 288, 386
source state, 304
Sperner property, 215
spurious keys, 61, 59-64

expected number of, 63

square-and-multiply algorithm, 127, 127, 131
Station-to-station Protocol, 272, 271-273
Stirling's formula, 68, 216
stream cipher, 20, 20-24, 360

cryptanalysis of, 37
synchronous, 21, 85

Subgroup Membership, 397
Subgroup Membership Interactive Proof System, 398
Subset Sum problem, 190, 190-191

modular transformation, 192
superincreasing, 191

substitution, 305
Substitution Cipher, 7, 7, 7-8

cryptanalysis of, 27-31
m-gram, 68

synchronous stream cipher, 21, 85
threshold scheme, 326, 326-331
timestamping, 252-254
transcript, 390
Transposition Cipher, 17
trapdoor, 116
trigram, 25
trusted authority, 258
unconditional security, 45
unicity distance, 63, 59-64
van Heyst-Pedersen Signature Scheme, 225, 224-229
Vandermonde matrix, 329

determinant of, 329

verification algorithm, 203
verifier, 385
Vernam One-time Pad, 50, 50
Vigenere Cipher, 12, 12-13, 40

cryptanalysis of, 31-36

zero-knowledge interactive argument, 406, 405-407

perfect, 407

zero-knowledge interactive proof, 385

computational, 398, 404, 400-404
perfect, 393, 388-397
perfect, for Vic, 391

Table of Contents


Cryptography: Theory and Practice

 [Spisak knjiga]
Cryptography: Theory and Practice
Douglas Stinson
Preface
Dedication

Chapter 1—Classical Cryptography
1.1 Introduction: Some Simple Cryptosystems
1.1.1 The Shift Cipher
1.1.2 The Substitution Cipher
1.1.3 The Affine Cipher
1.1.4 The Vigenere Cipher
1.1.5 The Hill Cipher
1.1.6 The Permutation Cipher
1.1.7 Stream Ciphers
1.2 Cryptanalysis
1.2.1 Cryptanalysis of the Affine Cipher
1.2.2 Cryptanalysis of the Substitution Cipher
1.2.3 Cryptanalysis of the Vigenere Cipher
1.2.5 Cryptanalysis of the LFSR-based Stream Cipher
1.3 Notes
Exercises
Chapter 2—Shannon’s Theory
2.1 Perfect Secrecy
2.2 Entropy
2.2.1 Huffman Encodings and Entropy
2.3 Properties of Entropy
2.4 Spurious Keys and Unicity Distance
2.5 Product Cryptosystems
2.6 Notes
Exercises
Chapter 3—The Data Encryption Standard
3.1 Introduction
3.2 Description of DES
3.2.1 An Example of DES Encryption
3.3 The DES Controversy
3.4 DES in Practice
3.4.1 DES Modes of Operation
3.5 A Time-memory Trade-off
3.6 Differential Cryptanalysis
3.6.1 An Attack on a 3-round DES
3.6.2 An Attack on a 6-round DES
3.6.3 Other examples of Differential Cryptanalysis
3.7 Notes and References
Exercises
Chapter 4—The RSA System and Factoring
4.1 Introduction to Public-key Cryptography
4.2 More Number Theory
4.2.1 The Euclidean Algorithm
4.2.2 The Chinese Remainder Theorem
4.2.3 Other Useful Facts
4.3 The RSA Cryptosystem
4.4 Implementing RSA
4.5 Probabilistic Primality Testing
4.6 Attacks On RSA
4.6.1 The Decryption Exponent
4.6.2 Partial Information Concerning Plaintext Bits
4.7 The Rabin Cryptosystem
4.8 Factoring Algorithms
4.8.1 The p - 1 Method
4.8.2 Dixon’s Algorithm and the Quadratic Sieve
4.8.3 Factoring Algorithms in Practice
4.9 Notes and References
Exercises
Chapter 5—Other Public-key Cryptosystems
5.1 The ElGamal Cryptosystem and Discrete Logs
5.1.1 Algorithms for the Discrete Log Problem
5.1.2 Bit Security of Discrete Logs
5.2 Finite Field and Elliptic Curve Systems
5.2.1 Galois Fields
5.2.2 Elliptic Curves
5.3 The Merkle-Hellman Knapsack System
5.4 The McEliece System
5.5 Notes and References
Exercises
Chapter 6—Signature Schemes
6.1 Introduction
6.2 The ElGamal Signature Scheme
6.3 The Digital Signature Standard
6.4 One-time Signatures
6.5 Undeniable Signatures
6.6 Fail-stop Signatures
6.7 Notes and References
Exercises
Chapter 7—Hash Functions
7.1 Signatures and Hash Functions
7.2 Collision-free Hash Functions
7.3 The Birthday Attack
7.4 A Discrete Log Hash Function
7.5 Extending Hash Functions
7.6 Hash Functions from Cryptosystems
7.7 The MD4 Hash Function
7.8 Timestamping
7.9 Notes and References
Exercises
Chapter 8—Key Distribution and Key Agreement
8.1 Introduction
8.2 Key Predistribution
8.2.1 Blom’s Scheme
8.2.2 Diffie-Hellman Key Predistribution
8.3 Kerberos
8.4 Diffie-Hellman Key Exchange
8.4.1 The Station-to-station Protocol
8.4.2 MTI Key Agreement Protocols
8.4.3 Key Agreement Using Self-certifying Keys
8.5 Notes and References
Exercises
Chapter 9—Identification Schemes
9.1 Introduction
9.2 The Schnorr Identification Scheme
9.3 The Okamoto Identification Scheme
9.4 The Guillou-Quisquater Identification Scheme
9.4.1 Identity-based Identification Schemes
9.5 Converting Identification to Signature Schemes
9.6 Notes and References
Exercises
Chapter 10—Authentication Codes
10.1 Introduction
10.2 Computing Deception Probabilities
10.3 Combinatorial Bounds
10.3.1 Orthogonal Arrays
10.3.2 Constructions and Bounds for OAs
10.3.3 Characterizations of Authentication Codes
10.4 Entropy Bound
10.5 Notes and References
Exercises
Chapter 11—Secret Sharing Schemes
11.1 Introduction: The Shamir Threshold Scheme
11.2 Access Structures and General Secret Sharing
11.3 The Monotone Circuit Construction
11.4 Formal Definitions
11.5 Information Rate
11.6 The Brickell Vector Space Construction
11.7 An Upper Bound on the Information Rate
11.8 The Decomposition Construction
11.9 Notes and References
Exercises
Chapter 12—Pseudo-random Number Generation
12.1 Introduction and Examples
12.2 Indistinguishable Probability Distributions
12.2.1 Next Bit Predictors
12.3 The Blum-Blum-Shub Generator
12.3.1 Security of the BBS Generator
12.4 Probabilistic Encryption
12.5 Notes and References
Exercises
Chapter 13—Zero-knowledge Proofs
13.1 Interactive Proof Systems
13.2 Perfect Zero-knowledge Proofs
13.3 Bit Commitments
13.4 Computational Zero-knowledge Proofs
13.5 Zero-knowledge Arguments
13.6 Notes and References
Exercises
Further Reading
Index


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 1
Classical Cryptography

1.1 Introduction: Some Simple Cryptosystems

The fundamental objective of cryptography is to enable two people, usually referred to as Alice and Bob, to communicate over an insecure channel in such a way that an opponent, Oscar, cannot understand what is being said. This channel could be a telephone line or computer network, for example. The information that Alice wants to send to Bob, which we call "plaintext," can be English text, numerical data, or anything at all — its structure is completely arbitrary. Alice encrypts the plaintext, using a predetermined key, and sends the resulting ciphertext over the channel. Oscar, upon seeing the ciphertext in the channel by eavesdropping, cannot determine what the plaintext was; but Bob, who knows the encryption key, can decrypt the ciphertext and reconstruct the plaintext.

This concept is described more formally using the following mathematical notation.

DEFINITION 1.1 A cryptosystem is a five-tuple [image], where the following conditions are satisfied:

1. is a finite set of possible plaintexts
2. is a finite set of possible ciphertexts
3. , the keyspace, is a finite set of possible keys
4. For each [image], there is an encryption rule eK [image] and a corresponding decryption rule [image]. Each [image] and [image] are functions such that dK(eK(x)) = x for every plaintext

The main property is property 4. It says that if a plaintext x is encrypted using eK, and the resulting ciphertext is subsequently decrypted using dK, then the original plaintext x results.

Alice and Bob will employ the following protocol to use a specific cryptosystem. First, they choose a random key [image] This is done when they are in the same place and are not being observed by Oscar, or, alternatively, when they do have access to a secure channel, in which case they can be in different places. At a later time, suppose Alice wants to communicate a message to Bob over an insecure channel. We suppose that this message is a string

for some integer n ≥ 1, where each plaintext symbol [image], 1 ≤ i ≤ n. Each xi is encrypted using the encryption rule eK specified by the predetermined key K. Hence, Alice computes yi = eK(xi), 1 ≤ i ≤ n, and the resulting ciphertext string

is sent over the channel. When Bob receives y1y2 . . . yn, he decrypts it using the decryption function dK, obtaining the original plaintext string, x1x2. . .xn. See Figure 1.1 for an illustration of the communication channel.

*
* Figure 1.1 The Communication Channel

Clearly, it must be the case that each encryption function eK is an injective function (i.e., one-to-one), otherwise, decryption could not be accomplished in an unambiguous manner. For example, if

where x1 ≠ x2, then Bob has no way of knowing whether y should decrypt to x1 or x2. Note that if [image], it follows that each encryption function is a permutation. That is, if the set of plaintexts and ciphertexts are identical, then each encryption function just rearranges (or permutes) the elements of this set.

1.1.1 The Shift Cipher

In this section, we will describe the Shift Cipher, which is based on modular arithmetic. But first we review some basic definitions of modular arithmetic.

DEFINITION 1.2 Suppose a and b are integers, and m is a positive integer. Then we write a ≡ b (mod m) if m divides b - a. The phrase a ≡ b (mod m) is read as "a is congruent to b modulo m." The integer m is called the modulus.

Suppose we divide a and b by m, obtaining integer quotients and remainders, where the remainders are between 0 and m - 1. That is, a = q1m + r1 and b = q2m + r2, where 0 ≤ r1 ≤ m - 1 and 0 ≤ r2 ≤ m - 1. Then it is not difficult to see that a ≡ b (mod m) if and only if r1 = r2. We will use the notation a mod m (without parentheses) to denote the remainder when a is divided by m, i.e., the value r1 above. Thus a ≡ b (mod m) if and only if a mod m = b mod m. If we replace a by a mod m, we say that a is reduced modulo m.

REMARK Many computer programming languages define a mod m to be the remainder in the range -m + 1, . . . , m - 1 having the same sign as a. For example, -18 mod 7 would be -4, rather than 3 as we defined it above. But for our purposes, it is much more convenient to define a mod m always to be nonnegative.

We can now define arithmetic modulo m: is defined to be the set {0, . . . , m-1}, equipped with two operations, + and ×. Addition and multiplication in [image] work exactly like real addition and multiplication, except that the results are reduced modulo m.

For example, suppose we want to compute 11 × 13 in [image]. As integers, we have 11 × 13 = 143. To reduce 143 modulo 16, we just perform ordinary long division: 143 = 8 × 16 + 15, so 143 mod 16 = 15, and hence 11 × 13 = 15 in [image].


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

These definitions of addition and multiplication in [image] satisfy most of the familiar rules of arithmetic. We will list these properties now, without proof:

1. addition is closed, i.e., for any
2. addition is commutative, i.e., for any [image], a + b = b + a
3. addition is associative, i.e., for any [image], (a + b) + c = a + (b + c)
4. 0 is an additive identity, i.e., for any [image], a + 0 = 0 + a = a
5. the additive inverse of any [image] is m-a, i.e., a+(m-a) = (m-a)+a = 0 for any
6. multiplication is closed, i.e., for any
7. multiplication is commutative, i.e., for any [image], ab = ba

*
* Figure 1.2 Shift Cipher

8. multiplication is associative, i.e., for any [image], (ab)c = a(bc)
9. 1 is a multiplicative identity, i.e., for any [image], a × 1 = 1 × a = a
10. multiplication distributes over addition, i.e., for any [image], (a+b)c = (ac) + (bc) and a(b + c) (ab) + (ac).

Properties 1, 3-5 say that [image] forms an algebraic structure called a group with respect to the addition operation. Since property 2 also holds, the group is said to be abelian.

Properties 1-10 establish that [image] is, in fact, a ring. We will see many other examples of groups and rings in this book. Some familiar examples of rings include the integers, [image]; the real numbers, [image]; and the complex numbers, [image]. However, these are all infinite rings, and our attention will be confined almost exclusively to finite rings.

Since additive inverses exist in [image], we can also subtract elements in [image]. We define a - b in [image] to be a + m - b mod m. Equivalently, we can compute the integer a - b and then reduce it modulo m.

For example, to compute 11 - 18 in [image], we can evaluate 11 + 13 mod 31 = 24. Alternatively, we can first subtract 18 from 11, obtaining -7 and then compute -7 mod 31 = 24.

We present the Shift Cipher in Figure 1.2. It is defined over [image] since there are 26 letters in the English alphabet, though it could be defined over [image] for any modulus m. It is easy to see that the Shift Cipher forms a cryptosystem as defined above, i.e., dK(eK(x)) = x for every

REMARK For the particular key K = 3, the cryptosystem is often called the Caesar Cipher, which was purportedly used by Julius Caesar.

We would use the Shift Cipher (with a modulus of 26) to encrypt ordinary English text by setting up a correspondence between alphabetic characters and residues modulo 26 as follows: A ↔ 0, B ↔ 1, . . . , Z ↔ 25. Since we will be using this correspondence in several examples, let's record it for future use:

A small example will illustrate.

Example 1.1

Suppose the key for a Shift Cipher is K = 11, and the plaintext is

                         wewillmeetatmidnight.

We first convert the plaintext to a sequence of integers using the specified correspondence, obtaining the following:

22 | 4 | 22 | 8 | 11 | 11 | 12 | 4 | 4 | 19
0 | 19 | 12 | 8 | 3 | 13 | 8 | 6 | 7 | 19

Next, we add 11 to each value, reducing each sum modulo 26:

7 | 15 | 7 | 19 | 22 | 22 | 23 | 15 | 15 | 4
11 | 4 | 23 | 19 | 14 | 24 | 19 | 17 | 18 | 4

Finally, we convert the sequence of integers to alphabetic characters, obtaining the ciphertext:

                         HPHTWWXPPELEXTOYTRSE

To decrypt the ciphertext, Bob will first convert the ciphertext to a sequence of integers, then subtract 11 from each value (reducing modulo 26), and finally convert the sequence of integers to alphabetic characters.

REMARK In the above example we are using upper case letters for ciphertext and lower case letters for plaintext, in order to improve readability. We will do this elsewhere as well.

If a cryptosystem is to be of practical use, it should satisfy certain properties. We informally enumerate two of these properties now.

1. Each encryption function eK and each decryption function dK should be efficiently computable.
2. An opponent, upon seeing a ciphertext string y, should be unable to determine the key K that was used, or the plaintext string x.

The second property is defining, in a very vague way, the idea of "security." The process of attempting to compute the key K, given a string of ciphertext y, is called cryptanalysis. (We will make these concepts more precise as we proceed.) Note that, if Oscar can determine K, then he can decrypt y just as Bob would, using dK. Hence, determining K is at least as difficult as determining the plaintext string x.

We observe that the Shift Cipher (modulo 26) is not secure, since it can be cryptanalyzed by the obvious method of exhaustive key search. Since there are only 26 possible keys, it is easy to try every possible decryption rule dK until a "meaningful" plaintext string is obtained. This is illustrated in the following example.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 1.2

Given the ciphertext string

                         JBCRCLQRWCRVNBJENBWRWN,

we successively try the decryption keys d0, d1, etc. The following is obtained:

                         jbcrclqrwcrvnbjenbwrwn
                         iabqbkpqvbqumaidmavqvm
                         hzapajopuaptlzhclzupul
                         gyzozinotzoskygbkytotk
                         fxynyhmnsynrjxfajxsnsj
                         ewxmxglmrxmqiweziwrmri
                         dvwlwfklqwlphvdyhvqlqh
                         cuvkvejkpvkogucxgupkpg
                         btujudijoujnftbwftojof
                         astitchintimesavesnine

At this point, we have determined the plaintext and we can stop. The key is K = 9.

On average, a plaintext will be computed after trying 26/2 = 13 decryption rules.

*
* Figure 1.3 Substitution Cipher

As the above example indicates, a necessary condition for a cryptosystem to be secure is that an exhaustive key search should be infeasible; i.e., the keyspace should be very large. As might be expected, a large keyspace is not sufficient to guarantee security.

1.1.2 The Substitution Cipher

Another well-known cryptosystem is the Substitution Cipher. This cryptosystem has been used for hundreds of years. Puzzle "cryptograms" in newspapers are examples of Substitution Ciphers. This cipher is defined in Figure 1.3.

Actually, in the case of the Substitution Cipher, we might as well take [image] and [image] both to be the 26-letter English alphabet. We used [image] in the Shift Cipher because encryption and decryption were algebraic operations. But in the Substitution Cipher, it is more convenient to think of encryption and decryption as permutations of alphabetic characters.

Here is an example of a "random" permutation, π, which could comprise an encryption function. (As before, plaintext characters are written in lower case and ciphertext characters are written in upper case.)

Thus, eπ(a) = X, eπ(b) = N, etc. The decryption function is the inverse permutation. This is formed by writing the second lines first, and then sorting in alphabetical order. The following is obtained:

Hence, dπ(A) = d, dπ(B) = l, etc.

As an exercise, the reader might decrypt the following ciphertext using this decryption function:

                  MGZVYZLGHCMHJMYXSSFMNHAHYCDLMHA.

A key for the Substitution Cipher just consists of a permutation of the 26 alphabetic characters. The number of these permutations is 26!, which is more than 4.0 × 1026, a very large number. Thus, an exhaustive key search is infeasible, even for a computer. However, we shall see later that a Substitution Cipher can easily be cryptanalyzed by other methods.

1.1.3 The Affine Cipher

The Shift Cipher is a special case of the Substitution Cipher which includes only 26 of the 26! possible permutations of 26 elements. Another special case of the Substitution Cipher is the Affine Cipher, which we describe now. In the Affine Cipher, we restrict the encryption functions to functions of the form

These functions are called affine functions, hence the name Affine Cipher. (Observe that when a = 1, we have a Shift Cipher.)

In order that decryption is possible, it is necessary to ask when an affine function is injective. In other words, for any [image], we want the congruence

to have a unique solution for x. This congruence is equivalent to

Now, as y varies over [image], so, too, does y - b vary over [image]. Hence, it suffices to study the congruence ax ≡ y (mod 26) [image].

We claim that this congruence has a unique solution for every y if and only if gcd(a, 26) = 1 (where the gcd function denotes the greatest common divisor of its arguments). First, suppose that gcd(a, 26) = d > 1. Then the congruence ax ≡ 0 (mod 26) has (at least) two distinct solutions in [image], namely x = 0 and x = 26/d. In this case e(x) = ax + b mod 26 is not an injective function and hence not a valid encryption function.

For example, since gcd(4, 26) = 2, it follows that 4x + 7 is not a valid encryption function: x and x + 13 will encrypt to the same value, for any

Let's next suppose that gcd(a, 26) = 1. Suppose for some x1 and x2 that

Then

and thus

We now make use of a property of division: if gcd(a, b) = 1 and a | bc, then a | c. Since 26| a(x1 - x2) and gcd(a, 26) = 1, we must therefore have that

i.e., x1 ≡ x2 (mod 26).

At this point we have shown that, if gcd(a, 26) = 1, then a congruence of the form ax ≡ y (mod 26) has, at most, one solution in [image]. Hence, if we let x vary over [image], then ax mod 26 takes on 26 distinct values modulo. 26. That is, it takes on every value exactly once. It follows that, for any [image], the congruence ax ≡ y (mod 26) has a unique solution for y.

There is nothing special about the number 26 in this argument. The following result can be proved in an analogous fashion.

THEOREM 1.1

The congruence ax ≡ b (mod m) has a unique solution [image] for every [image] if and only if gcd(a, m) = 1.

Since 26 = 2 × 13, the values of [image] such that gcd(a, 26) = 1 are a = 1, 3, 5, 7, 9, 11, 15, 17, 19, 21, 23, and 25. The parameter b can be any element in [image]. Hence the Affine Cipher has 12 × 26 = 312 possible keys. (Of course, this is much too small to be secure.)

Let's now consider the general setting where the modulus is m. We need another definition from number theory.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

DEFINITION 1.3 Suppose a ≥ 1 and m ≥ 2 are integers. If gcd(a, m) = 1, then we say that a and m are relatively prime. The number of integers in [image] that are relatively prime to m is often denoted by φ(m) (function is called the Euler phi-function).

A well-known result from number theory gives the value of φ(m) in terms of the prime power factorization of m. (An integer p > 1 is prime if it has no positive divisors other than 1 and p. Every integer m > 1 can be factored as a product of powers of primes in a unique way. For example, 60 = 22 × 3 × 5 and 98 = 2 × 72.) We record the formula for φ(m) in the following theorem.

THEOREM 1.2

Suppose

where the pi′s are distinct primes and ei > 0, 1 ≤ i ≤ n. Then

It follows that the number of keys in the Affine Cipher over [image] is mφ(m), where φ(m) is given by the formula above. (The number of choices for b is m, and the number of choices for a is φ(m), where the encryption function is e(x) = ax + b.) For example, when m = 60, φ(60) = 2 × 2 × 4 = 16 and the number of keys in the Affine Cipher is 960.

Let's now consider the decryption operation in the Affine Cipher with modulus m = 26. Suppose that gcd(a, 26) = 1. To decrypt, we need to solve the congruence y ≡ ax + b (mod 26) for x. The discussion above establishes that the congruence will have a unique solution in [image], but it does not give us an efficient method of finding the solution. What we require is an efficient algorithm to do this. Fortunately, some further results on modular arithmetic will provide us with the efficient decryption algorithm we seek.

We require the idea of a multiplicative inverse.

DEFINITION 1.4 Suppose [image]. The multiplicative inverse of a is an element [image] such that aa-1 ≡ a-1 a ≡ 1 (mod m).

By similar arguments to those used above, it can be shown that a has a multiplicative inverse modulo m if and only if gcd(a, m) = 1; and if a multiplicative inverse exists, it is unique. Also, observe that if b = a-1, then a = b-1. If p is prime, then every non-zero element of [image] has a multiplicative inverse. A ring in which this is true is called a field.

In a later section, we will describe an efficient algorithm for computing multiplicative inverses in [image] for any m. However, in [image], trial and error suffices to find the multiplicative inverses of the elements relatively prime to 26: 1-1 = 1, 3-1 = 9, 5-1 = 21, 7-1 = 15, 11-1 = 19, 17-1 = 23, and 25-1 = 25. (All of these can be verified easily. For example, 7 × 15 = 105 ≡ 1 mod 26, so 7-1 = 15.)

Consider our congruence y ≡ ax + b (mod 26). This is equivalent to

Since gcd(a, 26) = 1, a has a multiplicative inverse modulo 26. Multiplying both sides of the congruence by a-1, we obtain

*
* Figure 1.4 Affine Cipher

By associativity of multiplication modulo 26,

Consequently, x ≡ a-1(y - b) (mod 26). This is an explicit formula for x, that is, the decryption function is

So, finally, the complete description of the Affine Cipher is given in Figure 1.4. Let's do a small example.

Example 1.3

Suppose that K = (7, 3). As noted above, 7-1 mod 26 = 15. The encryption function is

and the corresponding decryption function is

where all operations are performed in [image]. It is good check to verify that dK(eK(x)) = x for all [image] Computing in [image], we get

*
* Figure 1.5 Vigenere Cipher

To illustrate, let's encrypt the plaintext hot. We first convert the letters h, o, t to residues modulo 26. These are respectively 7, 14, and 19. Now, we encrypt:

7 × 7 + 3 mod 26 | = | 52 mod 26 | = 0
7 × 14 + 3 mod 26 | = | 101 mod 26 | = 23
7 × 19 + 3 mod 26 | = | 136 mod 26 | = 6.

So the three ciphertext characters are 0, 23, and 6, which corresponds to the alphabetic string AXG. We leave the decryption as an exercise for the reader.

1.1.4 The Vigenere Cipher

In both the Shift Cipher and the Substitution Cipher, once a key is chosen, each alphabetic character is mapped to a unique alphabetic character. For this reason, these cryptosystems are called monoalphabetic. We now present in Figure 1.5 a cryptosystem which is not monoalphabetic, the well-known Vigenere Cipher. This cipher is named after Blaise de Vigenere, who lived in the sixteenth century.

Using the correspondence A ↔ 0, B ↔ 1, . . ., Z ↔ 25 described earlier, we can associate each key K with an alphabetic string of length m, called a keyword. The Vigenere Cipher encrypts m alphabetic characters at a time: each plaintext element is equivalent to m alphabetic characters.

Let's do a small example.

Example 1.4

Suppose m = 6 and the keyword is CIPHER. This corresponds to the numerical equivalent K = (2, 8, 15, 7, 4, 17). Suppose the plaintext is the string

                     thiscryptosystemisnotsecure.

We convert the plaintext elements to residues modulo 26, write them in groups of six, and then "add" the keyword modulo 26, as follows:

The alphabetic equivalent of the ciphertext string would thus be:

                     VPXZGIAXIVWPUBTTMJPWIZITWZT.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

To decrypt, we can use the same keyword, but we would subtract it modulo 26 instead of adding.

Observe that the number of possible keywords of length m in a Vigenere Cipher is 26m, so even for relatively small values of m, an exhaustive key search would require a long time. For example, if we take m = 5, then the keyspace has size exceeding 1.1 × 107. This is already large enough to preclude exhaustive key search by hand (but not by computer).

In a Vigenere Cipher having keyword length m, an alphabetic character can be mapped to one of m possible alphabetic characters (assuming that the keyword contains m distinct characters). Such a cryptosystem is called polyalphabetic. In general, cryptanalysis is more difficult for polyalphabetic than for monoalphabetic cryptosystems.

1.1.5 The Hill Cipher

In this section, we describe another polyalphabetic cryptosystem called the Hill Cipher. This cipher was invented in 1929 by Lester S. Hill. Let m be a positive integer, and define [image]. The idea is to take m linear combinations of the m alphabetic characters in one plaintext element, thus producing the m alphabetic characters in one ciphertext element.

For example, if m = 2, we could write a plaintext element as x = (x2, x2) and a ciphertext element as y = (y1, y2). Here, y1 would be a linear combination of x1 and x2, as would y2. We might take

Of course, this can be written more succinctly in matrix notation as follows:

In general, we will take an m × m matrix K as our key. If the entry in row i and column j of K is ki,j, then we write K = (ki,j). For x = (x1, . . . , xm) [image] and [image], we compute y = eK(x) = (y1, . . . , ym) as follows:

In other words, y = xK.

We say that the ciphertext is obtained from the plaintext by means of a linear transformation. We have to consider how decryption will work, that is, how x can be computed from y. Readers familiar with linear algebra will realize that we use the inverse matrix K-1 to decrypt. The ciphertext is decrypted using the formula x = yK-1.

Here are the definitions of necessary concepts from linear algabra. If A = (ai,j) is an [image] matrix and B = (bj,k) is an m × n matrix, then we define the matrix product AB = (ci,k) by the formula

for [image] and 1 ≤ k ≤ n. That is, the entry in row i and column k of AB is formed by taking the ith row of A and the kth column of B, multiplying corresponding entries together, and summing. Note that AB is an [image] matrix.

This definition of matrix multiplication is associative (that is, (AB)C = A(BC) but not, in general, commutative (it is not always the case that AB = BA, even for square matrices A and B).

The m × m identity matrix, denoted by Im, is the m × m matrix with 1′s on the main diagonal and 0′s elsewhere. Thus, the 2 × 2 identity matrix is

Im is termed an identity matrix since AIm = A for any [image] matrix A and ImB = B for any m × n matrix B. Now, the inverse matrix to an m × m matrix A (if it exists) is the matrix A-1 such that AA-1 = A-1 A = Im. Not all matrices have inverses, but if an inverse exists, it is unique.

With these facts at hand, it is easy to derive the decryption formula given above: since y = xK, we can multiply both sides of the formula by K-1, obtaining

(Note the use of the associativity property.)

We can verify that the encryption matrix above has an inverse in

since

Remember that all arithmetic operations are done modulo 26.)

Let's now do an example to illustrate encryption and decryption in the Hill Cipher.

Example 1.5

Suppose the key is

From the computations above, we have that

Suppose we want to encrypt the plaintext july. We have two elements of plaintext to encrypt: (9, 20) (corresponding to ju) and (11, 24) (corresponding to ly). We compute as follows:

and

Hence, the encryption of july is DELW. To decrypt, Bob would compute:

and

Hence, the correct plaintext is obtained.

At this point, we have shown that decryption is possible if K has an inverse. In fact, for decryption to be possible, it is necessary that K has an inverse. (This follows fairly easily from elementary linear algebra, but we will not give a proof here.) So we are interested precisely in those matrices K that are invertible.

The invertibility of a (square) matrix depends on the value of its determinant. To avoid unnecessary generality, we will confine our attention to the 2 × 2 case.

DEFINITION 1.5 The determinant of the 2 × 2 matrix A = (ai,j) is the value

REMARK The determinant of an m × m square matrix can be computed by elementary row operations: see any text on linear algebra.

Two important properties of determinants are that det Im = 1; and the multiplication rule det(AB) = det A × det B.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

A real matrix K has an inverse if and only if its determinant is non-zero. However, it is important to remember that we are working over [image]. The relevant result for our purposes is that a matrix K has an inverse modulo 26 if and only if gcd (det K, 26) = 1.

We briefly sketch the proof of this fact. First suppose that gcd(det K, 26) = 1. Then det K has an inverse in [image] Now, for ≤ i ≤ m, 1 ≤ j ≤ m, define Kij to be the matrix obtained from K by deleting the ith row and the jth column. Define a matrix K* to have as its (i, j)-entry the value (-1)i+j det Kji. (K* is called the adjoint matrix of K.) Then it can be shown that

Hence, K is invertible.

Conversely, suppose K has an inverse, K-1. By the multiplication rule for determinants, we have

Hence, det K is invertible in

REMARK The above formula for K-1 is not very efficient computationally, except for small values of m (say m = 2, 3). For larger m, the preferred method of computing inverse matrices would involve elementary row operations.

In the 2 × 2 case, we have the following formula:

THEOREM 1.3

Suppose A = (ai,j) is a 2 × 2 matrix over [image] such that det A = a1,1a2,2 - a1,2a2,1 is invertible. Then

Let's look again at the example considered earlier. First, we have

Now, 1-1 mod 26 = 1, so the inverse matrix is

as we verified earlier.

We now give a precise description of the Hill Cipher over [image] in Figure 1.6.

1.1.6 The Permutation Cipher

All of the cryptosystems we have discussed so far involve substitution: plaintext characters are replaced by different ciphertext characters. The idea of a permutation cipher is to keep the plaintext characters unchanged, but to alter their positions by rearranging them. The Permutation Cipher (also known as the Transposition Cipher) has been in use for hundreds of years. In fact, the distinction between the Permutation Cipher and the Substitution Cipher was pointed out as early as 1563 by Giovanni Porta. A formal definition is given in Figure 1.7.

As with the Substitution Cipher, it is more convenient to use alphabetic characters as opposed to residues modulo 26, since there are no algebraic operations being performed in encryption or decryption.

Here is an example to illustrate:

*
* Figure 1.6 Hill Cipher

*
* Figure 1.7 Permutation Cipher

Example 1.6

Suppose m = 6 and the key is the following permutation π:

Then the inverse permutation π-1 is the following:

Now, suppose we are given the plaintext

                      shesellsseashellsbytheseashore.

We first group the plaintext into groups of six letters:

                 shesel | lsseas | hellsb | ythese | ashore

Now each group of six letters is rearranged according to the permutation π, yielding the following:

                 EESLSH | SALSES | LSHBLE | HSYEET | HRAEOS

So, the ciphertext is:

                       EESLSHSALSESLSHBLEHSYEETHRAEOS.

The ciphertext can be decrypted in a similar fashion, using the inverse permutation π-1.

In fact, the Permutation Cipher is a special case of the Hill Cipher. Given a permutation of π of the set {1, . . . , m}, we can define an associated m × m permutation matrix Kπ = (ki,j) according to the formula

(A permutation matrix is a matrix in which every row and column contains exactly one "1," and all other values are "0." A permutation matrix can be obtained from an identity matrix by permuting rows or columns.)

It is not difficult to see that Hill encryption using the matrix Kπ is, in fact, equivalent to permutation encryption using the permutation π. Moreover, [image], i.e., the inverse matrix to Kπ is the permutation matrix defined by the permutation π-1. Thus, Hill decryption is equivalent to permutation decryption.

For the permutation π used in the example above, the associated permutation matrices are

and

The reader can verify that the product of these two matrices is the identity.

1.1.7 Stream Ciphers

In the cryptosystems we have studied to this point, successive plaintext elements are encrypted using the same key, K. That is, the ciphertext string y is obtained as follows:

Cryptosystems of this type are often called block ciphers.

An alternative approach is to use what are called stream ciphers. The basic idea is to generate a keystream z = z1z2 . . . , and use it to encrypt a plaintext string x = x1x2 . . . according to the rule

A stream cipher operates as follows. Suppose [image] is the key and x1x2 . . . is the plaintext string. The function fi is used to generate zi (the ith element of the keystream), where fi is a function of the key, K, and the first i - 1 plaintext characters:

The keystream element zi is used to encrypt xi, yielding [image]. So, to encrypt the plaintext string x1x2, . . . , we would successively compute

Decrypting the ciphertext string y1y2 . . . can be accomplished by successively computing

Here is a formal mathematical definition:


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

DEFINITION 1.6 A Stream Cipher is a tuple [image], where the following conditions are satisfied:

1. is a finite set of possible plaintexts
2. is a finite set of possible ciphertexts
3. , the keyspace, is a finite set of possible keys
4. is a finite set called the keystream alphabet
5. is the keystream generator. For i ≥ 1,

6. For each [image], there is an encryption rule [image] and a corresponding decryption rule [image] and [image] are functions such that dz(ez(x)) = x for every plaintext [image].

We can think of a block cipher as a special case of a stream cipher where the keystream is constant: zi = K for all i ≥ 1.

Here are some special types of stream ciphers together with illustrative examples. A stream cipher is synchronous if the keystream is independent of the plaintext string, that is, if the keystream is generated as a function only of the key K. In this situation, we think of K as a "seed" that is expanded into a keystream z1z2 . . ..

A stream cipher is periodic with period d if Zi+d = zi for all integers i ≥ 1. The Vigenere Cipher with keyword length m can be thought of as a periodic stream cipher with period m. In this case, the key is K = (k1, . . . , km). K itself provides the first m elements of the keystream: zi = ki, 1 ≤ i ≤ m. Then the keystream just repeats itself from that point on. Observe that in this stream cipher setting for the Vigenere Cipher, the encryption and decryption functions are identical to those used in the Shift Cipher: ez(x) = x + z and dz (y) = y - z.

Stream ciphers are often described in terms of binary alphabets, i.e., [image]. In this situation, the encryption and decryption operation are just addition modulo 2:

and

If we think of "0" as representing the boolean value "false" and "1" as representing "true," then addition modulo 2 corresponds to the exclusive-or operation. Hence, encryption (and decryption) can be implemented very efficiently in hardware.

Let's look at another method of generating a (synchronous) keystream. Suppose we start with (k1, . . . , km) and let zi = ki, 1 ≤ i ≤ m (as before), but we now generate the keystream using a linear recurrence relation of degree m:

where c0, . . . , [image] are predetermined constants.

REMARK This recurrence is said to have degree m since each term depends on the previous m terms. It is linear because zi+m is a linear function of previous terms. Note that we can take c0 = 1 without loss of generality, for otherwise the recurrence will be of degree m - 1.

Here, the key K consists of the 2m values k1, . . . , km, c0, . . . , cm-1. If

then the keystream consists entirely of 0's. Of course, this should be avoided, as the ciphertext will then be identical to the plaintext. However, if the constants c0, . . . , cm-1 are chosen in a suitable way, then any other initialization vector (k1, . . . , km) will give rise to a periodic keystream having period 2m - 1. So a "short" key can give rise to a keystream having a very long period. This is certainly a desirable property: we will see in a later section how the Vigenere Cipher can be cryptanalyzed by exploiting the fact that the keystream has short period.

Here is an example to illustrate.

Example 1.7

Suppose m = 4 and the keystream is generated using the rule

(i ≥ 1). If the keystream is initialized with any vector other than (0, 0, 0, 0), then we obtain a keystream of period 15. For example, starting with (1, 0, 0, 0), the keystream is

Any other non-zero initialization vector will give rise to a cyclic permutation of the same keystream.

Another appealing aspect of this method of keystream generation is that the keystream can be produced efficiently in hardware using a linear feedback shift register, or LFSR. We would use a shift register with m stages. The vector (k1, . . . , km) would be used to initialize the shift register. At each time unit, the following operations would be performed concurrently:

1. k1 would be tapped as the next keystream bit
2. k2, . . . , km would each be shifted one stage to the left
3. the "new" value of km would be computed to be

(this is the "linear feedback").

*
* Figure 1.8 A Linear Feedback Shift Register

*
* Figure 1.9 Autokey Cipher

Observe that the linear feedback is carried out by tapping certain stages of the register (as specified by the constants cj having the value "1") and computing a sum modulo 2 (which is an exclusive-or). This is illustrated in Figure 1.8, where we depict the LFSR that will generate the keystream of Example 1.7.

An example of a non-synchronous stream cipher that is known as the Autokey Cipher is given in Figure 1.9. It is apparently due to Vigenere.

The reason for the terminology "autokey" is that the plaintext is used as the key (aside from the initial "priming key" K). Here is an example to illustrate:


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 1.8

Suppose the key is K = 8, and the plaintext is

                             rendezvous.

We first convert the plaintext to a sequence of integers:

17 4 13 3 4 25 21 14 20 18

The keystream is as follows:

8 17 4 13 3 4 25 21 14 20

Now we add corresponding elements, reducing modulo 26:

25 21 17 16 7 3 20 9 8 12

In alphabetic form, the ciphertext is:

                            ZVRQHDUJIM.

Now let's look at how Alice decrypts the ciphertext. She will first convert the alphabetic string to the numeric string

25 21 17 16 7 3 20 9 8 12

Then she can compute

Next,

and so on. Each time she obtains another plaintext character, she also uses it as the next keystream element.

Of course, the Autokey Cipher is insecure since there are only 26 possible keys.

In the next section, we discuss methods that can be used to cryptanalyze the various cryptosystems we have presented.

1.2 Cryptanalysis

In this section, we discuss some techniques of cryptanalysis. The general assumption that is usually made is that the opponent, Oscar, knows the cryptosystem being used. This is usually referred to as Kerckhoff's principle. Of course, if Oscar does not know the cryptosystem being used, that will make his task more difficult. But we do not want to base the security of a cryptosystem on the (possibly shaky) premise that Oscar does not know what system is being employed. Hence, our goal in designing a cryptosystem will be to obtain security under Kerckhoff's principle.

First, we want to differentiate between different levels of attacks on cryptosystems. The most common types are enumerated as follows.

Ciphertext-only

The opponent possesses a string of ciphertext, y.

Known plaintext

The opponent possesses a string of plaintext, x, and the corresponding ciphertext, y.

Chosen plaintext

The opponent has obtained temporary access to the encryption machinery.
Hence he can choose a plaintext string, x, and construct the corresponding ciphertext string, y.

Chosen ciphertext

The opponent has obtained temporary access to the decryption machinery. Hence he can choose a ciphertext string, y, and construct the corresponding plaintext string, x.

In each case, the object is to determine the key that was used. We note that a chosen ciphertext attack is relevant to public-key cryptosystems, which we discuss in the later chapters.

We first consider the weakest type of attack, namely a ciphertext-only attack. We also assume that the plaintext string is ordinary English text, without punctuation or "spaces." (This makes cryptanalysis more difficult than if punctuation and spaces were encrypted.)

Many techniques of cryptanalysis use statistical properties of the English language. Various people have estimated the relative frequencies of the 26 letters by compiling statistics from numerous novels, magazines, and newspapers. The estimates in Table 1.1 were obtained by Beker and Piper.

On the basis of the above probabilities, Beker and Piper partition the 26 letters into five groups as follows:

1. E, having probability about 0.120
2. T, A, O, I, N, S, H, R, each having probabilities between 0.06 and 0.09
3. D, L, each having probabilities around 0.04
4. C, U, M, W, F, G, Y, P, B, each having probabilities between 0.015 and 0.028
5. V, K, J, X, Q, Z, each having probabilities less than 0.01.

It may also be useful to consider sequences of two or three consecutive letters called digrams and trigrams, respectively. The 30 most common digrams are (in decreasing order) TH, HE, IN, ER, AN, RE, ED, ON, ES, ST, EN, AT, TO, NT, HA, ND, OU, EA, NG, AS, OR, TI, IS, ET, IT, AR, TE, SE, HI, and OF. The twelve most common trigrams are (in decreasing order) THE ING, AND, HER, ERE, ENT, THA, NTH, WAS, ETH, FOR, and DTH.

Table 1.1 Probabilities of Occurrence of the 26 Letters
letter | probability | letter | probability
A | .082 | N | .067
B | .015 | O | .075
C | .028 | P | .019
D | .043 | Q | .001
E | .127 | R | .060
F | .022 | S | .063
G | .020 | T | .091
H | .061 | U | .028
I | .070 | V | .010
J | .002 | W | .023
K | .008 | X | .001
L | .040 | Y | .020
M | .024 | Z | .001


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

1.2.1 Cryptanalysis of the Affine Cipher

As a simple illustration of how cryptanalysis can be performed using statistical data, let's look first at the Affine Cipher. Suppose Oscar has intercepted the following ciphertext:

Example 1.9

Ciphertext obtained from an Affine Cipher

         FMXVEDKAPHFERBNDKRXRSREFMORUDSDKDVSHVUFEDK
         APRKDLYEVLRHHRH

The frequency analysis of this ciphertext is given in Table 1.2.

There are only 57 characters of ciphertext, but this is sufficient to cryptanalyze an Affine Cipher. The most frequent ciphertext characters are: R (8 occurrences), D (7 occurrences), E, H, K (5 occurrences each), and F, S, V (4 occurrences each). As a first guess, we might hypothesize that R is the encryption of e and D is the encryption of t, since e and t are (respectively) the two most common letters. Expressed numerically, we have eK(4) = 17 and eK(19) = 3. Recall that eK(x) = ax + b, where a and b are unknowns. So we get two linear equations in two unknowns:

Table 1.2 Frequency of Occurrence of the 26 Ciphertext Letters
letter | frequency | letter | frequency
A | 2 | N | 1
B | 1 | O | 1
C | 0 | P | 2
D | 7 | Q | 0
E | 5 | R | 8
F | 4 | S | 3
G | 0 | T | 0
H | 5 | U | 2
I | 0 | V | 4
J | 0 | W | 0
K | 5 | X | 2
L | 2 | Y | 1
M | 2 | Z | 0

This system has the unique solution a = 6, b = 19 (in [image]). But this is an illegal key, since gcd(a, 26) = 2 > 1. So our hypothesis must be incorrect.

Our next guess might be that R is the encryption of e and E is the encryption of t. Proceeding as above, we obtain a = 13, which is again illegal. So we try the next possibility, that R is the encryption of e and H is the encryption of t. This yields a = 8, again impossible. Continuing, we suppose that R is the encryption of e and K is the encryption of t. This produces a = 3, b = 5, which is at least a legal key. It remains to compute the decryption function corresponding to K = (3, 5), and then to decrypt the ciphertext to see if we get a meaningful string of English, or nonsense. This will confirm the validity of (3, 5).

If we perform these operations, we have dK(y) = 9y - 19 and the given ciphertext decrypts to yield:

           algorithmsarequitegeneraldefinitionsofarit
           hmeticprocesses

We conclude that we have determined the correct key.

1.2.2 Cryptanalysis of the Substitution Cipher

Here, we look at the more complicated situation, the Substitution Cipher. Consider the following ciphertext:

Table 1.3 Frequency of Occurrence of the 26 Ciphertext Letters
letter | frequency | letter | frequency
A | 0 | N | 9
B | 1 | O | 0
C | 15 | P | 1
D | 13 | Q | 4
E | 7 | R | 10
F | 11 | S | 3
G | 1 | T | 2
H | 4 | U | 5
I | 5 | V | 5
J | 11 | W | 8
K | 1 | X | 6
L | 0 | Y | 10
M | 16 | Z | 20

Example 1.10

Ciphertext obtained from a Substitution Cipher

           YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ
           NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ
           NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ
           XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR

The frequency analysis of this ciphertext is given in Table 1.3.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Since Z occurs significantly more often than any other ciphertext character, we might conjecture that dK(Z) = e. The remaining ciphertext characters that occur at least ten times (each) are C, D, F, J, M, R, Y. We might expect that these letters are encryptions of (a subset of) t, a, o, i, n, s, h, r, but the frequencies really do not vary enough to tell us what the correspondence might be.

At this stage we might look at digrams, especially those of the form -Z or Z-since we conjecture that Z decrypts to e. We find that the most common digrams of this type are DZ and ZW (four times each); NZ and ZU (three times each); and RZ, HZ, YZ, FZ, ZR, ZV, ZC, ZD, and ZJ (twice each). Since ZW occurs four times and WZ not at all, and W occurs less often than many other characters, we might guess that dK(W) = d. Since DZ occurs four times and ZD occurs twice, we would think that DK(D) ∈ {r, s, t}, but it is not clear which of the three possibilities is the correct one.

If we proceed on the assumption that dK(Z) = e and dK(W) = d, we might look back at the ciphertext and notice that we have ZRW and RZW both occurring near the beginning of the ciphertext, and RW occurs again later on. Since R occurs frequently in the ciphertext and nd is a common digram, we might try dK(R) = n as the most likely possibility.

At this point, we have the following:

              ------end---------e----ned---e------------
              YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ

              --------e----e---------n--d---en----e----e
              NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ

              -e---n------n------ed---e---e--ne-nd-e-e--
              NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ

              -ed-----n-----------e----ed-------d---e--n
              XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR

Our next step might be to try dK(N) = h, since NZ is a common digram and ZN is not. If this is correct, then the segment of plaintext ne - ndhe suggests that dK(C) = a. Incorporating these guesses, we have:

              ------end-----a---e-a--nedh--e------a-----
              YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ

              h-------ea---e-a---a---nhad-a-en--a-e-h--e
              NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ

              he-a-n------n------ed---e---e--neandhe-e--
              NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ

              -ed-a---nh---ha---a-e----ed-----a-d--he--n
              XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR

Now, we might consider M, the second most common ciphertext character. The ciphertext segment RNM, which we believe decrypts to nh-, suggests that h- begins a word, so M probably represents a vowel. We have already accounted for a and e, so we expect that dK(M) = i or o. Since ai is a much more likely digram than ao, the ciphertext digram CM suggests that we try dK(M) = i first. Then we have:

              -----iend-----a-i-e-a-inedhi-e------a---i-
              YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ

              h-----i-ea-i-e-a---a-i-nhad-a-en--a-e-hi-e
              NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ

              he-a-n-----in-i----ed---e---e-ineandhe-e--
              NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ

              -ed-a--inhi--hai--a-e-i--ed-----a-d--he--n
              XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR

Next, we might try to determine which letter is encrypted to o. Since o is a common letter, we guess that the corresponding ciphertext letter is one of D, F, J, Y. Y seem to be the most likely possibility, otherwise, we would get long strings of vowels, namely aoi from CFM or CJM. Hence, let's suppose dE(Y) = o.

The three most frequent remaining ciphertext letters are D, F, J, which we conjecture could decrypt to r, s, t in some order. Two occurrences of the trigram NMD suggest that dE(D) = s, giving the trigram his in the plaintext (this is consistent with our earlier hypothesis that dE(D) ∈ {r, s, t}). The segment HNCMF could be an encryption of chair, which would give dE(F) = r (and dE(H) = c) and so we would then have dE(J) = t by process of elimination. Now, we have:

              o-r-riend-ro--arise-a-inedhise--t---ass-it
              YIFQFMZRWQFYVECFMDZPCVMRZWNMDZVEJBTXCDDUMJ

              hs-r-riseasi-e-a-orationhadta-en--ace-hi-e
              NDIFEFMDZCDMQZKCEYFCJMYRNCWJCSZREXCHZUNMXZ

              he-asnt-oo-in-i-o-redso-e-ore-ineandhesett
              NZUCDRJXYYSMRTMEYIFZWDYVZVYFZUMRZCRWNZDZJJ

              -ed-ac-inhischair-aceti-ted--to-ardsthes-n
              XZWGCHSMRNMDHNCMFQCHZJMXJZWIEJYUCFWDJNZDIR

It is now very easy to determine the plaintext and the key for Example 1.10. The complete decryption is the following:

Our friend from Paris examined his empty glass with surprise, as if evaporation had taken place while he wasn't looking. I poured some more wine and he settled back in his chair, face tilted up towards the sun.1

1P. Mayle, A Year in Provence. A. Knopf, Inc., 1989.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

1.2.3 Cryptanalysis of the Vigenere Cipher

In this section we describe some methods for cryptanalyzing the Vigenere Cipher. The first step is to determine the keyword length, which we denote by m. There are a couple of techniques that can be employed. The first of these is the so-called Kasiski test and the second uses the index of coincidence.

The Kasiski test was first described by Friedrich Kasiski in 1863. It is based on the observation that two identical segments of plaintext will be encrypted to the same ciphertext whenever their occurrence in the plaintext is x positions apart, where x ≡ 0 mod m. Conversely, if we observe two identical segments of ciphertext, each of length at least three, say, then there is a good chance that they do correspond to identical segments of plaintext.

The Kasiski test works as follows. We search the ciphertext for pairs of identical segments of length at least three, and record the distance between the starting positions of the two segments. If we obtain several such distances d1, d2, . . . , then we would conjecture that m divides the greatest common divisor of the di′s.

Further evidence for the value of m can be obtained by the index of coincidence. This concept was defined by Wolfe Friedman in 1920, as follows.

DEFINITION 1.7 Suppose x = x1x2 . . . xn is a string of n alphabetic characters. The index of coincidence of x, denoted Ic(x), is defined to be the probability that two random elements of x are identical. Suppose we denote the frequencies of A, B, C, . . . , Z in x by f0, f1, . . . , f25 (respectively). We can choose two elements of x in [image] ways.2 For each i, 0 ≤ i ≤ 25, there are [image] ways of choosing both elements to be i. Hence, we have the formula

2The binomial coefficient [image] = n!/(k!(n - k)!) denotes the number of ways of choosing a subset of k objects from a set of n objects.

Now, suppose x is a string of English language text. Denote the expected probabilities of occurrence of the letters A, B, . . . , Z in Table 1.1 by p0, . . . , p25.

Then, we would expect that

since the probability that two random elements both are A is p02, the probability that both are B is p12, etc. The same reasoning applies if x is a ciphertext obtained by means of any monoalphabetic cipher. In this case, the individual probabilities will be permuted, but the quantity

will be unchanged.

Now, suppose we start with a ciphertext y = y1y2 . . . yn that has been constructed by using a Vigenere Cipher. Define m substrings y1, y2, . . . , y m of y by writing out the ciphertext, by columns, in a rectangular array of dimensions m × (n/m). The rows of this matrix are the substrings y i, 1 ≤ i ≤ m. If this is done, and m is indeed the keyword length, then each Ic(y i) should be roughly equal to 0.065. On the other hand, if m is not the keyword length, then the substrings y i will look much more random, since they will have been obtained by shift encryption with different keys. Observe that a completely random string will have

The two values 0.065 and 0.038 are sufficiently far apart that we will often be able to determine the correct keyword length (or confirm a guess that has already been made using the Kasiski test).

Let us illustrate these two techniques with an example.

Example 1.11

Ciphertext obtained from a Vigenere Cipher

              CHREEVOAHMAERATBIAXXWTNXBEEOPHBSBQMQEQERBW
              RVXUOAKXAOSXXWEAHBWGJMMQMNKGRFVGXWTRZXWIAK
              LXFPSKAUTEMNDCMGTSXMXBTUIADNGMGPSRELXNJELX
              VRVPRTULHDNQWTWDTYGBPHXTFALJHASVBFXNGLLCHR
              ZBWELEKMSJIKNBHWRJGNMGJSGLXFEYPHAGNRBIEQJT
              AMRVLCRREMNDGLXRRIMGNSNRWCHRQHAEYEVTAQEBBI
              PEEWEVKAKOEWADREMXMTBHHCHRTKDNVRZCHRCLQOHP
              WQAIIWXNRMGWOIIFKEE

First, let's try the Kasiski test. The ciphertext string CHR occurs in five places in the ciphertext, beginning at positions 1, 166, 236, 276 and 286. The distances from the first occurrence to the other three occurrences are (respectively) 165, 235, 275 and 285. The gcd of these four integers is 5, so that is very likely the keyword length.

Let's see if computation of indices of coincidence gives the same conclusion. With m = 1, the index of coincidence is 0.045. With m = 2, the two indices are 0.046 and 0.041. With m = 3, we get 0.043, 0.050, 0.047. With m = 4, we have indices 0.042, 0.039, 0.046, 0.040. Then trying m = 5, we obtain the values 0.063, 0.068, 0.069, 0.061 and 0.072. This also provides strong evidence that the keyword length is five.

Proceeding under this assumption, how do we determine the keyword? It is useful to consider the mutual index of coincidence of two strings.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

DEFINITION 1.8 Suppose x = x1x2 . . . xn′ and y = y1y2 . . . yn′ are strings of n and n′ alphabetic characters, respectively. The mutual index of coincidence of x and y, denoted MIc(x, y), is defined to be the probability that a random element of x is identical to a random element of y. If we denote the frequencies of A, B, C, . . . , Z in x and y by f0, f1, f25 and f′0, . . . , f′1, . . . , f′25, respectively, then MIc(x, y) is seen to be

Now, given that we have determined the value of m, the substrings y i are obtained by shift encryption of the plaintext. Suppose K = (k1, k2, . . . , km) is the keyword. Let us see if we can estimate MIc(yi, yj). Consider a random character in yi and a random character in yj. The probability that both characters are A is [image], the probability that both are B is [image], etc. (Note that all subscripts are reduced modulo 26.) Hence, we estimate that

Observe that the value of this estimate depends only on the difference ki - kj mod 26, which we call the relative shift of y i and yj. Also, notice that

so a relative shift of [image] yields the same estimate of MIc as does a relative shift of 26 - [image].

We tabulate these estimates, for relative shifts ranging between 0 to 13, in Table 1.4.

Table 1.4 Expected Mutual Indices of Coincidence
relative shift | expected value of MIc
0 | 0.065
1 | 0.039
2 | 0.032
3 | 0.034
4 | 0.044
5 | 0.033
6 | 0.036
7 | 0.039
8 | 0.034
9 | 0.034
10 | 0.038
11 | 0.045
12 | 0.039
13 | 0.043

The important observation is that, if the relative shift is not zero, these estimates vary between 0.031 and 0.045; whereas, a relative shift of zero yields an estimate of 0.065. We can use this observation to formulate a likely guess for [image], the relative shift of y i and y j, as follows. Suppose we fix y i, and consider the effect of encrypting y j by e0, e1, e2, . . .. Denote the resulting strings by [image], etc. It is easy to compute the indices [image], 0 ≤ g ≤ 25. This can be done using the formula

When g = [image], the MIc should be close to 0.065, since the relative shift of yi and [image] is zero. However, for values of g ≠ [image], the MIc should vary between 0.031 and 0.045.

By using this technique, we can obtain the relative shifts of any two of the substrings yi. This leaves only 26 possible keywords, which can easily be obtained by exhaustive key search, for example.

Let us illustrate by returning to Example 1.11.

Example 1.11 (Cont.)

We have hypothesized that the keyword length is 5. We now try to compute the relative shifts. By computer, it is not difficult to compute the 260 values [image], where 1 ≤ i ≤ j ≤ 5, 0 ≤ g 25. These values are tabulated in Table 1.5. For each (i, j) pair, we look for values [image] that are close to 0.065. If there is a unique such value (for a given (i, j) pair), we conjecture that

Table 1.5 Observed Mutual Indices of Coincidence

it is the value of the relative shift.

Six such values in Table 1.5 are boxed. They provide strong evidence that the relative shift of y1 and y2 is 9; the relative shift of y1 and y5 is 16; the relative shift of y2 and y3 is 13; the relative shift of y2 and y5 is 7; the relative shift of y3 and y5 is 20; and the relative shift of y4 and y5 is 11. This gives us the following equations in the five unknowns k1, k2, k3, k4, k5:


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

This allows us to express the five ki′s in terms of k1:

So the key is likely to be (k1, k1 + 17, k1 + 4, k1 + 21, k1 + 10) for some [image] Hence, we suspect that the keyword is some cyclic shift of AREVK. It now does not take long to determine that the keyword is JANET. The complete decryption is the following:

The almond tree was in tentative blossom. The days were longer, often ending with magnificent evenings of corrugated pink skies. The hunting season was over, with hounds and guns put away for six months. The vineyards were busy again as the well-organized farmers treated their vines and the more lackadaisical neighbors hurried to do the pruning they should have done in November.3

3P. Mayle, A Year in Provence, A. Knopf, Inc., 1989.

1.2.4 A Known Plaintext Attack on the Hill Cipher

The Hill Cipher is more difficult to break with a ciphertext-only attack, but it succumbs easily to a known plaintext attack. Let us first assume that the opponent has determined the value of m being used. Suppose he has at least m distinct pairs of m-tuples, xj = (x1,j, x2,j, . . . , xm,j) and yj = (y1,j, y2,j, . . . , ym,j) (1 ≤ j ≤ m), such that yj = eK(xj), 1 ≤ j ≤ m. If we define two m × m matrices X = (xi,j) and Y = (yi,j), then we have the matrix equation Y = XK, where the m × m matrix K is the unknown key. Provided that the matrix X is invertible, Oscar can compute K = X-1Y and thereby break the system. (If Y is not invertible, then it will be necessary to try other sets of m plaintext-ciphertext pairs.)

Let's look at a simple example.

Example 1.12

Suppose the plaintext friday is encrypted using a Hill Cipher with m = 2, to give the ciphertext PQCFKU.

We have that eK (5, 17) = (15, 16), eK (8, 3) = (2, 5) and eK (0, 24) = (10, 20). From the first two plaintext-ciphertext pairs, we get the matrix equation

Using Theorem 1.3, it is easy to compute

so

This can be verified by using the third plaintext-ciphertext pair.

What would the opponent do if he does not know m? Assuming that m is not too big, he could simply try m = 2, 3, . . . , until the key is found. If a guessed value of m is incorrect, then an m × m matrix found by using the algorithm described above will not agree with further plaintext-ciphertext pairs. In this way, the value of m can be determined if it is not already known.

1.2.5 Cryptanalysis of the LFSR-based Stream Cipher

Recall that the ciphertext is the sum modulo 2 of the plaintext and the keystream, i.e., yi = xi + zi mod 2. The keystream is produced from z1, . . . , zm using the linear recurrence relation

where [image] (and c0 = 1).

Since all operations in this cryptosystem are linear, we might suspect that the cryptosystem is vulnerable to a known-plaintext attack, as is the case with the Hill Cipher. Suppose Oscar has a plaintext string x1x2 . . . xn and the corresponding ciphertext string y1y2 . . . yn. Then he can compute the keystream bits zi = xi + yi mod 2, 1 ≤ i ≤ n. Let us also suppose that Oscar knows the value m. Then Oscar needs only to compute c0, . . . , cm-1 in order to be able to reconstruct the entire keystream. In other words, he needs to be able to determine the values of m unknowns.

Now, for any i ≥ 1, we have

which is a linear equation in the m unknowns. If n ≥ 2m, then there are m linear equations in m unknowns, which can subsequently be solved.

The system of m linear equations can be written in matrix form as follows:

If the coefficient matrix has an inverse (modulo 2), we obtain the solution

In fact, the matrix will have an inverse if m is the degree of the recurrence used to generate the keystream (see the exercises for a proof). Let's illustrate with an example.

Example 1.13

Suppose Oscar obtains the ciphertext string

101101011110010

corresponding to the plaintext string

011001111111000.

Then he can compute the keystream bits:

110100100001010.

Suppose also that Oscar knows that the keystream was generated using a 5-stage LFSR. Then he would solve the following matrix equation, which is obtained from the first 10 keystream bits:

It can be checked that

This yields

= (1, 0, 1, 1, 0).

Thus the recurrence used to generate the keystream is


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

1.3 Notes

Much of the material on classical cryptography is covered in textbooks, for example Beker and Piper [BP82] and Denning [DE82]. The probability estimates for the 26 alphabetic characters are taken from Beker and Piper. As well, the cryptanalysis of the Vigenere Cipher is a modification of the description given in Beker and Piper.

A good reference for elementary number theory is Rosen [Ro93]. Background in elementary linear algebra can be found in Anton [AN91].

Kahn's book "The Codebreakers" [KA67] is an entertaining and informative history of cryptography up to 1967. In it, Kahn states that the Vigenere Cipher is incorrectly attributed to Vigenere.

The Hill Cipher was first described in [HI29]. Much information on stream ciphers can be found in the book by Rueppel [RU86].

Exercises

1.1 Below are given four examples of ciphertext, one obtained from a Substitution Cipher, one from a Vigenere Cipher, one from an Affine Cipher, and one unspecified. In each case, the task is to determine the plaintext.

Give a clearly written description of the steps you followed to decrypt each ciphertext. This should include all statistical analysis and computations you performed.

The first two plaintexts were taken from "The Diary of Samuel Marchbanks," by Robertson Davies, Clarke Irwin, 1947; the fourth was taken from "Lake Wobegon Days," by Garrison Keillor, Viking Penguin, Inc., 1985.

(a) Substitution Cipher:
      EMGLOSUDCGDNCUSWYSFHNSFCYKDPUMLWGYICOXYSIPJCK
      QPKUGKMGOLICGINCGACKSNISACYKZSCKXECJCKSHYSXCG
      OIDPKZCNKSHICGIWYGKKGKGOLDSILKGOIUSIGLEDSPWZU
      GFZCCNDGYYSFUSZCNXEOJNCGYEOWEUPXEZGACGNFGLKNS
      ACIGOIYCKXCJUCIUZCFZCCNDGYYSFEUEKUZCSOCFZCCNC
      IACZEJNCSHFZEJZEGMXCYHCJUMGKUCY

HINT F decrypts to w.

(b) Vigenere Cipher:
      KCCPKBGUFDPHQTYAVINRRTMVGRKDNBVFDETDGILTXRGUD
      DKOTFMBPVGEGLTGCKQRACQCWDNAWCRXIZAKFTLEWRPTYC
      QKYVXCHKFTPONCQQRHJVAJUWETMCMSPKQDYHJVDAHCTRL
      SVSKCGCZQQDZXGSFRLSWCWSJTBHAFSIASPRJAHKJRJUMV
      GKMITZHFPDISPZLVLGWTFPLKKEBDPGCEBSHCTJRWXBAFS
      PEZQNRWXCVYCGAONWDDKACKAWBBIKFTIOVKCGGHJVLNHI
      FFSQESVYCLACNVRWBBIREPBBVFEXOSCDYGZWPFDTKFQIY
      CWHJVLNHIQIBTKHJVNPIST
(c) Affine Cipher:
      KQEREJEBCPPCJCRKIEACUZBKRVPKRBCIBQCARBJCVFCUP
      KRIOFKPACUZQEPBKRXPEIIEABDKPBCPFCDCCAFIEABDKP
      BCPFEQPKAZBKRHAIBKAPCCIBURCCDKDCCJCIDFUIXPAFF
      ERBICZDFKABICBBENEFCUPJCVKABPCYDCCDPKBCOCPERK
      IVKSCPICBRKIJPKABI
(d) unspecified cipher:
      BNVSNSIHQCEELSSKKYERIFJKXUMBGYKAMQLJTYAVFBKVT
      DVBPVVRJYYLAOKYMPQSCGDLFSRLLPROYGESEBUUALRWXM
      MASAZLGLEDFJBZAVVPXWICGJXASCBYEHOSNMULKCEAHTQ
      OKMFLEBKFXLRRFDTZXCIWBJSICBGAWDVYDHAVFJXZIBKC
      GJIWEAHTTOEWTUHKRQVVRGZBXYIREMMASCSPBNLHJMBLR
      FFJELHWEYLWISTFVVYFJCMHYUYRUFSFMGESIGRLWALSWM
      NUHSIMYYITCCQPZSICEHBCCMZFEGVJYOCDEMMPGHVAAUM
      ELCMOEHVLTIPSUYILVGFLMVWDVYDBTHFRAYISYSGKVSUU
      HYHGGCKTMBLRX

1.2

(a) How many 2 × 2 matrices are there that are invertible over [image]?
(b) Let p be prime. Show that the number of 2 × 2 matrices that are invertible over [image] is (p2 - 1)(p2 - p).

HINT Since p is prime, [image] is a field. Use the fact that a matrix over a field is invertible if and only if its rows are linearly independent vectors (i.e., there does not exist a non-zero linear combination of the rows whose sum is the vector of all 0′s).

(c) For p prime, and m ≥ 2 an integer, find a formula for the number of m × m matrices that are invertible over [image].

1.3 Sometimes it is useful to choose a key such that the encryption operation is identical to the decryption operation. In the case of the Hill Cipher, we would be looking for matrices K such that K = K-1 (such a matrix is called involutory). In fact, Hill recommended the use of involutory matrices as keys in his cipher. Determine the number of involutory matrices (over [image]) in the case m = 2.

HINT Use the formula given in Theorem 1.3 and observe that det A = ±1 for an involutory matrix over [image].

1.4 Suppose we are told that the plaintext
                            breathtaking

yields the ciphertext

                            RUPOTENTOSUP

where the Hill Cipher is used (but m is not specified). Determine the encryption matrix.

1.5 An Affine-Hill Cipher is the following modification of a Hill Cipher: Let m be a positive integer, and define [image]. In this cryptosystem, a key K consists of a pair (L, b), where L is an m × m invertible matrix over [image], and [image]. For x = (x1, . . . , xm) [image] and K = (L, b) [image], we compute y = eK(x) = (y1, . . . , ym) by means of the formula y = xL + b. Hence, if [image] and b = (b1, . . . , bm), then

Suppose Oscar has learned that the plaintext

                            adisplayedequation

is encrypted to give the ciphertext

                            DSRMSSIOPLXLJBZULLM

and Oscar also knows that m = 3. Compute the key, showing all computations.

1.6 Here is how we might cryptanalyze the Hill Cipher using a ciphertext-only attack. Suppose that we know that m = 2. Break the ciphertext into blocks of length two letters (digrams). Each such digram is the encryption of a plaintext digram using the unknown encryption matrix. Pick out the most frequent ciphertext digram and assume it is the encryption of a common digram in the list following Table 1.1 (for example, TH or ST). For each such guess, proceed as in the known-plaintext attack, until the correct encryption matrix is found.

Here is a sample of ciphertext for you to decrypt using this method:

        LMQETXYEAGTXCTUIEWNCTXLZEWUAISPZYVAPEWLMGQWYA
        XFTCJMSQCADAGTXLMDXNXSNPJQSYVAPRIQSMHNOCVAXFV

1.7 We describe a special case of a Permutation Cipher. Let m, n be positive integers. Write out the plaintext, by rows, in m × n rectangles. Then form the ciphertext by taking the columns of these rectangles. For example, if m = 4, n = 3, then we would encrypt the plaintext "cryptography" by forming the following rectangle:
                               cryp
                               togr
                               aphy

The ciphertext would be "CTAROPYGHPRY."

(a) Describe how Bob would decrypt a ciphertext (given values for m and n).
(b) Decrypt the following ciphertext, which was obtained by using this method of encryption:
       MYAMRARUYIQTENCTORAHROYWDSOYEOUARRGDERNOGW

1.8 There are eight different linear recurrences over [image] of degree four having c0 = 1. Determine which of these recurrences give rise to a keystream of period 15 (given a non-zero initialization vector).
1.9 The purpose of this exercise is to prove the statement made in Section 1.2.5 that the m × m coefficient matrix is invertible. This is equivalent to saying that the rows of this matrix are linearly independent vectors over [image].

As before, we suppose that the recurrence has the form

(z1, . . . , zm) comprises the initialization vector. For i ≥ 1, define

Note that the coefficient matrix has the vectors v1, . . . , vm as its rows, so our objective is to prove that these m vectors are linearly independent.

Prove the following assertions:

(a) For any i ≥ 1,

(b) Choose h to be the minimum integer such that there exists a non-trivial linear combination of the vectors v1, . . . , vh which sums to the vector (0, . . . , 0) modulo 2. Then

and not all the αj′s are zero. Observe that h ≤ m + 1, since any m + 1 vectors in an m-dimensional vector space are dependent.

(c) Prove that the keystream must satisfy the recurrence

for any i ≥ 1.

(d) Observe that if h ≤ m, then the keystream satisfies a linear recurrence of degree less than m, a contradiction. Hence, h = m + 1, and the matrix must be invertible.

1.10 Decrypt the following ciphertext, obtained from the Autokey Cipher, by using exhaustive key search:
                       MALVVMAFBHBUQPTSOXALTGVWWRG
1.11 We describe a stream cipher that is a modification of the Vigenere Cipher. Given a keyword (K1, . . . , Km) of length m, construct a keystream by the rule zi = Ki (1 ≤ i ≤ m), zi + m = zi + 1 mod 26 (i ≥ m + 1). In other words, each time we use the keyword, we replace each letter by its successor modulo 26. For example, if SUMMER is the keyword, we use SUMMER to encrypt the first six letters, we use TVNNFS for the next six letters, and so on.

Describe how you can use the concept of index of coincidence to first determine the length of the keyword, and then actually find the keyword.

Test your method by cryptanalyzing the following ciphertext:

          IYMYSILONRFNCQXQJEDSHBUIBCJUZBOLFQYSCHATPEQGQ
          JEJNGNXZWHHGWFSUKULJQACZKKJOAAHGKEMTAFGMKVRDO
          PXNEHEKZNKFSKIFRQVHHOVXINPHMRTJPYWQGJWPUUVKFP
          OAWPMRKKQZWLQDYAZDRMLPBJKJOBWIWPSEPVVQMBCRYVC
          RUZAAOUMBCHDAGDIEMSZFZHALIGKEMJJFPCIWKRMLMPIN
          AYOFIREAOLDTHITDVRMSE

The plaintext was taken from "The Codebreakers," by D. Kahn, Macmillan, 1967.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 2
Shannon's Theory

In 1949, Claude Shannon published a paper entitled "Communication Theory of Secrecy Systems" in the Bell Systems Technical Journal. This paper had a great influence on the scientific study of cryptography. In this chapter, we discuss several of Shannon's ideas.

2.1 Perfect Secrecy

There are two basic approaches to discussing the security of a cryptosystem.

computational security

This measure concerns the computational effort required to break a cryptosystem. We might define a cryptosystem to be computationally secure if the best algorithm for breaking it requires at least N operations, where N is some specified, very large number. The problem is that no known practical cryptosystem can be proved to be secure under this definition. In practice, people will call a cryptosystem "computationally secure" if the best known method of breaking the system requires an unreasonably large amount of computer time (but this is of course very different from a proof of security). Another approach is to provide evidence of computational security by reducing the security of the cryptosystem to some well-studied problem that is thought to be difficult. For example, it may be able to prove a statement of the type "a given cryptosystem is secure if a given integer n cannot be factored." Cryptosystems of this type are sometimes termed "provably secure," but it must be understood that this approach only provides a proof of security relative to some other problem, not an absolute proof of security.1
1This is a similar situation to proving that a problem is NP-complete: it proves that the given problem is at least as difficult as any other NP-complete problem, but it does not provide an absolute proof of the computational difficulty of the problem.

unconditional security

This measure concerns the security of cryptosystems when there is no bound placed on the amount of computation that Oscar is allowed to do. A cryptosystem is defined to be unconditionally secure if it cannot be broken, even with infinite computational resources.

When we discuss the security of a cryptosystem, we should also specify the type of attack that is being considered. In Chapter 1, we saw that neither the Shift Cipher, the Substitution Cipher nor the Vigenere Cipher is computationally secure against a ciphertext-only attack (given a sufficient amount of ciphertext).

What we will do in this section is to develop the theory of cryptosystems that are unconditionally secure against a ciphertext-only attack. It turns out that all three of the above ciphers are unconditionally secure if only one element of plaintext is encrypted with a given key!

The unconditional security of a cryptosystem obviously cannot be studied from the point of view of computational complexity, since we allow computation time to be infinite. The appropriate framework in which to study unconditional security is probability theory. We need only elementary facts concerning probability; the main definitions are reviewed now.

DEFINITION 2.1 Suppose X and Y are random variables. We denote the probability that X takes on the value x by p(x), and the probability that Y takes on the value y by p(y). The joint probability p(x, y) is the probability that X takes on the value x and Y takes on the value y. The conditional probability p(x|y) denotes the probability that X takes on the value x given that Y takes on the value y. The random variables X and Y are said to be independent if p(x, y) = p(x)p(y) for all possible values x of X and y of Y.

Joint probability can be related to conditional probability by the formula

Interchanging x and y, we have that

From these two expressions, we immediately obtain the following result, which is known as Bayes' Theorem.

THEOREM 2.1 (Bayes' Theorem)

If p(y) > 0, then

COROLLARY 2.2

X and Y are independent variables if and only if p(x|y) = p(x) for all x, y.

Throughout this section, we assume that a particular key is used for only one encrypion. Let us suppose that there is a probability distribution on the plaintext space, [image]. We denote the a priori probability that plaintext x occurs by [image]. We also assume that the key K is chosen (by Alice and Bob) using some fixed probability distribution (often a key is chosen at random, so all keys will be equiprobable, but this need not be the case). Denote the probability that key K is chosen by [image]. Recall that the key is chosen before Alice knows what the plaintext will be. Hence, we make the reasonable assumption that the key K and the plaintext x are independent events.

The two probability distributions on [image] and [image] induce a probability distribution on [image]. Indeed, it is not hard to compute the probability [image] that y is the ciphertext that is transmitted. For a key [image], define

That is, C(K) represents the set of possible ciphertexts if K is the key. Then, for every [image], we have that

We also observe that, for any [image] and [image], we can compute the conditional probability, [image] (i.e., the probability that y is the ciphertext, given that x is the plaintext) to be

It is now possible to compute the conditional probability [image] (i.e., the probability that x is the plaintext, given that y is the ciphertext) using Bayes' Theorem. The following formula is obtained:

Observe that all these calculations can be performed by anyone who knows the probability distributions.

We present a toy example to illustrate the computation of these probability distributions.

Example 2.1

Let [image] with [image]. Let [image] with [image]. Let [image], and suppose the encryption functions are defined to be [image]; [image]; and [image]. This cryptosystem can be represented by the following encryption matrix:

We now compute the probability distribution [image]. We obtain

Now we can compute the conditional probability distributions on the plaintext, given that a certain ciphertext has been observed. We have:

We are now ready to define the concept of perfect secrecy. Informally, perfect secrecy means that Oscar can obtain no information about plaintext by observing the ciphertext. This idea is made precise by formulating it in terms of the probability distributions we have defined, as follows.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

DEFINITION 2.2 A cryptosystem has perfect secrecy if [image] for all [image]. That is, the a posteriori probability that the plaintext is x, given that the ciphertext y is observed, is identical to the a priori probability that the plaintext is x.

In Example 2.1, the perfect secrecy property is satisfied for the ciphertext 3, but not for the other three ciphertexts.

We next prove that the Shift Cipher provides perfect secrecy. This seems quite obvious intuitively. For, if we are given any ciphertext element [image], then any plaintext element [image] is a possible decryption of y, depending on the value of the key. The following theorem gives the formal statement and proof using probability distributions.

THEOREM 2.3

Suppose the 26 keys in the Shift Cipher are used with equal probability 1/26. Then for any plaintext probability distribution, the Shift Cipher has perfect secrecy.

PROOF Recall that [image], and for 0 ≤ K ≤ 25, the encryption rule eK is eK (x) = x + K mod 26 [image]. First, we compute the distribution [image]. Let [image]; then

Now, for fixed y, the values y - K mod 26 comprise a permutation of [image], and [image] is a probability distribution. Hence we have that

Consequently,

for any [image].

Next, we have that

for every x, y, since for every x, y the unique key K such that eK(x) = y is K = y - x mod 26. Now, using Bayes' Theorem, it is trivial to compute

so we have perfect secrecy.

So, the Shift Cipher is "unbreakable" provided that a new random key is used to encrypt every plaintext character.

Let us next investigate perfect secrecy in general. First, we observe that, using Bayes' Theorem, the condition that [image] for all [image] is equivalent to [image] for all [image]. Now, let us make the reasonable assumption that [image] for all [image], then ciphertext y is never used and can be omitted from [image]). Fix any [image]. For each [image], we have [image]. Hence, for each [image], there must be at least one key K such that eK(x) = y. It follows that [image]. In any cryptosystem, we must have [image] since each encoding rule is an injection. In the boundary case [image], we can give a nice characterization of when perfect secrecy can be obtained. This characterization is originally due to Shannon.

THEOREM 2.4

Suppose [image] is a cryptosystem where [image]. Then the cryptosystem provides perfect secrecy if and only if every key is used with equal probability [image], every [image], and every [image], there is a unique key K such that eK(x) = y.

PROOF Suppose the given cryptosystem provides perfect secrecy. As observed above, for each [image] and [image] there must be at least one key K such that eK(x) = y. So we have the inequalities:

But we are assuming that [image]. Hence, it must be the case that

That is, there do not exist two distinct keys K1 and K2 such that [image]. Hence, we have shown that for any [image] and [image], there is exactly one key K such that eK(x) = y.

*
* Figure 2.1 One-time Pad

Denote [image]. Let [image] and fix a [image]. We can name the keys K1, K2,..., Kn, in such a way that [image]. Using Bayes' theorem, we have

Consider the perfect secrecy condition [image]. From this, it follows that [image], for 1 ≤ i ≤ n. This says that the keys are used with equal probability (namely, [image]). But since the number of keys is [image], we must have that [image] for every [image].

Conversely, suppose the two hypothesized conditions are satisfied. Then the cryptosystem is easily seen to provide perfect secrecy for any plaintext probability distribution, in a similar manner as the proof of Theorem 2.3. We leave the details for the reader.

One well-known realization of perfect secrecy is the Vernam One-time Pad, which was first described by Gilbert Vernam in 1917 for use in automatic encryption and decryption of telegraph messages. It is interesting that the One-time Pad was thought for many years to be an "unbreakable" cryptosystem, but there was no proof of this until Shannon developed the concept of perfect secrecy over 30 years later.

The description of the One-time Pad is given in Figure 2.1.

Using Theorem 2.4, it is easily seen that the One-time Pad provides perfect secrecy. The system is also attractive because of the ease of encryption and decryption.

Vernam patented his idea in the hope that it would have widespread commercial use. Unfortunately, there are major disadvantages to unconditionally secure cryptosystems such as the One-time Pad. The fact that [image] means that the amount of key that must be communicated securely is at least as big as the amount of plaintext. For example, in the case of the One-time Pad, we require n bits of key to encrypt n bits of plaintext. This would not be a major problem if the same key could be used to encrypt different messages; however, the security of unconditionally secure cryptosystems depends on the fact that each key is used for only one encryption. (This is the reason for the term "one-time" in the One-time Pad.)

For example, the One-time Pad is vulnerable to a known-plaintext attack, since K can be computed as the exclusive-or of the bitstrings x and eK(x). Hence, a new key needs to be generated and communicated over a secure channel for every message that is going to be sent. This creates severe key management problems, which has limited the use of the One-time Pad in commercial applications. However, the One-time Pad has seen application in military and diplomatic contexts, where unconditional security may be of great importance.

The historical development of cryptography has been to try to design cryptosystems where one key can be used to encrypt a relatively long string of plaintext (i.e., one key can be used to encrypt many messages) and still maintain (at least) computational security. One such system is the Data Encryption Standard, which we will study in Chapter 3.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

2.2 Entropy

In the previous section, we discussed the concept of perfect secrecy. We restricted our attention to the special situation where a key is used for only one encryption. We now want to look at what happens as more and more plaintexts are encrypted using the same key, and how likely a cryptanalyst will be able to carry out a successful ciphertext-only attack, given sufficient time.

The basic tool in studying this question is the idea of entropy, a concept from information theory introduced by Shannon in 1948. Entropy can be thought of as a mathematical measure of information or uncertainty, and is computed as a function of a probability distribution.

Suppose we have a random variable X which takes on a finite set of values according to a probability distribution p(X). What is the information gained by an event which takes place according to distribution p(X)? Equivalently, if the event has not (yet) taken place, what is the uncertainty about the outcome? This quantity is called the entropy of X and is denoted by H(X).

These ideas may seem rather abstract, so let's look at a more concrete example. Suppose our random variable X represents the toss of a coin. The probability distribution is p(heads) = p(tails) = 1/2. It would seem reasonable to say that the information, or entropy, of a coin toss is one bit, since we could encode heads by 1 and tails by 0, for example. In a similar fashion, the entropy of n independent coin tosses is n, since the n coin tosses can be encoded by a bit string of length n.

As a slightly more complicated example, suppose we have a random variable X that takes on three possible values x1, x2, x3 with probabilities 1/2, 1/4, 1/4 respectively. The most efficient "encoding" of the three possible outcomes is to encode x1, as 0, to encode x2 as 10 and to encode x3 as 11. Then the average number of bits in an encoding of X is

The above examples suggest that an event which occurs with probability 2-n can be encoded as a bit string of length n. More generally, we could imagine that an event occurring with probability p might be encoded by a bit string of length approximately - log2 p. Given an arbitrary probability distribution p1, p2, ..., pn for a random variable X, we take the weighted average of the quatities - log2 pi to be our measure of information. This motivates the following formal definition.

DEFINITION 2.3 Suppose X is a random variable which takes on a finite set of values according to a probability distribution p(X). Then, the entropy of this probability distribution is defined to be the quantity

If the possible values of X are xi, 1 ≤ i ≤ n, then we have

REMARK Observe that log2 pi is undefined if pi = 0. Hence, entropy is sometimes defined to be the relevant sum over all the non-zero probabilities. Since limx →0 x log2 x = 0, there is no real difficulty with allowing pi = 0 for some i. However, we will implicitly assume that, when computing the entropy of a probability distribution pi, the sum is taken over the indices i such that pi ≠ 0. Also, we note that the choice of two as the base of the logarithms is arbitrary: another base would only change the value of the entropy by a constant factor.

Note that if pi = 1/n for 1 ≤ i ≤ n, then H(X) = log2 n. Also, it is easy to see that H(X) ≥ 0, and H(X) = 0 if and only if pi = 1 for some i and pj = 0 for all j ≠ i.

Let us look at the entropy of the various components of a cryptosystem. We can think of the key as being a random variable K that takes on values according to the probability distribution [image], and hence we can compute the entropy H(K). Similarly, we can compute entropies H(P) and H(C) associated with plaintext and ciphertext probability distributions, respectively.

To illustrate, we compute the entropies of the cryptosystem of Example 2.1.

Example 2.1 (Cont.)

We compute as follows:

Similar calculations yield H(K) = 1.5 and [image].


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

2.2.1 Huffman Encodings and Entropy

In this section, we discuss briefly the connection between entropy and Huffman encodings. As the results in this section are not relevant to the cryptographic applications of entropy, it may be skipped without loss of continuity. However, this discussion may serve to further motivate the concept of entropy.

We introduced entropy in the context of encodings of random events which occur according to a specified probability distribution. We first make these ideas more precise. As before, X is a random variable which takes on a finite set of values, and p(X) is the associated probability distribution.

An encoding of X is any mapping

where {0, 1}* denotes the set of all finite strings of 0's and 1's. Given a finite list (or string) of events x1 ... xn, we can extend the encoding f in an obvious way by defining

where || denotes concatenation. In this way, we can think of f as a mapping

Now, suppose a string x1 ... xn is produced by a memoryless source such that each xi occurs according to the probability distribution on X. This means that the probability of any string x1 ... xn is computed to be p(x1) × . . . × p(xn). (Notice that this string need not consist of distinct values, since the source is memoryless. As a simple example, consider a sequence of n tosses of a fair coin.)

Now, given that we are going to encode strings using the mapping f, it is important that we are able to decode in an unambiguous fashion. Thus it should be the case that the encoding f is injective.

Example 2.2

Suppose X = {a, b, c, d}, and consider the following three encodings:

It can be seen that f and g are injective encodings, but h is not. Any encoding using f can be decoded by starting at the end and working backwards: every time a 1 is encountered, it signals the end of the current element.

An encoding using g can be decoded by starting at the beginning and proceeding sequentially. At any point where we have a substring that is an encoding of a, b, c, or d, we decode it and chop off the substring. For example, given the string 10101110, we decode 10 to b, then 10 to b, then 111 to d, and finally 0 to a. So the decoded string is bbda.

To see that h is not injective, it suffices to give an example:

From the point of view of ease of decoding, we would prefer the encoding g to f. This is because decoding can be done sequentially from beginning to end if g is used, so no memory is required. The property that allows the simple sequential decoding of g is called the prefix-free property. (An encoding g is prefix-free if there do not exist two elements x, y ∈ X, and a string z ∈ {0, 1}* such that g(x) = g(y) || z.)

The discussion this point has not involved entropy. Not surprisingly, entropy is related to the efficiency of an encoding. We will measure the efficiency of an encoding f as we did before: it is the weighted average length (denoted by [image]) of an encoding of an element of X. So we have the following definition:

where |y| denotes the length of a string y.

Now, our fundamental problem is to find an injective encoding, f, that minimizes [image]. There is a well-known algorithm, known as Huffman's algorithm, that accomplishes this goal. Moreover, the encoding f produced by Huffman's algorithm is prefix-free, and

Thus, the value of the entropy provides a close estimate to the average length of the optimal injective encoding.

We will not prove the results stated above, but we will give a short, informal description of Huffman's algorithm. Huffman's algorithm begins with the probability distribution on the set X, and the code of each element is initially empty. In each iteration, the two elements having lowest probability are combined into one element having as its probability the sum of the two smaller probabilities. The smaller of the two elements is assigned the value "0" and the larger of the two elements is assigned the value "1." When only one element remains, the coding for each x ∈ X can be constructed by following the sequence of elements "backwards" from the final element to the initial element x.

This is easily illustrated with an example.

Example 2.3

Suppose X = {a, b, c, d, e} has the following probability distribution: p(a) = .05, p(b) = .10, p(c) = .12, p(d) = .13 and p(e) = .60. Huffman's algorithm would proceed as indicated in the following table:

This leads to the following encodings:

x | f(x)
a | 000
b | 001
c | 010
d | 011
e | 1

Thus, the average length encoding is

Compare this to the entropy:


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

2.3 Properties of Entropy

In this section, we prove some fundamental results concerning entropy. First, we state a fundamental result, known as Jensen's Inequality, that will be very useful to us. Jensen's Inequality involves concave functions, which we now define.

DEFINITION 2.4 A real-valued function f is concave on an interval I if

for all x, y ∈ I. f is strictly concave on an interval I if

for all x, y ∈ I, x ≠ y.

Here is Jensen's Inequality, which we state without proof.

THEOREM 2.5 (Jensen's Inequality)

Suppose f is a continuous strictly concave function on the interval I,

and ai > 0, 1 ≤ i ≤ n. Then

where xi ∈ I, 1 ≤ i ≤ n. Further, equality occurs if and only if x1 = ... = xn.

We now proceed to derive several results on entropy. In the next theorem, we make use of the fact that the function log2 x is strictly concave on the interval (0, ∞). (In fact, this follows easily from elementary calculus since the second deriviative of the logarithm function is negative on the interval (0, ∞).)

THEOREM 2.6

Suppose X is a random variable having probability distribution p1, p2, pn, where pi > 0, 1 ≤ i ≤ n. Then H(X) ≤ log2 n, with equality if and only if pi = 1/n, 1 ≤ i ≤ n.

PROOF Applying Jensen's Inequality, we have the following:

Further, equality occurs if and only if pi = 1/n, 1 ≤ i ≤ n.

THEOREM 2.7

H (X, Y) ≤ H(X) + H(Y), with equality if and only if X and Y are independent events.

PROOF Suppose X takes on values xi, 1 ≤ i ≤ m, and Y takes on values yj, 1 ≤ j ≤ n. Denote pi = p(X = xi), 1 ≤ i ≤ m, and qj = p(Y = yj), 1 ≤ j ≤ n. Denote rij = p(X = xi, Y = yj), 1 ≤ i ≤ m, 1 ≤ j ≤ n (this is the joint probability distribution).

Observe that

(1 ≤ i ≤ m) and

(1 ≤ j ≤ n). We compute as follows:

On the other hand,

Combining, we obtain the following:

(Here, we apply Jensen's Inequality, using the fact that the rij's form a probability distribution.)

We can also say when equality occurs: it must be the case that there is a constant c such that piqj/rij = c for all i, j. Using the fact that

it follows that c = 1. Hence, equality occurs if and only if rij = piqj, i.e., if and only if

1 ≤ i ≤ m, 1 ≤ j ≤ n. But this says that X and Y are independent.

We next define the idea of conditional entropy.

DEFINITION 2.5 Suppose X and Y are two random variables. Then for any fixed value y of Y, we get a (conditional) probability distribution p(X|y). Clearly,

We define the conditional entropy H(X|Y) to be the weighted average (with respect to the probabilities p(y)) of the entropies H(X|y) over all possible values y. It is computed to be

The conditional entropy measures the average amount of information about X that is revealed by Y.

The next two results are straightforward; we leave the proofs as exercises.

THEOREM 2.8

H(X, Y) = H(Y) + H(X|Y).

COROLLARY 2.9

H(X|Y) ≤ H(X), with equality if and only if X and Y are independent.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

2.4 Spurious Keys and Unicity Distance

In this section, we apply the entropy results we have proved to cryptosystems. First, we show a fundamental relationship exists among the entropies of the components of a cryptosystem. The conditional entropy H(K|C) is called the key equivocation, and is a measure of how much information about the key is revealed by the ciphertext.

THEOREM 2.10

Let [image] be a cryptosystem. Then

PROOF First, observe that H(K, P, C) = H(C|K, P) + H(K, P). Now, the key and plaintext determine the ciphertext uniquely, since y = eK(x). This implies that H(C|K, P) = 0. Hence, H(K, P, C) = H(K, P). But K and P are independent, so H(K, P) = H(K) + H(P). Hence,

In a similar fashion, since the key and ciphertext determine the plaintext uniquely (i.e., x = dK(y)), we have that H(P|K, C) = 0 and hence H(K, P, C) = H(K, C).

Now, we compute as follows:

giving the desired formula.

Let us return to Example 2.1 to illustrate this result.

Example 2.1 (Cont.)

We have already computed [image] and [image]. Theorem 2.10 tells us that [image]. This can be verified directly by applying the definition of conditional entropy, as follows. First, we need to compute the probabilities p(Ki|j), 1 ≤ i ≤ 3, 1 ≤ j ≤ 4. This can be done using Bayes' Theorem, and the following values result:

Now we compute

agreeing with the value predicted by Theorem 2.10.

Suppose [image] the cryptosystem being used, and a string of plaintext

is encrypted with one key, producing a string of ciphertext

Recall that the basic goal of the cryptanalyst is to determine the key. We are looking at ciphertext-only attacks, and we assume that Oscar has infinite computational resources. We also assume that Oscar knows that the plaintext is a "natural" language, such as English. In general, Oscar will be able to rule out certain keys, but many "possible" keys may remain, only one of which is the correct key. The remaining possible, but incorrect, keys are called spurious keys.

For example, suppose Oscar obtains the ciphertext string WNAJW, which has been obtained by encryption using a shift cipher. It is easy to see that there are only two "meaningful" plaintext strings, namely river and arena, corresponding respectively to the possible encryption keys F (= 5) and W (= 22). Of these two keys, one will be the correct key and the other will be spurious. (Actually, it is moderately difficult to find a ciphertext of length 5 for the Shift Cipher that has two meaningful decryptions; the reader might search for other examples.)

Our goal is to prove a bound on the expected number of spurious keys. First, we have to define what we mean by the entropy (per letter) of a natural language L, which we denote HL. HL should be a measure of the average information per letter in a "meaningful" string of plaintext. (Note that a random string of alphabetic characters would have entropy (per letter) equal to log2 [image].) As a "first-order" approximation to HL, we could take H(P). In the case where L is the English language, we get [image] by using the probability distribution given in Table 1.1.

Of course, successive letters in a language are not independent, and correlations among successive letters reduce the entropy. For example, in English, the letter "Q" is always followed by the letter "U." For a "second-order" approximation, we would compute the entropy of the probability distribution of all digrams and then divide by 2. In general, define Pn to be the random variable that has as its probability distribution that of all n-grams of plaintext. We make use of the following definitions.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

DEFINITION 2.6 Suppose L is a natural language. The entropy of L is defined to be the quantity

and the redundancy of L is defined to be

REMARK HL measures the entropy letter of the language L. A random language would have entropy log2 [image]. So the quantity RL measures the fraction of "excess characters," which we think of as redundancy.

In the case of the English language, a tabulation of a large number of digrams and their frequencies would produce an estimate for H(P2). is one estimate obtained in this way. One could continue, tabulating trigrams, etc. and thus obtain an estimate for HL. In fact, various experiments have yielded the

empirical result that 1.0 ≤ HL ≤ 1.5. That is, the average information content in English is something like one to one and a half bits per letter!

Using 1.25 as our estimate of HL gives a redundancy of about 0.75. This means that the English language is 75% redundant! (This is not to say that one can arbitrarily remove three out of every four letters from English text and hope to still be able to read it. What it does mean is that it is possible to find a Huffman encoding of n-grams, for a large enough value of n, which will compress English text to about one quarter of its original length.)

Given probability distributions on [image] and [image], we can define the induced probability distribution on [image], the set of n-grams of ciphertext (we already did this in the case n = 1). We have defined Pn to be a random variable representing an n-gram of plaintext. Similarly, define Cn to be a random variable representing an n-gram of ciphertext.

Given y ∈ Cn, define

That is, K(y) is the set of keys K for which y is the encryption of a meaningful string of plaintext of length n, i.e., the set of "possible" keys, given that y is the ciphertext. If y is the observed sequence of ciphertext, then the number of spurious keys is |K(y)| - 1, since only one of the "possible" keys is the correct key. The average number of spurious keys (over all possible ciphertext strings of length n) is denoted by [image]. Its value is computed to be

From Theorem 2.10, we have that

Also, we can use the estimate

provided n is reasonably large. Certainly,

Then, if [image], it follows that

Next, we relate the quantity H(K|Cn) to the number of spurious keys, [image]. We compute as follows:

where we apply Jensen's Inequality (Theorem 2.5) with f(x) = log2 x. Thus we obtain the inequality

Combining the two inequalities (2.1) and (2.2), we get that

In the case where keys are chosen equiprobably (which maximizes H(K)), we have the following result.

THEOREM 2.11

Suppose [image] is a cryptosystem where [image] and keys are chosen equiprobably. Let RL denote the redundancy of the underlying language. Then given a string of ciphertext of length n, where n is sufficiently large, the expected number of spurious keys, [image], satisfies

The quantity [image] approaches 0 exponentially quickly as n increases. Also, note that the estimate may not be accurate for small values of n, especially since H(Pn)/n may not be a good estimate for HL if n is small.

We have one more concept to define.

DEFINITION 2.7 The unicity distance of a cryptosystem is defined to be the value of n, denoted by n0, at which the expected number of spurious keys becomes zero; i.e., the average amount of ciphertext required for an opponent to be able to uniquely compute the key, given enough computing time.

If we set [image] in Theorem 2.11 and solve for n, we get an estimate for the unicity distance, namely

As an example, consider the Substitution Cipher. In this cryptosystem, [image] and [image]!. If we take RL = 0.75, then we get an estimate for the unicity distance of

This suggests that, given a ciphertext string of length at least 25, (usually) a unique decryption is possible.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

2.5 Product Cryptosystems

Another innovation introduced by Shannon in his 1949 paper was the idea of combining cryptosystems by forming their "product." This idea has been of fundamental importance in the design of present-day cryptosystems such as the Data Encryption Standard, which we study in the next chapter.

For simplicity, we will confine our attention in this section to cryptosystems in which [image]: cryptosystems of this type are called endomorphic. Suppose [image] and [image] are two endomorphic cryptosystems which have the same plaintext (and ciphertext) spaces. Then the product of S1 and S2, denoted by S1 × S2, is defined to be the cryptosystem

A key of the product cryptosystem has the form K = (K1, K2), where [image] and [image]. The encryption and decryption rules of the product cryptosystem are defined as follows: For each K = (K1, K2), we have an encryption rule eK defined by the formula

and a decryption rule defined by the formula

That is, we first encrypt x with [image], and then "re-encrypt" the resulting ciphertext with [image]. Decrypting is similar, but it must be done in the reverse order:

Recall also that cryptosystems have probability distributions associated with their keyspaces. Thus we need to define the probability distribution for the keyspace [image] of the product cryptosystem. We do this in a very natural way:

*
* Figure 2.2 Multiplicative Cipher

In other words, choose K1 using the distribution [image], and then independently choose K2 using the distribution [image].

Here is a simple example to illustrate the definition of a product cryptosystem. Suppose we define the Multiplicative Cipher as in Figure 2.2.

Suppose M is the Multiplicative Cipher (with keys chosen equiprobably) and S is the Shift Cipher (with keys chosen equiprobably). Then it is very easy to see that M × S is nothing more than the Affine Cipher (again, with keys chosen equiprobably). It is slightly more difficult to show that S × M is also the Affine Cipher with equiprobable keys.

Let's prove these assertions. A key in the Shift Cipher is an element [image], and the corresponding encryption rule is eK(x) = x + K mod 26. A key in the Multiplicative Cipher is an element [image] such that gcd(a, 26) = 1; the corresponding encryption rule is ea(x) = ax mod 26. Hence, a key in the product cipher M × S has the form (a, K), where

But this is precisely the definition of a key in the Affine Cipher. Further, the probability of a key in the Affine Cipher is 1/312 = 1/12 × 1/26, which is the product of the probabilities of the keys a and K, respectively. Thus M × S is the Affine Cipher.

Now let's consider S × M. A key in this cipher has the form (K, a), where

Thus the key (K, a) of the product cipher S × M is identical to the key (a, aK) of the Affine Cipher. It remains to show that each key of the Affine Cipher arises with the same probability 1/312 in the product cipher S × M. Observe that aK = K1 if and only if K = a-1K1 (recall that gcd(a, 26) = 1, so a has a multiplicative inverse). In other words, the key (a, K1) of the Affine Cipher is equivalent to the key (a-1K1, a) of the product cipher S × M. We thus have a bijection between the two key spaces. Since each key is equiprobable, we conclude that S × M is indeed the Affine Cipher.

We have shown that M × S = S × M. Thus we would say that the two cryptosystems commute. But not all pairs of cryptosystems commute; it is easy to find counterexamples. On the other hand, the product operation is always associative: (S1 × S2) × S3 = S1 × (S2 × S3).

If we take the product of an (endomorphic) cryptosystem S with itself, we obtain the cryptosystem S × S, which we denote by S2. If we take the n-fold product, the resulting cryptosystem is denoted by Sn. We call Sn an iterated cryptosystem.

A cryptosystem S is defined to be idempotent if S2 = S. Many of the cryptosystems we studied in Chapter 1 are idempotent. For example, the Shift, Substitution, Affine, Hill, Vigenere and Permutation Ciphers are all idempotent. Of course, if a cryptosystem S is idempotent, then there is no point in using the product system S2, as it requires an extra key but provides no more security.

If a cryptosystem is not idempotent, then there is a potential increase in security by iterating several times. This idea is used in the Data Encryption Standard, which consists of 16 iterations. But, of course, this approach requires a non-idempotent cryptosystem to start with. One way in which simple non-idempotent cryptosystems can sometimes be constructed is to take the product of two different (simple) cryptosystems.

REMARK It is not hard to show that if S1 and S2 are both idempotent and they commute, then S1 × S2 will also be idempotent. This follows from the following algebraic manipulations:

(Note the use of the associative property in this proof.)

So, if S1 and S2 are both idempotent, and we want S1 × S2 to be non-idempotent, then it is necessary that S1 and S2 not commute.

Fortunately, many simple cryptosystems are suitable building blocks in this type of approach. Taking the product of substitution-type ciphers with permutation-type ciphers is a commonly used technique. We will see a realization of this in the next chapter.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

2.6 Notes

The idea of perfect secrecy and the use of entropy techniques in cryptography was pioneered by Shannon [SH49]. Product cryptosystems are also discussed in this paper. The concept of entropy was defined by Shannon in [SH48]. Good introductions to entropy, Huffman coding and related topics can be found in the books by Welsh [WE88] and Goldie and Pinch [GP91].

The results of Section 2.4 are due to Beauchemin and Brassard [BB88], who generalized earlier results of Shannon.

Exercises

2.1 Let n be a positive integer. A Latin square of order n is an n × n array L of the integers 1, ..., n such that every one of the n integers occurs exactly once in each row and each column of L. An example of a Latin square of order 3 is as follows:
1 | 2 | 3
3 | 1 | 2
2 | 3 | 1

Given any Latin square L of order n, we can define a related cryptosystem. Take [image]. For 1 ≤ i ≤ n, the encryption rule ei is defined to be ei(j) = L(i, j). (Hence each row of L gives rise to one encryption rule.)

Give a complete proof that this Latin square cryptosystem achieves perfect secrecy.

2.2 Prove that the Affine Cipher achieves perfect secrecy.
2.3 Suppose a cryptosystem achieves perfect secrecy for a particular plaintext probability distribution p0. Prove that perfect secrecy is maintained for any plaintext probability distribution.
2.4 Prove that if a cryptosystem has perfect secrecy and [image], then every ciphertext is equally probable.
2.5 Suppose X is a set of cardinality n, where 2k ≤ n < 2k+1, and p(x) = 1/n for all x ∈ X.

(a) Find a prefix-free encoding of X, say f, such that [image].

HINT Encode 2k+1 - n elements of X as strings of length k, and encode the remaining elements as strings of length k + 1.

(b) Illustrate your construction for n = 6. Compute [image] and H(X) in this case.

2.6 Suppose X = {a, b, c, d, e} has the following probability distribution: p(a) = .32, p(b) = .23, p(q) = .20, p(d) = .15 and p(e) = .10. Use Huffman's algorithm to find the optimal prefix-free encoding of X. Compare the length of this encoding to H(X).
2.7 Prove that H(X, Y) = H(Y) + H(X|Y). Then show as a corollary that H(X|Y) ≤ H(X), with equality if and only if X and Y are independent.
2.8 Prove that a cryptosystem has perfect secrecy if and only if H(P|C) = H(P).
2.9 Prove that, in any cryptosystem, H(K|C) ≥ H(P|C). (Intuitively, this result says that, given a ciphertext, the opponent's uncertainty about the key is at least as great as his uncertainty about the plaintext.)
2.10 Consider a cryptosystem in which [image]. Suppose the encryption matrix is as follows:

Given that keys are chosen equiprobably, and the plaintext probability distribution is [image], compute H(P), H(C), H(K), H(K|C) and H(P|C).

2.11 Compute H(K|C) and H(K|P, C) for the Affine Cipher.
2.12 Consider a Vigenere Cipher with keyword length m. Show that the unicity distance is 1/RL, where RL is the redundancy of the underlying language. (This result is interpreted as follows. If n0 denotes the number of alphabetic characters being encrypted, then the "length" of the plaintext is n0/m, since each plaintext element consists of m alphabetic characters. So, a unicity distance of 1/RL corresponds to a plaintext consisting of m/RL alphabetic characters.)
2.13 Show that the unicity distance of the Hill Cipher (with an m × m encryption matrix) is less than m/RL (Note that the number of alphabetic characters in a plaintext of this length is m2/RL.)
2.14 A Substitution Cipher over a plaintext space of size n has [image] Stirling's formula gives the following estimate for n!:

(a) Using Stirling's formula, derive an estimate of the unicity distance of the Substitution Cipher.
(b) Let m ≥ 1 be an integer. The m-gram Substitution Cipher is the Substitution Cipher where the plaintext (and ciphertext) spaces consist of all 26m m-grams. Estimate the unicity distance of the m-gram Substitution Cipher if RL = 0.75.

2.15 Prove that the Shift Cipher is idempotent.
2.16 Suppose S1 is the Shift Cipher (with equiprobable keys, as usual) and S2 is the Shift Cipher where keys are chosen with respect to some probability distribution [image] (which need not be equiprobable). Prove that S1 × S2 = S1.
2.17 Suppose S1 and S2 are Vigenere Ciphers with keyword lengths m1, m2 respectively, where m1 > m2.

(a) If m2 | m1, then show that S2 × S1 = S1.
(b) One might try to generalize the previous result by conjecturing that S2 × S1 = S3, where S3 is the Vigenere Cipher with keyword length lcm(m1, m2). Prove that this conjecture is false.

HINT If [image] mod m2, then the number of keys in the product cryptosystem S2 × S1 is than the number of keys in S3.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 3
The Data Encryption Standard

3.1 Introduction

On May 15, 1973, the National Bureau of Standards published a solicitation for cryptosystems in the Federal Register. This lead ultimately to the development of the Data Encryption Standard, or DES, which has become the most widely used cryptosystem in the world. DES was developed at IBM, as a modification of an earlier system known as LUCIFER. DES was first published in the Federal Register of March 17, 1975. After a considerable amount of public discussion, DES was adopted as a standard for "unclassified" applications on January 15, 1977. DES has been reviewed by the National Bureau of Standards (approximately) every five years since its adoption. Its most recent renewal was in January 1994, when it was renewed until 1998. It is anticipated that it will not remain a standard past 1998.

3.2 Description of DES

A complete description of DES is given in the Federal Information Processing Standards Publication 46, dated January 15, 1977. DES encrypts a plaintext bitstring x of length 64 using a key K which is a bitstring of length 56, obtaining a ciphertext bitstring which is again a bitstring of length 64. We first give a "high-level" description of the system.

The algorithm proceeds in three stages:

1. Given a plaintext x, a bitstring x0 is constructed by permuting the bits of x according to a (fixed) initial permutation IP. We write x0 = IP (x) = L0R0, where L0 comprises the first 32 bits of x0 and R0 the last 32 bits.
2. 16 iterations of a certain function are then computed. We compute LiRi,

*
* Figure 3.1 One round of DES encryption

1 ≤ i ≤ 16, according to the following rule:

where ⊕ denotes the exclusive-or of two bitstrings. f is a function that we will describe later, and K1, K2, . . . , K16 are each bitstrings of length 48 computed as a function of the key K. (Actually, each Ki is a permuted selection of bits from K.) K1, K2, . . . , K16 comprises the key schedule. One round of encryption is depicted in Figure 3.1

3. Apply the inverse permutation IP-1 to the bitstring R16L16, obtaining the ciphertext y. That is, y = IP-1(R16L16). Note the inverted order of L16 and R16.

The function f takes as input a first argument A, which is a bitstring of length 32, and a second argument J that is a bitstring of length 48, and produces as output a bitstring of length 32. The following steps are executed.

1. The first argument A is "expanded" to a bitstring of length 48 according to a fixed expansion function E. E(A) consists of the 32 bits from A, permuted in a certain way, with 16 of the bits appearing twice.

*
* Figure 3.2 The DES f function

2. Compute E(A) ⊕ J and write the result as the concatenation of eight 6-bit strings B = B1B2B3B4B5B6B7B8.
3. The next step uses eight S-boxes S1, . . . , S8. Each Si is a fixed 4 × 16 array whose entries come from the integers 0 - 15. Given a bitstring of length six, say Bj = b1b2b3b4b5b6, we compute Sj(Bj) as follows. The two bits b1b6 determine the binary representation of a row r of Sj (0 ≤ r ≤ 3), and the four bits b2b3b4b5 determine the binary representation of a column c of Sj (0 ≤ c ≤ 15). Then Sj(Bj) is defined to be the entry Sj(r, c), written in binary as a bitstring of length four. (Hence, each Sj can be thought of as a function that accepts as input a bitstring of length two and one of length four, and produces as output a bitstring of length four.) In this fashion, we compute Cj = Sj(Bj), 1 ≤ j ≤ 8.
4. The bitstring C = C1C2C3C4C5C6C7C8 of length 32 is permuted according to a fixed permutation P. The resulting bitstring P(C) is defined to be f(A, J).

The f function is depicted in Figure 3.2. Basically, it consists of a substitution (using an S-box) followed by the (fixed) permutation P. The 16 iterations of f comprise a product cryptosystem, as described in Section 2.5.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

In the remainder of this section, we present the specific functions used in DES.

The initial permutation IP is as follows:

IP
58 | 50 | 42 | 34 | 26 | 18 | 10 | 2
60 | 52 | 44 | 36 | 28 | 20 | 12 | 4
62 | 54 | 46 | 38 | 30 | 22 | 14 | 6
64 | 56 | 48 | 40 | 32 | 24 | 16 | 8
57 | 49 | 41 | 33 | 25 | 17 | 9 | 1
59 | 51 | 43 | 35 | 27 | 19 | 11 | 3
61 | 53 | 45 | 37 | 29 | 21 | 13 | 5
63 | 55 | 47 | 39 | 31 | 23 | 15 | 7

This means that the 58th bit of x is the first bit of IP(x); the 50th bit of x is the second bit of IP(x), etc.

The inverse permutation IP-1 is:

IP-1
40 | 8 | 48 | 16 | 56 | 24 | 64 | 32
39 | 7 | 47 | 15 | 55 | 23 | 63 | 31
38 | 6 | 46 | 14 | 54 | 22 | 62 | 30
37 | 5 | 45 | 13 | 53 | 21 | 61 | 29
36 | 4 | 44 | 12 | 52 | 20 | 60 | 28
35 | 3 | 43 | 11 | 51 | 19 | 59 | 27
34 | 2 | 42 | 10 | 50 | 18 | 58 | 26
33 | 1 | 41 | 9 | 49 | 17 | 57 | 25

The expansion function E is specified by the following table:

E bit-selection table
32 | 1 | 2 | 3 | 4 | 5
4 | 5 | 6 | 7 | 8 | 9
8 | 9 | 10 | 11 | 12 | 13
12 | 13 | 14 | 15 | 16 | 17
16 | 17 | 18 | 19 | 20 | 21
20 | 21 | 22 | 23 | 24 | 25
24 | 25 | 26 | 27 | 28 | 29
28 | 29 | 30 | 31 | 32 | 1


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

The eight S-boxes and the permutation P are now presented:

S1
14 | 4 | 13 | 1 | 2 | 15 | 11 | 8 | 3 | 10 | 6 | 12 | 5 | 9 | 0 | 7
0 | 15 | 7 | 4 | 14 | 2 | 13 | 1 | 10 | 6 | 12 | 11 | 9 | 5 | 3 | 8
4 | 1 | 14 | 8 | 13 | 6 | 2 | 11 | 15 | 12 | 9 | 7 | 3 | 10 | 5 | 0
15 | 12 | 8 | 2 | 4 | 9 | 1 | 7 | 5 | 11 | 3 | 14 | 10 | 0 | 6 | 13

S2
15 | 1 | 8 | 14 | 6 | 11 | 3 | 4 | 9 | 7 | 2 | 13 | 12 | 0 | 5 | 10
3 | 13 | 4 | 7 | 15 | 2 | 8 | 14 | 12 | 0 | 1 | 10 | 6 | 9 | 11 | 5
0 | 14 | 7 | 11 | 10 | 4 | 13 | 1 | 5 | 8 | 12 | 6 | 9 | 3 | 2 | 15
13 | 8 | 10 | 1 | 3 | 15 | 4 | 2 | 11 | 6 | 7 | 12 | 0 | 5 | 14 | 9

S3
10 | 0 | 9 | 14 | 6 | 3 | 15 | 5 | 1 | 13 | 12 | 7 | 11 | 4 | 2 | 8
13 | 7 | 0 | 9 | 3 | 4 | 6 | 10 | 2 | 8 | 5 | 14 | 12 | 11 | 15 | 1
13 | 6 | 4 | 9 | 8 | 15 | 3 | 0 | 11 | 1 | 2 | 12 | 5 | 10 | 14 | 7
1 | 10 | 13 | 0 | 6 | 9 | 8 | 7 | 4 | 15 | 14 | 3 | 11 | 5 | 2 | 12

S4
7 | 13 | 14 | 3 | 0 | 6 | 9 | 10 | 1 | 2 | 8 | 5 | 11 | 12 | 4 | 15
13 | 8 | 11 | 5 | 6 | 15 | 0 | 3 | 4 | 7 | 2 | 12 | 1 | 10 | 14 | 9
10 | 6 | 9 | 0 | 12 | 11 | 7 | 13 | 15 | 1 | 3 | 14 | 5 | 2 | 8 | 4
3 | 15 | 0 | 6 | 10 | 1 | 13 | 8 | 9 | 4 | 5 | 11 | 12 | 7 | 2 | 14

S5
2 | 12 | 4 | 1 | 7 | 10 | 11 | 6 | 8 | 5 | 3 | 15 | 13 | 0 | 14 | 9
14 | 11 | 2 | 12 | 4 | 7 | 13 | 1 | 5 | 0 | 15 | 10 | 3 | 9 | 8 | 6
4 | 2 | 1 | 11 | 10 | 13 | 7 | 8 | 15 | 9 | 12 | 5 | 6 | 3 | 0 | 14
11 | 8 | 12 | 7 | 1 | 14 | 2 | 13 | 6 | 15 | 0 | 9 | 10 | 4 | 5 | 3

S6
12 | 1 | 10 | 15 | 9 | 2 | 6 | 8 | 0 | 13 | 3 | 4 | 14 | 7 | 5 | 11
10 | 15 | 4 | 2 | 7 | 12 | 9 | 5 | 6 | 1 | 13 | 14 | 0 | 11 | 3 | 8
9 | 14 | 15 | 5 | 2 | 8 | 12 | 3 | 7 | 0 | 4 | 10 | 1 | 13 | 11 | 6
4 | 3 | 2 | 12 | 9 | 5 | 15 | 10 | 11 | 14 | 1 | 7 | 6 | 0 | 8 | 13

S7
4 | 11 | 2 | 14 | 15 | 0 | 8 | 13 | 3 | 12 | 9 | 7 | 5 | 10 | 6 | 1
13 | 0 | 11 | 7 | 4 | 9 | 1 | 10 | 14 | 3 | 5 | 12 | 2 | 15 | 8 | 6
1 | 4 | 11 | 13 | 12 | 3 | 7 | 14 | 10 | 15 | 6 | 8 | 0 | 5 | 9 | 2
6 | 11 | 13 | 8 | 1 | 4 | 10 | 7 | 9 | 5 | 0 | 15 | 14 | 2 | 3 | 12

S8
13 | 2 | 8 | 4 | 6 | 15 | 11 | 1 | 10 | 9 | 3 | 14 | 5 | 0 | 12 | 7
1 | 15 | 13 | 8 | 10 | 3 | 7 | 4 | 12 | 5 | 6 | 11 | 0 | 14 | 9 | 2
7 | 11 | 4 | 1 | 9 | 12 | 14 | 2 | 0 | 6 | 10 | 13 | 15 | 3 | 5 | 8
2 | 1 | 14 | 7 | 4 | 10 | 8 | 13 | 15 | 12 | 9 | 0 | 3 | 5 | 6 | 11

P
16 | 7 | 20 | 21
29 | 12 | 28 | 17
1 | 15 | 23 | 26
5 | 18 | 31 | 10
2 | 8 | 24 | 14
32 | 27 | 3 | 9
19 | 13 | 30 | 6
22 | 11 | 4 | 25


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Finally, we need to describe the computation of the key schedule from the key K. Actually, K is a bitstring of length 64, of which 56 bits comprise the key and 8 bits are parity-check bits (for error-detection). The bits in positions 8, 16, . . . , 64 are defined so that each byte contains an odd number of 1's. Hence, a single error can be detected within each group of 8 bits. The parity-check bits are ignored in the computation of the key schedule.

1. Given a 64-bit key K, discard the parity-check bits and permute the remaining bits of K according to a (fixed) permutation PC-1. We will write PC-1 (K) = C0D0, where C0 comprises the first 28 bits of PC-1(K) and D0 the last 28 bits.
2. For i ranging from 1 to 16, compute

and Ki = PC-2(CiDi). LSi represents a cyclic shift (to the left) of either one or two positions, depending on the value of i: shift one position if i = 1, 2, 9 or 16, and shift two positions otherwise. PC-2 is another fixed permutation.

The key schedule computation is depicted in Figure 3.3.

The permutations PC-1 and PC-2 used in the key schedule computation are as follows:

PC-1
57 | 49 | 41 | 33 | 25 | 17 | 9
1 | 58 | 50 | 42 | 34 | 26 | 18
10 | 2 | 59 | 51 | 43 | 35 | 27
19 | 11 | 3 | 60 | 52 | 44 | 36
63 | 55 | 47 | 39 | 31 | 23 | 15
7 | 62 | 54 | 46 | 38 | 30 | 22
14 | 6 | 61 | 53 | 45 | 37 | 29
21 | 13 | 5 | 28 | 20 | 12 | 4

*
* Figure 3.3 Computation of DES key schedule

PC-2
14 | 17 | 11 | 24 | 1 | 5
3 | 28 | 15 | 6 | 21 | 10
23 | 19 | 12 | 4 | 26 | 8
16 | 7 | 27 | 20 | 13 | 2
41 | 52 | 31 | 37 | 47 | 55
30 | 40 | 51 | 45 | 33 | 48
44 | 49 | 39 | 56 | 34 | 53
46 | 42 | 50 | 36 | 29 | 32

We now display the resulting key schedule. As mentioned above, each round uses a 48-bit key comprised of 48 of the bits in K. The entries in the tables below refer to the bits in K that are used in the various rounds.

Round 1
10 | 51 | 34 | 60 | 49 | 17 | 33 | 57 | 2 | 9 | 19 | 42
3 | 35 | 26 | 25 | 44 | 58 | 59 | 1 | 36 | 27 | 18 | 41
22 | 28 | 39 | 54 | 37 | 4 | 47 | 30 | 5 | 53 | 23 | 29
61 | 21 | 38 | 63 | 15 | 20 | 45 | 14 | 13 | 62 | 55 | 31

Round 2
2 | 43 | 26 | 52 | 41 | 9 | 25 | 49 | 59 | 1 | 11 | 34
60 | 27 | 18 | 17 | 36 | 50 | 51 | 58 | 57 | 19 | 10 | 33
14 | 20 | 31 | 46 | 29 | 63 | 39 | 22 | 28 | 45 | 15 | 21
53 | 13 | 30 | 55 | 7 | 12 | 37 | 6 | 5 | 54 | 47 | 23

Round 3
51 | 27 | 10 | 36 | 25 | 58 | 9 | 33 | 43 | 50 | 60 | 18
44 | 11 | 2 | 1 | 49 | 34 | 35 | 42 | 41 | 3 | 59 | 17
61 | 4 | 15 | 30 | 13 | 47 | 23 | 6 | 12 | 29 | 62 | 5
37 | 28 | 14 | 39 | 54 | 63 | 21 | 53 | 20 | 38 | 31 | 7

Round 4
35 | 11 | 59 | 49 | 9 | 42 | 58 | 17 | 27 | 34 | 44 | 2
57 | 60 | 51 | 50 | 33 | 18 | 19 | 26 | 25 | 52 | 43 | 1
45 | 55 | 62 | 14 | 28 | 31 | 7 | 53 | 63 | 13 | 46 | 20
21 | 12 | 61 | 23 | 38 | 47 | 5 | 37 | 4 | 22 | 15 | 54

Round 5
19 | 60 | 43 | 33 | 58 | 26 | 42 | 1 | 11 | 18 | 57 | 51
41 | 44 | 35 | 34 | 17 | 2 | 3 | 10 | 9 | 36 | 27 | 50
29 | 39 | 46 | 61 | 12 | 15 | 54 | 37 | 47 | 28 | 30 | 4
5 | 63 | 45 | 7 | 22 | 31 | 20 | 21 | 55 | 6 | 62 | 38

Round 6
3 | 44 | 27 | 17 | 42 | 10 | 26 | 50 | 60 | 2 | 41 | 35
25 | 57 | 19 | 18 | 1 | 51 | 52 | 59 | 58 | 49 | 11 | 34
13 | 23 | 30 | 45 | 63 | 62 | 38 | 21 | 31 | 12 | 14 | 55
20 | 47 | 29 | 54 | 6 | 15 | 4 | 5 | 39 | 53 | 46 | 22

Round 7
52 | 57 | 11 | 1 | 26 | 59 | 10 | 34 | 44 | 51 | 25 | 19
9 | 41 | 3 | 2 | 50 | 35 | 36 | 43 | 42 | 33 | 60 | 18
28 | 7 | 14 | 29 | 47 | 46 | 22 | 5 | 15 | 63 | 61 | 39
4 | 31 | 13 | 38 | 53 | 62 | 55 | 20 | 23 | 37 | 30 | 6

Round 8
36 | 41 | 60 | 50 | 10 | 43 | 59 | 18 | 57 | 35 | 9 | 3
58 | 25 | 52 | 51 | 34 | 19 | 49 | 27 | 26 | 17 | 44 | 2
12 | 54 | 61 | 13 | 31 | 30 | 6 | 20 | 62 | 47 | 45 | 23
55 | 15 | 28 | 22 | 37 | 46 | 39 | 4 | 7 | 21 | 14 | 53

Round 9
57 | 33 | 52 | 42 | 2 | 35 | 51 | 10 | 49 | 27 | 1 | 60
50 | 17 | 44 | 43 | 26 | 11 | 41 | 19 | 18 | 9 | 36 | 59
4 | 46 | 53 | 5 | 23 | 22 | 61 | 12 | 54 | 39 | 37 | 15
47 | 7 | 20 | 14 | 29 | 38 | 31 | 63 | 62 | 13 | 6 | 45

Round 10
41 | 17 | 36 | 26 | 51 | 19 | 35 | 59 | 33 | 11 | 50 | 44
34 | 1 | 57 | 27 | 10 | 60 | 25 | 3 | 2 | 58 | 49 | 43
55 | 30 | 37 | 20 | 7 | 6 | 45 | 63 | 38 | 23 | 21 | 62
31 | 54 | 4 | 61 | 13 | 22 | 15 | 47 | 46 | 28 | 53 | 29

Round 11
25 | 1 | 49 | 10 | 35 | 3 | 19 | 43 | 17 | 60 | 34 | 57
18 | 50 | 41 | 11 | 59 | 44 | 9 | 52 | 51 | 42 | 33 | 27
39 | 14 | 21 | 4 | 54 | 53 | 29 | 47 | 22 | 7 | 5 | 46
15 | 38 | 55 | 45 | 28 | 6 | 62 | 31 | 30 | 12 | 37 | 13

Round 12
9 | 50 | 33 | 59 | 19 | 52 | 3 | 27 | 1 | 44 | 18 | 41
2 | 34 | 25 | 60 | 43 | 57 | 58 | 36 | 35 | 26 | 17 | 11
23 | 61 | 5 | 55 | 38 | 37 | 13 | 31 | 6 | 54 | 20 | 30
62 | 22 | 39 | 29 | 12 | 53 | 46 | 15 | 14 | 63 | 21 | 28

Round 13
58 | 34 | 17 | 43 | 3 | 36 | 52 | 11 | 50 | 57 | 2 | 25
51 | 18 | 9 | 44 | 27 | 41 | 42 | 49 | 19 | 10 | 1 | 60
7 | 45 | 20 | 39 | 22 | 21 | 28 | 15 | 53 | 38 | 4 | 14
46 | 6 | 23 | 13 | 63 | 37 | 30 | 62 | 61 | 47 | 5 | 12

Round 14
42 | 18 | 1 | 27 | 52 | 49 | 36 | 60 | 34 | 41 | 51 | 9
35 | 2 | 58 | 57 | 11 | 25 | 26 | 33 | 3 | 59 | 50 | 44
54 | 29 | 4 | 23 | 6 | 5 | 12 | 62 | 37 | 22 | 55 | 61
30 | 53 | 7 | 28 | 47 | 21 | 14 | 46 | 45 | 31 | 20 | 63

Round 15
26 | 2 | 50 | 11 | 36 | 33 | 49 | 44 | 18 | 25 | 35 | 58
19 | 51 | 42 | 41 | 60 | 9 | 10 | 17 | 52 | 43 | 34 | 57
38 | 13 | 55 | 7 | 53 | 20 | 63 | 46 | 21 | 6 | 39 | 45
14 | 37 | 54 | 12 | 31 | 5 | 61 | 30 | 29 | 15 | 4 | 47

Round 16
18 | 59 | 42 | 3 | 57 | 25 | 41 | 36 | 10 | 17 | 27 | 50
11 | 43 | 34 | 33 | 52 | 1 | 2 | 9 | 44 | 35 | 26 | 49
30 | 5 | 47 | 62 | 45 | 12 | 55 | 38 | 13 | 61 | 31 | 37
6 | 29 | 46 | 4 | 23 | 28 | 53 | 22 | 21 | 7 | 63 | 39


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Decryption is done using the same algorithm as encryption, starting with y as the input, but using the key schedule K16, . . . , K1 in reverse order. The output will be the plaintext x.

3.2.1 An Example of DES Encryption

Here is an example of encryption using the DES. Suppose we encrypt the (hexadecimal) plaintext

0123456789ABCDEF

using the (hexadecimal) key

133457799BBCDFF1.

The key, in binary, without parity-check bits, is

00010010011010010101101111001001101101111011011111111000.

Applying IP, we obtain L0 and R0 (in binary):

The 16 rounds of encryption are then performed, as indicated.

Finally, applying IP-1 to R16L16, we obtain the ciphertext, which (in hexadecimal form) is:

85E813540F0AB405.

3.3 The DES Controversy

When DES was proposed as a standard, there was considerable criticism. One objection to DES concerned the S-boxes. All computations in DES, with the exception of the S-boxes, are linear, e.g., computing the exclusive-or of two outputs is the same as forming the exclusive-or of two inputs and then computing the output. The S-boxes, being the non-linear component of the cryptosystem, are vital to its security (We saw in Chapter 1 how linear cryptosystems, such as the Hill Cipher, could easily be cryptanalyzed by a known plaintext attack.) However, the design criteria of the S-boxes are not completely known. Several people have suggested that the S-boxes might contain hidden "trapdoors" which would allow the National Security Agency to decrypt messages while maintaining that DES is "secure." It is, of course, impossible to disprove such an assertion, but no evidence has come to light that indicates that trap-doors in DES do in fact exist.

In 1976, the National Security Agency (NSA) asserted that the following properties of the S-boxes are design criteria:

P0 Each row of each S-box is a permutation of the integers 0, . . . , 15.
P1 No S-box is a linear or affine function of its inputs.
P2 Changing one input bit to an S-box causes at least two output bits to change.
P3 For any S-box and any input x, S(x) and S(x ⊕ 001100) differ in at least two bits (here x is a bitstring of length 6).

Two other properties of the S-boxes were designated as "caused by design criteria" by NSA.

P4 For any S-box, for any input x, and for e, f ∈ {0, 1}, S(x) ≠ S(x ⊕ 11ef00).
P5 For any S-box, if one input bit is fixed, and we look at the value of one fixed output bit, the number of inputs for which this output bit equals 0 will be "close to" the number of inputs for which the output bit equals 1. (Note that if we fix the value of either the first or sixth input bit, then 16 inputs will cause a particular output bit to equal 0 and 16 inputs will cause the output to equal 1. For the second through fifth input bits, this will not be true, but the resulting distribution will be "close to" uniform. More precisely, for any S-box, if the value of any input bit is fixed, then the number of inputs for which any fixed output bit has the value 0 (or 1) is always between 13 and 19.)

It is not publicly known if further design criteria were used in the construction of the S-boxes.

The most pertinent criticism of DES is that the size of the keyspace, 256, is too small to be really secure. Various special-purpose machines have been proposed for a known plaintext attack, which would essentially perform an exhaustive search for the key. That is given a 64-bit plaintext x and corresponding ciphertext y, every possible key would be tested until a key K such that eK (x) = y is found (and note that there may be more than one such key K).

As early as 1977, Diffie and Hellman suggested that one could build a VLSI chip which could test 106 keys per second. A machine with 106 chips could search the entire key space in about a day. They estimated that such a machine could be built for about $20,000,000.

At the CRYPTO '93 Rump Session, Michael Wiener gave a very detailed design of a key search machine. The machine is based on a key search chip which is pipelined, so that 16 encryptions take place simultaneously. This chip can test 5 × 107 keys per second, and can be built using current technology for $10.50 per chip. A frame consisting of 5760 chips can be built for $100,000. This would allow a DES key to be found in about 1.5 days on average. A machine using 10 frames would cost $1,000,000, but would reduce the average search time to about 3.5 hours.

3.4 DES in Practice

Even though the description of DES is quite lengthy, it can be implemented very efficiently, either in hardware or in software. The only arithmetic operations to be performed are exclusive-ors of bitstrings. The expansion function E, the S-boxes, the permutations IP and P, and the computation of K1, K2, . . . , K16 can all be done in constant time by table look-up (in software) or by hard-wiring them into a circuit.

Current hardware implementations can attain extremely fast encryption rates. Digital Equipment Corporation announced at CRYPTO '92 that they have fabricated a chip with 50K transistors that can encrypt at the rate of 1 Gbit/second using a clock rate of 250 MHz! The cost of this chip is about $300. As of 1991, there were 45 hardware and firmware implementations of DES that had been validated by the National Bureau of Standards.

One very important application of DES is in banking transactions, using standards developed by the American Bankers Association. DES is used to encrypt personal identification numbers (PINs) and account transactions carried out by automated teller machines (ATMs). DES is also used by the Clearing House Interbank Payments System (CHIPS) to authenticate transactions involving over $1.5 × 1012 per week.

DES is also widely used in government organizations, such as the Department of Energy, the Justice Department, and the Federal Reserve System.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

3.4.1 DES Modes of Operation

Four modes of operation have been developed for DES: electronic codebook mode (ECB), cipher feedback mode (CFB), cipher block chaining mode (CBC) and output feedback mode (OFB).

ECB mode corresponds to the usual use of a block cipher: given a sequence x1x2 . . . of 64-bit plaintext blocks, each xi is encrypted with the same key K, producing a string of ciphertext blocks, y1y2 . . ..

In CBC mode, each ciphertext block yi is x-ored with the next plaintext block xi+1 before being encrypted with the key K. More formally, we start with a 64-bit initialization vector IV, and define y0 = IV. Then we construct y1, y2, . . . from the rule yi = eK(yi-1 ⊕ xi), i ≥ 1. The use of CBC mode is depicted in Figure 3.4.

*
* Figure 3.4 CBC mode

*
* Figure 3.5 CFB mode

In OFB and CFB modes, a keystream is generated which is then x-ored with the plaintext (i.e., it operates as a stream cipher, cf. Section 1.1.7). OFB is actually a synchronous stream cipher: the keystream is produced by repeatedly encrypting a 64-bit initialization vector, IV. We define z0 = IV, and then compute the keystream z1z2 . . . from the rule zi = eK(zi-1), i ≥ 1. The plaintext sequence x1x2 . . . is then encrypted by computing yi = xi ⊕ zi, i ≥ 1.

In CFB mode, we start with y0 = IV (a 64-bit initialization vector) and we produce the keystream element zi by encrypting the previous ciphertext block. That is, zi = eK (yi-1), i≥ 1. As in OFB mode, yi = xi ⊕ zi, i ≥ 1. The use of CFB is depicted in Figure 3.5 (note that the DES encryption function eK is used for both encryption and decryption in CFB and OFB modes).

There are also variations of OFB and CFB mode called k-bit feedback modes (1 ≤ k ≤ 64). We have described the 64-bit feedback modes here. 1-bit and 8-bit feedback modes are often used in practice for encrypting data one bit (or byte) at a time.

The four modes of operation have different advantages and disadvantages. In ECB and OFB modes, changing one 64-bit plaintext block, xi, causes the corresponding ciphertext block, yi, to be altered, but other ciphertext blocks are not affected. In some situations this might be a desirable property. For example, OFB mode is often used to encrypt satellite transmissions.

On the other hand, if a plaintext block xi is changed in CBC and CFB modes, then yi and all subsequent ciphertext blocks will be affected. This property means that CBC and CFB modes are useful for purposes of authentication. More specifically, these modes can be used to produce a message authentication code, or MAC. The MAC is appended to a sequence of plaintext blocks, and is used to convince Bob that the given sequence of plaintext originated with Alice and was not tampered with by Oscar. Thus the MAC guarantees the integrity (or authenticity) of a message (but it does not provide secrecy, of course).

We will describe how CBC mode is used to produce a MAC. We begin with the initialization vector IV consisting of all zeroes. Then construct the ciphertext blocks y1, . . . , yn with key K, using CBC mode. Finally, define the MAC to be yn. Then Alice transmits the sequence of plaintext blocks, x1 . . . xn, along with the MAC. When Bob receives x1, . . . xn, he can reconstruct y1, . . . , yn using the (secret) key K, and verify that yn is the same as the MAC that he received.

Note that Oscar cannot produce a valid MAC since he does not know the key K being used by Alice and Bob. Further, if Oscar intercepts a sequence of plaintext blocks x1 . . . xn, and changes one or more of them, then it is highly unlikely that Oscar can change the MAC so that it will be accepted by Bob.

It is often desirable to combine authenticity and secrecy. This could be done as follows: Alice first uses key K1 to produce a MAC for x1 . . . xn. Then she defines xn+1 to be the MAC, and she encrypts the sequence x1 . . . xn+1 using a second key, K2, yielding y1 . . . yn+1. When Bob receives y1 . . . yn+1, he first decrypts (using K2) and then checks that xn+1 is the MAC for x1 . . . xn using K1.

Alternatively, Alice could use K1 to encrypt x1 . . . xn, obtaining y1 . . . yn, and then use K2 to produce a MAC yn+1 for y1 . . . yn. Bob would use K2 to verify the MAC, and then use K1 to decrypt y1 . . . yn.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

3.5 A Time-memory Trade-off

In this section, we describe an interesting time-memory tradeoff for a chosen plaintext attack. Recall that in a chosen plaintext attack, Oscar obtains a plaintext-ciphertext pair produced using the (unknown) key K. So Oscar has x and y, where y = eK(x), and he wants to determine K.

A feature of this time-memory trade-off is that it does not depend on the "structure" of DES in any way. The only aspects of DES that are relevant to the attack are that plaintexts and ciphertexts have 64 bits, while keys have 56 bits.

*
* Figure 3.6 Computation of X (i, j)

We have already discussed the idea of exhaustive search: given a plaintext-ciphertext pair, try all 256 possible keys. This requires no memory but, on average, 255 keys will be tried before the correct one is found. On the other hand, for a given plaintext x, Oscar could precompute yK = eK(x) for all 256 keys K, and construct a table of ordered pairs (yK, K), sorted by their first coordinates. At a later time, when Oscar obtains the ciphertext y which is an encryption of plaintext x, he looks up the value y in the table, immediately obtaining the key K. Now the actual determination of the key requires only constant time, but we have a large memory requirement and a large precomputation time. (Note that this approach would yield no advantage in total computation time if only one key is to be found, since constructing the table takes at least as much time as an exhaustive search. The advantage occurs when several keys are to be found over a period of time, since the same table can be used in each case.)

The time-memory trade-off combines a smaller computation time than exhaustive search with a smaller memory requirement than table look-up. The algorithm can be described in terms of two parameters m and t, which are positive integers. The algorithm requires a reduction function R which reduces a bitstring of length 64 to one of length 56. (R might just discard eight of the 64 bits, for example.) Let x be a fixed plaintext string of length 64. Define the function [image] for a bitstring K0 of length 56. Note that g is a function that maps 56 bits to 56 bits.

In the pre-processing stage, Oscar chooses m random bitstrings of length 56, denoted X (i, 0), 1 ≤ i ≤ m. Oscar computes X(i, j) for 1 ≤ j ≤ t according to the recurrence relation X(i, j) = g (X(i, j - 1)), 1 ≤ i ≤ m, 1 ≤ j ≤ t, as indicated in Figure 3.6.

Then Oscar constructs a table of ordered pairs T = (X(i, t), X(i, 0)), sorted by their first coordinate (i.e., only the first and last columns of X are stored).

At a later time, Oscar obtains a ciphertext y which is an encryption of the chosen plaintext x (as before). He again wants to determine K. He is going to determine if K is in the first t columns of the array X, but he will do this by looking only at the table T.

*
* Figure 3.7 DES time-memory trade-off

Suppose that K = X (i, t -j) for some j, 1 ≤ j ≤ t (i.e., suppose that K is in the first t columns of X). Then it is clear that gj (K) = X (i, t), where gj denotes the function obtained by iterating g, j times. Now, observe that

Suppose we compute yj, 1 ≤ j ≤ t, from the recurrence relation

Then it follows that yj = X(i, t) if K = X(i, t - j). However, note that yj = X(i, t) is not sufficient to ensure that K = X(i, t - j). This is because the reduction function R is not an injection: The domain of R has cardinality 264 and the range of R has cardinality 256, so, on average, there are 28 = 256 pre-images of any given bitstring of length 56. So we need to check whether y = eX(i,t - j) (x), to see if X(i, t - j) is indeed the key. We did not store the value X(i, t - j), but we can easily re-compute it from X(i, 0) by iterating the g function t - j times.

Oscar proceeds according to the algorithm presented in Figure 3.7.

By analyzing the probability of success for the algorithm, it can be shown that if [image], then the probability that K = X(i,t - j) for some i, j is about 0.8mt/N. The factor 0.8 accounts for the fact that the numbers X(i,t) may not all be distinct. It is suggested that one should take [image] and construct about N1/3 tables, each using a different reduction function R. If this is done, the memory requirement is 112 × N2/3 bits (since we need to store 2 × N2/3 integers, each of which has 56 bits). The precomputation time is easily seen to be O(N).

The running time is a bit more dificult to analyze. First, note that step 3 can be implemented to run in (expected) constant time (using hash coding) or (worst-case) time O(log m) using a binary search. If step 3 is never satisfied (i.e., the search fails), then the running time is O(N2/3). A more detailed analysis shows that even when the running time of steps 4 and 5 is taken into account, the expected running time increases by only a constant factor.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

3.6 Differential Cryptanalysis

One very well-known attack on DES is the method of "differential cryptanalysis" introduced by Biham and Shamir. This is a chosen-plaintext attack. Although it does not provide a practical method of breaking the usual 16-round DES, it does succeed in breaking DES if the number of rounds of encryption is reduced. For instance, 8-round DES can be broken in only a couple of minutes on a small personal computer.

We will now describe the basic ideas used in this technique. For the purposes of this attack, we can ignore the initial permutation IP and its inverse (it has no effect on cryptanalysis). As mentioned above, we consider DES restricted to n rounds, for various values of n ≤ 16. So, in this setting, we will regard L0R0 as the plaintext, and LnRn as the ciphertext, in an n-round DES. (Note also that we are not inverting LnRn.)

Differential cryptanalysis involves comparing the x-or (exclusive-or) of two plaintexts to the x-or of the corresponding two ciphertexts. In general, we will be looking at two plaintexts L0R0 and [image] with a specified x-or value [image]. Throughout this discussion, we will use prime markings (′) to indicate the x-or of two bitstrings.

DEFINITION 3.1 Let Sj be a particular S-box (1 ≤ j ≤ 8). Consider an (ordered) pair of bitstrings of length six, say [image]. We say that the input x-or (of Sj) is [image] and the output x-or (of Sj) is [image].

Note that an input x-or is a bitstring of length six and an output x-or is a bitstring of length four.

DEFINITION 3.2 For any [image], define the set [image] to consist of the ordered pairs [image] having input x-or [image].

It is easy to see that any set [image] contains 26 = 64 pairs, and that

For each pair in [image], we can compute the output x-or of Sj and tabulate the resulting distribution. There are 64 output x-ors, which are distributed among 24 = 16 possible values. The non-uniformity of these distributions will be the basis for the attack.

Example 3.1

Suppose we consider the first S-box, S1, and the input x-or 110100. Then

For each ordered pair in the set Δ(110100), we compute output x-or of S1. For example, S1(000000) = E16 = 1110 and S1(110100) = 916 = 1001, so the output x-or for the pair (000000, 110100) is 0111.

If this is done for all 64 pairs in Δ(110100), then the following distribution of output x-ors is obtained:

In Example 3.1, only eight of the 16 possible output x-ors actually occur. This particular example has a very non-uniform distribution. In general, if we fix an S-box Sj and an input x-or [image], then on average, it turns out that about 75 - 80% of the possible output x-ors actually occur.

It will be convenient to have some notation to describe these distributions and how they arise, so we make the following definitions.

DEFINITION 3.3 For 1 ≤ j ≤ 8, and for bitstrings [image] of length six and [image] of length four, define

and

*
* Figure 3.8 Possible inputs with input x-or 110100

counts the number of pairs with input x-or equal to [image] which have output x-or equal to [image] for the S-box Sj. The actual pairs having the specified input x-ors and giving rise to the specified output x-ors can be obtained from the set [image]. Observe that this set can be partitioned into [image] pairs, each of which has (input) x-or equal to [image].

Observe that the distribution tabulated in Example 3.1 consists of the values [image]. The sets [image] are listed in Figure 3.8.

For each of the eight S-boxes, there are 64 possible input x-ors. Thus, there are 512 distributions which can be computed. These could easily be tabulated by computer.

Recall that the input to the S-boxes in round i is formed as B = E ⊕ J, where E = E(Ri-1) is the expansion of Ri-1 and J = Ki consists of the key bits for round i. Now, the input x-or (for all eight S-boxes) can be computed as follows:

It is very important to observe that the input x-or does not depend on the key bits J. (However, the output x-or certainly does depend on these key bits.)

We will write each of B, E and J as the concatenation of eight 6-bit strings:

and we write B* and E* in a similar way. Let us suppose for the moment that we know the values Ej and [image] for some j, 1 ≤ j ≤ 8, and the value of the output x-or for [image]. Then it must be the case that

where [image].

Suppose we define a set testj as follows:

DEFINITION 3.4 Suppose Ej and [image] are bitstrings of length six, and [image] is a bitstring of length four. Define

where [image].

That is, we take the x-or of Ej with every element of the set [image].

The following result is an immediate consequence of the discussion above.

THEOREM 3.1

Suppose Ej and [image] are two inputs to the S-box Sj, and the output x-or for Sj is [image]. Denote [image]. Then the key bits Jj occur in the set testj [image].

Observe that there will be exactly [image] bitstrings of length six in the set testj [image]; the correct value of Jj must be one of these possibilities.

Example 3.2

Suppose [image]. Since N1 (110100, 1101) = 8, there will be exactly eight bitstrings in the set test1 (000001, 110101, 1101). From Figure 3.8, we see that

Hence,

If we have a second such triple [image], then we can obtain a second set test1 of possible values for the keybits in J1. The true value of J1 must be in the intersection of both sets. If we have several such triples, then we can quickly determine the key bits in J1. One straightforward way to do this is to maintain an array of 64 counters, representing the 64 possibilities for the six key bits in J1. A counter is incremented every time the corresponding key bits occur in a set test1 for a particular triple. Given t triples, we hope to find a unique counter which has the value t; this will correspond to the true value of the keybits in J1.

3.6.1 An Attack on a 3-round DES

Let's now see how the ideas of the previous section can be applied in a chosen plaintext attack of a 3-round DES. We will begin with a pair of plaintexts and corresponding ciphertexts: and [image]. We can express R3 as follows:

can be expressed in a similar way, and hence

Now, suppose we have chosen the plaintexts so that [image], i.e., so that

*
* Figure 3.9 Differential attack on 3-round DES

Then [image] and so

At this point, [image] is known since it can be computed from the two ciphertexts, and [image] is known since it can be computed from the two plaintexts. This means that we can compute [image] from the equation

Now, f(R2, K3) = P(C) and [image], where C and C*, respectively, denote the two outputs of the eight S-boxes (recall that P is a fixed, publicly known permutation). Hence,

and consequently

This is the output x-or for the eight S-boxes in round three.

Now, R2 = L3 and [image] are also known (they are part of the ciphertexts). Hence, we can compute

and

using the publicly known expansion function E. These are the inputs to the S-boxes for round three. So, we now know E, E*, and C′ for the third round, and we can proceed, as in the previous section, to construct the sets test1, . . ., test8 of possible values for the key bits in J1, . . . , J8.

A pseudo-code description of this algorithm is given in Figure 3.9. The attack will use several such triples E, E*, C′. We set up eight arrays of counters, and thereby determine the 48 bits in K3, the key for the third round. The 56 bits in the key can then be computed by an exhaustive search of the 28 = 256 possibilities for the remaining eight key bits.

Let's look at an example to illustrate.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 3.3

Suppose we have the following three pairs of plaintexts and ciphertexts, where the plaintexts have the specified x-ors, that are encrypted using the same key. We use a hexadecimal representation, for brevity:

plaintext | ciphertxt
748502CD38451097 | 03C70306D8A09F10
3874756438451097 | 78560A0960E6D4CB
486911026ACDFF31 | 45FA285BE5ADC730
375BD31F6ACDFF31 | 134F7915AC253457
357418DA013FEC86 | D8A31B2F28BBC5CF
12549847013FEC86 | 0F317AC2B23CB944

From the first pair, we compute the S-box inputs (for round 3) from Equations (3.2) and (3.3). They are:

The S-box output x-or is calculated using Equation (3.1) to be:

From the second pair, we compute the S-box inputs to be

and the S-box output x-or is

From the third pair, the S-box inputs are

and the S-box output x-or is

Next, we tabulate the values in the eight counter arrays for each of the three pairs. We illustrate the procedure with the counter array for J1 from the first pair. In this pair, we have [image] and [image]. The set

Since E1 = 000000, we have that

Hence, we increment the values 0, 7, 40, and 47 in the counter array for J1.

The final tabulations are now presented. If we think of a bit-string of length six as being the binary representation of an integer between 0 and 63, then the 64 values correspond to the counts of 0, 1, . . . , 63. The counter arrays are as follows:

J1
1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0
0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 3
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1

J2
0 | 0 | 0 | 1 | 0 | 3 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0
0 | 1 | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0
0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 0
0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 2 | 0 | 0 | 0

J3
0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0
0 | 0 | 0 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1
0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0
0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0

J4
3 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 2 | 0 | 0 | 0 | 0
0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1
1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 0
0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 1

J5
0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0
0 | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
0 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 2 | 0

J6
1 | 0 | 0 | 1 | 1 | 0 | 0 | 3 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1
0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
0 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
1 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0

J7
0 | 0 | 2 | 1 | 0 | 1 | 0 | 3 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0
0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1
0 | 0 | 2 | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 | 1 | 2 | 1 | 1 | 0
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 1

J8
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 1 | 0 | 1
0 | 3 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

In each of the eight counter arrays, there is a unique counter having the value 3. The positions of these counters determine the key bits in J1, . . . , J8. These positions are (respectively): 47, 5, 19, 0, 24, 7, 7, 49. Converting these integers to binary, we obtain J1, . . . , J8:

J1 = 101111
J2 = 000101
J3 = 010011
J4 = 000000
J5 = 011000
J6 = 000111
J7 = 000111
J8 = 110001.

We can now construct 48 bits of the key, by looking at the key schedule for round 3. It follows that K has the form

0001101 | 0110001 | 01?01?0 | 1?00100
0101001 | 0000??0 | 111?11? | ?100011

where parity bits are omitted and "?" denotes an unknown key bit. The complete key (in hexadecimal, including parity bits), is:

1A624C89520DEC46.

3.6.2 An Attack on a 6-round DES

We now describe an extension of these ideas to a probabilistic attack on a 6-round DES. The idea is to carefully choose a pair of plaintexts with a specified x-or, and then to determine the probabilities of a specified sequence of x-ors through the rounds of encryption. We need to define an important concept now.

DEFINITION 3.5 Let n ≥ 1 be an integer. An n-round characteristic is a list of the form

which satisfies the following properties:

1. .
2. Let 1 ≤ i ≤ n, and let Li-1, Ri-1 and [image] be chosen such that [image] and [image]. Suppose Li, Ri and [image] are computed by applying one round of DES encryption. Then the probability that [image] and [image] is precisely pi. (Note that this probability is computed over all possible 48-tuples J = J1 . . . J8.)

The probability of the characteristic is defined to be the product p = p1 × . . . × pn.

REMARK Suppose we choose L0, R0 and [image] so that [image] and [image] and we apply n rounds of DES encryption, obtaining L1, . . . , Ln and R1, . . . ,Rn. Then we cannot claim that the probability that [image] and [image] for all i (1 ≤ i ≤ n) is p1 × . . . × pn. This is because the 48-tuples in the key schedule K1, . . . . , Kn, are not mutually independent. (If these n 48-tuples were chosen independently at random, then the assertion would be true.) But we nevertheless expect p1 × . . . × pn to be a fairly accurate estimate of this probability.

We also need to recognize that the probabilities pi in a characteristic are defined with respect to an arbitrary (but fixed) pair of plaintexts having a specified x-or, where the 48 key bits for one round of DES encryption vary over all 248 possibilities. However, a cryptanalyst is attempting to determine a fixed (but unknown) key. He is going to choose plaintexts at random (such that they have specified x-ors), hoping that the probabilities that the x-ors during the n rounds of encryption agree with the x-ors specified in the characteristic are fairly close to p1, . . . pn, respectively.

As a simple example, we present in Figure 3.10 a 1-round characteristic which was the basis of the attack on the 3-round DES (as before, we use hexadecimal representations). We depict another 1-round characteristic in Figure 3.11.

Let's look at the characteristic in Figure 3.11 in more detail. When f (R0, K1) and [image] are computed, the first step is to expand R0 and [image]. The resulting x-or of the two expansions is

*
* Figure 3.10 A 1-round characteristic

*
* Figure 3.11 Another 1-round characteristic

So the input x-or to S1 is 001100 and the input x-ors for the other seven S-boxes are all 000000. The output x-ors for S2 through S8 will all be 0000. The output x-or for S1 will be 1110 with probability 14/64 (since it can be computed that N1 (001100, 1110) = 14). So we obtain

with probability 14/64. Applying P, we get

which in hexadecimal is 0080820016. When this is x-ored with [image], we get the specified [image] with probability 14/64. Of course [image] always.

The attack on the 6-round DES is based on the 3-round characteristic given in Figure 3.12. In the 6-round attack, we will start with [image], L6R6 and [image], where we have chosen the plaintexts so that [image] and [image]. We can express R6 as follows:

*
* Figure 3.12 A 3-round characteristic

can be expressed in a similar way, and hence we get


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

(Note the similarity with the 3-round attack.)

is known. From the characteristic, we estimate that [image] and [image] with probability 1/16. If this is in fact the case, then the input x-or for the S-boxes in round 4 can be computed by the expansion function to be:

The input x-ors for S2, S5, S6, S7 and S8 are all 000000, and hence the output x-ors are 0000 for these five S-boxes in round 4. This means that we can compute the output x-ors of these five S-boxes in round 6 from Equation (3.4). So, suppose we compute

where each Ci is a bitstring of length four. Then with probability 1/16, it will be the case that [image] and [image] are respectively the output x-ors of S2, S5, S6, S7 and S8 in round 6. The inputs to these S-boxes in round 6 can be computed to be E2, E5, E6, E7 and E8, and [image] and [image], where

and

can be computed from the ciphertexts, as indicated in Figure 3.13.

We would like to determine the 30 key bits in J2, J5, J6, J7 and J8 as we did in the 3-round attack. The problem is that the hypothesized output x-or for round 6 is correct only with probability 1/16. So 15/16 of the time we will obtain random garbage rather than possible key bits. We somehow need to be able to determine the correct key from the given data, 15/16 of which is incorrect. This might not seem very promising, but fortunately our prospects are not as bleak as they initially appear.

DEFINITION 3.6 Suppose [image] and [image]. We say that the pair of plaintexts L0R0 and [image] is right pair with respect to a characteristic if [image] and [image] for all i, 1 ≤ i ≤ n. The pair is defined to be a wrong pair, otherwise.

*
* Figure 3.13 Differential attack on 6-round DES

We expect that about 1/16 of our pairs are right pairs and the rest are wrong pairs with respect to our 3-round characteristic.

Our strategy is to compute Ej, [image], and [image], as described above, and then to determine [image], for j = 2, 5, 6, 7, 8. If we start with a right pair, then the correct key bits for each Jj will be included in the set testj. If the pair is a wrong pair, then the value of [image] will be incorrect, and it seems reasonable to hypothesize that each set testj will be essentially random.

We can often identify a wrong pair by this method: If |testj| = 0, for any j ∈ {2, 5, 6, 7, 8}, then we necessarily have a wrong pair. Now, given a wrong pair, we might expect that the probability that |testj| = 0 for a particular j is approximately 1/5. This is a reasonable assumption since [image] and, as mentioned earlier, the probability that [image] is approximately 1/5. The probability that all five testj's have positive cardinality is estimated to be [image], so the probability that at least one testj has zero cardinality is about .67. So we expect to eliminate about 2/3 of the wrong pairs by this simple observation, which we call the filtering operation. The proportion of right pairs that remain after filtering is approximately


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 3.4

Suppose we have the following plaintext-ciphertext pair:

plaintext | ciphertext
86FA1C2B1F51D3BE | 1E23ED7F2F553971
C6F21C2B1B51D3BE | 296DE2B687AC6340

Observe that [image] and [image]. The S-box inputs and outputs for round 6 are computed to be the following:

Then, the sets testj are as follows:

We see that both test5 and test7 are empty sets, so this pair is a wrong pair and is discarded by the filtering operation.

Now suppose that we have a pair such that |testj| > 0 for j = 2, 5, 6, 7, 8, so that it survives the filtering operation. (Of course, we do not know if the pair is a right pair or a wrong pair.) We say that the bitstring J2J5J6J7J8 of length 30 is suggested by the pair if Jj ∈ testj for j = 2, 5, 6, 7, 8. The number of suggested bitstrings is

It is not unusual for the number of suggested bitstrings to be quite large (for example, greater than 80000).

Suppose we were to tabulate all the suggested bitstrings obtained from the N pairs that were not discarded by the filtering operation. For every right pair, the correct bitstring J2J5J6J7J8 will be a suggested bitstring. This correct bitstring will be counted about 3N/16 times. Incorrect bitstrings should occur much less often, since they will occur essentially at random and there are 230 possibilities (a very large number).

It would get extremely unwieldy to tabulate all the suggested bitstrings, so we use an algorithm that requires less space and time. We can encode any testj as a vector Tj of length 64, where the ith coordinate of Tj is set to 1 (for 0 ≤ i ≤ 63) if the bitstring of length six that is the binary representation of i is in the set testj; and the ith coordinate is set to 0 otherwise (this is essentially the same as the counter array representation that we used in the 3-round attack).

For each remaining pair, construct these vectors as described above, and name them [image], j = 2, 5, 6, 7, 8, 1 ≤ i ≤ N. For I ⊆ {1, . . . , N}, we say that I is allowable if for each j ∈ {2, 5, 6, 7, 8}, there is at least one coordinate equal to |I| in the vector

If the ith pair is a right pair for every i ∈ I, then the set I is allowable. Hence, we expect there to be an allowable set of size (approximately) 3N/16, which we hope will suggest the correct key bits and no other. It is a simple matter to construct all the allowable sets I by means of a recursive algorithm.

Example 3.5

We did some computer runs to test this approach. A random sample of 120 pairs of plaintexts with the specified x-ors was generated, and these were encrypted using the same (random) key. We present the 120 pairs of ciphertexts and corresponding plaintexts in hexadecimal form in Table 3.1.

When we compute the allowable sets, we obtain ni allowable sets of cardinality i, for the following values:

The unique allowable set of size 10 is

In fact, it does arise from the 10 right pairs. This allowable set suggests the correct key bits for J2, J5, J6, J7 and J8 and no others. They are as follows:

*
* Figure 3.14 Another 3-round characteristic

Note that all the allowable sets of cardinality at least 6, and all but three of the allowable sets of cardinality 5, arise from right pairs, since [image] and [image] for 6 ≤ i ≤ 10.

This method yields 30 of the 56 key bits. By means of a different 3-round characteristic, presented in Figure 3.14, it is possible to compute 12 further key bits, namely those in J1 and J4. Now only 14 key bits remain unknown. Since 214 = 16384 is quite small, an exhaustive search can be used to determine the remaining 14 key bits.

The entire key (in hexadecimal, including parity-check bits) is:

34E9F71A20756231.

As mentioned above, the 120 pairs are given in Table 3.1. In the second column, a * denotes that a pair is a right pair, while a ** denotes that the pair is an identifiable wrong pair and is discarded by the filtering operation. Of the 120 pairs, 73 are identified as being wrong pairs by the filtering process, so 47 pairs remain as "possible" right pairs.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

3.6.3 Other examples of Differential Cryptanalysis

Differential cryptanalysis techniques can be used to attack DES with more than six rounds. An 8-round DES requires 214 chosen plaintexts, and 10-, 12-, 14- and 16-round DESs can be broken with 224, 231, 239 and 247 chosen plaintexts, respectively. The attacks on more than 10 rounds are probably not practical at this time.

Several substitution-permutation product ciphers other than DES are also susceptible (to varying degrees) to differential cryptanalysis. These cryptosystems include several substitution-premutation cryptosystems that have been proposed in recent years, such as FEAL, REDOC-II, and LOKI.

Table 3.1 Cryptanalysis of 6-round DES
pair right pair? plaintext ciphertext
1 ** 86FA1C2B1F51D3BE
C6F21C2B1B51D3BE 		1E23ED7F2F553971
296DE2B687AC6340
2 ** EDC439EC935E1ACD
ADCC39EC975E1ACD 		0F847EFE90466588
93E84839F374440B
3 ** 9468A0BE00166155
D460A0BE04166155 		3D6A906A6566D0BF
3BC3B236398379E1
4 ** D4FF2B18A5A8AAC8
94F72B18A1A8AAC8 		26B14738C2556BA4
15753FDE86575A8F
5 09D0F2CF277AF54F
49D8F2CF237AF54F 		15751F4F11308114
6046A7C863F066AF
6 CBC7157240D415DF
8BCF157244D415DF 		7FCDC300FB9698E5
522185DD7E47D43A
7 0D4A1E84890981C1
4D421E848D0981C1 		E7C0B01E32557558
912C6341A69DF295
8 ** 6CE6B2A9B8194835
2CEEB2A9BC194835 		75D52E028A5C48A3
6C88603B48E5A8CE
9 ** 799F63C3C9322C1A
399763C3CD322C1A 		A6DA322B8F2444B5
6634AA9DF18307F4
10 ** 1B36645E381EDF48
5B3E645E3C1EDF48 		1F91E295D559091B
D094FC12C02C17CA
11 85CA13F50B4ADBB9
C5C213F50F4ADBB9 		ED108EE7397DDE0A
3F405F4A3E254714
12 ** 7963A8EFD15BC4A1
396BA8EFD55BC4A1 		8C714399715A33BA
C344C73CC97E4AC4
13 7BCFF7BCA455E65E
3BC7F7BCA055E65E 		475A2D0459BCCE62
8E94334AEF359EF8
14 0C505CEDB499218C
4C585CEDB099218C 		D3C66239E89CC076
9A316E801EE18EB1
15 6C5EA056CDC91A14
2C56A056C9C91A14 		BC7EBA159BCA94E6
67DB935C21FF1A8D
16 ** 6622A441A0D32415
262AA441A4D32415 		35F8616FEBA62883
4313E1925F5B64BC
17 C0333C994AFF1C99
803B3C994EFF1C99 		D46A4CF1C0221B11
D22B42DB150E2CE8
18 9E7B2974F00E1A6E
DE732974F40E1A6E 		172D286D9606E6FE
2217A91F8C427D27
19 ** CF592897BFD70C7E
8F512897BBD70C7E 		FB892B59E7DCE7EC
C328B765E1CC6653
20 E976CF19124A9FA1
A97ECF19164A9FA1 		905BF24188509FA6
9ADDBA0C23DD724F
21 ** 5C09696E7363675D
1C01696E7763675D 		92D60E5C71801A99
DD90908A4FE8168F
22 ** A8145AB3C1B2C7DE
E81C5AB3C5B2C7DE 		F68FC9F80564847B
51C041B5711B8132
23 47DF6A0BB1787159
07D76A0BB5787159 		52E36C4CA22EA5A2
373EAFD503F68DE4
24 * 7CE65464329B4E6D
3CEE5464369B4E6D 		832A9D7032015D9F
85E2CE665571E99C
25 ** 421FB6AD95791BA7
0217B6AD91791BA7 		D1E730BA1DB565E7
188E61735FA4F3CE
26 ** C58E9A361368FFD6
85869A361768FFD6 		795EB9D30CAE6879
26D37AC4867ACC61
27 ** DD86B6C74C8EA4E2
9D8EB6C7488EA4E2 		CC3B6915C9A348DF
104C2394555645F0
28 ** 43DB9D2F483CA585
03D39D2F4C3CA585 		E3E4DA503D1B9396
4EA02C0061332443
29 * 855A309F96FEA5EA
C552309F92FEA5EA 		85AD6E9E352AFAFA
929D22370ACAB80D
30 * AB3CA25B02BD18C8
EB34A25B06BD18C8 		0F7D768E9203F786
A1313BC26A99D353
31 ** A9F7A6F4A7C00E06
E9FFA6F4A3C00E06 		F26B385E6BA057FD
203D8384F8F54D19
32 ** 688B9ACD856D1312
28839ACD816D1312 		C41D99C107B4EF76
6CC817CA025A7DAC
33 ** 76BF0621C03D4CD9
36B70621C43D4CD9 		BBE1F95AFC1E052A
561F4801F2EB0C63
34 ** 014CF8D1F981B8EE
4144F8D1FD81B8EE 		D27091C4314CBFE8
B7976D6A80E3DB61
35 ** 487D66EDE0405F8C
087566EDE4405F8C 		8136325C0AEB84CE
8C638BC4495B69A0
36 ** DDCA47093A362521
9DC247093E362521 		51040CF16B600FAA
7FC75515AC3CAAF9
37 ** 45A9D34A3996F6D9
05A1D34A3D96F6D9 		F2004B854AE6C46C
546825016B03D193
38 ** 295D2FBFB00875EA
69552FBFB40875EA 		A309DF027E69C265
4F633FFB95A0C11E
39 964C8B98D590D524
D6448B98D190D524 		1FF1D0271D6F6C18
8CF2D8D401EBFC0F
40 60383D2BAF0836BC
20303D2BAB0836BC 		10A82D55FC480640
602346173581EF79
41 ** 5CF8D539A22A1CAD
1CF0D539A62A1CAD 		92685D806FBE8738
17006DAB2D28081C
42 F95167CAB6565609
B95967CAB2565609 		C52E2EB27446054E
0C219F686840E57A
43 49F1C83615874122
09F9C83611874122 		2680C8ECDF5E51CD
5022A7B69B4E75EF
44 ** ACB2EC1941B03765
ECBAEC1945B03765 		D6B593460098DEC5
D3190A0200FC6B9B
45 CCCC129D5CB55EC0
8CC4129D58B55EC0 		3AD22B7EF59E0D5E
A48C92CBEC17E430
46 ** 917FF8E2EE6B78D5
D177F8E2EA6B78D5 		EF847E058DB71724
F243F0554A00E4C5
47 ** 51DBCF028E96DE00
11D3CF028A96DE00 		574897CA1EE73885
9F0FD0A5B2C2B5FD
48 * 2094942E093463CE
609C942E0D3463CE 		59F6A018C6A0D820
799FE001432346C0
49 ** 50FB0723D7CD1081
10F30723D3CD1081 		16AF758395EA3A7D
CDCB23392D144BED
50 * 740815A4F6CDCABB
340015A4F2CDCABB 		4A84D2ED4D9351AB
5923D04CE94D6111
51 ** EDA46A1AE93735DC
ADAC6A1AED3735DC 		0B302A51B7E5476A
5F817F0ABC770E75
52 * 08BC39B766B2C128
48B439B762B2C128 		DFB5F3F500BC0100
B7B9FED8AC93EBFA
53 ** A74E29BBA98F2312
E74629BBAD8F2312 		A2B352B7F922E8DA
D6BC4B89CED2DEAC
54 ** D6F50D31EE4E68AB
96FD0D31EA4E68AB 		4D464847065C0938
7554D87AEDCE5634
55 * 06191AA594891CF5
46111AA590891CF5 		649C1D084F920F9E
BE12A10384365E19
56 5EA7EFD557946962
1EAFEFD553946962 		15E664293F4D77EE
E23396A758DC9CE6
57 ** 41FB7704781CC88A
01F377047C1CC88A 		8ABD385C441FD6CE
06DE8D55777AB65C
58 ** 9689B9123F7C5431
D681B9123B7C5431 		E1E63120742099BB
1AF88A2CF6649A4A
59 6F25032B4A309BFE
2F2D032B4E309BFE 		48FE50DE774288D7
47950691260D5E10
60 ** D8C4B02D8E8BF1E9
98CCB02D8A8BF1E9 		F34D565E6AE85683
A4D2DB548622A8E8
61 ** F663E8CCEE86805B
B66BE8CCEA86805B 		51BD62C9D5D0F0BB
D2ABB03CF9D26C0A
62 ** 428B29BFDFA838DB
028329BFDBA838DB 		006D62A65761089F
9FD73EF6124B0C11
63 ** 04BE2D22D81EDC66
44B62D22DC1EDC66 		26D99536D99B5707
94144EBDA0CDEB55
64 ** 667B779123A3EF80
2673779127A3EF80 		5D09CBF2CE7E5A69
5EFF8BFCA7BAA152
65 ** BC86D401D6572438
FC8ED401D2572438 		E05572AAA5F6C377
3C670BC455144F61
66 ** 6FE5E9547659E401
2FEDE9547259E401 		2C465BF6F52F864C
B71D106444F95F31
67 ** 27D3BAC6453BE3DE
67DBBAC6413BE3DE 		8F160E29000461CD
2A6660F46487F885
68 ** 1D864E7642A7023A
5D8E4E7646A7023A 		65F91EEBFD8A9C05
84761791B3C36661
69 ** 5256CA6894707CBA
125ECA6890707CBA 		91527F9349ABCF15
30F28F06A7B0A35A
70 ** C05383B8EFCD2BD7
805B83B8EBCD2BD7 		710B6EC61BF63E9C
53AC029D8E0179D5
71 50EB21CA13F9A96E
10E321CA17F9A96E 		26D95BA4DE4C85CF
8F01A90F638AFFF6
72 ** 60EB1229ACD90EDC
20E31229A8D90EDC 		3890EE8567782F96
EE404DF7BE537589
73 8E9A17D17B173B99
CE9217D17F173B99 		885C3933627EDEF0
B7ABB6DF5835E962
74 6EC8CD0802C98817
2ECDCD0806C98817 		A985ADFB1FEE013C
0428DE024B7E4604
75 ** 1E81712FF1145C06
5E89712FF5145C06 		417E667A99B3CFA5
5C24AA056EB1ADBA
76 ** DF3C5C13311AEC7C
9F345C13351AEC7C 		BF01675096F1C48A
243D99BCE12DB864
77 ** 7C34472994127C2D
3C3C472990127C2D 		713915DA311A7CF4
E9733D11D787E20B
78 ** 37304DABA75EAFB3
77384DABA35EAFB3 		EFB5C37FA0238ADF
A728F7407AF958B3
79 D03A16E4C2D8B54B
903216E4C6D8B54B 		423FC0AC24CEFEDD
047D8595DB4D372E
80 ** 8CED882B5D91832E
CCE5882B5991832E 		0006E2DE3AF5C2B5
00F6AA9ED614001B
81 ** 1BB0E6C79EFBEC41
5BB8E6C79AFBEC41 		E9AED4363915775A
655BC48F1FFB5165
82 D41B8346DA9E2252
94138346DE9E2252 		34F5E0BCC5B042EA
702D2C48CDBE5173
83 * 02A9D0A0A91F6304
42A1D0A0AD1F6304 		E2F1C10E59AF07C5
BDEE6AA00F25F840
84 ** 841B3E27C8F0A561
C4133E27CCF0A561 		2B288E554D712C92
FF8609C9E7301162
85 ** CDF0A8D6EE909185
8DF8A8D6EA909185 		5D661834D1C76324
22034D57D21FFB56
86 ** 4C31AC854F44EA34
0C39AC854B44EA34 		BD016309AEDB9BB1
C72EEDC4FA1D9312
87 DB3FC0703C972930
9B37C07038972930 		296ABCFBF01DF991
CA4700686F9F83A2
88 E4B362BFD6A7CFD1
A4BB62BFD2A7CFD1 		20FDAF335F25B1DA
008C24D75E14ACBD
89 F234232A0E0A4A28
B23C232A0A0A4A28 		90CFD699F2DEC5BD
2918D3DE0C1B689C
90 ** 71265345A5874004
312E5345A1874004 		3052CE3CE88710AE
38F0FC685DF30564
91 ** 3E6364548C857110
7E6B645488857110 		0E8581E42C9FEC6F
4DD1751861EC5529
92 * 464FBEDBD78900A7
0647BEDBD38900A7 		90F5F9ADEDED627A
2EF4C540425E339B
93 ** 373B75F847480BB0
773375F843480BB0 		5408B964F8442D16
805287D52599E9F0
94 ** D714E87810DE97AC
971CE87814DE97AC 		4EC4D623108FA909
0AA0725CED10D6A3
95 B9B5932EF54B2C60
F9BD932EF14B2C60 		4B438B3CCF36DEC9
054C6A337709280D
96 ** 2F283C38D2E4E1DD
6F203C38D6E4E1DD 		83515FB6DFEA90B8
09BCC4FF38C78C23
97 ** 1EB8ADAA43BBD575
5EB0ADAA47BBD575 		21A1E04813616E42
D044BA3F25DFD02A
98 ** 3164AA5454D9F991
716CAA5450D9F991 		9382C6C1883F1038
5CDFED4FF2117DEC
99 D78C1C5C6F2243D2
97841C5C6B2243D2 		1CCEB091E030E6A6
4DA2CD67CC449B21
100 BBE212A7D3CE3D14
FBEA12A7D7CE3D14 		2917C207B4D93E0D
A01D50E5A2B902D8
101 ** 104917795E98D0FB
504117795A98D0FB 		40916A71385C2803
413FD26EF671F46D
102 ** 4DDA114D6EFEEEB4
0DD2114D6AFEEEB4 		2E2C65E1D5CBAC31
A16FF03BC0913ED6
103 E0BED7B285BF0A77
A0B6D7B281BF0A77 		5D9EFEFF0AD10490
4C6CA1FAC36A8E5B
104 ** 0AE1555FA1716214
4AE9555FA5716214 		378400BCED39EB81
A1E0C758BD8912C2
105 ** 4657C26790FCB354
065FC26794FCB354 		588BA079B2E7ED20
DA90827AEED7A41F
106 ** 32BD719B0DC1B091
72B5719B09C1B091 		F3477C7552BCB05D
EFF444449D66BE9E
107 ** 0992F8C8C73A9BFE
499AF8C8C33A9BFE 		9F3FFD0F158295F6
C138358DCECC8FC7
108 02C3F061A237BBEB
42CBF061A637BBEB 		AC28B0307127EA7C
3FF1DAED9E0FCBC5
109 ** 80E529E69EDE6827
C0ED29E69ADE6827 		1DF1DB7B66BA1AF1
15700151A5804549
110 B55E84630067B8D5
F55684630467B8D5 		88321611FF9DA421
90649DTEACF91F9A
111 2749C2EBC603BFF2
6741C2EBC203BFF2 		A62B23A7348E2C3A
EB760A09C7FF5153
112 ** C4C5E14D4C5D9FF5
84CDE14D485D9FF5 		ABC2312FBFD94DF5
D2BB5954E5062D53
113 ** 1566BA21F2647E18
556EBA21F6647E18 		A247ED988457CB78
5E99F231005F5249
114 ** 2D093D426D922F92
6D013D4269922F92 		5DF62030B9F23AE9
5D92DA1FA3D07BA1
115 004518468E0C96C3
404D18468A0C96C3 		F28D85FF7E84F38F
52541B0443053C57
116 ** 437B70A98AE03344
037370A98EE03344 		04B3FBF9823B4CF7
14EBEC79DAD3093E
117 2D01F1073D3E375B
6D09F107393E375B 		F10B3E1EE356226C
6FF26DA5E3525B62
118 * 66573DD7E0D7F110
265F3DD7E4D7F110 		F2F26204C29FE51E
083A4ECE57E429AC
119 0846DB9538155201
484EDB953C155201 		F120D0D2AE788057
00CC914A33034782
120 ABB34FC195C820D1
EBBB4FC191C820D1 		5F17AE066B50FC81
2858DD63A2FA4B53


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

3.7 Notes and References

A nice article on the history DES is by Smid and Branstad [SB92]. Federal Information Processing Standards (FIPS) publications concerning DES include the following: description of DES [NBS77]; implementing and using DES [NBS81]; modes of operation of DES [NBS80]; and authentication using DES [NBS85].

Some properties of the S-boxes are studied by Brickell, Moore, and Purtill [BMP87].

The DEC DES chip is described in [EB93]. Wiener's key search machine was described at CRYPTO '93 [WI94].

The time-memory trade-off for DES is due to Hellman [HE80]. A more general time-memory trade-off is presented by Fiat and Naor in [FN91].

The technique of differential cryptanalysis was developed by Biham and Shamir [BS91] (see also [BS93A] and their book [BS93], where cryptanalysis of other cryptosystems is also discussed). Our treatment of differential cryptanalysis is based largely on [BS93].

Another new method of cryptanalysis that can be used to attack DES and other similar cryptosystems is the linear cryptanalysis of Matsui [MA94, MA94A].

Descriptions of other substitution-permutation cryptosystems can be found in the following sources: LUCIFER [FE73]; FEAL [MI9 1]; REDOC-II [CW91]; and LOKI [BKPS90].

Exercises

3.1 Prove that DES decryption can be done by applying the DES encryption algorithm to the ciphertext with the key schedule reversed.
3.2 Let DES(x, K) represent the encryption of plaintext x with key K using the DES cryptosystem. Suppose y = DES (x, K) and y′ = DES (c(x), c(K)), where c(·) denotes the bitwise complement of its argument. Prove that y′ = c(y) (i.e., if we complement the plaintext and the key, then the ciphertext is also complemented). Note that this can be proved using only the "high-level" description of DES — the actual structure of S-boxes and other components of the system are irrelevant.
3.3 One way to strengthen DES is by double encryption: Given two keys, K1 and K2, define [image] (of course, this is just the product of DES with itself). If it happened that the encryption function [image] was the same as the decryption function [image], then K1 and K2 are said to be dual keys. (This is very undesirable for double encryption, since the resulting ciphertext is identical to the plaintext.) A key is self-dual if it is its own dual key.

(a) Prove that if C0 is either all 0's or all 1's and D0 is either all 0's or all 1's, then K is self-dual.
(b) Prove that the following keys (given in hexadecimal notation) are self-dual:
0101010101010101
FEFEFEFEFEFEFEFE
1F1F1F1F0E0E0E0E
E0E0E0E0F1F1F1F1
(c) Prove that if C0 = 0101 . . . 01 or 1010 . . . 10 (in binary), then the x-or of the bitstrings Ci and C17-i is 1111 . . . 11, for 1 ≤ i ≤ 16 (a similar statement holds for the Di's).
(d) Prove that the following pairs of keys (given in hexadecimal notation) are dual:
E001E001F101F101
FE1FFE1FFE0EFE0E
E01FE01FF10EF10E 		01E001E001F101F1
1FFE1FFE0EFE0EFE
1FE01FE00EF10EF1

3.4 A message authentication code (MAC) can be produced by using CFB mode, as well as by using CBC mode. Given a sequence of plaintext blocks x1 . . . xn, suppose we define the initialization vector IV to be x1. Then encrypt x2 . . . xn using key K in CFB mode, obtaining y1 . . . yn-1 (note that there are only n - 1 ciphertext blocks). Finally, define the MAC to be eK(yn-1). Prove that this MAC is identical to the MAC produced in Section 3.4.1 using CBC mode.
3.5 Suppose a sequence of plaintext blocks, x1, . . . xn, is encrypted using DES, producing ciphertext blocks y1 . . . yn. Suppose that one ciphertext block, say yi, is transmitted incorrectly (i.e., some 1's are changed to 0's and vice versa). Show that the number of plaintext blocks that will be decrypted incorrectly is equal to one if ECB or OFB modes were used for encryption; and equal to two if CBC or CFB modes were used.
3.6 The purpose of this question is to investigate a simplified time-memory trade-off for a chosen plaintext attack. Suppose we have a cryptosystem in which [image], which attains perfect secrecy. Then it must be the case that [image] implies K = K1. Denote [image]. Let x be a fixed plaintext. Define the function g : Y → Y by the rule g(y) = ey(x). Define a directed graph G having vertex set Y, in which the edge set consists of all the directed edges of the form (yi, g(yi)), 1 ≤ i ≤ N.

(a) Prove that G consists of the union of disjoint directed cycles.
(b) Let T be a desired time parameter. Suppose we have a set of elements Z = {z1, . . . , zm} ⊆ Y such that, for every element yi ∈ Y, either yi is contained in a cycle of length at most T, or there exists an element zj ≠ yi such that the distance from yi to zj (in G) is at most T. Prove that there exists such a set Z such that

so |Z| is O(N/T)

(c) For each zj ∈ Z, define g-T (zj) to be the element yi such that gT (yi) = zj, where gT is the function that consists of T iterations of g. Construct a table X consisting of the ordered pairs (zj, g-T (zj)), sorted with respect to their first coordinates.

A pseudo-code description of an algorithm to find K, given y = eK (x), is presented in Figure 3.15. Prove that this algorithm finds K in at most T steps. (Hence the time-memory trade-off is O(N).)

*
* Figure 3.15 Time-memory trade-off

*
* Figure 3.16 Differential attack on 4-round DES

(d) Describe a pseudo-code algorithm to construct the desired set Z in time O(NT) without using an array of size N.

3.7 Compute the probabilities of the following 3-round characteristic:

3.8 Here is a differential attack on a 4-round DES. It uses the following characteristic, which is a special case of the characteristic presented in Figure 3.10:

(a) Suppose that the following algorithm presented in Figure 3.16 is used to compute sets test2, . . . test8. Show that Jj ∈ testj for 2 ∈ j ∈ 8.
(b) Given the following plaintext-ciphertext pairs, find the key bits J2, . . . , J8.
plaintext | ciphertext
18493AC485B8D9A0 | E332151312A18B4F
38493AC485B8D9A0 | 87391C27E5282161
482765DDD7009123 | B5DDD8339D82D1D1
682765DDD7009123 | 81F4B92BD94B6FD8
ABCD098733731FF1 | 93A4B42F62EA59E4
8BCD098733731FF1 | ABA494072BF411E5
13578642AAFFEDCB | FDEB526275FB9D94
33578642AAFFEDCB | CC8F72AAE685FDB1
(c) Compute the entire key (14 key bits remain to be determined, which can be done by exhaustive search).


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 4
The RSA System and Factoring

4.1 Introduction to Public-key Cryptography

In the classical model of cryptography that we have been studying up until now, Alice and Bob secretly choose the key K. K then gives rise to an encryption rule eK and a decryption rule dK. In the cryptosystems we have seen so far, dK is either the same as eK, or easily derived from it (for example, DES decryption is identical to encryption, but the key schedule is reversed). Cryptosystems of this type are known as private-key systems, since exposure of eK renders the system insecure.

One drawback of a private-key system is that it requires the prior communication of the key K between Alice and Bob, using a secure channel, before any ciphertext is transmitted. In practice, this may be very difficult to achieve. For example, suppose Alice and Bob live far away from each other and they decide that they want to communicate electronically, using e-mail. In a situation such as this, Alice and Bob may not have access to a reasonable secure channel.

The idea behind a public-key system is that it might be possible to find a cryptosystem where it is computationally infeasible to determine dK given eK. If so, then the encryption rule eK could be made public by publishing it in a directory (hence the term public-key system). The advantage of a public-key system is that Alice (or anyone else) can send an encrypted message to Bob (without the prior communication of a secret key) by using the public encryption rule eK. Bob will be the only person that can decrypt the ciphertext, using his secret decryption rule dK.

Consider the following analogy: Alice places an object in a metal box, and then locks it with a combination lock left there by Bob. Bob is the only person who can open the box since only he knows the combination.

The idea of a public-key system was due to Diffie and Hellman in 1976. The first realization of a public-key system came in 1977 by Rivest, Shamir, and Adleman, who invented the well-known RSA Cryptosystem which we study in this chapter. Since then, several public-key systems have been proposed, whose security rests on different computational problems. Of these, the most important are the following:

RSA
The security of RSA is based on the difficulty of factoring large integers. This system is described in Section 4.3.
Merkle-Hellman Knapsack
This and related systems are based on the difficulty of the subset sum problem (which is NP-complete1); however, all of the various knapsack systems have been shown to be insecure (with the exception of the Chor-Rivest Cryptosystem mentioned below). See Chapter 5 for a discussion of this cryptosystem.
1The NP-complete problems are a large class of problems for which no polynomial-time algorithms are known.
McEliece
The McEliece Cryptosystem is based on algebraic coding theory and is still regarded as being secure. It is based on the problem of decoding a linear code (which is also NP-complete). (See Chapter 5.)
ElGamal
The ElGamal Cryptosystem is based on the difficulty of the discrete logarithm problem for finite fields. (See Chapter 5.)
Chor-Rivest
This is also referred to as a "knapsack" type system, but it is still regarded as being secure.
Elliptic Curve
The Elliptic Curve Cryptosystems are modifications of other systems (such as the ElGamal Cryptosystem, for example) that work in the domain of elliptic curves rather than finite fields. The Elliptic Curve Cryptosystems appear to remain secure for smaller keys than other public-key cryptosystems. (See Chapter 5.)

One very important observation is that a public-key cryptosystem can never provide unconditional security. This is because an opponent, on observing a ciphertext y, can encrypt each possible plaintext in turn using the public encryption rule eK until he finds the unique x such that y = eK(x). This x is the decryption of y. Consequently, we study the computational security of public-key systems.

It is helpful conceptually to think of a public-key system in terms of an abstraction called a trapdoor one-way function. We informally define this notion now.

Bob's public encryption function, eK, should be easy to compute. We have just noted that computing the inverse function (i.e., decrypting) should be hard (for anyone other than Bob). This property of being easy to compute but hard to invert is often called the one-way property. Thus, we desire that eK be an (injective) one-way function.

One-way functions play a central role in cryptography; they are important for constructing public-key cryptosystems and in various other contexts. Unfortunately, although there are many functions that are believed to be one-way, there currently do not exist functions that can be proved to be one-way.

Here is an example of a function which is believed to be one-way. Suppose n is the product of two large primes p and q, and let b be a positive integer. Then define [image] to be

(For a suitable choice of b and n, this is in fact the RSA encryption function; we will have much more to say about it later.)

If we are to construct a public-key cryptosystem, then it is not sufficient to find a one-way function. We do not want eK to be a one-way function from Bob's point of view, since he wants to be able to decrypt messages that he receives in an efficient way. Thus, it is necessary that Bob possesses a trapdoor, which consists of secret information that permits easy inversion of eK. That is, Bob can decrypt efficiently because he has some extra secret knowledge about K. So, we say that a function is a trapdoor one-way function if it is a one-way function, but it becomes easy to invert with the knowledge of a certain trapdoor.

We will see in Section 4.3 how to find a trapdoor for the function f defined above. This will lead to the RSA Cryptosystem.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

4.2 More Number Theory

Before describing how RSA works, we need to discuss some more facts concerning modular arithmetic and number theory. Two fundamental results that we require are the Euclidean algorithm and the Chinese remainder theorem.

4.2.1 The Euclidean Algorithm

We already observed in Chapter 1 that [image] is a ring for any positive integer n. We also proved there that [image] has a multiplicative inverse if and only if gcd(b, n) = 1, and that the number of positive integers less than n and relatively prime to n is φ(n).

The set of residues modulo n that are relatively prime to n is denoted [image]. It is not hard to see that [image] forms an abelian group under multiplication. We already have stated that multiplication modulo n is associative and commutative, and that 1 is the multiplicative identity. Any element in [image] will have a multiplicative inverse (which is also in [image]). Finally, [image] is closed under multiplication since xy is relatively prime to n whenever x and y are relatively prime to n (prove this!).

At this point, we know that any [image] has a multiplicative inverse, b-1, but we do not yet have an efficient algorithm to compute b-1. Such an algorithm exists; it is called the extended Euclidean algorithm.

First, we describe the Euclidean algorithm, in its basic form, which is used to compute the greatest common divisor of two positive integers, say r0 and r1, where r0 > r1. The Euclidean algorithm consists of performing the following sequence of divisions:

Then it is not hard to show that

Hence, it follows that gcd(r0, r1) = rm.

Since the Euclidean algorithm computes greatest common divisors, it can be used to determine if a positive integer b < n has a multiplicative inverse modulo n, by starting with r0 = n and r1 = b. However, it does not compute the value of the multiplicative inverse (if it exists).

Now, suppose we define a sequence of numbers t0, t1, . . ., tm according to the following recurrence (where the qj's are defined as above):

Then we have the following useful result.

THEOREM 4.1

For 0 ≤ j ≤ m, we have that rj ≡ tjr1 (mod r0), where the qj's and rj's are defined as in the Euclidean algorithm, and the tj's are defined in the above recurrence.

PROOF The proof is by induction on j. The assertion is trivially true for j = 0 and j = 1. Assume the assertion is true for j = i - 1 and i - 2, where i ≥ 2; we will prove the assertion is true for j = i. By induction, we have that

and

Now, we compute:

Hence, the result is true by induction.

The next corollary is an immediate consequence.

COROLLARY 4.2

Suppose gcd(r0, r1) = 1. Then tm = r1-1 mod r0.

Now, the sequence of numbers t0 , t1, . . . tm can be calculated in the Euclidean algorithm at the same time as the qj's and the rj's. In Figure 4.1, we present the extended Euclidean algorithm to compute the inverse of b modulo n, if it exists. In this version of the algorithm, we do not use an array to keep track of the qj's, rj's and tj's, since it suffices to remember only the "last" two terms in each of these sequences at any point in the algorithm.

In step 10 of the algorithm, we have written the expression for temp in such a way that the reduction modulo n is done with a positive argument. (We mentioned earlier that modular reductions of negative numbers yield negative results in many computer languages; of course, we want to end up with a positive result here.) We also mention that at step 12, it is always the case that tb ≡ r (mod n) (this is the result proved in Theorem 4.1).

Here is a small example to illustrate:

Example 4.1

Suppose we wish to compute 28-1 mod 75. The Extended Euclidean algorithm proceeds as follows:

Hence, 28-1 mod 75 = 67.

*
* Figure 4.1 Extended Euclidean algorithm


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

4.2.2 The Chinese Remainder Theorem

The Chinese remainder theorem is really a method of solving certain systems of congruences. Suppose m1, . . . , mr are pairwise relatively prime positive integers (that is, gcd (mi, mj) = 1 if i ≠ j). Suppose a1, . . ., ar are integers, and consider the following system of congruences:

The Chinese remainder theorem asserts that this system has a unique solution modulo M = m1 × m2 × . . . × mr. We will prove this result in this section, and also describe an efficient algorithm for solving systems of congruences of this type.

It is convenient to study the function [image], which we define as follows:

Example 4.2

Suppose r = 2, m1 = 5 and m2 = 3, so M = 15. Then the function π has the following values:

Proving the Chinese remainder theorem amounts to proving that this function π we have defined is a bijection. In Example 4.2 this is easily seen to be the case. In fact, we will be able to give an explicit general formula for the inverse function π-1.

For 1 ≤ i ≤ r, define

Then it is not difficult to see that

for 1 ≤ i ≤ r. Next, for 1 ≤ i ≤ r, define

(This inverse exists since gcd(Mi, mi) = 1, and it can be found using the Euclidean algorithm.) Note that

for i ≤ i ≤ r.

Now, define a function [image] follows:

We will show that the function ρ = π-1, i.e., it provides an explicit formula for solving the original system of congruences.

Denote X = ρ(a1, . . ., ar), and let 1 ≤ j ≤ r. Consider a term aiMiyi in the above summation, reduced modulo mj: If i = j, then

since

On the other hand, if i ≠ j, then

since mj | Mi in this case. Thus, we have that

Since this is true for all j, 1 ≤ j ≤ r, X is a solution to the system of congruences.

At this point, we need to show that the solution X is unique modulo M. But this can be done by simple counting. The function π is a function from a domain of cardinality M to a range of cardinality M. We have just proved that π is a surjective (i.e., onto) function. Hence, π must also be injective (i.e., one-to-one), since the domain and range have the same cardinality. It follows that π is a bijection and π-1 = ρ. Note also that π-1 is a linear function of its arguments a1, . . . , ar.

Here is a bigger example to illustrate.

Example 4.3

Suppose r = 3, m1 = 7, m2 = 11 and m3 = 13. Then M = 1001. We compute M1 = 143, M2 = 91 and M3 = 77, and then y1 = 5, y2 = 4 and y3 = 12. Then the function [image] is the following:

For example, if x ≡ 5 (mod 7), x ≡ 3 (mod 11) and x ≡ 10 (mod 13), then this formula tells us that

This can be verified by reducing 894 modulo 7, 11 and 13.

For future reference, we record the results of this section as a theorem.

THEOREM 4.3 (Chinese Remainder Theorem)

Suppose m1, . . ., mr are pairwise relatively prime positive integers, and suppose a1, . . ., ar are integers. Then, the system of r congruences x ≡ ai (mod mi) (1 ≤ i ≤ r) has a unique solution modulo M = m1 × . . . × mr, which is given by

where Mi = M/mi and yi = Mi-1 mod mi, for 1 ≤ i ≤ r.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

4.2.3 Other Useful Facts

We next mention another result from elementary group theory, called Lagrange's Theorem, that will be relevant in our treatment of the RSA Cryptosystem. For a (finite) multiplicative group G, define the order of an element g ∈ G to be the smallest positive integer m such that gm = 1. The following result is fairly simple, but we will not prove it here.

THEOREM 4.4 (Lagrange)

Suppose G is a multiplicative group of order n, and g∈ G. Then the order of g divides n.

For our purposes, the following corollaries are essential.

COROLLARY 4.5

If [image], then bφ(n) ≡ 1 (mod n).

PROOF [image] is a multiplicative group of order φ(n).

COROLLARY 4.6 (Fermat)

Suppose p is prime and [image]. Then bp ≡ b (mod p).

PROOF If p is prime, then φ(p) = p - 1. So, for [image] (mod p), the result follows from Corollary 4.5. For b ≡ 0 (mod p), the result is also true since 0p ≡ 0 (mod p).

At this point, we know that if p is prime, then [image] is a group of order p - 1, and any element in [image] has order dividing p - 1. However, if p is prime, then the group [image] is in fact cyclic: there exists an element [image] having order equal to p - 1. We will not prove this very important fact, but we do record it for future reference:

THEOREM 4.7

If p is prime, then [image] is a cyclic group.

An element α having order p - 1 is called a primitive element modulo p. Observe that α is a primitive element if and only if

Now, suppose p is prime and α is a primitive element modulo p. Any element [image] can be written as β = αi, where 0 ≤ i ≤ p - 2, in a unique way. It is not difficult to prove that the order of β = αi is

Thus β is itself a primitive element if and only if gcd(p - 1, i) = 1. It follows that the number of primitive elements modulo p is φ(p - 1).

Example 4.4

Suppose p = 13. By computing successive powers of 2, we can verify that 2 is a primitive element modulo 13:

The element 2i is primitive if and only if gcd(i, 12) = 1; i.e., if and only if i = 1, 5, 7 or 11. Hence, the primitive elements modulo 13 are 2, 6, 7 and 11.

*
* Figure 4.2 RSA Cryptosystem

4.3 The RSA Cryptosystem

We can now describe the RSA Cryptosystem. This cryptosystem uses computations in [image], where n is the product of two distinct odd primes p and q. For such n, note that φ(n) = (p - 1) (q - 1).

The formal description of the cryptosystem is given in Figure 4.2. Let's verify that encryption and decryption are inverse operations. Since

we have that

for some integer t ≥ 1. Suppose that [image]; then we have

as desired. We leave it as an exercise for the reader to show that (xb)a ≡ x (mod n) if

Here is a small (insecure) example of the RSA Cryptosystem.

Example 4.5

Suppose Bob chooses p = 101 and q = 113. Then n = 11413 and φ(n) = 100 × 112 = 11200. Since 11200 = 26527, an integer b can be used as an encryption exponent if and only if b is not divisible by 2, 5 or 7. (In practice, however, Bob will not factor φ(n). He will verify that gcd(φ(n), b) = 1 using the Euclidean algorithm.) Suppose Bob chooses b = 3533. Then the Extended Euclidean algorithm will yield

Hence, Bob's secret decryption exponent is a = 6597.

Bob publishes n = 11413 and b = 3533 in a directory. Now, suppose Alice wants to send the plaintext 9726 to Bob. She will compute

and send the ciphertext 5761 over the channel. When Bob receives the ciphertext 5761, he uses his secret decryption exponent to compute

(At this point, the encryption and decryption operations might appear to be very complicated, but we will discuss efficient algorithms for these operations in the next section.)

The security of RSA is based on the hope that the encryption function eK (x) = xb mod n is one-way, so it will be computationally infeasible for an opponent to decrypt a ciphertext. The trapdoor that allows Bob to decrypt is the knowledge of the factorization n = pq. Since Bob knows this factorization, he can compute φ(n) = (p - 1)(q - 1) and then compute the decryption exponent a using the Extended Euclidean algorithm. We will say more about the security of RSA later on.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

4.4 Implementing RSA

There are many aspects of the RSA Cryptosystem to discuss, including the details of setting up the cryptosystem, the efficiency of encrypting and decrypting, and security issues. In order to set up the system, Bob follows the steps indicated in Figure 4.3. How Bob carries out these steps will be discussed later in this chapter.

One obvious attack on the cryptosystem is for a cryptanalyst to attempt to factor n. If this can be done, it is a simple manner to compute φ(n) = (p - 1)(q - 1) and then compute the decryption exponent a from b exactly as Bob did. (It has been conjectured that breaking RSA is polynomially equivalent2to factoring n, but this remains unproved.)

2Two problems are said to be polynomially equivalent if the existence of a polynomial-time algorithm for either problem implies the existence of a polynomial-time algorithm for the other problem.

*
* Figure 4.3 Setting up RSA

Hence, if the RSA Cryptosystem is to be secure, it is certainly necessary that n = pq must be large enough that factoring it will be computationally infeasible. Current factoring algorithms are able to factor numbers having up to 130 decimal digits (for more information on factoring, see Section 4.8). Hence, it is recommended that, to be on the safe side, one should choose p and q to each be primes having about 100 digits; then n will have 200 digits. Several hardware implementations of RSA use a modulus which is 512 bits in length. However, a 512-bit modulus corresponds to about 154 decimal digits (since the number of bits in the binary representation of an integer is log2 10 times the number of decimal digits), and hence it does not offer good long-term security.

Leaving aside for the moment the question of how to find 100 digit primes, let us look now at the arithmetic operations of encryption and decryption. An encryption (or decryption) involves performing one exponentiation modulo n. Since n is very large, we must use multiprecision arithmetic to perform computations in [image], and the time required will depend on the number of bits in the binary representation of n.

Suppose n has k bits in its binary representation; i.e., k = [log2 n] + 1. Using standard "grade-school" arithmetic techniques, it is not difficult to see that an addition of two k-bit integers can be done in time O(k), and a multiplication can be done in time O(k2). Also, a reduction modulo n of an integer having at most 2k bits can be performed in time O(k2) (this amounts to doing long division and retaining the remainder). Now, suppose that [image] (where we are assuming that 0 ≤ x, y ≤ n - 1). Then xy mod n can be computed by first calculating the product xy (which is a 2k-bit integer), and then reducing it modulo n. These two steps can be peformed in time O(k2). We call this computation modular multiplication.

*
* Figure 4.4 The square-and-multiply algorithm to compute xb mod n

We now consider modular exponentiation, i.e., computation of a function of the form xc mod n. As noted above, both the encryption and the decryption operations in RSA are modular exponentiations. Computation of xc mod n can be done using c - 1 modular multiplications; however, this is very inefficient if c is large. Note that c might be as big as φ(n) - 1, which is exponentially large compared to k.

The well-known "square-and-multiply" approach reduces the number of modular multiplications required to compute xc mod n to at most [image], where [image] is the number of bits in the binary representation of c. Since [image], it follows that xc mod n can be computed in time O(k3). Hence, RSA, encryption and decryption can both be done in polynomial time (as a function of k, which is the number of bits in one plaintext (or ciphertext) character).

Square-and-multiply assumes that the exponent, b say, is represented in binary notation, say

where [image]. The algorithm to compute z = xb mod n is presented in Figure 4.4. It is easy to count the number of modular multiplications performed by the square-and-multiply algorithm. There are always [image] squarings performed (step 3). The number of modular multiplications in step 4 is equal to the number of 1's in the binary representation of b, which is an integer between 0 and [image]. Thus, the total number of modular multiplications is at least [image] and at most [image].

We will illustrate the use of square-and-multiply by returning to Example 4.5.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 4.5 (Cont.)

Recall that n = 11413, and the public encryption exponent is b = 3533. Alice encrypts the plaintext 9726 by computing 97263533 mod 11413, using the square-and multiply algorithm, as follows:

i | bi | z
11 | 1 | 12 × 9726 = 9726
10 | 1 | 97262 × 9726 = 2659
9 | 0 | 26592 = 5634
8 | 1 | 56342 × 9726 = 9167
7 | 1 | 91672 × 9726 = 4958
6 | 1 | 49582 × 9726 = 7783
5 | 0 | 77832 = 6298
4 | 0 | 62982 = 4629
3 | 1 | 46292 × 9726 = 10185
2 | 1 | 101852 × 9726 = 105
1 | 0 | 1052 = 11025
0 | 1 | 110252 × 9726 = 5761

Hence, as stated earlier, the ciphertext is 5761.

It should be emphasized that the most efficient current hardware implementations of RSA achieve encryption rates of about 600 Kbits per second (using a 512 bit modulus n), as compared to 1 Gbit per second for DES. Stated another way, RSA, is roughly 1500 times slower than DES.

At this point we have discussed the encryption and decryption operations for RSA. In terms of setting up RSA, the generation of the primes p and q (Step 1) will be discussed in the next section. Step 2 is straightforward and can be done in time O((log n)2). Steps 3 and 4 involve the Euclidean algorithm, so let's briefly consider its complexity.

Suppose we compute the greatest common divisor of r0 and r1, where r0 > r1. In each iteration of the algorithm, we compute a quotient and remainder, which can be done in time O((log r0)2). If we can obtain an upper bound on the number of iterations, then we will have a bound on the complexity of the algorithm. There is a well-known result, known as Lamé's Theorem, that provides such a bound. It asserts that if s is the number of iterations, then fs+2 ≤ r0, where fi denotes the ith Fibonacci number. Since

it follows that s is O(log r0).

This shows that the running time of the Euclidean algorithm is O((log n)3). (Actually, a more careful analysis can be used to show that the running time is, in fact, O((log n)2).)

4.5 Probabilistic Primality Testing

In setting up the RSA Cryptosystem, it is necessary to generate large (e.g., 80 digit) "random primes." In practice, the way this is done is to generate large random numbers, and then test them for primality using a probabilistic polynomialtime Monte Carlo algorithm such as the Solovay-Strassen or Miller-Rabin algorithm, both of which we will present in this section. These algorithms are fast (i.e., an integer n can be tested in time that is polynomial in log2 n, the number of bits in the binary representation of n), but there is a possibility that the algorithm may claim that n is prime when it is not. However, by running the algorithm enough times, the error probability can be reduced below any desired threshold. (We will discuss this in more detail a bit later.)

The other pertinent question is how many random integers (of a specified size) will need to be tested until we find one that is prime. A famous result in number theory, called the Prime number theorem, states that the number of primes not exceeding N is approximately N/ln N. Hence, if p is chosen at random, the probability that it is prime is about 1/ln p. For a 512 bit modulus, we have [image]. That is, on average, of 177 random integers p of the appropriate size, one will be prime (of course, if we restrict our attention to odd integers, the probability doubles, to about 2/177). So it is indeed practical to generate sufficiently large random numbers that are "probably prime," and hence it is practical to set up the RSA Cryptosystem. We proceed to describe how this is done.

A decision problem is a problem in which a question is to be answered "yes" or "no." A probabilistic algorithm is any algorithm that uses random numbers (in contrast, an algorithm that does not use random numbers is called a deterministic algorithm). The following definitions pertain to probabilistic algorithms for decision problems.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

DEFINITION 4.1 A yes-biased Monte Carlo algorithm is a probabilistic algorithm for a decision problem in which a "yes" answer is (always) correct, but a "no" answer may be incorrect. A no-biased Monte Carlo algorithm is defined in the obvious way. We say that a yes-biased Monte Carlo algorithm has error probability equal to ∈ if, for any instance in which the answer is "yes," the algorithm will give the (incorrect) answer "no" with probability at most ∈. (This probability is computed over all possible random choices made by the algorithm when it is run with a given input.)

The decision problem called Composites is described in Figure 4.5.

Note that an algorithm for a decision problem only has to answer "yes" or "no." In particular, in the case of the problem Composites, we do not require the algorithm to find a factorization in the case that n is composite.

We will first describe the Solovay-Strassen algorithm, which is a yes-biased Monte Carlo algorithm for Composites with error probability 1/2. Hence, if the algorithm answers "yes," then n is composite; conversely, if n is composite, then the algorithm answers "yes" with probability at least 1/2.

*
* Figure 4.5 Composites

*
* Figure 4.6 Quadratic Residues

Although the Miller-Rabin algorithm (which we will discuss later) is faster than Solovay-Strassen, we begin by looking at the Solovay-Strassen algorithm because it is easier to understand conceptually and because it involves some number-theoretic concepts that will be useful in later chapters of the book. We begin by developing some further background from number theory before describing the algorithm.

DEFINITION 4.2 Suppose p is an odd prime and x is an integer, 1 ≤ x ≤ p - 1. x is defined to be a quadratic residue modulo p if the congruence y2 𕟁 x (mod p) has a solution [image]. x is defined to be a quadratic non-residue modulo p if [image] (mod p) and x is not a quadratic residue modulo p.

Example 4.6

The quadratic residues modulo 11 are 1, 3, 4, 5 and 9. Note that (±1)2 = 1, (±5)2 = 3, (±2)2 = 4, (±4)2 = 5 and (±3)2 = 9 (where all arithmetic is in [image]).

The decision problem Quadratic Residues is defined in Figure 4.6 in the obvious way.

We prove a result, known as Euler's criterion, that will give rise to a polynomialtime deterministic algorithm for Quadratic Residues.

THEOREM 4.8 (Euler's Criterion)

Let p be an odd prime. Then x is a quadratic residue modulo p if and only if

PROOF First, suppose x ≡ y2 (mod p). Recall from Corollary 4.6 that if p is prime, then xp-1 ≡ 1 (mod p) for any [image] (mod p). Thus we have

Conversely, suppose x(p-1)/2 ≡ 1 (mod p). Let b be a primitive element modulo p. Then x ≡ bi (mod p) for some i. Then we have

Since b has order p - 1, it must be the case that p - 1 divides i(p - 1)/2. Hence, i is even, and then the square roots of x are ±bi/2.

Theorem 4.8 yields a polynomial-time algorithm for Quadratic Residues, by using the "square-and-multiply" technique for exponentiation modulo p. The complexity of the algorithm will be O((log p)3).

We now need to give some further definitions from number theory.

DEFINITION 4.3 Suppose p is an odd prime. For any integer a ≥ 0, we define the Legendre symbol [image] as follows:

We have already seen that a(p-1)/2 ≡ 1 (mod p) if and only if a is a quadratic residue modulo p. If a is a multiple of p, then it is clear that a(p-1)/2 ≡ 0 (mod p). Finally, if a is a quadratic non-residue modulo p, then a(p-1)/2 ≡ -1 (mod p) since ap-1 ≡ 1 (mod p). Hence, we have the following result, which provides an efficient algorithm to evaluate Legendre symbols:

THEOREM 4.9

Suppose p is an odd prime. Then

Next, we define a generalization of the Legendre symbol.

DEFINITION 4.4 Suppose n is an odd positive integer, and the prime power factorization of n is [image]. Let a ≥ 0 be an integer. The Jacobi symbol [image] is defined to be


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 4.7

Consider the Jacobi symbol [image]. The prime power factorization of 9975 is 9975 = 3 × 52 × 7 × 19. Thus we have

Suppose n > 1 is odd. If n is prime then [image] (mod n) for any a. On the other hand, if n is composite, it may or may not be the case that [image] (mod n). If this equation holds, then n is called an Euler pseudo-prime to the base a. For example, 91 is an Euler pseudo-prime to the base 10, since

However, it can be shown that, for any odd composite n, n is an Euler pseudo-prime to the base a for at most half of the integers a such that 1 ≤ a ≤ n - 1 (see the exercises). This fact shows that the Solovay-Strassen primality test, which we present in Figure 4.7, is a yes-biased Monte Carlo algorithm with error probability at most 1/2. At this point it is not clear that the algorithm is a polynomial-time algorithm. We already know how to evaluate a(n-1)/2 mod n in time O((log n)3), but how do we compute Jacobi symbols efficiently? It might appear to be necessary to first factor n, since the Jacobi symbol [image] is defined in terms of the factorization of n. But, if we could factor n, we would already know if it is prime, so this approach ends up in a vicious circle.

Fortunately, we can evaluate a Jacobi symbol without factoring n by using some results from number theory, the most important of which is a generalization of the law of quadratic reciprocity (property 4 below). We now enumerate these properties without proof:

*
* Figure 4.7 The Solovay-Strassen primality test for an odd integer n

1. If n is an odd integer and m1 ≡ m2 (mod n), then

2. If n is an odd integer, then

3. If n is an odd integer then

In particular, if m = 2kt, where t is odd, then

4. Suppose m and n are odd integers. Then

Example 4.8

As an illustration of the application of these properties, we evaluate the Jacobi symbol [image] as follows:

Notice that we successively apply properties 4, 1, 3, and 2 in this computation.

In general, by applying these four properties, it is possible to compute a Jacobi symbol [image] in polynomial time. The only arithmetic operations that are required are modular reductions and factoring out powers of two. Note that if an integer is represented in binary notation, then factoring out powers of two amounts to determining the number of trailing zeroes. So, the complexity of the algorithm is determined by the number of modular reductions that must be done. It is not difficult to show that at most O(log n) modular reductions are performed, each of which can be done in time O((log n)2). This shows that the complexity is O((log n)3), which is polynomial in log n. (In fact, the complexity can be shown to be O((log n)2) by more precise analysis.)

Suppose that we have generated a random number n and tested it for primality using the Solovay-Strassen algorithm. If we have run the algorithm m times, what is our confidence that n is prime? It is tempting to conclude that the probability that such an integer n is prime is 1 - 2-m. This conclusion is often stated in both textbooks and technical articles, but it cannot be inferred from the given data.

We need to be careful about our use of probabilities. We will define the following random variables: a denotes the event

"a random odd integer n of a specified size is composite,"

and b denotes the event

"the algorithm answers 'n is prime' m times in succession."

It is certainly the case that prob(b|a) ≤ 2-m. However, the probability that we are really interested is prob(a|b), which is usually not the same as prob(b|a).

We can compute prob(a|b) using Bayes' theorem (Theorem 2.1). In order to do this, we need to know prob(a). Suppose N ≤ n ≤ 2N. Applying the Prime number theorem, the number of (odd) primes between N and 2N is approximately

Since there are [image] odd integers between N and 2N, we will use the estimate

Then we can compute as follows:

Note that in this computation, [image] denotes the event

*
* Figure 4.8 Error probabilities for the Solovay-Strassen test

"a random odd integer n is prime."

It is interesting to compare the two quantities (ln n - 2)/(ln n - 2 + 2m+1) and 2-m as a function of m. Suppose that [image], since these are the sizes of primes that we seek for use in RSA. Then the first function is roughly 175/(175 + 2m+1). We tabulate the two functions for some values of m in Figure 4.8.

Although 175/(175+2m+1) approaches zero exponentially quickly, it does not do so as quickly as 2-m. In practice, however, one would take m to be something like 50 or 100, which will reduce the probability of error to a very small quantity.

We conclude this section with another Monte Carlo algorithm for Composites which is known as the Miller-Rabin algorithm (it is also known as the "strong pseudo-prime test"). This algorithm is presented in Figure 4.9. It is clearly a polynomial-time algorithm: an elementary analysis shows that its complexity is O((log n)3), as in the case of the Solovay-Strassen test. In fact, the Miller-Rabin algorithm performs better in practice than the Solovay-Strassen algorithm.

We show now that this algorithm cannot answer "n is composite" if n is prime, i.e., the algorithm is yes-biased.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

THEOREM 4.10

The Miller-Rabin algorithm for Composites is a yes-biased Monte Carlo algorithm.

PROOF We will prove this by assuming the algorithm answers "n is composite" for some prime integer n, and obtain a contradiction. Since the algorithm answers "n is composite," it must be the case that [image] (mod n). Now consider the sequence of values b tested in the algorithm. Since b is squared in each iteration of the for loop, we are testing the values [image]. Since the algorithm answers "n is composite," we conclude that

for 0 ≤ i ≤ k - 1.

*
* Figure 4.9 The Miller-Rabin primality test for an odd integer n

Now, using the assumption that n is prime, Fermat's theorem (Corollary 4.6) tells us that

since n - 1 = 2km. Then [image] is a square root of 1 modulo n. Since n is prime, there are only two square roots of 1 modulo n, namely, ±1 mod n. This can be seen as follows: x is a square root of 1 modulo n if and only if

Since n is prime, either n | (x - 1) (i.e., x ≡ 1 (mod n)) or n | (x + 1) (i.e., x ± 1 (mod n)).

We have that

so it follows that

Then [image] must be a square root of 1. By the same argument,

Repeating this argument, we eventually obtain

which is a contradiction, since the algorithm would have answered "n is prime" in this case.

It remains to consider the error probability of the Miller-Rabin algorithm. Although we will not prove it here, the error probability can be shown to be, at most, 1/4.

4.6 Attacks On RSA

In this section, we address the question: are there possible attacks on RSA other than factoring n? Let us first observe that it is sufficient for the cryptanalyst to compute φ(n). For, if n and φ(n) are known, and n is the product of two primes p, q, then n can be easily factored, by solving the two equations

for the two "unknowns" p and q. If we substitute q = n/p into the second equation, we obtain a quadratic equation in the unknown value p:

The two roots of this equation will be p and q, the factors of n. Hence, if a cryptanalyst can learn the value of φ(n), then he can factor n and break the system. In other words, computing φ(n) is no easier than factoring n.

Here is an example to illustrate.

Example 4.9

Suppose the cryptanalyst has learned that n = 84773093 and φ(n) = 84754668. This information gives rise to the following quadratic equation:

This can be solved by the quadratic formula, yielding the two roots 9539 and 8887. These are the two factors of n.

4.6.1 The Decryption Exponent

We will now prove the very interesting result that any algorithm which computes the decryption exponent a can be used as a subroutine (or oracle) in a probabilistic algorithm that factors n. So we can say that computing a is no easier than factoring n. However, this does not rule out the possibility of breaking the cryptosystem without computing a.

Notice that this result is of much more than theoretical interest. It tells us that if a is revealed, then the value n is also compromised. If this happens, it is not sufficient for Bob to choose a new encryption exponent; he must also choose a new modulus n.

The algorithm we are going to describe is a probabilistic algorithm of the Las Vegas type. Here is the definition:

DEFINITION 4.5 Suppose 0 ≤ ∈ < 1 is a real number. A Las Vegas algorithm is a probabilistic algorithm such that, for any problem instance I, the algorithm may fail to give an answer with some probability ∈ (i.e., it can terminate with the message "no answer"). However, if the algorithm does return an answer, then the answer must be correct.

REMARK Las Vegas algorithm may not give an answer, but any answer it gives is correct. In contrast, a Monte Carlo algorithm always gives an answer, but the answer may be incorrect.

If we have a Las Vegas algorithm to solve a problem, then we simply run the algorithm over and over again until it finds an answer. The probability that the algorithm will return "no answer" m times in succession is ∈m. The average (i.e., expected) number of times the algorithm must be run in order to obtain an answer is in fact 1/(1 - ∈) (see the exercises).

Suppose that A is a hypothetical algorithm that computes the decryption exponent a from b and n. We will describe a Las Vegas algorithm that uses A as an oracle. This algorithm will factor n with probability at least 1/2. Hence, if the algorithm is run m times, then n will be factored with probability at least 1 - 1/2m.

The algorithm is based on certain facts concerning square roots of 1 modulo n, where n = pq is the product of two distinct odd primes. Recall that the congruence x2 ≡ 1 (mod p) has two solutions modulo p, namely x = ±1 mod p. Similarly, the congruence x2 ≡ 1 (mod q) has two solutions, namely x = ±1 mod q.

Now, since x2 ≡ 1 (mod n) if and only if x2 ≡ 1 (mod p) and x2 ≡ 1 (mod q), it follows that x2 ≪ 1 (mod n) if and only if x2 ≡ 1 mod p and x = ±1 mod q. Hence, there are four square roots of 1 modulo n, and they can be found using the Chinese remainder theorem. Two of these solutions are x = ±1 mod n; these are called the trivial square roots of 1 modulo p. The other two square roots are called non-trivial, and they are negatives of each other modulo n.

Here is a small example to illustrate.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 4.10

Suppose n = 403 = 13 × 31. The four square roots of 1 modulo 403 are 1, 92, 311 and 402. The square root 92 is obtained by solving the system x ≡ 1 (mod 13), x ≡ -1 (mod 31) using the Chinese remainder theorem. Having found this non-trivial square root, the other non-trivial square root must be 403 - 92 = 311. It is the solution to the system x ≡ -1 (mod 13), x ≡ 1 (mod 31).

Suppose x is a non-trivial square root of 1 modulo n. Then we have

but n divides neither factor on the right side. It follows that gcd(x + 1, n) = p or q (and similarly, gcd(x - 1, n) = p or q). Of course, a greatest common divisor can be computed using the Euclidean algorithm, without knowing the factorization of n. Hence, knowledge of a non-trivial square root of 1 modulo n yields the factorization of n with only a polynomial amount of computation. This important fact is the basis of many results in cryptography.

In Example 4.10 above, gcd(93,403) = 31 and gcd(312,403) = 13.

In Figure 4.10, we present an algorithm which, using the hypothetical algorithm A as a subroutine, attempts to factor n by finding a non-trivial square root of 1 modulo n. (Recall that A computes the decryption exponent a corresponding to the encryption exponent b.) We first do an example to illustrate the application of this algorithm.

Example 4.11

Suppose n = 89855713, b = 34986517 and a = 82330933, and the random value w = 5. We have

In step 6, v = 85877701, and in step 10, v = 1. In step 12, we compute

This is one factor of n; the other is n/9103 = 9871.

Let's now proceed to the analysis of the algorithm. First, observe that if we are lucky enough to choose w to be a multiple of p or q, then we can factor n

*
* Figure 4.10 Factoring algorithm, given the decryption exponent a

immediately. This is detected in step 2. If w is relatively prime to n, then we compute wr, w2r, w4r, . . . , by successive squaring, until

for some t. Since

we know that [image] (mod n). Hence, the while loop terminates after at most s iterations. At the end of the while loop, we have found a value v0 such that [image] (mod n) but [image] (mod n). If v0 ≡ -1 (mod n), then the algorithm fails; otherwise, v0 is a non-trivial square root of 1 modulo n and we are able to factor n (step 12).

The main task facing us now is to prove that the algorithm succeeds with probability at least 1/2. There are two ways in which the algorithm can fail to factor n:

1. wr ≡ 1 (mod n) (step 7)
2. (mod n) for some t, 0 ≤ t ≤ s - 1 (step 11)

We have s + 1 congruences to consider. If a random value w is a solution to at least one of these s + 1 congruences, then it is a "bad" choice, and the algorithm fails. So we proceed by counting the number of solutions to each of these congruences.

First, consider the congruence wr ≡ 1 (mod n). The way to analyze a congruence such as this is to consider solutions modulo p and modulo q separately, and then combine them using the Chinese remainder theorem. Observe that x ≡ 1 (mod n) if and only if x ≡ 1 (mod p) and x ≡ 1 (mod q).

So, we first consider wr ≡ 1 (mod p). Since p is prime, [image] is a cyclic group by Theorem 4.7. Let g be a primitive element modulo p. We can write w = gu for a unique integer u, 0 ≤ u ≤ p - 2. Then we have


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Let us write

where p1 is odd, and

where q1 is odd. Since

we have that

Hence

and

Now, the condition (p - 1) | [ur becomes 2ip1 | ur. Since p1 | r and r is odd, it is necessary and sufficient that 2i | u. Hence, u = k2i, 0 ≤ k ≤ p1 - 1, and the number of solutions to the congruence wr ≡ 1 (mod p) is p1.

By an identical argument, the congruence wr ≡ 1 (mod q) has exactly q1 solutions. We can combine any solution modulo p with any solution modulo q to obtain a unique solution modulo n, using the Chinese remainder theorem. Consequently, the number of solutions to the congruence wr ≡ (mod n) is p1q1.

The next step is to consider a congruence [image] (mod n) for a fixed value t (where 0 ≤ t ≤ s - 1). Again, we first look at the congruence modulo p and then modulo q (note that [image] (mod n) if and only if [image] (mod p) and [image] (mod q). First, consider [image] (mod p). Writing w = gu, as above, we get

Since g(p-1)/2 ≡ -1 (mod p), we have that

Since p - 1 = 2ip1, we get

Taking out a common factor of p1, this becomes

Now, if t ≥ i, then there can be no solutions since 2i+1 | 2t+1 but [image]. On the other hand, if t ≤ i - 1, then u is a solution if and only if u is an odd multiple of 2i-t-1 (note that r/p1 is an odd integer). So, the number of solutions in this case is

By similar reasoning, the congruence [image] (mod q) has no solutions if t ≥ j, and 2tq1 solutions if t ≤ j - 1. Using the Chinese remainder theorem, we see that the number of solutions of [image] (mod n) is

Now, t can range from 0 to s - 1. Without loss of generality, suppose i ≤ j; then the number of solutions is 0 if t ≥ i. The total number of "bad" choices for w is at most

Recall that p - 1 = 2ip1 and q - 1 = 2jq1. Now, j ≥ i ≥ 1 so p1q1 < n/4. We also have that

Hence, we obtain

Since at most (n - 1)/2 choices for w are "bad," it follows that at least (n - 1)/2 choices are "good" and hence the probability of success of the algorithm is at least 1/2.

4.6.2 Partial Information Concerning Plaintext Bits

The other result we will discuss concerns partial information about the plaintext that might be "leaked" by an RSA encryption. Two examples of partial information that we consider are the following:

1. given y = eK(x), compute parity(y), where parity(y) denotes the low-order bit of x
2. given y = eK(x), compute half(y), where half(y) = 0 if 0 ≤ x < n/2 and half(y) = 1 if n/2 < x ≤ n - 1.

We will prove that, given y = eK(x), any algorithm that computes parity(y) or half(y) can be used as an oracle to construct an algorithm that computes the plaintext x. What this means is that, given a ciphertext, computing the low-order bit of the plaintext is polynomially equivalent to determining the whole plaintext!

First, we prove that computing parity(y) is polynomially equivalent to computing half(y). This follows from the following two easily proved identities (see the exercises):

and from the multiplicative rule eK(x1)eK(x2) = eK (x1x2).

We will show how to compute x = dK(y), given a hypothetical algorithm (oracle) which computes half(y). The algorithm is presented in Figure 4.11. In steps 2-4, we compute

for 0 ≤ i ≤ log2 n. We observe that

and so on. Hence, we can find x by a binary search technique, which is done in steps 7-11. Here is a small example to illustrate.

*
* Figure 4.11 Decrypting RSA ciphertext, given an oracle for computing half(y)

Example 4.12

Suppose n = 1457, b = 779, and we have a ciphertext y = 722. eK(2) is computed to be 946. Suppose, using our oracle for half, that we obtain the following values yi in step 3 of the algorithm:

Then the binary search proceeds as shown in Figure 4.12. Hence, the plaintext is x = [999.55] = 999.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

4.7 The Rabin Cryptosystem

In this section, we describe the Rabin Cryptosystem, which is computationally secure against a chosen-plaintext attack provided that the modulus n = pq cannot be factored. The system is described in Figure 4.13.

*
* Figure 4.12 Binary search for RSA decryption

We will show that the encryption function eK is not an injection, so decryption cannot be done in an unambiguous fashion. In fact, there are four possible plaintexts that could be the encryption of any given ciphertext. More precisely, let w be one of the four square roots of 1 modulo n. Let [image]. Then, we can verify the following equations:

(Note that all arithmetic is being done in [image], and division by 2 and 4 is the same as multiplication by 2-1 and 4-1 modulo n, respectively.)

The four plaintexts that encrypt to eK(x) are x, -x - B, ω(x + B/2) - B/2 and -ω(x + B/2) - B/2, where ω is a non-trivial square root of 1 modulo n. In general, there will be no way for Bob to distinguish which of these four possible plaintexts is the "right" plaintext, unless the plaintext contains sufficient redundancy to eliminate three of these four possible values.

*
* Figure 4.13 Rabin Cryptosystem

Let us look at the decryption problem from Bob's point of view. He is given a ciphertext y and wants to determine x such that

This is a quadratic equation in the unknown x. We can eliminate the linear term by making the substitution x1 = x + B/2, or equivalently, x = x1 - B/2. Then the equation becomes

or

If we define C == B2 /4 + y, then we can rewrite the congruence as

So, decryption reduces to extracting square roots modulo n. This is equivalent to solving the two congruences

and

(There are two square roots of C modulo p and two square roots modulo q. Using the Chinese remainder theorem, these can be combined to yield four solutions modulo n.) We can use Euler's criterion to determine if C is a quadratic residue modulo p (and modulo q). In fact, C will be a quadratic residue modulo p and modulo q if encryption was performed correctly. But Euler's criterion does not help us find the square roots of C; it yields only an answer "yes" or "no."

When p ≡ 3 (mod 4), there is a simple formula to compute square roots of quadratic residues modulo p. Suppose C is a quadratic residue and p ≡ 3 (mod 4). Then we have that

Here we again make use of Euler's criterion, which says that if C is a quadratic residue modulo p, then C(p-1)/2 ≡ 1 (mod p). Hence, the two square roots of 7 modulo P are ±C(p+1)/4 mod p. In a similar fashion, the two square roots of 7 modulo q are ±C(q+1)/4 mod q. It is then straightforward to obtain the four square roots x1, of C modulo n using the Chinese remainder theorem.

REMARK It is interesting that for p ≡ 1 (mod 4) there is no known polynomial-time deterministic algorithm to compute square roots of quadratic residues modulo p. There is a polynomial-time Las Vegas algorithm, however.

Once we have determined the four possible values for x1, we compute x from the equation x = x1 - B/2 to get the four possible plaintexts. This yields the decryption formula


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 4.13

Let's illustrate the encryption and decryption procedures for the Rabin Cryptosystem with a toy example. Suppose n = 77 = 7 × 11 and B = 9. Then the encryption function is

and the decryption function is

Suppose Bob wants to decrypt the ciphertext y = 22. It is first necessary to find the square roots of 23 modulo 7 and modulo 11. Since 7 and 11 are both congruent to 3 modulo 4, we use our formula:

*
* Figure 4.14 Factoring a Rabin modulus, given a decryption oracle

and

Using the Chinese remainder theorem, we compute the four square roots of 23 modulo 77 to be ±10, ±32 mod 77. Finally, the four possible plaintexts are:

It can be verified that each of these plaintexts encrypts to the ciphertext 22.

We now discuss the security of the Rabin Cryptosystem. We will prove that any hypothetical decryption algorithm A can be used as an oracle in a Las Vegas algorithm that factors the modulus n with probability at least 1/2. This algorithm is depicted in Figure 4.14.

There are several points of explanation needed. First, observe that

so a value x will be returned in step 3. Next, we look at step 4 and note that [image] (mod n). It follows that x1 ≡ ±r (mod n) or x1 ≡ ±ωr (mod n), where ω is one of the non-trivial square roots of 1 modulo n. In the second case, we have

but n does not divide either factor on the right side. Hence, computation of gcd(x1 + r, n) (or gcd(x1 - r, n)) must yield either p or q, and the factorization of n is accomplished.

Let's compute the probability of success of this algorithm, over all n - 1 choices for the random value r. For two non-zero residues r1 and r2, define

It is easy to see that r ~ r for all r; r1 ~ r2 implies r2 ~ r1; and r1 ~ r2 and r2 ~ r3 together imply r1 ~ r3. This says that the relation ~ is an equivalence relation. The equivalence classes of [image] all have cardinality four: the equivalence class containing r is the set

where ω is a non-trivial square root of 1 modulo n.

In the algorithm presented in Figure 4.14, any two values r in the same equivalence class will yield the same value y. Now consider the value x returned by the oracle A when given y. We have

If r = ±y, then the algorithm fails; while it succeeds if r = ±ωy. Since r is chosen at random, it is equally likely to be any of these four possible values. We conclude that the probability of success of the algorithm is 1/2.

It is interesting that the Rabin Cryptosystem is provably secure against a chosen plaintext attack. However, the system is completely insecure against a chosen ciphertext attack. In fact the algorithm in Figure 4.14, that we used to prove security against a chosen plaintext attack, also can be used to break the Rabin Cryptosystem in a chosen ciphertext attack! In the chosen ciphertext attack, the oracle A is replaced by Bob's decryption algorithm.

4.8 Factoring Algorithms

There is a huge amount of literature on factoring algorithms, and a careful treatment would require more pages than we have in this book. We will just try to give a brief overview here, including an informal discussion of the best current factoring algorithms and their use in practice. The three algorithms that are most effective on very large numbers are the quadratic sieve, the elliptic curve algorithm and the number field sieve. Other well-known algorithms that were precursors include Pollard's rho-method and p - 1 algorithm, Williams' p + 1 algorithm, the continued fraction algorithm, and of course, trial division.

Throughout this section, we suppose that the integer n that we wish to factor is odd. Trial division consists of dividing n by every odd integer up to [image]. If n < 1012, say, this is a perfectly reasonable factorization method, but for larger n we generally need to use more sophisticated techniques.

*
* Figure 4.15 The p - 1 factoring algorithm


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

4.8.1 The p - 1 Method

As an example of a simple algorithm that can sometimes be applied to larger integers, we describe Pollard's p - 1 algorithm, which dates from 1974. This algorithm, presented in Figure 4.15, has two inputs: the (odd) integer n to be factored, and a "bound" B. Here is what is taking place in the p - 1 algorithm: Suppose p is a prime divisor of n, and q ≤ B for every prime power q | (p - 1). Then it must be the case that

At the end of the for loop (step 2),

so

since p | n. Now,

by Fermat's theorem. Since (p - 1) | [B!, we have that

(in step 3). Thus, in step 4,

and

so

The integer d will be a non-trivial divisor of n (unless a = 1 in step 3). Having found a non-trivial factor d, we would then proceed to attempt to factor d and n/d if they are composite.

Here is an example to illustrate.

Example 4.14

Suppose n = 15770708441. If we apply the p - 1 algorithm with B = 180, then we find that a = 11620221425 in step 3, and d is computed to be 135979. In fact, the complete factorization of n into primes is

in this case, the factorization succeeds because 135978 has only "small" prime factors:

Hence, by taking B ≥ 173, it will be the case that 135978 | B!, as desired.

In the algorithm, there are B - 1 modular exponentiations, each requring at most 2log2 B modular multiplications using square-and-multiply. The gcd computation can be done in time O((log n)3) using the Euclidean algorithm. Hence, the complexity of the algorithm is O(B log B(log n)2 + (log n)3). If the integer B is O((log n)i) for some fixed integer i, then the algorithm is indeed a polynomial-time algorithm; however, for such a choice of B the probability of success will be very small. On the other hand, if we increase the size of B drastically, say to [image], then the algorithm will be successful, but it will be no faster than trial division.

Thus, the drawback of this method is that it requires n to have a prime factor p such that p - 1 has only "small" prime factors. It would be very easy to construct an RSA modulus n = pq which would resist factorization by this method. One would start by finding a large prime p1 such that p = 2p1 + 1 is also prime, and a large prime q1 such that q = 2q1 + 1 is also prime (using one of the Monte Carlo primality testing algorithms discussed in Section 4.5). Then the RSA modulus n = pq will be resistant to factorization using the p - 1 method.

The more powerful elliptic curve algorithm, developed by Lenstra in the mid-1980's, is in fact a generalization of the p - 1 method. We will not discuss the theory at all here, but we do mention that the success of the elliptic curve method depends on the more likely situation that an integer "close to" p has only "small" prime factors. Whereas the p - 1 method depends on a relation that holds in the group [image], the elliptic curve method involves groups defined on elliptic curves modulo p.

4.8.2 Dixon's Algorithm and the Quadratic Sieve

Dixon's algorithm is based on a very simple idea that we already saw in connection with the Rabin Cryptosystem. Namely, if we can find [image] (mod n) such that x2 ≡ y2 (mod n), then gcd(x - y, n) is a non-trivial factor of n.

The method uses a factor base, which is a set [image] "small" primes. We first obtain several integers x such that all the prime factors of x2 mod n occur in the factor base [image]. (How this is done will be discussed a bit later.) The idea is to then take the product of several of these x's in such a way that every prime in the factor base is used an even number of times. This then gives us a congruence of the desired type x2 ≡ y2 (mod n), which (we hope) will lead to a factorization of n.

We illustrate with a carefully contrived example.

Example 4.15

Suppose n = 15770708441 (this was the same n that we used in Example 4.14). Let [image]. Consider the three congruences:

If we take the product of these three congruences, then we have


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Reducing the expressions inside the parentheses modulo n, we have

Then we compute

finding the factor 115759 of n.

Suppose [image] is the factor base. Let C be slightly larger than B (say C = B + 10), and suppose we have obtained C congruences:

for 1 ≤ j ≤ C. For each j, consider the vector

If we can find a subset of the aj's that sum modulo 2 to the vector (0, . . . , 0), then the product of the corresponding xj's will use each factor in [image] an even number of times.

We illustrate by returning to Example 4.15, where there exists a dependence even though C < B in this case.

Example 4.15 (Cont.)

The three vectors a1, a2, a3 are as follows:

It is easy to see that

This gives rise to the congruence we saw earlier that successfully factored n.

Observe that finding a subset of the C vectors a1, . . . , aC that sums modulo 2 to the all-zero vector is nothing more than finding a linear dependence (over [image]) of these vectors. Provided C > B, such a linear dependence must exist, and it can be found easily using the standard method of Gaussian elimination. The reason why we take C > B + 1 is that there is no guarantee that any given congruence will yield the factorization of n. Approximately 50% of the time it will turn out that x ≡ ±y (mod n). But if C > B + 1, then we can obtain several such congruences (arising from different linear dependencies among the aj's). Hopefully, at least one of the resulting congruences will yield the factorization.

It remains to discuss how we obtain integers xj such that the values xj2 mod n factor completely over the factor base [image]. There are several methods of doing this. One common approach is the Quadratic Sieve due to Pomerance, which uses integers of the form [image] The name "quadratic sieve" comes from a sieving procedure (which we will not describe here) that is used to determine those xj's that factor over [image]:

There is, of course, a trade-off here: if [image] is large, then it is more likely that an integer xj factors over [image]. But the larger B is, the more congruences we need to accumulate before we are able to find a dependence relation. The optimal choice for B is approximately

and this leads to an expected running time of

The number field sieve is a more recent factoring algorithm from the late 1980's. It also factors n by constructing a congruence x2 ≡ y2 (mod n), but it does so by means of computations in rings of algebraic integers.

4.8.3 Factoring Algorithms in Practice

The asymptotic running times of the quadratic sieve, elliptic curve and number field sieve are as follows:

The notation o(1) denotes a function of n that approaches 0 as n → ∞, and p denotes the smallest prime factor of n.

In the worst case, [image] and the asymptotic running times of the quadratic sieve and elliptic curve algorithms are essentially the same. But in such a situation, quadratic sieve generally outperforms elliptic curve. The elliptic curve method is more useful if the prime factors of n are of differing size. One very large number that was factored using the elliptic curve method was the Fermat number [image] in 1988 by Brent.

For factoring RSA moduli (where n = pq, p, q are prime, and p and q are roughly the same size), the quadratic sieve is currently the most successful algorithm. Some notable milestones have included the following factorizations. In 1983, the quadratic sieve successfully factored a 69-digit number that was a (composite) factor of 2251 - 1 (this computation was done by Davis, Holdridge, and Simmons). Progress continued throughout the 1980's, and by 1989, numbers having up to 106 digits were factored by this method by Lenstra and Manasse, by distributing the computations to hundreds of widely separated workstations (they called this approach "factoring by electronic mail").

More recently, in April 1994, a 129-digit number known as RSA-129 was factored by Atkins, Graff, Lenstra, and Leyland using the quadratic sieve. (The numbers RSA-100, RSA-110, . . . , RSA-500 are a list of RSA moduli publicized on the Internet as "challenge" numbers for factoring algorithms. Each number RSA-d is a d-digit number that is the product of two primes of approximately the same length.) The factorization of RSA-129 required 5000 MIPS-years of computing time donated by over 600 researchers around the world.

The number field sieve is the most recent of the three algorithms. It seems to have great potential since its asymptotic running time is faster than either quadratic sieve or the elliptic curve. It is still in developmental stages, but people have speculated that number field sieve might prove to be faster for numbers having more than about 125-130 digits. In 1990, the number field sieve was used by Lenstra, Lenstra, Manasse, and Pollard to factor [image] into three primes having 7, 49 and 99 digits.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

4.9 Notes and References

The idea of public-key cryptography was introduced by Diffie and Hellman in 1976. Although [DH76A] is the most cited reference, the conference paper [DH76] actually appeared a bit earlier. The RSA Cryptosystem was discovered by Rivest, Shamir and Adleman [RSA78]. The Rabin Cryptosystem was described in Rabin [RA79]; a similar provably secure system in which decryption is unambiguous was found by Williams [WI80]. For a general survey article on public-key cryptography, we recommend Diffie [DI92].

The Solovay-Strassen test was first described in [SS77]. The Miller-Rabin test was given in [MI76] and [RA80]. Our discussion of error probabilities is motivated by observations of Brassard and Bratley [BB88A, §8.6] (see also [BBCGP88]). The best current bounds on the error probability of the Miller-Rabin algorithm can be found in [DLP93].

The material in Section 4.6 is based on the treatment by Salomaa [SA90, pp. 143-154]. The factorization of n given the decryption exponent was proved in [DE84]; the results on partial information revealed by RSA is from [GMT82].

As mentioned earlier, there are many sources of information on factoring algorithms. Pomerance [PO90] is a good survey on factoring, and Lenstra and Lenstra [LL90] is a good article on number-theoretic algorithms in general. Bressoud [BR89] is an elementary textbook devoted to factoring and primality testing. Cryptography textbooks that emphasize number theory include Koblitz [KO94] and Kranakis [KR86]. Lenstra and Lenstra [LL93] is a monograph on the number field sieve.

Exercises 4.7-4.9 give some examples of protocol failures. For a nice article on this subject, see Moore [MO92].

Exercises

4.1 Use the Extended Euclidean algorithm to compute the following multiplicative inverses:

(a) 17-1 mod 101
(b) 357-1 mod 1234
(c) 3125-1 mod 9987.

4.2 Solve the following system of congruences:

4.3 Solve the following system of congruences:

HINT First use the Extended Euclidean algorithm, and then apply the Chinese remainder theorem.

4.4 Here we investigate some properties of primitive roots.

(a) The integer 97 is prime. Prove that x ± 0 is a primitive root modulo 97 if and only if [image] (mod 97) and [image] (mod 97).
(b) Use this method to find the smallest primitive root modulo 97.
(c) Suppose p is prime, and p - 1 has prime power factorization

where the pi's are distinct primes. Prove that x ± 0 is a primitive root modulo p if and only if [image] (mod p) for 1 ≤ i ≤ n.

4.5 Suppose that n = pq, where p and q are distinct odd primes and ab ≡ 1 (mod (p -1)(q - 1)). The RSA encryption operation is e(x) = xb mod n and the decryption operation is d(y) = ya mod n. We proved that d(e(x)) = x if [image]. Prove that the same statement is true for any [image].

HINT Use thr fact that x1 ≡ x2 (mod pq) if and only if x1 ≡ x2 (mod p) and x1 ≡ x2 (mod q). This follows from the Chinese remainder theorem.

4.6 Two samples of RSA ciphertext are presented in Tables 4.1 and 4.2. Your task is to decrypt them. The public parameters of the system are n = 18923 and b = 1261 (for Table 4.1) and n = 31313 and b = 4913 (for Table 4.2). This can be accomplished as follows. First, factor n (which is easy because it is so small). Then compute the exponent a from φ(n), and, finally, decrypt the ciphertext. Use the square-and-multiply algorithm to exponentiate modulo n.

In order to translate the plaintext back into ordinary English text, you need to know how alphabetic characters are "encoded" as elements in [image]. Each element of

Table 4.1 RSA Ciphertext
12423 | 11524 | 7243 | 7459 | 14303 | 6127 | 10964 | 16399
9792 | 13629 | 14407 | 18817 | 18830 | 13556 | 3159 | 16647
5300 | 13951 | 81 | 8986 | 8007 | 13167 | 10022 | 17213
2264 | 961 | 17459 | 4101 | 2999 | 14569 | 17183 | 15827
12693 | 9553 | 18194 | 3830 | 2664 | 13998 | 12501 | 18873
12161 | 13071 | 16900 | 7233 | 8270 | 17086 | 9792 | 14266
13236 | 5300 | 13951 | 8850 | 12129 | 6091 | 18110 | 3332
15061 | 12347 | 7817 | 7946 | 11675 | 13924 | 13892 | 18031
2620 | 6276 | 8500 | 201 | 8850 | 11178 | 16477 | 10161
3533 | 13842 | 7537 | 12259 | 18110 | 44 | 2364 | 15570
3460 | 9886 | 8687 | 4481 | 11231 | 7547 | 11383 | 17910
12867 | 13203 | 5102 | 4742 | 5053 | 15407 | 2976 | 9330
12192 | 56 | 2471 | 15334 | 841 | 13995 | 17592 | 13297
2430 | 9741 | 11675 | 424 | 6686 | 738 | 13874 | 8168
7913 | 6246 | 14301 | 1144 | 9056 | 15967 | 7328 | 13203
796 | 195 | 9872 | 16979 | 15404 | 14130 | 9105 | 2001
9792 | 14251 | 1498 | 11296 | 1105 | 4502 | 16979 | 1105
56 | 4118 | 11302 | 5988 | 3363 | 15827 | 6928 | 4191
4277 | 10617 | 874 | 13211 | 11821 | 3090 | 18110 | 44
2364 | 15570 | 3460 | 9886 | 9988 | 3798 | 1158 | 9872
16979 | 15404 | 6127 | 9872 | 3652 | 14838 | 7437 | 2540
1367 | 2512 | 14407 | 5053 | 1521 | 297 | 10935 | 17137
2186 | 9433 | 13293 | 7555 | 13618 | 13000 | 6490 | 5310
18676 | 4782 | 11374 | 446 | 4165 | 11634 | 3846 | 14611
2364 | 6789 | 11634 | 4493 | 4063 | 4576 | 17955 | 7965
11748 | 14616 | 11453 | 17666 | 925 | 56 | 4118 | 18031
9522 | 14838 | 7437 | 3880 | 11476 | 8305 | 5102 | 2999
18628 | 14326 | 9175 | 9061 | 650 | 18110 | 8720 | 15404
2951 | 722 | 15334 | 841 | 15610 | 2443 | 11056 | 2186

represents three alphabetic characters as in the following examples:

You will have to invert this process as the final step in your program.

The first plaintext was taken from "The Diary of Samuel Marchbanks," by Robertson Davies, 1947, and the second was taken from "Lake Wobegon Days," by Garrison Keillor, 1985.

4.7 This exercise exhibits what is called a protocol failure. It provides an example where ciphertext can be decrypted by an opponent, without determining the key, if a cryptosystem is used in a careless way. (Since the opponent does not determine the key, it is not accurate to call it cryptanalysis.) The moral is that it is not sufficient to use a "secure" cryptosystem in order to guarantee "secure" communication.

Suppose Bob has an RSA Cryptosystem with a large modulus n for which the factorization cannot be found in a reasonable amount of time. Suppose Alice sends

Table 4.2 RSA Ciphertext
6340 | 8309 | 14010 | 8936 | 27358 | 25023 | 16481 | 25809
23614 | 7135 | 24996 | 30590 | 27570 | 26486 | 30388 | 9395
27584 | 14999 | 4517 | 12146 | 29421 | 26439 | 1606 | 17881
25774 | 7647 | 23901 | 7372 | 25774 | 18436 | 12056 | 13547
7908 | 8635 | 2149 | 1908 | 22076 | 7372 | 8686 | 1304
4082 | 11803 | 5314 | 107 | 7359 | 22470 | 7372 | 22827
15698 | 30317 | 4685 | 14696 | 30388 | 8671 | 29956 | 15705
1417 | 26905 | 25809 | 28347 | 26277 | 7897 | 20240 | 21519
12437 | 1108 | 27106 | 18743 | 24144 | 10685 | 25234 | 30155
23005 | 8267 | 9917 | 7994 | 9694 | 2149 | 10042 | 27705
15930 | 29748 | 8635 | 23645 | 11738 | 24591 | 20240 | 27212
27486 | 9741 | 2149 | 29329 | 2149 | 5501 | 14015 | 30155
18154 | 22319 | 27705 | 20321 | 23254 | 13624 | 3249 | 5443
2149 | 16975 | 16087 | 14600 | 27705 | 19386 | 7325 | 26277
19554 | 23614 | 7553 | 4734 | 8091 | 23973 | 14015 | 107
3183 | 17347 | 25234 | 4595 | 21498 | 6360 | 19837 | 8463
6000 | 31280 | 29413 | 2066 | 369 | 23204 | 8425 | 7792
25973 | 4477 | 30989

a message to Bob by representing each alphabetic character as an integer between 0 and 25 (i.e., A ↔ 0, B ↔ 1, etc.), and then encrypting each residue modulo 26 as a separate plaintext character.

(a) Describe how Oscar can easily decrypt a message which is encrypted in this way.
(b) Illustrate this attack by decrypting the following ciphertext (which was encrypted using an RSA Cryptosystem with n = 18721 and b = 25) without factoring the modulus:

4.8 This exercise illustrates another example of a protocol failure (due to Simmons) involving RSA; it is called the common modulus protocol failure. Suppose Bob has an RSA Cryptosystem with modulus n and decryption exponent b1, and Charlie has an RSA Cryptosystem with (the same) modulus n and decryption exponent b2. Suppose also that gcd(b1, b2) = 1. Now, consider the situation that arises if Alice encrypts the same plaintext x to send to both Bob and Charlie. Thus, she computes [image] mod n and [image] mod n, and then she sends y1 to Bob and y2 to Charlie. Suppose Oscar intercepts y1 and y2, and performs the computations indicated in Figure 4.16.

(a) Prove that the value x1 computed in step 3 of Figure 4.16 is in fact Alice's plaintext, x. Thus, Oscar can decrypt the message Alice sent, even though the cryptosystem may be "secure."
(b) Illustrate the attack by computing x by this method if n = 18721, b1 = 43, b2 = 7717, y1 = 12677 and y2 = 14702.

4.9 We give yet another protocol failure involving RSA. Suppose that three users in a network, say Bob, Bart and Bert, all have public encryption exponents b = 3.

*
* Figure 4.16 RSA common modulus protocol failure

Let their moduli be denoted by n1, n2, n3. Now suppose Alice encrypts the same plaintext x to send to Bob, Bart and Bert. That is, Alice computes yi = x3 mod ni, 1 ≤ i ≤ 3. Describe how Oscar can compute x, given y1, y2 and y3, without factoring any of the moduli.

4.10 A plaintext x is said to be fixed if eK(x) = x. Show that, for the RSA Cryptosystem, the number of fixed plaintexts [image] is equal to gcd(b - 1, p - 1) × gcd(b -1, q - 1).

HINT Consider the system of two congruences eK(x) ≡ x (mod p), eK(x) ≡ x (mod q).

4.11 Suppose A is a deterministic algorithm which is given as input an RSA modulus n, an encryption exponent b, and a ciphertext y. A will either decrypt y or return no answer. Supposing that there are ∈(n - 1) ciphertexts which A is able to decrypt, show how to use A as an oracle in a Las Vegas decryption algorithm having success probability ∈.

HINT Use the multiplicative property of RSA that eK(x1)eK(x2) = eK(x1x2), where all arithmetic operations are modulo n.

4.12 Write a program to evaluate Jacobi symbols using the four properties presented in Section 4.5. The program should not do any factoring, other than dividing out powers of two. Test your program by computing the following Jacobi symbols:

4.13 For n = 837, 851 and 1189, find the number of bases 6 such that n, is an Euler pseudo-prime to the base b.
4.14 The purpose of this question is to prove that the error probability of the Solovay-Strassen primality test is at most 1/2. Let [image] denote the group of units modulo n. Define

(a) Prove that G(n) is a subgroup of [image]. Hence, by Lagranges theorem, if [image], then

(b) Suppose n = pk q, where p and q are odd, p is prime, k ≥ 2, and gcd(p, q) = 1. Let a = 1 + pk-1 q. Prove that

HINT Use the binomial theorem to compute a(n-1)/2.

(c) Suppose n = p1, . . . ps, where the pi's are distinct odd primes. Suppose a ≡ u (mod p1) and a ≡ 1 (mod p2p3 . . . ps), where u is a quadratic non-residue modulo p1 (note that such an a exists by the Chinese remainder theorem). Prove that

but

so

(d) If n is odd and composite, prove that |G(n)| ≤ (n - 1)/2.
(e) Summarize the above: prove that the error probability of the Solovay-Strassen primality test is at most 1/2.

4.15 Suppose we have a Las Vegas algorithm with failure probability ∈.

(a) Prove that the probability of first achieving success on the nth trial is pn = ∈n-1(1 - ∈).
(b) The average (expected) number of trials to achieve success is

Show that this average is equal to 1/(1 - ∈).

(c) Let δ be a positive real number less than 1. Show that the number of iterations required in order to reduce the probaility of failure to at most δ is

4.16 Suppose Bob has carelessly revealed his decryption exponent to be a = 14039 in an RSA Cryptosystem with public key n = 36581 and b = 4679. Implement the probablistic algorithm to factor n given this information. Test your algorithm with the "random" choices w = 9983 and w = 13461. Show all computations.
4.17 Prove Equations 4.1 and 4.2 relating the functions half and parity.
4.18 Suppose p = 199, q = 211 and B = 1357 in the Rabin Cryptosystem. Perform following computations.

(a) Determine the four square roots of 1 modulo n, where n = pq.
(b) Compute the encryption y = eK (32767).
(c) Determine the four possible decryptions of this given ciphertext y.

4.19 Factor 262063 and 9420457 using the p - 1 method. How big does B have to be in each case to be successful?


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 5
Other Public-key Cryptosystems

In this chapter, we look at several other public-key cryptosystems. The ElGamal Cryptosystem is based on the Discrete Logarithm problem, which we will have occasion to use in numerous cryptographic protocols throughout the rest of the text. Thus we devote a considerable amount of time to discussion of this important problem. In later sections, we give relatively brief treatments of some other well-known public-key cryptosystems. These include ElGamal-type systems based on finite fields and elliptic curves, the (broken) Merkle-Hellman Knapsack Cryptosystem and the McEliece Cryptosystem.

5.1 The ElGamal Cryptosystem and Discrete Logs

The ElGamal Cryptosystem is based on the Discrete Logarithm problem. We begin by describing this problem in the setting of a finite field [image], where p is prime, in Figure 5.1. (Recall that the multiplicative group [image] is cyclic, and a generator of [image] is called a primitive element.)

The Discrete Logarithm problem in [image] has been the object of much study. The problem is generally regarded as being difficult if p is carefully chosen. In particular, there is no known polynomial-time algorithm for the Discrete Logarithm problem. To thwart known attacks, p should have at least 150 digits, and p - 1 should have at least one "large" prime factor. The utility of the Discrete Logarithm problem in a cryptographic setting is that finding discrete logs is (probably) difficult, but the inverse operation of exponentiation can be computed efficiently by using the square-and-multiply method described earlier. Stated another way, exponentiation modulo p is a one-way function for suitable primes p.

ElGamal has developed a public-key cryptosystem based on the Discrete Logarithm problem. This system is presented in Figure 5.2.

The ElGamal Cryptosystem is non-deterministic, since the ciphertext depends on both the plaintext x and on the random value k chosen by Alice. So there will be many ciphertexts that are encryptions of the same plaintext.

*
* Figure 5.1 The discrete logarithm problem in

*
* Figure 5.2 ElGamal Public-key Cryptosystem in

Informally, this is how the ElGamal Cryptosystem works. The plaintext x is "masked" by multiplying it by βk, yielding y2. The value αk is also transmitted as part of the ciphertext. Bob, who knows the secret exponent a, can compute βk from βk. Then he can "remove the mask" by dividing y2 by βk to obtain x.

A small example will illustrate.

Example 5.I

Suppose p = 2579, α = 2, a = 765, and hence

Now, suppose that Alice wishes to send the message x = 1299 to Bob. Say k = 853 is the random integer she chooses. Then she computes

and

When Bob receives the ciphertext y = (435, 2396), he computes

which was the plaintext that Alice encrypted.

5.1.1 Algorithms for the Discrete Log Problem

Throughout this section, we assume that p is prime and a is a primitive element modulo p. We take p and α to be fixed. Hence the Discrete Logarithm problem can be phrased in the following form: Given [image], find the unique exponent a, 0 ≤ a ≤ p - 2, such that αa ≡ β (mod p).

Clearly, the Discrete Logarithm problem can be solved by exhaustive search in O(p) time and O(1) space (neglecting logarithmic factors). By precomputing all possible values αa, and sorting the ordered pairs (a, αa mood p) with respect to their second coordinates, we can solve the discrete log problem in O(1) time with O(p) precomputation and O(p) memory (again, neglecting logarithmic factors). The first non-trivial algorithm we describe is a time-memory trade-off due to Shanks.

*
* Figure 5.3 Shanks' algorithm for the discrete logarithm problem

Shanks' Algorithm

Denote [image]. Shanks' algorithm is presented in Figure 5.3. Some comments are in order. First, steps 1 and 2 can be precomputed, if desired (this will not affect the asymptotic running time, however). Next, observe that if (j, y) ∈ L1 and (i, y) ∈ L2, then

so

as desired. Conversely, for any β, we can write

where 0 ≤ j,i ≤ m - 1. Hence, the search in step 5 will be successful.

It is not difficult to implement the algorithm to run in O(m) time with O(m) memory (neglecting logarithmic factors). Note that step 5 can be done with one (simultaneous) pass through each of the two lists L1 and L2.

Here is a small example to illustrate.

Example 5.2

Suppose p = 809, and we wish to find log3 525. So we have α = 3, β = 525 and [image]. Then

First, we compute the ordered pairs (j, 99j mod 809) for 0 ≤ j ≤ 28. We obtain the list

(0, 1) | (1, 99) | (2, 93) | (3, 308) | (4, 559)
(5, 329) | (6, 211) | (7, 664) | (8, 207) | (9, 268)
(10, 644) | (11, 654) | (12, 26) | (13, 147) | (14, 800)
(15, 727) | (16, 781) | (17, 464) | (18, 632) | (19, 275)
(20, 528) | (21, 496) | (22, 564) | (23, 15) | (24, 676)
(25, 586) | (26, 575) | (27, 295) | (28, 81)

which is then sorted to produce L1.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

The second list contains the ordered pairs (i, 525 × (3i)-1 mod 809), 0 ≤ j ≤ 28. It is as follows:

(0, 525) | (1, 175) | (2, 328) | (3, 379) | (4, 396)
(5, 132) | (6, 44) | (7, 554) | (8, 724) | (9, 511)
(10, 440) | (11, 686) | (12, 768) | (13, 256) | (14, 355)
(15, 388) | (16, 399) | (17, 133) | (18, 314) | (19, 644)
(20, 754) | (21, 521) | (22, 713) | (23, 777) | (24, 259)
(25, 356) | (26, 658) | (27, 489) | (28, 163)

After sorting this list, we get L2.

Now, if we proceed simultaneously through the two sorted lists, we find that (10, 644) is in L1 and (19, 644) is in L2. Hence, we can compute

As a check, it can be verified that indeed 3309 ≡ 525 (mod 809).

The Pohlig-Hellman Algorithm

The next algorithm we study is the Pohlig-Hellman algorithm. Suppose

where the Pi's are distinct primes. The value a = logα β is determined (uniquely) modulo p - 1. We first observe that if we can compute a mod [image] for each i, 1 ≤ i ≤ k, then we can compute a mod (p - 1) by the Chinese remainder theorem. So, let's suppose that q is prime,

and

We will show how to compute the value

where 0 ≤ x ≤ qc - 1. We can express x in radix q representation as

where 0 ≤ ai ≤ q - 1 for 0 ≤ i ≤ c - 1. Also, observe that we can express a as

for some integer s.

The first step of the algorithm is to compute a0. The main observation is that

To see this, note that

so it suffices to show that

This will be true if and only if

However, we have

which was what we wanted to prove.

Hence, we begin by computing β(p-1)/q mod p. If

then a0 = 0. Otherwise, we successively compute

until

for some i. When this happens, we have a0 = i.

Now, if c = 1, we're done. Otherwise c > 1, and we proceed to determine a1. To do this, we define

and denote

It is not hard to see that

Hence, it follows that

So, we will compute β1(p-1)/q2 mod p, and then find i such that

Then we have a1 = i.

If c = 2, we are now finished; otherwise, we repeat this process c - 2 more times, obtaining a2,..., ac-1.

A pseudo-code description of the Pohlig-Hellman algorithm is given in Figure 5.4. In this algorithm, α is a primitive element modulo p, q is prime,

and

The algorithm calculates a0,..., ac-1, where

We illustrate the Pohlig-Hellman algorithm with a small example.

*
* Figure 5.4 Pohlig-Hellman algorithm to compute logαβ mod qc

Example 5.3

Suppose p = 29; then

Suppose α = 2 and β = 18, so we want to determine a = log2 18. We proceed by first computing a mod 4 and then computing a mood 7.

We start by setting q = 2 and c = 2. First,

and

Next,

Hence, a0 = 1. Next, we compute

and

Since

we have a1 = 1. Hence, a ≡ 3 (mod 4).

Next, we set q = 7 and c = 1. We have

and

Then we would compute

Hence, a0 = 4 and a ≤ 4 (mod 7).

Finally, solving the system

using the Chinese remainder theorem, we get a ≡ 11 (mod 28). That is, we have computed log2 18 in [image] to be 11.

The Index Calculus Method

The index calculus method for computing discrete logs bears considerable resemblence to many of the best factoring algorithms. We give a very brief overview in this section. The method uses a factor base, which, as before, is a set [image] of "small" primes. Suppose [image]. The first step (a preprocessing step) is to find the logarithms of the B primes in the factor base. The second step is to compute a discrete log of a desired element β, using the knowledge of the discrete logs of the elements in the factor base.

In the precomputation, we construct C = B + 10 congruences modulo p, as follows:

1 ≤ j ≤ C. Notice these congruences can be written equivalently as

1 ≤ j ≤ C. Given C congruences in the B "unknowns" logα pi (1 ≤ i ≤ B), we hope that there is a unique solution modulo p - 1. If this is the case, then we can compute the logarithms of the elements in the factor base.

How do we generate congruences of the desired form? One elementary way is to take a random value x, compute ax mod p, and then determine if ax mod p has all its factors in [image] (using trial division, for example).


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Now, given that we have already successfully carried out the precomputation step, we compute a desired logarithm logα β by means of a Las Vegas type probabilistic algorithm. Choose a random integer s (1 ≤ s ≤ p - 2) and compute

Now attempt to factor γ over the factor base [image]. If this can be done, then we obtain a congruence of the form

This can be written equivalently as

Since everything is now known except logαβ, we can easily solve for logαβ.

Here is a small, very artificial, example to illustrate the two steps in the algorithm.

Example 5.4

Suppose p = 10007 and α = 5 is the primitive element used as the base of logarithms modulo p. Suppose we take [image] as the factor base. Of course log5 5 = 1, so there are three logs of factor base elements to be determined.

Some examples of "lucky" exponents that might be chosen are 4063, 5136 and 9865.

With x = 4063, we compute

This yields the congruence

Similarly, since

and

we obtain two more congruences:

and

We now have three congruences in three unknowns, and there happens to be a unique solution modulo 10006, namely log5 2 = 6578, log5 3 = 6190 and log5 7 = 1301.

Now, let's suppose that we wish to find log5 9451. Suppose we choose the "random" exponent s = 7736, and compute

Since 8400 = 24315271 factors over [image], we obtain

To verify, we can check that 56087 = 9451 (mod 10007).

Heuristic analyses of various versions of the algorithm have been done. Under reasonable assumptions, the asymptotic running time of the precomputation is [image], and the time to find an individual discrete phase log is [image].

5.1.2 Bit Security of Discrete Logs

We now look at the question of partial information about discrete logs. In particular, we consider whether individual bits of a discrete logarithm are easy or hard to compute. To be precise, consider the problem presented in Figure 5.5, which we call the i th Bit problem.

*
* Figure 5.5 ith bit of discrete logarithm

We will first show that computing the least significant bit of a discrete logarithm is easy. In other words, if i = 1, the i th Bit problem can be solved efficiently. This follows from Euler's criterion concerning quadratic residues modulo p, where p is prime.

Consider the mapping [image] defined by

Denote by QR(p) the set of quadratic residues modulo p; then

First, observe that f(x) = f(p - x). Next note that

if and only if

which happens if and only if

It follows that

for every y ∈ QR(p), and hence

That is, exactly half the residues in [image] are quadratic residues and half are not.

Now, suppose a is a primitive element of [image]. Then αa ∈ QR(p) if a is even. Since the (p - 1)/2 elements α0 mod p, α2 mod p,..., αp-3 mod p are all distinct, it follows that

Hence, β is a quadratic residue if and only if logα β is even, that is, if and only it L1(β) = 0. But we already know, by Euler's criterion, that β is a quadratic residue if and only if

So we have the following efficient formula to calculate L1 (β):

Let's now consider the computation of Li(β) for values of i exceeding 1. Suppose

where t is odd. Then it can be shown that it is easy to compute Li(β) if i ≤ s. On the other hand, computing Ls+1 (β) is (probably) difficult, in the sense that any hypothetical algorithm (or oracle) to compute Ls+1 (β) could be used to find discrete logarithms in [image].

We shall prove this result in the case s = 1. More precisely, if p ≡ 3 (mod 4) is prime, then we show how any oracle for computing L2(β) can be used to solve the Discrete Log problem in [image].

Recall that, if β is a quadratic residue in [image] and p ≡ 3 (mod 4), then the two square roots of β modulo p are ± β(p+1)/4 mod p. It is also important that, for any β ≠ 0,

if p ≡ 3 (mod 4). We see this as follows. Suppose

then

Since p ≡ 3 (mod 4), the integer (p - 1)/2 is odd, and the result follows.

Now, suppose that β = αa for some (unknown) even exponent a. Then either

or

We can determine which of these two possibilities is correct if we know the value L2 (β), since

This fact is exploited in our algorithm, which we present in Figure 5.6.

At the end of the algorithm, the xi's comprise the bits in the binary representation of logα β; that is,

We will work out a small example to illustrate the algorithm.

*
* Figure 5.6 Computing discrete logs in [image] for p ≡ 3 (mod 4), given an oracle for L2(β)


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 5.5

Suppose p = 19, α = 2 and β = 6. Since the example is so small, we can tabulate the values of L1 (γ) and L2 (γ) for all [image]. (In general, L1 can be computed efficiently using Euler's criterion and L2 is an oracle.) These values are given in Table 5.1. The algorithm now proceeds as shown in Figure 5.7.

Hence, log2 6 = 11102 = 14, as can easily be verified.

It is possible to give formal proof of the algorithm's correctness using mathematical induction. Denote

For i ≥ 0, define

Also, define β0 to be the value of β in step 2 of the algorithm; and, for i ≥ 1, define βi to be the value of β in step 11 during the ith iteration of the while loop. It can be proved by induction that

Table 5.1 Values of L1 and L2 for p = 19, α = 2
γ | L1(γ) | L2(γ) | γ | L1(γ) | L2(γ) | γ | L1(γ) | L2(γ)
1 | 0 | 0 | 7 | 0 | 1 | 13 | 1 | 0
2 | 1 | 0 | 8 | 1 | 1 | 14 | 1 | 1
3 | 1 | 0 | 9 | 0 | 0 | 15 | 1 | 1
4 | 0 | 1 | 10 | 1 | 0 | 16 | 0 | 0
5 | 0 | 0 | 11 | 0 | 0 | 17 | 0 | 1
6 | 0 | 1 | 12 | 1 | 1 | 18 | 1 | 0

*
* Figure 5.7 Computation of log2 6 in

*
* Figure 5.8 The discrete logarithm problem in

for all i ≥ 0. Now, with the observation that

it follows that

i ≥ 0. Since

the algorithm is correct. The details are left to the reader.

5.2 Finite Field and Elliptic Curve Systems

We have spent a considerable amount of time looking at the Discrete Logarithm problem and the factoring. We will see these two problems again and again, underlying various types of cryptosystems and cryptographic protocols. So far, we have considered the Discrete Logarithm problem in the finite field [image], but it is also useful to consider the problem in other settings. This is the theme of this section.

The ElGamal Cryptosystem can be implemented in any group where the Discrete Log problem is intractible. We used the multiplicative group [image], but other groups are also suitable candidates. First, we phrase the Discrete Logarithm problem in a general (finite) group G, where we will denote the group operation by [image]. This generalized version of the problem is presented in Figure 5.8.

It is easy to define an ElGamal Cryptosystem in the subgroup H in a similar fashion as it was originally described in [image]. This is done in Figure 5.9. Note that encryption requires the use of a random integer k such that 0 ≤ k ≤ |H| - 1. However, if Alice does not know the order of the subgroup H, she can generate an integer k such that 0 ≤ k ≤ |G| - 1, and encryption and decryption will work without any changes. Also note that the group G need not be an abelian group (of course H is abelian since it is cyclic).

*
* Figure 5.9 Generalized ElGamal Public-key Cryptosystem

Let's now turn to the "generalized" Discrete Log problem. The subgroup H generated by any α ∈ G is of course a cyclic group of order |H|. So any version of the problem is equivalent, in some sense, to the Discrete Log problem in a cyclic group. However, the difficulty of the Discrete Log problem seems to depend in an essential way on the representation of the group that is used.

As an example to illustrate a representation where the problem is easy to solve, consider the additive cyclic group [image], and suppose gcd(α, n) = 1, so α is a generato, of [image]. Since the group operation is addition modulo n, an "exponentiation" operation, αa, corresponds to multiplication by a modulo n. Hence, in this setting, the Discrete Log problem is to find the integer a such that

Since gcd(α, n) = 1, α has a multiplicative inverse modulo n, and we can compute α-1 mod n easily using the Euclidean algorithm. Then we can solve for a, obtaining


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

We previously discussed the Discrete Log problem in the multiplicative group [image], where p is prime. This group is a cyclic group of order p - 1, and hence it is isomorphic to the additive group [image]. By the discussion above, we know how to compute discrete logs efficiently in this additive group. This suggests that we could solve the Discrete Log problem in [image] by "reducing" the problem to the the easily solved formulation in [image].

Let us think about how this could be done. The statement that [image] is isomorphic to [image] means that there is a bijection

such that

It follows easily that

so we have that

Hence, solving for a as described above, we have that

Consequently, if we have an efficient method of computing the isomorphism φ, then we would have an efficient algorithm to compute discrete logs in [image]. The catch is that there is no known general method to efficiently compute the isomorphism φ for an arbitrary prime p. Even though we know the two groups in question are isomorphic, we do not know an efficient algorithm to explicitly describe the isomorphism.

This method can be applied to the Discrete Log problem in any group G. If there is an efficient method of computing the isomorphism between H and [image], then the discrete log problem in G described above can be solved efficiently. Conversely, it is not hard to see that an efficient method of computing discrete logs yields an efficient algorithm to compute the isomorphism between the two groups.

This discussion has shown that the Discrete Log problem may be easy or (apparently) difficult, depending on the representation of the (cyclic) group that is used. So it may be useful to look at other groups in the hope of finding other settings where the Discrete Log problem seems to be intractible.

Two such classes of groups are

1. the multiplicative group of the Galois field GF(pn)
2. the group of an elliptic curve defined over a finite field.

We will discuss these two classes of groups in the next subsections.

5.2.1 Galois Fields

We have already discussed the fact that [image] is a field if p is prime. However, there are other examples of finite fields not of this form. In fact, there is a finite field with q elements if q = pn where p is prime and n ≥ 1 is an integer. We will now describe very briefly how to construct such a field. First, we need several definitions.

DEFINITION 5.1 Suppose p is prime. Define [image] to be the set of all polynomials in the indeterminate x. By defining addition and multiplication of polynomials in the usual way (and reducing coefficients modulo p), we construct a ring.

For [image], we say that f(x) divides g(x) (notation: f(x) | g(x)) if there exists [image] such that

For [image], define deg(f), the degree of f, to be the highest exponent in a term of f.

Suppose f(x), g(x), [image], and deg(f) = n ≥ 1. We define

if

Notice the resemblance of the definition of congruence of polynomials to that of congruence of integers.

We are now going to define a ring of polynomials "modulo f(x)" which we denote by [image]. The construction of [image] from [image] is based on the idea of congruences modulo f(x) and is analogous to the construction of [image] from [image].

Suppose deg(f) = n. If we divide g(x) by f(x), we obtain a (unique) quotient q(x) and remainder r(x), where

and

This can be done by usual long division of polynomials. Hence any polynomial in [image] is congruent modulo f(x) to a unique polynomial of degree at most n - 1.

Now we define the elements of [image] to be the pn polynomials in [image] of degree at most n - 1. Addition and multiplication in [image] is defined as in [image], followed by a reduction modulo f(x). Equipped with these operations, [image] is a ring.

Recall that [image] is a field if and only if m is prime, and multiplicative inverses can be found using the Euclidean algorithm. A similar situation holds for [image]. The analog of primality for polynomials is irreducibility, which we define as follows:

DEFINITION 5.2 A polynomial [image] is said to be irreducible if there do not exist polynomials [image] such that

where deg(f1) > 0 and deg (f2) > 0.

A very important fact is that [image] is a field if and only if f(x) is irreducible. Further, multiplicative inverses in [image] can be computed using a straightforward modification of the (extended) Euclidean algorithm.

Here is an example to illustrate the concepts described above.

Example 5.6

Let's attempt to construct a field having eight elements. This can be done by finding an irreducible polynomial of degree three in [image]. It is sufficient to consider the polynomials having constant term equal to 1, since any polynomial with constant term 0 is divisible by x and hence is reducible. There are four such polynomials:

Now, f1 (x) is reducible, since

(remember that all coefficients are to be reduced modulo 2). Also, f4 is reducible since

However, f2(x) and f3(x) are both irreducible, and either one can be used to construct a field having eight elements.

Let us use f2(x), and thus construct the field [image]. The eight field elements are the eight polynomials 0, 1, x, x + 1, x2, x2 + 1, x2 + x and x2 + x + 1.

To compute a product of two field elements, we multiple the two polynomials together, and reduce modulo x3 + x + 1 (i.e., divide by x3 + x + 1 and find the remainder polynomial). Since we are dividing by a polynomial of degree three, the remainder will have degree at most two and hence is an element of the field.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

For example, to compute (x2 + 1)(x2 + x + 1) in [image], we first compute the product in [image], which is x4 + x3 + x + 1. Then we divide by x3 + x + 1, obtaining the expression

Hence, in the field [image], we have that

Below, we present a complete multiplication table for the non-zero field elements. To save space, we write a polynomial a2x2 + a1 x + a0 as the ordered triple a2a1 a0.

Computation of inverses can be done by using a straightforward adaptation of the extended Euclidean algorithm.

Finally, the multiplicative group of the non-zero polynomials in the field is a cyclic group of order seven. Since 7 is prime, it follows that any non-zero field element is a generator of this group, i.e., a primitive element of the field.

For example, if we compute the powers of x, we obtain

which comprise all the non-zero field elements.

It remains to discuss existence and uniqueness of fields of this type. It can be shown that there is at least one irreducible polynomial of any given degree n ≥ 1 in [image]. Hence, there is a finite field with pn elements for all primes p and all integers n ≥ 1. There are usually many irreducible polynomials of degree n in [image]. But the finite fields constructed from any two irreducible polynomials of degree n can be shown to be isomorphic. Thus there is a unique finite field of any size pn (p prime, n ≥ 1), which is denoted by GF(pn). In the case n = 1, the resulting field GF(p) is the same thing as [image]. Finally, it can be shown that there does not exist a finite field with r elements unless r = pn for some prime p and some integer n ≥ 1.

We have already noted that the multiplicative group [image] (p prime) is a cyclic group of order p - 1. In fact, the multiplicative group of any finite field is cyclic: GF(pn)\{0} is a cyclic group of order pn - 1. This provides further examples of cyclic groups in which the discrete log problem can be studied.

In practice, the finite fields GF(2n) have been most studied. Both the Shanks and Pohlig-Hellman discrete logarithm algorithms work for fields GF(2n). The index calculus method can be modified to work in these fields. The precomputation time of the index calculus algorithm turns out to be

and the time to find an individual discrete log is

However, for large values of n (say n > 800), the discrete log problem in GF(2n) is thought to be intractible provided 2n - 1 has at least one "large" prime factor (in order to thwart a Pohlig-Hellman attack).

5.2.2 Elliptic Curves

We begin by defining the concept of an elliptic curve.

DEFINITION 5.3 Let p > 3 be prime. The elliptic curve y2 = x3 + ax + b over [image], is the set of solutions [image] to the congruence

where a, [image] are constants such that 4a3 + 27b2 ≠ 0 (mod p), together with a special point O called the point at infinity.1

1Equation 5.1 can be used to define an elliptic curve over any field GF(pn), for p > 3 prime. An elliptic curve over GF(2n) or GF(3n) is defined by a slightly different equation.

An elliptic curve E can be made into an abelian group by defining a suitable operation on its points. The operation is written additively, and is defined as follows (where all arithmetic operations are performed in [image]: Suppose

and

are points on E. If x2 = x1 and y2 = -y1, then [image]; otherwise P + Q = (x3, y3), where

and

Finally, define

for all P ∈ E. With this definition of addition, it can be shown that E is an abelian group with identity element [image] (most of the verifications are tedious but straightforward, but proving associativity is quite difficult).

Note that inverses are very easy to compute. The inverse of (x, y) (which we write as -(x, y) since the group operation is additive) is (x, -y), for all (x, y) ∈ E.

Let us look at a small example.

Example 5.7

Let E be the elliptic curve y2 = x3 + x + 6 over [image]. Let's first determine the points on E. This can be done by looking at each possible [image], computing x3 + x + 6 mod 11, and then trying to solve Equation 5.1 for y. For a given x we can test to see if z = x3 + x + 6 mod 11 is a quadratic residue by applying Euler's criterion. Recall that there is an explicit formula to compute square roots of quadratic residues modulo p for primes p ≡ 3 (mod 4). Applying this formula, we have that the square roots of a quadratic residue z are

The results of these computations are tabulated in Table 5.2.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Thus E has 13 points on it. Since any group of prime order is cyclic, it follows that E is isomorphic to [image], and any point other than the point at infinity is a generator of E. Suppose we take the generator α = (2, 7). Then we can compute the

Table 5.2 Points on the elliptic curve y2 = x3 + x + 6 over
x | x3+x+6 mod 11 | in QR(11)? | y
0 1 2 3 4 5 6 7 8 9 10 | 6 8 5 3 8 4 8 4 9 7 4 | no no yes yes no yes no yes yes no yes | 4,7 5,6 2,9 2,9 3,8 2, 9

"powers" of α (which we will write as multiples of α, since the group operation is additive). To compute 2α = (2, 7) + (2, 7), we first compute

Then we have

and

so 2α (5, 2).

The next multiple would be 3α = 2α + α = (5, 2) + (2, 7). Again, we begin by computing λ, which in this situation is done as follows:

Then we have

and

so 3α = (8, 3).

Continuing in this fashion, the remaining multiples can be computed to be the following:

α | = | (2,7) | 2α | = | (5,2) | 3α | = | (8,3)
4α | = | (10,2) | 5α | = | (3,6) | 6α | = | (7,9)
7α | = | (7,2) | 8α | = | (3,5) | 9α | = | (10,9)
10α | = | (8,8) | 11α | = | (5,9) | 12α | = | (2, 4)

Hence α = (2.7) is indeed a primitive element.

An elliptic curve E defined over [image] (p prime, p > 3) will have roughly p points on it. More precisely, a well-known theorem due to Hasse asserts that the number of points on E, which we denote by #E, satisfies the following inequality

Computing the exact value of #E is more difficult, but there is an efficient algorithm to do this, due to Schoof. (By "efficient" we mean that it has a running time that is polynomial in log p. School's algorithm has a running time of O((log p)8) bit operations and is practical for primes p having several hundred digits.)

Now, given that we can compute #E, we further want to find a cyclic subgroup of E in which the discrete log problem is intractible. So we would like to know something about the structure of the group E. The following theorem gives a considerable amount of information on the group structure of E.

THEOREM 5.1

Let E be an elliptic curve defined over [image], where p is prime, p > 3. Then there exist integers n1 and n2 such that E is isomorphic to [image]. Further, n2 | n1 and n2 | (p - 1).

Hence, if the integers n1 and n2 can be computed, then we know that E has a cyclic subgroup isomorphic to [image] that can potentially be used as a setting tot an ElGamal Cryptosystem.

Note that if n2 = 1, then E is a cyclic group. Also, if #E is a prime, or the product of distinct primes, then E must be a cyclic group.

The Shanks and Pohlig-Hellman algorithms apply to the elliptic curve logarithm problem, but there is no known adaptation of the index calculus method to elliptic curves. However, there is a method of exploiting an explicit isomorphism between elliptic curves and finite fields that leads to efficient algorithms for certain classes of elliptic curves. This technique, due to Menezes, Okamoto and Vanstone, can be applied to some particular examples within a special class of elliptic curves called supersingular curves that were suggested for use in cryptosystems. If the supersingular curves are avoided, however, then it appears that an elliptic curve having a cyclic subgroup of size about 2160 will provide a secure setting for a cryptosystem, provided that the order of the subgroup is divisible by at least one large prime factor (again, to guard against a Pohlig-Hellman attack).

Let's now look at an example of ElGamal encryption using the elliptic curve of Example 5.7.

Example 5.8

Suppose that α = (2, 7) and Bob's secret "exponent" is a = 7, so

Thus the encryption operaton is

where x ∈ E and 0 ≤ k ≤ 12, and the decryption operation is

Suppose that Alice wishes to encrypt the message x = (10, 9) (which is a point on E). If she chooses the random value k = 3, then she will compute

and

Hence, y = ((8, 3), (10, 2)). Now, if Bob receives the ciphertext y, he decrypts it as follows:

Hence, the decryption yields the correct plaintext.

There are some practical difficulties in implementing an ElGamal Cryptosystem on an elliptic curve. This system, when implemented in [image] (or in GF(pn) with n > 1) has a message expansion factor of two. An elliptic curve implementation has a message expansion factor of (about) four. This happens since there are approximately p plaintexts, but each ciphertext consists of four field elements. A more serious problem is that the plaintext space consists of the points on the curve E, and there is no convenient method known of deterministically generating points on E.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

A more efficient variation has been found by Menezes and Vanstone. In this variation, the elliptic curve is used for "masking," and plaintexts and ciphertexts are allowed to be arbitrary ordered pairs of (nonzero) field elements (i.e., they are not required to be points on E). This yields a message expansion factor of two, the same as in the original ElGamal Cryptosystem. The Menezes-Vanstone Cryptosystem is presented in Figure 5.10.

If we return to the curve y2 = x3 + x + 6 over [image], we see that the Menezes-Vanstone Cryptosystem allows 10 × 10 = 100 plaintexts, as compared to 13 in the original system. We illustrate encryption and decryption in this system using this same curve.

Example 5.9

As in the previous example, suppose that α = (2, 7) and Bob's secret "exponent" is a = 7, so

Suppose Alice wants to encrypt the plaintext

(note that x is not a point on E), and she chooses the random value k = 6. First, she computes

and

so c1 = 8 and c2 =3.

Next, she calculates

and

*
* Figure 5.10 Menezes-Vanstone Elliptic Curve Cryptosystem

The ciphertext she sends to Bob is

When Bob receives the ciphertext y, he first computes

and then

*
* Figure 5.11 Subset summ problem

Hence, the decryption yields thee correct plaintext.

5.3 The Merkle-Hellman Knapsack System

The well-known Merkle-Hellman Knapsack Cryptosytem was first described by Merkle and Hellman in 1978. Although this cryptosystem, and several variants of it, were broken in the early 1980's, it is still worth studying for its conceptual elegance and for the underlying design technique.

The term "knapsack" is actually a misnomer2.; the system is based on the Subset Sum problem which is presented in Figure 5.11.

2The Knapsack problem, as it is usually defined, is a problem involving selecting objects with given weights and profits in such a way that a specified capacity is not exceeded and a specified target profit is attained

The Subset Sum problem, as phrased in Figure 5.11, is a decision problem (i.e., we are required only to answer "yes" or "no"). If we rephrase the problem slightly, so that in any instance where the answer is "yes" we are required to find the desired vector x (which may not be unique), then we have a search problem.

*
* Figure 5.12 Algorithm for solving a superincreasing instance of the subset sum problem

The Subset Sum (decision) problem is one of the so-called NP-complete problems. Among other things, this means that there is no known polynomial-time algorithm that solves it. This is also the case for the Subset Sum search problem. But even if a problem has no polynomial-time algorithm to solve it in general, this does not rule out the possibility that certain special cases can be solved in polynomial time. This is indeed the situation with the Subset Sum problem.

We define a list of sizes, (s1, . . . , sn) to be superincreasing if

for 2 ≤ j ≤ n. If the list of sizes is superincreasing, then the search version of the Subset Sum problem can be solved very easily in time O(n), and a solution x (if it exists) must be unique. The algorithm to do this is presented in Figure 5.12.

Suppose s = (s1, . . . , sn) is superincreasing, and consider the function

defined by the rule

Is es a possible candidate for an encryption rule? Since s is superincreasing, es is an injection, and the algorithm presented in Figure 5.12 would be the corresponding decryption algorithm. However, such a system would be completely insecure since anyone (including Oscar) can decrypt a message that is encrypted in this way.

The strategy therefore is to transform the list of sizes in such a way that it is no longer superincreasing. Bob will be able to apply an inverse transformation to restore the superincreasing list of sizes. On the other hand Oscar, who does not know the transformation that was applied, is faced with what looks like a general, apparently difficult, instance of the subset sum problem when he tries to decrypt a ciphertext.

One suitable type of transformation is a modular transformation. That is, a prime modulus p is chosen such that

as well as a multiplier a, where 1 ≤ a ≤ p - 1. Then we define

1 ≤ i ≤ n. The list of sizes t = (t1,...,tn) will be the public key used for encryption. The values a, p used to define the modular transformation are secret. The complete description of the Merkle-Hellman Knapsack Cryptosystem is given in Figure 5.13.

The following small example illustrates the encryption and decryption operations in the Merkle-Hellman Cryptosystem.

Example 5.10

Suppose

is the secret superincreasing list of sizes. Suppose p = 2003 and a = 1289. Then the public list of sizes is

Now, if Alice wants to encrypt the plaintext x = (1, 0, 1, 1, 0, 0, 1, 1, 1), she computes

When Bob receives the ciphertext y, he first computes

Then Bob solves the instance I = (s, z) of the Subset Sum problem using the algorithm presented in Figure 5.12. The plaintext (1, 0, 1, 1, 0, 0, 1, 1, 1) is obtained.

*
* Figure 5.13 Merkle-Hellman Knapsack Cryptosystem

By the early 1980's, the Merkle-Hellman Knapsack Cryptosystem had been broken by Shamir. Shamir was able to use an integer programming algorithm of Lenstra to break the system. This allows Bob's trapdoor (or an equivalent trapdoor) to be discovered by Oscar, the cryptanalyst. Then Oscar can decrypt messages exactly as Bob does.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

5.4 The McEliece System

The McEliece Cryptosystem uses the same design principle as the MerkleHellman Cryptosystem: decryption is an easy special case of an NP-complete problem, disguised so that it looks like a general instance of the problem. In this system, the NP-complete problem that is employed is decoding a general linear (binary) error-correcting code. However, for many special classes of codes, polynomial-time algorithms are known to exist. One such class of codes, the Goppa codes, are used as the basis of the McEliece Cryptosystem.

We begin with some essential definitions.

DEFINITION 5.4 Let k, n be positive integers, k ≤ n. An [n, k] code, C, is a k-dimensional subspace of [image], the vector space of all binary n-tuples.

A generating matrix for an [n, k] code, C, is a k × n binary matrix whose rows form a basis for C.

Let x, [image], where x = (x1, . . . , xn) and y = (y1 , . . . , yn). Define the Hamming distance

i.e., the number of coordinates in which x and y differ.

Let C be an [n, k] code. Define the distance of C to be the quantity

An [n, k] code with distance d is denoted as an [n, k, d] code.

The purpose of an error-correcting code is to correct random errors that occur in the transmission of (binary) data through a noisy channel. Briefly, this is done as follows. Let G be a generating matrix for an [n, k, d] code. Suppose x is the binary k-tuple we wish to transmit. Then Alice encodes x as the n-tuple y = x G, and transmits y through the channel.

Now, suppose Bob receives the n-tuple r, which may not be the same as y. He will decode r using the strategy of nearest neighbor decoding. In nearest neighbor decoding, Bob finds the codeword y′ that has minimum distance to r. Then he decodes r to y′, and, finally, determines the k-tuple x′ such that y′ = x′G. Bob is hoping that y′ = y, so x′ = x (i.e., he is hoping that any transmission errors have been corrected).

It is fairly easy to show that if at most (d - 1)/2 errors occurred during transmission, then nearest neighbor decoding does in fact correct all the errors.

Let us think about how nearest neighbor decoding would be done in practice. |C| = 2k, so if Bob compares r to every codeword, he will have to examine 2k vectors, which is an exponentially large number compared to k. In other words, this obvious algorithm is not a polynomial-time algorithm.

Another approach, which forms the basis for many practical decoding algorithms, is based on the idea of a syndrome. A parity-check matrix for an [n, k, d] code C having generating matrix G is an (n - k) × n 0 - 1 matrix, denoted by H, whose rows form a basis for the orthogonal complement of C, which is denoted by C⊥ and called the dual code to C. Stated another way, the rows of H are linearly independent vectors, and GHT is a k × (n - k) matrix of zeroes.

Given a vector [image], we define the syndrome of r to be HrT. A syndrome is a column vector with n - k components.

The following basic results follow immediately from linear algebra.

THEOREM 5.2

Suppose C is an [n, k] code with generating matrix G and parity-check matrix H. Then [image] is a codeword if and only if

Further; if [image], and r = x + e, then H r T = H e T.

Think of e as being the vector of errors that occur during transmission of a codeword x. Then r represents the vector that is received. The above theorem is saying that the syndrome depends only on the errors, and not on the particular codeword that was transmitted.

This suggests the following approach to decoding, known as syndrome decoding: First, compute s = H r T. If s is a vector of zeroes, then decode r as r. If not, then generate all possible error vectors or weight 1 in turn. For each such e, compute H e T. If, for any of these vectors e, it happens that H e T = s, then decode r to r - e. Otherwise, continue on to generate all error vectors of weight 2, . . . , ⌊(d - 1)/2⌋. If at any time H e T = s, then we decode r to r - e and quit. If this equation is never satisfied, then we conclude that more than ⌊(d - 1)/2⌋ errors have occurred during transmission.

By this approach, we can decode a received vector in at most

steps.

This method will work on any linear code. For certain specific types of codes, the decoding procedure can be speeded up. However, a decision version of nearest neighbor decoding is in fact an NP-complete problem. Thus no polynomial-time algorithm is known for the general problem of nearest neighbor decoding (when the number of errors is not bounded by ⌊(d - 1)/2⌋).

As was the case with the subset sum problem, we can identify an "easy" special case, and then disguise it so that it looks like a "difficult" general case of the problem. It would take us too long to go into the theory here, so we will just summarize the results. The "easy" special case that was suggested by McEliece is to use a code from a class of codes known as the Goppa codes. These codes do in fact have efficient decoding algorithms. Also, they are easy to generate, and there are a large number of inequivalent Goppa codes with the same parameters.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

The parameters of the Goppa codes have the form n = 2m, d = 2t + 1 and k = n - mt. For a practical implementation of the public-key cryptosystem, McEliece suggested taking m = 10 and t = 50. This gives rise to a Goppa code that is a [1024, 524, 101] code. Each plaintext is a binary 524-tuple, and each ciphertext is a binary 1024-tuple. The public key is a 524 × 1024 binary matrix.

*
* Figure 5.14 McEliece Cryptosystem

A description of the McEliece Cryptosystem is given in Figure 5.14.

We present a ridiculously small example to illustrate the encoding and decoding procedures.

Example 5.11

The matrix

is a generating matrix for a [7, 4, 3] code, known as a Hamming code. Suppose Bob chooses the matrices

and

Then, the public generating matrix is

Now, suppose Alice encrypts the plaintext x = (1, 1, 0, 1) using as the random error vector of weight 1 the vector e = (0, 0, 0, 0, 1, 0, 0). The ciphertext is computed to be

When Bob receives the ciphertext y, he first computes

Next, he decrypts y1 to get x1 = (1, 0, 0, 0, 1, 1, 0) (note that e1 ≠ e due to the multiplication by P-1).

Next, Bob forms x0 = (1, 0, 0, 0) (the first four components of x1).

Finally, Bob calculates

This is indeed the plaintext that Alice encrypted.

5.5 Notes and References

The ElGamal Cryptosystem was presented in [EL85]. The Pohlig-Hellman algorithm was published in [PH78], and the material concerning individual bits of the Discrete Logarithm problem is based on Peralta [PE86]. For further information on the Discrete Logarithm problem, we recommend the articles by LaMacchia and Odlyzko [LO91] and McCurley [MC90].

The main reference book for finite fields is Lidl and Niederreiter [LN83]. McEliece [MC87] is a good textbook on the subject, and a research monograph on applications of finite fields was published by Menezes et al. [MBGMVY93]. A recent article on the Discrete Logarithm problem in GF(2n) is Gordon and McCurley [GM93].

The idea of using elliptic curves for public-key cryptosystems is due to Koblitz [KO87] and Miller [MI86]. Menezes [ME93] is a monograph on elliptic curve cryptosystems. See also Menezes and Vanstone [MV93] and Koblitz [KO94]. For an elementary treatment of elliptic curves, see Silverman and Tate [ST92]. The Menezes-Okamoto-Vanstone reduction of discrete logarithms from elliptic curves to finite fields is given in [MOV94] (see also [ME93]).

The Merkle-Hellman Cryptosystem was presented in [MH78]. This system was broken by Shamir [SH84], and the "iterated" version of the system was broken by Brickell [BR85]. A different knapsack-type system, due to Chor and Rivest [CR88], has not been broken. For more information, see the survey article by Brickell and Odlyzko [BO92].

The most important reference book for coding theory is MacWilliams and Sloane [MS77] There are many good textbooks on coding theory, e.g., Hoffman et al. [HLLPRW91] and Vanstone and van Oorschot [VV89]. The McEliece Cryptosystem was first described in [MC78]. A recent article discussing the security of this cryptosystem is by Chabaud [CH95].

Exercises

5.1 Implement Shanks' algorithm for finding discrete logarithms in [image], where p is prime and α is a primitive element. Use your program to find log106 12375 in [image] and log6 248388 in [image].
5.2 Implement the Pohlig-Hellman algorithm for finding discrete logarithms in [image], where p is prime and α is a primitive element. Use your program to find log5 8563 in [image] and log10 12611 in [image].
5.3 Find log5 896 in [image] using the algorithm presented in Figure 5.6, given that L2(β) = 1 for β = 25, 219 and 841, and L2(β) = 0 for β = 163, 532, 625 and 656.
5.4 Decrypt the ElGamal ciphertext presented in Table 5.3. The parameters of the system arc p = 31847, α = 5, a = 7899 and β = 18074. Each element of [image] represents three alphabetic characters as in Exercise 4.6.

The plaintext was taken from "The English Patient," by Michael Ondaatje, Alfred A. Knopf, Inc., New York, 1992.

5.5 Determine which of the following polynomials are irreducible over [image].
5.6 The field GF(25) can be constructed as [image]. Perform the following computations in this field.

(a) Compute (x4 + x2) × (x3 + x + 1).
(b) Using the extended Euclidean algorithm, compute (x3 + x2)-1
(c) Using the square-and-multiply algorithm, compute x25.

5.7 We give an example of the ElGamal Cryptosystem implemented in GF(33). The polynomial x3+2x2+1 is irreducible over [image] and hence [image] is the field GF(33). We can associate the 26 letters of the alphabet with the 26 nonzero field elements, and thus encrypt ordinary text in a convenient way. We will use a lexicographic ordering of the (nonzero) polynomials to set up the correspondence. This correspondence is as follows:

Suppose Bob uses α = x and a = 11 in an ElGamal system; then β = x + 2. Show how Bob will decrypt the following string of ciphertext:

(K, H) (P X) (N, K) (H,R) (T,F) (V, Y) (E, H) (F, A) (T, W) (J, D)(U, J)

Table 5.3 ElGamal Ciphertext
(3781, 14409) | (31552, 3930) | (27214, 15442) | (5809, 30274)
(54000, 31486) | (19936, 721) | (27765, 29284) | (29820, 7710)
(31590, 26470) | (3781, 14409) | (15898, 30844) | (19048, 12914)
(16160, 3129) | (301, 17252) | (24689, 7776) | (28856, 15720)
(30555, 24611) | (20501, 2922) | (13659, 5015) | (5740, 31233)
(1616, 14170) | (4294, 2307) | (2320, 29174) | (3036, 20132)
(14130, 22010) | (25910, 19663) | (19557, 10145) | (18899, 27609)
(26004, 25056) | (5400, 31486) | (9526, 3019) | (12962, 15189)
(29538, 5408) | (3149, 7400) | (9396, 3058) | (27149, 20535)
(1777, 8737) | (26117, 14251) | (7129, 18195) | (25302, 10248)
(23258, 3468) | (26052, 20545) | (21958, 5713) | (346, 31194)
(8836, 25898) | (8794, 17358) | (1777, 8737) | (25038, 12483)
(10422, 5552) | (1777, 8737) | (3780, 16360) | (11685, 133)
(25115, 10840) | (14130, 22010) | (16081, 16414) | (28580, 20845)
(23418, 22058) | (24139, 9580) | (173, 17075) | (2016, 18131)
(198886, 22344) | (21600, 25505) | (27119, 19921) | (23312, 16906)
(21563, 7891) | (28250, 21321) | (28327, 19237) | (15313, 28649)
(24271, 8480) | (26592, 25457) | (9660, 7939) | (10267, 20623)
(30499, 14423) | (5839, 24179) | (12846, 6598) | (9284, 27858)
(24875, 17641) | (1777, 8737) | (18825, 19671) | (31306, 11929)
(3576, 4630) | (26664, 27572) | (27011, 29164) | (22763, 8992)
(3149, 7400) | (8951, 29435) | (2059, 3977) | (16258, 30341)
(21541, 19004) | (5865, 29526) | (10536, 6941) | (1777, 8737)
(17561, 11884) | (2209, 6107) | (10422, 5552) | (19371, 21005)
(26521, 5803) | (14884, 14280) | (4328, 8635) | (28250, 21321)
(28327, 19237) | (15313, 28649)
5.8 Let E be the elliptic curve y2 = x3 + x + 28 defined over [image].

(a) Determine the number of points on E.
(b) Show that E is not a cyclic group.
(c) What is the maximum order of an element in E? Find an element having this order.

5.9 Let E be the elliptic curve y2 = x3 + x + 13 defined over [image]. It can be shown that #E = 34 and (9, 10) is an element of order 34 in E. The Menezes-Vanstone Cryptosystem defined on E will have as its plaintext space [image]. Suppose Bob's secret exponent is a = 25.

(a) Compute β = aα.
(b) Decrypt the following string of ciphertext:

((4, 9), 28, 7), ((19, 28), 9, 13), ((5,22), 20, 17), ((25, 16), 12, 27).

(c) Assuming that each plaintext represents two alphabetic characters, convert the plaintext into an English word. (Here we will use the correspondence A ↔ 1, . . . , Z ↔ 26, since 0 is not allowed in a (plaintext) ordered pair.)

5.10 Suppose the Merkle-Hellman Cryptosystem has as its public list of sizes the vector

Suppose Oscar discovers that p = 2503.

(a) By trial and error, determine the value a such that the list a-1t mod p is a permutation of a superincreasing list.
(b) Show how the ciphertext 5746 would be decrypted.

5.11 It can be shown that the matrix H shown below is a parity-check matrix for a [15, 7, 5] code called a BCH code.

Decode, it possible, each of the following received vectors r using the syndrome decoding method.

(a) r = (1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0).
(b) r = (1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0).
(c) r = (1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0).


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 6
Signature Schemes

6.1 Introduction

In this chapter, we study signature schemes, which are also called digital signatures. A "conventional" handwritten signature attached to a document is used to specify the person responsible for it. A signature is used in everyday situations such as writing a letter, withdrawing money from a bank, signing a contract, etc.

A signature scheme is a method of signing a message stored in electronic form. As such, a signed message can be transmitted over a computer network. In this chapter, we will study several signature schemes, but first we discuss some fundamental differences between conventional and digital signatures.

First is the question of signing a document. With a conventional signature, a signature is physically part of the document being signed. However, a digital signature is not attached physically to the message that is signed, so the algorithm that is used must somehow "bind" the signature to the message.

Second is the question of verification. A conventional signature is verified by comparing it to other, authentic signatures. For example, when someone signs a credit card purchase, the salesperson is supposed to compare the signature on the sales slip to the signature on the back of the credit card in order to verify the signature. Of course, this is not a very secure method as it is relatively easy to forge someone else's signature. Digital signatures, on the other hand, can be verified using a publicly known verification algorithm. Thus, "anyone" can verify a digital signature. The use of a secure signature scheme will prevent the possibility of forgeries.

Another fundamental difference between conventional and digital signatures is that a "copy" of a signed digital message is identical to the original. On the other hand, a copy of a signed paper document can usually be distinguished from an original. This feature means that care must be taken to prevent a signed digital message from being reused. For example, if Bob signs a digital message authorizing Alice to withdraw $100 from his bank account (i.e., a check), he only wants Alice to be able to do so once. So the message itself should contain information, such as a date, that prevents it from being reused.

A signature scheme consists of two components: a signing algorithm and a verification algorithm. Bob can sign a message x using a (secret) signing algorithm sig. The resulting signature sig(x) can subsequently be verified using a public verification algorithm ver. Given a pair (x, y), the verification algorithm returns an answer "true" or "false" depending on whether the signature is authentic.

Here is a formal defintion of a signature scheme.

DEFINITION 6.1 A signature scheme is a five-tuple [image], where the following conditions are satisfied:

1. is a finite set of possible messages
2. is a finite set of possible signatures
3. , the keyspace, is a finite set of possible keys
4. For each [image], there is a signing algorithm [image] and a corresponding verification algorithm [image]. Each sigK : and verK : → {true, false} are functions such that the following equation is satisfied for every message [image] and for every signature [image] :

For every [image], the functions sigK and verK should be polynomial-time functions. verK will be a public function and sigK will be secret. It should be computationally infeasible for Oscar to "forge" Bob's signature on a message x. That is, given x, only Bob should be able to compute the signature y such that ver(x, y) = true. A signature scheme cannot be unconditionally secure, since Oscar can test all possible signatures y for a message x using the public algorithm ver, until he finds the right signature. So, given sufficient time, Oscar can always forge Bob's signature. Thus, as was the case with public-key cryptosystems, our goal is to find signature schemes that are computationally secure.

As our first example of a signature scheme, we observe that the RSA public-key cryptosystem can be used to provide digital signatures. See Figure 6.1.

Thus, Bob signs a message x using the RSA decryption rule dK. Bob is the only person that can create the signature since dK = sigK is secret. The verification algorithm uses the RSA encryption rule eK. Anyone can verify a signature since eK is public.

Note that anyone can forge Bob's signature on a "random" message x by computing x = eK(y) for some y; then y = sigK(x). One way around this difficulty is to require that messages contain sufficient redundancy that a forged signature of this type does not correspond to a "meaningful" message x except with a very small probability. Alternatively, the use of hash functions in conjunction with signature schemes will eliminate this method of forging (cryptographic hash functions will be discussed in Chapter 7).

*
* Figure 6.1 RSA Signature Scheme

Finally, let's look briefly at how we would combine signing and public-key encryption. Suppose Alice wishes to send a signed, encrypted message to Bob. Given a plaintext x, Alice would compute her signature y = sig Alice(x), and then encrypt both x and y using Bob's public encryption function eBob, obtaining z = eBob (x, y). The ciphertext z would be transmitted to Bob. When Bob receives z, he first decrypts it with his decryption function dBob to get (x, y). Then he uses Alice's public verification function to check that ver Alice(x, y) = true.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

What if Alice first encrypted x, and then signed the result? Then she would compute

Alice would transmit the pair (z, y) to Bob. Bob would decrypt z, obtaining x, and then verify the signature y on x using ver Alice. One potential problem with this approach is that if Oscar obtains a pair (z, y) of this type, he could replace Alice's signature y by his own signature

(Note that Oscar can sign the ciphertext z = eBob(x) even though he doesn't know the plaintext x.) Then, if Oscar transmits (z, y′) to Bob, Oscar's signature will be verified by Bob using verOscar, and Bob may infer that the plaintext x originated with Oscar. Because of this potential difficulty, most people recommend signing before encrypting.

*
* Figure 6.2 ElGamal Signature Scheme

6.2 The ElGamal Signature Scheme

We now describe the ElGamal Signature Scheme, which was described in a 1985 paper. A modification of this scheme has been adopted as a digital signature standard by the National Institute of Standards and Technology (NIST). The ElGamal Scheme is designed specifically for the purpose of signatures, as opposed to RSA, which can be used both as a public-key cryptosystem and a signature scheme.

The ElGamal Signature Scheme is non-deterministic, as was the ElGamal Public-key Cryptosystem. This means that there are many valid signatures for any given message. The verification algorithm must be able to accept any of the valid signatures as authentic. The description of the ElGamal Signature Scheme is given in Figure 6.2.

If the signature was constructed correctly, then the verification will succeed, since

where we use the fact that

Bob computes a signature using both the secret value a (which is part of the key) and the secret random number k (which is used to sign one message, x). The verification can be accomplished using only public information.

Let's do a small example to illustrate the arithmetic.

Example 6.1

Suppose we take p = 467, α = 2, a = 127; then

Suppose Bob wants to sign the message x = 100 and he chooses the random value k = 213 (note that gcd(213, 466) = 1 and 213-1 mod 466 = 431). Then

and

Anyone can verify this signature by checking that

and

Hence, the signature is valid.

Let's look at the security of the ElGamal Signature Scheme. Suppose Oscar tries to forge a signature for a given message x, without knowing a. If Oscar chooses a value γ and then tries to find the corresponding δ, he must compute the discrete logarithm logγ αxβ-γ. On the other hand, if he first chooses δ and then tries to find γ, he is trying to "solve" the equation

for the "unknown" γ. This is a problem for which no feasible solution is known; however, it does not seem to be related to any well-studied problem such as the Discrete Logarithm problem. There also remains the possibility that there might be some way to compute γ and δ simultaneously in such a way that (γ, δ) will be a signature. No one has discovered a way to do this, but conversely, no one has proved that it cannot be done.

If Oscar chooses γ and δ and then tries to solve for x, he is again faced with an instance of the Discrete Logarithm problem, namely the computation of logα βγγδ. Hence, Oscar cannot sign a "random" message using this approach. However, there is a method by which Oscar can sign a random message by choosing γ, δ and x simultaneously: Suppose i and j are integers, 0 ≤ i ≤ p -2, 0 ≤ j ≤ p - 2, and gcd(j, p - 1) = 1. Then perform the following computations:

where j-1 is computed modulo (p - 1) (this is where we require that j be relatively prime to p - 1).

We claim that (γ, δ) is a valid signature for the message x. This is proved by checking the verification condition:

We illustrate with an example.

Example 6.2

As in the previous example, suppose p = 467, α = 2 and β = 132. Suppose Oscar chooses i = 99 and j = 179; then j-1 mod (p - 1) = 151. He would compute the following:

Then (117, 41) is a valid signature for the message 331, as may be verified by checking that

and

Hence, the signature is valid.

Here is a second type of forgery, in which Oscar begins with a message previously signed by Bob. Suppose (γ, δ) is a valid signature for a message x. Then it is possible for Oscar to sign various other messages. Suppose h, i and j are integers, 0 ≤ h, i, j ≤ p - 2, and gcd(hγ - jδ, p - 1) = 1. Compute the following:

where (hγ - jδ)-1 is computed modulo (p - 1). Then, it is tedious but straight-forward to check the verification condition:

Hence (λ, μ) is a valid signature for x′.

Both of these methods produce valid forged signatures, but they do not appear to enable an opponent to forge a signature on a message of his own choosing without first solving a discrete logarithm problem. Hence, they do not seem to represent a threat to the security of the ElGamal Signature Scheme.

Finally, we mention a couple of ways in which the ElGamal Scheme can be broken if it is used carelessly (these are further examples of protocol failures, some of which were discussed in the exercises of Chapter 4). First, the random value k used in computing a signature should not be revealed. For, if k is known, it is a simple matter to compute

Of course, once a is known, then the system is broken and Oscar can forge signatures at will.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Another misuse of the system is to use the same value k in signing two different messages. This also makes it easy for Oscar to compute a and hence break the system. This can be done as follows. Suppose (γ, δ1) is a signature on x1 and (γ, δ2) is a signature on x2. Then we have

and

Thus

Writing γ = αk, we obtain the following equation in the unknown k:

which is equivalent to

Now let d = gcd (δ1 - δ2, p - 1). Since d | (p - 1) and d | (δ1 - δ2), it follows that d | (x1 - x2). Define

Then the congruence becomes:

Since gcd(δ′, p′) = 1, we can compute

Then value of k is determined modulo p′ to be

This yields d candidate values for k:

for some i, 0 ≤ i ≤ d - 1. Of these d candidate values, the (unique) correct one can be determined by testing the condition

6.3 The Digital Signature Standard

The Digital Signature Standard (or DSS) is a modification of the ElGamal Signature Scheme. It was published in the Federal Register on May 19, 1994 and adopted as a standard on December 1, 1994 (however, it was first proposed in August, 1991). First, we want to motivate the changes that are made to ElGamal, and then we will describe how they are accomplished.

In many situations, a message might be encrypted and decrypted only once, so it suffices to use any cryptosystem which is known to be secure at the time the message is encrypted. On the other hand, a signed message could function as a legal document such as a contract or will, so it is very likely that it would be necessary to verify a signature many years after the message is signed. So it is important to take even more precautions regarding the security of a signature scheme as opposed to a cryptosystem. Since the ElGamal Scheme is no more secure than the Discrete Logarithm problem, this necessitates the use of a large modulus p. Certainly p should have at least 512 bits, and many people would argue that the length of p should be 1024 bits in order to provide security into the foreseeable future.

However, even a 512 bit modulus leads to a signature having 1024 bits. For potential applications, many of which involve the use of smart cards, a shorter signature is desirable. DSS modifies the ElGamal Scheme in an ingenious way so that a 160-bit message is signed using a 320-bit signature, but the computations are done using a 512-bit modulus p. The way that this done is to work in a subgroup of [image] of size 2160. The assumed security of the scheme is based on the belief that finding discrete logarithms in this specified subgroup of [image] is secure.

The first change we make is to change the "-" to a "+" in the definition of δ, so

This changes the verification condition to the following:

If gcd(x + aγ,p - 1) = 1, then δ-1 mod (p - 1) exists, and we can modify condition (6.1), producing the following:

Now here is the major innovation in the DSS. We suppose that q is a 160-bit prime such that q | (p - 1), and α is a qth root of 1 modulo p. (It is easy to construct such an α: Let α0 be a primitive element of [image], and define α = α0(p-1)/q mod p.) Then β and γ will also be qth roots of 1. Hence, any exponents of α, β and γ can be reduced modulo q without affecting verification condition (6.2). The tricky point is that γ appears as an exponent on the left side of (6.2), and again — but not as an exponent — on the right side of (6.2). So if γ is reduced modulo q, then we must also reduce the entire left side of (6.2) modulo q in order to perform the verification. Observe that (6.1) will not work if the extra reductions modulo q are done. The complete description of the DSS is given in Figure 6.3.

Notice that is necessary that [image] (mod q) since the value δ-1 mod q is needed to verify the signature (this is analogous to the requirement that gcd(δ, p-1) = 1 when we modified (6.1) to obtain (6.2)). If Bob computes a value δ ≡ 0 (mod q) in the signing algorithm, he should reject it and construct a new signature with a new random k. We should point out that this is not likely to cause a problem in practice: the probability that δ ≡ 0 (mod q) is likely to be on the order of 2-160, so for all intents and purposes it will almost never happen.

*
* Figure 6.3 Digital Signature Standard

Here is a small example to illustrate.

Example 6.3

Suppose we take q = 101 and p = 78q + 1 = 7879. 3 is a primitive element in [image], so we can take

Suppose a = 75; then

Now, suppose Bob wants to sign the message x = 22 and he chooses the random value k = 50, so

Then

and

The signature (94, 97) on the message 22 is verified by the following computations:

Hence, the signature is valid.

When the DSS was proposed in 1991, there were several criticisms put forward. One complaint was that the selection process by NIST was not public. The standard was developed by the National Security Agency (NSA) without the input of U. S. industry. Regardless of the merits of the resulting scheme, many people resented the "closed-door" approach.

Of the technical criticisms put forward, the most serious was that the size of the modulus p was fixed at 512 bits. Many people would prefer that the modulus size not be fixed, so that larger modulus sizes could be used if desired. In reponse to these comments, NIST altered the description of the standard so that a variety of modulus sizes are allowed, namely, any modulus size divisible by 64, in the range from 512 to 1024 bits.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Another complaint about the DSS was that signatures can be generated considerably faster than they can be verified. In contrast, if RSA is used as a signature scheme and the public verification exponent is very small (say 3, for example), then verification can be performed much more quickly than signing. This leads to a couple of considerations concerning the potential applications of the signature scheme:

1. A message will only be signed once. On the other hand, it might be necessary to verify the signature many times over a period of years. This suggests that a faster verification algorithm would be desirable.

*
* Figure 6.4 Lamport Signature Scheme

2. What types of computers are likely to be doing the signing and verifying? Many potential applications involve smart cards, with limited processing power, communicating with a more powerful computer. So one might try to design a scheme so that fewer computations are likely to be done by a card. But one can imagine situations where a smart card would generate a signature, and other situations where a smart card would verify a signature, so it is difficult to give a definitive answer here.

The response of NIST to the question of signature generation/verification times is that it does not really matter which is faster, provided that both can be done sufficiently quickly.

6.4 One-time Signatures

In this section, we describe a conceptually simple way to construct a one-time signature scheme from any one-way function. The term "one-time" means that only one message can be signed. (The signature can be verified an arbitrary number of times, of course.) The description of the scheme, known as the Lamport Signature Scheme, is given in Figure 6.4.

Informally, this is how the system works. A message to be signed is a binary k-tuple. Each bit is signed individually: the value zi,j corresponds to the ith bit of the message having the value j (j = 0, 1). Each zi,j is the image of yi,j under the one-way function f. The ith bit of the message is signed using the preimage yi,j of the zi,j corresponding to the ith bit of the message. The verification consists simply of checking that each element in the signature is the preimage of the appropriate public key element.

We illustrate the scheme by considering one possible implementation using the exponentiation function f(x) = αx mod p, where α is a primitive element modulo p.

Example 6.4

7879 is prime and 3 is a primitive element in [image]. Define

Suppose Bob wishes to sign a message of three bits, and he chooses the six (secret) random numbers

Then he computes the images of the y's under the function f:

These z's are published. Now, suppose Bob wants to sign the message

The signature for x is

To verify this signature, it suffices to compute the following:

Hence, the signature is valid.

Oscar cannot forge a signature because he is unable to invert the one-way function f to obtain the secret y's. However, the signature scheme can be used to sign only one message. For, given signatures for two different messages, it is (usually) an easy matter for Oscar to construct signatures for further messages (different from the first two).

For example, suppose the messages (0, 1, 1) and (1, 0, 1) are both signed using the same scheme. The message (0, 1, 1) would have as its signature the triple (y1,0, y2,1, y3,1), and the message (1, 0, 1) would be signed with (y1,1, y2,0, y3,1). Given these two signatures, Oscar can manufacture signatures for the messages (1, 1, 1) (namely, (y1,1, y2,1, y3,1)) and (0, 0, 1) (namely, (y1,0, y2,0, y3,1)).

Even though this scheme is quite elegant, it is not of great practical use due to the size of the signatures it produces. For example, if we use the modular exponentiation function, as in the example above, then a secure implementation would require that p be at least 512 bits in length. This means that each bit of the message is signed using 512 bits. Consequently, the signature is 512 times as long as the message!

We now look at a modification due to Bos and Chaum that allows the signatures to be made somewhat shorter, with no loss of security. In the Lamport Scheme, the reason that Oscar cannot forge a signature on a (second) message, given a signature on one message, is that the y's corresponding to one message are never a subset of the y's corresponding to another (distinct) message.

Suppose we have a set [image] of subsets of a set B such that [image] only if B1 = B2, for all B1, [image]. Then [image] is said to satisfy the Sperner property. Given a set B of even cardinality 2n, it is known that the maximum size of a set [image] of subsets of B having the Sperner property is [image]. This can easily be obtained by taking all the n-subsets of B: clearly no n-subset is contained in another n-subset.

Now suppose we want to sign a k-bit message, as before, and we choose n large enough so that

Let |B| = 2n and let [image] denote the set of n-subsets of B. Let [image] be a publicly known injection. Then we can associate each possible message with an n-subset in [image]. We will have 2n y's and 2n z's, and each message will be signed with n y's. The complete description of the Bos-Chaum Scheme is given in Figure 6.5.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

The advantage of the Bos-Chaum Scheme is that signatures are shorter than with the Lamport Scheme. For example, suppose we wish to sign a message of six bits (i.e., k = 6). Since 26 = 64 and [image], we can take n = 4. This allows a six-bit message to be signed with four y's, as opposed to six with Lamport. As well, the key is shorter, consisting of eight z's as opposed to twelve with Lamport.

*
* Figure 6.5 Bos-Chaum Signature Scheme

The Bos-Chaum Scheme requires an injective function φ that associates an n-subset of a 2n-set with each possible binary k-tuple x = (x1, ..., xk). We present one simple algorithm to do this in Figure 6.6. Applying this algorithm with x = (0, 1, 0, 0, 1, 1), for example, yields

In general, how big is n in the Bos-Chaum Scheme as compared to k? We need to satisfy the inequality [image]. If we estimate the binomial coefficient

using Stirling's formula, we obtain the quantity [image]. After some simplification, the inequality becomes

*
* Figure 6.6 Computation of φ in the Bos-Chaum Scheme

Asymptotically, n is about k/2, so we obtain an almost 50% reduction in signature size by using the Bos-Chaum Scheme.

6.5 Undeniable Signatures

Undeniable signatures were introduced by Chaum and van Antwerpen in 1989. They have several novel features. Primary among these is that a signature cannot be verified without the cooperation of the signer, Bob. This protects Bob against the possibility that documents signed by him are duplicated and distributed electronically without his approval. The verification will be accomplished by means of a challenge-and-response protocol.

But if Bob's cooperation is required to verify a signature, what is to prevent Bob from disavowing a signature he made at an earlier time? Bob might claim that a valid signature is a forgery, and either refuse to verify it, or carry out the protocol in such a way that the signature will not be verified. To prevent this from happening, an undeniable signature scheme incorporates a disavowal protocol by which Bob can prove that a signature is a forgery. Thus, Bob will be able to prove in court that a given forged signature is in fact a forgery. (If he refuses to take part in the disavowal protocol, this would be regarded as evidence that the signature is, in fact, genuine.)

Thus, an undeniable signature scheme consists of three components: a signing algorithm, a verification protocol, and a disavowal protocol. First, we present the signing algorithm and verification protocol of the Chaum-van Antwerpen Undeniable Signature Scheme in Figure 6.7.

*
* Figure 6.7 Chaum-van Antwerpen Undeniable Signature Scheme

We should explain the roles of p and q in this scheme. The scheme lives in [image]; however, we need to be able to do computations in a multiplicative subgroup G of [image] of prime order. In particular, we need to be able to compute inverses modulo |G|, which is why |G| should be prime. It is convenient to take p = 2q + 1 where q is prime. In this way, the subgroup G is as large as possible, which is desirable since messages and signatures are both elements of G.

We first prove that Alice will accept a valid signature. In the following computations, all exponents are to be reduced modulo q. First, observe that

Since

we have that

Similarly,

implies that

Hence,

as desired.

Here is a small example.

Example 6.5

Suppose we take p = 467. Since 2 is a primitive element, 22 = 4 is a generator of G, the quadratic residues modulo 467. So we can take α = 4. Suppose a = 101; then

Bob will sign the message x = 119 with the signature

Now, suppose Alice wants to verify the signature y. Suppose she chooses the random values e1 = 38, e2 = 397. She will compute c = 13, whereupon Bob will respond with d = 9. Alice checks the response by verifying that

Hence, Alice accepts the signature as valid.

We next prove that Bob cannot fool Alice into accepting a fradulent signature as valid, except with a very small probability. This result does not depend on any computational assumptions, i.e., the security is unconditional.

THEOREM 6.1

If [image] (mod p), then Alice will accept y as a valid signature for x with probability 1/q.

PROOF First, we observe that each possible challenge c corresponds to exactly q ordered pairs (e1, e2) (this is because y and β are both elements of the multiplicative group G of prime order q). Now, when Bob receives the challenge c, he has no way of knowing which of the q possible ordered pairs (e1, e2) Alice used to construct c. We claim that, if [image] (mod p), then any possible response d ∈ G that Bob might make is consistent with exactly one of the q possible ordered pairs (e1, e2).

Since α generates G, we can write any element of G as a power of α, where the exponent is defined uniquely modulo q. So write c = αi, d = αj, x = αk, and [image], where [image] and all arithmetic is modulo p. Consider the following two congruences:

This system is equivalent to the following system:

Now, we are assuming that

so it follows that

Hence, the coefficient matrix of this system of congruences modulo q has non-zero determinant, and thus there is a unique solution to the system. That is, every d ∈ G is the correct response for exactly one of the q possible ordered pairs (e1, e2), Consequently, the probability that Bob gives Alice a response d that will be verified is exactly 1/q, and the theorem is proved.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

We now turn to the disavowal protocol. This protocol consists of two runs of the verification protocol and is presented in Figure 6.8.

Steps 1-4 and steps 5-8 comprise two unsuccessful runs of the verification protocol. Step 9 is a "consistency check" that enables Alice to determine if Bob is forming his responses in the manner specified by the protocol.

The following example illustrates the disavowal protocol.

Example 6.6

As before, suppose p = 467, α = 4, a = 101 and β = 449. Suppose the message x = 286 is signed with the (bogus) signature y = 83, and Bob wants to convince Alice that the signature is invalid.

*
* Figure 6.8 Disavowal protocol

Suppose Alice begins by choosing the random values e1 = 45, e2 = 237. Alice computes c = 305 and Bob responds with d = 109. Then Alice computes

Since 149 ≠ 109, Alice proceeds to step 5 of the protocol.

Now suppose Alice chooses the random values f1 = 125, f2 = 9. Alice computes C = 270 and Bob responds with D = 68. Alice computes

Since 25 ≠ 68, Alice proceeds to step 9 of the protocol and performs the consistency check. This check succeeds, since

and

Hence, Alice is convinced that the signature is invalid.

We have to prove two things at this point:

1. Bob can convince Alice that an invalid signature is a forgery.
2. Bob cannot make Alice believe that a valid signature is a forgery except with a very small probability.

THEOREM 6.2

If [image] (mod p), and Alice and Bob follow the disavowal protocol, then

PROOF Using the facts that

and

we have that

A similar computation, using the facts that [image] (mod p) and β ≡ αa (mod p), establishes that

so the consistency check in step 9 succeeds.

Now we look at the possibility that Bob might attempt to disavow a valid signature. In this situation, we do not assume that Bob follows the protocol. That is, Bob might not construct d and D as specified by the protocol. Hence, in the following theorem, we assume only that Bob is able to produce values d and D which satisfy the conditions in steps 4, 8, and 9 of the protocol presented in Figure 6.8.

THEOREM 6.3

Suppose y ≡ xa (mod p) and Alice follows the disavowal protocol. If

and

then the probability that

is 1 - 1/q.

PROOF Suppose that the following congruences are satisfied:

We will derive a contradiction.

The consistency check (step 9) can be rewritten in the following form:

where

is a value that depends only on steps 1-4 of the protocol.

Applying Theorem 6.1, we conclude that y is a valid signature for d0 with probability 1 - 1/q. But we are assuming that y is a valid signature for x. That is, with high probability we have

which implies that x = d0.

However, the fact that

means that

Since

we conclude that x ≠ d0 and we have a contradiction.

Hence, Bob can fool Alice in this way with probability 1/q.

6.6 Fail-stop Signatures

A fail-stop signature scheme provides enhanced security against the possibility that a very powerful adversary might be able to forge a signature. In the event that Oscar is able to forge Bob's signature on a message, Bob will (with high probability) subsequently be able to prove that Oscar's signature is a forgery.

In this section, we describe a fail-stop signature scheme constructed by van Heyst and Pedersen in 1992. This is a one-time scheme (only one message can be signed with a given key). The system consists of signing and verification algorithms, as well as a "proof of forgery" algorithm. The description of the signing and verification algorithms of the van Heyst and Pedersen Fail-stop Signature Scheme is presented in Figure 6.9.

It is straightforward to see that a signature produced by Bob will satisfy the verification condition, so let's turn to the security aspects of this scheme and how the fail-stop property works. First we establish some important facts relating to the keys of the scheme. We begin with a definition. Two keys (γ1, γ2, a1, a2, b1, b2) and [image] are said to be equivalent if [image] and [image]. It is easy to see that there are exactly q2 keys in any equivalence class.

We establish several lemmas.

LEMMA 6.4

Suppose K and K′ are equivalent keys and suppose that verK (x, y) = true. Then verK′ (x, y) = true.

PROOF Suppose K = (γ1, γ2, a1, a2, b1, b2) and K′ = ( [image]), where

and

Suppose x is signed using K, producing the signature y = (y1, y2), where

Now suppose that we verify y using K′:

Thus, y will also be verified using K′.

*
* Figure 6.9 van Heyst and Pedersen Fail-stop Signature Scheme

LEMMA 6.5

Suppose K is a key and y = sigK(x). Then there are exactly q keys K′ equivalent to K such that y = sigK′ (x).

PROOF Suppose γ1 and γ2 are the public components of K. We want to determine the number of 4-tuples (a1, a2, b1, b2) such that the following congruences are satisfied:


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Since α generates G, there exist unique exponents [image] such that

and

Hence, it is necessary and sufficient that the following system of congruences be satisfied:

This system can, in turn, be written as a matrix equation in [image], as follows:

Now, the coefficient matrix of this system can be seen to have rank 1 three: Clearly, the rank is at least three since rows 1, 2 and 4 are linearly independent over [image]. And the rank is at most three since

1the rank of a matrix is the maximum number of linearly independent rows it contains

where ri denotes ith row of the matrix.

Now, this system of equations has at least one solution, obtained by using the key K. Since the rank of the coefficient matrix is three, it follows that the dimension of the solution space is 4 - 3 = 1, and there are exactly q solutions. The result follows.

By similar reasoning, the following result can be proved. We omit the proof.

LEMMA 6.6

Suppose K is a key, y = sigK(x), and verK (x′, y′) = true, where x′ ≠ x. Then there is at most one key K′ equivalent to K such that y = sigK′(x) and y′ = sigK′(x′).

Let's interpret what the preceding two lemmas say about the security of the scheme. Given that y is a valid signature for message x, there are q possible keys that would have signed x with y. But for any message x′ ≠ x, these q keys will produce q different signatures on x′. Thus, the following theorem results.

THEOREM 6.7

Given that sigK(x) = y and x′ ≠ x, Oscar can compute sigK(x′) with probablity 1/q.

Note that this theorem does not depend on the computational power of Oscar: the stated level of security is obtained because Oscar cannot tell which of q possible keys is being used by Bob. So the security is unconditional.

We now go on to look at the fail-stop concept. What we have said so far is that, given a signature y on message x, Oscar cannot compute Bob's signature y′ on a different message x′. It is still conceivable that Oscar can compute a forged signature y″ ≠ sigK(x′) which will still be verified. However, if Bob is given a valid forged signature, then with probability 1 - 1/q he can produce a "proof of forgery." The proof of forgery is the value a0 = logα β, which is known only to the central authority.

So we assume that Bob possesses a pair (x′, y″) such that verK(x′, y″) = true and y″ ≠ sigK(x′). That is,

where y″ = ( [image]). Now, Bob can compute his own signature on x′, namely y′ = ( [image]), and it will be the case that

Hence,

Writing [image] mod p, we have that

or

This simplifies to give

Now, [image] (mod q) since y′ is a forgery. Hence, ( [image])-1 mod q exists, and

Of course, by accepting such a proof of forgery, we assume that Bob cannot compute the discrete logarithm logα β by himself. This is a computational assumption.

Finally, we remark that the scheme is a one-time scheme since Bob's key K can easily be computed if two messages are signed using K.

We close with an example illustrating how Bob can produce a proof of forgery.

Example 6.7

Suppose p = 3467 = 2 × 1733 + 1. The element α = 4 has order 1733 in [image]. Suppose that a0 = 1567, so

(Recall that Bob knows the values of α and β, but not a0.) Suppose Bob forms his key using a1 = 888, a2 = 1024, b1 = 786 and b2 = 999, so

and

Now, suppose Bob is presented with the forged signature (822, 55) on the message 3383. This is a valid signature since the verification condition is satisfied:

and

On the other hand, this is not the signature Bob would have constructed. Bob can compute his own signature to be

Then, he proceeds to calculate the secret discrete log

This is the proof of forgery.

6.7 Notes and References

For a nice survey of signature schemes, we recommend Mitchell, Piper, and Wild [MPW92]. This paper also contains the two methods of forging ElGamal signatures that we presented in Section 6.2.

The ElGamal Signature Scheme was presented in ElGamal [EL85]. The Digital Signature Standard was first published by NIST in August 1991, and it was adopted as a standard in December 1994 [NBS94]. There is a lengthy discussion of DSS and the controversy surrounding it in the July 1992 issue of the Communications of the ACM. For a response by NIST to some of the questions raised, see [SB93].

The Lamport Scheme is described in the 1976 paper by Diffie and Hellman [DH76]; the modification by Bos and Chaum is in [BC93]. The undeniable signature scheme presented in Section 6.5 is due to Chaum and van Antwerpen [CVA90]. The fail-stop signature scheme from Section 6.6 is due to van Heyst and Pedersen [VHP93].

Some examples of well-known "broken" signature schemes include the Ong-Schnorr-Shamir Scheme [OSS85] (broken by Estes et al. [EAKMM86]); and the Birational Permutation Scheme of Shamir [SH94] (broken by Coppersmith, Stern, and Vaudenay [CSV94]). Finally, ESIGN is a signature scheme due to Fujioka, Okamoto, and Miyaguchi [FOM91]. Some versions of the scheme were broken, but the variation in [FOM91] has not been broken.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Exercises

6.1 Suppose Bob is using the ElGamal Signature Scheme, and he signs two messages x1 and x2 with signatures (γ, δ1) and (γ, δ2), respectively. (The same value for γ occurs in both signatures.) Suppose also that gcd(δ1 - δ2, p - 1) = 1.

(a) Describe how k can be computed efficiently given this information.
(b) Describe how the signature scheme can then be broken.
(c) Suppose p = 31847, α = 5 and β = 25703. Perform the computation of k and a, given the signature (23972, 31396) for the message x = 8990 and the signature (23972, 20481) for the message x = 31415.

6.2 Suppose I implement the ElGamal Signature Scheme with p = 31847, α = 5 and β = 26379. Write a computer program which does the following.

(a) Verify the signature (20679, 11082) on the message x = 20543.
(b) Determine my secret exponent, a, using the Shanks time-memory tradeoff. Then determine the random value k used in signing the message x.

6.3 Suppose Bob is using the ElGamal Signature Scheme as implemented in Example 6.1: p = 467, α = 2 and β = 132. Suppose Bob has signed the message x = 100 with the signature (29, 51). Compute the forged signature that Oscar can then form by using h = 102, i = 45 and j = 293. Check that the resulting signature satisfies the verification condition.
6.4 Prove that the second method of forgery on the ElGamal Signature Scheme, described in Section 6.2, also yields a signature that satisfies the verification condition.
6.5 Here is a variation of the ElGamal Signature Scheme. The key is constructed in a similar manner as before: Bob chooses [image] to be a primitive element, a is a secret exponent (0 ≤ a ≤ p - 2) such that gcd (a, p - 1) = 1, and β = αa mod p. The key K = (α, a, β), where α and β are public and a is secret. Let [image] be a message to be signed. Bob computes the signature sig(x) = (γ, δ), where

and

The only difference from the original ElGamal Scheme is in the computation of δ. Answer the following questions concerning this modified scheme.

(a) Describe how a signature (γ, δ) on a message x would be verified using Bob's public key.
(b) Describe a computational advantage of the modified scheme over the original scheme.
(c) Briefly compare the security of the original and modified scheme.

6.6 Suppose Bob uses the DSS with q = 101, p = 7879, α = 170, a = 75 and β = 4567, as in Example 6.3. Determine Bob's signature on the message x = 52 using the random value k = 49, and show how the resulting signature is verified.
6.7 In the Lamport Scheme, suppose that two k-tuples, x and x′, are signed by Bob. Let [image] denote the number of coordinates in which x and x′ differ. Show at Oscar can now sign [image] new messages.
6.8 In the Bos-Chaum Scheme with k = 6 and n = 4, suppose that the messages x = (0, 1, 0, 0, 1, 1) and x′ = (1, 1, 0, 1, 1, 1) are signed. Determine the new messages that be signed by Oscar, knowing the signatures on x and x′.
6.9 In the Bos-Chaum Scheme, suppose that two k-tuples x and x′ are signed by Bob. Let [image] Show that Oscar can now sign [image] new messages.
6.10 Suppose Bob is using the Chaum-van Antwerpen Undeniable Signature Scheme as in Example 6.5. That is, p = 467, α = 4, a = 101 and β = 449. Suppose Bob is presented with a signature y = 25 on the message x = 157 and he wishes to prove it is a forgery. Suppose Alice's random numbers are e1 = 46, e2 = 123, f1 = 198 and f2 = 11 in the disavowal protocol. Compute Alice's challenges, c and d, and Bob's responses, C and D, and show that Alice's consistency check will succeed.
6.11 Prove that each equivalence class of keys in the Pedersen-van Heyst Fail-stop Signature Scheme contains q2 keys.
6.12 Suppose Bob is using the Pedersen-van Heyst Fail-stop Signature Scheme, where p = 3467, α = 4, a0 = 1567 and β = 514 (of course, the value of a0 is not known to Bob).

(a) Using the fact that a0 = 1567, determine all possible keys

such that sigK(42) = (1118, 1449).

(b) Suppose that sigK(42) = (1118, 1449) and sigK(969) = (899, 471). Without using the fact that a0 = 1567, determine the value of K (this shows that the scheme is a one-time scheme).

6.13 Suppose Bob is using the Pedersen-van Heyst Fail-Stop Signature Scheme with p = 5087, α = 25 and β = 1866. Suppose the key is

Now, suppose Bob finds the signature (2219, 458) has been forged on the message 4785.

(a) Prove that this forgery satisfies the verification condition, so it is a valid signature.
(b) Show how Bob will compute the proof of forgery, a0, given this forged signature.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 7
Hash Functions

7.1 Signatures and Hash Functions

The reader might have noticed that the signature schemes described in Chapter 6 allow only "small" messages to be signed. For example, when using the DSS, a 160-bit message is signed with a 320-bit signature. In general, we will want to sign much longer messages. A legal document, for example, might be many megabytes in size.

A naive attempt to solve this problem would be to break a long message into 160-bit chunks, and then to sign each chunk independently. This is analogous to encrypting a long string of plaintext by encrypting each plaintext character independently using the same key (e.g., ECB mode in the DES).

But there are several problems with this approach in creating digital signatures. First of all, for a long message, we will end up with an enormous signature (twice as long as the original message in the case of the DSS). Another disadvantage is that most "secure" signature schemes are slow since they typically use complicated arithmetic operations such as modular exponentiation. But an even more serious problem with this approach is that the various chunks of a signed message could be rearranged, or some of them removed, and the resulting message would still be verified. We need to protect the integrity of the entire message, and this cannot be accomplished by independently signing little pieces of it.

The solution to all of these problems is to use a very fast public cryptographic hash function, which will take a message of arbitrary length and produce a message digest of a specified size (160 bits if the DSS is to be used). The message digest will then be signed. For the DSS, the use of a hash function h is depicted diagramatically in Figure 7.1

When Bob wants to sign a message x, he first constructs the message digest z = h(x), and then computes the signature y = sigK(z). He transmits the ordered pair (x, y) over the channel. Now the verification can be performed (by anyone) by first reconstructing the message digest z = h(x) using the public hash function h, and then checking that verK(z, y) = true.

*
* Figure 7.1 Signing a message digest

7.2 Collision-free Hash Functions

We have to be careful that the use of a hash function h does not weaken the security of the signature scheme, for it is the message digest that is signed, not the message. It will be necessary for h to satisfy certain properties in order to prevent various forgeries.

The most obvious type of attack is for an opponent, Oscar, to start with a valid signed message (x, y), where y = sigK(h(x)). (The pair (x, y) could be any message previously signed by Bob.) Then he computes z = h(x) and attempts to find x′ ≠ x such that h(x′) = h(x). If Oscar can do this, (x′, y) would be a valid signed message, i.e., a forgery. In order to prevent this type of attack, we require that h satisfy the following collision-free property:

DEFINITION 7.1 Let x be a message. A hash function h is weakly collision-free for x if it is computationally infeasible to find a message x′ ≠ x such that h(x′) = h(x).

Another possible attack is the following: Oscar first finds two messages x ≠ x′ such that h(x) = h(x′). Oscar then gives x to Bob and persuades him to sign the message digest h(x), obtaining y. Then (x′, y) is a valid forgery.

This motivates a different collision-free property:

DEFINITION 7.2 A hash function h is strongly collision-free if it is computationally infeasible to find messages x and x′ such that x′ ≠ x and h(x′) = h(x).

Observe that a hash function h is strongly collision-free if and only if it in computationally infeasible to find a message x such that h is not weakly collision-free for x.

Here is a third variety of attack. As we mentioned in Section 6.2, it is often possible with certain signature schemes to forge signatures on random message digests z. Suppose Oscar computes a signature on such a random z, and then he finds a message x such that z = h(x). If he can do this, then (x, y) is a valid forgery. To prevent this attack, we desire that h satisfy the same one-way property that was mentioned previously in the context of public-key cryptosystems and the Lamport Signature Scheme:

DEFINITION 7.3 A hash function h is one-way if, given a message digest z, it is computationally infeasible to find a message x such that h(x) = z.

We are now going to prove that the strongly collision-free property implies the one-way property. This is done by proving the contrapositive statement. More specifically, we will prove that an arbitrary inversion algorithm for a hash function can be used as an oracle in a Las Vegas probabilistic algorithm that finds collisions.

This reduction can be accomplished with a fairly weak assumption on the relative sizes of the domain and range of the hash function. We will assume for the time being that the hash function h : X → Z, where X and Z are finite sets and |X| ≥ 2|Z|. This is a reasonable assumption: If we think of an element of X as being encoded as a bitstring of length log2 |X| and an element of Z as being encoded as a bitstring of length log2 |Z|, then the message digest z = h(x) is at least one bit shorter than the message x. (Eventually, we will be interested in the situation where the message domain X is infinite, since we want to be able to deal with messages of arbitrary length. Our argument also applies in this situation.)

We are assuming that we have an inversion algorithm for h. That is, we have an algorithm A which accepts as input a message digest z ∈ Z, and finds an element A(z) ∈ X such that h(A(z)) = z.

We prove the following theorem.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

THEOREM 7.1

Suppose h : X → Z is a hash function where |X| and |Z| are finite and |X| ≥ 2|Z|. Suppose A is an inversion algorithm for h. Then there exists a probabilistic Las Vegas algorithm which finds a collision for h with probability at least 1/2.

PROOF Consider the algorithm B presented in Figure 7.2. Clearly B is a probabilistic algorithm of the Las Vegas type, since it either finds a collision or returns no answer. Thus our main task is to compute the probability of success. For any x ∈ X, define x ~ x1 if h(x) = h(x1). It is easy to see that ~ is an equivalence relation. Define

Each equivalence class [x] consists of the inverse image of an element of Z, so the number of equivalence classes is at most |Z|. Denote the set of equivalence classes by C.

Now, suppose x is the element of X chosen in step 1. For this x, there are |[x]| possible x1's that could be returned in step 3. |[x]| - 1 of these x1's are different from x and thus lead to success in step 4. (Note that the algorithm A does not know the representative of the equivalence class [x] that was chosen in step 1.) So, given a particular choice x ∈ X, the probability of success is (|[x]| - 1)/|[x]|.

*
* Figure 7.2 Using an inversion algorithm A to find collisions for a hash function h

The probability of success of the algorithm B is computed by averaging over all possible choices for x:

Hence we have constructed a Las Vegas algorithm with success probability at least 1/2.

Hence, it is sufficient that a hash function satisfy the strongly collision-free property, since it implies the other two properties. So in the remainder of this chapter we restrict our attention to strongly collision-free hash functions.

7.3 The Birthday Attack

In this section, we determine a necessary security condition for hash functions that depends only on the cardinality of the set Z (equivalently, on the size of the message digest). This necessary condition results from a simple method of finding collisions which is informally known as the birthday attack. This terminology arises from the so-called birthday paradox, which says that in a group of 23 random people, at least two will share a birthday with probability at least 1/2. (Of course this is not a paradox, but it is probably counter-intuitive). The reason for the terminology "birthday attack" will become clear as we progress.

As before, let us suppose that h : X → Z is a hash function, X and Z are finite, and |X| ≥ 2|Z|. Denote |X| = m and |Z| = n. It is not hard to see that there are at least n collisions — the question is how to find them. A very naive approach is to choose k random distinct elements x1,...,xk ∈ X, compute zi = h(xi), 1 ≤ i ≤ k, and then determine if a collision has taken place (by sorting the zi's, for example).

This process is analogous to throwing k balls randomly into n bins and then checking to see if some bin contains at least two balls. (The k balls correspond to the k random xi's, and the n bins correspond to the n possible elements of Z.)

We will compute a lower bound on the probability of finding a collision by this method. This lower bound will depend on k and n, but not on m. Since we are interested in a lower bound on the collision probability, we will make the assumption that [image] for all z ∈ Z. (This is a reasonable assumption: if the inverse images are not approximately equal, then the probability of finding a collision will increase.)

Since the inverse images are all (roughly) the same size and the xi's are chosen at random, the resulting zi's can be thought of as random (not necessarily distinct) elements of Z. But it is a simple matter to compute the probability that k random elements z1,...,zk ∈ Z are distinct. Consider the zi's in the order z1,...,zk. The first choice z1 is arbitrary; the probability that z2 ≠ z1 is 1 - 1/n; the probability that z3 is distinct from z1 and z2 is 1 - 2/n, etc.

Hence, we estimate the probability of no collisions to be

If x is a small real number, then [image]. This estimate is derived by taking the first two terms of the series expansion

Then our estimated probability of no collisions is

So we estimate the probability of at least one collision to be

If we denote this probability by ∈, then we can solve for k as a function of n and ∈.

If we ignore the term -k, then we estimate

If we take ∈ = .5, then our estimate is

So this says that hashing just over [image] random elements of X yields a collision with a probability of 50%. Note that a different choice of ∈ leads to a different constant factor, but k will still be proportional to

If X is the set of all human beings, Y is the set of 365 days in a non-leap year (i.e., excluding February 29), and h(x) denotes the birthday of person x, then we are dealing with the birthday paradox. Taking n = 365 in our estimate, we get [image] Hence, as mentioned earlier, there will be at least one duplicated birthday among 23 random people with probability at least 1/2.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

This birthday attack imposes a lower bound on the sizes of message digests. A 40-bit message digest would be very insecure, since a collision could be found with probability 1/2 with just over 220 (about a million) random hashes. It is usually suggested that the minimum acceptable size of a message digest is 128 bits (the birthday attack will require over 264 hashes in this case). The choice of a 160-bit message digest for use in the DSS was undoubtedly motivated by these considerations.

*
* Figure 7.3 Chaum-van Heijst-Pfitzmann Hash Function

7.4 A Discrete Log Hash Function

In this section, we describe a hash function, due to Chaum, van Heijst, and Pfitzmann, that will be secure provided a particular discrete logarithm cannot be computed. This hash function is not fast enough to be of practical use, but it is conceptually simple and provides a nice example of a hash function that can be proved secure under a reasonable computational assumption. The Chaum-van Heijst-Pfitzmann Hash Function is presented in Figure 7.3. We now prove a theorem concerning the security of this hash function.

THEOREM 7.2

Given one collision for the Chaum-van Heijst-Pfitzmann Hash Function h, the discrete logarithm logα β can be computed efficiently.

PROOF Suppose we are given a collision

where (x1, x2) ≠ (x3, x4). So we have the following congruence:

or

Denote

Since p - 1 = 2q and q is prime, it must be the case that d ∈ {1,2,q,p - 1}. Hence, we have four possibilities for d, which we will consider in turn.

First, suppose that d = 1. Then let

We have that

so we can compute the discrete logarithm logα β as follows:

Next, suppose that d = 2. Since p - 1 = 2q where q is odd, we must have gcd(x4 - x2, q) = 1. Let

Now

for some integer k, so we have

since

So we have

It follows that

or

We can easily test which of these two possibilities is the correct one. Hence, as in the case d = 1, we have calculated the discrete logarithm logα β.

The next possibility is that d = q. But

and

so

So it is impossible that gcd(x4 - x2,p - 1) = q; in other words, this case does not arise.

The final possibility is that d = p - 1. This happens only if x2 = x4. But then we have

so

and x1 = x3. Thus (x2, x2) = (x3, x4), a contradiction. So this case is not possible, either.

Since we have considered all possible values for d, we conclude that the hash function h is strongly collision-free provided that it is infeasible to compute the discrete logarithm logα β in [image].

We illustrate the result of the above theorem with an example.

Example 7.1

Suppose p = 12347 (so q = 6173), α = 2 and β = 8461. Suppose we are given the collision

Thus x1 = 5692, x2 = 144, x3 = 212 and x4 = 4214. Now, gcd (x4 - x2, p -1) = 2, so we begin by computing

Next, we compute

Now it is the case that logα β ∈ {y′, y′ + q mod (p - 1)}. Since

we conclude that

As a check, we can verify that

Hence, we have determined logα β.

7.5 Extending Hash Functions

So far, we have considered hash functions with a finite domain. We now study how a strongly collision-free hash function with a finite domain can be extended to a strongly collision-free hash function with an infinite domain. This will enable us to sign messages of arbitrary length.

Suppose h : is a strongly collision-free hash function, where m ≥ t + 1. We will use h to construct a strongly collision-free hash function h* : , where

We first consider the situation where m ≥ t + 2.

We will think of elements of X as bit-strings. |x| denotes the length of x (i.e., the number of bits in x), and x || y denotes the concatenation of the bit-strings x and y. Suppose |x| = n > m. We can express x as the concatenation

where

and

where 0 ≤ d ≤ m - t - 2. Hence, we have that

We define h* (x) by the algorithm presented in Figure 7.4.

*
* Figure 7.4 Extending a hash function h to h* (m ≥ t + 2)

Denote

Observe that yk is formed from xk by padding on the right with d zeroes, so that all the blocks yi (1 ≤ i ≤ k) are of length m - t - 1. Also, in step 3, yk+1 should be padded on the left with zeroes so that |yk+1| = m - t - 1.

In order to hash x, we first construct y(x), and then "process" the blocks y1, y2,...,yk+1 in a particular fashion. It is important that y(x) ≠ y(x′) whenever x ≠ x′. In fact, yk+1 is defined in such a way that the mapping [image] will be an injection.

The following theorem proves that h* is secure provided that h is secure.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

THEOREM 7.3

Suppose h : is a strongly collision-free hash function, where m ≥ t + 2. Then the function h* : , as constructed in Figure 7.4, is a strongly collision-free hash function.

PROOF Suppose that we can find x ≠ x′ such that h*(x) = h*(x′). Given such a pair, we will show how we can find a collision for h in polynomial time. Since h is assumed to be strongly collision-free, we will obtain a contradiction, and thus h* will be proved to be strongly collision-free.

Denote

and

where x and x′ are padded with d and d′ 0's, respectively, in step 2. Denote the values computed in steps 4 and 5 by g1,...,gk+1 and [image], respectively.

We identify two cases, depending on whether or not |x| ≡ |x′| (mod m-t-1).

case 1: (mod m - t - 1).

Here d ≠ d′ and [image]. We have

which is a collision for h since [image].

case 2: |x| ≡ |x′| (mod m - t - 1).

It is convenient to split this into two subcases:

case 2a: |x| = |x′|.

Here we have [image] and [image]. We begin as in case 1:

If [image], then we find a collision for h, so assume [image]. Then we have

Either we find a collision for h, or [image] and [image]. Assuming we do not find a collision, we continue working backwards, until finally we obtain

If [image], then we find a collision for h, so we assume [image]. But then [image] for 1 ≤ i ≤ k + 1, so y(x) = y(x′). But this implies x = x′ since the mapping [image] is an injection. Since we assumed x ≠ x′, we have a contradiction.

*
* Figure 7.5 Extending a hash function h to h* (m = t + 1)

case 2b: |x| ≠ |x′|.

Without loss of generality, assume |x′| > |x|, so [image]. This case proceeds in a similar fashion as case 2a. Assuming we find no collisions for h, we eventually reach the situation where

But the (t + 1)st bit of 0t+1 || y1 is a 0 and the (t + 1)st bit of [image] is a 1. So we find a collision for h.

Since we have considered all possible cases, we have the desired conclusion.

The construction of Figure 7.4 can be used only when m ≥ t + 2. Let's now look at the situation where m = t + 1. We need to use a different construction for h*. As before, suppose |x| = n > m. We first encode x in a special way. This will be done using the function f defined as follows:

The algorithm to construct h* (x) is presented in Figure 7.5.

The encoding [image], defined in step 1, satisfies two important properties:

1. If x ≠ x′, then y(x) ≠ y(x′) (i.e., [image] is an injection).
2. There do not exist two strings x ≠ x′ and a string z such that y(x) = z || y(x′). (In other words, no encoding is a postfix of another encoding. This is easily seen because each string y(x) begins with 11, and there do not exist two consecutive 1's in the remainder of the string.)

THEOREM 7.4

Suppose h : is a strongly collision-free hash function. Then the function h* : , as constructed in Figure 7.5, is a strongly collision-free hash function.

PROOF Suppose that we can find x ≠ x′ such that h* (x) = h* (x′). Denote

and

We consider two cases.

case 1: .

As in Theorem 7.3, either we find a collision for h, or we obtain y = y′. But this implies x = x′, a contradiction.

case 2: .

Without loss of generality, assume [image]. This case proceeds in a similar fashion. Assuming we find no collisions for h, we have the following sequence of equalities:

But this contradicts the "postfix-free" property stated above.

We conclude that h* is collision-free.

We summarize the two constructions of in this section, and the number of applications of h needed to compute h*, in the following theorem.

THEOREM 7.5

Suppose h : is a strongly collision-free hash function, where m ≥ t + 1. Then there exists a strongly collision-free hash function

The number of times h is computed in the evaluation of h* is at most

where |x| = n.

7.6 Hash Functions from Cryptosystems

So far, the methods we have described lead to hash functions that are probably too slow to be useful in practice. Another approach is to use an existing private-key cryptosystem to construct a hash function. Let us suppose that [image] is a computationally secure cryptosystem. For convenience, let us assume also that [image]. Here we should have n ≥ 128, say, in order to prevent birthday attacks. This precludes using DES (as does the fact that the key length of DES is different from the plaintext length).

Suppose we are given a bitstring

where [image]. (If the number of bits in x is not a multiple of n, then it will be necessary to pad x in some way, such as was done in Section 7.5. For simplicity, we will ignore this now.)

The basic idea is to begin with a fixed "initial value" g0 = IV, and then construct g1,...,gk in order by a rule of the form

where f is a function that incorporates the encryption function of our cryptosystem. Finally, define the message digest h(x) = gk.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Several hash functions of this type have been proposed, and many of them have been shown to be insecure (independent of whether or not the underlying cryptosystem is secure). However, four variations of this theme that appear to be secure are as follows:

*
* Figure 7.6 Constructing M in MD4

7.7 The MD4 Hash Function

The MD4 Hash Function was proposed in 1990 by Rivest, and a strengthened version, called MD5, was presented in 1991. The Secure Hash Standard (or SHS) is more complicated, but it is based on the same underlying methods. It was published in the Federal Register on January 31, 1992, and adopted as a standard on May 11, 1993. (A proposed revision was put forward on July 11, 1994, to correct a "technical flaw" in the SHS.) All of the above hash functions are very fast, so they are practical for signing very long messages.

In this section, we will describe MD4 in detail, and discuss some of the modifications that are employed in MD5 and the SHS.

Given a bitstring x, we will first produce an array

where each M[i] is a bitstring of length 32 and N ≡ 0 mod 16. We will call each M[i] a word. M is constructed from x using the algorithm presented in Figure 7.6.

In the construction of M, we append a single 1 to x, then we concatenate enough 0's so that the length becomes congruent to 448 modulo 512, and finally we concatenate 64 bits that contain the binary representation of the (original) length of x (reduced modulo 264, if necessary). The resulting string M has length divisible by 512. So when we break M up into 32-bit words, the resulting number of words, denoted by N, will be divisible by 16.

Now we proceed to construct a 128-bit message digest. A high-level description of the algorithm is presented in Figure 7.7. The message digest is constructed as the concatenation of the four words A, B, C and D, which we refer to as registers. The four registers are initialized in step 1. Now we process the array M 16 words at a time. In each iteration of the loop in step 2, we first take the "next" 16 words of M and store them in an array X (step 3). The values of the four registers are then stored (step 4). Then we perform three "rounds" of hashing. Each round consists of one operation on each of the 16 words in X (we will describe these operations in more detail shortly). The operations done in the three rounds produce new values in the four registers. Finally, the four registers are updated in step 8 by adding back the values that were stored in step 4. This addition is defined to be addition of positive integers, reduced modulo 232.

*
* Figure 7.7 The MD4 hash function

The three rounds in MD4 are different (unlike DES, say, where the 16 rounds are identical). We first describe several different operations that are employed in these three rounds. In the following description, X and Y denote input words, and each operation produces a word as output. Here are the operations employed:

Note that all of these operations are very fast, and the only arithmetic operation that is used is addition modulo 232. If MD4 is actually implemented, it will be necessary to take into account the underlying architecture of the computer it is run on in order to perform addition correctly. Suppose a1a2a3a4 are the four bytes in a word. We think of each ai as being an integer in the range 0,...,255, represented in binary. In a big-endian architecture (such as a Sun SPARCstation), this word represents the integer

In a little-endian architecture (such as the Intel 80xxx line), this word represents the integer

MD4 assumes a little-endian architecture. It is important that the message digest is independent of the underlying architecture. So if we wish to run MD4 on a big-endian computer, it will be necessary to perform the addition operation X + Y as follows:

1. Interchange x1 and x4; x2 and x3; y1 and y4; and y2 and y3.
2. Compute Z = X + Y mod 232
3. Interchange z1 and z4; and z2 and z3.

Rounds 1, 2, and 3 of MD4 respectively use three functions f, g and h. Each of f, g and h is a bitwise boolean function that takes three words as input and produces a word as output. They are defined as follows:

The complete description of Rounds 1, 2 and 3 of MD4 are presented in Figures 7.8-7.10.

MD4 was designed to be very fast, and indeed, software implementations on Sun SPARCstations attain speeds of 1.4 Mbytes/sec. On the other hand, it is difficult to say something concrete about the security of a hash function such as MD4 since it is not "based" on a well-studied problem such as factoring or the Discrete Log problem. So, as is the case with DES, confidence in the security of the system can only be attained over time, as the system is studied and (one hopes) not found to be insecure.

*
* Figure 7.8 Round 1 of MD4

Although MD4 has not been broken, weakened versions that omit either the first or the third round can be broken without much difficulty. That is, it is easy to find collisions for these two-round versions of MD4. A strengthened version of MD4, called MD5, was proposed in 1991. MD5 uses four rounds instead of three, and runs about 30% slower than MD4 (about .9 Mbytes/sec on a SPARCstation).


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

The Secure Hash Standard is yet more complicated, and slower (about .2 Mbytes/sec on a SPARCstation). We will not give a complete description, but we will indicate a few of the modifications employed in the SHS.

1. SHS is designed to run on a big-endian architecture, rather than a little-endian architecture.
2. SHS produces a 5-register (160-bit) message digest.

*
* Figure 7.9 Round 2 of MD4

3. SHS processes the message 16 words at a time, as does MD4. However, the 16 words are first "expanded" into 80 words. Then a sequence of 80 operations is performed, one on each word.

The following "expansion function" is used. Given the 16 words X[0],..., X[15], we compute 64 more words by the recurrence relation

The result of Equation 7.1 is that each of the words X[16],..., X[79] is formed as the exclusive-or of a predetermined subset of the words X[0],...,X[15].

For example, we have

*
* Figure 7.10 Round 3 of MD4

The proposed revision of the SHS concerns the expansion function. It is proposed that Equation 7.1 be replaced by the following:

As before, the operation " [image] " means a circular left shift of one position.

7.8 Timestamping

One difficulty with signature schemes is that a signing algorithm may be compromised. For example, suppose that Oscar is able to determine Bob's secret exponent a in the DSS. Then, of course, Oscar can forge Bob's signature on any message he likes. But another (perhaps even more serious) problem is that the compromise of a signing algorithm calls in to question the authenticity of all messages signed by Bob, including those he signed before Oscar stole the signing algorithm.

*
* Figure 7.11 Timestamping a signature on a message x

Here is yet another undesirable situation that could arise: Suppose Bob signs a message and later wishes to disavow it. Bob might publish his signing algorithm and then claim that his signature on the message in question is a forgery.

The reason these types of events can occur is that there is no way to determine when a message was signed. This suggests that we consider ways of timestamping a (signed) message. A timestamp should provide proof that a message was signed at a particular time. Then, if Bob's signing algorithm is compromised, it would not invalidate any signatures he made previously. This is similar conceptually to the way credit cards work: if someone loses a credit card and notifies the bank that isssued it, it becomes invalid. But purchases made prior to the loss of the card are not affected.

In this section, we will describe a few methods of timestamping. First, we observe that Bob can produce a convincing timestamp on his own. First, Bob obtains some "current" publicly available information which could not have been predicted before it happened. For example, such information might consist of all the major league baseball scores from the previous day, or the values of all the stocks listed on the New York Stock Exchange. Denote this information by pub.

Now, suppose Bob wants to timestamp his signature on a message x. We assume that h is a publicly known hash function. Bob will proceed according to the algorithm presented in Figure 7.11. Here is how the scheme works: The presence of the information pub means that Bob could not have produced y before the date in question. And the fact that y is published in the next day's newspaper proves that Bob did not compute y after the date in question. So Bob's signature y is bounded within a period of one day. Also observe that Bob does not reveal the message x in this scheme since only z is published. If necessary, Bob can prove that x was the message he signed and timestamped simply by revealing it.

*
* Figure 7.12 Timestamping (zn, yn, IDn)

It is also straightforward to produce timestamps if there is a trusted timestamping service available (i.e., an electronic notary public). Bob can compute z = h(x) and y = sigK(z) and then send (z, y) to the timestamping service, or TSS. The TSS will then append the date D and sign the triple (z, y, D). This works perfectly well provided that the signing algorithm of the TSS remains secure and provided that the TSS cannot be bribed to backdate timestamps. (Note also that this method establishes only that Bob signed a message before a certain time. If Bob also wanted to establish that he signed it after a certain date, he could incorporate some public information pub as in the previous method.)

If it is undesirable to trust the TSS unconditionally, the security can be increased by sequentially linking the messages that are timestamped. In such a scheme, Bob would send an ordered triple (z, y, ID(Bob)) to the TSS. Here z is the message digest of the message x; y is Bob's signature on z; and ID(Bob) is Bob's identifying information. The TSS will be timestamping a sequence of triples of this form. Denote by (zn, yn, IDn) the nth triple to be timestamped by the TSS, and let tn denote the time at which the nth request is made.

The TSS will timestamp the nth triple using the algorithm in Figure 7.12. The quantity Ln is "linking information" that ties the nth request to the previous one. (L0 will be taken to be some predetermined dummy information to get the process started.)

Now, if challenged, Bob can reveal his message xn, and then yn can be verified. Next, the signature sn of the TSS can be verified. If desired, then IDn-1 or IDn+1 can be requested to produce their timestamps, (Cn-1, sn-1, IDn) and (Cn+1, sn+1, IDn+2), respectively. The signatures of the TSS can be checked in these timestamps. Of course, this process can be continued as far as desired, backwards and/or forwards.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

7.9 Notes and References

The discrete log hash function described in Section 7.4 is due to Chaum, van Heijst, and Pfitzmann [CvHP92]. A hash function that can be proved secure provided that a composite integer n cannot be factored is given by Gibson [GIB91] (see Exercise 7.4 for a description of this scheme).

The material on extending hash functions in Section 7.5 is based on Damgård [DA90]. Similar methods were discovered by Merkle [ME90].

For infomation concerning the construction of hash functions from private-key cryptosystems, see Preneel, Govaerts, and Vandewalle [PGV94].

The MD4 hashing algorithm was presented in Rivest [RI91], and the Secure Hash Standard is described in [NBS93]. An attack against two of the three rounds of MD4 is given by den Boer and Bossalaers [DBB92]. Other recently proposed hash functions include N-hash [MOI90] and Snefru [ME90A].

Timestamping is discussed in Haber and Stornetta [HS91] and Bayer, Haber, and Stornetta [BHS93].

A thorough survey of hashing techniques can be found in Preneel, Govaerts, and Vandewalle [PGV93].

Exercises

7.1 Suppose h : X → Y is a hash function. For any y ∈ Y, let

and denote sy = |h-1(y)|. Define

Note that N counts the number of unordered pairs in X that collide under h. Answer the following:

(a) Prove that

so the mean of the sy 's is

(b) Prove that

(c) Prove that

*
* Figure 7.13 Hashing 4m bits to m bits

(d) Using the result proved in part (c), prove that

Further, show that equality is attained if and only if

for every y ∈ Y.

7.2 As in Exercise 7.1, suppose h : X → Y is a hash function, and let

for any y ∈ Y. Let ∈ denote the probability that h(x1) = h(x2), where x1 and x2 are random (not necessarily distinct) elements of X. Prove that

with equality if and only if

for every y ∈ Y.

7.3 Suppose p = 15083, α = 154 and β = 2307 in the Chaum-van Heijst-Pfitzmann Hash Function. Given the collision

compute logα β.

7.4 Suppose n = pq, where p and q are two (secret) distinct large primes such that p = 2p1 + 1 and q = 2q1 + 1, where p1 and q1 are prime. Suppose that α is an element of order 2p1q1 in [image] (this is the largest order of any element in [image]). Define a hash function h : {1,...,n2} → [image] by the rule h(x) = αx mod n.

Now, suppose that n = 603241 and α = 11 are used to define a hash function h of this type. Suppose that we are given three collisions for h: h(1294755) = h(80115359) = h(52738737). Use this information to factor n.

7.5 Suppose h1 : is a strongly collision-free hash function.

(a) Define h2 : as in Figure 7.13. Prove that h2 is strongly collision-free.
(b) For an integer i ≥ 2, define a hash function hi : recursively from hi-1, as indicated in Figure 7.14. Prove that hi is strongly collision-free.

7.6 Using the (original) expansion function of the SHS, Equation 7.1, express each of X[16],..., X[79] in terms of X[0],..., X[15]. Now, for each pair X[i], X[j],

*
* Figure 7.14 Hashing 2i m bits to m bits

where 1 ≤ i < j ≤ 15, use a computer program to determine λij, which denotes the number of X[k]'s (16 ≤ k ≤ 79) such that X[i] and X[j] both occur in the expression for X[k]. What is the range of values λij?


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 8
Key Distribution and Key Agreement

8.1 Introduction

We have observed that public-key systems have the advantage over private-key systems that a secure channel is not needed to exchange a secret key. But, unfortunately, most public-key systems are much slower than private-key systems such as DES, for example. So, in practice, private-key systems are usually used to encrypt "long" messages. But then we come back to the problem of exchanging secret keys.

In this chapter, we discuss several approaches to the problem of establishing secret keys. We will distinguish between key distribution and key agreement. Key distribution is defined to be a mechanism whereby one party chooses a secret key and then transmits it to another party or parties. Key agreement denotes a protocol whereby two (or more) parties jointly establish a secret key by communicating over a public channel. In a key agreement scheme, the value of the key is determined as a function of inputs provided by both parties.

As our setting, we have an insecure network of n users. In some of our schemes, we will have a trusted authority (denoted by TA) that is reponsible for such things as verifying the identities of users, choosing and transmitting keys to users, etc.

Since the network is insecure, we need to protect against potential opponents. Our opponent, Oscar, might be a passive adversary, which means that his actions are restricted to eavesdropping on messages that are transmitted over the channel. On the other hand, we might want to guard against the possibility that Oscar is an active adversary. An active adversary can do various types of nasty things such as the following:

1. alter messages that he observes being transmitted over the network
2. save messages for reuse at a later time
3. attempt to masquerade as various users in the network.

The objective of an active adversary might be one of the following:

1. to fool U and V into accepting an "invalid" key as valid (an invalid key could be an old key that has expired, or a key chosen by the adversary, to mention two possibilities)
2. to make U or V believe that they have exchanged a key with other when they have not.

The objective of a key distribution or key agreement protocol is that, at the end of the protocol, the two parties involved both have possession of the same key K, and the value of K is not known to any other party (except possibly the TA). Certainly it is much more difficult to design a protocol providing this type of security in the presence of an active adversary as opposed to a passive one.

We first consider the idea of key predistribution in Section 8.2. For every pair of users {U, V}, the TA chooses a random key KU,V = KV,U and transmits it "off-band" to U and V over a secure channel. (That is, the transmission of keys does not take place over the network, since the network is not secure.) This approach is unconditionally secure, but it requires a secure channel between the TA and every user in the network. But, of possibly even more significance is the fact that each user must store n - 1 keys, and the TA needs to transmit a total of [image] keys securely (this is sometimes called the "n2 problem"). Even for relatively small networks, this can become prohibitively expensive, and thus it is not really a practical solution.

In Section 8.2.1, we discuss an interesting unconditionally secure key predistribution scheme, due to Blom, that allows a reduction in the amount of secret information to be stored by the users in the network. We also present in Section 8.2.2 a computationally secure key predistribution scheme based on the discrete logarithm problem.

A more practical approach can be described as on-line key distribution by TA. In such a scheme, the TA acts as a key server. The TA shares a secret key KU with every user U in the network. When U wishes to communicate with V, she requests a session key from the TA. The TA generates a session key K and sends it in encrypted form for U and V to decrypt. The well-known Kerberos system, which we describe in Section 8.3, is based on this approach.

If it is impractical or undesirable to have an on-line TA, then a common approach is to use a key agreement protocol. In a key agreement protocol, U and V jointly choose a key by communicating over a public channel. This remarkable idea is due to Diffie and Hellman, and (independently) to Merkle. We describe a few of the more popular key agreement protocols. A variation of the original protocol of Diffie and Hellman, modified to protect against an active adversary, is presented in Section 8.4.1. Two other interesting protocols are also discussed: the MTI scheme is presented in Section 8.4.2 and the Girault scheme is covered in Section 8.4.3.

8.2 Key Predistribution

In the basic method, the TA generates [image] keys, and gives each key to a unique pair of users in a network of n users. As mentioned above, we require a secure channel between the TA and each user to transmit these keys. This is a significant improvement over each pair of users independently exchanging keys over a secure channel, since the number of secure channels required has been reduced from [image] to n. But if n is large, this solution is not very practical, both in terms of the amount of information to be transmitted securely, and in the amount of information that each user must store securely (namely, the secret keys of the other other n - 1 users).

Thus, it is of interest to try to reduce the amount of information that needs to be transmitted and stored, while still allowing each pair of users U and V to be able to (independently) compute a secret key KU,V. An elegant scheme to accomplish this, called the Blom Key Predistribution Scheme, is discussed in the next subsection.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

8.2.1 Blom's Scheme

As above, we suppose that we have a network of n users. For convenience, we suppose that keys are chosen from a finite field [image], where p ≥ n is prime. Let k be an integer, 1 ≥ k ≥ n - 2. The value k is the largest size coalition against which the scheme will remain secure. In the Blom Scheme, the TA will transmit k + 1 elements of [image] to each user over a secure channel (as opposed to n - 1 in the basic key predistribution scheme). Each pair of users, U and V, will be able to compute a key KU,V = KV,U, as before. The security condition is as follows: any set of at most k users disjoint from {U, V} must be unable to determine any information about KU,V (note that we are speaking here about unconditional security).

We first present the special case of Blom's scheme where k = 1. Here, the TA will transmit two elements of [image] to each user over a secure channel, and any individual user W will be unable to determine any information about KU,V if W ≠ U, V. Blom's scheme is presented in Figure 8.1. We illustrate the Blom Scheme with k = 1 in the following example.

Example 8.1

Suppose the three users are U, V and W, p = 17, and their public elements are rU = 12, rV = 7 and rW = 1. Suppose that the TA chooses a = 8, b = 7 and c = 2, so the polynomial f is

The g polynomials are as follows:

*
* Figure 8.1 Blom Key Distribution Scheme (k = 1)

The three keys are thus

U would compute KU,V as

V would compute KU,V as

We leave the computation of the other keys as an exercise for the reader.

We now prove that no one user can determine any information about the key of two other users.

THEOREM 8.1

The Blom Scheme with k = 1 is unconditionally secure against any individual user.

PROOF Let's suppose that user W wants to try to compute the key

The values rU and rV are public, but a, b and c are unknown. W does know the values

and

since these are the coefficients of the polynomial gW (x) that was sent to W by the TA.

What we will do is show that the information known by W is consistent with any possible value [image] of the key KU,V. Hence, W cannot rule out any values for KU,V. Consider the following matrix equation (in [image]):

The first equation represents the hypothesis that [image]; the second and third equations contain the information that W knows about a, b and c from gW (x).

The determinant of the coefficient matrix is

where all arithmetic is done in [image]. Since rW ≠ rU and rW ≠ rV, it follows that the coefficient matrix has non-zero determinant, and hence the matrix equation has a unique solution for a, b, c. In other words, any possible value [image] of KU,V is consistent with the information known to W.

On the other hand, a coalition of two users, say {W, X}, will be able to determine any key KU,V where [image] W and X together know that

Thus they have four equations in three unknowns, and they can easily compute a unique solution for a, b and c. Once they know a, b and c, they can form the polynomial f(x, y) and compute any key they wish.

It is straightforward to generalize the scheme to remain secure against coalitions of size k. The only thing that changes is step 2. The TA will use a polynomial f(x, y) having the form

where [image], and ai.j = aj,i for all i, j. The remainder of the protocol is unchanged.

8.2.2 Diffie-Hellman Key Predistribution

In this section, we describe a key predistribution scheme that is a modification of the well-known Diffie-Hellman key exchange protocol that we will discuss a bit later, in Section 8.4. We call this the Diffie-Hellman Key Predistribution Scheme. The scheme is computationally secure provided a problem related to the Discrete Logarithm problem is intractible.

We will describe the scheme over [image], where p is prime, though it can be implemented in any finite group in which the Discrete Logarithm problem is intractible. We will assume that a is a primitive element of [image], and that the values p and α are publicly known to everyone in the network.

In this scheme, ID(U) will denote certain identification information for each user U in the network, e.g., his or her name, e-mail address, telephone number, or other relevant information. Also, each user U has a secret exponent aU (where 0 ≤ aU ≤ p - 2), and a corresponding public value

The TA will have a signature scheme with a (public) verification algorithm verTA and a secret signing algorithm sigTA. Finally, we will implicitly assume that all information is hashed, using a public hash function, before it is signed. To make the procedures easier to read, we will not include the necessary hashing in the description of the protocols.

Certain information pertaining to a user U will be authenticated by means of a certificate which is issued and signed by the TA. Each user U will have a certificate

where bU is formed as described above (note that the TA does not need to know the value of aU). A certificate for a user U will be issued when U joins the network. Certificates can be stored in a public database, or each user can store his or her own certificate. The signature of the TA on a certificate allows anyone in the network to verify the information it contains.

It is very easy for U and V to compute the common key

as shown in Figure 8.2.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

We illustrate the algorithm with a small example.

Example 8.2

Suppose p = 25307 and α = 2 are publicly known (p is prime and α is a primitive root modulo p). Suppose U chooses aU = 3578. Then she computes

which is placed on her certificate. Suppose V chooses aV = 19956. Then he computes

*
* Figure 8.2 Diffie-Hellman Key Predistribution

which is placed on his certificate.

Now U can compute the key

and V can compute the same key

Let us think about the security of this scheme in the presence of a passive or active adversary. The signature of the TA on users' certificates effectively prevents W from altering any information on someone else's certificate. Hence we need only worry about passive attacks. So the pertinent question is: Can a user W compute KU,V if W ≠ U, V? In other words, given αau mod p and αav mod p (but not aU nor aV), is it feasible to compute αaUaV mod p? This problem is called the Diffie-Hellman problem, and it is formally defined (using an equivalent but slightly different presentation) in Figure 8.3. It is clear that Diffie-Hellman Key Predistribution is secure against a passive adversary if and only if the Diffle-Hellman problem is intractible.

*
* Figure 8.3 The Diffie-Hellman problem

If W could determine aU from bU, or if he could determine aV from bV, then he could compute KU,V exactly as U (or V) does. But both these computations are instances of the Discrete Log problem. So, provided that the Discrete Log problem in [image] is intractible, Diffie-Hellman Key Predistribution is secure against this particular type of attack. However, it is an unproven conjecture that any algorithm that solves the Diffie-Hellman problem could also be used to solve the Discrete Log problem. (This is very similar to the situation with RSA, where it is conjectured, but not proved, that breaking RSA is polynomially equivalent to factoring.)

By the remarks made above, the Diffie-Hellman problem is no more difficult than the Discrete Log problem. Although we cannot say precisely how difficult this problem is, we can relate its security to that of another cryptosystem we have already studied, namely the ElGamal Cryptosystem.

THEOREM 8.2

Breaking the ElGamal Cryptosystem is equivalent to solving the Diffie-Hellman problem.

PROOF First we recall how ElGamal encryption and decryption work. The key is K = (p, α, a, β), where β = αa mod p (a is secret and p, α, and β are public). For a (secret) random number [image],

where

and

For [image],

Suppose we have an algorithm A to solve the Diffie-Hellman problem, and we are given an ElGamal encryption (y1, y2). We will apply the algorithm A with inputs p, α, y1, and β. Then, we obtain the value

Then, the decryption of (y1, y2) can easily be computed as

Conversely, suppose we have an algorithm B that performs ElGamal decryption. That is, B takes as inputs p, α, β, y1, and y2, and computes the quantity

Now, given inputs p, α, β, and γ for the Diffie-Hellman problem, it is easy to see that

as desired.

8.3 Kerberos

In the key predistribution methods we discussed in the previous section, each pair of users can compute one fixed key. If the same key is used for a long period of time, there is a danger that it might be compromised. Thus it is often preferable to use an on-line method in which a new session key is produced every time a pair of users want to communicate (this property is called key freshness).

If on-line key distribution is used, there is no need for any network user to store keys to communicate with other users (each user will share a key with the TA, however). Session keys will be transmitted on request by the TA. It is the responsibility of the TA to ensure key freshness.

Kerberos is a popular key serving system based on private-key cryptography. In this section, we give an overview of the protocol for issuing session keys in Kerberos. Each user U shares a secret DES key KU with the TA. In the most recent version of Kerberos (version V), all messages to be transmitted are encrypted using cipher block chaining (CBC) mode, as described in Section 3.4.1.

*
* Figure 8.4 Transmission of a session key using Kerberos

As in Section 8.2.2, ID(U) will denote public identification information for user U. When a request for a session key is sent to the TA, the TA will generate a new random session key K. Also, the TA will record the time at which the request is made as a timestamp, T, and specify the lifetime, L, during which K will be valid. That is, the session key K is to be regarded as a valid key from time T to time T + L. All this information is encrypted and transmitted to U and (eventually) to V. Before going into more details, we will present the protocol in Figure 8.4.

The information transmitted in the protocol is illustrated in the following diagram:

We will now explain what is going on in the various steps of the protocol. Although we have no formal proof that Kerberos is "secure" against an active adversary, we can at least give some informal motivation of the features of the protocol.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

As mentioned above, the TA generates K, T, and L in step 2. In step 3, this information, along with ID(V), is encrypted using the key KU shared by U and the TA to form m1. Also, K, T, L, and ID(U) are encrypted using the key Kv shared by V and the TA to form m2. Both these encrypted messages are sent to U.

U can use her key to decrypt m1, and thus obtain K, T, and L. She will verify that the current time is in the interval from T to T + L. She can also check that the session key K has been issued for her desired communicant V by verifying the information ID(V) decrypted from m1.

Next, U will relay m2 to V. As well, U will use the new session key K to encrypt T and ID (U) and send the resulting message m3 to V.

When V receives m2 and m3 from U, he decrypts m2 to obtain T, K, L and ID(U). Then he uses the new session key K to decrypt m3 and he verifies that T and ID(U), as decrypted from m2 and m3, are the same. This ensures V that the session key encrypted within m2 is the same key that was used to encrypt m3. Then V uses K to encrypt T + 1, and sends the result back to U as message m4.

When U receives m4, she decrypts it using K and verifies that the result is T + 1. This ensures U that the session key K has been successfully transmitted to V, since K was needed in order to produce the message m4.

It is important to note the different functions of the messages transmitted in this protocol. The messages m1 and m2 are used to provide secrecy in the transmission of the session key K. On the other hand, m3 and m4 are used to provide key confirmation, that is, to enable U and V to convince each other that they possess the same session key K. In most key distribution schemes, (session) key confirmation can be included as a feature if it is not already present. Usually this is done in a similar fashion as it is done in Kerberos, namely by using the new session key K to encrypt known quantities. In Kerberos, U uses K to encrypt ID (U) and T, which are already encrypted in m2. Similarly, V uses K to encrypt T + 1.

The purpose of the timestamp T and lifetime L is to prevent an active adversary from storing "old" messages for retransmission at a later time (this is called a replay attack). This method works because keys are not accepted as valid once they have expired.

*
* Figure 8.5 Diffie-Hellman Key Exchange

One of the drawbacks of Kerberos is that all the users in the network should have synchronized clocks, since the current time is used to determine if a given session key K is valid. In practice, it is very difficult to provide perfect synchronization, so some amount of variation in times must be allowed.

8.4 Diffie-Hellman Key Exchange

If we do not want to use an on-line key server, then we are forced to use a key agreement protocol to exchange secret keys. The first and best known key agreement protocol is Diffie-Hellman Key Exchange. We will assume that p is prime, α is a primitive element of [image], and that the values p and α are publicly known. (Alternatively, they could be chosen by U and communicated to V in the first step of the protocol.) Diffie-Hellman Key Exchange is presented in Figure 8.5.

At the end of the protocol, U and V have computed the same key

This protocol is very similar to Diffie-Hellman Key Predistribution described earlier. The difference is that the exponents aU and aV of users U and V (respectively) are chosen anew each time the protocol is run, instead of being fixed. Also, in this protocol, both U and V are assured of key freshness, since the session key depends on both random exponents aU and aV.

8.4.1 The Station-to-station Protocol

Diffie-Hellman Key Exchange is supposed to look like this:

Unfortunately, the protocol is vulnerable to an active adversary who uses an intruder-in-the-middle attack. There is an episode of The Lucy Show in which Vivian Vance is having dinner in a restaurant with a date, and Lucille Ball is hiding under the table. Vivian and her date decide to hold hands under the table. Lucy, trying to avoid detection, holds hands with each of them and they think they are holding hands with each other.

An intruder-in-the-middle attack on the Diffie-Hellman Key Exchange protocol works in the same way. W will intercept messages between U and V and substitute his own messages, as indicated in the following diagram:

At the end of the protocol, U has actually established the secret key [image] with W, and V has established a secret key [image] with W. When U tries to encrypt a message to send to V, W will be able to decrypt it but V will not. (A similar situation holds if V sends a message to U.)

Clearly, it is essential for U and V to make sure that they are exchanging messages with each other and not with W. Before exchanging keys, U and V might carry out a separate protocol to establish each other's identity, for example by using one of the identification schemes that we will describe in Chapter 9. But this offers no protection against an intruder-in-the-middle attack if W simply remains inactive until after U and V have proved their identities to each other. Hence, the key agreement protocol should itself authenticate the participants' identities at the same time as the key is being established. Such a protocol will be called authenticated key agreement.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

We will describe an authenticated key agreement protocol which is a modification of Diffie-Hellman Key Exchange. The protocol assumes a publicly known prime p and a primitive element α, and it makes use of certificates. Each user U will have a signature scheme with verification algorithm verU and signing algorithm sigU. The TA also has a signature scheme with public verification algorithm verTA. Each user U has a certificate

where ID(U) is identification information for U.

*
* Figure 8.6 Simplified Station-to-station Protocol

The authenticated key agreement known as the Station-to-station Protocol (or STS for short) is due to Diffie, Van Oorschot, and Wiener. The protocol we present in Figure 8.6 is a slight simplification; it can be used in such a way that it is conformant with the ISO 9798-3 protocols.

The information exchanged in the simplified STS protocol (excluding certificates) is illustrated as follows:

Let's see how this protects against an intruder-in -the-middle attack. As before, W will intercept [image] and replace it with [image]. W then receives [image] from V. He would like to replace [image] with [image] as before. However, this means that he must also replace [image] by [image]. Unfortunately for W, he cannot compute V's signature on [image] since he doesn't know V's signing algorithm sigV. Similarly, W is unable to replace [image] by [image] because he does not know U's signing algorithm.

This is illustrated in the following diagram:

It is the use of signatures that thwarts the intruder-in-the-middle attack.

The protocol, as described in Figure 8.6, does not provide key confirmation. However, it is easy to modify so that it does, by defining

in step 4 and defining

in step 6. (As in Kerberos, we obtain key confirmation by encrypting a known quantity using the new session key.) The resulting protocol is known as the Station-to-station Protocol. We leave the remaining details for the interested reader to fill in.

8.4.2 MTI Key Agreement Protocols

Matsumoto, Takashima, and Imai have constructed several interesting key agreement protocols by modifying Diffie-Hellman Key Exchange. These protocols, which we call MTI protocols, do not require that U and V compute any signatures. They are two-pass protocols since there are only two separate transmissions of information performed (one from U to V and one from V to U). In contrast, the STS protocol is a three-pass protocol.

We present one of the MTI protocols. The setting for this protocol is the same as for Diffie-Hellman Key Predistribution. We assume a publicly known prime p and a primitive element α. Each user U has an ID string, ID(U), a secret exponent aU (0 ≤ aU ≤ p - 2), and a corresponding public value

The TA has a signature scheme with a (public) verification algorithm verTA and a secret signing algorithm sigTA.

*
* Figure 8.7 Matsumoto-Takashima-Imai Key Agreement Protocol

Each user U will have a certificate

where bU is formed as described above.

We present the MTI key agreement protocol in Figure 8.7. At the end of the protocol, U and V have both computed the same key

We give an example to illustrate this protocol.

Example 8.3

Suppose p = 27803 and α = 5 are publicly known. Assume U chooses aU = 21131; then she will compute

which is placed on her certificate. As well, assume V chooses aV = 17555. Then he will compute

which is placed on his certificate.

Now suppose that U chooses rU = 169; then she will send the value

to V. Suppose that V chooses rV = 23456; then he will send the value

to U.

Now U can compute the key

and V can compute the key

Thus U and V have computed the same key.

The information transmitted during the protocol is depicted as follows:

Let's look at the security of the scheme. It is not too difficult to show that the security of the MTI protocol against a passive adversary is exactly the same as the Diffie-Hellman problem — see the exercises. As with many protocols, proving security in the presence of an active adversary is problematic. We will not attempt to prove anything in this regard, and we limit ourselves to some informal arguments.

Here is one threat we might consider: Without the use of signatures during the protocol, it might appear that there is no protection against an intruder-in-the-middle attack. Indeed, it is possible that W might alter the values that U and V send each other. We depict one typical scenario that might arise, as follows:

In this situation, U and V will compute different keys: U will compute

while V will compute

However, neither of the key computations of U or V can be carried out by W, since they require knowledge of the secret exponents aU and aV, respectively. So even though U and V have computed different keys (which will of course be useless to them), neither of these keys can be computed by W (assuming the intractibility of the Discrete Log problem). In other words, both U and V are assured that the other is the only user in the network that could compute the key that they have computed. This property is sometimes called implicit key authentication.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

8.4.3 Key Agreement Using Self-certifying Keys

In this section, we describe a method of key agreement, due to Girault, that does not require certificates. The value of a public key and the identity of its owner implicitly authenticate each other.

The Girault Scheme combines features of RSA and discrete logarithms. Suppose n = pq, where p = 2p1 + 1, q = 2q1 + 1, and p, q, p1, and q1 are all large primes. The multiplicative group [image] isomorphic to [image]. The maximum order of any element in [image] is therefore the least common multiple of p - 1 and q - 1, or 2p1q1. Let α be an element of order 2p1q1. Then the cyclic subgroup of [image] generated by α is a suitable setting for the Discrete Logarithm problem.

In the Girault Scheme, the factorization of n is known only to the TA. The values n and α are public, but p, q, p1, and q1 are all secret. The TA chooses a public RSA encryption exponent, which we will denote by e. The corresponding decryption exponent, d, is secret (recall that d = e–1 mod ø (n)).

Each user U has an ID string ID(U), as in previous schemes. A user U obtains a self-certifying public key, pU, from the TA as indicated in Figure 8.8. Observe that U needs the help of the TA to produce pU. Note also that

can be computed from pU and ID(U) using publicly available information.

The Girault Key Agreement Protocol is presented in Figure 8.9. The information transmitted during the protocol is depicted as follows:

*
* Figure 8.8 Obtaining a self-certifying public key from the TA

At the end of the protocol, U and V each have computed the key

Here is an example of key exchange using the Girault Scheme.

Example 8.4

Suppose p = 839 and q = 863. Then n = 724057 and ø(n) = 722356. The element α = 5 has order 2p1q1 = ø(n)/2. Suppose the TA chooses d = 125777 as the RSA decryption exponent; then e = 84453.

Suppose U has ID(U) = 500021 and aU = 111899. Then bU = 488889 and pU = 650704. Suppose also that V has ID(V) = 500022 and aV = 123456. Then bV = 111692 and pV = 683556.

Now, U and V want to exchange a key. Suppose U chooses rU = 56381, which means that sU = 171007. Further, suppose V chooses rV = 356935, which means that sV = 320688.

Then both U and V will compute the same key K = 42869.

Let's consider how the self-certifying keys guard against one specific type of attack. Since the values bU, pU, and ID(U) are not signed by the TA, there is no way for anyone else to verify their authenticity directly. Suppose this information is forged by W (i.e., it is not produced in cooperation with the TA), who wants to masquerade as U. If W starts with ID(U) and a fake value [image], then there is no way for her to compute the exponent [image] corresponding [image] if the Discrete Log problem is intractible. Without [image] computation cannot be performed by W (who is pretending to be U).

*
* Figure 8.9 Girault Key Agreement Protocol

The situation is similar if W acts as an intruder-in-the-middle. W will be able to prevent U and V from computing a common key, but W is unable to duplicate the computations of either U or V. Thus the scheme provides implicit key authentication, as did the MTI protocol.

An attentive reader might wonder why U is required to supply the value aU to the TA. Indeed, the TA can compute pU directly from bU, without knowing aU. Actually, the important thing here is that the TA should be convinced that U knows the value of aU before the TA computes pU for U.

We illustrate this point by showing how the scheme can be attacked if the TA indiscriminately issues public keys pU to users without first checking that they possess the value aU corresponding to their bU. Suppose W chooses a fake value [image] and computes the corresponding value

Here is how he can determine the corresponding public key

W will compute

and then given [image] and ID(W) to the TA. Suppose the TA issues the public key

to W. Using the fact that

it is immediate that

Now, at some later time, suppose U and V execute the protocol, and W substitutes information as follows:

Now V will compute the key

whereas U will compute the key

W can compute K′ as

Thus W and V share a key, but V thinks he is sharing a key with U. So W will be able to decrypt messages sent by V to U.

8.5 Notes and References

Blom presented his key predistribution scheme in [BL85]. Generalizations can be found in Blundo et al. [BDSHKVY93] and Beimel and Chor [BC94].

Diffie and Hellman presented their key exchange algorithm in [DH76]. The idea of key exchange was discovered independently by Merkle [ME78]. The material on authenticated key exchange is taken from Diffie, van Oorschot, and Wiener [DVW92].

Version V of Kerberos is described in [KN93]. For a recent descriptive article on Kerberos, see Schiller [SC94].

The protocols of Matsumoto, Takashima, and Imai can be found in [MTI86]. Self-certifying key distribution was introduced by Girault [GIR91]. The scheme he presented was actually a key predistribution scheme; the modification to a key agreement scheme is based on [RV94].


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Two recent surveys on key distribution and key agreement are Rueppel and Van Oorschot [RV94] and van Tilburg [VT93].

Exercises

8.1 Suppose the Blom Scheme with k = 1 is implemented for a set of four users, U, V, W and X. Suppose that p = 7873, rU = 2365, rV 6648, rW = 1837 and rX = 2186. The secret g polynomials are as follows:

(a) Compute the key for each pair of users, verifying that each pair of users obtains a common key (that is, KU,V = KV,U, etc.).
(b) Show how W and X together can compute KU,V.

8.2 Suppose the Blom Scheme with k = 2 is implemented for a set of five users, U, V, W, X and Y. Suppose that p = 97, rU = 14, rV = 38, rW = 92, rX = 69 and rY = 70. The secret g polynomials are as follows:

(a) Show how U and V each will compute the key KU,V = KV,U.
(b) Show how W, X and Y together can compute KU,V.

8.3 Suppose that U and V carry out the Diffie-Hellman Key Exchange with p = 27001 and α = 101. Suppose that U chooses aU = 21768 and V chooses aV 9898. Show the computations performed by both U and V, and determine the key that they will compute.
8.4 Suppose that U and V carry out the MTI Protocol where p = 30113 and α = 52. Suppose that U has aU = 8642 and chooses rU = 28654, and V has aV = 24673 and chooses rV = 12385. Show the computations performed by both U and V, and determine the key that they will compute.
8.5 If a passive adversary tries to compute the key K constructed by U and V by using the MTI protocol, then he is faced with an instance of what we might term the MTI problem, which we present in Figure 8.10. Prove that any algorithm that can be used to solve the MTI problem can be used to solve the Diffie-Hellman problem, and vice versa.
8.6 Consider the Girault Scheme where p = 167, q = 179, and hence n = 29893. Suppose α = 2 and e = 11101.

(a) Compute d.
(b) Given that ID(U) = 10021 and aU = 9843, compute bU and pU. Given that ID(V) = 10022 and aV = 7692, compute bV and pV.
(c) Show how bU can be computed from pU and ID(U) using the public exponent e. Similarly, show how bV can be computed from pV and ID(V).

*
* Figure 8.10 The MTI problem

(d) Suppose that U chooses rU = 15556 and V chooses rV = 6420. Compute sU and sV, and show how U and V each compute their common key.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 9
Identification Schemes

9.1 Introduction

Cryptographic methods enable many seemingly impossible problems to be solved. One such problem is the construction of secure identification schemes. There are many common, everyday situations where it is necessary to electronically "prove" one's identity. Some typical scenarios are as follows:

1. To withdraw money from an automated teller machine (or ATM), we use a card together with a four-digit personal identification number (PIN).
2. To charge purchases over the telephone to a credit card, all that is necessary is a credit card number (and the expiry date).
3. To charge long-distance telephone calls (using a calling card), one requires only a telephone number together with a four-digit PIN.
4. To do a remote login to a computer over a network, it suffices to know a valid user name and the corresponding password.

In practice, these types of schemes are not usually implemented in a secure way. In the protocols performed over the telephone, any eavesdropper can use the identifying information for their own purposes. This could include the person who is the recipient of the information; many credit card "scams" operate in this way. An ATM card is somewhat more secure, but there are still weaknesses. For example, someone monitoring the communication line can obtain all the information encoded on the card's magnetic strip, as well as the PIN. This could allow an imposter to gain access to a bank account. Finally, remote computer login is a serious problem due to the fact that user IDs and passwords are transmitted over the network in unencrypted form. Thus they are vulnerable to anyone who is monitoring the computer network.

The goal of an identification scheme is that someone "listening in" as Alice identifies herself to Bob, say, should not subsequently be able to misrepresent herself as Alice. Furthermore, we should try to guard against the possibility that Bob himself might try to impersonate Alice after she has identified herself to him. In other words, Alice wants to be able to prove her identity electronically without "giving away" her identifying information.

*
* Figure 9.1 Challenge-and-response protocol

Several such identification schemes have been discovered. One practical objective is to find a scheme that is simple enough that it can be implemented on a smart card, which is essentially a credit card equipped with a chip that can perform arithmetic computations. Hence, both the amount of computation and the memory requirements should be kept as small as possible. Such a card would be a more secure alternative to current ATM cards. However, it is important to note that the "extra" security pertains to someone monitoring the communication line. Since it is the card that is "proving" its identity, we have no extra protection against a lost card. It would still be necessary to include a PIN in order to establish that it is the real owner of the card who is initiating the identification protocol.

In later sections, we will describe some of the more popular identification schemes. But first, we give a very simple scheme that can be based on any private-key cryptosystem, e.g., DES. The protocol, which is described in Figure 9.1, is called a challenge-and-response protocol. In it, we assume that Alice is identifying herself to Bob, and Alice and Bob share a common secret key, K, which specifies an encryption function eK.

We illustrate this protocol with a small example.

Example 9.1

Alice and Bob use an encryption function which does a modular exponentiation:

Suppose Bob's challenge is x = 77835. Then Alice responds with y = 100369.

Virtually all identification schemes are challenge-and-response protocols, but the most useful schemes do not require shared keys. This idea will be pursued in the remainder of the chapter.

9.2 The Schnorr Identification Scheme

We begin by describing the Schnorr Identification Scheme, which is one of the most attractive practical identification schemes. The scheme requires a trusted authority, which we denote by TA. The TA will choose parameters for the scheme as follows:

1. p is a large prime (i.e., p ≥ 2 512) such that the discrete log problem in [image] is intractible.
2. q is a large prime divisor of p - 1 (i.e., q ≥ 2 140).
3. has order q (such an α can be computed as the (p - 1)/qth power of a primitive element).
4. A security parameter t such that q > 2t. For most practical applications, t = 40 will provide adequate security.
5. The TA also establishes a secure signature scheme with a secret signing algorithm sigTA and a public verification algorithm verTA.
6. A secure hash function is specified. As usual, all information is to be hashed before it is signed. In order to make the protocols easier to read, we will omit the hashing steps from the descriptions of the protocols.

The parameters p, q, and α, the public verification algorithm verTA and the hash function are all made public.

A certificate will be issued to Alice by the TA. When Alice wants to obtain a certificate from the TA, the steps in Figure 9.2 are carried out. At a later time, when Alice wants to prove her identity to Bob, say, the protocol of Figure 9.3 is executed.

As mentioned above, t is a security parameter. Its purpose is to prevent an impostor posing as Alice, say Olga, from guessing Bob's challenge, r. For, if Olga guessed the correct value of r, she could choose any value for y and compute

She would give Bob γ in step 1, and then when she receives the challenge r, she would supply the value y she has already chosen. Then γ would be verified by Bob in step 6.

*
* Figure 9.2 Issuing a certificate to Alice

The probability that Olga will guess the value of r correctly is 2-t if r is chosen at random by Bob. Thus, t = 40 should be a reasonable value for most applications. (But notice that Bob should choose his challenge r at random every time Alice identifies herself to him. If Bob always used the same challenge r, then Olga could impersonate Alice by the method described above.)


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Basically, there are two things happening in the verification protocol. First, the signature s proves the validity of Alice's certificate. Thus Bob verifies the signature of the TA on Alice's certificate to convince himself that the certificate itself is authentic. This is essentially the same way that certificates were used in Chapter 8.

The second part of the protocol concerns the secret number a. The value a functions like a PIN in that it convinces Bob that the person carrying out the identification protocol is indeed Alice. But there is an important difference from a PIN: in the identification protocol, the value of a is not revealed. Instead Alice (or more accurately, Alice's smart card) "proves" that she/it knows the value of a in step 5 of the protocol by computing the value y in response to the challenge r issued by Bob. Since the value of a is not revealed, this technique is called a proof of knowledge.

*
* Figure 9.3 The Schnorr identification scheme

The following congruences demonstrate that Alice will be able to prove her identity to Bob:

Thus Bob will accept Alice's proof of identity (assuming he is honest), and the protocol is said to have the completeness property.

Here is a small (toy) example illustrating the challenge-and-response aspect of the protocol.

Example 9.2

Suppose p = 88667, q = 1031 and t = 10. The element α = 70322 has order q in [image]. Suppose Alice's secret exponent is a = 755; then

Now suppose Alice chooses k = 543. Then she computes

and sends γ to Bob. Suppose Bob issues the challenge r = 1000. Then Alice computes

and sends y to Bob. Bob then verifies that

So Bob believes that he is communicating with Alice.

Next, let's consider how someone might try to impersonate Alice. An imposter, Olga, might try to impersonate Alice by forging a certificate

where v′ ≠ v. But s′ is supposed to be a signature of (ID (Alice), v′), and this is verified by Bob in step 3 of the protocol. If the signature scheme of the TA is secure, Olga will not be able to forge a signature s′ which will subsequently be verified by Bob.

Another approach would be for Olga to use Alice's correct certificate, which is C(Alice) = (ID(Alice), v, s) (recall that certificates are not secret, and the information on a certificate is revealed each time the identification protocol is executed). But Olga will not be able to impersonate Alice unless she also knows the value of a. This is because of the "challenge" r in step 4. In step 5, Olga would have to compute y, but y is a function of a. The computation of a from v involves solving a discrete log problem, which we assume is intractible.

We can prove a more precise statement about the security of the protocol, as follows.

THEOREM 9.1

Suppose Olga knows a value γ for which she has probability ∈ ≥ 1/2t-1 of successfully impersonating Alice in the verification protocol. Then Olga can compute a in polynomial time.

PROOF For a fraction ∈ of the 2t possible challenges r, Olga can compute a value y which will be accepted in step 6 by Bob. Since ∈ ≥ 1/2 t-1, we have that 2t ∈, and therefore Olga can compute values y1, y2, r1 and r2 such that

and

It follows that

Since v = α-a, we have that

Now, 0 < |r2 - r1| < 2t and q > 2t is prime. Hence gcd(r2 - r1, q) = 1, and Olga can compute

as desired.

The above theorem proves that anyone who has a non-negligible chance of successfully executing the identification protocol must know (or be able to compute in polynomial time) Alice's secret exponent a. This property is often referred to as soundness.

We illustrate with an example.

Example 9.3

Suppose we have the same parameters as in Example 9.2: p = 88667, q = 1031, t = 10, α = 70322, α = 755 and v = 13136. Suppose Olga learns that

Then she can compute

and thus discover Alice's secret exponent.

We have proved that the protocol is sound and complete. But soundness and completeness are not sufficient to ensure that the protocol is "secure." For example, if Alice simply revealed the value of her exponent a to prove her identity to Olga (say), the protocol would still be sound and complete. However, it would be completely insecure, since Olga could subsequently impersonate Alice.

This motivates the consideration of the secret information released to a verifier (or an observer) who takes part in the protocol (in this protocol, the secret information is the value of the exponent a). Our hope is that no information about a can be gained by Olga when Alice proves her identity, for then Olga would be able to masquerade as Alice.

In general, we could envision a situation whereby Alice proves her identity to Olga, say, on several different occasions. Perhaps Olga does not choose her challenges (i.e., the values of r) in a random way. After several executions of the protocol, Olga will try to determine the value of a so she can subsequently impersonate Alice. If Olga can determine no information about the value of a by taking part in a polynomial number of executions of the protocol and then performing a polynomial amount of computation, then we would be convinced that the protocol is secure.

It has not been proven that the Schnorr Scheme is secure. But in the next section, we present a modification of the Schnorr Scheme, due to Okamoto, that can be proved to be secure given a certain computational assumption.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

The Schnorr Scheme was designed to be very fast and efficient, both from a computational point of view and in the amount of information that needs to be exchanged in the protocol. It is also designed to minimize the amount of computation performed by Alice. This is desirable because in many practical applications, Alice's computations will be performed by a smart card with low computing power, while Bob's computations will be performed by a more powerful computer.

For the purpose of discussion, let's assume that ID (Alice) is a 512-bit string. v also comprises 512 bits, and s will be 320 bits if the DSS is used as a signature scheme. The total size of the certificate C(Alice) (which needs to be stored on Alice's smart card) is then 1344 bits.

Let us consider Alice's computations: step 1 requires a modular exponentiation to be performed; step 5 comprises one modular addition and one modular multiplication. It is the modular exponentiation that is computationally intensive, but this can be precomputed offline, if desired. The online computations to be performed by Alice are very modest.

It is also a simple matter to calculate the number of bits that are communicated during the protocol. We can depict the information that is communicated in the form of a diagram:

Alice gives Bob 1344 + 512 = 1856 bits of information in step 2; Bob gives Alice 40 bits in step 4; and Alice gives Bob 140 bits in step 6. So the communication requirements are quite modest, as well.

*
* Figure 9.4 Issuing a certificate to Alice

9.3 The Okamoto Identification Scheme

In this section, we present a modification of the Schnorr Scheme due to Okamoto. This modification can be proved secure, assuming the intractibility of computing a particular discrete logarithm in [image].

To set up the scheme, the TA chooses p and q as in the Schnorr Scheme. The TA also chooses two elements [image] both having order q. The value [image] is kept secret from all the participants, including Alice. We will assume that it is infeasible for anyone (even a coalition of Alice and Olga, say) to compute the value c. As before, the TA chooses a signature scheme and hash function. The certificate issued to Alice by the TA is constructed as described in Figure 9.4. The Okamoto Identification Scheme is presented in Figure 9.5.

Here is an example of the Okamoto Scheme.

Example 9.4

As in previous examples, we will take p = 88667, q = 1031, and t = 10. Suppose α1 = 58902 and α2 = 73611 (both α1 and α2 have order q in [image]). Now, suppose α1 = 846 and α2 = 515; then v = 13078.

*
* Figure 9.5 The Okamoto identification scheme

Suppose Alice chooses k1 = 899 and k2 = 16; then γ = 14574. If Bob issues the challenge r = 489 then Alice will respond with y1 = 131 and y2 = 287. Bob will verify that

So Bob will accept Alice's proof of identity.

The proof that the protocol is complete (i.e., that Bob will accept Alice's proof of identity) is straightforward. The main difference between Okamoto's and Schnorr's scheme is that we can prove that the Okamoto Scheme is secure provided that the computation of the discrete logarithm [image] is intractible.

The proof of security is quite subtle. Here is the general idea: As before, Alice identifies herself to Olga polynomially many times by executing the protocol. We then suppose (hoping to obtain a contradiction) that Olga is able to learn some information about the values of Alice's secret exponents α1 and α2. If this is so, then we will show that (with high probability) Alice and Olga together will be able to compute the discrete logarithm c in polynomial time. This contradicts the assumption made above, and proves that Olga must be unable to obtain any information about Alice's exponents by taking part in the protocol.

The first part of this procedure is similar to the soundness proof for the Schnorr Scheme.

THEOREM 9.2

Suppose Olga knows a value γ for which she has probability ∈ ≥ 1/2t-1 of successfully impersonating Alice in the verification protocol. Then, in polynomial time, Olga can compute values b1 and b2 such that

PROOF For a fraction ∈ of the 2t possible challenges r, Olga can compute values y1, y2, z1, z2, r and s with r ≠ s and

Define

b1 = (y1 - z1)(r - s)-1 mod q

and

b2 = (y2 - z2)(r - s)-1 mod q.

Then it is easy to check that

as desired.

We now proceed to show how Alice and Olga can together compute the value of c.

THEOREM 9.3

Suppose Olga knows a value γ for which she has probability ∈ ≥ 1/2t-1 of successfully impersonating Alice in the verification protocol. Then, with probability 1 - 1/q, Alice and Olga can together compute [image] in polynomial time.

PROOF By Theorem 9.2, Olga is able to determine values b1 and b2 such that

Now suppose that Alice reveals the values α1 and α2 to Olga. Of course

so it must be the case that

Suppose that (a1, a2) ≠ (b1, b2). Then (a2 - b2)-1 mod q exists, and the discrete log

can be computed in polynomial time.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

There remains to be considered the possibility that (a1, a2) = (b1, b2). If this happens, then the value of c cannot be computed as described above. However, we will argue that (a1, a2) = (b1, b2) will happen only with very small probability 1/q, so the procedure whereby Alice and Olga compute c will almost surely succeed.

Define

That is, [image] consists of all the possible ordered pairs that could be Alice's secret exponents. Observe that

where [image]. Thus [image] consist of q ordered pairs.

The ordered pair (b1, b2) computed by Olga is certainly in the set [image]. We will argue that the value of the pair (b1, b2) is independent of the value of the pair (a1, a2) that comprises Alice's secret exponents. Since (a1, a2) was originally chosen at random by Alice, it must be the case that the probability that (a1, a2) = (b1, b2) is 1/q.

So, we need to say what we mean by (b1, b2) being "independent" of (a1, a2). The idea is that Alice's pair (a1, a2) is one of the q possible ordered pairs in the set [image], and no information about which is the "correct" ordered pair is revealed by Alice identifying herself to Olga. (Stated informally, Olga knows that an ordered pair from [image] comprises Alice's exponents, but she has no way of telling which one.)

Let's look at the information that is exchanged during the identification protocol. Basically, in each execution of the protocol, Alice chooses a γ; Olga chooses an r; and Alice reveals y1 and y2 such that

Recall that Alice computes

and

where

But note that k1 and k2 are not revealed (nor are a1 and a2).

The particular quadruple (γ, r, y1, y2) that is generated during one execution of the protocol appears to depend on Alice's ordered pair (a1, a2), since y1 and y2 are defined in terms of a1 and a2. But we will show that each such quadruple could equally well be generated from any other ordered pair [image]. To see this, suppose [image] and [image] where 0 ≤ θ ≤ q - 1. We can express y1 and y2 as follows:

and

where all arithmetic is performed in [image]. That is, the quadruple (γ,r,y1,y2) is also consistent with the ordered pair [image] using the random choices [image] and [image] to produce (the same) γ. We have already noted that the values of k1 and k2 are not revealed by Alice, so the quadruple (γ,r,y1,y2) yields no information regarding which ordered pair in [image] Alice is actually using for her secret exponents. This completes the proof.

This security proof is certainly quite elegant and subtle. It would perhaps be useful to recap the features of the protocol that lead to the proof of security. The basic idea involves having Alice choose two secret exponents rather than one. There are a total of q pairs in the set [image] that are "equivalent" to Alice's pair (a1, a2). The fact that leads to the ultimate contradiction is that knowledge of two different pairs in [image] provides an efficient method of computing the discrete logarithm c. Alice, of course, knows one pair in [image]; and we proved that if Olga can impersonate Alice, then Olga is able to compute a pair in [image] which (with high probability) is different from Alice's pair. Thus Alice and Olga together can find two pairs in [image] and compute c, which provides the desired contradiction.

Here is an example to illustrate the computation of [image] by Alice and Olga.

Example 9.5

As in Example 9.4, we will take p = 88667, q = 1031 and t = 10, and assume that v = 13078.

Suppose Olga has determined that

Then she can compute

b1 = (131 - 890)(489 - 199)-1 mod 1031 = 456

and

b2 = (287 - 303)(489 - 199)-1 mod 1031 = 519.

Now, using the values of a1 and a2 supplied by Alice, the value

c = (846 - 456)(519 - 515)-1 mod 1031 = 613

is computed. This value c is in fact [image] as can be verified by calculating

58902613 mod 88667 = 73611.

Finally, we should emphasize that, although there is no known proof that the Schnorr Scheme is secure (even assuming that the discrete logarithm problem is intractible), neither is there any known weakness in the scheme. Actually, the Schnorr Scheme might be preferred in practice to the Okamoto Scheme simply because it is somewhat faster.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

9.4 The Guillou-Quisquater Identification Scheme

In this section, we describe another identification scheme, due to Guillou and Quisquater, that is based on RSA.

The set-up of the scheme is as follows: The TA chooses two primes p and q and forms the product n = pq. The values of p and q are secret, while n is public. As is usually the case, p and q should be chosen large enough that factoring n is intractible. Also, the TA chooses a large prime integer b which will function as a security parameter as well as being a public RSA encryption exponent; to be specific, let us suppose that b is a 40-bit prime. Finally, the TA chooses a signature scheme and hash function.

The certificate issued to Alice by the TA is constructed as described in Figure 9.6. When Alice wants to prove her identity to Bob, say, the protocol of Figure 9.7 is executed. We will prove that the Guillou-Quisquater Scheme is sound and complete. However, the scheme has not been proved to be secure (even assuming that the RSA cryptosystem is secure).

*
* Figure 9.6 Issuing a certificate to Alice

*
* Figure 9.7 The Guillou-Quisquater identification scheme

Example 9.6

Suppose the TA chooses p = 467 and q = 479, so n = 223693. Suppose also that b = 503 and Alice's secret integer u = 101576. Then she will compute

Now, let's assume that Alice is proving her identity to Bob and she chooses k = 187485; then she gives Bob the value

Suppose Bob responds with the challenge r = 375. Then Alice will compute

and gives it to Bob. Bob then verifies that

Hence, Bob accepts Alice's proof of identity.

As is generally the case, proving completeness is quite simple:

Now, let us consider soundness. We will prove that the scheme is sound provided that it is infeasible to compute u from v. Since v is formed from u by RSA encryption, this is a plausible assumption to make.

THEOREM 9.4

Suppose Olga knows a value γ for which she has probability ∈ > 1/b of successfully impersonating Alice in the verification protocol. Then, in polynomial time, Olga can compute u.

PROOF For some γ, Olga can compute values y1, y2, r1, r2 with r1 ≠ r2, such that

Suppose, without loss of generality, that r1 > r2. Then we have

Since 0 < r1 - r2 < b and b is prime, t = (r1 - r2)-1 mod b exists, and it can be computed in polynomial time by Olga using the Euclidean algorithm. Hence, we have that

Now,

for some positive integer [image], so

or equivalently,

Now raise both sides of the congruence to the power b-1 mod φ(n), to get the following:

Finally, compute the inverse modulo n, of both sides of this congruence, to obtain the following formula for u:

Olga can use this formula to compute u in polynomial time.

Example 9.7

As in the previous example, suppose that n = 223693, b = 503, u = 101576 and v = 89888. Suppose Olga has learned that

She will first compute

*
* Figure 9.8 Issuing a value u to Alice

Next, she calculates

Finally, she can obtain the secret value u as follows:

Thus Alice's secret exponent has been compromised.

9.4.1 Identity-based Identification Schemes

The Guillou-Quisquater Identification Scheme can be transformed into what is known as an identity-based identification scheme. This basically means that certificates are not necessary. Instead, the TA computes the value of u as a function of Alice's ID string, using a public hash function h with range [image]. This is done as indicated in Figure 9.8. The identification protocol now works as described in Figure 9.9. The value v is computed from Alice's ID string via the public hash function h. In order to carry out the identification protocol, Alice needs to know the value of u, which can be computed only by the TA (assuming that the RSA cryptosystem is secure). If Olga tries to identify herself as Alice, she will not succeed because she does not know the value of u.

*
* Figure 9.9 The Guillou-Quisquater identity-based identification scheme

9.5 Converting Identification to Signature Schemes

There is a standard method of converting an identification scheme to a signature scheme. The basic idea is to replace the verifier (Bob) by a public hash function, h. In a signature scheme obtained by this approach, the message is not hashed before it is signed; the hashing is integrated into the signing algorithm.

We illustrate this approach by converting the Schnorr Scheme into a signature scheme. See Figure 9.10. In practice, one would probably take the hash function h to be the SHS, with the result reduced modulo q. Since the SHS produces a bitstring of length 160 and q is a 160-bit prime, the modulo q reduction is necessary only if the message digest produced by the SHS exceeds q; and even in this situation it is necessary only to subtract q from the result.

In proceeding from an identification scheme to a signature scheme, we replaced a 40-bit challenge by a 160-bit message digest. 40 bits suffice for a challenge since an impostor needs to be able to guess the challenge in order to precompute a response that will be accepted. But in the context of a signature scheme, we need message digests of a much larger size, in order to prevent attacking the scheme by finding collisions in the hash function.

Other identification schemes can be converted to signature schemes in a similar fashion.

*
* Figure 9.10 Schnorr Signature Scheme


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

9.6 Notes and References

The Schnorr Identification Scheme is from [SC91], the Okamoto Scheme was presented in [OK93], and the Guillou-Quisquater Scheme can be found in [GQ88]. Another scheme that can be proved secure under a plausible computational assumption has been given by Brickell and McCurley [BM92].

Other popular identification schemes include the Feige-Fiat-Shamir Scheme [FFS88] (see also [FS87]) and Shamir's Permuted Kernel Scheme [SH90]. The Feige-Fiat-Shamir Scheme is proved secure using zero-knowledge techniques (see Chapter 13 for more information on zero-knowledge proofs).

The method of constructing signature schemes from identification schemes is due to Fiat and Shamir [FS87]. They also describe an identity-based version of their identification scheme.

Surveys on identification schemes have been published by Burmester, Desmedt, and Beth [BDB92] and de Waleffe and Quisquater [DWQ93].

Exercises

9.1 Consider the following possible identification scheme. Alice possesses a secret key n = pq, where p and q are prime and p ≡ q ≡ 3 (mod 4). The values n and ID(Alice) are signed by the TA, as usual, and stored on Alice's certificate. When Alice wants to identify herself to Bob, say, Bob will present Alice with a random quadratic residue modulo n, say x. Then Alice will compute a square root y of x and give it to Bob. Bob then verifies that y2 ≡ x (mod n). Explain why this scheme is insecure.
9.2 Suppose Alice is using the Schnorr Scheme where q = 1201, p = 122503, t = 10 and α = 11538.

(a) Verify that α has order q in [image].
(b) Suppose that Alice's secret exponent is a = 357. Compute v.
(c) Suppose that k = 868. Compute γ.
(d) Suppose that Bob issues the challenge r = 501. Compute Alice's response y.
(e) Perform Bob's calculations to verify y.

9.3 Suppose that Alice uses the Schnorr Scheme with p, q, t and α as in Exercise 9.2. Now suppose that v = 51131, and Olga has learned that

Show how Olga can compute Alice's secret exponent a.

9.4 Suppose that Alice is using the Okamoto Scheme with q = 1201, p = 122503, t = 10, α1 = 60497 and α2 = 17163.

(a) Suppose that Alice's secret exponents are a1 = 432 and a2 = 423. Compute v.
(b) Suppose that k1 = 389 and k2 = 191. Compute γ.
(c) Suppose that Bob issues the challenge r = 21. Compute Alice's response, y1 and y2.
(d) Perform Bob's calculations to verify y1 and y2.

9.5 Suppose that Alice uses the Okamoto Scheme with p, q, t, α1, and α2 as in Exercise 9.4. Suppose also that v = 119504.

(a) Verify that

(b) Use this information to compute b1 and b2 such that

(c) Now suppose that Alice reveals that a1 = 484 and a2 = 935. Show how Alice and Olga together will compute [image].

9.6 Suppose that Alice is using the Guillou-Quisquater Scheme with p = 503, q = 379, and b = 509.

(a) Suppose that Alice's secret u = 155863. Compute v.
(b) Suppose that k = 123845. Compute γ.
(c) Suppose that Bob issues the challenge r = 487. Compute Alice's response, y.
(d) Perform Bob's calculations to verify y.

9.7 Suppose that Alice is using the Guillou-Quisquater Scheme with n = 199543, b = 523 and v = 146152. Suppose that Olga has discovered that

Show how Olga can compute u.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 10
Authentication Codes

10.1 Introduction

We have spent a considerable amount of time studying cryptosystems, which are used to obtain secrecy. An authentication code provides a method of ensuring the integrity of a message, i.e., that the message has not been tampered with and that it originated with the presumed transmitter. Our goal is to achieve this authentication capability even in the presence of an active opponent, Oscar, who can observe messages in the channel and introduce messages of his own choosing into the channel. This goal is accomplished in the "private-key" setting whereby Alice and Bob share a secret key, K, before any message is transmitted.

In this chapter, we study codes that provide authentication but no secrecy. In such a code, a key is used to compute an authentication tag which will enable Bob to check the authenticity of the message he receives. Another application of an authentication code is verify that data in a large file has not been tampered with. An authentication tag would be stored with the data; the key used to generate and verify the authenticator would be stored separately, in a "secure" area.

We should also point out that, in many respects, an authentication code is similar to a signature scheme or to a message authentication code (MAC). The main differences are as follows: The security of an authentication code is unconditional, whereas signature schemes and MACs are studied from the point of view of computational security. Also, when an authentication code (or a MAC) is used, a message can be verified only by the intended receiver. In comparison, anyone can verify a signature using a public verification algorithm.

We now give a formal definition of the terminology we use in the study of authentication codes.

DEFINITION 10.1 An authentication code is a four-tuple [image], where the following conditions are satisfied:

1. is a finite set of possible source states

*
* Figure 10.1 Impersonation by Oscar

2. is a finite set of possible authentication tags
3. , the keyspace, is a finite set of possible keys.
4. For each [image], there is an authentication rule [image].

The message set is defined to be [image].

REMARK Note that a source state is analogous to a plaintext. A message consists of a plaintext with an appended authentication tag; it could be more precisely referred to as a signed message. Also, an authentication rule need not be an injective function.

In order to transmit a (signed) message, Alice and Bob follow the following protocol. First, they jointly choose a random key [image]. This is done in secret, as in a private-key cryptosystem. At a later time, suppose that Alice wants to communicate a source state [image] to Bob over an insecure channel. Alice computes a = eK(s) and sends the message (s, a) to Bob. When Bob receives (s, a), he computes a′ = eK(s). If a′ = a, then he accepts the message as authentic; otherwise, he rejects it.

We will study two different types of attacks that Oscar might carry out. In both of these attacks, Oscar is an "intruder-in-the-middle." These attacks described are as follows:

Impersonation
Oscar introduces a message (s, a) into the channel, hoping to have it accepted as authentic by Bob. This is depicted in Figure 10.1.
Substitution
Oscar observes a message (s, a) in the channel, and then changes it to (s′, a′), where s′ ≠ s, again hoping to have it accepted as authentic by Bob. Hence, he is hoping to mislead Bob as to the source state. This is depicted in Figure 10.2.

Associated with each of these attacks is a deception probability, which represents the probability that Oscar will successfully deceive Bob, if he (Oscar) follows an optimal strategy. These probabilities are denoted by Pd0 (impersonation) and Pd1 (substitution). In order to compute Pd0 and Pd1, we need to specify probability distributions on [image] and [image]. These will be denoted by [image] and [image], respectively. We assume that the authentication code and these two probability distributions are known to Oscar. The only information that Alice and Bob possess that is not known to Oscar is the value of the key, K. This is analogous to the way that we studied the unconditional security of private-key cryptosystems.

*
* Figure 10.2 Substitution by Oscar


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

10.2 Computing Deception Probabilities

In this section, we look at the computation of deception probabilities. We begin with a small example of an authentication code.

Example 10.1

Suppose

and

For each [image] and each [image], define

It will be useful to study the authentication matrix, which tabulates all the values eij(s). For each key [image] and for each [image], place the authentication tag eK(s) in row K and column s of a [image] matrix M. The array M is presented in Figure 10.3.

Suppose that the key is chosen at random, i.e., [image] for each [image]. We do not specify the probability distribution [image] since it turns out to be immaterial in this example.

Let's first consider an impersonation attack. Oscar will pick a source state s, and attempt to guess the "correct" authentication tag. Denote by K0 the actual key being used (which is unknown to Oscar). Oscar will succeed in deceiving Bob if he guesses the tag [image]. However, for any [image] and [image], it is easy to verify that there are exactly three (out of nine) authentication rules [image] such that eK(s) = a. (In other words, each symbol occurs three times in each column of the authentication matrix.) Hence, it follows that Pd0 = 1/3.

*
* Figure 10.3 An authentication matrix

Substitution is a bit more complicated to analyze. As a specific case, suppose Oscar observes the message (0, 0) in the channel. This does give Oscar some information about the key: he now knows that

Now suppose Oscar replaces the message (0, 0) with the message (1, 1). Then, he will succeed in his deception if and only if K0 = (1, 0). The probability that K0 is the key is 1/3, since the key is known to be in the set {(0, 0), (1, 0), (2, 0)}.

A similar analysis can be done for any substitution that Oscar might make. In general, if Oscar observes the message (s, a), and replaces it with any message (s′, a′) where s′ ≠ s, then he deceives Bob with probability 1/3. We can see this as follows. Observation of (s, a) restricts the key to one of three possibilities. Then, for each choice of (s′, a′), there is one key (out of the three possible keys) under which a′ is the authentication tag for s′.

Let's now discuss how to compute the deception probabilities in general. First, we consider Pd0. As above, let K0 denote the key chosen by Alice and Bob. For [image] and [image], define payoff (s, a) to be the probability that Bob will accept the message (s, a) as being authentic. It is not difficult to see that

That is, payoff (s, a) is computing by selecting the rows of the authentication matrix that have entry a in column s, and summing the probabilities of the corresponding keys.

In order to maximize his chance of success, Oscar will choose (s, a) such that payoff (s, a) is a maximum. Hence,

Note that Pd0 does not depend on the probability distribution [image].

Pd1 is more difficult to compute, and it may depend on the probability distribution [image]. Let's first consider the following problem: Suppose Oscar observes the message (s, a) in the channel. Oscar will substitute some (s′, a′) for (s, a), where s′ ≠ s. Hence, for [image], and [image], we define payoff (s′, a′; s, a) to be the probability that a substitution of (s, a) with (s′, a′) will succeed in deceiving Bob. Then we can compute the following:

The numerator of this fraction is found by selecting the rows of the authentication matrix that have the value a in column s and the value a′ in column s′, and summing the probabilities of the corresponding keys.

Since Oscar wants to maximize his chance of deceiving Bob, he will compute

The quantity ps,a denotes the probability that Oscar can deceive Bob with a substitution, given that (s, a) is the message observed in the channel.

Now, how do we compute the deception probability Pd1? Evidently, we have to compute a weighted average of the quantities Ps,a with respect to the probabilities [image] of observing messages (s, a) in the channel. That is, we calculate Pd1 to be

*
* Figure 10.4 An authentication matrix

The probability distribution [image] is as follows:

In Example 10.1,

for all s, a, so Pd0 = 1/3. Also, it can be checked that

for all s, s′, a, a′, s ≠ s′. Hence, Pd1 = 1/3 for any probability distribution [image]. (In general, though, Pd1 will depend on [image].)

Let's look at the computation of Pd0 and Pd1 for a less "regular" example.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 10.2

Consider the authentication matrix of Figure 10.4. Suppose the probability distributions on [image] and [image] are

1 ≤ i ≤ 4; and

The values payoff (s, a) are as follows:

Hence, Pd0 = 3/4. Oscar's optimal impersonation strategy is to place any of the messages (1, 1), (3, 1) or (4, 2) into the channel.

Now we turn to the computation of Pd1. First, we present the various values payoff (s′, a′; s, a) in the form of a matrix. The entry in row (s, a) and column (s′, a′) is the value payoff (s′, a′; s, a).

Thus we have p1,1 = 2/3, p2,2 = 1/2, and ps,a = 1 for all other s, a. It is then a simple matter to evaluate Pd1 = 7/8. An optimal substitution strategy for Oscar is as follows:

This strategy indeed yields Pd1 = 7/8.

The computation of Pd1 in Example 10.2 is straightforward but lengthy. We can in fact simplify the computation of Pd1 by observing that we divide by the quantity payoff (s, a) in the computation of ps,a, and then later multiply by payoff (s, a) in the computation of Pd1. Of course, these two operations cancel each other out. Suppose we define

for all s, a. Then we have the following more concise formula for Pd1:

10.3 Combinatorial Bounds

We have seen that the security of an authentication code is measured by the deception probabilities. Hence, we want to construct codes so that these probabilities are as small as possible. But other considerations are also important. Let's consider the various objectives that we might strive for in an authentication code:

1. The deception probabilities Pd0 and Pd1 must be small enough to obtain the desired level of security.
2. The number of source states must be large enough so that we can communicate the desired information by appending an authentication tag to one source state.
3. The size of the key space should be minimized, since the value of the key must be communicated over a secure channel. (Note that the key must be changed every time a message is communicated, as is done with the One-time Pad.)

In this section, we determine lower bounds on the deception probabilities, which will be computed in terms of other parameters of the code. Recall that we have defined an authentication code to consist of a four-tuple [image]. Throughout this section, we will denote [image].

Suppose we fix a source state [image]. Then we can compute:

Hence, for every [image], there exists an authentication tag a(s) such that

The following theorem follows easily.

THEOREM 10.1

Suppose [image] is an authentication code. Then [image], where [image]. Further, [image] if and only if

for every [image].

Now, we turn our attention to substitution. Suppose we fix s, a and s′, where s′ ≠ s. Then we have the following:

So, there exists an authentication tag a′(s′, s, a) such that

The next theorem follows as a consequence.

THEOREM 10.2

Suppose [image] is an authentication code. Then [image], where [image]. Further, [image] if and only if

for every [image].

PROOF We have

Further, equality occurs if and only if [image] for ever (s, a). But this is in turn equivalent to the condition that [image] for every (s, a).

Combining Theorems 10.1 and 10.2, we get the following:

THEOREM 10.3

Suppose [image] is an authentication code, where [image]. Then [image] if and only if

for every [image].

PROOF Equations (10.4) and (10.5) imply Equation (10.6). Conversely, Equation (10.6) implies Equations (10.4) and (10.5).

If the keys are equiprobable, then we obtain the following corollary:

COROLLARY 10.4

Suppose [image] is an authentication code where [image], and keys are chosen equiprobably. Then [image] if and only if

for every [image].

10.3.1 Orthogonal Arrays

In this section, we look at the connections between authentication codes and certain combinatorial structures called orthogonal arrays. First, we give a definition.

*
* Figure 10.5 An OA(3, 3, 1)

DEFINITION 10.2 An orthogonal array OA(n, k, λ) is a λn2 × k array of n symbols, such that in any two columns of the array every one of the possible n2 pairs of symbols occurs in exactly λ rows.

Orthogonal arrays are well-studied structures in combinatorial design theory, and are equivalent to other structures such as transversal designs, mutually orthogonal Latin squares and nets.

In Figure 10.5, we present an orthogonal array OA(3, 3, 1) which is obtained from the authentication matrix of Figure 10.3. Any orthogonal array OA(n, k, λ) can be used to construct an authentication code with Pd0 = Pd1 = 1/n, as stated in the following theorem.

THEOREM 10.5

Suppose there is an orthogonal array OA(n, k, λ). Then there is an authentication code [image], where [image] and Pd0 = Pd1 = 1/n.

PROOF Use each row of the orthogonal array as an authentication rule with equal probability 1/(λn2). The correspondences are as follows:

Since Equation (10.7) is satisfied, we can apply Corollary 10.4, obtaining a code with the stated properties.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

10.3.2 Constructions and Bounds for OAs

Suppose that we construct an authentication code from an OA(n, k, λ). The parameter n determines the number of authenticators (i.e., the security of the code), while the parameter k determines the number of source states the code can accommodate. The parameter λ relates only to the number of keys, which is λn2. Of course, the case λ = 1 is most desirable, but we will see that it is sometimes necessary to use orthogonal arrays with higher values of λ.

Suppose we want to construct an authentication code with a specified source set [image], and a specified security level ∈ (i.e., so that Pd0 ≤ ∈ and Pd1 ≤ ∈). An appropriate orthogonal array will satisfy the following conditions:

1. n ≥ 1/∈
2. (observe that we can always delete one or more columns from an orthogonal array and the resulting array is still an orthogonal array, so we do not require [image])
3. λ is minimized, subject to the two previous conditions being satisfied.

Let's first consider orthogonal arrays with λ = 1. For a given value of n, we are interested in maximizing the number of columns. Here is a necessary condition for existence:

THEOREM 10.6

Suppose there exists an OA(n, k, 1). Then k ≤ n + 1.

PROOF Let A be an OA(n, k, 1) on symbol set X = {0, 1, …, n - 1}. Suppose π is a permutation of X, and we permute the symbols in any column of A according to the permutation π. The result is again an OA(n, k, 1). Hence, by applying a succession of permutations of this type, we can assume without loss of generality that the first row of A is (00 … 0).

We next show that each symbol must occur exactly n times in each column of A. Choose two columns, say c and c′, and let x be any symbol. Then for each symbol x′, there is a unique row of A in which x occurs in column c and x′ occurs in column c′. Letting x′ vary over X, we see that x occurs exactly n times in column c.

Now, since the first row is (00 … 0), we have exhausted all occurrences of ordered pairs (0, 0). Hence, no other row contains more than one occurrence of 0. Now, let us count the number of rows containing at least one 0: the total is 1 + k(n - 1). But this total cannot exceed the total number of rows in A, which is n2. Hence, 1 + k(n - 1) ≤ n2, so k ≤ n + 1, as desired.

We now present a construction for orthogonal arrays with λ = 1 in which k = n. This is, in fact, the construction that was used to obtain the orthogonal array presented in Figure 10.5.

THEOREM 10.7

Suppose p is prime. Then there exists an orthogonal array OA(p, p, 1).

PROOF The array will be a p2 × p array, where the rows are indexed by [image] and the columns are indexed by [image]. The entry in row (i, j) and column x is defined to be ix + j mod p.

Suppose we choose two columns, x, y, x ≠ y, and two symbols a, b. We want to find a (unique) row (i, j) such that a occurs in column x and b occurs in column y of row (i, j). Hence, we want to solve the two equations

for the unknowns i and j (where all arithmetic is done in the field [image]). But this system has the unique solution

Hence, we have an orthogonal array.

We remark that any OA(n, n, 1) can be extended by one column to form an OA(n, n + 1, 1) (see the Exercises). Hence, using Theorem 10.7, we can obtain an infinite class of OA's that meet the bound of Theorem 10.6 with equality.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Theorem 10.6 tells us that λ > 1 if k > n + 1. We will prove a more general result that places a lower bound on λ as a function of n and k. First, however, we derive an important inequality that we will use in the proof.

LEMMA 10.8

Suppose b1, …, bm are real numbers. Then

PROOF Apply Jensen's Inequality (Theorem 2.5) with f(x) = -x2 and ai = 1/m, 1 ≤ i ≤ m. The function f is continuous and concave, so we obtain

which simplifies to give the desired result.

THEOREM 10.9

Suppose there exists an OA(n, k, λ). Then

PROOF Let A be an OA(n, k, λ) on symbol set X = {0, 1,…, n - 1}, where, without loss of generality, the first row of A (00 … 0) (as in Theorem 10.6).

Let us denote the set of rows of A by [image] let r1 denote the first row, and let [image]. For any row r of A, denote by xr the number of occurrences of 0 in row r. It is easy to count the total number of occurrences of 0 in [image]. Since each symbol must occur exactly λn times in each column of A, we have that

Now, the number of times the ordered pair (0, 0) occurs in rows in [image] is

Applying Lemma 10.8, we obtain

and hence

On the other hand, in any given pair of columns, the ordered pair (0, 0) occurs in exactly λ rows. Since there are k(k - 1) ordered pairs of columns, it follows that the exact number of occurrences of the ordered pair (0, 0) in rows in [image] is (λ - 1)k(k - 1). We therefore have

and hence

If we divide out a factor of k, we get

Expanding, we have

This simplifies to give

or

Finally, taking out a factor of λ(n - 1), we obtain

which is the desired bound.

Our next result establishes the existence of an infinite class of orthogonal arrays that meet the above bound with equality.

THEOREM 10.10

Suppose p is prime and d ≥ 2 is an integer. Then there is an orthogonal array OA(p, (pd - 1)/(p - 1), pd-2).

PROOF Denote by [image] the vector space of all d-tuples over [image]. We will construct A, an OA(p,(pd - 1)/(p - 1), pd-2) in which the rows and columns are indexed by certain vectors in [image]. The entries of A will be elements of [image]. The set of rows is defined to be [image]; the set of columns is

consists of all vectors in [image], so [image]. consists of all non-zero vectors that have the first non-zero coordinate equal to 1. Observe that

and that no two vectors in [image] are scalar multiples of each other.

Now, for each [image] and each [image], define

where · denotes the inner product of two vectors (reduced modulo p).

We prove that A is the desired orthogonal array. Let [image] be two distinct columns, and let [image]. We will count the number of row [image] such that [image] and [image]. Denote [image] and [image]. The two equations [image] can be written as two linear equations in [image]:

*
* Figure 10.6 An OA(2, 7, 2)

This is a system of two linear equations in the d unknowns r1, … rd. Since [image] and [image] are not scalar multiples, the two equations are linearly independent. Hence, this system has a solution space of dimension d - 2. That is, the number of solutions (i.e., the number of rows in which x occurs in column [image] and y occurs in column [image]) is pd-2, as desired.

Let's carry out a small example of this construction.

Example 10.3

Suppose we take p = 2, d = 3. Then we will construct an OA(2, 7, 2). We have

and

The orthogonal array in Figure 10.6 results.

10.3.3 Characterizations of Authentication Codes

To this point, we have studied authentication codes obtained from orthogonal arrays. Then we looked at necessary existence conditions and constructions for orthogonal arrays. One might wonder whether there are better alternatives to the orthogonal array approach. However, two characterization theorems tell us that this is not the case if we restrict our attention to authentication codes in which the deception probabilities are as small as possible.

We first prove the following partial converse to Theorem 10.5:

THEOREM 10.11

Suppose [image] is an authentication code where [image] and Pd0 = Pd1 = 1/n. Then [image]. Further, [image] if and only if there is an orthogonal array OA (n, k, 1) where [image], and [image] for every key [image].

PROOF Fix two (arbitrary) source states s and s′, s ≠ s′, and consider Equation (10.6). For each ordered pair (a, a′) of authentication tags, define

Then [image] for every pair (a, a′). Also, the n2, sets [image] are disjoint. Hence, [image].

Now, suppose that [image]. Then [image] for every pair (a, a′), and Equation (10.6) tells us that [image] for every key [image].

It remains to show that the authentication matrix forms an orthogonal array OA(n, k, 1). Consider the columns indexed by the source states s and s′. Since [image] for every (a, a′), we have every ordered pair occurring exactly once in these two columns. Since, s and s′ are arbitrary, we see that every ordered pair occurs exactly once in any two columns.

The following characterization is more difficult; we state it without proof.

THEOREM 10.12

Suppose [image] is an authentication code where [image] and Pd0 = Pd1 = 1/n. Then [image]. Further, [image] if and only if there is an orthogonal array OA(n, k, λ), where λ = (k(n - 1) + 1)/n2, and [image] for every key [image].

REMARK Notice that Theorem 10.10 provides an infinite class of orthogonal arrays that meet the bound of Theorem 10.12 with equality.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

10.4 Entropy Bound

In this section, we use entropy techniques to obtain bounds on the deception probabilities. The first of these is a bound on Pd0.

THEOREM 10.13

Suppose that [image] is an authentication code. Then

PROOF From Equation (10.1), we have

Since the maximum of the values payoff (s, a) is greater than their weighted average, we obtain

Hence, by Jensen's inequality (Theorem 2.5), we have

Recalling from Section 10.2 that

we see that

Now, we observe that [image] (i.e., the probability that a is the authenticator, given that s is the source state). Hence,

by the definition of conditional entropy. We complete the proof by showing that - H(A|S) = H(K|M) - H(K). This follows from basic entropy identities. On one hand, we have

On the other hand, we compute

where we use the facts that H(A|K, S) = 0 since the key and source state uniquely determine the authenticator, and H(K, S) = H(K) + H(S) since the source and key are independent events.

Equating the two expressions for H(K, A, S), we obtain

But a message m = (s, a) is defined to consist of a source state and an authenticator (i.e., [image]). Hence, H(K|A, S) = H(K|M) and the proof is complete.

There is a similar bound for Pd1 which we will not prove here. It is as follows:

THEOREM 10.14

Suppose that [image] is an authentication code. Then

We need to define what we mean by the random variable M2. Suppose we authenticate two distinct source states using the same key K. In this way, we obtain an ordered pair of message [image]. In order to define a probability distribution on [image], it is necessary to define a probability distribution on [image], with the stipulation that [image] for every [image] (that is, we do not allow source states to be repeated). The probability distribution on [image] and [image] will induce a probability distribution on [image], in the same way that the probability distributions on [image] and [image] induce a probability on [image].

As an illustration of the two bounds, we consider our basic orthogonal array construction and show that the bounds of Theorems 10.13 and 10.14 are both met with equality. First, it is clear that

since each of the λn2 authentication rules are chosen with equal probability. Let's next turn to the computation of H(K|M). If any message m = (s, a) is observed, this restricts the possible keys to a subset of size λn. Each of these λn keys is equally likely. Hence, H(K|m) = log λn, for any message m. Then, we get the following:

Thus we have

so the bound is met with equality.

If we observe two messages which have been produced using the same key (and different source states), then the number of possible keys is reduced to λ. Using similar reasoning as above, we have that H(K|M2) = log λ. Then

so this bound is also met with equality.

10.5 Notes and References

Authentication codes were invented in 1974 by Gilbert, MacWilliams, and Sloane [GMS74]. Much of the theory of authentication codes was developed by Simmons, who proved many fundamental results in the area. Two useful survey articles by Simmons are [SI92] and [SI88]. Another good survey is Massey [MA86].

The connections between orthogonal arrays and authentication codes has been addressed by several researchers. The treatment here is based on three papers by Stinson [ST88], [ST90] and [ST92]. Orthogonal arrays have been studied for over 45 years by researchers in statistics and in combinatorial design theory. For example, the bound in Theorem 10.9 was first proved by Plackett and Berman in 1945 in [PB45]. Many interesting results on orthogonal arrays can be found in various textbooks on combinatorial design theory such as Beth, Jungnickel, and Lenz [BJL85].

Finally, the use of entropy techniques in the study of authentication codes was introduced by Simmons. The bound of Theorem 10.13 was first proved in Simmons [SI85]; a proof of Theorem 10.14 can be found in Walker [WA90].


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Exercises

10.1 Compute Pd0 and Pd1 for the following authentication code, represented in matrix form:

The probability distributions on [image] and [image] are as follows:

What are the optimal impersonation and substitution strategies?

10.2 We have seen a construction for an orthogonal array OA(p, p, 1) when p is prime. Prove that this OA(p, p, 1) can always be extended by one extra column to form an OA(p, p + 1, 1). Illustrate your construction in the case p = 5.
10.3 Suppose A is an OA(n1, k, λ1) on symbol set {1, …, n1} and suppose B is an OA(n2, k, λ2) on symbol set {1, …, n2}. We construct C, an OA(n1n2, k, λ1 λ2) on symbol set {1, …, n1} × {1, …, n2}, as follows: for each row r1 = (x1, …, xk) of A and for each row s1 = (y1, …, yk) of B, define a row

of C. Prove that C is indeed an OA(n1 n2, k, λ1 λ2).

10.4 Construct an orthogonal array OA(3, 13, 3).
10.5 Write a computer program to compute H(K), H(K|M) and H(K|M2) for the authentication code from Exercise 10.1. The probability distribution on sequences of two sources is as follows:

Compare the entropy bounds for Pd0 and Pd1 with the actual values you computed in Exercise 10.1.

HINT To compute [image], use Bayes' formula

We already know how to calculate [image]. To compute [image], write m = (s, a) and then observe that [image] if ek(s) = a, and [image] otherwise.

To compute [image], use Bayes' formula

can be calculated as follows: write m1 = (s1, a1) and m2 = (s2, a2). Then

(Note the similarity with the computation of p(m).) To compute [image], observe that [image] and ek(s2) = a2, and [image], otherwise.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 11
Secret Sharing Schemes

11.1 Introduction: The Shamir Threshold Scheme

In a bank, there is a vault which must be opened every day. The bank employs three senior tellers, but they do not trust the combination to any individual teller. Hence, we would like to design a system whereby any two of the three senior tellers can gain access to the vault, but no individual teller can do so. This problem can be solved by means of a secret sharing scheme, the topic of this chapter.

Here is an interesting "real-world" example of this situation: According to Time-Magazine1, control of nuclear weapons in Russia involves a similar "two-out-of-three" access mechanism. The three parties involved are the President, the Defense Minister and the Defense Ministry.

1 Time Magazine, May 4, 1992, p. 13

We first study a special type of secret sharing scheme called a threshold scheme. Here is an informal definition.

DEFINITION 11.1 Let t, w be positive integers, t ≤ w. A (t, w) -threshold scheme is a method of sharing a key K among a set of w participants (denoted by [image]), in such a way that any t participants can compute the value of K, but no group of t - 1 participants can do so.

Note that the examples described above are (2, 3)-threshold schemes.

The value of K is chosen by a special participant called the dealer. The dealer is denoted by D and we assume [image]. When D wants to share the key K among the participants in [image], he gives each participant some partial information called a share. The shares should be distributed secretly, so no participant knows the share given to another participant.

At a later time, a subset of participants [image] will pool their shares in an attempt to compute the key K. (Alternatively, they could give their shares to a trusted authority which will perform the computation for them.) If |B| ≥ t, then they should be able to compute the value of K as a function of the shares they collectively hold; if |B| < t, then they should not be able to compute K.

*
* Figure 11.1 The Shamir (t, w)-threshold scheme in

We will use the following notation. Let

be the set of w participants. is the key set (i.e., the set of all possible keys); and [image] is the share set (i.e., the set of all possible shares).

In this section, we present a method of constructing a (t, w)-threshold scheme, called the Shamir Threshold Scheme, which was invented in 1979. Let [image] where p ≥ w + 1 is prime. Also, let [image]. Hence, the key will be an element of [image], as will be each share given to a participant. The Shamir threshold scheme is presented in Figure 11.1. In this scheme, the dealer constructs a random polynomial a(x) of degree at most t - 1 in which the constant term is the key, K. Every participant Pi obtains a point (xi, yi) on this polynomial.

Let's look at how a subset B of t participants can reconstruct the key. This is basically accomplished by means of polynomial interpolation. We will describe a couple of methods of doing this.

Suppose that participants [image] want to determine K. They know that

1 ≤ j ≤ t, where [image] is the (secret) polynomial chosen by D. Since a(x) has degree at most t - 1, a(x) can be written as

where the coefficients a0, . . . , at-1 are unknown elements of [image], and a0 = K is the key. Since [image], 1 ≤ j ≤ t, B can obtain t linear equations in the t unknowns a0, . . . , at-1, where all arithmetic is done in [image]. If the equations are linearly independent, there will be a unique solution, and a0 will be revealed as the key.

Here is a small example to illustrate.

Example 11.1

Suppose that p = 17, t = 3, and w = 5; and the public x-co-ordinates are xi = i, 1 ≤ i ≤ 5. Suppose that B = {P1, P3, P5} pool their shares, which are respectively 8, 10, and 11. Writing the polynomial a(x) as

and computing a(1), a(3) and a(5), the following three linear equations in [image] are obtained:

This system does have a unique solution in [image]. The key is therefore K = a0 = 13.

Clearly, it is important that the system of t linear equations has a unique solution, as in Example 11.1. We show now that this is always the case. In general, we have

1 ≤ j ≤ t, where

and

The system of linear equations (in [image]) is the following:

This can be written in matrix form as follows:

Now, the coefficient matrix A is a so-called Vandermonde matrix. There is a well-known formula for the determinant of a Vandermonde matrix, namely

Recall that the xi's are all distinct, so no term [image] in this product is equal to zero. The product is computed in [image], where p is prime, which is a field. Since the product non-zero terms in a field is always non-zero, we have that det A ≠ 0. Since the determinant of the coefficient matrix is non-zero, the system has a unique solution over the field [image]. This establishes that any group of t participants will be able to recover the key in this threshold scheme.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

What happens if a group of t - 1 participants attempt to compute K? Proceeding as above, they will obtain a system of t - 1 equations in t unknowns. Suppose they hypothesize a value y0 for the key. Since the key is a0 = a(0), this will yield a tth equation, and the coefficient matrix of the resulting system of t equations in t unknowns will again be a Vandermonde matrix. As before, there will be a unique solution. Hence, for every hypothesized value y0 of the key, there is a unique polynomial ay0(x) such that

1 ≤ j ≤ t - 1, and such that

Hence, no value of the key can be ruled out, and thus a group of t - 1 participants can obtain no information about the key.

We have analyzed the Shamir scheme from the point of view of solving systems of linear equations over [image]. There is an alternative method, based on the Lagrange interpolation formula for polynomials. The Lagrange interpolation formula is an explicit formula for the (unique) polynomial a(x) of degree at most t that we computed above. The formula is as follows:

It is easy to verify the correctness of this formula by substituting [image]: all terms in the summation vanish except for the jth term, which is [image]. Thus, we have a polynomial of degree at most t - 1 which contains the t ordered pairs [image], 1 ≤ j ≤ t. We already proved above that this polynomial is unique, so the interpolation formula does yield the correct polynomial.

A group B of t participants can compute a(x) by using the interpolation formula. But a simplification is possible, since the participants in B do not need to know the whole polynomial a(x). It is sufficient for them to compute the constant term K = a(0). Hence, they can compute the following expression, which is obtained by substituting x = 0 into the Lagrange interpolation formula:

Suppose we define

1 ≤ j ≤ t. (Note that these values bj can be precomputed, if desired, and their values are not secret.) Then we have

Hence, the key is a linear combination of the t shares.

To illustrate this approach, let's recompute the key from Example 11.1.

Example 11.1 (Con't.)

The participants {P 1, P3, P5} can compute b1, b2, and b3 according to the formula given above. For example, they would obtain

Similarly, b2 = 3 and b3 = 11. Then, given shares 8, 10, and 11 (respectively), they would obtain

as before.

The last topic of this section is a simplified construction for threshold schemes in the special case w = t. This construction will work for any key set [image] with [image]. (For this scheme, it is not required that m be prime, and it is not necessary that m ≥ w + 1.) If D wants to share the key [image], he carries out the protocol of Figure 11.2.

*
* Figure 11.2 A (t, t)-threshold scheme in

Observe that the t participants can compute K by the formula

Can t - 1 participants compute K? Clearly, the first t - 11 participants cannot do so, since they receive t - 1 independent random numbers as their shares. Consider the t - 1 participants in the set [image], where 1 ≤ i ≤ t - 1. These t - 1 participants possess the shares

and

By summing their shares, they can compute K - yi. However, they do not know the random value yi, and hence they have no information as to the value of K. Consequently, we have a (t, t)-threshold scheme.

11.2 Access Structures and General Secret Sharing

In the previous section, we desired that any t of the w participants should be able to determine the key. A more general situation is to specify exactly which subsets of participants should be able to determine the key and which should not. Let Γ be a set of subsets of [image]; the subsets in Γ are those subsets of participants that should be able to compute the key. Γ is called an access structure and the subsets in Γ are called authorized subsets.

Let [image] be the key set and let [image] be the share set. As before, when a dealer D wants to share a key [image], he will give each participant a share from [image]. At a later time a subset of participants will attempt to determine K from the shares they collectively hold.

DEFINITION 11.2 A perfect secret sharing scheme realizing the access structure Γ is a method of sharing a key K among a set of w participants (denoted by [image]), in such a way that the following two properties are satisfied:

1. If an authorized subset of participants, [image] pool their shares, then they can determine the value of K.
2. If an unauthorized subset of participants [image] pool their shares, then they can determine nothing about the value of K.

Observe that a (t, w)-threshold scheme realizes the access structure

Such an access structure is called a threshold access structure. We showed in the previous section that the Shamir scheme is a perfect scheme realizing the threshold access structure.

We study the unconditional security of secret sharing schemes. That is, we do not place any limit on the amount of computation that can be performed by an unauthorized subset of participants.

Suppose that B ∈ Γ and [image]. Suppose the subset C wants to determine K. Since B is an authorized subset, it can already determine K. Hence, the subset C can determine K by ignoring the shares of the participants in C\B. Stated another way, a superset of an authorized set is again an authorized set. What this says is that the access structure should satisfy the monotone property:

In the remainder of this chapter, we will assume that all access structures are monotone.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

If Γ is an access structure, then B ∈ Γ is a minimal authorized subset if A ∉ Γ whenever [image]. The set of minimal authorized subsets of Γ is denoted Γ0 and is called the basis of Γ. Since Γ consists of all subsets of [image] that are supersets of a subset in the basis Γ0, Γ is determined uniquely as a function of Γ0. Expressed mathematically, we have

We say that Γ is the closure of Γ0 and write

Example 11.2

Suppose [image] and

Then

Conversely, given this access structure Γ, it is easy to see that Γ0 consists of the minimal subsets in Γ.

In the case of a (t, w)-threshold access structure, the basis consists of all subsets of (exactly) t participants.

11.3 The Monotone Circuit Construction

In this section, we will give a conceptually simple and elegant construction due to Benaloh and Leichter that shows that any (monotone) access structure can be realized by a perfect secret sharing scheme. The idea is to first build a monotone circuit that "recognizes" the access structure, and then to build the secret sharing scheme from the description of the circuit. We call this the monotone circuit construction.

Suppose we have a boolean circuit C, with w boolean inputs, x1, . . . , xw (corresponding to the w participants P1, . . . , Pw), and one boolean output, y. The circuit consists of "or" gates and "and" gates; we do not allow any "not" gates. Such a circuit is called a monotone circuit. The reason for this nomenclature is that changing any input xi from "0" (false) to "1" (true) can never result in the output y changing from "1" to "0." The circuit is permitted to have arbitrary fan-in, but we require fan-out equal to 1 (that is, a gate can have arbitrarily many input wires, but only one output wire).

If we specify boolean values for the w inputs of such a monotone circuit, we can define

i.e., the subset of [image] corresponding to the true inputs. Suppose C is a monotone circuit, and define

where C (x1, . . . , xw) denotes the output of C, given inputs x1, . . . , xw. Since the circuit C is monotone, it follows that Γ(C) is a monotone set of subsets of [image].

It is easy to see that there is a one-to-one correspondence between monotone circuits of this type and boolean formulae which contain the operators [image] ("and") and [image] ("or"), but do not contain any negations.

If Γ is a monotone set of subsets of [image], then it is easy to construct a monotone circuit C such that Γ(C) = Γ. One way to do this is as follows. Let Γ0 be the basis of Γ. Then construct the disjunctive normal form boolean formula

In Example 11.2, where

we would obtain the boolean formula

Each clause in the boolean formula corresponds to an "and" gate of the associated monotone circuit; the final disjunction corresponds to an "or" gate. The number of gates in the circuit is |Γ0| + 1.

Suppose C is any monotone circuit that recognizes Γ (note that C need not be the circuit described above.) We describe an algorithm which enables D, the dealer, to construct a perfect secret sharing scheme that realizes Γ. This scheme will use as a building block the (t, t)-schemes constructed in Figure 11.2. Hence, we take the key set to be [image] for some integer m.

The algorithm proceeds by assigning a value [image] to every wire W in the circuit C. Initially, the output wire Wout of the circuit is assigned the value K, the key. The algorithm iterates a number of times, until every wire has a value assigned to it. Finally, each participant Pi is given the list of values f(W) such that W is an input wire of the circuit which receives input xi.

A description of the construction is given in Figure 11.3. Note that, whenever a gate G is an "and" gate having (say) t input wires, we share the "key" f(WG) among the input wires using a (t, t)-threshold scheme.

Let's carry out this procedure for the access structure of Example 11.2, using the circuit corresponding to the boolean formula (11.1).


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Example 11.3

We illustrate the construction in Figure 11.4. Suppose K is the key. The value K is given to each of the three input wires of the final "or" gate. Next, we consider the "and" gate corresponding to the clause [image]. The three input wires are assigned values a1, a2, K - a1 - a2, respectively, where all arithmetic is done in [image]. In a similar way, the three input wires corresponding to [image] are assigned values b1, b2, K - b1 - b2. Finally, the two input wires corresponding to [image] are assigned values ci, K - c1. Note that a1, a2, b1, b2 and c1 are all independent random values in [image]. If we look at the shares that the four participants receive, we have the following:

1. P1 receives a 1, b1.
2. P2 receives a2, c1.
3. P3 receives b2, K - c1.
4. P4 receives K - a1 - a2, K - b1 - b2.

*
* Figure 11.3 The monotone circuit construction

Thus, every participant receives two elements of [image] as his or her share.

Let's prove that the scheme is perfect. First, we verify that each basis subset can compute K. The authorized subset {P1, P2, P4} can compute

The subset {P1, P3, P4} can compute

Finally, the subset {P2, P3} can compute

*
* Figure 11.4 A monotone circuit

Thus any authorized subset can compute K, so we turn our attention to the unauthorized subsets. Note that we do not need to look at all the unauthorized subsets. For, if B1 and B2 are both unauthorized subsets, [image], and B2 cannot compute K, then neither can B1 compute K. Define a subset [image] to be a maximal unauthorized subset if B1 ∈ Γ for all [image]. It follows that it suffices to verify that none of the maximal unauthorized subsets can determine any information about K. Here, the maximal unauthorized subsets are

In each case, it is easy to see that K cannot be computed, either because some necessary piece of "random" information is missing, or because all the shares possessed by the subset are random. For example, the subset {P1, P2} possesses only the random values a1, b1, a2, c1. As another example, the subset {P3, P4} possesses the shares b2, K - c1, K - a1 - a2, K - b1 - b2. Since the values of c1, a1, a2, and b1 are unknown random values, K cannot be computed. In each possible case, an unauthorized subset has no information about the value of K.

We can obtain a different scheme realizing the same access structure by using a different circuit. We illustrate by returning again to the access structure of Example 11.2.

Example 11.4

Suppose we convert the formula (11.1) to the so-called conjunctive normal form:

(The reader can verify that this formula is equivalent to the formula (11.1).) If we implement the scheme using the circuit corresponding to formula (11.2), then we obtain the following:

1. P1 receives a1, a2.
2. P2 receives a1, a3, a4.
3. P3 receives a2, a3, K - a1 - a2 - a3 - a4.
4. P4 receives a4, K - a1 - a2 - a3 - a4.

We leave the details for the reader to check.

We now prove that the monotone circuit construction always produces a perfect secret sharing scheme.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

THEOREM 11.1

Let C be any monotone boolean circuit. Then the monotone circuit construction yields a perfect secret sharing scheme realizing the access structure Γ(C).

PROOF We proceed by induction on the number of gates in the circuit C. If C contains only one gate, then the result is fairly trivial: If C consists of one "or" gate, then every participant will be given the key. This scheme realizes the access structure consisting of all non-empty subsets of participants. If C consists of a single "and" gate with t inputs, then the scheme is the (t, t)-threshold scheme presented in Figure 11.2.

Now, as an induction assumption, suppose that there is an integer j > 1 such that, for all circuits C with fewer than j gates, the construction produces a scheme that realizes Γ(C). Let C be a circuit on j gates. Consider the "last" gate, G, in the circuit; again, G could be either an "or" gate or an "and" gate. Let's first consider the case where G is an "or" gate. Denote the input wires to G by Wi, 1 ≤ i ≤ t. These t input wires are the outputs of t sub-circuits of C, which we denote Ci, 1 ≤ i ≤ t. Corresponding to each Ci, we have a (sub-)scheme that realizes the access structure ΓCi, by induction. Now, it is easy to see that

Since every Wi is assigned the key K, it follows that the scheme realizes Γ(C), as desired.

The analysis is similar if G is an "and" gate. In this situation, we have

Since the key K is shared among the t wires Wi using a (t, t)-threshold scheme, it follows again that the scheme realizes Γ(C). This completes the proof.

Of course, when an authorized subset, B, wants to compute the key, the participants in B need to know the circuit used by D to distribute shares, and which shares correspond to which wires of the circuit. All this information will be public knowledge. Only the actual values of the shares are secret. The algorithm for reconstructing the key involves combining shares according to the circuit, with the stipulation that an "and" gate corresponds to summing the values on the input wires modulo m (provided these values are all known), and an "or" gate involves choosing the value on any input wire (with the understanding that all these values will be identical).

11.4 Formal Definitions

In this section, we will give formal mathematical definitions of a (perfect) secret sharing scheme. We represent a secret sharing scheme by a set of distribution rules. A distribution rule is a function

A distribution rule represents a possible distribution of shares to the participants, where f(Pi) is the share given to Pi, 1 ≤ i ≤ w.

Now, for each [image], let [image] be a set of distribution rules. will be distribution rules corresponding to the key having the value K. The sets of distribution rule [image] are public knowledge.

Next, define

is the complete set of distribution rules of the scheme. If [image] is the value of the key that D wishes to share, then D will choose a distribution rule [image], and use it to distribute shares.

This is a completely general model in which we can study secret sharing schemes. Any of our existing schemes can be described in this setting by determining the possible distribution rules which the scheme will use. The fact that this model is mathematically precise makes it easier to give definitions and to present proofs.

It is useful to develop conditions which ensure that a set of distribution rules for a scheme realizes a specified access structure. This will involve looking at certain probability distributions, as we did previously when studying the concept of perfect secrecy. To begin with, we suppose that there is a probability distribution [image] on [image]. Further, for every [image], D will choose a distribution rule in [image] according to a probability distribution [image].

Given these probability distributions, it is straightforward to compute the probability distribution on the list of shares given to any subset of participants, B (authorized or unauthorized). This is done as follows. Suppose [image]. Define

where the function f|B denotes the restriction of the distribution rule f to B. That is, [image] is defined by

for all Pi ∈ B. Thus, [image] is the set of possible distributions of shares to the participants in B.

The probability distribution on [image], denoted [image], is computed as follows: Let [image]. Then

Also

for all [image] and [image].

Here now is a formal definition of a perfect secret sharing scheme.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

DEFINITION 11.3 Suppose Γ is an access structure and [image] is a set of distribution rules. The [image] is a perfect secret sharing scheme realizing the access structure Γ provided that the following two properties are satisfied:

1. For any authorized subset of participants [image], there do not exist two distribution rules [image] and [image] with K ≠ K', such that f|B = f'|B. (That is, any distribution of shares to the participants in an authorized subset B determines the value of the key.)
2. For any unauthorized subset of participants [image] and for any distribution of shares [image] for every [image]. (That is, the conditional probability distribution on [image], given a distribution of shares fB to an unauthorized subset B, is the same as the a priori probability distribution on [image]. In other words, the distribution of shares to B provides no information as to the value of the key.)

*
* Figure 11.5 Distribution rules for a secret sharing scheme

Observe that the second property in Definition 11.3 is very similar to the concept of perfect secrecy; this similarity is why the resulting secret sharing scheme is termed "perfect."

Note that the probability [image] can be computed from probability distributions exhibited above using Bayes' theorem:

Let us now illustrate these definitions by looking at a small example.

Example 11.5

We will present the distribution rules for the scheme constructed in Example 11.4 when it is implemented in [image]. Each of [image] and [image] contains 16 equiprobable distribution rules. For conciseness, we replace a binary k-tuple by an integer between 0 and 2k - 1. If this is done, then [image] and [image] are as depicted in Figure 11.5, where each row represents a distribution rule.

This yields a perfect scheme for any probability distribution [image] on the keys. We will not perform all the verifications here, but we will look at a couple of typcial cases to illustrate the use of the two properties in Definition 11.3.

The subset {P2, P3} is an authorized subset. Thus the shares that P2 and P3 receive should (together) determine a unique key. It can easily be checked that any distribution of shares to these two participants occurs in a distribution rule in at most one of the sets [image] and [image]. For example, if P2 has the share 3 and P3 has the share 6, then the distribution rule must be the eighth rule in [image] and thus the key is 0.

On the other hand, B = {P1, P2} is an unauthorized subset. It is not too hard to see that any distribution of shares to these two participants occurs in exactly one distribution rule in [image] and in exactly one distribution rule in [image]. That is,

for any [image] and for K = 0, 1. Next, we compute

Now, we use Bayes' theorem to compute [image]:

so the second property is satisfied for this subset B.

Similar computations can be performed for other authorized and unauthorized sets, and in each case the appropriate property is satisfied. Hence we have a perfect secret sharing scheme.

11.5 Information Rate

The results of Section 11.3 prove that any monotone access structure can be realized by a perfect secret sharing scheme. We now want to consider the efficiency of the resulting schemes. In the case of a (t, w)-threshold scheme, we can construct a circuit corresponding to the disjunctive normal form boolean formula which will have [image] gates. Each participant will receive [image] elements of [image] as his or her share. This seems very inefficient, since a Shamir (t, w)-threshold scheme enables a key to be shared by giving each participant only one "piece" of information.

In general, we measure the efficiency of a secret sharing scheme by the information rate, which we define now.

DEFINITION 11.4 Suppose we have a perfect secret sharing scheme realizing an access structure Γ. The information rate for Pi is the ratio

(Note that [image] denotes the set of possible shares that Pi might receive; of course [image] The information rate of the scheme is denoted by ρ and is defined as

The motivation for this definition is as follows. Since the key K comes from a finite set [image], we can think of K as being represented by a bit-string of length [image], by using a binary encoding, for example. In a similar way, a share given to Pi can be represented by a bit-string of length [image]. Intuitively, Pi receives [image] bits of information (in his or her share), but the information content of the key is [image] bits. Thus ρi is the ratio of the number of bits in a share to the number of bits in the key.

Example 11.6

Let's look at the two schemes from Section 11.2. The scheme produced in Example 11.3 has

However, in Example 11.4, we get a scheme with

Hence, the first implementation is preferable.

In general, if we construct a scheme from a circuit C using the monotone circuit construction, then the information rate can be computed as indicated in the following theorem.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

THEOREM 11.2

Let C be any monotone boolean circuit. Then there is a perfect secret sharing scheme realizing the access structure Γ(C) having information rate

where ri denotes the number of input wires to C carrying the input xi.

With respect to threshold access structures, we observe that the Shamir scheme will have information rate 1, which we show below is the optimal value. In contrast, an implementation of a (t, w)-threshold scheme using a disjunctive normal form boolean circuit will have information rate [image], which is much lower (and therefore inferior) if 1 < t < w.

Obviously, a high information rate is desirable. The first general result we prove is that ρ ≤ 1 in any scheme.

THEOREM 11.3

In any perfect secret sharing scheme realizing an access structure Γ, ρ ≤ 1.

PROOF Suppose we have a a perfect secret sharing scheme that realizes the access structure Γ. Let B ∈ Γ0 and choose any participant Pj ∈ B. Define B′ = B\{Pj}. Let [image]. Now, B′ ∉ Γ, so the distribution of shares g|B′ provides no information about the key. Hence, for each [image], there is a distribution rule [image] such that gK|B′ = g|B′. Since B ∈ Γ, it must be the case that gK (Pj) ≠ gK′ (Pj) if K ≠ K′. Hence, [image] and thus ρ ≤ 1.

Since ρ = 1 is the optimal situation, we refer to such a scheme an ideal scheme. The Shamir schemes are ideal schemes. In the next section, we present a construction for ideal schemes that generalizes the Shamir schemes.

11.6 The Brickell Vector Space Construction

In this section, we present a construction for certain ideal schemes known as the Brickell vector space construction.

Suppose Γ is an access structure, and let [image] denote the vector space of all d-tuples over [image], where p is prime and d ≥ 2. Suppose there exists a function

which satisfies the property

*
* Figure 11.6 The Brickell scheme

In other words, the vector (1, 0, . . . , 0) can be expressed as a linear combination of the vectors in the set {φ(Pi) : Pi ∈ B} if and only if B is an authorized subset.

Now, suppose there is a function φ that satisfies Property (11.3). (In general, finding such a function is often a matter of trial and error, though we will see some explicit constructions of suitable functions φ for certain access structures a bit later.) We are going to construct an ideal secret sharing scheme with [image]. The distribution rules of the scheme are as follows: for every vector [image], define a distribution rule [image], where

for every [image], and the operation "·" is the inner product modulo p.

Note that each [image] contains pd-1 distribution rules. We will suppose that each probability distribution [image] is equiprobable: for every [image]. The Brickell scheme is presented in Figure 11.6

We have the following result.

THEOREM 11.4

Suppose φ satisfies Property (11.3). Then the sets of distribution rules [image], comprise an ideal scheme that realizes Γ.

PROOF First, we will show that if B is an authorized subset, then the participants in B can compute K. Since

we can write

where each [image]. Denote by si the share given to Pi. Then

where [image] is an unknown vector chosen by D and

By the linearity of the inner product operation,

Thus, it is a simple matter for the participants in B to compute

What happens if B is not an authorized subset? Denote by e the dimension of the subspace 〈φ(Pi) : Pi ∈ B〉 (note that e ≤ |B|). Choose any [image], and consider the system of equations:

This is a system of linear equations in the d unknowns a1, . . . , ad. The coefficient matrix has rank e + 1, since

Provided the system of equations is consistent, the solution space has dimension d - e - 1 (independent of the value of K). It will then follow that there are precisely pd-e-1 distribution rules in each [image] that are consistent with any possible distribution of shares to B. By a similar computation as was performed in Example 11.5, we see that [image] for every [image], where fB(Pi) = si for all Pi ∈ B.

Why is the system consistent? The first |B| equations are consistent, since the vector [image] chosen by D is a solution. Since

(as mentioned above) the last equation is consistent with the first |B| equations. This completes the proof.

It is interesting to observe that the Shamir (t, w)-threshold scheme is a special case of the vector space construction. To see this, define d = t and let

for 1 ≤ i ≤ w, where xi is the x-coordinate given to Pi. The resulting scheme is equivalent to the Shamir scheme; we leave the details to the reader to check.

Here is another general result that is easy to prove. It concerns access structures that have as a basis a collection of pairs of participants that forms a complete multipartite graph. A graph G = (V, E) with vertex set V and edge set E is defined to be a complete multipartite graph if the vertex set V can be partitioned into subsets [image] such that {x, y} ∈ E if and only if x ∈ Vi, y ∈ Vj, where i ≠ j. The sets Vi are called parts. The complete multipartite graph is denoted by [image] if [image]. A complete multipartite graph K1, . . . , 1 (with [image] parts) is in fact a complete graph and is denoted [image].


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

THEOREM 11.5

Suppose G = (V, E) is a complete multipartite graph. Then there is an ideal scheme realizing the access structure cl (E) on participant set V.

PROOF Let [image] be the parts of G. Let [image] be distinct elements of [image], where [image]. Let d = 2. For every participant v ∈ Vi, define φ(v) = (xi, 1).

It is straightforward to verify Property (11.3). By Theorem 11.4, we have an ideal scheme.

To illustrate the application of these constructions, we will consider the possible access structures for up to four participants. Note that it suffices to consider only the access structures in which the basis cannot be partitioned into two non-empty subsets on disjoint participant sets. (For example, Γ0 = {{P1, P2}, {P3, P4}} can be partitioned as {{P1, P2}} ∪ {{P3, P4}} so we do not consider it.) We list the non-isomorphic access structures of this type on two, three, and four participants in Table 11.1 (the quantities ρ* are defined in Section 11.7).

Of these 18 access structures, we can already obtain ideal schemes for ten of them using the constructions we have at our disposal now. These ten access structures are either threshold access structures or have a basis which is a complete multipartite graph, so Theorem 11.5 can be applied. One such access structure is # 9, whose basis is the complete multipartite graph K1,1,2. We illustrate in the following example.

Example 11.7

For access structure # 9, take d = 2, p ≥ 3, and define φ as follows:

Table 11.1 Access structures for at most four participants
 | w | subsets in Γ0 | ρ* | comments
1. | 2 | P1P2 | 1 | (2, 2)-threshold
2. | 3 | P1P2, P2P3 | 1 | Γ0 ≅ K1,2
3. | 3 | P1P2, P2P3, P1P3 | 1 | (2, 3)-threshold
4. | 3 | P1P2P3 | 1 | (3, 3)-threshold
5. | 4 | P1P2, P2P3, P3P4 | 2/3 |
6. | 4 | P1P2, P1P3, P1P4 | 1 | Γ0 ≅ K1,3
7. | 4 | P1P2, P1P4, P2P3, P3P4 | 1 | Γ0 ≅ K2,2
8. | 4 | P1P2, P2P3, P2P4, P3P4 | 2/3 |
9. | 4 | P1P2, P1P3, P1P4, P2P3, P2P4 | 1 | Γ0 ≅ K1,1,2
10. | 4 | P1P2, P1P3, P1P4, P2P3, P2P4, P3P4 | 1 | (2, 4)-threshold
11. | 4 | P1P2P3, P1P4 | 1 |
12. | 4 | P1P 3P4, P1P2, P2P3 | 2/3 |
13. | 4 | P1P3P4,P1P2, P2P3, P2P4 | 2/3 |
14. | 4 | P1P2P3, P1P2P4 | 1 |
15. | 4 | P1P2P3, P1P2P4, P3P4 | 1 |
16. | 4 | P1P2P3, P1P2P4, P1P3P4 | 1 |
17. | 4 | P1P2P3, P1P2P4, P1P3P4, P2P3P4 | 1 | (3, 4)-threshold
18. | 4 | P1P2P3P4 | 1 | (4, 4)-threshold


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Applying Theorem 11.5, an ideal scheme results.

Eight access structures remain to be considered. It is possible to use ad hoc applications of the vector space construction to construct ideal schemes for four of these: # 11, # 14, # 15 and # 16. We present the constructions for # 11 and # 14 here.

Example 11.8

For access structure # 11, take d = 3, p ≥ 3, and define φ as follows:

First, we have

Also,

Hence,

and

Now, it suffices to show that

if B is a maximal unauthorized subset. There are three such subsets B to be considered: {P1, P2}, {P1, P3}, and {P2, P3, P4}. In each case, we need to establish that a system of linear equations has no solution. For example, suppose that

where [image]. This is equivalent to the system

The system is easily seen to have no solution. We leave the other two subsets B for the reader to consider.

Example 11.9

For access structure # 14, take d = 3, p ≥ 2 and define φ as follows:

Again, Property (11.3) is satisfied and hence an ideal scheme results.

Constructions of ideal schemes for the access structures # 15 and # 16 are left as exercises. In the next section, we will show that the remaining four access structures cannot be realized by ideal schemes.

11.7 An Upper Bound on the Information Rate

Four access structures remain to be considered: # 5, # 8, # 12, and # 13. We will see in this section that in each case, there does not exist a scheme having information rate ρ > 2/3.

Denote by ρ* = ρ* (Γ) the maximum information rate for any perfect secret sharing scheme realizing a specified access structure Γ. The first result we present is an entropy bound that will lead to an upper bound on ρ* for certain access structures. We have defined a probability distribution [image] on [image]; the entropy of this probability distribution is denoted H(K). We have also denoted by [image] the probability distribution on the shares given to a subset [image]. We will denote the entropy of this probability distribution by H(B).

We begin by giving yet another definition of perfect secret sharing schemes, this time using the language of entropy. This definition is equivalent to Definition 11.3.

DEFINITION 11.5 Suppose Γ is an access structure and [image] is a set of distribution rules. Then [image] is a perfect secret sharing scheme realizing the access structure Γ provided that the following two properties are satisfied:

1. For any authorized subset of participants
2. For any unauthorized subset of participants [image].

We will require several entropy identities and inequalities. Some of these results were given in Section 2.3 and the rest are proved similarly, so we state them without proof in the following Lemma.

LEMMA 11. 6

Let X, Y and Z be random variables. Then the following hold:

We next prove two preliminary entropy lemmas for secret sharing schemes.

LEMMA 11.7

Suppose Γ is an access structure and [image] is a set of distribution rules realizing Γ.

Suppose B ∉ Γ and [image], where [image]. Then

PROOF From Equations 11.5 and 11.6, we have that

and

so

Since, by Property 2 of Definition 11.5, we have

and, by Property 1 of Definition 11.5, we have

the result follows.

LEMMA 11.8

Suppose Γ is an access structure and [image] is a set of distribution rules realizing Γ. Suppose [image], where [image]. Then H (A|B) = H (A|BK).

PROOF As in Lemma 11.7, we have that

Since

and

the result follows.

We now prove the following important theorem.

THEOREM 11.9

Suppose Γ is an access structure such that

and

Let [image] be any perfect secret sharing scheme realizing Γ. Then H (XY) ≥ 3H (K).

PROOF We establish a sequence of inequalities:

H(K) | = | H(Y|WZ) - H(Y|WZK) | by Lemma 11.7
 | ≤ | H(Y|WZ) | by (11.7)
 | ≤ | H(Y|W) | by (11.8)
 | = | H(Y|WK) | by Lemma 11.8
 | ≤ | H(XY|WK) | by (11.9)
 | = | H(X|WK) + H(Y|WXK) | by (11.5)
 | ≤ | H(X|WK) + H(Y|XK) | by (11.8)
 | = | H(X|W) - H(K) + H(Y|X) - H(K) | by Lemma 11.7
 | ≤ | H(X) - H(K) + H(Y|X) - H(K) | by (11.7)
 | = | H(XY) - 2H(K) | by (11.4).

Hence, the result follows.

COROLLARY 11.10

Suppose that Γ is an access structure that satisfies the hypotheses of Theorem 11.9. Suppose the [image] keys are equally probable. Then ρ ≤ 2/3.

PROOF Since the keys are equiprobable, we have

Also, we have that

By Theorem 11.9, we have that

Hence it follows that

Now, by the definition of information rate, we have

and

It follows that

Hence, ρ ≤ 2/3.

For the access structures # 5, # 8, # 12, and # 13, the hypotheses of Theorem 11.9 are satisfied. Hence, ρ* ≤ 2/3 for these four access structures.

We also have the following result concerning ρ* in the case where the access structure has a basis Γ0 which is a graph. The proof involves showing that any connected graph which is not a multipartite graph contains an induced subgraph on four vertices that is isomorphic to the basis of access structure # 5 or # 8. If G = (V, E) is a graph with vertex set V and edge set E, and [image], then the induced subgraph G[V1] is defined to be the graph (V1, E1), where


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

THEOREM 11.11

Suppose G is a connected graph that is not a complete multipartite graph. Let Γ(G) be the access structure that is the closure of E, where E is the edge set of G. Then ρ*(Γ(G)) ≤ 2/3.

PROOF We will first prove that any connected graph that is not a complete multipartite graph must contain four vertices w, x, y, z such that the induced subgraph G[w, x, y, z] is isomorphic to either the basis of access structure # 5 or # 8.

Let GC denote the complement of G. Since G is not a complete multipartite graph, there must exist three vertices x, y, z such that xy, yz ∈ E(GC) and xz ∈ E(G). Define

where dG denotes the length of a shortest path (in G) between two vertices. Then d ≥ 2. Without loss of generality, we can assume that d = dG (y, x) by symmetry.

Let

be a path in G, where yo = y. We have that

and

It follows that G[yd-2, yd-1, x, z] is isomorphic to the basis of access structure # 5 or # 8, as desired.

So, we can assume that we have found four vertices w, x, y, z such that the induced subgraph G[w, x, y, z] is isomorphic to either the basis of access structure #5 or # 8. Now, let [image] be any scheme realizing the access structure Γ(G). If we restrict the domain of the distribution rules to {w, x, y, z}, then we obtain a scheme [image] realizing access structure # 5 or # 8. It is also obvious that [image]. Since [image], it follows that [image]. This completes the proof.

Since ρ* = 1 for complete multipartite graphs, Theorem 11.11 tells us that it is never the case that 2/3 < ρ* < 1 for any access structure that is the closure of the edge set of a connected graph.

11.8 The Decomposition Construction

We still have four access structures in Table 11.1 to consider. Of course, we can use the monotone circuit construction to produce schemes for these access structures. However, by this method, the best we can do is to obtain information rate ρ = 1/2 in each case. We can get ρ = 1/2 in cases # 5 and # 12 by using a disjunctive normal form boolean circuit. For cases # 8 and # 13, a disjunctive normal form boolean circuit will yield ρ = 1/3, but other monotone circuits exist which allow us to attain ρ = 1/2. But in fact, it is possible to construct schemes with ρ = 2/3 for each of these four access structures, by employing constructions that use ideal schemes as building blocks in the construction of larger schemes.

We present a construction of this type called the "decomposition construction." First, we need to define an important concept.

DEFINITION 11.6 Suppose Γ is an access structure having basis Γ0. Let [image] be a specified key set. An ideal [image] decomposition of Γ0 consists of a set {Γ1, . . . Γn} such that the following properties are satisfied:

1.
2.
3. for 1 ≤ k ≤ n, there exists an ideal scheme with key set [image], on the subset of participants

for the access structure having basis Γk.

Given an ideal [image] -decomposition of an access structure Γ, we can easily construct a perfect secret sharing scheme, as described in the following theorem.

THEOREM 11.12

Suppose Γ is an access structure having basis Γ0. Let [image] be a specified key set, and suppose {Γ1,...Γn} is an ideal [image] -decomposition of Γ. For every participant Pi, define

Then there exists a perfect secret sharing scheme realizing Γ, having information rate ρ 1/R, where

PROOF For 1 ≤ k ≤ n, we have an ideal scheme realizing the access structure with basis Γk, with key set [image], having [image] as its set of distribution rules. We will construct a scheme realizing Γ, with key set [image]. The set of distribution rules [image] is constructed according to the following recipe. Suppose D wants to share a key K. Then, for 1 ≤ k ≤ n, he chooses a random distribution rule [image] and distributes the resulting shares to the participants in [image].

We omit the proof that the scheme is perfect. However, it is easy to compute the information rate of the resulting scheme. Since each of the component schemes is ideal, it follows that

for 1 ≤ i ≤ w. So

and

which is what we were required to prove.

Although Theorem 11.12 is useful, it is often much more useful to employ a generalization in which we have [image] ideal [image] -decompositions of Γ0 instead of just one. Each of the [image] decompositions is used to share a key chosen from [image]. Thus, we build a scheme with key set [image]. The construction of the scheme and its information rate are as stated in the following theorem.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

THEOREM 11.13 (Decomposition Construction)

Suppose Γ is an access structure having basis Γ0, and [image] is an integer. Let [image] be a specified key set, and for 1 [image], suppose that [image] is an ideal decomposition of Γ0. Let [image] denote the participant set for the access structure Γj,k. For every participant Pi, define

Then there exists a perfect secret sharing scheme realizaing Γ, having information rate [image] where

PROOF For [image] and 1 ≤ k ≤ n, we have an ideal scheme realizing the access structure with basis Γj,k, with key set [image], having [image] as its set of distribution rules. We construct a scheme realizing Γ, with key set [image]. The set of distribution rules [image] is constructed according to the following recipe. Suppose D wants to share a key [image]. Then for [image] and 1 ≤ k ≤ n, he chooses a random distribution rule [image] and distributes the resulting shares to the participants in

The information rate can be computed in a manner similar to that of Theorem 11.12.

Let's look at a couple of examples.

Example 11.10

Consider access structure # 5. The basis is a graph that is not a complete multi-partite graph. Therefore we know from Theorem 11.11 that ρ* ≤ 2/3.

Let p be prime, and consider the following two ideal [image] -decompositions:

where

and

where

Each decomposition consists of a K2 and a K1,2, so they are indeed ideal [image] -decompositions. Either of them yields a scheme with ρ = 1/2. However, if we "combine" them by applying Theorem 11.13 with [image], then we get a scheme with ρ = 2/3, which is optimal.

One implementation of the scheme, using Theorem 11.5, is as follows. D will choose four random elements (independently) from [image], say b11, b12, b21, and b22. Given a key [image], D distributes shares as follows:

1. P1 receives b11, b21.
2. P2 receives b11 + K1, b12, b21 + K2.
3. P3 receives b12 + K1, b21, b22.
4. P4 receives b12, b22 + K2.

(All arithmetic is performed in [image].)

Example 11.11

Consider access structure # 8. Again, ρ* ≤ 2/3 by Theorem 11.11, and two suitable ideal compositions will yield an (optimal) scheme with ρ = 2/3.

Take [image] for any prime p ≥ 3, and define two ideal [image] -decompositions to be:

where

and

where

consists of a K2 and a K3, and [image] consists of a K2 and a K1,3, so both are ideal [image] -decompositions. Applying Theorem 11.13 with [image], we get a scheme with ρ = 2/3.

One implementation, using Theorem 11.5, is as follows. D will choose four random elements (independently) from [image], say b11, b12, b21, and b22. Given a key [image], D distributes shares as follows:

1. P1 receives b11 + K1, b21 + K2.
2. P2 receives b11, b12, b21.
3. P3 receives b12 + K1, b21 + K2, b22.
4. P4 receives b12 + 2K1, b21 + K2, b22 + K2.

(All arithmetic is performed in

To this point, we have explained all the information in Table 11.1 except for the values of ρ* for access structures # 12 and # 13. These values arise from a more general version of the decomposition construction which we do not describe here; see the notes below.

11.9 Notes and References

Threshold schemes were invented independently by Blakley [BL79] and Shamir [SH79]. Secret sharing for general access structures was first studied in Ito, Saito, and Nishizeki [ISN87]; we based Section 11.2 on the approach of Benaloh and Leichter [BL90]. The vector space construction is due to Brickell [BR89A]. The entropy bound of Section 11.7 is proved in Capocelli et al. [CDGV93], and some of the other material from this section is found in Blundo et al. [BDSV93].

In this chapter, we have emphasized a linear-algebraic and combinatorial approach to secret sharing. Some interesting connections with matroid theory can be found in Brickell and Davenport [BD91]. Secret sharing schemes can also be constructed using geometric techniques. Simmons has done considerable research in this direction; we refer to [SI92A] for an overview of geometric techniques in secret sharing. Further discussion of these topics, as well as constructions for schemes having information rate 2/3 for access structures # 12 and # 13, can be found in the expository paper by Stinson [ST92A].


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Exercises

11.1 Write a computer program to compute the key for a Shamir (t, w)-threshold scheme implemented in [image]. That is, given t public x-coordinates, x1, x2, . . . , xt, and t y-coordinates y1, . . . , yt, compute the resulting key. Use the Lagrange interpolation method, as it is easier to program.

(a) Test your program if p = 31847, t = 5 and w = 10, with the following

shares:

x1 = 413 y1 = 25439
x2 = 432 y2 = 14847
x3 = 451 y3 = 24780
x4 = 470 y4 = 5910
x5 = 489 y5 = 12734
x6 = 508 y1 = 12492
x7 = 527 y2 = 12555
x8 = 546 y3 = 128578
x9 = 565 y4 = 20806
x10 = 584 y5 = 21462

Verify that the same key is computed by using several different subsets of five shares.

(b) Having determined the key, compute the share that would be given to a participant with x-coordinate 10000. (Note that this can be done without computing the whole secret polynomial a(x).)

11.2 A dishonest dealer might distribute "bad" shares for a Shamir threshold scheme, i.e., shares for which different t-subsets determine different keys. Given all w shares, we could test the consistency of the shares by computing the key for every one of the [image] t-subsets of participants, and verifying that the same key is computed in each case. Can you describe a more efficient method of testing the consistency of the shares?
11.3 For access structures having the following bases, use the monotone circuit construction to construct a secret sharing scheme with information rate ρ = 1/3.

(a) Γ0 = {{P1, P2}, {P2, P3}, {P2, P4}, {P3, P4}}.
(b) Γ0 = {{P1, P3, P4}, {P1, P2}, {P2, P3},{P2, P4}}.
(c) Γ0 = {{P1, P2}, {P1, P3}, {P2, P3, P4}, {P2, P4, P5}, {P3, P4, P5}}.

11.4 Use the vector space construction to obtain ideal schemes for access structures having the following bases:

(a) Γ0 = {{P1, P2, P3}, {P1, P2, P4}, {P3, P4}}.
(b) Γ0 = {{P1, P2, P3}, {P1, P2, P4},{P1, P3, P4}}.
(c) Γ0 = {{P1, P2}, {P1, P3}, {P2, P3}, {P1, P4, P5}, {P2, P4, P5}}.

11.5 Use the decomposition construction to obtain schemes with specified information rates for access structures having the following bases:

(a) Γ0 = {{P1, P3, P4}, {P1, P2}, {P2, P3}}, ρ = 3/5.
(b) Γ0 = {{P1, P3, P4}, {P1, P2}, {P2, P3},{P2, P4}}, ρ = 4/7.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 12
Pseudo-random Number Generation

12.1 Introduction and Examples

There are many situations in cryptography where it is important to be able to generate random numbers, bit-strings, etc. For example, cryptographic keys are to be generated at random from a specified keyspace, and many protocols require random numbers to be generated during their execution. Generating random numbers by means of coin tosses or other physical processes is time-consuming and expensive, so in practice it is common to use a pseudo-random bit generator (or PRBG). A PRBG starts with a short random bit-string (a "seed") and expands it into a much longer "random-looking" bit-string. Thus a PRBG reduces the amount of random bits that are required in an application.

More formally, we have the following definition.

DEFINITION 12.1 Let k, [image] be positive integers such that [image] (where [image] is a specified polynomial function of k). A [image] - pseudo-random bit generator (more briefly, a [image] -PRBG) is a function [image] that can be computed in polynomial time (as a function of k). The input [image] is called the seed, and the output [image] is called a pseudo-random bit-string.

The function f is deterministic, so the bit-string f(s0) is dependent only on the seed. Our goal is that the pseudo-random bit-string f(s0) should "look like" truly random bits, given that the seed is chosen at random. Giving a precise definition is quite difficult, but we will try to give an intuitive description of the concept later in this chapter.

One motivating example for studying this type PRBG is as follows. Recall the concept of perfect secrecy that we studied in Chapter 2. One realization of perfect secrecy is the One-time Pad, where the plaintext and the key are both bitstrings of a specified length, and the ciphertext is constructed by taking the bitwise exclusive-or of the plaintext and the key. The practical difficulty of the One-time Pad is that the key, which must be randomly generated and communicated over a secure channel, must be as long as the plaintext in order to ensure perfect secrecy. PRBGs provide a possible way of alleviating this problem. Suppose Alice and Bob agree on a PRBG and communicate a seed over the secure channel. Alice and Bob can then both compute the same string of pseudo-random bits, which will be used as a One-time Pad. Thus the seed functions as a key, and the PBRG can be thought of as a keystream generator for a stream cipher.

*
* Figure 12.1 Linear Congruential Generator

We now present some well-known PRBGs to motivate and illustrate some of the concepts we will be studying. First, we observe that a linear feedback shift register, as described in Section 1.1.7, can be thought of as a PRBG. Given a k-bit seed, an LFSR of degree k can be used to produce as many as 2k - k - 1 further bits before repeating. The PRBG obtained from an LFSR is very insecure: we already observed in Section 1.2.5 that knowledge of any 2k consecutive bits suffice to allow the seed to be determined, and hence the entire sequence can be reconstructed by an opponent. (Although we have not yet defined security of a PRBG, it should be clear that the existence of an attack of this type means that the generator is insecure!)

Another well-known (but insecure) PRBG, called the Linear Congruential Generator, is presented in Figure 12.1. Here is a very small example to illustrate.

Example 12.1

We can obtain a (5, 10)-PRBG by taking M = 31, a = 3 and b = 5 in the Linear Congruential Generator. If we consider the mapping [image] mod 31, then [image], and the other 30 residues are permuted in a cycle of length 30, namely 0, 5, 20, 3, 14, 16, 22, 9, 1, 8, 29, 30, 2, 11, 7, 26, 21, 6, 23, 12, 10, 4, 17, 25, 18, 28, 27, 24, 15, 19. If the seed is anything other than 13, then the seed specifies a starting point in this cycle, and the next 10 elements, reduced modulo 2, form the pseudo-random sequence.

Table 12.1 Bit-strings produced by the linear congruential generator
seed | sequence
0 | 1010001101
1 | 0100110101
2 | 1101010001
3 | 0001101001
4 | 1100101101
5 | 0100011010
6 | 1000110010
7 | 0101000110
8 | 1001101010
9 | 1010011010
10 | 0110010110
11 | 1010100011
12 | 0011001011
13 | 1111111111
14 | 0011010011
15 | 1010100011
16 | 0110100110
17 | 1001011010
18 | 0101101010
19 | 0101000110
20 | 1000110100
21 | 0100011001
22 | 1101001101
23 | 0001100101
24 | 1101010001
25 | 0010110101
26 | 1010001100
27 | 0110101000
28 | 1011010100
29 | 0011010100
30 | 0110101000

The 31 possible pseudo-random bit-strings produced by this generator are illustrated in Table 12.1.

We can use some concepts developed in earlier chapters to consrtruct PRBGs. For example, the output feedback mode of DES, as described in Section 3.4.1, can be thought of as a PRBG; moreover, it appears to be computationally secure.

*
* Figure 12.2 RSA Generator


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Another approach in constructing very fast PRBGs is to combine LFSRs in some way that the output looks less linear. One such method, due to Coppersmith, Krawczyk and Mansour, is called the Shrinking Generator. Suppose we have two LFSRs, one of degree k1 and one of k2. We will require a total of k1 + k2 bits as our seed, in order to initialize both LFSRs. The first LFSR will produce a sequence of bits, say a1, a2, …, and the second produces a sequence of bits b1, b2, ….Then we define a sequence of pseudo-random bits z1, z2, … by the rule

where ik is the position of the kth 1 in the sequence b1, b2, …. These pseudo-random bits comprise a subsequence of the bits produced by the first LFSR. This method of pseudo-random bit generation is very fast and is resistent to various known attacks, but there does not seem to be any way to prove that it is secure.

In the rest of this chapter, we will investigate PRBGs that can be proved to be secure given some plausible computational assumption. There are PRBGs based on the fundamental problems of factoring (as it relates to the RSA public-key cryptosystem) and the Discrete Logarithm problem. A PRBG based on the RSA encryption function is shown in Figure 12.2, and a PRBG based on the Discrete Logarithm problem is discussed in the exercises.

We now give an example of the RSA Generator.

Table 12.2 Bits produced by RSA generator
i | si | zi
0 | 75634 |
1 | 31483 | 1
2 | 31238 | 0
3 | 51968 | 0
4 | 39796 | 0
5 | 28716 | 0
6 | 14089 | 1
7 | 5923 | 1
8 | 44891 | 1
9 | 62284 | 0
10 | 11889 | 1
11 | 43467 | 1
12 | 71215 | 1
13 | 10401 | 1
14 | 77444 | 0
15 | 56794 | 0
16 | 78147 | 1
17 | 72137 | 1
18 | 89592 | 0
19 | 29022 | 0
20 | 13356 | 0

Example 12.2

Suppose n = 91261 = 263 × 347, b = 1547, and s0 = 75364. The first 20 bits produced by the RSA Generator are computed as shown in Table 12.2. Hence the bit-string resulting from this seed is

10000111011110011000.

12.2 Indistinguishable Probability Distributions

There are two main objectives of a pseudo-random number generator: it should be fast (i.e., computable in polynomial time as a function of k) and it should be secure. Of course, these two requirements are often conflicting. The PRBGs based on linear congruences or linear feedback shift registers are indeed very fast. These PRBGs are quite useful in simulations, but they are very insecure for cryptographic applications.

Let us now try to make precise the idea of a PRBG being "secure." Intuitively, a string of km bits produced by a PRBG should look "random." That is, it should be impossible in an amount of time that is polynomial in k (equivalently, polynomial in [image]) to distinguish a string of [image] pseudo-random bits produced by a PRBG from a string of [image] truly random bits.

This motivates the idea of distinguishability of probability distributions. Here is a definition of this concept.

DEFINITION 12.2 Suppose p0 and p1 are two probability distributions on the set [image] of bit-strings of length [image]. Let [image] be a probabilistic algorithm that runs in polynomial time (as a function of [image]). Let ε > 0. For j = 0, 1, define

We say that A is an ε-distinguisher of p0 and p1 provided that

and we say that p0 and p1 are ε-distinguishable if there exists an ε-distinguisher of p0 and p1.

REMARK If A is a deterministic algorithm, then the conditional probabilities

always have the value of 0 or 1.

The intuition behind this definition is as follows. The algorithm A tries to decide if a bit-string [image] of length [image] is more likely to have arisen from probability distribution p1 or from probability distribution p0. This algorithm may use random numbers if desired, i.e., it can be probabilistic. The output [image] represents the algorithm's guess as to which of these two probability distributions is more likely to have produced, [image]. The quantity EA (pj) represents the average (i.e., expected) value of the output of A over the probability distribution pj, for j = 0, 1. This is computed by summing over all possible sequences [image] the product of the probability of the [image] -tuple [image] and the probability that A answers "1" when given [image] as input. A is an ε-distinguisher provided that the values of these two expectations are at least ε apart.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

The relevance to PRBGs is as follows. Consider the sequence of [image] bits produced by the PRBG. There are [image] possible sequences, and if the bits were chosen independently at random, each of these [image] sequences would occur with equal probability [image]. Thus a truly random sequence corresponds to an equiprobable distribution on the set of all bit-strings of length [image]. Suppose we denote this probability distribution by p0.

Now, consider sequences produced by the PRBG. Suppose a k-bit seed is chosen at random, and then the PRBG is used to obtain a bit-string of length [image]. Then we obtain a probability distribution on the set of all bit-strings of length [image], which we denote by p1. (For the purposes of illustration, suppose we make the simplifying assumption that no two seeds give rise to the same sequence of bits. Then, of the [image] possible sequences, 2k sequences each occur with probability 1/2k, and the remaining [image] sequences never occur. So, in this case, the probability distribution p1 is very non-uniform.)

Even though the two probability distributions p0 and p1 may be quite different, it is still conceivable that they might be ε-distinguishable only for small values of ε. This is our objective in constructing PRBGs.

Example 12.3

Suppose that a PRBG only produces sequences in which exactly [image] bits have the value 0 and [image] bits have the value 1. Define the function A by

In this case, the algorithm A is deterministic. It is not hard to see that

and

It can be shown that

Hence, for any fixed value of ε < 1, p0 and p1 are ε-distinguishable if [image] is sufficiently large.

12.2.1 Next Bit Predictors

Another useful concept in studying PRBGs is that of a next bit predictor, which works as follows. Let f be a [image] -PRBG. Suppose we have a probabilistic algorithm Bi, which takes as input the first i-1 bits produced by f (given an unknown seed), say z1, …, zi-1, and attempts to predict the next bit zi. The value i can be any value such that [image]. We say that Bi is an ε-next bit predictor if Bi can predict the ith bit of a pseudo-random sequence with probability at least 1/2 + ε, where ε > 0.

We can give a more precise formulation of this concept in terms of probability distributions, as follows. We have already defined the probability distribution p1 on [image] induced by the PRBG f. We can also look at the probability distributions induced by f on any of the [image] pseudo-random output bits (or indeed on any subset of these [image] output bits). So, for [image], we will can think of the ith pseudo-random output bit as a random variable that we will denote by zi.

In view of these definitions, we have the following characterization of a next bit predictor.

THEOREM 12.1

Let f be a [image] -PRBG. Then the probabilistic algorithm Bi is an ε-next bit predictor for f if and only if

PROOF The probability of correctly predicting the ith bit is computed by summing over all possible (i - 1)-tuples (z1, … , zi-1) the product of the probability that the (i - 1)-tuple, (z1,…, zi-1) is produced by the PRBG and the probability that the ith bit is predicted correctly given the (i - 1)-tuple, (z1, … , zi-1).

The reason for the expression 1/2 + ε in this definition is that any predicting algorithm can predict the next bit of a random sequence with probability 1/2. If a sequence is not random, then it may be possible to predict the next bit with higher probability. (Note that it is unnecessary to consider algorithms that predict the next bit with probability less than 1/2, because in this case an algorithm that replaces every prediction z by 1 - z will predict the next bit with probability greater than 1/2.)

We illustrate these ideas by producing a next-bit predictor for the Linear Congruential Generator of Example 12.1.

Example 12.1 (Cont.)

For any i such that 1 ≤ i ≤ 9, Define Bi(z) = 1 - z. That is, Bi predicts that a 0 is most likely to be followed by a 1, and vice versa. It is not hard to compute from Table 12.1 that each of these predictors Bi is a [image] -next bit predictor (i.e., they predict the next bit correctly with probability 20/31).

We can use a next bit predictor to construct a distinguishing algorithm A, as shown in Figure 12.3. The input to algorithm A is a sequence of bits, [image], and A calls the algorithm Bi as a subroutine.

*
* Figure 12.3 Constructing a distinguisher from a next bit predictor

THEOREM 12.2

Suppose Bi is an ε-next bit predictor for the [image] -PRBG f. Let p1 be the probability distribution induced on [image] by f, and let p0 be the uniform probability distribution on [image]. Then A, as described in Figure 12.3, is an ε-distinguisher of p1 and p0.

PROOF First, observe that

Also, the output of A is independent of the values of [image]. Thus we can compute as follows:


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

On the other hand, any predictor Bi will predict the ith bit of a truly random sequence with probability 1/2. Then, it is not difficult to see that EA (p0) = 1/2. Hence |EA (p0) - EA (p1)| ≥ ε, as desired.

One of the main results in the theory of pseudo-random bit generators, due to Yao, is that a next bit predictor is a universal test. That is, a PRBG is "secure" if and only if there does not exist an ε-next bit predictor except for very small values of ε. Theorem 12.2 proves the implication in one direction. To prove the converse, we need to show how the existence of a distinguisher implies the existence of a next bit predictor. This is done in Theorem 12.3.

THEOREM 12.3

Suppose A, is an ε-distinguisher of p1 and p0, where p1 is the probability distribution induced on [image] by the [image] -PRBG f, and p0 is the uniform probability distribution on [image]. Then for some i, [image], there exists an [image] -next bit predictor Bi for f.

PROOF For [image], define qi to be a probability distribution on [image] where the first i bits are generated using f, and the remaining [image] bits are generated at random. Thus q0 = p0 and [image]. We are given that

By the triangle inequality, we have that

Hence, it follows that there is at least one value [image], such that

Without loss of generality, we will assume that

We are going to construct an ith bit predictor (for this specified value of i). The predicting algorithm is probabilistic in nature and is presented in Figure 12.4. Here is the idea behind this construction. The predicting algorithm in fact produces an [image] -tuple according to the probability distribution qi - 1, given that z1,…,zi-1are generated by the PRBG. If A answers "0," then it thinks that the [image] -tuple was most likely generated according to the probability distribution qi. Now qi-1 and qi differ only in that the ith bit is generated at random in qi-1, whereas it is generated according to the PRBG in qi. Hence, when A answers "0," it thinks that the ith bit, zi, is what would be produced by the PRBG. Hence, in this case we take zi as our prediction of the ith bit. If A answers "1," it thinks that zi is random, so we take 1 - zi as our prediction of the ith bit.

We need to compute the probability that the ith bit is predicted correctly. Observe that if A answers "0," then the prediction is correct with probability

*
* Figure 12.4 Constructing a next bit predictor from a distinguisher

where p1 is the probability distribution induced by the PRBG. If A answers "1," then the prediction is correct with probability

For brevity, we denote [image]. In our computation, we will make use of the fact that

This can be proved easily as follows:

Now we can perform our main computation:

which was what we wanted to prove.

12.3 The Blum-Blum-Shub Generator

In this section we describe one of the most popular PRBGs, due to Blum, Blum, and Shub. First, we review some results on Jacobi symbols from Section 4.5 and other number-theoretic facts from other parts of Chapter 4.

Suppose p and q are two distinct primes, and let n = pq. Recall that the Jacobi symbol

Denote the quadratic residues modulo n by QR (n). That is,

Recall that x is a quadratic residue modulo n if and only if

Define

Thus

An element [image] is called a pseudo-square modulo n.

The Blum-Blum-Shub Generator, as well as some other cryptographic systems, is based on the Quadratic Residues problem defined in Figure 12.5. (In Chapter 4, we defined the Quadratic Residues problem modulo a prime and showed that it is easy to solve; here we have a composite modulus.) Observe that the Quadratic Residues problem requires us to distinguish quadratic residues modulo n from pseudo-squares modulo n. This can be no more difficult than factoring n. For if the factorization n = pq can be computed, then it is a simple matter to compute [image], say. Given that [image], it follows that x is a quadratic residue if and only if [image].

*
* Figure 12.5 Quadratic Residues

*
* Figure 12.6 Blum-Blum-Shub Generator

There does not appear to be any way to solve the Quadratic Residues problem efficiently if the factorization of n is not known. So this problem appears to be intractible if it is infeasible to factor n.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

The Blum-Blum-Shub Generator is presented in Figure 12.6. The generator works quite simply. Given a seed s0 ∈ QR(n), we compute the sequence [image] by successive squaring modulo n, and then reduce each si modulo 2 to obtain zi. It follows that

.

Table 12.3 Bits produced by BBS generator
i | si | zi
0 | 20749 |
1 | 143135 | 1
2 | 177671 | 1
3 | 97048 | 0
4 | 89992 | 0
5 | 174051 | 1
6 | 80649 | 1
7 | 45663 | 1
8 | 69442 | 0
9 | 186894 | 0
10 | 177046 | 0
11 | 137922 | 0
12 | 123175 | 1
13 | 8630 | 0
14 | 114386 | 0
15 | 14863 | 1
16 | 133015 | 1
17 | 106065 | 1
18 | 45870 | 0
19 | 137171 | 1
20 | 48060 | 0

We now give an example of the BBS Generator.

Example 12.4

Suppose n = 192649 = 383 × 503 and s0 = 1013552 mod n = 20749. The first 20 bits produced by the BBS Generator are computed as shown in Table 12.3. Hence the bit-string resulting from this seed is

11001110000100111010.

Here is a feature of the BBS Generator that is useful when we look at its security. Since n = pq where p ≡ q ≡ 3 mod 4, it follows that for any quadratic residue x, there is a unique square root of x that is also a quadratic residue. This square root is called the principal square root of x. It follows the mapping [image] mod n used to define the BBS Generator is a permutation on QR(n), the set of quadratic residues modulo n.

12.3.1 Security of the BBS Generator

In this section, we look at the security of the BBS Generator in detail. We begin by supposing that the pseudo-random bits produced by the BBS Generator are ε-distinguishable from [image] random bits and then see where that leads us. Throughout this section, n = pq, where p and q are primes such that p ≡ q ≡ 3 mod 4, and the factorization n = pq is unknown.

We have already discussed the idea of a next bit predictor. In this section we consider a similar concept that we call a previous bit predictor. A previous bit predictor for a [image] -BBS Generator will take as input [image] pseudo-random bits produced by the generator (as determined by an unknown random seed s0 ∈ QR(n)), and attempt to predict the value z0 = s0 mod 2. A previous bit predictor can be a probabilistic algorithm, and we say that a previous bit predictor B0 is an ε-previous bit predictor if its probability of correctly guessing z0 is at least 1/2 + ε, where this probability is computed over all possible seeds s0.

We state the following theorem, which is similar to Theorem 12.3, without proof.

THEOREM 12.4

Suppose A, is an ε-distinguisher of p1 and p0, where p1 is the probability distribution induced on [image] by the [image] -BBS Generator, f, and p0 is the uniform probability distribution on [image]. Then there exists an [image] -previous bit predictor B0 for f.

We now show how to use an [image] -previous bit predictor, B0, to construct a probabilistic algorithm that distinguishes quadratic residues modulo n from pseudo-squares modulo n with probability 1/2 + ε. This algorithm A, presented in Figure 12.7, uses B0 as a subroutine, or oracle.

THEOREM 12.5

Suppose B0 is an ε-previous bit predictor for the [image] -BBS Generator f. Then the algorithm A, as described in Figure 12.7, determines quadratic residuosity correctly with probability at least 1/2 + ε, where this probability is computed over all possible inputs [image].

PROOF Since n = pq and p ≡ q ≡ 3 mod 4, it follows that [image] so [image]. Hence, if [image] then the principal square root s0 = x2 is x if x ∈ QR(n); and -x if [image]. But

so it follows that algorithm A gives the correct answer if and only if B0 correctly predicts z. The result then follows immediately.

*
* Figure 12.7 Constructing a quadratic residue distinguisher from a previous bit predictor

Theorem 12.5 shows how we can distinguish pseudo-squares from quadratic residues with probability at least 1/2 + ε. We now show that this leads to a Monte Carlo algorithm that gives the correct answer with probability at least 1/2 + ε. In other words, for any [image], the Monte Carlo algorithm gives the correct answer with probabilty at least 1/2 + ε. Note that this algorithm is an unbiased algorithm (it may give an incorrect answer for any input) in contrast to the Monte Carlo algorithms that we studied in Section 4.5 which were all biased algorithms.

The Monte Carlo algorithm A1 is presented in Figure 12.8. It calls the previous algorithm A as a subroutine.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

THEOREM 12.6

Suppose that algorithm A determines quadratic residuosity correctly with probability at least 1/2 + ε. Then the algorithm A1, as described in Figure 12.8, is a Monte Carlo algorithm for Quadratic Residues with error probability at most 1/2 + ε.

PROOF For any given input [image], the effect of step 2 in algorithm A1 is to produce an element x′ that is a random element of [image] whose status as a quadratic residue is known.

The last step is to show that any (unbiased) Monte Carlo algorithm that has error probability at most 1/2 + ε can be used to construct an unbiased Monte Carlo algorithm with error probability at most δ, for any δ > 0. In other words, we can make the probability of correctness arbitrarily close to 1. The idea is to run the given Monte Carlo algorithm 2m + 1 times, for some integer m, and take the "majority vote" as the answer. By computing the error probability of this algorithm, we can also see how m depends on δ. This dependence is stated in the following theorem.

*
* Figure 12.8 A Monte Carlo algorithm for Quadratic Residues

THEOREM 12.7

Suppose A1 is an unbiased Monte Carlo algorithm with error probability at most 1/2 + ε. Suppose we run A1 n = 2m + 1 times on a given instance I, and we take the most frequent answer. Then the error probability of the resulting algorithm is at most

PROOF The probability of obtaining exactly i correct answers in the n trials is at most

The probability that the most frequent answer is incorrect is equal to the probability that the number of correct answers in the n trials is at most m. Hence, we compute as follows

as required.

Suppose we want to lower the probability of error to some value δ, where 0 < δ < 1/2 - ε. We need to choose m so that

Hence, it suffices to take

Then, if algorithm A is run 2m + 1 times, the majority vote yields the correct answer with probability at least 1 - δ. It is not hard to show that this value of m is at most c/(δε2) for some constant c. Hence, the number of times that the algorithm must be run is polynomial in 1/δ and 1/ε.

Example 12.5

Suppose we start with a Monte Carlo algorithm that returns the correct answer with probability at least .55, so ε = .05. If we desire a Monte Carlo algorithm in which the probability of error is at most .05, then it suffices to take m = 230 and n = 461.

Let us combine all the reductions we have done. We have the following sequence of implications:

*

Since it is widely believed that there is no polynomial-time Monte Carlo algorithm for Quadratic Residues with small error probability, we have some evidence that the BBS Generator is secure.

We close this section by mentioning a way of improving the efficiency of the BBS Generator. The sequence of pseudo-random bits is constructed by taking the least significant bit of each si, where [image] mod n. Suppose instead that we extract the m least significant bits from each si. This will improve the efficiency of the PRBG by a factor of m, but we need to ask if the PRBG will remain secure. It has been shown that this approach will remain secure provided that m ≤ log2 log2 n. So we can extract about log2 log2 n pseudo-random bits per modular squaring. In a realistic implementation of the BBS Generator, [image], so we can extract nine bits per squaring.

12.4 Probabilistic Encryption

Probabilistic encryption is an idea of Goldwasser and Micali. One motivation is as follows. Suppose we have a public-key cryptosystem, and we wish to encrypt a single bit, i.e., x = 0 or 1. Since anyone can compute eK (0) and eK (1), it is a simple matter for an opponent to determine if a ciphertext y is an encryption of 0 or an encryption of 1. More generally, an opponent can always determine if the plaintext has a specified value by encrypting a hypothesized plaintext, hoping to match a given ciphertext.

The goal of probabilistic encryption is that "no information" about the plaintext should be computable from the ciphertext (in polynomial time). This objective can be realized by a public-key cryptosystem in which encryption is probabilistic rather than deterministic. Since there are "many" possible encryptions of each plaintext, it is not feasible to test whether a given ciphertext is an encryption of a particular plaintext.

Here is a formal mathematical definition of this concept.

DEFINITION 12.3 A probabilistic public-key cryptosystem is defined to be a six-tuple [image] where [image] is the set of plaintexts, [image] is the set of ciphertexts, [image] is the keyspace, [image] is a set of randomizers, and for each key [image], [image] is a public encryption rule and [image] is a secret decryption role. The following properties should be satisfied:

1. Each [image] and [image] are functions such that

for every plaintext [image] and every [image]. (In particular, this implies that [image].)

2. Let ε be a specified security parameter. For any fixed [image] and for any [image], define a probability distribution pK,x on [image], where pK,x(y) denotes the probability that y is the ciphertext given that K is the key and x is the plaintext (this probability is computed over all [image] ). Suppose [image], x ≠ x′, and [image]. Then the probability distributions pK,x and pK,x′ are not ε-distinguishable.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Here is how the system works. To encrypt a plaintext x, choose a randomizer [image] and compute y = eK(x,r). Any such value y = eK(x,r) can be decrypted to x. Property 2 is stating that the probability distribution of all encryptions of x cannot be distinguished from the probability distribution of all encryptions of x′ if x′ ≠ x. Informally, an encryption of x "looks like" an encryption of x′. The security parameter ε should be small: in practice we would want to have [image] for some small c > 0.

We now present the Goldwasser-Micali Probabilistic Public-key Cryptosystem in Figure 12.9. This system encrypts one bit at a time. A 0 bit is encrypted to a random quadratic residue modulo n; a 1 bit is encrypted to a random pseudo-square modulo n. When Bob recieves an element [image] he can use his knowledge of the factorization of n to determine whether y ∈ QR(n) or whether [image]. He does this by computing

then

*
* Figure 12.9 Goldwasser-Micali Probabilistic Public-key Cryptosystem

A more efficient probabilistic public-key cryptosystem was given by Blum and Goldwasser. The Blum-Goldwasser Probabilistic Public-key Cryptosystem is presented in Figure 12.10. The basic idea is as follows. A random seed s0 generates a sequence of [image] psuedorandom bits [image] using the BBS Generator. The zi's are used as a keystream, i.e., the are exclusive-ored with the [image] plaintext bits to form the ciphertext. As well, the [image] element [image] mod n is transmitted as part of the ciphertext.

When Bob receives the ciphertext, he can compute s0 from [image], then reconstruct the keystream, and finally exclusive-or the keystream with the [image] ciphertext bits to obtain the plaintext. We should explain how Bob derives s0 from [image]. Recall that each si-1 is the principal square root of si. Now, n = pq with p ≡ q ≡ 3 mod 4, so the square roots of any quadratic residue x modulo p are ±x(p+1)/4. Using properties of Jacobi symbols, we have that

It follows that x(p+1)/4 is the principal square root of x modulo p. Similarly, x(q+1)/4 is the principal square root of x modulo q. Then, using the Chinese remainder theorem, we can find the principal square root of x modulo n.

*
* Figure 12.10 Blum-Goldwasser Probabilistic Public-key Cryptosystem

More generally, [image] will be the principal [image] root of x modulo p and [image] will be the principal [image] root of x modulo q. Since [image] has order p - 1, we can reduce the exponent [image] modulo p - 1 in the computation [image] mod p. In a similar fashion, we can reduce the exponent [image] modulo q - 1. In Figure 12.10, having obtained the principal [image] roots of [image] modulo p and modulo q (steps 1-4 of the decryption process), the Chinese remainder theorem is used to compute the principal [image] root of [image] modulo n.

Here is an example to illustrate.

Example 12.6

Suppose n = 192649, as in Example 12.4. Suppose further that Alice chooses r = 20749 and wants to encrypt the 20-bit plaintext string

She will first compute the keystream

exactly as in Example 12.4, and then exclusive-or it with the plaintext, to obtain the ciphertext

which she transmits to Bob. She also computes

and sends it to Bob.

Of course Bob knows the factorization n = 383 × 503, so (p + 1)/4 = 96 and (q + 1)/4 = 126. He begins by computing

and

Next, he calculates

and

Now Bob proceeds to solve the system of congruences

to obtain Alice's seed r = 20749. Then he constructs Alice's keystream from r. Finally, he exclusive-ors the keystream with the ciphertext to get the plaintext.

12.5 Notes and References

A lengthy treatment of PRBGs can be found in the book by Kranakis [KR86]. See also the survey paper by Lagarias [LA90].

The Shrinking Generator is due to Coppersmith, Krawczyk, and Mansour [CKM94]; another practical method of constructing PBRGs using LFSRs has been given by Gunther [GU88]. For methods of breaking the Linear Congruential Generator, see Boyar [BO89].

The basic theory of secure PRBGs is due to Yao [YA82], who proved the universality of the next bit test. Further basic results can be found in Blum and Micali [BM84]. The BBS Generator is described in [BBS86]. The security of the Quadratic Residues problem is studied by Goldwasser and Micali [GM84], on which we based much of Section 12.3.1. We have, however, used the approach of Brassard and Bratley [BB88A, Section 8.6] to reduce the error probability of an unbiased Monte Carlo algorithm.

Properties of the RSA Generator are studied in Alexi, Chor, Goldreich, and Schnorr [ACGS88]. PRBGs based on the Discrete Logarithm problem are treated in Blum and Micali [BM84], Long and Wigderson [LW88], and Håstad, Schrift, and Shamir [HSS93]. A sufficient condition for the secure extraction of multiple bits per iteration of a PRBG was proved by Vazirani and Vazirani [VV84].

*
* Figure 12.11 Discrete Logarithm Generator

The idea of probabilistic encryption is due to Goldwasser and Micali [GM84]; the Blum-Goldwasser Cryptosystem is presented in [BG85].


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Exercises

12.1 Consider the Linear Congruential Generator defined by si = (asi-1 + b) mod M. Suppose that M = qa + 1 where a is odd and q is even, and suppose that b = 1. Show that the next bit predictor Bi(z) = 1 - z for the i bit is an ε-next bit predictor, where

12.2 Suppose we have an RSA Generator with n = 36863, b = 229 and seed s0 = 25. Compute the first 100 bits produced by this generator.
12.3 A PRBG based on the Discrete Logarithm problem is given in Figure 12.11. Suppose p = 21383, the primitive element α = 5 and the seed s0 = 15886. Compute the first 100 bits produced by this generator.
12.4 Suppose that Bob has knowledge of the factorization n = pq in the BBS Generator.

(a) Show how Bob can use this knowledge to compute any si from so with 2k multiplications modulo φ(n) and 2k multiplications modulo n, where n has k bits in its binary representation. (If i is large compared to k, then this approach represents a substantial improvement over the i multiplications required to sequentially compute s0, …, si.)
(b) Use this method to compute s10000 if n = 59701 = 227 × 263 and s0 = 17995.

Table 12.4 Blum-Goldwasser Ciphertext
E1866663F17FDBD1DC8C8FD2EEBC36AD7F53795DBA3C9CE22D
C9A9C7E2A56455501399CA6B98AED22C346A529A09C1936C61
ECDE10B43D226EC683A669929F2FFB912BFA96A8302188C083
46119E4F61AD8D0829BD1CDE1E37DBA9BCE65F40C0BCE48A80
0B3D087D76ECD1805C65D9DB730B8D0943266D942CF04D7D4D
76BFA891FA21BE76F767F1D5DCC7E3F1D86E39A9348B3
12.5 We proved that, in order to reduce the error probability of an unbiased Monte Carlo algorithm from 1/2 - ε to δ, where δ + ε < 1/2, it suffices to run the algorithm m times, where

Prove that this value of m is O(1/(δε2).

12.6 Suppose Bob receives some ciphertext which was encrypted with the Blum-Goldwasser Probabilistic Public-key Cryptosystem. The original plaintext consisted of English text. Each alphabetic character was converted to a bitstring of length five in the obvious way: A ↔ 00000, B ↔ 00001, …, Z ↔ 11001. The plaintext consisted of 236 alphabetic characters, so a bitstring of length 1180 resulted. This bitstring was then encrypted. The resulting ciphertext bitstring was then converted to a hexadecimal representation, to save space. The final string of 295 hexadecimal characters is presented in Table 12.4. Also, s1181 = 20291 is part of the ciphertext, and n = 29893 is Bob's public key. Bob's secret factorization of n is n = pq, where p = 167 and q = 179.

Your task is to decrypt the given ciphertext and restore the original English plaintext, which was taken from "Under the Hammer," by John Mortimer, Penguin Books, 1994.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Chapter 13
Zero-knowledge Proofs

13.1 Interactive Proof Systems

Very informally, a zero-knowledge proof system allows one person to convince another person of some fact without revealing any information about the proof. We first discuss the idea of an interactive proof system. In an interactive proof system, there are two participants, Peggy and Vic. Peggy is the prover and Vic is the verifier. Peggy knows some fact, and she wishes to prove to Vic that she does.

It is necessary to describe the kinds of computations that Peggy and Vic will be allowed to perform, and also to describe the interaction that takes place. It is convenient to think of both Peggy and Vic as being probabilistic algorithms. Peggy and Vic will each perform private computations, and each of them has a private random number generator. They will communicate to each other through a communication channel. Initially, Peggy and Vic both possess an input x. The object of the interactive proof is for Peggy to convince Vic that x has some specified property. More precisely, x will be a yes-instance of a specified decision problem II.

The interactive proof, which is a challenge-and-response protocol, consists of a specified number of rounds. During each round, Peggy and Vic alternately do the following:

1. receive a message from the other party
2. perform a private computation
3. send a message to the other party.

A typical round of the protocol will consist of a challenge by Vic, and a response by Peggy. At the end of the proof, Vic either accepts or rejects, depending on whether or not Peggy successfully replies to all of Vic's challenges. We define the protocol to be an interactive proof system for the decision problem II if the following two properties are satisfied whenever Vic follows the protocol:

*
* Figure 13.1 Graph Isomorphism

completeness
If x is a yes-instance of the decision problem II, then Vic will always accept Peggy's proof.
soundness
If x is a no-instance of II, then the probability that Vic accepts the proof is very small.

We will restrict our attention to interactive proof systems in which the computations performed by Vic can be done in polynomial time. On the other hand, we do not place any bound on the computation time required by Peggy (informally, Peggy is "all-powerful").

We begin by presenting an interactive proof system for the problem of Graph Non-isomorphism. The Graph Isomorphism problem is described in Figure 13.1. This is an interesting problem since no polynomial-time algorithm to solve it is known, but it is not known to be NP-complete.

We will present an interactive proof system which will allow Peggy to "prove" to Vic that two specified graphs are not isomorphic. For simplicity, let us suppose that G1 and G2 each have vertex set {1, …, n}. The interactive proof system for Graph Non-isomorphism is presented in Figure 13.2.

We present a toy example.

Example 13.1

Suppose G1 = (V, E1) and G2 = (V, E2), where V = {1, 2, 3, 4}, E1 = {12, 14, 23, 34} and E2 = {12, 13, 14, 34}.

Suppose in some round of the protocol that Vic gives Peggy the graph H = (V, E3), where E3 = {13, 14, 23, 24} (see Figure 13.3). The graph H is isomorphic to G1 (one isomorphism from H to G1 is the permutation (1 3 4 2)). So Peggy answers "1."

It is easy to see that this proof system satisfies the completeness and soundness properties. If G1 is not isomorphic to G2, then j will equal i in every round, and Vic will accept with probability 1. Hence, the protocol is complete.

*
* Figure 13.2 An interactive proof system for Graph Non-isomorphism

*
* Figure 13.3 Peggy's non-isomorphic graphs and Vic's challenge

On the other hand, suppose that G1 is isomorphic to G2. Then any challenge graph H submitted by Vic is isomorphic to both G1 and G2. Peggy has no way of determining if Vic constructed H as an isomorphic copy of G1 or of G2, so she can do no better than make a guess j = 1 or 2 for her response. The only way that Vic will accept is if Peggy is able to guess all n choices of i made by Vic. Her probability of doing this is 2-n. Hence, the protocol is sound.

Notice that Vic's computations are all polynomial-time. We cannot say anything about Peggy's computation time since the Graph Isomorphism problem is not known to be solvable in polynomial time. However, recall that we assumed that Peggy has infinite computing power, so this is allowed under the "rules of the game."


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

13.2 Perfect Zero-knowledge Proofs

Although interactive proof systems are of interest in their own right, the most interesting type of interactive proof is a zero-knowledge proof. This is one in which Peggy convinces Vic that x possesses some specified property, but at the end of the protocol, Vic still has no idea of how to prove (himself) that x has this property. This is a very tricky concept to define formally, and we present an example before attempting any definitions.

In Figure 13.4, we present a zero-knowledge interactive proof for Graph Isomorphism. A small example will illustrate the workings of the protocol.

Example 13.2

Suppose G1 = (V, E1) and G2 = (V, E2), where V = {1, 2, 3, 4}, E1 = {12, 13, 14, 34} and E2 = {12, 13, 23, 24}. One isomorphism from G2 to G1 is the permutation σ = (4 1 3 2).

Now suppose in some round of the protocol that Peggy chooses the permutation π = (2 4 1 3). Then H has edge set {12, 13, 23, 24} (see Figure 13.5).

If Vic's challenge is i = 1, then Peggy gives Vic the permutation π and Vic checks that the image of G1 under π is H. If Vic's challenge is i = 2, then Peggy gives Vic the composition ρ = π [image] σ = (3 2 1 4) and Vic checks that the image of G2 under ρ is H.

Completeness and soundness of the protocol are easy to verify. It is easy to see that the probablity that Vic accepts is 1 if G1 is isomorphic to G2. On the other hand, if G1 is not isomorphic to G2, then the only way for Peggy to deceive Vic is for her to correctly guess the value i that Vic will choose in each round, and write a (random) isomorphic copy of Gi on the communication tape. Her probability of correctly guessing Vic's n random challenges is 2-n.

*
* Figure 13.4 A perfect zero-knowledge interactive proof system for Graph Isomorphism

*
* Figure 13.5 Peggy's isomorphic graphs

All of Vic's computations can be done in polynomial time (as a function of n, the number of vertices in G1 and G2). Although it is not necessary, notice that Peggy's computations can also be done in polynomial time provided that she knows the existence of one permutation a such that the image of G2 under σ is G1.

Why would we refer to this proof system as a zero-knowledge proof? The reason is that, although Vic is convinced that G1 is isomorphic to G2, he does not gain any "knowledge" that would help him find a permutation σ that carries G2 to G1. All he sees in each round of the proof is a random isomorphic copy H of the graphs G1 and G2, together with a permutation that carries G1 to H or G2 to H (but not both!). But Vic can compute random isomorphic copies of these graphs by himself, without any help from Peggy. Since the graphs H are chosen independently and at random in each round of the proof, it seems unlikely that this will help Vic find an isomorphism from G1 to G2.

Let us look carefully at the information that Vic obtains by participating in the interactive proof system. We can represent Vic's view of the interactive proof by means of a transcript that contains the following information:

1. the graphs G1 and G2
2. all the messages that are transmitted by both Peggy and Vic
3. the random numbers used by Vic to generate his challenges.

Hence, a transcript T for the above interactive proof of Graph Isomorphism would have the following form:

The essential point, which is the basis for the formal definition of zero-knowledge proof, is that Vic (or anyone else) can forge transcripts — without participating in the interactive proof — that "look like" real transcripts. This can be done provided that the input graphs G1 and G2 are isomorphic. Forging is accomplished by means of the algorithm presented in Figure 13.6. The forging algorithm is a polynomial-time probabilistic algorithm. In the vernacular of zero-knowledge proofs, a forging algorithm is often called a simulator.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

The fact that a simulator can forge transcripts has a very important consequence. Anything that Vic (or anyone else) can compute from the transcript could also be computed from a forged transcript. Hence, participating in the proof system does not increase Vic's ability to perform any computation; and in particular, it does not enable Vic himself to "prove" that G1 and G2 are isomorphic. Moreover, Vic cannot subsequently convince someone else that G1 and G2 are isomorphic by showing them the transcript T, since there is no way to distinguish a legitimate transcript from one that has been forged.

We still have to make precise the idea that a forged transcript "looks like" a real one. We give a rigorous definition in terms of probability distributions.

*
* Figure 13.6 Forging algorithm for transcripts for Graph Isomorphism

DEFINITION 13.1 Suppose that we have a polynomial-time interactive proof system for a decision problem II, and a polynomial-time simulator S. Denote the set of all possible transcripts that could be produced as a result of Peggy and Vic carrying out the interactive proof with a yes-instance x by [image], and denote the the set of all possible forged transcripts that could be produced by S by [image]. For any transcript [image], let [image] denote the probability that T is the transcript produced from the interactive proof. Similarly, for [image], let [image] denote the probability that T is the (forged) transcript produced by S. Suppose that [image], and for any [image], suppose that [image]. (In other words, the set of real transcripts is identical to the set of forged transcripts, and the two probability distributions are identical.) Then we define the interactive proof system to be perfect zero-knowledge for Vic.

Of course we can define zero-knowledge however we like. But it is important that the definition captures our intuitive concept of what "zero-knowledge" should mean. We are saying that an interactive proof system is zero-knowlege for Vic if there exists a simulator that produces transcripts with an identical probability distribution to those produced when Vic actually takes part in the protocol. (This is a related but stronger concept than that of indistinguishable probability distributions that we studied in Chapter 12.) We have observed that a transcript contains all the information gained by Vic by taking part in the protocol. So it should seem reasonable to say that whatever Vic might be able to do after taking part in the protocol he could equally well do by just using the simulator to generate a forged transcript. We are perhaps not defining "knowledge" by this approach; but whatever "knowledge" might be, Vic doesn't gain any!

We will now prove that the interactive proof system for Graph Isomorphism is perfect zero-knowledge for Vic.

THEOREM 13.1

The interactive proof system for Graph Isomorphism is perfect zero-knowledge for Vic.

PROOF Suppose that G1 and G2 are isomorphic graphs on n vertices. A transcript T (real or forged) contains n triples of the form (H, i, p), where i = 1 or 2, ρ is a permutation of {1, …, n}, and H is the image of Gi under the permutation ρ. Call such a triple a valid triple and denote by [image] the set of all valid triples. We begin by computing [image], the number of valid triples. Evidently [image] since each choice of i and p determines a unique graph H.

In any given round, say j, of the forging algorithm, it is clear that each valid triple (H, i, p) occurs with equal probability 1/(2 × n!). What is the probability that the valid triple (H, i, p) is the jth triple on a real transcript? In the interactive proof system, Peggy first chooses a random permutation π and then computes H to be the image of G1 under π. The permutation ρ is defined to be π if i = 1, and it is defined to be the composition of the two permutations π and ρ if i = 2.

We are assuming that the value of i is chosen at random by Vic. If i = 1, then all n! permutations ρ are equiprobable, since ρ = π in this case and π was chosen to be a random permutation. On the other hand, if i = 2, then p = π [image] ρ, where π is random and σ is fixed. In this case as well, every possible permutation ρ is equally probable. Now, since the two cases i = 1 and 2 are equally probable, and each permutation ρ is equally probable (independent of the value of i), and since i and p together determine H, it follows that all triples in [image] are equally likely.

Since a transcript consists of the concatenation of n independent random triples, it follows that

for every possible transcript T.

The proof of Theorem 13.1 assumes that Vic follows the protocol when he takes part in the interactive proof system. The situation is much more subtle if Vic does not follow the protocol. Is it true that an interactive proof remains zero-knowledge even if Vic deviates from the protocol?

In the case of Graph Isomorphism, the only way that Vic can deviate from the protocol is to choose his challenges i in a non-random way. Intuitively, it seems that this does not provide Vic with any "knowledge." However, transcripts produced by the simulator will not "look like" transcripts produced by Vic if he deviates from the protocol. For example, suppose Vic chooses i = 1 in every round of the proof. Then a transcript of the interactive proof will have ij = 1 for 1 ≤ j ≤ n; whereas a transcript produced by the simulator will have ij = 1 for 1 ≤ j ≤ n only with probability 2-n.

The way around this difficulty is to show that, no matter how a "cheating" Vic deviates from the protocol, there exists a polynomial-time simulator that will produce forged transcripts that "look like" the transcripts produced by Peggy and (the cheating) Vic during the interactive proof. As before, the phrase "looks like" is formalized by saying that two probability distributions are identical.

Here is a more formal definition.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

DEFINITION 13.2 Suppose that we have a polynomial-time interactive proof system for a given decision problem II. Let V* be any polynomial-time probabilistic algorithm that (a possibly cheating) verifier uses to generate his challenges. (That is, V* represents either an honest or cheating verifier.) Denote the set of all possible transcripts that could be produced as a result of Peggy and V* carrying out the interactive proof with a yes-instance x of II by [image]. Suppose that, for every such V*, there exists an expected polynomial-time probabilistic algorithm S* = S*(V*) (the simulator) which will produce a forged transcript. Denote the set of possible forged transcripts by [image]. For any transcript [image], let [image] denote the probability that T is the transcript produced by V* taking part in the interactive proof. Similarly, for [image], let [image] denote the probability that T is the (forged) transcript produced by S*. Suppose that [image], and for any [image], suppose that [image]. Then the interactive proof system is said to be perfect zero-knowledge (without qualification).

In the special case where V* is the same as Vic (i.e., when Vic is honest), the above definition is exactly the same as what we defined as "perfect zero-knowledge for Vic."

In order to prove that a proof system is perfect zero-knowledge, we need a generic transformation which will construct a simulator S* from any V*. We proceed to do this for the proof system for Graph Isomorphism. The simulator will play the part of Peggy, using V* as a "restartable subroutine." Informally, S* tries to guess the challenge ij that V* will make in each round j. That is, S* generates a random valid triple of the form (Hj, ij, ρj), and then executes the algorithm V* to see what its challenge is for round j. If the guess ij is the same as the challenge i′j (as produced by V*), then the triple (Hj, ij, ρj) is appended to the forged transcript. If not, then this triple is discarded, S* guesses a new challenge ij, and the algorithm V* is restarted after resetting its "state" to the way it was at the beginning of the current round. By the term "state" we mean the values of all variables used by the algorithm.

We now give a more detailed description of the simulation algorithm S*. At any given time during the execution of the program V*, the current state of V* will be denoted by state(V*). A pseudo-code description of the simulation algorithm is given in Figure 13.7.

It is possible that the simulator will run forever, if it never happens that ij = i′j. However, we can show that the average running time of the simulator is polynomial, and that the two probability distributions [image] and [image] are identical.

*
* Figure 13.7 Forging algorithm for V* for transcripts for Graph Isomorphism

THEOREM 13.2

The interactive proof system for Graph Isomorphism is perfect zero-knowledge.

PROOF First, we observe that, regardless of how V* generates its challenges, the probability that the guess ij of S* is the same as the challenge i′j is 1/2. Hence, on average, S* will generate two triples for every triple that it concatenates to the forged transcript. Hence, the average running time is polynomial in n.

The more difficult task is to show that the two probability distributions [image] and [image] are identical. In Theorem 13.1, where Vic was honest, we were able to compute the two probability distributions and see that they were identical. We also used the fact that triples (H, i, ρ) generated in different rounds of the proof are independent. However, in the current setting, we have no way of explicitly computing the two probability distributions. Further, triples generated in different rounds of the proof need not be independent. For example, the challenge that V* presents in round j may depend in some very complicated way on challenges from previous rounds and on the way Peggy replied to those challenges.

The way to handle these difficulties is to look at the probability distributions on the possible partial transcripts during the course of the simulation or interactive proof, and proceed by induction on the number of rounds. For 0 ≤ j ≤ n, we define probability distributions [image] and [image] on the set of partial transcripts [image] that could occur at the end of round j. Notice that [image] and [image]. Hence, if we can show that the two distributions [image] and [image] are identical for all j, then we will be done.

The case j = 0 corresponds to the beginning of the algorithm; at this point the transcript contains only the two graphs G1 and G2. Hence, the probability distributions are identical when j = 0. We use this for the start of the induction.

We make an inductive hypothesis that the two probability distributions [image] and [image] on [image] are identical, for some j ≥ 1. We now prove that the two probability distributions [image] and [image] on [image] are identical.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Consider what happens during round j of the interactive proof. The probability that V*'s challenge i′j = 1 is some real number p1 and the probability that his challenge i′j = 2 is 1 - p1, where p1 depends on the state of the algorithm V* at the beginning of round j. We noted earlier that in the interactive proof, all possible graphs H are chosen by Peggy with equal probability. As well, any permutation ρ occurs with equal probability, independent of the value of p1, since all permutations are equally likely for either possible challenge i′j. Hence, the probability that the jth triple on the transcript is (H, i, ρ) is p1/n! if i = 1, and (1 - p1)/n! if i = 2.

Next, let's do a similar analysis for the simulation. In any given iteration of the repeat loop, S* will choose any graph H with probability 1/n!. The probability that ij = 1 and V*'s challenge is 1 is p1/2; and the probability that ij = 2 and V*'s challenge is 2 is (1 - p1)/2. In each of these situations, (H, ij, ρ) is written as the jth triple on the transcript. With probability 1/2, nothing is written on the tape during any given iteration of the repeat loop.

Let us first consider the case ij = 1. As mentioned above, the probability that V*'s challenge is 1 is p1. The probability that a triple (H, 1, ρ) is written as the jth triple on the transcript during the [image] iteration of the repeat loop is

Hence, the probability that (H, 1, ρ) is the jth triple on the transcript is

The case ij = 2 is analyzed in a similar fashion: the probability that (H, 2, ρ) is written as the jth triple on the transcript is (1 - p1)/n!

Hence, the two probability distributions on the partial transcripts at the end of round j are identical. By induction, the two probability distribution [image] and [image] are identical, and the proof is complete.

It is interesting also to look at the interactive proof system for Graph Non-isomorphism. It is not too difficult to prove that this proof is perfect zero-knowledge if Vic follows the protocol (i.e., if Vic chooses each challenge graph to be a random isomorphic copy of Gi where i = 1 or 2 is chosen at random). Further, provided that Vic constructs each challenge graph by taking an isomorphic copy of either G1 or G2, the protocol remains zero-knowledge even if Vic chooses his challenges in a non-random fashion. However, suppose that our ubiquitous troublemaker, Oscar, gives a graph H to Vic which is isomorphic to one of G1 or G2, but Vic does not know which Gi is isomorphic to H. If Vic uses this H as one of his challenge graphs in the interactive proof system, then Peggy will give Vic an isomorphism he didn't previously know, and (possibly) couldn't figure out for himself. In this situation, the proof system is (intuitively) not zero-knowledge, and it does not seem likely that a transcript could be forged by a simulator.

*
* Figure 13.8 A perfect zero-knowledge interactive proof system for Quadratic Residues

It is possible to alter the proof of Graph Non-isomorphism so it is perfect zero-knowledge, but we will not go into the details.

We now present some other examples of perfect zero-knowledge proofs. A perfect zero-knowledge proof for Quadratic Residues (modulo n = pq, where p and q are prime) is given in Figure 13.8. Peggy is proving that x is a quadratic residue. In each round, she generates a random quadratic residue y and sends it to Vic. Then, depending on Vic's challenge, Peggy either gives Vic a square root of y or a square root of xy.

*
* Figure 13.9 Subgroup Membership

It is clear that the protocol is complete. To prove soundness, observe that if x is not a quadratic residue, then Peggy can answer only one of the two possible challenges since, in this case, y is a quadratic residue if and only if xy is not a quadratic residue. So Peggy will be caught in any given round of the protocol with probability 1/2, and her probability of deceiving Vic in all log2 n rounds is only [image]. (The reason for having log2 n rounds is that the size of the problem instance is proportional to the number of bits in the binary representation of n, which is log2 n. Hence, the deception probability for Peggy is exponentially small as a function of the size of the problem instance, as in the zero-knowledge proof for Graph Isomorphism.)

Perfect zero-knowledge for Vic can be shown in a similar manner as was done for Graph Isomorphism. Vic can generate a triple (y, i, z) by first choosing i and z, and then defining

Triples generated in this fashion have exactly the same probability distribution as those generated during the protocol, assuming Vic chooses his challenges at random. Perfect zero-knowledge (for an arbitrary V*) is proved by following the same strategy as for Graph Isomorphism. It requires building a simulator S* that guesses V*'s challenges and keeps only the triples where the guesses are correct.

We now present one more example of a perfect zero-knowledge proof, this one for a decision problem related to the Discrete Logarithm problem. The problem, which we call Subgroup Membership, is defined in Figure 13.9. Of course, the integer k (if it exists) is just the discrete logarithm of β.

We present a perfect zero-knowledge proof for Subgroup Membership in Figure 13.10. The analysis of this protocol is similar to the others that we have looked at; the details are left to the reader.

*
* Figure 13.10 A perfect zero-knowledge interactive proof system for Subgroup Membership


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

13.3 Bit Commitments

The zero-knowledge proof system for Graph Isomorphism is interesting, but it would be more useful to have zero-knowledge proof systems for problems that are known to be NP-complete. There is theoretical evidence that perfect zero-knowledge proofs do not exist for NP-complete problems. However, we can describe proof systems that attain a slightly weaker form of zero-knowledge called computational zero-knowledge. The actual proof systems are described in the next section; in this section we describe the technique of bit commitment that is an essential tool used in the proof system.

Suppose Peggy writes a message on a piece of paper, and then places the message in a safe for which she knows the combination. Peggy then gives the safe to Vic. Even though Vic doesn't know what the message is until the safe is opened, we would agree that Peggy is committed to her message because she cannot change it. Further, Vic cannot learn what the message is (assuming he doesn't know the combination of the safe) unless Peggy opens the safe for him. (Recall that we used a similar analogy in Chapter 4 to describe the idea of a public-key cryptosystem, but in that case, it was the recipient of the message, Vic, who could open the safe.)

Suppose the message is a bit b = 0 or 1, and Peggy encrypts b in some way. The encrypted form of b is sometimes called a blob and the encryption method is called a bit commitment scheme. In general, a bit commitment scheme will be a function f : {0, 1} × X → Y, where X and Y are finite sets. An encryption of b is any value f(b, x), x ∈ X. We can informally define two properties that a bit commitment scheme should satisfy:

concealing
For a bit b = 0 or 1, Vic cannot determine the value of b from the blob f(b, x).
binding
Peggy can later "open" the blob, by revealing the value of x used to encrypt b, to convince Vic that b was the value encrypted. Peggy should not be able to open a blob as both a 0 and a 1.

If Peggy wants to commit any bitstring, she simply commits every bit independently.

One way to perform bit commitment is to use the Goldwasser-Micali Probabilistic Cryptosystem described in Section 12.4. Recall that in this system, n = pq, where p and q are primes, and [image]. The integers n and m are public; the factorization n = pq is known only to Peggy. In our bit commitment scheme, we have [image] and

Peggy encrypts a value b by choosing a random x and computing y = f(b, x); the value y comprises the blob.

Later, when Peggy wants to open y, she reveals the values b and x. Then Vic can verify that

Let us think about the concealing and binding properties. A blob is an encryption of 0 or of 1, and reveals no information about the plaintext value x provided that the Quadratic Residues problem is infeasible (we discussed this at length in Chapter 12). Hence, the scheme is concealing.

Is the scheme binding? Let us suppose not; then

for some [image]. But then

which is a contradiction since [image].

We will be using bit commitment schemes to construct zero-knowledge proofs. However, they have another nice application, to the problem of coin-flipping by telephone. Suppose Alice and Bob want to make some decision based on a random coin flip, but they are not in the same place. This means that it is impossible for one of them to flip a real coin and have the other verify it. A bit commitment scheme provides a way out of this dilemma. One of them, say Alice, chooses a random bit b, and computes a blob, y. She gives y to Bob. Now Bob guesses the value of b, and then Alice opens the blob to reveal b. The concealing property means that it is infeasible for Bob to compute b given y, and the binding property means that Alice can't "change her mind" after Bob reveals his guess.

We now give another example of a bit commitment scheme, this time based on the Discrete Logarithm problem. Recall from Section 5.1.2 that if p = 3 (mod 4) is a prime such that the Discrete Logarithm problem in [image] is infeasible, then the second least significant bit of a discrete logarithm is secure. Actually, it has been proved for primes p ≡ 3 (mod 4) that any Monte Carlo algorithm for the Second Bit problem having error probability 1/2 = ∈ with ∈ > 0 can be used to solve the Discrete Log problem in [image]. This much stronger result is the basis for the bit commitment scheme.

This bit commitment scheme will have X = {1, …, p - 1} and [image]. The second least significant bit of an integer x, denoted by SLB(x), is defined as follows:

The bit commitment scheme f is defined by

In other words, a bit b is encrypted by choosing a random element having second last bit b, and raising α to that power modulo p. (Note that SLB(p - x) ≠ SLB (x) since p ≡ 3 (mod 4).)

The scheme is binding, and by the remarks made above, it is concealing provided that the Discrete Logarithm problem in [image] is infeasible.


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

13.4 Computational Zero-knowledge Proofs

In this section, we give a zero-knowledge proof system for the NP-complete decision problem Graph 3-Colorability, which is defined in Figure 13.11. The proof system uses a bit commitment scheme; to be specific, we will employ the bit commitment scheme presented in Section 13.3 that is based on probabilistic encryption. We assume that Peggy knows a 3-coloring φ of a graph G, and she wants to convince Vic that G is 3-colorable in a zero-knowledge fashion. Without loss of generality, we assume that G has vertex set V = {1, …, n}. Denote m = |E|. The proof system will be described in terms of a commitment scheme f : {0, 1} x X → Y which is made public. Since we want to encrypt a color rather than a bit, we will replace the color 1 by the two bits 01, the color 2 by 10 and the color 3 by 11. Then we encrypt each of the two bits representing the color by using f.

*
* Figure 13.11 Graph 3-Colorability

The interactive proof system is presented in Figure 13.12. Informally, what happens is the following. In each round, Peggy commits a coloring that is a permutation of the fixed coloring φ. Vic requests that Peggy open the blobs corresponding to the endpoints of some randomly chosen edge. Peggy does so, and then Vic checks that the commitments are as claimed and that the two colors are different. Notice that all Vic's computations are polynomial-time, and so are Peggy's, provided that she knows the existence of one 3-coloring φ.

Here is a very small example to illustrate.

Example 13.3

Suppose G is the graph (V, E), where

and

Suppose that Peggy knows the 3-coloring φ where φ(1) = 1, φ(2) = φ(4) = 2 and φ(3) = φ(5) = 3. Suppose also that the bit commitment scheme is defined as f(b, x) = 156897bx2 mod 321389, where b = 0, 1 and [image].

Suppose that Peggy chooses the permutation π = (1 3 2) in some round of the proof. Then she computes:

*
* Figure 13.12 A computational zero-knowledge interactive proof system for Graph 3-colorability

She will encode this coloring in binary as the 10-tuple

0111101110

and then compute commitments of these ten bits. Suppose that she does this as follows:

Then Peggy gives Vic the ten values f(b, x) computed above.

Next, suppose that Vic chooses the edge 34 as his challenge. Then Peggy opens four blobs: the two that correspond to vertex 3 and the two that correspond to vertex 4. So Peggy gives Vic the ordered pairs

Vic will first check that the two colors are distinct: 10 encodes color 2 and 11 encodes color 3, so this is all right. Next, Vic verifies that the four commitments are valid and hence this round of the proof is completed successfully.

As in previous proof systems we have studied, Vic will accept a valid proof with probability 1, so we have completeness. What is the probability that Vic will accept if G is not 3-colorable? In this case, for any coloring, there must be at least one edge ij such that i and j have the same color. Vic's chances of choosing such an edge are at least 1/m. Peggy's probability of fooling Vic in all m2 rounds is at most

Since (1 - 1/m)m → e-1 as m ∞, there exists an integer m0 such that (1 - 1/m)m ≤ 2/e for m ≥ m0. Hence [image]. Since (2/e)m approaches zero exponentially quickly as a function of m = |E|, we have soundness as well.

Let's now turn to the zero-knowledge aspect of the proof system. All that Vic sees in any given round of the protocol is an encrypted 3-colouring of G, together with the two distinct colours of the endpoints of one particular edge, as previously committed by Peggy. Since the colors are permuted in each round, it seems that Vic cannot combine information from different rounds to reconstruct the 3-coloring.

The proof system is not perfect zero-knowledge, but it does provide a weaker form of zero-knowledge called computational zero-knowledge. Computational zero-knowledge is defined exactly as perfect zero-knowledge, except that the relevant probability distributions of transcripts are required only to be polynomially indistinguishable (in the sense of Chapter 12) rather than identical.

We begin by showing how transcripts can be forged. We give an explicit algorithm that will forge transcripts that cannot be distinguished from those produced by an honest Vic. If Vic deviates from the protocol, then it is possible to construct a simulator which uses the algorithm V* as a restartable subroutine to construct forged transcripts. Both forging algorithms follow the pattern of the related algorithms for the Graph Isomorphism proof system.

Here, we consider only the case where Vic follows the protocol. A transcript T for the interactive proof of Graph 3-colorability would have the form

where Aj consists of 2n blobs computed by Peggy, the edge uv chosen by Vic, the colors assigned by Peggy in round j to u and v, and the four random numbers used by Peggy to encrypt the colors of these two vertices. A transcript is forged by means of the forging algorithm presented in Figure 13.13.

Proving (computational) zero-knowledge for Vic requires showing that the two probability distributions on transcripts (as produced by the Vic taking part in the protocol, and as produced by the simulator) are indistinguishable. We will not do this here, but we will make a couple of comments. Notice that the two probability distributions are not identical. This is because virtually all the Rij's in a forged transcript are blobs encrypting 1; whereas the Rij's on a real transcript will (usually) be encryptions of more equal numbers of 0's and 1's. However, it is possible to show that the two probability distributions cannot be distinguished in polynomial time, provided that the underlying bit commitment scheme is secure. More precisely, this means that the probability distribution on blobs encrypting color c are indistinguishable from the probability distribution on blobs encrypting color d if c ≠ d.

Readers familiar with NP-completeness theory will realize that, having given a zero-knowledge proof for one particular NP-complete problem, we can obtain a zero-knowledge proof for any other problem in NP. This can be done by applying a polynomial transformation from a given problem in NP to the Graph 3-coloring problem.

*
* Figure 13.13 Forging algorithm for transcripts for Graph 3-colorability


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

13.5 Zero-knowledge Arguments

Let us recap the basic properties of the computational zero-knowledge proof for Graph 3-colorability presented in the last section. No assumptions are needed to prove completeness and soundness of the protocol. A computational assumption is needed to prove zero-knowledge, namely that the underlying bit commitment scheme is secure. Observe that if Peggy and Vic take part in the protocol, then Vic may later try to break the bit commitment scheme that was used in the protocol (for example, if the scheme based on quadratic residuosity were used, then Vic would try to factor the modulus). If at any future time Vic can break the bit commitment scheme, then he can decrypt the blobs used by Peggy in the protocol and extract the 3-coloring.

This analysis depends on the properties of the blobs that were used in the protocol. Although the binding property of the blobs is unconditional, the concealing property relies on a computational assumption.

An interesting variation is to use blobs in which the concealing property is unconditional but the binding property requires a computational assumption. This leads to a protocol that is known as a zero-knowledge argument rather than a zero-knowledge proof. The reader will recall that we have assumed up until now that Peggy is all-powerful; in a zero-knowledge argument we will assume that Peggy's computations are required to be polynomial-time. (In fact, this assumption creates no difficulties, for we have already observed that Peggy's computations are polynomial-time provided she knows one 3-coloring of G.)

Let us begin by describing a couple of bit commitment schemes of this type and then examine the ramifications of using them in the protocol for Graph 3-coloring.

The first scheme is (again) based on the Quadratic Residues problem. Suppose n = pq, where p and q are prime, and let m ∈ QR(n) (note that in the previous scheme m was a pseudo-square). In this scheme neither the factorization of n nor the square root of m should be known to Peggy. So either Vic should construct these values or they should be obtained from a (trusted) third party.

Let [image] and Y = QR(n), and define

As before, Peggy encrypts a value b by choosing a random x and computing the blob y = f(b, x). In this scheme all the blobs are quadratic residues. Further, any y ∈ QR(n) is both an encryption of 0 and an encryption of 1. For suppose y = x2 mod n and m = k2 mod n. Then

This means that the concealing property is achieved unconditionally. On the other hand, what happens to the binding property? Peggy can open any given blob both as a 0 and as a 1 if and only if she can compute k, a square root of m. So, in order for the scheme to be (computationally) binding, we need to make the assumption that it is infeasible for Peggy to compute a square root of m. (If Peggy were all-powerful, then she could, of course, do this. This is one reason why we are now assuming that Peggy is computationally bounded.)

As a second bit commitment scheme of this type, we give an example of a scheme based on the Discrete Logarithm problem. Let p be a prime such that the discrete log problem in [image] is infeasible, let α be a primitive element of [image] and let [image]. The value of β should be chosen by Vic, or by a trusted third party, rather than by Peggy. This scheme will have [image], and f is defined by

It is not hard to see that this scheme is unconditionally concealing, and it is binding if and only if it is infeasible for Peggy to compute the discrete logarithm logα β.

Now, suppose we use one of these two bit commitment schemes in the protocol for Graph 3-colorability. It is easy to see that the protocol remains complete. But now the soundness condition depends on a computational assumption: the protocol is sound if and only if the bit commitment scheme is binding. What happens to the zero-knowledge aspect of the protocol? Because the bit commitment scheme is unconditionally concealing, the protocol is now perfect zero-knowledge rather than just computational zero-knowledge. Thus we have a perfect zero-knowledge argument.

Table 13.1 Comparison of Properties of Proofs and Arguments
property | zero-knowledge proof | zero-knowledge argument
completeness | unconditional | unconditional
soundness | unconditional | computational
zero-knowledge | computational | perfect
binding blobs | unconditional | computational
concealing blobs | computational | unconditional

Whether one prefers an argument to a proof depends on the application, and whether one wants to make a computational assumption regarding Peggy or Vic. A comparison of the properties of proofs and arguments is summarized in Table 13.1. In the column "zero-knowledge proof," the computational assumptions pertain to Peggy's computing power; in the column "zero-knowledge argument," the computational assumptions refer to Vic's computing power.

13.6 Notes and References

Most of the material in this chapter is based on Brassard, Chaum, and Crépeau [BCC88] and on Goldreich, Micali, and Wigderson [GMW91]. The bit commitment schemes we present, and a thorough discussion of the differences between proofs and arguments, can be found in [BCC88] (however, note that the term "argument" was first used in [BC90]). Zero-knowledge proofs for Graph Isomorphism, Graph Non-isomorphism and Graph 3-colorability can be found in [GMW91]. Another relevant paper is Goldwasser, Micali, and Rackoff [GMR89], in which interactive proof systems are first defined formally. The zero-knowledge proof for Quadratic Residues is from this paper.

The idea of coin-flipping by telephone is due to Blum [BL82].

A very informal and entertaining illustration of the concept of zero-knowledge is presented by Quisquater and Guillou [QG90]. Also, see Johnson [JO88] for a more mathematical survey of interactive proof systems.

*
* Figure 13.14 An interactive proof system for Quadratic Non-residues


Cryptography: Theory and Practice

 Cryptography: Theory and Practice - Douglas Stinson

Exercises

13.1 Consider the interactive proof system for the problem Quadratic Non-residues presented in Figure 13.14. Prove that the system is sound and complete, and explain why the protocol is not zero-knowledge.
13.2 Devise an interactive proof system for the problem Subgroup Non-membership. Prove that your protocol is sound and complete.
13.3 Consider the zero-knowledge proof for Quadratic Residues that was presented in Figure 13.8.

(a) Define a valid triple to be one having the form (y, i, z), where y ∈ QR(n), i = 0 or 1, [image] and z2 ≡ xiy (mod n). Show that the number of valid triples is 2(p - 1)(q - 1), and each such triple is generated with equal probability if Peggy and Vic follow the protocol.
(b) Show that Vic can generate triples having the same probability distribution without knowing the factorization n = pq.
(c) Prove that the protocol is perfect zero-knowledge for Vic.

13.4 Consider the zero-knowledge proof for Subgroup Membership that was presented in Figure 13.10.

(a) Prove that the protocol is sound and complete.
(b) Define a valid triple to be one having the form (γ, i, h), where [image], i = 0 or 1, 0 ≤ h ≤ [image] - 1 and αh ≡ βiγ (mod n). Show that the number of valid triples is [image], and each such triple is generated with equal probability if Peggy and Vic follow the protocol.
(c) Show that Vic can generate triples having the same probability distribution without knowing the discrete logarithm logα β.
(d) Prove that the protocol is perfect zero-knowledge for Vic.

13.5 Prove that the Discrete Logarithm bit commitment scheme presented in Section 13.5 is unconditionally concealing, and prove that it is binding if and only if Peggy cannot compute logα β.
13.6 Suppose we use the Quadratic Residues bit commitment scheme presented in Section 13.5 to obtain a zero-knowledge argument for Graph 3-coloring. Using the forging algorithm presented in Figure 13.13, prove that this protocol is perfect zero-knowledge for Vic.


 Selections from Cryptologia: History, People, and Technology - Cipher A. Deavours et al
Table of Contents

Appendix A: The Artech House Telecommunications Library

Vinton G. Cerf, Series Editor

Access Networks: Technology and V5 Interfacing, Alex Gillespie

Advanced High-Frequency Radio Communications, Eric E. Johnson, Robert I. Desourdis, Jr., et al.

Advanced Technology for Road Transport: IVHS and ATT, Ian Catling, editor

Advances in Computer Systems Security, Vol. 3, Rein Turn, editor

Advances in Telecommunications Networks, William S. Lee and Derrick C. Brown

Advances in Transport Network Technologies: Photonics Networks, ATM, and SDH, Ken-ichi Sato

An Introduction to International Telecommunications Law, Charles H. Kennedy and M. Veronica Pastor

Asynchronous Transfer Mode Networks: Performance Issues, Second Edition, Raif O. Onvural

ATM Switches, Edwin R. Coover

ATM Switching Systems, Thomas M. Chen and Stephen S. Liu

Broadband: Business Services, Technologies, and Strategic Impact, David Wright

Broadband Network Analysis and Design, Daniel Minoli

Broadband Telecommunications Technology, Byeong Lee, Minho Kang and Jonghee Lee

Cellular Mobile Systems Engineering, Saleh Faruque

Cellular Radio: Analog and Digital Systems, Asha Mehrotra

Cellular Radio: Performance Engineering, Asha Mehrotra

Cellular Radio Systems, D. M. Balston and R. C. V. Macario, editors

CDMA for Wireless Personal Communications, Ramjee Prasad

Client/Server Computing: Architecture, Applications, and Distributed Systems Management, Bruce Elbert and Bobby Martyna

Communication and Computing for Distributed Multimedia Systems, Guojun Lu

Communications Technology Guide for Business, Richard Downey, et al.

Community Networks: Lessons from Blacksburg, Virginia, Andrew Cohill and Andrea Kavanaugh, editors

Computer Networks: Architecture, Protocols, and Software, John Y. Hsu

Computer Mediated Communications: Multimedia Applications, Rob Walters

Computer Telephone Integration, Rob Walters

Convolutional Coding: Fundamentals and Applications, Charles Lee

Corporate Networks: The Strategic Use of Telecommunications, Thomas Valovic

The Definitive Guide to Business Resumption Planning, Leo A. Wrobel

Digital Beamforming in Wireless Communications, John Litva and Titus Kwok-Yeung Lo

Digital Cellular Radio, George Calhoun

Digital Hardware Testing: Transistor-Level Fault Modeling and Testing, Rochit Rajsuman, editor

Digital Switching Control Architectures, Giuseppe Fantauzzi

Digital Video Communications, Martyn J. Riley and Iain E. G. Richardson

Distributed Multimedia Through Broadband Communications Services, Daniel Minoli and Robert Keinath

Distance Learning Technology and Applications, Daniel Minoli

EDI Security, Control, and Audit, Albert J. Marcella and Sally Chen

Electronic Mail, Jacob Palme

Enterprise Networking: Fractional T1 to SONET, Frame Relay to BISDN, Daniel Minoli

Expert Systems Applications in Integrated Network Management, E. C. Ericson, L. T. Ericson, and D. Minoli, editors

FAX: Digital Facsimile Technology and Applications, Second Edition, Dennis Bodson, Kenneth McConnell, and Richard Schaphorst

FDDI and FDDI-II: Architecture, Protocols, and Performance, Bernhard Albert and Anura P. Jayasumana

Fiber Network Service Survivability, Tsong-Ho Wu

Future Codes: Essays in Advanced Computer Technology and the Law, Curtis E. A. Karnow

Guide to Telecommunications Transmission Systems, Anton A. Huurdeman

A Guide to the TCP/IP Protocol Suite, Floyd Wilder

Implementing EDI, Mike Hendry

Implementing X.400 and X.500: The PP and QUIPU Systems, Steve Kille

Inbound Call Centers: Design, Implementation, and Management, Robert A. Gable

Information Superhighways Revisited: The Economics of Multimedia, Bruce Egan

Integrated Broadband Networks, Amit Bhargava

International Telecommunications Management, Bruce R. Elbert

International Telecommunication Standards Organizations, Andrew Macpherson

Internetworking LANs: Operation, Design, and Management, Robert Davidson and Nathan Muller

Introduction to Document Image Processing Techniques, Ronald G. Matteson

Introduction to Error-Correcting Codes, Michael Purser

An Introduction to GSM, Siegmund Redl, Matthias K. Weber and Malcom W. Oliphant

Introduction to Radio Propagation for Fixed and Mobile Communications, John Doble

Introduction to Satellite Communication, Bruce R. Elbert

Introduction to T1/T3 Networking, Regis J. (Bud) Bates

Introduction to Telephones and Telephone Systems, Second Edition, A. Michael Noll

Introduction to X.400, Cemil Betanov

LAN, ATM, and LAN Emulation Technologies, Daniel Minoli and Anthony Alles

Land-Mobile Radio System Engineering, Garry C. Hess

LAN/WAN Optimization Techniques, Harrell Van Norman

LANs to WANs: Network Management in the 1990s, Nathan J. Muller and Robert P. Davidson

Minimum Risk Strategy for Acquiring Communications Equipment and Services, Nathan J. Muller

Mobile Antenna Systems Handbook, Kyohei Fujimoto and J. R. James, editors

Mobile Communications in the U.S. and Europe: Regulation, Technology, and Markets, Michael Paetsch

Mobile Data Communications Systems, Peter Wong and David Britland

Mobile Information Systems, John Walker

Networking Strategies for Information Technology, Bruce Elbert

Packet Switching Evolution from Narrowband to Broadband ISDN, M. Smouts

Packet Video: Modeling and Signal Processing, Naohisa Ohta

Personal Communication Networks: Practical Implementation, Alan Hadden

Personal Communication Systems and Technologies, John Gardiner and Barry West, editors

Practical Computer Network Security, Mike Hendry

Principles of Secure Communication Systems, Second Edition, Don J. Torrieri

Principles of Signaling for Cell Relay and Frame Relay, Daniel Minoli and George Dobrowski

Principles of Signals and Systems: Deterministic Signals, B. Picinbono

Private Telecommunication Networks, Bruce Elbert

Radio-Relay Systems, Anton A. Huurdeman

RF and Microwave Circuit Design for Wireless Communications, Lawrence E. Larson

The Satellite Communication Applications Handbook, Bruce R. Elbert

Secure Data Networking, Michael Purser

Selections from Cryptologia: History, People, and Technology, Cipher A. Deavours, et al.

Service Management in Computing and Telecommunications, Richard Hallows

Signaling in ATM Networks, Raif O. Onvural, Rao Cherukuri

Smart Cards, Jose Manuel Oton and Jose Luis Zoreda

Smart Card Security and Applications, Mike Hendry

Smart Highways, Smart Cars, Richard Whelan

Successful Business Strategies Using Telecommunications Services, Martin F. Bartholomew

Super-High-Definition Images: Beyond HDTV, Naohisa Ohta, et al.

Telecommunications Deregulation, James Shaw Television Technology: Fundamentals and Future Prospects, A. Michael Noll

Telecommunications Technology Handbook, Daniel Minoli

Telecommuting, Osman Eldib and Daniel Minoli

Telemetry Systems Design, Frank Carden

Teletraffic Technologies in ATM Networks, Hiroshi Saito

Toll-Free Services: A Complete Guide to Design, Implementation, and Management, Robert A. Gable

Transmission Networking: SONETand the SDH, Mike Sexton and Andy Reid

Troposcatter Radio Links, G. Roda

Understanding Emerging Network Services, Pricing, and Regulation, Leo A. Wrobel and Eddie M. Pope

Understanding GPS: Principles and Applications, Elliot D. Kaplan, editor

Understanding Networking Technology: Concepts, Terms and Trends, Mark Norris

UNIX Internetworking, Second Edition, Uday O. Pabrai

Videoconferencing and Videotelephony. Technology and Standards, Richard Schaphorst

Voice Recognition, Richard L. Klevans and Robert D. Rodman

Wireless Access and the Local Telephone Network, George Calhoun

Wireless Communications in Developing Countries: Cellular and Satellite Systems, Rachael E. Schwartz

Wireless Communications for Intelligent Transportation Systems, Scott D. Elliot and Daniel J. Dailey

Wireless Data Networking, Nathan J. Muller

Wireless LAN Systems, A. Santamaria and F. J. Lopez-Hernandez

Wireless: The Revolution in Personal Telecommunications, Ira Brodsky

Writing Disaster Recovery Plans for Telecommunications Networks and LANS, Leo A. Wrobel

X Window System User's Guide, Uday O. Pabrai

For further information on these and other Artech House titles, including previously considered out-of-print books now available through our In-Print-ForeverTM (IPFTM) program, contact:

Artech House | Artech House
685 Canton Street | Portland House, Stag Place
Norwood, MA 02062 | London SW1E 5XA England
781-769-9750 | +44 (0) 171-973-8077
Fax: 781-769-6334 | Fax: +44 (0) 171-630-0166
Telex: 951-659 | Telex: 951-659
email: artech@artech-house.com | email: artech-uk@artech-house.com

Find us on the World Wide Web at:

www.artech-house.com

Table of Contents


 [Spisak knjiga]

Selections from Cryptologia: History, People, and Technology
Cipher A. Deavours et al

Introduction to the History section of the third collection of CRYPTOLOGIA articles
Illumination from the Past
The Cryptographic Services of the Royal (British) and Italian Navies
Pearl Harbor And The Inadequacy Of Cryptanalysis
The Unsolved Messages Of Pearl Harbor
The Fall And Rise Of Cryptanalysis In Canada
Origins Of Cryptology: The Arab Contributions
Roosevelt, Magic, And Ultra
The Papal Cipher Section In The Early Nineteenth Century
Cryptanalysis for Peacetime: Codebreaking and the Birth and Structure of the United Nations
Italian Diplomatic Cryptanalysis in World War I
Faded Lustre: Vatican Cryptography, 1815-1920
Bletchley Park 1941-1945
Why Was Safford Pessimistic About Breaking the German Enigma Cipher Machine in 1942?
S. I. S./CB
Agnes Meyer Driscoll
Letters to the Editor
MY “Purple” Trip to England in 1941
Introduction to the Technology section of the third collection of CRYPTOLOGIA articles
Cryptologic Technology
Viet Cong SIGINT and U.S. Army COMSEC in Vietnam
Some Victorian Periodic Polyalphabetic Ciphers
Chaocipher Enters the Computer Age when its Method is Disclosed to CRYPTOLOGIA’S Editors
The Key-Vowel Cipher of Charles L. Dodgson
The Cylinder-Cipher
A World War II German Army Field Cipher and How We Broke It
An Introduction to an Historic Computer Document: The 1946 Pendergass Report—Cryptanalysis and the Digital Computer
Enigma Before ULTRA Polish Work and the French Contribution
Enigma Before ULTRA The Polish Success And Check (1933-1939)
Enigma Avant ULTRA (Enigma Before ULTRA)
The Turing Bombe: Was It Enough?
An Enigma Chronology
The Siemens Cipher Teletype in the History of Telecommunications
New Information on the History of the Siemens and Halske T52 Cipher Machines
The Norwegian Modification of the Siemens and Halske T52e Cipher M-chines
The Breaking of the Japanese Army Administrative Code
The Story of the Hagelin Cryptos
The Lorenz Cipher Machine SZ42
The Autoscritcher
Appendix A


Introduction to the History section of the third collection of CRYPTOLOGIA articles

Illumination from the Past

by David Kahn

This third volume of selected papers from issues of CRYPTOLOGIA again glows with the quality of its history. Some readers might have feared—as I did, both with the previous volume and with this one—that the quality would decline as the field was mined. In fact, the quality has grown. Mining has not depleted the history of cryptology; it has augmented it. It has raised new questions. It has repeatedly shown, in a variety of contexts, the importance of crytology in people's lives. It has demonstrated the richness of the sources available. The flow of papers to CRYPTOLOGIA continues to increase; the quality and variety from a growing number of authors are better than ever before.

Some of the superior examples of CRYPTOLOGIA articles are printed here. A few are works of the first order in the history of cryptology. For example, Ibrahim A. Al-Kindi's study of early Arab cryptology pushes the known origins of that field back some 300 years. Stephen Schlesinger's study of American solutions of foreign codes to help establish a United Nations the way the United States wanted it is one of the few illustrations of a major use of cryptanalysis in peacetime. David Alvarez's study of the cryptography of that almost legendary body, the papal curia, dispassionately analyzes the sad decline of an agency that once was supreme in its field and still is bathed in a nimbus of mystery messages.

Frederick Parker maintains that, had the Japanese messages intercepted before the Japanese attack on Pearl Harbor been solved at the time, instead of retrospectively after the War, as they were, the attack would have been foreseen and so perhaps prevented. I disagree with this, thinking that the conclusion is rooted in hindsight, but approved the article for publication because I thought the view was legitimate and worthy of dissemination. My own article holds that no cryptanalytic information pointed, or would have pointed, to an attack.

Also included is Susan Lujan's profile of a colorful and important, but too often forgotten, woman cryptanalyst. The two studies of Italian cryptology provide badly needed illumination of an area only sketched in elsewhere.

The other historical articles likewise add details to history or fill in chinks in the record. We therefore hope that the cryptologic community will concur with our confidence in the interest and worth of these articles.

The Cryptographic Services of the Royal (British) and Italian Navies

A comparative analysis of their activities during World War II

Rear Admiral (disch.) Luigi Donini1
Translated by Augusto Buonafalce2.

ADDRESS: (1) Via della Motonautica 9, 00194 Rome ITALY and (2) Via Rogoredo 113, 20138 Milano ITALY.

[This article is a translation of a paper which was published in the January 1983 issue of Rivista Marittima. Rivista Marittima, founded in 1868, is a monthly publication of the Italian Naval General Staff, Via Romeo Romei, n. 5 - 00136 Rome ITALY. The article is printed with permission of the author, the editors of Rivista Marittima, and the translator.]

1 INTRODUCTION

1.1

The interesting article by Captain (Engineering Duty Corps) Gino Jori appearing in the May 1982 issue of Rivista Marittima under the title La crittologia nelle operazioni navali in Mediterraneo (1940-43), offers a clear and sufficiently enlightened picture of the influence that the secret services - especially the cryptological one - had on the naval operations in the Mediterranean in the Second World War. This article, however, conveys something less than the title seems to promise, because it deals extensively with the influence of these services in the direction Royal Navy - Italian Navy, but very little in the opposite direction. The author must not be blamed for this. In the first instance he had at his disposal the abundant documentation furnished with some reticence and adjustments, involuntary or deliberate, by the first two volumes of the official work British Intelligence in the Second World War1. In the second instance he had nothing else, official or unofficial, than the precious material contained in the 21 volumes of the Ufficio Storico della Marina La Marina italiana nella Seconda Guerra Mondiale. This work says little or nothing of our secret services and particularly nothing about the cryptographic service in our Navy and even less about its influence on the conduct of the naval warfare. As of now an official, or at least unofficial, documentation is missing, and it is not foreseen that this gap may be filled in a totally credible manner, as all papers relating to our cryptological work performed during the war period, and even before, were destroyed by superior orders in the days immediately following the armistice.

1Hereinafter quoted as B.I.

Jori himself was conscious of this gap and has expressed in his article the wish that it may be filled by others more directly cognizant of this particular matter. This is what I am going to do.

The aforesaid abnormal situation, while it prevents at the present time a documented, equitable summing-up of the contribution furnished to the relevant war operations by the two rival cryptographic services, has caused even an authoritative historian to make some statements that should have been verified. For instance, dealing with the war in the Mediterranean, it was written that "our formidable decrypting ability" did not cause any damage to our adversaries because "almost always it was our ships that were involved, our men who died."

I regard this judgment not much considered and rather unfair because the British or Allied ships, whether military or merchant, sunk or damaged in the Mediterranean were more than one. In addition to that, it should be pointed out that the British naval ciphers [Transl. note: Includes codes everywhere in this article.] broken by our cryptographic service were in general use, i.e. not only in the Mediterranean, but in all the naval warfare theaters, and that the effects of those breaks weighed heavily on our opponents, especially in the Atlantic, both Northern and Southern, and in the Arctic Seas.


1.2

Although there are not sufficient elements to draw a final balance, I believe it conceivable to make a comparative analysis of the activities of both services regardless of whether their higher commands have utilized them or not in strategic and tactical fields. In other words I will make a purely technical comparison of the structure and quality of the cryptosystems, of the degree of difficulty of the relevant cryptographic work, of the capability/incapability shown of reaching a success or suffering a defeat. This analysis is in order because several authors show a tendency to minimize or criticize what was done by us in this field and they exalt what the British did. For example, some gave a general judgment on our enciphering systems defining them, little sophisticated. Our opponent has been defined as skillful, while it has been conceitedly written about our cryptanalysts that "after all they had obtained appreciable results." The fact that we had been unable to prevent the British from breaking our mechanical enciphering is described as "bordering guilt" while our opponent is not to be criticized for our cracking of their principal naval ciphers.

An analysis of this kind will help clarify the cause of the British success in breaking our mechanical encipherments, and must take into consideration both offensive cryptography or decryption, i.e., attacking the enemy signals, and defensive cryptography, i.e. the protection of one's encipherments.

It seems right to me to limit my analysis to the systems of general usage for the wireless messages of high degree of secrecy, i.e. the superenciphered naval ciphers (British and Italian) and the mechanical encipherments (Italian). About the latter I think I can disregard, in this article, the Enigma machine used by the SIS (Navy Intelligence). One reason is because it was not of general naval use, having been used by Supermarina (Navy Hqtrs.) in rare occasions (B.I. on page 210, vol 1 says: "only one or two signals per day" and only during the first 9 months of war.) Another reason is because its history has been absorbed by the history of the Enigma machine, used by the German armed forces, which was broken by the British in 1940.

1.3

In the following chart, whose banality will be forgiven by my readers, the signs + or - indicate respectively a success or failure in attacking enemy systems in case of offensive cryptography and failure in cracking (+) or actual cracking (-) of one's systems by the opponent in case of defensive cryptography.2

2The Royal Navy, in contrast with the other Armed Forces, did not use mechanical encipherment during the conflict with Italy. They implemented it only in November 1943.
 |  | Italy | Great Britain
 | Naval ciphers | + | -
Attack
 | Cipher machine |  | +
 | Naval ciphers | + | -
Defense
 | Cipher machine | -

As the reader can see, while forgiving such oversimplification, we have two plus signs and a minus sign for Italy versus two minus signs and only one plus sign for Great Britain. Let's examine in detail what the systems were like and what happened to them in that secret war of means and brains.

2 THE SUPERENCIPHERED NAVAL CIPHERS

2.1

As is well known to the majority of readers, this cipher system is divided into two successive operations. The first operation consists of substituting the words or phrases of the plaintext with secret literal or numerical groups (4-digit groups for both Navies in question) found in the cipher book or code book proper. For instance (words and 4-digit groups imaginary):

Plaintext | Hush | most | secret | stop | Cincmed | from | Adm London | stop …
First encipherment | 1782 | 8130 | 6615 | 8469 | 0983 | 3561 | 1565 | 8469

The second operation, called second encipherment or superencipherment, consists of subtracting one digit at a time, without borrowing, i.e. modulo 10, the first - encipherment groups from the groups of the so called superencipherment keystream as taken from suitable tables. For instance:

Keystream | (V) | 2916 | 3410 | 9226 | 0880 | 4534 | 8117 | 4036 | 1912 | …
-1st enc. | (C) | -1782 | 8130 | 6615 | 8469 | 0983 | 3561 | 1565 | 8469 | …
Superenciph. | (S) | 1234 | 5380 | 3611 | 2421 | 4651 | 5656 | 3571 | 3553


That is: S = V - C to perform superencipherment and C = V - S to eliminate superencipherment.

It is obvious that no cryptanalyst, even if in possession of the enemy cipher, will ever be able to decrypt a message superenciphered with a random or incoherent keystream, not known to him and used only for that message.

However, if in the mass of material under examination there are pairs, triples etc. of messages superenciphered with the same keystream, then a small gap is open for penetration on the basis of a very simple algebraic consideration.

In fact the difference between two second encipherment groups S2 and S1, superenciphered with the same keystream group V, is equivalent to the difference between the two first encipherment groups (C1 and C2) which have generated the relevant superenciphered groups:

S1 | = | V - C1
S2 | = | V - C2
S1 - S2 | = | C2 - C1

It follows that only the pairs, triples etc. of messages enciphered with the same cipher and superenciphered with the same keystream are the basis with which to start the penetration. To do this the cryptanalyst must:

• identify, if any, all pairs, triples etc. of messages superenciphered with the same keystream
• execute, for each identified pair, the difference between the corresponding groups, i.e. between the groups occupying the same position in both messages
• keep track of the differences which have appeared with frequency noticeably greater than that expected according to the laws of probability
• for each difference (S1 - S2 = C2 - C1) accepted as meaningful, assign arbitrary numerical values (C2 and C1) to the groups of code which have generated it. He should then try to put together such arbitrary values so as to reduce them all to the same relative basis. These arbitrary values should then differ from the corresponding true code groups (still unknown) by a common constant which in time may perhaps be identified, although this is not necessary¹.

The above work will have to be continued until a sufficient quantity of material is collected so as to assign meaning to some of the arbitrary code groups.

The reconstruction of the used code or cipher proper can then be started and developed together with the relevant superencipherment tables.

2.2

It is not necessary to show here how hard and laborious is the work I have briefly described and which the cryptographic service of our Navy managed to mplete for the different British ciphers and codes which had been utilized from 1938 through 1942.

Similarly, "British attempts against our naval ciphers of the war period met with complete failure, honestly admitted by our adversaries and repeatedly regretted also in the two volumes of B.I.", as has been pointed out by Jori in his above mentioned article. For instance, at page 210 of B.I. vol. 1 one can read: "Italy's main naval book-cyphers, which were the cyphers used by her fleet for most of its communications, were never read again after July 1940, except for a few brief intervals as a result of captures after the middle of 1941." We do not know how much of this failure was due to their lesser ability or how much was due to the possibly smaller intensity of our radio traffic or how much was due to a greater efficiency of the expedients and countermeasures which we introduced in our cipher/superencipherment system in mid-July 1940. It was because we could use the knowledge of the faults found in the corresponding British system and the tricks we had learned to use in order to break it.


2.3

The weaknesses of the British system were varied and of differing weight. I will limit myself to citing a few:

a) total reuse of keystream, very frequently up to 3/4 times for the same keystream; partial re-use very frequent;
b) period of validity (even though for a few days) of the superencipherment tables was badly commensurated with the intensity of radio traffic;
c) imperfect control of the wearing out of the keystream tables. When rescission of overused keystreams was ordered by enciphered message, the damage had already occurred;
d) within the tables containing the superencipherment keystream whole lines existed here and there already existing in rescinded tables (perhaps for quicker compilation);
e) the indicator groups placed at the beginning of the cryptograms to indicate to the recipient which keystream was used were utilized for many years without masking, i.e. not disguised. The possible reuse of a keystream was then directly revealed by the identical indicators;
f) when, from December 1940 through the better part of 1941, the British introduced the masking of the indicators, this masking was so banal and moreover so badly used by the encipherers that it only caused us a two week crisis;
g) the ciphers did not contain homophones, i.e. did not assign two or more different secret groups to encipher a certain word of very frequent usage. For instance fullstop could be enciphered only in one manner (in our ciphers: a score of different ways), similarly for from etc.;
h) geographical names were coupled, as an alternate interpretation, to vocabulary entries beginning with the same two or three letters. e.g.: 7184 = Give/Gibraltar, 0921 = Last/La Spezia, 4650 = Make/Malta, 2935 = All Concerned/Alexandria, 7714 = Left/Leghorn. This was of great help in identifying the geographical term when we had already identified the vocabulary term coupled thereto and vice versa;
i) the signal almost invariably began with the complete address: (to)…from;
j) the cipher books were made of two sections. The first section, which was mostly used to encipher the address, contained the names of the Commands, of the Authorities, of the Naval Units, and, in addition to that, numbers, dates, ranks and minutes of latitude and longitude; the second section was the vocabulary proper. In both sections the secret enciphering groups (4 figures) were chosen at random from the natural series from 0001 to 9999, and almost all the 4 digit groups were found in both the first and second section.

If the address used up 1,2,3, …, 9 groups, the first group of the message in first encipherment was 1111, 2222, 3333, …, 9999 respectively to indicate to the addressee that for 1,2,3,…, 9 groups he was to use the first section of the cipher book. If the address used more than 9 groups, the first group was 9900 and at the end of the address the group 0099 was placed to indicate that the second section was to be used. 0099 was also placed as first group when the second section and not the first section was to be used from the start.

These peculiarities obviously had a mnemonic purpose, and were kept in use for several years. That meant that the differences pertaining to the first group of message pairs having identical indicator group, i.e. superenciphered with the same keystream,3 only had the following values 0000, 1111, 2222, …, 9999, 1122, 2233, …, 8899, 2211, 3322, …, 9988, 1199, 9911. From the observation of this peculiarity to the formulation of the logical explanation was a short step for us and that marked the opening of the first crack for penetration.

3See Paragraph 2.1. It should be remembered that the difference between two second encipherment groups superenciphered with the same keystream is equal to the difference between the relevant groups in first encipherment.

Among other information, we were immediately able to know how many groups were used for the addresses, where the secret group for from (which had no homophones, see Subparagraph g) was located and where the actual text was beginning, with its stereotyped initial phrases (reference …, answer …, continuation, …, etc.).

In my opinion, however, the most serious British carelessness or inadvertence was to keep in use for the whole first year of war with Germany the same principal naval cipher which was in force since the time of the Spanish civil war, broken by us (and by the Germans) in 1938.

When I had to leave the command of a submarine unit to assume the direction of the SIS Cryptographic Branch (where I had worked with few interruptions since 1934) I was surprised - but also elated - to notice that the main target was still the same I was confronted with from 1938 to the beginning of 1940.

This was the reason why our cryptographic section had decrypted, on July 4, 1940, the operation order of the Commander of the Mediterranean Fleet, which caused the passage of arms of Punta Stilo on the 9th of that month. This was the first occurrence in history of a clash between the two fleets; an encounter which was to be inconclusive.4

4As Jori mentioned in his article, it appears that the British had obtained information, albeit incomplete, on the intention of our fleet, by intercepting our plain language signals and also by decrypting some radio messages which were still enciphered with our pre-war cipher. This had already been broken by the British. It was substituted only in mid-July 1940, probably owing to organizational difficulties.

Unfortunately only two weeks after my return to the cryptographic service, the British discontinued that cipher, substituting it with two different ciphers, and they introduced some devices which did not prevent us from re-assuming control of the situation for both ciphers within a few months.

To sum up the brief technical analysis I performed in reference to the cipher warfare, I believe I can state that our Navy's cryptographic service surpassed the homologous British service greatly.


3 MECHANICAL ENCIPHERMENT OF THE ITALIAN NAVY

I shall use the same designation C38m used by B.I. for our cipher machine, which was a modified version of almost no cryptographic importance of a Swedish machine commercially available.

3.1

The following describes the adoption of the C38m according to my personal recollection. It is well noted that a previous version of that machine, offered to our Navy by its inventor and producer in 1934-35, was refused by SIS after an accurate test performed by a board of 4 cryptographers, the present writer included. In 1935 that same machine, obviously considered efficient by the French specialists, was acquired in large quantities by their armed forces.

A few years later, perhaps in 1938, the Swedish maker presented to our General Staff a new model he had somewhat improved, increasing the number of rotors from 5 to 6; but the original principle of the enciphering mechanism was still that of 1935.

I do not know whether the SIS was again consulted as I was absent and on sea duty. The fact is that later on, I believe during 1940, that machine was adopted, but limited to the secret communication of a few High Commands, on shore and on board. In the meantime it had also been acquired by the US Army, which utilized it as a medium level system from Division Command down to Battalion. I can only guess that the improvements introduced by the makers and the knowledge of the bulk purchases made by France and USA may have persuaded our technical administrative bodies to disregard the unfavorable opinion expressed by the SIS some years earlier. Moreover B.I. states in a note on page 22 of vol 2 that C38m was forced on us by the Germans, who considered our prewar system insecure, and perhaps with good reason (see Footnote 4). If this is true then one could assess that the Germans, involuntarily caused us a two-fold damage: with the Enigma and with the C38m. It is easy to be wise after the event and condemn the decision without an appeal. Some may even find it convenient. I maintain that considering the situation of cryptology at that time and the circumstances connected with it, this choice is more than justified.5 Another discussion will have to be made hereafter on the manner in which the C38m was utilized, especially during the crucial period of our transport activities to North Africa.

5It should not be forgotten that the C38 was used by the U.S. Army under the designation M-209 throughout the war.

3.2. Working on C38m - Encipherment and decipherment.

3.2.1

The C38m produced with its mechanism a polyalphabetic substitution encipherment, by using 26 inverse normal cipher alphabets obtained by shifting one place at a time the normal English alphabet in reverse order. These alphabets are partially shown in Table 1, which can be used to manually perform this type of polyalphabetic encipherment. Each letter of the plain text is substituted with the letter corresponding to it in the secret alphabet used in turn according to a succession prescribed by the key or enciphering keystream.

For instance, utilizing Table 1.

Table 1. Table for the manual polyalphabetic encipherment with normal reverse alphabets.
Cipher Alphabets
plaintext |  | 0 | 1 | 2 | 3 | … | 12 | 13 | 14 | 15 | … | 23 | 24 | 25 | plaintext
0 | a | A | B | C | D | … | M | N | O | P | … | X | Y | Z | a
1 | b | Z | A | B | C | … | L | M | N | O | … | W | X | Y | b
2 | c | Y | Z | A | B | … | K | L | M | N | … | V | W | X | c
3 | d | X | Y | Z | A | … | J | K | L | M | … | U | V | W | d
4 | e | W | X | Y | Z | … | I | J | K | L | … | T | U | V | e
5 | f | V | W | X | Y | … | H | I | J | K | … | S | T | U | f
6 | g | U | V | W | X | … | G | H | I | J | … | R | S | T | g
7 | h | T | U | V | W | … | F | G | H | I | … | Q | R | S | h
8 | i | S | T | U | V | … | E | F | G | H | … | P | Q | R | i
9 | j | R | S | T | U | … | D | E | F | G | … | O | P | Q | j
10 | k | Q | R | S | T | … | C | D | E | F | … | N | O | P | k
11 | l | P | Q | R | S | … | B | C | D | E | … | M | N | O | l
12 | m | O | P | Q | R | … | A | B | C | D | … | L | M | N | m
13 | n | N | O | P | Q | … | Z | A | B | C | … | K | L | M | n
14 | o | M | N | O | P | … | Y | Z | A | B | … | J | K | L | o
15 | p | L | M | N | O | … | X | Y | Z | A | … | I | J | K | p
16 | q | K | L | M | N | … | W | X | Y | Z | … | H | I | J | q
17 | r | J | K | L | M | … | V | W | X | Y | … | G | H | I | r
18 | s | I | J | K | L | … | U | V | W | X | … | F | G | H | s
19 | t | H | I | J | K | … | T | U | V | W | … | E | F | G | t
20 | u | G | H | I | J | … | S | T | U | V | … | D | E | F | u
21 | v | F | G | H | I | … | R | S | T | U | … | C | D | E | v
22 | w | E | F | G | H | … | Q | R | S | T | … | B | C | D | w
23 | x | D | E | F | G | … | P | Q | R | S | … | A | B | C | x
24 | y | C | D | E | F | … | O | P | Q | R | … | Z | A | B | y
25 | z | B | C | D | E | … | N | O | P | Q | … | Y | Z | A | z
plaintext |  | 0 | 1 | 2 | 3 | … | 12 | 13 | 14 | 15 | … | 23 | 24 | 25 | plaintext
Cipher Alphabets

Keystream | 23 | 14 | 00 | 12 | 25 | 02 | 13 | 01 | 15 | 12 | 00
Plaintext | r | i | f | e | r | i | m | e | n | t | o
Ciphertext | G | G | V | I | I | U | B | X | C | T | M
Note: Keystream refers to the number of secret alphabets used.


The same table can be used for decipherment:

Keystream | 23 | 14 | 00 | 12 | 25 | 02 | 13 | 01 | 15 | 12 | 00
Ciphertext | G | G | V | I | I | U | B | X | C | T | M
Plaintext | r | i | f | e | r | i | m | e | n | t | o

3.2.2

The enciphering mechanism was made of:

a) a certain number of metallic elements both in the rotary and fixed parts of the machine: pins, guide-arms, lugs; each one of them could be set by the operator into an active or inactive position, thus forming the so called internal key. This was, in reality, made up of two components called W and X. The number of different internal keys that the operator could set was astronomical, nearly 1060 (a 60 digit number) but the setting was laborious and required much attention and time. The result was that the same internal key was kept in use for a long time.
b) 6 rotors having like diameters, mounted side by side on a horizontal axle, each one rotatable, manually or not, by means of gears having a different number of teeth for each rotor, precisely 26, 25, 23, 21, 19, 17.

On the outer rim of each rotor were carved, in normal order, the letters of the English alphabet, respectively A to Z, A to Z excluding W, A to X excluding W, A to U, A to S and A to Q.

Along the edge of the right face of each rotor were housed respectively 26, 25, 23, 21, 19, 17 pins corresponding to as many letters carved around the rim. As already mentioned each pin could be set by the operator into an active or inactive position.

The number and position of the active pins made up the X component of the internal key, as I said before. The number of different X components that the operator could form was extremely great, 2131 (a 40-digit number), even though many components were not advisable because they were too simple.

Note for Table 2: The external key EATJHP corresponds to position 30,000,000 in the keystream because dividing 30,000,000 by 26, 25, 23, 21, 19, and 17, yields respectively the following remainders:
4( = E), 0( = A), 19( = T), 9( = J), 7( = H), and 15( = P).
Table 2. Generation scheme of a complete keystream relevant to a given internal key. Encipherment of a message with a given external key.
 |  |  | Rotor |  |  |  | Position | Keystream
 | 1 | 2 | 3 | 4 | 5 | 6 | within | (# of cipher
# of teeth | 26 | 25 | 23 | 21 | 19 | 17 | Keystream | Alphabet)
 | A | A | A | A | A | A | 0 | 8
 | B | B | B | B | B | B | 1 | 11
 | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | [vellip]
 | Q | Q | Q | Q | Q | Q | 16 | 17
 | R | R | R | R | R | A | 17 | 2
 | S | S | S | S | S | B | 18 | 11
 | T | T | T | T | A | C | 19 | 12
 | U | U | U | U | B | D | 20 | 1
 | V | V | V | A | C | E | 21 | 18
 | W | X | X | B | D | F | 22 | 2
 | X | Y | A | C | E | G | 23 | 12
 | Y | Z | B | D | F | H | 24 | 8
 | Z | A | C | E | G | I | 25 | 13
 | A | B | D | F | H | J | 26 | 5
 | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | See Table 1
External Key | E | A | T | J | H | P | 30 000 000 | 14 | Plain text | Cipher text
 | F | B | U | K | I | Q | 30 000 001 | 23 | r | G
 | G | C | V | L | J | A | 30 000 002 | 14 | i | G
 | H | D | X | M | K | B | 30 00 0003 | 0 | f | V
 | I | E | A | N | L . | C | 30 000 004 | 12 | e | I
 | J | F | B | O | M | D | 30 000 005 | 25 | r | I
 | K | G | C | P | N | E | 30 000 006 | 2 | i | U
 | L | H | D | Q | O | F | 30 000 007 | 13 | m | B
 | M | I | E | R | P | G | 30 000 008 | 1 | e | X
 | N | J | F | S | Q | H | 30 000 009 | 15 | n | C
 | O | K | G | T | R | I | 30 000 010 | 12 | t | T
 | P | L | H | U | S | J | 30 000 011 | 0 | o | m
 | Q | M | I | A | A | K | 30 000 012 | 3 | v | I
 | R | N | J | B | B | L | 30 000 013 | 14 | o | A
 | S | O | K | C | C | M | 30 000 014 | 24 | s | G
 | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | [vellip] | [vellip]
 | Y | Y | V | T | R | P | 101 405 848 | 13
 | Z | Z | X | U | S | Q | 101 405 849 | 15
 | A | A | A | A | A | A | 0 | 8
 | B | B | B | B | B | B | 1 | 11

Table 3. Generation of encipherment keystream for message in Table 2.
 | Rotors | (1) | (2) | (3) | (4) | (5) | (6) | (0)
W component of internal key |  | w1 | w2 | w3 | w4 | w5 | w6 | w0 | Keystream (# of secret alphabet)
 |  | 5 | 1 | 8 | 2 | 6 | 3 | 2 |
position | 30 000 000 | E | A | T | J | H | P |  |
 |  | +5 |  | +8 | +2 | 6 |  | +2 | = 23
position | 30 000 001 | F | B | U | K | I | Q |  |
 |  | · | +1 | · | +2 | +6 | +3 | +2 | = 14
position | 30 000 002 |
 |  | +5 | · | +8 | +2 | +6 | +3 | +2 | = 26 ≡ 0
position | 30 000 003 |
 |  | · | · | +8 | +2 | · | · | +2 | = 12
position | 30 000 004 |
 |  | +5 | +1 | +8 | · | +6 | +3 | +2 | = 25
position | 30 000 005 |
 |  | · | · | · | · | · | · | +2 | = 2
position | 30 000 006 |
 |  | +5 | · | · | · | +6 | · | +2 | = 13
position | 30 000 007 |
 |  | +5 | +1 | +8 | +2 | +6 | +3 | +2 | = 27 ≡ 1
position | 30 000 008 |
 |  | +5 | · | +8 | · | · | · | +2 | = 15
position | 30 000 009 |
 |  | · | +1 | · | · | +6 | +3 | +2 | = 12
position | 30 000 010 |
 |  | +5 | · | +8 | +2 | +6 | +3 | +2 | = 26 ≡ 0
position | 30 000 011 |
 |  | · | +1 | · | · | · | · | +2 | = 3
position | 30 000 012 |
 |  | · | +1 | +8 | · | · | +3 | +2 | = 14
position | 30 000 013 | R | N | J | B | B | L |
 |  | ↑ | ↑ | ↑ | ↑ | ↑ | ↑ | ↑ | Keystream
 |  |  | X component of internal key |  |  |  |  |
 |  |  | + pin in active position · pin in inactive position |  |  |  |  |  | (See Table 1 for secret substitution alphabets.)


3.2.3

Manually rotating each of the 6 rotors, their respective letters could be aligned along a reference line made up of 6 windows on the machine cover. This alignment executed by the operator was the so-called external key, which was to be formed before the beginning of the encipherment of the message. During the letter by letter enciphering operation the 6 rotors advanced simultaneously by one step at each stroke (see Table 2). Thus to encipher each struck letter one of the 26 secret substitution alphabets of the above cited Table 1 was used, according to a succession called worm (keystream), which was only apparently chaotic. It actually depended on the formation law generated by the two components W and X of the internal key as set by the operator.

To better demonstrate the effect of the combined W and X components of the internal key, it will be useful to make an example, even though somewhat simplified (see Table 3).

We have already described the X component; the 131 (26+25+23+21+19+17) pins partly extended and partly retracted. The W component6 was in reality the sum of 7 components, one for each rotor (w1w2w3w4w5w6) and one (w0) independent of the rotors. When the operator had to set the internal key, he could assign to w0 one of the values 1 or 2 and to each of the other w's the values from 1 to 9.7 The sum however must always be 27. Let's consider the alignment which, in Table 2, corresponds to position 30,000,000 of the keystream generated by the internal key as set by the operator whose W component is shown below (see Table 3):

6For brevity we will omit the description of the quite complicated mechanical structure of the W component. We will only say that it was a cylinder with a number of beveled lugs and guide arms which could be positioned in active or inactive position by the operator.
7What we have called for simplicity values 1 or 2 and values from 1 to 9 were in reality the number of guide arms set in active position by the operator (see Footnote 6 above).
rotors | 1 | 2 | 3 | 4 | 5 | 6 | 0 |
W component | 5 | +1 | +8 | +2 | +6 | +3 | +2 | = 27
alignment | E | A | T | J | H | P |  |
(external key) |  |  |  |  |  |  |  |

According to the X component of the preset internal key, the 6 pins housed in the 6 rotors below the 6 letters of the given alignment (and the subsequent alignments caused by the enciphering operation) could be active or inactive.

The active pins interacted with their respective w while the inactive ones left their respective w ineffective.

The w0 was independent from the rotors, then always active. At each simultaneous advancement of the 6 rotors the internal mechanisms of the machine brought into play the secret substitution alphabet having the serial number (see Table 1) equal to the sum, modulo 26, of the active w's. This alphabet enciphered the struck letter. This stroke also caused the advancement of the rotors.

Now we can see, from Table 3, how that portion of the keystream which enciphered the text shown in Table 2 has been generated. As to the external key, we have already seen that it consisted in the alignment made by the operator before beginning the encipherment, during which the 6 rotors advanced simultaneously one step at a time. Since a complete turn of each of the 6 rotors involved a different number of steps: 26, 25, 23, 21, 19, 17, a given alignment set at the beginning presented itself identically only after 26 x 25 x 23 x 21 x 19 x 17 = 101, 405, 850 simultaneous steps of the 6 rotors (see Table 2). This was the total length of the keystream as generated by the internal key. Obviously each possible internal key generated a total keystream of identical length but having a different structure (i.e. succession of secret alphabets).


3.3. Decryption of the C38m

3.3.1

Let us resume the encipherment example shown in Subparagraph 3.2.1 and in Table 2:

Keystream | (V) | 23 | 14 | 00 | 12 | 25 | 02 | 13 | 01 | 15 | 12 | 00 | 03 | 14 | 24
Plaintext | (C) | r | i | f | e | r | i | m | e | n | t | o | v | o | s
Ciphertext | (S) | G | G | V | I | I | U | B | X | C | T | M | I | A | G

Let us substitute for each letter the number indicating the position it occupies in the normal alphabet (A = 0, Z = 25 see Table 1) and let's transcribe everything (see Table 4, Section I). We can observe that each secret letter can be obtained by subtracting algebraically, modulo 26, the plaintext letter from the relevant keystream element (V - C = S):

23 - 17 = 6, 14 - 8 = 6, 0 - 5 = -5 + 26 = 21; … ; 2 - 8 = -6 + 26 = 20; etc.

* Fig.
Table 4. Example of decryption of two cryptograms with same keystreams and recovery of relevant portion of keystream by the probable word method.

Now the enciphering process is identical with the superencipherment operation in the case of numerical ciphers described in Subparagraph 2.1, with the enormous advantage for the codebreaker that in this case there is no secret code. One may say that the code is so well known to him that he knows it by heart (if he masters the language).

3.3.2

This being the situation, the decryption of two messages enciphered with the same keystream is no longer a serious problem. In fact, if one can assume the existence in one of the texts, of probable words or phrases (this often happens especially in the initial and final portions of the messages), one can directly verify the validity of the hypothesis on the other text (see Table 4, Section II). Since S1 - S2 = C2 - C1 then S1 - S2 + C1 = C2 and S2 - S1 + C2 = C1 as was said in Subparagraph 2.1. Usually a few trials are sufficient to start a decryption which can be continued step by step without great difficulty, thus furnishing at one time the elements of the keystream portion which has enciphered the two messages: V - S2 = C2 then S2 + C2 = V (see Table 4, Section III).

3.3.3

After completing the decryption of the two messages (even though incomplete) the cryptanalyst has at his disposal a good portion of the keystream generated by the internal key set by the operator. At this point no assumptions would be possible as to the remaining keystream portions, in case of numerical ciphers superenciphered with random or incoherent keystreams (see Paragraph 2). In the case of C38m a skilled cryptanalyst was able, if he had a deep knowledge of the machine, to recover the whole keystream (101,405,850 elements) after reconstructing the W and the X components of the internal keys (see Table 3). He could do this by writing and solving equations with 1 to 6 unknowns or empirically by successive trials. Usually it is sufficient to have about 100 keystream elements to reconstruct the internal key. In luckier instances about 40 elements can suffice.

It should be noted that the keystream so obtained is identical with the keystream generated by the encipherer's machine but it is shifted with respect to it because the element which has arbitrarily been assigned the 0 position (external key AAAAAA) by the solver actually occupies an unknown position between 0 and 101,405,849 in the original keystream. The shifts will be initially different for different internal keys but it will be possible to gradually bring them to a common arbitrary shift.


3.3.4

Once in possession of the internal key and consequently of an entire keystream, the British cryptanalysts would have been able to decrypt all the messages enciphered with that internal key and with any external key without being disturbed by the above said shift. Of course, owing to the length of the keystream, the number of necessary trials was so great that it is impossible for me to hazard a statement as to the possible efficiency of this work, which depended on the quantity of people employed and, above all, on the substantial aid of special ad-hoc equipment which the joint British-American technology was in a position to produce in 1941.8 Such decryptions were in general timely only on occasion and certainly the most productive work in terms of quantity and timeliness was that of attacking each message directly by assuming the existence of probable words or phrases. These, if confirmed, would furnish a portion of keystream which was not difficult to position within the total keystream, i.e. identify the (fictitious or arbitrary) external key set by the enemy operator and then decrypt the whole message. Table 5 shows an example of determination of the external key of a message (S), with known internal key X and W, starting from a portion of keystream (V) obtained by assumption on the beginning of plaintext (C):

8 B.I. repeatedly mentions the use of such equipment (the electro-mechanical Bombes; the Colossus, an electronic computer) in the penetration work of enciphered Enigma radio traffic. We also had developed a simple machine to accelerate the work against the British naval superenciphered codes in 1942. Using punched cards it would perform the many trials necessary to detect which portions of a long keystream, already recovered, had been used to superencipher other messages still to be decrypted.
(S) | G | H | R | V | O | L | J | Q | W | N | X | A | REUHLDREYUY
 | 06 | 07 | 17 | 21 | 14 | 11 | 09 | 16 | 22 | 13 | 23
+ (C) | s | u | p | e | r | m | a | r | i | n | a
 | 18 | 20 | 15 | 04 | 17 | 12 | 00 | 17 | 08 | 13 | 00
 | 24 | 27 | 32 | 25 | 31 | 23 | 09 | 33 | 30 | 26 | 23
 |  | -26 | -26 |  | -26 |  |  | -26 | -26 | -26
= (V) | 24 | 01 | 06 | 25 | 05 | 23 | 09 | 07 | 04 | 00 | 23

* Fig.
Table 5. Recovery of External Key (LBGFA pos. 30 082, 401) and consequent solution of a message by the "probable word" method.

It may seem that such a method would depend on chance only, but when cryptanalysts have a deep experience and knowledge of the message subject matter, the encipherers' habits, the way the texts usually start and end, the stereotyped phrases etc., then the above method may become the normal routine method and yield timely solutions too.

3.3.5

Two or more messages enciphered with an identical keystream can be immediately identified among the intercepted traffic if the indicator groups of the external key used are not masked, as in the example of Table 4; otherwise like portions of ciphertext in different messages must be found at the beginning and at the end of the secret message.

3.3.6

If no cases exist of keystream reuse,(this is very possible and actually probable when the adversary has issued careful directions for the use of the machine), one can attack the messages one at a time by the probable word or phrase method until one has some portions of keystream sufficiently reliable and long enough to permit reconstruction of the internal keys and proceed as in Subparagraph 3.3.3. and 3.3.4.

3.3.7

To sum up, it is clearly shown from the above that the C38m enciphering machine lent itself to incomparably greater chances of decryption than the naval superenciphered code described in Paragraph 2.


3.4. Usage of the C38m by the Italian Navy during the war.

3.4.1

I will state in advance that the contents of this paragraph are based both on what the British say in B.I. and on what I could come up with by consulting a few documents of that time still available in the archives of the Naval Staff.

It should be kept in mind that in contrast with what has been repeatedly said and written, the choice, the production, the regulation, the distribution of the cryptosystems and the control over their utilization were not made by the SIS Cryptographic Branch. These were performed by the Telecommunications Branch (TLC). This was, in my opinion, a considerable fault of our secret communications organization. The Cryptographic Branch only furnished consulting when so required. I know with certainty that this was done and was decisive for the wartime enciphering-superenciphering system's security. I do not know, however, whether that was true for the C38m or to what extent it was done. I believe that direct or indirect co-responsibility for our Cryptographic Branch cannot be excluded.

3.4.2

The regulations of the time did not make a neat distinction between the material that could be enciphered with the superenciphered naval ciphers and that which could be enciphered with the C38m. The two systems were practically considered alternates and the guiding criterion for the commanders probably was, or in the long run became, that of greater practicality and speed, clearly favorable for the C38m. One can therefore think that in the crucial circumstances of more intense operations or radio traffic, those who had both systems, often preferred, perhaps to an excessive extent, the mechanical encipherment to the manual encipherment with cipher book and superencipherment tables, which was safer but more laborious and slow.

3.4.3

According to the distribution lists contained in the documents still available, the authorities who were issued the C38m were very few9: only High Commands on shore and on board (see Table 6 and Subparagraph 3.4.5), and no single surface or submarine unit.

9They were few in comparison with the number of users of the general superenciphered naval cipher, which had been issued to almost every naval unit and minor Navy Command.
Table 6. Quantities of External Keys Assigned to the Different Commands.
2nd edition in force March 1942 |  |  | 3rd edition (April 1942) enforced
 |  | R.P. | in May/June 1942
Supermarina e M.M. | 3 500 | 300 | Supermarina | 1 200
Reparto Comunicazioni |  | 200 | Reparto Comunicazioni | 1 800
Com. FF.NN. e Com 1 Sq. | 500 | 100 | Com. FF.NN. e Com 1 Sq. | 300
Com. 2 Sq. e Com 5 Div | 300 | 100 | Com. 2 Sq. e Com 5 Div | 200
Com. 3 Div. | 200 | 100 | Com. 3 Div. | 100
Com. 7 Div. | 200 | 100 | Com. 7 Div. | 100
Com. 8 Div. | 200 | 100 | Com. 8 Div. | 100
Com. 9 Div. | 200 | 100 | Com. 9 Div. | 100
Marina Napoli | 200 | 100 | Marina Napoli | 200
" Taranto | 200 | 100 | " Taranto | 200
" Messina | 200 | 100 | " Messina | 200
" Brindisi | 200 | 100 | " Brindisi | 00
" Albania | 500 | 100 | " Albania | 500
" Morea | 1 500 | 100 | " Morea | 1 200
" Sud est | 800 | 100 | " Sud est | 800
" Egeo | 800 | 100 | " Egeo | 1 200
" Lero | 300 | 100 | " Lero | 600
" Libia | 2 000 | 100 | " Libia | 1 500
" Tripoli | 1 000 | 100 | " Tripoli | 800
" Bengasi | 1 000 | 100 | " Bengasi | 800
" Tobruk | — | 100
Reserve | 1 700 | 600 | Reserve | 2 900
Total 15 000 |  | 3 000 | Total 15 000

I cannot exclude, however, that in some particular cases some single unit may have been temporarily given the machine for use with assigned keys.


3.4.4

The astronomical number of possible internal keys, around 1060, theoretically offered very good security, which could not be practically utilized because the setting of the internal keys was delicate, laborious and long. For the X component alone, one had to manipulate 131 pins. This could be not be done as frequently as desired; therefore it was decided to use internal keys common to all users and change them each month. This rule was in force at least throughout 1942, but probably until the armistice. It was certainly acceptable during the first war period, with C38m radio traffic not intense (B.I. says: "during the month of August 1941, 600 signals")10. In my opinion it was unsafe to keep it when the traffic was gradually growing (B.I. says: 4,000 messages during the most intense month, July 1942). In that situation the preference was to increase the number of external keys available, switching from 18,000 to about 40,000.

10B.I. vol 2 page283.

3.4.5

It was understood that the greatest danger was in the reuse of the keystream, that is two or more messages enciphered with the same external key, within the same month. For this reason this practice was positively forbidden and all authorities issued a C38m were assigned series of keys not to be used by other authorities. Of course I cannot exclude that violation of the rules may have existed but this must have been the exception, considering the large number of keys available.

In Table 6 a tabulation is given of the external keys assigned in the second and third edition of the so-called Table of External Keys for Cipher Machine.

I found no trace of the first edition. The second edition, enforced on March 1, 1942, contained 15,000 normal keys plus another 3,000 for the R.P. messages (Restricted-Personal). The third edition introduced in May or June of the same year contained the same number of normal keys and presumably another 3,000 for the R.P. On October 1, 1942 new editions were implemented with a total of 40,000 keys counting both normal and R.P. These quantities were, in my opinion, judiciously commensurate with the foreseeable radio traffic intensity for the C38m in a month, i.e. within the life span of the internal key.

3.4.6

All right so far; but let us see how the external keys were chosen, then we shall see how the utilized key was indicated. As we have seen in the preceding paragraph, the number of external keys theoretically possible were 101,405,850, but it was deemed necessary to avoid the possibility that the initial part of a message should be casually enciphered with the portion of keystream which had or would later encipher the tail of another message.

This was meant to avoid providing our opponent with a reusage of keystream, even partial. For that purpose, since a maximum length of 500 letters was prescribed for each message, the external keys, within the total keystream generated by the internal key, had to be distanced by a number of elements greater than 500. An average distance of about 1,000 elements would have been abundantly sufficient while not reducing the quantity of available keys. This would have reduced the number of possible keys to a still reasonable amount, i.e. about 100,000 (101, 405, 850/1,000).

A greater interval was chosen instead and this decreased the available external keys to about 40,000. Obviously, it was thought with good reason, this number was more than sufficient to cover the requirements of one month. The above reduction would have brought no appreciable advantage to the British cryptanalysts. After recovering, by lucky decipherments, the month's internal key, they would still have to try all the 100 million possible external keys to find the external key of any other message during that month. I have used the conditional because unfortunately a fixed number of elements (2497) was prescribed by our planning. This being the situation, our adversary, after identifying some external keys (arbitrary) by means of some decryption of intercepted messages within that month, could observe that they were all at the same distance of a multiple of 249711 in the total keystream. At this point he was able to identify, by means of a simple calculation, all the 40,611 (arbitrary) external keys prescribed by our plans, thus reducing the trials necessary to decrypt every other message in that month12 from about one hundred million to only about 40,000 in the worst case13. I cannot believe that the British overlooked this unsecure feature of our planning; they must have noticed it fairly early. They certainly made great use of it to decrypt those messages believed to be important which had resisted the direct attack by the probable word or phrase (see Subparagraph 3.3.5) method.

11Table 7 shows some external keys chosen at random from our Tables of External Keys in use in 1941/2 with their positions in the total keystream, the partial distance and their prime factors.
12Something similar happened during our work against the British superenciphered naval ciphers. We found out that the keystream extracted from like blocks of superencipherment tables were always at a distance of a multiple of 20 elements (5 tetragraphs). Thus we could reduce the necessary trials by 80%.
13Actually the cryptanalyst did not know the position in the original keystream where our breakdown of 40,611 keys with the interval of 2,497 was started. Since 101,405,850 is not a multiple of 2,497, the necessary trials were in the worst case 80,000 to start with. When the number of identified keys increased, the trials decreased to a maximum of 40,000 in the worst case.

* Fig.
Table 7. Determining the planning scheme for the external keys.


3.4.7

Let us look at the indicator groups, or those agreed-upon groups that the encipherer placed as first and last groups of the cryptogram to communicate to the receiver which external key had been set on the machine before starting the encipherment of that message.

The books containing the Tables of External Keys had a certain number of pages, subdivided among the different users, on the basis of the quantity of keys assigned to each user. (See Table 6.)

Each page contained a number of keys chosen at random, not repeated, among the 40,611 of the general plan. They were listed in rows and columns. The encipherer would each time select within the relevant pages the key he was to use and checked it off in pencil to avoid reutilization before checking off all the keys at his disposal. After finishing the encipherment he would place an indicator group as first and last group of the cryptogram. By an appropriate index (not as explicitly as we have shown in Table 4, Section II for simplicity), he would furnish the receiver page, column and row numbers pertaining to the key.

Obviously with this procedure only those who had the above mentioned secret books were in a position to interpret the indicator groups, i.e. know the external key used in the messages14. With a certain imprudence, up to the fall of 1942 the indicator groups were used without masking in such a way that like indicator groups meant like external keys to the adversaries. This imprudence was similar to that committed by the British in superenciphering their naval ciphers (see Paragraph 2.3.e) which proved much more dangerous. It was probably caused by the confidence that the rules positively prohibiting reuse of the keys were strictly adhered to. Anyhow, let us see which consequences could come from not masking the indicator groups, to the advantage of the British cryptanalysts. We will list the different situations:

14The above statement about the structure and function of the indicator groups has been derived by the examination of second and subsequent editions of Tables of External Keys in force after March 1, 1942. I cannot say anything about the structure and function of the indicators used before that date because I found no trace of the first edition of these Tables in the Navy Staff Archives.
a) indicator groups used only once. No usefulness to the decryptors;
b) indicator groups used twice during a given period of validity of the external key (i.e. within the month). This was the typical case of two cryptograms enciphered with the same keystream: very serious consequences (see Sub-paragraph 3.3.2 through 3.3.5 and Table 4). However, I repeat, such a circumstance should not occur and if it rarely happened, this does not justify, in my opinion, the systematic and timely penetration alleged by B.I.;
c) indicator groups used twice during two different months, i.e. with different internal keys, i.e. not with the same keystream.
This latter event, also forbidden by the regulations, could present itself in some months' time when the keys assigned to a specific user were exhausted (however reserve keys were provided, see Table 6).
This fact could be to some advantage, if the two messages were decrypted, to match the total keystream relevant to the two recovered internal keys (see Subparagraph 3.3.3) and then utilize, at best, other possible reemployment of the indicator groups between the two relevant months.

With the new issue of the Tables of External Keys adopted on October 1, 1942 a good procedure for masking the indicator groups was finally implemented. This makes one think that during the summer of 1942, a period of maximum C38m radio traffic intensity, some violations of the rules may actually have been made and discovered by the control service performed by the Communication Branch of the Naval Staff. One can infer that serious breaches may not have previously happened since the non-masking of the indicator groups was maintained in the editions enforced in March and in May/June 1942.

3.4.8. Plaintext language of the secret messages.

In my opinion this was another weak point of our C38m radio traffic. It may have progressively helped - after an initially laborious and lucky penetration - the British cryptanalysts when they acquired experience and became familiar with the subject matters and the fashion in which they were discussed in the messages.

Stereotyped beginnings and endings, habits of the encipherers and other details may have allowed the decryptors to guess probable words and phrases (as mentioned in Subparagraph 3.3.2) with which to attack single messages, without waiting for a keystream to be reused, a very rare occurrence. During the period of maximum radio traffic intensity this may have been the more generalized system to widely penetrate our C38m communications.

3.4.9. Mistakes made by the encipherers.

In 1978, the noted British writer, Ronald Lewin, wrote in his work Ultra Goes to War. "Italian cipher-clerks were too often haphazard and negligent, whilst the cryptanalysts of Vincent's section - as elsewhere at BP - developed a rigorous and effective professionalism. And so, for example, it became almost customary to pass on advance information to British headquarters in the Middle East about the make-up, content, port of departure and port of arrival for the Axis convoys that transported supplies and reinforcements to Rommel's army in Africa."15.

15The fact that the British had largely penetrated our secret communications relevant to the ship and convoy traffic to North Africa was given out by some daily press as an unheard of discovery or revelation in 1981. It had been known for at least three years.

Nothing is surprising in Lewin's statement about pretended carelessness of our encipherers; this has always happened, in peace time and even more so in war time, to us and to others. I must stress that in the case of C38m, such negligence as the reuse of keystreams or other typical errors, which I will not enumerate, help the cryptanalysts in an often decisive way. This help cannot be compared with similar circumstances that may occur with more complex systems such as, for instance, the superenciphered ciphers.


3.5. British penetration of the Italian Navy's C38m.

3.5.1

The C38m, like other Hagelin machines which had preceded it in the market for two decades utilizing for the encipherment the same original concept devised by the inventor and constructors, was not unknown to the British. They did not make the confusion found in a book published last year where the Italian author has written that the Hagelin machine was a German ciphering machine firm which had made a variant of the Enigma machine for the Germans. This is an error: the firm was Swedish with a plant in Stockholm and its proprietor, a businessman, also Swedish, was not even pro-German. Just after the Nazi occupation of Norway he had adventurously crossed Germany and Italy with the dismantled parts of the last version of this machine (almost identical with the C38m) hidden in his luggage. From Genoa he reached the United States which he had visited a few years before and he immediately placed about 50 units with the United States armed forces. Back in 1935, as mentioned in Paragraph 3.1., he had supplied the French armed forces. The machine must have been well known to the British who keenly studied it from the cryptographic security angle. It is not unlikely that the diffidence which was shown up to November 1943 - as stated in B.I. - by the Royal Navy, in contrast with the other armed forces, toward the ciphering machines in general may have originated or been strengthened precisely after the study of the Hagelin (and Enigma) machines. One may suppose, with grounds, that owing to the alliance with France and the tight bonds with the Americans, some specimens very similar to C38m may have been in their hands at the beginning of the conflict.

Nothwithstanding this markedly advantageous situation, the British, after observing the use of the C38m in the secret radio communications of the Italian Navy, did not obtain any results for a good six months, from December 1940 to June 1941. I have heard someone's opinion that the Royal Navy had not concerned itself with this problem during those months.

I cannot wrong our adversaries of that time. In fact - according to B.I.16 - "after the end of July 1940 the black-out of SIGINT from Italian naval communications was broken only by fitful gleams from the systems of the lower grades and by the first success of GC and CS (Government Code and Cypher School) against an Italian naval machine cypher ENIGMA, broken in September 1940. Unfortunately, it carried only one or two messages a day up to the summer of 1941, when it was withdrawn from naval use."

16B.I. vol 1, page 210.

It is unlikely that in the darkness only broken by those few Enigma messages, and with the persistent failure of any attempts to penetrate our superenciphered naval ciphers, the British Intelligence service should have neglected for so many months trying to break those mechanical ciphers of ours that must have been regarded by their specialists as vulnerable, thanks to their knowledge, albeit theoretical, of the Hagelin machine.

3.5.2

Let's see what the British say. In vol. 2 of B.I. page 22, one can read: "The Italian C38m, which had first been intercepted in December 1940, was broken by GC and CS since the summer of 1941," and on pages 283-284: "By the end of June 1941 GC and CS had broken the C38m settings for May and June and was reading a few messages currently. The first intelligence from the source for the Middle East was sent out on June 23 …. On July 10 GC and CS broke the July settings. Thereafter, with routine and progressively shorter interruptions at the end of each month, when the Italians introduced new settings, it was going on reading the traffic with little or no delay until the Italian armistice. By the end of 1941 the monthly interruption had been reduced to less than three days; by the end of 1942 it was less than 24 hours."

Thus between December 1940 and June 1941;

a) there were no cases of keystream reutilization, which is reasonable owing to the positive prohibition by our rules, especially in relation to the still sluggish C38m cipher traffic; or
b) if some reutilization occurred, the British did not manage to avail themselves of it, which is not likely;
c) having failed to decrypt our messages, the British cryptanalysts acquired neither such experience as to apply the probable word or phrase method (see Subparagraphs 3.3.4 and 3.3.6) nor the knowledge of the weak point in our external keys planning (see Subparagraph 3.4.3).


Nothwithstanding all this, at the end of June 1941 the British cracked the C38m so as to recover the internal keys for the previous month, reconstruct the internal keys for July in ten days and from that moment onwards began a systematic and timely decryption "…to give from July 1942 the operational authorities advance notice of virtually every convoy and independent ship that sailed with troops or supplies across the Mediterranean, and usually to identify the ships and escorting forces involved."17

17B.I. vol 2, page 284.

What sudden flash had enlightened the minds of the British cryptanalysts toward the end of spring 1941? Frankly such a rapid and substantial jump in quality and quantity puzzles me. Even more so if I consider what had happened at the beginning of March 1942, i.e. the serious interruption of the decryption exactly when the situation for the British cryptanalysts was supposed to be better with respect to June 1941, owing to the much greater intensity of our C38m traffic (B.I. reads: "600 messages in August 1941, 4000 in July 1942") and the experience acquired in the preceding 10 months of intensive decryptions. In fact B.I. Vol 2, page 348 reads: "At the beginning of March 1942 the Italians introduced a new indicator for the C38m signals and GC and CS temporarily read only a small portion of the traffic and with considerable delay. At the end of March GC and CS regained its ability to read C38m currently." What the British thought was an introduction of new indicators, was the implementation of our second edition of the Tables of External Keys (see Subparagraph 3.4.5.) where a good number of keys were also contained in the first edition, since they were all taken from the series of 40,611 keys envisaged by our planning.

If the appearance of new indicators had caused such a serious interruption in the decryptions, one must conclude that previously the indicators (those of the first edition) had been precious for the penetration work. We have seen (Subparagraph 3.4.7 c) that the indicators could be of some limited use for the cryptanalysts only if they had previously appeared in decrypted messages. This was not the case in March 1942 because the indicators were new. One can understand the crisis which followed to some extent. It was not the case in June/July 1941 either, because there were no messages previously decrypted. Still the British penetration of C38m was begun and suddenly developed with great success at that time. These considerations make one think that upon the discontinuance of the first edition of the Tables of External Keys on March 1, 1942, the British must have missed some detail that had to do with the indicators. This detail had permitted a particularly profitable type of work which had facilitated the initial cracking and the subsequent systematic and timely utilization of it from 1941 through February 1942.

The sudden lack of this detail must have forced the British to devise some different working method, which required almost a month's running in. This method must have been independent from the consideration of the indicators. This seems to be proved by the fact that the next change of the indicators accompanying the third edition of our Tables of External Keys, implemented in May or June 1942, does not seem to have caused any interruption in the decipherments. B.I. only mentions one or two serious interruptions, that of March 1942 and that considerably less serious of October 1942, when we introduced the masking of the indicators with our fourth edition (see last section of Paragraph 3.4.7). I am not in a position to say with certainty what this detail connected with the indicators of the first edition of our Tables of External Keys consisted of. Perhaps before March 1942, with the first edition of said Tables, the indicators, instead of furnishing the key indirectly, through an index as described in Subparagraph 3.4.7., furnished it explicitly, or with some simple disguise that the British ended up discovering after some months' attempts.

As I have already said, I have found no trace of the first edition of the Tables of External Keys in the Archives of the Navy Staff, therefore I cannot verify this hypothesis.

CONCLUSION

From what I have explained, it is confirmed, according to my conviction, that during WW II the Italian Navy's Cryptographic service had nothing to envy the British counterpart from the strictly technical viewpoint, and allowing for the different degrees of difficulty and vulnerability of the opposed systems. The contrary is true. Had I to reexamine the banal chart shown at the beginning of this article I would revise it as below.

 |  | Italy | Great Britain
 | Naval ciphers | +++ | -
Attack
 | Cipher machine |  | +
 | Naval ciphers | +++ | - - -
Defense
 | Cipher machine | - - -

I wish to make it clear that the above oversimplified evaluation has no absolute value for a comparison with our opponents of that time. I cannot overlook the vast and efficient British penetration of the radio communications enciphered by the German Enigma. That problem comprised a greater degree of difficulty than the C38m. I lack, for the time being, sufficient information to make a critical treatment of that interesting subject.


BIOGRAPHICAL SKETCHES

Luigi Donini, a retired Admiral, served the Italian Navy on surface ships and submarines. For a long time he served as executive in the Cryptographic Branch of Naval Intelligence, largely decrypting, above all, before World War II, French naval and diplomatic codes (CST, BDG, and TBM) and British naval codes (Fleet code, Aux. code, and Nav. cipher 1). During the War he worked on the principal British naval ciphers (Nav. ciphers 1-2-3, Nav. codes 1-2). These performances earned him the Second Class Gold Medal for works of interest to the Navy.

Augusto Buonafalce was born in Livorno (Leghorn) in 1930 and attended high school there. He studied Economics at Pisa University and worked in electronic data processing during the 1960's. At that time he became interested in information theory and efficient codification in general. He now heads the Patents and Licensing office with an Italian State oil concern. He was "hooked" into cryptography by David Kahn's book, The Codebreakers. He is a member of the American Cryptogram Association and regularly submits solutions of ciphers to the ACA publication The Cryptogram. Doing research on cryptology is one of his many hobbies, which also include archaeology and ancient music.

Pearl Harbor And The Inadequacy Of Cryptanalysis

David Kahn

ADDRESS: 120 Wooleys Lane, Great Neck NY 11023 USA.

ABSTRACT: American codebreakers achieved remarkable success in solving Japanese codes, but could not prevent the surprise attack on Pearl Harbor because the Japanese never transmitted anything about the attack on the airwaves.

KEYWORDS: Pearl Harbor, PURPLE, JN25, Rowlett, Friedman, Grotjan, J19.

Fifty years later, Pearl Harbor still haunts Americans. The nation remains mystified about how Japan's surprise attack on the proud Pacific Fleet could have succeeded. The joint congressional committee that in 1945 and 1946 investigated the attack put the question sharply:

Why, with some of the finest intelligence available in our history, with the almost certain knowledge that war was at hand, with plans that contemplated the precise type of attack that was executed by the Japanese on the morning of December 7 — Why was it possible for a Pearl Harbor to occur?1

1U.S., Congress, Joint Committee on the Investigation of the Pearl Harbor Attack, Investigation of the Pearl Harbor Attack, Report, 79th Congress, 2d Session (Document No. 244) (Washington: Government Printing Office, 1946), 253.

The "finest intelligence" came from codebreaking. This is nearly always the best form of intelligence. It is faster and more trustworthy than spies, who have to write up and transmit their reports and who are always suspected of setting up a deception. It sees further into the future than aerial reconnaissance, which detects only what is present. It is broader in scope than the interrogations of prisoners, who know little more than what they have experienced. And it is usually cheaper and less obtrusive, hence more secret, than all of these. But it has a serious double-barreled failing. It cannot provide information that a nation has not put onto the airwaves, while its apparent omniscience and its immediacy seduce its recipients into thinking they are getting all the other nation's secrets.

This is one of the lessons of Pearl Harbor. American codebreakers performed prodigies, giving remarkable insight into Japanese thinking. But that insight was not total, and so even the extraordinary U.S. cryptanalysis could not warn policymakers of Japan's secret intentions.

This article attempts, therefore, two things: to provide information about the codebreaking leading up to Pearl Harbor that has become available since the 1967 publication of The Codebreakers, and to explain why even the voluminous information that cryptanalysis provided did not prevent surprise.


After World War I, which taught or retaught the great powers the value of cryptanalysis, they extended or refounded their codebreaking agencies to provide in peace the benefactions gained in war.2 The United States was one of these. In 1919, the head of the wartime Section 8 of the Military Intelligence Division of the Army General Staff, or MI-8, persuaded officials to establish a joint State Department-War Department codebreaking agency, formally called the Cipher Bureau but later widely known as the American Black Chamber. He was Herbert O. Yardley, 30, short, balding, and charismatic; he set up shop in New York. The Navy, which after a short-lived effort by naval intelligence had let MI-8 do its codebreaking during the war,3 talked about organizing its own agency4 but in fact did nothing in cryptanalysis until 1923. In that year, naval intelligence rifled the steamer trunk of a Japanese naval office in New York, yielding a 1918 Japanese naval codebook.5. This trove inspired the Navy to establish a codebreaking agency6 under the director of naval communications, calling it the Research Desk for security.

2"Codebreaking in World Wars I and II: The Major Success and Failures, Their Causes and Their Effects," in David Kahn, Kahn on Codes: Secrets of the New Cryptology (New York: Macmillan, 1983), 102-103; David Kahn, Seizing the Enigma: The Race to Break the German U-Boat Codes, 1939-1943 (New York: Houghton Mifflin, 1991), 82-85.
3J. S. Holtwick, "Naval Security Group History to World Was ii," Appendix I, p. 003, SHR-355, Record Group 457, National Archives (since all SRH documents are from this record group, that designation is omitted in other references); William F. Friedman, "A Brief History of the Signal Intelligence Service," SRH-029, p. 01.
4SRH-355, p. 026.
5Jeffery M. Dorwart, Conflict of Duty: The U. S. Navy's Intelligence Dilemma, 1919-1945 (Annapolis: Naval Institute Press, 1983), 39-40, giving March 1923 as the terminus ad quem; I have been unable to find the report in the source given by Dorwart, apparently through some error; Laurance F. Safford, "A Brief History of Communications Intelligence in the United States," pp. 3-12 in Ronald H. Spector, ed., Listening to the Enemy: Key Documents on the Role of Communications Intelligence in the War with Japan (Wilmington, Delaware: Scholarly Resources, 1988), 6, giving date as "about 1922" Ladislas Farago, The Broken Seal: The Story of "Operation Magic" and the Pearl Harbor Disaster (New York: Random House, 1967), 36-37, giving 1920 as the date. Farago's source is the memoir of Rear Admiral Ellis M. Zacharias, Secret Missions (New York: Putnam, 1946), which Farago co-authored
6L. F. Safford, "The Undeclared War: 'History of R[adio]. I[ntelligence].' 15 November 1943," SRH-305, p. 03.

Its first head was Lieutenant Laurance F. Safford, who thus became the father of U. S. naval cryptography. Fifteenth in the Annapolis class of 1916, "Saffo" — rumpled, with wavy blond hair that seemed permanently on end ("as if he had been scratching his head in perplexity," a staff member said), whose speech emerged in disconnected bursts — had a flair for mechanics and mathematics and a love for cryptology,7 though his abilities lay more in making ciphers and cipher machines than in breaking them. He began work in January 19248 with four civilians in Room 16219 of Main Navy, a temporary building on what is now Constitution Avenue, near the Lincoln Memorial. One of the first things he did was to start setting up radio intercept stations in the Pacific to furnish more material for codebreaking than was provided by the haphazard monitoring by ships and the naval radio station in Shanghai.10

7Edwin T. Layton with Roger Pineau and John Costello, "And I Was There": Pearl Harbor and Midway — Breaking the Secrets (New York: Morrow, 1985), 32.
8Russell Willson, "The Birthday of the Naval Security Group," in A History of Communications Intelligence in the United States, ed. Graydon A. Lewis (Denver: Naval Cryptologic Veterans Association, 1982), 41.
9Layton, 33
10Safford, "A Brief History," 5; Willson, 41.

Safford concentrated on Japan because it was generally felt that she constituted the greatest danger to the United States. She had, not many years before, defeated China and then Russia to become mistress of the western Pacific. Now she was building a fleet to match America's and had occupied, under a League of Nations mandate, islands that menaced the ocean routes to the Philippines. Japan was a main concern of America's foreign policy, a main focus of America's naval war planning,11 a main target of America's codebreaking. Yardley, for example, sent Japanese cables that he had cryptanalyzed to U. S. diplomats at the Washington disarmament conference of 1921-22. This gave them knowledge of Japan's negotiating position that they used to drive her to a lower capital-ship ratio with the United States than she wanted, costing her the equivalent of three battleships.

11Edward S. Miller War Plan Orange: The U. S. Strategy to Defeat Japan, 1897-1945 (Annapolis: Naval Institute Press, 1991).


Soon thereafter, Safford took one of his most important strides in building up his agency when, in August 1924, he hired Mrs. Agnes Meyer Driscoll, 32, as a cryptanalyst. Tall, slender, secretive, a graduate of Ohio State University, she had taught mathematics and music in Texas, had worked in cryptography for the Navy, had studied cryptanalysis at the Riverbank Laboratories, had worked for Yardley's Cipher Bureau for five months and then for cipher-machine inventor Edward H. Hebern in California before accepting Safford's offer and rejoining the Navy, this time as a cryptanalyst.12 She soon proved outstanding and became the Navy's principal civilian cryptanalyst (most of the Navy's cryptanalysts were officers). Among her first assignments was to work on the photographed code. The Research Desk had found that it was not used plain; rather, its codegroups were themselves enciphered by a columnar transposition. The first job of Miss Aggie, as she was called, was to solve the transposition. At first, she found, the key remained the same for several weeks; later it changed more frequently. Incessantly turning the pages of the photographed book with the rubber tip of her eraser to check codegroups, she took a couple of years to complete the job,13 enabling the Research Desk to turn the coded messages into Japanese. A husband-and-wife pair translated the Japanese into English.14

12Susan M. Lujan, "Agnes Meyer Driscoll," Cryptolog (Naval Cryptologic Veterans Association), 9 (August Special 1988), 4-6.
13"Reminiscences of Captain Joseph J. Rochefort, U. S. N., Ret." Transcript of oral history interview (U. S. Naval Institute, 1969), 30; Layton, 31.
14SRH-305, p. 01; Layton, 31-32.

By then, in 1926, Safford had gone to sea and had been succeeded by one of the first American naval officers to have studied Japanese in Japan: Lieutenant Joseph J. Rochefort. He was a "mustang"—a former enlisted man who had won a commission. This had made him tough and independent in a world dominated by Annapolis graduates; he neutralized his caustic speech with a conciliatory smile. Rochefort became one of the very few Americans capable both in Japanese and in codebreaking.15

15David kahn, The Codebreakers: The Story of Secret Writing (New York: Macmillan, 1967), 7 (henceforth as Kahn); Layton, 33, 34; Jasper Holmes, Double-Edged Secrets: U. S. Naval Intelligence Operations in the Pacific During World War II (Annapolis: Naval Institute Press, 1979), 3.

One of his subordinates described work at the unit:

Hours went by without any of us saying a word, just sitting in front of piles of indexed sheets on which a mumbo jumbo of figures or letters was displayed in chaotic disorder, trying to solve the puzzle bit by bit like fitting together the pieces of a jigsaw puzzle. We were just a few then …, young people who gave ourselves to cryptography with the same ascetic devotion with which young men enter a monastery.16

16Zacharias 89.

The hardest part of breaking a code is the beginning. Rochefort explained it in colorful terms:

It first off involved what I call the staring process. You look at all of these messages that you have, you line them up in various ways, you write them one below the other, and you'd write them in various forms and you'd stare at them. Pretty soon you'd notice a pattern; you'd notice a definite pattern between these messages. This was the first clue.… You notice a pattern that when you follow through, you say this means so-and-so; you'd run that through, and it doesn't work out. Then you'd proceed on some other effort and eventually, if you're lucky and the other fellow makes mistakes, which he invariably will, then you come up with a solution that will stand up under test, and this gives your first lead-in.17

17Rochefort, 17.

Rochefort said he felt good while doing this work "because you have defied these people who have attempted to use a system they thought was secure, that is, it was unreadable. It was always somewhat of a pleasure to defeat them or challenge them." But the work took its toll. While engaged in the actual cryptanalysis, he said, he generally felt frustrated. The tension was so great that after work he had to lie down for two or three hours before he could eat anything; he developed ulcers anyway, and this, together with the fact that duty in communications intelligence hurt a man's career, drove him to get out of the work when his tour in the Research Desk ended in 1927.18 At least for a while.

18Rochefort, 16, 17, 45-46.

Safford returned in June 1929. He had the photographed code, called Code No. 1, whose translation had originally been put together in 10 "volumes" with metal-strip Acco office binders, retyped in four copies on huge 12-by-18-inch forms and bound in two volumes in red buckram McBee binders, far more convenient to use. This gave the code its more common name, the Red Code.19 On 1 December 1930, however, the Japanese replaced it with a new code. But Miss Aggie, who had by then learned the ships' names, the communications patterns, and the frequently used phrases of the Japanese fleet, solved its transposition encipherment and then reconstructed the 85,000-group, two-part code, later called, from its binding, the Blue Code. It was a remarkable feat of cryptanalysis, and it gave the U. S. Navy insight into Japanese forces and tactics for years.20

19SRH-305, p. 03.
20Safford, 8-9; Layton, 46; SRH-305, pp. 11-12.

Two events in 1929 led the Army to expand its codebreaking. In May, after giving the new secretary of state a little time to understand the realities of the job, Yardley passed him some solved messages. Henry L. Stimson was shocked at what he regarded as a dishonorable and counterproductive activity—"Gentlemen do not read each other's mail," he said later, maintaining that "the way to make men trustworthy is to trust them." He withdrew State Department support from the Cipher Bureau.21 In the meantime, the Army had decided that Yardley was not doing what it needed most: training cryptanalysts for immediate use in case of war.22 These events doomed the unit, which was dissolved as of October 31, 192923—two days after the great stock market crash.

21Louis Kruh, "Stimson, The Black Chamber, and the 'Gentlemen's Mail' Quote," Cryptologia, 12 (April 1988), 65-89.
22SRH-029, Paragraph 6; Frank B. Rowlett, "Comments on [an early draft of this] David Kahn Article," 22 May 1991.
23Kahn, 360.


Its people were released; its papers went to the Army Signal Corps, which had charge of Army codes and ciphers.24 The Signal Corps had in 1921 hired a 29-year-old who was on his way to becoming the world's greatest cryptologist. William F. Friedman, natty, uptight, brilliant, had written some theoretical treatises of landmark importance and had solved German codes in France during World War I. His new job was nominally to improve the Army's own cryptosystems. But doing this properly required him to test cryptosystems offered to it. This gave him experience in cryptanalysis and expanded the Army's knowledge of it.25 With the closing of Yardley's agency, the Signal Corps added codebreaking to its responsibilities, and Friedman became the head of a new Signal Intelligence Service. He hired as junior cryptanalysts at $2,000 a year three young men who knew languages and mathematics. The first to report was Frank B. Rowlett, 21, a Virginian, a former teacher, fair-haired, with an all-American look to him, who at 8 a.m. April 1, 1930, found himself entering Room 3406 of the Munitions Building on B Street (now Constitution Avenue) near the Lincoln Memorial in Washington.26

24SRH-029, Paragraph 8.
25Kahn, 384-385.
26Theodore M. Hannah, "Frank B. Rowlett: A Personal Profile," Cryptologic Spectrum, 11 (Spring 1981), 4-21 at 5.

Two months later, he and his colleagues were excitedly combing through the secret files of Yardley's organization. This most clandestine and most valuable form of intelligence thrilled them. They went on to study basic cryptography and the solution of machine ciphers, clearly the wave of the future. In 1932, their training completed, they attacked Japanese diplomatic cryptosystems, working on messages provided by the Army's new intercept service. They first cracked a simple code, the LA, so called from the indicator group that preceded its messages. It did little more than put kata kana, a Japanese syllabary, into Latin letters for telegraphic transmission and to secure some abbreviations for cable economy. It was rather regular in construction. For example, all kata kana syllables that ended in e had as code equivalents groups beginning with A, so that ke=AC and se=AD. This arrangement facilitated solution. When the young cryptanalysts discovered that LA encrypted only insignificant messages, such as those dealing with expenses or vacations, and when they had gained experience and knowledge of Japanese diplomatic language and communications practices, they turned their attention to the more important messages, apparently protected by a machine system. It served on two main Japanese diplomatic communication networks — one covering the Far East, one linking Tokyo with major world capitals.27

27Ibid., passim.

* Fig.
Frank D. Rowlett on 2 May 1930, his 22nd birthday, a month after joining the Signal Intelligence Service.

The cryptanalysts recognized messages in it because they consisted of 5-letter groups like RWOBA preceded by a 5-digit group like 77181. Study showed that vowels had a relatively higher frequency than consonants. It appeared that the machine divided the 26 letters of the Latin alphabet into two subsets, the 6 vowels and the 20 consonants. They perhaps got some help from Navy Lieutenant Jack S. Holtwick's solution of a Japanese cipher machine that enciphered kana. Eventually, the Army cryptanalysts discovered that the machine they were working on used the same cryptographic principles as the Holtwick machine but enciphered not 50 syllables but the 26 letters of the Latin-alphabet form of Japanese, called Romaji, with a mechanism appropriately adapted.28 Rowlett and one of the other original junior cryptanalysts, Solomon Kullback, one day struck gold. Among their tentative recoveries of partial plaintext were three letters, an unknown, and another letter: oyo?i. They knew then they had broken the system, because oyobi was Japanese for and. They named it RED, the first of the primary colors.29 By 1937, for the first time in American history solutions began going to the White House,30 probably to President Franklin D. Roosevelt. The intercepts revealed, for instance, advance information about Italys' possible adherence to the German-Japanese Anti-Comintern Pact — providing this in March 1937, six months before American diplomats began reporting on it. Later they provided part of the text of the treaty.31

28Rowlett, "Answers to David Kahn's Questions," 22 March and 31 May 1991. The U. S. Navy solved the following Japanese cipher machines: (1) The IKA machine, perhaps so called because messages encrypted it used the Japanese character ika, written as a double slash, as a separator. (2) Its successor, called the M1, a kana machine used by the Japanese navy for about four years starting in 1933 whose analogue was designed by Holtwick, who presumably had solved the machine. (3) M2, contemporaneous with M1 but used by naval attaches and producing text in Latin letters rather than kana. It passed through several models, each more complicated than its predecessor. (4) M3, used by the Japanese Foreign Office. The Navy gave this an additional, alphabetical designation and called it the A machine. The Army called it RED. (5) M4. No other information available. (6) M5, which the Navy called the B machine and the Army PURPLE. (SRH-355, Appendix VIII.) M2 would appear to be the Romaji machine utilizing the same cryptographic principles as the kana machine on which it was based, namely the machine solved by the Army, but M3 is stated to be RED, or that machine. I cannot clear up the discrepancy.
29Hannah, 14-15. I have written (Kahn, 20) that the machine solved by the Army was probably codenamed RED as a successor to the likely codename ORANGE — the Navy's codeword for Japan — for the machine Holtwick solved, and that RED's successor, PURPLE, was so named because the colors deepened as did the strength of the machine. Alas for this elegant theory, both Rowlett, in his Answers, and Holtwick, in SRH-355, Appendix VIII, pp. 090E-090F, deny that this was the basis for the naming. Holtwick even specifies that "These colors had no connection with color covernames for foreign nations in war plans, such as ORANGE for Japan."
30Frank Rowlett, telephone interview, 22 April 1986.
31Solved Japanese messages declassified by the National Security Agency; United States, State Department, Foreign Relations of the United States: Diplomatic Papers, 1937, 1 (Washington: Government Printing Office, 1954), 608.

* Fig.
Women work with the U. S. Army analogue of the Japanese PURPLE cipher machine.


The next year, messages began to appear suggesting that a new machine would supplement and probably eventually replace the older one, which was physically wearing out. On March 20, 1939, three messages in the new system were intercepted, and over the next three months messages in RED gradually disappeared. Japan's major diplomatic messages had become unreadable. Faced with the loss of the nation's paramount intelligence, the Signal Intelligence Service mounted a concentrated attack to solve the new machine. Friedman put Rowlett in charge, with he himself exercising overall supervision. The Americans called the new machine PURPLE.32

32Hannah, 15-16.

In the absence of any American spies almost anywhere in the world,33 these half-dozen cryptanalysts would be able to provide the United States with its best secret intelligence on Japan as relations with that nation, which persisted in its aggression against China, deteriorated. They plunged into their work in Rooms 3416 and 3418 in the Munitions Building. Room 3418, about 25 feet square with a steel door and barred windows, was known as the vault. As additional cryptanalysts were assigned to the PURPLE problem, the group moved into larger quarters, finally occupying about eight rooms. Rowlett worked in 3416. He arrived at 7, an hour early, and left at 5, an hour late. His desk was usually neat, for he spread out his worksheets on a nearby table. Rowlett was extremely focused on the work. He never hummed or chewed his pencil or muttered to himself; he looked out the window only when something attracted his attention; he never drank coffee at work, but did puff on a pipe. His mind didn't dwell on the cryptanalytic problems during the 15-minute drive to work from Arlington County, but he would awaken each night after a few hours of sleep and, lying there in the dark, would review the day's work and think of ways of improving it.34 In the morning, at work he would exchange ideas with the other cryptanalysts — Assistant Cryptanalyst Robert O. Ferner, Rowlett's principal helper, and Junior Cryptanalysts Albert W. Small, Genevieve Grotjan, and Samuel S. Snyder, occasionally assisted by Cryptographic Specialists Cyrus C. Sturgis, Jr., Kenneth D. Miller and Glenn S. Landig35 (their names deserve to be remembered!). After the conference, all would return to their desks. Quiet reigned as they pored over the intercepts, most of which had been teletyped in from the monitoring stations; sometimes they puzzled over statistical and alphabetical tables compiled by hand from intercepts. To Rowlett, the atmosphere was like that of a library. Only the rustling of papers and the scratching of pencils and an occasional discussion broke the brooding silence.

33United States, Congress, Joint Committee on the Investigation of the Pearl Harbor Attack, Pearl Harbor Attack, Hearings (79th Congress, 1st and 2nd Sessions) (Washington: Government Printing Office, 1946), 2:785 (henceforth PHA, following by volume and page number); Anthony Cave Brown, Wild Bill Donovan: The Last Hero (New York: Times Books, 1982), 176-177; Thomas F. Troy, Donovan and the CIA: A History of the Establishment of the Central Intelligence Agency (Frederick, Maryland: University Publications of America, 1981), 105-107.
34Rowlett, Answers.
35William F. Friedman, "Preliminary Historical Report on the Solution of the 'B' Machine," October 14, 1940, SRH-159, p. 8.

PURPLE had carried over from RED the 6-and-20 division of letters. But this time the "sixes" were not exclusively vowels. Nevertheless, within a few weeks, the cryptanalysts ascertained how the sixes were enciphered. This enabled them to recover the plaintext for those letters. The process was slow and painstaking. A new recruit, Leo Rosen, was assigned to devise a way to mechanize this pencil-and-paper process. He hit upon the idea of using standard 6-level, 25-point telephone selector switches, used in dialing. They worked like a dream, and the solution process was considerably speeded.36

36Rowlett, Answers.

Despite Rosen's remarkable advance, the totality of PURPLE still resisted the Americans. Friedman, who had been supervising the work rather loosely, was asked by his bosses — all extremely supportive, financially as well as psychologically37 — to participate personally.38 But he was busy with administrative and communication security and, aside from the fundamental and important job of assembling the team, did not contribute directly to solving PURPLE.39 The Navy, though its main effort continued to be against Japan's naval codes, lent a hand. It and the Army organized their files the same way to facilitate cooperation. After about four months, the Navy dropped its effort40 and returned to Japanese naval cryptosystems. The S. I. S. pushed ahead. Within Rowlett's group, teamwork was extremely close. No one complained that a task was too menial. Determination pervaded the team. Rowlett was confident from the start that they would break PURPLE the way he and others had broken RED. He never grew depressed, even as one month after another went by without a solution.41

37Ibid.
38PHA, 34:84-85.
39Rowlett, telephone interview.
40Ibid.
41Ibid.

As the breakthrough was being sought, the cryptanalysts spent much of their time matching proposed plaintext — a guess as to the text of the original message — against the text of the cryptograms, the ciphertext. The proposed plaintext came from various sources. During the gradual replacement of RED with PURPLE, circular messages were sent to embassies and legations around the world in both systems. RED's solution gave the plaintext to the circulars in PURPLE. In a few cases, the State Department gave them the texts of notes to or from the Japanese ambassadors, which the codebreakers used as cribs. And the crypt-analysts' guessed at plaintexts. They knew, for example, that many diplomatic dispatches began "I have the honor to inform Your Excellency that …" and they often tried that as the plaintext start.


Among the things they were looking for, apparently, were letter intervals that would reveal the advancement of PURPLE's mechanism. Exactly what these intervals were has never been revealed, but one possibility is that two occurrences of a probable plaintext letter, such as a, stood at a certain distance from one another, say two places, as in Japan, and that the ciphertext letters that represented them were separated in the alphabet by the same interval, as X and Z. This would show that the machine had moved forward one space each time a letter was enciphered. C. A. Deavours thinks that the search was for identical ciphertext letters, so that in a situation like

1 | 2 | 3 | 4 | 5 | 6 | 7
a | d | v | a | n | c | e
X | G | A | R | X | G | V

the cryptanalysts looked for an alphabet where the a in alphabet 1 was represented by the same letter as plaintext n in alphabet 5, and d in alphabet 2 was represented by the same ciphertext letter as c in alphabet 6. A plugboard, which monoalphabetically enciphered the cipher alphabet table, complicated things. Either way, two or more intervals were needed for the hypothesis to be confirmed.

More than a year of tedious work passed.42 Then, about 2 p. m. on Friday, 20 September 1940, a sunny day on which the temperature reached 88 degrees, in the middle of Roosevelt's running for an unprecedented third term, as Britain anxiously awaited a German invasion from occupied France, Albert Small noticed that Grotjan, a dignified, 26-year-old statistician, seemed to be concentrating extremely intently. He spoke to her about it, and learned that she had just discovered a couple of the needed intervals and was looking hard for others. He took her in to see Rowlett. He was conferring with Ferner. She said she had something to show him.43 All moved to her desk. She pointed to her instances, then a third leaped out at the codebreakers. At once, they grasped the significance of what she was showing them. The ebullient Small dashed around the room, hands clasped above his head. Ferner, normally phlegmatic, shouted "Hooray!" Rowlett jumped up and down, crying "That's it!" Everybody crowded around. Friedman came in. "What's all the noise about?" he asked. Rowlett showed him Grotjan's findings. He understood immediately. Grotjan's discovery verified the team's theory of how the PURPLE machine worked. It marked the climax of one of the greatest cryptanalyses of all time. And how did the egghead codebreakers celebrate this great moment in the history of American codebreaking? They sent out for Cokes!44

42Rowlett, Answers, says that nothing in the romantic and hypothetical paragraph on pages 21-22 of Kahn is accurate.
43Grotjan, interview, 12 May 1991.
44Hannah, 5.

When the euphoria and the effects of the Coca-Colas had worn off, the crypt-analysts drifted back to work. Grotjan, who seems to have gotten excited about the breakthrough mainly because everybody else was, regarded it soberly as just one step in a series of steps. A week later, the day after Japan started to occupy French Indo-China and the very day the Tripartite Pact establishing the Rome-Berlin-Tokyo axis was signed, the Signal Intelligence Service handed in its first two solutions of PURPLE messages.45 This did not mean its work was done. The settings for the machine changed each day, and the cryptanalysts had to recover these. But this work was facilitated by Rosen's construction of two American analogues of the Japanese PURPLE machine. The cost: $684.65.46 Later, other copies of the machine were built — several at the Washington Navy Yard; some were given to the Navy, which had rejoined the PURPLE work to help with the heavy volume of solutions, and some to the British, so they could read the messages without having to wait for American solutions to be forwarded to them. Soon Navy Lieutenant Francis A. Raven discovered a pattern to the daily machine setting changes. With this knowledge, the United States was able to read reports from and instructions to Japan's ambassadors on average within a day or so, sometimes within hours. It had gained access to the most secret diplomatic dispatches of Japan as U. S. relations with that empire continued to worsen.

45Ibid., 18
46Ibid.


In cryptology, the beginning of wisdom is to know that there is no such thing as "the" Japanese code. PURPLE was not the only cryptosystem of the Foreign Ministry, much less of the empire. The Foreign Ministry employed a hierarchy of systems, of which PURPLE was the apex. Under it lay several codes that — unlike PURPLE, which served embassies exclusively — were used at both embassies and consulates. LA, the simplest, lay at the bottom.47 Above it rested a two- part system, PA-K2. The PA was a code, like LA, though more extensive and with its codegroups disarranged in relation to their kata kana plaintexts. The K2 part shuffled the letters of the PA codegroups. American cryptanalysts could restore them to their original positions and then solve the PA code message in three days on the average.48 More complex still was the J series of codes. These codes were more extensive and more thoroughly disarranged than PA, and their codegroups were transposed in a far more complicated manner than the K2: an irregular columnar transposition with a pattern of blanks that broke up the columns. (See Appendix.) Moreover, the code was changed every several months, and the transposition blank pattern — called K plus a number by the Americans — was, by November 1941, changed every 10 days; the transposition's columnar transcription key changed daily, so the codebreakers had to make a fresh analysis for each day's messages.49 Thus J17-K6 was replaced on 1 March 1941 by J18-K8, and that in turn by J19-K9, whose K9 transposition was itself replaced by K10, which served from 11 to 20 November.50 Some 10 to 15 percent of J-code messages were not solved at all, and those that were took an average of a week from interception through translation to distribution. This situation contrasts with that of PURPLE, in which most messages were solved within hours and all but 2 to 3 percent of keys were recovered. Did the Japanese err in assessing the security of their cryptosystems? Yes and no. PURPLE was a much more difficult system to solve in the first place, but once solved it was easier to keep up with.

47Kahn, 14-15.
48Ibid., 15-16. This does seem long for this system, but this was in fact the time given.
49Ibid., 15-18.
50Some J19-K10 solved intercepts and transposition worksheets are in Box 13, Records of the Pearl Harbor Liaison Office, Record Group 80, National Archives.

While the Army was concentrating on Japan's diplomatic systems (though the Navy later joined in recovering daily keys and solving and translating messages), the Navy's codebreaking agency — except for some help to the Army — focused on Japan's naval systems. This agency, again under Safford, now a commander, was now called OP-20-G: the OP for the Office of the Chief of Naval Operations, the 20 for Office of Naval Communications, the G for its Communication Security Section, which included codebreaking. During the 1930s it read messages in the Blue code, gaining considerable knowledge about Japan's naval maneuvers.51 The Japanese replaced this with a new code on 1 November 1938. But the paucity of intercepts in it meant that almost no progress was made in reading it.52

51Safford in Listening to the Enemy, 8-9, 12; SRH-305, 12-19; Layton, 46.
52Safford in Listening to the Enemy, 8; Layton, 77.

On 1 June 1939, the Japanese introduced yet another code. Called JN25 by the Americans, for the 25th Japanese naval code attacked, it soon became the main fleet code, enshrouding messages dealing with naval operations. It was a two-part code of 33,333 five-digit groups, each of which had the peculiarity of being divisible by 3: for example, 52884; this checked against garbles. It was superenciphered with an additive — a string of numbers, printed in a book, that the cipher clerk added to the codegroups, disguising them. Miss Aggie attacked it, greatly helped by reserve Lieutenant Prescott Currier. She stripped off the additive, exposing the original codegroups, then recovered their meanings. A year and a half later, in almost the very week that the Army was producing its first PURPLE solutions, the first JN25 solutions emerged.53 The Navy's satisfaction did not last long. On 1 December 1940, the Imperial Navy replaced it with a new edition, called by the Americans JN25b. But Japan's cryptographers foolishly kept the first edition's last additive book in force for the first two months of the second edition's service. Since this additive book had been reconstructed, OP-20-G gained easy access to the basic codegroups of JN25b and quickly determined the meaning of 2,000 of them. When new additive tables went into effect, its I. B. M. tabulators generated the difference tables necessary for stripping off the additives.54 As Safford remarked, "We could always trust the Japanese, themselves, to do something that would assist us in the solution of their systems: they have never yet failed us."55 Nevertheless, progress continued slow.

53Safford in Listening to the Enemy, 9; Layton, 77-78.
54Layton, 78.
55SRH-305, 12.


So in March 1941, OP-20-G in Washington ordered the radio intelligence unit on Corregidor Island in the Philippines to help with the solution of JN25b. A British codebreaking unit in Singapore exchanged recoveries of JN25b codegroups with Corregidor.56 But despite all these efforts, the code remained readable to but a very small degree. By December 1941, only 10 to 15 percent of each message could be understood.57

56Kahn, 10. James Rusbridger and Eric Nave, in Betrayal at Pearl Harbor: How Churchill Lured Roosevelt into World War II (New York: Summit Books, 1991), claim that the British unit was reading JN25 pretty fully and was passing its knowledge of Japanese fleet movements, and hence its foreknowledge of the Pearl Harbor attack, to Britain, where Prime Minister Winston Churchill suppressed it to make sure the attack would bring the United States into the war on the side of Britain. Aside from the fact that Churchill wanted the United States to fight Germany, not Japan, the claim is not only not substantiated by any documents (it is based chiefly on hypotheses and "must have beens"), but is vitiated by technical errors. The chief one is the belief that JN25b was merely a new additive book to the unchanged basic code. In fact JN25b was a new, improved, and enlarged codebook, which is why the Americans had made such little progress into it. It is improbable that the British would have made much more, and also unlikely that they would have limited exchanging codegroup recoveries with the Americans, when they would have benefited as much if not more than the Americans from learning as much as they could about the Japanese.
57Layton, 231-232, 547. Captain Thomas H. Dyer, U. S. N., states, in his U. S. Naval Institute oral history, p. 231, that JN25 produced no intelligence about the Pearl Harbor attack.

This, then, was the cryptanalytic situation early in December 1941: the main Japanese diplomatic system could be read rapidly and completely; Japanese consular systems could be read with a few days' delay; the main naval system could be read only spottily. The diplomatic solutions provided Washington with the instructions to and reports from Japan's ambassadors to the United States and so with considerable insight into Japan's foreign plans, and it did so with great speed, accuracy, and volume. By November 1941, 50 to 75 messages a day poured out of the U. S. codebreaking agencies.

Why did not all this information enable the United States to foresee the Pearly Harbor attack? The most basic answer is this: Japan never sent any message to anybody saying anything like "We shall attack Pearl Harbor." The ambassadors in Washington were never told of the plan. Nor were any other Japanese diplomats or consular officials. The ships of the strike force were never radioed any message mentioning Pearl Harbor. It was therefore impossible for the cryptanalysts to solve one.

Some historians have contended that, if only the intelligence officers of the Army and the Navy, and perhaps State Department officials as well, had found the time to analyze all the intercepts as a group, they would have discerned a pattern that pointed to Pearl Harbor. This argument resembles that made by Roberta Wohlstetter in her book Pearl Harbor: Warning and Decision,58 in which she holds that the noise of the false evidence drowned out the indications of the true signals. "We failed to anticipate Pearl Harbor not for want of the relevant materials, but because of a plethora of irrelevant ones."59 This is wrong. There were no true signals, no clear indications of the attack.

58(Stanford: Stanford University Press, 1962).
59Ibid., 387.

What then is the answer to the joint congressional committee's question? What about that "finest intelligence"? The simple answer is that fine though it was, it was not good enough. A nation can't cryptanalyze what is not there. In that elementary fact lies the inadequacy of cryptanalysis at Pearl Harbor.

APPENDIX: A JAPANESE CRYPTOSYSTEM

The most complicated of Japan's pencil-and-paper cryptosystems was the J19-K10 enciphered code. J19 was a two-part code. A Japanese plaintext was first encoded using the J19 code, then this codetext was transposed using the K10 transposition.

An example of this system is one of the messages from Tokyo to Washington setting up the so-called "winds" code. If it had been drafted in English, Circular 2352 would have read:

Regarding the broadcast of a special message in an emergency.
In case of emergency (danger of cutting off our diplomatic relations), and the cutting off of international communications, the following warning will be added in the middle of the daily Japanese language short wave news broadcast.
(1) In case of Japan-U. S. relations in danger: HIGASHI NO KAZE AME.
(2) Japan-U. S. S. R. relations: KITA NO KAZE KUMORI.
(3) Japan-British relations (including Thailand, Malaya, and Netherlands East Indies): NISHI NO KAZE HARE.
This signal will be given in the middle and at the end as a weather forecast and each sentence will be repeated twice. When this is heard please destroy all code papers, etc. This is as yet to be a completely secret arrangement.
Forward as urgent intelligence.

The phrases relate to the position of the various countries from Japan and mean, respectively, "East wind rain," "North wind cloudy," and "West wind clear."

The text in romaji (Japanese syllabary in Latin letters) version began: Kancho fugo atsukai. Kokusai jigyo no hippaku no kekka itsu saiaku no jitai ni tachi itaru kamo. … This was first encoded in J19 (see Figure 1).

* Fig.
* Fig. Figure 1. The "East wind rain" message encoded in J19.


Most of the codewords are two-letter groups that replace syllables in kata kana, punctuation, numbers, and whole words. But some are four-letter groups that replace words. An M was added at the end, perhaps as a dummy to fill out the last five-letter group. The dash before the B in the third to last group indicates that that letter is missing from the intercepted texts.

The code clerk next took out a sheet of paper with that day's transposition block printed on it (see Figure 2). This consisted of a grid of lines and columns in which several of the cells were blacked out [image] topped by a row of numbers from 1 to 19 in scrambled order. He wrote the coded message — XEICNCSTWYNYKYES… — horizontally into the grid, skipping the blacked out squares [image].

* Fig.
* Fig. Figure 2. Code clerk's sheet of the day's tranposition block.

Then he transcribed the letters vertically by columns beginning with the column numbered 1. The result

NEWZUCNFXALEKFKOHTZVGONHEWKWWK…

was the cryptogram that would be sent. He divided it into five-letter groups for ease of transmission and to detect any dropped letters, placed the system and key indicators in two five-letter groups at the top, some other indicators at the end, typed it up, and handed it to a communications officer for transmission (probably by a commercial radio company) to the embassy of the United States.

When a cryptogram like this was intercepted by U. S. Army or Navy radio monitors and teleprinted to Washington, cryptanalysts there faced two steps. They had, first, to determine the pattern of blanks in the grid and the order of the keynumbers at the top. They did this by trying various arrangements until repetitions of pairs of quadruplets or letters indicated that that arrangement had produced codewords. Then the codebreakers had to ascertain the original Japanese words or syllables for which the codewords stood. These two steps could not be completed on the basis of a single message; dozens were needed. When a message was finally solved, translators would render it into English. Then it would be distributed to the very limited circle of those high officials entitled to see the intercepts.

* Fig.
* Fig. Figure 3. The Japanese cryptogram of Circular 2353, as intercepted in two part by American monitors.

BIOGRAPHICAL SKETCH

David Kahn, an op-ed editor and member of the editorial board at Newsday, the Long Island daily, is the author of The Codebreakers and, most recently, of Seizing the Enigma: The Race to Break the German U-Boat Codes, 1939-1943 (Houghton Mifflin).

* Fig.
Genevieve Grotjan in 1935.

THE WOMAN WHO MADE THE KEY DISCOVERY

In a small brick ranch house exactly like the others lined along the wooded suburban street near Washington, D. C., lives an important piece of America's secret history — and of its feminist. The young neighbors on either side have no inkling that the structure between them houses a woman who detected data that led to perhaps the nation's most important cryptanalytic breakthrough. To them, she and her husband are just an old couple with a less than manicured lawn.

But the woman, the former Genevieve Grotjan, discovered, on 20 September 1940, significant characteristics in intercepted Japanese code messages. A week later, the team of U. S. Army civilian codebreakers, of which she was a part, had solved Japan's main diplomatic cipher machine, which the Americans called the PURPLE machine. Solutions of the messages enciphered in it gave President Franklin D. Roosevelt and other high-ranking members of the U. S. Government extraordinary insight into the thoughts and plans of the Foreign Ministry and its envoys in Washington through all of 1941.

Grotjan was born in Buffalo on April 30, 1913. She attended the University of Buffalo, hoping to become a mathematics teacher. But no such work was available at the time, so she accepted a position in Washington as a statistical clerk with the Railroad Retirement Board. Later she took an examination for a professional grade of mathematician and was offered a job by the Signal Intelligence Service, which naturally never told her that she would be breaking codes but asked only whether she would be interested in working with codes. She was; she was hired; she proved good enough to be assigned to the premier task of the unit — and she became the one who made the critical find.

Why her? "Maybe I was just lucky in getting the right series of papers," she said. "I perhaps had a little more patience than [Al] Small. [Bob] Ferner might have done it but he was working on something else." When she spotted the intervals, she aside, "I was elated up to a point but I didn't think 'This is it!' because there was so much more to do. I was looking forward to working on the mechanism. I regarded it more as just one step in a series of steps."

So she makes no great claims for herself. Interviewed in her cluttered living room, she thought long before offering a carefully worded answer to each question. She was evidently trying both to reconstruct transient events of more than half a century ago and to be as accurate as she could. She did not exaggerate her importance, did not suggest that without her discovery PURPLE would not have been cracked or the war lost. She recognized that only chance gave her the breakthrough. It was a commendable display of modesty.

Though the solution of PURPLE did not prevent Pearl Harbor, it had enormous effects on the war. Not the war in the Pacific, but that in Europe. "Our main basis of information regarding Hitler's intentions in Europe is obtained from [Japanese ambassador] Baron Oshima's messages from Berlin," wrote Army Chief of Staff General George C. Marshall. These and other solutions, he noted, "contribute greatly to the victory and tremendously to the saving in American lives." It was a remarkable tribute to America's cryptanalysts: including Grotjan, who lives obscure, forgotten, unsung in a low house with a scruffy lawn, but who helped, more than most, to shorten America's biggest war.


The Unsolved Messages Of Pearl Harbor

Frederick D. Parker

ADDRESS: 721 Holly Dr. N., Annapolis MD 21401

ABSTRACT: In 1941, the United States Navy intercepted enough Japanese naval messages to predict the attack on Pearl Harbor if the code which protected them had been solved. The messages would have disclosed that, while the Japanese government cynically conducted diplomatic negotiations with the United States, the Japanese Combined Fleet under Admiral Isoroku Yamamoto, beginning as early as September 1941, systematically prepared ships, crews, weapons, tactics, and a complicated fuel supply system for the most ambitious operation ever undertaken by the Japanese navy. Details of Japan's intentions were hidden from Navy cryptanalysts because of limited manpower resources and because all Japanese navy codes were assigned a lower priority than Japanese diplomatic codes and German submarine threat.

KEYWORDS: Communications intelligence, Pearl Harbor, World War II, cryptanalysis, Japanese codes, JN25, Purple, OP-2o-G, traffic analysis, 14th Naval District, 16th Naval District, radio silence, tactical warning, Pacific Fleet, Japanese navy, Imperial Fleet, Combined Fleet, First Air Force, STrike Force, Second Fleet, Third Fleet, intentions, capabilities, Southern Operation, Northern Operation, strategic warning, Mount Niitaka.

A study of cryptologic archives in Crane, Indiana, and Washington, D. C., reveals that, in 1941, the United States Navy intercepted enough encoded and enciphered Japanese naval messages to predict the attack on Pearl Harbor. If they had been solved, the intercepts would have disclosed Japanese preparations for the surprise attack, including most significantly the positioning of a tanker where its only purpose could have been to refuel a long-range strike force. But the code that protected these messages could not then be penetrated by U. S. Navy codebreakers working in an obscure but important element of the Navy Communications Directorate, OP-20-G.

Beginning in September 1941, the Japanese Combined Fleet under Admiral Isoroku Yamamoto systematically prepared ships, crews, weapons, tactics and a complicated fuel supply system for the most ambitious operation ever undertaken by the Japanese navy. Their goal was to duplicate the successful British carrier assault on the Italian fleet at Taranto in November 1940 by attacking the U. S. Pacific fleet at Pearl Harbor.

Details of Japan's intentions were hidden from OP-20-G cryptanalysts by the Japanese navy's General Purpose code, designated by OP-20-G as JN25. The first, or Able, version of this code was introduced in 1939 and gradually became the primary cryptographic system used by the Japanese navy. The failure of OP-20-G to develop code group meanings in this five-digit enciphered code, particularly in 1941, can be traced, however, to causes other than superior Japanese cryptography. Simply stated, in Washington in 1941, Japanese navy codes were assigned a lower priority than Japanese diplomatic codes and the German submarine threat.1

1History of OP-20-GYP, Classified Naval Archives, Crane, Indiana. Hereafter History of OP-20-GYP.

Since the inception of OP-20-G in the 1920s, the U. S. Navy entrusted cryptography (the development of codes and ciphers) and cryptanalysis (the exploitation of codes and ciphers) almost exclusively to commissioned officers.2 The goal of the officers assigned to OP-20-G in Washington was to improve the security of U. S. Navy cryptography. Japanese navy codes and ciphers were studied only for their cryptographic characteristics, not for the intelligence value of the underlying messages. Accordingly, the meaning of a code group was only incidental to their work, and, unless an additive cipher period lasted for an unusual length of time, they seldom if ever worked with current material. Conversely, the officers in the 14th and 16th Naval Districts, who supported the Pacific and Asiatic Fleets respectively, were vitally concerned with developing codegroup meanings and solving the current additive cipher. This knowledge would be critical to their attempts to read in real time any intercepted Japanese navy message sent in an enciphered code such as JN25.3

2History of OP-20-GYP.
3Military Study [of] Communications Intelligence Research Activities, Author, Admiral Joseph Wenger USN (Ret) SRH-151 Record Group 457, National Archives. This study was done in 1937, in conjunction with an update of the Navy's Orange (Japan) War Plan. Page 003 of the study contains Article 102, Mission, of the Naval Communications Service, about 1929, which, in subparagraphs 5 and 6, specifies the two distinct efforts of research and exploitation. On pages 007 and 012 the study specifically notes that Washington and Pearl Harbor and the Philippines are charged with research and exploitation. A similar study was done in 1935, also in conjunction with War Plan Orange. It too assigned research activity to OP-20-G in Washington. Since all SRH studies are from Record Group 457 that designation will be omitted henceforth.


By late 1941, 80 officers were assigned to OP-20-G, and the 14th and 16th Naval Districts, 53 of them to OP-20-G. Of these only eight studied JN25. this small number represented a significant increase from 1939, 1940, and 1941, when usually two and never more than five analysts were assigned to analyze Japanese navy codes including JN25; JN20, a minor system for construction activity; JN22, JN39, and JN40, merchant vessel-navy liaison systems; JN61, a merchant vessel-navy five-letter cipher; and JNA20, a naval attachè cipher. The other 45 officer-analysts at OP-20-G, in addition to administrative duties, developed secure codes and ciphers for the navy, operated the communications and processing centers for a 24-hour Atlantic direction-finding (D/F) net tracking German submarines, and solved Japanese diplomatic messages in cooperation with the Army.4

4History of OP-20-GYP. See also SRH-355 pp. 158, 159, where Holtwick describes how a lack of trained manpower in sufficient number meant that no current system could be read, and how difficult it was to find competent trainees.

Throughout 1941, intelligence staffs in Washington and London increased pressure on OP-20-G for more rapid processing of D/F bearings and dissemination of fixes to convoy managers. At the same time pressure increased on Army and Navy cryptanalysts and linguists working on the diplomatic messages to supply the president and the State, War and Navy departments with the Japanese government's latest instructions to its ambassadors.5 The officers at OP-20-G excelled in both these endeavors, and the unrelenting pressures to produce results surely led OP-20-G to concentrate on diplomatic solutions and to deflect attention from the unreadable Japanese naval codes and ciphers.

5History of OP-20-GYP.

Another cause of the Navy's failure to read any of the Japanese navy's codes before the war was the primitive communications facilities which existed at the time. Movement of intercepted radio traffic and the exchange of ideas between OP-20-G and the 14th and 16th Naval Districts was, even by those days' standards, incredibly slow. Some Japanese navy radio messages, probably those which contained a coding or enciphering error, were airmailed to the West Coast and thence sent to Washington by rail. This process probably consumed more than a week since the rail journey alone in 1940 was about six days. The bulk of Japanese navy radio traffic and all technical exchanges except warnings, which late in 1941 were given special treatment, were sent in weekly shipments by a combination of ship and rail, which at best took weeks and in prolonged bad weather might take months. Under these circumstances it is easy to see why little or no headway against current material was possible either in Washington or in the 14th and 16th Naval Districts.6

6History of OP-20-GYP. See also SRH-012, The Role of Radio Intelligence in the American-Japanese Naval War, August 1941-June 1942, Vol. I. Prior to November 1941, daily comint reports written in Hawaii were customarily received in Washington 10 days to two weeks later! The first electrically transmitted reports from the Pacific were sent on 26 and 27 November 1941! See Com 14 26011ZNov41 and Com 16 261331ZNov41 in Com 14TI Summaries, SRMN012, RG457, NA.

Nevertheless, success came early to the OP-20-G cryptanalysts, if only briefly. By 30 November 1940, the original Japanese code, JN25 Able, was fully diagnosed and analysts were certain that they were able to produce an accurate and original five-digit text, though lacking most code group meanings. But on 1 December all progress stopped. Japanese cryptographers introduced a greatly enlarged five-digit enciphered code, designated by OP-20-G JN25 Baker. It proved to be a much larger code with a new and larger additive cipher book. It also employed several new and unfamiliar cipher techniques. Thanks to Japanese communications errors each successive JN25 Baker cipher, numbered Baker 1, Baker 2, etc., was successfully penetrated by the analysts in Washington and Corregidor until the additive book designated Baker 8 was introduced on 4 December 1941. This cipher, ironically, proved to be only a minor variation on its predecessor though it completely frustrated analysis for several months. The 1 December 1940 change in the code itself also resisted all efforts at a solution until after the Japanese attack on Pearl Harbor. Thus, JN25 messages could not be solved in time to play a part in U. S. policy or military decisions during this critical period. Thousands of Japanese navy messages intercepted in the last six months of 1941, particularly those in JN25, were not read because, due to manpower shortages and different priorities, the underlying code values remained unrecovered.7

7History of OP-20-GYP.

To appreciate fully the value of these unread radio messages in 1941 it is necessary to review briefly what the United States knew about Japanese intentions from communications intelligence not derived from the text of naval messages, i. e. , from traffic analysis, direction finding, and from Japanese diplomatic messages.

By recovering the daily keys for the B Japanese diplomatic cipher machine, called PURPLE, the U. S. could read all messages between Tokyo and the two Japanese ambassadors in Washington. Often they were being read as soon as the Japanese recipients in Washington read them and sooner than the addressees in Tokyo did. The messages gave the U. S. government a complete picture of the diplomatic negotiations. All those reading the diplomatic messages were aware of Japan's warlike intentions and, beginning in November 1941, knew that the Japanese diplomats were waiting for a war between the United States and Japan to begin.8

8SRH-012.

After monitoring Japanese navy communications for many years, U. S. Navy communications analysts were completely familiar with the Japanese fleet organization both afloat and ashore. This included the Combined Fleet and its communications infrastructure down to individual fleets, divisions, home ports and navy yards. They were also completely familiar with communications serving Guard Districts and Base Forces, as well as the Combined Air Force, including the 11th Air Fleet, and its airfields with their resident squadrons.9

9SRMN-012.

At the very top of the structure the analysts were familiar with the radio networks serving Japanese radio intelligence, the imperial and navy general staff organizations, and the navy minister. They also routinely identified battleships, cruisers, carriers, submarines and many of the Navy's cargo vessels, particularly while they were at sea.10

10Ibid.


Augmented by direction finding, message headings, and the knowledge gained from following the maneuvers and imperial fleet operations since the 1930s, communications analysts in Hawaii and Corregidor throughout 1941 frequently issued reports which today would be called strategic and tactical warnings. They regularly recognized and reported the formation of task forces, the creation of new fleets, the movements of large and small air and fleet units, build-ups of ground-based air units, and deployments of key fleet units such as carrier divisions. It was clear in the daily reports filed after 1 July 1941 that Admiral Thomas Hart, Commander in Chief, Asiatic Fleet, and Admiral Husband Kimmel, Commander in Chief, Pacific Fleet, wanted the analysts to follow the imperial fleet wherever it went and report on any threat to the Asiatic and Pacific Fleets. Primary emphasis was obviously placed on carriers, battleships, cruisers, and submarines. Until December 1941, even without the substance of the Japanese navy's messages, simply from the form of Japanese navy communications, the analysts more than fulfilled this mission.11

11Ibid.

Beginning as early as July 1941, but becoming more emphatic by October, communications intelligence reports from the 14th and 16th Naval Districts warned of a "Southern Operation." The warnings stemmed from ground-based air unit deployments to Taiwan, Hainan, Viet Nam, and to the Japanese mandate islands of the Marianas, Carolines and Palaus, and on the movements in late November and on 2 December 1941 of two new task forces formed around the Second and Third Fleets.12 Warnings of a threat to "Southern Objectives" dominated comint reports until 27 November when Admiral Harold R. Stark, chief of naval operations, issued the first of his "war warning" messages to the Pacific commands.13 The alarming news concerning Japanese fleet movements provided by the comint reports from Hawaii and Corregidor was probably responsible for Stark's warnings.14 Figure 1, depicting the deployment locations for Japanese gound-based air units, shows an unmistakable threat to the Philippines and the Netherlands East Indies and reflects what the chronology of comint reports and Stark warnings suggests: that comint played a major role in the decision-making processes in Washington.15 But there were no warnings from any quarter of a threat to Pearl Harbor.

12Ibid.
13SRH-012.
14Edwin T. Layton, with Roger Pineau and John Costello, And I Was There (New York: Morrow, 1985) 187,212,234.
15SRMN-012.

* Fig.
* Fig. Figure 1. Japanese ground-based air buildups in the Pacific, 1941.

As startling and ominous as the reports from Hawaii and Corregidor were, particularly after 1 September, they pale when compared with the texts of unread messages which the analysts in the 14th and 16th Naval Districts dutifully forwarded to OP-20-G for study. After the war, Navy analysts in Washington discovered that, during the period from July to December 1941, Navy monitors had intercepted more than 7000 original messages per month — and more than twice this volume if duplicates were counted. After studying this enormous backlog of unexploited prewar material, analysts in 1945 and 1946 decrypted 26,581 Japanese navy messages in seven different cryptographic systems intercepted between 5 September and 4 December 1941.16

16Unpublished study of Pre-Pearl Harbor Japanese Naval Dispatches by OP-20-G, Classified Naval Archives, Crane, Indiana.

Between 15 March 1946 and 20 August 1947, OP-20-G analysts and linguists from ONI undertook to study this huge body of solved messages to discover what Japanese secrets might be hidden in their contents. Of the 26,581 decryptions, only 2413 were considered important enough for full translation. Of these, 188 were isolated as pertaining specifically to the events of 7 December 1941 (8 December Japanese time). Had they been available at the time, decision makers in 1941 would have recognized these messages for what they were; pieces missing from the picture of Japanese fleet intentions already partially revealed by the comint reports written by the analysts in the 14th and 16th Naval Districts between September and December 1941.

As early as 5 September, long before the diplomatic messages began to reflect the possibility, the naval messages contained an indirect reference to an approaching war when the chief of staff, Second Fleet, directed this dispatch to the ships of the entire Second Fleet:17

17This message, and others that follow, have been lightly edited for clarity. Two messages, 5 September and 4 November, do not have SRN numbers. All SRNs are individual translations of Japanese naval messages found in Record Group 457. Where no SRN number is given this message was found only in the unpublished OP-20-G study of Japanese naval dispatches.
A STATE OF COMPLETE READINESS FOR BATTLE OPERATIONS MUST BE ACHIEVED BY THE
FIRST OF NOVEMBER … EXPECT EVEN GREATER EFFORTS TOWARD ACHIEVING MAXIMUM
FIGHTING STRENGTH…

Similar sentiments were reflected four days later in a dispatch from the chief of staff, Combined Fleet, to all fleet chiefs of staff and all fleet commanders in chief that referred to impending war operations.18

18SRN 115533.
AS CONDITIONS BECOME MORE AND MORE CRITICAL, EACH AND EVERY SHIP UNIT WILL
AIM AT BEING FULLY PREPARED FOR COMMENCING WAR OPERATIONS BY THE FIRST PART
OF NOVEMBER … COMPLETING ALL PERSONNEL CHANGES ORDERED 3 AUGUST AS SOON
AS POSSIBLE.


Beginning in late September and early October 1941, specific details concerning Japanese plans for wax began to appear in the messages. The Combined Fleet and particularly the First Air Fleet were observed exchanging messages almost daily, organizing the forces and conducting drills on various aspects of forthcoming operations. In addition to providing these vital details, the message headings revealed the structure of a First Air Fleet carrier Strike Force and parts of its support train.

References to torpedo and dive-bombing drills against battleships occurred in messages from the First Air Fleet staff between 6 October and 4 November 1941. The following is typical:19

19SRN 117453. Akagi, Kaga, Sooryuu, and Hiryuu are all carriers. All names are given as they appear in the translation.
     FROM: FIRST AIR FLEET STAFF       6 OCTOBER 1941
     ACTION:  COMMANDER CARRIER DIVISION 2
              KAGOSHIMA AIR GROUP COMMANDER
              COMMANDER CARRIER DIVISION 1
              COMMANDING OFFICER YUUZUKI
              COMBINED FLEET STAFF
IN FIRST AIR FLEET AERIAL TORPEDO ATTACK DRILL #13, WHICH IS SCHEDULED TO
BE CONDUCTED ON 21 OCTOBER AGAINST BATTLESHIP DIVISION 1, AKAGI AND KAGA
ARE EACH ALLOTED 9 TORPEDOES AND SOORYUU AND HIRYUU ARE EACH ALLOTED 6
TORPEDOES.

Several important details about the torpedoes being tested were revealed in a group of messages concerning torpedoes, of which this is typical:20

20SRN 117301. Parentheses added by translator.
    FROM: CHIEF OF STAFF FIRST AIR FLEET       28 OCTOBER 1941
    TO:  CHIEF GENERAL AFFAIRS SECTION NAVAL TECHNICAL BUREAU
         GENERAL AFFAIRS SECTION AERONAUTICS BUREAU
         CHIEF FIRST SECTION NAVAL SECTION IMPERIAL HEADQUARTERS
         CHIEF SECOND SECTION NAVAL SECTION IMPERIAL HEADQUARTERS
    INFO:  CHIEF OF STAFF COMBINED FLEET
           KURE NAVY YARD
ON 30 OCTOBER, THIS FLEET WILL PICK UP FROM 5 TO 10 NEAR SURFACE(?)
TORPEDOES AT SASEBO MILITARY STORES DEPARTMENT(.) CLASSES ON THIS TORPEDO
WILL BE HELD AT KANOYA FOR ABOUT FIVE DAYS FROM THE 31ST AND THEN WILL BE
SHIFTED TO FIRING PRACTICE .… BY WORKING NIGHT AND DAY, IT SHOULD BE
POSSIBLE TO COMPLETE 10 (A SPECIAL ATTACHMENT FOR TORPEDOES, PROBABLY BOW
OR STERN PLANES) BY 5 NOVEMBER.

The most significant point was that modifications being made on a crash basis were probably intended to produce shallow-running torpedoes. This would have been of particular interest in the 14th Naval District and Washington, where a current assessment stated that Pearl Harbor was too shallow for a conventional torpedo attack.21

21Layton, 74.

One message revealed that three carrier divisions totaling six carriers were to be equipped with the new torpedoes22 and another disclosed that practice torpedo drills were to be conducted against anchored capital ships.23

22SRN 116323.
23SRN 117665.

Of equal significance, given the Japanese reputation for surprise attacks, was the message on 3 November from the staff of the First Air Fleet to the commander of Saeki Air Base:24

24SRN 117665.
IN THE 3RD SPECIAL DRILL IN AMBUSHING, 54 SHIPBOARD BOMBERS WILL CARRY OUT
A BOMBING AND STRAFING ATTACK IN SIGHT OF SAEKI BASE FROM 0815 ON THE 4TH,
0715 ON THE 5TH AND 0815 ON THE 6TH, AND ABOUT AN HOUR OR AN HOUR AND A
HALF AFTERWARDS 54 SHIPBOARD ATTACK PLANES WILL CARRY OUT A SIMILAR BOMBING
ATTACK.

The evidence of these translations is overwhelming: the Combined Fleet was planning another Taranto. The First Air Fleet was feverishly training its carrier pilots in dive bombing and torpedo attack techniques. And shipyards were hurriedly modifying large numbers of conventional torpedoes to run in shallow water. These preparations were for a surprise carrier ambush of an unsuspecting fleet at anchor, just as the British had done to the Italian fleet anchored at Taranto on 12 November 1940. The question was, What was the objective?

Other messages revealed that the objective of the Japanese First Air Fleet was in the north Pacific, and extraordinary distance from Japan and far beyond the Combined Fleet's normal capabilities. One group of messages, of which this intercept is typical, concerned the preparation of tankers by the First Air Fleet and the almost daily drills between 4 October and 15 November in refueling carriers and other vessels while under way:25

25SRN 116588. Parentheses added by translator.
    FROM: CHIEF OF STAFF FIRST AIR FLEET       30 OCTOBER 1941
    ACTION: KUROSHIO MARU OR SHINKOKU MARU
    INFO: KURE NAVY YARD
          KURE SANDA SECTION
          MAIZURU NAVY YARD CHIEF
          COMMANDER CARRIER DIVISION 2
WHEN INSTALLATION OF GEAR FOR REFUELING UNDER TOW AND PREPARATIONS FOR
ACTION HAVE BEEN COMPLETED, KUROSHIO (KOKU/CHOO) MARU AND SHINKOKU
(KAMI/KUNI) MARU WILL DEPART SASEBO AND KURE RESPECTIVELY ON THE 13TH AND
PROCEED TO KAGOSHIMA BAY, CONDUCTING EXERCISES WITH CARRIERS ENROUTE.
REQUEST THEY LOAD FUEL OIL FOR REFUELING PURPOSES BEFORE THEY DEPART.

Another group of messages concerned the preparation of large quantities of fuel in drums and tins. For example:26

26SRN 117180.
    FROM: YOKOSUKA MILITARY STORES DEPARTMENT CHIEF    3 NOVEMBER 1941
    TO: CHIEF OF STAFF FIRST AIR FLEET
    INFO: TOKYO SUPPLY DEPARTMENT CHIEF
…THE AMOUNT WHICH WE CAN PREPARE BY THE 7TH IS 150 TONS (750 DRUMS) AND
140 TONS (8000 KEROSENE TINS) REQUIRED OF US BY MILITARY AFFAIRS BUREAU
SERIAL ______, READY BY THE 7TH. WE HAVE ASKED THE BUREAU TO MAKE
ARRRANGEMENTS FOR OBTAINING THIS FROM SOME OTHER MILITARY STORES
DEPARTMENT. HOWEVER, WE CAN PUT THIS UP IN DRUMS, IF SATISFACTORY. PLEASE
ADVISE.


By the end of October 1941 it was clear to the First Air Fleet staff that they could not come up with enough tanker capacity to supply the carriers and their escorts. This conclusion is based on a message originating with the First Air Fleet on 1 November.27 A second message not only revealed how much fuel in tins and drums was to be carried but also described how the additional weight was to be distributed on board the carriers.28

27SRN 117150.
28SRN 116566.
    FROM: CHIEF BUREAU MILITARY AFFAIRS SECTION      5 NOVEMBER 1941
    TO: CHIEF OF STAFF FIRST AIR FLEET
    INFO: CHIEF OF STAFF COMBINED FLEET
1. REGARDING THE LOAD OF DRUMS OF FUEL OIL ON SHIPS OF YOUR FLEET, ONLY
THAT WHICH IS ABSOLUTELY NECESSARY FOR OPERATIONAL REASONS [should be
loaded], AND THEN ONLY AFTER CAREFULLY CONSIDERING THE FOLLOWING POINTS
SHOULD THIS BE DONE, AS IT WILL EFFECT [sic] THE STRENGTH OF THE HULL AND
THE SHIPS PERFORMANCE.
    A. AMOUNT TO BE LOADED: AKAGI, UNDER 600 TONS; SOORYUU AND HIRYUU
UNDER 400 TONS: AND IN EITHER CASE THE MINIMUM REQUIRED AMOUNT, AND
WHENEVER POSSIBLE, AN EQUIVALENT WEIGHT WILL BE REMOVED.
    B. IN THE CASE OF AKAGI AND HIRYUU, LOAD AMIDSHIPS AND AVOID BOW AND
STERN.
    C. IN THE CASE OF SOORYUU, LOAD EVENLY, DISTRIBUTING THE WEIGHT OVER
THE LENGTH OF THE SHIP.
2. WE HAVE PREPARED 1400 TONS OF FUEL OIL IN DRUMS.

One of the most significant of the intercepted messages was sent on 1 December 1941. It offered a tantalizing clue as to the whereabouts of the objective. One of the tankers assigned to the Strike Force was Shiriya. It was at Yokosuka delayed by repairs from joining the operation. Destined to support Destroyer Division 7, the escort of Carrier Division 1, one of the last anxious messages concerning this vessel pointed directly toward the future whereabouts of the First Air Fleet:29

29SRN 115398.
    FROM: SHIRIYA                      1 DECEMBER 1941
    ACTION: COMMANDER DESTROYER DIVISION 7
    INFO: CHIEF OF STAFF COMBINED FLEET
THIS SHIP IS PROCEEDING DIRECT TO POSITION 30.00 N, 154.20E. EXPECT TO
ARRIVE AT THAT POINT ON 3 DECEMBER. THEREAFTER WILL PROCEED EASTWARD ALONG
30 DEGREE NORTH LATITUDE LINE AT SPEED 7 KNOTS.

Starting at 30 degrees north latitude 154.20 degrees east longitude, this course would have placed Shiriya at approximately 30 north, 170 east on the 8th of December (7 December in Hawaii), northeast of Wake and northwest of Midway. If solved before 7 December, this message would have shown clearly that the carriers of the First Air Fleet were not involved in a southern operation (see Figure 2).

* Fig.
* Fig. Figure 2. Aproximate position of Japanese Shiriya on 2 and 7 December 1941 shown against the route of the Japanese strike force.

In mid-November 1941, the messages intercepted since September showed the Japanese fleet planned to go on a war footing and planned a torpedo and dive bombing attack on anchored capital ships in the northern Pacific with a force of six carriers. But where was the Japanese Strike Force? Seven messages intercepted between 17 and 20 November would have provided the answer: Hitokappu Wan or Tankan Bay on a tiny island in the remote Kurile Islands.

The first three messages concerned a person named Suzuki. He was to be transported from Tokyo to the headquarters of the First Air Fleet, temporarily situated aboard the battleship Hiei located at Hitokappu Wan. This person was probably Lieutenant Commander Shugru Suzuki, who had been a Japanese spy at Pearl Harbor until early November 1941.31,32,33,34

31,Gordon Prange, At Dawn We Slept, (New York: McGraw-Hill, 1984), 315-319, 365-372.
32,SRN 116436.
33,SRN 116643.
34SRN 116920. Kunajiri, a heavy cruiser, was in the Kurile Islands Base Force.
    FROM: CHIEF FIRST SECTION NAVY GENERAL STAFF    17 NOVEMBER 1941
    TO: CHIEF OF STAFF FIRST AIR FLEET
    INFO: COMMANDER BATTLESHIP DIVISION 3
SUZUKI (1776) IS BEING SENT TO YOUR HEADQUARTERS ON BOARD HIEI TO REPORT
INSPECTION RESULTS.
    FROM: CHIEF FIRST SECTION NAVY GENERAL STAFF    18 NOVEMBER 1941
    TO: CHIEF OF STAFF, OMINATO GUARD DISTRICT
    INFO: CHIEF OF STAFF, FIRST AIR FLEET
PLEASE ARRANGE TO HAVE SUZUKI (1776) WHO WAS SENT TO 1ST AIR FLEET ON
BUSINESS PICKED UP AT ABOUT 23 OR 24 NOVEMBER AT HITTOKAPU WAN BY … OF
YOUR COMMAND.
    FROM: CHIEF OF STAFF OMINATO GUARD DISTRICT    19 NOVEMBER 1941
    TO: CHIEF FIRST SECTION NAVY GENERAL STAFF
HE (SUZUKI) WILL BE TAKEN ABOARD THE KUNAJIRI.

These messages show this spy being taken to the "standby location" mentioned in a a fourth message on 19 November from Admiral Isoroku Yamamoto, Commander in Chief of the Combined Fleet:35

35SRN 115678. Parentheses added by translator.
    FROM: COMMANDER IN CHIEF COMBINED FLEET    19 NOVEMBER 1941
    TO: ALL FLAGSHIPS COMBINED FLEET
1. THE FOURTH SERIES OF COMMUNICATIONS TESTS FOR THE COMBINED FLEET WILL BE
HELD AS FOLLOWS: (ON NOVEMBER 22 AND 23)
2. PARTICIPATING FORCES: (HOWEVER, THE FORCES PRESENTLY ENROUTE TO THE
STANDBY LOCATION WILL NOT RECEIVE TESTS.)
3. PRINCIPAL TOPIC FOR CONSIDERATION IN THEIR DRILL: INVESTIGATION AND
STUDY OF THE COMMUNICATIONZ REQUIRED TO EFFECTIVELY HANDLE THE SITUATION
UPON OPENING OF HOSTILITIES.


The last three of the seven messages also point toward the Tankan Bay location by suggesting that the headquarters of the First Air Fleet was beyond Ominato. As this one does, each concerned a submarine departing Yokosuka on the 20th and 21st. The messages traced the route of each submarine from Yokosuka to Ominato and, from there to the First Air Fleet Flagship communications Zone (see Figure 3).36

36SRN 116329/116990. See also SRN 117673, 117666 and 117674.
    FROM: STAFF SUBMARINE DIVISION TWO       20 NOVEMBER 1941
    TO: COMMANDER OMINATO COMMUNICATIONS UNIT
        YOKOSUKA COMMUNICATIONS UNIT STAFF SIXTH FLEET
        STAFF FIRST AIR FLEET
I-19 WILL LEAVE YOKOSUKA COMM ZONE ON NOVEMBER 21, AND ENTER OMINATO COMM
ZONE. AT 1600 NOVEMBER 22 WILL ENTER 1ST AIR FLEET FLAGSHIP COMM ZONE.

* Fig.
* Fig. Figure 3. Japanese submarine route.

But what was the Strike Force's objective? Four other messages intercepted in late November would, if solved, have helped unravel this mystery. Navy weather analysts, who in 1946-47 reviewed three weather messages intercepted on 27, 28, and 30 November 1941, were positive in their evaluation that these messages could easily have been solved at the time with the aid of the weather information available from the Pan American Airways weather service. Addressed to the "Striking Force," these messages, of which the following is typical, probably reflected weather conditions along the route to be followed in the northern Pacific by the carriers and their escorts.37

37SRN 116668. See also SRN 115460 and 115690.
     FROM: TOKYO COMMUNICATIONS UNIT                   27 NOVEMBER 1941
     TO: ALL FLEETS
     FROM: FIRST SECRETARY NAVY SECTION CHIEF, IMPERIAL HEADQUARTERS
     TO: STRIKING FORCE
   CHIEF OF STAFF COMBINED FLEET
1. WEATHER REPORT. THE LOW PRESSURE CENTER OF 740MM WHICH WAS "NERI O
NA" TODAY AT NOON IS ADVANCING AT A SPEED OF 45KM. WIND SPEED IS OVER 15M
[i.e., 15 meters per second] WITHIN 1000 KM TO SW OF CENTER AND ABOUT 26M
NEAR THE CENTER. THE HIGH PRESSURE CENTER IN "RU" AREA CONTINUES TO
PROCEED EASTWARD AT A SPEED OF 45KM. (Quotes as in text)
2. PRINCE HIROYASU FUSHIMI SENDS THE FOLLOWING MESSAGE TO COMMANDER IN
CHIEF NAGUMO: I PRAY FOR YOUR LONG AND LASTING BATTLE FORTUNES.

Another message concerned Russian ships in the northern Pacific. It registered definite Strike Force interest in the area as emphatically as the reported intentions of the Shiriya to follow the 30 degree north line eastward.38

38SRN 116667.
    FROM: FIRST SECRETARY NAVY SECTION IMPERIAL HEADQUARTERS
                                                         27 NOVEMBER 1941
    TO: STRIKING FORCE
   CHIEF OF STAFF, COMBINED FLEET
ALTHOUGH THERE ARE INDICATIONS OF SEVERAL SHIPS OPERATING IN THE ALEUTIANS
AREA,THE SHIPS IN THE NORTHERN PACIFIC APPEAR CHIEFLY TO BE RUSSIAN
SHIPS … THEY ARE UZBEKISTAN (ABOUT 3000 TONS … 12 KNOTS) AND AZERBAIDJAN
(6114 TONS LESS THAN 10 KNOTS). BOTH ARE WEST BOUND (FROM SAN FRANCISCO).

The messages cited so far, if solved, would have been viewed by U. S. Navy communications analysts and decision-makers as conclusive evidence that the Japanese navy was planning a northern offensive in addition to the southern one. In the messages associated with a northern operation, those concerning ambushing of anchored capital ships by carrier planes, torpedoes modified to run near the surface, a six-carrier strike force, fueling at sea, fuel as deck cargo and weather surveys of the northern Pacific would have been viewed as unambiguous indications that the attack was aimed at Pearl Harbor. Finally, the messages contained evidence which suggested an answer to the final question: When will the attack take place?

Radio silence was recognized by Navy communications analysts in Hawaii and Corregidor as an indication of either impending operational activity or total inactivity when a ship or fleet was in home port. The three messages which follow, intercepted between 10 and 25 November 1941, would probably have inspired warnings to Admirals Hart and Kimmel. The most critical message by far was the last of this group, that from Admiral Yamamoto on the 25th. In view of the other messages intercepted during this period, there is little doubt that any analyst would have misunderstood their implications.39,40

39,SRN 117687.
40SRN 116866.
    FROM: COMMANDER IN CHIEF, SIXTH FLEET       10 NOVEMBER 1941
    TO: SIXTH FLEET SUBMARINES
    INFO: ALL FLEET COMMANDERS IN CHIEF
MAINTAIN WARTIME RADIO SILENCE ON SHORTWAVE COMMENCING 0000 NOVEMBER 11.
    FROM: COMMANDER IN CHIEF ELEVENTH AIR FLEET    20 NOVEMBER 1941
    TO: ELEVENTH AIR FLEET
FROM 26 NOVEMBER WHEN PLANES (OR PLANE UNITS) ARE SHIFTED, MAINTAIN
PRECAUTIONARY SHORTWAVE SILENCE.
    FROM: COMMANDER IN CHIEF COMBINED FLEET       25 NOVEMBER 1941
    TO:ALL FLAGSHIPS
FROM 26 NOVEMBER SHIPS OF COMBINED FLEET WILL OBSERVE COMMUNICATIONS
PROCEDURE AS FOLLOWS:
1. EXCEPT IN EXTREME EMERGENCY THE MAIN FORCE AND ITS ATTACHED FORCE WILL
   CEASE COMMUNICATING.

The last message is familiar to all students of the Pearl Harbor attack.41

41SRN 115376.
    FROM: COMMANDER IN CHIEF COMBINED FLEET        2 DECEMBER 1941
    TO:COMBINED FLEET
THIS DISPATCH IS TOP SECRET. THIS ORDER IS EFFECTIVE AT 1730 ON 2
DECEMBER. CLIMB NIITAKAYAMA 1208, REPEAT 1208. (CLIMB MOUNT NIITAKA
DECEMBER 8).


Intercepted on 2 December 1941, rather than the 6th as recorded in the congressional hearings, this message was the signal to open hostilities. There was no doubt in 1945 that the underlying meaning of this message-"Attack on 8 December"-would have been understood by communications analysts in December 1941.42

42U. S., Congress Joint Committee on the Investigation of the Pearl Harbor Attack, Pearl Harbor Attack, Hearings, 79th Congress, 2nd Session (Washington: Government Printing Office, 1946), 1:185.

In 1941, Japan had one the newest and strongest fleets in the world. However, alarming gaps were discovered in the Japanese navy's capability to actually conduct a successful attack against the U. S. Pacific Fleet at Pearl Harbor. By mid-1941, the Japanese government was covering these gaps with a cynical continuation of diplomatic negotiations designed not to avert war but to buy time. Many of the hurried Japanese preparations for war were detailed in secret naval radio messages intercepted by the U. S. Navy's communications intelligence installations in Hawaii and the Philippines. Ironically the Navy's principal capability to solve these secret radio messages was diverted to higher priority tasks in the last six months of 1941 with the result that the Japanese navy, in spite of its shortcomings, attained an almost total surprise on 7 December 1941.

The accuracy of the prewar reports from Hawaii and the Philippines based only on traffic analysis (the analysis of communications activity, message headings, and operational characteristics) show without a doubt that the analysts who wrote about task forces aimed at southern objectives were intimately familiar with the operational habits of the Japanese navy. If the Japanese navy messages had enjoyed a higher priority and were assigned more analytic resources, could the U. S. Navy have predicted the Japanese attack on Pearl Harbor from Japanese navy radio communications? Most emphatically yes! If the secret messages were read at the time, the analysts would have quickly deduced Japanese intentions to conduct a major northern campaign with a six-carrier strike force. Only for one place did all the northern Pacific pieces and clues fit precisely together. An objective extraordinarily far from Japan, the ambush of anchored capital ships, shallow-running torpedoes, six major carriers in a strike force, carrier fuel stored on deck, and a demonstrated interest in the waters of the northern Pacific-all these pointed inexorably to Pearl Harbor.

BIOGRAPHICAL SKETCH

Frederick D. Parker is a senior historian in the Center for Cryptologic History at the National Security Agency. His field of concentration is the role of communications intelligence in the U. S. Navy's decision-making processes during the American-Japanese war in the Pacific. He lives in Annapolis, Maryland. This article is based on his still classified book, "A New View to Pearl Harbor," published in 1988, which traced the development of communications intelligence in the U. S. Navy from 1924 to 7 December 1941. His latest book, A Priceless Advantage: The Role of Communications Intelligence in the Battles of Coral Sea, Midway, and the Aleutians, will soon be published in the Cryptologic History Series by the National Security Agency. He is currently working on a book concerning the role of communications intelligence in the Battle for Guadalcanal.

Mr. Parker, a former Marine Corps Reserve Officer who served on active duty from 1943-1945 and 1950-1952, is a 1950 graduate of Georgetown University, School of Foreign Service, where he majored in history and economics. Retiring from NSA in 1984, he began a second career as a cleared part-time employee producing classified histories based on still classified material. The views expressed in this unclassified article are his own and not those of the National Security Agency.

The Fall And Rise Of Cryptanalysis In Canada

Bill Robinson

ADDRESS: Conrad Grebel College, Waterloo, Ontario N2L 3G6 CANADA.

ABSTRACT: In the late 1970s, Canada's Communications Security Establishment (CSE) was an organization in decline. By 1979 only one person was still doing real cryptanalysis. When Ed Cheramy died late in 1980, Canadian cryptanalysis almost died with him. Earlier in 1980, however, CSE had sent Tom Johnston to NSA to investigate what it would take to re-enter the field. This effort culminated in the purchase of a Cray X-MP supercomputer, installed in 1985, and the hiring of a new staff of cryptanalysts to make use of it. Only with these developments did CSE enter the modern age of cryptanalysis.

KEYWORDS: Canada, CBNRC, Cray, cryptanalysis, CSE, NSA.

The Communications Security Establishment (CSE) is Canada's "ultra secret" signals intelligence agency.[26] Assisted by the radio intercept stations of the Canadian Forces Supplementary Radio System, it collects SIGINT "to provide the government with foreign intelligence on the diplomatic, military, economic, security and commercial activities, intentions, and capabilities of foreign governments, individuals, and corporations."[8] The primary target of Canada's SIGINT efforts is the Soviet Union, which CSE monitors in close co-operation with the American and British SIGINT agencies, the National Security Agency (NSA) and Government Communications Headquarters (GCHQ). In comparison to these SIGINT giants, CSE is a tiny intelligence agency. But Canada's arctic geography has made CSE an important player in the SIGINT game. According to Seymour Hersh, the protocol that divides monitoring responsibilities between the SIGINT allies, the SIGINT Combined Operating List, assigns CSE primary responsibility "for collection in the polar regions," including the northern Soviet Union.[11]


SIGINT satellites have reduced the role of Canadian geography in SIGINT collection, but CSE's contribution remains an important one. It was CSE, for example, that wrote the SIGINT community's "final after-action report" on the shoot-down of Korean Air Lines flight 902 over the Kola peninsula in 1978.[11]

It is CSE's ability to contribute useful data and analyses to the broader SIGINT community that ensures it significant access to the SIGINT data and analyses produced by NSA and GCHQ. Patrick Spearey, who served as the Canadian Special Liaison Officer to NSA (CANSLO/W) from 1978 to 1980,[1] became director-general production late in 1980,[33] and is now serving as the Canadian Special Liaison Officer to GCHQ (CANSLO/L),[1] has stated that CSE's reputation as a useful contributor is "extremely important" in maintaining the flow of information from its allies: "CSE must maintain its ability to inform; it must continue to gather and obtain information from other countries." [33] To do otherwise risks being reduced to a mere collection arm of NSA/GCHQ, receiving in return only the scraps those agencies consider it useful to throw.

* Fig.
CSE headquarters, the Sir Leonard Tilley building, June 1990. The virtually-windowless concrete structure is the new addition to the Tilley building under construction.

During the late 1970s CSE was running exactly this risk. Canada's codebreaking abilities were literally on the point of extinction, and CSE as a whole was understaffed and ill-equipped. It was to take a concerted and expensive renewal effort lasting most of the 1980s, involving both extensive hiring and the purchase of a multi-million dollar supercomputer, to revive Canadian cryptanalysis and the Canadian SIGINT effort in general. This article tells the story of that revival. Emphasis is placed on the details of the Canadian cryptanalysis effort.

Canada first entered the codebreaking business during the Second World War. A permanent postwar SIGINT agency, the Communications Branch of the National Research Council (CBNRC), was created in 1946, the product of the amalgamation of Canada's wartime civilian and military SIGINT efforts.[15] Starting with a staff of 62,[15] CBNRC was fully grown by 1962, at which time it had a staff of about 600.[29] It had about the same number in 1975, when it was transferred to the Department of National Defence and became the Communications Security Establishment.[14,28] CSE operates out of the Sir Leonard Tilley building in Ottawa, which has been its headquarters since 1962.

By the late 1970s Canadian cryptanalysis was virtually extinct. CSE's targets "had become very sophisticated and difficult to analyze" and its cryptanalysis unit, the O1 section of O Group, "had a poor reputation as a dead end, being unproductive." The section's computer resources were limited and much of its work was still being done by hand. Only one O1 staffer, O1A member Ed Cheramy, was doing full-fledged cryptanalysis, and he worked only on manual cryptosystems. The other members of O1A had "moved away from cryptanalysis" and were engaged instead in "linguistic analysis," the recovery of garbled plain text, and "depth reading," which this author believes to be a form of superimposition, a method for recovering the keys of known cryptosystems (see Frank Lewis, "The Day of the Dodo," Cryptologia, January 1990, p. 19). O1B, the other subunit of O1, recovered high-frequency radio call signs through traffic analysis.

It was widely recognized within CSE that it was "no longer its commitment to serve Canada and the other allied countries." O1 "had become obsolete and unreliable" and CSE felt that it "had to bring this area up to what was expected."[33] The decision to modernize the section, and the rest of CSE, appears to have been made early in 1980. The beginning of the 1980s was a time of growing tension between East and West, and allied (particularly American) pressure is the most probable explanation for the timing of the modernization decision. However, it may also have had something to do with the 1979 appointment of Peter Hunt to the position of director-general production, replacing Jack Dornan, who had held the job since 1957.[15,31] Hunt was a man on his way to the top. Serving in all the right jobs, analyst/reporter, directorate assistant (a key policy and planning position),[31] CANSLO/W,[2] L group head, and director-general administration, he had probably already been tapped to become CSE chief by the time he returned from a year of French language training to take up the position of director-general production. By the beginning of 1980, Hunt may already have been reshaping the organization he was soon to direct (he became chief in September of 1980).[31]

Whatever the cause, by 1980 change was on its way. In December 1979 O1 head John Orobko and two of his Second World War SIGINT vets, Frank Cumming and Camille Fiset, retired[22,30,33] and in the spring of 1980 Tom Briscoe, the new head of O1, sent O1A staffer Thomas Johnston south to NSA for a week of consultations. Johnston's mission was "to assess and study the cryptoanalysis [sic] performed there and recommend to CSE how it could reenter the area of cryptanalysis." A physics PhD and registered engineer who was at home in the world of advanced mathematics, Johnston had joined CSE only in 1977. Now he was to become the key figure in the organization's cryptanalytic renaissance.[33]

Johnston's report made it clear that the only way for CSE to re-enter the cryptanalysis business was to hire a new cryptanalysis staff with advanced mathematical training and purchase a powerful supercomputer to support their work. However, although CSE's management committee accepted this conclusion, in 1980 the agency didn't have the money for a supercomputer.[33] CSE's largest computers at the time were its IBM 370 mainframes;[3] O1 was accessing the SIGINT databases kept on the mainframes, but using obsolescent PDP-8 and PDP-11 computers to run its linguistic analysis and call sign recovery software. The section's budget for extra purchases was only $8,000, and it would take time to get approval for a multi-million dollar computer purchase.[33]


While they were waiting, therefore, Johnston recommended that O1 try to use one of the IBMs for cryptanalysis. NSA doubted that any useful results could be achieved with this system, but it nonetheless assisted Johnston's efforts, supplying daily key information on the system he was trying to break.[33]

Meanwhile, O1A went ahead with hiring new mathematicians to replace the people retiring or transferring out of the unit. The first of this crop, John Mulholland (who was already a CSE employee) and Luc Laplante, joined O1A later that year, and immediately set to work helping Johnston (who had since become head of O1A) on the "machine cryptanalysis" project.[33]

It was none too soon. Late in 1980, while Johnston's team was still labouring over its software, Canada unexpectedly found itself out of the codebreaking business. Ed Cheramy, CSE's sole working cryptanalyst, had died.[32] During the period between Cheramy's death and January 1981, when Michel Jette was transferred from M Group and took over Cheramy's work, nobody at CSE was actually "breaking codes." (Jetté later did "outstanding" work on manual systems, but in 1982 he was reassigned to machine cryptanalysis and CSE's manual work was effectively dropped. Former cryptanalyst Celia McInnis has stated that she saw only one "manually generated code" during the entire period – August 1985 to March 1988 – that she worked in O1.)[33]

In the spring of 1981, Johnston and co. finally achieved a minimal cryptanalysis capability on the IBM. Nevertheless, all was not well on the machine side, either. Estimating that the IBM "would cease to be effective [for cryptanalysis] within one to two years," Johnston (by then head of O1) recommended again in 1981 that CSE purchase a "suitable scientific computer" for cryptanalysis. At the time NSA was considering buying the Control Data Corporation's Cyber 740, so Johnston recommended this computer to his bosses. The Cyber 740 would have cost only $3-5 million, not as much as some supercomputers, but CSE was still unconvinced it could afford such an expenditure, so the purchase was again put off.[33]

Fortunately for Johnston, CSE's money problems were not to last forever. In fact, it was that same year that CSE's budget constraints began to ease and the agency entered its first significant period of growth in nearly 20 years.[9]

Late in 1983 or early in 1984, the purchase of O1's cryptanalysis computer was finally approved (the computer approved may have been the Cyber 740 or a Cray supercomputer; the evidence is unclear on this point). At the same time, 14 new positions were authorized in O1A: 8 cryptanalyst- mathematicians, 3 computer operators, 2 engineers, and 1 assistant. By the end of 1984, most of these positions had been filled,[33] and CSE had grown to a total staff of about 670.[10]

In April 1984, with the computer project approved and under way, Johnston again visited NSA, where he learned to his dismay that the Americans had decided to use a different model of computer. CSE had "a 40-year catch-up" to accomplish in machine cryptanalysis, and Johnston knew it would be necessary to use NSA software and expertise in the effort, so he returned to Ottawa and recommended that CSE purchase a more expensive Cray X-MP. The X-MP is Cray's second generation of supercomputer; developed in 1982, it superseded the original Cray-1 model developed in 1976. The purchase was formally approved in June 1984, under the name Project PM-40 (or Project Elevator), and a purchase contract was signed with Cray Research Inc. the following December.[4,33]

CSE's Cray X-MP/11 (modified) supercomputer (see Figure 1) was delivered to the Sir Leonard Tilley building in March 1985. Purchased for a mere $8,536,000 (Canadian), CSE's Cray was the budget version of the X-MP, containing just 1 central processing unit and 1 megaword of central memory (hence the designation X-MP/11).[4] The top-of-the-line X-MP/416 contains 4 CPUs and 16 megawords of central memory.

* Fig.
* Fig. Figure 1. The proposed system.[4]

Nonetheless, it was an impressive machine. At the time of its purchase, it was the most powerful computer in Canada (Environment Canada and the University of Toronto have since purchased more powerful X-MPs). It is still more powerful than anything possessed by the Soviet Union.

Six 600-megabyte disk drives were needed just to keep the data-hungry computer fed. "Front-end interfaces" and a magnetic tape processing system gave the Cray access to additional data stored on CSE's other computer systems. Sixteen terminals connected the cryptanalysts and operators to their number-cruncher.[4]

Even the maintenance of the machine was a multi-million dollar job. Cray Research's Canadian subsidiary, Cray Canada Inc., was contracted to conduct all major maintenance under a five-year deal worth $3,545,000 (later increased to more than $4 million to allow for inflation), making the complete purchase and maintenance cost of the computer $12,082,000.[4]

The maintenance of the Cray is no simple job. The machine is required to run 24 hours a day, 7 days a week, 90 percent of the time. Cray Canada's maintenance personnel work on-site in the Tilley building and must be Canadian nationals cleared for Top Secret-Special Access, the same general clearance that CSE's cryptanalysts hold[4,33] (of course, the cryptanalysts also hold additional, specific SIGINT clearances).

The computer has undergone continuous upgrades since its arrival in 1985. The first element to be upgraded was the disk subsystem, which was traded in for the DD-49 system, with twice the capacity, in April 1986. The trade cost CSE an extra $849,848.[5] In 1987-1988, a much more ambitious upgrade was undertaken. Three megawords of central memory were added to the X-MP/11, making it an X-MP/14. This additional central memory enables more complex problems to be tackled without slowing the system down by resorting to external storage systems. At the same time, a special 64-megaword Solid-state Storage Device (SSD) was added to the system. According to Cray Research, an SSD "allows the development of algorithms to support [input/output]-intensive applications such as out-of-memory solution techniques. It is used for large prestaged or intermediate files generated and manipulated repeatedly by user programs." This upgrade – apparently planned from the beginning – cost CSE a total of $4,340,880,[6] more than half the cost of the original X-MP/11, for a total expenditure on the Cray of about $13,726,938 (more than $19 million including maintenance fees to date).


The NSA contribution to CSE's Cray was indispensable. NSA not only supplied the basic Cray SIGINT software, the Folklore operating system,[4,33] it also trained a number of O1's cryptanalysts in the use of the machine. "Operations required that O1 have in residence the expertise to continuously modify the operating system. Thus, Dr. Johnston arranged for the American counterparts to train the computer operators and the O1 staff for CSE."[33]

The first O1 staff member sent for training at NSA was Christina Sattler, a CO-3 (trainee) cryptanalyst, who had joined O1B in 1982 to run the call sign software on the PDP-11. Early in March 1984, Sattler started a 14-month stint at Fort Meade, returning to Ottawa late in April 1985, just in time to help get the Cray up and running. Sattler was promoted to CO-4 (a fully-trained cryptanalyst) upon her return to CSE. Ultimately, however, she proved to be something of a disappointment, spending 6 months analyzing "particular sign targets" and having "difficulties breaking the system."Johnston "found her more competent in support work to the computer than as a cryptanalyst" and in 1986 Sattler transferred to another section of O Group where she helped run the Cray for O1.[33]

John Mulholland, then head of O1A, also went to NSA for training in 1984, starting a 12-month stint at Fort Meade in April and returning, like Sattler, just in time to help get the Cray running.[33]

Finally, upon the return of Mulholland and Sattler, Luc Laplante was sent to NSA for 3 years of training, returning to CSE in the summer of 1988.[33]

In addition to NSA's training assistance, CSE also conducted its own training to prepare for the Cray's arrival. One such course, run in February 1985 "as a result of direct operational requirements," was shift-register cryptology, which required 3 weeks of full-time study. Among the 16 students who took the course were 5 members of O1, nearly half of O1's working cryptanalysts. (Most of the other students were from S Group in CSE's COMSEC arm.)[23]

Modern cryptanalysts need an extensive background knowledge of both mathematics and computer programming. As Johnston wrote in 1986, "Although it is fair to say that one need not necessarily be a graduate Mathematician to be a good cryptanalyst, modern cryptologic developments make it extremely unlikely that one can be a good cryptanalyst without having a good basis in Mathematics. Our recruitment processes have therefore been aimed at the selection of personnel with good analytic abilities who also happen to have the requisite formal mathematical education."[16]

Even with this background, new members of O1 undergo up to 3 years of classroom and on-the-job training before they become fully-qualified cryptanalysts.[24] Required internal courses include:

• CA-105 – Introduction to Cryptography and Exploitation of Manual Cryptosystems
• CA-107 – Exploitation of Manual Cryptosystems
• CA-123 – Shift-Register Cryptology
• CA-223 – Advanced Shift-Register Cryptology
• CA-247 – Vocoders for Cryptanalysts
• MA-213 – PTAH
• MA-246 – Cryptomathematics for Mathematicians
• MA-256 – Signal Processing Mathematics[16]

The cryptanalysts also require extensive computer programming knowledge and specific training on using the Cray, starting with Introduction to IMP/Folklore.[24] (See Appendix for more details on qualifications for cryptanalysts.)

In addition to the acquisition of the Cray, a number of organizational changes were also undertaken as O1 was transformed during the 1980s. In 1982 depth reading was transferred out of O1A.[33]

In June 1984 O1B was disbanded entirely, with its call sign recovery work, more formally known as the "[deleted] Identification Systems Task," transferred to M Group, which previously had been the recipient of O1B's product.[21] The 4 staff positions in O1B were transferred to M4B,[33] leaving only the head of O1, the section clerk, a cryptanalytic engineer, and O1A in O1 section.

By the summer of 1985 O1's Cray was finally running and O1A had grown to a staff of about 16.[20] Administering the unit had become too much work for one unit head (Mulholland) to handle and so, effective 2 July 1985, O1A was broken into 2 parts: a smaller O1A (Cryptanalysis Research and Diagnosis) with 8 members, and a new O1B (Computer Production) also with 8 members. O1E (the Cryptanalytic Engineer) remained unchanged with its single member, leaving the new O1 with a total staff of 19 (see Figure 2).[19]

* Fig.
* Fig. Figure 2. Organization Chart of O1, 1985.

Finally, in 1986 O Group itself was reorganized. O1B's computer support personnel were transferred out of the section and combined with O2's computer support personnel (O2 was a traffic analysis section) to create a new O3 section, with 6 or 7 members, which became responsible for the "pure computer sciences support" for O Group. Production cryptanalysis runs remained with the 6 remaining members of O1B, and O1A was unchanged.

These final changes, although minor, effectively completed O1's six-year transformation into a real cryptanalysis section. About 17 personnel were left in the O1 section as a whole: 15 of these had extensive and up-to-date cryptanalytic training and 12 were working cryptanalysts.

The renewal of Canadian cryptanalysis was expensive, but, according to Patrick Spearey, it was worth it: "since the acquisition of the Cray computer and the hiring of mathematical and computer sciences graduates, CSE's reputation has gone up immeasurably and it has enhanced its capability in the business."[33]


According to one source, the overall modernization of CSE during the 1980s cost Canadians about $100 million.[27] In addition to a continuing increase in staff (now more than 900)[7] and the purchase of the Cray, several other initiatives were undertaken to bring CSE into the modern age of SIGINT. Since 1988, a number of CSE's analysis-reporting sections, apparently including M4, have become 24 hour/7 day operations and have established quick reaction reporting teams, enabling them to provide more timely SIGINT reporting and work more closely with CSE's Watch Office, the CANSOC (CANSOC probably stands for Canadian SIGINT Operations Centre).[17] The Supplementary Radio System detachment at the Tilley building has been enlarged and upgraded to the status of an independent unit, the 771 Communications Research Squadron.[13] Finally, construction has begun on a $35.1 million addition to the Tilley building that will nearly double the size of the facility.[25] The addition, scheduled for completion in 1992, uses "certain innovative construction techniques… to prevent the emission of potentially compromising electromagnetic radiation." Its construction will enable CSE to consolidate its newly-enlarged staff in a single, modern, secure site.[34]

Ironically, CSE has managed to accomplish all this just as the "Soviet Threat" it was largely directed toward crumbled away. There is still much to be monitored in the Soviet Union (in some respects, the usefulness of such monitoring is greater now than ever), but many future targets will be elsewhere. What, and who, will be the targets in the post-Cold War world? CSE's long-range plan, the CSE 2000 document, reportedly underwent a rapid rewrite in 1990 to try to answer just that question. But the results of the rewrite will remain highly secret. Will CSE continue to grow? Will it remain the same size? Will its targets change? Where will CSE's eavesdropping powers be directed?

As the parliamentary committee reviewing the Canadian Security Intelligence Service recently reported, Canadians are right to be concerned about the potential for the abuse of CSE's powers.[12] The Canadian government currently shows few signs of permitting any public "oversight" of the operations of CSE, refusing even to tell members of Parliament how many employees CSE has (905 as of December 1990) or how much money the agency spends ($36.2 million in salaries alone in fiscal year 1989-1990).[7]

Nevertheless, the day of oversight is probably coming. CSE is said to be already preparing internally for the advent of some form of oversight committee, parliamentary or otherwise. The concern that such a committee would spill too many of CSE's secrets is unjustified. Indeed, if Canada's experience with the Security Intelligence Review Committee is any guide, the public will learn less about CSE's "secrets" from the committee than it will from this article, which was collected entirely from public sources.

APPENDIX: CSE REQUIREMENTS FOR CRYPTANALYSTS, EARLY 1980S [18] (FORMERLY CLASSIFIED TOP SECRET UMBRA)

This position requires a very good knowledge of Mathematics, acquired through University graduation in a relevant discipline, and a good understanding of its application to cryptanalysis. The incumbent would normally have up to eight years experience in Mathematics and cryptanalysis. Rated experience would include:

• relevant undergraduate and graduate studies in related Scientific/Engineering disciplines,
• training as a cryptanalyst as a result of participation in [NSA?] integration/training programs, and
• in-house training and experience as a cryptanalyst.

Knowledge of the following areas of Mathematics is required:

• Probability and Statistics including Stochastic and Markov Processes,
• Linear Algebra and Abstract Algebra including Finite Field Theory, Shift Register and Polynomial Theory,
• Communications, Information and Coding Theory,
• Sampling Theory,
• Digital Filtering,
• Numerical Methods, and
• Fast Transformation Algorithms.

Extensive knowledge of computer programming and operating systems is required. Operating system proficiency is necessary for the VAX, IBM 370 and the CRAY XMP and entails familiarity with:

• VMS and UNIX,
• VM/SP, CMS, and VS1SYS,
• FOLKLORE.

Programming languages required are:

• DEC Assembler,
• IBM Assembler,
• CRAY Assembler,
• Register IMP Assembler,
• FORTRAN,
• PL1,
• IMP.

The incumbent must be familiar with networking and protocol hardware, systems and architectures employed throughout the cryptanalytic community such as:

• Network Systems hardware,
• Netex protocol system,
• Transf bulk transfer system.

Adoption of other computer languages, operating systems, networking and forwarding systems may be required as the task demands. The incumbent must be flexible and quite adaptable to change as technology and practices improve.

The incumbent must develop and maintain familiarity with standard data formats and retrieval procedures used throughout the [cryptanalytic community?] to determine their impact upon our processing systems.

The incumbent must be fully aware of the various applications software packages available in the cryptanalytic community and their particular relevance and usefulness to current projects.

A broad knowledge of various modulation methods and multiplexing schemes is required such as:

• Amplitude Modulation,
• Frequency Modulation,
• Phase Modulation,
• Pulse Modulation,
• Frequency Division Multiplexing (FDM),
• Time Division Multiplexing (TDM),
• Time Division Multiple Access (TDMA).

Knowledge of various signal types, employed by potential targets, and their accessibility is necessary such as:

• teletypewriter,
• analog and digital voice and vocoder,
• facsimile,
• data systems.


REFERENCES

1. Canadian Representatives Abroad. 1978. Ottawa, Ont: Department of External Affairs. June. Also November 1980, November 1988, November 1990.
2. Canadian Representatives Abroad and Representatives of Other Countries in Canada. 1967. Ottawa, Ont: Department of External Affairs. August.
3. Computer Programmer (careers ad). 1981. Ottawa Citizen. January 10. 40.
4. Contract 12213-4-9120. 1984. Ottawa, Ont: Department of Supply and Services. Also Amendments 001-004. Released in severed form under Access to Information Act.
5. Contract 12213-5-0250. 1985. Ottawa, Ont: Department of Supply and Services. Released in severed form under Access to Information Act.
6. Contract W2213-7-4161. 1987. Ottawa, Ont: Department of Supply and Services. Also amendment 001. Released in severed form under Access to Information Act.
7. Dominguez, Christine, analyst, Statistics Canada. 1991. Letter to author. May 8.
8. Elcock, Ward, Deputy Clerk, Security and Intelligence and Counsel, Privy Council Office. 1990. Minutes of Proceedings and Evidence of the Special Committee on the Review of the CSIS Act and the Security Offences Act. Ottawa, Ont: House of Commons. Number 27, April 24. 5-6.
9. Federal Government Employment, Catalogue 72-004 Quarterly. 1981. Ottawa, Ont: Statistics Canada. January-March. Also April-June, July-Sept, Oct-Dec.
10. Federal Government Employment. 1984. Ottawa, Ont: Statistics Canada. October-December.
11. Hersh, Seymour. 1987. "The Target is Destroyed". New York, NY: Vintage. 67n, 347n.
12. In Flux But Not in Crisis, Report of the House of Commons Special Committee on the Review of the CSIS Act and the Security Offences Act. 1990. Ottawa, Ont: House of Commons. September. 152-153.
13. National Defence Telephone Directory, National Capital Region. 1989. Ottawa, Ont: Department of National Defence. Spring. E-161.
14. Ninth Annual Report, 1975-1976. 1976. Ottawa, Ont: Public Service Staff Relations Board (PSSRB). 114.
15. O'Neill, Kevin, ed. 1987. History of CBNRC. Ottawa, Ont: CSE. Released in severed form under Access to Information Act.
16. PSSRB file 166-13-17850, exhibit 31 (Requirements for a Career in Cryptanalysis, CSE file O.109-86, 3 March 1986). 1990. Ottawa, Ont: PSSRB.
17. PSSRB file 166-13-17850, exhibit 63 (Edwina Slattery: CSE competition no. M-144, [1987]). 1990. Ottawa, Ont: PSSRB.
18. PSSRB file 166-13-17850, exhibit 107 (untitled, undated). 1990. Ottawa, Ont: PSSRB.
19. PSSRB file 166-13-17850, exhibit 110-A (Reorganization of O1, CSE file O.350-85, 17 June 1985). 1990. Ottawa, Ont: PSSRB.
20. PSSRB file 166-13-17850, exhibit 111 (O1 Position Code Numbers, CSE file O.359-85, 17 June 1985). 1990. Ottawa, Ont: PSSRB.
21. PSSRB file 166-13-17850, exhibit 112 (Resubordination of the [deleted] Identification Systems Task, CSE file ORG/6-4/ORG/6-2, 3 May 1984). 1990. Ottawa, Ont: PSSRB.
22. PSSRB file 166-13-17850, exhibit 113 (O1 - 1979-1986, undated). 1990. Ottawa, Ont: PSSRB.
23. PSSRB file 166-13-17850, exhibit 132 (Staff training and development: course notice, CSE file U2/012-85, 7 January 1985). 1990. Ottawa, Ont: PSSRB.
24. PSSRB file 166-13-17850, exhibit 133 (O1 training requirements, CSE file O.19-86, 9 January 1986). 1990. Ottawa, Ont: PSSRB.
25. Public Works Canada Annual Report 1989-90. 1991. Ottawa, Ont: Department of Public Works. 42.
26. Seaborn, Blair, former Intelligence and Security Co-ordinator, Privy Council Office. 1990. Minutes of Proceedings and Evidence of the Special Committee on the Review of the CSIS Act and the Security Offences Act. Ottawa, Ont: House of Commons. Number 16, February 20. 10.
27. Spy agency was given millions, CBC reports. 1989. Vancouver Sun. March 21. A8.
28. Tenth Annual Report, 1976-1977. 1977. Ottawa, Ont: PSSRB. 118.
29. Tillian (CBNRC/CSE newsletter). 1964. Ottawa, Ont: CSE. May. 4. Released in severed form under Access to Information Act. Data for other years collected by the author from a variety of sources.
30. Tillian. 1980. Ottawa, Ont: CSE. Spring. 16, 38. Released in severed form under Access to Information Act.
31. Tillian. 1980. Ottawa, Ont: CSE. Autumn. 2. Released in severed form under Access to Information Act.
32. Tillian. 1981. Ottawa, Ont: CSE. Number 1. 17. Released in severed f.orm under Access to Information Act.
33. Wexler, Muriel Korngold. 1990. Record of decision, file 166-13- 17850. Ottawa, Ont: PSSRB. March 12. All statements attributed to CSE personnel from this source are paraphrases by Wexler of oral testimony presented during the adjudication proceedings.
34. Yonson, Doug. 1989. The secret's out: Spy headquarters to get $30-million addition. Ottawa Citizen. January 22. A1-A2.


BIOGRAPHICAL SKETCH

ll Robinson is a researcher for Project Ploughshares, the Canadian Council of Churches project on militarism and disarmament. In his spare time he is working on a book about Canadian signals intelligence from the Second World War to the present. He would appreciate contact with anyone who has information on this subject.

Origins Of Cryptology: The Arab Contributions*

*This paper is based on two presentations given by the author at the Swedish Royal Institute of Technology in Stockholm and at the University of Linkoping in Linkoping SWEDEN during September 1990.

Ibrahim A. Al-Kadi†

† The author is on temporary leave from his permanent position as an Associate Professor, Electrical Engineering Department, College of Engineering, King Saud University, P. O. Box 800, Riyadh 11421 SAUDI ARABIA.

ADDRESS: Advanced Electronics Company, Ltd., P. O. Box 90916, Riyadh 11623, SAUDI ARABIA.

ABSTRACT: Recently discovered old manuscripts show that the origin of cryptology, and the Arab contributions to it, are older and more extensive than previously thought. The word 'cipher' in European languages comes from the Arabic word [underdot s]ifr. The 9th-century Arab scientist al-Kind[overbar i] is the author of the oldest known book on cryptology, antedating any other by more than 300 years. This paper highlights the specific contributions of some Arab cryptologists based on newly discovered documents that include books of al-Kind[overbar i], ibn Adl[overbar a]n and ibn ad-Duraihim. Factors behind the emergence and advancement of Arab cryptology are discussed. The discoveries reported in this paper push the frontiers of the history of cryptology back by about 500 years.

KEYWORDS: History of cryptology, Arab cryptology, al-Khal[overbar i]l, al-Kind[overbar i], ibn Dunain[overbar i]r, ibn Adl[overbar a]n, ibn ad-Duraihim, combinatorics, statistics, statistical cryptanalysis.

1. INTRODUCTION

In his celebrated history of cryptology, The Codebreakers, David Kahn asserts that "Cryptology was born among the Arabs" [9, p. 93]. Kahn attributes most of his account of Arab cryptology to the section on cryptology in the 1412 encyclopedia Subh al-A'sha, written by the Arab scholar al-Qalqashand[overbar i] [2,9,13]. Nearly all of al-Qalqashand[overbar i]'s writings on cryptology was taken from an earlier and more extensive book of ibn ad-Duraihim (1312-1361) entitled Mift[overbar a][underdot h] al-Kun[overbar u]z f[overbar i] I[underdot d][overbar a][underdot h] al-Marm[overbar u]z (Treasured Key for Clarifying Ciphers), which was considered among the lost books of cryptology. This book was recently discovered and published [12].

Actually, Arab cryptology is much older than Kahn's account suggests, and Arab contributions to it are much more extensive than previously thought. Recently discovered old manuscripts show that the origin of cryptology is attributed to the 9th-century Arab scientist al-Kindi. His study of cryptology is the oldest available book on the subject, antedating any other by about three centuries, although his predecessor al-Khalil (718-786) is reported to have written Kitab al-Mu'amma (The Book of Cryptographic Messages) about a century earlier. Unfortunately this book has not been found.

The aim of this paper is to report and discuss not-widely-known Arab contributions to cryptology. The study is largely based on the recent discovery of three old Arab manuscripts of cryptology including al-Kindi's book, a book by ibn Adlan, in addition to the long-reported but never-seen book by ibn ad-Duraihim. These books were edited and published (in Arabic) by a group of researchers at the Arab Academy of Damascus [12].

The next section briefly discusses the factors that led to the emergence and advancement of cryptology among the Arabs. The origin of the word 'cipher' is touched upon in Section 3. Section 4 introduces some known Arab cryptologists and their works in general. The following five sections (5 to 9) highlight the contributions of al-Khalil, al-Kindi, ibn Adlan, ibn Dunainir and ibn ad-Duraihim, respectively. The original contributions of al-Kindi, in particular, are discussed in some detail in Section 6.


2. FACTORS BEHIND THE ARAB ADVANCEMENT IN CRYPTOLOGY

Many factors contributed to the early emergence and advancement of cryptology in the Arab civilization. These can be grouped into five main related factors, each of which is discussed in the following.

2.1 Translation

Like many other rising civilizations, the Arabs considered knowledge as a shared human wealth. They started, shortly after their emergence from Arabia, to absorb the learning of older neighboring civilizations. One important means of their acquiring knowledge was through translation. In particular, the first three centuries of the Islamic civilization (700-1000 A.D.) witnessed intensive translation activities into Arabic of major works written in Greek, Indian, Farsi, Syriac, Armenian, Hebrew, Roman and other languages. The Caliph al-Ma'mun (reigned 813-833) established the famous Bait al-Hikmah (The House of Wisdom) in Baghdad as a national translation academy, state library and research center [1,3]. So much emphasis did the Arabs place on acquiring other cultures' knowledge that Greek manuscripts were obtained from the Byzantine Empire through peace treaties in return for maintaining the status quo. The main subjects of translation included philosophy, astronomy, mathematics, physics, chemistry and medicine. Some source books were written in dead languages forcing the Arab scholars to study them as ciphertexts and to classify their letters and symbols. They developed cryptanalytic techniques to try to understand those writings. In addition, some writings, were enciphered, especially in the fields of philosophy, theology1, witchcraft and chemistry2.

1Some extremist sects of Islam cultivated cryptography to conceal their writings from the orthodox [9, p. 93].
2Chemistry was considered a precious secret because it was believed that through chemistry cheap metals could be converted to gold!

A number of Arab scholars have written on other languages and alphabets. Thob[overbar a]n al-Masry (? - 859) wrote a book entitled [underdot H]all ar-Rum[overbar u]z wa Br[overbar a]'a al-Asq[overbar a]m f[overbar i] O[underdot s][overbar u]l al-Lugh[overbar a]t wal-Aql[overbar a]m (Solution of Symbols and Answers of Doubts on Principles of Languages and Scripts) [12]. The book of Ahmad ibn Wahshiyyah (? - 919), Shawq al-Mustah[overbar a]m f[overbar i] Ma'rifat Rum[overbar u]z al-Aql[overbar a]m (Seekers' Joy in Learning about Other Languages' Written Symbols) contained 93 alphabets of different languages [12]. Nine centuries later, this book was translated from Arabic into English by the orientalist Joseph von Hammer and was published in both languages in London in 1806 under the title Ancient Alphabets and Hieroglyphic Characters Explained [2,9,7]. The French scholar Sylvestre de Sacy published a study about the book in Paris in 1810 that might have helped the famous archeologist J. F. Champollion in deciphering hieroglyphics [12].

2.2 Linguistic Studies

Being the language of the Holy Qur'[overbar a]n, Arabic spread swiftly across vast areas of the world. It also served in different parts of the Islamic empire as a unifying official and scientific language. The first four centuries of the Arab civilization, in particular, witnessed intensive activities in studies of different linguistic aspects of Arabic, including phonetics, morphology, syntax, semantics, lexicography, grammar, prosody and computational linguistics. Such studies were prerequisites for the advancement of cryptography and cryptanalysis. Consequently many of the Arab cryptologists were also prominent linguists (see Sections 4-9).

2.3 Administrative Studies

The need of the emerging Islamic state for administrative organization and services led the Arabs early on to concentrate on developing secretarial and management skills especially in the fields of composition and correspondence. The practical need to protect sensitive state affairs, particularly in the mails, was one of the main driving forces behind using, and therefore writing about, cryptography and cryptanalysis. The hands-on experience of practicing message concealment and codebreaking is evident in the available biographies of many Arab writers and secretaries who were considered among the most important personnel in government offices, especially in the Caliph''s court [1,12]. Use of special codes by tax officials and accountants is well documented [9,10].

Many Arab administrative scientists wrote on cryptology either within their books of general administration or separately. Among those were Ab[overbar u] Bakr as-[underdot S][overbar u]l[overbar i] (? - 946) in his book Adab al-Kutt[overbar a]b (The Secretaries' Manual), Is-[underdot h][overbar a]q al-K[overbar a]tib (∼10 Cent.) in his book al-Burh[overbar a]n f[overbar i] W[overbar o]j[overbar o]h al-Bay[overbar a]n (The Authoritative Guide to the Art of Articulation), the '1;father of sociology' Abdur-Rahm[overbar a]n ibn Khald[overbar u]n (1322 - 1406) in his book al-Muqaddimah (The Introduction [to his history Encyclopedia]) which was called by the British historian Arnold Toynbee "undoubtedly the greatest work of its kind that has ever yet been created by any mind in any time or place", and al-Qalqashand[overbar i] in [underdot S]ub[underdot h] al-A'sh[overbar a] [2,9,10,12,13].


2.4 Public Literacy

The Arab-Islamic civilization witnessed an expansion of reading and writing skills never approached by any previous civilization. This unprecedented level of literacy advanced cryptology in two ways:

• It increased the need to protect sensitive written messages. (In illiterate societies a mere writing of messages guaranteed their secrecy).
• The larger pool of educated people bred many qualified scientists in fields related to cryptology, such as translation, linguistics, mathematics, and administrative sciences.

2.5 Advanced Mathematics

The Arabs acquired the mathematical knowledge of their neighbors, especially the Greeks and Indians. During the first three centuries of Islam, in particular, they embarked on an extensive campaign of translation and absorption of major mathematical works of the time including the books of Euclid (Elements), Ptolemy (Tetrabiblos and Al-magest), Archimedes, Aristotle, Diophantus (Arithmetica) and Brahmagupta (Siddh[overbar a]nta which the Arabs called Sindhind). By the early ninth century, Arab mathematicians started to produce major contributions of their own in the established fields of mathematics (arithmetic, trigonometry and geometry), but, more importantly, they pioneered in establishing two new fields, namely algebra and statistics, both of which are extremely important in modern cryptology. The glorious history of Arab mathematics cannot possibly be given a fair treatment here, but excellent references abound (see for example [1-6, 11, 14-16]). Here only aspects related to cryptology will be discussed.

Al-Khw[overbar a]rizm[overbar i] (780 - 847?) is the most famous Arab mathematician. Algorismus, as he was known in Latin, was probably the best mathematician of his time [11]. He wrote two books on arithmetic and algebra, which played important roles in the history of mathematics. The original Arabic version of his first book (written around 820) on arithmetic is lost, but its contents have survived in different Latin translations. It was through those Latin versions of al-Khw[overbar a]rizm[overbar i]'s book that the (Hindu-) Arabic numerals were introduced into Europe (see Section 3). Al-Khw[overbar a]rizm[overbar i]'s second book, al-Jabr wal-Muq[overbar a]balah, established the new field of algebra. It was translated in Spain into Latin by the Englishman Robert of Chester in the 12th century under the title Algebra et Almucabala [3, 11, 9]. From al-Khw[overbar a]rizm[overbar i] and his works stem a number of common technical terms such as 'zero', 'cipher', 'algorithm', 'algebra' and 'Arabic numerals'. As Boyer wrote in his history of mathematics [3, p. 252], to al-Khw[overbar a]rizm[overbar i] belongs the title "the father of algebra."

The Arabic foundation of algebra is widely acknowledged, but this is hardly the case with statistics. Historians of mathematics (see Boyer [3, p. 397] for example) attribute the first writings on probability and statistics to correspondence between Pascal and Fermat in 1654. In his recently discovered manuscript, al-Kind[overbar i] gave the first description of statistical methods in cryptanalysis. He even explicitly required texts to be long enough to allow letter statistics to be meaningful (see Section 6). This is the world's first known writings in statistics, antedating those of Pascal and Fermat by about 800 years3. Statistical techniques were routinely used by Arab cryptologists after al-Kind[overbar i] (see Sections 6 to 9).

3Historians of science and mathematics are urged to investigate the evidences of the new findings reported here.

3. ORIGIN OF "CIPHER"

Up to the thirteenth century (A.D.), the West was using Roman numerals (I, V, X, L, C, D, M), which were very cumbersome and made the simplest arithmetic operation (e.g. addition) extremely complicated. The reason is that the Roman number system is not a decimal system and has no symbol for an empty space (the zero). The lack of the concept of zero was the great stumbling block for the medieval mathematicians in Europe [11].

The decimal number system and the zero were originally developed in India. The Arabs became aware of the new numerals,in the early ninth century, through their translation of Brahmagupta's Siddh[overbar a]nta from Sanskrit into Arabic. The new numerals together with the zero were quickly adopted and put into widespread use throughout the Islamic empire from China in the east to Spain in the west. The Hindu-Arabic numerals were later introduced to Europeans through their contacts with the Arab civilization in Spain, where these numerals became known simply as the Arabic numerals. The spread of the new numerals throughout Europe started in the twelfth century with the Latin translations of al-Khw[overbar a]rizm[overbar i]-s book on arithmetic first by Robert of Chester and then by John of Seville [11]. In the thirteenth century, many authors helped to popularize the new Arabic numerals. Among those were the French Alexander de Villa Die (∼1225), the English John of Halifax (∼1200-1256), also known as Sacrobosco, through his book Algorismus vulgaris, and the Italian Leonardo of Pisa (∼1180-1250), best known as Fibonacci, who studied under a Muslim teacher and traveled in Egypt, Syria and Greece. Fibonacci later wrote his book Libre abaci, in which he strongly advocated the use of Arabic numerals [3].


The new concept of zero confused people in the Middle Ages Europe[11]. In Sanskrit, the zero was called sunya or "empty". The Arabs translated the Indian name into its Arabic equivalent [underdot s]ifr. When the Europeans became aware of the new digit, they no longer translated the name, but adopted the concept and the symbol as well as its Arabic name [underdot s]ifr which was transformed into the Latin words cifra and cephirum (as written by Fibonacci). The two Latin words worked their way into different European languages. In Italian cephirum was changed to zefiro, zefro, and zevero which was later shortened to zero.

The French formed the word chiffre, and also took over the Italian zero. The English used zero and cipher from which the word ciphering was some times used in the sense of 'computationѺ. The German used ziffer and chiffer [11].

The concept of the zero or [underdot s]ifr or cipher was so confusing and ambiguous to common Europeans at first that a person used to say in arguments or discussions that he is "talking about something clear and comprehensible, and not about some ambiguous and far-fetched thing like the cipher." From such common confusion developed the use of the word cipher to mean concealment of clear meaning of messages or simply encryption [12]. In conclusion, the Arabic word [underdot s]ifr for the digit "zero" (0) developed into European technical terms that mean encryption.

Table 1. Some known Arab Cryptologists.
 | Scientist Name | Period | Comments | References
1 | Al-Khal[overbar il | 718 - 786 | see Section 5 | [9,12]
2 | J[overbar a]bir ibn Hayy[overbar a]n (The Chemist) [known in Europe as Geber] | ? - 815 | Hall ar-Rum[overbar u]z wa Miftah al-Kun[overbar u]z (Code Solving and Key to Tresures) | [1,4,12]
3 | Thob[overbar a]n al-Masry | ? - 859 | See Subsection 2.1 | [12]
4 | Al-Kind[overbar i] | 801 - 973 | See Section 6 | [1,4,12]
5 | Ibn Wahshiyyah | ? - >919 | See Subsection2.1 | [2,9,12,15]
6 | Mohammad ibn Ahmad ibn Kais[overbar a]n | 9th Cent. | Wrote a book on cryptography and prosody | [12]
7 | Mohammad ibn Ahmad ibn Tab[overbar a]tab[overbar a] | ? - 934 | Wrote a book on cryptanalysis preserved in the Ottoman Archive | [12]
8 | Is-h[overbar a]q al-K[overbar a]tib | ~10th Cent. | See Subsection 2.3 | [12]
9 | As'as [overbar i]bn Muhadhdhab ibn Mamm[overbar a]ti | 1149 - 1209 | Khas[overbar a]'is al-Ma'rifah f[overbar i] al-Mu'ammiy[overbar a]t (Guides of Knowledge on Cryptograms) | [12]
10 | Ibn Adl[overbar a]n | 1187 - 1268 | See Section 7 | [12]
11 | Ibn Dunain[overbar i]r | 1187 - 1229 | Section 8 | [12]
12 | Ibn ad-Duraihim | 1312 - 1361 | See Section 9 | [2,9,12,13]
13 | Ali ibn Mohammad ibn Aidamur al-Jaldak[overbar i] | ? - >1341 | Kanz al-Ikhtis[overbar a]s f[overbar i] Ma'rifat Asr[overbar a]r Ilm al-Khaw[overbar a]s (Treasures of Specialization in Learning Secrets of Elites' Science) | [12]
14 | Al-Qalqashand[overbar i] | 1355 - 1418 | See Sections 1 and 9 | [2,9,13]


4. EARLY ARAB CRYPTOLOGISTS

Cryptography has been practiced to conceal messages since antiquity by different civilizations, including the ancient Egyptian, Chinese, Indian, Mesopotamian, Greek and Roman. But in none of them was there any cryptanalysis; only cryptography existed [9]. Cryptology, the science of both making ciphers (cryptography) and breaking them (cryptanalysis), was born among the Arabs shortly after the rise of the Arab-Islamic empire. Many Arab scholars wrote on, and excelled in practicing, both branches of cryptology. Table 1 lists, in a chronological order, Arab cryptologists known to us who wrote on the subject. The space limitations, and the meager information available on some of them, preclude writing in detail on each of the individuals in the table, but we will highlight the contributions of some of them in the following five sections.

5. AL-KHAL[overbar I]L

Ab[overbar u] Abdur-Rahm[overbar a]n al-Khal[overbar i]l ibn A[underdot h]mad ibn 'amr ibn Tamm[overbar a]m al-Far[overbar a]h[overbar i]d[overbar i] (718-786), who was among the greatest Arab linguists of all times, was also the world's first great philologist. Among other achievements, his record of 'firsts' includes the following:

1. He was the first to conceive the idea of, and produce, a comprehensive dictionary. Al-Ain, a dictionary of Arabic vocabulary, is the oldest known Arabic book of its kind.
2. He was the first to study and classify the prosody of Arabic poetry.
3. He wrote the first known book on cryptology, Kit[overbar a]b al-Mu'amma (The Book of Cryptographic Messages). This book is apparently lost [9,12].
4. He used combinatorics to count all possible Arabic words by considering all different combinations of letters with and without vowels. Appendix B presents a sample of al-Khal[overbar i]l's writings on this subject. His writings on the subject are eloquent and easy to understand.

Although considered the greatest linguist of his time, al-Khal[overbar i]l was not known to be a mathematician. Yet his spontaneous use and apparent command of combinatorics (which may as well be the first such use in history) without much fanfare or elaboration raise a number of interesting questions that are worth consideration by historians of mathematics. Did the Arabs develop combinatorics and use it to such an extent that a nonmathematician linguist found it natural to employ it implicitly within a dictionary? Or was it the modesty of this great scholar that made him unaware of the importance of his ideas that could have contributed greatly to mathematics early in the 8th century?

* Fig.
* Fig. Figure 1. The first page of al-Kindi's manuscript [12].

6. AL-KIND[overbar I]

Ab[overbar u] Y[overbar u]s[overbar u]f Ya'q[overbar u]b ibn Is-[underdot h][overbar a]q ibn a[underdot s]-[underdot S]abb[overbar a]h ibn 'omr[overbar a]n ibn Isma[overbar i]l al-Kind[overbar i], the author of the oldest known book on cryptology, was born around the year 801 A.D. in al-Kufah, where his father was a governor. He grew up in Baghdad, where he got his education. He excelled in many fields including philosophy, medicine, astronomy, mathematics, linguistics and music. Al-Kind[overbar i], the first philosopher of Islam, was known as "The Philosopher of the Arabs"4. He became the caliph's court physician, and was later named the first director of Bait al-Hikmah Library and Translation Academy, where he led the ongoing translation campaigns (see Subsection 2.1). Three of the caliphs al-Kind[overbar i] served supported his research pursuits, but the unsympathetic attitude of the fourth caliph towards his philosophical thoughts brought about loss of prestige and personal fortune. His large famous personal library, the al-Kindiyyah, was confiscated and he was publicly beaten. He returned home sad and with a broken spirit; he later died in Baghdad in 873 at the age of seventy-two.

4The Arabs reserved the more prestigious title, "The Philospher" for Aristotle, for whom they held much respect and admiration

Al-Kind[overbar i] left a wealth of knowledge in diversified fields. His writings were authoritative and encyclopedic, covering about 290 titles in different subjects. One of his books, Ris[overbar a]lah f[overbar i] Istikhr[overbar a]j al-Mu'amm[overbar a] (A Manuscript on Deciphering Cryptographic Messages), is the oldest available book on cryptology, antedating any other by more than 300 years. A manuscript of it was recently discovered in the Sulaimaniyyah Ottoman Archive in Istanbul. It was edited and published by the Arab Academy of Damascus in 1987 [12]. Figure 1 shows the first page of al-Kind[overbar i]'s manuscript.


The main contributions to cryptology of al-Kind[overbar i] are briefly discussed in the following:

1. Cryptanalysis Techniques: Al-Kind[overbar i] started by discussing principles of cryptanalysis. He divided cryptograms into two types-those of normal texts and those of poetry. For normal texts he specified four cryptanalysis methods: quantitative techniques (which he called 'quantitative tricks'), qualitative techniques ('tricks'), probable words (which he called 'opening statements and praises'), and vowel-consonant combinations. For poetry he called for using prosody in addition to all the above methods of normal texts.
2. Cipher Types: Al-Kind[overbar i] gave a tree-diagram classification of the major types of cipher systems, described each, and explained how to cryptanalyze them. Figure 2a shows his tree diagram as it appeared in the manuscript, while a translation of it is given in Figure 2b. Al-Kind[Overbar i] also introduced super encipherment, but he did not discuss it in much detail. All he wrote about it is the following:

* Fig.
* Fig. Figure 2a. Al-Kind[overbar i]'s tree diagram classification of cipher types as it appears in his manuscript [12].

* Fig.
* Fig. Figure 2b. A translation of al-Kind[overbar i]'s tree-diagram classification of cipher types.

… [Super-encipherment] is composed of a number of simple [ones], by using two or more possible combinations of simple encipherments
… Solving it is more difficult [12, pp. 224 and 234].
3. Arabic Phonetics: He classified Arabic phonetics into consonants (he used the term 'silent'), long vowels (which he called "al-mu[Underdot s][Underdot s]awiyt[Overbar a]t al-'i[Overbar z][Overbar a]m" = major louds) and short vowels ("al-mu[Underdot s][Underdot s] a]sawiyt[overbar a]t a[Underdot s]-[Underdot s]igh[overbar a]r" = minor louds).
4. Arabic Syntax: Al-Kind[Overbar i] gave a detailed study of Arabic syntax, in which he explained possible and impossible letter combinations. Figure 3 shows his study in the manuscript.

* Fig.
* Fig. Figure 3. A page of al-Kind[Overbar i]'s manuscript, in which he discussed Arabic Syntax and impossible letter combinations [12].

5. Statistical Cryptanalysis: The above contributions are remarkable, but the most important and original of al-Kind[overbar i]'s contributions comprises the world's first description and use of statistical techniques in cryptanalysis (or in any other application, for that matter). In his discussion of the quantitative techniques in cryptanalysis, al-Kind[overbar i] clearly described how to use letter frequency statistics of the cryptogram to solve it. He also explained how to find these letter frequencies (by using a sample of the same language). Equally important and original is the condition he set on the length of the ciphertext under consideration; he required texts to be long enough to allow letter statistics to be meaningful. This concept, introduced by al-Kind[overbar i] more than 1100 years ago, is extremely important in statistics today. The ideas of al-Kind[overbar i] are remarkably clear in his manuscript. He wrote:
One way to solve an encrypted message, if we know its [original] language, is to find a [different clear] text of the same language long enough to fill one sheet or so and then we count [the occurrences of] each letter of it. We call the most frequently occurring letter the "first", the next most occurring the "second", the following most occurring the "third" and so on, until we finish all different letters in the cleartext [sample]. Then we look at the cryptogram we want to solve and we also classify its symbols. We find the most occurring symbol and change it to the form of the "first" letter [of the cleartext sample], the next most common symbol is changed to the form of the "second" letter, and the following most common symbol is changed to the form of the "third" letter and so on, until we account for all symbols of the cryptogram we want to solve.
It could happen sometimes that the cryptogram is too short to have all different letters. The high and low [frequency] counts will not be correct, for high and low counts are only correct in long enough messages to correspond to all places of frequent and rare occurrences so that if some letters are [too] few in one segment of the message, they will be [too] many in others. But if the cryptogram is too short, equivalence does not apply, letter ranks are not correct and [consequently] a second trick should be used to recover letters. Such a trick is qualitative which is … [here al-Kind[overbar i] wrote in detail on possible letter combinations in a language like 'al' in Arabic… etc.] [12, p. 216].
6. Arabic Letter Frequency: Later in his book, al-Kind[Overbar i] set out the first tally of Arabic letter frequencies. His sample consisted of "seven pages of Arabic" totaling 3667 letters. Al-Kind[Overbar i]'s results are shown in Figure 4, together with results of a much more recent study [8]. Not only did al-Kind[Overbar i] pioneer in computing letter frequencies, but he also analyzed his results and explained what might appear as a partial contradiction between the assertion at the beginning of his book, that vowels are the most common letters on all languages, and the fact that his results show that although the Arabic vowel 'Alif' is the most common letter, the other two vowels 'W[Overbar a]w' and 'Y[Overbar a]' come only fifth and sixth after the consonants 'L[Overbar a]m' (l), 'M[Overbar i]m' (m) and 'H[Overbar a]' (h). He explained that vowels are underrepresented in written Arabic texts because short vowels are not usually written explicitly in Arabic scripts (see Appendix A). Al-Kind[Overbar i] wrote:
We said before that vowels are naturally the most common letters in all languages… Our results show that [the consonant] L[Overbar a]m (l) is more frequent in the Arabic language than [the vowels] W[Overbar a]w and Y[Overbar a]. So are [the consonants M[Overbar i]m (M) and] H[Overbar a] (h). This is not a contradiction to what we previously said, because vowels appear in Arabic writing [only] if they are major [or long]. Minor [i.e., short] vowels do not appear in Arabic scripts… We have explained this in our [other] book [entitled] f[Overbar i][Underdot S]in[Overbar a]'at ash-Sh'ir [On the Art of Poetry]… [12, p. 236].

* Fig.
* Fig. Figure 4. Al-Kind[Overbar i]'s computed letter frequencies in Arabic, compared with recent results.

Al-Khal[Overbar i]l's combinatorics and al-Kind[Overbar i]'s statistics are, most probably, the world's first writings in the field of computational linguistics.

Thus it appears that al-Kind[Overbar i] is the father not only of cryptology but of statistics as well (see Subsection 2.5).


7. IBN ADL[Overbar A]N

Af[Overbar i]f ad-D[Overbar i]n Al[Overbar i] ibn [underdot H]amm[Overbar a]d ibn Al[Overbar i] al-Mou[underdot s]il[Overbar i] an-Na[Underdot h]wi al-Mutarjim was born in Mou[Underdot s]il (1187), educated in Baghdad, lived in Damascus for some time, and taught in Cairo, where he died in 1268 at the age of eighty-one. He was a poet who also excelled in grammar and linguistics, but he did not write many books. He left four, two of which were on cryptology. The first, al-M'ulam (The Told [Book]), is lost, while the second, al-Mu'allaf lil-Malik al-Ashraf (The [Book] Written for King al-Ashraf), was recently discovered and published [12]. Figure 5 shows the title page and the first page of the document. The book is a manual or user-guide for cryptanalysis; unlike al-Kind[Overbar i], ibn Adl[Overbar a]n does not go into cryptography much. At the beginning, he discusses qualifications necessary for cryptanalysts, examples of simple substitution ciphers and Arabic letter combinations. The main body of the book is devoted to techniques of cryptanalysis where Ibn-Adl[Overbar a]n gives 20 rules in 9 topics. In the conclusions, he works out a detailed example of solving a cipher as a training exercise for the reader. Ibn Adl[Overbar a]n's main contributions include the following:

• The concept of variable key for simple substitution. (He called it the 'controller', and used different verses of Arabic poetry as a variable key.)
• A detailed study of word spaces, identification of word starting points in a cryptogram, probable words and doubled letters.
• An explicit lower bound on the required ciphertext length. He wrote:
The text to be solved should be [at least] 90 letters long approximately because letters will have circulated about three times [3 x 28 = 84]. [12, p. 276]
• A classification of Arabic alphabet characters according to their frequencies into three groups-common (7), medium (11) and rare (10).
• An eloquent articulation of cryptanalysis methodology. In Rule 18, ibn Adl[Overbar a]n clearly stated the consecutive steps of solving a cryptogram-moving from ciphertext to plaintext systematically from the unknown (ciphertext) through the set of all possible solutions, to suspected solutions, to probable ones and finally to the confirmed solution (the correctly solved plaintext).
He briefly but lucidly explained that one should consider all possible solutions, fix the sure parts of them, neglect the impossible combinations, and try all suspected solutions using elimination to arrive at the final solution. This is probably his most impressive contribution.

* Fig.
* Fig. Figure 5. The title page and the first page of ibn Adl[Overbar a]n's manuscript [12]

8. IBN DUNAIN[Overbar I]R

Ibr[Overbar a]h[Overbar i]m ibn Mo[Underdot h]ammad ibn Dunain[Overbar i]r was born in Mou[Underdot s]il (1187), lived in Cairo and Damascus, and died in Baniyas (1229) at the age of forty-two. He wrote at least one book on cryptology, entitled Maq[Overbar a][underdot s]id al-Fu[Underdot s][Overbar u]l al-Mutarjamah an [Underdot H]all at-Tarjamah (Clear Chapters' Goals on Solving Ciphers). The historical manuscript was recently discovered but has not yet been published [12]. In addition to his use of statistical technique in cryptanalysis pioneered earlier by al-Kind[Overbar i] (see Section 6), ibn Dunain[Overbar i]r was the first to describe an arithmetical cipher, or what he called "sentence reckoning", in which cleartext characters are converted into numbers and then some simple arithmetic operations are performed on them (see Figure 7 for some examples). He was also the first to describe some elementary cipher mechanisms, such as the chessboard [12].

9. IBN AD-DURAIHIM

T[Overbar a]j ad-D[Overbar i]n Al[Overbar i] ibn Mo[Underdot h]mmad ibn Abdul'az[Overbar i]z ibn ad-Duraihim was born in Mou[Underdot s]il in 1312. A wealthy merchant who traveled between Damascus and Cairo, he held various teaching and official positions in both cities. Ibn ad-Duraihim died in Qaus on his way to Eritrea in 1361 at the age of forty-nine. He excelled in theology, arithmetic, linguistics and cryptology. He wrote about 80 books on different subjects. Ibn ad-Duraihim's most extensive work on cryptology is contained in his book Mift[Overbar a][Underdot h] al-Kun[Overbar u]z f[Overbar i] I[Underdot d][Overbar a][Underdot h] al-Marm[Overbar u]z (Treasured Key for Clarifying Ciphers) to which most of al-Qalqashand[Overbar i]' writings on the subject in [Underdot S]ub[Underdot h] al-A'sh[Overbar a] are attributed. The book of ibn ad-Duraihim was considered lost [9, p. 95] until fairly recently when it was found, in Istanbul's Sulaimaniyyah Ottoman Archives, and published in a book by Mrayati et.al. [12]. Figure 6 shows two pages of the manuscript. Our discussion of ibn ad-Duraihim's contributions will be brief and will concentrate on aspects not sufficiently covered by other authors [2,9,13].

* Fig.
* Fig. Figure 6a. The first page of ibn ad-Duraihim's manuscript [12].

* Fig.
* Fig. Figure 6b. An example of cryptanalysis as it appears in ibn ad-Duraihim's manuscript [12].

Ibn ad-Duraihim starts his book by briefly outlining the prerequisites of successful cryptanalysis. He then, also briefly, discusses the alphabets of 15 different languages including Greek, Farsi, Hindi and French. (Al-Qalqashand[Overbar i] included less than a third of this discussion in his book [13]). Like his predecessors, ibn ad-Duraihim gives an extensive study of the linguistic characteristics of Arabic, which was preserved nearly completely in al- Qalqashand[Overbar i]'s writings [2,13].


One of the most important contributions of ibn ad-Duraihim is his detailed and exquisite description of 8 systems of cipher with many variations and examples. His systems include the following:

1. Transposition with 24 variations and 31 examples including column transposition.
2. Substitution with 11 variations and 24 examples including the variable key substitution pioneered by ibn Adlan.
3. Letter addition (such as "ssifr" in "sifr") and omission (like "ifr" in "sifr").
4. Simple cipher mechanisms discussed earlier by ibn Dunain{Overbar i]r. An example of such mechanisms is a board with holes corresponding to letters of the alphabet and a string passed through holes of the intended message-a device described a millennium earlier by the Greek writer Aeneas the Tactician.
5. Arithmetical cipher. Ibn ad-Duraihim expanded on this method, pioneered by ibn Dunain[Overbar i]r, and gave 8 variations and 8 examples. Some of these examples are illustrated in Figure 7.

* Fig.
* Fig. Figure 7. Some variations and examples of arithmetic cipher as given by ibn ad-Duraihim. (See Appendix A.)

6. So-called letter-word substitution of four main types. One of these hides plaintext in the first (or second or last . . . etc.) letter of words in innocent-looking but meaningful sentences. An example of which is hiding the plaintext Ali in the ciphertext sentence pAint aLl pInk.
7. Substituting for plaintext letters, under the control of a key, names from some species or families (e.g. animals, birds, cities, plants, etc.).
8. Use of some invented symbols for letters.

In his discussion of substitution cipher (No. 2 above), ibn ad-Duraihim analyzed the various possibilities and introduced the idea of what is now known as a 'Vigenère' table two centuries before Vigenère. Ibn ad-Duraihim's extensive and lucid study of cipher types was condensed beyond recognition (to about 10 percent) in al-Qalqashand[Overbar i]'s book. Even though the presentation in [Underdot S]ub[Underdot h] al-A'sh[Overbar a] was deformed by contraction, ibn ad-Duraihim's study of cipher types was commended by Kahn as remarkable and important [9, p. 98].

At the end, ibn ad-Duraihim gives a systematic methodology for cryptanalysis including the statistical techniques of al-Kind[Overbar i] and ibn Adl[Overbar a]n. He falls short of them, however, in not giving actual frequency counts of Arabic letters but only their order of usage (rank). He also differs in that his ranking is based on the Holy Qur'[Overbar a]n. Next, ibn ad-Duraihim works out two detailed examples of solving sample cryptograms (see Figure 6b). Both the methodology and the two examples of ibn ad-Duraihim appear in their entirety in al-Qalqashand[Overbar i]'s book, and were later described by Kahn as "the first exposition on cryptanalysis in history"5[9].

5Al-Kind[Overbar i]'s exposition was 500 years earlier!

10. CONCLUSIONS

Cryptology originated, and developed to an advanced state, in the Arab-Islamic civilization (650 - 1400 A.D.). Among the factors that led to the early emergence and advancement of Arab cryptology were translation, linguistic studies, administrative sciences, public literacy and advanced mathematics. The origin of cryptology is much older, and Arab contributions to it are much more extensive, than previously reported. The new findings, reported in this paper, push back the frontiers of the known history of cryptology by more than 500 years. These findings are largely based on 8th century and later manuscripts written by the prominent Arab cryptologists al-Kind[Overbar i], ibn Adl[Overbar a]n and ibn ad-Duraihim. The contributions of these scientists, in addition to ibn Dunain[Overbar i]r, to cryptology are remarkable and important. Al-Kind[Overbar i]'s contributions, in particular, are original and pioneering. He is the father of cryptology.

Among the new findings, reported in this paper, is the fact that the first writings in statistics are due to al-Kind[Overbar i] about 800 years before Pascal and Fermat. In addition, the great Arab linguist, al-Khal[Overbar i]l, used combinatorics more than 1200 years ago.

There are, however, some unanswered questions on Arab cryptology that cannot be cleared up yet. Arab cryptologists do not mention unbreakable ciphers, unlike many later inventors of cipher systems who claimed, and genuinely believed, that their systems were unbreakable.

It should be kept in mind that the sudden and quick decline of Arab civilization may have led to the loss of many books on the subject, and may have set back cryptology [9, pp. 98-99]. That decline seriously delayed the progress of human knowledge in many other fields as well.


APPENDIX A
THE ARABIC SCRIPT

The basic Arabic alphabet consists of a set of 28 letters. The Arabic script is extended to some 90 elements by additional shapes, marks and short vowels formally recognized in Arabic morphology. Arabic, written from right to left in elegant cursive form, does not differentiate between upper and lower case. Table A-1 shows Arabic letters, together with their transliteration as used in the etymologies and the numerical values of the different letters as adopted by early Arab linguists and cryptologists. In standardizing these values, early Arab scholars adopted the "abjadi" order of letters, which is different from the alphabetical order shown in the table. They gave the first nine "abjadi" letters consecutive numbers 1 to 9; the following nine letters were assigned numbers 10, 20, 30, . . . , 90; letters 19 through 27 were given numerical values 100,200,300, . . . ,900; the last letter in the "abjadi" order, which is ghayn, is assigned the numerical value of 1000.

* Fig.
Table A-1. (Opposite) Arabic letters, their forms, transliteration, and "abjadi" values.

Notes:

1. [Overbar a], [Overbar i], and [Overbar u] are pronounced like a in father, i in machine and u in rude, respectively.
2. The form of the letter when it stands alone.
3. The form of the letter when it is joined to the preceding letter only.
4. The form of the letter when it is joined to the following letter only.
5. The form of the letter when it is joined to both the preceding and following letters.
6. Alif represents no sound in itself, but is used principally as an indicator of the presence of a glottal stop (transliterated ' medially and finally; not transliterated when initial) and as the sign of a long a (transliterated [Overbar a]; see Note (1) above).
7. When 'o' has two dots above it (ö), it is called ta marb[Overbar u]tah and, if it precedes a vowel, is transliterated t instead of h.

Arabic letters are all primarily consonants; three of which (alif, w[Overbar a]w and y[Overbar a]) are also used to represent long vowels. Full indication of short vowels or phonetic effects are not usually provided in normal Arabic scripts except in important texts or where confusion may arise. When they are provided, it is by means of a number of strokes and diacritical signs which are similar to accent marks in some European languages. Those signs are written above or below characters to mark short vowels and emphasize or loosen a letter's sound. They can also be mixed to produce composite phonetic effects.

APPENDIX B
AL-KHAL[Overbar I]L'S COMBINATORICS

The great Arab linguist al-Khal[Overbar i]l, the author of the oldest dictionary of Arabic words, used combinatorics in the eighth century to count all possible Arabic words by considering all different combinations of letters with or without vowels. In his dictionary, Al-Ain, al- Khal[Overbar i]l wrote:

If you want to exhaustively know all of the Arabic language double-letter words, either meaningful or not, which the Arabs either used or rejected, such as qd, km, an . . . etc., take the [Arabic] alphabet letters which are 28, then multiply them with each other to get 784 [= 282]. A single letter is not a word. If you take two letters [without reversal], you get 392 [= 784/2] such as dm and the like. If you reverse [the two letter positions] it comes back to 784, 28 of which have identical letters like hh which do not change when reversed. 600 of these [784 - 28 = 28 x 27 = 756 words) are perfect words [i.e., consonants only] with no W[Overbar a]w, Y[Overbar a] or Hamzah [these are the three basic vowels in Arabic), which come to 300 before reversal [(28 - 3)(27 - 3)/2 = 300]. 150 words [of the 756] contain one of these [vowels]: Y[Overbar a], W[Overbar a]w and Hamzah, with 75 before reversal [25 x 3]. 6 words [of the 756] contain two [different] vowels [3 x 2], with three before reversal. 3 double-letter words [of the 784] contain the same vowel, 25 [double-letter words], contain identical consonants. You should understand what I just explained to you of the double-letter word counts which the Arabs spoke or rejected.

If you want to count all triplets [three-letter words], multiply the 3 vowels by the 9 double-letter words, they become 27 structures with three letters all of which are vowels [33 = 27]. Then multiply the 3 vowels by the 150 double-letter words where one letter is a consonant and the other is a vowel; you get 450 forms of three letters, two of which are vowels and one is a consonant. Also multiply the 3 vowels by the 600 double-consonant words; you get 1800 forms of three letters-2 consonants and 1 vowel. Multiply the 25 [consonant] letters by the 600 double-consonant words; they become 15625 three-consonant words [al-Khal[Overbar i]l might have made a computational error here (it should be 15000), but we believe this to be a slip; see below]. This is all that can result of three letters.

If you want to count the quaternary [four-letter] words, use the same method. Multiply the 3 vowels by the 27 all-vowel triplets, then multiply by 450, then by 1800. Then multiply the 25 consonants by the 15000 three-consonant words [al-Khal[Overbar i]l is right this time, which proves that the 15625 above was indeed a slip). These are all the quaternary structures. The same method is used for five-letter words. Six-letter [root] words are not possible except by using superfluous letters [a property of Arabic morphology].

In another part of al-Ain, al-Khal[Overbar i]l wrote:

A double letter word has two variations like qd, dq, and [Underdot s]d, d[Underdot s] [= 2!]. A triple-letter word has 6 variations [= 3!]; the collection of the six variations is called a hexa-word, an example of which is: [Underdot d]rb, [Underdot d]br, br[Underdot d], b[Underdot d]r, r[Underdot d]b, rb[Underdot d]. A four-letter word has 24 variations because its letters, which are 4 are multiplied by the three-letter word variations which are 6 and the result is 24 [= 4! = 4 x 3!]. A five-letter word has 120 variations because its letters which are 5, are multiplied by the four-letter word variations which are 24, and the result is 120 [= 5!] . . .6

6The exact Arabic text of these two quotations is given by Mrayati et al [12]. The translation given here is the attempt of the author [texts between square brackets are added to clarify the original writings]. Apology is extended to the great al-Kahl[Overbar i]l for the inevitable weakness and loss of eloquence of this translation in conveying the actual meaning and beauty of the original words.


ACKNOWLEDGEMENTS

The author spent part of his sabbatical leave as a Visiting Researcher at the Data Transmission Division of the Electrical Engineering Department (Institutonen för Systemteknik) of Linköping University, Linköping SWEDEN, during which time this paper was written. I would like to express my gratitude to Prof. Thomas Ericson for hosting my stay in Sweden, and to Linköping University for using its services and facilities. I am grateful to Dr. Shakir Abdul-Jabbar for the helpful discussions and critical comments on the manuscript. I would also like to acknowledge the critical review and expert editing of Dr. David Kahn, which helped make the paper more focused and, hopefully, easier to read.

REFERENCES

1. Badeau, J. S. et al. 1983. The Genius of Arab Civilization: Source of Renaissance. Second Edition. Cambridge: MIT Press.
2. Bosworth, C. E. 1963. The Section on Codes and Their Decipherment in Qalqashand[Overbar i]'s Subh Al-A'sh[Overbar a]. Journal of Semitic Studies. VIII-i (Spring): 17-33.
3. Boyer, Carl B. 1985. A History of Mathematics. Princeton: Princeton University Press.
4. Bynum, W. F., E. J. Browne and R. Porter, eds. 1983. Dictionary of History of Science. London: Macmillan Press Ltd.
5. ad-Dafffa', Ali. A 1978. Nawabigh Ulama'i al-Arab wal-Muslimeen fi arriyadiyyat (The Genius of Arab and Muslim Mathematicians). in Arabic. New York: John Wiley & Sons.
6. Fauvel, John and Jermy Gray, eds. 1987. The History of Mathematics: A Reader. London: Macmillan Press Ltd.
7. Hammer, Joseph von. 1806. Ancient and Hieroglyphic Characters Explained. London: W. Bulmer & Co.
8. al-Kadi, Ibrahim A. 1991. Cryptography and Data Security: Cryptographic Properties of Arabic. In Arabic. Proceedings of the Third Saudi Engineering Conference. Riyadh, Saudi Arabia: Nov 24-27. Vol. 2: 910-921.
9. Kahn, David. 1967. The Codebreakers: The Story of Secret Writing. New York: Macmillan.
10. Khald[Overbar u]n, Abdur-Rahm[Overbar a]n ibn. 1958. The Muqaddimah : An Introduction to History. Trans. Franz Rosenthal. New York: Pantheon Books.
11. Menninger, Karl. 1969. Number Words and Number Systems: A Cultural History. Trans. Paul Broneer. Cambridge: MIT Press.
12. Mrayati, Mohammad, Yahya Meer Alam and Hassan al-Tayyan. 1987. Ilm at-Ta'miyah wa Istikhr[Overbar a]j al-Mu'amm[Overbar a] Ind al-Arab. (Origins of Arab Cryptography and Cryptanalysis), Volume I : Analysis and Editing of Three Arabic Manuscripts of Al-Kind[Overbar i]-Ibn-Adl[Overbar a]n and Ibn ad-Duraihim. In Arabic. Damascus: The Arab Academy of Damascus.
13. al-Qalqashand[Overbar i], Ahmad ibn Ali. 1412. [Underdot S]ub[Underdot h] al-A'sh[Overbar a]f[Overbar i] [Underdot S]ina'at al-Insh[Overbar a]. (The Light of the Blind in the Profession of Writing). in Arabic. Section 8 of Chapter 2 in Part 4. Reprinted in Cairo (1957): Ministry of Culture and National Guidance.
14. Sarton, George. 1927-1948. Introduction to the History of Science. 3 vol. Baltimore: Carnegie Institution.
15. Waerden, B. L. van der. 1985. A History of Algebra: From al-Khw[Overbar a]rizm[Overbar i] to Emmy Noether. Berlin: Springer-Verlag.
16. Youshkevitch, A. P. 1964. Geschichte der Mathematik im Mittelatter. Liepzig, Germany: Teubner.

BIOGRAPHICAL SKETCH

Ibrahim Al-Kadi was born in Onaizah, SAUDI ARABIA on 19 January 1954. He received the BS from Riyadh University (currently King Saud University), Saudi Arabia in 1978, the MS from the University of Michigan, Ann Arbor MI, in 1980 and the PhD from Stanford University, Stanford CA, in 1984 all in electrical engineering. He worked as a graduate assistant at Riyadh University (1978-79), as a research assistant at the Communication Satellite Planning Center of Stanford University (1981-84), and as an assistant professor at King Saud University (1984-88) where he is now an associate professor. He teaches undergraduate and graduate courses in systems, electromagnetics and communications, and performs research and consultation. He is a senior member of the IEEE. His areas of interest include communication systems, coding and cryptography, spectrum management, radio wave propagation, remote sensing, technology transfer, satellite positioning, and education.

Starting September 1991, he is on a temporary leave from the University working as Vice President for Engineering at the Advanced Electronics Company (AEC).


Roosevelt, Magic, And Ultra*

*Reprinted with slight changes from Historians and Archivists, ed. George O. Kent (Fairfax, Va.: George Mason University Press, 1991).

David Kahn

ADDRESS: 120 Wooleys Lane, Great Neck NY 11023 USA.

ABSTRACT: Exhaustive research finds little about what solved German and Japanese intercepts went to President Franklin D. Roosevelt during World War II and nothing about how he used them. However, a picture is given of Military Intelligence's Special Branch, which selected and edited the intercepts for presentation to the president and other high officials, and of the intercepts' presentation to and reception by the president.

KEYWORDS: Roosevelt, MAGIC, ULTRA, PURPLE, Special Branch, McCormack, Clarke, and Corderman.

I

What MAGIC and ULTRA information went to President Franklin D. Roosevelt during World Wax II, and how did he use it?

It may as well be said at once that exhaustive research provides only a sketchy answer to the first question and merely unproved hypotheses for the second.

Extant records specify only a few of the solved German and Japanese messages that went to the president. Moreover, no documents or recollections have come to light suggesting any action that he took based on, or even influenced by, these intercepts.

Despite these disappointing results, it may be worthwhile to set out what has been learned about the production of MAGIC (Japanese solutions) and ULTRA (German) and their selection for and presentation to Roosevelt. Doing so will at least outline how one intelligence organization operated in preparing information for the highest authority in the land. And it will provide a framework in case any papers are discovered that answer, or at least shed more light on, the fundamental questions about Roosevelt, MAGIC, and ULTRA.

Foreign intercepts began going to the White House for the first time in 1937.1 They came from the Army's Signal Intelligence Service, the Signal Corps' codemaking and codebreaking agency. Founded in 19292 under the great cryptanalyst William F. Friedman, SIS directed its first nontraining codebreaking at Japan, starting in 1932.3 At first it solved Japanese diplomatic codes.4 But these turned out to encrypt only insignificant messages. High-level, important traffic was apparently protected by a machine cipher.5 In 1935, when SIS realized this and had obtained more experience and greater knowledge of Japanese communication habits from its code solutions, it attacked the machine.6 By 1936, cryptanalysts Frank B. Rowlett and Solomon Kullback,7 perhaps helped by information about a simpler Japanese Navy machine cipher that the U. S. Navy had solved,8 cracked what the Japanese Foreign Office called its "A" machine and what the Americans called RED. With this, the United States gained access to more and better intelligence than ever before.9

1Rowlett. I know of no evidence that earlier American signals intelligence results—during the Civil War, World War I, or the Washington Disarmament Conference of 1922, to name the three cryptanalytically most likely events—went to the White House.
2The Origin and Development of the Army Security Agency 1917-1929 (Laguna Hills, Calif.: Aegean Park Press, 1978), 8-9.
3 Hannah, 11-14.
4Rowlett; SRH-001, 303-04.
5Ibid., 305.
6Ibid., 301; Hannah, 14-15.
7Hannah, 14.
8Lewis, ed., 12; SRH-305, 020; SRH-149, 8; SRH-355, 161-62; Layton, 79. Rowlett disputes this (Hannah, 15), but the several naval officers' statements suggest that the Navy contributed something, though perhaps more in a general than a specific way.
9Hannah, 15; Rowlett. On RED and PURPLE, see Cipher A. Deavours and Louis Kruh, Machine Cryptography and Modern Cryptanalysis (Dedham, Mass.: Artech House, 1985), 211-245. Rowlett says, on p. 2 of a memorandum of 1 March 1980 prepared for author John Toland, that "G2 showed no more than a passing interest in the S. I. S. output [of Japanese diplomatic solutions] until a partially decoded circular message from Tokyo indicated that a secret codicil was being formulated to the tripartite treaty which Japan was negotiating with Germany and Italy." Actually, this intercept could only have concerned not the "tripartite treaty," which was not signed until 17 September 1940, but negotiations for the Anti-Comintern Pact between Germany and Japan. This was signed 25 November 1936, the year in which the A machine was solved; negotiations for Italy's adherence did not begin until 1937. The intercepts reflect this. Italy adhered to the pact on 6 November 1937 but was never told of its secret codicil between Germany and Japan. Parrish, 43, likewise errs in ascribing the army interest to negotiations on a "tripartite treaty." Ironically, the Japanese message of 14 November 1936 that provided a synopsis of the secret codicil was not solved until 13 November 1937 (from solved Japanese diplomatic messages dealing with the Anti-Comintern Pact kindly declassified for me by the National Security Agency). No "partially decoded circular message" was included in the messages supplied to me. (The agreement between Germany and Italy that in effect created the Axis, the so-called Pact of Steel, was signed 25 October 1936.)


This included advance information starting in March 1937 about Italy's possible adherence to the German-Japanese anti-comintern pact of the previous November. The intercepts revealed the early indications that Italy might join, the start of negotiations, the discussion over whether Italy should accede to the existing treaty or sign a separate agreement with Japan, a statement that Hitler wanted Italy to participate in the existing agreement, Italy's apparent acceptance, the emperor's approval, and part of the text of the treaty, which was signed 6 November 1937.10

10Solved Japanese diplomatic messages declassified by N. S. A. In the following citations, the date of the telegram is given first, followed, in parentheses, by the date the codebreakers submitted the translation to the Army G-2. Note the increasing speed of solution. All dates are 1937. Berlin-Tokyo 19 January (8 March) and Berlin-Tokyo 9 March (5 April) for early indications; Tokyo-Rome 15 September (1 October) for start of negotiations; Rome-Tokyo 2 August (23 August) for existing treaty or separate; Berlin-Tokyo 7 October (15 October) for Hitler; Tokyo-Rome 26 October (3 November) for acceptance; Tokyo-Rome 4 November (8 November) for emperor's approval; Tokyo-Rome 31 October, 31 October and 1 November (all 5 November) for part of text. Some of the memoranda to G-2 bear the handwritten initials of Friedman, Rowlett, and John B. Hurt, a translator.

U. S. diplomats did not start reporting until October on this rapprochement between three aggressive nations, to the east and to the west of the United States—and even then their information was far less specific than the intercepts.11 So these seized the attention of officials in Army and Navy intelligence, in the State Department, and in the White House. Information from solved foreign cryptosystems, which had never before been of sufficient value to reach the topmost level of government, began going to the White House,12 probably to the president.

11FRUS, 1937, 1:608. British cryptanalysts likewise seemed to have obtained information about these negotiations only in September-October 1937 (John D. Ferris, "From Broadway House to Bletchley Park: The Diary of Captain Malcolm D. Kennedy, 1934-1946," Intelligence and National Security, 4 [July 1989], 421-450 at 447-448).
12Rowlett.

On 20 February 1939, the Japanese introduced a new and more complicated cipher machine to supersede RED, whose mechanisms were wearing out. The Japanese called it their "B" machine; the Americans, PURPLE.13 There followed nineteen months of concentrated cryptanalytic attack by a team under the general supervision of Friedman and the immediate direction of Rowlett, with some help from the Navy (which resumed diplomatic cryptanalysis to assist with PURPLE after having abandoned that field around 1938 or 1939 to concentrate on Japanese naval solutions). Their efforts were crowned with success in September 1940.14 Thus, even after Japan discontinued use of the RED machine on 21 August 1941,15 the PURPLE solution assured American officials of being able to read Japan's most secret diplomatic messages.

13SRH-145, 001, 008-09.
14Hannah, 5, 15-19; Kahn, 20-22. Lewis, ed., 14-15 for Navy diplomatic solutions.
15SRH-001, 308.

II

Growing cryptanalytic production—from an average of about one diplomatic solution a day in the first half of 193716 to fifty a day in the last half of 194117—compelled the Army and the Navy to share the burden. In August 1940, SIS and the Navy's OP-20-G arranged that diplomatic messages originating in Tokyo on odd-numbered days would be handled by the Navy, those on even-numbered days, by the Army.18 The two agencies exchanged the solutions.

16S. I. S. serial 320 was solved on 12 February, serial 461 on 26 June—141 messages in 135 days.
17Kahn, 29.
18PHA, 8:3923, 18:3313-14; Lewis, ed., 15; SRH-270, 024.

Then, on 23 January 1941, the Army and the Navy agreed that their codebreaking results, which had been given the covername MAGIC, would be delivered to the White House by the Army in odd months, by the Navy in even.19 But after a few months the Army stopped taking MAGIC to the White House.20 In part this was because it felt that diplomatic intelligence should go to the presiden.t through the State Department, in part because the president's military aide, Brigadier General Edwin M. (Pa) Watson, had breached security by forgetfully throwing a MAGIC summary into his wastebasket some time after the Army's turn either in May or in July.21 The final delivery in one or the other of those months seems to have been the last time an Army officer brought intercepts to Roosevelt. Henceforth, the naval aide was the person by whose hand MAGIC went to him.22 This apparently came about because of the following events.

19PHA, 2:288, 4:1734.
20Kahn, 30.
21Ibid ., 26. PHA, 11:5475, implies May. But Watson was receiving occasional deliveries of MAGIC in July.
22Clarke, in reply to my question of 2 December 1985 as to who brought solutions to Roosevelt, wrote: "Navy—always. They never let their poor relations near the old con man." PHA, 2:789, which states that from 1 November 1941 intercepts to the president were delivered through the naval aide, may refer only to the time up to Pearl Harbor.


Near the end of September, an Army month, during which no intercepts were delivered to the White House, the president said he wanted to see them. In October, the Navy continued to send him memoranda based on MAGIC. Early in November, an Army month, during which again nothing was delivered to him, Roosevelt again told his naval aide, Captain John R. Beardall, that he wanted to see MAGIC. The aide told him it was an Army month. The president replied that he knew that and that he was either seeing MAGIC or being told what it said by Secretary of State Cordell Hull but that he still wanted to see the original intercepts. Beardall thought the reason was that the president grasped things more quickly by eye than by ear. On Monday, 10 November 1941, a conference changed the distribution setup. Henceforth, the Army would bring MAGIC to the State Department and the Navy to the White House. At 4:15 p. m. Wednesday, 12 November, the first intercepts were delivered to the White House under this system.23

23PHA, 11:5475-76.

Neither war nor changes in the production of MAGIC affected this arrangement. On 25 January 1942, a month and a half after hostilities began, the cryptological officers of the Army and the Navy agreed orally that the Army would intercept and cryptanalyze all diplomatic traffic as well as enemy army and air force messages. This was formalized in a written agreement of 30 June 1942, which specified that the Navy would solve enemy naval messages and that the Army would continue to give the Navy the diplomatic solutions. The Joint Chiefs of Staff approved this on 6 July.24 Another agreement, on distribution matters, made 25 May 1942 between representatives of the Army, the Navy, and the Federal Bureau of Investigation, likewise did not alter Navy responsibility for White House delivery. It merely stated that, while the president should see diplomatic intelligence, he should not be given enemy naval, enemy military, western hemisphere clandestine (spy messages), or international clandestine intercepts.25 (The reason was certainly not to conceal information from him but rather not to burden him with that which he could barely use.) Thus, although the Navy no longer generated the cryptanalytic intelligence that went to the White House, the naval aide continued to take it in to Roosevelt, who had been an assistant secretary of the Navy in World War I. By then, it was coming from a new Army agency.

24SRH-200, 184, 198, 042.
25Howe, 6; SRH-200, 038.

III

After recovering from the shock of Pearl Harbor, the secretary of war, Henry L. Stimson, decided that a centralized analysis of the intercepted Japanese messages might have prevented the surprise.26 He felt that an organization should be set up to study the intelligence from cryptanalysis more fully and carefully than before. Stimson, a lawyer, consulted with his assistant secretary, John McCloy, a lawyer, and they decided that the job should be handled by a lawyer, particularly one having experience in large cases involving many and complicated facts. They settled upon a former partner of McCloy in one of New York's most prestigious Wall Street firms who had represented Edwin Armstrong in his suits to protect his invention of the regenerative circuit. This was Alfred McCormack, a big, brilliant, demanding Brooklyn-born graduate of Princeton University and Columbia Law School.27 He was appointed special assistant to Stimson on 19 January 1942, six days after his 41st birthday.28

26SRH-116, 006-07; Henry L. Stimson and McGeorge Bundy, On Active Service in Peace and War (New York: Harper & Row, 1947), 454-55.
27Kai Bird, The Chairman: John J. McCloy; The Making of the American Establishment (New York: Simon & Schuster, 1992), 142; SRH-185, 77-83; Who Was Who in America, 3:572, s. v. McCormack, Alfred; "Alfred McCormack, Lawyer, Dies; Former Director of Intelligence," The New York Times (12 July 1956), 23:4-5; Shute; Nelson, Fontana.
28SRH-116, 007.

Because the Signal Corps had for years produced the raw material—the intercepted and solved messages—that went to Army intelligence, one of the two slots allotted to the Signal Corps in the General Staff was in G-2, or intelligence. It was filled, at that time, by Colonel Carter W. Clarke, a tall, slim, energetic career officer with strong likes and dislikes, salty modes of expression, and very good common sense.29 Clarke had no signal intelligence background—he had replaced an officer who had signal intelligence experience but had been shifted to prepare for censorship as war approached30—but was extremely capable. On 15 May 1942, he was placed in charge of the new organization; McCormack, who was commissioned in June, became his deputy.31 They worked very well together, each recognizing that the other had strengths that he lacked. McCormack was ideally qualified to run an intelligence unit while Clarke was just right to fit it into the Army.

29Shute; Ervin; Nelson; Clarke (1963), "Carter W. Clarke Dies at 90, An Army Intelligence Officer," The New York Times (7 September 1987), A24:5.
30Corderman.
31Howe, 6; SRH-185, 002.

The new unit, named Special Branch, was a highly secret element of the Military Intelligence Division;32 the coordinate branches were administration, intelligence, counterintelligence, operations, and plans. All depended directly from the Army's assistant chief of staff for intelligence, the G-2.33 A reorganization of the Military Intelligence Division in June 1944 changed some titles—McCormack became director of intelligence within the Military Intelligence Service, of which Clarke was deputy director34—but had little effect on the work and output of Special Branch.

32SRH-146, 001.
33SRH-185, 015.
34SRH-062, 001, 080; SRH-146, 002. For the 1944 reorganization, see Otto L. Nelson, National Security and the General Staff (Washington: Infantry Journal Press, 1946), 526-535.


The intercepts being distributed early in 1942 still consisted merely of the translations of Japanese messages with some footnotes. These gave the dates and intercept numbers of earlier messages referred to in those texts or said that those earlier messages were "not available."35 This was raw intelligence. McCormack felt that the persons who got this exceptionally valuable information were entitled to have each detail in it illuminated by other information and the intercept itself placed into a larger context.36 This would provide the recipients with finished—and therefore much more useful—intelligence. It would be presented to them in the form of a daily report. Moreover, surveying the intercepts over days or weeks would develop data that could be processed into long-range and specialized intelligence.37

35See intercepts in PHA, 12:1-248.
36SHR-116, 008-9.
37Ibid., 009.

For these jobs, McCormack sought men whose training "teaches them to deal with evidence, to be inquisitive and skeptical, to pursue an investigation through to a conclusion, to meet unfamiliar situations of fact and, most important, to do very detailed work without losing their sense of values."38 He found these abilities not among historians, not among journalists, not among scientists nor detectives nor even parents of quarreling siblings, but exclusively among lawyers, and he began recruiting among members and former members of his law firm, Cravath, de Gersdorff, Swaine and Wood.

38SRH-185, 004.

Clarke, meanwhile, handled administrative, security, and external matters. He prevailed upon the Signal Corps to expand its intercept facilities—the radio posts that monitored the foreign transmissions and so supplied the codebreakers with cryptograms to solve. He instructed the Signal Corps on assignment priorities—which cryptosystems intelligence wanted broken, which enemy circuits should be monitored. Perhaps most important of all, he alleviated the residual friction between SIS, a Signal Corps unit, and Special Branch, the glamorous organ of intelligence.39

39SRH-185, 00203; SRH-116, 010.

IV

When Special Branch started, its offices were on the second floor of the front wing of Washington's Munitions Building, an ugly World War I temporary structure on Constitution Avenue between the Washington Monument and the Lincoln Memorial. In the rear wing of the same floor was SIS. Both moved in the fall of 1942: Special Branch to the newly built Pentagon, SIS to Arlington Hall, a former girls' school in Arlington, Virginia, half a dozen miles from the Pentagon.40 As Special Branch grew, it expanded from its original single room into neighboring offices. The walls were temporary partitions that were constantly being shifted for new people. From time to time the branch moved;41 for a while, at least, it was located in Rooms 2D715, 719, and 732.42

40SRH-117, 007, 008.
41Ervin; Shute; Fontana.
42SRH-062, 084-85.

Though Clarke, McCormack, and some of the specialists had individual offices or offices shared with one or two people, the men preparing the daily intelligence reports—the heart of the operation— worked in a largish open area with desks facing each other in two rows. This enabled them to talk easily back and forth, to pass papers to one another, to go quickly and informally to another's desk to look at something to discuss it.43

43Ibid.

The branch grew rapidly. From the dozen officers and civilians at the start, it swelled to 400 officers, enlisted men, and civilians in Washington and 80 abroad.44

44SRH-145, 009.

Officers of Special Branch could not at first enter the supersensitive codebreaking agency,45 even though the two organizations adjoined one another. But many questions arose in their work:46 Was a message translated correctly? Did the codebreakers have other messages on the same subject? Could the intercept facilities pick up wanted traffic? In the first months, these matters were resolved by Clarke's consulting personally with the head of the Signal Intelligence Service, a fellow Signal Corps officer, at first Lieutenant Colonel Rex Minckler, and after April 1942, Lieutenant Colonel Frank W. Bullock.47 Clarke carried written questions from his staff members to the codebreakers. Later a single officer of each agency handled liaison. But it was still a bottleneck.

45SRH-117, 008.
46SRH-132, 6.
47Kahn, 575.

Then, in February of 1943, Lieutenant Colonel W. Preston (Red) Corderman, who had studied in the SIS School in the 1930s, took charge of the codebreaking agency, now called the Signal Security Agency (SSA). It was Corderman who had been moved to handle censorship and whom Clarke had replaced in intelligence in the General Staff; they had known each other since 1927, when Corderman reported at Fort Monmouth, New Jersey, to the 51st Signal Battalion's B Company party, whose commander was Lieutenant Clarke.48

48Ibid.; Corderman.

At once the liaison situation eased. Special Branch added several officers to handle the work. Secret teletype and secure telephone communications were established between Arlington Hall and the Pentagon.49 The chief liaison officer with the codebreakers, Captain Thomas Ervin, a lawyer who had a flair for telling stories, would spend half his day with Special Branch and half with the codebreakers. He brought them questions Special Branch's analysts wanted answered and got from them both question and requests for information that would help in their solving, such as which one of five possible solutions of a name was the right one.50 Finally, toward the end of the war, the ultimate step was taken and many Special Branch evaluators began working in the same room as the codebreakers,51 greatly facilitating the work of both sides.

49SRH-117, 009.
50Ervin.
51SRH-113, 010.


Once a day, and perhaps later in the war several times a day, a courier driven in a staff car brought the hundreds of daily translated intercepts to Special Branch from Arlington Hall.52 Urgent ones were teletyped.53 Most were Japanese army or diplomatic—the Japanese army material comprised 150 intercepts a day in the middle of the war, 700 at the end—a few were low-level German diplomatic, the higher-level German diplomatic codes not having been solved.54 Italian messages did not figure prominently. It seems likely that every message solved and translated was sent to Special Branch: if Arlington Hall thought it worth translating, it was sent over.55 In addition, an American unit under Colonel Telford Taylor at the British codebreaking agency at Bletchley Park, the Government Code and Cypher School, selected and then cabled to Special Branch, as it did to high military commands in the European theater, German military, air, and naval solutions, called ULTRA.56 This began 27 August 1943.57

52SRH-132, 6; Ervin.
53SRH-062, 43.
54Ervin.
55Ibid.
56Shute. For the origins of this unit: Howe, 10-12, 90, 125- 131.
57Parrish, 181-2.

All messages, when they arrived at Special Branch, went to McCormack. He skimmed them to keep track of what was going on,58 then, from 1943, divided them into diplomatic and military and distributed them respectively to Henry Rigby and Benjamin Shute. These two lawyers, both in their early thirties (Rigby formerly of Cravath, and Shute on leave from it),59 read the intercepts more closely. They kept those that dealt with immediate matters and gave those that dealt with longer-range or background matters to the specialists.60

58Fontana.
59SRH-185, 003, 013; Shute.
60Bell, 52.

* Fig.
Brigadier General Carter Clarke, head of the Special Branch, which processed German and Japanese intercepts for delivery to President Franklin D. Roosevelt and other high officials, congratulates his deputy, Alfred McCormack, a lawyer in civilian life, whom he has just awarded the Distinguished Service Medal for his work during World War II.

They carried MAGIC and ULTRA to Roosevelt.

* Fig.
Captain John R. Beardall, naval aide

* Fig.
Captain John McCrea, naval aide

* Fig.
Rear Admiral Ross T. McIntire, physician and acting naval aide

* Fig.
Lieutenant Commander William C. Mott, assistant naval aide

* Fig.

* Fig.
Brigadier General W. Preston Corderman, head of the Signal Security Agency

* Fig.
Marine Corps Colonel Ronald A. Boone, deliverer of MAGIC and ULTRA to the president, arrives on assignment in Alaska.

* Fig.
Roosevelt smiles as he pins a Distinguished Service Medal on his departing naval aide, Vice Admiral Wilson Brown, 28 April 1942.

Many of the specialists were academics with backgrounds in particular areas. They related the intercepts to other sources of information that flowed to Special Branch,61 such as newscast transcripts and prisoner-of-war interrogations, to produce a full-fledged report on a particular aspect of an enemy nation's economy, military organization, internal unrest, new technology, and the like.62 These were published as completed or as needed.

61SRH-141, 220-23.
62SRH-116, 015-19; Bell, 55, 58.

The messages that were kept by Rigby, Shute, and later another lawyer, William R. Perdue, were edited by them and by the two other lawyers under each to produce the daily intercept reports.63 These were the MAGIC Summary (later called the MAGIC Diplomatic Summary), edited by Rigby; the Military and Diplomatic Supplement to the MAGIC Summary (which began in the summer of 1943 and which from 1 July 1944 was called the MAGIC European Summary), edited by Shute; and the Far Eastern Summary (which began 10 February 1944 and dealt with Japanese military matters), edited by Perdue.64 For these, the teams studied the day's file of translated decodes from the Signal Security Agency. They passed some to the specialists for information or comment.65 They scanned other material for background. The editors working on the Far Eastern Summary, for example, looked at the 300 daily Japanese Navy intercepts; at cables from the field; outgoing cables on MAGIC intelligence; diplomatic and commercial intercepts; other MAGIC intelligence such as Japanese order of battle; the codebreaking agency's Weekly Report, giving the status of cryptanalysis of Japanese army systems; at the Boston Series of plaintext German diplomatic messages obtained by the Office of Strategic Services; and at noncodebreaking intelligence material, such as field command intelligence reports, photo interpretations, and Allied operational cables.66

63SRH-146, 1.
64SRH-062, 087-88; SRH-035, 18; Bell, 50, 52, 56; SRH-062, 049; Perdue, who was in charge from 1942 of what was then called the Japanese Army Supplement; Rigby. Though SRH-062, 038, says that F. A. O'Connell Jr. prepared the Far Eastern Summary, this may refer to a later time.
65Fontana; SRH-146, 5.
66SRH-062, 043-44. Anthony Cave Brown, The Last Hero: Wild Bill Donovan (New York: Times Books, 1982), 278-286, for Boston Series.


From the translated decodes the editors selected the most important messages. That was their basic job. Most of their effort went into deciding what to leave out.67 In addition, they spoke with the cryptanalysts and translators, often over the secure telephone, to rectify errors or fill in lacunae caused by garbles in the ciphertext and to clarify ambiguities in the Japanese and in the English. Even after this, many messages were, by themselves, scarcely intelligible.68 To squeeze the maximum out of them, the editors called for files on items mentioned in the intercepts and sought elucidation.69 They asked research personnel at the Military Intelligence Service to provide information about lesser known people, places, or things, about obscure events or allusions to forthcoming operations. They visited the who's who section, in which each mention of a name or operation was card-indexed, and determined who the person or what the event was.70 This would be inserted as a footnote to the intercept. The editors conferred with one another on the meaning of a word. Some important intercepts were run verbatim in the summaries, but most were summarized. Rarely, if ever, were comments made beyond identifications; the editors did not evaluate the significance of the information.71 The editors assembled the several summaries or texts and the identifications into a single writing, which went to Rigby and then to McCormack for final editing. McCormack was an extremely meticulous editor, who often spent a great deal of time making sure that the reports would be quickly and fully intelligible to their readers.72

67SRH-116, 12.
68SRH-062, 044.
69Fontana.
70SRH-146, 5; Ervin; Shute; Nelson; Fontana.
71The summaries themselves.
72Fontana; Ervin.

The manuscript then went for typing to one of the two chief secretaries, both male legal stenographers who had worked at Cravath. One of them, at least, Vincent Fontana, followed the Cravath form book in typing the summaries: indent text 15 spaces from the left, indent 25 spaces for a paragraph start, indent 20 fewer for a quotation set apart in the text.73 Reports averaged ten to twelve pages. The typescripts then were reproduced in bluish-purple ink by the Ditto process in about ten copies—later apparently more. Each was bound in a spiral binder, using a device that Special Branch had for this.74 Then each was put in a locked leather briefcase,75 called a pouch, and distributed by officer couriers in regularly scheduled staff cars. The diplomatic summary went out early in the afternoon; the military ones were reproduced and bound in the evening and distributed the next morning.76

73Fontana.
74Ibid.
75Mott; SRH-123, 3.
76SRH-132, 3.

The Army delivered its copies of the MAGIC Summary to the secretary of war (Stimson), the chief of staff (General George C. Marshall), the assistant chief of staff for war plans, the assistant chief of staff for intelligence, the Navy for further distribution, the secretary of state (Cordell Hull) and the assistant secretary of state who followed signal intelligence (probably Adolf A. Berle).77 The Navy passed the 10 copies it got to the White House, the secretary of the navy (Frank Knox), the commander-in-chief of the U. S. Fleet and chief of naval operations (Admiral Ernest J. King), the chief of war plans, the director of naval intelligence, the vice chief of naval operations, the director of naval communications, the radio intelligence center, the domestic intelligence section and a file.78 The Far Eastern Summary also went to the president. It was at first distributed in 23 copies; by the end of the war the number of copies rose to 75.79

77SRH-035, 18.
78SRH-145, 049.
79SRH-062, 049.

V

Delivery to the president was made by the naval aide. This had been the practice since before Pearl Harbor. For a few months in 1941, Roosevelt's physician, Admiral Ross T. McIntire, acted as naval aide and so brought the intercepts to the chief executive.80 Once, in the summer, when McIntire was not available, a naval intelligence specialist, who did not want to entrust the pouch to the president's private secretary and who felt that the intercept he had could not wait, brought a message to the president. He was Lieutenant Commander Alwin D. Kramer, a Japanese language expert who headed the translation section of OP-20-G and who, because of his expertise on Japan and MAGIC, carried the intercepts to the high-level naval recipients so that he could answer their questions on MAGIC.81 The intercept perhaps concerned Japanese-American negotiations the next day.82

80PHA, 9:3985.
81Kahn, 11.
82Ibid., 30.

When, a little later, Rear Admiral John Beardall became naval aide, Kramer—a methodical and dedicated man whom some called "The Shadow"83—took the intercepts to him for delivery to the president.84 Kramer explained to Beardall points in the summaries about which the president was asking. Beardall, whose office, like Kramer's, was in the Navy Building, would ask Kramer late in the afternoon whether any MAGIC would be coming over for the president. Kramer would say, "There is no MAGIC ready for the president now" or "There might be something later" or "Nothing until tomorrow morning."85 When there was something, Beardall would receive the intercepts in his office from Kramer and would carry them to the White House, a few blocks away, arriving around 5:30 or 6:00.86

83Mott.
84PHA, 11:5475.
85Ibid., 5269, 5277.
86Ibid., 11:5277, 5278.


About the first week in December 1941, a junior naval officer was assigned to bring MAGIC to the president when Beardall was off duty or at dinner or otherwise unavailable. This was a communications intelligence specialist, Lieutenant Lester R. Schulz.87 He was assisted, beginning 6 December, by an Ensign Carson.88 This was the system in effect at the time of Pearl Harbor.

87Ibid., 10:4660, 11:5278. He was technically on temporary duty at the White House as communications assistant to the naval aide.
88Ibid., 10:4660, 11:5280.

Beardall had a key to the MAGIC pouch and sometimes opened it for the president, sometimes not. Normally, the president returned the sheaf of papers to Beardall when he had finished with it, though on 2 or 3 December 1941 he said he wanted to keep a copy of a Tokyo-Berlin intercept warning that "War may suddenly break out between the Anglo-Saxon nations and Japan …quicker than anyone dreams." Kramer prepared the copy and gave it to Beardall; he never got it back.89 Though Beardall never discussed the intercepts with the president, he sometimes called attention to what he considered were important ones. On 4 or 5 December, for example, he said to Roosevelt of an intercept revealing that the Japanese were burning their codes: "Mr. President, this is a very significant dispatch." Roosevelt read it carefully, and then said to Beardall, "When do you think it will happen?" Beardall, thinking that Roosevelt meant the start of war, said, "Most any time."90

89Ibid., 9:4072.
90Ibid., 11:5284.

The most dramatic case of Roosevelt's receiving intercepts took place on the evening of Saturday, 6 December 1941. Schulz had been ordered by Beardall to stand by and bring some MAGIC expected that evening to the president. It was the first time that MAGIC was delivered after the close of the ordinary working day.91 Schulz was waiting in Beardall's small office in the corner of the White House's basement mail room when Kramer arrived about 9:15. He was bringing the first thirteen parts of a fourteen-part PURPLE message; the Japanese had deliberately delayed transmitting the fourteenth part, which would break off negotiations with the United States. Schulz obtained permission to bring the MAGIC to the president and an usher accompanied him to the oval study on the second floor and announced him.

91Ibid., 11:5278.

Roosevelt, who had excused himself from a dinner party, was seated at his desk. Only his confidante, Harry Hopkins, was with him in the quiet lamplit room. Schulz unlocked the briefcase in which the MAGIC was distributed with the key Beardall had given him, removed the sheaf of intercepts, and handed it to the president. Roosevelt read the thirteen parts in about ten minutes, while Hopkins paced slowly up and down. Then Hopkins read them. The thirteenth part rejected the final American offer and, when Hopkins passed the papers back to the president, Roosevelt said, in effect, "This means war." Hopkins agreed, and for about five minutes they discussed the situation, the deployment of Japanese forces, the movement towards Indochina, and similar matters. The president mentioned a message for peace he had sent the emperor of Japan. Hopkins replied that it was too bad that the United States could not strike the first blow and prevent any kind of surprise in the inevitable war.

"No," the president said in effect, "we can't do that. We are a democracy and a peaceful people." He raised his voice. "But we have a good record." He tried unsuccessfully to get the chief of naval operations on the telephone, deciding against having him paged at the National Theater for fear of causing undue alarm.

The president then returned the papers to Schulz, and, about half an hour after he had entered the study, Schulz left. He returned the papers to Kramer, who was waiting on a bench in the mail room, and soon thereafter went home.92

92Ibid., 10:4659-665.

The next morning, Beardall brought the president the fourteenth part, which had been intercepted and solved during the night. It expressed the Japanese government's regret that "it is impossible to reach an agreement through further negotiations." It was about 10 a. m., and the president was in his bedroom. He said "Good morning" to Beardall, read the contents of the pouch, and then commented, in substance, "It looks like the Japanese are going to break off negotiations." There was no further discussion; Beardall took the pouch back to the Navy Department, and was as surprised as everybody else when news came of the attack on Pearl Harbor.93

93Ibid., 5283.

Beardall was replaced as naval aide in January 1942 by Captain John McCrea. He saw or telephoned the president almost every day. To him went the MAGIC intercepts, brought over by an army courier. He also received copies of important naval dispatches that high naval officers thought should be shown to the president. Much of this arrived at his office in the Navy Building. Twice a day, McCrea picked out the naval dispatches he thought would be of interest to the president, added to these all the MAGIC intercepts, stuck them in his pocket or his briefcase, and headed for the White House, where he was expected.


When McCrea arrived in the morning, Roosevelt would usually be either in bed, in which case McCrea would hand him the papers, or in the bathroom, shaving. If the latter, the naval officer would go in, close the toilet cover, sit down on it, and in that inglorious setting would read the leader of the most powerful nation in the world some of the most secret documents of the greatest war in history. In the afternoon, the president, if he were not in the Map Room, a chart-lined office in the basement of the White House, would be seated in a neighboring room in a chair like a dentist's, having his sinuses packed by McIntire or his polio-deformed legs rubbed down by a masseur. Here again, McCrea would read the documents aloud to him.94

94McCrea. As an interesting confirmation, Franklin D. Roosevelt Library, Map Room File, Box 195, Folder Chart Room Log and Standing Orders, p. 67, notes that McCrea saw the president in the doctor's office on 14 and 16 May 1942.

When the president was at his home in Hyde Park or at his Maryland mountain retreat, Shangri-La, McCrea would tell him about the documents by telephone.95

95McCrea.

In 1942, the MAGIC documents were reaching McCrea's office in the Navy Department about 2:30 p. m.96 OP-20-G had, since before Pearl Harbor, flagged the important items by sliding paper clips on them.97 McCrea commented that although the selection was in general "accurate as to the possibility of their interest to the president," he often spotted other items that only he knew would also interest Roosevelt. After showing the documents to the president, he returned them to the courier for destruction by OP-20-G. In the summer of 1942, when Admiral William D. Leahy became Roosevelt's chief of staff, MAGIC documents may have been addressed to him,98 though he himself rarely took them in to Roosevelt.99 That function was still mainly exercised by McCrea, his 1943 successor Rear Admiral Wilson Brown, Brown's assistant, Commander John A. Tyree, and a young naval officer working in the Map Room under Leahy, Lieutenant Commander William C. Mott. All briefed the president orally in the morning and in the afternoon, often when he was getting his sinuses packed and was, Mott said, "a captive audience." They reported the intercept intelligence with the other information that they thought he would want to know and that they had boiled down for his consumption. If Roosevelt wanted to read some papers, they would give them to him; the intercept material was on separate sheets. Mott often underlined on these pages what he thought the president ought to know. The president struck the aides as extremely well informed.100

96SRH-200, 052.
97Kahn, 30.
98Adams, 237. Smedberg said that "probably Leahy brought it [MAGIC] around to him [Roosevelt]." Tending to support this is that Leahy knew of MAGIC since, in the late 1930s, he was chief of naval operations, when a cryptanalyst said of his attitudes to signal intelligence that "We couldn't ask for better. He is thoroughly sold" (SRH-355, 259). On the other hand, Elsey doubted that any MAGIC or ULTRA documents went through Leahy, and Mott said that Leahy did not have access to the intercepts he was bringing to Roosevelt.
99Mott; inferred from Elsey.
100Mott; Tyree.

During Roosevelt's trip to the Casablanca conference with British Prime Minister Winston Churchill in January 1943, he was sent messages from Washington that were based on MAGIC Diplomatic Summaries. They were "from Colonel Boone"—almost certainly Marine Corps Colonel Ronald Aubry Boone, an Annapolis graduate who had been in naval intelligence in Washington since February 1939 and had spent eight years in Peking and Shanghai on language, intelligence, diplomatic, and public-relations missions.101 One such "Colonel Boone" message, of 17 January 1943, identical with MAGIC Diplomatic Summary 296 of the day before, states that "Yamaguchi in Berlin reports to Tokyo that Nazis axe upset about American military supplies reaching Vladivostok in Russian bottoms X. They reach Moscow quickly by rail X Germans say until this supply route is cut off, any advance in Russia quite impossible X …"102 On 27 January, "Colonel Boone has indicated for last three days that a Jap movement in strength toward southern Solomons is shaping up X …"103 There are about half a dozen such messages to Roosevelt during the conference.

101Service record, Serial No. 093, National Personnel Records Center, St. Louis, Missouri. Clarke said that the colonel delivering to the White House was not in the Army but was a Marine. This would keep delivery in the Navy Department and eliminates a Colonel Boone who was in Army intelligence at the time—Colonel Joseph W. Boone, O15341 (U. S. War Department, Adjutant General's Office, Army Directory, April 20, 1943 [Washington], 174). The only Marine Corps Colonel Boone at the time was Ronald Aubry Boone. Department of the Navy, Marine Corps, Headquarters, History and Museums Branch, Reference Section.
102Franklin D. Roosevelt Library, Map Room File, Box 15, Folder Casablanca Conference Trip File, Item QMU 171938Z (UTAH 53), and SRS 839, "Magic" Summary No. 490, 19 July 1943.
103Ibid., Item WHITE 21. Cochran, "The Influence of 'Magic' Intelligence," 348, errs in stating that item WHITE 8 in the above folder is identical to MAGIC Diplomatic Summary of 7/29/43; they are different.


Churchill sometimes sent Roosevelt ULTRA—which the prime minister referred to by his own codename for it, BONIFACE. While they were at the Quebec conference in September, 1944, for example, Churchill passed some to him, which he later said "appeared to me to be of profound significance." More often Churchill gave him documents based on intercepts or referred to information based on them. In a note to the president of 13 August 1943, while both were at the president's home in Hyde Park, New York, Churchill provided an analysis of the situation in Yugoslavia of which he said, "Much of it is taken from BONIFACE sources." In October, he wrote about movements in Italy that "We know from BONIFACE that the enemy is withdrawing to the north.104

104Winston S. Churchill and Franklin D. Roosevelt, The Complete Correspondence, ed. Warren F. Kimball (Princeton: Princeton University Press, 1984), 3:321 for "appeared to me," 2:389 for "Much of it," 2:502-503 for "We know." Other references to intercepts at 2:4, 389, 502-503, 666-667, 672-673, 677, 3:229, 256, 321, 341, 388. Kimball says that all the intelligence files have not been declassified and suggests that intercepts sent by Churchill to Roosevelt but not yet published might be found therein. This is reinforced by Martin GIlbert's footnote in Randolph Churchill, then Martin Gilbert, Winston Churchill (Boston: Houghton Mifflin, 1966-1988), 6 (1983): Finest Hour, 1939-1941, at 1260, based on a personal communication, that Churchill sent Roosevelt intercepts via "C," the chief of the Secret Intelligent Service, Sir Stewart Menzies, minuting on them "Make sure the President knows this" or "Make sure the President sees this."

Likewise, during Roosevelt's visit to Canada for the QUADRANT conference of July and August 1943 and his subsequent fishing trip, he was sent two Colonel Boone messages: One was on 21 August: "Colonel Boone indications that Jap barge traffic from Shortland to Kolombangara and Vella area (in the Solomon Islands) may be proceeding along south Choiseul coast to vicinity Sami Head (7 22 S 156 08 E) then across the channel to destination."105 The other, four days later, dealt with a report of the Japanese ambassador in Hanoi.106

105Franklin D. Roosevelt Library, Map Room File, Box 16, Folder Canadian Fishing Trip and Quadrant Conference, WHITE 112.
106Ibid., WHITE 128.

In November of 1943, a Marshall aide and Churchill sent Roosevelt intercepts that they wanted him to see. But he was away at the conferences at Cairo and Teheran when they arrived. He first saw them when he returned on 16 December 1943. After transferring from the battleship Iowa to the presidential yacht Potomac for the trip up to Washington, he was handed a brown manila envelope in a briefcase that a Map Room watch officer, naval reserve Lieutenant (j. g.) R. H. Myers, had brought with him on the yacht. The intercepts dealt with German forces and forts that would be defending against the Allied invasion of Europe. The Japanese ambassador, General Hiroshi Oshime, and his aides had inspected the Atlantic Wall and reported to Tokyo in great detail on the German measures: "… All the German fortifications on the French coast are very close to the shore and it is quite clear that the Germans plan to smash any enemy attempt to land as close to the water edge as possible. … The coastal defence divisions are distributed as follows: Netherlands Defence Army (covering the Netherlands down to the mouth of the Rhine), 4 divisions; Fifteenth Army (covering the area extending from the mouth of the Rhine to west of Le Havre), 9 divisions; Seventh Army (extending thence to the southern bank of the Loire), 8 divisions. …" These messages, all probably decrypted from PURPLE radiograms intercepted at Asmara, Ethiopia, where SSA monitored Berlin-Tokyo traffic in the dun-colored block building of a former Italian radio station, have been called the greatest military intelligence coup of all time—and it certainly gave an insight into enemy plans unprecedented in history in both detail and scope. In July of 1944, Churchill asked that Roosevelt be shown another message. But, Clarke noted in a memo, that one had already been reported in Special Branch's Far Eastern Summary of 20 July 1944 and had been delivered to the White House that day. It was a Japanese message saying that the enemy must not be given any pretext to use chemical warfare.107

107SRH-111; Carl Boyd, "Significance of MAGIC and the Japanese Ambassador to Berlin: (V) News of Hitler's Defense Preparations for Allied Invasion of Western Europe," Intelligence and National Security, 4 (July 1989), 461-481 at 472-474; Hinsley, 3:2:32-33, 771-775.


VI

Roosevelt's confidence in the MAGIC decrypts had been settled well before Pearl Harbor. When he was given the solution of the Japanese ambassador's report of a private audience with himself, he appeared much impressed and commented, as he read the translation, "Yes, I said that" and "That is correct; he said that."108

108SRH-305, 24.

But did ULTRA and MAGIC, which had to compete with so many other messages and problems, capture his interest? The evidence is contradictory.109

109Gerhard L. Weinberg, Reflections on Running a War: Hitler, Churchill, Stalin, Roosevelt, Tojo, Transcription of the 6th annual Phi Alpha Theta distinguished lecture on history at the State University of Albany on March 20, 1986 (n. p., n. d.), 11, is no doubt correct in observing that Roosevelt got through an enormous amount of paperwork each day and that he "kept track of the 'ultra' materials." Likewise, Cochran, "The Influence of 'Magic' Intelligence," 346, concludes that Roosevelt had "more than a passing interest in intelligence." On the other hand, Adams says, 237, that while Leahy continued to bring ULTRA to the president as it seemed pertinent, "Roosevelt was not much interested in details." None of these comments answers the significant question of whether he much used the intercepts in running the war. On January 1944, Roosevelt, in a message to Churchill, cited an intercept as a possible answer to a puzzling international incident (Kimball, ed., 2:672-673, 677). This might indicate that he was reading the intercepts voluminously and carefully. On the other hand, his staff might have provided this information to him-an impression reinforced by Kimball's putting a bracketed "WDL?" at the end of the message to suggest that Leahy might have drafted it for him.

In February 1944, Marshall wrote the president, "I have learned that you seldom see the Army summaries of 'Magic' material." Marshall's remedies—improving the presentation, separating the wheat from the chaff and correlating the items with other information, binding the summaries in a Black Book for convenience of reading—suggest that the president was not particularly interested in the information.110

110National Archives, Record Group 218, Leahy Papers, Folder 126, p. 189.

On the other hand, McCormack said of his MAGIC summaries that the president and his advisers "read them avidly."111 He commented that "the fight for personnel was made easier by high praise which the [Special] Branch received from time to time by General Marshall and other high officials, and occasionally from the President."112 General Corderman said that although none of the Signal Service Agency's many commendations were ever known to have come from the president, he had the impression that Roosevelt was a great user of the intercepts and a great supporter of those who produced them.113 Frank Rowlett, the cryptanalyst whose solution of the RED machine had started it all, said that the usefulness of the intercepts to the top leadership was made clear to the cryptanalysts in the most valuable way possible: the cryptanalysts were asked, "Do you have anything more on this? Give it priority!" That request for more of their results was, said Rowlett, "better than any pat on the back."114

111SRH-185, 013.
112Ibid., 004.
113Corderman.
114Rowlett.

No doubt the president was glad for the information he got—hence the appreciation sensed by the codebreakers and the evaluators. Nevertheless, the impression remains that the intercepts had only a secondary interest for Roosevelt. The reason may be that—unlike the military and naval men—he could not much use the information they gave him.

During hostilities, intercepts on diplomatic affairs—either of the Axis or of neutral nations—dealt at best with nonbelligerents. But geography and the war situation influenced them far more than Allied pressure. Roosevelt understood this, and so perhaps interested himself but little in the intercepts dealing with these matters. For instance, the Allied invasion of North Africa had given rise to fears that Spain might admit German troops who could close the Strait of Gibraltar and trap Allied forces in the Mediterranean.115 McCormack later claimed that Special Branch had provided the best intelligence about Spain's postinvasion intentions.116 But however valuable the information Roosevelt may have gleaned from MAGIC, he seems not to have acted upon it: he sent no instructions to his ambassador in Madrid and did not mention guarding against the German threat in any of his postinvasion messages to Churchill.117 In August of 1942, however, a MAGIC solution seemed of unusual importance. Marshall pointed out to Roosevelt intercepts showing that Japan had decided not to attack the Soviet Union and urged that Roosevelt tell Soviet leader Joseph Stalin about this. Roosevelt did—without, however, specifying the source of his information, which he affirmed was "definitely authentic."118

115Dallek, 362, 370.
116SRH-035, 18-19; SRH-116, 014.
117Nothing in Kimball, ed., 2, checking especially 20, 41-42, 75, 76; FRUS, 1942, 2:303-315; FRUS, 1943, 2:595-97; Willard L. Beaulac (an American diplomat in Spain at the time), Franco: Silent Ally in World War II (Carbondale: Southern Illinois University Press, 1986); James W. Cortada, Two Nations Over Time: Spain and the United States, 1776-1977 (Westport, Conn.: Greenwood Press, 1978); or in Carlton J. H. Hayes (the ambassador), Wartime Mission to Spain (New York: Macmillan, 1945).
118Franklin D. Roosevelt Library, Map Room Files, Box 167, Naval Aide's File: A/16-Japan and Japanese Islands, 1942-1945, Memo for the President from Marshall, 5 Aug 1942; FRUS, 1942, 3:616. I am obliged to Dr. Carl Boyd for this information.


The MAGIC and ULTRA dealing with military matters was, of course, of extraordinary value to the generals and admirals, and Roosevelt urged it upon them. "Be sure Ernie King sees that!" he would sometimes tell McCrea when shown an intercept. But neither McCrea nor Mott nor Tyree saw cases in which an intercept impelled the president to take a particular action.119 In part this was because he delegated the detailed running of the war to his military chiefs and seldom interfered.

119McCrea; Mott; Tyree.

In this Roosevelt contrasted sharply with Churchill. Soon after messages were solved at the codebreaking center at Bletchley Park, they were sent, together with their interpretations, to MI6 headquarters at Broadway Buildings in London, where a selection was made, put in a special buff-colored box, and taken by messenger to 10 Downing Street, five minutes away. The prime minister, who carried the key to the box on his key ring, sometimes read up to 200 ULTRA intercepts a day, discussed them at his daily meetings with his chiefs of staff, drew the attention of his field commanders to individual items, was told of such technical successes as the solution of a U-boat key, and in general utilized the intercepts in directing the war. But Churchill was much closer to the fighting and bombing than Roosevelt and, as minister of defense, much more intimately involved in the details. The intercepts were therefore of far greater value to him. Not to know what he saw would distort the picture of how he worked during World War II.120

120Gilbert, 6:611, 613, and the numerous references under "Enigma decrypts" in 7 (Road to Victory); Martin Gilbert, memorandum, 17 March 1987; Hinsley, 2:4 and entries under "Churchill" in the index.

Can the same be said of Roosevelt? Not with our present knowledge. Perhaps he used MAGIC and ULTRA extensively in his running of the war, and perhaps he did not. All known sources do not provide enough facts to permit a conclusion one way or the other. He and his advisors cannot be asked, and the chances seem slight that papers yielding direct evidence on the matter will come to light. Possibly comparisons of MAGIC and ULTRA with actions taken in the war will develop a pattern convincing enough to let historians say that the one affected the other. But here, too, the uncertainty even about what intercepts Roosevelt saw and the tenuousness of such influence even with solid data reduce the likelihood of success. Of the two efforts made so fax, one fails to convince—even its proposer offers it diffidently—and the other, more plausible, awaits a documentary basis.121 The almost complete lack of traces of Roosevelt's use of MAGIC and ULTRA despite the great volume of material may indeed suggest that Roosevelt did not much use them. But this is at best a negative argument, and in no way conclusive. In the end, we historians can only say that we do not now know how Roosevelt used MAGIC and ULTRA. And probably we never will.

121The effort that fails to convince is that of Cochran. He says, at 346, that the decision to get Italy out of the war "could well have been based upon Magic information" about "major strategic differences between the Axis powers, desperate economic and social conditions in Italy, and growing political instability among the fascist elite," citing 17 reports from April to June 1943. But not only did many nonintelligence factors contribute to this decision, not only had the idea been considered even before the North African invasion, but, as Cochran admits, no evidence links the intelligence to the decision, and without evidence his hypothesis remains unproved. The more plausible effort is that of Rohwer. He argues that Churchill knew via ULTRA of Hitler's 21 June 1941 order to U-boats that they not attack large warships whose nationality they did not know, that Churchill passed this information to Roosevelt, and that it encouraged the president to support Britain in the Atlantic more energetically than if he feared German retaliation. Rohwer says that U. S. Navy situation maps show the U-boats dispositions and that this is "clear evidence of the re-transmission of 'Ultra' intelligence from London to Washington." But National Archives, Record Group 457, SRMN-033, Part I, "Cominch File of Messages on U-Boat, Estimates and Situation Reports October 1941-September 1942," which gives individual submarine positions and estimated positions, does not reach back to June, the Navy's Operational Archives does not have any earlier files, and none of the messages in Kimball, ed., 1:210-281 (21 June to 7 December 1941) deal with any such instructions to U-boats. Thus, plausible as Rohwer's suggestion is, it must await further documentation before winning at least my acceptance.


NOTES

A name alone refers to an interview. Numbers for SRH items beginning with 0 or 00 when under 10 or 100 are the page numbers stamped on the document by archivists; they differ from the document's original typewritten page numbers.

Note

Though this article concludes that cryptologic information was little used by Roosevelt, such material was of enormous value in military operations, including at the highest level. Special Branch prepared the material for use by these high officials. But because this article focuses on Roosevelt, it excludes many details of the workings of the Special Branch. Almost every sentence dealing with that operation could—and some day should—be expanded into an article. The sources offer clues to this.

My claim of exhaustive research requires that I list the sources that I checked for information about Roosevelt and MAGIC and ULTRA. They are:

• Franklin D. Roosevelt Library, Hyde Park, New York. The only records of MAGIC for the president are those for some of his trips away from Washington. These are cited below. The Map Room files have no logs of ULTRA or MAGIC documents or the summaries based on them that went to the president, nor any mention of the couriers who brought him ULTRA or MAGIC documents, nor any record of what the naval aides (as Captain McCrea) brought him in its noting of their visits to him.
• William Emerson, at the time of my inquiry director of the Roosevelt Library. He knew of no unpublished documentary material concerning Roosevelt's receipt of or use of ULTRA or MAGIC.
• Robert Dallek, Franklin D. Roosevelt and American Foreign Policy, 1932-1945 (New York: Oxford University Press, 1979). The work does not mention ULTRA, and Dallek, who went through all or nearly all the papers dealing with Roosevelt and foreign policy, said on the telephone 14 April 1986 he does not remember seeing anything about it in the documents. He believes that such material would have jumped out at him and that he would have used it in his book as a good story.
• Robert Myers of the White House Map Room staff said that both he and George Elsey of that staff picked up intercepts from the commander in chief, U. S. Fleet (Admiral Ernest King), and Admiral John Tyree, assistant to the naval aide, said that Myers picked up intercepts from the Navy. On the other hand, Elsey said that, although he dealt with messages of Winston Churchill and Joseph Stalin to Roosevelt, he knew nothing about MAGIC or ULTRA documents or summaries going to the president. I have not been able to clear up the contradiction. Asked for any intercepts sent by the Navy to the White House during World War II, the Navy Security Group Command headquarters replied in a letter of 18 April 1988 (Ser GH/4057) that it had only duplicates of Army-originated diplomatic intercepts.
• All of the interviewees were asked about how the Special Branch summaries went to Roosevelt and what use he made of that information; they knew little about the distribution and none knew anything about the president's use of the material.
• Archive of Contemporary History, University of Wyoming. I was told 16 April 1986 by Professor Emmet Chisum that he has never seen anything about distribution of MAGIC to Roosevelt in the papers of Laurance Safford, Dundas Tucker, and C. C. Hiles, all naval officers who worked in or knew about communications intelligence.
• Dundas P. Tucker, "Rhapsody in Purple: A New History of Pearl Harbor," ed. Greg Mellen, Cryptologia, 6 (July 1982), 193-228, (October 1982), 346-367, has nothing on MAGIC or ULTRA to Roosevelt.
• Library of Congress, Manuscript Division, Papers of William D. Leahy, Diary. A skim showed nothing dealing with ULTRA or MAGIC. Moreover, these papers became available to the public before ULTRA material was declassified.
• National Archives, Record Group 218, Joint Chiefs of Staff, Leahy Papers. None of the following folders have anything about MAGIC or ULTRA to Roosevelt:
- Folder 54, Value of OSS
- Folder 82, Miscellaneous Messages 1945-46
- Folder 83, Dispatches 1944 Miscellaneous
- Folder 84, Miscellaneous 1944
- Folder 89, Conference Teheran 1943
- Folder 100, Prime-President 1944
- Folder 101, Prime-President 1943
- Folder 102, Hurley—"Special"—and Harriman
- Folder 104, High-Level (not Prime), 1945
- Folder 105, High-Level Message 1943 1944 not Prime
- Folder 125, Memoranda to and from President 1945
- Folder 127, Memos to and from President July 1, 1943 to Nov. 1943
- Folder 128, Correspondence signed by the President
- Folder 138, Messages between President and Admiral Leahy
• U. S. Naval Institute, Annapolis, Maryland, "Reminiscences of Vice Admiral William R. Smedberg III, U. S. Navy, Retired," an oral history, July 1979, has nothing on Roosevelt on pages 281-313, which deal with Smedberg's service from September 1944 to March 1946 as combat intelligence officer on the staff of the commander in chief, U. S. Fleet.

I have not credited or used the statements in William Stevenson's A Man Called Intrepid: The Secret War (New York: Harcourt Brace Jovanovich, 1976) ascribed to Sir William Stephenson, head of British Security Coordination during World War II, that he brought ULTRA information to Roosevelt. These statements are unsupported by other evidence and my confidence in the book's unsubstantiated statements has been undermined by its basic and numerous errors.


SOURCES

Published Documents

United States. Congress. Joint Committee on the Investigation of the Pearl Harbor Attack. Pearl Harbor Attack. Hearings. 79th Congress: 1st and 2nd Sessions. Washington: Government Printing Office, 1946. 39 vols. Cited as PHA followed by volume and page number.

__________ Department of State. Foreign Relations of the United States: Diplomatic Papers. Washington: Government Printing Office. Cited as FRUS with year followed by volume and page.

1937, 1: General; 2: British Commonwealth, Europe, Near East, Africa; 3 and 4: The Far East (all 1954).
Japan, 1931-1941, 2 (1943).
1942, 2: Europe (1961); 3: Europe (1961).
1943, 2: Europe (1964).

Unpublished Documents

National Archives. Record Group 457. Records of the National Security Agency.

SRH-061. "Allocation of Special Security Offices to Special Branch, Military Intelligence Service, War Department, 1943-45." 33 pp.
SRH-111. "MAGIC Reports for the Attention of the President 1943-1944." 32 pp.
SRH-116. "Origin, Functions and Problems of the Special Branch, M. I. S." [A memorandum by McCormack to Clarke, 15 April 1943.] 55 pp.
SRH-118. "Incidental Exhibits re Pearl Harbor Investigation" [Collection of Japanese Diplomatic Messages 12 July 1938-21 January 1942. Department of the Army Intelligence Files.] 481 pp.
SRH-141. "Papers from the Personal Files of Alfred McCormack." Parts 1 and 2. 570 pp.
SRH-145. "Collection of Memoranda on Operations of S. I. S. [Signal Intelligence Service] Intercept Activities and Dissemination 1942-45." 293 pp.
SRH-153. "M. I. S., War Department, Liaison Activities in the U. K. 1943- 1945." 20 pp.
SRH-185. "War Experience of Alfred McCormack." 83 pp.
SRH-270. "Army-Navy-F. B. I. Comint [Communications Intelligence] Agreements of 1942." 5 pp.
SRDJ-113,785-114,000. Japanese Diplomatic Translations. Station "C" Intercept Message File. [10 September 1940-7 April 1941.]
SRS. Diplomatic "MAGIC" Summaries 1943. September, October, December.
__________ Record Group 218. Joint Chiefs of Staff. Leahy Papers. Folder 126, Memos to the President from General Marshall.

Franklin D. Roosevelt Library, Hyde Park, New York. Map Room File.

Box 15. Presidential Trips 27 September 1942-28 April 1943. Folders Casa- blanca Conference Trip, Presidential Trips Hyde Park 6 February-5 April 1943.
Box 16. Presidential Trips 14 May-7 November 1943. Folder Canadian Fishing Trip and Quadrant Conference 31 July-29 August 1943.
Box 192. Map Room Logs—Incoming Dispatches. 1941-July 1943. Folder Log of Incoming Dispatches April May 1942. Folder Log of Incoming Dispatches December 1941 January February March 1942. Folder Log Sheets September 1942.
Box 195. Map Room Logs: 1942-1945. Folder White House Chart Room—Special Log President-Churchill Dispatches. Folder Chart Room Log and Standing Orders [a log of calls and visitors]. Folder 1-A [log of Map Room].

National Security Agency.

Translated solutions of intercepted Japanese diplomatic messages dealing with the accession of Italy to the Anti-Comintern Pact.

Interviews

In Person

Carter W. Clarke. Head of Special Branch. 6 December 1963.
Thomas Ervin. Liaison officer in Special Branch. 11 March 1986.
Vincent Fontana. Typist and production manager in Special Branch. 12 March 1986.
C. Roger Nelson. Assistant editor of MAGIC Diplomatic Summary. 12 December 1985.
Benjamin R. Shute. Editor of German military summary. 8 March 1986.

By Telephone

Cletus Beard. Head of Asmara intercept post in 1950s. 20 April 1986.
Carter W. Clarke. 10 January and 19 October 1986.
Arthur Compton. Liaison officer in Special Branch. 27 October 1986.
W. Preston Corderman. Head of Signal Security Agency. 20 April 1986.
George Elsey. Deputy to Leahy in Map Room. 12 February 1986.
Forrest McCluney. Head of Special Branch couriers. 9 March 1986.
John McCrea. Naval aide to Roosevelt. 4 and 12 March 1986.
William Mott. Assistant to Leahy. 13 January and 23 March 1987.
Robert Myers. Assistant in Map Room. 26 April 1987.
William R. Perdue. Editor of Far Eastern Summary. 16 February 1987.
William McKinley Rigdon. Assistant to Leahy. 9 March 1986.
Frank Rowlett. Head of cryptanalytical teams that solved RED and PURPLE. 22 April 1986.
W. R. Smedberg. Head of naval combat intelligence. 12 May 1980.
John A. Tyree. Assistant to naval aide. 19 May 1987.


Letters

Carter W. Clarke. December 1985.
W. Preston Corderman. Commander of Signal Service Agency, 1943-1945. 5 December 1983.
John J. McCloy. Assistant secretary of war. 18 July 1984.
Henry W. Rigby. Editor of MAGIC Summary. 10 February 1987.
Lester R. Schultz. Delivered MAGIC to Roosevelt. 3 and 24 January 1986.
J. C. Sharp. Deputy chief, information policy, National Security Agency. 4 November 1986 (Serial J9420A).
Telford Taylor. Head of Special Branch unit at Bletchley Park. 6 January 1985.

Secondary Sources

(Includes only items cited more than once in the notes or of basic importance)

Adams, Henry H. Witness to Power: The Life of Fleet Admiral William D. Leahy. Annapolis, Maryland: Naval Institute Press, 1985.
Bell, Ernest L. An Initial View of ULTRA as an American Weapon. Keene, New Hampshire: TSU Press, 1977. (A reprint of SRH-005, "Use of [CX/MSS ULTRA] by the United States War Department, 1943-1945," 82 pp., and of SRH-006, "Synthesis of Experiences in the Use of ULTRA Intelligence by the U. S. Army Field Commands in the European Theatre of Operations," 29 pp.)
Cochran, Alexander S., Jr. "The Influence of 'Magic' Intelligence on Allied Strategy in the Mediterranean." In Craig L. Symonds, ed., New Aspects of Naval History: Selected Papers Presented at the Fourth Naval History Symposium, United States Naval Academy, 25-26 October 1979 (Annapolis, Maryland: Naval Institute Press, 1981), at 340-348.
__________ The MAGIC Diplomatic Summaries: A Chronological Finding Aid. New York: Garland Publishing, 1982.
Hannah, Theodore M. "Frank B. Rowlett—A Personal Profile." Cryptologic Spectrum (National Security Agency), 11 (April 1981), 4-22.
Hinsley, F. H., et al., British Intelligence in the Second World War Cambridge and New York: Cambridge University Press, 1979-1988.
Howe, George F. American Signal Intelligence in Northwest Africa and Western Europe. [Unclassified]. United States Cryptologic History, Series IV: World War II, Vol. 1. [Fort Meade, Maryland]: National Security Agency/Central Security Service, 1980.
Jones, F. C. Japan's New Order in Asia: Its Rise and Fall, 1937-1945. London: Oxford University Press, 1954.
Kahn, David. The Codebreakers: The Story of Secret Writing. New York: Macmillan, 1967.
Kesaris, Paul, ed. The MAGIC Documents: Summaries and Transcripts of the Top-Secret Diplomatic Communications of Japan, 1938-1945. Index compiled by David Wallace. Frederick, Maryland: University Publications of America, 1982.
Lewis, Graydon A., ed. A History of Communications Intelligence in the United States with Emphasis on the United States Navy. Denver: Naval Cryptologic Veterans Association, 1982.
Parrish, Thomas. The Ultra Americans: The U. S. Role in Breaking the Nazi Codes New York: Stein and Day, 1986.
Rohwer, Jürgen. "Ultra, the United States and the Battle of the Atlantic 1941." Paper delivered at the 2nd U. S. Army War College International Conference on Intelligence and Military Operations, Carlisle Barracks, Pennsylvania, 12-14 May 1987.

National Archives. Record Group 457. Records of the National Security Agency.

SRH-001. "Historical Background of the Signal Security Agency." Vol. 3. "The Peace." Preface signed by Historian, Army Security Agency. [Assistant Chief of Staff, G-2, 12 April 1946.] 416 pp. Reprinted as The Origin and Development of the Army Security Agency 1917-1947 (Laguna Hills, Calif.: Aegean Park Press, 1978).
SRH-029. "A Brief History of the Signal Intelligence Service." By William F. Friedman, June 19, 1942. 18 pp. Reprinted in Kesaris, ed., 17-31.
SRH-035. "History of the Special Branch, MIS [Military Intelligence Service], War Department, 1943-1944." 63 pp.
SRH-041. "M. I. S. Contributions to the War Effort." December 1945. 22 pp.
SRH-061. "Allocation of Special Security Officers to Special Branch, Military Intelligence Service, War Department, 1943-45." 33 pp.
SRH-062. "History of Military Intelligence Service. Reports Unit." September 1945. 116 pp.
SRH-117. "History of Special Branch, M. I. S." June 1944-September 1945. Unsigned. 11 pp. Reprinted in Kesaris, ed., 55-67.
SRH-132. "History of the Special Distribution Branch." 7 pp.
SRH-146. "Handling of Ultra Within the Military Intelligence Service (M. I. S.) 1941-1945." [A memorandum by Lieutenant Colonel Thomas E. Ervin, 29 January 1946.] 7 pp.
SRH-149. "A Brief History of Communications Intelligence in the United States." By Laurance F. Safford, Captain, U. S. N., Ret. 22 pp. Reprinted in Kesaris, ed., 1-16.
SRH-152. "Historical Review of OP-20-G." 17 February 1944. [Unsigned]. 13 pp.
SRH-154. "Signal Intelligence Disclosures in the Pearl Harbor Investigations." 47 pp.
SRH-159. "Preliminary Historical Report on the Solution of the 'B' Machine." By William F. Friedman. 14 October 1940. 10 pp.
SRH-305. "The Undeclared War: 'History of R. I.' [radio intelligence]." By L. F. Safford. 15 November 1943. 29 pp.
SRH-349. "The Achievements of the Signal Security Agency in World War II." February 1946. 115 pp.
SRH-355. "Naval Security Group History to World War II." Prepared and compiled by Captain J. S. Holtwick, Jr., U. S. N., Ret. June 1971. Part I, 464 pp.


The Papal Cipher Section In The Early Nineteenth Century

David Alvarez

ADDRESS: Department of Government, Saint Mary's College, Moraga CA 94575 USA.

ABSTRACT: In the early 19th century, papal cryptography was moribund. The pope's Secretariat maintained a small cipher section, but this unit was sadly neglected. It shunned cryptanalysis, and limited itself to handling the few secret communications that passed between the Vatican and its diplomatic representative abroad. Papal ciphers were simple, and provided only modest security.

KEYWORDS: Papacy, cryptography, ciphers.

Recent papal cryptography has received little attention from historians. Specialists have illuminated the early history of the subject, namely during the Renaissance, when the papacy was a major force in international politics. But they have not been inclined to pursue the subject into the modern period. As a result, little is known about the cipher practices of the Vatican in the 19th and 20th centuries. This note makes a preliminary foray into a relatively unexplored area.

At the beginning of the 19th century, papal cryptography was the responsibility of the Secretariat of Ciphers, a distinct department in the papal administration. Its staff consisted of three "cifristi" (cipher officers) under the direction of the Cipher Secretary.1. The department worked exclusively for the papal Secretariat of State which, under the direction of the Cardinal Secretary of State, supervised the external and internal affairs of the Papal States, the wide band of territory across central Italy under the temporal rule of the popes. In 1814, a reorganization abolished the Secretariat of Ciphers and the office of Cipher Secretary, and moved the cifristi into the Secretariat of State, where they became a separate class of officials. The reorganization also established the new post of "Substitute Secretary of State and Cipher Secretary". As deputy to the Cardinal Secretary of State, the Substitute assumed direction of the day-to-day-affairs of the department, including (in a purely titular capacity) responsibility for codes and ciphers.

1The post of Cipher Secretary was established in 1555. For a survey of early papal cryptography, see Aloys Meister, Die Geheimschrift im Dienste der Papstlichen Kurie (Paderborn, 1906)

In this period papal cryptography was closely linked with a single family, the Evangelisti. Giuseppe Evangelisti entered papal service in 1793 as a "scrittore" (copyist) in the Secretariat of State. His father, Francesco, had served the church as a cipher officer in the pontificate of Pope Pius VI (1775-1799). In 1801, Giuseppe followed the paternal example by joining the cifristi. He accompanied Cardinal Ercole Consalvi to the Congress of Vienna in 1814-1815. At his death in 1847, he had been the Secretariat's senior cipher officer for more than four decades. Filippo Evangelisti, Giuseppe's brother, maintained the family tradition by joining the cipher office in 1806. A graduate of the elite Collegio Romano, where he won academic prizes, Filippo served in the cipher section until 1840.2.

2Students of the modern papacy owe an enormous debt to Lajos Pasztor, whose researches have illuminated the history of the Secretariat of State in the 19th century. Unless otherwise indicated, the material for this article has been drawn from his magisterial La Segreteria di Stato e il suo archivio, 1814-1833, 2 volumes (Stuttgart, 1984)

Two other individuals worked in papal cryptography during this period. Vincenzo Scagliosi, of whom little is known, served in the old Secretariat of Ciphers from 1804 until 1814 when that department was incorporated into the Secretariat of State. His successor, Giuseppe Golt, remained in the cipher section until his retirement in 1847.

The cipher section was not an important unit in the Secretariat of State. Papal cryptography had declined since those years in the late Renaissance when the Argenti family, the illustrious predecessors of the Evangelisti, had made the papacy a leader in the field. In the first half of the 19th century, the cryptographic requirements of the Vatican were few, largely because of the modest position of the papacy in international affairs. In 1830 the papacy accredited diplomatic missions to only 11 states. By mid-century the number had increased to 13. These missions generated few secret communications. The record of the nunciature (embassy) in Madrid, one of the more important posts, is revealing. In 1828 the Secretariat of State sent only 12 enciphered dispatches to Madrid, while the nunciature sent no enciphered messages at all to Rome. In 1848 the Secretariat forwarded only 3 enciphered dispatches to Madrid, which responded with 7 of its own. Months might pass without secret correspondence between the Vatican and its nuncio (ambassador) in the Spain.3

3Giuseppe de Marchi, Le Nunziature apostoliche dal 1800 al 1956 (Rome, 1957), passim; Vicente Carcel Orti, Correspondencia Diplomatica del Nuncio Tiberi, 1827-1834 (Pamplona, 1976), xvii and passim; idem. "Los despachos de la Nunciatura de Madrid, 1847-1857," Archivum Historiae Pontificiae, 13 (1975), 311-400.


The volume of enciphered correspondence that passed between the Vatican and its representatives abroad had diminished to such an extent that there was insufficient work for the cifristi.4 The senior officer, Giuseppe Evangelisti, was usually able to handle all the enciphering and deciphering and still have time for other tasks around the Secretariat. After 1815, Filippo Evangelisti worked less and less with ciphers as he assumed duties in the registry of the Secretariat. It is unlikely that Giuseppe Golt was concerned with secret communications at all. Although assigned to the cipher section, Golt performed archival work and served as secretary to Monsignor Carlo Mauri, who held the post of Substitute for 16 years. After Mauri's death in 1830, Golt assumed responsibility for various legal matters of concern to the Secretariat, although he remained nominally attached to the cipher section.

4Lajos Pasztor, "La Segreteria di Stato di Gregorio XVI, 1833-1846," Archivum Historiae Pontificiae, 15 (1977), 322, 331.

* Fig.
The nuncio in Madrid reports an audience with the Queen of Spain, 15 June 1847. Deciphered in the papal Secretariat of State.

At any given time, the Vatican had in service a variety of codes and ciphers. The nuncio in The Hague, Monsignor Francesco Capaccini, used several systems.5 In the simplest procedure, Capaccini would compose a dispatch in which the paragraphs or sections were numbered consecutively. In the even-numbered sections there was no attempt at subterfuge; the sentences meant what they said. In the odd-numbered sections, however, the true meaning of a sentence was the opposite of that written. The statement "our position is strong" was meant to be understood as "our position is weak". The message was then sent in clear. Capaccini also used a jargon code in which an apparently open dispatch contained codewords. In this code, for example, Vatican = MUSEUM, Austria = MR. CERINI, Spain = MR. CARLI, Prince of Orange = MR. FREY, and ambassador = BOOKKEEPER. A third system consisted of two-, three-, and four-digit number groups. Words, listed in alphabetical order, were replaced by codegroups listed in numerical order and beginning with 0 or 1, while individual letters and syllables, listed separately, but again alphabetically, were replaced by groups beginning with 2. Each word could be represented by any of 10 sequential numbers, and proper nouns had, in addition, a corresponding codeword. Austria for example, could be represented by any number in the sequence 090 … 099 or by the codeword MOROCCO. The Prince of Orange was 1340 … 1349 or ERNESTO.

5Segreteria di Stato, Spogli: F. Capaccini, "Chiavi di cifre imaginate da Mr. Capaccini," Archivio Segreto Vaticano (ASV), Vatican City.

Capaccini was an enthusiast who delighted in creating new methods for secret communication. More typical, perhaps, of papal cryptography was a modified two-part nomenclator used by the nuncio in Vienna in 1814. Letters of the alphabet and syllables were represented by two-digit number groups from 00 = F to 99 = to. A list of four-digit groups, each beginning with 7, served for common nouns (7282 = emperor), proper names (7690 = Napoleon Bonaparte), and parts of speech (7070 = this). The number 5 was a null, but there were no homophones. In the 1840's the Madrid nunciature had in service three monoalphabetic ciphers. In the first (introduced by the indicator 4), the letters of the alphabet were replaced by ordered pairs of numbers (a = 1/2, b = 1/3, c = 1/4 …). In the second system (indicator 6), the letters were replaced by two-digit numerical groups in ascending order (a = 11, b = 12, c = 13, d = 14, ldots. In the third (indicator 1), the letters were replaced by either letters or numbers in a system in which the letters in the first half of the alphabet were replaced by those in the last half in reverse order (a = Z, b = Y, c = X …) and the remaining letters by numbers in declining order (n = 12, o = 11, p = 10, q = 9 …)/. This latter system occasionally substituted symbols for certain proper nouns [image] = France, C3 = Pope). In all three systems numbers were spelled out, and punctuation was provided by symbols (comma = >, period = √). None of the ciphers used homophones or nulls.6

6The Vienna and Madrid systems have been reconstructed by a comparison of the enciphered and deciphered versions of dispatches. For Vienna, see Segretaria di Stato, 1814, rubrica 242, busta 385, fascicolo 4, ASV. For Madrid see Segreteria di Stato (Esteri), 1847, rubrica 11, fogli 43-46, ASV.


These simple ciphers show how sadly neglected papal cryptography was.7 This neglect manifested a pervasive inertia and lassitude that made the papal administration a byword for inefficiency in the early 19th century. In the Secretariat of State the infrequent use of ciphers gave little incentive to improve established practices. Senior officials had no interest in cryptography. One cardinal secretary of state, Giuseppe Albani, considered a cipher section "totally useless" since diplomatic affairs could be conducted more easily through foreign ambassadors in Rome than through the nuncios in distant capitals.8 Even Francesco Capaccini, an amateur cryptographer and the only official with an appreciation for the subject, considered the cifristi superfluous. Upon his promotion to the dual post of Substitute and Cipher Secretary Capaccini promptly suggested the abolition of the cipher section and the transfer of its few tasks to an ordinary clerk. But the reform was not instituted at the time.

7The ciphers used by the Madrid nuncio in 1847 were inferior to the cipher prepared for a predecessor by Giovanni Battista Argenti in 1588. Compare Meister, 405-407.
8Between 1815 and 1850 several states (Russia, Prussia, Saxony, Hannover, Wurtemburg) maintained diplomatic missions in Rome without accepting a nuncio in their own capitals.

The cipher section was moribund and provided little control or direction. There was no serious attempt at cryptanalysis.9 Hardly more than clerks, the cifristi performed the mechanical tasks of encipherment and decipherment, but apparently gave little systematic thought to the problems of communications security. In one dangerous practice, for example, nuncios were allowed to invent their own codes and ciphers. While at The Hague, Monsignor Capaccini used at least 10 different systems in three years. Often a nuncio would use more than one system in a single dispatch; part of the text would be written in a numerical cipher, part in a letter cipher, and part in a jargon code. At times, officials in the Secretariat would despair at their inability to make any sense out of an incoming dispatch. Apparently, security did not compensate for the confusion. In 1801, the vatican's special envoy in Paris had no confidence in his ciphers. At the Congress of Vienna, Cardinal Consalvi correctly surmised that his cipher had been compromised by the Austrians. Fourteen years later, Capaccini complained that the Vatican's codes were open to everyone.10

9When papal officers intercepted two enciphered letters addressed to Napoleon's entourage on Elba, the Secretariat of State sent them to Cardinal Consalvi in Vienna with the suggestion that he or his cipher clerk, Giuseppe Evangelisti, try to solve them. Consalvi promptly turned the unbroken ciphers over to the Austrians. Ilario Rinieri, Corrispondenza inedita dei cardinali Consalvi e Pacca nel tempo del Congresso di Vienna, 1814-1815 (Turin, 1903), 232.
10Boulay de la Meurthe (ed.), Documents sur la Négociation du Concordat, 1 (paris, 1895), xxiii; Alessandro Roveri, La Missione Consalvi e il Congresso di Vienna, 3 (Rome, 1970), 545, 567-68; J. P. de Valk (ed.), Lettres de Francesco Capaccini (Brussels, 1983), 256.

Political decline, administrative enervation, and personal obtuseness all may have contributed to the decline of papal cryptography in the early 19th century. Whatever the explanation, the papacy had clearly slipped to a minor place in the world of secret communications.

BIOGRAPHICAL SKETCH

David Alvarez, PhD, is a professor of government at Saint Mary's College of California. His current research focuses on the role of the Vatican as an intelligence center during the world wars.


Cryptanalysis for Peacetime: Codebreaking and the Birth and Structure of the United Nations

Stephen Schlesinger

ADDRESS: 500 West 111th St. #4A, New York NY 10025 USA.

ABSTRACT: This paper examines how the United States used the information it obtained from the confidential diplomatic cable traffic of allies that it intercepted in the six months before and the two months during the conference in San Francisco drafting the charter of the United Nations in 1945 to influence the outcome of that meeting.

KEYWORDS: United Nations, Argentina, U.S.S.R., Poland, Ultra, Magic, peacetime, cryptanalysis, diplomatic, veto, colonies, trusteeship, UN veto, UN Security Council, San Francisco Conference.

[Editor's Note: This is one of the most important articles in the historiography of cryptology. Many are the stories about how codebreaking played a major role in wars. Only two cases are known in which cryptanalysis has significantly affected world events in peacetime. The first was the American solution under Herbert O. Yardley of Japanese diplomatic code messages which enabled the United States, at the Washington naval arms limitation conference of 1921-22, to push Japan into accepting fewer warships than it wanted. This article reports the second.

David Kahn]

The interception of foreign diplomatic traffic by the United States played a major role in enabling America to fashion the United Nations into the organization it wished at the San Francisco conference in 1945. The United States, the primary strategist behind the creation of the U.N., had a war-created cryptanalytic program that included the interception and solution of the embassy cables not only of its enemies, but also of its allies and of neutrals. As World War II wound down, America employed it to uncover the interests of the San Francisco participants in order to mold the organization's charter to its liking.

Secret U.S. files recently released under the Freedom of Information Act reveal how Pentagon operatives eavesdropped on friendly nations in the weeks leading up to the San Francisco meeting to find out how they were preparing for it and during the two months of the conference to find out how they were reacting to it. These documents suggest that, in producing a United Nations that the United States envisaged, it was indulging not only in altruism but also in national self-interest. Such revelations indicate that, in retrospect, some revisions in conventional historical judgments on the origins of the United Nations may now be in order.

* Fig.

The 635 pages of diplomatic messages came from the Army's Signal Security Agency, which broke codes and solved intercepts, and its Special Branch, which evaluated, edited, and distributed them as "Magic" Diplomatic Summaries. These summaries reported information obtained from intercepted and solved foreign diplomatic cryptograms. Reproduced in purple ink by the Ditto process and issued daily, they averaged 15 pages each. All were divided into three categories — military, political, and economic — and, where needed, into two others: psychological and subversive and miscellaneous. Sometimes annexes amplified some items or gave the full text of important documents; occasionally maps were included. Within each category, headings described the subject of the intercept or intercepts. The scope was extremely great, ranging from German military plans as reported by the Japanese ambassador to Afghan attitudes to Japanese intelligence activities. The documents do not include any British or Soviet messages.1

1The National Archives and Records Administration, Record Group 457 (Records of the National Security Agency / Central Security Service), " 'Magic' Diplomatic Summaries," 1942-1945. The NSA declassified portions of the summaries for me 28 October 1993 on the basis of my request of 13 August 1993 under the Freedom of Information Act. All are from 1945. The intercept numbers on the declassified portions were found in the full summaries by David Kahn, and sources in each summary are cited below by date (excluding "1945") and page. For the organization and operation of Special Branch, which produced and distributed the summaries, see David Kahn, "Roosevelt, Magic, and Ultra," Cryptologia, 16 (October 1992), 289-319.


The "Magic Summaries" reveal that:

• Washington knew in advance the negotiating positions of almost all the 50 or so countries that assembled in San Francisco; (Opposite) First page of a "Magic" Diplomatic Summary, showing security restrictions. A.C. of S., G-2 is the army assistant chief of staff for intelligence. C.W.C. are the initials of Carter W. Clarke, General Staff Corps, head of the Military Intelligence Divisions' Special Branch, which produced the summaries. The six-digit numbers in the margin are the numbers of intercepts; the DS 15 may be the number of a special study and the WCM are the initials of the drafter of item number 1.
• On key issues — whom to admit to the U.N., decolonization, the Security Council veto, the role of smaller countries, even Soviet views — the U.S. had crucial intelligence beforehand;
• Most nations, including the U.S., sought to push their own interests over those of the world community;
• The U.S. apparently used its surveillance reports to set the agenda of the U.N., to control the debate, to pressure nations to agree to its positions, and to write the U.N. Charter mostly according to its own blueprint.

Whether the U.S. was morally right to make use of Ultra - as the solutions were covernamed - is not an easy question to answer; America found itself in desperate times in the 1940s. Undoubtedly every country that had the capacity to intercept cable traffic was ready to take advantage of this capacity to ferret out as much as possible about the strategies of other governments. And surely the creation of as important a body as the U.N., given past historic failures, also merited special attention. Still, Ultra was not the proper way to treat allies and was of dubious legality.2

2Under the Federal Communications Act of 1934, "no person receiving…any interstate or foreign communication by wire or radio shall divulge or publish the existence, contents, substance …thereof…to any person other than the addressee…" (18 United States Code 605).

But President Franklin D. Roosevelt knew he was going to have a difficult time pushing a complicated U.N. structure, even if designed in large part to forestall another League of Nations disaster, through the U.S. Senate. Isolationist sentiment was still strong in the land. To make the United Nations palatable to Congress, Roosevelt, beginning at a conference of the leading Allied nations he convened on 21 August 1944, at Dumbarton Oaks, a study center in Washington, campaigned for a Security Council tightly controlled by the major powers.3 The United States, Great Britain, the Soviet Union, and China agreed at Dumbarton Oaks to the draft outlines of the United Nations, which provided for a General Assembly with rather modest authority, a Secretariat subject to major-power control, a Military Staff Committee composed of the Big Five (the four nations that met at Dumbarton plus France, which was also invited to be the fifth permanent Council member), and, of course, the all-powerful Security Council, an 11-member body of which the Big Five were to be permanent members with veto powers.

3See Robert Hilderbrand, Dumbarton Oaks: The Origins of the United Nations and the Search for Postwar Security (Chapel Hill: University of North Carolina Press, 1990).

The adoption of the latter veto provision which at first as left in limbo at Dumbarton Oaks reflected F.D.R.'s belief that the Security Council would actually run the United Nations and that, since these five nations were the only ones that possessed the forces to police the world, this prerogative was required. Extending the veto to all nations (as had been done for the Executive Council of the League of Nations) would invite gridlock and inaction. Four months later, at Yalta, Churchill and Stalin, at F.D.R.'s insistence, completed voting procedures reflecting this veto system for the United Nations.

With these building blocks in place, F.D.R. believed in the spring of 1945 that his special conception of the United Nations would have the best chance of acceptance by the other nations of the world. Not only was it in America's national interest, but it basically met the needs of all the earth's capitals. Still, he anticipated that smaller countries would fear the expansive authority of the organizing nations and would probably dispute the draft U.N. Charter. He knew he would probably have to twist arms to get his way.

The intercepted diplomatic notes nonetheless show that in the months leading up to the San Francisco conference, which began 25 April 1945, as well as during the two-month meeting itself, the U.S. used information from its codebreaking to help get its way on the U.N. issues about which it vitally cared.


I

One of the most vexing questions facing the conference from the onset was whether Argentina should be admitted. Soviet Premier Joseph Stalin opposed the admission of Argentina on the grounds that it was a crypto-Nazi country that had helped Hitler during the war. The "United Nations" was, in Stalin's reasoning, originally a wartime alliance of nations opposed to the Axis, not yet a global security organization, and thus like-minded countries should be its members. Most Latin American nations, however, for reasons of hemispheric solidarity, threatened to boycott the U.N. if Argentina's application were turned down. Although the U.S. had objected to Argentina's neutrality during nearly all the war, Washington knew that it had to resolve the question of Argentina's admission or face the possibility that either Moscow or Latin America would refuse to join the U.N., crippling the fledgling organization from the outset.

The general outlines of the Argentine dispute were known to the U.S. through the diplomatic contacts of Roosevelt's coordinator of inter-American affairs, Nelson Rockefeller. But Ultra told the United States precisely what tactics the Latin nations were beginning to pursue on Argentina's behalf, and what actions Moscow was, in turn, taking to prevent its admission.

Hemispheric countries, according to Ultra, already were aware that Roosevelt was sidestepping an urgent Argentine proposal in late 1944 to convene a meeting of the Pan American Union on postwar hemisphere problems. Argentina, which had severed its relations with Germany and Japan in January, was going to present its arguments for a continued neutrality at this meeting. Instead, Washington, to isolate Argentina within the Americas to force it to abandon its position, suggested that the Latin republics that had collaborated with Washington in the war effort meet in a conference outside the Pan American Union in Mexico and there consider the Argentine request.4 The United States may have supposed that, once it induced Buenos Aires to abandon its neutrality and declare war on Germany, it could recast that nation into a more acceptable applicant for U.N. membership.

46 January, 7-9. See also "The Inter-American Conference on Problems of War and Peace. Held at Mexico City, February 21 - March 8, 1945 (The Chapultec Conference)," in United States, Department of State, Foreign Relations of the United States: Diplomatic Papers, 1945, 9: The American States (Washington: Government Printing Office, 1969), 1-153; also 223-230.

The Ultra intercepts enabled the United States to track the reaction to F.D.R.'s proposal around the hemisphere. For example, it read an Ecuadoran message of 30 December 1944 to its chargé in Washington, Duran Ballen, instructing him to tell representatives of Chile and Colombia that Ecuador would back Buenos Aires' request. In his answer, Ballen stated that since 16 Latin countries had already accepted invitations to the conference, he advised de-emphasizing strong support for Argentina at the Pan American Governing Board. "I therefore suggest that I limit myself to stating Ecuador's point of view without seeking to contradict the plans prepared by nearly all the other countries," he wrote in a message that U.S. officials undoubtedly read with satisfaction. Ballen then did what he proposed doing.5

5Ibid.

U.S. codebreakers also solved a dispatch of 6 January 1945 from the Argentine chargé in Washington to his government. This reported that the Chilean ambassador was assuring him that the conference in Mexico - now scheduled for February - "should not be considered as an unfriendly act toward Argentina but rather as a means for reaching a solution." The Chilean soothed the Argentinian. Perhaps the meeting would "convert" itself into the Pan American Union conference on postwar problems that Argentina wanted. In a message to its Washington embassy, Chile's foreign ministry said that it had turned down a U.S. request that it demand that the Pan American Governing Board rebuff Buenos Aires' entreaty. And a day earlier, the Chilean ambassador in Brazil informed Santiago that Brazil's acting foreign minister, while not approving of the Mexican parley, nevertheless planned to attend it. In mid-February, shortly before the assemblage in Mexico City, another flurry of messages scanned by Ultra reported on new developments in Latin attitudes on the Argentina problem. Paraguay now proposed to Colombia, Ecuador, and Venezuela that pressure be put on Argentina directly to join the Allied war effort. Colombia, though, objected to that course as likely to be "badly received" and offered a gentler plan.

All these tidbits may have helped Washington intensify pressure on the other Latin countries, both within and without the Mexico meeting, to compel Argentina to reassess its ideological course.6 For example, the Chilean envoy in Washington telegraphed Santiago in mid-January about a luncheon with Nelson Rockefeller at the Cuban embassy. He and seven other ambassadors agreed there that Buenos Aires had to be told to "adopt concrete and effective measures to eradicate the conviction of all the American peoples that she has been antagonistic toward the democracies of the continent…" The relentless U.S. blitz against Buenos Aires was gradually generating a hemispheric condemnation that was starting to be felt by Argentine officials - at least those in Washington. Around the time of the Rockefeller lunch, the Argentine chargé in Washington informed his government that, while the Mexico conference would "cover a discussion of the diplomatic misunderstanding" (copying the same word as one of his Latin colleagues), his country might have to think about a compromise.7 By 21 March 1945, the Argentine military attaché in the U.S. cabled President Farrell of Argentina that it was "decidedly advisable" for Buenos Aires to back an idea recently floated by its foreign minister: the country should hold a plebiscite for or against a declaration of war against Germany and Japan. Ultra enhanced Washington's ability to anticipate objections from, evaluate shifts in the mood of, and calibrate the growing demands upon Argentina by, its neighbors. On 27 March 1945, Argentina entered the war against the Axis powers.8

68 January, 4-6; 11 January, 9.
719 January, 9-12; 22 January, 13.
826 March, 5-6.


Ecuador now tried to convince the other recalcitrant party to this drama, the U.S.S.R., to reconsider its opposition to Argentina's admission to the U.N. Ecuador enlisted Peru, Mexico, Cuba, Panama, Venezuela, and Brazil in this campaign. On 7 April, Colombia's foreign minister instructed his envoy in Moscow to tell the Soviet regime that if it were now to oppose Buenos Aires, "we fear it would be a bad beginning for her relations with the American countries, which, for the most part, have established relations with Russia without much conviction and against considerable internal opposition."9 With this information, the U.S. was in a position in its negotiations with Moscow at San Francisco to persuade Stalin's envoys to admit Argentina, thereby placating the Latins.10

916 April, 10-11.
10George McJimsey says in Harry Hopkins: Ally of the Poor and Defender of Democracy (Cambridge, Mass.: Harvard University Press, 1987), at 382, that Stalin gave in on Argentina's seating after "both Hopkins and Stalin were willing to concede issues that lay in the other's area of special interest."

II

A second great dispute arose over the U.S. desire to establish a Trusteeship Council at the U.N. Both the British and French feared that such a body might force them to give up their colonies after the war. The U.S. became keenly aware of French apprehensions as early as mid-January, 1945, through Ultra. France's provisional foreign minister, Georges Bidault, telegraphed his ambassador in Washington about reports that the Americans at a preliminary U.N. conference were pressing for an international U.N. committee for colonies modeled on the controversial Mandates Commission of the League of Nations. A French emissary reassured Bidault on the matter: "The American tendency of hastening the normal evolution of colonies toward autonomy, dominion status, or independence was met by objections — and even the protests — of qualified delegates…including ours."11 But the issue was not dead.

116 February, 5-7.

After the Yalta meeting in early February (from which the French had been excluded), the French representative in Moscow sought out Soviet Foreign Minister Vyacheslav Molotov to obtain additional assurances that the U.N. would not act against Paris's colonial possessions. Molotov responded that Paris should take the matter up with the White House, which, he said, was now assuming primary responsibility over the colonial matter. In any event, he informed the French that a system of "trusteeship" had been "defined only in principle" at Yalta.12 Solutions of French intercepts revealed these discussions to the Americans.

1224 February, 8-10.

The French later shared their concerns with the British. After meeting with British Foreign Secretary Anthony Eden in London, Bidault, according to Ultra, expressed his satisfaction with the results: "Mr. Eden explained to me that the idea [of the trusteeship] was an American one and would permit the United States to lay hands chastely on the Japanese islands in the Pacific. The system is not to be applied to any region in Europe nor to any colonies belonging to the Allied countries. The English are determined that no misunderstanding arise in this regard." In their talks, Eden and Prime Minister Winston Churchill approved a continuation of France's "privileged position" in Syria and Lebanon.13

136 March, 8-11.

But the French, not convinced of the British assurances, undertook a broader offensive to protect their colonies. For example, they convened talks in Paris between General Charles de Gaulle and the foreign minister of the Netherlands, which also had overseas territories. Both men agreed, according to Bidault, on "the impossibility of surrendering to an international authority any of their colonies." By early April, Bidault wired all his envoys that France would flatly reject any plan of international control "over all or part of her colonial empire or of the countries placed under her protection." She would, moreover, uphold trusteeships for former Japanese or Italian territories. Bidault told the Greek ambassador that the U.S. was promoting trusteeships simply because it "wants to exercise influence on other people's colonies for selfish political and economic reasons."14 The codebreaking helped shape U.S. policy. Just as the San Francisco conference was commencing, the U.S. began to reassess its thinking about France. It wanted France to join the Security Council and drop its championship of smaller nations opposed to the Big Five rule. It began to ease its pressure somewhat on the trusteeship issue. America's conciliatory stance and respect toward France's amour-propre had an immediate calming influence, the Ultra documents show.

1429 March, 9-10; 12 April, 6-8; 14 April, 8-10.

Paris began displaying more pragmatism on the colonial issue. A French official in the U.S. with experience in territorial problems advised his home office midway through the conference that France should not "turn down a text in which independence is set as the eventual goal for trusteeship, for we would be approximately the only ones to do so." By the end of the San Francisco conference, Bidault himself was imploring his delegates to get U.N. approval for an impartial commission to investigate France's treaty status in Syria and Lebanon. At the conference's conclusion, France's verdict on trusteeship was now cautiously favorable and no longer harshly antagonistic. Its delegate wrote Paris: "The settlement of the trusteeship problem fulfills in broad outlines the instructions of the French government."15 American diplomacy, guided in part by intelligence from Ultra, had maneuvered France into a satisfactory stand-down.

155 June, 14-15; 25 June, 7-9; 30 June, 4-6.


III

The third serious matter that roiled U.N. members from the outset was the exclusive veto power over U.N. actions that the Big Five as permanent members of the Security Council would hold. Before the conference, Washington had had hints of the profound misgivings of certain mid-sized countries about this allocation of power. France had provisionally turned down becoming the fifth permanent member of the Security Council (after the U.S., the Soviet Union, China, and the United Kingdom) with the accompanying right to the veto because of its ire over its exclusion from Yalta. It criticized the veto and held to a romantic notion of becoming the leader of the U.N.'s smaller nations. Ironically, in the months preceding the San Francisco conclave, F.D.R. also privately clashed with Stalin over the breadth of the veto. Stalin wanted the power to bar even discussion of issues in the Security Council, not merely to bar action by the Security Council, as F.D.R. wished. Whatever the source, F.D.R.'s effort to get the veto in any form was in much more serious trouble from U.N. members than many historians have previously thought.

* Fig.

First, U.S. intelligence obtained a message in mid-March 1945 from the Chilean foreign minister offering his "personal opinion" to his envoys that "the procedure devised at the Crimea Conference [Yalta] for voting in the Security Council is not in accord with the sovereign equality of all peace-loving states and, in operation, would put the permanent members of the Council above the law which will govern all nations." The minister requested his diplomats abroad to find out how other countries regarded the special status of the great powers at the U.N. He received at least five replies - from Costa Rica, Cuba, Italy, Switzerland and the Vatican - agreeing with Chile's reservations. This group was probably representative of the 50 or so states that were to convene at San Francisco, and their reservations suggested that an alarmingly large opposition to the U.S. position was in the making.16

1611 April, 12-13.

In a dispatch in late March, Turkish officials also expressed doubts about the voting procedures. They told French diplomats in Ankara that the setup "seemed destined to make lawful the projects of the large powers against the small - with the system of voting in the Security Council ensuring them impunity." The Turks warned that "the small states are inevitably going to be reduced to the status of satellites of the great." They also feared that bilateral alliances, as, for example, their 1939 mutual assistance pact with England, could be overridden during a crisis, say, with the U.S.S.R., by the veto. But they also conceded the "futility" of modifying the U.N. Charter. Instead they hoped to increase "the number of non-permanent members on the Security Council from six to nine in order to give the Great Powers a less preponderant majority."17

1713 April, 7-9.

(Opposite) Page 7 of the "Magic" Diplomatic Summary for 2 May 1945, reporting on a French intercept.

* Fig.
The eavesdropper and his victim, U. S. Secretary of State Edward Stettinius, left, and French Foreign Minister Georges Bidault smile at one another. Would Bidault have been so pleased if he knew that Stettinius was reading his most secret cables?

Likewise, France's initial worries with respect to Security Council procedures grew as the conference neared. In early March, Bidault instructed his ambassador in Moscow to advise the Soviets of his concern that, under the veto arrangement, regional and bilateral treaties could be "subordinated to the previous agreement of the Security Council." This is dangerous, he added, because the "automatic nature of regional pacts is …the essential element of collective security …" The French enlisted the Belgians and sounded out the British on an amendment to "clarify" the section on regional agreements in the U.N. Charter.18

187 March, 6-8; 9 March, 6-7.

With such intelligence data, the United States was able to develop its arguments on behalf of the veto well in advance and thus disable the opposition. Washington contended that, as a matter of realpolitik, there simply was not going to be a viable U.N. organization unless the four or five most powerful nations received the veto. Without it, none of these countries, the U.S. insisted, would entrust any of its sovereignty to an international group, especially not to a resurrected intrusive and powerless League of Nations-like body. The U.S. Congress, for instance, would not ratify the pact and would probably allow America to return to isolationism. Nor would the Soviets join without the veto. And, as a matter of practicality, U.S, delegates added, the other nations in the United Nations would never have enough weapons or influence to impose U.N. edicts on the great powers except with the concurrence of the Big Five themselves. Finally, even with an organization that might distribute authority in a way that would seem unsatisfactory to lesser powers, the superpowers would still always remain in the dock of world opinion. This would stabilize the peace system better than none at all.19

19Clark Eichelberger, Organizing for Peace: A Personal History of the United Nations (New York: Harper amp; Row, 1977), 165; Evan Luard, Conflict and Peace in the Modern International System (New York: Macmillan, 1988), 212-215; Harry Truman, Memoirs: Years of Decision (New York: Doubleday, 1955), 284-285; Sumner Welles, Seven Decisions That: Shaped History (New York: Harper, 1950), 185.


Alerted to France's hesitancy by its reading of the French diplomatic transmissions, Washington decided to focus its campaign for the veto on Paris, which had now made itself the leader of the recalcitrant member-states. As the San Francisco conference commenced, America intensified its overtures to France to reconsider its decision to forgo its role as the fifth permanent member of the Security Council. This approach, coming at a time when France was finding it increasingly difficult to act as the champion of the smaller nations, and flattering France's pretensions to being a great power and salving its hurt over Yalta, reignited the Quai d'Orsay's interest.20 The French soon decided to accept their earlier assigned spot on the Security Council. With France's decision, the campaign to thwart the veto collapsed.

20Hilderbrand, 40, 120-121; "France Lining Up With Big Powers" (25 April 1945), "France's Position Put by De Gaulle" (26 April 1945), "France Position Still in Doubt" (28 April 1945), "Hints France Asks Major-Power Role" (3 May 1945), all The New York Times.

French diplomats saw fresh virtues in the arrangement, Ultra showed. While ostensibly staying above the battle, they were signaling a change of heart. The French U.N. delegate cabled Paris: "However far apart we [the Four Powers and small and medium powers] still are, a conciliatory solution is not impossible, for everyone is beginning to realize that the veto is a necessity and that its limits could not be further defined without risks for which no one wishes seriously to assume responsibility." Later, in summing up his country's achievements at San Francisco, the French delegate reflected: "although it [the veto] may in some cases seem an annoyance - and a very grave annoyance - it may also in others be a means of preventing the Council from meddling unduly in affairs which are our own or which we intend to settle through other channels."21 Finally, discussions at San Francisco had rendered moot the concern of France and Turkey that bilateral and regional pacts would be undermined by the veto. Several Ultra dispatches showed that France no longer worried about this. The American victory was complete.

2130 June, 4-6.

IV

The fourth event of importance illuminated by the Ultra files were the chronically touchy U.S.-Soviet relations. Intercepted cables - none of them Soviet - contained conflicting assessments of Soviet intentions. On the one hand, they showed the Soviets holding firm to their agreement with F.D.R. forged at Yalta, which had included a commitment to free elections in Poland. On the other hand, Moscow's surreptitious resistance to democratic government in Poland, and its heavy-handed pressures on such border countries as Iran and Turkey, revealed a Soviet Union that was growing increasingly obstinate about its territorial security. Ironically, even after the intercepts disclosed potentially alarming Russian moves, the United States remained puzzled about Moscow's intentions and unsure of how to act toward Stalin. American reactions swung between confrontation and conciliation.

Washington learned from intercepted messages, for example, that Moscow, at least in private conversations with some European powers, was treating the Yalta agreements as sacrosanct. Just days after Yalta, Molotov told the French ambassador to the Soviet Union, according to that envoy's dispatch to Paris on 19 February, that Stalin had expressed great confidence at the outcome of the meeting with Roosevelt and Churchill. And Molotov, according to the official, said that Stalin guaranteed that France would receive "an equal place" with Great Britain, Russia and the U.S. at the U.N.22

2226 February, 8-10.

But Stalin was not willing to jeopardize his agreements with Washington and London to please Paris over the latter's aggressive insistence that the three powers reword the joint summons to San Francisco - an invitation in which France had been asked to join by Moscow - to promise preservation of regional and bilateral pacts. The French ambassador messaged home his guess as to why the Soviets refused to rephrase the invitation: "I have reason to believe that at the present stage one of the prime concerns of Soviet policy is carefully to avoid anything that could weaken the assertion of perfect unity of viewpoint and action proclaimed by the Three Powers meeting at Yalta." Why? Because, he concluded, Moscow was "the principal beneficiary of the accord."23

2316 March, 6-7.

* Fig.
Soviet Foreign Minister Vyacheslav Molotov at the podium in San Francisco. His cables, at least, were not read. Photo - National Archives

On the other hand, America was picking up ominous reports from foreign emissaries in Moscow and elsewhere concerning Poland and other Soviet border nations. The crux of the issue with respect to Poland was Stalin's demand that his puppet regime in Lublin be admitted to the U.N. and the U.S. response that Stalin first democratize Lublin by including members of the prewar Polish government-in-exile in London. At Yalta, Stalin had agreed to a three-way British-American-Soviet commission to try to work out a coalition government in Poland. But Washington soon knew from the Ultra records and from its own ambassador in Moscow, Averell Harriman, that Stalin was reneging on that deal. France's delegate to Stalin's puppet regime in Lublin wired Paris that Lublin's policies were gradually turning it into a "quasi-protectorate" of the Soviet Union. Meantime, in early March, France's Moscow envoy reported that the commission was "running into great difficulties" because, according to information from Harriman, the Soviets didn't like "the choice of Polish leaders to be consulted." By mid-March, the French diplomat had had conversations with Poland's pro-Russian emissary to Moscow and informed Paris that the Soviets would probably impose their own regime on Poland.24

2410 March, 8-9; 22 March, 7-9.

The impasse over Poland persisted for weeks, gravely imperiling the United Nations' ratification. Stalin was, according to some observers, on the edge of aborting the whole idea of a global organization - a "Western" idea about which he was never enthusiastic - over the matter. Eventually President Harry Truman, in the midst of the San Francisco conference, sent the late F.D.R.'s closest confidante, the seriously ill Harry Hopkins, who knew Stalin from past encounters, as his emissary to the Soviet leader to work out a settlement on Lublin. Hopkins, though terminally ill, arrived in Moscow in late May and conferred with the Soviet chieftain for 10 days.


Hopkins eventually obtained Stalin's agreement to a high-level meeting in Moscow on the beleaguered nation's future to be supervised by the three-way commission. The French liaison in London to the Polish government-in-exile learned of the Hopkins deal from his contacts and, in a message read in Washington, informed Paris in early June that the meeting would include all Polish leadership inside and outside the country. But he noted that the Polish exiles were disappointed that so few representatives of the democratic parties had been invited. Harriman later told the French in md-June that "it is not impossible that a successful conclusion will be reached" at the upcoming Moscow conclave on Poland. By the end of June, the session did produce a Provisional Government of National Unity that nominally included members of all political parties; as a result of this settlement, by early July, the U.S. and Great Britain formally recognized the regime and thereafter it gained U.N. membership. Ultra had warned Washington how stubborn and unyielding Stalin was likely to be on Poland and thereby had encouraged a U.S. backdown on the issue.

During this time, Washington had taken its own action to ward off what it saw as a potential Soviet intrusion in its own sphere of influence in Latin America. It began to caution Latin states throughout the hemisphere against Communist subversion. Ultra revealed that a Venezuelan diplomat telegraphed his home office on 7 May after a session with Rockefeller, the coordinator of inter-American affairs, that "Rockefeller communicated to us the anxiety of the United States government about the Russian attitude." American officials, the envoy wrote, were "beginning to speak of Communism as they once spoke of Nazism, and are invoking continental solidarity and hemispheric defense against it."

Still, as the Ultra documents illuminate, even as it maneuvered its way between various Soviet moves and countermoves, the United States continued to concentrate on the main target at hand - overcoming the Soviet Union's reservations about the United Nations, especially on such matters as Argentina's admission, the breadth of the veto, and Poland. Thus it showed a distinct unwillingness to allow any of these outside events to derail the conference. While F.D.R. at Yalta, and Truman in his first days in office, bargained hard on most issues with Stalin and Molotov, both always judged that the U.S. national interest in establishing the U.N. made it necessary to accommodate Stalin where otherwise they might have challenged him.

At Yalta, F.D.R. accepted the dictator's desire for three votes at the U.N. (one for the Soviet Union, and one for each of its "territories," the Ukraine and Byelorussia). Truman eventually backed a vague settlement of the Polish situation and gave in on a host of other protocol matters at San Francisco. In turn, though, the U.S. gained Stalin's assent to what is now generally regarded as an essential international body. That that assent was a necessity was well understood even then. As the under secretary of state, Sumner Welles, wrote a few years later, without the U.N. "war between the Soviet Union and Western powers would already have been inevitable, and the fate of our civilization would today be trembling in the balance."25

25Welles, xviii.

The San Francisco conference ended in late June 1945. Helped by Ultra, Washington by and large achieved what it wanted: a Security Council controlled by the five Allies, a weaker General Assembly, and a malleable Secretariat and military commission (though the latter soon vanished in the Cold War mists.) The Ultra intercepts gave the United States advance warnings about problems with members' admissions to the U.N., with decolonization, with the veto and with U.S.-Soviet relations - all essential to America's various bargaining positions. The Ultra intelligence thus gave Washington an edge in its public and in its behind-the-scenes efforts at the conference.

In the spring of 1945, the U.S. already had a very big edge - Franklin Roosevelt had formally crafted the idea for the United Nations, he had organized the founding meeting in an American city (incidentally making interception easier for the Americans), nearly every country in the world wanted to join it, and the United States was now the most powerful nation on earth, possessing the richest economy and the strongest military. Nonetheless, Washington had to be absolutely certain of gaining its objectives at San Francisco, or the United Nations it desired might have fallen apart. America consequently used every weapon in its arsenal, including one of its most secret - Ultra. And it achieved what it sought.

* Fig.
Stettenius signs the charter of the United Nations for the United States, as President Harry S Truman watches.

BIOGRAPHICAL SKETCH

Stephen Schlesinger is currently a visiting scholar at the Taub Urban Research Center at New York University. He recently ended service as director of international organizations in the New York State government. In the 1970s, he was a staff writer at Time magazine and editor of The New Democrat. A graduate of Harvard Law School, he is also the author of The New Reformers (Houghton Mifflin, 1974) and co-author of Bitter Fruit (Doubleday, 1982).


Italian Diplomatic Cryptanalysis in World War I

David Alvarez

ADDRESS: Department of Government, St. Mary's College, Moraga CA 94575 USA.

ABSTRACT: After its entry into the First World War in May 1915, Italy established a cryptanalytic unit to attack the military and diplomatic cryptosystems of other governments. This unit retrospectively solved an Austrian diplomatic system and currently read an American system, but it succeeded mainly against the codes and ciphers of minor powers.

KEYWORDS: Italy, diplomatic cryptanalysis, World War I.

1 INTRODUCTION

Although the combatants in the First World War eventually exploited signals intelligence on a large scale, few entered the conflict able to intercept and read secret diplomatic communications. In 1914 France and Russia supported small establishments of codebreakers in their foreign offices and interior ministries, and the Austro-Hungarian secret service purchased the occasional codebook from a venal embassy clerk, but other governments remained indifferent to the possibility of eavesdropping on friends and foes. Italy, which cast its lot with the Entente by declaring war on Austria-Hungary on 5 May 1915, was one of the powers that belatedly recognized the potential of signals intelligence. Though modern cryptology had practically been invented on their peninsula in the early Renaissance, the Italians had long since lost touch with their distinguished heritage and, like most of the belligerents, they entered the war with no codebreaking capability. In 1915 Italy's Foreign Ministry and Interior Ministry had cipher offices, but these units limited their effort to compiling new cryptosystems and encoding and decoding messages from Italy's diplomatic missions and regional prefectures. The cryptologic service of the Italian army consisted of a single officer whose limitations were revealed before the war when he labored two months to solve a correspondence encoded in two commercial systems without being aware that several years earlier a French expert had developed a mechanical solution to such a problem that would have reduced his labor to hours.

With such resources Italy was woefully unprepared to attack the various diplomatic messages that it had begun to intercept even before it entered the war. An appeal to France for assistance was rebuffed, probably because the French cryptanalysts, who had been regularly cracking Italian diplomatic codes since 1887, were reluctant to share their skills with their new ally.1 Within a year of joining the conflict (circumstantial evidence suggests a date no earlier than the fall of 1915), Italian military intelligence created a three-man cryptanalytic unit and placed it under Luigi Sacco, a lieutenant of engineers who as head of the army's radio intercept service had begun to work on encrypted Austrian messages in his spare time.2 This Reparto crittografico (Cryptographic Bureau), which eventually grew to several score officers, was responsible for both diplomatic and military cryptanalysis. While historians have recorded some of its military successes, such as the solution of Austria's CW and CARNIA codes, systems of 1000 groups for use within a single army, almost nothing is known of its work against the diplomatic cryptosystems of other powers, mainly because of the absence of official records and documentation.3 It is possible, however, to catch a glimpse of the bureau's diplomatic work in the personal papers of Sidney Sonnino, Italy's wartime foreign minister. Military intelligence passed to the foreign minister various diplomatic intercepts and these documents allow us to sketch the broad outlines of Italian diplomatic cryptanalysis.4

1David Kahn, The Codebreakers (New York, 1967), 318; Christopher Andrew, "Déchiffrement et Diplomatie: Le cabinet noir du Quai d'Orsay sous la Troisième République," Relations internationales, 5 (1976), 37-64, at 42-43.
2The birthdate of the Cryptographic Bureau remains obscure. In his memoirs the wartime director of the intelligence service, Odoardo Marchetti, recalled that for a period following the declaration of war the Italian army did nothing but file the cryptograms it intercepted. Then "for a long time" Lieutenant Sacco worked alone on the intercepts before his results convinced his superiors to establish the Cryptographic Bureau. A date in late 1915 or early 1916 seems reasonable. Odoardo Marchetti, Il servizio informazione dell' esercito italiano nella grande guerra (Rome, 1937), 88.
3Repeated inquiries at the historical office of the Italian army have failed to determine whether the records of the cryptographic bureau still exist.
4The Sonnino Papers are in the family archive in Montespertoli (Tuscany), Italy. A microfilm copy is in the Hoover Library, Stanford University, Palo Alto, California.


2 ENEMIES

Italy eventually declared war on Germany, Bulgaria, and Turkey, but the Austro-Hungarian Empire remained throughout the war its principal antagonist. The bulk of the Italian army was engaged against the Austrians along the common border, which ran through the rugged peaks of the Dolomites and the Julian Alps, and it was from the Austrians that Italy expected to win territory in the Trentino and along the eastern shore of the Adriatic Sea. Italian intelligence began intercepting Austrian diplomatic messages as early as March 1915, during negotiations over Italy's neutrality. A member of the Triple Alliance and, therefore, nominally an ally of Austria and Germany, Italy had remained on the sidelines during the crisis of July-August 1914 when its partners had gambled on war. By the spring of 1915 Rome was bargaining in London and in Vienna to determine which alliance was prepared to pay the most for Italian military support. Like all diplomatic missions in the Eternal City, the two Austrian embassies (one accredited to the Italian government and the other to the papacy) used the telegraphic facilities of the Italian postal service to communicate with their foreign ministry. The Austrians, who were old hands at intercepting secret communications, assumed correctly that the Rome telegraph office passed to Italian intelligence copies of all embassy cables, but they relied on their codes to protect the secrecy of their messages. They were not disappointed. Lacking a cryptanalytic capability (the Reparto crittografico was yet to be created), the Italians could do nothing but stare in frustration at the rows of numerical groups on the telegraph forms delivered each day by the central post office.

The inability to penetrate Austrian cryptosystems placed the Italians at a serious disadvantage during the negotiations in the spring of 1915 since cryptanalysts in Vienna were reading the encoded correspondence of the Italian embassy.5 The inability to monitor the exchanges between Austrian diplomats was especially disappointing to Italian policy-makers since they did not entirely trust their ambassador in Vienna (who was a noted Austrophile) to report accurately the true intentions of the Dual Monarchy.6 For some time after the creation of the Reparto crittografico the Austrian diplomatic codes resisted attack. Luigi Sacco's codebreakers did not register their first success until the late summer of 1916 and though this target remains unidentified it was almost certainly not an Austrian diplomatic system.7 Indeed, the earliest diplomatic decrypts in the Sonnino Papers date from December 1916, and consist of a handful of Vatican telegrams.

5In 1919 the Italian Foreign Ministry learned that the Austrians had been able to read at least four of its cryptosystems. After interrogating an officer in the Austrian cipher service, Italian intelligence concluded, "The Italian diplomatic code [sic] has for a long time been known to Austria which easily decodes any Italian telegram." The report noted that Austria's penetration of Italy's codes began before the war. "Promemoria," Commando della 1 Armata, 23 March 1919, reel 51, Sonnino Papers.
6William Renzi, In the Shadow of the Sword: Italy's Neutrality and Entrance into the Great War, 1914-1915 (New York, 1987), 71, 176-177.
7Cesare Amè, Guerra Segreta in Italia, 1940-1943 (Rome, 1954), 48. Sacco's unit did not achieve a complete solution of an Austrian military cyptogram until August 1917. Kahn, The Codebreakers, 318.

The first evidence of progress on the Austrian codes appears in September 1917, when military intelligence sent Sonnino plaintext versions of the encoded telegrams exchanged between the foreign ministry in Vienna and its embassy at the Vatican during the period March-May 1915. Though substantially complete, these solutions contain unrecovered groups and tentative identifications, which indicate that Sacco's cryptanalysts were still reconstructing the code. In October 1917 Sonnino received solutions of four telegrams exchanged between Vienna and its embassy to the Italian government in April and May 1915; military intelligence assured the foreign minister that additional telegrams were forthcoming. Though these four telegrams were also incomplete, the codebreakers had clearly made a significant (though tardy) entry into the Austrian systems of 1915.8 Surprisingly, there was little further progress for several months. In April 1918 military intelligence sent the foreign minister "corrected" versions of the four cables. Later that month Sonnino received plaintext versions of all telegrams sent or received by the Austrian embassy in Rome during the period March to May 1915, though these decrypts still contained a few unsolved groups and some tentative identifications.9

8"Raccolta di telegrammi scambiati tra il Ministro degli Esteri Austro-Ungarico e l'ambasciatore A.U. presso la Sante Sede nel periodo 13 marzo 1915 - 22 maggio 1915," reel 52, Sonnino Papers; Chief of Section R to Sonnino, 22 October 1917, reel 47, Sonnino Papers. The characteristics of the Austrian system(s) remain unknown.
9Chief of the Intelligence Service to Sonnino, April 1918, reel 47, Sonnino Papers.

During the war Italian intelligence organized several clandestine incursions into diplomatic missions in various countries in search of secret documents, but the progress against the Austrian diplomatic codes apparently depended upon pure cryptanalysis, though Sacco's codebreakers were able to compare the intercepted codetexts of certain Austrian cables with the plaintext versions which Vienna (in an uncharacteristic indiscretion) included in the "Red Book" of selected documents that it published in the summer of 1915 to justify its policy toward Italy.10 Of course the success against the 1915 messages was of limited value in 1917. Sacco later claimed that during the war his unit solved a superenciphered Austrian diplomatic11 Without further details it is impossible to determine whether he is referring to the 1915 messages or to a second effort against the Austrians. There is no evidence of a second penetration of Austrian diplomatic codes in the papers of Sidney Sonnino.

10When Italy entered the war Italian intelligence did not possess any foreign codebooks or cipher tables. Marchetti, Il servizio informazione, 87.
1111 Kahn, The Codebreakers, 319.

Though modest, Italy's achievement against Austrian diplomatic codes starkly contrasted with its even worse record against the cryptosystems of the other Central Powers. The Sonnino Papers offer no evidence that Italian cryptanalysts were able to crack the diplomatic codes of Germany, Bulgaria, or Turkey.


3 ALLIES

One of the ironies of international relations is that a government may expend as much effort to collect intelligence on friendly powers as it does to secure information about hostile ones. Because Italy was constantly concerned that Britain, France and Russia might abandon her (as she had abandoned Germany and Austria-Hungary in 1914) by arranging a separate peace with Austria-Hungary or reneging on secret agreements for the distribution of the territorial spoils of victory, she kept a wary eye on her allies throughout the war. The papers of the foreign minister contain reports from Italian army and naval intelligence on such subjects as Allied politicians, French morale, British secret service operations, and alleged contacts between British, French or Russian diplomats and their Austrian and German counterparts. This discreet intelligence effort included attempts to monitor the secret communications of the Allied governments. The Reparto crittografico apparently had no success against the encrypted communications of Britain, France, and Russia, but it was able to crack at least some of the cryptosystems of Greece and the United States. For example, a cable from the Greek ambassador in London to his foreign ministry, dated 12 June 1918 and sent to Sonnino by the Reparto crittografico on 5 July ("Deciphering delayed by a change of cipher") revealed that British intelligence was assisting a Bulgarian revolutionary named Bonbolof to return to his country to foment a coup that would remove the king and separate Bulgaria from the Central Powers.12 In the last days of the war an intercepted telegram from the State Department in Washington to the American embassy in Rome exposed Anglo-American concerns over financial conditions and currency stabilization in Archangel and Siberia, Russian territories occupied by or under the influence of Allied military forces, which had invaded the areas as an anti-Bolshevik measure. When passed to Sonnino, this intercept contained 22 undeciphered groups, suggesting that Italian cryptanalysts were still reconstructing this American code at the end of the war. By the time the peace conference convened at Versailles, however, the codebreakers had sufficient command of American cryptosystems to allow Sonnino to eavesdrop as the State Department and its embassy in Rome discussed the prospects for replacing him as foreign minister with an individual more amendable to American plans for the postwar world.13

12"3 telegrammi decifrati," reel 52, Sonnino Papers. Italian intelligence dated this telegram 12 July 1918, but since it was already decrypted and in Sonnino's hands by 5 July, the later date is clearly a slip of the pen.
13"5 telegrammi decifrati," reel 51, Sonnino Papers; Denis Mack Smith, Italy and its Monarchy (New Haven, 1989), 234.

4 NEUTRALS

The communications of neutral powers also attracted the attention of Italian intelligence. During the wax these governments maintained their diplomatic representation in Berlin, Vienna, Constantinople, and Sofia, and the confidential telegrams of their embassies offered Italy an opportunity to gain some insight into the political, military and economic affairs of the Central Powers. Also, the reports cabled to their respective foreign ministries by neutral embassies in Allied capitals could usefully supplement the information reported by Italian representatives in those cities. Finally, the surveillance of neutral communications might alert Italian authorities to the activities of the army of spies, revolutionaries, secret emissaries, and confidence tricksters who found Berne, Madrid and Stockholm secure locales for their intrigues.

By the end of the war the Reparto crittografico had penetrated at least some of the diplomatic cryptosystems of Spain and Switzerland and were reading messages on the Madrid-Rome and Berne-Rome circuits. Spanish codes (which seem to have been read by almost everybody) were broken as early as September 1917. A decrypted cable to Madrid from the Spanish embassy to the Vatican alerted Sonnino that Pope Benedict XV was pressing British Prime Minister David Lloyd George to remain open to negotiations in the face of Berlin's unsatisfactory response to the pope's peace proposal of August. The telegrams of the Spanish and Swiss embassies in Rome were intercepted at the local telegraphic office, but some Spanish radio messages were plucked from the air by Italian listening posts. On 2 July 1918, for example, the Italians intercepted and decrypted a radiogram from the Spanish embassy in Vienna. It was forwarding to Madrid a message from the Spanish embassy in Constantinople concerning Turkey's interest in a separate peace.14

14"3 telegrammi decifrati," reel 52, Sonnino Papers.


The Vatican, the smallest of the states, provided the Reparto crittografico with its greatest success and its richest harvest of diplomatic intelligence. Throughout the war Italy considered the Vatican a major intelligence target; indeed, Rome's intelligence effort against the papacy was matched only by its effort against its principal military opponent, Austria-Hungary. The preoccupation with the Vatican reflected, in part, a continuing concern with the "Roman Question." Ever since the occupation of Rome by the forces of the House of Savoy in 1870 and the incorporation of the Papal States into the Kingdom of Italy, the papacy and the Italian government had glared at each other across the symbolic boundary of the Tiber. The popes had never renounced their claims to temporal power and the Italian authorities suspected, with some reason, that they were alert for any opportunity to reassert those claims. With the outbreak of war Italy became especially apprehensive that the Vatican would find an opportunity in the political dislocations of the conflict and its aftermath to seek a restoration of at least some of its former territory. These apprehensions were sufficiently great that Rome insisted that the 1915 Treaty of London, which formalized its adhesion to the Allied cause, include a secret clause (Article 15) pledging London, Paris, and St. Petersburg to support Italy in opposing any role for the Vatican in a future peace conference. For the Italian government suspected the political sympathies of the papacy. The Vatican had long considered the Austro-Hungarian Empire its strongest supporter in Europe, while its attitude toward anti-clerical France was exceeded in coolness only by its feelings toward Italy itself. The Italians feared that the Vatican might place its considerable influence at the service of the Central Powers in return for a commitment from Berlin and Vienna to support the restoration of the popes to some form of temporal power at the expense of Italy.15

15David Alvarez, "A German Agent in the Vatican: The Gerlach Affair," Intelligence and National Security (forthcoming).

The earliest Vatican decrypts in the Sonnino papers, a few telegrams between the papal Secretariat of State and its nuncio (ambassador) in Vienna, date from the end of 1916, but by 1918 Italian intelligence was routinely intercepting and reading the communications of nuncios in Austria, Belgium, Germany, Holland, Spain, Switzerland, Turkey, and the United States. The Reparto crittografico had such a command of papal codes that it was able to read messages which, as a result of mistakes by papal code clerks, were gibberish to their intended recipients. When the Secretariat of State mistakenly sent a telegram to the nunciature (embassy) in Germany in a system reserved for the nunciature in Austria, Sacco's cryptanalysts immediately caught the mistake and were able to read the message while papal officials were still trying to sort things out. On another occasion the codebreakers expressed their frustration with the sloppy cryptographic practices of papal diplomats by complaining at the end of one decrypt: "This telegram contains so many errors, omissions and repetitions that it is in some parts indecipherable."16 The effort against papal communications faltered only in the last year of the war when the Vatican improved its cryptographic procedures and introduced superencipherment for its more confidential messages.17

16Pacelli to Gasparri, 11 October 1917, and Ragonesi to Gasparri, 18 August 1917, reel 48, Sonnino Papers.
17David Alvarez, "Faded Lustre: Vatican Cryptography, 1815-1920," Cryptologia (forthcoming).

Through the surveillance of papal communications, Italian intelligence was privy to all aspects of the Vatican's secret diplomacy. The cryptanalysts must have smiled as they decoded the warning, "Destroy this message and maintain absolute secrecy in this matter," in a telegram from the Cardinal Secretary of State to the nuncios in Munich (where the nunciature to Germany was located) and Vienna concerning an opportunity to detach Italy from its Entente partners.18 Intercepts proved especially useful in the late summer of 1917 as the belligerent and neutral capitals reacted to Pope Benedict XV's seven-point peace proposal of 1 August. Decrypts of traffic on the Munich-Vatican circuit revealed that the Vatican had discussed the terms of the proposal with Berlin before presenting it to the Allies. Decrypts also revealed that in its effort to sell the proposal to the Allies the Vatican exaggerated the extent to which Berlin and Vienna were prepared to make political and territorial concessions.19 Surveillance of papal communications also alerted the Italian foreign ministry to the Vatican's efforts, despite Article 15 of the Treaty of London, to secure representation at the Versailles peace conference by mobilizing Catholic opinion and lobbying statesmen such as U. S. President Woodrow Wilson and British Prime Minister David Lloyd George. Forewarned, Foreign Minister Sonnino counteracted such papal maneuvers as a plan to stimulate a press campaign against Article 15 and an effort to enlist the sympathies of the British monarch, George V.20

18Gasparri to Aversa and Valfre, 23 February 1917, reel 48, Sonnino Papers.
19Pacelli to Gasparri, 27 September 1917, and Valfre to Gasparri, 25 August 1917, reel 48, Sonnino Papers; Italo Garzia, La Questione Romana durante la prima guerra mondiale (Naples, 1981), 161-164.
20Ragonesi to Gasparri, 14 January 1918, Gasparri to Ogno, 15 December 1918 and 20 April 1919, reel 48, Sonnino Papers; Dragan Zivojinovic, The United States and the Vatican Policies, 1914-1918 (Boulder, 1978), 134-138.


Occasionally, the Vatican's diplomatic messages contained items of military interest. This was especially true in the fall of 1917 when Austro-German forces smashed the Italians at Caporetto, advanced across northeastern Italy, and threatened Venice. From Berne, Munich, and Vienna, papal diplomats cabled to the Vatican daily reports on the offensive. Gleaned from official bulletins, contacts in the host government, and conversations with colleagues in the local diplomatic communities, these reports included references to the situation at the front ("Colle di Lana lost, province of Belluno threatened, Italians retreating along a front of 150 kilometers"), Italian losses ("From 24 October to 1 November, 250,000 prisoners and over 2000 pieces of artillery"), the direction of the attack ("On the lower Piave German troops are nearing the Vidor bridge."), and the intentions of Italy's enemies ("Probably the plan is to separate Italy from France in order to aggravate Italy's economic and military situation.").21 Delivered each day to the Foreign Ministry, these intercepts were especially welcome to Sonnino since the Italian High Command deliberately issued misleading or false reports to civilian authorities in order to obscure the scope of the debacle and the incompetence of the general staff.22

21Marchetti to Gasparri, 6 and 11 November 1917, and Pacelli to Gasparri, 7 and 12 November 1917, reel 48, Sonnino Papers.
22Smith, Italy and its Monarchy, 220, 227.

The papal messages, especially those from Berne, Munich, and Vienna, produced a trove of information which permitted Italian intelligence to monitor not only the activities of the Vatican, but also political developments in Austria and Germany. The telegrams of the Munich nuncio, Monsignor Eugenio Pacelli, allowed the Italians to chart the military's growing influence in German politics and to follow the political maneuvers in Berlin that led to the replacement of Theobald von Bethmann-Hollweg as chancellor by Georg Michaelis in July 1917 and the latter's removal in favor of Georg von Hertling in November of that year.23 In November 1918 the reports of the Munich and Vienna nunciatures described the deteriorating economic and political conditions in Germany and Austria and provided early indications of the overthrow of the Wittelsbach dynasty in Bavaria and the creation of a Bavarian socialist republic.24

23For Pacelli's numerous reports on German domestic politics, see reel 48, Sonnino Papers. Monsignor Pacelli capped a distinguished career in the papal diplomatic service by serving as Cardinal Secretary of State for nine years (1930-1939). In 1939 he was elected to the papacy as Pope Pius XII (1939-1958).
24Pacelli to Gasparri and Valfre to Gasparri, 8 November 1918, reel 48 Sonnino Papers.

5 CONCLUSION

Any appraisal of Italian diplomatic cryptanalysis that depends upon the Sonnino Papers must acknowledge the limitations of that source. The decrypts in the collection probably represent only a portion of the diplomatic messages intercepted and decrypted by the Reparto crittografico. Some messages may have been destroyed or returned to the intelligence service, while others may have remained in the Foreign Ministry's files when Sonnino left his post in the summer of 1919. Also, the foreign minister may have received and retained only those messages relevant to his principal diplomatic concerns. A selection process can be inferred from the Vatican decrypts. The majority of these messages originated in the important nunciatures in Munich and Vienna, the heart of enemy territory. There are relatively few telegrams from the nunciatures in Berne, Brussels, Madrid, and Washington, and none at all from papal missions in such capitals as Lima and Buenos Aires. Since it is unlikely that the Reparto crittografico would have been able to read the cryptosystems used by the papal Secretariat of State to communicate with its most important missions but remain unable to penetrate the less sophisticated systems in service at minor papal missions, the absence of decrypts from the Latin American nunciatures probably reflects the interests and priorities of the foreign minister rather than the limitations of the Cryptographic Bureau.25

25The Secretariat of State tried to give its most important missions more secure systems. When, in 1918, it introduced superenciphered codes, the new system was initially reserved for the nunciatures in Munich, Vienna, Berne, and Madrid.

Nonetheless, the intercepts in the Sonnino Papers remain the only primary evidence of Italian diplomatic cryptanalysis during the First World War. This evidence indicates that Italy was a latecomer to wartime signals intelligence and that its codebreakers never equaled the achievements of their counterparts in Britain and France.26 The Reparto crittografico succeeded, at best, only moderately in reading the diplomatic communications of other governments. The retrospective reading of an early Austrian code and the current reading of an American code were probably the highpoints of a cryptanalytic effort whose success was mainly limited to the cryptosystems of a few minor powers.27 Of course, much of the cryptanalytic history of the First World War remains unwritten, but the available evidence suggests that when the full record is revealed, Italy will not stand in the first rank.

26During the war French cryptanalysts solved the diplomatic cryptosystems of Austria-Hungary, Bulgaria, Germany, Greece, Italy, Spain, and Turkey, while British codebreakers read the systems of Austria-Hungary, Denmark, Germany, Greece, Japan, Norway, Spain, Sweden, and the United States. Andrew, "Déchiffrement et Diplomatie," 58-59; A. G. Denniston, "The Government Code and Cypher School Between the Wars," Intelligence and National Security, 1 (January, 1986), 53-54.
27It is possible that the Reparto crittografico cracked British, French or German codes but that no trace of such successes remains in the Sonnino Papers. It is, however, unlikely that Sonnino would save the odd Greek or Swiss intercept but retain not a single intercept from the major belligerents, especially in view of the fact that his papers contain many intelligence reports on Britain, France and Germany drawn from sources other than intercepted communications.

BIOGRAPHICAL SKETCH

David Alvarez is a professor at Saint Mary's College of California. His current research focuses on SIGINT and the European neutrals in World War II.


Faded Lustre: Vatican Cryptography, 1815-1920

David Alvarez

ADDRESS: Department of Government, St. Mary's College, Moraga CA 94575 USA.

ABSTRACT: The Vatican, whose cryptology was once perhaps the finest in the West, lagged behind other states in responding to advances in cryptography in the period 1815-1920. It depended upon nomenclators long after other states had abandoned that system, and it did not begin to encipher its codes until decades after the practice had become common in the foreign ministries of other governments. Only during the First World War did the Vatican seriously attempt to upgrade its diplomatic cryptosystems.

KEYWORDS:Vatican, cryptography, nomenclators, polyalphabetics, enciphered codes

I. ORGANIZATION

Papal cryptography has a long and often distinguished history. The popes were among the first in modern Western civilization to recognize the utility of codes and ciphers. The first modern code, a short list of single letter substitutions for nouns, can be found in the archives of the Vatican. This code dates from the first half of the fourteenth century, a period during which the papal curia (central administration) was already using simple ciphers for some of its correspondence with cardinals and archbishops. In 1379 a papal secretary compiled a set of cryptographic keys that are the oldest extant and that include the first nomenclator, a system that combines a cipher system of substitutes for individual letters with a code list of equivalents for entire words. The post of cipher secretary was created in the curia in 1555, and in the late 1580s the office fell under the control of a remarkable family, the Argentis. As cipher secretaries to six popes, Giovanni Batista Argenti and Matteo Argenti, his nephew and successor, elevated papal cryptography to a dominant position in the late Renaissance. Their accomplishments included the first use of a mnemonic key to mix a cipher alphabet, an early solution of a polyalphabetic cipher, and the preparation of a manual of cryptography that represented the state of the art at the time.1

1David Kahn, The Codebreakers (New York, 1967), 106-107, 113; Aloys Meister, Die Geheimschrift im Dienste der Päpstlichen Kurie (Paderborn, 1906).

At the beginning of the nineteenth century, papal cryptography was the responsibility of the Secretariat of Ciphers, a distinct department in the papal administration. Its staff consisted of three cifristi (cipher clerks) under the direction of the cipher secretary. The office worked exclusively for the Secretariat of State, the pope's foreign ministry. In 1814, a reorganization abolished the Secretariat of Ciphers and the position of cipher secretary and moved the cifristi into the Secretariat of State, where they became a separate class of officials responsible for ciphering and deciphering dispatches, maintaining secure custody of the codes and ciphers, and creating new cryptographic systems. The reorganization also established the new post of substitute secretary of state and cipher secretary. As deputy to the cardinal secretary of state, the substitute was responsible for the day-to-day operation of the department, including the supervision of cryptography. From 1814 to 1840, the total staff of the Secretariat of State, which for most of the period supervised both the external and internal affairs of the Papal States, consisted of the cardinal secretary of state, the substitute, four or five minutanti (chief clerks), three cifristi, four or five scrittori (copyists), and two archivisti (archivists).2

2For the organizational history of the Secretariat of State in this period, see Lajos Pásztor, La Segreteria di Stato e il suo Archivio, 1814-1833 (Stuttgart, 1984).

Throughout this period, papal cryptography was closely linked with another family, the Evangelisti. Giuseppe Evangelisti entered the Secretariat of State in 1793 as a copyist. His father and uncle had served as cipher clerks in the old Secretariat of Ciphers in the pontificates of Clement XIV (1769-1774) and Pius VI (1775-1799). In 1801 Giuseppe continued the family tradition by joining the cifristi. He accompanied Cardinal Ercole Consalvi to the Congress of Vienna (1814-1815), where he was responsible for the cardinal's voluminous secret correspondence. At his death in 1847, he had been the Secretariat's senior cipher officer for four decades. Filippo Evangelisti, Giuseppe's younger brother, followed the expected path by joining the cifristi in 1806. A graduate of the elite Collegio Romano, where he won several academic prizes, Filippo was included in the small suite of officials who followed Pius VII to Genoa in March 1815 when the pontiff, reacting to Napoleon's escape from Elba and the invasion of the Papal States by King Murat of Naples, sought refuge in northern Italy. Filippo Evangelisti served in the cipher section of the Secretariat until 1840.

Two other individuals worked in papal cryptography during this period. Vincenzo Scagliosi, of whom little is known, served in the old Secretariat of Ciphers from 1804 until 1814, when that department was incorporated into the Secretariat of State and he transferred into the ranks of the minutanti. His successor, Giuseppe Golt, remained in the cipher section until his retirement in 1847.3

3Ibid., 78, and chapter 5.


II. SYSTEMS, 1815-1870

In the first decades of the nineteenth century, the Vatican had in service a variety of codes and ciphers. Francesco Capaccini, for instance, used several systems while serving as nuncio (papal ambassador) in the Low Countries between 1828 and 1831. In the simplest procedure, Capaccini would compose a dispatch in which the paragraphs were numbered consecutively. In the even-numbered paragraphs there was no attempt at subterfuge; the sentences meant what they said. In the odd- numbered sections, however, the true meaning of a sentence was the opposite of that written. The statement "our position is strong" was meant to be understood as "our position is weak." Such a message was sent in clear. Monsignor Capaccini also used at least two jargon codes which substituted codewords for the names of certain individuals or places in an otherwise open dispatch. In one such code, which had more than 70 equivalents and also included a small cipher, Rome = AUNT, France = MR. NANTINI, Spain = MR. CARLI, Amsterdam = SAN ORESTE, and Prince of Orange = MR. FREY. A third system consisted of two-, three-, and four-digit numerical groups. Nouns could be represented by any of 10 sequential numbers, and many had, in addition, a corresponding codeword. Austria, for example, could be represented by any group in the sequence 090…099, or by the codeword MOROCCO. The Pope was 1240…1249 or MERCURIO. Capaccini also used a null code in which the first words of certain lines in an apparently innocuous letter would, if separated from the text and combined in the correct order, form a message. The reader could identify the relevant words in the appropriate lines because the last letter of one was connected to the first letter of the next without the pen having been lifted from the page. The correspondents agreed ahead of time on the order in which the words would be read.4

4Vatican City. Archivio Segreto Vaticano (ASV). Segreteria di Stato, Spogli: F. Capaccini, busta 3, fasc. A.

Capaccini's cryptographic repertory was probably atypical among papal diplomats, since he was an enthusiast who delighted in creating methods of secret communication. In the first half of the nineteenth century the papacy's basic system for secret communication was the nomenclator, that system, half code and half cipher, which had dominated western cryptography since the fifteenth century. The Argentis had discussed the use of nomenclators in their famous cipher manual, and in an institution as tradition-bound as the Vatican, the Evangelisti were not inclined to challenge the authority of their illustrious predecessors.

The system prepared in 1856 for Monsignor Alessandro Bianchi, the last nuncio to the Grandduchy of Tuscany before its absorption into the Kingdom of Italy, is representative of the papal nomenclators of the period.5 It was a modified two-part system with three-digit numerical equiv.alents (from 242 to 989, skipping any group with a 0, 1, or 3) for the letters of the alphabet and almost 200 common bigrams and trigrams. Four-digit groups (3008 to 3989) served for nouns, proper names, parts of speech, and commonly used prefixes and suffixes. For punctuation there were three-digit groups each beginning and ending in 1 (comma = 121, period = 161). In the alphabet, each vowel was represented by three homophones, as were the consonants n, r, and t. The other consonants, excepfor h, l, the combination qu, and the (in Italian) rarely used j, w, x, and y, each had two homophones. In Italian, a preposition will have different endings depending upon the gender and number of its object. In Franchi's nomenclator, the compiler has given each variation its own group; thus, the preposition da (by, from) is represented by the group 857, while its variants dagli, dai, dal, dalla, and dalle are represented, respectively, by the groups 542, 986, 758, 752, and 928.

55ASV. Segreteria di Stato, Spogli: A. Franchi, v. 1.

A section labeled "Precautions" provided directions on the use of the system.

The dispatch, or the portion of it written in code, should be entirely in numbers, without paragraph divisions. It should not be divided by commas, periods, and accent marks, and even less by [unenciphered] words, even if they are insignificant. All must be done with numbers, which should be clear, intelligible, and always placed at an equal distance from each other.

The numbers 1 and 0 are nulls, except a 0 which has value as part of a four-digit group that always begins with the number 3, a number that appears nowhere else.

A 1, separated from another 1 by a single number, serves as punctuation, and, by itself, serves to distinguish one word from another to avoid confusion in deciphering. For this reason do not omit to use it. The 0 can also be used for the purpose of dividing words, except in cases where 0 is used at the end of a four-digit group. This will facilitate decoding.


In the first half of the nineteenth century, the Vatican also made occasional use of ciphers. In the 1840s the nunciature in Madrid had in service three monoalphabetic substitutions. The first system substituted ordered pairs of numbers (a = 1/2, b = 1/3, c = 1/4, …) for the letters of the alphabet. In the second system the letters were replaced by two-digit numerical groups in ascending order (a = 11, b = 12, c = 13, …). In the third system the letters in the first half of the alphabet were replaced by those in the last half in reverse order (a = Z, b = Y, c = X, …), and the remaining letters by numbers in descending order (n = 12, O = 11, p = 10, … ). The latter system occasionally substituted symbols for certain proper nouns (France = F, Pope = ∞). The three ciphers used symbols for punctuation (comma = [image], period = √) and spelled-out all numbers. Apparently, there were neither homophones nor nulls. Provincial delegates (governors) who administered various cities and territories within the Papal States, that broad swath of territory curving across central Italy from Rome in the southwest to Ravenna in the northeast, also used simple ciphers to protect their telegraphic communications with Rome. In the spring of 1860, the delegate in the Adriatic port of Ancona had a system that substituted two-digit numerical elements for letters of the alphabet (a = 28, b = 69) and common Italian bigrams and trigrams (al = 58, co = 49, pre = 64). In the same year, the governor of Pesaro depended upon a cipher that merely mixed the alphabet in a system of reciprocal substitution (a = U, u = A, b = H, h = B).6

6The ciphers of the Madrid nuncio and the Ancona and Pesaro delegates have been partially reconstructed by the author from their dispatches. For Madrid, see ASV. Segreteria di Stato, Esteri, 1847, rubrica 11, fogli 43-46. For the delegates, see ASV. Segreteria di Stato, 1860, rubrica 165, fasc. 2-3.

Given the relative simplicity of these ciphers, it is rather surprising to discover that the Vatican seriously experimented with systems of polyalphabetic substitution. Such a system had been conceived in the fifteenth century by the great Renaissance polymath Leon Battista Alberti, but it had never caught on in the government chancelleries and embassies, where the professional cryptographers much preferred the nomenclator. While recognizing that a polyalphabetic cipher avoided that fixed replacement of a letter with the same cipher symbol throughout a message which is the particular weakness of monoalphabetic systems, the professionals felt that the more complex system required too much time and attention. The frequently changing alphabets could too easily confuse a code clerk for whom the traditional nomenclator seemed more straightforward and less difficult to use.7

7On the development of polyalphabetic substitution, see Kahn, The Codebreakers, 125ff.

Papal cryptographers were no strangers to polyalphabetic ciphers. Such ciphers were known to and used by the Argentis in the sixteenth century. Matteo Argenti considered them "the noblest and the greatest [ciphers] in the world" and discussed them in his famous manual. While reluctant to adopt polyalphabetics for general service, the Argentis occasionally prepared one for the use of a particular papal official.8 Subsequently, the papacy, like other European governments, came to depend upon the nomenclator to protect its secret communications, but the memory of polyalphabetics apparently survived.

8Ibid., 150-151.

The private papers of Francesco Capaccini, a career diplomat whose service between 1815 and 1844 included senior posts in the Secretariat of State, as well as ambassadorial assignments in Holland and Portugal, include two polyalphabetic systems.9 In the first, the letters of the alphabet, in column, are numbered from 1 to 26 and, again, from 27 to 52. A central column lists the alphabet again, this time with each letter associated with a key word (a = PARAVAMI). To encipher the phrase, Il Re di Marocco (The King of Morocco), the cipher clerk would write below it a keyword, e.g. PEPELAGRO. Referring to the first column, he would add the numerical value of each plaintext letter to the numerical value of the keyword letter below it. The sum would then be converted, again by reference to the first column, to a letter, creating the ciphertext YQHJPJTSGEHST. The clerk would place at the front of the ciphertext the letter identifying the keyword.

9ASV. Segreteria di Stato, Spogli: F. Capaccini, busta 3, fasc. A.
plain | i | l | r | e | d | i | m | a | r | o | c | c | o
+ | 9 | 12 | 18 | 5 | 4 | 9 | 13 | 1 | 18 | 15 | 3 | 3 | 15
key | P | E | P | E | L | A | G | R | O | P | E | P | E
 | 16 | 5 | 16 | 5 | 12 | 1 | 7 | 18 | 15 | 16 | 5 | 16 | 5
cipher | 25 | 17 | 34 | 10 | 16 | 10 | 20 | 19 | 33 | 31 | 8 | 19 | 20
 | y | q | h | j | p | j | t | s | g | e | h | s | t


To decipher a message like QEROMGSB, the process would be reversed. The first letter of the ciphertext would indicate which keyword was in use. This keyword would be written under the ciphertext. The numerical value of each letter in the keyword would be subtracted from that of the cipher letter above it, and the result converted into a plaintext letter to reveal the words i Masoni (the Masons).

indicator Q = VENTRES
cipher | e | r | o | m | g | s | b
- | 31 | 18 | 15 | 39 | 33 | 19 | 28
key | V | E | N | T | R | E | S
 | 22 | 5 | 14 | 20 | 18 | 5 | 19
clear | 9 | 13 | 1 | 19 | 15 | 14 | 9
 | i | m | a | s | o | n | i

The second polyalphabetic in the Capaccini papers consists of two tables, one for enciphering and one for deciphering. The former has 26 columns, one for each letter of the alphabet. Within each column, the letters of the alphabet are listed in order with each letter associated with a number from 50 to 78. These numbers are in mixed order, and the numbers 55, 66, and 77 are not represented. Along the bottom of the table are 52 key phrases, in groups of four, each of the first 26 identified by a letter of the alphabet, and each of the remainder by a number between 10 and 38 (skipping 11, 22, and 33). To encipher "abbey" with the keyword CABA, the writer would copy the plaintext under the key, search down the column represented by the first letter of the keyword to find the first letter of the plaintext, and then enter the corresponding number as the first element of the ciphertext. He would proceed in the same way with the second letter and so on to the end of the message.

A | B | C |  | Y | Z
a+a = 63 | b+a = 62 | c+a = 74 | … | y+a = 51 | z+a = 68
a+b = 62 | b+b = 65 | c+b = 53 | … | y+b = 60 | z+b = 67
a+c = 74 | b+c = 53 | c+c = 64 | … | y+c = 70 | z+c = 62
a+d = 69 | b+d = 52 | c+d = 73 | … | y+d = 71 | z+d = 50
a+e = 78 | b+e = 56 | c+e = 51 | … | y+e = 63 | z+e = 73
[vellip] | [vellip] | [vellip] | [vellip] | [vellip] |
a+y = 51 | b+y = 60 | c+y = 70 | [vellip] | y+y = 74 | z+y = 56
a+z = 68 | b+z = 67 | c+z = 62 | [vellip] | y+z = 56 | z+z = 57

key | C | A | B | A | C
plain | a | b | b | e | y
cipher | 74 | 62 | 65 | 78 | 70

To indicate which keyphrase he had selected from the 52 available at the bottom of the table, the writer would place at the head of the ciphertext the letter or number that identified the keyword. Two-digit groups served to denote punctuation (period = 11, comma = 22) and to indicate a change in the key (88).

The presence of these ciphers in the papers of an important papal diplomat is intriguing in view of established opinion that in the early nineteenth century governments avoided polyalphabetics and depended almost exclusively upon nomenclators for their diplomatic correspondence This opinion is, at first glance, reinforced by the fact that Capaccini, distrusting the official cryptographic systems of the Secretariat of State, often created codes and ciphers for his private use. Were the papal polyalphabetic ciphers aberrations concocted by Capaccini for his personal amusement or private use? Circumstantial evidence suggests that the polyalphabetics may have seen more general service in the Secretariat of State.

The first of the two polyalphabetics is handwritten on a single sheet of paper, and bears the notation, Questa chiave Mgr. Leardi ("Monsignor Leardi's key"). This is a reference to Paolo Leardi who was the papal nuncio in Vienna from 1816 to 1823, and it indicates that this polyalphabetic cipher was in service at the Vienna nunciature during his tenure. Indeed, as late as 1825 the nunciature was still using this cipher to communicate with the Secretariat of State.10

10See the correspondence in ASV. Segreteria di Stato, 1823-1825, rubrica 247.


The second of the polyalphabetics is even more suggestive. It is printed on long sheets of paper. The keywords along the bottom have been written by hand next to the printed letters and numbers which serve as indicators. The Capaccini papers contain several "blank" tables; that is, tables where the keywords have not been entered at the bottom. This indicates that the tables had been prepared ahead of time for future use, and that someone anticipated using the tables with sufficient frequency to justify printing multiple copies. There are two "complete" tables; that is tables in which the keywords have been filled in by hand. A marginal notation on one table indicates that it was prepared for Antonio Tosti who was the pope's diplomatic representative in Turin from 1822 to 1828. In the second complete table, the keywords differ from the first. What is interesting about this second table is the fact that one cluster of four keywords has the notation Parigi (Paris), another cluster of four has Lisbona (Lisbon), another has Vienna, while two clusters are marked Comune (common). It was not unusual for the Secretariat of State to issue two ciphers to a nuncio; one, the cifra particolare (private cipher), was for confidential communication with the Vatican, the other, the cifra comune (common cipher), was for communication between nuncios. One can conclude that the Secretariat of State was using the polyalphabetic cipher to communicate with papal diplomatic missions in Austria, France, and Portugal, and that the same cipher was in use among the nuncios in those capitals. In 1831 Capaccini used this system to report to the Vatican from his post as nuncio in the Low Countries.

The presence of these polyalphabetic ciphers in the personal papers of Monsignor Capaccini lends additional credence to the argument that such systems were, at least for a time, in general service in the papal Secretariat of State. From 1815 to 1824, Capaccini was the minutante (chief clerk) responsible for drafting the Secretariat's correspondence with the nuncios in foreign capitals. Apparently the polyalphabetics were used for the most confidential messages, and only Capaccini and Luigi Armellini, his successor as minutante for diplomatic affairs, were allowed to encipher and decipher dispatches in these systems. Reports in more routine systems, such as nomenclators, were left to the personnel of the cipher section.11

11Pásztor, La Segreteria di Stato, 207-208.

It is even likely that Capaccini designed the polyalphabetic ciphers, for in June 1829 the amateur cryptographer, then on a diplomatic mission to Holland, recalled an interesting encounter for Cardinal Albani.

Since our old ciphers were known to everyone, when I was in the Secretariat of State I composed, with much effort, a new one which is the one I now use. Pope Leo XII laughed at me, saying that no cipher is indecipherable, but when he saw that a professional cryptologist, who was then in Rome, could not break mine and had to give up, he authorized me to use it for correspondence. This cipher, however, is very difficult and requires much time to cipher, and perhaps even more to decipher, as Your Eminence will have already discovered …12

12J. P. de Valk (ed.), Lettres de Francesco Capaccini (Brussels, 1983), 256. Leo XII was pope from September 1823 to February 1829.

The cryptographic systems of the papacy were clearly of uneven quality. The polyalphabetic ciphers (to the extent that they were used) certainly represented the best effort of the Secretariat of State. At a time when other governments depended upon the familiar and more easily managed nomenclator for their diplomatic communications (even though they considered the polyalphabetics unbreakable), these papal ciphers were probably as good as any system then in use in Europe.13 As for the Vatican's nomenclators, the one in service at the nunciature in Florence in 1856 (supra) was probably indicative of the best in the repertory of the Secretariat. Although rather short (fewer than 900 plaintext elements in a period when nomenclators of 3,000 elements were not unknown), it was a respectable system. It was as good as some of those used by France in the first half of the century, and in its use of homophones and nulls was superior to a contemporary American system. On the other hand, it was less sophisticated than a Spanish system which, in addition to homophones and nulls, utilized such devices as special groups to indicate a doubled letter or to cancel a preceding group.14 Unfortunately, the nomenclator of the nuncio in Florence was an anachronism on the day it entered service since, in response to the expansion of telegraphic communications, European foreign offices at mid-century were rushing to replace their nomenclators with full-fledged codebooks.15 The flimsy substitution ciphers used by papal governors and, at least for a time, the nuncio in Madrid offered scant security and would not have withstood even the most cursory professional scrutiny. Of course, in the case of the governors, the danger of interception was practically nil since their telegrams and dispatches moved along internal lines. Their schoolboy ciphers probably sufficed to protect a message from the casual curiosity of a clerk, messenger, or cable operator.

13A general solution for polyalphabetic ciphers was not available until 1863. Kahn, The Codebreakers, 191.
14Ralph Weber, United States Diplomatic Codes and Ciphers, 1775-1938 (Chicago, 1979), 478-489. The French and Spanish systems are among the papers of Britain's Decyphering Branch in the British Library (London), Add. MSS. 32265 and 32301.
15On this process, which was substantially completed by the 1860s, see Kahn The Codebreakers, 190.


III. DECLINE

Whatever their merits, the cryptographic systems of the Secretariat of State engendered little confidence among papal officials. The Vatican's special envoy to Paris in 1801 did not trust his ciphers. At the Congress of Vienna, the pope's representative, Cardinal Consalvi, correctly surmised that his nomenclator had been compromised by the Austrian intelligence service. Fourteen years later, Francesco Capaccini complained that the Vatican's ciphers (except, of course, those of his own devising) were open to everyone, and Pope Leo XII, who before his election to the Throne of St. Peter had led diplomatic missions to France, Germany, and Switzerland, expressed his conviction that no cryptographic system was secure. As late as 1859, the nuncio in Vienna warned the Secretariat of State that his cipher was insecure.16

16Boulay de la Meurthe (ed.), Documents sur la Négociation du Concordat (Paris, 1895), xxiii; Alessandro Roveri (ed.), La Missione Consalvi e il Congresso di Vienna (Rome, 1970), v. 3, 545, 567-68; Carla Meneguzzi Rostagni (ed.), Il Carteggio Antonelli-DeLuca, 1859-1861 (Rome, 1983), 61.

Several factors, aside from the inherent weaknesses of some of the systems, undermined the integrity of the Vatican's ciphers. Cryptographic materials were a natural target for the foreign agents who occasionally insinuated themselves into the households of nuncios.17 Official dispatches might easily be intercepted and copied in transit, thereby providing foreign cryptanalysts an opportunity to study and attack papal ciphers.18 This threat was not diminished by the poor communications practices of papal diplomats. To save time and effort, nuncios usually enciphered only those portions of a dispatch which they considered the most sensitive. Appraisals of sensitivity varied, naturally, from individual to individual. Often a nuncio would encipher only the names of people and places; sometimes only one or two words in an entire message might be enciphered. Such practices (especially the latter) immeasurably aided the efforts of anyone seeking to break the cipher. With most of the message in clear, the cryptanalyst had a context of subject matter, time, and locale in which to place the unknown text.

17During the Congress of Vienna the Austrian secret police obtained documents from an agent inside the papal nunciature. One nuncio in Madrid discovered that his valet was reading his confidential correspondence. A confidant of Francesco Capaccini during the latter's mission to the Low Countries may well have been an agent of the local Neapolitan ambassador.
18Lacking a regular courier service of its own, the Secretariat of State often depended upon the diplomatic couriers of Austria, France and Spain. The Austrians were especially skillful at opening dispatches entrusted to their care.

Communications security was further compromised by the low priority of cryptography in the Secretariat of State. This situation reflected a decline in the use of codes and ciphers. In the first half of the nineteenth century, the cryptographic requirements of the Vatican were few, largely because of the modest position of the papacy in world affairs. In 1830, for example, the papacy accredited diplomatic missions to only 11 countries. By midcentury the number had increased slightly to 13. These missions generated relatively few secret communications. The record of the nunciature in Madrid, one of the more important posts in the Vatican's diplomatic network, is revealing. In 1828 the Secretariat of State sent only 12 enciphered dispatches to Madrid, while the nunciature sent no enciphered messages at all to Rome. In 1848 the Secretariat forwarded only three enciphered dispatches to Madrid, which responded with seven of its own. Months might pass without secret correspondence between the Vatican and its nuncio in Spain.19

19Vicente Carcel Orti (ed.), Correspondencia Diplomatica del Nuncio Tiberi, 1827-1834 (Pamplona, 1976), xvii and passim; idem, "Los Despachos de la Nunciatura de Madrid, 1847-1857," Archivum Historiae Pontificiae, 13 (1975), 311-400. The totals for enciphered messages, low as they are, still exaggerate the use of ciphers by the Madrid nunciature since only portions (at times only a few words) of a dispatch would be enciphered.

The volume of enciphered correspondence that passed between the Vatican and its representatives abroad had diminished to such an extent that there was insufficient work for the cifristi.20 The senior officer, Giuseppe Evangelisti, was usually able to handle most of the enciphering and deciphering and still have time for other tasks around the Secretariat. After 1815, Filippo Evangelisti worked less and less with ciphers as he assumed duties in the registry of the Secretariat. The third member of the section, Giuseppe Golt, was not concerned with secret communications at all. He performed archival work and served as secretary to Monsignor Carlo Mauri, who held the post of substitute for 16 years. Upon Mauri's death in 1830, Golt assumed responsibility for various legal matters of interest to the Secretariat, although he remained nominally attached to the cipher section.

20Lajos Pásztor, "La Segreteria di Stato di Gregorio XVI, 1833-1846" Archivum Historiae Pontificiae, 15 (1977), 322.


Papal cryptography was rather neglected in the last decades of the Papal States. For some time, perhaps more than a century, the cipher section had been coasting, propelled (ever less forcefully) by the memories of past accomplishments. By the middle of the nineteenth century it had coasted almost to a stop. The Secretariat of State and its cipher section were not immune to that malaise marked by inertia, lassitude, and a "qualitative loss of energy" that pervaded the papal administration in the 18th and 19th centuries, and that made the Roman bureaucracy a byword in Europe for inefficiency and incompetence.21 Having obtained their posts through family connections, favoritism, or bribery, papal administrators in general had little training or even interest in their duties. They performed their assigned tasks with varying degrees of industry and competence, but they resisted any hint of reform or innovation which threatened traditional routines and prerogatives. Describing the papal bureaucracy, one Austrian representative concluded, "In general, they are individuals with little or no knowledge of the duties entrusted to them," while a second agent bemoaned the "total lack of all system and method."22 In the Secretariat the cifristi performed the mechanical tasks of enciphering and deciphering the few dispatches which required their attention, but gave little systematic thought to communications security. There were certainly signs of laxness in cryptographic matters. In one dangerous practice, nuncios were allowed to invent their own codes and ciphers. Francesco Capaccini, alone, used at least 10 different systems during his three years in The Hague. Often a nuncio used more than one system in a single dispatch; part of the text would be written in a numerical cipher, part in a letter cipher, and part in a jargon code. Officials in the Secretariat would sometimes despair of their ability to make any sense of a particular dispatch.

21Hanns Gross, Rome in the Age of the Enlightenment: The Post-Tridentine Syndrome and the Ancien Regime (Cambridge, 1990), 10-11.
22Alan Reinerman, Austria and the Papacy in the Age of Metternich, v. 2 (Washington, DC, 1989), 182, 187, 193; Carte segrete e atti ufficiali della Polizia austriaca in Italia (no editor), v. 1 (Capolago, 1851), 382.

Inertia led the cipher section to adopt a conservative (not to say antiquarian) approach to cryptography. Furthermore, the relatively infrequent use of ciphers provided little incentive to improve established practices, especially in an institution as tradition-bound as the Roman Curia. The result was a tendency to continue tried- and-true procedures and to eschew innovation. The experiment with polyalphabetic ciphers in the late 1820s is the exception that confirms the rule. The temporary adoption of polyalphabetic systems reflected the personal enthusiasm of the senior minutante, Francesco Capaccini, rather than the considered judgment of the cipher section. Polyalphabetics were notoriously laborious and prone to errors in ciphering and deciphering as correspondents tried to keep track of the constant shift in alphabets. There is evidence that Capaccini's systems caused some confusion in the Secretariat of State. The margins of dispatches bear such notations as "Because of a mistake in enciphering it isn't possible to read the name of the person indicated" or "Armellini gave up trying to decipher this dispatch." Several lines of ciphertext might remain unintelligible, a sure sign of confusion over the proper key.23 It is easy to imagine officials cursing these new systems as a needless complication and urging a return to the familiar systems which had served well in the past. With the exception of some of Monsignor Capaccini's creations (which apparently did not survive his departure from the Secretariat of State), the cryptographic systems in service at the Vatican in the early nineteenth century differed little from those in service 300 years earlier. The simple substitution cipher in which a nuncio to Spain composed some of his reports in the 1840s (a = 11, b = 12, c = 13, d = 14 …) was similar to a cipher used by a nuncio to France in the 1570s (a = 1, b = 3, c = 4, d = 6 … ), with the latter, because of its broken sequence of numerical equivalents, revealing a marginally greater imagination. Indeed, in an interesting case of historical regression, the substitution ciphers of the Madrid nuncio were greatly inferior to the cryptographic systems prepared for his predecessors by Giovanni Battista Argenti in the late sixteenth century.24

23de Valk, Lettres de Francesco Capaccini, xxvi, 100, 179.
24Pierre Hurtubise, Correspondance du nonce en France, Antonio Maria Salviati, 1572-1578, v. 1 (Rome, 1975), viii; Meister, Die Geheimschrift im Dienste der Päpstlichen Kurie, 355-357, 405-407.

The lethargy of the cipher section might have been overcome by strong leadership and direction from the senior officers in the Secretariat of State, but none was forthcoming. These officers had no interest in cryptography. Ercole Consalvi, the great cardinal secretary of state who skillfully navigated the papacy through the shoals of the post-Napoleonic years, was sensitive to the problems of communications security, but thought that trustworthy messengers, cover addresses and clandestine mail routes provided greater security than ciphers. Consalvi retired in 1823, and his successors as cardinal secretary of state exhibited even less interest in secret communications. Cardinal Giulio Della Somaglia was an octogenarian when he assumed direction of the Secretariat and had little energy for even the most routine tasks of his office. Cardinal Tommaso Bernetti was a skillful diplomatist, but he paid scant attention to the administration of his department. One Secretary of State, Cardinal Giuseppe Albani, actually saw no need for ciphers. Discussing the office of cipher secretary, Albani argued,

Today this title and responsibility of Cipher Secretary is totally useless. At other times there was a separate cipher department, because the Secretariat of State conducted all of its affairs with foreign courts through its nuncios, and it was necessary to make use of the cipher in correspondence, but now that the number of foreign ambassadors to the Holy See has increased and all prefer to conduct affairs here in Rome, the cipher is used little or not at all …25

25Pásztor, La Segreteria di Stato, 208.

Unfortunately, indifference at the top was not balanced by interest at the next level of the Secretariat. Ideally, the substitute, who after 1814 also held the title of cipher secretary, should have manifested such interest since he was administratively responsible for all cryptographic matters. In practice, however, this responsibility was purely nominal. With one exception the substitute/cipher secretaries during this period knew nothing of codes and ciphers, and without exception they relegated cryptography to a priority far below more pressing problems of departmental finance, personnel, and general administration.


Monsignor Carlo Mauri, the first official to combine the offices of substitute and cipher secretary, had little understanding of communications security. During the Congress of Vienna, his naive faith in the inviolability of papal correspondence drove Cardinal Consalvi to distraction. Mauri would send letters directly to Consalvi rather than to the pre-arranged cover address in Vienna, believing that the cardinal's nomenclator would protect the confidential messages from prying eyes. The cardinal secretary of state had a higher opinion of Austrian cryptanalysis, but his repeated warnings were of no avail. In a letter to his friend Cardinal Pacca, Consalvi exploded, "I confess that when reading the cipher messages that are sent to me directly through the mail, I would like to slam the head of monsignor Substitute against the wall."26 Mauri preferred to devote his attention to various legal questions of interest to the Secretariat of State, and it was on such matters that he spent most of his time until he died in office in 1830.

26Roveri, La Missione Consalvi, 568.

His successor as substitute and cipher secretary, Francesco Capaccini, seemed prepared by appetite and experience to assume the responsibilities of both offices. Unlike most of his colleagues who came to the Secretariat of State with a background in theology or law, Capaccini had studied mathematics and astronomy, and had been teaching the latter science at the University of Naples when he was recruited into the Secretariat as a minutante in 1815. He was a man of many gifts, and when he temporarily left the Secretariat in 1824 the Prussian ambassador commented that the department had lost the only official who understood diplomatic affairs. He might have added that it had lost the only official who understood cryptography. Capaccini had a keen interest in codes and ciphers, and he considered the Vatican's cryptographic systems hopelessly outdated or compromised. It was almost certainly because of his influence that the Secretariat of State, at least for a time, adopted polyalphabetic ciphers for its most confidential messages.

Capaccini came to his high post from a stint as nuncio to the Low Countries where he exhibited an obsessive concern with the security of his communications. Surprisingly, this concern abated when he returned to Rome in 1831 to take up his duties as substitute/cipher secretary. He turned his energies to administrative matters, in particular plans to reorganize the Secretariat of State. When he thought of the cipher section, it was only to consider ways to reduce its presence in the department. This rather unexpected attitude in one previously so sensitive to the importance of cryptography was almost certainly a reaction to financial exigencies which were pressing upon the Secretariat. By 1831 the financial affairs of the papacy had reached such a dismal state that the Prussian minister in Rome informed his government that "bankruptcy is at the door."27 Revolutionary disturbances in the pope's northern provinces had required extraordinary expenditures while impeding the collection of taxes in the region. This condition, combined with "normal" maladministration in the pontifical Treasury, created a situation where there were doubts that the pope could pay the salaries of his officials. In this climate Capaccini, who as substitute was the chief administrative officer of the Secretariat, concluded that the cryptographic needs of the department had declined to such an extent that a cipher section was an extravagance. In fact, there was no longer enough work for even a full-time cipher clerk. As early as 1823, Capaccini, while still a minutante, had advised the cardinal secretary of state that

27Reinerman, Austria and the Papacy in the Age of Metternich, 178, 374(n.1).

the two most senior and most faithful copyists would be able to learn in half a day the very simple art of ciphering and deciphering, so that when there is an occasion to write in cipher or to decipher, it could be done as quickly as now, but without the expense of over 100 scudi a month for the Cipher Section.28

28Pásztor, La Segreteria di Stato, v. 2, 378.

As substitute, Capaccini continued to argue for the abolition of the Cipher Section. Shortly after his appointment to the second-ranking position in the Secretariat, he proposed the elimination of the cifristi and the transfer of their duties to one of the officials responsible for the correspondence with the nuncios. Capaccini's proposal was not formally adopted, but its purposes were achieved in time. In 1840 the cifristi were reduced to two when Filippo Evangelisti retired. On 1 March 1847 Giuseppe Evangelisti died after forty-six years of service as a cipher officer for the papacy. That same day Giuseppe Golt retired, and with his departure the cifristi effectively disappeared as a separate group of officials in the Secretariat of State. The post of prima cifrista, which had been in the Evangelisti family for over half a century, was abolished, and responsibility for cryptography passed to a junior official in the Secretariat.29

29As late as 1848 the roster of the Secretariat of State still included two cifristi, but one performed only archival duties, and his title was merely titular. By the 1850s, a certain Monsignor Emidio Gentilucci was responsible for codes and ciphers. Lajos Pásztor, "La Segreteria di Stato di Pio IX durante il triennio 1848-1850," Annali della Fondazione italiana per la storia amministrativa, 3 (1966), 308-365.

Between 1841 and 1844, Francesco Cappacini was absent from Rome on special diplomatic missions to Holland and Portugal. Although he retained his office of substitute and cipher secretary, he clearly could not fulfill its responsibilities from The Hague or Lisbon, so that the Secretariat was again without the services of the one man who had both an appetite and aptitude for cryptography. In 1844 he was made a cardinal and transferred to the largely honorific post of Auditor General of the Apostolic Chamber. He died in 1845. His immediate successors in the Secretariat of State were more or less competent administrators, but they demonstrated little appreciation of the requirements of secret communications. The experience of one substitute, who in the midst of the crisis surrounding the Roman revolution of November 1848 could not determine which cipher was in use at the Brussels nunciature, suggests how far affairs had been allowed to slip.30

30Michele Fatica, Le Relazioni diplomatiche fra lo Stato Pontificio e la Francia (Rome, 1971), 389.


IV. CONSOLIDATION

On 7 September 1870, after a brief skirmish with papal troops at the northern gates to the city, an Italian army occupied Rome and brought to a close a thousand years of papal temporal power in central Italy. The incorporation of the Eternal City into the Kingdom of Italy concluded a decades-long effort by the Piedmontese House of Savoy to unify the various duchies and principalities of the peninsula under a single crown. The new rulers of Italy had no wish to undercut the pope's position as an independent spiritual leader, and they promised to guarantee his sovereignty over the Vatican complex of palaces and gardens and his right to enter into diplomatic relations with other states. But, inconsolable over the loss of his territories and convinced that a government that granted rights could repeal them, Pope Pius IX refused any accomodation with the new regime. The uncompromising pontiff registered his protest by withdrawing into the vastness of his palace as the self-styled "prisoner of the Vatican," and refusing any contact with the "usurpers."

The loss of the temporal power wrought a sea change in papal diplomacy. For centuries the papacy had been a significant presence in European affairs, but after the occupation of Rome this presence was much reduced. With the pope a self-proclaimed prisoner in his palace, and his Secretariat of State reduced to hurling a string of protests against the "usurpation" by the House of Savoy, the Vatican quickly slipped to the margins of international affairs. Its diplomatic establishment contracted accordingly. By 1880 the entire staff of the Secretariat of State, from the cardinal secretary to the humblest underarchivist, numbered only 11 individuals supervising a diplomatic network shrunk to eight nunciatures. This establishment remained fairly stable through the pontificate of Pope Leo XIII (1878-1903) despite that pontiff's aggressive efforts to project a more visible and important role for the Holy See in world affairs. In 1904, when Leo's successor, Pius X, had been on the papal throne for a year, the strength of the Secretariat declined to 10 officials, although the number of nunciatures had increased by one to nine.

In 1908 Pius X reorganized his diplomatic machinery by incorporating into the Secretariat of State certain hitherto independent offices whose responsibilities partook of a diplomatic character. While the cardinal secretary of state remained the pope's foreign minister, the reorganization divided his department into three sections. The First Section (Extraordinary Affairs) was responsible for issues arising from the Vatican's relations with foreign governments, especially issues relating to concordats (treaties). This section was supervised of a secretary who was assisted by an undersecretary. The Second Section (Ordinary Affairs) handled the routine administrative tasks of the Secretariat, such as accounts, personnel and communications. The substitute, who retained the nominal title of cipher secretary, now directed only this section. The Third Section (Apostolic Briefs) prepared papal messages of a formal or ceremonial nature. Its director had the title of chancellor. This enlarged Secretariat still numbered scarcely 16 or 17 individuals, and the level of activity remained modest. In 1910 the undersecretary for extraordinary affairs, in a letter to the cardinal secretary of state asking to be relieved of his position, complained that though he diligently appeared in his office each day, "nine times out of ten, there was nothing to do."31

31Benigni to Merry del Val, 21 June 1910, ASV. Segreteria di Stato, Spogli: U. Benigni, busta 31.

The cryptographic requirements of the Secretariat of State changed significantly in the period following the loss of the temporal power. The volume of encoded traffic declined with the number of nunciatures and with the more modest diplomatic agenda of the Holy See, which required fewer secret communications. Even after the turn of the century an entire year might pass without an exchange of coded messages between the Vatican and one of its less important diplomatic outposts.32 As mentioned, the Secretariat of State, in the 1850s, had abandoned any effort to maintain a distinct cipher office. After the surrender of Rome, the Secretariat was content to continue the practice of assigning responsibility for cryptographic matters to one of the department's junior officers under the general supervision of the substitute secretary of state, who also held the purely formal title of cipher secretary. Until the end of the century that junior officer was Monsignor Filippo Frassenelli, a minutante who was also listed on the department's roster as cipher officer (incaricato della Cifra). After Frassenelli's retirement, the Secretariat no longer bothered to distinguish the cipher officer with a separate title or notation on the rolls. Rather than a security measure to hide the identity of this official, the new practice probably reflected the insignificant status of cryptography in the department. Frassenelli's replacement was another minutante, the layman Camillo Barluzzi, who served until his retirement in 1903. His immediate successor cannot be identified. After the reorganization of the Secretariat in 1908, cryptography fell within the purview of the Second Section (Ordinary Affairs). A junior officer in this section, Monsignor Domenico Spada, seems to have been assigned the responsibility for codes and ciphers.

32Falconi to Merry del Val, 28 July 1906 and 1 January 1907, ASV. Archivio Nunziatura Stati Uniti (ANSU), sezione 1, fasc. 62.


The cipher officer encoded and decoded messages, and prepared new ciphers and codes for the nunciatures. The latter was not an onerous task since papal cryptography underwent a certain consolidation after 1870. In the first half of that century, the Secretariat of State had used an array of cryptographic systems including jargon codes, null codes, nomenclators, and monoalphabetic and polyalphabetic ciphers. By the 1870s, however, the foreign ministry of the Vatican had abandoned diversity for standardization. For the next 40 years the standard was that familiar standby, the two-part nomenclator, which the Secretariat issued to each of its diplomatic missions. Though these codes differed from each other in certain details such as the equivalencies of specific codegroups or the size of the plaintext repertory, they all followed the same model. This is well illustrated by the system used by the Vatican's special envoy to Mexico at the turn of the century.33

33ASV. Segreteria di Stato, Visita Apostolica nel Mexico, 1896-1900, busta 1.

This nomenclator consisted of 729 three-digit groups from 000 to 888 (skipping any trio that included the number 9) and 100 four-digit groups from 9900 to 9999. Except when part of a group introduced by 99, 9 was a null as were the groups 854, 855 and 856. There were groups for letters of the alphabet (a = 857, b = 310), numbers (one = 555, fifty = 331), punctuation (comma = 9978), common Italian prefixes, suffixes, and syllables (mente = 766, er = 9914), and complete words (Paris = 648, cardinal = 9930). The presence of codegroups for such user-specific terms as Yucatan (330) and Vera Cruz (240) confirms that each diplomatic mission had a version of the standard system adapted to its particular need. Scattered throughout were numerical groups with no plaintext equivalents so as to add to the lexicon as required. There were no homophones.

Between 1870 and 1914, similar systems (two-part nomenclators of three- and four- digit numerical groups with a null digit that was doubled to introduce the four-digit groups) were in service at papal diplomatic missions around the world, although toward the end of the period the large folded sheets of paper which were the distinctive format of the nomenclator began to give way to small codebooks. Occasionally, one mission would borrow the code of another. This was especially true for temporary missions or those whose arrangements had to be expedited. When Monsignor Alessandro Franchi, temporarily recalled to Rome from his post as nuncio to Spain, was sent as the pope's special envoy to Constantinople in 1871, he carried a copy of the nomenclator then in service at his Madrid nunciature. When, in 1900, the archbishop of Havana, then engaged in delicate negotiations with the secular authorities over the status of church lands in Cuba, asked the Secretariat of State for a code, he was authorized to use the system of the apostolic delegation in Washington.34

34For Franchi's code, compare the encoded correspondence from Madrid and Constantinople in ASV. Segreteria di Stato, Spogli: Artibani, busta 1, fasc. 2 and Franchi, busta 1. For the archbishop of Havana, see Sbarretti to Martinelli, 21 March 1900, ASV. ANSU, sezione 1, fasc. 17.

The Secretariat also continued its practice (dating back to the 16th century) of issuing two codes to each nunciature: a cifra particolare for correspondence with the Vatican, and a cifra comune for correspondence with other nuncios. The practice, however, sometimes proved imperfect as the distinction between "private" and "common" codes proved too much for the pope's ambassadors. In 1893 the new nuncio in Madrid, Monsignor Serafino Cretoni, used "Cifra X," the designated cifra comune, in a letter to his more experienced colleague in Paris, Monsignor Domenico Ferrata, seeking advice on how to prepare for the visit to Madrid of the Queen of Portugal. Monsignor Ferrata returned Cretoni's letter unread, saying that the Paris nunciature did not have a code corresponding to that used by the Madrid nuncio. Cretoni then sent the apparently useless "Cifra X" to the Secretariat, and asked for a system which would allow him to communicate with Paris. A rather puzzled Secretariat promptly returned the same code to Cretoni, insisting that "Cifra X" was still in service as the cifra comune, but leaving unanswered the question why the Paris nunciature seemed never to have heard of such a code.35

35Cretoni to the Secretariat of State, 13 July 1893 and Barluzzi to Cretoni, 29 July 1893, ASV. Archivio Nunziatura Madrid (ANM), scatola 585.


Although the two-part numerical code dominated papal cryptography until the First World War, the Secretariat of State occasionally used other systems. During his tenure as nuncio to Spain (1879-1881), Monsignor Angelo Bianchi used a homophonic substitution in which the letters of the alphabet were replaced by two- digit numerical groups from 00 to 99 in mixed order.36

36ASV. Spogli: A Bianchi, busta 2. The author has not been able to discover whether similar ciphers were in service at other nunciatures at this time.
a | b | c | d | e | f | g | h | i | k | l | m
07 | 14 | 11 | 09 | 02 | 08 | 01 | 70 | 04 | 52 | 54 | 20
28 | 24 | 38 | 35 | 19 | 15 | 16 | 82 | 15 |  | 67 | 41
39 | 30 | 65 | 49 | 25 | 55 | 83 | 93 | 23 |  | 88 | 75
48 | 59 | 71 | 63 | 44 | 98 | 91 |  | 42 |  | 97 | 99
86 | 87 | 89 | 85 | 79 |  |  |  | 60 |  |
95 |  | 92 |  | 96 |  |  |  | 76 |  |

n | o | p | q | r | s | t | u | v | x | y | z
21 | 05 | 33 | 45 | 06 | 27 | 17 | 00 | 43 | 03 | 69 | 37
61 | 12 | 40 | 74 | 13 | 31 | 26 | 34 | 66 | 51 |  | 57
80 | 22 | 56 |  | 36 | 84 | 64 | 47 | 77 |  |  | 72
90 | 46 | 68 |  | 62 | 94 | 81 | 58 |  |  |  |
 | 78 |  |  |  |  |  |  |  |  |  |

period(.) 10
nulls 29, 32, 50, 53, 73


Except for the letters K and Y (both rarely used in Italian) each letter had several cipher equivalents, with the more frequently used letters having up to six homophones. The system included five null groups which could be inserted in the middle of an enciphered word as well as between enciphered words. Instructions also directed the cipher clerk to begin and end each line of ciphertext with any single-digit number he pleased. The first and last numbers of each line would then be disregarded during deciphering. Using this system, the message "Arrived Madrid" could be transformed (with nulls) into the following rather unwieldy cryptogram:

- | A | R | R | I | V | E | D | - | M | A | - | D | R | I | D | -
6 | 07 | 62 | 13 | 23 | 77 | 02 | 09 | 73 | 41 | 28 | 50 | 85 | 36 | 04 | 49 | 9

Arrived Madrid = 60762132377020973412850853604499

In its continued reliance on nomenclators and cumbersome monoalphabetic ciphers, the Vatican lagged behind other governments in responding to recent advances in cryptography which made such traditional systems hopelessly obsolete. Unlike the papal Secretariat of State, many foreign ministries, by the 1870s, had abandoned the small, two-part nomenclators that had served for centuries in favor of large, one-part codebooks with lexicons of up to 50,000 entries. With their numerical or alphabetic equivalents for thousands of words and phrases, these codes shortened messages sent by telegraph, which had come to be widely used for diplomatic communications, and so lowered message costs. In 1876 (three years before Monsignor Bianchi carried his unwieldy cipher to Madrid) the U. S. State Department had introduced its Red Code of more than a thousand pages of words and phrases, each of which had a corresponding codeword and five-digit codenumber. With this, a clerk could reduce a 16 word plaintext to as few as five codewords.37 Many governments also sought further security by superenciphering their coded messages, either by transposing the digits of the codegroups or, more commonly, by adding to each codegroup a keynumber, the so-called "additive." By the end of the century enciphered codes were in service at almost every foreign ministry. The Red Code (1876) and Blue Code (1899) of the State Department, for example, used several different methods of encipherment including transposition, additives and, in the Blue Code, subtractives. Even minor powers adopted the practice. The foreign ministry of the Netherlands communicated with its diplomatic missions by means of a system based on the commercial code developed by F. J. Sittler, a former director of French posts and telegraphs. In Sittler's codebook each word or phrase was represented by a four-digit numerical group in which the first two digits indicated the page and the second two the line on which the word or phrase could be found. The Dutch adapted this system by scrambling the page numeration and reversing the order of the codegroup pairs so that the first two digits now indicated the line and the last two the page. A cipher table was then used to transform each numerical codegroup pair into a letter pair, producing an enciphered code of four-letter groups.38 Such procedures, which were increasingly widespread in western chancelleries at the turn of the century, clearly provided more security than the old-fashioned nomenclators and ciphers of the Vatican.

37Ralph Weber, Masked Dispatches: Cryptograms and Cryptology in American History, 1775-1900 (Fort George Meade MD, 1993), 203.
38Ibid., 213, 229; David Alvarez, "A Dutch Enciphered Cod2e," Cryptologia, 19(3) (July 1995), 261-264.


V. TOWARDS REFORM

Though sw to abandon traditional methods, the Vatican was not insensitive to the need to improve its cryptographic systems. The Secretariat of State was even willing to consider systems invented by private cryptographers. In 1907, for example, it asked one of its nuncios to evaluate an enciphered code offered to the Vatican by one Ricardo Candriani, an Argentine of Italian extraction.39 Invariably, however, the Secretariat returned to its own resources. In 1902 the substitute instructed Camillo Barluzzi, the official then responsible for the ciphers of the Secretariat, to solicit from the nuncios "opinions and suggestions that would be useful in the event that a new cipher is adopted."40 The response to this solicitation is not recorded, but in the spring of 1907 the Secretariat of State introduced a new system in which the codegroups combined letters and numbers. The characteristics of this system cannot be determined from the available evidence, but the code quickly proved impractical. Some cable companies would not accept a mixed group such as 4KA14 as a single group for transmission purposes; instead, the companies insisted on treating it as three groups, 4, KA, 14, and charging for three groups rather than one. Shortly after the new system went into service, nuncios began to complain that their cable expenses had increased dramatically. Then in the summer, the Italian parliament enacted new regulations which prohibited the combination of numbers and letters in encoded telegrams. Since the Vatican depended upon Italian cable facilities for the transmission and reception of all its telegrams, this legislation doomed a system whose utility was already in question because of financial considerations. After only a few months, the Secretariat of State withdrew the system and reintroduced the old two-part numerical codes.41 The fact that the Secretariat had to return the old codes in the same packet that contained the letter announcing the recall of the new system suggests that each nunciature had, at any given time, only one system with which to communicate with the Vatican.

39The nuncio judged the system "ingenious" and "most secure," but noted that it was cumbersome to use since encoding required three separate operations. The Secretariat declined Candriani's offer. Locatelli to Merry del Val, 19 June and 21 October 1907, ASV. Archivio Nunziatura Argentina (ANA), scatola 61, fasc. 7.
40Barluzzi to Marchetti, 3 May 1902, ASV. ANSU, fasc. 18.
41Dolci to Merry del Val, 26 June 1907, ASV. Archivio Nunziatura Peru, busta 55, fasc. 3; Locatelli to Merry del Val, 24 July 1907 and Della Chiesa to Locatelli, 20 August 1907, ASV. ANA, scatola 61, fasc. 7.

In 1913, the Secretariat of State had the idea of improving its cryptographic systems through superencipherment. In July of that year, the nuncio in Brazil acknowledged receipt of a new code in a "perfectly sealed package which bore no signs of tampering." The package had been entrusted to one Bruno Chaves (a priest returning to Brazil from Rome), and the nuncio informed the Secretariat that the old code (which had been in service since 1902) would be returned by hand of the bishop of Bahia, who was travelling to Rome on church business.42 The new code, which the Secretariat considered "much more complete, serviceable, and economical" than the old, incorporated two important innovations.43 As we have seen in the case of the code of the Vatican's envoy to Mexico, the standard papal code of this period identified a number (e.g. 5) which, when appearing alone, was a null, but which could be doubled to introduce a legitimate four-digit codegroup (e.g., 5512). The new code for the Brazilian nunciature revised this feature by allowing the doubled null to complete as well as introduce a four-digit group. The new procedure doubled the number of four-digit groups by adding, for example, groups from 0055 to 9955 to the usual sequence of groups which ranged from 5500 to 5599. A more significant innovation introduced superencipherment tables consisting of 100 numerical pairs from 00 to 99 with letter bigrams (always beginning with a consonant) corresponding to each numerical pair (00 = SE, 01 = PO … 99 = HO). A message would be encoded in the numerical code and then assembled into five-digit groups (e.g., 38306 36005 …), as required for telegraphic transmission. These groups would then be divided into numerical pairs (38 30 63 60 05 …). Each pair would then be replaced by its corresponding bigram from the cipher table (38 = TO, 30 = KU, 63 = DA, 60 = ME, 05 = FO …) and the letter pairs combined to form "words" of 10 letters (TOKUDAMEFO …). International telegraphic conventions authorized the transmission of such artificial words, each of which represented two of the original numerical codegroups, as a single group. Although the procedure of superencipherment enhanced security, it appealed to the Secretariat of State primarily because it cut cable costs by half. For some unknown reason, this system proved short-lived for it was not in service when, the next year, an assassination in Sarajevo set off a world war.

42Aversa to the Cardinal Secretary of State, 10 July 1913 and 18 August 1913, ASV. Archivio Nunziatura Brasile, n. 136, fasc. 681.
43Merry del Val to Aversa, 4 June 1913, Ibid.


VI. WARTIME CRYPTOGRAPHY

The Vatican did not replace its cryptographic systems when World War I began. As had long been the custom in the Secretariat of State, each papal diplomatic mission had its own code. The creation and distribution of unique codebooks for each of almost two dozen missions were tasks clearly beyond the capacity of the small Secretariat where, among the staff of 14 in 1914, a single officer was responsible for cryptography.44 Consequently, for some time the nuncios had to depend upon their prewar cryptographic systems. These systems were seriously deficient for wartime use. The lexicons of 700 to 800 plaintext elements were actually smaller than those of many papal codes of the 19th century, and the codes' utility was further restricted by the omission of many secular words such as offensive, armistice, belligerents, submarine, and blockade and their code equivalents. The limitations of this papal cryptography are suggested by comparison to a German diplomatic code in use, early in the war, at German missions in Central and South America. This code had 25,000 plaintext elements and used many homophones. The wartime director of the cryptanalysis unit of the Italian Supreme Command believed that a good diplomatic code should have at least 10,000 elements. Even the U. S. State Department (hardly a leader in cryptography) began the war with a codebook of almost 1500 pages, much harder to reconstruct than the paltry 14 pages of the codebook in service at the Lisbon nunciature in August 1914.45

44Monsignor Domenico Spada, who joined the Secretariat in 1909, was responsible for codes and ciphers during the war. He was assisted by Count Stanislao Caterini, a lay employee of the department since 1912. Scapinelli to Gasparri, 21 May 1915, ASV. Segreteria di Stato, Guerra 1914-18, rubrica 244, fasc. 4.
45Kahn, The Codebreakers, 289; Luigi Sacco, Manual of Cryptography (1936, trans. Laguna Hills, CA, 1977), 58; Weber, United States Diplomatic Codes and Ciphers, 1775-1938, 246. For the Lisbon codebook, see David Alvarez, "A Papal Diplomatic Code," Cryptologia, 16(2) (April, 1992), 174-176.

Thus the Vatican's cryptosystems at the start of the war provided illusory security against the scrutiny of foreign governments seeking information about papal attitudes and intentions. One government was a particular threat. Suspicious of the Vatican's political sympathies and concerned that the papacy would use the dislocations of the war and its aftermath to reassert its claims to temporal sovereignty in the Eternal City (the so-called "Roman Question"), the Italian government was especially eager for intelligence on the Vatican.

Communications intelligence became an important instrument of that surveillance. Under the leadership of Luigi Sacco, an enthusiastic young lieutenant of engineers, a cryptanalysis unit had been established in the intelligence division of the army high command. In addition to attacking the cryptographic systems of the Austrian army, this unit also monitored the diplomatic traffic passing in and out of Rome. In the course of the war Sacco's unit read, with more or less regularity, the secret diplomatic messages of several countries including Greece, Spain, Switzerland, and the United States. By the end of 1916 (at the latest), Italian intelligence was routinely intercepting and reading the encoded communications between the Secretariat of State and papal diplomatic missions in Austria-Hungary, Belgium, Germany, the Netherlands, Spain, Switzerland, Turkey, and the United States. Plaintext transcriptions of such communications were often delivered to Italy's foreign minister, Sidney Sonnino, within 24 hours of their transmission.46

46David Alvarez, "Italian Diplomatic Cryptanalysis in World War I," Cryptologia, 20(1) (January 1996), 1-10.

From the available evidence it is difficult to determine whether Italy's success in reading the Vatican's secret communications stemmed from cryptanalysis or from the clandestine acquisition of papal codebooks. Sacco's solution in 1917 of an Austrian diplomatic code suggests why theft may be likely. The texts of the messages, which passed between the Austrian foreign ministry and its two embassies in Rome (one accredited to the Italian government and one to the Vatican), include tentative identifications and undeciphered groups. These imply that the code is being reconstructed.47 The intercepts of the papal telegrams, on the other hand, are completely deciphered from the start. This suggests possession of a codebook. In fact during the war Italian agents burglarized the diplomatic missions of neutral as well as enemy powers in search of secret documents including codebooks. Given Italy's concern over Vatican policy, the Secretariat of State would have been a prime target for covert operations. At least one papal nunciature was infiltrated: Italian intelligence had an agent inside the nunciature in Madrid, which was a collection center for papal diplomatic mail to and from South America. The nunciature in Berne was under especially close surveillance.48

47Ibid.
48Marchetti to Sonnino, 10 January 1918, reel 52, Sidney Sonnino Papers (microfilm), Hoover Library, Stanford University, Palo Alto CA.


While Italy was the principal transgressor, other governments successfully attacked the Vatican's codes during the war. In 1917 the Swiss were reading at least one of the systems in use at the Munich nunciature. Although direct evidence is lacking, it is highly likely that Britain's skilled cryptanalysts in Room 40 (who successfully attacked cryptographic systems far more difficult than those of the papacy) were able to read encoded papal messages. They perhaps paid special attention to those that passed through the transatlantic cables, which the British closely monitored. Germany may have achieved the most dramatic success. Through Matthias Erzberger, the head of the Catholic Center Party and a familiar figure in Vatican circles, the German Foreign Ministry apparently sought, in the spring of 1915, to recruit the services of a junior officer in the Munich nunciature. The Germans hoped to convince this monsignor to provide, among other items, the codebook of the nunciature, ostensibly to allow Berlin to communicate directly with the Vatican in the event of war with Italy. A successful approach would have paid unexpected dividends, for the papal diplomat was soon promoted to a senior position in another European nunciature. While it is impossible from the available evidence to determine conclusively that the papal diplomat was a German informant, it seems that he received covert payments from Franz von Stockhammern, the director of German espionage in Switzerland, and that these payments continued at least until the summer of 1916. Whatever their success with this monsignor, German intelligence closely monitored the communications of the nunciature in Munich throughout the war and read some of the encoded traffic on the Munich-Berne-Vatican circuit.49

49Hort to Pacelli, 10 May 1922, ASV. Archivio Nunziatura Baviera (ANB), busta 307, pos. 1; "Fidelis" to the Undersecretary of State of His Holiness (undated, but probably the summer of 1917), ASV.'Segreteria di Stato, Guerra 1914-18, rubrica 244, fasc. 6. Wilhelm Patin, Beiträge zur Geschichte der deutsch-vatikanischen Beziehungen in den letzten Jahrzehnten (Berlin, 1942), 51.

Various factors undermined the integrity of papal codes. In the early years of the war, each nunciature had a single, unique code. All encoded communications beween the nunciature in Madrid and the Secretariat of State, for example, were in one system. This practice would ease the work of foreign cryptanalysts seeking to break into the Madrid-Vatican circuit. Lacking cable or wireless facilities of its own, the Vatican was compelled to use the facilities of the Italian telegraph service. Italian military intelligence was thus able to obtain copies of every telegram sent or received by the Secretariat of State, thereby enhancing its effort against papal codes. In foreign capitals the nuncios were also dependent on local facilities. Messages to and from distant posts often passed through several sets of hands. Cables for the papal representative in Warsaw, Monsignor Achille Ratti (the future Pope Pius XI), for example, moved through Italian, Swiss, German, and (after the armistice) Polish telegraph offices. In 1920, when he was on a special, fact-finding mission in Upper Silesia (then a disputed area under a temporary Inter-Allied Plebiscite Commission), Monsignor Ratti actually sent and received encoded messages through the facilities of the Polish consulate in that territory.50 Since Poland claimed parts of the disputed territory, the consular officials would have been most interested in the nuncio's confidential reports, and they perhaps passed copies of such reports to the fledgling Polish cryptanalytic service, the Biuro Szyfrów (Cipher Bureau), which had already achieved notable success against the cryptographic systems of the new Bolshevik regime in Russia.

50Farolfi to Ratti, 14 November 1920, ASV. Archivio Nunziatura Varsavia (ANVa), scatola 190, fasc. 1.

Papal representatives were sometimes lax in their communications practices. The Secretariat of State specifically warned the nuncios against repeating in their reports the plaintext of encoded cables, but there were lapses. Once the Vatican's representative in Washington acknowledged, in an open dispatch, the receipt of a coded message from the Secretariat, the contents of which he then summarized. This same officer often repeated verbatim the plaintext of coded messages from the Vatican in his open letters to Cardinal James Gibbons of Baltimore. Monsignor Bonaventura Cerretti, a senior officer in the Secretariat on a special mission to Paris, entrusted to the British diplomatic pouch an open letter to the cardinal secretary of state which included the dates, reference numbers and summaries of three coded messages he had recently cabled to the Vatican. Trusting in the inviolability of the diplomatic pouch, papal diplomats usually did not encode dispatches entrusted to couriers. Since the security of the pouch was, in fact, problematic, this was a potentially dangerous practice. Nuncios often prepared dispatches to follow and elaborate upon developments which they had briefly reported by encoded telegram. Any power which surreptitiously gained access to the Vatican bag could, after comparing dates, use the nuncios' open reports as cribs to attack the intercepted cables. Some nuncios would encode only the more sensitive passages of a dispatch despite an explicit warning from the Secretariat of State that such practice greatly facilitated the efforts of foreign cryptanalysts. In one extreme case, the nuncio in Berne sent a long message in which only two words were in code.51

51Bonzano to Gasparri, 9 June 1915, ASV. ANSU, sezione 1, fasc. 62; Cerretti to Gasparri, 27 November 1918, Vatican City. Archivio della Congregazione per gli Affari Ecclesiastici Straordinari (AAES), Stati Ecclesiastici, 214, Questione Romana, III, 1917-20; Marchetti to Gasparri, 20 June 1917, AAES, Guerra Europa, 1914-18, Iniziative Pontificie per la Pace.


The Secretariat of State, itself, was not beyond reproach. In May 1915 it prepared a new code for the nuncio in Vienna, and naively gave it to a homeward bound Austrian diplomat for delivery. While there is no evidence that this messenger betrayed his trust, it would have been an exceptional diplomat, indeed, who would have, in wartime, resisted the temptation to sneak a look at the cryptographic system of an important neutral. The following month the Secretariat gave one of its codes to the Prussian ambassador to the Holy See in order to facilitate communication after the ambassador was compelled by Italian belligerency to move his mission from Rome to Switzerland. Since all papal codes in this period were variants of the same system (two-part, numerical codes with three- and four-digit groups and a null number), this courtesy helped not only the German diplomat but the German cryptanalysts as well.52

52Scapinelli to Gasparri, 21 May 1915, ASV. Segreteria di Stato, Guerra 1914-18, rubrica 244, fasc. 4; Tedeschini to Muhlberg, 6 June 1915, ibid., fasc. 2.

The experience of the nuncio to Bolivia suggests that, as late as the final year of the war, a degree of amateurism still tainted the Secretariat's attitude toward cryptography. In the spring of 1918, Monsignor Rudolfo Caroli, concluding that the codebook of the La Paz nunciature was hopelessly inadequate, decided to pass the time at his distant and quiet post by creating a new system. The result was a four volume code of more than 30,000 words and phrases with provision for a daily superencipherment key. Caroli proudly offered this work to the Secretariat, graciously noting that he would understand if the department wished to retain its current codes, even though none were as clever and efficient as his "marvel." The Secretariat of State was underwhelmed. Cardinal Gasparri eventually thanked Monsignor Caroli for his "colossal" efforts, but hastened to remind the nuncio that the Vatican's codes were the product of long experience and careful consideration. Left unspoken was the implicit suggestion that the good monsignor (who, before his appointment to La Paz in 1913, had presumably sharpened his cryptographic skills as the bishop of a small Italian town and, before that, as an expert on religious ceremonial) might seek other diversions and leave cryptography to the Secretariat of State. The cardinal secretary of state, however, was reluctant to disappoint the nuncio after so much labor, so he authorized him to use his code to communicate with the Vatican. Allowing diplomats to invent their own codes was a dangerous practice, since only experienced cryptologists could judge the security of a new system. Fortunately, Caroli's "marvel" proved too inconvenient for the clerks in the Secretariat, and after a few months the nuncio was instructed to return to his standard system.53

53See the Caroli-Gasparri correspondence in ASV. Archivio Nunziatura Bolivia, fasc. 108.

The Vatican was not completely oblivious to the threat to its confidential communications. In October 1915, the Secretariat of State warned its diplomatic missions that it was aware of "grave abuses" concerning the custody of confidential papers, "some of which, though most sensitive and important, had come into the possession of foreigners." Diplomatic personnel were informed that anyone, of whatever rank, who removed documents from the archive of a nunciature or delegation without authorization would be immediately suspended not only from his post but also from the priesthood. To lend substance to this warning, the first secretary in the nunciature to Belgium was sacked for leaking to an Italian journalist information concerning the Vatican's interest in a separate peace between Belgium and Germany.54

54See the circular in ASV. Archivio Nunziatura Vienna (ANV), scatola 190, fasc. 148.

The Secretariat of State was aware that its encoded cables were intercepted and studied by foreign governments, and at least some of the nuncios worried about the integrity of their cryptographic systems. As late as the spring of 1916, the nuncio in Germany hesitated to commit his most secret messages to the telegraph office because he doubted the security of his code.55 To protect its cryptographic systems, the Secretariat of State tightened its procedures for handling encoded messages. Previously, it had been common for the code clerk to write the plaintext of an incoming dispatch or telegram above the codetext or on an attached sheet. Both the codetext and the plaintext would then go into the department's files where they could be examined (and compared) by anyone who gained access to the files. By the middle of the war, the Secretariat had abolished this practice in favor of a method by which the clerk decoded a cable onto a separate piece of paper. Only this plaintext would enter the files, while the codetext would be destroyed. In dispatches where only a portion of the message was in code, the plaintext might still be written above the codetext, but now the latter was obliterated. The goal was to make it impossible for unauthorized eyes to compare the codetext of a message with the plaintext. Unfortunately, the nunciatures did not always follow the new procedures. The papal mission to Poland, for example, continued to file the plaintext with the codetext of both outgoing and incoming messages well into 1920.56

55Frühwirth to Gasparri, 14 May 1916, ASV. Segreteria di Stato, Guerra 1914-18, rubrica 244, fazc. 29.
56ASV. ANVa, scatola 190, fasc. 1 and 5.


The Secretariat of State also responded to the threat against its communications security by changing its codes more frequently. Whereas before 1914 a nunciature might have used the same codebook for more than a decade, during the war (especially after Italian belligerency) the more important posts received new codes as often as every six months. For delivery of new codes, the Secretariat decided that it could not trust the ordinary mail or the diplomatic couriers of friendly or neutral states. Special couriers were now employed for this sensitive task- usually papal diplomats traveling to their posts, as when the nuncio in Madrid received a new codebook from a monsignor returning to the nunciature in Argentina, or when the secretary to the nuncio in Munich returned from a leave in Rome with a new codebook for the nunciature. Nuncios were now required to confirm that the sealed packets containing cryptographic materials arrived "absolutely intact and without any sign of tampering." The more important posts were given additional codes to allow them to vary the cryptography of their messages. When Monsignor Eugenio Pacelli (the future Pope Pius XII) was appointed nuncio to Germany in the spring of 1917, he departed for his post with not one, but two fresh codebooks in his baggage.57

57Cardinal Gasparri was annoyed by Monsignor Pacelli's practice of signing some of his cables "Pacelli" and others "Pacelli, Nunzio Apostolico." The nuncio explained that when he had taken custody of the two new codebooks he had been instructed by the Secretariat's cipher officer to indicate the use of the first by signing the message with his surname, and to indicate the use of the second by adding "Nunzio Apostolico" to his name.

Like other states during this period, the Vatican concluded (somewhat belatedly) that its cryptographic systems were inadequate for the challenges of the war. In the spring of 1916, the Secretariat of State distributed new codebooks that were "larger and more complete" than those previously in service. These were two-part codes of four-digit numerical groups, a single null number and an expanded plaintext dictionary.58 In January 1918 the Secretariat took a more significant step by introducing a superencipherment for use by its missions in Berne, Madrid, Munich, and Vienna. In a letter to the nuncio in Munich, Cardinal Gasparri explained the Secretariat's action:

58For an example from the Vienna nunciature, see Valfre to Gasparri, 20 September 1917, AAES, Guerra Europa, 1914-18, fasc. 12.

The need to guarantee the secrecy of the code has led the Holy See to introduce certain modifications in its normal use.

It is reasonable to assume that copies of encoded telegrams are retained by government agencies, and that these copies are carefully studied by experts in cryptology who, by means of careful comparison of texts and attention to information from newspapers and other sources, succeed, more or less quickly, in exposing the codes of the Holy See.

To deal with this serious problem, it is necessary to confuse these cryptologists by undermining the basis of their studies. This can be achieved by the use of "transformation tables" that do not replace the system currently in use, but complement it by transforming and distorting coded telegrams before they are consigned to cable offices for transmission.59

59Gasparri to Pacelli, 26 January 1918, ASV. ANB, busta 307, pos.1.

Each mission received a new code along with so-called "transformation tables" consisting of two columns, each containing 100 pairs of digits from 00 to 99. In column A these groups were arranged sequentially (00, 01, 02 …), while in column B they were arranged randomly (29, 43, 09 …). To prepare a secret communication, the nuncio would first use his codebook to encode his message in the usual way. Next he would divide the codetext into groups of two digits. He would then find each of these bigrams in column A of his cipher (transformation) table, substitute the corresponding bigrams from column B, and arrange the latter into the standard five- digit group for transmission.60 The codetext 16042 97831, for example, would be divided into 16 04 29 78 31. The cipher table would indicate that

60Ibid.
16 | corresponds to | 08
04 | " | 31
29 | " | 88
78 | " | 54
31 | " | 17

The table enciphers the original codetext 16042 97831 into 08318 85417.

By early spring Monsignor Pacelli in Germany was using three different cipher tables with his codebooks. Each table carried identifying indicators: the first, the prime numbers 1, 5, and 7; the second, the number 2 and its multiples 4 and 8; and the third the number 3 and its multiples 6 and 9. Whoever enciphered a message would indicate which table was in use by making the first group in the enciphered text consist of four rather than five digits. He would then expand this into a five digit group by prefixing one of the three table indicators. An enciphered message from Munich beginning with 2, 4, or 8, therefore, would mean that Pacelli had used his second table.


Only the most secret communications were to be sent in this cifra transformata, while less sensitive material would be sent in straight code without encipherment. The Secretariat of State was confident that this new system would guarantee "full and absolute" secrecy.61 The available evidence suggests that this confidence was not entirely misplaced. The Secretariat's enciphered codes immediately began to cause problems for Italian cryptanalysts. While continuing to read many coded telegrams on the important Munich-Vatican circuit, they were unable, well into the summer of 1918, to break the new system in which the more sensitive messages had been enciphered. This lapse, which was in marked contrast to earlier success in reading practically all of Pacelli's communications, was probably due to frequent changes of code and the selective use of the cifra transformata. The nuncio in Germany used the new system to discuss with his superiors at the Vatican Austria's interest in a negotiated peace, discussions which apparently escaped the scrutiny of Italian cryptanalysts. Similarly, reports from the papal representative in Switzerland concerning Austrian overtures in September 1918 for a separate peace remained hidden from Italian eyes. By the last month of the war, Italian intelligence was reading some of the cables from Berne, but one of the nuncio's codes remained unbroken, as did one of the codes of the nuncio in Vienna.62

61Ibid.
62See the notations on the intercepts from the Munich-Vatican circuit for 31 March, 31 May, and 27 June 1918; from the Berne-Vatican circuit for 24 and 25 September 1918; and from the Vienna-Vatican circuit for 7 and 28 October 1918, reel 48, Sonnino Papers.

Despite the apparent success of the cifra transformata, the Secretariat of State did not put the system into general service. The unenciphered, two-part numerical code remained the workhorse of the papal diplomatic corps right through the end of the war. When Monsignor Ratti departed in the summer of 1918 to become the Vatican's new representative in Poland, he carried a codebook that exemplified the "larger and more complete" systems first introduced by the Secretariat of State in the spring of 1916. This was a two-part, four-digit code with groups for nouns (Russia = 6281, information = 0036), the alphabet (A = 5004, Z = 6210), common Italian syllables, prefixes, and suffixes (es = 3641, st = 3487, mente = 0679), parts of speech (without = 2816), numbers (3 = 6523, 100 = 5425), and punctuation (period[.] = 7074). Similar codes in service at other papal diplomatic missions included some sort of null feature, but Monsignor Ratti used no such feature when encoding his reports. As late as the summer of 1919, this code was not enciphered. Still, the plaintext repertory of several thousand elements was a marked improvement over the small codes of several hundred elements with which papal missions began the war. In 1920 the Secretariat of State put into service a new code on the Vatican-Warsaw circuit. This was again a two-part code, but of five- rather than four-digit groups. It was notably more efficient than its predecessor. The latter, for example, required three groups for the word corriere (mail/courier): corr = 4608, i = 5310, ere = 3608, while the new code used a single group: corriere = 08902. It too was routinely used without superencipherment. Sometime in early 1920, however, the Warsaw nunciature began to superencipher its telegrams, but from the available evidence it is impossible to determine if this represented the introduction of the cifra transformata or a system based on additives.63

63Monsignor Ratti's codes have been partially reconstructed by the author from the telegrams in ASV. AnVa, scatola 190, fasc. 1 and 5.

The Vatican occasionally employed jargon codes in which an apparently open message contained code words or phrases. As early as January 1915, the nuncio in Vienna was using a short code in which Italy = MONSIGNOR G, Austria = MONSIGNOR T, Germany = ABBOT S, Pope = MISTER PRIMO, and Emperor Franz Joseph = MISTER T. During secret exchanges with Austria concerning a separate peace in the fall of 1918, Monsignor Pacelli sought additional security by employing a jargon in which the Austrian emperor was the ABBOT PRIMATE, Vienna was the CENTRAL ABBEY, and the world war was the WELL-KNOWN CONTROVERSY, while the entire communication was composed in such a manner as to suggest that its subject was ecclesiastical business. Such a message would then be encoded by the usual two-part code. It is unlikely that such a transparent cover would have fooled experienced cryptanalysts who almost certainly would have guessed what "well-known controversy" was so secret as to require encoding. But it may have deterred them by eliminating from the plaintext words they would look for, such as Charles, the new emperor of Austria.64

64Scapinelli to Gasparri, 19 January 1915, ASV. Segreteria di Stato, Guerra 1914-18, rubrica 244, fasc. 28; Pacelli to Gasparri, 12 September 1918, AAES. Guerra Europa, 1914-18, fasc. 18-22. Occasionally the Vatican used codes to protect the secrecy of internal church administration, but usually codes were used only for diplomatic affairs.

Monsignor Bonaventura Cerretti carried a similar jargon code on his special mission to the Versailles Peace Conference which opened in January 1919. Cerretti was ostensibly at the conference to ask the victorious Allies to refrain from expelling German missionaries from former German colonies in Africa, China, and the Pacific islands. In fact, he was the Vatican's unofficial representative to the peace conference, where he met with various world leaders and opened secret negotiations with Italian Prime Minister Vittorio Orlando on a reconciliation between the Vatican and the Kingdom of Italy. To communicate with the Secretariat of State, Cerretti had a jargon code of two sections.65 In the first, various individuals or subjects were identified by ecclesiastical terms. For example,

65"Missione di Mons. Cerretti a Parigi," AAES. Asia (1919), fasc. 86.
Woodrow Wilson = ABBOT GENERAL
Colonel House = BENEDICTINE ABBOT
Lloyd George = VICAR GENERAL
Vittorio Orlando = MASTER GENERAL
Sidney Sonnino = ASSISTANT GENERAL
Georges Clemenceau = LAY BROTHER
League of Nations = GENERAL CHAPTER
Peace Conference = PROVINCIAL CHAPTER, etc.


In the second, a series of code phrases allowed Cerretti to report secretly the progess of his confidential discussions on the Roman Question with Orlando. The code phrases included such statements as,

ENRICO HAS BEEN ADMITTED TO THE EXAMS.
Orlando has recognized the necessity of a territorial solution.
ENRICO PREFERS THE MILITIA.
Orlando agrees, to negotiations between Italy and the Vatican.
ENRICO HAS MADE HIS FIRST COMMUNION.
Orlando will not accept a [Vatican] outlet to the sea.

Whatever the benefits of jargon codes or the cifra transformata, there can be little doubt that throughout the war the Holy See was plagued by poor communications security. The problem complicated the administration of a global church and seriously undermined papal diplomacy. Italy, the state for which the relevant cryptanalytical evidence is most readily available, provides a case in point. A power with which the Vatican's relations were cool and distant when not actually hostile, Italy was privy to most of the confidential communications of the Holy See. In February 1917 it listened as Cardinal Gasparri cabled the nuncios in Vienna and Munich about an opportunity to detach Italy from its Entente partners. Rome's cryptanalysts must have smiled at the cardinal's warning to the nuncios to 'destroy this message and maintain absolute secrecy in this matter.'66 When intercepts in the summer of 1917 revealed the papal Secretariat of State discussing with Berlin (but not London, Paris, Rome or Washington) the terms of Pope Benedict's peace proposal before its presentation to the Allies on 9 August, Italian suspicions that the papacy favored the Central Powers could only have been fueled. Reluctant to pursue any settlement that did not address its territorial ambitions, the Italian government was fortified in its opposition to the pope's peace proposal by cryptanalytical evidence that the Secretariat of State was exaggerating the extent to which Berlin and Vienna were prepared to make concessions. In late September 1917, Cardinal Gasparri assured an emissary of the Italian government that Germany was prepared to reestablish Belgium and pay an indemnity to Brussels and that Austria was open to some of Italy's territorial aspirations. The Italian foreign minister, however, knew from the intercepts that the nuncio in Germany had informed the Vatican that Berlin's position on Belgium remained vague and equivocal, while the nuncio in Austria had reported that Vienna had excluded any territorial concessions.67

66Gasparri to Aversa and Valfre, 23 February 1917, reel 48, Sonnino Papers.
67Pacelli to Gasparri, 27 September 1917 and Valfre to Gasparri, 25 August 1917, reel 48, Sonnino Papers; Italo Garzia, La Questione Romana durante la prima querra mondiale (Naples, 1981), 161-164.

VII. CONCLUSION

Between 1815 and 1920 the Vatican lagged behind other states in responding to developments in cryptographic practice. Throughout most of the period it entrusted its confidential communications to outdated cryptosystems of uncertain security. Particular factors may account for the Vatican's failure to keep pace with advances in the field of diplomatic codes and ciphers. For reasons of economy the Secretariat of State lacked professional cryptologists and delegated cryptography to junior officers who picked up what knowledge they could on the job, often while performing other duties within the department. An exaggerated respect for tradition and a preference for proven and long-established procedures (characteristic elements of the administrative culture of the Vatican) discouraged innovation in all areas, including cryptography. Finally, the Vatican's modest profile in international diplomacy reduced the demand for secure communications and provided little incentive to change established practices. Without more research into the cryptographic practices of minor powers in the same period it is impossible to determine whether the experience of the Vatican was common to small states. It is possible, however, to conclude that the reputation of papal cryptography, once so lustrous, had sadly faded.

BIOGRAPHICAL SKETCH

David Alvarez is a professor at Saint Mary's College of California. He is completing a study of German espionage at the Vatican during the Second World War.


Bletchley Park 1941-1945

William F. Clarke

[William F. Clarke, barrister, received a commission as an Assistant Paymaster RNVR at the beginning of 1915. He was the son of Oscar Wilde's counsel and had been educated at Uppingham and Magdalen College, Oxford, and was fluent in German. In 1916, the Director of Intelligence, William R. " Blinker" Hall, recruited him for Room 40. Clarke was only an average cryptanalyst, but his legal training made him a good Intelligence Officer. After WW I he and Frank Birch, another cryptanalysts, were assigned to compile a secret record of the activities of the German fleet. Clarke was one of the few people who remained with Room 40's successor, the Government Code and Cypher School (GC&CS) from its inception through the end of WW II. He wrote extensively about the history of the organizations in which he served, his experiences, views and co-workers.

This is the third of four sections from Clarke's papers and it was written nine years after he ended a 29 year cryptologic career which began in 1916. Clarke's papers are held by the Churchill Archives Centre, Churchill College Cambridge, with reference identification CLKE 3. The copyright to these papers is held by Clarke's son, Judge Edward Clarke QC, and all excerpts are printed with his permission.]

Forsan et haec olim memisse iuvabit.

[Perhaps someday it will be pleasant, to remember this.]

The receipt yesterday of a letter from my oldest and most valued co-worker asking for some information for a very old colleague of mine from whom circumstances have estranged me about a detail relating to our common work in the 1914 - 18 war has led my thoughts back to these years which were certainly the most unhappy of a very happy life so I will try to write my account of them as fairly as I can. I recognise [sic] that the events which made them so unhappy for me may have been partly my own fault and partly the result of a sequence of events over which none of us had complete control but compelled as I was to be in the role of an onlooker without any power or responsibility, I may be able to judge fairly.

At the beginning of the year all seemed going well; the Italian section which was under my control was going well despite difficulties; it is true that thanks largely to the stupidity of the detached section in Egypt, which had been established in spite of my protests while I was away for an operation, thanks to an intrigue by Titterton, more care had been taken by the Italians with regard to their codes which were now much more difficult but valuable work by two junior members of my staff Grey and Clarke had shown that a most important part of their communications were ciphered on a Swedish machine, obtainable in the market whose possible use by them had been betrayed by a plain language message. I had handed over to the Head of the German section Bodsworth, my most capable cryptographer and the others were fully employed on other Italian codes. I urged that work on the machine material should be undertaken at once by our general cryptographic section under Tiltman but it was some time before I could get this done.

In the meantime important developments had occurred with relation to the German codes; their machine had come into our hands and great progress was being made. Also the entry of Italy into the war had made the task of Naval section one especially as combined action between the two allies was developing. So in March 1941 I suggested the two sections should be amalgamated under Birch, he was in favour [sic] and I was quite willing to work under him as I had in 1919 but at first the proposal did not seem to find favour [sic] among the high-ups. At a meeting in April I managed to get my way re the machine and the cryptographic section took on the task which was beyond the powers of my heavily reduced staff whose work was fatally hampered by the retention in Egypt of an Italian captured code book which was in current use by Italian submarines but which in their inexperience they had put aside as useless; it finally reached us after peace with Italy. Early in June the machine secret was solved and a large part of the Italian traffic became legible. This was decisive and on June 15 the merger of the German and Italian sections took place; Birch very properly was to be head and Bodsworth his second, I was to be on the staff in a rather undefined capacity. Bodsworth's position was, I thought and still think was a mistake; he was very bad at managing staff; worse still he took over the management of the Italian section with a Professor Vincent as No 2. Vincent was Professor of Italian at Cambridge, a fine scholar but like many of his class good at his own classics but very poor at the language used by naval officers; when he was working under me my first task on arrival at the office when he had been on night duty was to see how many mistakes in translation he had made and send in the necessary corrections.

Bodsworth was too busy on his own special work to devote much time to the Italian part and was, I think suffering from an inferiority complex as regards Vincent whom as his elder at one time, I think, tutor or at any rate superior at Cambridge, he regarded with some awe. As a result Vincent ran one section with a protegee of his, Crichton, who was useless and idle.

However all went well for a time; the drastic change produced by the solution of the important German and Italian codes necessitated much increase of staff and reorganisation [sic]; in this I seemed to be useful. Kinsley, extremely capable had my old 1917 - 18 job of intelligence officer in chief which he kept till the end of the war. A few young R.N.V.R. special branch officers were brought in as I.O.s to maintain constant watch on current traffic; the head of these was Gorley Putt who was excellent in every way. In their early days I used to visit them early in the morning, before Birch arrived and try to train them in their job which after a little experience they carried out most successfully. At first the decode sections worked separately but on August 21st the German and Italian decode sections were merged and all their decodes were passed to Admiralty under one head, Walter Ettinghausen, a very able Oxonian who now I believe is a prominent member of the Israeli government under another name; he was excellent and one pleasant memory of unhappy years is the continued good terms which continued until I left, between him and myself.


The next memorable event which took place was a visit from the Prime Minister on September 6th. This had for me some humourous [sic] sides. I had known Winston Churchill for many years, my father was a great admirer and defender of his and I had helped him in the writing of his well known books. The day before his visit I was told by our chief that it was proposed to introduce to him some of the senior staff, including myself and Jimmie Turner. He duly arrived and was shown a sort of composite section, of all kinds, including Vincent who under the guidance of one Wall, a very competent cryptographer of my staff in the pre- amalgamation days, had managed to solve a submarine Italian code which should have been solved a long time before if Vincent had done his work properly. Turner and I were not presented. Winston made us a very nice speech of thanks and went off to lunch at Blenheim. He had arrived in the usual cortege of cars with flags flying and must have been spotted by the local inhabitants; for lunch the B.P. staff crowded out into the town and no doubt talked about the visit to all and sundry; late that afternoon an order came round to say that his visit was to be kept secret, another instance, if more were needed of our wonderful security which had allowed motors bearing the name of a well known tabulating firm to ply to and from our premises.

* Fig.
The Victorian mansion, Bletchley Park, which was the headquarters of the G.C.&C.S. in WW II.

Late in October our I.O.s were put into watches.

Then came the first big row; it concerned one of my most able helps for many years; she was a Miss White who sent originally to be my secretary, I had found to have the makings of a quite passable cryptographer. She had done remarkably good work helping me; when De Grey came to bring up to date his knowledge in that line, I handed him over to her, so I did Lord Lytton who at one time was anxious to join us but whom I persuaded that he would be better employed elsewhere. On the reorganisation [sic] she had been left to work under Vincent and Crichton both of whom were her enemies and who were jealous of her, realising [sic] she was better at the job than they; in fact she often had had to teach them their job and point out their mistakes in the past so having attained power they were bent on revenge. In this they were helped by her temperament which was not easy. In spite of my efforts to prevent what I regarded as a calamity, they persuaded Birch to ask for her transfer; Bodsworth collaborated in this in spite of the fact that he had previously told me he could not spare her when I asked she might be lent to me for what I thought was a very important job. Another section was delighted to have her but her work was lost to the section. The full correspondence on this matter will be found in my files labeled The White Conspiracy.

This most unfortunate episode was my first break with Birch who had preferred others advice to mine; I think it shook his previous good opinion of me, certainly his attitude to me in after years showed a great change. All this happened in November and very soon afterwards I was taken ill with a complaint that was to lead to a severe operation a few years later.

Things however apparently settled down and our work went on but during my rather prolonged absence many changes took place. It should be stated here that, acting in accordance with the lessons of the previous war, it had always been my practice to circulate appreciations of messages decoded rather than the text; our experience in the early days of that war had shown that we were not merely a decoding but an intelligence section and that the higher ups made too many mistakes by their incorrect deductions. This was more easy in the days of the old type of codes when some words had to be guessed but when machine codes came in all the words were normally legible, only the language difficulty remaining.

While the two sections were separate I kept to the old method but as it involved a little more work, it was unpopular with my staff and they had while I was away adopted the easier course and the actual text was sent to the O.I.C. whose functions are described in "The War at Sea" Vol. I, p18. I personally think this was a mistake; during the prewar years I had always urged that we should be in close proximity to Operations Division as had been the case at the end of the previous war; the only result had been as I have stated in previous chapters that my German section had been removed to the Admiralty where it was completely useless and I had to build up a new one as the head of which I was lucky to get Birch. This development had taken from me all control over the Italian traffic circulated which I had exercised before my unfortunate illness.

However I found plenty to do, in spite of the fact that relations between me and many of the section were for ever strained. The mass of deciphered intercepts necessitated a largely increased staff and the establishment of a system of indexing and arranging the intercepts to ensure rapid ascertainment of previous information on matters dealt with in a current message. This needed a great deal of supervision as the staff entrusted with these tasks was of necessity inexperienced and many mistakes were made.

Another amusing task I had was to compose differences between Dilly Knox and Birch; Dilly was a genius in cryptography as well as in other matters but he was very temperamental and had to be handled tactfully or else he refused to play; I happened to be one of the few people who understood how to manage him and when these clashes occurred I was called into restore peace and ward off the resignation which he threatened. He died soon after and his death is one of the many for which the high ups at B.P. were responsible, another was Leslie Lambert alias A.J. Alan. Both were sick men and no consideration was shown them. This was characteristic; in the years before the war I had a valuable colleague, Rees, who had had his lungs injured by shrapnel in the First World War; in consequence when there was fog he was grievously distressed and not fit to work but when at these times I sent him away I was reprimanded by Denniston. The same thing happened when I varied the hours of one of my staff who was only temporary and was preparing for an examination; in spite of my protests she was dismissed only to be employed in an important job by the S.I.S.


In the meantime in March 1942 great changes had taken place at B.P. The diplomatic section under Denniston had been moved back to London; if any part were to be moved to this danger area it should have been Naval section or some part of it so as to be nearer Operations Division. This meant that most of my old and experienced colleagues went with it and so I lost most of my friends. Travis was now in supreme command under Quex's successor, a very unworthy heir, named Menzies who disliked me as he was connected by marriage with the Wilson family who had been the subject of bitter comment by my father in the famous Baccarat case. Of my real friends only Strachey was left and he was a sick man. Quex's death early in the war was a real calamity, he was a very brilliant chief who trusted his subordinates and to whom one could go knowing that if your application was wise, and he always investigated fully, you would get what you wanted, be it staff or improvements in organisation [sic].

B.P. now consisted of the three service sections and the usual competition started as to which was to be the largest, one had seen this in the former war and its evil results; staff was added to do useless jobs and thanks to the poor material enrolled the work was badly done. Advertisements even appeared in the Times asking for recruits. Naval section was no less to blame in this respect than the other sections.

Quite early on tabulating machines had been brought into use by us and while they were of great use in certain circumstances, they did, in my opinion, a good deal of harm as many idle people thought they relieved them of personal indexing; this, I think, led to the failure to solve many of the hand codes, as contrasted with the machine types.

The increase of staff led to the building of more commodious quarters and a large new block was built for naval section to which we duly removed.

During 1942 the war at sea was chiefly concerned with the submarine menace, apart from the escape of Scharnhorst and Gneisenau from Brest and very important the interception of Italian transports carrying men and supplies to Africa. Thanks to the solving of the Swedish machine this last was very successful. The work of the section had become largely a matter of routine, the Swedish messages were dealt with most successfully by a small section and the German naval traffic outside, the only really active work being in the hands of the circulating party under Ettinghausen and intelligence appreciations by Hinsley and his aides. The larger part of the staff was employed on record keeping. This required constant overseeing and a large part of my time was devoted to this. I was officially shown on so called Naval Section state as part of the Staff but Staff papers were carefully kept from me and most of my activities were sub rosa; however I found most of the subsections glad of my suggestions. I had learnt that my formal submissions were either ignored or turned down so except in more important matters I did not indulge in them. The section was now approaching the 1000 mark as compared with the 24 who had gone down in August 1939.

The Italian section had made no progress with the main codes; the famous submarine code displayed to the P.M. had been changed within a week of his visit, a strange coincidence which might have seemed curious but it had only been solved after being compromised by long use. Work on the German naval traffic had been halted for a while by their employing additional wheels but this had been soon overcome.

So matters went on in the early part of 1943 but the course of events was changing in our favour [sic]; things were going badly for our enemies all along the various fronts and our political leaders had started to talk about "Unconditional Surrender". Early in the year a copy of an Italian code "Sigma" came into our possession and helped our anti Italian operations. But Italy was already at its last gasp, the Axis forces had been driven out of Africa by early May and in July Italian territory was invaded. Negotiation for her surrender started and in August the Italian section personnel were dispersed to other duties; on September 9th came the armistice and charge of its remnants was given to me; these consisted of 4 or 5 and their job was to keep a watch on such Italian communications as were permitted and the more interesting task of investigating old traffic which had never been deciphered with the aid of a number of old code books which had come into our hands; these were very incomplete as no condition had been imposed as to their surrender; this was because it was an armistice and not a complete surrender so we had the same situation as we had had in 1918. It was however a quite amusing job and gave me work to do which was really worth doing, a welcome change from the preceding period. The Spanish and French sections were also put under me so again I had plenty to do and as the staff transferred was mostly old staff of mine less the senior members who had got themselves more showy posts, I was happier than I had been for some time. It was nice to be doing cryptographic work again even if it was of very minor importance.


The Spanish section had carried on bravely with its rather monotonous task as the traffic was of no great import but it had been recognised [sic] that there was always a possibility of Franco taking sides against us and in this work continuity was of the utmost importance. Once the codes of a nation are put on one side as of no moment it is very difficult to pick up the threads. This had been shown in the case of the French; in 1937 we were reading their traffic with comparative ease but they then had recourse to a system which, though we found the system, was too hard to decipher on the very small amount of traffic intercepted. They found it too complicated and reverted to an easier one but no effort was made to deal with it except as a very part time job of one or two of the diplomatic staff. My staff which the Admiralty kept on reuseing the moment world affairs looked calmer, had their hands too full especially as in 1938, just before the Munich affair I had managed, thanks to Quex's intervention, I having bypassed Denniston, to start a German section. This sounds hardly credible but it is a fact. This section had a narrow escape from extinction, as Quex, impatient of the little information we had been able to get at that time sent a minute to Denniston, asking if he thought our work was going to be of any use in view of the increasing difficulty of foreign codes. Denniston always a pessimist said " No" but I wrote a very strong minute, taking the opposite view; in it I said that machines were in use and this type of ciphering would spread and that in war there was a good chance of their capture. This turned out to be an accurate prediction. When war broke out this work stopped but was resumed shortly after the collapse of France; I was not told of this though I had studied the subject for many years. The staff employed had practically no experience of French methods and made little progress and the section did not last long. Now in 1944 we started again when we had the advantage of dealing with material that was that of a fleet that was allied with us; the traffic was scanty and progress was slow though now the Americans were collaborating but we had seldom found them of great assistance; the staffs they employed were very much larger than ours but our smaller numbers generally produced better results.

But in the meantime we knew of the proposed landing in France and attention was naturally focused on this; it took place in June and success after success followed with only a very occasional setback. The end seemed very near. My mothers death in October upset me and within a fortnight my old trouble returned; it is curious to note that these attacks of mine seemed to have some strange connection with my mental state; an immediate operation was averted; I wanted badly to be in at the death but I had to be away for over two months during which the last desperate German offensive after some initial success was repulsed and Allied progress resumed. My prolonged absence had of course put me out of touch with things and it took me a long time to pick up the threads and Birch's Staff had never liked me and resented my occasional interferences, they treated me with scanty courtesy, and took every opportunity to subject me to minor inconveniences.

V.E. day soon came. We had a thanksgiving service conducted by the local parson; I was deputed to make the arrangements as I had become a sort of office church warden when I arranged every Sunday for the conduct of the very nice little services which the padre, a charming person called Milne who could not do too much for us, always willing to arrange extra services at hours to suite our convenience. Some amusement was caused by Nigel de Grey reading a message to us from Travis whom he described as " Our beloved chief" a description I think Travis himself would have been the first to disclaim; able he was especially in working his way up to get the top job, pushing out Denniston than whom he was a much more efficient organiser [sic]; he was always very nice to me.


After this the work of Naval section really came to an end, after almost exactly 21 years of existence during 16 of which I had been its head and its founder; it had started with two or three members, never exceeded 30 before the war and finished up over 1000 strong; I sent a message of my thanks to Hope (N.S.L.), Rees and Miss White who had been with me in the early days, all three were now in London with the diplomatic show. Now two jobs remained, the first the future of the whole organisation and the second the preparation of a history.

The first was considered by a small committee on which there was a lamentable absence of those who had had experience of peace conditions which are so different from those of war; I gave evidence before it but my views carried little weight, I fear, though I never learnt what recommendations were made though I did last long enough to see some of the first steps in reorganisation with which I profoundly disagreed.

The second, as to which I can only speak of the Naval Section side, was complicated by the masses of captured documents which had fallen into hands. These were in the first place entrusted to a section which was quite unsuited for the task and my efforts to have the material divided into historical and cryptographical sections was only considered by of the junior Staff with no real knowledge either of history or cryptography and so remained without success. The task of writing the history of Naval section during the war was entrusted to the various existing subsections who apparently on the assumption that all had started from scratch in 1939 or even later; thus the identification of call signs, a very important job was stated to have begun with a Miss Linahan who had not had anything to do with it until very late in the war it had actually been very important from the very start of Naval Section in 1924. The Roman historian Livy says somewhere Semper enim quod postremum adjectum est, id rem totam videtur traxisse, a translation of which sage remark is "For the last stroke to a business seems to get the credit for the whole performance". Of course the question of a history on the lines of that in which I had assisted Birch in 1919 was under discussion and now (October 1954) nine years afterwards seems from the query to which I have referred in the opening lines of this chapter of events to be still in course of preparation was another matter for consideration and Birch at any rate hinted that I might be of use to him in this; he was given the job but I never heard any more about it; as it has turned out this is fortunate for me at least as I have enjoyed nearly nine years of peace in retirement. Travis informed me of the fact that I was to be retired on September 26th 1945, I received the formal notice from the F.O. on October 8th, a couple of days after I had made the acquaintance of Mr. Ernest Bevin, then Foreign Minister, at my London club of which somewhat surprisingly he had recently become a very popular member.

My days in G.C. & C.S., our official description, were numbered and on October 17th my old complaint flared up again and this time an operation was necessary; I left B.P. in a hurry that day and never went back; some of my papers were sent to me as they happened to be locked up in a box of which I had the key but most of them were taken by Birch and I shall never see them again; I expect most of them have either been destroyed or placed with the records of Room 40 O.B. which Birch and I among others so carefully put in order in 1919. Anyway they have gone and these reminiscences have had to be compiled without them which must be my excuse for such misstatements as have been made. I have tried to be fair even to my most bitter enemies to all of whom I must be grateful for nine years of peace.

Commenced October 24th 1954.

Finished October 28th 1954.


Why Was Safford Pessimistic About Breaking the German Enigma Cipher Machine in 1942?

Louis Kruh

ADDRESS: 17 Alfred Road W., Merrick NY 11566 USA.

ABSTRACT: In 1942, Captain Lawrence F. Safford doubted that the German Enigma would be broken. This implied that he was unaware that the British had already broken the Enigma Cipher. Various authorities and colleagues of Safford speculate on his assertion.

KEYWORDS: Safford, Enigma, Weeks, Currier, Rosen, Sinkov, Rowlett, Kahn.

This article is reprinted, with permission, from the Winter 1988 (Volume 9, Number 2, pp. 3, 11), issue of Cryptolog. Cryptolog is a publication of the Naval Cryptologic Veterans Association, 3421 Stack St., Eugene OR 97404 USA.

In a memorandum dated March 18, 1942, Captain Lawrence F. Safford wrote, "Our prospects of ever breaking the German 'Enigma' cipher machine are rather poor." (See section 2 in the complete memo which follows.)

Safford's assertion implies that he was not aware that the British had solved German Enigma ciphers. But, he wrote this memo a year after Navy Lieutenant Robert H. Weeks and Ensign Prescott Currier along with Captain Abraham Sinkov and Lieutenant Leo Rosen of the Signal Intelligence Service had visited Britain's Government Code and Cipher School at Bletchley Park.

Some authors have speculated that the British may not have informed the American mission of the extent of their success with the Enigma.[1, p.312] Also, the April 11, 1941 report prepared by Sinkov and Rosen on their trip did not mention the Enigma. (Information is unavailable on any Weeks-Currier report.) The Signal Security Agency official history, however, points out that for security reasons the omission of Enigma was deliberate. The British had obtained a special pledge of secrecy from the American officers who agreed to reveal what they learned to just three persons: the Assistant Chief of Staff, G-2; the Chief, SIS; and Mr. William F. Friedman. Only then did the British disclose "the existence of their 'E' operations (a successful attack upon traffic enciphered by the German high- security cryptographic machine known as the Enigma . . .)."[2, pp.12-13]

Op-20-Q MEMORANDUM FOR OP-20 March 18, 1942

Subject: Security of information obtained from enemy communications.

1. During World War I (1914-18), the British Admiralty was very successful in reading German naval messages and tracking German submarines by D/F. This was a well-guarded secret during the war. After the war, the British let down the bars of secrecy and told the world what they had done and how good they were. Apparently, it never occurred to the British that the Germans would profit by these revelations.
2. In World War II, the German ciphers and communication procedure are such that the information obtainable by radio is substantially zero. With the exception of "weather codes" and certain other "minor systems" the only German naval messages which have been read have been the result of captures. Our prospects of ever breaking the German "Enigma" cipher machine are rather poor.
3. We are much more fortunately situated with regard to the Japanese Navy. Our solution of Japanese naval systems began back in 1925. We are able to read most of the messages in the most important Japanese naval ciphering system and have completely solved one of the secondary systems. This is the result of sixteen years of unremitting effort along these lines. We are also getting considerable information by direction finding. We are faced with constant danger that news of our success will leak out, whereupon the Japanese will adopt steps which will nullify our efforts.
4. In 1938 and 1939, the Japanese Navy used a system which we have never succeeded in solving. They abandoned this system in 1940 and adopted a system which we can solve. The reason for abandoning the secure system is not known, but we surmise it was due to its slowness and complexity. There is nothing that will stop their return to it if they find out that we are reading their messages.
5. What is even worse, they may adopt the German Navy cipher machine if news of our success leaks out. If this happens, our source of information will be reduced to direction finding and traffic analysis.
6. This memorandum is submitted for the purpose of impressing upon all concerned that we must not compromise our one sure source of enemy information by a too wide dissemination of information.
Respectfully,
 | L. F. Safford,
 | Captain, U. S. Navy, Op-20-Q.

Presumably, Weeks and Currier made similar pledges — the Signal Security Agency history does not discuss Navy details — but surely Safford, head of the Navy's cryptographic unit, OP-20-G, would be among the Navy people permitted to receive the information. However, if Safford knew about British success with Enigma ciphers, what is the explanation for his seemingly uninformed comment.

Frank B. Rowlett, a key member of SIS who knew Safford, was asked for his opinion on the apparent discrepancy. He wrote,

I do not consider it possible that Safford was not aware of the British success with the Enigma at the time he wrote the memorandom in 1942. As you know, an American team of cryptologists visited England in 1941 and were told of the UK work on the Enigma intercepts. I do not see how the information on the German Enigma brought back by Weeks and Currier could have been kept from Safford, considering the position he held in the Navy cryptologic organization.

One possible reason for Safford implying that the Enigma had not been broken is perhaps best described by the word "disinformation." The subject of Safford's memo was "Secrecy of information obtained from enemy communications" and what better way to protect Enigma derived intelligence then to highlight dismal prospects of ever breaking Enigma ciphers. Or as Winston Churchill said in 1943, "In war-time, truth is so precious that she should always be attended by a bodyguard of lies."[3, p.338]


Another view comes from David Kahn, author of The Codebreakers and the leading historian on cryptologic matters. He says that Safford was referring only to the Naval Enigma, which had been decrypted with the assistance of clues gathered from captured U-boats and other German ships. Kahn also suggests that Safford may have been alluding specifically to the then current inability to crack the new 4-rotor Enigma Shark cipher which was introduced 1 February 1942. (It was not solved until early December 1942 with the help of a codebook taken from the captured U-559.)[4, pp.749-750]

The opinions of naval officers who had worked with Safford were also solicited. None of the replies had a definitive answer but two of them had some pertinent information.

A retired Lieutenant Commander wrote:

Prior to the Pearl Harbor incident, Captain Safford, though chiefly concerned with the Japanese problem, did devote considerable time with the embryonic German Naval section, discussing at length with Mrs. Driscoll matters associated with the German Naval problem.

Around July or August 1940, the section was comprised of four persons. Since Mrs. Driscoll was severely handicapped in walking because of an earlier auto accident, it was my job to relay the various requests and questions directly to Captain Safford - this occurred frequently and I spent much time in his office.

Such was the climate of our operations until either late 1940 or early 1941, when Commander Wenger became our boss and we no longer had contact with Safford. I must stress that I am not certain of these dates - it could have been later, but I do clearly recall that this shift did take place.

By March 1942 (the date of the memorandom in question) the German Naval section had expanded considerably.

It may have been around June 1941 when the first of the English visited us - and by the fall of 1941, plans had already started for the development of the U.S. Navy version of "Bombe". Again, I'm pretty vague as to exact months.

I have no knowledge as to why Captain Safford made the statement you quoted from the memorandom - but one may draw some conclusions:

1. Since he had been out of touch with the German section since either late 1940 or early 1941, he may not have been up to date in the progression of things since the development of Bombe plans in mid 1941, though he was aware of the information that Weeks and Currier brought back from England.
2. The memorandom was written in March 1942, 3 months after Pearl Harbor. Things were in a great state of flux, and the matter of the Winds message must have placed a great stress and inner turmoil on his over-worried mind, so that he may have overlooked or forgotten the actual facts.

Another retired officer explained that "My personal knowledge of the naval Enigma was in February and March 1941. At the time the British had several copies of the machine and sets of rotors and umkehrwalz, some of which, at least, had been taken in the Lofoten raid. They had only the key/starting setting material for a few days, and their big problem was trying to solve the coding of each message's starting settings. As you are aware from the extensive literature on the Enigma, they apparently did solve this, but I do not know when."

In regard to the Currier-Weeks trip to Bletchley Park, he said they returned with,

…several days' settings and a "paper" Enigma, with all the rotor and umkehrwalz laid out. With this material we were able to decipher messages taken from our intercept logs of the days covered. Captain Safford's "except by capture" seems to bear this out. What the exact situation was by March of 1942, I do not know. I feel sure he would have known of any recent successes in the matter.

One final conjecture is that Safford's pessimism was based on the apparently remote possibility of a purely cryptanalytical solution of the Naval Enigma cipher in contrast to one assisted by capture of rotors, key lists, etc., which had been the situation up to that time.

Supporting or contrary views are welcome.

REFERENCES

1. Bamford, J. 1982. The Puzzle Palace. Boston: Houghton Mifflin Co.
2. U.S. Army Security Agency, History of the Signal Security Agency, Vol. Two. 1947. The General Cryptanalytic Problems. Washington, D.C.: Army Security Agency.
3. Churchill, W. 1949. The Second World War, Vol. V. Boston: Houghton Mifflin Co.
4. Hinsley, F. H., et al. 1981. British Intelligence in the Second World War, Vol. II. London: HMSO.

BIOGRAPHICAL SKETCH

Louis Kruh is an indefatigable researcher and collector of cryptologic memorabilia, particularly cipher devices. His interest in cryptology spans more than 40 years and he responds knowledgeably and promptly to all queries regarding literature and equipment. He is co-author with Dr. C. A. Deavours, of Machine Cryptography and Modern Cryptanalysis, (Artech House, Inc. 1985). He retired as Director of Advertising for New York Telephone in June 1990 and he is setting about practicing the law he studied.


S. I. S./CB

Henry D Ephron

ADDRESS: 800 Woodworth Ave., Missoula MT 59801 USA.

ABSTRACT: From Fort Monmouth to Vint Hill Farms to SWPA: S.I.S./CB, Brisbane (Col. Abraham Sinkov, CO).

An introduction to the Japanese method of superenciphermant, by means of additives, of their encoded messages and a detailed description of some of our (successful) efforts at solving this superencipherment — by means of step-by-step solution of an imaginary worksheet. Later the Japanese improved the security of their communications, superenciphering by passing the digits through a numerical square (a 10x10 table) instead of by mere addition. Encipherment is demonstrated in detail by means of a step- by-step solution of an English encoded tect superenciphmered by this method.

KEYWORDS: Japanese, codegroups, superencipherment, additives, encicode groups, cryptanalysis, dendai, redundancies, shift to squares (10x10 tables), solving the square.

PART I

It was the fateful day of May 7, 1942 that a dozen American SIS men arrived in Melbourne, Australia, and joined a handful of Aussies working in a beautiful mansion in a fashionable suburb under the command of Captain A. W. Sanford, A.I.F. (that is, Central Bureau, just activated, April 15, 1942). Two of the four American officers stayed with the outfit: (then) Captain Harry L. Clark and Lt. Charles E. Girhard. The enlisted men were Cpl. Joseph Fitzpatrick and seven Pfcs: Lester Truex, Tris Johnson, Leo Tepperwein, DeWitt Barnett, Chet Young, William Estep, and John Dalas. Who would have guessed then that from this bit of seed would grow and flourish an outfit of well over two thousand officers and men (including WACS), generously sprinkled with awards, citations, and promotions, and including even the Legion of Merit?

Major Abraham Sinkov (later Colonel) arrived to take charge of the entire outfit, with a few others, on July 1, and a few more personnel reached the southern port on July 22. It would be presumptuous on my part to attempt to point out the genius of Col. Sinkov. Whether as a cryptanalyst or as a Commanding Officer, this scholar and mathematician seemed able to keep a finger on every facet of the work that went on in the outfit from the interception of the radio communiques, through the collateral channels and collateral information and the grinding by the IBM Business Machines which were the predecessors of the computer, to the individual efforts and results of the personnel engaged in various cryptanalytic activities. I will merely say that I considered myself and my colleagues lucky to have him at the head of the outfit. To attempt to comment on and name the personnel as the outfit grew would be an impossible task, of course. Perhaps, however, I should mention T/4 Joseph E. Richard whose genius made the first successful cryptanalytic break in the continual, agonizing efforts of the group and finally started the outfit off to happier results. For which Joe, as a deserved reward, became Chief Warrant Officer Richard. At the other extreme I cannot refrain from mentioning, without naming him, one permanent army man who, when it became possible for men over a certain age to ask for a discharge, immediately did, saying to me, "I didn't join the army to fight in any wars."

According to an article in the Sigaffe Clamor (October 14, 1944) the "Melbourne idyll" ended with the move to Brisbane on September 25, 1942, a move "greeted by no less a personage than 'Tokyo Rose' from Radio Tokyo. 'So you have moved to Brisbane,' she said. 'We'll be over soon to visit you with bombers.' We are still waiting, lady."1

1For two years the SIS organization had been shunted from pillar to post before it was allowed to join the army as a separate organization with its own quarters (July 25, 1944): the Signal Intelligence Service, United States Armed Forces in the Far East (USAFFE). Permission was then granted to publish the Sigaffe Clamor twice monthly: "by and for the personnel of Signal Intelligence Service (USAFFE), Col. A. Sinkov, commanding." Of particular interest was this additional sentence: "The Clamor may be mailed home." It was about this time that the USAFFE censorship regulations changed our official address from "Somewhere in Australia" to "Brisbane, Queensland." The information I have given here can be found in greater detail in various issues of the Clamor and/or the S.I.S. Record, which was published in 1946 with the added longer title, Signal Intelligence Service in the Far East 1942-1946, "An Historical and Pictorial Record." I have checked details with these publications as far as possible.

It was not until sometime in 1943 that the small select group which included myself joined the outfit, but because it happened before the great increase in size began, we too were always numbered among the anasazi, "the ancient ones," and were included in the picture taken at the (original Brisbane) Henry Street Headquarters. I have used the word "select" advisedly, but since we are dealing with the army way of doing things, the word is not necessarily used here in an ameliorative sense. To appreciate the irony of our becoming members of this excellent outfit for the duration it is necessary to point out that we were indeed carefully selected, but not, I repeat "not," for this purpose.


I had enlisted in 1942 at the urging of the Office of the Chief Signal Officer, War Department, Washington ("The Chief Signal Officer directs me to advise that * * * *"), with an assignment at his request from The Adjutant General to the Cryptographic Division of the Signal Corps School at Fort Monmouth, New Jersey (actually, in plain language, the school for Cryptanalysis). It was my understanding that the purpose was for further, efficient training in the art, but, of course, I had not yet been initiated into army ways and "Catch 22." When the school moved on October 5, 1942 to its new quarters in the new army camp at Vint Hill Farms in Virginia, we found that the attitude of those in charge was that the school was a leftover of the old days and no longer of any value. For the most part grammar school dropouts were put in charge, apparently with instructions to put those "Ph.D.s" in their place. It was an extremely unfair situation to put this practically-illiterate cadre (they couldn't even count correctly), up against some of the army's most brilliant and educated personnel. It took the school about a month to get rid of all these men and replace them with its own staff. However, I myself heard the C.O. of the whole camp, speaking to another officer, say that since cryptanalysis was passe and the school really served no purpose, we were to serve as KPs and do guard duty for the much larger radio school. This practice was carried to the point where on one occasion of which I happened to be aware, someone came into a German class and took the instructor away to put him on KP! Fortunately for us the army, playing its usual game of musical chairs with its officers, soon replaced its unfortunate choice of a C.O.

Although I was on the staff technically as a "teacher," I also ranked as a student-no matter what your duties or rank you continued your studies during whatever hours were available to you. Along with other "students" I was given a refresher course in German. There were classes in various European languages as well as a class in Japanese. In early 1943 a request came to the school from Brigadier General Akin, Chief Signal Officer, SWPA, to "select six of your best men." We were to join five enlisted men from Arlington Hall and be attached as a group of eleven men (and four officers) to the CSO himself. The experience became another great lesson in the army way of doing things: "Hurry up and wait!" It was to be done "pronto," for we would in fact be flown across the Pacific, almost unheard of then for enlisted men in view of the shortage of space even on one of the cargo planes (C-47), such as we were crammed into along with a number of officers. But before that we were relieved of all duties and spent more than two weeks doing such things as running the obstacle course, hiking, policing the area, and other busywork.

I noted with puzzlement that all six of the men chosen were from those who had had refresher or cram courses in European languages. I did not know that we were slated for the Philippine underground, and Military Intelligence on one of its periodic visits looking for personnel had promised me that they would not insist if I preferred to remain in the work I was doing-although they had the clout to take me if they wanted to. I therefore had no objection to being one of those "selected," but out of curiosity, I naively asked why "they" were sending men from the European language classes to SWPA instead of taking them from the Japanese language class. The answer was that they had a good Japanese class going and didn't want to do anything to disturb it. It made sense of a sort-but I learned afterward that the men who had studied Japanese in that class all ended up in the European theater.

There were in the school at this time three men who were considered nuisances, silly, undependable, "gold-brick," or "Good Soldier Schweik" types. All three were selected for inclusion in our group of six, and thus in one fell stroke the school got rid of the three men who were considered the greatest detriment to the school. Poor fellows! In the field they got short shrift. Anybody for the front lines?

Of course, the destiny of our own little group lay forward of the front lines, for once inside the Philippines we would presumably not have been heard from again unless and until our forces successfully retrieved the Islands at some future date. And even then, of course, it would have been "iffy." However, it was fated that I should remain in cryptanalysis, even if it was just by luck. I've always insisted that regardless of what you have enlisted for and regardless of what the army has promised, you must depend on luck to get into and to remain in the field you wanted. Had a call come in for a large number of permanent KPs, I have no doubt that my "spec" number would have been changed to that of a permanent KP. You do not tell the general that you are unable to fill his order.

When I arrived at the School at Fort Monmouth with my assignment from The Adjutant General I found that insofar as the School was concerned, it was a worthless piece of paper and I was immediately shipped off to basic training without any promise of being accepted. The School made its own decisions as to whom to accept. One of my friends, an excellent professional photographer enlisted in the Signal Corps for photography. Once in the Corps he too was sent to basic training and from there was picked for cryptanalysis school just as I was-and a number of others-willy-nilly.

What happened was that after hurrying up and waiting, and then hurrying up again by flying across the Pacific Ocean, we found it necessary to wait again. The Chief Signal Officer, SWPA, Brigadier General S. B. Akin, was not there to welcome us. He had returned to Washington in what proved a successful effort to get himself another star. Meanwhile the Base Commander, who had been desperately trying to get a few additional men for Major Sinkov's hard-pressed little unit on Henry Street, found himself with an unexpected gift, a group of eleven (trained) men. He accepted five of the group for About-to-be-Major General Akin's special project and sent the other six to Major Sinkov without ever mentioning what he had done to the CSO. Four of us fitted happily into the SIS outfit, and thus, purely by luck, I ended up doing the very work I had enlisted for. Meanwhile the other five, having hurried up, began their next wait.


We found a small but efficient, well-integrated group of enlisted men and officers working side by side, and enthusiasm making up for experience, it was not long before we were helping them quite effectively in what they were doing. They were working on one of the Japanese systems mentioned by Kahn in his chapter on "The Scrutable Orientals."2 The messages, which, like all radio-sent material, were available to anyone who wished to intercept them, were encoded with four-figure code groups which were then superenciphered by means of four-figure additive groups.3

2David Kahn, The Codebreakers (The Macmillan Company, 1969), p. 587 (Kahn).
3I shall go into greater detail in this article, but I touched upon the subject in "An American Cryptanalyst in Australia," The Australian Rationalist Quarterly, No. 2, Apr-May-Jun 1980, pp. 11 f., and No. 9, Jan-Feb-Mar 1982, p. 7; also Cryptologia, Vol. IX, October 1985, pp. 337 ff. (Crypt-Australia).

Although four-figure groups allowed for as many as ten thousand code groups, the Japanese knew as well as we did that the frequent repetition of many of the groups used, especially those necessary under war conditions, would allow our cryptanalysts easy entry into their secret code. Kahn, on page 402, explains the use of such additives and demonstrates how effectively they conceal the repetitions of the plaintext codegroups. As in the case of the codegroups the four-figure numerical additives allowed for a series of 10,000 different groups without a single repeat.4 All the cryptographer had to do was to write the additives in order over the codegroups of his message and add each pair of groups without writing or carrying the tens digits (e.g., 5986 + 7259 = 2135): a simple task and very effective.

4Joseph E. Richard, mentioned earlier, reminded me in a personal letter dated 1 February 1988, after reading my Australian article in Cryptologia (see note 3), that additive books were composed of several books each of 10,000 such groups: "A few had only 3 books, i.e., 30,000, but most had 5 books, i.e., 50,000."

Before our decoders could even begin to try to solve the meanings of the codegroups we had to solve these keys, i.e., the additives, and subtract them from the four-figure encicode groups of the message. And before we could even begin to think of trying to do that we had to find a number of instances of consecutive text enciphered by the same string of additives. Such instances of text were typed on extra-wide sheets of paper with the coinciding groups underneath each other so that all groups superenciphered by the same additive were in the same column. The rest of the texts of the messages were also written in before and/or after these groups so arranged. Messages incorrectly placed wasted time but did no further harm-they merely yielded no results.

On my first morning with the outfit, my personal secret came out: I could type! Apparently in the army, if on a typing test you did forty words a minute, you were rated a typist, but at the crypt school at Fort Monmouth where you were tested not on words but on code groups, a mere twenty such (errorless) groups of five characters per minute gave you the rating of a typist. I was immediately placed in front of a typewriter with the longest roller I had ever seen and asked to prepare the superwide worksheets of anywhere from five to ten messages per sheet.

Officially I did that for several days, but actually I spent most of my time learning how this superencipherment worked and how others went about seeking to extract the hidden additives and the meaning of the codegroups. Since none of our little group had had any training at all in Japanese, the decoding aspect of the work was not for us. Instead we concentrated on the cryptanalysis of the superencipherment, for which a knowledge of the Japanese language was not a requisite, although I personally believe that the ability to think like a Nipponese would have been of considerable help in the cryptanalysis. Only rarely did we use terms in their language, but rather a rough translation of the meaning in English. For example, although we did say and write such expressions as den dai, roughly "Message No.," we did not bother putting the number into Japanese, saying and writing instead, "one, two"-rather than ichi, ni.

We used every military term we could think of as probable words in our decipherment attempts, as well as collateral information concerning events which might affect the content of individual messages, such as, for example, the fact that Okinawa had just been bombed about the time we intercepted a flurry of messages radioed from there. One might then look for words like, "planes" or "pilots," and even such face-saving expressions as "damage slight," or "damage none," which were almost mandatory. But identification was absolutely mandatory for the benefit of the recipient, and as I got into the work I found that of particular interest and help.5 Whatever Col. Sinkov's reason for choosing me for the task shortly after I arrived, it was he that saw the possibilities in having a complete record of all dendai-information as it became available from the intercepted messages. Keeping that record became my highest priority, and I was permitted for a time to work swing shift all by myself in order to be able to bring the dendai-record book up to date during hours when no one was using the worksheets.

5See Kahn, p. 753, on the sliding ratio between the element of time and need for security, and especially, on p. 587, on how disastrous a special attempt at extra secrecy by the Japanese actually turned out for them. Even with the relatively simple methods used in the cryptosystem we were working on, the intended recipients of the messages (cryptographers) must often have found their decipherment difficult or even impossible, in whole or in part, because of errors in encoding, in enciphering, in transmission, in reception—not to mention mere typographical errors.


As I have written elsewhere,6 in those pre-computer days I became a sort of living computer with the information stored in my memory bank, and some of my co-workers, instead of looking up the information in the record book would call out to me, e.g., "Dendai, I've got Blank Shipping on _____," and ask for the dendais on that date. If the information was available, I would give it to them; for example, 125 to 127. Putting the correct number into the text (by digits) gave us two or three more additives to add to the string already recorded.7 With his Australian fondness for Gilbert and Sullivan, an Aussie cobber from CB (Central Bureau), punning on Deadeye Dick, dubbed me Dendai Dick.

6Crypt-Australia, 1980, p. 12; 1985, p. 340.
7This was important because, if you obtained an additive which was one of a known string, you could try out the remainder of the string in the adjacent columns in the hope that the additive was from the same book of ten thousand. Thus a good deal more of the messages on the worksheet could be (at least) deciphered.

To illustrate in a bit of detail how we worked with such a worksheet I have created a portion of an imaginary sheet, using words, phrases, and kana strings typical of those we encountered in our work, such as bu ra tsu ku ui do (= "black widow"), "[bi ta mi n B]" (= "vitamin B"), (Blank) "Shipping Company den dai" (+ numerals), "to be cont.," "the end," and so forth. (See Figure 1.) Assume that we are at an early stage of our work and that the majority of the plaincode groups cannot be translated even after the additive has been removed, since we did work under this handicap during the period I am describing. Even they could help, however, if they were groups that had been met several times before, for obtaining recognizable groups could show that we were on the right track. I will refer to them as "friendly" groups. Note that the messages in lines 6 and 8 begin in the middle of this portion of the worksheet and those in lines 3 and 7 end in the middle. This phenomenon was a constant on a full sized worksheet, and because of it only a limited number of columns were likely to have any real depth to work with.

* Fig.
* Fig. Figure 1. Text superenciphered by means of additives.

It must be understood that I have deliberately brought together on one portion of a worksheet too many redundancies, and so forth. In actual practice one would have had to work on several worksheets to find so many obviously helpful codegroups. However, the time was to come when we were able to follow the travels of a frustrated Japanese general for six months from island to island. He had been ordered to report to Tokyo, but because of the sad shape of the Imperial Navy he was finding it virtually impossible to get there. After he had been marooned on an island for a time, a submarine might arrive and he would get aboard and travel to another island, but at the end of the six months of vicissitudes of which I was personally aware from the deciphered messages he had not yet succeeded in reaching Tokyo.

To help the reader follow the demonstration of the cryptanalysis, I give a small list of (assumed) plaincode groups whose meanings have already been decoded. By assumption no plaincode group used in these messages, which is not in the list, has been decoded. For realism a few undecoded groups of more frequent occurrence ("friendly" groups) have also been assumed.

LIST OF PLAINCODE GROUPS
0068 in | 2865 (Eng.) B | 5565 colon (:) | 7959 two | friendly groups
0140 three |  | 5752 box | 7984 began | 1234
0229 daybreak | 3443 the end | 5878 nine |  | 1597
0631 KU | 3661 four | 5939 DO | 8166 boat | 2148
0978 RA | 3871 none | 5948 BI | 8216 (kana) N | 3701
 | 3974 DAI |  | 8403 zero | 3851
1357 part |  | 6170 metal | 8437 ] | 6489
1648 damage | 4003 general | 6295 six | 8570 attacked | 6942
1713 five | 4012 Kyoto | 6530 planes | 8651 TSU | 7340
1726 and | 4241 DEN | 6711 TA | 8686 HQ | 8250
1733 bombed | 4412 MI |  |  | 8639
 | 4504 seven | 7315 [ | 9101 company | 8756
2036 eight | 4927 shipping | 7324 injured | 9122 officer | 9165
2332 to be cont. | 4942 water | 7409 at | 9213 codebook
2457 slight |  | 7658 men | 9327 one
2486 locked | 5393 pilot | 7802 UI | 9705 sunk
2550 BU | 5439 killed

The worksheet has now been prepared, and I can have considerable confidence in the line-up of the messages considering the number of repeated groups it has yielded. Presumably, as a relative beginner, I work at first with various redundant and routine expressions which have yielded entry in the past, but somehow, for a long time nothing works. It often happens that the expert scans the fresh worksheet, casually tries several "hunches," and almost immediately succeeds in correctly placing that all-important first plaincode group, and the time would come when that could happen to me too. But, for whatever reason, that doesn't usually work for the beginner or the less gifted. Of the old-timer it may sometimes be said that he uses the sense of smell, or ESP, or just a common old-fashioned hunch, but never of the tyro. Somehow hunches do work, but even then it should be noted that they never work if you deliberately try to use them. A thought occurs to you unbidden, you try something and it works. You had a hunch! But try deliberately to use them, and nothing works. And at first nothing worked for me. No luck! The gifted expert, of course, creates his own luck.


To some extent it perhaps only seemed to take so very long. The tyro is easily discouraged. Sooner or later one of the words that should work, does work. In this case let us say that I finally try out the plaincode groups 3443, "the end," and 2332, "to be continued." I run "to be continued" through the first message, and lo and behold at the very end of line 1, I strike paydirt, and I learn how easily the feeling of depression can be dispelled. In line 1, column 14, 6395 - 2332 = 4063, which is then tried as additive for the whole of column 14. Line 2, 9087 - 4063 = 5024 (?). But line 4, 3164 - 4063 = 9101, plaincode for "company." Lines 5 and 8 yield 5654 and 7213, both unknown and therefore meaningless, but line 6, 8066 - 4063 = 4003, "general," and line 9, 8905 - 4063 = 4942, "water." Keep in mind (as I have indicated above) that we are not "carrying" in our addition and subtraction (for example, 6 + 4 = 0).

There was always a good deal of comment, both humorous and serious, between co-workers in the hutment, although each of us worked alone on a worksheet in his own table-space. This rather uninhibited conversation stimulated thought and prevented dullness from setting in, and along with an occasional friendly look over a shoulder at the results being obtained on a worksheet, often led to quite helpful suggestions. I myself, let us assume, looking at the three isolated words, company, general, and water, am wondering, "What next?" A co-worker, however, looking over my shoulder at this point suggests that I ought to try "shipping" in line 4, column 13, before the word "company." 0248 - 4927 gives me the possible additive 6321 and that in turn subtracted from 2851 in line 1, yields 6530, "planes," a good possibility. However, in the rest of the column the plaincode results yield no recognizable words. And now I am unable to proceed further with these words. I must begin the process over again of trying to break into the worksheet from scratch.

I begin trying DEN in line two (without results) but notice the shortness of line 3 (only five groups), and a streak of laziness overcomes me. I shift from line 2 to line 3, and my "silly" hunch pays off. Encicode 2305 minus plaincode 4241 yields the possible additive 8164. None of the rest of the column below yields anything except a "friendly" group in line 7: 4543 - 8164 = 6489, but the repeated group in lines 1 and 2 (4694 - 8164 = 6530) yields the word "planes," a corroborating coincidence for column 13, where line 1 also was deciphered (at least tentatively) to 6530, "planes." Suddenly the world looks brighter. I now have something to work with in adjacent columns. And we may now, with some confidence, inscribe the additive 8164 in column 1 along with the additives 6321 and 4063 in columns 13 and 14.

I enter DAI, 3974, after DEN, in line 3, column 2: 7512 - 3974 = 4648, the additive for column 2. In line 2, 2118 - 4648 = 8570, "attacked." In line 9, 2704 - 4648 = 8166, "boat" (note "water" in column 14, obtained earlier).

Presumably a number follows, but since I do not know the sender, my specialized work on "dendais" does not help me here. For the repeated group in column 3, 5792, we may begin by testing 9327, the numeral "one": 5792 - 9327 = 6475, which subtracted from 5597 in line 7, gives 9122, "officer," as well as 7409, "at," in line 2. As I have indicated, the army method of sending those trained in Japanese to the European Theatre and those in the European languages to the Southwest Pacific, forced us to work with English translations or rough approximations of the various groups' meanings. So "at" was good enough here, as was "Message No." for "dendai." At any rate, we can now enter the additive 6475 in column 3.

Assuming that the message number is not simply "one," I now try for another digit in column 4, beginning again with "one": 0476 - 9327 = 1159. In line 2, 1378 - 1159 yields 0229, "daybreak" (note, "planes attacked at daybreak"), and in line 8, 7860 - 1159 = 6711, kana "TA." With the aid of IBM machines we had been able to list all words previously encountered spelled out in kana-in (English) alphabetical order by first syllable, as well as by second syllable, third, and so forth. We therefore now have the possibility of trying out all such listed words with TA in line 8, as well as the number we have been working on.

Having entered the additive 1159 in column 4, let us assume a message number above one hundred for line 3. Trying the digits in order in column 5, when we reach "eight," we get 5552 - 2036 = 3526, which as the additive gives us the friendly group 2148 in lines 1 and 9, 0475 - 3526 = 7959, "two," in line 4, and more importantly as it turned out, 7938 - 3526 = 4412, kana "MI," in line 8. We may enter 3526 with full confidence as the additive in column 5.

With TA and MI in columns 4 and 5 we find only one listed word to test in the IBM alphabetical run-off: [BI-TA-MI-N B], "vitamin B," giving us, if correct, kana "N" (8216) in column 6, the English letter "B" (2865) in 7, and the closing bracket, "]" (8437) in 8. In column 6, "N" gives us "men" in line 4 after "two," and "HQ" in line 5. We may therefore inscribe the additive 7215 in column 6.


"B" (2865) in column 7 yields 5393, "pilot," in line 1, 5439, "killed" (after "men"), in line 4, 1733, "bombed" (after "HQ"), in line 5, and 9213, "codebook," in line 9, and we may enter the additive 2724 for column 7.

"]" (8437) in line 8 yields 7324, "injured" (after "pilot") in line 1, the friendly group 6942 in line 4, 2550, kana "BU" in line 7, and we are warranted in entering the additive 5296 in column 8. For kana "BU" the IBM alphabetical list gives as one possibility "BU-RA-TSU-KU UI-DO," "Black Widow," the name of the plane, which seems like the best choice.

Kana "RA" in column 9 yields the friendly group 7340 in line 4, and in line 9, 6170, which has been assumed to be the plaincode for "metal," perhaps insufficient reason for confidence. We try TSU in column 10 and again get a friendly group, 8250, in line 1, and 5752 in line 9, a possible group for "box." More careful consideration would add to our confidence, but for the moment we try KU in column 11, the last group in that message, hoping for stronger corroboration. (As for its being the last group in the message, please remember that these messages have been cut in two at some arbitrarily-chosen intermediate point, and that then the end has been placed before the beginning.)

Placement of KU gives us 1648, "damage," for the repeated group, 6470, in lines 5 and 6, an encouraging result since the word is frequently encountered in these messages because of the importance of face-saving in the Nipponese culture. I first try 3871, "none," in column 12, line 5, without satisfactory results, but when I try it in line 6, I get 2457, "slight," in line 5: "Damage none" in one message and "Damage slight" in the other.

Returning to columns 9 and 10, I find that my results for the two columns when considered together were quite corroborative of my assumptions, for we have a metal box in line 9 associated with the plaincode group 9213, "codebook" in column 7, altogether an interesting and suggestive combination. We may enter the additives 2530, 9353, 5832, and 1406 in columns 9-12 and thus complete this mini-illustration of our work in the field with the additives. But I must repeat my cautionary reminder that in actual practice, in the early days of the work, we met so much success only working with a number of worksheets-not on a single one, and certainly not on only part of one.

Once in the field when this context of a codebook locked in a metal box appeared in a message, it turned out that the boat (see line 9, column 2) was on fire and doomed and that the message was intended to inform the authorities that the codebook had been placed in a metal box which was both locked and carefully bound before being thrown overboard. For like ourselves the Nipponese knew that the best way to solve a code was to get hold of the codebook one way or another! By placing it in a strong metal box, binding the box and locking it, and then throwing it overboard in very deep water, they were able to assure themselves that we could not possibly get our hands on that copy of the codebook.

Conversely, since we knew as well as they did that the most successful method of solving a code was to get hold of the codebook, we actually had in the hutment in which I worked at Ascot Park, a large map of the area stuck with colored pins, each denoting a spot where we knew that a codebook had been sunk or where a ship had gone down which might have contained a codebook. Unfortunately the ocean in the area was much too deep as a rule for any of our "frogs" to be able to go down and bring up one of these codebooks. As a rule! But I do remember a conference with the Chief Signal Officer, SWPA, in which we studied all these pins and their locations and found one where the water was only a little too deep for a diver. The CSO pointed to it and said, a bit dramatically to be sure, "We'll get that one." And a diver did go down in a successful bid for that codebook, which he brought up locked inside the metal box as described!

It turned out that even though the "fireproof" box had kept the code and additive books from burning up in the fire that sank the ship, the intense heat of the flames had charred the pages into solid black sheets on which not a thing was visible. The technology, of course, had long been available for bringing the print on such pages back automatically, and one of our technicians demonstrated how with a camera (infra red, I presume) he could photograph an entire page and bring the contents out quite clearly. However, we did it painstakingly by rubbing the "now-you-see-it-now-you-don't" print with a piece of cotton soaked in an alcoholic liquid. I remember doing my share of this work to some of the additive pages, one group at a time. As long as the spot was wet the numerals remained quite clear, although they became invisible again quickly enough-but not before I had called them off to a T-5, who recorded them in the additive book we were re-creating.


PART II

As I have pointed out, the Nipponese knew that we would be making every effort to read their (radio) mail and did what they thought necessary to keep us from being able to do so. At some point they had begun to realize that we were being successful in our efforts, that the superencipherment by means of additives was insufficient to render us impotent. The "Yanks" had turned out to be a harder nut to crack than they had expected and, it appeared, smarter too! They would have to work out a more difficult and safer system-and they did. We became aware of this only when the additive system suddenly no longer worked for us.

The additive books were still being used for the superencipherment, it turned out, but the digits of the numerical groups were now being passed through a square, through a sort of numerical Vigenere. (See Figure 2.) Each row had all ten digits arranged in random order (if we may take the liberty of calling zero a digit). The rows and the columns were each numbered in order from zero to nine (or one to zero). The column numbers were the plaincode numerals, the row numbers those of the keys (the former additives), and the digits inside the square (at row-and-column intersection) the encicode numerals.

* Fig.
* Fig. Figure 2. Square used in encipherment.

It was a simple enough matter for the Japanese cryptographer to take a plaincode group, such as 8352, and encipher it with a keygroup, such as 9716, thus: row 9, column 8 = 6; row 7, column 3 = 9; row 1, column 5 = 9; row 6, column 2 = 6: encicode 6996. It was equally simple for him to decipher encicode 6996 with the key 9716, thus: 6 in row 9 = 8, 9 in row 7 = 3, 9 in row 1 = 5, 6 in row 6 = 2: plaincode 8352. However, it was much more of a nuisance for us working without the key to attempt to decipher the encicode, for as you can see by looking down the columns in Figure 2 almost every column has repeated digits, with the result that even assuming the correct plaincode group will not necessarily give you the key, but may rather give you several possible keys. If you look for the cipher digits 6, 9, 9, 6 in columns 8, 3, 5, 2 respectively, you will find two 9's in both columns 3 and 5, and have to work with the possible keygroups, 9316, 9366, 9766, and 9716. Often there were many more choices than that.

The technique involved in the cryptanalysis remained essentially the same as I described it with the additives, except for this new method of finding and using the key. The use of the square did not stop us although it was a nuisance which did slow us down considerably. In addition to that, the enemy had foisted upon us the additional task of solving the square itself before we could work with it. And showing that they knew well enough what we were up to, they began to introduce new squares at a fast pace.

I did not myself ever work on the solving of the squares themselves but did spend a great deal of time with them deciphering messages and extracting keys. There is nothing additional I can offer of actual material from the war period, and it would take a whole cryptanalytic outfit to demonstrate methods of solution working with several books of 10,000 keys and a 10,000 group codebook. I do, however, want to demonstrate, insofar as I can, the decipherment and cryptanalysis involved in working with squares of this sort. I think I can illustrate that with a simple problem based on one which someone else prepared for me, one simple enough so that I was able to solve it.

In 1946, with my World War II military service and cryptanalytic activity well behind me, I met someone in Chicago whom I scarcely knew, but who seemed to know me well enough, and who had been engaged in similar work. He sent me some humorous (English) encicode problems he had constructed using numerical code groups superenciphered in the manner I have described. I left Chicago shortly afterward and never heard from him again. Included were some explanatory materials and a quasi-code book of perhaps a thousand numerical groups with meanings he had assigned them.

Insofar as these problems in English used additives, they were quite simple to solve. Of two problems, however, using squares for the superencipherment, I solved only one, which, I must admit, was made easier for me because I knew what sort of thing to expect from my earlier solutions of the simpler problems. This problem is the only one I have been able to locate—nor have I found the quasi-code book.

Almost a decade ago when I decided to discuss this work for the benefit of the American Cryptogram Association I studied this worksheet in an attempt to recall how I went about solving the problem. In the process I was able to reconstruct a part of the codebook, but after that I was too familiar with the worksheet to be able to demonstrate how one might go about solving the problem from scratch, since I would have been remembering the decipherment of many of the numerical groups. To illustrate a step at a time what might happen in attempting a solution I needed material I would not recognize. To obtain such material, the steps of whose solution might integrate with those of my successful solution in 1946, I re-enciphered the plaintext with modifications in the encoding as well as in the text itself, and, of course, with a totally new set of keys. And in order to keep the numerical groups meaningless, I enciphered only one of the digits at a time, as, for example, all the 9's, wherever I found them. In other words, I did not work at any time with complete groups, either key or code.


After the enciphering was complete, I copied off the encicode text of the messages on separate sheets, mixed them up, and put them away for several months. (See Figure 3.) Thus, before I could begin to re-solve the messages, I was faced with the task of lining them up correctly, as I had been in 1946, with all groups of each column under the same (though unknown) key. (See Figure 4, and note the repeated groups within the columns.) I then proceeded to re-solve the encicode messages without resorting to any of their special context—which I might have remembered. Instead of assuming the sort of text I had assumed for the Japanese example, of course, I assumed English terms, but only of the sort I had assumed in 1946, when I had begun with no idea of the special content. Undoubtedly it was an easier task this second time around, and perhaps that was fortunate for my special purpose, for which success was mandatory. It is interesting to note, however, that any memory I might have retained of the fifty keys I used would have proved absolutely worthless since insofar as I know there is no way to reproduce the original order of the rows of the square, and so any keys obtained by cryptanalysis would bear no resemblance whatever to the original keys used in the enciphering. (The same phenomenon, of course, held true for additives too as long as every bit of our knowledge of the system had been obtained by cryptanalysis and there had been no opportunity of correcting our original assumptions via an authentic [original] code or additive book.)

* Fig.
* Fig. Figure 3. The mixed bag. Ten parts of a message in encicode.

* Fig.
* Fig. Figure 4. The worksheet. The parts aligned under their keys.

I began then, as I had in 1946, by looking for some of the inevitable redundancies, for such common words as "the, and, of, to," and such grammatical endings as "-ing, -ly, -s, -d," etc. And although the bisection technique helped hide the redundancy of the beginning of each message, the weakness inherent in the use of groups for identification purposes proved as helpful to me here as they did for us in the Southwest Pacific during World War II.8

8See Crypt-Australia, 1980, p. 12, or 1985, p. 340. See also Kahn, s.v. "Redundancy."

It may be thought that working with a small codebook (about 1000 groups instead of 10,000) would make the problem simpler, and to some extent that is undoubtedly true. But I rendered the question moot in 1946 by not even working with the thousand. Instead I went through the list picking out the commoner "words" and prepared a list of only a few hundred. It was with this and even smaller lists that I worked, rather than with the thousand, looking for words most likely to appear in any text—especially during the earlier, more difficult stages, of the solution. And I lined up these smaller lists in numerical order by first, second, third, and final digits—for obvious reasons. For example, when I noted that in column 19, rows 1 and 10, I had 0168 and 0160, I wasted little time finding in my shorter list the excellent fit, plaincode 2882, "-ing" and 2888, "-ment." In column (C) 10, rows (R) 3, 4, and 5, I had 3357, 2350, 2357, and here too my list suggested 2882 and 2888 for the latter two, although it showed more than one possibility for the first (but did include the correct plaincode group). I merely made a note of this at the time, but I mention it because it turned out that "-ing" and "-ment" were correct both times.9

9In order to avoid confusion, since I will be referring constantly to the rows and columns of both the messages and the square, I will refer to the rows of the square as "keys" (symbol K) and the columns as plaincode (symbol PC). Rows (symbol R) and columns (symbol C) will refer to the aligned messages on the worksheet (Figure 4). The solved groups will also be referred to as keys (K) and plaincode (PC). EC will designate encicode (numeral) groups.

I will not waste time on the tentative assumptions I made which proved futile after I had lined up the ten messages as shown in Figure 4. What I sought to begin with were two (or more) assumptions which seemed to corroborate each other, and which would enable me to place two or more digits in the same row (K) and/or the same column (PC) of the (so far empty) square. I paid especial attention to columns containing repeated groups and this finally paid off in column (C) 45. Note 0650 in rows (R) 1 and 6 of C45, 6656 in R7, and 0040 in R4. Compare especially 6656 and 0040. I assumed that the key number in positions 1, 2, and 4 was identical and that one of these groups was plaincode (PC) 9999, "the." Experimenting with the three groups, I found that if 6656 were PC9999 and 0040 were PC2222, ":" (colon), 0650 would automatically become plaincode PC2992, "-ly." Curiously 6656 reappeared as a repeat in C25, R5 and 6. I assumed that here too we had PC9999, "the," and that since presumably no two key groups were identical, at least one key number must be different of the key groups of C25 and 45. The odds were, of course, 3 to 1 on the 6, and so I assumed a 6 in two rows (K) of the square in column (PC) 9 (see Figure 5A).

* Fig.
* Fig. Figure 5. Genesis of the square; first three rows.

Note carefully which are the key groups, which the encicode groups, and which the plaincode groups (column 45):

Key (K): | 2232 | 2232 | 2232
Encicode (EC): | 6656 | 0040 | 0650
Plaincode (PC): | 9999 | 2222 | 2992

Note again that the numbering of my rows is arbitrary. And since even if our assumptions are correct we do not know which 6 is in row (K) 1, we may write the key for C25 as 1/2 1/2 3 1/2, where the solidus or slash mark may be read as "or."

In C45, R9, we have 0919 = 2 x y x. With one digit known there were not many groups to fit the pattern in my high frequency list--actually seven all told: "the, colon, E, G, -ing, -(e)D, -ment," but with all plaincode 2's and 9's already used in C45 I could eliminate these four: "the, colon, -ing (2929), E (2525)." Since the two best possibilities (PC2585 = "(e)D" and PC2888 = "-ment") have a common 8 in the third position, I added 1 in K3 under PC8. Assuming PC2585 = "(e)D," I pencilled 9 lightly in K2 under PC5. I was, of course, at all times ready to erase an assumption as necessary and used my eraser freely.


In C25, R1 and 2 we have the same pattern: EC0686 and 2616, but this time we know (by assumption, at least) that the repeated 6's are PC9's: x9x9. I found six possibilities: 0909 = "begin," 4989 = "P," 9919 = "red," 2929, 1999, and 9999. The last three have PC2 or 9 in the third position (K3), and so may be eliminated. 1 in K3 is PC8 and so PC4989 is "P" if our previous assumptions are not at fault. That leaves the likely PC0909 = "begin," for EC0686 (or 9919 = "red"). In either case the first key number cannot be 2, for 0 in K2 = PC2. Therefore change the key for C25 to 1 1/2 3 1/2. So for PC4989, "P," I placed 2 in K1 under 4, and assuming 0909, "begin," for EC0686, I placed a lightly penciled 0 in K1 under 0, and 8 in K3 under 0. For the inclusion of these assumptions in the square see Figure 5B.

Since in 1946 I had found the search for "the" fruitful I had tried PC9999 in other places before meeting with the success already described. C11 and 15 contained the only double repetitions, EC0323 and 0179 respectively, and I had assumed 9999 for 0323 without success. I had, however, experimented with a (blank) square and filled in a number of cells based on the assumptions that 0323 (C11) = PC6505, "to," and 0179 (C15) = PC9999, "the." Returning to this square now, where EC0323 = PC6505, "to," had led to EC7324 = PC7502, "seven," in R7, corroborating my placement of the seventh part of the message, I found that I could rearrange the K-rows of this square to mesh with the other square without any conflicts.

I assumed, therefore, R2, C11 and 12, EC0323 5616, to be PC6505 9999, "to the," and R7, EC7324 4720, to be PC7502 2222, "seven, colon" (a standard redundancy), along with the assumption that the triple repeat in C15, EC0179, was PC9999, "the." Omitting a few of my riskier assumptions, and assigning the K-rows arbitrarily, this gave me the results shown in Figure 6A. Comparison with Figure 5B shows that this row K1 is our previous row K3, K4 is K2, and K2 is K1. Combining the two squares we obtain the square shown in Figure 6B.

* Fig.
* Fig. Figure 6. The square: first assumptions.

The new keys then are C11, K[5]323, C12, K3162, and C15, K4605. Comparison of EC1564 (C11, R8) = PCx992 and EC7171 (C15, R6) = PCx99x, shows that the assumption that in K5, EC1 = PC2, will give us PC2992 = "-ly" for both. We may add EC7 under 2 in K4 and EC1 under 2 in K5. I also assumed for EC6139 (C15, R5) = PCx9x9, PC0909, "begin." That added EC6 under 0 in K4, and EC3 under 0 in K0. We may also add the 0 under 6 in K5 and 2 under 0 in K2, both from our PC6505, "to," in C11, as well as 7 under 7 in K5 from our PC7502 = "seven" in R7.

In C11-12, R3, EC0122 2207 = PC680x x4xx, the assumption of EC2 = PC1 in K3 gives us PC6801 14xx, "when," followed by the possible excellent context PC1418, "ever," which I assumed, placing 0 under 1 in K6 and 7 under 8 in K2. For EC1787 (C12, R5) = 82x8, I assumed 8288, "possible," and placed 8 in K6 under 8. (See Figure 6C.)

For R4, C12, EC3023 = PC502x I confidently assumed "an" after "to." Usually three sure digits out of four provided an easy choice, but obviously not always, for "an" turned out to be wrong. I mention this failed assumption because of the confidence with which I assumed it. Actually I frequently had to try out and drop assumptions for which at the time I could not find sufficient corroboration, including some which later turned out to be correct. Note R7, C10, where I assumed EC6146 = PC4609, "part," preceding "seven," but dropped it when I found no way to place the assumed values in their K-rows even though EC2720 in C9 worked well for PC0909, "begin." "Begin part 7" was, of course, a type of redundancy which we found very helpful in the field during the war.

We have only one place where two consecutive groups are repeated, C29 and 30, R4 and 5, EC4557 1950. Experimenting with these two groups I found that if I assumed 1950 = 9999, "the," and the key for C30, K6534, EC2940 (C30, R2) became PC2929, "-ing." EC1974 = PC99xx (R6) compared with EC0164 = 12xx (R9) showed that if EC4 = PC6 in K4, the first could be PC9966, "without," and the second PC1276, "period," with the placement of 7 under 6 and 6 under 7 in K3, without creating a conflict in the now well-filled row K3. (Obviously the odds were that wrong assumptions would produce one or more conflicts in a well-filled K-row.)

In R10, for EC2427 = 2x12, I assumed PC2112, "his," and placed 4 in K5 under 1. At this point I had a bit of serendipity. Debating whether to assume that EC4126 = PCx210 (R3) = 5210, "situation," I tried the group preceding it in C29, EC0615, as 9999, "the," for "the situation," and found that this assumption gave me K4163 for C29, which in turn gave me PC2222, "colon," in R1 for EC7724, and PC2992, "-ly," in R8 for EC7614. In R2, preceding "-ing," I needed a verb for EC3009 = x01x and assumed PC7014, "love," which gave me 3 under 7 in K4, 9 under 4 in K3, without a conflict, and, of course, 0 under 3, completing the row K3.

I refrained from making a final decision on C29, R4, EC4557 = 6xx6 = PC6136?, "of," preceding "the", and on C30, R1, EC0581 = 1x0x = PC1001?, "according," R3, "situation," R7, EC2122 = 221x = PC2215?, "de," and R8, EC2001 = 263x = PC2631?, "pro-." Instead I went back to C9 and 10, R7, to the possible assumption for EC2720 6146 of PC0909 4609, "begin part," preceding "seven." As indicated earlier it was in C10, R3, 4, and 5 (EC3357, 2350, and 2357) that my list suggested PC2882 and 2888, "-ing" and "-ment." K2 in the fourth position (EC6 = PC9) gives EC0 = PC2 and EC7 = PC8, and so we can assume for R3, 4, and 5, PCx888, 2882, "-ing," and 2888, "-ment." EC2 = PC2 gives us K6 in position 1, and so we can place 6 under 4 in K6. With "-ment" eliminated, I assumed R3, EC3357 = PC6888, "haste," and placed 3 under 6 in K6.


I now had EC4257 = PC62x7 (C15, R10), for which I assumed PC6277, "on," and I placed 5 under 7 in K0. This gave me EC1250 = x276 in R1, and I assumed PC1276, "period," and placed 1 under 1 in K4. C10, R2, EC1057 = 90x8 became PC9088, "example," since EC5 = PC8, although not yet assigned to K7. I did now assign the third position to K7, placing 5 under 8 and from EC6146 = PC4609, "part," 4 under 0 in K7. For R10, EC9646 = x909, I assumed PC0909, "begin" (before "to"), and placed 9 under 0 in K6.

After C9, R7, "begin" (EC2720 = PC0909), I assumed K2024 for C9. Then R6, EC6221 = 9x01 = PC9801, "(e)D," yielded 2 under 8 in K0; R10, EC6556 = 97x0 = PC9740, "-ence," 5 under 4 in K2. R9, EC5222 = 480x = PC4805, "be," placed 2 under 5 in K4; R8, EC5358 = 404x = PC4048, "mistake," 8 under 8 in K4. For R5, EC2515 = PC07x 3/4 (i.e., 3 or 4) I assumed PC0774, "arrange," before "-ment" in C10. I placed 1 under 7 in K2, 5 under 4 in K4, and added 9 under 3 to finish K4. For R4, EC0332 = 20x5 I assumed PC2035, "refuse," before "-ing" and placed 3 under 3 in K2.

In C10, R1 (K6172), I now had EC3033 = 60x3 and assumed PC6023, "one," giving me 3 under 2 in K7. But in C9, R2, I had EC8309 = x023, and so I recognized "one" again: 6023. I placed 8 under 6 in K2 and added 4 under I to finish K-row 2. (See Figure 7A for the current state of the square.)

* Fig.
* Fig. Figure 7. Square recovery.

I now returned to the colon in message 1 (C29). If my placement was correct, C27 and 28 should be (EC2411 8844), "part one": PC4609 6023. Positions 2 and 3 of the key for C28 seemed to be K3, but positions 1 and 4 gave me trouble. I finally put 4 under 3 in K8 (position 4). EC8 = PC6 in K2 gave me a conflict, but compared with 8844 and using Kx338, EC8524 in R7 gave me PC6913, "this." EC8209 in R6 = 613x gave me a satisfactory PC6136, "of," and I placed 9 under 6 in K8. Also EC2931 in R4, C27, compared with EC2411, gave me 43x9, which I assumed to be PC4319, "were." So I accepted "part one" as correct and placed 3 under 1 in K8.

Then, assuming that the repeated group in C27, EC2097 = 49xx (with the assumption EC7 = PC7 in K6) is PC4967, "and," I labelled the two keys (C27 and 28) K1486 x338. PC4609, "part," gave me 1 under 0 in K8. C27, R3, EC5996 = x364 I assumed as PC1364, "they," and placed 5 under 1 in Kl. In R7, EC7121 = 21x9 gave me PC2199, "how," and I placed 2 under 9 in K8. R9, EC2814 = 480x I assumed as PC4805, "be," and placed 4 under 5 in K6, and then added 5 under 3 to finish K6.

EC3394 in C28, R10, = x543, I assumed as PC0543, "so," and so EC8 = PC6 should probably have been placed in K0 for position 1 of this column (C28) rather than in K2. Checking all the values down this column with K0 in the first position, I found no conflicts and so assumed 8 under 6 and 9 under 1 in K0. K0338 for C28 allowed me to assume EC9076 in R4 = PC1364, "they," already shown to be the subject in R3, C27, and placed 6 under 4 in K8 as well as corroborating the 9 under 1 in K0.

If we go back to C25, K1 1/2 3 1/2, we can now correct the key to K1131. In R4, EC7637 = PC2952, "other." In R3 we may assume EC5761 = PC1276, "period," and place I under 6 in K1. The EC7 = PC2 in the second and fourth positions corrected the 1/2 in the key to 1 in each case. So it turned out that while K2232 gave us EC6656 = PC9999, "the," in C45, all three 6's in C25 derived their 9's from K1, and the complete key for C25 turned out to be K1131. In R7 EC6013 = 908x became PC9088, "example," and I placed 3 under 8 in K1.

In C25-29, R1, we now have PC0909 xxxx 4609 6023 2222, "begin xxxx part one:". For EC4939 in C26 I decided to try "message" (PC1600), giving me K5/2 806 for the key to C26. This yielded EC0343 = PC6136, "of," in R7 and 8, correcting the key to K5806, and placing 4 under 3 in K0. EC0734 in R9 gave me PC6505, "to," placing 7 under 5 in K8. EC3491 in R3, becoming PC4319, "were," placed 3 under 4 in K5. "Were they" in R3, C26-27, helped corroborate and make firm the "were they" in R4, C27-28. I was, of course, usually aware of such corroboration before I added a new value to the square, since I was frequently working on more than one column at a time--as well as trying out and discarding alternate values here and there.


Returning to C15, R5, PC0909, "begin," I assumed that it was probably followed in C16-18 by PC4609 5174 2222, "part five:". The first corroboration I found was in C18, where EC1401 = 2222, ":," gave me K5325, which in turn gave me EC9124 = PC9801, "(e)D," in R3, and EC3729 = PC4609, "part," in R6. I could assume then in R6, C17-20, EC3270 3729 2992 2614 = PC0909 4609 0499 2222, "begin part six:". (Lest this be considered as something which deliberately made this problem easier, let me point out that in the field during the wax the phenomenon of redundancies multiplying redundancies proved a constant help, because of the necessity of the sender, as I have shown in the earlier part of this article, to identify the system, the sender, and the message and part numbers, and to use such groups as "begin, message, the end, to be cont.," necessary punctuation, and so forth. Note that this problem has the redundancies of English grammatical verbiage.)

EC3270 = PC0909 gave K0794 (C17), for 7 under 0 can by this time be placed only in K9. EC9361 (R1) gave PC 1221, "very," placing 6 under 2 in K9. EC6125 in R5, already assumed as 5174, "five," placed 6 under 5 in K0, 1 under 1 in K7, and 2 under 7 in K9. (See Figure 7B for the current state of the square.)

EC8114 in R9 (C17) = PC6136, "of," placed 1 under 3 in K9. In C16, R5, EC9742 = PC4609, "part," produced K3378?, which was corrected to 3377 by R2, EC9031 = PC4321, "well," and R6, EC0383 = PC3542, "to be cont." I placed 8 under 4 and 2 under 9 in K7. Note that "to be cont." immediately precedes "begin part 6." Obviously what we have here is the beginning of the message immediately following its last group. This phenomenon was a constant in our work during the war because of the bisection technique used to hide the redundant groups by cutting the message at some arbitrary point and placing the latter part before the beginning.

C16, R8, EC4350 = PC2585, "(e)D," placed 0 under 5 in K7. R10, EC6507 = PC7957, "a," placed 7 under 7 in K7. EC2992 = PC0499, "six," in C19, R6, gave me K235 7/8 for C19. This was corrected to K2358 by R1, EC0168 = PC2882, "-ing," and R10, EC0160 = PC2888, "-ment." This enabled me to place 6 under 8 in K5, 8 under 2 and 0 under 8 in K8. R2, EC8229 = PC6136, "of," placed 2 under 3 in K5, and R8, EC2552 = PC0909, "begin," placed 5 under 0 in K5 and enabled me to add 8 under 5 to finish the K-row. We can also assume here in R8, EC6035 9153 0661 = PC 4609 3285 2222, "part eight colon."

C20, R6, EC2614 = PC2222, ":," and R8, EC6035 = PC4609, "part," supplied K6903 for C20 and placed 1 under 2 in K0 and 0 under 6 in K9. I added 0 under 4 in K0 to finish that K-row. Continuing with C20, EC5383 in R3 = PC3165, "place," and EC0473 in R5 = PC1495, "would," placed 3 under 1 and 4 under 4 in K9 without creating a conflict. Moving on to C22, R8, EC0661 = PC2222, ":," gave K2995, with which the repeated EC6688 became PC9295, "upon," and placed 8 under 9 in K9. R6, EC2855 = PC0980, "design," yielded 5 under 8 in K9, and I added 9 under 5 to finish K9.

In C30 (K6534), R1, I had EC0581 = PC1001, "according." Moving to C31, EC9377 = PC6505, "to," was an obvious assumption and could now give the manageable K 7/8 398. Several groups in the column corrected the key to K7398, such as R5, EC3598 = PC2952, "other," and I placed 9 under 6 in K7 (and 6 under 3 to finish K7). In C10 (K6172), R8, EC2428 = PC2596, "N," added to the preceding "mistake," gave me "mistaken," and I placed 4 under 5 in K1. Then moving back to C25, K1131, R8, EC9784 = PC3205, "habit," gave me 9 under 3 in K1, making it possible to finish that K-row with 8 under 7. Both of these values were then corroborated by R10, EC8049 = PC7023, "with," placing 9 under 3 and 8 under 7 in K1. C28 (K0338), R4, EC9076 = PC1364, "they," tentatively assumed earlier, placed 6 under 4 in K8. EC5 under PC7 then finished the row K8 as well as the square. See Figure 8 for the completed square, keeping in mind that this square, which resulted from the decipherment, is quite different from the square which was used in the encipherment, for which see above, Figure 2. Note, however, that it is only the numbering of the rows which is different, not the numbering of the columns.

* Fig.
* Fig. Figure 8. Square as deciphered.


ADDENDA

Without any intention of guessing what sort of experimentation one or another reader of this article may wish to try, if any, I have decided to include here a moderate amount of the materials which might be necessary for such experimentation. First the fifty keys obtained by decipherment, arranged consecutively and divided and numbered in groups of five.

9627 2656 4435 1591 9371  #5  1609 8531 8729 2024 6172 #10  5323
3162 9158 1447 4605 #15  3377 0794 5325 2358 6903 #20  4073 2995
7401 9792 1131 #25 5806 1486 0338 4163 6534 #30  7398 0722 6245
3719 5067 #35  8838 3529 1040 7497 2811 #40 9474  6533 4201 7659
2232 #45  1946 4723 3889 0535 1467 #50

Following are the first three messages as deciphered and still bisected, with the individual groups shown separated by a dash (-). C8 (etc.) indicates the column in which the enciphered message began.

C1: which—go—to—make—up—a—marriage—name—ly—one—man—and
—one—woman—period—this—very—suprise—ing—state—ment
concerning—marriage—to—be—cont.—begin—message—part—one
colon—according—to—recent—important—find—ing—s—by—men
of—science—there—are—two—absolute—ly necessary—part—s
C8:  period—one—example—to—the—point—is—the—well—known
case—of—j—a—c—k—s—p—rat—and—his—love—ing—wife
period—to becont.—begin—part—two—colon—may—not—at—once
seem—self—evident—but—let—us—examine—it—care—ful—ly—in
—detail—C15:—the—cook—dare—d—to—place—lean—upon—her
plat—period—were—they—single—the—situation—would—have
been—grave—for—to becont.—period—begin—part—three—colon
now—j—him—self—could—not—bear—the—very—thought—of—fat
—while—mrs—s—retire—d—in—haste—when—ever

For anyone who might wish to try out the deciphering technique with or without consulting the list of keys given above, I add, in numerical order, a list of every plaincode group which appears in messages four through seven. Seven includes the end of the J.S. portion of the whole message.

0145, or.   0239, same.   0266, social.   0404, refer.  0415,
certain. 0471, general.   0499, six.   0608, own.   0774,
arrange.   867,  fashion.  0909, begin.   0925, men.   0980,
design.   0990, there.   1045, continue.   1191, serve.   1218,
up.   1276, period.   1348, finally.   1364, they.   1495, would.
1760, no.   2014, y.   2035, refuse.   2057, al.  2112, his.
2199, how.   2212, de.   2222, colon.   2458, now.   2525, e.
2585, (e)d.  2743, com.  2824, long.    2829, new.   2877, se.
2882, ing.  2888, ment.  2952, other.    2992, ly.   3003,
actual.   3100, match.  3296, x.   3366,  strong.   3422,
practical.   3474, meat.  3481, such.     3509, take.
3542, to be cont.   3631, being.  3769,  success.   4070, as.
4101, application.   4319, were.  4598,  ies.   4609, part.
4693, marriage.   4774, us.   4805, be.   4910, mr.   4967, and.
 4989, p.   5021, an.  5023, work.   5043, but.   5083, for.
5102, have.   5133, individual.   5173, in.   5174, five.   5230,
human.   5342, waste.   5390, develop.   5555, is.   5562, could.
5971, regard.   5843, ful.   5997, it.   6008, hand.   6023,
one    6136, of.   6242, fare.   6277, on.   6411, insist.
6505, to .  6566, that.   6650, let.   6789, might.   6913, this
6927, share.   6996, war.   7022, dispose.   7040, will.   7311,
cook.   7375, variety.   7491, vice.  7502, seven.   7528, good.
 7660, put.   7704, image.     7957, a.   8012, hold.   8111,
paragraph.   8242, urge.  8288, possible.    8323, i.   8434,
difficult.   8479, L.   8544, er.   8727,  women.  8781, all.
8888, can.   8902, any.  9088,  example.   9134, house.  9295,
upon.   9740, ence.   9751, ar.  9779, less.    9801, (e)d.
9833, both.  9843, better.    9876, each.    9895, set.  9919,
red.  9930,  four.  9966, without.    9977,  prefer.   9999, the.

BIOGRAPHICAL SKETCH

Henry D Ephron is co-editor with his wife Marguerite of Xenocrypts (foreign-language ciphers) for the American Cryptogram Association. He was introduced to cryptanalysis in the 1920's by a pulp magazine's "Yardleygrams," and one of Yardley's captains in WW I, Latin Professor Charles H. Beeson of the University of Chicago. Educated there in Classical Greek and Latin, Professor Ephron then spet the war years in cryptanalysis and later did research in Meycenaean (pre-alphabetic) writing, a task which shared the difficulties and the fun of cryptanalysis. His cryptanalytic mentors in the Army Signal Corps were Lambros Calimahos, William Bundy, and, of course, his CO in the field, Abraham Sinkov.


Agnes Meyer Driscoll

Lt. Susan M. Lujan, USNR

[This paper was submitted as an entry in the Rear Admiral Don H. McDowell Essay Contest, established in 1986 by the Commander, Naval Security Group Command. The Contest seeks to stimulate critical thinking, to improve writing skills, and to foster an appreciation for the historical development of the cryptologic profession among junior cryptologic officers. Papers are written on technical, operations, or historical topics. Submission of a paper is a graduation requirement for the Cryptologic Division Officer Course (CDOC), a formal nineteen week school taught in Washington DC.

Lt. Susan M. Lujan, USNR, submitted the paper to the CDOC on 30 October 1987.]

INTRODUCTION

As history is rewritten with the declassification of previously classified information, people who were once behind the scenes are often revealed to be heroes who provided information and services to military and political leaders which allowed those leaders to make decisions based on insightful judgments. Mrs. Agnes May (Meyer) Driscoll spent forty-one years of devoted government service from 1918 to 1959 in the field of cryptology. She was often the first in a group of skilled analysts to break new systems and codes considered milestones in the history of the field, including solutions to the Hebern cipher machine, the Japanese Orange Grand Naval Maneuvers cipher, the Japanese Red Book and Blue Book superencipherments, and more. Due to the sensitive nature of her work, Mrs. Driscoll led a private lifestyle characterized by anonymity. Thus, time and lack of documentation have obscured some aspects of her life, making her as mysterious in some ways as the cryptanalysis she performed. This paper is a tribute to this remarkable historical character.

I. EARLY LIFE

Agnes May Meyer was born on 24 July 1889 in Geneseo, Illinois and was raised there by her parents, Dr. Gustav Frederick Meyer (PhD) and Lucy Andrews Shaw Meyer. She attended Osterbein University from 1907 to 1909 and later received an A.B. degree from Ohio State University in 1911; her main undergraduate subjects were mathematics, physics, music, and languages. Interestingly, in the context of the Victorian era, Miss Meyer was an atypical woman since she pursued technical subjects. "As recently as a hundred years ago, it was assumed that because men are larger and heavier their brains must be larger and heavier too."1

1Tobias, Sheila. Overcoming Math Anxiety (1978), p. 80.

After college, Miss Meyer moved to Amarillo, Texas, where she worked at Lowry Phillips Military Academy as Director of Music from 1911 to 1914. She was later the Head of Mathematics Department at Amarillo High School from 1914 to 1918.

II. WORLD WAR I

Woodrow Wilson won his second term on the slogan, "He kept us out of war." Shortly after his inauguration in 1917, however, the United States entered World War I.

Rapid wartime expansion rocked the country which had already been experiencing a growing labor shortage. "While the general citizenry was not aware in 1916 of how great was the probability of the United States being drawn into the war in Europe, the military in this country was already involved in preparations that had to be made if we were not to be caught short in an emergency. This was particularly true of the leadership of the Navy."2

2Hancock, Joy (Captain, U.S. Navy, Retired). Lady in the Navy, A Personal Reminiscence (1972), p. 22.

The shortage of personnel to do work that was immediately necessary was a problem that the Secretary of the Navy, Josephus Daniels solved with a far-reaching decision:

'Is there any law that says a yeoman must be a man'? I asked my legal advisers. The answer was that there was not but that only men had heretofore been enlisted. The law did not say 'male.' 'Then enroll women in the Naval Reserve as yeoman,' I said. 3

3Daniels, Josephus. Our Navy At War (1922), pp. 328-329.

Although the enrollment of women in the Navy was shocking, women from all points of the compass answered the Call to Colors. Agnes May Meyer, a first generation American, was one of those answering the call, and she was recruited in the highest rating obtainable for a woman in World War I, Chief Yeoman (F). Recruiting in such a rating was only done in a few instances. "Other qualifications being equal, in the selection of chief yeoman, preference was given to those who had a working knowledge of Stenography."4.Stenography was but one of Miss Meyer's talents. In addition, she had a background in statistics, math, physics, engineering, clerical skills, teaching, and was proficient in German, French, Latin, and Japanese. She enlisted on 22 June 1918 and began active duty on 23 July 1918. She worked in communications, where she began her career as a cryptographer and cryptanalyst.

4Dessey, Eunice C. The First Enlisted Women 1917-1918 (1955), p. 30.

With demobilization, the government gave the yeoman (F) an option: "All Yeoman (F) on duty in the Navy department who do not desire to accept civil service appointments will be released by July 15th or as soon thereafter as your office can handle the releases. . . . Those who accept civil service appointments will be released from active duty as reservists on July 31st, and their civil service appointments will date from the day following."5 Miss Meyer's records show that her active duty ended on 31 July 1919 and that she was honorably discharged on 5 February 1920.

5Ibid., p. 66.


III. EARLY WORK IN CRYPTOLOGY

The Office of the Director of Naval Communications (DNC) hired Agnes Meyer as a stenographer, and in 1920 she was given the title clerk. "It was during this time that a turning point came. Miss Meyer was invited in 1920 to join an elite group of people studying at the Department of Ciphers at Riverbank Laboratories in Chicago, Illinois, owned and overseen by millionaire George Fabyan. Fabyan paid her expense and her salary and the Navy gave its blessing to the venture."6In a letter of 1 March 1920 to Commander Milo F. Draemel, Officer in Charge, Navy Code and Signal Section, Colonel Fabyan wrote about Miss Meyer, "We were very favorably impressed with the young lady and should the condition arise that it is deemed the part of wisdom to dispense with her services at Washington, I want to say, for your personal information, that you are authorized to send her to Riverbank at my expense and I will give her employment in the Cipher Department, at salary which is equal to that which she is now receiving and, furthermore, should occasion arise that you wanted her returned to the Department, I will then release her for that purpose and, in the meantime, she has been kept in touch with that sort of work and would be available."7

6Pioneers In U.S. Cryptology, Part II. The NSA Family 35 Year Celebration. History and Publications Division, National Security Agency (July 1987), p. 13.
7Colonel George Fabyan's letter of March 1, 1920 to Commander Milo F. Draemel.

IV. CIPHER MACHINES

Early in 1921, the inventor of the first cipher machine to employ the rotor principle, Edward Hugh Hebern, "advertised an 'unbreakable cipher' in a marine magazine, but Miss Agnes Meyer, a cryptanalyst in the Navy's Code and Signal Section, solved the sample message."8Whether Miss Meyer had actually returned to Washington or remained at Riverbank at that time is unknown.

8Kahn, David. The Codebreakers, The Story of Secret Writing (1967), p. 415.

Between 1920 and 1922 the sequence of events and activities in Miss Meyer's life were not documented. Evidence does show, however, that she was "attached to the (Yardley's) New York Office for five months in 1920."9Under the Education heading on applications for government service submitted after 1920, she did list a course in cryptanalysis at New York Laboratories, a place which became known as the Black Chamber.

9Holtwick, J. S. (U.S. Navy, Retired) Naval Security Group History To World War II, Part I, p. 35.

On 15 January 1923, Miss Meyer resigned, and in February 1923 Edward Hebern hired her as Technical Advisor and Liaison with the Navy at The Hebern Electric Code Company which he had incorporated in 1921 with encouragement from the Navy. Although the Navy was very interested in 1923 in purchasing the Hebern cipher machine once it was perfected, the Navy had not provided Hebern with any financial assistance. He was unable to bear the burden of the initial investment (including building a factory, machine dies, molds, patterns, etc.), and "in the spring of 1924, the firm defaulted on the interest on its $100,000 mortgage."10

10Kahn, Codebreakers, p. 418.

In August 1924, Mrs. Driscoll returned to the Navy department, having married a Washington lawyer, Michael Bernard Driscoll, in the interim. When or where they met cannot be determined, but they may have known each other for some time since Mr. Driscoll was from Cairo, Illinois.

During the mid-1930's, Mrs. Driscoll testified before the Senate Naval Affairs Committee that in 1922 she had helped invent a device called the Communication Machine or Cipher Machine. She worked with Lieutenant Commander William F. Gresham on the development of this device. "Later after GRESHAM's death in August 1935, a claim was made by his widow for recompense . . . . In the course of hearings on this bill, Mrs. Agnes DRISCOLL, the Navy's principal civilian cryptanalyst for many years, claimed a share in the invention. A clipping from the Manila Daily Bulletin of Tuesday, June 8, 1937, datelined Washington (by airmail), quotes Senator David L. WALSH, Chairman of Senate Naval Affairs Committee, that 'The Navy now possesses a device which prevents decoding of its secret radio communications. . . .' "11 Mrs. Driscoll was awarded $15,000 for her part in inventing the device.

11Ibid., p. 18.


V. THE RED BOOK

"In the spring of 1920 the secret ONI [Office of Naval Intelligence] fund had financed a major undercover operation that involved not just one but a whole series of break-ins at the Japanese Consulate in New York City."12 The Japanese Fleet code book was photographed page by page and then translated over the course of four years by Dr. Emerson J. Haworth and his wife. This translation was called the Red Book because it was bound into two volumes with red bindings.

12Layton, Edwin T. (Rear Admiral, U.S. Navy, Retired), and Pineau, Roger (Captain, U.S. Naval Reserve, Retired). "And I Was There" (1985). p. 31.

Lieutenant Lawrence F. Safford, who was destined to become the father of naval cryptology, was appointed head of the newly established Cryptographic Research Desk when he arrived at the Naval Code and Signal Section in January 1924. After she returned to DNC, Mrs. Driscoll began to teach Lieutenant Safford the science of cryptology. He reportedly had a natural talent; however, if he is known as the father of naval cryptology, then Miss Aggie (as she was known) would be the grandmother of cryptology.

The Research Desk assumed responsibility for deciphering and analyzing messages sent by Japanese naval radio stations. In 1925, Lieutenant (j.g.) Joseph J. Rochefort joined the group and also came under the tutelage of Miss Aggie. After Safford had rotated to sea duty in February, Rochefort took charge of the research desk and "actual cryptanalysis of the 'Red Book' messages was first undertaken. Mrs. Driscoll was responsible for the initial solution and for the solution of most of the new ciphers and 'transposition forms' used with this code."13"This was no small achievement because the two-volume Red Book contained some 100,000 code groups, with as many as 3 groups assigned to each Japanese word or expression."14Mrs. Driscoll received a promotion and a raise in 1925.

13Safford, L. F. (Captain, U.S. Navy). The Undeclared War (""History of R. I.") (15 November 1943), p. 05.
14Layton, "And I Was There", p. 34.

Miss Aggie's diligence paid off in other ways. According to Safford, information gained from this deciphered code included:

"(1) Form, phraseology, and subject matter of secret Japanese naval messages.
(2) Various accidents and casualties of Japanese men-of-war.
(3) General knowledge that Japanese naval maneuvers where much more realist that ours, particularly in night torpedo attacks.
(4) Exact knowledge of Japanese fuel supplies (oil, coal, and gasoline).
(5) Early knowledge of Japanese advances in naval aviation."15
15Safford, The Undeclared War, p. 05.

VI. GRAND MANUEVERS

In the spring of 1930, the Japanese Imperial Navy conducted exercises known as the Grand Fleet Maneuvers. "The Japanese introduced a new cipher system for these maneuvers plus a daily change of key, but used the good old 'Red Book.' All hands turned to these messages, Mrs. Driscoll got the first break as usual, and the various daily keys were solved without too much effort."16Her work resulted in an intelligence windfall which revealed how the Japanese Fleet would conduct combat operations against the United States. Even more significantly, further analysis of the data indicated that the Japanese had successfully exploited a weakness in U.S. operations. "Their [Japanese] naval intelligence had apparently succeeded in obtaining a very accurate picture of the latest modifications in our war plan Orange."17The use of cryptology to discover what the enemy knew about our own forces probably gave senior officials a new appreciation for the importance of cryptology.

16Ibid., p. 08.
17Layton, "And I Was There", p. 35.

The Research Desk (Op-20-G) received machines during this time "for relieving the cryptanalytic section of a good deal of their purely mechanical labor. Apparently Mrs. DRISCOLL, the principal cryptanalyst, was not too enthusiastic about this approach, having for more than 10 years operated with pencil and cross-section paper."18

18Holtwick, Naval Security Group History To World War II, Part I, p. 79.

VII. THE BLUE BOOK

In the fall of 1931, Mrs. Driscoll was the first to notice that the Japanese had instituted a new code. This new code, known as the Blue Book, superseded the Red Book. Faced with solving code and cipher simultaneously, the OP-20-G team worked to recover the code by cryptographic analysis and reconstruction. Using an IBM tabulating machine facilitated the task of keeping track of the nearly 85,000 code groups and changing cipher keys. No other cryptologists had ever broken a complex code and cipher system without assistance from information obtained by compromise of either the code or system. Devoting all her time to pure cryptanalysis, Mrs. Driscoll was the first to break the code. Thanks to her efforts, in 1936 important information was learned:

The most important and certainly the most dramatic incident connected with the 'Blue Book' was the message reporting [the battleship] NAGATO's post-modernization trials. . . . The NAGATO's new speed was 26+ knots. . . . This created consternation in the higher echelons of the Navy Department because. . . our new battleships (then only in the blueprint stage) were going to have a speed of only 24 knots.19

19Ibid., p. 90.

As a result of this information, our battleships were redesigned to have the edge over their Japanese counterparts with a speed which ultimately was to reach 28 knots. Safford was quoted as saying "that this coup 'paid for our peacetime RI [Radio Intelligence] organization a thousand times over.' " 20The Blue Book was not superseded by another code until 1938.

20Layton, "And I Was There", p. 58.


VIII. TO CATCH A SPY

In 1936, while working on intercepted Japanese foreign office traffic, Mrs. Driscoll consulted the Japanese linguist, Lieutenant Layton, and identified a word T-O-M-I-M-U-R-A, which Layton said could be rendered as Tomison or Thompson. Mrs. Driscoll responded, "Lieutenant Layton, I want to thank you. You helped me to solve part of a cipher that has been bothering me for some time. The name is Thompson."21After investigation, Thompson was revealed to be a Pacific Fleet radioman.

21Ibid., p. 59.

The American who was his accomplice and referred to as Agent K in the same message turned out to be John S. Farnsworth, a cashiered navy officer. It was he who masterminded an espionage ring that turned over engineering, and gunnery secrets. . . . Farnsworth and Thompson were arrested in 1936. Although the evidence that unmasked Harry H. Thompson and led to the pair's apprehension could never be presented in court, they were convicted the following year of conspiring to violate federal espionage laws and sentenced to four to twelve years in prison.22

22Ibid., p. 59.

IX. AGIE, WE HARDLY KNEW YE

The brief character sketch of Agnes May (Meyer) Driscoll which follows is an excerpt from the late Rear Admiral Edwin T. Layton's book, And I Was There:

I had been warned not to patronize Madam 'X,' as her colleagues sometimes referred to her, because she was sensitive to her role as a woman in a man's world. Because of this she kept to herself as much as possible and none of us was invited to socialize with her and her lawyer husband. While she could be warm and friendly, she usually affected an air of intense detachment, which was heightened by her tailored clothes and shunning of makeup. It was surprising to hear Miss Aggie curse, which she frequently did - as fluently as any sailor whom I have ever heard.

In the navy she was without peer as a cryptanalyst. Some of her pupils. . . were more able mathematicians but she had taught cryptanalysis to all of them, and none ever questioned her superb talent and determination in breaking codes and ciphers. She understood machines and how to apply them. . . . But her principal talent was her ability to get to the root of a problem, sort out its essential components, and find a way to solve it. Among her uniformed colleagues, she was held in the highest esteem throughout her long career. . . .23

23Ibid., p. 58.

In October 1937, Mrs. Driscoll was in an automobile accident in which two of the other passengers were killed. "She suffered two broken jaws and a broken leg. . . not returning fully to duty until September 1938."24When she did return with the use of a cane, she was reportedly "physically changed, but her personality and attitude had apparently been affected as well."25

24Holtwick, Naval Security Group History To World War II, Part I, p. 160.
25Pioneers In U.S. Cryptology, Part II. The NSA Family 35 Year Celebration.

* Fig.

Agnes Meyer Driscoll (r) at the retirment of her assistant, Mrs. Talley (c) with General Samford (l) in 1956.

Photo Department of Defense.

X. DESIGNING THE PERFECT MOUSETRAP

By 1 June 1939 Japan was using an operations code cipher (JN-25) which was radically different from any other Japanese navy cipher.

It was a five-numeral, two-part code, consisting of 33,333 groups in the 'dictionary book,' to each of which were aed by false arithmetic (no borrowing or carrying) a five-digit number selected in sequence according to the start key from a second book of random additives. The resulting encipherment consisted of a series of five-digit numbers divisible by three to provide a convenient check against garbles.26

26Layton, "And I Was There", p. 77.

Mrs. Driscoll recognized the traffic as machine generated, and she developed a manual decryption for this Japanese machine ("M1"). Working from known characteristics of the cipher, Lieutenant Jack S. Holtwick built an electric coding machine which decrypted the Japanese Navy's machine cipher. "This analog worked well enough. . . until it [the cipher] was abandoned in 1938. But the real value of the 'M1' machine was that it gave Friedman's team a headstart in constructing the device that would imitate the 'M3' machine which was then being used by the Japanese foreign ministry in 1935 to encode its most confidential communications."27

27Ibid., p. 79.

XI. CONCLUSION

From this point until her retirement at age 70 in 1959, there is no documentation of Agnes Driscoll's specific achievements. What we do know of her as a cryptologist is that her most significant contributions were to the development of Naval cryptology and to the Navy's victory in World War II. She modestly described her job in cryptology as scientific duties; she probably approached her job with quiet loyalty and dedication (forty years of government service) grounded in a solid understanding of the importance of cryptology. In addition to this unpretentious dignity, Mrs. Driscoll had a certain drive to accomplish wholeheartedly any task at hand, as evidenced by her proficiency and some devoted achievements in her life. Her unique character and the important role she played as a pioneer in the burgeoning field of cryptology in the early part of this century immediately appealed to me as a topic for the Rear Admiral McDowell Essay.


Letters to the Editor

The following two letters relate to the article, "Why Was Safford Pessimistic About Breaking the German Enigma Cipher Machine in 1942," by Louis Kruh which appeared in Cryptologia, (1990) 14(3): 253-257.

From Colin Burke, 6319 Red Haven Rd, Columbia MD 21045 USA.

October 1990

Dear Editors

I enclose some general comments re the Safford memo, p. 254, and my conclusions.

a) The political setting: When Safford wrote this memo he was well on the way to being permanently shoved out of any significant role in OP20 but was not yet put totally aside. And, he may have been continuing his attempt to return to a role in "G". The Redman group was on its way in and was beginning to forge its new policies.
I think Safford was giving policy advice to the new group. The memo has to be placed in a political as well as a technical context to appreciate it:
1) he is advising the new group to stand against political pressures and to keep Magic to as few people as possible that—might have been one of his responsibilities as 'Q'
2) he is warning the new group, which included Wenger, who had always been an advocate for the use high speed equipment, not to count on a quick way into Enigma, a system he had been tackling since early 1941
3) he is perhaps warning the new men not to give Britain too much about or from Magic
b) Conclusions: I agree with your final conjecture, p. 257. Placed in the appropriate contexts, Safford's assertions that only "captures" had allowed the previous entries into Naval Enigma and that prospects for re-entry were rather poor, seem reasonable. Given what England had probably told us about methods by early 1941, his history was well grounded. The importance of the later capture to the re-entry into submarine Enigma at the end of 1942 validated his prediction. If we add the helpful but not critical assumption that OP20G had not been informed of the "bombe" and its allied methods by early 1942, then Safford's memo is explained.1 And, even if the assumption about the bombe is dropped, Safford's statements were those of a well informed man. Note that the British bombes were in operation long before the mid-1941 break into the 3 wheel Naval Enigma which suggests that some captures might have been needed to initiate bombe usage, even for the simplest 3 wheel systems.
1Joe Eachus' surprise when he discovered the use of the diagonal board in the Turing bombe when he went to England in late summer, 1942, is only one among many small indications that Britain had not told OP20G all about its Enigma capability.

Finally, the materials in this article reinforce my beliefs about how much and how soon Britain shared the Ultra secrets with us. Again, why were there no references to the bombes, even guarded ones, by Safford or the two Navy men you quote?

Comments on Particular Points

I) p. 253: "Our prospects of ever breaking the German "Enigma" cipher machine are rather poor." There are several ways to interpret this. Let me cite some that are not emphasized in your article.
1) "our" could well refer to OP20G, not to both Britain and America—thus he could have known of the earlier 3 wheel break by Britain and yet made his assertion. The following support this.
a) Safford did not trust England nor did he approve of sharing American secrets with it
b) Safford thought ("Rhapsody in Purple") that Britain had not kept its promises concerning the February-March 1941 exchange
c) he wanted an independent American capability for all systems for all nations
2) "ever breaking"
a) the Weeks-Currier trip came before England had cracked even the three wheel Naval Enigma so that it could not have told the Americans how to enter it via any general method (also see your article, p. 257)
b) I have seen no hard evidence that Britain told OP20G the necessary details to allow it to enter the 3 wheel system by itself after England's entry in mid-1941 when more 'pure' techniques were available (also see you article, p. 257)
c) I have not seen any evidence that Britain gave OP20G knowledge of more general techniques (pure cryptanalytic) for Naval Enigma systems before 1942
d) Safford's reference seems to be to "naval" Enigma — he could have known of the more pure or general British approaches to air force or other Enigma but was correct that Naval systems awaited captures
e) certainly, in March 1942, the prospects of either England or America cracking the 4 wheel system were low and, I believe, Safford wanted an independent American capability
i) I have some reason to believe that when the Americans started their bombe project (Feb-March) 1942, the design was very different from Turing's and the logical approach to a solution was also very different
ii) the U. S. surely knew that Britain had been locked out of the 4 wheel system and that it could not promise a speedy re-entry
II) p. 255: "A Retired Lieutenant Commander wrote. . . ." This man seems to have his dates wrong, he is consistently one year too early
1) the official history or OP20G states that no Enigma work was begun until early 1941 (NOT 1940)
2) Wenger did not return from "sea duty" until late 1941 and probably did not begin any OP20G operational work until early 1942
3) Safford did not leave (get kicked out) as head of G until 1942
4) although the Americans decided to build some sort of bombe on their own in early 1942 it was not until the fall of 1942 that anything like a firm commitment to a design was made
5) because of the above we should conclude that Safford was well aware of what was going on in March, 1942
III) p. 256: "Another retired officer. . . ." This man also seems to be somewhat confused about dates, but is perhaps a few months rather than a year off the mark. Despite that possible 'slip' he says much that seems acceptable and much that is very important to the questions of how much and when Britain shared it methods, and the Safford memo, as well.
1) Didn't the Lofoten raid come after the conclusion of the Weeks-Currier visit (and if this isn't one of those men someone else who I should know of was at GC&CS in 1941)? [Editor's note: The Lofoten raid occurred on March 4, 1941, and it is known that the two Army members of the U. S. intelligence mission to England remained there through the end of March; presumably so did Weeks and Currier.]
2) if this man was either Weeks or Currier and if England sent over the techniques which allowed them into the three wheel Enigma in mid-1941, he should have known more about all this
IV) Your comment, p. 257: "One final conjecture...." Much of my wandering depends upon a meaning for the elusive concept, "a purely cryptanalytic solution" and this section is directed more to the general question of how much Britain told us than to the Safford memo in your article.
1) In conjunction with the "several days' settings" quote Safford's "captures" assertion strongly suggests that England did not tell us about their "bombe" and its use of 'cribs' in 1941
a) why wasn't the bombe mentioned along with the mention, p. 257, of the paper Enigma given the Americans?
b) why didn't Safford include something about the Turing bombe approach in the memo, at least to directly discount the possibilities of such an approach?


From Ralph Erskine who can be reached c/o Cryptologia, Rose-Hulman Institute of Technology, Terre Haute IN 47803 USA.

10 October 1990 - Belfast NORTHERN IRELAND

Sir

In a recent article, Louis Kruh asked for views on Captain Lawrence Safford's knowledge of the British breaking of Enigma.

Safford's memorandum was expressly intended to emphasise to others the paramount need to ensure that the US Navy breaking of, in particular, the JN 25 series of codes was not revealed to the Japanese. He was anxious that this might occur due to the resulting intelligence being disseminated too widely. His passing reference to Enigma, occupying only about one line, is clearly not an exhaustive statement of the Allied capability to break any form of Enigma.

Safford was right to be concerned. His memorandum is dated 18 March 1942. On 30 August 1942, 3 Skl II Staffel in the Seekriegsleitung (the German Navy's war staff) is apparently recorded as stating -

When the Japanese were presented with some captured documents from the prize ship "Nankin", the [Japanese] Navy suspected that its signal methods were no longer unknown to the enemy and it requested German assistance in revising its coding systems.2

2Report on documents from 3 Skl II Staffel (NID 24/T50/45): PRO ADM 223/255.

The Australian ss Nankin, bound from Freemantle, had been captured by the German raider Thor on 5 May. Documents seized from the Nankin were transferred to a German supply ship, Regenberg, and reached the German naval attache in Japan, Rear-Admiral Paul Wenneker, only on 18 July. It would seem that the papers seized included intelligence summaries which could only have been derived from the Allied breaking of Japanese naval codes. On 29 August, after being authorised by Berlin to do so, Wenneker showed some of the reports to the Japanese — hence the remarks by 3 Skl quoted above.3

3See James Rusbridger, "The Sinking of the "Automedon", the Capture of the "Nankin," Encounter 64(5) (May 1985), 8.

The further event that Safford feared — the supply of Enigma by the Germans to the Japanese also took place, although perhaps not until around 1943. Fortunately, it would appear that the Germans did not hand over the plugboard version of Enigma. If they had done so and Enigma's rotors had been rewired (as elementary security would have required), the US Navy would have had the utmost difficulty in penetrating "Japanese" naval Enigma — if the Japanese could have produced it quickly and distributed it in sufficient numbers.

Safford was writing purely in a naval context and not referring to Luftwaffe or Army ciphers, machine or otherwise. Moreover, since naval Enigma ("Enigma M") presented special problems to the cryptanalysts, it was almost irrelevant whether Luftwaffe or Army Enigma ciphers were being broken by the British or not. In the last sentence of paragraph 2 and the first of paragraph 3, Safford is clearly referring to US Navy cryptanalytical work. It is therefore reasonable to read the penultimate sentence of paragraph 2 (stating, in effect, that Enigma M — "German naval messages" which were not "weather codes [or] other minor systems" — had been read only as a result of captures), as also referring to American naval work on Enigma, and therefore only to Enigma M.

The first effective breaking of naval Enigma ("Enigma M") by Bletchley Park ("BP") started on 12 March, after the capture of Heimisch (the main Kriegsmarine Enigma cipher at that time) keylists for at least part of February on 4 March 1941 during a commando raid in the Lofotens.4 However, the British had not then recovered the bigram tables used to encode the message key on Enigma M, which held BP back badly, despite the fact that the keylists gave the main settings — another indication just how difficult it was to penetrate Enigma M.5 BP read or broke 26 Heimisch daily keys for dates between I February and 9 May, which led to the decipherment of about 340 signals — probably less than 10 per cent of the naval traffic intercepted during that period. The British captured further important Enigma M material in late April and mid-May, including keylists and a weather short signal book, the Wetterkurzschlüssel, which was immediately to prove a crucially important source of cribs.

4Ralph Erskine, "Breaking Heimisch and Triton," Intelligence and National Security, (1988) 3: 162.
5Cf. the retired officer quoted in Kruh, 256.

According to Safford, and the second retired US Navy officer quoted in the article, the US Navy also read some Enigma M traffic, probably around May 1941 (after Lieutenant Weeks and Ensign Currier returned to the States).6 The British must, at the minimum, have supplied the US Navy with —

6Kruh, 254 (paragraph 3).

a "paper" copy of Enigma M (the wiring of its 8 rotors and its reflector); some keylists (probably those for part of February 1941).7

7Kruh, 257.

BP's possession of this material was one of the United Kingdom's most closely kept secrets. Only a select few on each side of the Atlantic knew about the captures, but Safford was clearly one. And since the US Navy in Washington would have realised, from at least September 1941 on, that the British were breaking naval Enigma, it would be most surprising if Safford, in view of his position in Op 20 G, was not aware of that fact.8

8See e.g. Ultra signal 1334A/8 September 1941 to CNO Washington, giving U-boat positions: PRO ADM 223/102.


Because the US Navy was primarily concerned with escorting the Atlantic convoys, it was really only interested in the radio traffic to and from the Atlantic U-boats, and not in the signals to surface vessels. On 1 February 1942 the Atlantic U-boats had begun to use four-rotor Enigma (M4) on Triton (known to the Allies as "Shark") — a cipher introduced specially for the Atlantic U-boats on 5 October 1941. Worse still, a second edition of the Wetterkurzschlüssel had been introduced on 20 January 1942. Lacking that edition, BP was therefore without its only source of Triton cribs for the bombes (fast key-finding aids) and, for the first time, blinded by the Triton traffic. In the result, BP broke Triton on only three days between 1 February and 11 December 1942.

Since there was no prospect of BP reconstructing the new Wetterkurzschlüssel cryptanalytically in 1942, the Allies were dependent on capturing it, probably from a U-boat, without the Germans realising that the code had been compromised. The codebook was printed using water soluble ink, so that recovering it intact from a sunk U-boat would have been virtually impossible.

The capture of the Wetterkurzschlüssel (and another short signal book, the Kurzsignalheft) required something approaching a minor miracle, and the supreme sacrifice by Lieutenant Tony Fasson and AB Colin Grazier (both from HMS Petard) on 30 October 1942. They had boarded the sinking U 559 in the Eastern Mediterranean, but were unable to escape from the submarine when it suddenly went down. For their bravery, which undoubtedly helped significantly to shorten the war, so saving many thousands of lives, they were awarded the George Cross (the second highest British medal). Tommy Brown, a 16 year old canteen assistant who helped them, received the George Medal for his part in the capture. But it was not until 2 December that BP became convinced that it could break into Triton again.9 And that happened because, by great good fortune, the U-boats sent weather short signals on Triton using M4 in three-rotor mode, so that the absence of four-rotor bombes was not too much of a handicap; the three-rotor bombes sufficed — just about. Given the run of luck required by the Allies before M4 could be broken, it is little wonder that Safford was pessimistic in early 1942.

9See Erskine "Breaking Heimisch and Triton", 170.

A further factor in Safford's thinking may have been that the knowledge then available to the Allies possibly suggested that four-rotor bombes would also be required to break Triton on M4. And he would have probably have known that there was little prospect of those bombes entering service before July 1943. In March 1942 who, then, would have rated the chances of breaking Triton as higher than, in Safford's words "rather poor"?

So far as is known, the US Navy did not break Enigma M cryptanalytically in 1941 — or indeed in 1942. All of Safford's statements are entirely consistent with his referring in March 1942 only to Enigma M4. Read in that light, they are extremely accurate, showing Safford to have been very well-informed indeed, since breaking M4 was truly a problem of horrifying proportions.

Safford refers to the breaking of the German naval meteorological cipher by the US Navy. This once again highlights the lack of liaison between the US Navy and the US Army on cryptanalytical matters. The German navy met cipher was in fact a code which was super-enciphered using trigram tables. It was broadcast by Norddeich (call sign DAN) on 125 and 150 kHz, and included reports from numerous met stations in occupied Europe, Portugal and other countries. Apparently, although the US Navy had broken the met cipher before March 1942, the US Army meteorological cryptanalytical section (under Captain Edmund J. Wrigley) was unaware of that fact in mid-1942, notwithstanding that the Army was interested in breaking it. It would be interesting to know whether the US Navy was still breaking the met cipher at that time.

Perhaps the most significant part of the memorandum is Safford's statement that "German [non-minor — that is Enigma M] naval messages... have been read". As we have seen, it means that the British must have given the US Navy Enigma M's wiring, and some captured keylists. It is confirmed by the remarks to the same effect (by the second retired US Navy officer) quoted in the article.10 This directly controverts Safford's post-war claim that the British reneged on a bargain to supply Enigma material in exchange for the Purple machine handed over to BP by the Sinkov/Weeks/Currier mission in January 1941.11 Since the memorandum was written less than a year after the events and is supported by other evidence, it is to be preferred to Safford's post-war account. Further evidence on the issue would be welcome, in view of the seriousness of Safford's charges of bad faith.

10Kruh, 257.
11Dundas P. Tucker "Rhapsody in Purple: A New History of Pearl Harbor — Part I," Cryptologia (1982) 6: 214-7.


MY "Purple" Trip to England in 1941

Prescott Currier

ADDRESS: Prescott Currier died in 1995.

ABSTRACT: The United States brings the Purple machine to Britain and gets information in return.

KEYWORDS: "Purple," Bletchley Park, Agnes Driscoll, Leo Rosen, Robert Weeks, Abraham Sinkov, Alastair Denniston, John Tiltman, Edward Travis, Humphrey Sandwith, Clive Loehnis, Frank Birch, Don Seiler, Enigma.

[Introduction to Currier's talk at a National Security Agency historical meeting: Captain Prescott Courier was a cryptanalyst in the U. S. Navy's Op20G from 1936 to 1941. He was a member of the negotiating team sent to GC&CS at Bletchley Park [in January 1941], our first mission of cooperation with the British. In February, 1943, he moved to Nebraska Avenue [headquarters of naval communications intelligence] and was assigned to Op 20GC as a senior watch officer, where he stayed until 1945. From there he went to the Russian Language School at Boulder; 1946 to '48, he was the naval representative to Britain's Government Communications Headquarters; 1949 director of research at the Naval Security Group; in 1951 office of Naval Intelligence; 1952 to '54 officer in charge of the Naval Security Group stationed at Kamiseya, Japan; 1955 to '59, senior U. S. liaison officer, London; 1959, director of operations Naval Security Group and then back at NSA. This is one of the real giants in the business. Captain Currier.]

I'm no giant. It took me two breaths to get up those stairs. I'm reminded of something - before I start - some years ago I served on a petit jury along with a friend of mine who was a retired professor from the University of Maine. I complained to him after we got about halfway through that I was never selected for the final 12. I never sat on a jury and tried a case. And he said, you know why? He said it's that bow tie of yours. He said the lawyers don't trust you. They're afraid you know too much.

I thought that some of you might be interested in knowing what life was really like between 1936 and 1940, what we did and how many of us there were. Op 20G in those days was in the back end of the first floor of the sixth wing of the Navy Department. There were eight rooms, total. In the bark room were three Japanese linguists, all civilians, Phil Cate, Fred Woodrough, and Dr. Hoffman, who had been a missionary. In the middle room there was a great hodgepodge of everything under the sun including files, papers and that sort of thing. There were probably on the order of about eight or a dozen people in this room at various times, on and off, crypto clerks, cipher people, floor-sweepers and an occasional cryptanalyst - me. In the next room, there were two of us - Mrs. [Agnes] Driscoll and myself. We were the only acknowledged cryptanalysts there at the time. There had been others. There were other naval officers who had been taught, some by Mrs. Driscoll, some who had learned it on their own, but at that particular time - early on, '36, say - there were only two of us. There were two and sometimes three language officers in the front room. On the righthand side of the corridor was our machine section, which was composed of two card punchers, one collator, one sorter and one printer. We rented them from IBM for about, gee, I don't know, $50 a month, something - anyway it was not very much. And that was it.

By December, 1940, our personnel had tripled and we had moved out of the back of the sixth wing, but we were essentially the same kind of an organization. I had gone on leave home. I'd only been married a few months. The day after Christmas I got a note from Joe Wenger who said, come back immediately. So I got back to Washington as quickly as I could in the wintertime and when I arrived I was told that I was going to be making a trip to England. They didn't know exactly when or under what circumstances. I was told almost nothing else. It later turned out in about a week that I was to be one of two Navy representatives [the other was Robert Weeks] who, along with Abe Sinkov and Leo Rosen from the Army, were going to England to bring with us two copies of the Purple machine, what we had done on the JN25 codes, a copy of the typewriter that the U. S. Navy intercept operators used and an enormous amount of additional material on DF and various other items that the Navy considered to be important in the coding business on this side of the Atlantic. I can't remember precisely what was crated because I never saw it before I left. It turned out - I didn't know this, by the way, before I left - but it turned out that there had been something called the Tizard commission [a British unit], which had been debating a proposition that would have allowed the U. S. and the U. K. to exchange information on intelligence, along with other technical things. Originally this arrangement did not include comint but it was eventually decided, not long before I was asked to return, that we would in fact include an exchange of comint material. Bill Friedman was supposed to have been the man who led the group, but about a week before - which occasioned my being called off leave - had gone to the hospital with a breakdown and he was replaced by Abe Sinkov.


Now, strangely enough, I was given no instructions. I wasn't told where I was going. I knew roughly what we had with us. I was not told what to ask for. I wasn't instructed how to behave or what to offer or what to say or even who I was to see, which certainly is rather strange now, but it didn't [seem so] at the time. I didn't really give it a second thought. In any event, the four of us one cold day in January - about the 15th, I guess - eventually were told we'd be leaving. We went down to Annapolis, waited on the dock down there for several hours. It turned out that we were going to England on the HMS King George V, one of the two new battleships that the British had just built. The Prince of Wales was her sister ship. The Prince of Wales, by the way, was sunk by the Japanese a little bit later. We waited there for a longish length of time and eventually us and all our gear were loaded into a motorwheeler and we went out into the road where the King George was anchored. When we got there we couldn't go aboard because she also had aboard Lord Halifax, who was to be the ambassador to the U. S. He had not seen fit to debark so we couldn't go aboard until he left. So we sat out there in a bumpy sea in an open boat full of a lot of gear that we didn't want to get wet waiting until the future ambassador to the United States came ashore. He eventually did, of course. We immediately went aboard. All the gear was taken care of beautifully. Arrangements obviously had been made and they not only expected us but they knew precisely who we were, what we had and how much room it would take to store it. The trip was relatively uneventful. We went out east, joined a beef convoy coming up from Argentina, and it took something like 12 or 13 days, we arrived at Scapa Flow on the 7th of February, I believe, with no trouble on the way, not a shot fired in anger.

At Scapa Flow we were told we would be flown from there south in a Short flying boat, of which there were two there waiting for us. It turned out that the people who ordered the flying boat hadn't measured the hatch through which all our gear was supposed to go and it was only about half wide enough. A cruiser, the Newcastle, I believe, was there and her captain let the captain of the King George V know that he was going south and he'd be very pleased to take us along. So, we got alongside and we loaded all our gear, transferred it to the Newcastle - sorry Neptune, wasn't the Newcastle, the Neptune - and we left the following morning alone. We were going down the swept channel on the east coast of England where all of the convoys traveled. The following noon we approached a convoy - we were going a lot faster than they - and so our captain obviously elevated to get by and get south as soon as he could. In the course of doing this, a reconnaissance bomber came overhead and saw the convoy and saw us and we all knew what this meant - that in about 20 minutes it would begin to be noisy. True enough, it did, and we all went below decks and sat there. I was trying to eat some soup but my mouth was so dry I couldn't swallow it. I've never been so scared in my life. We heard a bomb landing on one side and one on the other side and the ship would bounce out of the water and back down again. Then I heard something that sounded like someone dragging chains along the deck. The German were making strafing runs and eventually when it was all over, we went topside. All our gear, by the way, which was in large wooden crates, had been stowed topside. We went to look at them and fortunately all of the ammunition was antipersonnel which exploded on contact and when it hit the heavy wood crating, it exploded and left a little hollow but it never penetrated so that none of the gear that we brought with us was damaged in the least. We put into Sheerness that night and we were met by two staff cars and a truck. We were introduced to the - I've forgotten who it was, Sandwith, I think, Navy captain, plus the drivers. We were driven through the night up to Bletchley, which was about 120 miles to the north. We arrived there well after dark - eight or nine o' clock probably - and were driven through the gates to a large, forbidding-looking brick building with all the blackout curtains drawn. [We were taken] into a room and there standing behind the desk were four men. One of them, [was] Alastair Dennison [the director of GC&CS], one of the former members of Room 40 in the Admiralty. Jumbo Travis was also there. He later became director. John Tiltman, whom many of you know, was there in his full regimentals with his hands behind his back. We were introduced and we made a few inconsequential remarks and we were driven off to our billet. Now our billet was the country home of the chairman of the board of the, at the time, Anglo-Persian Oil Company, and an extremely wealthy man. He had turned over his entire house to these four traveling Americans, fully staffed, the larder full, with a butler and three upstairs maids and a cook to take care of us for the time that we were there.

The British split us up. The Navy group and the Army group never traveled together. We each had a car and our own driver, so [the Navy] never went to the same place as they [Army] did - at least [not] at the same time. We weren't intentionally kept apart. We saw each other almost every night, but we often did our own thing and saw the things that we, as service representatives, were interested in. I was struck by the real friendly feeling of complete cooperation which belies some things that have been said about this little trip of ours by others who've written about it since then. We went everywhere, including Hut 6. We watched the operation of the bombe. We were told in great detail the solution of the Enigma, who worked on it, how the bombe worked, what they had to do in order to get a readable text. Bob Weeks and I were not Enigma mathematicians or cryptanalysts but Abe and Leo Rosen were and it was left to them to do whatever would be done in making the necessary notes and records that we would take back with us.


We spent a good deal of time at Bletchley, Bob Weeks and I, principally in the Naval Section but around in various others also. We spent many days traveling around to the north of England and to the south visiting Royal Navy intercept stations, up at Scarborough, for instance, where we were introduced to - we had tea in the Wrens barracks - and we were introduced to the immobile Wrens and the mobile Wrens. The mobile Wrens were the ones who could be sent to Singapore. We also spent time at Flowerdown, at Winchester, at the intercept station there and several days traveling around the south of England, one of them which I think is quite memorable, really, we went down to Dover Castle and we went up on the cliffs where the British had installed their radar, which all of you have seen in pictures, I'm sure. The radar was manned by [the women's auxiliary of] the Royal Air Force - alleged to be the prettiest women in the armed services, and they'd never let anyone forget it. There were, as a matter of fact. We went in and watched the screen of CRTs, watched the screens, and then of course while we were there, a German bomber group took off just over the Channel and one of the girls came up to me and tapped me on the shoulder and said, in about three minutes, go outside and look up. So I went outside - all of us did - looked up and here came the German squadron of bombers, contrails very obvious. And over this side, the Spitfires taking off. And we just watched them as they came together. And then they got rather mixed up for a bit and - you could follow this whole thing from the ground - a whole squadron of bombers suddenly turned around and back they went and the Spitfires came back and landed. That was during the time when daylight bombing was still being undertaken by both the British and the Germans, but it was a very expensive way of doing business and in that particular case the Germans decided they'd try it again another day.

The extent of what we actually learned and what we were told was as much as we wanted to know and we were told a good many things that we didn't know we wanted to know until after we were told that the information existed. Now of the material that we brought the Purple machines of course, were the most important. That was the sort of thing that the British wanted the most. One of the things that we wanted from them the most was all the information that they had on the Enigma, the structure and manufacturing of the bombe, which was an Enigma analog actually. They did not have a spare Enigma machine to give us. But they did allow us to spend as much time as we wanted to in Hut 6 watch the operation of the bombe, have someone there to explain precisely what was going on, how the menu was fed in, what happened when the red lights went on and when they discovered that they had a match. All extremely interesting and very valuable. Even though we did not in fact bring back an Enigma, we brought bark all of the information that we really wanted and there was never any question that anyone was holding anything from us.

We spent a lot of time visiting around. For instance, I spent a lot of time over in the Naval Section which was run, by the way, by a man named Frank Birch, who was a movie director. His wife was an actress. He ended up, by the way, by writing the [GC&CS] history, some eight or nine volumes, an enormous work. I got to know him fairly well. In fact I stayed in his house a couple of nights. They had, by the way, a lot of material that I thought might be interesting to the Navy, all German material, such as coastal charts and recognition codes for the E-boats, mostly captured. There were no copying machines in those days. Cameras, yes, but not Xerox. So I spent something on the order of two weeks tracing maps of the Channel and copying recognition codes. It would have taken possibly 20 minutes total if we had had a copier at the time.

The Admiralty, in the form of NSD9, which was the equivalent of Op20G, [was] commanded by Captain Humphrey Sandwith at the time. His No. 2 was Clive Loehnis, who later became the director of GC&HQ and whom we met then and whom we've kept in touch with all these years. But I was advised never to ride with him. He drove what was called a flying bedpan and he offered me a lift down to Winchester to see the intercept station and I was strongly advised by several other people in NSD9, don't take him up on it. Make your own way. Which we did. However, he later, by the way, when he was director of GCHQ, drove his Jag up to Cheltenham and ran off the road and injured himself to the point where they weren't quite sure he was alive. But that's the only bad accident he ever had.

The feeling that I got when I was there was only one of extreme friendship and willingness to produce and provide to us anything we saw that we wanted. And they did. We could go absolutely anywhere and they were very good at answering questions and providing the documents that they had.

There were various things, such as learning how to play rounders. Barbara Kenworthy taught me how to play rounders. She married Joe Eachus by the way. He went over there later. But Barbara was my introduction to rounders and she taught me how to play and eventually it turned out that I could hit more home runs than anybody else and I wore myself out running around the bases because I couldn't stop. See, once you'd hit a ball, you had to run until you got back to the home plate.

There were several other very pleasurable instants. I must mention that there were two girls who said they had a car but no petrol and if we could get some petrol, we could go for a ride up in the Cotswolds. So, I went to my Scottish driver and said, Do you have a hose. Can you get me a gallon of petrol out of the car? He said, Sure. So I got a gallon and we gave it to the gals and took a lovely ride up into the Cotswolds.

On another visit, our driver was a Scottie with a very strong Highland accent and none of the English could understand it. We were driving up north and I don't remember where we were going at the time, but we were stopped at a road block by local police. This was done all over England. And here they had a [military] vehicle with two men in civilian clothes in the back seat, obviously not British, a Scottish driver that they couldn't understand, and he got out there and stamped his feet - I cannot imitate him. "See, ya conno do this, they're on a secret mission." And the sergeant nodded his head and off we went to the police station. But it did get cleared up. We did get out and we did come back.


The only other trip that we took where the Navy gathered some material which the Army either didn't know about or didn't want was a new DF [direction finder], some new DF equipment being built by Marconi out in Norfolk. We went out to get it and we were given a complete rundown on its good points and how it was to be set up, all the instructions and that sort of thing, and it was all crated and ready to go and we arranged to have it sent. About two weeks later we finished what we were supposed to do - more of the same that I've been telling you - and we headed north to Helensbrough, up in the Clyde, where we were to board the HMS Revenge for a trip home. We got there and we went aboard and we checked all of the gear that had been put aboard and the only thing that wasn't there was all of this DF equipment. So Bob Weeks and I started at one end of Clydeside and walked through all the buildings and warehouses. It probably was only about a half a mile, but it seemed like three miles. And we found it under an old blanket in the corner of one of the warehouses, alongside some I-beams that nobody knew what to do with. How it got there, I don't know, but we found it. We told them about it and we got someone to come and get it and they put it aboard. We were told that we were in a hurry, that we had to get it aboard that night because they wanted everything squared away. They were leaving early the next morning with a convoy.

Well, early the next morning dawned and we weren't going anywhere. The convoy wasn't ready so we were told that we could take a couple of days off. So we walked up to Loch Lomond, had a couple of good lunches, walked around, came back and the following morning we left. It was the biggest and fastest convoy that had left the Clyde since the beginning of the wax. There were 20 liners, all over 20,000 tons, all capable of at least 20 knots, loaded with troops for North Africa. We accompanied them out to the middle Atlantic, where we left. Now the Revenge was a battleship built in 1911, I believe, for duty in the North Sea, very low freeboard, and it was bumpy during that trip coming back and more often than not there was green water over the bow, so we didn't spend a lot of time out walking around.

But we were very fortunate in having aboard the next high commissioner to Canada, [Malcolm] MacDonald [M.P.], the premier in exile of Poland with his staff, and a couple of other people who were never identified to me and I don't know who they were, but they were important because they all had their own staffs, too. The prime minister of Poland on his staff had a young musician who was a private major in the British Army. Whenever the weather was fair enough, he would move the little piano that was in the wardroom out onto the deck and have a concert for the entire ship's company. Then in the evenings MacDonald - provided the weather was good - would have a discussion group on international politics including the prime minister of Poland and his group, the high commissioner to Canada's group and this other character whom I never identified but was important. And we spent several hours, many evenings, being lectured to by these giants. So coming back was more fun than going over.

Going over, by the way, there's just one thing I might mention, I forgot. In the course of our trip across, eastward, I became friendly with the wireless officer who was a young man about my age and so he took me up into their communications center and took me through the area and into the radar room. That night at dinner the young RNVR [Royal Navy Volunteer Reserve] officer, wavy Navy [sleeve stripes] type, got up. He had had more than his share of drinks before dinner and he called on the captain of the ship. He said, You've allowed these American spies to see British radar. I want you to do something about it. Well, they took him below and stuck him in his bunk. But that shows you how at that time the feeling - at least in the British Navy - was that their radar was the only instrument that was keeping them from being beaten during the war and they damn well weren't going to release the details of it to anyone, including the Americans. I didn't get that feeling once I was in England, but this young man obviously felt that way and he called upon his captain to do something about us, not allow us to see any more British radar and how it worked.

The outcome of all of this was when we got back to Halifax - we hadn't been there more than an hour - when a U. S. destroyer pulled up alongside, a four-stacker, the USS Overton, Bob Weeks and I were on deck and a man got off and walked across. It was a man named Stefanic, a lieutenant commander whom Bob Weeks and I knew, and I thought, Well, isn't this lucky, here we now have a ride back to Washington. I didn't realize at the time that it had all been planned and that he was up there to meet us and take us back to Washington. We got all our gear loaded into the destroyer, and we lashed it down on the deck, lashed it into the bunks as best we could, and we left that night after dark, running without lights. As we got out to the entrance of Halifax harbor, two or three trawlers were coming into the harbor. The captain didn't see them until the last minute because they were also running without lights and we did a quick 180 and everything that we had so carefully laid out and lashed down in the captain's cabin came crashing down onto the floor, everything, all over the place. It was a mess. The deck was covered. Fortunately, there was nothing breakable so we got it all back. It was a bad trip going back. It was bumpy all the way. A young seaman came up to me just before we got back to Washington and wanted to know whether there was anything I could do to get him off the Overton. He didn't like it. But we pulled into Washington Navy Yard in the middle of the day. We were tired. We probably looked a bit bedraggled because we'd been up a lot of the night. We did have a lot of gear that we had to get ashore and as we came ashore, Don Seiler, who had a machine shop down in the Navy Yard - he was the one, by the way, who made the first analog of the RED machine - he just happened to be there. He saw us coming off and he made a remark to one of his friends, Here they come, they look like whipped dogs and they don't have anything. This got round and you can imagine what some writers made of this remark of his that later got out, got in print and the first thing you know that became gospel: We went over there, we gave them everything we had, we came back emptyhanded with nothing. It took a long time before that view of our trip and the results of our negotiation ever got put to rest, but eventually it was. I trust that this [talk] has let some of you know precisely what went on in those days, how we did it. We were not giants. The problems were simple.


Introduction to the Technology section of the third collection of CRYPTOLOGIA articles

Cryptologic Technology

by Cipher Deavours

It seems only yesterday that Artech published the first collection of CRYPTOLOGIA articles. But, it was 10 years ago. In that time, much information has come to light concerning the history of cryptologic technology.

The reader will find this section of Selections from CRYPTOLOGIA: History, People, and Technology comprised of 19 articles of which 14 are concerned with the all important World War II era, and of these, six articles deal with various aspects of the Enigma cipher machine, including the English translation of Gilbert Block's meticulously researched Enigma chronology, Enigma Avant Altra. As more details are revealed about cyptologic development between 1920 to 1945, it becomes truly difficult to believe that so much could have happened in such a relatively short span of years. Machine cryptography was born during this time and matured rapidly during the catastrophe known as World War II. What might be called the world's first digital computers were developed during this war to cope with intricacies necessary to unravel the increasingly complex ciphers generated by the new cipher machines. Many of those early cryptanalytic devices performed computational feats with speeds that leave modern microprocessors in the dust, as has been demonstrated repeatedly.

Interest in the German Enigma cipher machine remains as high as it was 10 years ago. At least one site on the World Wide Web is devoted exclusively to "Enigmatology." Historians and cryptologists keep finding new and significant information about the Enigma story. The "autoscritcher," an interesting and powerful derivative of the British bombes, is described in one article of this series. Even more recently, details concerning the Enigma variant used by the Abwehr have been disclosed. The Enigma chronology is important because we now know so much about the politics, technology, and use of that machine that we can see more clearly, than in most other instances, just how these factors work together and contribute to the security or insecurity of a cryptographic system.

Gordon Welchman, the author of the Hut 6 Story and inventor of the bombe's diagonal plugboard, often remarked that he believed that hard lessons concerning communications security learned with the Enigma were being lost. There are many people in the cryptologic field whose lack of knowledge about the history of their subject causes them to repeat the same errors made decades ago—only this time with a computer. College and university courses in cryptology abound these days, and, in these courses, "classical" cryptology is often dismissed with a few waves of the hand as interesting but largely irrelevant to the computer era. The fault in this case is with the instructors who sometimes do not have enough background in this subject to see connections between the past and present. Old techniques and methods are rediscovered and thought to be new. History is important. Without the anchor of historical knowledge, technological advancement is not assured and one is adrift in a lonely sea of disconnected results that cannot be synthesized into useful cryptologic knowledge. CRYPTOLOGIA and Artech House are making available an archive not likely to be matched anywhere else. It is a treasure to read, appreciate, and use.

Although the importance of the World War II era to cryptologic technology can hardly be overestimated, there exists much material from earlier days which has gone largely unnoticed. The Victorian age was an especially remarkable one. Two articles in this section present new material on the ingenuity and efforts of Victorian cryptographers to construct secure cipher systems. Charles Dodgson (a.k.a. Lewis Carroll), the author of Alice in Wonderland, had an intense interest in cryptography and some of his work with ciphers is presented in this volume.

We have only one article related to the cryptology of the Vietnam era. The cryptologic side of that war is an interesting and varied story that remains to be told. All types of cryptographic systems were in use during the Vietnamese war, ranging from simple cardboard devices, to commercial cipher machines, to high tech electronic systems. This fact may come as a shock to those who believe that the electronic age has relegated hand and mechanical systems to the history books. As the years pass, we hope to know more about the cryptologic history of that conflict.

So, we recommend that readers consider these samples from the history of secret writing and draw whatever lessons seem relevant to their current pursuit. Lambros Callimahos, who was, in many respects, William Friedman's successor, was once asked if he thought that something had been lost when cryptology became electronic. The coldness of computer algorithms and the mysteries of the circuit board had replaced the older mechanical and hand systems where one could achieve a good grasp of how things worked. Callimahos immediately made a remark which, I for one, did not understand until years later. He said, "Well, somebody has to run the machines, right?" Indeed, he was right.

Viet Cong SIGINT and U.S. Army COMSEC in Vietnam

Lt. Gen. Charles R. Myer

[Very little information has come to public notice about Viet Cong successes in reading the messages of the United States Army and the Army of the Republic of Vietnam during the Vietnam War, or about U.S. Army attempts to secure its messages. The following extract may be regarded as a first attempt to at least partially fill that gap. It tells, for example, about how Viet Cong interception enabled those forces to avoid the devastation of B-52 bombings, and, on the other hand, about American difficulties in getting soldiers to secure communications and in getting equipment to work. The extract consists of chapters 7 and 8 of an official history in the series Vietnam Studies, Division-Level Communications 1962-1973, by Lieutenant General Charles R. Myer, a career signals officer who served twice in Vietnam. The work was published by the Department of the Army in 1982. — David Kahn, Ed.]


Chapter VII
COMMUNICATIONS SECURITY THREAT

Few communications subjects can arouse more controversy among professional soldiers than that of security. The argument centers on security versus responsiveness. The points under debate usually involve the techniques used to try to gain some degree of communications security, such as changing call signs and frequencies and using codes for the coordinates of locations and other critical data. To many people, the ideal solution to all but the frequency problem would probably be a small, lightweight, durable, unclassified black box which would automatically scramble and unscramble transmissions so that only those people with a similar black box could understand the transmission. The black box was not available when U.S. troops were committed in Vietnam.

The security measure of changing call signs was very controversial on two points. One was the confusion created by changes. Colonel Sid Berry, the commander of the 1st Brigade, 1st Infantry Division, from June 1966 to February 1967, spoke for many commanders when he said, "It simplifies communications for units and individuals to keep the same frequencies and particularly call signs. Frequent changing of call signs confuses friends more effectively than enemies." The other point was that new call signs frequently cast a poor image or were too long. Although there were many complaints about the denigrating image reflected by call signs, one of the most vivid was provided by Lieutenant Colonel Norman E. Archibald when he recalled his 1970 experiences as division signal officer of the 1st Cavalry Division: "The signal battalion still had a difficult time selling call signs like 'Supreme Capon' (. . . the commander said, 'What? You want everyone to call my troops castrated chickens?') . . ." Brigadier General William S. Coleman, in 1967-68 assistant commander of the 1st Infantry Division, expressed both objections: "Another change that I hated to see was the fancy call signs that the division was ordered to adopt. Imagine being on the radio and saying, 'Sailing Gerta Delta One One Juliet, this Mister is Taboo Four Four.' Why, by the time you get the call signs out, you're forgotten what you wanted to say."

Although these and other complaints were understandable, a communications security problem did exist in Vietnam and had to be dealt with. Early in the war, many people apparently had assumed that the enemy was unsophisticated and that communications security did not warrant much concern. That may have been true when American troops first arrived, but the enemy quickly adjusted.

It soon became evident that he was using poor communications security to good advantage: U.S. troops found their own radio equipment when they swept enemy positions; the enemy might disappear from a location just before a planned U.S. attack; B-52 bomber strikes did not produce expected results because the enemy apparently anticipated them. Some plans probably leaked to enemy agents operating within the Vietnamese political structure at the province and district levels, where Americans had to get political clearance for operational areas to ensure that there was no conflict with local defense forces. But even these suspected leaks could not explain the problems in established free fire zones and already authorized areas of temporary operations. There had to be another source from which the enemy was obtaining critical information.

There was. On 20 December 1969 elements of the 2nd Battalion, 2nd Infantry, and of the 2nd Battalion, 28th Infantry, both from the 1st Brigade, 1st Infantry Division, overran an enemy installation about four kilometers north of Ben Suc in the II Corps zone and captured twelve enemy soldiers. They also seized an assortment of documents and communications equipment, including three U.S. Army FM radios of the AN/PRC-25/77 variety; one Chinese Communist AM receiver, compatible with the U.S. AM radios of the AN/GRC series; seven Sony transistor radios; one Panasonic receiver; one homemade receiver; and one homemade transmitter. All the equipment was in excellent operating condition, and the homemade receiver and transmitter reflected a very high quality of workmanship. With this equipment, this Viet Cong and North Vietnamese Army radio intercept unit could monitor and exploit virtually all nonsecure voice and manual Morse code communications among U.S. and allied tactical units within receiving range. Interrogated prisoners indicated that, along with building up equipment, the enemy had instituted a large number of English linguists. These linguists became an integral part of many Viet Cong and North Vietnamese units such as the intercept unit.


Among the captured documents were several booklets containing extensive instructions on proper intercept techniques and detailed analyses of the communications procedures and exploitable weaknesses of U.S. and allied units. Specific comments were included on the communications procedures of the 1st Infantry Division, 1st Cavalry Division, 4th Infantry Division, 25th Infantry Division, 11th Armored Cavalry Regiment, 5th Vietnamese Division, and the various Military Assistance Command advisory teams. These booklets were current, lengthy, and very detailed. On point of origin codes the instruction was, ". . . they usually use landmarks or a PO (point of origin) from which they use LEFT, RIGHT, UP and DOWN to designate a position." In one journal entry a 1st Infantry Division unit message had been copied: "Presently my one six element is at CPT Coutine (actually Checkpoint Canteen) R.6U2.2." The actual coordinates were written above the entry. That decoding points of origin and shackle codes appeared frequently throughout the journal indicated that deciphering such codes apparently posed no major problem.

The unit also noted the importance of obtaining information from warning nets. By monitoring these nets the enemy could get data on artillery and air attacks and transport of wounded. He could extract all intelligence concerning the units engaged, fire bases, landing zones, and air reconnaissance. He could also get information that would help analyze the traffic intercepted from U.S. infantry communications. The journal identified U.S. advisory and South Vietnamese unit nets as very productive for intelligence data. It further noted that the South Vietnamese made monitoring easy because they never changed call signs or encrypted messages. The journal gave examples of B-52 warning messages. It instructed the monitors to be sure to get the coordinates because they represented the box, or target area, which had to be reported for the security of their own troops.

The captured radio intercept unit was one small team targeted on several American and Republic of Vietnamese units. A logical assumption is that by 1969, when this unit was captured, the enemy could field a number of such teams. This attribute of the enemy in Vietnam must be accorded any future enemy; that is, his ability to adjust to American operational techniques and to improve his capacities. The enemy in 1969 was not what he had been. As the situation changed, he changed.

These examples are but a few of how the enemy used U.S. communications means and procedures to gather intelligence and to assist his operations. There are numerous instances on record of the enemy jamming radio frequencies and sending false messages. These bogus transmissions used imitation to try to turn fires or forces to an area chosen by the enemy. In one case the enemy tapped the internal telephone lines of a defensive base and diverted reserve forces from the area where he attacked.

General Abrams, then head of the Military Assistance Command, after examining the captured documents and being briefed on the radio intercept team incident, summed up his reaction: "This work is really rather startling; the attention to detail, complete accuracy, and thorough professionalism is amazing. These guys are reading our mail, and everyone will be informed that they are."

Chapter VIII
SECURITY RESPONSE

In the history of strategic military operations, it has long been public knowledge that Allied operations in the Pacific during World War II were indebted to the breaking of the Japanese operational communications codes. More recently it has been revealed that the British were able to break the strategic communications code by which Hitler received information and issued directives to his senior commanders. High command levels should have and did get top priority in American attempts to safeguard communications against similar breaches. With the men at the tactical level not really wanting signal security gear, there was a relatively modest attempt in the U.S. Army to develop new equipment for that level until the need became more evident.


Signal security equals a combination of people, equipment, and systems. People participating in tactical operations have a natural dislike for signal security measures; they prefer to conduct business in the simplest, most direct manner possible. Early in Vietnam, possible in most cases meant whatever superiors would permit in the tactical situation involved. The people part of the equation did not improve until the chain of command became convinced of the need for strict communications discipline. The conviction grew in proportion to information about the threat, and interest and participation grew as equipment and systems improved. The signal security problem was never fully resolved during the Vietnam era despite many improvements and much hard work. It was, and still is, one of the most difficult problems facing tactical communicators.

Along with the need to communicate as rapidly and directly as possible during combat, another factor worked against effective communications security: a soldier in the thick of battle feels the need for friendly assurance. This assurance can come from talking—to anyone else who has a radio. When this happens, chances are that little thought is given to what information is actually being transmitted.

Before secure voice equipment was in general use, oral communications requiring any secrecy had to be painstakingly coded and decoded manually. All too often coding was circumvented in favor of simplicity, speed, and assurance. The tactical communicator needed some type of voice encryption device.

In 1965, when he was the U.S. Army, Vietnam, signal officer, Colonel Kenneth Ring recalled that there were some eight hundred KY-8 security devices in a stateside depot. These devices had no mounting brackets or connecting cables because their application had not been settled. Colonel Ring requested, on behalf of U.S. Army, Vietnam, that a new equipment training team be sent to Vietnam to demonstrate the devices and discuss their tactical uses; the team arrived in August 1965. From this start, the security device applications and improvements progressed through some tortuous evolutions.

Voice security devices were issued to the field units in Vietnam beginning in 1965 with the KY-8 for stationary or vehicular use; the KY-8 was fully distributed by the third quarter of fiscal year 1968. The KY-28 was issued for use in aircraft beginning in 1967 and ending a year later. The KY-38, for manpack or mobile use, also was issued initially in 1967 and was fully distributed in 1968.

The voice security gear, like most newly developed equipment, had its problems. The main cause of failure was heat. The KY-8 had to be kept in well ventilated surroundings and away from direct sunlight. During a 1969 presidential visit, an overheated KY-8 temporarily stopped secure voice communications between II Field Force headquarters and Tan Son Nhut.

The 199th Light Infantry Brigade station at Xuan Loc, in the II Field Force commanding general's secure FM voice net, continually broke down. Many pieces of equipment were generating heat in a poorly ventilated bunker and causing the KY-8 to fail. When it was moved to a cooler location, the KY-8 operated normally. Commanders and communicators became aware that, for reliable communications, the security equipment had to operate in the coolest possible environment.

In at least one case the ingenuity of a well intentioned communicator backfired. First, he placed a filled Lyster (water) bag over the KY-8 to cool it, but the device still overheated. Next, he removed the cover of the KY-8 to increase ventilation. That improved the operation of the KY-8 but violated security by exposing the equipment to view and giving the enemy an opportunity to intercept intelligible signals.

In a successful innovation, the 125th Signal Battalion engineered and installed a secure land line to expedite spot reports between the 372nd Radio Relay Company and the 25th Infantry Division G-2. The battalion, operating two KY-8's in essentially a back-to-back test mode, replaced the short back-to-back cable with spiral-four cable of the required length. This system worked with almost no trouble and with very high quality.


During September 1968, the PRC-77 and KY-38 combination was issued down to the infantry battalion level, providing for the first time secure voice communications to infantry companies and in the battalion command nets. At the company level plans and operations could be discussed rapidly, safely, and explicitly without fear of enemy detection. Field units used two methods to offset the weight disadvantage (over 50 pounds) of the PRC-77/KY-38 package. One was to use the equipment fully by having one man carry the PRC-77 radio and another the KY-38. The second was to move the field unit as normal, then have supporting aircraft fly in the equipment for use in static or night defensive positions.

In the fall of 1969, reports on NESTOR (code name given narrow-band secure voice equipment) utilization were revealing. Many units under the operational control of the II Field Force were not using their KY-38s as much as desired. The PRC-77/KY-38 when carried by one man, as intended, was just too heavy. Other units reported that they needed VRC-12/KY-38 interconnecting cables. They wanted to use the smaller KY-38 in lieu of the KY-8. The KY-8 was prone to overheat and ran on generated power. Power was limited at fire support bases, and the field troops thought it unwise to run the noisy generators at night. The KY-38, on the other hand, was battery powered. Acting on this feedback, Electronics Command Laboratories went into an emergency production of interconnecting cables for the VRC-12 and the KY-38 and by late 1969 had produced three hundred. A substitution of the KY-38 for the KY-8 in vehicles was also agreed upon.

One of the most serious equipment problems in the field was the lack of kits and special cables for installing NESTOR gear in aircraft and vehicles. The installation kits and most of the cables was a supply responsibility of the Electronics Command. There was a variety of kits which adapted NESTOR equipment to the tactical series radios used in vehicles and aircraft. The initial supply of kits, once they had been developed, was adequate early in fiscal year 1969. As the use and uses of the equipment increased, however, kits and cable components became increasingly hard to find. The so-called X-mode cable for the KY-8 was also in short supply. Another aspect of the kit shortage came to light during the U.S. withdrawal from Vietnam. Although retrieving the NESTOR equipment from vehicles was no problem, redistributing it was often haphazard. A similar situation existed with KY-28's in aircraft, principally helicopters. Losses and evacuations of aircraft which had been fitted for the equipment made the shortage worse, since most of the replacement aircraft did not come with the kits installed.

The number of KY-28 keying devices for the NESTOR family of equipment was adequate when units were operating in one locale; when divisional units were dispersed (the normal practice), more keying devices were needed. The 101st Airborne Division recommended that keying devices for the KY-28 and KY-38 be supplied as required. In July 1969, II Field Force established a common NESTOR key list for all units operating in the III Corps zone. To maintain compatibility, key changes had to occur simultaneously in all units. The time chosen for this change was midnight, tactically the worst possible time because the greatest number of enemy contacts occurred from 2200 to 0200. Moreover, where several units shared the same keying device, having to move at night to change key settings was inconvenient and dangerous and added to the reasons for not using the equipment. Later the time of the daily NESTOR key change was moved to 0600.

As the use of communications security equipment increased in Vietnam, so did the need for logistical support. A reorganization was approved by the commanding general of U.S. Army, Vietnam, in mid-1967. The revised structure consisted of the Communications Security Logistics Support Center, Vietnam, which functioned as a general support facility serving U.S. Army, Vietnam, with six subordinate units strategically located in the four corps areas. The center was also assigned the mission of organizing, training, and deploying seven contact detachments to augment direct support to the combat divisions. Support began to improve.

Security codes were another problem. The early units had not been very conscious of communications security, and the unauthorized practice of using homemade communications security codes and shackles eventually became widespread. This dangerous practice, which persisted throughout the conflict and which gave users a false sense of security, was probably more dangerous than communicating in the clear. As the number of units burgeoned and the need for compartmenting code systems grew, U.S. Army, Vietnam, requested and received from the National Security Agency a whole series of preprinted codes to cover the expansion. These codes, while not perfect, were a great improvement in both the employment and distribution of such systems. Getting the infantryman under fire, or the helicopter pilot supporting him, to employ the codes was another matter.

The continuing need for easier authentication and more reliable means for passing limited classified traffic led to a new development in the later stages of the conflict: the KAL-55B authentication wheel, or "whiz wheel," a circular authentication table in a plastic covered disc. This innovation simplified communications security measures to a level acceptable to most users.

Signal security, particularly in voice radio transmissions, was a major problem area throughout the period of combat operations in Vietnam. Army Regulation 380-40, Department of the Army Policy for Safeguarding Communications Security Information, assigns responsibility for this type of security to both the commander and the individual. All users of communications facilities were more or less aware of their vulnerability to enemy intercept, analysis, and decoding, and of the need for authentication and encoding. The gap between this knowledge and actual practice was immense, and in Vietnam it seemed at times an insurmountable problem.


Some Victorian Periodic Polyalphabetic Ciphers

Francine Abeles and Stanley H. Lipson

ADDRESS: Department of Mathematics, Kean College of New Jersey, Union NJ 07083 USA.

ABSTRACT: Charles L. Dodgson, better known as Lewis Carroll, invented several polyalphabetic ciphers by 1868. The two that he published are put into historical perspective in this paper. His Alphabet Cipher produces a Vigenère enciphered text, while his Telegraph Cipher is equivalent to a Beaufort cipher. In constructing the Telegraph Cipher, Dodgson used two sliding alphabets, similar to the St. Cyr slide invented by Auguste Kerckhoff fifteen years later. A program is available to explore these ciphers and another, the Variant Beaufort.

KEYWORDS: Dodgson, Carroll, cryptography, cipher, computer, Beaufort, Vigenère.

BACKGROUND

In 1843 the first electric telegraph was opened from Paddington to Slough on the Great Western Railway. The telegraph increased the volume of communications, particularly signal communication - command and reconnaissance messages. The telegraph allowed commanding officers, for the first time, the opportunity to control large armies in the field. But as military communications became more effective, the possibility of message interception increased. Tactical messages required protection - telegraph wires could be tapped. The older methods of secret code reliant on code books became outmoded because they were too easy to capture and too hard to reissue rapidly to the many and widely dispersed telegraph stations. The solution lay in a forgotten branch of cryptography - the cipher: simple, easily acquired, easily read, and secure. Ciphers could be printed at low cost on one sheet of paper and readily distributed. By using different keywords secrecy was guaranteed and even if one of the keys was learned, the entire system would not be compromised. So the field cipher was born.

The first system to be used by the military already existed. It was the Vigenère cipher, a form of polyalphabetic substitution invented in the 16th century. It fell out of favor because it was comparatively slow, difficult to use, and prone to error compared with nomenclators, the prevailing system, consisting of lists of names, words, syllables and numeric replacements. This cipher was considered unbreakable (until 1917) when in reality it had been broken in 1863 by the Prussian Major Friedrich Kasiski. His 95 page book, Die Geheimschriften und die Dechiffrir-Kunst went unnoticed, however, so the world did not know that all periodic polyalphabetic ciphers (those with repeating keywords) were not secure!

THE VIGENÈRE CIPHER

The cipher known by this name in the 19th century is actually not the one invented by Blaise de Vigenère (1585) but a simpler and less secure form of it. Nevertheless, even in its degraded form, it is the archetype of the most famous cipher system in history.

                 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
             A   A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
             B   B C D E F G H I J K L M N O P Q R S T U V W X Y Z A
             C   C D E F G H I J K L M N O P Q R S T U V W X Y Z A B
             D   D E F G H I J K L M N O P Q R S T U V W X Y Z A B C
             E   E F G H I J K L M N O P Q R S T U V W X Y Z A B C D
             F   F G H I J K L M N O P Q R S T U V W X Y Z A B C D E
             G   G H I J K L M N O P Q R S T U V W X Y Z A B C D E F
             H   H I J K L M N O P Q R S T U V W X Y Z A B C D E F G
             I   I J K L M N O P Q R S T U V W X Y Z A B C D E F G H
             J   J K L M N O P Q R S T U V W X Y Z A B C D E F G H I
             K   K L M N O P Q R S T U V W X Y Z A B C D E F G H I J
             L   L M N O P Q R S T U V W X Y Z A B C D E F G H I J K
             M   M N O P Q R S T U V W X Y Z A B C D E F G H I J K L
             N   N O P Q R S T U V W X Y Z A B C D E F G H I J K L M
             O   O P Q R S T U V W X Y Z A B C D E F G H I J K L M N
             P   P Q R S T U V W X Y Z A B C D E F G H I J K L M N O
             Q   Q R S T U V W X Y Z A B C D E F G H I J K L M N O P
             R   R S T U V W X Y Z A B C D E F G H I J K L M N O P Q
             W   S T U V W X Y Z A B C D E F G H I J K L M N O P Q R
             T   T U V W X Y Z A B C D E F G H I J K L M N O P Q R S
             U   U V W X Y Z A B C D E F G H I J K L M N O P Q R S T
             V   V W X Y Z A B C D E F G H I J K L M N O P Q R S T U
             W   W X Y Z A B C D E F G H I J K L M N O P Q R S T U V
             X   X Y Z A B C D E F G H I J K L M N O P Q R S T U V W
             Y   Y Z A B C D E F G H I J K L M N O P Q R S T U V W X
             Z   Z A B C D E F G H I J K L M N O P Q R S T U V W X Y

The example that follows is a version of this polyalphabetic substitution system called The Alphabet Cipher, authored by C. L. Dodgson.[1, p. 1156-57] It was reinvented by Dodgson probably in 1868. The keyword, VIGILANCE is repeated over the message to be enciphered, called the plaintext. The intersection of the column headed by V and the row headed by M gives the cipher-letter H, etc. This encipherment can be represented as

Key + Plain = Cipher.

Keyword | VIGILANCEVIGILANCE
Plaintext | MEETMEONTUESDAYEVE
Ciphertext | HMKBXEBPXPMYLLYRXI

Note that different cipher letters could result from the same letter of plaintext, e.g. M is encoded by V and by L to give H and X. Geometrically, the cipher alphabet is translated behind (to the left of) the plain alphabet a number of places determined by the key letter. This can be seen by using the same letter, say D, encoding the entire alphabet, and observing the three place shift.

Key | DDDDD
 | 01234
Plain | ABCDE
Cipher | DEFGH
 | 34567

So Cipher = Plain + 3, e.g. E = B + 3.


THE BEAUFORT CIPHER

In 1857, another polyalphabetic substitution cipher was reinvented by retired Admiral Sir Francis Beaufort of the British Royal Navy. Its origin goes back to the early 18th century and is ascribed to Giovanni Sestri (1710). Like the Vigenère, it uses an alphabetic square which differs only in the repetition of the alphabet on all four sides, placing an "A" in all four corners.

WARWARWARWA | Keyword
MEETMEATSIX | Plaintext
KWNDONWHZOD | Ciphertext

To encipher the first letter of this example, the table is entered at row M, the plaintext letter, and followed to the tabular entry of the keyletter W; then the column heading above that entry gives the cipher letter K.

Again, a different cipher letter may result from the same letter of plaintext, e.g. E is encoded as both W and N. Geometrically, the cipher alphabet is a reversed alphabet translated forward (to the right) of the plain alphabet. Using the same keyletter, D and encoding the plaintext we obtain

Key | DDDDD
Plain | ABCDE
 | 01234
Cipher | DCBAZ
 | 321025

So Cipher = Plain - 3; e.g. A = D - 3. This encipherment can be represented as

Key - Plain = Cipher,

which in decipherment becomes

Key - Cipher = Plain.

Another cipher, derived from the Beaufort cipher, and called Variant Beaufort can be given by its encipherment scheme,

Plaintext - Key = Cipher.

If we compare this with deciphering using the Vigenère cipher:

Cipher - Key = Plain,

we see that each of these two processes is the inverse of the other.

THE TELEGRAPH CIPHER

In his diary entry of April 22, 1868 Dodgson wrote, "sitting up at night I invented a new cipher, which I think of calling The Telegraph Cipher."[3, vol 2 p.268-9] Both the Alphabet Cipher and the Telegraph Cipher were published anonymously and undated. There is no mention of the Alphabet Cipher in his diary. Dodgson's biographer and relative, Stuart Collingwood, and his bibliographer, Falconer Madan, ascribed 1868 to both ciphers. But we know that Dodgson had produced two ciphers ten years earlier, in February, 1858 — both appear in his unpublished diaries. Roger L. Green had this to say about their omission:

The cipher is too long and complicated to reproduce here. Moreover, Dodgson printed an improved and much shorter method, The Alphabet Cipher, in 1868...and another variant, The Telegraph Cipher, in the same year.[3, vol 1, p, 140]

Apparently Green was unaware that there were actually two different ciphers in the entries of February 23 and 26 and although these are also periodic polyalphabetic ciphers, they are unrelated to the Alphabet and Telegraph ciphers. Because of their novel construction, each will be the subject of a separate paper.

From Dodgson's statement, it would seem that the Telegraph Cipher followed the Alphabet Cipher chronologically.[2] Dodgson communicated his invention to George Ward Hunt, First Lord of the Admiralty. No response is known. Dodgson's enciphering scheme is interesting. Unlike the Vigenère and Beaufort tables, it uses two sliding alphabets: a key letter alphabet A-Z and a plaintext alphabet A-Z,A.

KEY-ALPHABET

            a b c d e f g h i j k l m n o p q r s t u v w x y z
            a b c d e f g h i j k l m n o p q r s t u v w x y z a

MESSAGE-ALPHABET


Dodgson's instructions for using the cipher are to find the keyletter on the key alphabet, slide the plaintext alphabet under it so that the plaintext letter appears under the Keyletter. Then using the appropriate "A' on the plaintext alphabet, the cipher letter appears on the key alphabet above that "A". This scheme produces a Beaufort enciphered text. It is entirely probable that Dodgson was aware of the 1857 cipher. What is curious, however, is that the identification of the polyalphabetic tableau and this cryptographic slide (as we will call the sliding alphabet) was not made until 1883 by Auguste Kerckhoffs, a Dutchman, in a book entitled La Cryptographie Militaire, considered one of the best books on cryptology. He named his cryptographic slide, the St. Cyr system, after the French military academy where it was taught. Kerckhoff's instructions showed the equivalence of the cryptographic slide and the Vigenère tableau. It was not until 1910 that Felix Delastelle, by shifting the position of the key, plaintext, ciphertext and index letters in the St. Cyr slide showed that other codes like the Beaufort cipher would result. He published his work in a book entitled, Traite Elementaire de Cryptographie.

CONCLUSION

Like Beaufort and Charles Babbage, Dodgson was a dilettante in cryptography. Although he did not systematically study ciphers, his inventions show that his knowledge of them was state of the art for his time. Codes produced by periodic polyalphabetic systems were considered best from the mid-nineteenth century until 1939. Indeed the German Enigma machine (WW II) was just an automatic (electric) encipherment of periodic polyalphabetic ciphers yielding incredible complexity! The deciphering techniques used to break it were critical to the birth of the computer. Mathematically, cipher systems enjoy group properties — indeed they are early applications of abstract groups; but more than this, they have philosophical importance. Darwin had shown earlier in the century that the determinism of natural selection depends on the disorder, the random mutation of individual genes. Cipher systems demonstrate the opposite — that disorder in the form of ciphertext can arise from a determinate system, plaintext. By the end of the century it was becoming clear that science recognized many faces of determinism. The march toward the determinism of the computing machine had begun.

NOTE ON SOFTWARE

The authors have written a program, "CIPHER_PLAY," in Pascal for compilation with Borland's Turbo Pascal 4.0. Information about the program is available from Dr. Lipson.

REFERENCES

1. Dodgson, C. L. (Lewis Carroll). 1982. The Penguin Complete Lewis Carroll. Harmondsworth, England: Penguin Books.
2. Dodgson, C. L. "The Telegraph Cipher." Unpublished Manuscript. Berol Collection, New York University.
3. Green, R. L., ed. 1954. The Diaries of Lewis Carroll, v.2. New York: Oxford University Press. 268-9.
4. Kahn, D. 1967. The Codebreakers. New York: Macmillan.

BIOGRAPHICAL SKETCH

Both Francine Abeles and Stanley Lipson hold doctorate degrees in mathematics from Columbia University and have recently acquired masters degrees in computer science from Stevens Institute in New Jersey. Both teach mathematics and computer science at Kean College of New Jersey. Dr. Abeles is the editor of The Mathematical Pamphlets of Charles L. Dodgson (in progress.)

APPENDIX

Readers will probably have little difficulty in finding Dodgson's errors and in recovering both the plaintext and the key for the following1:

1From an unpublished manuscript by C. L. Dodgson in the Houghton Collection of the Pierpont Morgan Library: Houghton Alice dep. 529. Cited with the permission of the Estate of Charles L. Dodgson.
"Jgmu qjl vgrv x ugemdt pupdeto?" wxxl x ugmh vj f jji,
"Ge'n ijsk tuckcbb qfds fb qrug eq xud eyk cxdmfit  ddjdef:
Fbu cgkskg mglb gf mstutt, had vbj, okc Ljudz jgmu nt jxxmh,
Pa St'ok krv eykgb vj dogoq grukc vqb zmfvtn as tuemak exjmh
Wrs-brs, brs-brs, wrs-brs, brs-brs; "Vrct fdjsi!" fgu Ugmh.
Njj-sjj, sjj-sjj, njj-sjj, sjj-sjj; Jji wxxl "Urbe ek fxdmd."
"Ljs'v efdn ma lb as zapsi hlgvv," wxxl Srr, "zru qrsmxbr yadsl,
Lh jtbv xok urbt jgey zxmkkm imlk, eyoe nvpdef mhtt vj mht zxjlbu,
Xv pn cz rsk xbcbbexak — na Urdmh btblk'm wvrdu,
nht dok'm gkkkkm o ifvtsv ruut mhxm'w brxey genstxiqm gkzamc.""
Njj-sjj, sjj-sjj, njj-sjj, sjj-sjj; "G urbe dogb jjo Ljudz!"
Eab-jab, jab-jab, eab-jab, jab-jab; Ugmh wxxl "Eyoe'n jjudz."


Chaocipher Enters the Computer Age when its Method is Disclosed to CRYPTOLOGIA'S Editors

John Byrne1, Cipher A. Deavours2 and Louis Kruh3

ADDRESS: (1) c\o Cryptologia, Rose-Hulman Institute of Technology, Terre Haute IN 47803 USA, (2) Department of Mathematics and Computer Science, Kean College of New Jersey, Union NJ 07083 USA, and (3) 17 Alfred Road West, Merrick NY 11566 USA.

ABSTRACT: John F. Byrne invented Chaocipher in 1918 and tried unsuccessfully for almost 40 years to interest the U.S. government in his cipher system. He offered a reward to anyone who could break his cipher but the reward was never claimed. In 1989, John Byrne, son of John F. Byrne, demonstrated Chaocipher to two Cryptologia editors to determine if it had any commercial value. After making some improvements and providing additional information they jointly issue a new challenge to would-be solvers.

KEYWORDS: John F. Byrne, Chaocipher, Silent Years, perfect accuracy, computer algorithm, John Byrne.

BACKGROUND

In his autobiography, Silent Years[1], John F. Byrne, a lifelong friend of James Joyce, devoted the last chapter to Chaocipher which he had invented in 1918. Byrne described his attempts starting in 1920 to interest the State, War, and Navy Departments in his indecipherable cipher and his frustration with the disinterest shown by William F. Friedman and other cryptanalytic experts after he had demonstrated his machine.

In a letter to Friedman, February 17, 1957, who had written earlier about "why you put in that book the completely extraneous matter of ciphers when you had so much of interest to tell of Joyce," Byrne admitted that "my main reason for writing the book was to be on record in the matter of my 'Chaocipher'." Byrne was still so convinced of its indecipherability that he included an enciphered message in his book and offered a $5,000 reward to anyone who solved it within three months after the book was published. He specifically challenged Bell Laboratories, where he had also demonstrated his machine, all "cryptanalytic and cryptographic" experts, and the American Cryptogram Association and its local affiliate, the New York Cipher Society, to solve the cipher. The reward was never claimed.

Since then, however, many attempts have been made to solve the Chaocipher. (For an excellent analysis by G. E. Mellen see [4].)

Over the years many hypotheses about the cipher have been suggested and most of them were demolished in Mellen's article, e.g., it is a crude rotor system, contains an element of transposition, autokey cipher, Vernam tape system, and a form of polyalphabetic cipher with a random nonrepeating key.

An overlooked, possibly valuable clue is in The Editor's Notebook of the American Cryptogram Association 1952-1956. This is a detailed chronological diary kept by Henry E. Langen who was editor of the ACA's magazine, The Cryptogram, during that period. The diary contains a letter from Byrne, December 10, 1953, in answer to Langen's request for a demonstration of Chaocipher. Byrne said he would be delighted to demonstrate his apparatus and after many postponements, Byrne, accompanied by his wife, visited Langen in New Jersey on May 25, 1954. For two hours he discussed his book and Chaocipher but he did not bring the cipher machine "explaining that it was too heavy and cumbersome." He did bring blueprints which Langen did not understand. According to Langen, "He did explain that the machine is made up somewhat like a typewriter with two revolving disks with the alphabets arranged along the periphery in a complete disorder." Langen commented parenthetically that "With only two disks used; I am a bit confused as to how this can result in such utter chaotification of the plaintext message."

SEARCHING FOR INFORMATION

In 1973, Louis Kruh, seeking an interesting program for the 1974 annual convention of the ACA which he had volunteered to organize, thought of the unsolved Chaocipher. If he could obtain information on Chaocipher from Byrne's family it would make a fascinating program. Thus began a search for Byrne's heirs. After several months of research a letter about his hunt which appeared in the Irish Echo, resulted in a letter from John Byrne, son of John F, Byrne, with his telephone number. When contacted, John Byrne said he had helped his father with the cipher and was totally familiar with it. He said that perfect accuracy was essential when enciphering a message because one error would distort the rest of the cipher. But, unfortunately, he did not want to reveal how Chaocipher worked.

Kruh, not easily discouraged, continued to call or write to Byrne periodically for the next 15 years. In 1978, Byrne said his mother had died and that he was sorting out his father's papers. He also expressed concern that if Chaocipher had commercial application it shouldn't be taken from the family which was why he was reluctant to explain how it worked. In a call near the end of 1989, Byrne agreed that he had to make a decision on what to do with Chaocipher, i.e., determine whether it had commercial value and proceed accordingly or reveal its methodology in an article. Kruh suggested they meet to discuss how Byrne might make that determination and that to assist he would ask Dr. Cipher A. Deavours to attend the meeting. He explained that Deavours was a professor of computer science, co-author with Kruh of Machine Cryptography and Modern Cryptanalysis [2], an accomplished cryptanalyst, and an internationally known consultant in computer security matters.


Byrne agreed to the meeting and a few weeks later all parties met in Kruh's office in Manhattan. Byrne discussed his involvement with the cipher and his ambivalent feelings about revealing its secret before knowing if it had any value for commercial use. Deavours explained how Chaocipher could be analyzed for use as a computer security algorithm, how it could be protected if developed commercially, and some ways to market it. Byrne asked many questions and was genuinely interested but still hesitant when the meeting ended. But a few days later he called and said he was ready to demonstrate the Chaocipher for our evaluation.

CHAOCIPHER BECOMES A COMPUTER ALGORITHM

Saturday afternoon, March 10, 1990, in Deavours' home in New Jersey, Byrne showed Deavours and Kruh how the Chaocipher worked. They were quickly impressed with its simplicity, ease of operation and security. Deavours immediately suggested how the cipher could be expanded and adapted for implementation in computer systems and the next morning he wrote a computer program to simulate the original Chaocipher. This program was then used to decipher portions of Exhibit 1 taken from Silent Years in order to verify its accuracy.

Attempts to cryptanalyze the cipher appear to substantiate the elder Byrne's claim that ". . . any person on earth using a device similar to my own home-made contraption, could produce a cipher message which would be indecipherable by any other person except the one to whom the message is directed." Certainly, Chaocipher was a very secure cryptographic system for its time and even the original system would not be easy to solve today using a computer. And, an important shortcoming of the cipher - make one error during encipherment and the message is garbled beyond repair - is eliminated by its implementation on a computer.

Byrne says, "Without daring to suggest that my father would have been in the least surprised, I wish he could have experienced the speed, ease, and accuracy of computerized encryption. For me this is a stunning change indeed, considering my memories of tense and tedious hours striving for perfect accuracy while working with my father and his 'contraption' at the dining room table."

A CHALLENGE

In a letter, September 7, 1922, William F. Friedman, responding to a previous question of Byrne's about the type of material he needed to solve the Chaocipher, said, ". . . a series of fifty messages of approximately twenty-five words each might be sufficient . . ."

Of course, in 1922, Friedman was asking for enough text to mount a pencil and paper attack on the cipher. Given the availability of computers today the need for adequate material should be greatly reduced. Accordingly, below are three messages of approximately 25 words each as an "in depth" specimen of Chaocipher (as opposed to the original exhibits in Silent Years, each of which was enciphered in a single continuous key). The plaintext, which is typical English prose, is taken from Stewart C. Easton's book, Rudolf Steiner: Herald of a New Epoch [3].

Thirty-seven years ago, John F. Byrne challenged "experts" to solve an enciphered message and collect a $5,000 reward. That message was never solved and now Byrne's son and his two associates, who have provided additional information about Chaocipher, challenge the readers of Cryptologia to solve the following messages within three months after publication. There is no monetary reward but all solvers who describe their method will receive a photograph of the Chaocipher equipment plus recognition in a future article.

The reader may consider this as Exhibit 5, continuing where Byrne left off in his book Silent Years.

     MESSAGE 1:
MHRMC  DEGBC  WWQFN  SDDQC  UJHYF  YNPYT  QCXVR  CSFUX  QZZLH  VVWIV  SBFDF
QGEXV  NXEAN  SDEJF  RVKYA  RVRED  QDEKZ  QENIL  QZVEV  DXBZP  URCQY  QSHYY
VPLXP  IKMGX  Q
     MESSAGE 2:
ENWSC  EAQGI  VIDEM  WUMSN  ZMNTV  UFDLB  JKKMR  HHSNB  KTJBH  VPTWH  FMQQJ
PGRWF  FVJMD  HFUZO  XEOZT  MKZSA  MJYRL  SQSXU  ZYEKR  JBFRE  SGGFX  FEGXL
PWTWL  ZAVIM  TBDTQ  BLVRZ  VEMMT  LXITZ
     MESSAGE 3:
JZHAS  QNRTK  TTLZD  YOWLN  VDMWN  YHSMG  XJMGZ  QTHRI  WTIFL  XYHTK  BOXUY
EANJU  DXNVO  GFZHM  JEGRD  GGPUG  SXVBA  CBEPK  WHVSB  IJGOH  KVKAI BBNKF
HFFLS  FMIIN  TTXJX  UHWA  PTSNB  TBBNK  FUCBP  IONQS  MVEHU  XTLMR RA

REFERENCES

1. Byrne, John F. 1953. Silent Years. New York: Farrar, Straus & Young.
2. Deavours, C. A. and L. Kruh. 1985. Machine Cryptography and Modern Cryptanalysis. Norwood MA: Artech House, Inc.
3. Easton, Stewart C. 1980. Rudolf Steiner: Herald of a New Epoch. Spring Valley NY: Anthroposophic Press.
4. Mellen, Greg. 1979. J. F. Byrne and the Chaocipher, Work in Progress. Cryptologia. 3(3): 136-154.

BIOGRAPHICAL SKETCHES

John Byrne is an independent design consultant specializing in exhibits and visual merchandising. He is a member of I.E.A. (International Exhibitors Association) and N.A.M.E. (National Association for Museum Exhibition). After living in Ireland, San Francisco and New York, he now travels to clients from his home in Vermont where he and his wife remodeled an old slaughter-house in their "magic circle" . . . a clearing in the woods beside a river. He professes almost complete ignorance of any cipher except the one he grew up with, his father's Chaocipher.

Cipher A. Deavours teaches cryptography courses at Kean College of New Jersey. He also does extensive consulting in the field of Electronic Funds Transfer.

Louis Kruh is an indefatigable researcher and collector of cryptologic memorabilia, particularly cipher devices. His interest in cryptology spans more than 40 years and he responds knowledgeably and promptly to all queries regarding literature and equipment. He is co-author with Dr. C. A. Deavours, of Machine Cryptography and Modern Cryptanalysis, (Artech House, Inc. 1985). Now that he has retired as Director of Advertising for New York Telephone, he is starting a new career as an attorney.


The Key-Vowel Cipher of Charles L. Dodgson

Stanley H. Lipson and Francine Abeles

ADDRESS: Department of Mathematics, Kean College of New Jersey, Morris Avenue, Union NJ 07083 USA.

ABSTRACT: In his diary entry of 23 February 1858, Charles L. Dodgson (Lewis Carroll) described the first of two ciphers he invented within three days of each other. Analyzing this cipher, we show that it is a subset of a Vigenère cipher with the addition of systematically placed nulls. Discussing the Key-Vowel cipher and the later one, the Matrix cipher, we focus on the security of the key-word. Two exercises for the reader are also provided.

KEYWORDS: Dodgson, Carroll, cryptography, cipher, nulls, Vigenère, algorithm.

BACKGROUND

In two earlier papers [1,6] we have reported on Dodgson's work on ciphers. In the first paper we analyzed ciphers invented in 1868; in the second, a cipher from 1858. The subject of this paper is an earlier 1858 cipher that Dodgson described in his diary entry of February 23 [4]:

Invented a system of cipher, which I think looks promising, as it may be carried entirely in the head.

1. Agree on some word as the key: for every vowel in the key you write a letter representing some letter in the message, and for every consonant you write any letter at random.
2. The rule of the vowels is as follows: Suppose M the letter to be represented: imagine it written
                  a  e i  o u
                  K L M N O.

Then if "a" is the key - vowel, write K, and so on.
3. Begin with some unmeaning letters, the number regulated by the first vowel of the key [a = 1, e = 2, i = 3, o = 4, u = 5] and end with some, regulated by the last key vowel.

EXAMPLE

Let the key be i m a g i n e. As there are 4 vowels every 4 letters of the message will use it once, producing 7 letters of cypher. Let the message be come directly. We begin with 3 unmeaning letters, e.g. L A T - then for each letter of the key we proceed as follows:

for (i) symbol for c = C
(m) unmeaning - = H
(a) symbol for o = M
(g) unmeaning - = E
(i) symbol for m = M
(n) unmeaning - = S
(e) symbol for e = D

this gives us LATCHMEMSD.

To retranslate this suppose it written

                        i m a  g i n  e
                  LAT C H M E M S D

Discard the first 3 letters, and every other one with a consonant over it -

As [image] has an i it = c. For [image] we write [image] and run the alphabet on until till we get under i - this gives o. The next [image] having i over it = m and [image] by the same rule gives e.

ANALYSIS

If it is not immediately apparent, the Key-Vowel Cipher can be considered a subset of a Vigenère Cipher by replacing the key - vowels (a e, i, o, and u) respectively with Y, Z, A, B, and C and using just five of the twenty-six available alphabets. Which alphabets are used, and the precise sequence in which they are used, depends solely on the pattern of vowels within the key word. Consider the following tableau:

 | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z
A | Y | Z | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X
E | Z | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y
I | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z
O | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | A
U | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | A | B


Using the vowels in the keyword IMAGINE to index the rows, and the letters of the plaintext COMEDIRECTLY to index the columns, we locate the letters of the ciphertext in the table:

              I M A G I N E I M A G I N E I M A G I N E
              C   O   M   E D   I   R   E C   T   L   Y
              C _ M _ M _ D D _ G _ R _ D C _ R _ L _ X

Dodgson filled in the consonants with random letters:

              C H M E M S D D F G E R X D C O R D L M X,

then added three random nulls, L A T, at the beginning since i-3 is the first vowel, and two, D N, at the end since e-2 is the last vowel in the key - word. Thus the complete encipherment is:

              L A T C H M E M S D D F G E R X D C O R D L M X D N.

Observe that if a key word contains even multiple instances of a single vowel, the resulting ciphertext would be the equivalent of a Caesar type cipher with nulls embedded at regular intervals. Dodgson's message enciphered with SEVENTEEN, for example yields:

      S E V E N T E E N S E V E N T E E N S E V E N T E E N
        C   O     M E     D   I     R E     C   T     L Y
    _ _ B _ N _ _ L D _ _ C _ H _ _ Q D _ _ B _ S _ _ K X _ _

Furthermore, if the key word contained only the vowel 'i', as does MISSISSIPPI, the resulting ciphertext would contain the message in clear hidden amongst the nulls.

Commenting on the Key-Vowel Cipher Dodgson concluded his diary entry:

The great advantage which I think this has is, that no one, even if he knows the system, can read a piece of cypher done by it, without also knowing the key - word. And this the correspondents may change as often as they like - or even use a whole sentence as key.

For his day, Dodgson may very well have been close to being correct. The introduction of nulls into the message, even in such a regular fashion, was probably too much for Victorian cryptanalysts to handle. These meaningless extra characters obscured the normal word divisions that Charles Babbage had exploited so successfully in the solution of polyalphabetic systems.[5, p.205] Étienne Bazeries, who is credited as being one of the greatest natural cryptanalysts, is known for reconstructing the "Grand Chiffre de 1691," a code with about 600 entries including nulls.[5, pp.245-246] However, the introduction of six nulls at the beginning and end of a polyalphabetic cipher in 1892 gave him such trouble that when he finally did solve it, he came to regard that solution as one of his best.[5, pp.245-246]

Dodgson abandoned and never published the Key-Vowel Cipher: indeed his diary entry three days later described the Matrix Cipher as being "far better".[3] On what basis? The advantages he claimed for the Matrix Cipher repeated the language he used with his earlier effort, with one important difference. For the Matrix Cipher, Dodgson added the claim that even if the plaintext is known, it is not possible to recover the key-word. If Dodgson had found reason to believe that the key-word in the Key-Vowel Cipher was vulnerable, he made no mention of it. Nevertheless his focus on it in the second cipher is sufficient motivation to examine the security of the key-word in the first cipher.

Consider Dodgson's example, and suppose that both the plaintext and the cipher text are available:

                  C O M E D I R E C T L Y
    L A T C H M E M S D D F G E R X D C O R D L M X D N.

The general strategy is to slide the plaintext along the ciphertext looking for possible matches while watching for the inevitable repetitions of pattern to reveal the length of the key. Expanding the plaintext over the five alphabets:

                  A - a m k c b g p c a
                  E - b n l d c h q d b
                  I - C O M E D I R E C ...
                  O - d p n f e j s f d
                  U - e q o g f k t g e

The first letter of the plaintext C could match with the second or fourth letters of the ciphertext. If it matches with the A, its key-vowel is A indicating one null, 'L', at the beginning. If it enciphers as the fourth letter, C, its key-vowel must be I, which indicates three nulls 'L A T' at the beginning. Since the second letter O of the plaintext can only be enciphered as M, N, O, P, or Q, it must correspond to either the 6th or 8th letters of the ciphertext indicating a key-vowel of A in either case. If, however, it corresponds to the 8th letter, there is no appropriate ciphertext for the next letter M until the 19th, and that would leave ten consecutive consonants. E, D, and I could map anywhere within 'D D F G', but if I does not correspond to G, once again there will be a gap with too many consonants in a row. Likewise, R must encipher to the 15th letter which yields:

       Known               O   M         I   R
                           a   i         a   i
                 L A T C H M E M S D D F G E R X D C O R D L M X D N.
                                   E
       Possible    C   C         E D D


At this point it is tempting to try a key-word length of 7 which forces:

                   C   O   M   E D   I   R
                   i   a   i   e i   a   i
             L A T C H M E M S D D F G E R X D C O R D L M X D N

The rest follows quickly from the pseudo-key 'I _ A _ I _ E'. A check of the dictionary reveals at least six words that will do as equally well as 'I M A G I N E'. This method works quite well and generally reveals a solution within two lengths of the key.

Recovery of the plaintext from the ciphertext alone is another matter, but modern tools provide sufficient entries. Surprisingly enough, statistical methods are sufficiently sensitive to reveal the periodicity in spite of the presence of a large percentage of nulls. The cipher may be solved simply by applying a kappa test on decimations to learn the length of the key-word.[3, pp.16-17] Next, a test of the distribution at each successive position will reveal the location of the nulls and permit them to be stripped off. At this point, the cipher has been reduced to a Vigenère and the solution is straightforward.

By now the reader may have anticipated the potential for a disasterous flaw to enter the system. If multiple copies of a message are sent to different individuals, possibly by different clerks, but using the same keyword, as might be the case if the cipher were used in the field, the ciphertexts would differ only in the randomly assigned nulls. A comparison of any two such texts would render the solution trivial.

Whatever reason Dodgson had for switching to a "far better" system, several patterns are evident in his exposition that were to characterize his approach both to ciphers and to mathematics in general for most of his life. First of all, he saw an underlying regularity and mathematical structure in the system:

"As there are 4 vowels [in IMAGINE] every 4 letters of the message will use it once, producing 7 letters of cypher." He exploited this cyclic feature to the utmost in producing the Matrix Cipher. The second characteristic was his insistence that the system be "carried entirely in the head." Third, and possibly the most significant was the procedural manner in which he presented the cipher. Much of Dodgson's work is characterized by a prescription of steps to be carried out. This algorithmic approach was not unique to Dodgson, of course, and cipher systems are particularly well described by a set of instructions.

In his approach both to cryptography and to mathematics in general, Charles Dodgson, like that other nineteenth century cryptographer, Charles Babbage, seems to have anticipated developments that were to evolve over the next century.

REFERENCES

1. Abeles, F. and S. H. Lipson. Some Victorian Periodic Polyalphabetic Ciphers. Cryptologia. 14(2): 128-134.
2. Candela, R. 1938. The Military Cipher of Commandant Bazeries. New York: Cardanus Press.
3. Deavours, C. 1987. Cryptanalytic Programs for the IBM PC. Laguna Hills CA: Aegean Park Press.
4. Dodgson, C. L. Diaries, 23 February 1858.
5. Kahn, D. 1967. The Codebreakers. New York: Macmillan.
6. Lipson, S. H. and F. Abeles. The Matrix Cipher of C. L. Dodgson. Cryptologia. 14(1): 28-36.

BIOGRAPHICAL SKETCH

Both Francine Abeles and Stanley Lipson hold doctorate degrees in mathematics from Columbia University and have recently earned masters degrees in computer science from Stevens Institute of Technology. Both teach mathematics and computer science at Kean College of New Jersey.

Dr. Abeles is the editor of The Mathematical Pamphlets of Charles L. Dodgson (in progress.)

EXERCISES

1. Recover the key in the following ciphertext having the plaintext:
                   LEWIS CARROLL IS CHARLES DODGSON
      T P F S M X G E W J K Z T I E G A S M T S C Q B L M V G J J U
      N C I P S B L T W L F J Q T N F K O E S O H P U X O O F W T C
2. Recover both the plaintext and the key.
   DHQDR QXECC OPAAV CAERW YJPXO JEFTM HUKYE BMPLH AYHYI WENTD EDBJX
   TCTFR RWOLO XMMTT HEKTE LGCIN PJLSH KPERV TLEBE XBXTD AJDBL QMONV
   EVJVI CVSAE NLAXO NITHI EVXIR GIMAX TCDVT XCERE BMWBU ITTMM OTIUD
   JDERW NOMQT QLOOM LJCDS EGPRT OYMEV KMIDW EBVHS HOUFV FIKHI NBCIC
   EDSBN UPTEJ NTUIT QKRIE ETBNS RQSUI RPGWP IMRID SYKJI QXNGN LQCOY
   LBENT OVIHU FXGHL SFWKT BHATY IJKLS METIN CIEDA TNLMM EBJIT HWHOT
   DIDNS FVARE EUTKS PYUFX FJTCI UVCID EXYIN KCTLH YRYOS KOENV SJJQI
   JHTIQ VGYLE NSTON RADBE UHVEA AJIOL WFTHQ EEVDP OWERU IBBAO LLDIO
   CDHBI ODTYM IEBFN YLSPM IHCIT HKOOK FINUT YNHEP PRVFR KCESG EDKYN
   GLCEQ OADTF AFALC ALLSR TUGEG PUKGE PHRCF ERLCN HETAR GUCUE CKOFX
   NQEGU OGLLW SH


The Cylinder-Cipher

David W. Gaddy

ADDRESS: The Riverside, Box 46, Tappahannock VA 22560-9765 USA.

ABSTRACT: Questions Thomas Jefferson's attributed invention of the "cylinder cipher," notes similar devices before and after his time, and introduces a "mystery device" of this nature in the National Cryptologic Museum, Fort George G. Meade MD.

KEYWORDS: CYLINDER CIPHER, Thomas Jefferson, Étienne Bazeries, Parker Hitt, Joseph Mauborgne, wheel cipher, Charles Babbage, Baron Fridric Gripenstierna, Francis Bacon, "Star" cipher, William F. Friedman, Riverbank Laboratories, M-94, CSP 488, CSP 493, French language cipher

Friedman and Riverbank Laboratories called it "multiplex," or "the Star cipher;" Bazeries, the cylinder - orcylindrical-cipher; Thomas Jefferson used the term "wheel cipher." All three described the same type of cipher device: on a shaft a given number of disks (each individually numbered) are arranged in a prescribed order and placed side by side. Each disk or wheel (or, to use Bazeries' term, rondelle) bears on its outer edge scrambled alphabetical letters. Aligning any one row to the desired plain text, another row is copied as cipher text. The recipient arranges the line of cipher text on a row of his device (with disks set in the same order as his correspondent) and turns the whole to find and read off the plain text. French army Major Étienne Bazeries described it in his 1901 book, Les Chiffres Secrets Dévoilés, published after a decade of frustration because French authorities declined to adopt the system, which was believed to have originated with the author (Figure 1). Two decades later a similar device, developed in the United States by Captain Parker Hitt of the U.S. Army, familiar with Bazeries' device, and enhanced by Major Joseph O. Mauborgne, was introduced by the Signal Corps in 1921 as its M-94 (CSP 488 in the Navy six years later; CSP 493 in the Coast Guard by 1939) (Figure 2). Virtually on the date of its introduction into military service, a scholar at the Library of Congress chanced upon a description of such a device in the papers of Thomas Jefferson. Marvelous! was the reaction: Jefferson, the intellectual giant, had anticipated the twentieth century inventions. Given his reputation, attribution came easy, and it was flattering to American ego that their exceptional fellow countryman had accomplished yet another feat. This is the accepted, romantic story surrounding the M-94, unwitting copy of Jefferson's "wheel cipher".1

1See, as a basic reference, David Kahn, The Codebreakers (New York: The Macmillan Company, 1967), pp. 181-186. The eminent American cryptologist, William F. Friedman, accepted the Jefferson story, as reflected in Friedman's "Lectures" of the mid- to late 1950s. See National Security Agency, Center for Cryptologic History, [Donald K. Snyder, ed.,] The Friedman Legacy: A Tribute to William and Elizebeth Friedman, 1992, pp. 192-193, at 193: "Jefferson anticipated Bazeries by over a century, and the Hitt-Mauborgne combination by almost a century and a half." (Bazeries had discussed his idea in an earlier (1891) paper.) In his seminal work, U.S. Diplomatic Codes and Ciphers, 1775-1938 (Chicago: Precedent Publishing Inc, 1979), pp. 170-173, 178-179, Ralph E. Weber found no reason to challenge the Jeffersonian origin. The "official" army history of strip cipher devices (SRH-366) credited Jefferson as the originator (p. 20), doubtless reflecting Friedman's view. (The present article intentionally skirts the relationship of strips to disks and the evolution of the M-94.)

* Fig.
* Fig. Figure 1. The Bazeries device:. The plain text along its middle line boasts "I am undecipherable" in French.

* Fig.
* Fig. Figure 2. U.S. Army Cipher Device M-94.

In the early 1980s, the National Security Agency acquired for its museum collection an obviously old, but otherwise unidentified, device of this general type. Its size made it a "desk model," as opposed to a hand-held device (Figure 3). Accented letters showed its application to the French language. Its evident age and source (it was acquired from West Virginia) made speculation of Jeffersonian association tantalizing and reopened curiosity about antecedents of the cylinder-cipher. The device obviously is incomplete: A brass frame, comprised of a base and two uprights, holds a wooden shaft on which are placed 35 wooden disks (probably boxwood) numbered in red ink - from spacing and numbering, evidently five are missing. The disks had been placed arbitrarily, some reading upside down with respect to others (i.e., front and back are not obvious as the device is viewed).2 Remains of a cloth covering - perhaps linen - can still be seen on the shaft, possibly placed there to keep the disks from slipping too freely while being adjusted. Other pieces are missing: evidently a horizontal brass bar held the two uprights against the shaft and perhaps served as an aid to viewing; the disks have 42 holes or perforations near the outer edge and were evidently locked in place by a thin rod passing through an upright and through the aligned holes. On the edge of each disk, 42 characters (letters, including accented vowels; numerals; punctuation) are hand-printed (not stamped or incised) in black ink. (The accumulated grime and friction of handling and age blur or obliterate many of the inscriptions; some have been lost in the separation of chips of wood from the edge.) Each disk is numbered on the face in red ink (numbers 35, 36, and presumed 38, 39, and 40 are missing.) Informal examination of the device by specialists at the Smithsonian in 1991 concluded that it was of European origin, possibly French, made, in all probability, during the second quarter of the 19th Century. (Jefferson died in 1826.)

2An elderly lady recalled playing with the device as a child. Further, missing screws from the base have been crudely replaced with wood screws.

* Fig.
* Fig. Figure 3. Unidentified Cylinder-Cipher in the U.S. National Cryptologic Museum.

But what is it? Nicely executed, it is no toy, but who used it, and for what purpose of communication? Does its companion lie in some museum of French or European cryptographic relics, or, is it described in a manuscript? How did it come to West Virginia? From such questions came, first, a reexamination of the Jefferson "invention" and devices of this design.


A recent Jefferson study by Silvio Bedini of the Smithsonian3 repeats the attribution of the seminal idea to Jefferson and quotes one of two copies of the manuscript in which the device is described. While accepting the idea that Jefferson invented it, Bedini was nevertheless careful and cautious with qualifying remarks which could support the notion that Jefferson was not inventing but adopting an idea seen elsewhere - not an unusual practice on his part.4

3Silvio A. Bedini, Thomas Jefferson, Statesman of Science (New York: Macmillan Publishing Company, 1990).
4Bedini (ibid.) also pictures between pages 462-463 in Figure 29 a Vigenère variant - a conventional, keyed alphabetical tableau, but with the addition of the ampersand at the end of the alphabet, and a provision for enciphering digits - possibly Jefferson's innovations. This President Jefferson provided to Lewis and Clark for their 1804 expedition of discovery in the west. If no earlier version of the Vigenére were known, Jefferson may as well have been credited with inventing that cipher.

Jefferson's device, that of Bazeries, and the M-94 have a remarkable similarity in size and make-up - they can easily be held in one hand and manipulated with the other. The M-94 uses 25 disks bearing the 26 letters of the alphabet, mixed. Bazeries used 20 disks of 25 letters. In the surviving description of Jefferson's version,5 the dating of which is speculative, but seems to have been around the turn of the nineteenth century,6 he does not specify the number of disks, but simple arithmetic converts his "cylinder of wood" 6 or 8 inches long, sliced in disks of about one-sixth of an inch thickness, into 36 or 48 disks (not allowing for kerf waste).7

5Bedini, 504-505, Document 7: "Project of a Cipher."
6Bedini, 236, concluded c1792-1793, but other scholars suggest a later date. Weber, op.cit., 170, suggested "sometime prior to March 22, 1802."
7Mr. Bedini had a replica made after the Jeffersonian description, using 36 disks with the 26 letters Jefferson prescribed stamped into the wood with dies. The Bedini replica is executed in a different, and evidently softer, wood than the boxwood of the "mystery" device in Figure 4. The disks of the latter have a hard, thin outer layer that would resist, if not flake off under, the pressure of a die. Bedini's speculation that Jefferson intended to impress his letters with a die was based upon the discovery that Jefferson had purchased a set of "punches" perhaps for that very purpose . . . or to mark his garden tools. While impressed letters would have avoided the tendency over time to rub off, as has been the case with the "India ink" of the mystery device, whether or not Jefferson used one technique or the other remains open. There is no indication that Bazeries intended other than inscription, raising the question of how early stamped or incised lettering might have been employed for such purposes (Sumerians and Babylonians excluded). See Bedini, 236-243, with speculation concerning the 1792 purchase of an unspecified "set of punches" from a Philadelphia type-founder at 237.

* Fig.
* Fig. Figure 4. M-136 (1933), "Mystery Device," M-94 (1921).

In a 1980 article,8 cryptographic collector-scholar Louis Kruh traced these relationships (as well as the relationship between disks and strips to produce the same effect) and suggested that, in concept, they stemmed from ancient letter locks of similar design, not unlike today's bicycle combination locks that use digits instead of letters. Mr. Kruh also called attention to an overlooked statement by British mathematician-scientist Charles Babbage, published in 1854, that cipher systems of this general type were not new, and that "the best form is, perhaps, rings of box-wood placed side by side on a cylinder, and having the twenty-six letters on the circumference of each," a concise description of the Jefferson-Bazeries "invention."

8Louis Kruh, "The Genesis of the Jefferson/Bazeries Cipher Device." Cryptologia , October 1981 5(4): 193-208. (An earlier version appeared in The Army Communicator, Winter 1980 5(1): 48-54, under the title, "The Evolution of Communications Security Devices.") Cryptanalysis of this type of device has appeared in Greg Mellen and Lloyd Greenwood, "The Cryptology of Multiplex Systems," Cryptologia , January 1977 1(1): 4-16 and April 1977 1(2): 150-165, reprinted in Cipher A. Deavours, et al., eds., Cryptology - Yesterday, Today, and Tomorrow (Norwood, MA: Artech House, 1987.)

The theory of independent invention continues to be an interesting subject, but surely logic compels that that be a last-choice explanation. Jefferson did not say or even imply that he had invented something new, any more than in the case of the Vigenère-variant tableau he supplied to the Lewis-Clark expedition, which could as well (barring evidence to the contrary) be attributed to the fertile brain of Jefferson. It would seem more reasonable to assume that, in both cases, Jefferson was aware, through his wide reading, his acquaintances, or his direct observation, of useful ideas, which he adopted directly or with innovations of his own.


In fact, Jefferson's use of the term, "wheel cipher," may well point us to an earlier source: Francis Bacon's 1605 The Advancement of Learning (in the 1640 English translation by Gilbert Wats) recounts various forms of cipher, among which are the "Wheele-Cyphars," not further elaborated. (Bacon's 1623 De Augmentis Scientarium renders the Latin as "Ciphare Rotae").9

9Quoted in Penn Leary, The. Second. Cryptographic. Shake-speare. The Enlarged Second Edition. Omaha, Nebraska: Westchester House Publishers, 1990, pp. 123 and 125-6.

Over a century and a half later a somewhat comparable device was put forward in 1786 as the invention of a Swedish officer, Baron Fridric Gripenstierna. He offered to King Gustav III: "a wooden construction in the form of a cylinder with a number of alphabetic rings."10

10Josef Garlinski, The Enigma War. New York: Charles Scribner's Sons, 1979. Some additional detail is given in Boris Hagelin, "The Story of the Hagelin-Cryptos." Zug, Switzerland: Crypto-AG, 1981, pp. 5-7. A conjectural illustration of a "desk model" operated by two people is shown by Garlinski following p. 60, as in Hagelin, p. 7.

As an alternative theory, then, suppose that, behind the scenes, unknown to the outside world, certain devices were used by cryptographers serving the courts of Europe - devices known only within that small corps of initiates, learned about through espionage or other means, but rarely documented or falling into the hands of outsiders. It is reasonable, then, to believe that - through Bacon, the manuscript describing the Swedish device, the Jefferson manuscript, the Babbage allusion . . . and now this "mystery device" - we have been afforded those rare glimpses, hinting at, but not clearly giving us, a sense of connectivity behind the scenes: That Jefferson had seen an earlier version of the device he described. That Brazeries, too, was putting forward a device not entirely of his own invention. Hitt and Mauborgne relate to Bazeries. Somewhere in that continuum behind the scenes we place this "mystery" device.

Finally (for the moment), we can add another ingredient to this mystery of behind-the- scenes continuum, another "desk model," as similar to the "mystery device" as the M-94 was to the Bazeries or Jefferson: the U.S. Army Signal Corps M-136 of 1933 (Figure 4). Friedman was experimenting with a method whereby the advantages of the M-94 could be expanded by being able, not only to re-order the disks themselves on the shaft, but to change the letters appearing on the edge of the disk. "Cipher device, type M-136" was the result: It employed 25 disks, each bearing an alphabetical strip of 26 mixed letters, the strips held in place by a spring-and-clip arrangement. The resultant device (evidently only one of which was constructed before the idea was abandoned) bears a striking similarity to the "mystery device" in size and appearance, even to the inclusion of a rod running the length of the shaft through the disks to hold them in place.11 No information at this writing suggests that Friedman was aware of our "mystery device," or was thinking in terms other than modifying the M-94.

11The only known model is in the collection of the National Cryptologic Museum, Center for Cryptologic History, at NSA. SRH-366 (pp. 43-44) implies that only one was made (by National Electric Supply Company, Washington DC, at a cost of $84) and that no instructions were ever written.

These musings have in common three points: doubt that Jefferson was the inventor or originator of the device associated with him thus far; curiosity as to the identity of the "mystery device;" and a renewed conviction that, given the nature of state cryptography, one must be cautious in ascribing "firsts" in that field as isolated and fragmentary examples come to light. The "mystery device" is on display at NSA's National Cryptologic Museum as the oldest such device known to its curator. Perhaps a reader can solve the mystery.

BIOGRAPHICAL SKETCH

After a forty-one year career in cryptology, Mr. Gaddy retired in 1994 as chief historian of the National Security Agency, having established the Center for Cryptologic History and the National Cryptographic Museum, both at Fort Meade, Maryland. He is a student of American Civil War history, with emphasis on the Signal Corps, cryptography, and secret service activities of the Confederate States of America.


A World War II German Army Field Cipher and How We Broke It

Charles David

ADDRESS: 75 N. Strathmore Street, Valley Stream NY 11581 USA.

ABSTRACT: This article offers a first-hand account of the breaking of a German medium-grade cipher during Word War II and personal recollections of the process and history.

KEYWORDS: Vint Hill Farms Station, Weymouth St. London, Signal Security Detachment "D", Doppelkastenschlüssel, cryptanalytic procedures.

In 1942 the U. S. Army Signal Corps was looking for college graduates to train in radar work, which was then new and promising. I enlisted and after six months of pre-radar courses in calculus, physics, chemistry, and radio theory at Rutgers University, I found myself in Camp Crowder, Missouri, the Signal Corps basic training center.

Once there, a classification sergeant informed me that Camp Murphy, the radar facility in Florida, was overcrowded, and that I'd be assigned to some other training. Checking my academic record and Army Classification Test score, he suddenly asked me if I had ever heard of "cryptography".

As he thumbed through his manual, my memory reverted to a cryptanalysis course that my fiancée had taken at Brooklyn College, given by Professor Jack Wolfe of the math department. We sat in a sunny meadow in Prospect Park, and I helped with the frequency counts. My reverie was broken by the information that I was to be sent to Vint Hill Farms Station, near Warrenton, Virginia—close enough to my New York home and my wife-to-be to please me no end.

Vint Hill was an unusual army camp—in a bucolic setting, with evergreen trees surrounding the barracks, and woods all around. It was a hush-hush place, and we were constantly warned to keep it so. One of its two parts was devoted to cryptanalysis studies, and the other was a working radio facility.

The students were men of strong academic backgrounds and achievements. Of course, there was a large supporting cast of administrative cadre who tormented us with the usual army routine—drills, hikes, KP, inspections and the like. However, there was communion among the "crypt" people due to the learning atmosphere, and the intellectual interest engendered by these new and uncommon studies.

Our classes were taught by sergeants. Once in a while an officer would appear from Arlington Hall, but Vint Hill was an enlisted men's camp and school. We religiously followed the texts of William Friedman, and considered him our mentor. Men left after unpredictable time periods to join active units in both main theaters of the war. Most seemed to end up in our own signal centers, and were involved with security matters.

After seven months I was called in for an interview, and asked if I knew any German. I had studied the language for two years in high school, and was able to read and translate a newspaper handed to me by an officer. He said that would do just fine, and I was assigned to a special class being formed.

The new class of about 30 men was taught by a very bright Sergeant Dineen. He had just returned from Arlington Hall where he had been briefed on two German systems—one a lower-grade code, and the other a medium-grade field cipher which the British had started to break. We learned about the cryptographic nature of the systems, and then something of the analysis procedures. I knew then that working on them would be intriguing beyond compare. I found out I was right when I got a chance to work on the cipher as a member of Signal Security Detachment D, an element of General Omar Bradley's 12th Army Group.

In April, 1944, I and other embryonic codebreakers debarked from a large troop ship in the bombed out port of Liverpool, England. Our group was brought to London and billeted near Marble Arch. The next day we were brought to an apartment building in Weymouth Street that had been given to the Signal Corps for its intelligence work. A number of comrades and I were assigned to a breaking section working on the German army medium grade cipher. We had been familiarized with it during our last month at Vint Hill and told that the British had made good progress with the breaking. The system was used by all levels of the German army, from army groups on down, seemingly where the highest-level means (Enigma) was not necessary or available. The encryption method was a clever variation of the Playfair that rendered breaking very difficult. We called it NI, short for Non Indicator, as the early intercepts showed no indicator. To the enemy it was Doppelkastenschlussel (Two-Box Cipher).

At Weymouth Street it became apparent that all the operational work was performed by us enlisted men. This contrasted with the British Army, where our equivalents were officers. Our men and officers were mostly professional people and scholars. On the whole they were older than the average G.I.

Our first weeks were spent on practice problems, and perfecting our skills. We also took a class in military German. Although all of the men had had some past school contact with the language, this taught us likely army vocabulary. Oddly, however, many of the people who succeeded best in codebreaking had limited facility with German. The language experts, both trained and German-born, tended to see more than really was present in cipher text, and neglected the more important principles as frequency and combination which led to gradual, but surer results.

Soon we were thrown in with the earlier arrivals, and we began to work on current intercepts. Before the 1944 invasion, the enemy signal units on the Atlantic Wall practiced their cryptography and radio transmission constantly. Often they sent personal messages, newspaper articles, nursery rhymes, and the like. As they practiced, so could we, and thus our skills were improved.

When the Normandy invasion began, the breaking proceeded apace. We gleaned both tactical and strategic information which was coordinated with other forms of signal intelligence, such as direction finding and traffic analysis. As our armies pressed inland, it was decided to form a mobile unit to accompany the advancing allied armies. This was necessary for two reasons. Firstly, our radio intercept companies had to be near the front in order to read enemy traffic more accurately. Secondly, they could send intercepts to us quickly by messenger.

I was chosen for this unit along with a mixture of the newer and the more experienced men. The unit was called Signal Security Detachment D, afterwards always S.S.D.D., and was commanded by Lieutenant Colonel Charles Allen. We were attached to General Omar Bradley's 12th Army Group, but we always stayed by ourselves in the field.


S.S.D.D. consisted of various sections representing all phases of signal intelligence:

1. Cipher-breaking team. This was my group. It was headed by Sergeant Howard Arnold, a fine cryptanalyst from Providence, Rhode Island who later became president of a large department store in his home city. Besides continuing his own breaking activity, Howard supervised the group in a laid-back and thoughtful manner.
2. Codebreaking section, headed by Sergeant James Wallace of North Dakota, a skilled and assiduous codebreaker. After the war Jim became a newspaper columnist in Brainerd, Minnesota. The enemy used code for lower-grade messages and for smaller units. He and his men were very successful.
3. Traffic analysis was the largest group. It studied message headings, radio frequencies, code signs of sending and receiving units, volumes of traffic, and more, to derive indications of enemy placements and intentions. Led by Sergeant George Bauer, it was consistently productive.
4. Direction-finding team, headed by my tentmate Sergeant Leonard Netzorg, successful Yale Law School graduate. It located and identified enemy units by triangulating bearings. Len later became an eminent attorney in Portland, Oregon, whose career and views were the subject of a recent magazine article.
5. Emending Section. These German language experts removed mistakes from interpreted deciphered messages. Much of what was broken was so garbled that it required people very conversant in the language to make it coherent. Sergeant Hyman Sobel, a Harvard instructor, was in charge. The group's members were amazed at how we cryptanalysts broke traffic that was so full of errors, and we were surprised at their ability to make it understandable.
6. Intelligence Coordination Section. This group collated the information, evaluated it, and passed reports based on it to the 12th Army Group headquarters. It was headed by a very bright Master Sergeant Samuel Lieberman, whose ability was respected by all of us.

There was also a small photographic unit, a signal center, and the usual administrative personnel, motor transport, mess people, etc.

S.S.D.D. landed in Normandy. We started by living in the field, and working under tents. We bivouacked in the countryside near such towns as Coutances, Laval, Chartres, and Meaux, and in the tiny Lorraine village of Mangiennes. Then came three months in the city of Luxembourg, where we were quartered in a school not far from the headquarters of Generals Bradley and Dwight Eisenhower in the Hotel Alfa. After the Battle of the Bulge in December 1944, we went through parts of Belgium, entered Germany at Aachen, and after four months of movement, ended the war in Forchheim, north of Nuremberg.

As noted previously, all the operational work was performed by enlisted men. Many of the officers were sent by the Signal Corps to supervise subsidiary functions, as transport and billeting, and had no knowledge of our work. Occasionally, men from the intercept units drifted in and out of our unit, seemingly to absorb some notion of what was being done. Of course we had our own intelligence officers, and two in particular deserve special note for their hands-on attitude and productive work.

The officer supervising my cipher breaking section was the very erudite Captain Benjamin Schwartz. He was a family man, over 45, who had volunteered to contribute his expertise to the war effort. Schwartz was skilled in Sanskrit, other ancient Indic languages, and several modern tongues and had headed departments for those exotic languages in the New York Public Library and the Library of Congress. He instructed us in cryptanalysis, watched over our daily efforts and assumed a rather paternal role toward his men. He was a person who cared, and that meant a great deal.

Captain Howard Mendel supervised the traffic analysis team. He worked at it incessantly, and was an intelligent and effective participant throughout. He was wounded in Luxembourg when a stray airplane bullet went through his side, but much to the relief of his men, their Howie soon recovered and came back.

Soon after we landed, an unexpected development struck our cipher-breaking section. Many of the approximately 16 cryptanalysts—no matter how adept they had been in training classes—seemed unable to cope with a real life situation in which a solution might not exist. A substantial portion of the enemy traffic could not be solved, and this uncertainty created a psychological barrier that stopped men before they started. As a consequence, many faltered, and produced few results.

Four of our men did most of our breaking. They were George Hussey of Bronxville, New York, and Eric Porter of El Segundo, California who worked marvelously together, Howard Arnold, and this writer, Charles David.

* Fig.
SSD - Cipher or Code Breakers - Emenders - Intelligence control. September 1944 in Mangiennes FRANCE.

Though we were the signal intelligence arm of the 12th Army Group, S. S. D. D. traveled by itself in the field. We were an isolated outfit, known only to the intelligence people. Our security and secrecy were superb, even within the outfit. People in one section knew very little about the doings in other sections. On a few occasions, senior officers from headquarters stopped by to express appreciation for our results, and would mention a specific case or two that were outstanding.

Our daily intelligence production was notable from the start, but reached a crescendo during the three months in Luxembourg. Our deciphered intercepts, traffic analysis, and direction finding indicated unusual enemy activity before the Battle of the Bulge began on 16 December 1944. I judge that people at army group headquarters simply did not evaluate these data properly. It probably seemed impossible to them for the Germans to stage a major armored offensive over unfavorable terrain, especially when the bulk of their forces was tied up on the Russian front. After the Ardennes offensive was stopped a month later, the cipher section broke the radio traffic to the German units retreating from the Bulge. It revealed the detailed instructions to each unit as to exactly where it should be virtually every minute of the day. With the skies now clear, the allied air forces made very effective use of this information.

In January, S.S.D.D. moved again through different parts of Belgium and entered Germany, remaining there until the end of the war. We stayed near Cologne, crossed the Rhine at Remagen, camped at Bad Wildungen, and were in Forchheim, Bavaria on V.E.Day. We then moved to the town of Russelsheim, where everyone wondered what our destination would be in the war with Japan. But that soon ended, and gradually we worked our separate ways home.

Many of us had a strong affection for S. S. D. D. Our fellow soldiers were often quite accomplished and thoughtful. There were professors, attorneys, classical musicians, writers, and businessmen—all on their way to join the hopeful post-war world.

A short personal epilogue: Several years later, I visited Captain Schwartz in Washington, D. C. He had stayed in the army, and had become a colonel in Army Signal Intelligence. We were happy to see each other and had much to discuss. He thanked me for my wartime efforts, and then offered an immediate commission to re-enlist and come to work with him. I'm afraid that my destiny was as a civilian, but it was gratifying to have the praise and appreciation of the boss.


NI DESCRIPTION AND BREAKING

The medium-grade field cipher was called "Doppelkastenschluessel"—double box cipher by the Germans. Because the first messages when intercepted by the British bore no indicator, the Allies called it the Non Indicator or NI system.

The NI was used from army groups to all lower units down to company level and carried both tactical and strategic information. It ranked just below the Enigma, the high grade electro-mechanical machine cipher. The Enigma could not be distributed to the multitude of lower field units, and thus the NI served as a more practical hand system.

During the North African campaign the British overran a German signal center and discovered the nature of the system.

System Rules

The NI cipher combines a transposition, or more precisely, a seriation of the plaintext, and then two Playfair-like encipherments. The apparatus of the system consists of two 5 x 5 random alphabet squares (which omit J) such as those in Figure 1 below.

* Fig.
* Fig. Figure 1. Enciphering Boxes.

A plaintext bigram, say EU, is enciphered by finding the first letter in Box 1, and the second letter in Box 2. In this case E is on row 2 of Box 1 and U is on row 3 of Box 2. The two letters are considered as diagonal corners of a rectangle, and the intermediate cipher bigram is those two letters which complete the rectangle, namely VR.

* Fig.
* Fig. Figure 2. First Encipherment.

The bigram VR is now itself enciphered just as the original plaintext bigram was, and the resultant diagonals form the final cipher letters,— QZ.

* Fig.
* Fig. Figure 3. Second Encipherment.

In this way EU becomes QZ in cipher.

If the two letters of a bigram to be encrypted lie along the same line in both boxes, we take the letters immediately to the left as cipher values. These we called liners. For example, DL = KC. The rectangular cases we called diagonals.

* Fig.
* Fig. Figure 4. Linear Encipherment.

For cryptanalytic purposes it is important to see that four combinations of diagonals (D) and liners (L) exist:

                               D-D    D-L    L-D    L-L
                Plaintext      en     ea     gi     yr
                Intermediate   FL     HK     UW     OZ
                Ciphertext     NC     TT     XP     HM

The seriation of the plaintext consists of writing in blocks of two lines of 21 letters each. For example, using the first line of Heine's poem "Die Lorelei":

Ich weiss nicht was soll es bedeuten
dass ich so traurig bin.

and replacing the common ch combination with q:

                 i q w e i s s n i q t w a s s o l l e s b
                 e d e u t e n d a s s i q s o t r a u r i
                 g b
                 i n

The vertical bigrams (I-E, Q-D, W-E, etc.) are the pairs actually enciphered:

           G G Y Q Q D C M A R N E R Q W M B G Q T B
           i q w e i s s n i q t w a s s o l l e s b
           e d e u t e n d a s s i q s o t r a u r i
           U Z Y Z S Z I Q I Q I Y A R U F S T Z U N
           X B
           g b
           i n
           P O

The cipher text is then read off horizontally in groups of five letters for transmission.

           GGYQQ   DCMAR   NERQW   MBGQT   BUZYZ   SZIQI   WIYAR
           UFSTZ   UNXBP   O

German Communication Protocols

Each German division had its own set of cipher boxes. It was assigned six different boxes for each day. These were paired in different combinations for each day's eight three-hour periods. In effect, there were eight keys per day.

The German encipherers followed certain conventions, though they grew lax as the war progressed. They placed an X before and after all numbers, proper names, place names, sentences, within abbreviations, and at the end of plaintexts having an odd number of letters. As a result, plaintext X had an abnormally high frequency and tended to mask the identification of the usual high frequency German letters.

The word zwei ("two") was changed to zwo, and ch plaintext was changed to q, distorting the frequency of q also. The use of occasional "quatsch" (German for nonsense) sequences was encouraged to distort combinations and frequencies.

All in all, the Germans seemed to feel that the NI system was impervious to timely cryptanalysis. They had readied a different system on general security grounds, but kept the NI right to the end.

Some Cipher Security Considerations

The seriation (vertical reading of the plaintext bigrams) ensured that the enciphered bigrams were not letter combinations of normal German text. The most frequent ciphertext bigrams certainly did not reflect the most frequent German combinations (en, er, ei, ie, in, etc.). What the ciphertext bigrams did reflect was the individual frequencies of disconnected German letters taken two at a time. The most frequent NI cipher bigram had a good possibility of representing plaintext ee, for example, but this was by no means always the case.

Even knowing the mechanics of the system, we found that the reconstruction of the cipher boxes was extremely difficult because of the double encipherment feature. No doubt the German signal experts counted heavily on this.

By using different boxes for each division, and by changing the boxes every three hours of every day, the Germans made the cipher uncomfortably close to a one-time pad system—unbreakable in theory and practice. Nevertheless we broke it, and generally in useful time.


AIDS TO CRYPTANALYSIS: THE INITIAL BREAKS

Each message was preceded by three-letter call signs of the sending and receiving units and the time of day. Our traffic-analysis teams were quite adept at identifying them. This plus the assigned radio frequency used aided in sorting the messages by unit—and therefore by key.

In cryptanalyzing the NI system, we used very large sheets of paper, colored pencils, and good erasers. These sheets and colors cannot be fairly reproduced in a journal. I will describe our analytic tools, show how we used them, and then leave it to the initiative of the reader to complete the analysis to reach the final plaintext.

Usually we needed a satisfactory number of letters in the same key to make inroads into the cipher. We were delighted if many more letters arrived for a three hour period, but often not enough traffic was intercepted to afford us an entry.

To illustrate the solution of the cipher, I have composed a sample problem of 18 lines of ciphertext supposedly intercepted from units of one division during a three-hour period. Also listed are the times of transmission and the call signs of the sending and receiving radio stations. Each pair of lines is given a line-identification letter (Figure 5).

* Fig.
* Fig. Figure 5. Transcribed messages for a single period.

The messages have been manipulated to allow showing a number of techniques in a relatively short space. This sample is much less difficult than the messages we encountered, although the language is typical of the German radio traffic of the time.

Knowing how the system worked, we were able immediately to rewrite the intercepted messages in 2 lines of 21 letters each. Once the messages were in this form, we made a bigram frequency—one of our most useful tools. The count was made in a 25 x 25 matrix on graph paper.

Each line of the problem was given an identifying letter at the left, as can be seen in Figure 6. When a bigram was entered on the frequency count that line-identification letter was entered in the appropriate square instead of a tally mark allowing us to locate the occurrences rapidly (Figure 6).

* Fig.
* Fig. Figure 6. Line location and frequency count of cipher bigrams.

With the frequency count as a guide, we attacked the messages using a technique we called anagramming. This started with an assumption—an educated guess—followed by a good deal of trial and error. The intent was to substitute these plaintext assumptions throughout the problem, check surrounding values, and search for clues to expand our guesses.

We could not immediately start reconstructing the boxes even when our assumptions were corroborated because of the double-encipherment feature. The middle bigram is unknown, and a workable number of plaintext values must be ascertained before a start can be made in forming the boxes.

In the frequency count, we find that KK ciphertext stands out with the highest count by far. We can start by assuming that it represents ee. We replace KK with ee throughout the text, and see where it leads us.

The Germans often sent very short messages, which reflected basic military necessities, and which were therefore constantly repeated. The most common 6-, 8-, 10-, and 12-letter messages were known to us. Curiously, we were almost never given information about the military situation, that might yield more specific probable words for our attacks. I don't know why. Perhaps it was security, perhaps ignorance, perhaps lack of time.

Note Line O in our sample problem:

                                    e
                  Line O -    T  C  K  U
                              N  U  K  O
                                    e

To the experienced eye, this a dead giveaway. The Germans were constantly asking "What is your position?" or wie lage. The phrase might be preceded by a "please" or "request" or an addressee, but time was critical, and the eight-letter message was common.

We thus have assurance that our K K = e e assumption is probably correct, and we have:

                              w  i  e  l
                  Line O -    T  C  K  U
                              N  U  K  O
                              a  g  e  x


Our initial assumption has grown: We now have plaintext pairs w a, w g, and l x.

The Germans used many stereotyped openings. With experience, we cryptanalysts could recognize them. For instance, messages frequently began with an ("to") followed by the person or unit receiving the signal.

We are fortunate in having three of our practice messages begin with KK ciphertext. In German military text, about 95 percent of the time a message starts with e, the opening word will be one of four: erbitte ("request"), eigene ("our, our, own"), ein ("a") and eins ("one"). Let us see how this works in our text.

If we try erbitte in Line I, the final e falls on ciphertext KZ, which has a high frequency. (Frequencies are always our strong ally.)

                          e r b i t t e
            Line I  -     K N D C V M K S A . . .
                          K S U P O K Z K A . . . .
                          e

K Z also occurs in line L in position 6. This is a perfect opening for eigene:

                     e i g e n e           e
      Line L  -      K U G Y X K W S K R M K A . . .
                     K K B W W Z H M M Z U K E . . .
                     e                     e

Although these are only the upper values of bigrams, when they are substituted throughout the text, we gain opportunities for assumptions for additional anagramming. For example, bigram CP in Line I has a rather high frequency, and it is a good assumption that the bottom plaintext letter is e.

We now examine other lines, with our assumed plaintext entered, and see what can be done:

                                   e  e n   n    e
                     Line A - VDLVVKXRKCXVRBXABASKD
                              DFXEGKMNZIWBTANPFYOKE
                                   e

Units in the field constantly sent reports to various headquarters. German operations officer in a unit's staff (G-3 in American nomenclature) had the designator 1a. Messages to higher headquarters often began "an roem eins Anton" ("to roman numeral one Anton" — "Anton" being the German phonetic equivalent of our "Able"). Line A has several values that fit this possibility:

                              anxroemxeinsxanton e
                     Line A - VDLVVKXRKCXVRBXABASKD
                              DFXEGKMNZIWBTANPFYOKE
                                   e             e

Often this opening was followed by "enemy something or other" as "enemy tanks, or planes, or troops". ("Enemy" is "Feind" (noun) or "Feindliche" (adjective) in German.) That seems a likely entry here because of the KK (e e) second from the end on top, and around the bottom where "Feindliqe" connects with it. Now we have six more probable equivalents to substitute throughout.

                              anxroemxeinsxantonfei
                     Line A - VDLVVKXRKCXVRBXABASKD
                              DFXEGKMNZIWBTANPFYOKE
                              ndliqe

Let us examine one more short message to see if anything can be made of it:

                                        t t e   a
                           Line F - M U M V I B V O
                                    P Q K O I W D U

That first word just has to be bitte ("please"). But please what? A common request was for a situation report, and a frequent 16-letter message was therefore bitte lage meldung ("situation report, please"). Checking the frequency chart also shows higher counts for a few of the cipher bigrams that accord well with their plaintext equivalents (e.g., I I = e d has a count of 5 and V D = a n a count of 3). We always looked for frequency indications that tend to confirm what are still only assumptions. Thus we have eight more probable bigrams:

                                    b i t t e l a g
                           Line F - M U M V I B V O
                                    P Q K O I W D U
                                    e m e l d u n g

It was a good break if we found two messages with the same beginning. It was especially helpful if the beginning was longer than 21 letters and thus continues on the second line. Such a case may be seen in Lines A and P. We have already anagrammed some of Line A and it seems as though it may share the first 27 letters with Line P. Note the six-bigram repeated block at the beginning of Lines A and P and the repetition of vertical bigrams at positions 1 and 14 of Line P.

                              anxroemxeinsxantonfei
                     Line A - VDLVVKXRKCXVRBXABASKD
                              DFXEGKMNZIWBTANPFYOKE
                              ndliqe
                              anxroemxeinsxantonfei
                     Line P - VDLVVKBUOUSHWVDYEBQOG
                              DFXEGKZWZKAZXDFPDOAZQ
                              ndliqe       nd

Thus it seems we have another 15 upper letters to substitute in the problem.

One of the greatest helps to the cryptanalyst is to know about what the enemy is likely to talk. There were plenty of the German equivalents of "enemy" , "aircraft", "troops", "armor", and so on. But by far the most common words were the cardinal numbers: eins, zwo, drei, etc. Supply reports, casualty figures, radio frequencies, times of day, location grids and much more all required numbers.

Often when no other openings were visible, numbers could be spotted in the middle of messages—after some judicious assumptions based on frequency and combinations. The 10 digits were frequently used in groups (e.g., 4 digits for the time of day plus uhre, "hours"), and were separated one from another by X.

Security violations and operator mistakes greatly helped us. Of course, we were always looking for that rare happening—the same message sent in plain and ciphertext—but that was rare.

Let us examine Line G of the problem:

                              e      e  ie
                     Line G - KLOAZWSEZUCKSPKOTXCYU
                              KSLFQMFDXDPZQXNMOVKDS
                              e         e

We already have some letters of Line G, one of them being an initial e. Now look in Figure 5 at the header of the message:

                                       ZBT
                                       QLS
                                       1240


The alert cryptanalyst now rubs his hands in glee, because he realizes that the careless German code clerk has included the time of day at the beginning of the message. (This occurred once in a long while.) We now have the following:

                              einsxzwoxvierxnulluhr
                     Line G - KLOAZWSEZUCKSPKOTXCYU
                              KSLFQMFDXDPZQXNMOVKDS
                              e         e

There is a slight problem here because the code clerk accidentally omitted the X after null, but that is seen when uhre fits into the bottom e of cipher K K.

The above examples should give a good idea of how the initial plaintext wedges were made. Again, I caution that the sample problem is contrived— it was never this easy. Very few of these entry points, if any, appeared within the same three-hour period. It took a very patient cryptanalyst to find correct plaintext in the average set of messages.

This involved daring assumptions, and detailed tracking of surrounding bigrams to find promising combinations. Trial and error, persistence, and some sixth sense could lead to progress. We stuck with a three-hour period as long as it seemed to show promise—a large volume, a good frequency count, likely plaintexts. Conversely, we abandoned a period for a new one—usually on the basis of instinct and experience—when we were getting nowhere.

Once a sufficient number of bigrams was recovered—one cannot quantify it exactly—we set about trying to reconstruct the cipher squares, a process we called "boxing". This could be the most daunting task of all. As the squares were completely random, we got no help from any pattern or keyword.

The first step was to make "encipher" and "decipher" charts. These were large sheets with 25 x 25 matrices, each cell with four quadrants (Figure 7). The charts were over a foot square, and were preprinted for us in horizontal sets of two which made it easier for us to fill in and work from both at the same time.

* Fig.
* Fig. Figure 7. Portion of ENCIPHER CHART

In the encipher chart, the bigram coordinates of the matrix represented the plaintext, with the cipher equivalents written in the appropriate square. The first letter of the cipher bigram was written in the upper right quadrant of the square, and the second letter in the lower left quadrant, thus serving as a reminder of which letter belonged in which box.

The decipher chart was the reverse. The coordinates were the cipher bigrams, with the plaintext bigrams written in the upper right and lower left quadrants of the inner squares. The upper left and lower right quadrants were reserved for the unknown intermediate bigrams of the cipher which might be discovered as we progressed with pencil and paper. These charts now served as the software for the computer in the cryptanalyst' mind.

With graph paper and pencils, the problem then was to somehow string these letters together so as to recover the original cipher boxes. For a start we used numbers to represent the middle letters, and then by using other related values tried to "hook" some of the letters together. Except in the unlikely event of a very great amount of anagrammed plaintext, this became an eye-twisting exercise in frustration for several reasons.

First, we had no idea of the middle bigram. Second, as I explained earlier, the final cipher bigram can be the result of four possible enciphering combinations: diagonal-diagonal, liner-diagonal, liner-diagonal, and liner-liner. Third, we had no idea of which letters in each box were on the same line, or in the same column, or opposite which line in the other box.

Fortunately, the inherent properties of the boxes and the enciphering method offered ways to help reconstruct the boxes. I will use the cipher boxes of Figure 1 to illustrate the three most useful phenomena.

1. Reversibles — If the middle bigram of the encipherment is a repeated letter, the result of the second must be the reverse of the original plaintext bigram:
ed = HH = DE

This can only result from a double diagonal, which is a clue we really need in our boxing effort.
2. Reciprocal — Assume a plaintext bigram enciphered via two diagonals. If the reverse of its cipher bigram happens to be a plaintext bigram, its own cipher equivalent is the reverse of the original plaintext bigram:
                      (1)      ne  =  AW  =  XM
                      (2)      mx  =  WA  =  EN

Thus if we find two of our anagrammed bigrams in this forward-reverse relationship we know that they can only result from a diagonal-diagonal encipherment. We encircle these in red on the encipherment and decipherment charts. Eliminating the possibility of liners is invaluable in rebuilding the boxes.
3. Appendixing — "Appendixing" aided the analyst in determining which letters were on the same line in a box. If any letters are on the same line (e.g., K, X, Z, N, or Y in Box 1 of Figure 1), and are combined with one particular letter in Box 2—say the letter E in the fourth row—then the first letter of the middle bigram must be the same for all the bigrams:
                             ke = AR = XX
                             xe = AA = EX
                             ne = AW = XM
                             ze = AG = UX
                             ye = AU = XR

When the middle bigram is re-enciphered, since the first letter (in this case A) is always the same, the final bigram can consist only of one of five values in each box, instead of the usual twenty-five: (in this case U, I, E, M, or X in Box 2, and X, M, Q, A, or S of Box 1).

So if we find a pattern of repeats going down a column of our charts, there is a greater than normal chance that the corresponding outside letters on the left are on the same line in their box. This can be fortified if another column shows a similar pattern. None of this is certain, but it bolsters the odds.

From this point, the cryptanalyst resorts to trial-and-error boxing. If the reader has experience with a particularly difficult Playfair cipher, where it appears that some letters must be all at once on the same row, in the same column, and on the diagonal with others, he will have some appreciation of what we faced.


Common Boxes

As mentioned earlier, each German unit was supplied with a group of six boxes for the day. Since two boxes were necessary for each of eight 3-hour periods, some of the boxes were used more than once. For instance, Boxes 1 through 6 might be used as follows:

                          Time of Day  Box Combination
                           0001-0300        1&2
                           0301-0600        3&4
                           0601-0900        4&1
                           0901-1200        5&6
                           1201-1500        3&5
                           1501-1800        2&6
                           1801-2100        5&2
                           2101-2400        6&3

The difficult and crucial breaking was of the first workable period that appeared during the day. This was not necessarily the 0001-0300 slot, but the first period with enough text to attack. We called that the original period. Which it would be was not predictable: no period was regularly the busiest. After that a search was made for other periods that used one of the two boxes now known. If one was discovered, its period was a rather easy one to solve.

Locating a common box in the double encipherment was not simple. A period had to be anagrammed and boxing started before there was a basis for comparison. However, once a common box was spotted, an experienced analyst could reconstruct the second one quickly.

I would like to discuss the major reasons for the NI cipher-breaking being so difficult and daunting.

1. Volume: The enemy changed its boxes every three hours of every day. The quantity of cipher text available in a three-hour period was often insufficient to work with, or just marginally sufficient and might not bear fruit.
2. Peculiarities: Even with adequate volume, a particular text might have a make-up that frustrated continuity, even after sensible beginnings. Worst of all was an aberrant frequency count, which would result in interminable false starts. Though this is common with limited message volume, many longer texts displayed abnormal frequency indications. Despite this, we had the feeling that given time, we could break into most problems, which brings up the next obstacle.
3. Time strictures: Messages enciphered in NI dealt with both tactical and strategic matters. However, after a day or two, the intelligence became stale, and we had to go on to new and fresh material. Working under this kind of pressure proved impossible for some of the men.
4. Garbles: This was the greatest obstacle. The German cryptographers made mistakes, their radio men transmitted wrongly, and our interceptors made errors in picking up what was often a weak signal. At times we had the same messages from three or four intercept companies and the variation could be most marked. If breaking assumptions are based on wrong cipher values, the analyst is dead in the water from the start. Sometimes we displayed an almost sixth sense about incorrect letters, but garbles greatly hindered us.

A Simplification—Single Encipherment

With the invasion of Normandy in June 1944, much of the enemy traffic began to be enciphered only singly, the second, double, encipherment being omitted. The German communication experts perhaps realized that the double encipherment took too long and was too prone to error at both ends to use in combat situations.

To illustrate its use, here are the opening words of "Die Lorelei" singly enciphered in the boxes of Figure 1:

  Y  Y  I  V  Q  D  L  A  Y  N  L  U  X  L  P  W  Q  Y  V  T  F
  i  q  w  e  i  s  s  n  i  q  t  w  a  s  s  o  l  l  e  s  b
  e  d  e  u  t  e  n  d  a  s  s  i  q  s  o  t  r  a  u  r  i
  U  S  A  R  T  A  Q  C  Y  X  Y  A  Q  X  X  C  K  K  R  X  G
      Ct:YYIVQ DLAYN LULXP WQYVT FUSAR TAQCY XYAQX XCKKR XG

It will be seen that our task was much simplified. The diabolical middle bigram no longer exists, and the analyst can use a reciprocal process of simultaneous anagramming and boxing. Some salient helping points are as follows:

1. When enciphered, a plaintext letter must result in a different cipher letter in the same box. Thus, for example, bigram ab cannot possibly show a cipher B on top, or a cipher A on bottom. This helps prevent wrong anagramming.
2. Single plaintext values allow a start to be made in boxing. If we know that plaintext u- is cipher XY, then we also know that X in Box 2 is on the same line as e in Box 1, and that Y in Box 1 is in the same column as e in Box 1, or in the same line adjacent to e. Single value are far less useful in double encipherment.
3. Plaintext assumptions lead more quickly to boxing contradictions than they do in double encipherment, letting us discover anagramming errors much earlier.

Unfortunately, while these aids are valid, they were rather moot in practice. Often the reason was the lack of sufficient traffic in any three- hour period. Add to this the numerous garbles, and breaking the cipher was still difficult—especially in a usable time frame. Nevertheless a large volume of the single encipherment was broken in time to be of use. Due to security necessities and our isolation, we were largely in the dark as to the use of our information, but were assured by officers from General Bradley's headquarters that it was of constant and great value.

To sum up, I found that breaking the NI cipher system was a complex and often mind-bending process. It would be interesting to learn if it were the last of the "pencil-and-paper" systems used by a major army in a major conflict. It was tempting to think of it as "interesting" or "intriguing", but in wartime that seemed incongruous. Perhaps now, many years later, we can study it with a greater degree of academic equanimity.

BIOGRAPHICAL SKETCH

Charles David was was born in Brooklyn NY in 1922 to a well-known songwriter father and a school teacher mother. He graduated from Erasmus Hall High School in June 1937 and from Brooklyn College in 1941. Interrupting graduate studies at Columbia University he enlisted in the Army Signal Corps in 1942 and was discharged in January 1946. He immediately married his fiancé, Helen Robertson, a high school biology teacher, and entered the business world as a sales representative. Mr. David and his wife are both retired now and enjoy extensive and exotic travels.


An Introduction to an Historic Computer Document: The 1946 Pendergass Report—Cryptanalysis and the Digital Computer

Colin Burke

ADDRESS: 6319 Red Haven Road, Columbia MD 21045 USA.

ABSTRACT: In 1946 James T. Pendergrass of the American Navy's code-breaking agency, OP20G, attended the University of Pennsylvania's seminar on electronic computers. That experience convinced him that the navy should turn from the many special-purpose cryptanalytic machines it was building and develop a general-purpose electronic computer. His argument was persuasive and the navy went on to sponsor the Atlas computer. Built by Engineering Research Associates and delivered to the navy in 1950, Atlas became one of the first modern computers in operation. Pendergrass' 1946 report was classified top-secret for decades because it showed how the most significant problems, such as the Ultra-Enigma, could be conquered by a general-purpose computer.

KEYWORDS: Computer, cryptanalysis, OP20G, Pendergrass, Wenger, Atlas, Able, Abner, Monogram, Moore School, Enigma, Hagelin, Vannevar Bush, von Neumann, ENIAC, ERA.

The code-breakers of the American Navy's OP20G had struggled since the early 1930s to obtain the funds needed to design and build high-speed electronic analytic machines.[11, 2] Unfortunately, their mid-1930s project, which was led by Vannevar Bush of MIT, did not produce a truly successful device. The developmental funds and the institutional supports provided by the navy were insufficient for the challenging move into digital electronics. The relatively special-purpose, optical- electronic Comparator that Bush constructed for the navy in 1938 was unreliable and OP20G began World War II with only a handful of IBM electromechanical tabulators and the hope that massive funding of electronics research would finally allow the creation of the more powerful Rapid Analytic machines Bush had suggested to the navy's S. C. Hooper and Joseph Wenger in 1936.[21] They hoped that such machines, distant cousins of the programmed digital electronic computer, would allow OP20G to create and apply advanced mathematical and statistical cryptanalytic techniques.

A stable, long-term program to develop the Rapid Machines might have led OP20G to be among the very first to conceive of and build a modern electronic computer, but the chance for such a program had been lost in the 1930s. It could not be reborn during the war. The demands created by Pearl Harbor and the Battle of the Atlantic turned OP20G away from speculative projects. Although the outbreak of World War II and a new faith in cryptanalysis led to the allocation of millions of dollars for machine development and the creation of a special group for machine design and construction, there were too many pressures to allow OP20G to do more than build a series of very limited and special-purpose electromechanical and electronic devices during the war. Most did not even attain the flexibility of Bush's original mid-1930s design and few provided significant help to advanced statistical and mathematical cryptanalysis.

The need to immediately read the ever-changing codes and ciphers of the Axis powers caused the OP20G research group under Joseph Wenger and Howard Engstrom to take the cryptanalytical and technological paths of least resistance. In the early 1940s, least resistance usually meant brute-force cryptanalytic methods and the creation of machines that were either physical representations of an enemy's encryption devices or fixed physical embodiments of decryption processes. The men in OP20G's research team knew that such a strategy could lead to expensive machines that suddenly became useless when enemy cryptographers made slight changes in their cipher systems. They realized, however, that the creation of more general-purpose machines based on then blue-sky cryptanalytic and electronic techniques was impractical if not irresponsible. When the only sound cryptanalytic alternative was to attack systems through statistical methods, more general-purpose computing devices were constructed. But, their architecture had to be sparse and their components based upon as much off-the-shelf technology as possible.[9]

The crises of World War II were too grave and too frequent to allow creative urges to have priority over practical solutions. Reliable electromechanics and analogue electronics thus triumphed over digital technology at OP20G. Wile easy to construct and relatively fast special-purpose devices were chosen over more general-purpose designs, the young engineers that OP20G plucked from the leading electronic computer projects in American industry and academia lobbied for the exploration of technological frontiers and for more general-purpose machines. But they never recommended the option of constructing one machine to perform all cryptanalytic functions. A "universal" cryptanalytic machine, let alone a high-speed device to perform all logical and mathematical routines, was something at the farthest edge of imagination in the early 1940s. Even Alan Turing, Bletchley Park's mathematical genius, left his philosophical defense of the possibility of a universal mathematical device as a set of formula in an abstract article. [1,13,27]


Immediate problems, not long-term goals, shaped the accomplishments of the Rapid Machine group during the war. OP20G's teams at the National Cash Register Company, Eastman Kodak and OP20G's Nebraska Avenue Annex in Washington DC, created a host of amazing machines that frequently used components that were far beyond the state-of-the-art in electronics and electromechanics.[12] But all were compromises between engineering visions and practical needs. The machines created between 1942 and 1945 were limited in technology as well as in their range of application. Although most used some electronics, many were based on analogue rather than digital logic and components. A few were unashamed reversions to much older electromechanical technology. Such choices were rational in the context of the war. Analogue methods and electrical and electromechanical parts were reliable and inexpensive, ran faster than electronics in some applications and demanded much less development time than the still experimental digital electronics. For example, in the attack against the German's Enigma systems, it proved quite efficient to use a special-purpose design and to base the American Bombe on electromechanics. The early 1942 dream of a fully electronic and perhaps relatively general-purpose digital Bombe, or even one centered on electronic versions of the wheels of the Enigma, devolved into a variation on England's combination of spinning shafts and electric commutators.[14, 15] Electronic digital counters played a very secondary role as the hard-rubber and brass wheels of the Bombe proved to be a combination of speed and reliability that surpassed the power of the tubes and circuits of the early 1940s.[16, 8, 26]

Even when OP20G's engineering wizards were told to create devices to speed the application of general cryptanalytic methods, such as the Index of Coincidence, or to go far beyond the power of tabulating machines to perform searches for repetitions and correlates of code-patterns, they had to follow that least resistant path. For example, digraph analysis machines were hurriedly constructed using the very primitive technologies of mechanical teletype readers and electromechanical industrial counters.[9, 10: p. 23] And, while a new version of Bush's 1930s Comparator was constructed, it barely achieved the power of its mid-1930s design. It merely tallied 'coincidences' of varying lengths on electronic counters as off-set messages punched on special paper-tapes flew by an optical reading- station.[9]

Other war-time applications of Bush's original ideas were even more limited. The hand operated, table-top IC machine was based on analogue electronics. Its operators had to slide two photographic plates over each other until optical-cells sensed enough light passing through the off-set messages to indicate a desired level of coincidence.[9] The later and more sophisticated optical-electronic Copperheads, some of which were built for code-pattern look-ups, were also essentially analogue devices. They did not even count or tally, they just stopped when the messages on the two high-speed tapes had enough coincident code spots. The operator of the machines had to rewind the films by hand to locate the position of the code overlap.[9]

Other machines built by OP20G's Naval Computing Machine Laboratory contained more advanced electronics and logic. But even the complex extensions of the Bombe remained special-purpose as did the electronic digital machines built to model the encryption devices of the Japanese.[7, 5] OP20G was not alone in achieving less than its young engineers might have desired. The American Army's cryptologic machine builders were also limited by the state of digital electronic technology and the need to respond to emergencies. For the same reasons, England's engineers did not build a universal machine.[22, 20]

In 1945, when there were fewer crises, the engineers and mathematicians at OP20G urged the higher-ups to allow them to go beyond what they had accomplished during the critical years of the war. The crew at OP20G knew of the similar and in some ways more advanced work in England, work which led to the famous electronic pre-computer, the Colossus.[22] But they wanted to do more than catch-up with their British counterparts. They had their own dreams of extending the power of digital electronics, optical memory devices and magnetic data recording.

The new mathematical cryptanalysts at OP20G also wanted to continue and extend the World War II automation efforts but they feared that peace would mean the end of the Rapid Machines project. Luckily, the sacrifices and accomplishments of the war were appreciated by the navy's leaders. The ongoing contributions of Ultra and Magic and the role of cryptanalysis at critical times, such as the battle of Midway, had finally won recognition for the code-breakers. Unlike the situation at the end of World War I, when funding was suddenly ended, OP20G was promised several million dollars for post war technological development. Immediately, great plans were laid to produce the type of machines that would have been built during World War II if there had been time to develop digital electronics and to overcome the barrier that frustrated all computer builders of the period, the lack of high-speed memory.[19, 27]

By the end of 1945, OP20G had its Program Monogram computing wish-list ready. No one envisioned a universal machine but there were great hopes that the special devices of World War could be replaced by much more general mathematical engines. A new, much faster, multi- purpose digital Comparator was to be designed and built. It was to be one that would be able to perform the functions that had been assigned to separate machines during the war. Along with the search for the new Comparator was a quest for the best memory medium. Microfilm and the unproven magnetic recording were to be explored for use on the machine that later became known as Goldberg. There was even a great deal of thought given to the possibility of constructing a Super-bombe, America's version of the British Colossus.[19]

There was no thought given to the creation of a general- purpose, electronic and program driven computer, however. Monogram was not a computer project, it was an effort to develop what we would now call powerful special-purpose machines. OP20G's team included some of the best electronics engineers, mathematicians and logicians in the nation but even in 1945 they did not think a general-purpose computer was a realistic option for 1940s code- breaking. Cryptanalytic functions, they thought, had to be hard-wired into a machine. Thus, Monogram's original list did not include a 'programmed' computer.


Then, OP20G's technical leader, Howard Engstrom, decided to send one of his men to see what the University of Pennsylvania was offering at its Moore School's 1946 summer course on computers. The Moore School's computer group had just completed the ENIAC, the astounding yet special-purpose digital electronic version of the giant analogue differential analyzers. Unlike the engineers at the cryptanalytic agencies, the Moore School's team had been allowed enough time to experiment because Army Ordnance's computing crises were not of the same order as those of the cryptologic agencies. It could balance future usefulness against meeting production schedules.[24]

Although the huge ENIAC was a special-purpose machine, it was fully electronic and its construction led to something more significant, the design of universal devices that later became known as the EDVAC and IAS computers. By 1945, the Moore School's men had sketched out a machine that would perform any mathematical routine through what we now call 'software'. When the young Philadelphia engineers, led by John Mauchly and Presper Eckert, were joined by the famous and powerful John von Neumann, their idea for a universal engine became more precise and gained recognition in the scientific community.

Although torn by policy and personal conflicts, the original ENIAC group was so famous that engineers and mathematicians from the most advanced computer projects flocked to Philadelphia in the summer of 1946 to gain insights into the latest computer developments. Von Neumann's design for a more sophisticated universal machine, which he planned to build at the Institute for Advanced Study, was revealed as were the ideas of Eckert and Mauchly who were beginning to sketch the outlines of their own new computer, the UNIVAC. They were joined by a host of others who described the advances in digital electronic hardware and logic achieved at other American computer centers since the onset of the war.[4, 24]

Because of the continued security restrictions on OP20G's work, it could not reveal its computing secrets. But, Engstrom accepted an invitation to send a representative to the summer meeting. He was to listen and learn. At the same time, Engstrom and Wenger began to establish a long-term relationship with John von Neumann. The contacts with von Neumann and the Philadelphia group were important to OP20G because although they were aware of the general nature of the work at all the American and British computer centers, OP20G's men had not yet realized the implications and potentials of the design of the proposed EDVAC and IAS computers. They, like most others, did not fully appreciate the critical importance of the idea of a general-purpose programmed computer.

James T. Pendergrass, a young academic physicist who had been drafted into naval cryptanalysis, was selected to represent OP20G at the summer conference and workshops. He was impressed by the size of the group and the credentials of the men who attended the near month long meeting. He was even more impressed with what he heard. He very quickly became an advocate for the construction and use of a general-purpose digital computer for code-breaking, one based upon John von Neumann's scheme.

During his weeks in Philadelphia, Pendergrass convinced himself that most if not all cryptanalysis could be performed through digital methods. He came to believe that general-purpose digital computers would soon be fast enough to compete with the special-purpose and analogue devices being constructed under Monogram. His next step was to persuade the navy that OP20G should have its own version of the proposed IAS universal mathematical and information machine.

With the blessing of the operational head of OP20G, Joseph Wenger, Pendergrass devoted much time to preparing a long and detailed report. His two part document was submitted to OP20G in late 1946. It remained Top Secret for decades. Only recently has one part been declassified and the censors continue to black-out and delete significant portions of that text.1[3]

1In the appended copy of the report the censor's deletions are indicated by the use of '[DELETED]'. Because the copy provided by the government contained some words and partial sentences that were unreadable, the symbol [UNREADABLE] will be found in a few places in the retyped copy. To allow computerized publication, some alterations were made to Pendergrass' report. Certain brackets used in the original text were deleted and superscript plus and minus signs were placed after variable names. The agony of interpreting and retyping the original text was borne by Mary Claire Monroe who coaxed Word Perfect into doing things that seemed impossible.

There is a good reason for the continued caution about the Pendergrass Report for it contains secret information. Pendergrass had to demonstrate to the navy that a programmed computer could perform all the critically important cryptanalytic tests that were embedded in the many special-purpose devices, both analogue and digital, that were in OP20G's operational center in Building Four in Washington, D.C. Those tests included the secrets of Ultra and Magic. His report had to go further, however. He had to show that cryptanalytic procedures of a more advanced nature, which were to be built into the planned special-purpose Project Monogram machines, could be done with a computer.

The nature of his audience shaped the content and style of Pendergrass' report. The men at OP20G did not need a tutorial on basic electronic technology. They were leaders in the field and thought they would be able to convince the navy's Bureau of Ships, the agency that supervised all of OP20G's materials acquisitions, of the viability of electronic machines. What OP20G's cryptanalysts did need from Pendergrass was a demonstration that the yet to be born 'programming', digital methods and the non-existent general-purpose computer were reasonable cryptanalytic options. It was critically important to build a convincing argument that a computer could attack the new automatic enciphering devices that had led to the first Rapid Machine of the 1930s. Pendergrass and his allies had to show that the general-purpose machine could attack the Purple, Enigma and the more worrisome machines like those used in the German Fish system.


Thus, Pendergrass went into great detail about the logic of a general-purpose computer and the programming of cryptanalytic tests. His report outlined the architecture of the new computer, gave a version of John von Neumann's programming language and hinted that a more practical computer language would soon emerge. More importantly, the report showed how the most difficult World War II cryptanalytic problems could be programmed. Because time was always critical to the code-breakers, his reports also contained estimates of how long it would take the programs to solve problems familiar to the crew at 'G'.

Because upgraded versions of Bush's old designs were central to the plans for Monogram, Pendergrass had to prove that his proposed machine could replace the Comparator. Thus, one section of the report was devoted to the description of the steps necessary to compile the statistics needed for the ubiquitous Index of Coincidence analysis.

Then, Pendergrass went on to OP20G's truly secret techniques and to the problems that had been the focus of its attention during the war. Unfortunately, only one of the perhaps three World War II special cryptanalytic procedures Pendergrass explored in the first report have survived the censor's scissors and black pen. He demonstrated the solution to the now famous target of the American 'Bombes', the four wheel Naval Enigma machine. And, in Appendix II, Pendergrass programmed a solution to the Enigma 'Grenade' problem. It was one of the less time-consuming tasks assigned to the Bombes. A Grenade demanded a shorter run than other anti-Enigma procedures but followed the general logic of the Turing-Welchman attack on the Enigma.[26] Their critical method was crib-based and analogue and very fast. So, Pendergrass had to prove that his all-purpose digital machine could at least match the Bombe's efficiency.

Pendergrass wanted to show that his device would be useful to everyone at OP20G. In Appendix III he demonstrated how the digital computer could be used as an ultra high-speed version of the Hagelin ciphering machine. With a very different mechanism and logic than the Enigma, the Hagelin devices were used by several nations, including the United States, during and after World War II.[10] To aid the attack on Hagelin systems used by an enemy, and perhaps to illustrate how to speed the decryption of friendly messages, OP20G had constructed an electrical analog of the Hagelin during the war. Pendergrass showed how his all-purpose electronic digital machine could take its place.[18]

The particular systems discussed in the missing Appendices of the first report are unknown but they perhaps included an example of how to program a computer for an attack against Japanese encryption machines and a demonstration of how to solve the more sophisticated machine ciphers such as those produced by the American ECM. The other part of his report, which remains classified[3], perhaps contained proof of the ability of a universal digital computer to replace the dozens of tabulating machines and special-purpose Rapid Machine devices that were arrayed against code problems. Pendergrass perhaps demonstrated how to strip additives and, although he realized that there was no mass memory technology that could keep pace with the speed of electronics in 1945, showed how the computer could search for code groups.[10]

The Pendergrass report hit its mark, at least with the operational head of OP20G. As a result, Joseph Wenger began to lobby the very reluctant Bureau of Ships to allow yet another 'computer' project. The job of convincing the Bureau was not easy. 'G' already had the more special-purpose Goldberg and other machines with something like proven track records on the way to completion. The Bureau wondered if it wasn't wiser to let others, such as John von Neumann, carry the long and costly burden of the first stages of computer research.

The Bureau's men must also have wondered if the new computer would be cost-effective. The electromechanical Bombes which worked so well against the Germans cost $45,000 each and the other World War II machines were built for similar amounts. The ENIAC, however, had cost hundreds of thousands of dollars and the projections for the EDVAC and IAS computers were even higher.[12]

Even Pendergrass' optimistic estimates of the electronic computer's run times must have raised some concern. The machine he proposed would take seven minutes to run the regular Enigma problem. The inexpensive and reliable conglomeration of spinning wheels, the Bombe, took twenty minutes plus machine set-up time. A perhaps three fold advantage at a probable ten fold cost didn't seem a wise choice. An outside observer, even one who realized the other advantages of a general- purpose machine, might well have thought that OP20G was as or more interested in 'engineering' than cryptanalysis. [17]

Wenger won his battle but it took some time to secure the go-ahead from the Bureau of Ships and to piece together the funds to study and then build OP20G's general- purpose Atlas computer. Its design went through several alterations as the engineering team learned more about computer logic. The army's cryptanalysts were also inspired by the Pendergrass report and soon began their quest which led to the Abner computer. [22, 23]

Neither service found it easy to fund or build a machine, however. Using the private company established by the men who had been the core of OP20G's World War II computer program, Engineering Research Associates, the navy had a working computer at the end of 1950, some three and one-half years after letting the initial contracts. The army had an even more difficult time arranging for the construction of its device which was finally delivered in the spring of 1952.[25, 23]

The price of the Atlas was much greater than expected, more than twenty times that of a Bombe. Just the assembled hardware cost some $950,000. That was much, much more than the navy paid for the many fast special- purpose machines ERA and other contractors continued to build throughout the 1940s and 1950s.[22 p. 8]

The special-purpose machines continued to fill many cryptanalytic needs but the great advances in computer technology in the 1950s vindicated Pendergrass and the others who had been willing to take a very great leap of faith in 1946. Although it was more than a decade before the general-purpose computer, even when augmented with special-purpose add-ons, could match the speed of dedicated devices for all cryptanalytic problems, the code-breakers held to their commitment and continued to be major sponsors of the development of universal computers and their underlying technologies. [22]


REFERENCES

1. Bashe, Charles J. et al. 1986. IBM's Early Computers. Cambridge MA: MIT Press, 59.
2. Bureau of Engineering, November 18, 1931. To CNO, 'Automatic Machines for OP20G.'
3. Campaigne, H. and J. T. Pendergrass, December 12, 1946. Second Report on Cryptanalytic Use of High-Speed Digital Computing Machines. OP20G, Washington DC.
4. Campbell-Kelly, Martin and Michael R. Williams (eds). 1985. The Moore School Lectures: Theory and Techniques for the Design of Electronic Digital Computers. Cambridge MA: MIT Press.
5. Chief of Naval Operations, U. S. Naval Communications, 1945. Technical Paper TS-6, "Rattler".
6. Chief of Naval Operations, U. S. Naval Communications, May, 1945. Communications Intelligence Technical Paper-41, "Copperhead I Punch and Scanner".
7. Chief of Naval Operations, U. S. Naval Communications, March, 1946. "Duenna Operations Manual". Washington, D.C.
8. Chief of Naval Operations, U. S. Naval Communications, May, 1946. Communications Technical Paper TS-43, "The Bombe". Washington DC.
9. Chief of Naval Operations, U. S. Naval Communications, June, 1946. "Machine Comparisons". Washington, D.C.
10. Deavours, Cipher A. and Louis Kruh. 1985. Machine Cryptography and Modern Cryptanalysis. Norwood MA: Artech House.
11. Hooper, S. C., DNC, September 26, 1930. To OP20G, 'Automatic Computing', NARA RG457, SRH355, "Naval Security Group History to World War II," p. 79.
12. Meader, Ralph. USNR, January 21, 1949. To Capt. J. N. Wenger, USN, 'Report of 14 Day Training Duty.'
13. OP20G, November 3, 1941. 'Report of Meeting to Discuss New RAM Comparator.'
14. OP20G, May, 1942. 'Memo to Station X, Future E Policy.'
15. OP20G, August 5, 1942. Wenger to Ely, 'Electronic Wheel.'
16. OP20G, September 15, 1942. To Joseph Desch, 'Report on Proposed Bombe Design.'
17. OP20G, February 21, 1944. W. Wright to OP20G, 'Report on Meeting with ASA.'
18. OP20G, September 7, 1944. OP20G to NCML, 'Electrical Hagelin.'
19. OP20G, December 20, 1945. 'Project Monogram'.
20. Randell, Brian. 1980. The Colossus. In N. Metropolis et al. (eds) A History of Computing in the Twentieth Century. New York: Academic Press. pp. 47-92.
21. Safford, L. R. to S. C Hooper, November 19, 1940. 'Comparator'. NARA RG457, SRH355. p. 404.
22. Snyder, Samuel S. 1964. History of NSA General-Purpose Electronic Digital Computers. Washington: Department of Defense.
23. Snyder, Samuel S. 1980. ABNER: The ASA Computer. Part 1: Design. NSA Technical Journal. 25(2): 49-67.
24. Stern, Nancy. 1981. From ENIAC to UNIVAC: An Appraisal of the Eckert-Mauchly Computers. Bedford MA: Digital Press.
25. Tomash, Erwin. 1980. The Start of an ERA: Engineering Research Associates, Inc., 1946-1955. In, N. Metropolis t al. (eds) A History of Computing in the Twentieth Century. New York: Academic Press. pp. 485-495.
26. Welchman, Gordon. 1982. The Hut Six Story: Breaking the Enigma Codes. New York: McGraw-Hill.
27. Williams, Michael R. 1985. A History of Computing Technology. New York: Prentice-Hall.æ

BIOGRAPHICAL SKETCH

Colin Burke is an historian who became interested in the history of computers after many years researching the course of American politics and education. He resides in Columbia MD and is now completing a history of computer development in the American cryptanalytic community from 1930 to 1950.

Enigma Before ULTRA Polish Work and the French Contribution

Gilbert Bloch1
Translated by C. A. Deavours2

ADDRESS: (1) 7 Rue du Cher, 75029 Paris FRANCE and (2) Department of Mathematics, Kean College of New Jersey, Union NJ 07083 USA.

INTRODUCTION FROM THE TRANSLATOR

The Enigma story seems to be an unending series of revelations and corrections to previous revelations. It is then with genuine happiness that Cryptologia begins, in the current issue, to publish a series of chapters from the French work Enigma Avant Ultra (Enigma Before Ultra). The author of this work, M. Gilbert Bloch, has done a real service to those of us interested in this subject. Mr. Bloch has examined all of the publicly available documents and put together what is the most comprehensive documentation on the subject of Enigma which has yet to appear. He has been able, in many cases, to clarify and correct dates and to establish the sequence of events. Where there are discrepancies and mysteries remaining, this too has been pointed out for the reader's benefit. In particular, the crucial period between May and September 1940 has received the author's attention. It now seems likely that the explanation of how the British got by may yet prove more interesting than the story told by the late Gordon Welchman in his remarkable book, The Hut Six Story.

I am not a professional translator as will, no doubt, be evident. Although Mr. Bloch will examine every bit of material published before its appearance, the final guilt for any mistranslations or missed meanings is with me. In making the translation, I have tried to stay somewhere between the just give me the facts school of translation where the author's personality and style disappears and is replaced by that of the translator, and the don't change a comma school, which, in this case, would mean quarter of a page sentences with accompanying clauses and parenthetical remarks. As the old maxim goes "What is not clear is not French" — but, then again, it's not English either. Au boulot!…

Avis à Nos Lecteurs Francophones

Il y a disponible, en quantité strictemente limitée, des exemplaires de cette oeuvre en version originale. On peut en obtenir une copie (gratuite) du traducteur (ne pas la demander de l'auteur, svp).


PRESENTING "ENIGMA BEFORE ULTRA"

Books and articles on what could be called The Enigma Saga are many; it is even tempting to say, taking into account the irrelevance of part of these publications, too many … The production of a new study and its partial reproduction (not contemplated initially!) in Cryptologia necessitate therefore some explanation, if not an excuse.

The fact that the author of this new study is a Frenchman (and consequently, the original text written in French) does not constitute a recommendation. Rather on the contrary … French literature on Enigma has until recently been coarse and poor. One could therefore expect that the main purpose of a French study would be the filling, in French knowledge, of gaps that do not exist anymore in Anglo-Saxon minds; but, on some important points, new documentation, explanations and interpretations are brought to light.

It is only fair, before listing the areas where the reader can expect to find new developments, to warn on the limitations of the study.

1. Enigma before Ultra (Enigma avant Ultra), as indicated by its title, embraces only the 1930-1940 period.
2. The study is not a technical one.
3. The extraordinary spying process through which the French obtained secret German documents is not examined in detail.

Turning now to the more positive aspects, Enigma before Ultra will, among other things, provide the reader with improved views in the following fields:

1. For the first time, a precise and comprehensive time-table of the events is given, particularly for the 1931-1933 period, where the real course had been until now difficult to retrace. The interrelations between France and Poland are shown and dated: a coherent picture emerges from this process.
2. The knowledge of the exact time-table of events shows clearly how the Polish mathematician M. Rejewski was able to organize his work during the 4 decisive ending months of 1932.
3. As could be expected from a French study, particular attention has been given to the French contribution to the Polish success. One could have feared at that stage a burst of French chauvinisme. This is not the case, and it is made perfectly clear that the French contribution was limited to obtaining secret German documents and transmitting them to the Poles, who receive full credit for their work. But, what is shown beyond any doubt is that
a) The documents transmitted by the French conditioned the Polish success (and not only facilitated it, as it has till now been widely believed).
b) Besides the four documents (explicitly identified, and with their dates of transmission duly indicated) that were put at Rejewski's disposal, the Poles received, from 1931 till the end of 1938, a lot of other Enigma documents. These were not transmitted to the Polish mathematicians by their superiors, but were used for monitoring the quality of their work, and perhaps warned the Poles of procedural changes.
4. Hypotheses liable to explain how the British re-entered the 22nd May 1940 Enigma Luftwaffe net (after having lost all their prior means of access with the dropping, the 1st May, of the duplication of the message key by the Germans) are examined in detail.

In such a field as the Enigma story, nothing will every be considered as final; but, on many aspects, the new study hopes to clarify things, and to allow future research to make further advances.

Cryptologia will not furnish a complete translation of Enigma avant Ultra. The introduction and the two first chapters (devoted respectively to the description of the Enigma machine, to a review of its procedures of use — and the evolution through time of these procedures) are omitted, as Cryptologia readers should be familiar with such aspects. Also omitted will be an Annex 1, devoted to the problem of Marine Enigma: more specialized studies on this subject already exist, and better ones are in preparation.

The introduction of Enigma avant Ultra includes acknowledgments that it would be unfair to omit. The author owes large debts of gratitude to Sir F. H. Hinsley (Cambridge), Colonel T. Lisicki (London), J. Garlinski (London), P. Calvocoressi (Bath), R. Erskine (Belfast), and on the other side of the hill, to Pr. J. Rohwer (Stuttgart). Enigma instructions have been obligingly furnished by the Militärgeschichtliches Forschungsamt (Freiburg in Breisgau) and other German archives.

The study would never have been written without the constant encouragement of Colonel P. Paillole.

NOTES

1. It must nevertheless be remembered that the book that started the unveiling of the Enigma mystery was published in France in 1973. Enigma, ou la plus grande énigme de la guerre 1939-45 by General Gustave Bertrand (Plon ed.) was not the first publication on Enigma: a Polish book on the subject Bitwa o tajemnice (Battle for Secrets) was issued as early as 1967, under the signature of Colonel W. Kozaczuk, but went practically unnoticed…Without Bertrand's book, it seems doubtful that Group Captain F. W. Winterbotham would have been granted the authorisation to publish in 1974 The Utra Secret. Bertrand's book, despite the crucial role played by the author in the first phase of the Enigma story, is unfortunately, both from the historical and technical points of view, not very satisfactory: it is furthermore marred by errors on dates (some of these errors are simply misprints!) that have prevented many historians (Garlinski and Stengers are notable exceptions!) to draw a correct time-table of the events. Winterbotham simply did not take notice of Bertrand's book, and what he wrote in his second chapter on the Birth of Utra for the period 1930-39 was nonsensical.
2. These aspects of the story are fully covered in the recent book of Colonel Paillole, Notre espion chez Hitler, (Laffont ed. 1986.) An English translation is due to appear shortly.


CHAPTER III
POLISH WORK AND THE FRENCH CONTRIBUTION

A. THE POLES AND THE INTRODUCTION OF THE ENIGMA

The reconstituted Polish State of 1919, sandwiched between two equally hostile neighbors, thought cryptology could bring some factor of security to its precarious existence. The Polish cipher bureau, always very active, included excellent specialists. Initially, the decryption of the radio messages of the Reichwehr and Kriegsmarine was accomplished without great difficulty.

This favorable state of affairs was not to last. The naval messages became unreadable beginning in February 1926. The same thing occurred with the army traffic in July 1928.

The German ciphered messages, which were still being intercepted by listening stations, resisted all the established methods of cryptanalysis.

The Poles rapidly arrived at the conclusion that the German armed forces were using a new method of encipherment by machine. This presented the Poles with a triple problem, or rather, three problems whose solutions depended upon one another.

1. To determine the type of machine being used
2. To obtain or reconstruct the machine
3. To develop methods of decipherment.

The study of these problems fell, logically, in the Services, i.e., the Second Department of the General Staff of the Army at Warsaw.

This Army Department, as well as its directors, had undergone changes during the period under review. For our purposes, it is sufficient to note that the division charged with interception, radio intelligence, and cryptology directed until January 15, 1930 by Major Pokorny was led thereafter by Major (later Lieutenant Colonel) Gwido Langer. In the middle of 1931, this division became the Biuro Szyfrow (Cipher Bureau) which was in turn divided into sections covering each geographic zone. The most important of the sections was BS4 which was responsible for Germany. This section was placed under the direction of Captain (later Major) Maksymilian Ciezki.

From 1928, the work on the Enigma was confined to a team of three officers: Ciezki, Michalowski, and Czajsner.

B. DETERMINATION OF THE MACHINE TYPE

The attention of the Poles seems to have been drawn to the commercial Enigma machine and its possible use by the German Army during 1927-28. A curious story about the interception by customs officials at the railroad station in Warsaw of a package containing an Enigma and its subsequent examination over the weekend by Polish technicians is found, in several versions and with conflicting dates, in the works of Lewin, Garlinski, and Rejewski[1]. At the indicated dates, the machine examined could only have been a commercial model of the Enigma. Our interest in this incident is only to indicate at what time the Poles became interested in the Enigma. The result was a purchase (perfectly legal but done under a cover address) of an Enigma machine by the Poles.

It was logical for the Poles to believe that there was some connection between the commercial Enigma which they had just acquired and the new model of the machine used by the German Army. They began by checking, with the aid of their commercial model, whether or not the undecipherable messages could have been generated by a similar machine. The answer to this question was in the affirmative.

The Poles were not devoid of sources of information. Their intelligence service soon began investigations in Germany, and the Poles rapidly acquired confirmation that the machine introduced into the German armed forces was a modified Enigma (in fact, significantly modified with the introduction of the Type I model on June 1, 1930).

C. RECONSTRUCTION OF THE MACHINE

1. Differences Between the Commercial Enigma and the Military Model

The commercial Enigma was a machine having 3 movable rotors and a reversing drum but it did not have a plugboard (steckerverbindung).

The military Type I Enigma model of 1930 differed from the commercial model in 4 ways:

a. The three movable rotors had completely different wirings from those of the commercial machine;
b. The reversing drum (Umkehrwalze) was of a new type and not settable;
c. A plugboard with changeable connecting plugs (the steckers) was added to the machine;
d. The wiring connecting the keyboard to the entry/exit drum (via the plugboard) was different from the corresponding wiring on the commercial machine which connected the keyboard directly with the entry text drum.


2. 1928-1931. The Initial Deceptive Attempt.

The first task of the Polish team was to identify with precision the differences between the commercial Enigma and the military version. Up to the end of 1931, only slight progress was made in this direction.

It is certain that the Poles, who had at their disposal at least one copy of the commercial Enigma, received from their intelligence service some information about the military Enigma[2]. It was thus that the Stöpstellstellung, the early version of the steckers was identified and described. But, the information remained imprecise and fragmentary. At the end of 1931, the Poles were still very far from having a clear and precise notion of the general structure of the machine and, therefore, could not envision that they might ever be able to reconstruct the internal wirings. In fact, research on the machine was abandoned.

One all important point had, however, been noted. Systematic studies of the encrypted German messages had shown that the first 6 letters of these texts had some particular significance. Very probably, these first 6 letters formed some sort of key.

3. The French Contribution (1931-1932)

In October 1931[3], an employee of the Chiffrierstelle of the Reichswehrministerium (Cipher Bureau of the German Ministry of Defense)[4] proposed to deliver documents to the agents of the French Service de Renseignement, (S.R.F.).

Since October 30, 1930, the S.R.F. included a Service Crypto (called Section D) whose role was to obtain information concerning foreign codes and ciphers. (Section D had no decrypting or analytic functions and was limited to transmitting any information gained to the experts of the Service de Chiffre). Captain (later General) Bertrand, head of the crypto service of the S.R.F. (and at this date about the only person in its service), was put in charge as case officer of individual contacts with new agents.

It was thus that Captain Bertrand received from Asche or H.E. (the pseudonyms of Hans Thilo Schmidt) [5], on the 8th of November 1931, the first set of secret documents of which the most important were:

a. Gebrauchsanleitung für die Chiffriermaschine Enigma
(Instructions for Using the Enigma Cryptograph). Document H.Dv. g 13 of the Army (H=Heer) and L.Dv g 13 of the Luftwaffe.
b. Schlüsselanleitung zur Chiffriermaschine Enigma
(Instructions for Ciphering on the Enigma Cryptograph). Document H.Dv g 14 of the Army and L.Dv g 14 of the Luftwaffe.

These two documents, in fact, the instructions given to machine operators, furnished a detailed description (with photos) and the operating instructions for the military Enigma I (Eins), put into service in 1930. Nevertheless, nothing about the internal wirings of the machine was included [6].

Bertrand transmitted his acquisitions to the S.C.F. which declared immediately that the Enigma was impossible to solve, the documents useless, and removed itself from further discussion of the question. This unfortunate state resulted, perhaps, from the character of Bertrand himself — a character which hardly facilitated relations with the other services.

Bertrand, to his credit, was not convinced by these peremptory affirmations, but he was practically alone and his limited cryptologic competence prevented him from personally attacking the problem. He then had the idea, and was able to obtain from his superiors the necessary authorization, to transmit the documents to the services of France's allies and to suggest to them a common attack on the problem.

London was contacted first: the Government Code and Cypher School — the modest name of the British cryptologic service, better known later under the name of Bletchley Park — received the documents, carefully filed them, and gave no follow up offer of cooperation [7].

From the 7th to the 11th of December 1931 [8] Bertrand himself visited Warsaw and recontacted Major Langer, the very competent head of the Biuro Szyfrow, who understood immediately that the documents furnished would allow resumption of the interrupted Enigma studies. Bertrand and Langer, provided with codenames BOLEK and LUC respectively, promised each other to share all information. This meant that in exchange for these documents (and those to follow), the Poles promised Bertrand to keep him informed of the results which they obtained.

Major Langer, Captain Ciezki, and his team renewed their work with enthusiasm. The documents brought by Bertrand indicated clearly the points where the military Enigma differed from the commercial models: different wirings, the new type of reversing drum, addition of the plugboard and its plugs. The Poles also learned the frequency and procedures for changing the machine's settings as well as the significance of the first six ciphertext letters of each message. In other words, the Poles knew, from this time, that in each message the first and the fourth letters, the second and fifth letters, and the third and sixth letters resulted from the encipherment of the same cleartext letter.

Enthusiasm waned rapidly. All efforts to reconstruct the inner wirings of the machine failed: the magic mountain which had been opened by Bertrand's documents had ended in a blind alley — and this in spite of the receipt of new documents from Asche. The visits of Captain Bertrand to Warsaw on 9-11 May and 17-21 September 1932 [8], and, probably, other transmissions of documents through the Polish Services in Paris, placed in Major Langer's hands monthly schedules giving the daily settings of the Enigma. However, in the absence of a complete reconstruction of the machine (the difficulty of which has been discussed in Chapter II), all these documents were, apparently, useless.


4. Mathematicians to the Rescue

Being in despair, Langer and Ciezki decided to seek the intervention of other persons. For a long time, they had had the idea that the level of mathematical competence necessary for cryptanalysis surpassed that of the Army cryptologic personnel available and would require professional mathematicians. Beginning in 1929, they had organized a course of study in cryptology at the University of Poznan [9]. This course was taken by about 20 students already skilled in higher mathematics. The professors were officers in civil guise: Major Pokorny, then responsible for the interception service and cryptology (the predecessor of Langer), Captain Ciezki, future chief of the German section of the Cipher Bureau and a radio specialist, A. Palluth, employed as a high level technician by the Cipher Bureau.

Among the students, three distinguished themselves particularly: Marian Rejewski, Henryk Zygalski and Jerzy Rozycki. Rejewski, who had a degree in Mathematics (1st Class), spent the academic year 1929-30 at the University of Göttingen in Germany in order to pursue a special course of actuarial mathematics and returned in the summer of 1930 to take a post as assistant at the University of Poznan. At Poznan, he recontacted Rozycki and Zygalski who both worked several hours per day in an annex that the Cipher Bureau had set up in the town. They were occupied with the study of cryptologic problems related to the double transposition cipher. Rejewski rejoined his two fellow workers. The problem of Enigma was not considered.

At the beginning of 1932, the three mathematicians were warned of the impending closing of the annex. On the 1st of September 1932, they were officially enlisted into the Cipher Bureau at Warsaw. There, for more than a month, they studied the decryption of a German Naval code which they mastered. Suddenly, about the middle of October, Major Ciezki assigned Rejewski to work, alone and in great secrecy, on a new problem.

5. The Extraordinary Success of Mr. Rejewski (October 1932 - January 1933 [10].)

What Major Ciezki had asked of Rejewski was to attempt the complete reconstruction of the German military Enigma. Rejewski was very young — born in 1905 he was then 27 years old — and had never even heard mention of the Enigma machine. But proving himself to be a worker of prodigious talent, he succeeded in 10 weeks (from mid-October to the end of December 1932) in establishing a mathematical theory of the workings of the Enigma, in reconstructing the internal wirings, and in devising the first cryptanalytic methods to decipher the Enigma messages.

a. The First Phase (approximately October 15 - November 15, 1932). Rejewski initially had at his disposal the following information and tools:
i. The two basic German documents on Enigma — GebrauchsAnleitung and Schlüsselanleitung — obtained from Asche and brought by Bertrand on December 7, 1931;
ii. A brand new model of the commercial Enigma;
iii. The continuous flow of German ciphertext messages captured by the Polish interception stations.

The two Asche documents allowed Rejewski to become rapidly familiar with the structure of the machine and the procedures used to operate it.

In studying the implications of the repetition of the message key — the first six letters — Rejewski realized that in 81% of the cases (21 out of 26) the typing of the six letters on the machine's keyboard caused only a movement of the righthand rotor. The rest of the machine could then be considered as a single fixed ensemble. Moreover, the first and the fourth letters, the second and the fifth letters, and the third and the sixth letters each corresponded to the same cleartext letter.

Rejewski then deduced the mathematical consequences implied by his two observations. Namely, that it was possible to express the encipherment of these six letters by equations representing the six successive permutations involved. Further, if given a sufficient number of messages from the same day (which would therefore use the same settings of the machine), one could draw out certain characteristics of the permutations of that day. The system of six equations was therefore theoretically capable of a solution which would, in turn, yield the wiring of the rightmost rotor.


Since the Germans changed the order of the rotors periodically, each of the three movable rotors would, from time to time, find itself placed on the extreme right. This allowed the three rotor wirings to eventually be reconstructed. In a similar manner, the reversing drum (Umkehrwalze) wirings could also be found.

The method which had just been described was theoretically applicable even if the machine settings (order of rotors, Ringstellung, Grundstellung, and plugboard setting) were completely unknown. Nevertheless, in this case, carrying out the actual calculations involved was practically impossible because no powerful calculating devices such as today's computers were available. Theoretically, one could dream of a solution but it implied use of a very large number of messages and a succession of luck too extraordinary to be believable

Once again, the work seemingly landed in a blind alley.

b. The Second Phase (approximately November 15 - December 31, 1932).

In October, Major Ciezki had only furnished Rejewski with the two basic documents on Enigma. No doubt he wished to determine the ability of the young mathematician to use them. Experience having decided the question (beyond all possible expectations) Ciezki gave Rejewski, probably around November 15, two new documents also acquired from Asche, and transmitted by Bertrand. These documents were two monthly schedules giving the daily settings for the Enigma. Rejewski had at his disposal the ciphertexts of the messages intercepted during these two months. He knew from the latest documents the order of the rotors, the setting of the rings, the rotor starting positions (for enciphering the message key), and the plugboard connections. All this information allowed Rejewski to simplify his equations enough to solve them. By luck, the two months covered in the schedules fell in different trimesters of the year [13]. Because the rotor order was altered each trimester, two different rotors occupied the rightmost position. The solution of the wiring of these two rotors allowed the solution of the wiring of the third rotor. With this done, the wirings of the reversing drum (Umkehrwalze) could be determined without difficulty.

Rejewski immediately threw himself into his calculations. Initially, no valid results were obtained. In seeking the cause for this failure, Rejewski had the idea that his assumption about the wiring from the plugboard to the entry/exit drum had been wrong. In the commercial Enigma, the wiring was straight through from the keyboard to the 26 contacts of the entry/exit drum. This placed the wires in the same order as the letters on the keyboard of the standard German typewriter: the letter Q connected to contact 1, W to contact 2, E to contact 3, etc.

Rejewski had thought initially that the military Enigma was wired similarly. The keys being wired according to a straight alphabetic correspondence (key A joined to plugboard connection A, B to B, etc.) and the plugboard letters joined in the order of the entry keyboard letters with the contacts of the entry/exit drum.

Rejewski told himself that if the order had been modified, the methodical Germans would not have changed to a random wiring. He then tried the wiring where the letters were arranged in regular alphabetic order (A connected to the first contact, B to the second, C to the third, etc.). This inspiration was correct and, as if by a miracle, the internal wirings began to emerge from his calculations.

By the end of 1932, the combination of mathematical genius displayed by Rejewski and the trove of secret Asche documents sent to the Poles by Bertrand, as well as a good dose of luck (Rejewski's intuition concerning the entry wiring), had led to the complete reconstruction of the German military Enigma.

This reconstruction was not only carried out on paper. An eminent technician, the engineer Antoni Palluth, one of the directors of the AVA radio equipment fabrication factory in Vasovie and also a consultant for the Cipher Bureau had, following Rejewski's progress, modified a commercial Enigma into the military model of the machine. This perfectly operational prototype served first to verify Rejewski's results. Second, this machine was used as an industrial prototype for the AVA manufacture of replicas of the military Enigma destined for the Polish cryptologic services.

D. THE FIRST METHODS OF DECIPHERMENT

The knowledge that the first six letters of each message was the twice enciphered key to that specific message furnished Rejewski with the means of reconstructing the internal wirings of the Enigma. It also led to his decipherment of the messages coming from the machine.

A detailed description of the decrypting methods developed by Rejewski while at the same time reconstructing the Enigma machine will not be attempted here. Suffice it to say that these methods allowed, from some characteristics of the 6 letter groups corresponding to the twice repeated enciphered message keys, one to determine the settings of the machine compatible with these characteristics, to test these settings and finally to find out the exact message key used. After this had been done, the decipherment could be carried out under the same conditions as the German clerks to whom the messages were addressed had. In other words, using the same machine settings, each message could be deciphered by typing the ciphertext on the keyboard of the machine.

During his work in establishing the first methods of decipherment, Rejewski could exploit certain errors committed by the German operators. In particular, the use of particular groupings of letters such as AAA, BBB, SSS, etc. as message keys. The use of such letter groups was forbidden to the operators at the end of 1932, which was, of course, too late.

From the beginning of 1933, the decipherment methods perfected by Rejewski showed themselves to be entirely adequate.

The Polish success was complete.


E. THE ROLE AND IMPORTANCE OF THE FRENCH CONTRIBUTION

The Poles and only the Poles attempted the reconstruction and cryptanalysis of the German military Enigma. Alone, they carried out the totality of intellectual, cryptologic, and mathematical investments necessary. And they alone arrived at the solution. Chapter IV will show that they were the only ones, between 1933 and the beginning of 1939, to retain mastery over the Enigma and its decipherment. For these extraordinary exploits, the Poles must receive all of the credit and retain all of the glory.

It does not diminish the value of their accomplishment to recall that there was a French contribution and to try to evaluate its importance.

The French contribution to the Polish success was both mediocre and fundamental:

a. Mediocre, because that contribution was limited to the transmission by Bertrand of the documents obtained from Asche;
b. Fundamental, because the documents supplied by Bertrand played a decisive role. The writings of Rejewski and a careful study of the timetable of events leads to the following inescapable conclusions: the first two documents, Gebrauchsanleitung and Schlüsselanleitung, constituted the indispensable base upon which Rejewski's genius built his reasoning and equations; the two monthly schedules of daily machine settings allowed the theory thus established to be actually carried out. Without these four documents, success could not have been attained.

It would not be necessary to emphasize the role of the French contribution had that role been correctly described and appreciated in the available documentation. But, if the existence of a French contribution is acknowledged at all, most works on the Enigma appear to agree that the documents furnished by the French intelligence service merely facilitated the tasks of the Polish mathematicians and, that, in their absence, the same result would have been obtained with only an additional delay. These affirmations do not correspond to reality. The documents furnished by Bertrand not only facilitated the Polish success — they made it possible. One may consider that the methodical and obstinate efforts of the Poles have a value superior to the stroke of luck which was offered to the S.R.F by Hans Thilo Schmidt. But, without the Asche documents furnished to the Poles, it is nearly certain that the Enigma would have kept its secrets a lot longer [14].

There remains a secondary problem to discuss in regard to the French contribution. At first glance, this problem may appear quite strange. Rejewski has stated publicly, and there is every reason to believe him, that he only received four secret German documents (the two basic ones and the two monthly sets of machine settings). Bertrand has indicated that, thanks to Asche, a very large number of documents (notably, the monthly machine setting schedules from December 1931 to December 1932) were obtained, and it would seem implicit that this bulk of information was immediately repeated to the Poles, i.e., Langer and Ciezki. (However, there do not exist accompanying letters or memoranda explicitly proving that this transmission took place.)

Could this situation be explained by the fact that Rejewski was only allowed to see a fraction of the material received by his superiors? On second thought, such a situation would not appear surprising. The need to know principle is the rule in intelligence work. Rejewski, a completely new employee of the Biuro Szyfrow, had proved his worth when confronted with two documents (GebrauchsAnleitung and Schlüsselanleitung) which were, in turn, followed by two other documents (the two monthly schedules of machine settings). Those documents were sufficient to lead to the success of his enterprise and there was no cause to reveal the existence of the other papers (which were no longer indispensable to his work but would serve to verify it). Finally, it is worth noting that, up to 1939, Rejewski ignored that the base of his work had been furnished by the S.R.F. Only Langer and Ciezki knew of the liaison with Bertrand.


NOTES FOR CHAPTER III

1. R. Lewin — Ultra Goes To War. See note on page 30, paperback edition. J. Garlinski — Intercept: p 2-3. M. Rejewski — How the Polish Mathematicians Broke Enigma — English translation of an article which appeared in 1980 (posthumously) in a Polish mathematical magazine; this translation is to be found in the appendix of the book Enigma, by W. Kozaczuk, p. 246. Weltkrieg, p. 67.
2. T. Lisicki Die Funkaufklärung und Ihre Rolle im zweiten Welkrieg, p. 67.
3. And not 1932 as indicated by Bertrand (Enigma, 23). The mixture of dates in Bertrand's book which sometimes place the events of 1931 in 1932 did not help the Enigma historians. The chronology now established beyond any doubt.
Most historians of Enigma (excluding Garlinski and Stengers) have been deceived by the date 1932 which has, in turn, led to inextricable difficulties in reconstructing the correct timetable of events. A more serious fact is that Rejewski himself in his last articles also adopted the date 1932. Existing documents including accounts by Bertrand himself leave no doubt as to the actual date.
4. The Chiffrierstelle was not the only German service dealing with cryptology. E. Höttenhain in Die Funkaufklärung und ihre Rolle im zweiten Weltkrieg, p. 100 citation 8, shows clearly the waste and disorder resulting from the lack of coordination.
5. The extraordinary story of Hans Thilo Schmidt, whose importance extends beyond Enigma material, is recorded by Paul Paillole in his work Notre espion chez Hitler, Editions Laffont, 1985.
6. The wartime destruction in the Polish, French, and German archives was such that no copy of the original 1930 documents exists any longer in continental Europe. The only place where, perhaps, these documents might still be found is in the archives of the British Government Code and Cypher School (GC&CS), which received copies of them from Bertrand in 1931. These archives are inaccessible.
The original 1930 documents did not carry the reference "L" indicating Luftwaffe since this organization did not exist officially until 1935. The documents Dv. g 13 and 14 were the object of numerous editions which incorporated the successive modifications of the Enigma machine and its method of use. These modifications do not seem, at least before May 1940, to have changed any essential aspect or spirit of the documents. Thanks to the amiability of the Militärgeschichtliches Forschungsamt (the historical service of the German Army located in Freibourg in Brisgau), the author of this study possesses copies of the following:
a. The Gebrauchsanleitung of January 12, 1937 (the edition is of January 1940 but specifies that the text is unchanged); this is a document of 16 pages of which 11 are text and the remaining are photographs of the machine pointing out its different parts.
b. The Schlüsselanleitung of January 13, 1940. 14 pages.
7. Cf. Hinsley, British Intelligence in the Second World War, Vol. 1, p.488. Bertrand did not go to London on this occasion. The documents were given to the representative of the I.S. in Paris, Commander Wilfred Dunderdale, who carried them to England. (He left on the 23rd of November for England. Source: P. Paillole, Notre espion chez Hitler, p. 39)
8. The dates of Bertrand's visits to Warsaw must be connected with those of the Asche meetings, each of which resulted in the transmission of documents. In 1931 and 1932, Bertrand met Asche four times
• November 7-8 1931 at Verviers
• December 19-20 1931 at Verviers
• May 7-8 1932 at Verviers
• October 29-30 at Liège.

Besides this, REX (aka Lemoine), the intermediary of the S.R.F., met Asche in Berlin August 1-2, 1932 and received from him documents which arrived in Paris August 10 by diplomatic pouch.
9. The University of Poznan was excellent. All of the students there spoke both Polish and German.
10. Rejewski's complete and detailed explanation of the mathematical theory of the Enigma, the corresponding system of permutations and their solution is beyond the scope of this study. The interested reader can consult the writings of Rejewski himself, writings from (1984) are now available in English. These writings consist of appendices D and E from the book Enigma, by W. Kozaczuk. Appendix B of the same book, a liberal transcription of an interview with Rejewski recorded July 24, 1978, furnishes an extraordinarily vivid illustration of Rejewski's career and work.
Another work of Rejewski cited (with numerous extracts) but not furnished in extenso in the work of Kozaczuk must be quoted. This article has been, fortunately, fully translated and published in English in Cryptologia, 6, No. 1, Jan. 1982, pp. 75-83. The Remarks on Appendix 1 to "British Intelligence in the Second World War" consists of Rejewski's comments of the text published in Volume I of Hinsley and furnishes precise corrections.)
11. And it appears that the computer time required may, even today, be prohibitive.
12. Cf. Rejewski — How the Polish Mathematicians Broke Enigma. Appendix D of the book Enigma, W. Kozaczuk, p.258. Rejewski explicitly indicates that, without further information, it was impossible to solve the permutation equations.
13. The fact that the two monthly schedules belonged to different trimesters is perfectly clear. By contrast, the months covered by the two schedules and the date and circumstances of their possession by the Poles present several problems whose elements are given below:
a. Bertrand (Enigma, p. 32) indicates having received from Asche (and then transmitted to the Poles) les tableau mensuels de configurations (comportant une configuration par jour) pour Décembre 1931, les années 1932, 1933, et 1934 (1er semestre).
b. Rejewski states in his works, and there is every reason to believe him, that only two monthly schedules of machine settings were put at his disposition covering two months belonging to different trimesters. However, he has varied the dates. In his last article (posthumous, 1980) he indicates the months of September and October 1932 which appear not only plausible but probable (see below). But, he has also indicated that these documents were put at his disposal December 9, 1932 (cf. Kozaczuk p.256). This date, apparently based on the erroneous date (December 7, 1932 instead of December 7, 1931) of the first visit of Bertrand to Warsaw, is evidently false. It would have left Rejewski only three weeks to finish all of his calculations before December 31.
In the first paper that Rejewski wrote, in France during 1942 under conditions hardly favorable to composing memoirs, he quotes the months of October and December 1931 — a double impossibility. Bertrand has said that he did not have any table of keys at his disposal until December 1931. The months of October and December belong to the same trimester. (It is strange that, even in the conditions of 1942, Rejewski had forgotten a technical detail of such importance.)
c. The date of the Asche meetings, i.e., the dates of Bertrand's visits to Warsaw, do not allow one to settle, a priori, this question.

The monthly schedules could have only been transmitted by Asche during his meetings with Bertrand (or, in one case, with another agent of the S.R.F., Lemoine). These meetings took place November 7-8 and December 19-20 in 1931 and May 7-8 and October 29-30 in 1932. The meeting with Lemoine was on August 1-2, 1932.
Bertrand visited Warsaw from the 7th to the 11th of December 1931, from the 9th to the 11th of May, and from the 17th to the 21st of September 1932. (All of the preceding dates were furnished to the author by Colonel Paillole.)
Assuming, logically, but, as will appear, wrongly, that the monthly schedules of keys were not available in Germany until a few days before the first of the month in which they were to be used, one is led to think that the schedules for September-October 1932 could have been furnished by Asche only during the meeting of October 29-30. This would be too late to have been provided by Bertrand to Warsaw at the time of his visit of September 17-21, 1932. There would nevertheless remain other possibilities: the schedules in question could have been transmitted to Langer by an intermediary representative of the Polish services in Paris (the contacts here were constant). It would be equally possible that Langer himself came to Paris at the beginning of November 1932 to take delivery of these documents. Whatever hypothesis is adopted, the schedules provided by Asche to Bertrand during October 29-30 could have found themselves in Warsaw about November 15 without difficulty. This fits perfectly into the timetable of events.
The book of P. Paillole, Notre espion chez Hitler (p.53), provides the solution to the problem.
The monthly schedules for September and October 1932 (the October schedule in provisional form) were provided by Asche to Lemoine in Berlin at the beginning of August 1932. This is early enough to have been given to the Poles by Bertrand at the time of his stay in Warsaw, September 17-21.
The date (approximate) of November 15, 1932 when Rejewski received the two monthly schedules of keys suggests they referred to the months September and October 1932. It was logical to furnish Rejewski with the latest schedules available since these schedules would correspond to the most recent ciphertexts intercepted by the Polish listening stations and Rejewski already had these ciphertexts.
14. The relative misestimation of the French contribution is understandable. The only French work concerning Enigma was that of Bertrand , Published in 1973. This work had the distinction of being the first one to be describe what had happened. But, the style of this book, the understandable tendency for the author to discuss more his personal role in the events rather than to provide a complete, clear and technical documentation of the facts, arouses suspicion. Historians cannot be blamed for having preferred to utilize two short reports (unpublished but easily accessible) drawn up by Colonel Mayer in London during 1974in response to the books of Bertrand and Winterbotham. Colonel Mayer was the former head of the PolishIntelligence Services. Mayer has written, "At the stage in which the research of the Enigma was at that time in the Polish cryptological section, these documents [those sent by Bertrand] appeared not to be indispensable for the final solution of the problem. But, unquestionably, they facilitated it." In writing this sentence Colonel Mayer was certainly sincere, but, even if he had closely supervised the work of Langer, Ciezkim and Rejewski, this supervision would not entail familiarity with the difficulties encountered, the methods used, nor the solutions found.
The works of Rejewski, partially in English, have been available only little while. Their interpretation is sometimes made difficult by the fact that Rejewski had ended up being persuaded that the first visit of Bertrand to Warsaw was on December 7, 1932 instead of 1931.
Of course, recourse to the original documents is impossible since the French and Polish archives were destroyed or lost during the war.
The book of P. Palileo, Notre espion chez Hitler, published in September 19785 describes in detail the extraordinary role played by Hans Thilo Schmidt, alias Asche, and, finally, permits one to appreciate exactly the magnitude of the Asche documents and their importance to the Poles.

BIOGRAPHICAL SKETCHES

Gilbert Bloch is retired and lives in Paris. He is member of the French Military Reserves. His hobbies include cryptology and parachute jumping.

C. A. Deavours teaches a unique sequence of courses in cryptology at Kean College of New Jersey. The courses address both technical and historical issues in cryptology.


Enigma Before ULTRA The Polish Success And Check (1933-1939)

Gilbert Bloch1
Translated by C. A. Deavours2

ADDRESS: (1) 7 Rue du Cher, 75029 Paris FRANCE and (2) Department of Mathematics, Kean College of New Jersey, Union NJ 07083 USA.

We continue the translation of selected sections of Gilbert Bloch's book, Enigma Avant Ultra (Enigma Before Ultra). See the previous issue of this journal for the first part of the publication, Chapter III — Polish Work and the French Contribution

CHAPTER IV

A. OVERVIEW

At the beginning of 1933, the Poles had attained their objectives:

a. They had completely reconstructed the German military Enigma. By this time, they possessed a replica of the machine — a prototype, of course, but fully operational;
b. They had developed methods which, exploiting the German error of the twice enciphered message key, permitted the determination of the machine settings and the individual message keys, and, therefore, the decipherment of intercepted messages on an Enigma replica with as much ease as that of the German addressees.

From 1933 until the end of 1938, the Poles kept their advantage. The Biuro Szyfrow read the majority of German radio Enigma messages transmitted by the Army, the Luftwaffe, and the S.S. [1] All through this period, the Germans changed certain workings of the machine, increased the frequency of key changes, and altered ciphering procedures. The Poles always, and with surprisingly short delays, kept up with these changes and continued their decipherments. To better aid them in their tasks, the Poles — particularly the trio of mathematicians Rejewski, Rozycki, and Zygalski — devised numerous methods, invented hand and mechanical devices, and, finally, created electro-mechanical machines which, from characteristic data of the intercepted messages, could find the daily settings used for the encipherment. During all of this period, the Poles alone were able to accomplish these things. [2]

On the 15th of December 1938, following new German modifications, the Polish lost the ability to decipher messages even though they had correctly diagnosed the new changes made, and reconstructed the new workings. In order to successfully confront these changes, the Poles would have needed to augment considerably (in fact, tenfold) their available materiel. For them, this was impossible.

On the 24th of July 1939 — one month before the beginning of the Second World War — the Poles revealed, during a meeting with French and British experts, their results. They gave to each of their astonished allies a copy of their German Enigma replica (complete with 5 rotors!), a detailed description of all decipherment methods, and schematics of all the devices they had developed to decipher the machine in which the Germans placed — and kept — complete confidence.

Until this date, the Poles had kept absolute secrecy even towards Bertrand who, from 1933 to 1939, continued to make numerous trips to Warsaw (compensated for by an equal number of trips of Langer to Paris) in order to transmit documents. This absence of reciprocity, this lack of tit for tat, should not surprise. Examples abound which show that relations between the Services are never free of hidden motives and that an eye for an eye is not always the rule. [3]

The Polish gift of 24 July 1939, which would have never been possible without the French aid received, was to play in its turn an essential role in the future British success.

B. THE CONSTRUCTION OF THE ENIGMA REPLICAS

Possession of only one copy of the Enigma machine was, even in 1933, not sufficient, and it was clear that in the future many more machines would be needed. Consequently, the Poles launched the industrial manufacture of Enigma replicas. This task was the responsibility of A. Palluth, one of the associate directors of the AVA company in Warsaw who specialized in the fabrication of radio equipment. He had been closely connected with the work of the Biuro Szyfrow and it was certainly he, following Rejewski's progress in reconstructing the schematics of the Enigma, who translated these schematics into actual pieces of equipment.

The AVA factory produced the first Enigma copies in 1934. In 1939, the Poles had 17 Enigmas of which two were given to the Allies (one to the French and one to the British). Taking into account the considerable number of Enigma rotors used in the mechanical devices invented by the Poles (cyclometers and bombes) to aid them in the tasks of decipherment, the number of Enigmas (or parts of Enigmas) produced by the factory between 1934 and 1939 was much greater (by at least 4 or 5 times) than one would think from the number existing in 1939.


C. THE POLISH SUCCESS (FROM 1933 TO THE END OF 1938)

To achieve mastery of the Enigma, the Polish aimed at numerous methods "from the simplest to the most complicated, from manual to mechanical, from the cheapest to the most costly." [4]

1. Manual methods

As has already been mentioned (c.f. Chapter III, Polish Work and the French Contribution), the methods used by the Poles consisted, essentially, of listing first characteristics from the first six letters of the encrypted part of the intercepted German messages — the portion corresponding to the encipherment of the message key (one group of three letters repeated twice). From these groups, one could then determine the settings of the machine which were compatible with these characteristics and, if the number of messages were sufficient, these compatible settings could be reduced to a small enough number that could be tested on the Enigma replicas. The tests permitted one to determine the actual setting used for the day under consideration (the daily key) and, by another process, the specific key for each message. It sufficed then to read the messages using the Enigma replicas.

2. The First Mechanical Methods

In order to make their work easier and faster, the Poles invented an apparatus, the cyclometer [5] composed of elements (rotors) from two Enigma machines joined together. This instrument allowed one to setup mechanically a catalogue of compatible settings, at least as long as the Germans made use of a daily master Grundstellung. The cyclometer began service in 1937.

3. The First Enigma Modifications — 1937

From 1930 to 1937, The German military Enigma hardly changed.

Of course, the method of use evolved: increase in the frequency — ending in a daily alteration of all elements of the settings — increase in the number of steckers used. [6] This evolution was not of such a nature as to seriously impede the Polish work.

It was the 1st of November 1937 when the Germans brought to their machine the first important modification. The original reversing drum (Umkehrwalze) was replaced by a new one (Umkehrwalze B). The Polish reconstruction of the wirings of the new reversing drum was very rapid and decipherments continued as in the past.

The facility with which the Poles adapted to this change would seem to pose a problem. It is certain that, once having determined the nature of the modification (the change of the Umkehrwalze), the Poles (by their mathematical theory of the machine, by the permutation equations which resulted from this theory, and by their methods of solving these equations) found themselves perfectly well able to determine the wirings of the new drum. [7]

But how was it possible for the Poles to establish an exact diagnosis, that is, how could they find the exact nature of the unexpected change? The Polish answer to this question is apparently clear: the Germans, always strongly methodical, sent by Enigma to all users a text reminding them that, beginning a 00:00 hours on 1 November, they must mount in their machines — formerly called the Wehrmacht Enigma, the new reversing drum. [8] The Poles, thanks to their decipherments, were therefore forewarned and mastered the new machine as well as the old one.

In January 1938, a test of two weeks by the Polish Cipher Bureau, showed that 75% of the intercepted messages had been decrypted. [9]

4. Procedural Modifications from September 1938 and the Polish Bombe

The 15th of September 1938, the Germans abandoned the daily master Grundstellung, used until then, for the encipherment of the specific message key for all messages of that day. From now on, a particular Grundstellung for each message had to be randomly chosen by the operator (and indicated in the clear in the first part of the message).

Here again, the Poles were very quickly informed of the nature of the change, and, equally quickly, found the means of parrying it.

a. A new electro-mechanical machine, the bomba incorporating elements (rotors) of six Enigmas was invented. Six such bombes, each corresponding to one possible rotor order of the Enigma, were constructed. These bombes became operational in November 1938.
b. The mathematician H. Zygalski devised a manual system based on the utilization of a series of perforated sheets. Each perforation corresponded to one configuration of the machine compatible with one characteristic of a message key. The juxtaposition, on a light table, of a certain number of sheets corresponding to the characteristics of the keys made apparent the cases in which these characteristics were juxtaposed by permitting light from the table to shine through perforations in the pile of sheets. These perforations corresponded to compatible settings. Once the sheets were established (by an initially long process, necessitating complicated calculations and mathematical checks, but done only once), the selections could be rapidly made in a few minutes.


D. THE POLISH CHECK (DECEMBER 1938-JULY 1939)

On December 15, 1938, the Germans added to the first set of I, II, and III (which had six possible rotor orders) two more rotors, IV and V. The machine itself was not modified and always used only three rotors, but the number of possible rotor orders was increased from 6 to 60.

This time also, the Poles are said to have had immediate knowledge of the nature of the change: they had known for several months — from intelligence — that in case of extreme international tension, the German operators would receive two new rotors (already stockpiled!) baptized mobilization rotors (Mobilisierungswalzen), and that the simple ordering of the three existing rotors would become in this case a selection of three rotors out of five, followed by an ordering of the three rotors selected in the machine. Moreover, once again, the Poles are said to have intercepted the Enigma message giving the order to put the new procedure into practice.

The Poles were even able to reconstruct the wirings of the two new rotors thanks to the S.S. During September 1937, a new network of Enigma users appeared transmitting S.S. and Sicherheitsdienst messages. Before being enciphered on the Enigma, these messages were enciphered manually using a method that the Poles easily mastered. For some reason, after the 15th of December 1938, the S.S. and the Sicherheitsdienst used, like everyone else, the entire set of five rotors but kept the old keying procedure instead of the Grundstellung modification of September 15, 1938. Under these conditions, the experienced Polish mathematicians were able to reconstruct the internal wirings of rotors IV and V, and continued to read the Enigma messages of the S.S. and the Sicherheitsdienst (until July 1939).

On the other hand, the Enigma messages of the other networks had become unreadable to the Poles in spite of the reconstruction of the two new rotors. This occurred because of a lack of technical resources and not because of a lack of cryptanalytic mastery. It is easy to see why: in order to read the Enigma using a set of three rotors the Poles had six Bombes. The addition of two more rotors multiplied the number of rotor orders by a factor of 10 (i.e., from 6 to 60). In order to expand the number of Bombes (i.e., to build 54 additional Bombes), the Poles would have had to quickly produce 54 · 6 = 324 more sets of Enigma rotors (since each Bombe used six Enigma rotor sets). (The Poles had only 17 Enigmas in service.) This task largely surpassed the production capacity of the AVA factory. In spite of their intellectual mastery, the Polish were circumvented by the lack of materiel. Also, the increase in the daily number of plugs (Steckers) used (beginning January 1, 1939) limited the effectiveness of the Polish Bombes.

Even if the lack of materiel could have been overcome, the Polish could not have hoped in so short a time to increase their cryptanalytic personnel for dealing with the work which had increased tenfold.

Zygalski's perforated sheets method suffered the same problems. The number of sheets required was also to be increased tenfold. The Poles put themselves to the task, but the magnitude of the necessary calculations made the process very slow, and only very partial results were obtained before the declaration of war.

It is Bletchley Park that, thanks to a prodigious concentration — both quantitative and qualitative — of staff and materiel, mastered the new problems. But the fact of having taken advantage of the Polish experience and having had access to the Polish work represented an enormous advantage for the English.

E. WAS THERE A FRENCH CONTRIBUTION TO THE POLISH SUCCESS OF 1933-38?

The French contribution to the Polish success of 1932 is universally recognized even if its real role may appear misestimated.

For the period 1933-38, there is no mention, either in Polish documents or other sources and available references, of a French contribution. Does this silence correspond to reality?

a. The Polish mathematicians, Rejewski, Rozycki, and Zygalski have always said that they had used no secret German documents other than the four used by Rejewski during the last trimester of 1932. There is every reason to believe them. But, this does not imply that their superiors, Langer and Ciezki, did not have possession of other documents which, while not indispensable to the mathematicians' work, which went along magnificently, permitted control of the quality of their results.
b. Bertrand states in his book that Asche delivered during the period 1931-34 a number of important documents relative to Enigma besides the basic ones, the Gebrauchsanleitung and the Schlässelanleitung. [10] It is practically certain that these documents were transmitted to the Poles.
c. Bertrand mentions (Enigma, p. 33) that in 1934 Asche left the Chiffrierstelle; the reader is then led to the conclusion that beginning at this date, Asche was no longer in a position to provide cryptologic information on the Enigma.
The reality is quite different. Use of new — and irrefutable — documents allowed Colonel Paillole, in his book Notre espion chez Hitler, to show that Hans Thilo Schmidt alias Asche did not leave the Chiffrierstelle in 1934 but only his role there changed. Formerly, he was in charge of the liaison with the Forschungsamt, a department reporting to the Air Ministry (i.e., Marshall Goering) and one of the most important German cryptologic agencies. It was not until September 28, 1938 that Asche was administratively attached to the Forschungsamt (at an important level since he was director of the Templin station).
Up to this date, at least, he was marvelously placed to secure cryptologic documents and he made considerable use of this advantage.
Hans Thilo Schmidt had, in 1931-32, met six times with the agents of the French Services. Bertrand was present at four of these interviews (c.f., note 8, Chapter III and Appendix II of this work).
From 1933 to 1939, the records show 14 meetings between Asche and Bertrand (and there were other meetings at which Bertrand was not present). Each one of these encounters resulted in a transmission of documents.
d. Bertrand went to Poland three times during 1931-32 (c.f., Chapter III). From 1933 to 1939, there were 10 trips to Warsaw (and one to Czechoslovakia), [see Appendix II of this work].
Langer went to Paris (on dates not precisely known) as many times as Bertrand went to Warsaw. Bertrand never went to Warsaw, and Langer never returned from Paris, empty-handed.
e. One part of the documents delivered by Asche between 1934 and 1940 contained general (but important) information on German armed forces and politics. These documents were handled by the German department of the French Services. But the mere fact that Bertrand (whose prerogative was limited to the cryptologic field) had continued to meet Asche between 1934 and 1938 shows unequivocally the duration of the deliveries of the cryptologic information. Bertrand himself gives some examples, not relative to Enigma.
Colonel Paillole (op. cit.) is able to show that, during the course of this period, Asche continued to deliver Enigma documents and, in particular, a considerable number of monthly key setting schedules (see Appendix II of this work). At each interview, Asche furnished two of these schedules!
These documents were surely transmitted to the Poles and, beginning in 1937-38, at least partly to the English.
f. It is therefore absolutely certain that Enigma documents continued to be transmitted after 1934. What were these documents and what role did they play?
i. Appendix II gives a list (non-exhaustive) of the monthly key setting schedules furnished by Asche and their date of delivery.
It seems certain that in the Polish case, no key setting schedules (apart from the two schedules which play a decisive role for Rejewski in the reconstruction of the machine) were ever delivered to the mathematicians who, by their own methods, determined the daily keys. Langer and Ciezki would have then used these documents to check on the work of the mathematicians. (Taking into account the work rules of a cryptologic service, this hypothesis is not unreasonable. [11])
In the case of the British, this problem is studied in Chapter V.
ii. Aside from the transmission of the monthly key setting schedules, no explicit mention has been found as to whether or not Asche transmitted information about the procedural and technical modifications made by the Germans to their system between 1934 and 1938. These modifications, which have been detailed in Chapter II, are numerous. It cannot be excluded that such information had indeed been transmitted and would have been of the sort mentioned to the Poles. (The reasons that the Polish have given themselves to explain their knowledge could be cover stories — told in all good faith.)
g. There is a possible objection: if Enigma documents were transmitted after 1934, then the absence of mention of this fact, understandable in Polish reports and studies, in Bertrand's book is strange (to say the least).
This argument, while valuable, is not decisive. Bertrand wrote his book in 1973. He knew that from 1933 to 1939, the Poles whom he had pestered with questions about the course of their work had beat around the bush up until July 24, 1939. He could speak of what he had furnished from 1931 to 1934. Psychologically, it was perhaps difficult for him to admit that he had continued to go on helping the Poles for five years without receiving anything in exchange.


NOTES FOR CHAPTER IV

1. Enigma messages sent by the German Navy were decrypted only rarely. The lack of a sufficient number of intercepted messages is the reason.
2. The British seem to have begun study of the Enigma beginning in 1936. During the Spanish Civil War (beginning in 1937) the English deciphered German, Italian, and Spanish Nationalist messages enciphered on commercial models of the Enigma (slightly modified, but without the plugboard). In July 1939, at the moment of the Polish gift, the English had not yet reconstructed the German military Enigma.
3. Political factors can explain secrecy observed by the Poles. Political relations between Poland and France from 1933 to 1939 were far from always being excellent and information received from the Poles, partly due to their decryptments, made the Poles doubt the willingness of the French to aid Poland in case of attack, and also of the capacity of the French to keep secret the fact that the Enigma had been decrypted. As for the English, they gave their guarantee to Poland only March 31, 1939.
Besides, Bertrand was not a cryptologist and did not share the fraternity which joins specialists. It would not have been easy to explain to him the mathematical and technical complexities of the Polish decipherments.
4. T. Lisicki in Die Funkaufklärung und ihre Rolle in 2. Weltkrieg, p. 78.
5. So called because of peculiarities revealed in the encipherment of message keys leading to the determination of cycles of letters.
6. For this subject, see the part of Chapter II dedicated to setting changes.
7. Welchman (The Hut Six Story, p. 113-115) tells that following the interception of a message using a new trial Umkehrwalze, Bletchley Park reconstructed the wirings in a few hours (less than one night).
8. Source: interview with Colonel Lisicki.
9. Source: Memoranda of Colonel Mayer.
The percentages refer, without doubt, to messages sent by the Army and the Air Force. The alterations in machines and procedures used by the Navy rendered its messages practically inaccessible.
10. G. Bertrand — Enigma, p. 32
"...the monthly key setting schedules (one key per day): Heeres-M., for December 1931, the years 1932, 1933 and 1934 (1st semester).
...10 documents concerning the Enigma cipher machine (1930), that became Enigma I in 1931 (when Enigma II was put into service) and, finally, Enigma type Wehrmacht (1937), when its use was extended to all of the Wehrmacht, among which two were of decisive value:
...a piece of information showing, for a given setting of the machine, the ciphered letters obtained by pressing each letter of the keyboard.
...a ciphertext, in several parts, with the corresponding cleartext and daily key used."
11. The following figures permit one to appreciate the importance that can be assigned to the delivery of the monthly Enigma key setting schedules. The combination of data furnished in the works of Bertrand and Paillole shows that, for the period extending from December 1931 to September 1938 inclusive (a total of 82 months), Hans Thilo Schmidt gave the French Intelligence Service 54 monthly schedules (at least!). Therefore, 66% of the settings of the Enigma used during this time interval were known.
The Poles received the greater part, if not all, of these schedules. In a letter sent to the author on January 15, 1986, Colonel Lisicki admitted the possible existence in the Biuro Szyfrow of a Section (whose existence would have been hidden from the Polish mathematicians) which decrypted the Enigma messages using the monthly key settings sent by Bertrand. For its work, such a section, which could not have begun operating until 1934 (the year of the first delivery of the Enigma copies by the AVA factory), would have been able to determine using the methods perfected by the mathematicians the particular key for each message.)

BIOGRAPHICAL SKETCHES

Gilbert Bloch is retired and lives in Paris. He is member of the French Military Reserves. His hobbies include cryptology and parachute jumping.

C. A. Deavours teaches a unique sequence of courses in cryptology at Kean College of New Jersey. The courses address both technical and historical issues in cryptology.


Enigma Avant ULTRA (Enigma Before ULTRA)

Gilbert Bloch1
Translated by C. A. Deavours2

ADDRESS: (1) 7 Rue du Cher, 75029 Paris FRANCE and (2) Department of Mathematics Kean College of New Jersey, Union NJ 07083 USA.

[We continue the translation of selected sections of Gilbert Bloch's book, Enigma Avant Ultra (Enigma Before Ultra). See the July 1987 (Vol. XI, Number 1) and October 1987 (Vol. XI, Number 2) issues of this journal for the first parts of the publication, Chapter III — Polish Work and the French Contribution and Chapter IV — Polish Success and Check, respectively.]

CHAPTER V
A. THE ENGLISH, THE FRENCH, AND THE POLISH

The English do not seem to have shown much interest in the birth and childhood of the military Enigma; perhaps, because Germany, at the end of the 20's and beginning of the 30's, was not considered a potential source of trouble for them. When Bertrand tried, at the end of November 1931, to interest them in the documents which Asche had just provided, the English carefully filed the photocopies furnished (which consisted of a description of the Enigma I and its operating instructions, i.e., the Gebrauchsanleitung and the Schlüsselanleitung) and gave no follow up offer of collaboration.[1]

By 1936, the situation in Germany was not so reassuring, and the British Services had changed their tune; they asked the French if any new information was available. The English received the texts of intercepted messages as well as several goniometric fixes of the transmitting stations.[1] In 1938, probably at Bertrand's initiative, new contacts took place. The English then received the text, both clear and ciphered, of four Enigma messages [2], and a report on the radio network of the German Army. This time, they were quite interested and were provided with other documents as well, in particular, they were provided with monthly key settings for the Enigma.

By this date, there is no doubt that the English had begun their own studies of the Enigma and had come up with several of their own ideas on the subject.[3] They possessed two commercial Enigmas bought in 1928 at the request of the Admiralty. (The British cipher machine, "Type X," christened finally "TYPEX" and adopted in January 1935, was an improved Enigma.[4]) The documents furnished by Bertrand in 1931 described the military Enigma and its first operating instructions. But, unlike the Polish, the English were unable to reconstruct the internal wiring of the machine.

At the initiative of Bertrand, the Polish, French, and English experts held a meeting in Paris on the 9th and 10th of January 1939. The Polish revealed nothing of their success nor of their problems since the 15th of December 1938. The English avoided talking about their work and, as far as cryptology, Bertrand had nothing to offer. The contacts were useful, nothing more.

At the beginning of July, it was the Polish who summoned everyone together in Warsaw. During the 24th and 25th of July 1939, they revealed to their audience all of their work. They gave to France and to England two of their Enigma replicas (one for each country), the plans for their Bombe, copies of their perforated sheets, in brief, all they had accomplished in 10 years of effort. The Enigma destined for England was taken to London on the 16th of August by Bertrand and given to Colonel S. Menzies, the second in command of the SIS (Special Intelligence Service). [5] This machine had previously traveled from Warsaw to Paris by French diplomatic pouch. Langer himself, after a visit to Paris, had arrived in London several days before the machine and cemented more tightly relations with the British. [6]

The English were prompt to utilize the Polish gift. They had, for that, an organization and individuals which were to prove extraordinarily efficient.

B. THE GOVERNMENT CODE AND CIPHER SCHOOL

During the First World War, cryptanalytic work had be assigned, for specifically British reasons, to the Admiralty. The organization responsible for this work was known, both during and after the war, by the nondescript title of Room 40. Room 40 acquitted itself well in its tasks.

When England reorganized its intelligence services in 1922, cryptanalytic work (as well as the S.I.S.) was placed under the Ministry for Foreign Affairs. This guardianship was purely nominal; if the Foreign Office was among the consumers of the products of the services, it was perfectly incapable of telling them what they should produce or how.

The cryptanalytic service was modestly referred to as the Government Code and Cipher School, abbreviated GC&CS. In September of 1939, this service left London to move to its war headquarters which were about 80 kilometers northwest of the capital on a property known as Bletchley Park.

In 1938, the staff of GC&CS was numerically small but at the head of the organization were outstanding men. The director was Commander A. Denniston with the chief cryptologist being D. Knox. Both individuals were highly qualified specialists from the old Room 40 of the Admiralty.[7]

Denniston and Knox had grasped the necessity to cryptanalyse the Enigma machine along with other machines. They had, therefore, recruited a mathematical genius and Oxford professor, A. Turing, who was able to comprehend the entire gamut of possible electromechanical calculations (electronic means being not available at that time) and the practical methods of translating these visions into real and functioning cryptanalytic machinery. H. Keen, an engineer for the British Tabulating Machine Company (the English counterpart of the American IBM company), played the role that A. Palluth had played in Poland; i.e., he supervised the production of the machine which issued from the imagination of Turing. Turing had, perhaps, no need to know about the Polish work to conceive of his own version of a Bombe. This bombe, whose name was carried over from the Polish work, operated on principles different from those used by the Polish. These differences, as will be seen, made the British bombe far more powerful than its Polish namesake.

The beginning of the war coincided with the development of what came later on to be known as Bletchley Park. Among the new arrivals was another Cambridge mathematician, G. Welchman, whose role was to become considerable. From his mathematical acumen came proposals (immediately endorsed by Turing) of modifications to the first conceptions of Turing's bombe which greatly increased its power. Welchman also possessed managerial talents and laid some of the organizational foundations for the work at Bletchley Park.


C. FIRST WORK AT BLETCHLEY PARK

Examination of the Polish copy of the Enigma gave the British specialists a happy surprise. By making a few modifications to their TYPEX, this latter machine could be used to simulate the German military Enigma. [8] The Polish had had to set up an industrial production to obtain the AVA Enigmas; Bertrand had to order 40 Enigma replicas to be produced in France [9]; the English were instantly in possession of their own Enigma copies, and in quantity.

Blessed by this favorable occurrence, Bletchley Park began work in three directions:

1. Design (by a team of mathematicians led by another Cambridge Professor, J. F. R. Jeffreys) of new perforated sheets identical to those provided formerly by the Polish but taking into account the alterations introduced by the Germans at the end of 1938.
In other words, the new sheets, consisting of 60 different sets, permitted the mastery of the new rotor orders whose number had been increased from 6 to 60.
2. Research into new methods for finding the machine settings and message keys. These new methods should not be based on the use of the twice repeated message key. The British were of the opinion that, sooner or later, the Germans would realize their mistake and, consequently, would change their procedures. The English did not want such an occurrence to take them unprepared.
3. Development and construction of new Bombes. As has been already pointed out, these bombes were based on more general principles than those of the Polish and would be more effective. While the Polish bombes determined the compatible settings using the twice enciphered keys from a set of messages, the British bombes could determine the compatible settings from a single message using certain words, parts of a phrase, or even entire stereotyped phrases used by the Germans. These stereotypic phrases were called cribs by the English. The game for the English consisted of constructing a menu for their bombes using these cribs. If the menu had been appropriately prepared, the bombe would determine from the portion of the message corresponding to the crib, the compatible settings which would produce the ciphertext. At that point, as with the Polish, it was sufficient to test the possible settings on the Enigma replicas to determine which setting was correct and then to establish the key for each message. The decipherment could then proceed in the same way as it was done by the German addressees. [10]

As can be seen, the British bombes bypassed the use of the repeated key altogether (cf. 2 above).

D. THE DIFFICULT BEGINNING (September 1939-April 1940)

1. P. C. Bruno [11]

The French also had their cryptanalytic service. Under the direction of Bertrand, "P. C. Bruno" regrouped at the Château de Vignolles (some 50 kilometers northeast of Paris) about 50 persons. Work on the Enigma messages was essentially the purview of team Z which was composed of 15 Polish from the Biuro Szyfrow, who, commanded by Langer, were able to escape capture during the invasion of their homeland by taking refuge in Romania from whence they were quickly transferred to France. [12]

The English proposal for a Franco-English cryptanalytic center was rejected by the French. However, P. C. Bruno had a British liaison officer, Captain MacFarlan, who possessed a good knowledge of cryptology; Bruno was connected to Bletchley Park by teletypewriter. This teletype liaison was both close and continuous.[13]

2. First Results

While waiting for the first fruits of labor from Bletchley Park, P. C. Bruno and Bletchley, during the first months of the war, collected Enigma messages and identified the various groups of users.[14] At the end of December 1939, Bletchley Park had completed the perforated sheets. A set of these sheets was immediately sent to P. C. Bruno. According to all sources, it was there that Polish team Z, in the presence of the British expert who had brought the sheets, cracked the first daily setting for the German Enigma during the war. This event occurred on 17 January 1940.[15] The setting found was for 28 October 1939 and referred to the Green net which consisted of communications among the 20 German military regions. Immediately afterwards, Bletchley Park found, for the same net, the daily setting for 25 October.

This was not an auspicious beginning in view of the content of the messages (administrative material) nor the delay - nearly three months - which separated the interception from the decipherment. Things rapidly improved. Between 17 January and the end of March 1940, the method based on the perforated sheets allowed the solution of about 50 daily settings from three user nets which were named after the pencil colors which Bletchley Park used to distinguish among them.

i. Blue- Traffic consisting of test messages sent by Enigma operators being trained by the Luftwaffe.
ii. Green- Traffic between the 20 German military regions.
iii. Red- General traffic from the Luftwaffe including operational and liaison operations between the Air Force and the Army. This traffic was quite important. The first daily setting was determined was that of 6 January 1940.

On 10 April 1940, a new net of messages appeared. These were the Yellow messages which concerned operations in Norway. The daily setting and the message keys were broken daily beginning on 15 April and, thereafter, until the middle of May (at which date the traffic disappeared). Channels for forwarding the information extracted from the decrypts were still embryonic and, therefore, its utilisation by the Allied staff could only be limited. Practically no operational use was made of this intelligence material.

The Blue traffic was without interest and the Green rarely decipherable (probably because of lack of messages). Efforts were therefore concentrated on the Red traffic which was of operational interest and for which there were plentiful messages. At the end of April, the results obtained were encouraging.[16]

3. Bletchley Park and P. C. Bruno

Officially, the British (Hinsley, Vol I, p. 493) state that between January and the 23rd of June 1940, (the date at which, due to the French defeat, Bertrand's organization was dismantled) Paris and the GC&CS worked in close cooperation.

Unofficial English authors, notably Lewin, as well as Polish sources are severely critical of the atmosphere, the rhythm, and the work methods of P. C. Bruno; the Polish team's expertise does not appear to have been fully utilized. It is also clear that P. C. Bruno, having taken to the road on the 10th of June in order to flee from the advancing Germans, found itself in such a state of disorder that useful work simply could not be performed.

Bertrand's book gives an impressive number of decrypted messages (4,789) which appear to be corroborated by Langer's notebooks. However, Bertrand's figures may include non-Enigma decrypts, but they do not furnish any information on the importance of the Enigma decrypts or on the delay in decryptment. Moreover, it is not indicated whether the daily settings were discovered at Bletchley Park or at Vignolles. The only (unpublished) account on this subject is to be found in the memoranda of Colonel Mayer (Head of the Polish S.R.). Colonel Mayer indicates that a division of labor was established and that Bletchley carried out the greater part of the work by determining 83% of the 126 daily settings found up until 16 June 1940. Taking into account the existing circumstances, it appears likely that the preponderance of work was, indeed, done by Bletchley.


NOTES FOR CHAPTER V

1. Sources: Hinsley, British Intelligence in the Second World War, Vol 1, p. 488. Paillole, Notre espion chez Hitler, p. 39.
According to Paillole, the documents were given to the representative of the Intelligence Service in Paris, Commander Wilfred Dunderdale. He left for London on 23 November 1931 and returned on the 26th with the British refusal for a joint study.
2. Source: Hinsley. Vol 1, p. 491. It was most certainly the Asche documents.
3. During the Spanish Civil War, the British, beginning in 1937, deciphered German, Italian, and Spanish Nationalist messages enciphered on commercial models of the Enigma (slightly modified, but without the plugboard).
4. The Typex was notably superior to the Enigma in three important respects:
A. The Typex was more convenient to use. Whereas the Enigma only enciphered letters (necessitating that numbers and figures in words be spelled out) the Typex enciphered letters and numbers;
B. The Typex incorporated a printing mechanism (ultimately, the Typex could be used to produce perforated paper tapes for direct radio transmission);
C. The Typex offered greater security because the rotor movement was more irregular.
For further information of this subject see L. Kruh and C. A. Deavours:
i. The Typex Cryptograph, Cryptologia: April 1983, pp. 145-163.
ii. Machine Cryptography and Modern Cryptanalysis. Artech Books, 1984.
5. The death of Admiral H. Sinclair made Menzies the head of the SIS on 28 November 1939.
6. Source: Paillole, op. cit. p. 167.
7. Denniston had to leave his post for reasons of health in 1942 and was replaced by his deputy Travis. Knox, one of the best cryptologists of that day, died in the beginning of 1943. The high degree of professionalism of the heads of the English and Polish teams contrasts with the situation on the French side where Bertrand seems to have possessed only a limited cryptologic knowledge and was unable to obtain competent aid. This fact possibly explains the situation which developed from 1931 to 1939 between Bertrand and the Polish. The Polish accepted the documents provided by Bertrand but probably never discussed with him the technical aspects of the problems which they faced and progressively mastered. Until July 1939, Bertrand was ignorant of the successes which began in 1933. This led him to believe that the success had only just occurred.
8. Source: G. Welchman, The Hut Six Story, p. 75 (note).
9. The deliveries had not yet begun at the time of the Armistice of 25 June 1940.
After the Armistice, Bertrand, in the course of dangerous trips to the occupied zone, gathered the parts necessary for the completion of four Enigma copies which were used at P. C. "Cadix" situated in the free zone near Uzès. One of these machines survived and is now at the Sikorski Museum in London.
10. On the British Bombe, see: G. Welchman - The Hut Six Story, p. 77-83, 295-309.
L. Kruh and C. A. Deavours - Machine Cryptography and Modern Cryptanalysis.
While a technical description of the British bombe will not be furnished here (besides, the author of the present work would not be able to do so), it is important to mention that in the original conception of Turing, the British bombe utilized the existence of loops to solve the problems. Loops were closed chains of letters. Welchman showed that by modifying this attack (by the introduction of a diagonal board into the machine) that the loops were superfluous. This increased the power of the machine dramatically.
11. The history of P. C. Bruno in 1939-40 has yet to be written. From what Bertrand says in his book, as well as from the few facts given in the writings of Rejewski which have been translated into English, one cannot reconstruct the activities that took place or judge the results obtained. One paper by Langer written in 1940 at Algiers (after the Armistice) could not be consulted.
12. Against the wishes of the British who were keen to have the Polish specialists in England. Perhaps, also against the wishes of the Polish themselves who had seen at the meeting of 24-25 July at Pyry that the English experts were better qualified than the French ones. Moreover, the Polish seemed to have quickly noticed that better possibilities of work existed at Bletchley Park.
When the Polish mathematicians Rejewski, Rozycki, and Zygalski arrived in Bucharest on 18 September 1939, they went first to the British embassy. Politely refused, they then went to the French embassy where they received a most hospitable reception. The French took immediate measures to see that they were transferred along with their military superiors (who had been interned by the Rumanians at Calimanesti) to France where they arrived the 28-30 September. At this date, France was still the main military adversary of Germany. The Polish brought with them two Enigma copies. P. C. Bruno possessed, therefore, three Enigmas (the third was the one offered on 25 July by the Polish). One of these Enigmas was dismantled in order to facilitate the production of the French replicas of the machine.
13.In fact, beginning in November 1939, after a visit from Langer at Bletchley, the teleprinter circuit which carried the Enigma information was no longer under the supervision of MacFarlan. The English and Polish cryptographers exchanged messages on this teleprinter circuit. These messages were, most certainly, enciphered before transmission on the Polish copies of the Enigma!
14. Interception was, for the English, the jurisdiction of "Service Y" and for France, the Centres d'ècoute spècialisès. The problems associated with interception, the solutions which were found to these problems, and the efforts which were directed towards those solutions, would constitute a study in itself.
15. According to Polish sources, and, there is every reason to believe them. Curiously, the Bletchley archives seem to place this event at the end of December 1939. (See Hinsley, British Intelligence in the Second World War, Vol. 1, p. 493). According to Colonel Paillole, (op. cit. p. 80, and by a personal communication to the author) the Journal de Marche of Colonel Rivet, head of the 5ème Bureau SR-SCR mentions in an entry dated 3 January 1940 for the return from London of Captain MacFarlan, carrying new material for the decipherment of Enigma. This refers probably to the new perforated sheets. In order to explain the delay between the 3rd and the 17th of January, it may be supposed that, if Captain MacFarlan transported the heavy and voluminous package represented by the perforated sheets, effective utilization of the sheets could not have taken place until after the arrival at Bruno of the Bletchley specialist (Knox or Turing himself).
16. A table established by G. Welchman (From Polish Bomba to British Bombe, p. 105) on the basis of information appearing in the notebooks of Langer allows one to follow the increasing speed of the decipherments. For the Enigma intercepts of January 1940, 11 settings (for 10 different days) were determined with delays ranging from a maximum of 43 days to a minimum of 5 days (average delay, 21 days). For the February messages (13 settings found) the average delay was 13 days. This delay was reduced to 8 days during March (24 settings found). By April (25 settings found) the greater part of the decipherments were realized with a one day delay.
The Journal de Marche of Colonel Rivet mentions that, from 6 March 1940, the tactical exploitation of the decipherments appeared possible (Paillole, op. cit. p. 181).

BIOGRAPHICAL SKETCHES

Gilbert Bloch is retired and lives in Paris with cryptology and parachute jumping as hobbies. He is a member of the French Military Reserves. C. A. Deavours teaches courses in cryptology (both technical and historical issues) at Kean College of New Jersey.


The Turing Bombe: Was It Enough?

C. A. Deavours1 and Louis Kruh2

ADDRESS: (1) Department of Mathematics, Kean College of New Jersey, Union NJ 07083 USA and (2) 17 Alfred Road West, Merrick NY 11566 USA.

ABSTRACT: The effectiveness of Alan Turing's original bombe design is discussed in this article along with organizational methods of implementing it via a microcomputer. To illustrate the material presented, a three part German Werhmacht message is solved.

KEYWORDS: Turing bombe, Enigma, cryptanalysis.

It is well known that Polish methods of breaking the German Enigma cipher were based on the use of a twice enciphered rotor starting position. There were two such methods used by the Germans before May 1940 and the Poles overcame both methods in order to solve the Enigma ciphers. Beginning in 1939 when the Poles turned over the results of their work to the British, some method was sought which would not depend on the fact that the Germans used a twice enciphered key for each message since this keying method was likely to be changed in time (as it was during May 1940).

There were two such methods examined by the British. The first method was based on stepping through all rotor positions and computing, at each position, certain statistics which would indicate the correct cylinder positions had been attained. This method was a ciphertext only cryptographic attack and will not be discussed here. This first method was, apparently, never put into use, probably because of German foresight in keeping the individual message lengths to about 200 characters (the maximum allowed length varied from time to time).

The second method, and the one used by the British, was based on cribs (probable plaintext). In this second method, a crib of of suitable length was assumed to be in a certain position of a message and a device, the bombe, stepped through rotor positions trying to determine at each position whether the assumed plain/cipher correspondence was possible. Bletchley Park, where most of this codebreaking was done, had a separate section devoted entirely to obtaining cribs based on close study of previously deciphered texts. A fuller discussion of how the British bombe worked may be found in [1] and [2].

It was the Enigma plugboard that caused most of the difficulties in solution and the bombe was the machine that largely overcame these difficulties with the aid of available cribs.

The bombe as conceived by the British was the product of two ingenious individuals, Alan Turing and Gordon Welchman. Originally, the bombe was to be built according to principles devised by Turing but, reportedly, before the first bombe had been built, Welchman had an idea for an addition to the device which later came to be called the diagonal board. This board greatly augmented the power of the bombe. It is the purpose of this paper to inquire as to just how well the British would have done without Welchman's idea. To this end, a number of simulations have been prepared and used. These programs will be referred to in the following discussion and are discussed in the Appendix to this paper.

There are several reasons for interest in this subject. First, Turing claimed that his original idea for the bombe design came from Polish work. This cannot be wholly true since the Poles, who were themselves very able cryptanalysts, never seem to have come across the same idea. The author once discussed this issue at length with Mr. Welchman. The conclusion of that conversation was that Turing must have meant that the essential trick involved in the bombe, the use of plugboard invariance, was suggested to him by Polish work. At any rate, it would seem that someone other than Turing might have come up with his idea. This is not so of the diagonal board. How this board works is by no means obvious and even Turing, when first confronted by Welchman with the idea, took some time before he saw the wisdom in its use. It is unlikely that anyone other than Welchman would have come upon the concept of using a diagonal board.

Second, the British bombe relied heavily on the fact that the Enigma plugboard substitution was reciprocal, i.e., if E is mapped into Y then Y is mapped into E. The Germans were introducing an Enigma attachment called the "ühr box" towards the end of the war. The effect of this gadget was to make the plugboard encipherment non-reciprocal which would have, in turn, rendered the bombes ineffective. The British would most likely have returned to Turing's original conception which was based on loops and did not require, in its simplest form, the reciprocity of the plugboard.

Thirdly, the British TYPEX cryptograph, which was a sister to the German Enigma, used five rotors plus a reflector. The two entrance rotors did not move during encipherment and thus formed a sort of non-reciprocal plugboard. Could the Germans have solved this machine had they discovered Turing's idea?

To see how Turing's solution method worked, let us first look at a message enciphered on an Enigma with no plugboard. Suppose we have the following message beginning with suspected crib.

                                   1  1  1  1  1  1  1  1  1  1
     0  1  2  3  4  5  6  7  8  9  0  1  2  3  4  5  6  7  8  9
     F  O  R  F  I  R  S  T  D  I  V  I  S  I  O  N  S  T  O  P
     Q  L  L  K  Q  G  D  E  M  D  R  G  I  V  A  S  F  G  L  T.

From this piece of text, we next draw up a menu. The menu consists of loops found in the above plaintext/ciphertext (pl/cr) pairs. In this case, the loops are

              O——>L——>O            1   18
              S——>D——>I——>S        6    9   12
              F——>S——>I——>Q——>F   16   12    4   0
              R——>V——>I——>G——>R   10   13   11   5

The first loop tells us that if we encipher O at position 1, we will get the letter L, which, when enciphered at position 18 gives 0 back again and thus closes the loop. The other loops are interpreted in a similar manner. Clearly a loop can start with any letter in its cycle. For our purposes however we only count a given loop once and ignore its rotations since these provide nothing new to us of computational value in this case.

If U represents the substitution produced by the machine at position 1, and V is the substitution produced by the machine at position 18, then we have

(o)U = l (l)V = o or, (o)UV = o.


That is, the letter "o" is a fixed point of the rotor path UV. Similarly, "1" is a fixed point for VU

Now, if we try every one of the 263 rotor positions for position 0, we can test for the presence of the four loops shown above. A positive result would indicate that we, most likely, have found the correct starting position for the coding cylinders. We do this calculation for every rotor order (e.g., 6 orders with 3 rotors). Actually, only 3 loops should be sufficient to exclude nearly all false alarms for a given rotor order. This is because the chance of a given letter enciphering to the same letter over any chosen path is about 1/26. Therefore the chance that an incorrect position will have three prescribed loops is about 1/263 so that the expected number of false alarms in the trial of one rotor order is about

263(1/263) = 1.

There is, however, an important problem. We know that the fast rotor moves once for every character enciphered. Further, except for an infrequent anomaly when the medium and slow rotors move together, the middle or medium rotor moves once for every 26 letters enciphered. We have no way of knowing if or where the second rotor moved during the encipherment of the crib text. If, for instance, we decide that the middle rotor did not move at all during the crib encipherment then, taking pair 0 as cylinder position 0,0,0, then the position menu for the four loops given above would be (recall that the rotor order in machine is SLOW MEDIUM FAST):

1 18: | 0,0,1 | 0,0,18 |  |
6 9 12: | 0,0,6 | 0,0,9 | 0,0,12
16 12 4 0: | 0,0,16 | 0,0,12 | 0,0,4 | 0,0,0
10 13 11 5: | 0,0,10 | 0,0,13 | 0,0,11 | 0,0,5

Suppose now that the medium rotor stepped between pl/cr pairs 11 and 12 (as it did). This means that at positions 12, 13, and 18 the medium rotor now occurs at a new position. In this case, the relevant positions in the above menu should be altered to 0,1,12 0,1,13 and 0,1,18 respectively. In the absence of further information, a number of runs through each rotor order might be necessary in order to determine the answer. Since the crib is 20 characters long, one might assume that 20 possible stepping positions for the medium rotor must be tried. This is not so since some stepping positions do not affect the loops because these positions are not used in forming them.

For the situation above, we see below assumed stepping positions which are equivalent for this set of loops. Stepping position means that the medium rotor steps between the indicated position and the next position.

EQUIVALENT STEPPING POSITIONS:
0 | 1 | 4 | 5 | 6 | 7 | 9 | 10 | 11 | 12 | 13 | 14 | 18
 | 2 |  |  |  | 8 |  |  |  |  |  | 15 | 19
 | 3 |  |  |  |  |  |  |  |  |  | 16
 |  |  |  |  |  |  |  |  |  |  | 17

Obviously, if all of these positions must be tried, those having the longest span (e.g., 14 15 16 17) should be tried first to save time.

Suppose now that an exhaustive search of rotor orders and positions has revealed that the required loops do exist when position 0,0,0 corresponds to coding cylinder positions JIV and the stepping position of the medium rotor was between pl/cr pairs 11 and 12. The solution is completed by determining the alphabet ring settings which will make the medium rotor step at the required location as well as the (adjusted) rotor starting positions.

              A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
              ----------------------------------------------------
            Z: Z Y X W V U T S R Q P O N M L K J I H G F E D C B A
            A: A Z Y X W V U T S R W P O N M L K J I H G F E D C B
            B: B A Z Y X W V U T S R Q P O N M L K J I H G F E D C
            C: C B A Z Y X W V U T S R Q P O N M L K J I H G F E D
            D: D C B A Z Y X W V U T S R Q P O N M L K J I H G F E
            E: E D C B A Z Y X W V U T S R Q P O N M L K J I H G F
            F: F E D C B A Z Y X W V U T S R Q P O N M L K J I H G
            G: G F E D C B A Z Y X W V U T S R Q P O N M L K J I H
            H: H G F E D C B A Z Y X W V U T S R Q P O N M L K J I
            I: I H G F E D C B A Z Y X W V U T S R Q P O N M L K J
            J: J I H G F E D C B A Z Y X W V U T S R Q P O N M L K
            K: K J I H G F E D C B A Z Y X W V U T S R Q P O N M L
            L: L K J I H G F E D C B A Z Y X W V U T S R Q P O N M
            M: M L K J I H G F E D C B A Z Y X W V U T S R Q P O N
            N: N M L K J I H G F E D C B A Z Y X W V U T S R Q P O
            O: O N M L K J I H G F E D C B A Z Y X W V U T S R Q P
            P: P O N M L K J I H G F E D C B A Z Y X W V U T S R Q
            Q: Q P O N M L K J I H G F E D C B A Z Y X W V U T S R
            R: R Q P O N M L K J I H G F E D C B A Z Y X W V U T S
            S: S R Q P O N M L K J I H G F E D C B A Z Y X W V U T
            T: T S R Q P O N M L K J I H G F E D C B A Z Y X W V U
            U: U T S R Q P O N M L K J I H G F E D C B A Z Y X W V
            V: V U T S R Q P O N M L K J I H G F E D C B A Z Y X W
            W: W V U T S R Q P O N M L K J I H G F E D C B A Z Y X
            X: X W V U T S R Q P O N M L K J I H G F E D C B A Z Y
            Y: Y X W V U T S R Q P O N M L K J I H G F E D C B A Z
                               Beaufort Tableau.
ALPHABET RING SETTING TABLE
LEFTMOST COLUMN = LETTER READ FROM WINDOW
TOP ROW = CODING CYLINDER POSITION
INTERSECTION OF ROW AND COLUMN = ALPHABET RING SETTING

Enigma rotors 1, 2, and 3, when used in the fast rotor position, step the medium rotor when their alphabet rings reach locations Q, E, and V, respectively in the setting window of the machine (see [2]). From the above data, we conclude the following concerning the positions of the coding cylinders and alphabet ring settings in the crib. (The coding cylinder location is known from the rotor search while the ring positions are known from the stepping position of the medium rotor.)

                                                   1 1 1 1 1 1 1 1 1 1
                               0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
                         (1) J K L M N O P Q R S T U V W X Y Z A B C D
                         (2) U V W X Y Z A B C D E F G H I J K L M N O

Here, (1) is FAST ROTOR RING and (2) is FAST CYLINDER LOC.


Thus, the starting position of the fast rotor (as read from the viewing window of the machine) was J. At this time, the alphabet ring of this rotor was set so that the coding cylinder location was U. What is the ring setting?

The ring setting can be determined by consulting the ordinary Beaufort tableau.

We find the ring setting of the fast rotor to be P. The ring position for the medium rotor cannot be determined until the slow rotor moves. Hence, we should take as our solution at this point:

ROTOR ORDER = 1,2,3 RING SETTING = AAP ROTOR START = JIJ.

In the case of the plugboardless Enigma, the above method would not be the best one. First, a hand method, the baton method is available if we have the proper lookup tables. Second, if we had a machine capable of running through all of the rotor positions it would be much better to use a consecutive 3 or 4 letter crib in the solution in order to minimize the possibility of the medium rotor having stepped. For example, we might search for consecutive rotor positions such that the correspondence

PLAIN: | F | O | R
CIPHER: | Q | L | L

holds. This point deserves attention because, after World Wax II, the bombes were used on non-plugboard models of the Enigma in the manner above.

If the Poles had thought of the loop principle they could have built a bombe, which differed only slightly from their bomba, that would have been capable of solving Enigma ciphers based on cribs as long as not all letters were involved in the plugboard substitution. In order to see how this would have worked, consider the following sample crib:

                   O B E R K O M M A N D O D E R W E H R M A C H T
                   I F I X M W W X W V L T Z S L C L P E P J X K C

The relationship between the letters in the crib and the rotor positions is shown in Figure 1.

* Fig.
* Fig. Figure 1. The relationship between the letters in the crib and the rotor positions.

The following loops are present.

                                 (WCXM)
                                 (ERL)
                                 (TOWC)
                                 (KMPH)
                                 (IERXCWO)

Now, if the initial letter of a loop is unchanged by the plugboard, the loop still exists regardless of the effect of the plugboard on any intermediate letters in the loop. In order to see this fact, consider the loop (ERL). Suppose that U, V, and W are the three rotor positions involved and P represents the plugboard substitution. We have, noting that P = P-1, U = U-1, etc.,

 | (e)PU P-1 = r (r)PV P-1 = l (l)PW P-1 = e
or
 | (e)PU P-1PV P-1PW P-1 = (e)PUVW P-1 = e.

This last expression may be further written

(e)PUVW = (e)P or, (e)UVW = e

since (e)P = e if this letter is unchanged by the plugboard.

Amazingly enough, all plugboard references are missing from the final equation provided that the initial loop letter is unchanged by the plugboard.

This observation suggests a very simple form of bombe. Referring to Figure 1, we see that many points belong to two or more loops. We could, for instance, use as menus the following loops, all all of which have been rearranged to have the same initial letter.

                           (WCXM)      (TCWO)
                           (WCTO)      (TCXMWO)
                           (WOIERXC)   (TOIERXC)
                           _________   _________
                           (CXMW)      (XMWC)
                           (CTOW)      (XCWOIER)
                           (CWOIERX)   (XCTOWM)
                           _________   _________

If a letter such as W, C, T, or X is unchanged by the plugboard, then, we should find the correct cylinder setting and rotor order. Since there are three loops, there will be few, if any, false alarms. We now compare the solutions for position 0,0,0 looking for agreement between the different menus. Only two letters need be unchanged by the plugboard in order to reveal the correct solution. The fact that this form of the bombe is virtually identical to the ones which had already been built by the Poles is the most convincing evidence available that the Turing solution, based on loops, had not been thought of previously.

One can see from looking at Figure 1 that some letters do not fall into loops (e.g., Z, D, L, B, F,. . . ). By the term Turing bombe we shall mean any solution procedure which uses only letters which are found in the crib loops. The diagonal board provided by Welchman permitted the use of all letters, whether in loops or not. Returning to the question of the Turing bombe, we may ask what modifications are needed when the plugboard is taken into consideration? All of the previous discussion regarding the canceling out of the plugboard substitutions in a loop goes through without change. The only difference is that the initial letter of a loop may have been effected by the plugboard. Thus, in the previous example, our final equation for the ERL loop would read

(e)PUVW = (e)P.

No matter what the plugboard substitution is, the effect of P on any letter can only be to produce another letter. There are only 26 possibilities for (e)P, for it can be only one of the letters A,B,C,. . . Z. All of these possibilities can be tested simultaneously by applying input to all 26 rotor input contacts and using double ended rotor sets where the reverse path of the current does not pass along the same rotor input paths. The current then flows through the resulting maze determined by the menu and indicates whether or not the rotor setup is correct or not. Once again, the reader is referred to [1] and [2] for further description.


For a loop like the one shown above, we can get candidates for the plugboard mapping by computing all of the fixed points, i.e., the solutions of

(x)UVW = x.

Any such solution gives a possible plugboard mapping via

x = (e)P, or, equivalently e = (x)P.

The program TBOMBE.BAS (TBOMBE.EXE) emulates the solution process as it would have been carried out by the Turing bombe. First, the crib is examined and all loops are written out. Rotations of the same loop and loops derived from previously listed loops are deleted from the list. Next, the loops are rearranged to provide as many loops as possible which begin with the same letter. This arrangement greatly reduces the number of false alarms that the program finds. We shall call the loops arranged as indicated above primary loops. A secondary loop is any rotation of a primary loop.

To prepare the menu for a given rotor order, the program LOOPFIND.BAS is now run and the primary loops and their rotor positions relative to 0,0,0 are entered. The program will compute the secondary loops and add these to the menu and store the resultant menu on the disk. Note that something about the stepping position of the rotors must be assumed here in entering the rotor positions in the menu.

The main program TBOMBE.BAS (TBOME.EXE) is now run and the rotor order is entered while the menu is read from the disk. The solution process uses the following logic:

A. At each rotor position all fixed points for each loop are computed one at a time. If any loop fails to have at least one fixed point, then, no such loop exists and this rotor positioning is discarded.
B. If a loop has one or more fixed points, then the program tries to eliminate them using data from the other loops. If the elimination is successful, then the rotor position can be discarded.
C. If the elimination process fails, the cylinder positions and deduced plugboard connections are printed and the program proceeds to the next rotor position.

As an example, suppose the loops (and derived secondary loops) are as shown.

PRIMARY LOOPS:

 | (EV)
 | (EM)
 | (XAU)
SECONDARY LOOPS (ROTATIONS OF PRIMARY LOOPS):
 | (VE)
 | (ME)
 | (AUX)
 | (UXA).

Computing the fixed points of the (EV) loop we might find two such points, say, F and U. In terms of the plugboard mapping, we may have either f = (e)P or u = (e)P. We now try to eliminate these two fixed points so as to rule out this rotor position. The other loops all either begin with the same initial letter, E, a fixed point letter, F or U, or some other letter.

If another loop begins with the same letter, such as loop 2, then we try the fixed points for the first loop at this position. On the other hand, if another loop begins with a fixed point letter, such as the 7th loop which begins with U, then we can test the assumption e = (f)P, which is the same as f = (e)P, by trying the letter E as a fixed point at this position. The letter U would be tested in a similar way. Loops that begin with neither letter are of no use in this testing process.

If we fail to rule out a loop, we move to the next loop and try again. If no loop can be ruled out then the surviving set of fixed points, which represents possible plugboard connections is printed.

We note that in the testing of the loops which begin with a fixed point of another loop, the reciprocity of the patchpanel is assumed. Thus, the Turing loop method does not preclude the use of this property of the plugboard as has been commonly assumed. In practice, however, this additional testing does not result in a greatly increased computational ability because the number of letters in the loops as well as the number of loops is usually small unless the crib is unusually long.

To illustrate the entire process as well as the program outputs, we will solve an actual set of three Enigma messages. These are seen in an actual Enigma message in Figure 2. These messages were kindly provided by David Kahn.

* Fig.
* Fig. Figure 2. Hardcopy of the three Enigma messages sent to HEERSGRUPPENKOMMANDO 2 on 21 September 1939.

The messages in question are shown as they were received. In actuality we have one long message which has been divided into three parts. These messages were sent on 21 September 1939 to HEERSGRUPPENKOMMANDO 2. At this time only three Enigma rotors were in use and the 9 letter message key system in which the sender selected a groundsetting with which to encipher the rotor starting positions (twice) was the current message keying system.

The message preamble is

2109 - 1750 - 3 TLE - FRX FRX - 1 TL - 172

The first two groups are the date and time (21 SEPT, 17:50 HRS). The next groups indicate that the message is in 3 parts. The groundsetting, FRX, is then repeated in the clear. Evidently, the same groundsetting has been used for all three parts. Finally, comes the indication that this is the first part of the message which is 172 letters in length. The term FERNSCHREIBEN indicates that these messages were received over land lines, i.e., teletype.

Fortunately, whoever deciphered the messages has left us some useful notations on the sheet. The first 6 letters of each message have been ruled out. These are the twice enciphered rotor starting positions. The third group of each message has also been ruled out. This is the indicator group for each message. A number of indicator groups were generally provided for each key schedule in order to make the process of sorting the cryptograms more difficult for interceptors.

The numbers 06 18 24 shown on the right border are the groundsetting FRX converted into numbers. The decipherer's machine had numbers, 01 - 26 on the rotor circumference instead of the letters A - Z. Inside the hood of the Enigma machine was a chart used to make this conversion, A=01, B=02, etc.

Best of all, the deciphered rotor starting positions have been written for each each message on the left side of the paper. For part 1, these are AGI (01 07 09). One can see where the clerk has checked off the letters during the second decipherment of the key. This fact will save us some computation since we will know the turnover position for the middle rotor without having to guess it.


We now need a crib. This was provided by the fact that the plaintext of the message was also available. After some consideration the first 28 letters of the first message were selected for use. Having the plaintext does not make the solution process automatic since abbreviations, the use of X's as word and sentence dividers, and other complications often arise.

The crib is shown.

     (1) I J K L M N O P Q R S T U V W X Y Z A B C D E F G H I J K
                               1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0
           0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7
     (2)   A U F B E F E H L D E S O B E R S T E N B E F E H L S H
     (3)   Q K R Q W U Q T Z K F X Z O M J F O Y R H Y Z W V B X Y

Here row (1) is FIRST ROTOR, row (2) is PLAIN, and row (3) is CIPHER.

The top line indicates the fast rotor position using a starting position of I. At this point, the menu loops are sought.

MENU LOOPS:

* Fig.

To prepare the bombe menu, the program BOMBMENU.BAS is run and the primary loop data is entered. The output from this program is on the following pages.

MENU GENERATOR:

ENTER NUMBER OF PRIMARY LOOPS TO BE ENTERED? 5
ENTER LENGTH OF LOOP 1: ? 2
ENTER LENGTH OF LOOP 2: ? 2
ENTER LENGTH OF LOOP 3: ? 2
ENTER LENGTH OF LOOP 4: ? 4
ENTER LENGTH OF LOOP 5: ? 5
NUMBER OF PRIMARY LOOPS = 5
NUMBER OF SECONDARY LOOPS = 10
ENTER LETTER CYCLE FOR LOOP # 1 :? EW
ENTER POSITION # 1 ? 0,0,4
ENTER POSITION # 2 ? 0,1,23
ENTER LETTER CYCLE FOR LOOP # 2 :? EY
ENTER POSITION # 1 ? 0,1,18
ENTER POSITION # 2 ? 0,1,21
ENTER LETTER CYCLE FOR LOOP # 3 :? SX
ENTER POSITION # 1 ? 0,0,11
ENTER POSITION # 2 ? 0,1,0
ENTER LETTER CYCLE FOR LOOP # 4 :? HTOB
ENTER POSITION # 1 ? 0,0,7
ENTER POSITION # 2 ? 0,1,17
ENTER POSITION # 3 ? 0,1,13
ENTER POSITION # 4 ? 0,1,20
ENTER LETTER CYCLE FOR LOOP # 5 :? HBQEY
ENTER POSITION # 1 ? 0,1,20
ENTER POSITION # 2 ? 0,0,3
ENTER POSITION # 3 ? 0,0,6
ENTER POSITION # 4 ? 0,1,18
ENTER POSITION # 5 ? 0,1,1
BOMBE MENU
PRIMARY LOOPS:
1 | E | 2 | 0 | 0 | 4 | 0 | 1 | 23 |  |  |  |  |  |  |  |  |
2 | E | 2 | 0 | 1 | 18 | 0 | 1 | 21 |  |  |  |  |  |  |  |  |
3 | S | 2 | 0 | 0 | 11 | 0 | 1 | 0 |  |  |  |  |  |  |  |  |
4 | H | 4 | 0 | 0 | 7 | 0 | 1 | 17 | 0 | 1 | 13 | 0 | 1 | 20
5 | H | 5 | 0 | 1 | 20 | 0 | 0 | 3 | 0 | 0 | 6 | 0 | 1 | 18 | 0 | 1 | 1
SECONDARY LOOPS:
6 | W | 2 | 0 | 1 | 23 | 0 | 0 | 4 |  |  |  |  |  |  |  |  |
7 | Y | 2 | 0 | 1 | 21 | 0 | 1 | 18
8 | X | 2 | 0 | 1 | 0 | 0 | 0 | 11
9 | T | 4 | 0 | 1 | 17 | 0 | 1 | 13 | 0 | 1 | 20 | 0 | 0 | 7
10 | O | 4 | 0 | 1 | 13 | 0 | 1 | 20 | 0 | 0 | 7 | 0 | 1 | 17
11 | B | 4 | 0 | 1 | 20 | 0 | 0 | 7 | 0 | 1 | 17 | 0 | 1 | 13
12 | B | 5 | 0 | 0 | 3 | 0 | 0 | 6 | 0 | 1 | 18 | 0 | 1 | 1 | 0 | 1 | 20
13 | Q | 5 | 0 | 0 | 6 | 0 | 1 | 18 | 0 | 1 | 1 | 0 | 1 | 20 | 0 | 0 | 3
14 | E | 5 | 0 | 1 | 18 | 0 | 1 | 1 | 0 | 1 | 20 | 0 | 0 | 3 | 0 | 0 | 6
15 | Y | 5 | 0 | 1 | 1 | 0 | 1 | 20 | 0 | 0 | 3 | 0 | 0 | 6 | 0 | 1 | 18


The menu is now stored on the disk. Three separate menus must be prepared (depending on the stepping position of the fast rotor). We will need menus for rotor orders 123 and 213 (stepping position V), orders 132 and 312 (position E), and orders 321 and 231 (position Q). The menu shown above is prepared for orders 123 and 213.

We can run the main program, TBOMBE.BAS (TBOMBE.EXE). After specifying the rotor order and name of the bombe menu which is to be loaded from the disk the solution process commences. When rotor order 213 was tested the following output was printed.

CYLINDER POSITION=BKG
E | E | H | H | S | W | Y | T | O | B | B | Q | E | Y | X
Z | Z | H | H | O | R | Y | C | D | L | L | Q | Z | Y | P
 |  |  |  | P | Z |  | P | J |  |  |  |  |  |
 |  |  |  | S |  |  | T | P |  |  |  |  |  |
 |  |  |  |  |  |  | Z | S |  |  |  |  |  |

This program output shows the coding cylinder locations and the final fixed point data. We see that the letters H, Y, and Q are unchanged by the plugboard. Probably S and T are also unchanged. The plug connections which have been found so far are (EZ), (BL), and (XP). The connection (WR) can also be inferred since W must be mapped to either R or Z but the letter E is already mapped to Z.

Thusfar, we know the following key data:

ROTOR ORDER = 2,1,3.
PLUGBOARD: (EZ) (BL) (XP) (WR)
CODING CYLINDER STARTING LOCATIONS = BKF
ROTOR STARTING POSITIONS = AGI (from message sheet).

Using the Beaufort table, the ringsettings are quickly found to be ZWD.

We now have only to work out the remaining plug settings (if any). Using the above data and deciphering the cryptogram using the program ENIGMA.BAS we have

PLAIN: A I F B E R E H L D E S J U R G S W E N B E F E Q L S H

The decipherment is correct at 19 of the 28 positions. It is now comparatively easy to work out the remaining plug settings which are found to be (IU), (VM) and (JO).

The rotor starting positions for the three parts are found from deciphering the enciphered rotor positions to be AGI, YBE, and LUN, respectively. The messages can now be deciphered.

DECIPHERMENT (WITH GARBLES):

PART I:

       AUF BEFELD DES OBERSTEN BEFEHLSHABERS SIND IM
       FALLE XZXZTX UNWAHRSCHEINLICHEN FRANZOESISQEN
       ANGRIFFS DIE WESTBEFESTIGUNGEN JEDERZAHLENMAESSGEN
       UEBERLEGENHEIT ZUM TROTZ ZU HALTEN X

PART II:

       FUEHRUNG UND TRUPPE MESSEN VON DIESER EHRENFPLIQT
       DURQDRUNGEN SEIN X ABS X DEMGEMAESS BEHALT IQ MIR
       DIE ERMAEQTIGUN ODER AUQ OON TEILEN AUSDRUECKLIQ

PART III:

       PERSONLIQ VOR X ABS X AENDERUNG DERUNGGERANWEISUNG
       X OKH X GEN X ST X D X HXERSTE ABT X NR X DREI
       DREI ZWO EINS DREI AQT G X KDOS X VOM JULI EIN X
       WEUNDREI AQT BLEIBT VORBEHRLTEN X DER
       OBERBEFEHLSHABER DES HEERES

The following conventions are to be noted.

X = PERIOD OR WORD DIVIDER
ABS = NEW PARAGRAPH
AQT = /
XZXZTX = ? MAP COORDINATES?
Q = CH.

The erratic use the X's, which has been noted in [3], must have been a tremendous problem in establishing cribs.

If Turing's loop method had been the sole principle used by the British bombes, could penetration of the Enigma ciphers have been successful? The answer rests mostly on the availability of cribs. Welchman states in [1] that the British bombes contained 10, later 12, rotor sets. This indicates that the cribs used by the British were about that length. This figure is disputed by persons who worked on and serviced the bombes, who recall 25-30 sets of rotors per machine. (A drawing in [4] shows a bombe with two rows of 12 rotor sets each.) In [2], it is revealed that bombe personnel were given the following table for false alarms.

BOMBE SETUPS NECESSARY FOR ONE STOP
PER ROTOR ORDER
NUMBER OF LOOPS | NUMBER OF NON-LOOP LETTERS
4 | 0
3 | 9
2 | 11
1 | 12
0 | 13


With a 12 letter crib, most loops are 2, 3, or perhaps 4 letters in length. Thus, the above table indicates that the cribs were about 12-18 letters in length. We also know from Welchman that Turing was basing his machine on the fact that 3 loops could be found in a crib. Turing must have known from available data that this was a practical possibility. The table below shows an experiment in which 20 cribs of lengths 12, 15, 18, and 21 letters were tested and the numbers of loops found were recorded.

LENGTH OF CRIB
# LOOPS | 12 | 15 | 18 | 21
0 | 13 | 9 | 5 | 0
1 | 6 | 8 | 6 | 1
2 | 1 | 3 | 6 | 9
3 |  |  | 3 | 6
4 |  |  |  | 4
AV. # LOOPS | .40 | .70 | 1.35 | 2.65

Unless a very large number of cribs are available it would seem that about 20 or so letters would be needed to have a good chance of securing the three loop situation. Further, since bombe operators were provided with a "4 loop" figure it appears that this case could occur, at least occasionally. This suggests once again that cribs of 20 or so letters must have been available.

Turing based his need for at least 3 loops on the following reasoning. Without the plugboard 3 loops are sufficient to produce only about 1 false alarm per rotor order. Because of the plugboard 26 more possibilities need to be investigated for the initial loop letter. Therefore 3 loops could produce about 26 false alarms per rotor order. This figure could be coped with but the actual situation was usually not so bad. A frequently occurring letter in a crib was quite likely to be in more than one loop. When this happens, the number of false alarms falls off. If, for instance, the first loop turns out to have 1 fixed point then this is the only letter which has to be tested in a second loop which begins with the same letter - the other 25 possibilities having already been ruled out.

From the above considerations, it would appear that, although the loop method requires a somewhat longer crib than if non-loop letters can be used, the more lengthy cribs appear to have been obtainable.

A more serious problem is the stepping position of the medium rotor. We do not know how this problem was handled. Even if short 12 letter cribs were used, there is a good chance (12/26) that the middle rotor will step somewhere in the crib. This would mean rerunning the test using different assumed stepping positions. Such a requirement could multiply greatly the time required to find the solution. Whether or not the diagonal board is used in the bombe, this obstacle is major. It would seem that, in many cases, the stepping position must have been known. One can conceive of how this might have happened from reading about the operator carelessness that was so prevalent, but, until more is known, we cannot say for sure. In any case, both methods would suffer from this problem.

In conclusion, it would seem that breaks in the Enigma system could have been made even if Welchman's diagonal board had not been conceived. The solutions could not have been accomplished with the same degree of work, but, the additional work necessary and the crib data required for the solution appear to have been available to the best of our current knowledge.

PROGRAM DESCRIPTION

The programs used in this article are described below. These programs are available from the author on an MS-DOS diskette. Most programs are in BASIC but machine language versions have been supplied when the execution time of the program was considered too slow. Both the programs and brief instructions and problems can be obtained from the author at the address given in the front of this journal. The price is $10 and covers the costs of preparation, shipping, and handling.

ENIGMA.BAS
- Simulation of the 3 rotor Wehrmacht Enigma.
BOMBE.BAS (BOMBE.EXE) -
- This program runs through all rotor positions trying to match preassigned data. It is intended primarily for solution of plugboardless Enigma messages. The compiled version of the program, BOMBE.EXE, examines a rotor order in 30-45 seconds (4 MHZ).
LOOPFIND.BAS
- Plaintext and matching ciphertext are entered and a list of primary loops is generated.
BOMBMENU.BAS
- Primary loops and positions are entered and secondary loops are generated and stored on the disk. This program is used to prepare menus for the TBOMBE program.
TBOMBE.BAS (TBOMBE.EXE)
- Simulation of the Turing bombe. TBOMBE.EXE requires 7-12 minutes to examine one rotor order.

REFERENCES

1. Welchman, Gordon. 1982. The Hut Six Story. New York: McGraw-Hill.
2. Deavours, Cipher A. and Louis Kruh. 1985. Machine Cryptography and Modern Cryptanalysis. Norwood MA: Artech House.
3. Calvocoressi, Peter. 1980. Top Secret Ultra. New York: Pantheon Books.
4. Johnson, Brian. 1978. The Secret War. London: Methuen.

BIOGRAPHICAL SKETCHES

C. A. Deavours teaches cryptography courses at Kean College of New Jersey. He also does extensive consulting in the field of Electronic Funds Transfers. His wife says that he has mastered the art of speaking in ciphertext and he continues a full schedule of teaching, consulting, and writing about cryptology at Kean College of New Jersey.

Louis Kruh is an indefatigable researcher and collector of cryptologic memorabilia, particularly cipher devices. His interest in cryptology spans more than 40 years and he responds knowledgeably and promptly to all queries regarding literature and equipment. He is co-author with Dr. C. A. Deavours, of Machine Cryptography and Modern Cryptanalysis, (Artech House, Inc. 1985). He is recently retired from his position as Director of Advertising for New York Telephone.


An Enigma Chronology

David Kahn

ADDRESS: 120 Wooleys Lane, Great Neck NY 11023 USA.

ABSTRACT: A list of important events in the making and breaking of the German Enigma cipher machine of World War II is given in the order of their occurrences.

KEYWORDS: Enigma, chronology.

In view of the importance of ULTRA to the history of intelligence, and in view of the errors in dating that appear in so many accounts, I thought it would be worthwhile to establish a list of important dates in the story of the Enigma cipher machine and its solution. Most deal with the naval machine.

This chronology comes mainly from my book, Seizing the Enigma (New York: Houghton Mifflin, 1991), which is based as much as possible on primary or on solid secondary sources, chiefly F. H. Hinsley et al's magisterial History of British Intelligence in the Second World War (Cambridge: Cambridge University Press, 1979-1988). For my book, the Naval Historical Branch in London made available to me a key primary source for much naval dating, but I promised not to cite it by name, a promise that I am honoring. Information about the times of naval solution comes from analyses of intercepts in the Public Record Office, London, and in the National Archives, Washington. To avoid burdening this list with a heavy biographical apparatus, I have given as sources the page numbers in Hinsley or in Seizing the Enigma that specify these dates or detail those incidents. Notes in the form 2:659 refer to Hinsley; those in the form 140 refer to Kahn, in which the notes to the text cite the source. "GC&CS" is the Government Code and Cypher School, the British codebreaking agency at Bletchley Park. Names in capital letters of colors, insects, and birds are GC&CS's covernames for Enigma keys.

An important element in the history of the solution of Enigma is that the Luftwaffe Enigma was solved more easily by GC&CS than the naval Enigma, though until 1 February 1942 both used the identical machine with identically wired rotors. One reason is that the air force (like the army) chose the three rotors inserted into its machines from a set of five rotors while the navy chose its three from a set of eight. In addition, air force key settings could often be recovered in the first few months of the war from the key indicators at the head of the cryptograms and later from blunders by the ciphering clerks in making up their part of the key. This gave GC&CS insight into Luftwaffe messages, providing enough cribs for a daily recovery of keys. The naval settings, on the other hand, were taken from books and permitted neither the indicators nor the blunders approach; without some early solutions, GC&CS could not at first get enough messages to yield future guesses of plaintext for cribs. Captures of the books were thus the necessary first step for solution of the naval Enigma. The Enigma keys used by the army were seldom solved because the army cipher clerks, more experienced than those of the Luftwaffe, seldom made blunders.

Solution of the Enigma consisted of guessing an intercepted cryptogram's plaintext (either from knowledge of the traffic previously passed between posts or from solutions of the same messages in other cryptosystems) and then seeing whether the bombes (which were in effect multiple Enigmas) found a setting of the Enigma rotors that would produce such a ciphertext from the assumed plaintext. If they did, that setting was the key for the day on that cipher net and permitted the reading of other messages on the net. The process had to be repeated for each net each day.

Not included here are minor matters, such as the dates of early improvements in Enigma construction or keying, of techniques the Poles used to overcome these changes, or of the solution of secondary keys, such as the Luftwaffe's practice key. (I consider keys minor if they did not contribute significantly to the fighting. Some people may of course differ with my judgment.) A full list of solution dates is given in Hinsley, 2:658-668, 3:1:483-487, 3:2:855-857. In some cases, Hinsley does not state for how long the solutions continued. The date of 24 May 1943 at 2:667 is a misprint for 24 May 1945, Hinsley told me.


Date |  | Enigma | The World
1918 | Feb 23 | Arthur Scherbius files for a patent for the Enigma cipher machine (Patentschrift Nr. 416291) |
 | Apr 15 | Arthur Scherbius offers the Enigma to the German navy (31) |
1918 | Nov 11 |  | World War I ends
1921 | Feb 19 |  | France and Poland sign accord for"concerted measures for the defense of their territory"
1926 |  | German navy begins using Enigma (40-41) |
1928 | Jul 15 | German army begins using Enigma (51) |
1931 | Nov 8 | Hans-Thilo Schmidt betrays army Enigma instruction manual and keying methods to the French, who give them to the Poles (59) |
1932 | Dec | With the help of keys betrayed by Schmidt and given by the French to the Poles, cryptanalyst Marian Rejewski reconstructs the wiring of the three Enigma rotors, enabling the Poles to begin solving German army Enigma messages (65-66) |
1933 | Jan 30 |  | Führer of the Nazi party, Adolf Hitler, named chancelor of Germany
1938 | Dec 15 | Two new rotors (making a toitl of five) go into service on army Enigma; Rejewski recovers wiring, but tenfold increase in number of keys overwhelms Poles, requiring them to ask aid from their allies (77)
1939 | Mar 30 |  | Britain and France guarantee to help Poland if she is invaded by Germany as Czechoslovakia was
 | Jul 24 | Poles reveal success to British and French (79) |
 | Sep 1 |  | Germany invades Poland
 | Sep 3 |  | Britain and France declare war on Germany
1940 | Jan 6 | British break into Luftwaffe Enigma using hand methods (1:108, 2:659) |
 | Feb 12 | British seize two of the three unknown rotors used in the naval Enigma from a crew member of the U-33 captured after the submarine was sunk (114; 3:2:957) |
 | Apr 9 |  | Germany invades Denmark and Norway
 | May | First bombe installed at GC&CS (1:494, more likely than August installation date given at 1:184 because regular solution of RED began May 22) |
 | May | First break into naval Enigma: using documents salvaged 26 April from the trawler Polares (303,3:2:957), GC&CS solves April messages (117) |
 | May 10 |  | Germany opens massive offensive against France; Winston Churchill,a long-time supporter of communications intelligence, named prime minister
 | May 22 | GC&CS begins solution of Luftwaffe general purpose key (RED) continuing virtually uninterrupted to end of war (1:109, 144) |
 | Jun 10 |  | Last Norwegian troops surrender
 | Jun 22 |  | France surrenders
 | Aug | The last unknown naval rotor obtained from a naval capture (3:2:957) |
 | Aug 12 |  | Adlertag (Eagle Day): Germany attempts to win air superiority over England as invasion preparation
 | Sep 15 |  | Battle of Britain D Day: it becomes clear that Germany's attempt has failed
 | Oct 12 |  | Hitler postpones invasion of Britain until spring
 | Dec 10 | SS (Schutzstaffel) general-purpose key (ORANGE I) first broken by GC&CS (2:668) |
1941 | Jan 28 | Luftwaffe operational key for Africa first solved by GC&CS; solutions end 31.12.42 with introduction the next day of separate Fliegerkorps keys (2:660) |
 | Feb- Mar | 2 U. S. Army and 2 U. S. Navy cryptanalysts visit GC&CS and learn Enigma cryptanalysis (235-237) |
 | Feb 15 |  | General Erwin Rommel named commander of German forces in North Africa
 | Mar 12 | Documents seized from patrol ship Krebs permit reading of some March and all April and May naval messages in the Home Waters key net, also used for U-boat messages (137) |
 | Jun 1 | GC&CS reads Home Waters (and U-boat) messages for a month with keys seized from weather ship München and U-110 (169) |
 | Jun 27 | GC&CS breaks German army Russian front key (VULTURE I); solutions last until spring 1944 (2:662) |
 | Jun 22 |  | Germany invades the Soviet Union
 | Jul 2 | GC&CS reads Home Waters (and U-boat) messages for a month with keys seized from weather ship Lauenburg (182) |
 | mid- Aug | Home Waters key (DOLPHIN), then also used by U-boats, begins to be solved daily in 36 hours without seized keys on bases of knowledge gained in two preceding months; solutions continue to 7 May 1945 (1:338, 2:663-664) |
 | Sep 6 | Churchill visits GC&CS at Bletchley Park (184-85) |
 | Oct 5 | TRITON key net for U-boats separates U-boat keys from Home Waters key (205); solutions continue on both nets in times ranging (in January) from 12 to 120 hours |
 | Dec | GC&CS first solves Enigma of the Abwehr, the armed forces espionage service; solutions, called ISK (Intelligence Service [Dillwyn] Knox), continue to end of 1944 (2:668) |
 | Dec 12 |  | Germany declares war on the United States
1942 | Jan 1 | Keys for Luftwaffe's Fliegerkorps IX (WASP) and Fliegerkorps X (GADFLY) broken on day of first appearance and continuing to end of war; key for Fliegerkorps IV (HORNET) broken to December 1943 (2:660) |
 | early | New Short Weather Key goes into service (2:750), ending GC&CS's ability to use meteorological messages as cribs to read naval Enigma (189-190) |
 | Feb 1 | 4-rotor Enigma goes into service on TRITON; U-boat solution blackout begins (2:179) |
 | Apr 3 |  | Rommel begins offensive against British
 | Apr 22 | GC&CS breaks the Luftwaffe key for ground-air cooperation in Africa (Fliegerführer Afrika, or SCORPION); this ends February 1943 |
 | Jun 30 |  | Rommel halts offensive at El Alamein
 | fall | U. S. Navy liaison officer starts at GC&CS |
 | Oct 23 |  | British attack at El Alamein begins German rout
 | Oct 30 | New Short Weather Key captured from U-559 (220, 226) |
 | Nov 8 |  | Americans and British invade North Africa
 | Dec 13 | GC&CS, using new Short Weather Key to get cribs, breaks 4-rotor Enigma keys (SHARK); blackout ends (226, 2:667) but solutions irregular and at first slow (sometimes 8 days), by end of month much faster (around 12 hours) |
1943 | Jan and Feb Mar 10- Jun 30 | SHARK solutions stutter, but when and working some are fast enough to be of operational value (229) 90 SHARK keys solved in 112 days (230)
 | Mar 14-20 |  | Greatest convoy battle of the war, in which 21 ships are sunk, leading Allies to fear that the Atlantic lifeline to Britain could be cut
 | May 24 |  | Commander of U-boats Admiral 24 Karl Dönitz withdraws his submarines from the North Atlantic, signifying a major Allied victory in the Battle of the Atlantic
 | Aug | SHARK being read almost solid and fast (242) |
1944 | Jun 6 |  | Allies invade Normandy
 | Nov | Individual U-boat keys end SHARK solutions, but so great is Allies' strength in ships and air-power that this no longer matters much (262) |
1945 | May 8 | Germany surrenders |

* Fig.
12 Feb 1940 A series of sketches – artist not indicated – showing the end of the U-33 after it surfaced in a vain attempt to escape the minesweeper H. M. S. Gleaner in Scotland's Firth of Clyde 12 February 1940. The sketches show (1) a shower of sparks from the coning tower; (2) crew members in the water as they abandon ship; (3) more in the water as the submarine starts to sink; (4) the U-boat in her final dive with survivors swimming away from her. Three rotors were recovered from a crew member who forgot to take them from his pocket and drop them in the ocean.(Public Record Office, ADM 199/123, p. 54)

* Fig.
May 1940 The former trawler Julius Pickenpack, now a German attack vessel disguised as the Dutch ship Polares. From one of its canvas bags, thrown overboard on the approach of British warships but saved by a British sailor, the British obtained cryptographic documents useful in solving German naval Enigma traffic.(Public Record Office, ADM 199/476, p. 243)

* Fig.
1 June 1941 The officers and crew of the U-110, in port, await an approaching inspecting officer while the skipper, Lieutenant Fritz-Julius Lemp, 28, approaches the gangplank to greet him. On patrol, the submarine was seized by the British and, before it sank, yielded an Enigma machine and keying documents. These enabled the British to read U-boat messages for June 1941. Lemp was lost during the action.(David Kahn collection)

* Fig.
2 July 1941 Hinrich Gewald, 58, skipper of the Lauenburg, whose capture by the British led to the seizure of Enigma keying documents that enabled the British to read U-boat messages for July 1941.(David Kahn collection)

* Fig.
2 July 1941 The Lauenburg, just before being boarded on 28 June 1941.(David Kahn collection)


The Siemens Cipher Teletype in the History of Telecommunications

WOLFGANG W. MACHE

ADDRESS: Zielstatt St. 133, D-8000 Munich 70, WEST GERMANY.

ABSTRACT: This paper spans a period from early pioneers in electrical telegraphy and telephony to the roots of the Geheimschreiber: the (electric) Siemens Schnelltelegraphy and the development of the first electrical teletypewriter "T25" by Wernerwerk 1, Berlin, in 1928. The paper expresses the opinion that the Geheimschreiber T52d/e was not cracked by Bletchley Park's deciphering machines during WWII.

KEYWORDS: Teletype, electric teletypewriter, Geheimschreiber, Schlüsselzusatz, Bletchley Park, sägefisch, Fish, Tunny, Sturgeon, COLOSSUS.

The following is a translation of a speech given at the 23rd Meeting of the Association of Former Navy Teletype Operators held at the Telecommunications School of the Federal Navy, Flensburg-Muerwik WEST GERMANY on 26 August 1984.

Much appreciation goes to the Siemens Museum (Munich) and Deutsches Museum (Munich) for supplying photographs.

Today when one hears the word America in discussions of countries that have produced great inventions, in communications technology what people associate with it are names like Samuel Finley Morse for the telegraph, Alexander Graham Bell for the telephone, and John William Mauchly for the computer. All this is a one-sided view, however, and does not do justice to any serious history of technology. It is true that the men I have just named have made widely known and immortal contributions, but in regard to the title of the inventor or the first we must differentiate.

It was from this point of view that I welcomed and supported the article by Dr. Jaxotskiy of the Post Office in Moscow in remembrance of the 150th birthday of Johann Philipp Reis, born 7 January 1834 in the Barbarossa town of Gelnhausen, as a pioneer in telephony in No. 1/1984 of the technical journal Electrical Communications. The author made use of a quotation, you see, which I would like to share with you.

It runs: "A Critical history of technology would generally show that hardly any invention of the 18th and 19th centuries belongs to one individual or another."

Incidentally, this quotation comes from a gentleman who is known to all of you. He is the jurist, philosopher and utopian Karl H. Marx.

I think we must agree with Marx, because his comment also holds true to a remarkable extent in regard to the development of telegraph equipment.

The processes of the development of telephone and computer technology also bring up a multitude of names, which I will not enumerate here.

Instead, let us stay in the realm of teletype technology. Its rapid development was based on discoveries by scientists whose names, gentlemen, you know as well as I do. I remind you of Galvani, Oerstedt, Faraday, Henry, Ohm, and Ampere, to name only a few.

                                           Control Number 1135
STAFF INFORMATION. SECRET!
The Siemens SFM T52d Cipher Teletype Machine
SECRET STAFF INFORMATION!
      1. This is a state secret within the meaning of Section 88 of the
         Reichs Criminal Code.
      2. To be forwarded only by handcarrying or addressed by name, in
         double envelope, and upon written acknowledgement of receipt.
      3. Transmission by courier or by a trusted person if possible; with
         a declared valuation of more than 1000 Reichsmarks if mailed.
      4. Reproduction of any kind or taking extracts is forbidden.
      5. Storage is upon the responsibility of the recipient, to be in a
         safe or under exceptional conditions in a steel locker with a
         combination lock.
      6. Violations of these rules will be most severely punished.
Printed Naval Regulation No. 35 IV

Cover sheet for the Siemens SFM T52d Cipher Teletype Machine

But how did it begin?

On the question of origin in the field of pre-Philipp Reis electrical telegraphy, I may emphasize its germ in Germany, namely the electrochemical telegraphy of the physician and member of the Royal Academy of Sciences at Munich, Samuel Thomas von Soemmerring, born in 1755 at Thorn.

After the invention of the electrical pile by the Italian physicist Volta in 1800, Soemmerring succeeded in setting up an impressive but slow parallel-wire telegraphic system over a distance of around 1000 meters, making use of the releasing of gas. Soemmerring demonstrated his method in Munich at the Academy in 1809, but he was unable to make any headway against the optical telegraphy (i.e., semaphore) which was dominant at that time, (e.g., the Chappe System, used by Napoleon Bonaparte at that time). However, his labors were of inspirational value for the later telegraph systems that Schilling von Canstatt and Moritz Jacobi developed in Petersburg. They determined the further progress of two-wire telegraphy. For electrical telegraphy, in Russia 21 October 1982 was the 150th anniversary of the invention of the electromagnetic telegraph, the day when the German-Russian Baron Paul Schilling von Canstatt successfully demonstrated his 6-pin apparatus for the Russian alphabet to the Czar and the Academy in Petersburg in 1832. After his death in 1837 his work was continued until 1874 by Jacobi, a Prussian master builder in Koenigsberg and from 1837 on a scientist in St. Petersburg.

Let us not trouble now to go into the numerous events in the evolution of telegraphy from 1833 to the start-stop teleprinter of the 1920's in detail. But emphasis should be given to the essentials. Some systems established milestones and others sooner or later were replaced. Thus, for example, the needle telegraphs did not achieve lasting prevalence, whereas the American painter Samuel Morse, in the years from 1837 to 1844, in creating so-called recording telegraphs and inventing the alphabet that is named after him made an inestimable contribution that is still influential at the present time. Morse gave credit publicly to the German Steinheil, who first proved to use the ground as the return loop, which saved the installation of a 2nd wire and eased the introduction of the Morse system in the US Continent in 1844.

Still more successful was the technology of the type printing telegraphs that came into use in 1854. A well known representative of this type of equipment is the so-called Hughes printer, a very remarkable device with a life in service of almost 8 decades.


I have been told that there are several members represented in your Association who were trained on this machine here in Muerwik at the Torpedo and Communications School before the beginning of the 1930's. If you should ever come to Frankfurt am Main some time, I recommend to you a visit to the Federal Postal Museum. There on display are two different models of the Hughes printer, in working condition. In 1900 there were already 3500 of his teletypes in existence in the whole world when Edward Hughes dies a millionaire and one of the leading consultants in telegraphy. As early as 1895, Hughes printers were being developed further at Siemens, and by the beginning of the teletype generation of the 1920's they were being built. A parallel development in 1903, that of the Siemens automatic telegraphs with a speed of 2000 characters per minute, (i.e., 33 characters per second), was far in advance of its time. It was followed in 1924 at Siemens by the 56 Baud keyboard printing telegraph, which still came in two pieces (i.e., separate sender and receiver. See picture). That became the starting point for the electrical teletype. You see, there was more demand for a machine that would send, receive, be operated by only one person, and still be cheap as well. What was aimed at was a machine similar to the type bar carriage typewriter in common use at that time, which had been factory made since the second half of the 19th century. It was in this way that the start-stop teleprinter patents of the German-American Krum originated in the USA, financed by Morton and with the trademark MORKRUM, and in 1923 for Kleinschmidt, a German-American from Bremen. The two firms merged in 1924 to form the Morkrum-Kleinschmidt Corporation. Next came the development of the "Teletypewriter Model 14," which was imported under the name Springschreiber (i.e., start-stop teleprinter) in 1926 by Lorenz of Berlin. Lorenz copied the "Teletype 14," and Siemens & Halske successfully met the challenge from America too. Between 1927 and 1933 Wernerwerk of Berlin likewise developed a series of different teletype machines in rapid succession. In the course of doing so, a transition was made from steps under patent to taking new directions.

* Fig.
* Fig. Figure 1. Soemmerring's electrochemical telegraph. Munich, 1809. Later improved by Schweigger for use over only two wires, 1811 (but not realized).

* Fig.
* Fig. Figure 2. Schilling von Canstatt's 6-pin telegraph. Petersburg, 1832. Courtesy: Deutsches Museum, Munich.

* Fig.
* Fig. Figure 3. Siemens high-speed telegraph. Berlin, 1903. Courtesy: Siemens Museum, Munich.

In 1927, when the German licensee of the principle of the lengthened stop signal in the mechanical teletype, patented by Kleinschmidt, would not share his monopoly with other manufacturers, F. Geissler, a designer in our firm, had the idea of cleverly circumventing the Kleinschmidt patent through an electrical plan. There came into existence an electrical teletype machine, "T type 25", whose further development - available in 1932 - is familiar to all of you gentlemen. I refer to the T of "T type 36." Until now it remained unclear why it was this particular machine, which later served as the basic module for a secret teletype machine, that attracted the interest of the Navy in the first place. The Navy preferred the electrical machine over the mechanical machine because of its superior stability at sea.

This machine superseded the Hughes printer that had been in use up to then, and the sounder. The change from the piano-like keyboard to a completely new set of keys may have caused a few problems at the time. Perhaps one or another of you can confirm that.

Of the further developments that followed at Siemens, I would like to point out the following mass-production teletypes, all of them page printers:

• 1933: T type 37a (successor to the T type 26a of 1929),
• 1957: T 100, 50 Baud capacity,
• 1961: T 100, 100 Baud capacity,
• 1976: T 1000, already highly integrated.

The following figures point to the worldwide significance of these developments:

By 1984 Siemens had supplied over 1 million teletype machines and had become the unquestioned market leader among 13 manufacturers.

Our firm's share in the total number of 1.6 million telex machines in the world is 30%.

By the end of the second World War, 15,000 machines had been produced; in recent years, 1978 to 1983, it was 70,000 machines a year, just on the average.

From the point of view of types of technology, we get the following distribution figures:

• 55% electromechanical, including Model T100
• 40% electronic, first generation
• 5% electronic, second generation, microprocessor controlled.

After this short outline of the history of the development of telegraphy, I would like to take up a machine that you all know, gentlemen, and which even yet - almost 40 years after the end of the was - is surrounded by secrets. I mean the Siemens cipher printer "T type 52", models "a" through "f". On the construction and manner of operation of this machine I do not need to say much. You, gentlemen, have worked with this machine during your time of active service. Many of you have been at Siemens in Berlin, and have taken part in the course of instruction there. Instead of going into that, allow me to make a few remarks on the development of this machine and the enemy efforts at decipherment.

* Fig.
* Fig. Figure 4. Siemens keyboard printing telegraph. Berlin, 1924.

* Fig.
* Fig. Figure 5. T type 25 without protective cover.

In 1969 a colleague in our firm, Ehrhard Rossberg, director of the Technical Division of Switching Technology of Siemens retired. The patent department marked the departure of this extraordinarily skillful engineer with a separate volume which contained more than 100 of his patent descriptions. Mr. Rossberg, who was born in 1904, attended the Chemnitz Industrial Academy after a period of practical training, and came to Siemens & Halske in Berlin-Siemensstadt in 1926.

* Fig.
* Fig. Figure 6. Electrical teletype, T type 36. Berlin, 1932 (without casing).

At Siemens, Mr. Rossberg was on the scene at the new starting point of the T type 25 electrical teletype and in 1928, when the Navy in Kiel selected it. A productive collaboration ensued shortly with additional, very successful colleagues such as Honorary Dr. of Engineering Herbert Wuesteney, Honorary Dr. of Engineering August Jipp, Eberhard Hettler, Fritz Geissler, and others.

The Navy, concerned because of the British decipherment successes in the first World War, made it a requirement that the electrical teletype should be provided with an enciphering device.


This idea was unusual, because at that time teletype text requiring protection were enciphered before transmission in a fussy and time-consuming manner. But what they had in mind now was the truly sensational development of what today is called an on-line machine. It is possible also to ascribe this to the design advantages of the electrical teletype, whose creation was provoked (i.e., by the Kleinschmidt patent problem), but was fortunate.

* Fig.
* Fig. Figure 7. T25 with cipher device. Berlin, 1930/31. (Laboratory model of the cipher printer).

On the way there, we come in 1930 to the first applications for patent with the title Arrangement of Switching for the Transmission of Information in Cipher.

A further application from the year 1931 contains the principle of irregular cam discs, i.e., cam discs of different sizes, set up chiefly on the basis of prime numbers. In the same year the patent application was filed in the USA by Jipp, Rossberg and Hettler, there US patent 1,912,983 was granted June 6, 1933.

* Fig.
* Fig. Figure 8. Cipher printer, T52a/b, with power supply in the lid, before WW II.

It has been reported by several veterans that the Navy was the first service branch to be interested in using the cipher teletype machine, also called a cipher printer (Geheimschreiber), or G-Schreiber for short. The Navy wanted secure written communications, for example, from a ship at the pier to the high command of the fleet.

I owe it to a fortunate circumstance that in 1982 in Munich I was able to discover - or rather rescue - a photograph that shows the laboratory model from this first couple of developmental years. It still consisted of two separate machines, the T type 25 electrical teletype and the cipher device with 2 x 10 adjustable wheels. This attachment was later, in the commercial version of 1932, united with the T36, in one unit, the "T52".

* Fig.
* Fig. Figure 9. Cipher printer, T52c, without casing.

Now we go to the introduction of the Siemens T type 52a cipher teleprinter.

In 1932 the Hungarian armed forces were the next to be supplied with machines and complete documentation.

Navy tests soon showed that the cipher printer interfered with the sensitive radio receivers. For this reason the model T type 52a/b with noise suppression circuitry was created in 1934.

A further step in the development of the cipher printer was the T type 52c, for which the Air Force gave specifications in 1936.

* Fig.
* Fig. Figure 10. Cipher printer, T52e, without casing, a Norway machine

In the meantime, the security authorities of the Third Reich imposed secrecy measures which still make themselves felt today in the considerable scarcity of information.

Nevertheless, after the signing of the German-Russian treaty on 28 September 1939 in Moscow, the Siemens SFM T52 cipher teletype machine was demonstrated and shown opened up to Soviet military delegation in Berlin-Siemens-stadt.

Development continued during the second World War. Models T type 52d and T type 52e were created, and towards the end of the war Model T type 52f was in the works, at least in specification form.

In 1944 or '45 the technical documentation in the factory for the strictly secret T52 became lost. For one thing, the production facility in Berlin was badly hit by Allied bombs in October 1944. But what stands today as the main reason for unsuccessful endeavors to obtain authentic factory records and technical descriptions is the rigorous security regulations and their application before the occupation in 1945.

After the end of the war Dr. Wuesteney was brought to the British headquarters in Berlin-Gatow and given the task, together with Fricke, a Siemens mechanic who worked on T52 assembly, of reconstructing the circuit diagram of a T52. At the end of a week the British discontinued the project without result, although they are said to have gotten possession of no documents, but only machines, with Mr. Fricke, among others, serviced. The machines were later put to use on teletype connections between the British Sector in Berlin and the headquarters of the Army of the Rhine.

In Norway, too, a number of T52e's survived the end of the war, and were later put into use again there. Of these there are specimens in operating condition on display today in the Siemens Museum in Munich, in the Science Museum in London, and in the Technology Museum in Oslo. The exhibit in the Military Museum at Dieppe-Pourville on the other hand, a "d" (in phonetic alphabet Dora) machine (T52d), which I saw last month, is incomplete, poorly maintained, and hence not in working condition, although the T rls 43a telegraph relays, for example, look like new.

From the former armed forces' telecommunications equipment depot which was in in Elmshorn between 1948 and 1960, the remaining cipher printers, which had previously been demilitarized, i.e., partly dismantled, were brought to Trier and there at the Electromechanical and Electronic Plant were put in circulation, as the French security authorities had decided on utilization of the cipher printers. Trier supplied a total of 280 T52d's and e's, which is an estimated 25% of all T52's supposed to have been produced in Berlin up to 1944. The cipher printer was classified down to the present time, and may still be in use somewhere. Investigations become all the more difficult; informants prove to be a decided rarity.

This may be sufficient at this point as a presentation of the development of the cipher printer.

But now to the question of the extent to which the enemy was able to break into our machine's cipher texts. I can anticipate the answer: Up to the present time we have not been able to obtain any reliable information — but let us nevertheless attempt an approximation.

Soon after the first opening of the British secret records in 1975, 30 years after the second World War, the British journalist Brian Johnson wrote in his book The Secret War about technical facilities of the British decipherment center at Bletchley Park. According to that book, there were the so-called bombes for processing the Enigma material, and forerunners of the computer, designated Colossus, were available for deciphering the cipher printer texts. This statement went from one book to another, and was even broadcast by the BBC in both words and pictures.

* Fig.
* Fig. Figure 11. Decipherment computer Colossus, Bletchley 1944.


Knowledge of our own and indications from England over the past years have confirmed us more and more in the belief that the above-named author's investigations were in behalf of the Siemens & Halske Geheimschreiber inadequate. Recently our belief has become a certainty. Because the third volume of the official British history of the operations of the intelligence service in the second World War has been published. In it, in Part I, the principal mission of the Colossus decipherment computer is described: Actually, it was used in decrypting German 5-character code messages. These messages however came not from the Siemens cipher printer, but rather from another teletype cipher printer, the model SZ40 and SZ42 cipher attachment from the firm of Lorenz — a machine, therefore, with which you may have had less contact, gentlemen.

In order to substantiate this statement further, at this point let a short review of the development of radio teletype technology be permitted:

As a result of the constantly increasing distances between the fronts, in 1940 a need grew up in the German armed forces for a serviceable radio teletype system that cipher equipment could be incorporated into.

To meet this need, in the course of 4 years a series of various devices was developed and in varying quantities was requested and operated by users, chiefly the Army. The first of these shortwave radio teletype sets with telegraphy channels went by the covername of Saegefisch (i.e. Sawfish) on the German side, by the way. The British intelligence service found out this designation from a decrypted Enigma message, and for their part used the codeword fish for it. Since about 1978, this covername has been incorrectly applied to the Siemens cipher printer, and indeed even when in reality what is being discussed is the German 5-character code traffic in general, or even the decipherment of Army communications done on the Lorenz cipher attachment.

It would be dishonest to the history of the field if it went unmentioned that the T52 designed by Siemens for wire communications lines was also used in its "b" and "c" models for the first radio teletype trials ([19]40/41). It soon turned out, however, that both models were suited only for limited use. This was because if it happened, for example, that as a result of atmospheric disturbances even a single letter of cipher text was suppressed, then the encipherment's synchronization was thrown off and subsequent text was decrypted incorrectly. The problem was overcome, however, with the Lorenz machines already mentioned.

Thus when the British systematically monitored the German experimental radio teletype link between Vienna and Athens in 1941, they thought they were on the trail of the Siemens cipher printer. We must even concede today that British mathematicians had then already succeeded, by awkward manual procedures, in deciphering German teletype messages. In so doing, they realized that only automatic decrypting machines could give effective support to their own military operations. These were actually put into use in June 1942, and had to be constantly modified because of their severe rate of wear and breakdown.

* Fig.
* Fig. Figure 12. SZ42, without cover. (1984 pictures, Navy Telecommunications School, Flensburg).

The British were more successful with the Colossus computer, which was not put into use until February 1944, however. An example on this point:

Beginning on 6 January 1944 the radio teletype link belonging to the Commander in Chief in the west (Army Group D) between Paris and the OKW [Supreme Command of the Armed Forces] in Berlin (Stratssberg radio station) was intercepted, and starting in March, about 3 months before the beginning of the invasion, they were in a position to decrypt teletype enciphered on the SZ42 (Schüsselzusatz 42).

But the "d" model of the Siemens cipher printer was also intended for radio teletype operation. Here is the proof: It is from among you, gentlemen, that valuable training materials from the Naval Communications School at Waren-Mueritz dating from 1944 have been made available to the Archive of the Naval Telecommunication School. They contain operating directions for the machine just mentioned. So far, however, it has remained unknown whether the Navy has at any time ever operated communications as we see them shown here. It would be very nice if this matter could be clarified with your kind assistance.

* Fig.
* Fig. Figure 13. SFM T43, basic model of mechanical page printer, T type 37f.


With that I come to my conclusion:

I believe it has become clear that we are only at the beginning of the truth. The official British historiography has up to now not been prepared to make any statement regarding the extent of the decipherment of teletype texts. Might this perhaps be because they would have to admit to not having had unqualifiedly splendid success? I ask you to consider: Up to the present time we still have no information on how the attempts at decipherment turned out in so far as they relate to the T type 52 "d" and "e" cipher printer or the "SFM T43" cipher teletype machine also developed by Siemens late in World War II. Wernerwerk T in Berlin built the "T43" on the basis of 2 types of teletype machine. These were the T type 37f mechanical page printer and the T type 34n mechanical tape printer, of which a hand full of machines started working in 1944 on a one-time-pad principle, e.g., between Paris and Berlin.

* Fig.
* Fig. Figure 14. SFM T43, basic model of mechanical tape printer, T type 34n.

Thus for the present the question remains open as to the extent to which the teletype decipherment added to the significance of the Enigma decipherment or qualified its value. In the same way it must be kept for a later historical study to elucidate the degree to which the decipherment successes that have been presented influenced the strategy and tactics of the Allied invasion.

Gentlemen, and naturally ladies also, in so far as you were active in the communications service, the aim of my remarks has been, among other things, to present to you some facts about the T type 52 Siemens cipher printer which cannot have become known to you at any time, at least, during your period of active duty.

If I have succeeded in this, then it is an additional pleasure to me that I have been able to comply with your invitation to the Flensburg inlet, which I know well from former years. I would also like to express my obligation to the commandant of this school, Kapitaen zur See (rank of US Captain) Ehlert and Kapitaenleutnant (rank of US Lieutenant Commander) Timm for their support for this presentation: I wish all of you unforgettable days here in Muerwik.

WORDS USED IN FIGURES

1. |  | Sender | transmitter
2. |  | Empfaenger | receiver
 |  | Sauerstoffblsen | oxygen bubbles
3. |  | Locher | perforator
4. |  | 56 Bd | 56 Bauds
5. | a. | Tastenwerk | keyboard unit
 | b. | Kontakteinrichtung | contact device
 | c. | Sender | transmitter
 | d. | Empfänger | receiver
 | e. | Übersetzer | converter
 | f. | Druckwerk | printing mechanism
 | g. | Übersetzerrelais | converter relay
 | h. | Locherschalterlais | perforator switching relay
 | i. | Auslöserelais | release relay
 | k. | Regler | control
 | l. | Weckerrelais | relay controlling local bell circuit
12. |  | Gleichlaufeinrichtung | synchronizing device
 |  | Funkfernschreibbetrieb | radio teletype operation
14. |  | Abfühleinrichtung | tape reading device
 |  | Vorshubeinrichtung | feed device
 |  | Stanzeinrichtung | punching device
 |  | Gleichlaufschalter | synchronization switch

BIOGRAPHICAL SKETCH

Wolfgang W. Mache, member of VDE/ITG, derived his interest in WW II cryptology from Brian Johnson's book The Secret War (1978), whilst he wrote the Lexikon der Text- und Datenkommunikation (1980, Oldenbourg). He joined Siemens for telephone technician training, received a Dipl. Ing. degree in Electrical Engineering (1955) and then worked in Siemens' Telecom development Centers, Munich-Sendling for Power-line telephony and FDM/TDM-ARQ radio teleprinter systems. After time as Telecom Sales Manager in South America he is in marketing in the field of data transmission and switching.


New Information on the History of the Siemens and Halske T52 Cipher Machines

D. W. Davies

ADDRESS: Fair Winds, 15 Hawkewood Road, Sunbury-on-Thames, Middlesex TW16 6HL GREAT BRITAIN.

ABSTRACT: Two earlier papers in Cryptologia [1, 2] described the WWII German on-line cipher machine known as T52. The circuit which controls the irregular movement of the cam wheels appeared in two forms and it seemed likely that alterations had been made. The reason for these alterations can now be explained. The paper which follows this one is from the person who made the changes.

KEYWORDS: Cipher machine, Teletype, Geheimschreiber, World War II

BACKGROUND

The T52 Cipher Machines were Siemens teletype machines, modified by adding extra equipment which could encipher their transmissions and decipher what they received, on-line. They were used extensively by the German forces during World War II. Until near the end of the war, this secret traffic was not sent by radio, only by lines. Consequently it was not routinely broken by the Allied cryptanalysts.

The T52 machine is sometimes known in recent literature as a "Geheimschreiber", but there seems to be no clear evidence that it was called by this name. German documents of the time call it "Die Siemens" - "Schlusselfernschreibermaschine". These cipher machines should not be confused with the Lorenz cipher attachments, SZ40 and SZ42, which were the source of the "fish" traffic broken so effectively by the Colossus machines.

EARLIER REPORTS ON THE T52 MACHINE

Models T52 a/b, c, d and e were produced. I wrote a detailed account of the T52e [1]. Then [2] I traced the development of the T52 through its four models (a and b were logically similar, differing only in the suppression of electrical noise in model b).

All models featured 10 cam wheels which rotated step-by-step as Baudot characters were transmitted or received. The numbers of steps in a complete rotation of each of the cams were mutually prime so that, if all cams stepped each time, the maximum total cycle length would be achieved. The cam profiles produced a pseudo-random string of bits. The outputs of the 10 cam contacts were used both for modulo 2 addition (XOR) and for permuting bits in the 5-unit Baudot code. Details are given in the two papers quoted above.

An important feature of models d and e was that cam wheel movement was intermittent. Any given cam would move approximately three quarters of the time, its movement depending on a logical function derived from the outputs of the cams themselves. The outputs used for encipherment and for controlling cam movement came from different points on the cam periphery.

In the 1982 and 1983 papers there were unexplained features of the cam movement control circuits. New information has come to light which helps to explain the history of these machines. In this paper I will give more details of the circuit information that was included in the earlier papers. A companion paper by Ernst Selmer describes how the changes came to be incorporated in the cam movement control circuits.

THE TWO VERSIONS OF CAM MOVEMENT CONTROL CIRCUITS

My first observations of the wiring of these circuits showed that changes had been made. Many of the wires forming these circuits were attached to a multi-point switch called KTF or Klar-Text Function which, when set to the ON position, complicated the whole process by making control of cam wheel movement also dependent on one of the previously transmitted plaintext Baudot elements. (With KTF in operation, a single error in transmission would put sender and receiver permanently out of synchronization.)

Of the 10 wafers of the KTF switch, only 4 were in use but these and the others had loose ends of wire attached. I concluded that at some time all the wafers of the switch had been used and that a radical rewiring had taken place, but its purpose and history was obscure.

The machines I had studied were at the Siemens Museum in Munich and at the Science Museum in London, on loan from the Norwegian Technical Museum in Oslo. Both came from Norway and it soon became clear that most of the T52 machines still in existence had their origin in Norway.

Then new sources of information appeared. Circuits from the Oberkommando der Kriegsmarine dated March 1944 showed a different logic for controlling cam movement. Another source was Ing. Willi Reichert who, at the end of WWII bought T52 parts at scrap prices from a store at Elmshorn and rebuilt them into T52 machines in Trier. His circuits showed the same version as the Kreigsmarine. I understand that Reichert's machines were supplied to the French Government.

Finally, I had the opportunity to examine a T52 machine in the care of the ZFCh (German Cipher Bureau) in Bad Godesburg and found again the same circuit. It therefore seemed that all T52 machines with intermittent wheel movement, whether T52d or T52e, had the same movement control circuit and that only the Norwegian machines had the alternative version. My paper of October 1982 had been misleading because it showed in Figures 7 and 8 a circuit that resulted from a Norwegian modification of the machine, not the original as I had supposed.

The figures in my two earlier papers used different notations. to clarify the nature of the changes a new figure is given here comparing the circuits in the condition "without KTF". the wheels are labelled A to K. The figures show the cam outputs as changeover contacts, which are approximately balanced between the 0 and 1 states. The magnets are therefore operated approximately one quarter of the time. Each magnet, by means of an interposer, stops the stepping of its corresponding wheel. Note that contacts do not influence their own wheels, except in one case which is the C wheel in the modified circuit. Also in the modified circuit the B wheel might be expected to stop on about 3/8 of occasions.


WHENCE THE MODIFICATIONS?

At the Oslo Defence Museum I was told that T52 machines had been modified for use by the Norwegian Border Guards. This was all the information I had until a report from Eurocrypt '93 that Dr. Ernst S. Selmer had spoken about his own role in the "capture" of these machines and their modification. His part of the story is given in a companion paper.

He questions whether the modifications improved the machine cryptographically. One objective would be to obtain a very long cycle before the wheels were all back in a previous position and the whole cipher repeated. It is very hard to estimate the cycle lengths produced. In the case of KTF, the cycle structure depends on the text being transmitted, which rather reduces its importance.

The original wiring produced more difference between the behavior with and without KTF. I believe that KTF caused so much operational difficulty (and weakness due to restarting after a failure) that after a time it went out of use. The significant circuit is the one "without KTF"

* Fig.

Comparing the "without KTF" circuits it is noteworthy that in the original configuration cam wheels A, B, C and D always moved together. This ensured that any cycle would be a multiple of 73.71.69.67 (=23,961,009) steps. How this compares with the (less easily estimated) performance of the modified circuit is an interesting question. Perhaps this was considered a large enough cycle to exceed the life of one key period.

OTHER MODIFICATIONS OF THE T52?

Two other features of the machines studied might be evidence of modifications.

(I) The 10 binary outputs of the cam wheels go through switches or a plugboard which permute them. This permutation is the main cryptographic key. Then in models T52c and T52e the outputs go through the logical transformation of the H relays to operate the encoding relays SR. This logic is different in the two models. Compare October 1982, Figure 10 with July 1983, Figure 8.
I believe that this is a real difference between models T52c and T52e, since there is no evidence of extensive wiring changes and nothing else suggests that a change has been made.
(II) From the start of my work it appeared that the cam profiles were fixed - that they remained the same throughout the whole series of models and machines. These profiles are recorded in Figure 3 of the October 1982 paper. But when I came to examine the T52 machine at ZFCh the profiles were different.

The mechanical explanation was that the same cam parts had been used, but assembled differently. Each cam is fixed by three screws and can be assembled in one of three positions by rotation through 0, 1/3 or 2/3 of a turn. Also a cam can be flipped over, so that the sequence of output bits is reversed, making a total of six possible orientations. The ZFCh machine had different orientations from the Norwegian machines. It is not the kind of change that could easily be made in the field because there are consequential adjustments of contact lift and timing.

I found the modified cam orientations nowhere else. Either these were the wartime positions and all Norwegian machines had been modified or else it was the ZFCh machine that had been modified.

There is no conclusive answer. Ernst Selmer has no recollection of the change, which would have been a major task. Therefore I incline to the view that, for some reason, the ZFCh machine was modified and that in all the other machines we see the cam profiles in WWII.

REFERENCES

1. Davies, D. W. 1982. The Siemens and Halske T52e Cipher Machine. Cryptologia. 6(4): 289-308.
2. Davies, D. W. 1983. The Early Models of the Siemens and Halske T52 Cipher Machines. Cryptologia. 7(3): 235-253.

BIOGRAPHICAL SKETCH

Donald W. Davies helped to build one of the first electronic digital computers at the National Physical Laboratory. He was active in computer research and pioneered the concept of packet switching from 1965 onwards. Then his interests moved to computer and communications security. Since 1984 he has provided security consultancy to financial institutions, telecommunications providers and pay TV broadcasters, among others. He has received a number of honours, including Fellowship of the Royal Society.


The Norwegian Modification of the Siemens and Halske T52e Cipher M-chines*

*Extract from the Eurocrypt '93 Proceedings [5]

Ernst S. Selmer

ADDRESS: Idrettsvn. 20, N-1400 Ski NORWAY.

ABSTRACT: Two earlier papers in Cryptologia [1, 2] described the WWII German on-line cipher machine known as T52. The circuit which controls the irregular movement of the cam wheels appeared in two forms and it seemed likely that alterations had been made. The reason for these alterations are given in this paper, by the person who made the changes.

KEYWORDS: Cipher machine, Teletype, Geheimschreiber, World War II

In 1946, my friend Nils Stordahl became head of the Norwegian military cipher office, a position he held until 1983. One of his early initiatives was to engage me as a consultant.

My first big task, and my most fantastic cryptological experience, was to establish a (hopefully) safe communication system for the Norwegian equivalent of MI5 and Scotland Yard Special Branch ("Overvakningspolitiet" = "watch-over police"). We based it on the German Siemens teleprinter ("Fernschreiber" = "remote writer"), with an additional unit for encryption/decryption ("Geheimschreiber" = "secret writer"). There were many of them in Norway, and the Norwegian (public) Telephone and Telegraph Company had collected them to dismantle the cipher unit and use them as ordinary teleprinters. We had to prevent this, and could not let it pass official channels. I cooperated with another of Stordahl's men, a young electrical engineer named Asbjörn Mathisen. Some time in 1946, we were supplied with two large military trucks and 20 German prisoners of war. We commanded them in our school-Germanm, and drove to the Telecompany store just outside Oslo. The few attendants protested vigorously, but did not stand a chance against us and the Germans. Including a wooden case, each G-Schreiber weighed 180 kg, so the POW's were really needed. We got everything onto the trucks and drove it to a safe hiding place.

Mathisen used wires and torch bulbs to reconstruct the coupling diagram of the G-Schreibers, including 20 relays. I knew absolutely nothing about relays then, but had to find out how everything worked, from a maze of unsystematized wires.

The same reconstruction has been performed much later by other people. The Norwegian Technical Museum in Oslo had managed to get hold of 3 G-Schreibers. Two of them were given - for exchange purposes - to museums in London and Munich, and these were analyzed by Donald W. Davies [1, 2]. Some more historical information was supplied by Wolfgang Mache [4].

The cipher unit contained 10 large notched wheels, stepped by a pawl mechanism. The number of steps in a revolution varied from 47 to 73, all pairwise coprime. The cam pattern was the same for all machines. The cams activated two contacts for each wheel. Over 20 relays, one contact set controlled an irregular stepping of the cam wheels, while the other set performed the ciphering of each 5 bit teleprinter symbol, in two rounds: one bitwise addition key, and one permutation (only 32 out of the 5! = 120 were used). Details can be found in [1, 2, 4].

I did not want to use the German outfit exactly as it was, so I made some modifications which Mathisen implemented. I could not change the notched wheels (with contact sets), but I fiddled with the above-mentioned relay control of the cam wheel stepping. I thought I made it more complicated, but had nobody to check whether the outfit had become cryptologically stronger.

The G-Schreibers came in successive versions 52 a/b, c, d, e. The first version was broken by the Swedish mathematician Arne Beurling (cf. Kahn [3]) from the traffic passing Sweden between Norway and Germany. Later versions were "occasionally, but not routinely" broken by the British cipher office at Bletchley. - All this has become known later; in my case, I just had to hope for the best.

In my later designs of cipher machines (for NATO), I always said like the waiter: "Not my table", about all questions regarding protocols, key distribution, initiating and closing routines, etc. But for the police cipher, I had to work this out all by myself. In 1948, we summoned 20 police officers from different stations to Oslo, where I drilled them in the use of the G-Schreiber for a 3 weeks course. And the system was used from 1949 until around 1960.

COMMENTS ADDED TO REPRINT

Andrew J. Clark, after attending the Eurocrypt '93 seminar, has established contact between me and Donald Davies, who wrote the above-mentioned two papers on the G-Schreiber in Cryptologia. In a third, companion paper, he now describes in more detail the differences between the original and the "Norwegian" versions of Model 52e, in other words, my modifications.

All my notes on the G-Schreiber, from the late '40s, were stored at the cipher office, and were unfortunately destroyed a long time ago. Of course, it is impossible for me to recollect the modifications I made, and it is very satisfactory that this has now been described by Donald Davies.

The only relevant archive information I have been able to trace is a note at the Norwegian Technical Museum, stating that they got their 3 machines from the Military High Command in 1961, so they were in all likelihood police machines which had been replaced by more modern equipment.

I have also been in contact with one of my police pupils from 1948. He remembered that they were instructed to use KTF (Klar-Text Funktion, cf. [1]) only for top secret messages.

REFERENCES

1. Davies, D. W. 1982. The Siemens and Halske T52e Cipher Machine. Cryptologia. 6(4): 289-308.
2. Davies, D. W. 1983. The Early Models of the Siemens and Halske T52 Cipher Machine. Cryptologia. 7(3): 235-253.
3. Kahn, D. 1967. The Codebreakers. New York: Macmillan.
4. Mache, W. 1986. Geheimschreiber. Cryptologia. 10(4): 230-247.
5. Selmer, E. 1994. From the Memoirs of a Norwegian Cryptologist. In Advances in Cryptology - EUROCRYPT '93, ed. T. Helleseth. New York: Springer-Verlag.

BIOGRAPHICAL SKETCH

Ernst S. Selmer received his Dr. philos. degree (Mathematics) from the University of Oslo in 1952. From 1957 to 1990 he was professor of pure mathematics at the University of Bergen, Norway. His main mathematical interest has been number theory, but he has also been strongly engaged in computer activities and cryptological work for the Norwegian government.


The Breaking of the Japanese Army Administrative Code

David Mead

ADDRESS: 1229 Glen Meadow Road, East Lansing MI 48823 USA.

ABSTRACT: This memoir relates the writer's experience as a Signal Corps cryptanalyst in World War II. The major focus is the effort of the Signal Intelligence Service at Arlington Hall to crack the Japanese Army Administrative Code. The technical operation of the code and the ultimate breakthrough into the indicator system are detailed. The process of decoding and translating messages is described, and the memoir concludes with comments on the influence of cryptanalysis on the prosecution of the war in the Far East.

KEYWORDS: Japanese Army Administrative Code, solution problems, 1943 breakthrough, decoding messages, impact of cryptanalysis.

* Fig.
Legion of Merit ceremony for Lt. David Mead at Arlington Hall, 1945. (l) General Preston Corderman, Commanding Officer, Arlington Hall, and (r) Lt. David Mead.

In the summer of 1942, World War II occupied the mind of the American people and was requiring more and more of their energy. The Allies were preparing for the invasions of North Africa and of Guadalcanal. I was a graduate student at Ohio State University and had just completed the general exams for a doctorate in English; my dissertation, however, would have to wait a few years. Like many other young people, I decided to put aside my studies and to try to help in some small way in the war effort.

The armed services were expanding that summer at an unprecedented rate and had numerous openings for civilian specialists in language, mathematics and other fields. Although I had no specific knowledge of cryptology, I had one of the cryptanalyst's essential traits — an infatuation with solving puzzles, whether they were acrostics in the old Saturday Review of Literature or problems in identifying Middle English dialects in a medieval seminar. My years of literary study may also have given me the illusion that solving enemy codes would be a life of romantic adventure.

In any event, I wrote the Army Signal Corps, detailing my biography and aspirations to become involved in cryptography, as we then called cryptology. I soon received an offer to come to Washington as a research analyst in the Signal Intelligence Service. After a brief FBI investigation confirming that I was sound in character and innocent in politics, I reached Washington and was assigned to a four-week training course in the Munitions Building. The small class was supervised by Evelyn Akeley, an able teacher who immersed us in the cryptographic manuals of William F. Friedman, the eminent cryptanalyst who later gained fame for his part a few years before in solving the Purple diplomatic cipher. He was for many years technical chief of the Signal Intelligence Service. I met Friedman on a couple of occasions, but never really knew him. He was an impressive figure, a neat, urbane man who seemed in a nervous hurry to get on with whatever was engaging his restless spirit.

I still recall that August in Washington, partly because it was so oppressively hot that Akeley fed us salt tablets, but mainly because of the excitement I experienced solving Friedman's problems. Most of them were ciphers, using mixed alphabets with keywords, irregular form and other concealing devices. After working on them all day, I took them home at night, so that my meals and sleeping hours became as irregular as Friedman's alphabets. But I made satisfactory progress, although the ciphers which concluded the course were ingenious in construction, and their solution was painfully slow. Among other things, they taught me that patience is not merely helpful to the cryptanalyst, it is critical.

I was next assigned to B Branch, the intelligence section of S.I.S., where worked in the cryptanalytic unit under the direction of Major Solomon Kullback, an expert mathematician and cryptanalyst. Kullback had been recruited by Friedman in the early years of the S.I.S. He was a blunt, forceful administrator, well respected for his organizational skills. S.I.S. soon outgrew its quarters in the Munitions Building. By the time I undertook my permanent assignment, S.I.S. had acquired a new home at Arlington Hall, a former school for girls in the Virginia suburb across the Potomac. The Hall, or "The Salt Mine," as we often called it, became the nerve center of American signal intelligence, the army counterpart of England's Bletchley Park, where British code operations were housed. When I arrived in Arlington Hall, it had a few hundred workers. Before I left in 1946 it had several thousand.

Kullback placed me in a section of about 15 cryptanalysts supervised by Frank Lewis, a small, wiry civilian with astonishing nervous energy. A former music teacher from Maryland, he was also gifted in mathematics. Lewis generously taught me his cryptanalytic skills. I also learned a great deal from Lieutenant Karl Elmquist, a linguist from Texas A&M. He was an innovator, who liked to try unusual approaches to cryptanalytic problems when conventional methods proved fruitless. I worked most often with Elmquist, but I had the advantage of knowing and working with other talented cryptanalysts. Among them was Eli Schuman, a fine mathematician. Like Elmquist, he was a good chess player. I often ate lunch with one or the other while we played chess on a portable board. They both beat me regularly. Other able cryptanalysts included Sam Snyder, a mathematician, and Delia Sinkov, wife of Lieutenant Colonel Abe Sinkov, cryptanalytic officer for U.S. Army Forces in the Far East. I was fortunate in having the opportunity to work at various times with them, as well as with other capable people in my code unit.

The major focus of our activity in the fall of 1942 was the Japanese Army Administrative Code, a machine-encoded system of four-digit groups layered over with additive to disguise the code text. We called the code the 7890 (seven-eight-nine-oh). I am not sure who coined the term, though I first heard it from Frank Lewis; nor do I recall that it had any special meaning beyond its convenience. The 7890 emanated mainly from Tokyo, although it was used wherever the Japanese had army units with radio communication facilities. The 7890 intercepts were sent to the Hall's IBM unit, where they were bound in thick sheafs of printouts with the messages arranged chronologically. These were the raw material for cryptanalysts to study. The volume of messages was so heavy that, even before the code was broken, the Signal Corps' skill in traffic analysis and radio direction finding sometimes furnished clues to Japanese troop positions, impending operations and other matters useful to military intelligence.


During the next few months I learned why Arlington Hall was called "The Salt Mine." Through repeated trial and error we tried to find a way of penetrating the Japanese code. (We knew from contacts with liaison officers at the Hall that cryptanalysts in the Far East and in England were sharing our difficulties.) Our group occupied a spacious room where each of us had a desk, file cabinets and a long table where we could spread out messages. Both our work-station environment and even our methodology seem somewhat crude by modern standards, but in 1942 the electronic explosion had not yet occurred. Instead of computers we had piles of graph paper and pencils that could write in four colors. We were confident that the encoders were using an additive book of four-digit numbers and that somewhere in the messages there were indicators informing the recipient how to decode them. Of course the Japanese could increase their security by periodically changing the indicator system, the additive tables, or even the codebook.

One attack on the system was to search for code groups or paired digits that could serve as indicators pointing to messages that would overlap because they used the same additive numbers. If the encoders were using whole groups as indicators, this approach would be feasible. However, if they were using scattered pairs (as they were), the possibilities of success would be more remote. In this approach, as with others, we sometimes found interesting correspondences that encouraged us but that then, perhaps after lengthy analysis, proved meaningless. The cryptographer's random numbers often function in nonrandom ways that promise hope but end in disappointment.

The opposite strategy was to begin with encoded messages and work backward toward the indicator system. The Hall's IBM unit could provide us with message printouts and make a machine search for repeated groups that formed some kind of pattern. We tried various strategies, such as bunching messages from a single source or within a narrow time span. Identical groups in separate messages were clues for careful study. They were nearly always random coincidences, but occasionally the correspondence of a second group would encourage us to believe we had a true overlap. However, we did not have enough evidence to identify the indicator system.

Around the beginning of 1943 some of us stopped working in pairs or small groups, and I set out on my own. Maybe our nerves were beginning to show. Even before going to Arlington Hall, I had had my share of meetings with failure, but none so protracted as my experience with the 7890. (Fortunately my wife came to Arlington from Columbus, Ohio, and stayed for the duration. She was a welcome distraction from my cryptographic misfortunes. We luckily found an apartment near the post, and since she was an experienced social worker, she was soon happily employed in Washington.)

In February my life took another turn. Almost from the time I arrived at Arlington Hall, I felt somewhat uncomfortable not being in the Army Signal Corps. I worked daily among people in uniform, and some of them had become good friends. It is hard to define this longing to be part of the armed services in wartime, this feeling that somehow you belong there. It is a little like being homesick for a place you've never been. Anyhow, I wrote my draft board in Ohio and asked to be drafted, requesting also that I be inducted in Virginia. The board obliged; they did not care where I was inducted as long as they got credit for one more draftee. So in a few weeks I entered the army in Charlottesville as a private.

Northern Virginia can be cold in February, and at Camp Lee, where I was sent for processing, I found cold, drizzly weather and an outbreak of pharyngitis along with other draftees I was kept busy washing dishes and shoveling gravel while I awaited orders. Since Major Kullback had requested my assignment to his code unit, I anticipated a prompt release from Camp Lee and a rapid return to Arlington Hall. What I got was a bad case of pharyngitis and a slow trip to the hospital at Fort Myer. A few days later, however, I was fully recovered and back at my desk in the Hall.

My work was much as before except that some of my time was given to military exercises such as weapons training, drill sessions and special functions of the Second Signal Service Battalion, which intercepted messages for cryptanalysis. I enjoyed these activities. They were good for camaraderie as well as my health, both of mind and body. In March I received a corporal's rating (Technician 5th Grade), which was not very impressive at a post like Arlington Hall Station, but was progress.

The first signs of a breakthrough in the Japanese Army code came in mid-April. I had asked the IBM unit on numerous occasions for printouts of messages sorted on specific code groups in selected positions in the text. One of these revealed a pair of messages sent on the same date from Tokyo and apparently beginning with the same additive numbers. This could happen by chance, but more likely a Japanese operator carelessly or inadvertently sent almost identical messages without shifting to a different starting point in the additive table. The number of hits, or matching groups was much greater than random. Further IBM runs on these matching groups uncovered two more messages, neither sent from Tokyo, using the same table, but with different starting points. By close study of these overlaps I made a tentative identification of digits used in the indicator system.

Although I felt reasonably confident about these findings, I did not want to report them without strong confirmation. I requested a new IBM run based on the apparent indicator positions. Then unfortunately I had to delay the project for a brief time because my name was posted in the orderly room for KP duty. Privates and corporals were so few at Arlington Hall that they, and often sergeants, were expected to take their regular turn in the mess hall. Sometimes I was the subject of joking remarks that I was better at breaking dishes than breaking codes.

The new IBM printouts disclosed several overlaps that supported my conclusions about the indicator system. These were that eight indicator digits, which were scattered in the message heading as well as in the message text, represented the following:


BPPS RCRC

The B indicated which one of the ten additive books (1-0) was used to encode the message. PP revealed the page (01-00) in the 100-page book used to begin the message. The S was a sum check. For example, if BPP totaled to 23, S would be its last digit, or 3. The first RC marked the row and column on the additive page where the encoding began. The final RC denoted the row and column where the message ended.

I first showed these findings to Elmquist. He was an able cryptanalyst; moreover, I knew that if he found my solution to be flawed, he would react with understanding and useful criticism. More than once he had eased my depressed feelings with friendly encouragement. After studying my results briefly, Elmquist responded excitedly and hurried me off to report the news to Kullback. Unluckily, he was not available, as he was holding a staff meeting. To my dismay, Elmquist grabbed a sheet of paper and wrote in large block letters

WE'RE READING THE 7890!

Then he brashly shoved this premature announcement under the door. I never learned Kullback's immediate reaction to this interruption. Maybe I was afraid to ask. But he emerged from his office in about an hour and wanted an explanation of the note. After our brief account, he assigned Elmquist and Sam Snyder, an experienced cryptanalyst, to check my findings further and report back to him. They reviewed them carefully, and together we returned to assure a relieved Kullback that the Japanese Army Administrative Code had finally been broken.

As I walked the two miles home that day, I tried to sort out my feelings about the breakthrough. Relief, quiet elation, even a tinge of disbelief all combined in a kind of emotional tangle. I wondered, of course, what influence the breakthrough would have on the solution of the code. Most of all, I felt a bond, a sense of community, with the Allied forces who were enduring the hazards and hardships of the war in the Pacific.

The breakthrough apparently came on April 30, 1943. (Truthfully, I do not remember the precise date, but on May 10 I received a letter from the Commanding Officer at Arlington Hall Station, Colonel [later General] Preston Corderman, which began: "I wish to extend to you my commendation for your very important accomplishment of April 30….")

Kullback told me, along with some other military and civilian personnel, to report to the post that evening. It was a night to remember. Besides the Hall's headquarters officers, officials from the Military Intelligence Service and other Washington agencies attended the "celebration" of the long- awaited breakthrough. There was brisk discussion among them about it and its implications for their particular offices. I had little share in this, but sat at my desk observing the dignitaries and feeling somewhat out of place among them. Several came by to see the "evidence." On a table I had spread out several overlaps and circled the indicator digits with a red pencil. Most people looked at the exhibit like museum-goers scanning the Rosetta stone, not really understanding it, but realizing it was important.

The rapid decoding and translation of the Japanese messages was achieved mainly by a group of about 50 specially trained Signal Corps enlistees who had studied Japanese, mostly at Columbia, Harvard and Yale. They were supervised and given further intensive instruction in Japanese military language by Captain Edwin Reischauer at Arlington Hall. These talented students could read Japanese messages almost like natives. Reischauer later became a foremost advocate of U.S.-Japanese understanding, especially as our ambassador to Japan. The solved and translated messages of his group went to intelligence assessors and other military intelligence personnel for use in the prosecution of the war.

Reconstruction of the Japanese additive tables and codebook was aided not only by increasing traffic, but also by some curious practices of the Japanese themselves. Sometimes messages used stereotyped patterns or commonly repeated phrases. For example, Japanese syllables with the same pronunciation but different meanings were often identified after a code group meaning "open Chinese telegraph." Several groups later would come the group for "close Chinese telegraph." Decoders were adept at finding and interpreting such patterns on overlaps. Even more helpful was the carelessness of Japanese signal officers when they changed their additive books, usually every few months. Apparently they had complete faith in the security of their code and thought we were incapable of breaking it, because they regularly sent their new additive tables to their stations throughout the Pacific area by using the old tables to encode them. As a result, we received their new additive books at the same time their cryptographers did. Probably most successes in military cryptanalysis benefit from the careless mistakes of the enemy. (I can recall a couple of rare cases in which the Japanese operator, apparently suffering from burnout or too much sake, omitted the additive and sent the raw code undisguised.)

By 1944 virtually every message between Japanese stations was being intercepted, and useful information such as the date, place of origin and destination of messages was being recorded on IBM equipment. On a high hill somewhere in America there ought to be a lasting monument to IBM and its creative personnel for their invaluable contribution to our success in breaking enemy codes in World War II. For all branches of the service, they supplied the materials, often ingeniously arranged and sorted, that aided the work of cryptanalysts. (I have sometimes wondered whether I could ever have broken into the Japanese Army Administrative Code without their help. It certainly would have taken a lot longer.)


The enormous volume of Japanese messages sent in a variety of codes and ciphers actually became a problem as well as an asset to the Military Intelligence Service. In describing "the importance of the 1943 breakthrough into the Japanese Army's signal traffic," the historian Ronald Lewin noted that at the direction of General George C. Marshall a "Black Book," summarizing intelligence information from the mass of intercepts, was sent daily to the White House. In a letter to President Franklin D. Roosevelt, Marshall wrote: "The recent discovery of the Japanese Army machine code has added a tremendous amount of such material…. The problem is how to avoid being buried under the mass of information, and I think the present arrangement satisfactorily meets that difficulty."1

1The American Magic: Codes, Ciphers and the Defeat of Japan (New York: Farrar, Straus and Giroux, 1982) p.199. The entire letter is reproduced on p.201. I have frequently relied on Lewin's well-researched study for historical details about which I have little or no personal knowledge. The word "Magic" was a general term used to designate secret intelligence in both the European and Far East Theaters. The naval and military signal traffic of the Japanese was known as "Ultra."

A high proportion of Japanese Army intercepts did not qualify for Roosevelt's "Black Book" because they dealt with routine matters, such as local orders and reports. However, many messages had important operational value for Allied commanders. They provided information about the disposition and strength of Japanese forces, as well as troop movements and their strategic intentions. Other intercepts related to the enemy's shipping organization and the stocks of oil and other resources necessary for defense of its island garrisons. The sifting of all this material for useful intelligence was a prodigious task which required increasing numbers of intelligence assessors and other skilled personnel at Arlington Hall. "In July 1943, B Branch, the intelligence wing of the Army's Signal Security Agency [a later designation of the S.I.S.], numbered 1,804. By July 1944 it had expanded to 2,574, of whom 82 percent worked on the signal traffic of the Japanese Army."2

2Lewin, p. 198. The S.I.S. in 1942 became the Signal Security Service and was renamed the Signal Security Agency in 1943.

To illustrate the staggering volume of messages to be evaluated, we had a "show but don't tell" exhibit of overlaps which extended from the ceiling to the floor - overlapped messages eight feet deep using the same additive page. The display also illustrated how messages using additive could be read vertically before they were read horizontally.

After the breakthrough into the Japanese Army code, I found my later experiences at Arlington Hall somewhat anticlimactic. For a while I worked on overlaps to recover or confirm additive numbers. I was also assigned to instruct new personnel in my cryptographic section. By early 1944 I had been promoted to Technician Third Grade (which finally released me from KP chores, though I never found them very onerous). I spent the summer in Officers' Candidate School at Fort Monmouth, New Jersey, and was commissioned a second lieutenant in the Signal Corps in July 1944.

During the rest of my stay at Arlington Hall I had various supervisory assignments, but little opportunity for meaningful cryptanalysis. We also solved a few minor codes, but the Japanese Army Administrative Code dwarfed them in volume and importance. For several months I worked on a three-digit army code. Its solution was not especially difficult, but message traffic dwindled and was insufficient to warrant a serious effort to recover the additive and code books.

In the final stages of the war, when the Japanese empire was collapsing in the face of Allied assaults, my work in cryptanalysis virtually ended. In mid-1945, at the request of now-Lieutenant Colonel Kullback, I wrote a detailed analysis of the Japanese Army Administrative Code and its operation. I suppose it is somewhere in Signal Corps records. Later in the year, at a ceremony in Arlington Hall, I was awarded the Legion of Merit.


My final assignment was writing citations for military officers ending their wartime service. (The practice would prove useful later in my academic career.) In the early spring of 1946 I was promoted to first lieutenant, and in May I was discharged at the Pentagon. A week later, together with my family (which now included infant twin daughters), I returned to Columbus to write the doctoral dissertation (a study of 19th-century New England lecturers in the Middle West) that I had put aside four years earlier.

It would be difficult to assess the significance of the 1943 breakthrough and its influence on the war. Such judgments are better left to scholars and historians whose knowledge of the war's events and personalities is much greater than mine. Several observations, however, can safely be made. The Japanese Army Administrative Code was one of a group of cracked enemy codes and ciphers, all of which had their special importance. Among them were diplomatic and navy ciphers and a water transport code, broken in early 1943, that traced the activities of the Japanese Army's own fleet of supply vessels. The war in the Pacific was mainly a maritime conflict, and the dramatic breakthrough into the main Japanese Navy code by Captain Joseph Rochefort in the spring of 1942 was of paramount importance.3 Intelligence gained from the code was largely responsible for the American victory at Midway, and for many other successful attacks on Japanese-held islands. The solution of the main Japanese Army Air Force code in early 1944 added to the flood of information available to Admiral Chester Nimitz, General Douglas MacArthur and other Allied commanders.4

3The mistreatment of Captain Rochefort and the struggle in the Office of Naval Communications in Washington to claim credit for his achievement are narrated in a number of war histories. A first-hand account is given in the memoirs of Rear Admiral Edwin T. Layton, "And I Was There" (New York: William Morrow, 1985). Layton, the Pacific fleet's Intelligence Officer, died in 1984, and his book was completed by Captain Roger Pineau and John Costello. Captain Rochefort, who died in 1976, was awarded the Distinguished Service Medal posthumously in 1985.
4Unlike Nimitz, Marshall and other leaders, MacArthur disliked agencies "whose responsibility did not run directly to himself," and was rarely willing to share credit for his striking exploits with military intelligence, though the intelligence gleaned from the Japanese Army Administrative Code and others must have been invaluable to him. In 1961 MacArthur was given an honorary Doctor of Laws degree at Michigan State University, where I was the English Department chairman. The citation which accompanied the degree was a well-deserved tribute to his military successes in the Far East. I enjoyed writing it.

In the end, of course, the war in the Pacific was not won by cryptanalytic intelligence. It was won by superior strength on land, at sea and in the air, by astute military and civilian leaders, by a supportive civilian population, and by an industrial complex that could turn out five Liberty ships every day. Still, cryptography had its impact. In assessing the navy and army code breakthroughs of 1942 and 1943, Ronald Lewin wrote that, had they not been achieved, "the war in the Pacific might not have been lost, but victory would have been incalculably more expensive in time, effort and blood."5

5The American Magic, p. 18.

In the years after the war I seldom thought about the Japanese Army Administrative Code or my experiences as a cryptanalyst. My civilian life was crowded with new interests and events. Moreover, I was so conscious of security that I avoided speaking to anyone about Arlington Hall. Occasionally there were unexpected reminders. For example, while waiting for a traffic light to change, I would be startled to see a half- remembered number, like 2042, on an automobile license plate. That one was the code group for "owari," meaning "end of message." I suppose that everyone who has been involved in cryptography is at times pursued by ghosts of the past. Now, like other people in their retirement years, I give more space in my life to reminiscence, and perhaps even nostalgia. When by chance I meet a veteran of the war in the Far East, I cannot help wondering whether, in some cryptic way, our lives may have touched a half-century ago.

BIOGRAPHICAL SKETCH

David Mead is a retired Michigan State University professor. In 1955-57 he directed an advisory group at the University of the Ryukyus, Okinawa, where he developed English language and literature programs. In 1964, as a Fulbright lecturer in the Philippines, he established the American Studies Association of the Philippines with his seminar students as its charter members. He is the author of numerous scholarly articles and books on American culture and co-author of the Prentice-Hall Handbook for Writers, a college text now in its eleventh edition. He was English advisory editor for Dodd, Mead & Company (1963-75) and editor of The Centennial Review (1966-81).


The Story of the Hagelin Cryptos

Boris C. W. Hagelin
Edited by David Kahn

INTRODUCTION

"The Story of the Hagelin Cryptos" could never have been written if Russia after the revolution had become a democratic republic. The expected defeat of the Bolshevik minority and the return of a moderate government would have caused me to remain an employee of the Nobel Brothers Oil Production Company, and there would have been no Crypto AG - the present manufacturer of ciphering machines.

I was born on July 2, 1892, in Adshikent, a small summer resort in Azerbaijan. My father, a Swede who had been born in St. Petersburg in 1860, was at that time manager of the Nobel company's oilfields and refineries in Baku. Ha had joined the company in 1879 when it was founded by an older brother of the famous Alfred Nobel. My father was made a director of the company in 1899, and moved with his family to St. Petersburg. I went to a Russian school until 1904, when my father sent me to a Swedish school. After finishing this school I entered the Royal Technical University in Stockholm where I graduated in 1914 with a degree in mechanical engineering. It was taken for granted that my future would lie in Russia with the company, where my father held a leading position and had become the closest friend of Emanuel Nobel, head of this famous family.

My first job would have been the supervision of the construction of an electric power station on one of the Nobel oil fields in Baku. As I was a mechanical engineer, I had first to broaden my knowledge of electrical engineering. Nobels had ordered the equipment from the large firm ASEA in Vasteraos, and therefore it seemed to be a good idea for me to work in different departments of that firm for a year.

However, the Russian Revolution broke out, and it became obvious that the project would have to be delayed until normal conditions returned. I thus stayed on at ASEA in their foreign department since I spoke five languages.

In 1920 the Nobel family made an agreement with the Standard Oil Company of New Jersey for a future collaboration in Russia, and I was sent to the USA where I worked in their general engineering department during 1921. At the end of that year it became clear that stable moderate conditions in Russia could not be expected under the rule of the Bolsheviks, who had in the meantime confiscated all private enterprises, including the Nobel works.

I could have stayed with Standard Oil, but I did not feel at home in America and wanted to get back to Sweden. There was no future for me in the oil business but my ties with Emanuel Nobel remained unbroken. The Nobels managed to retain their non-Russian businesses, but had to live in exile, like my father.

Emanuel Nobel was very generous towards me and financed the establishment of a small engineering office in Stockholm. During my stay in the US I had acquired some inventions, which I developed and which made me financially independent. The decisive turning point in my life came, however, when Emanuel Nobel entrusted me with the supervision of a small company which he had begun to finance in 1921 - Aktiebolaget Cryptograph. This company was founded in 1915 with the objective to develop and manufacture ciphering machines invented by the Swedish engineer Arvid G. Damm. In 1925 I assumed the management of the company as well as the development of saleable products. This was a fascinating task although I did not have any knowledge of cryptography. Mr. Damm died in 1927. In 1932 A. B. Cryptograph was liquidated and replaced by A. B. Cryptoteknik.

A. B. Cryptoteknik manufactured only mechanical and electromechanical ciphering machines. After World War II the need for ciphered telegraph transmission became obvious. In order to be able to work without the interference of the Swedish government - ciphering machines were at that time considered war material - I decided to move to Zug, Switzerland. I first collaborated with the Swiss inventor Dr. E. Gretener, but later established a small independent laboratory. Crypto AG (Aktiengesellschaft) was incorporated on May 13, 1952, and had at first just one employee. My Swedish activities were transferred to Crypto AG, and since the name "Hagelin Cryptos" had become well known before World War II, the enterprise grew so fast that in 1966 a new manufacturing and administration building was built in the Steinhausen section of Zug.

The basis for the growth of Crypto AG was the mechanical machines. which were conceived and developed in Sweden - the original "Hagelin Cryptos." The first products of Crypto AG were the "Telecryptos". They were succeeded by the electronic ciphering equipment which, since the mid-1970s, has become predominant. The old mechanical machines are, in decreasing quantities, still being produced. Without their basis, however, the new developments which have culminated in a large number of different versions would never have evolved. The trade name "Hagelin Cryptos" has therefore been applied to all the products of Crypto AG. It would be presumptuous for me to claim the credit for the success of Crypto AG; however, it was internationally acknowledged as the most important manufacturer of ciphering equipment before I retired from the company in 1970.

The art of invention - and I have mainly become known as an inventor - has for me been sudden ideas or the combination of existing designs for totally different purposes. But the hard detail work to obtain the finished products has been performed by many collaborators, for all of whom I retain a sincere thankfulness. In this context I feel that I must mention several names. First, C. A. Lindmark, who became indispensable to me when I developed my best known machine, the C-type, in 1934.

My son Boris Jr., who met a tragic death in 1970, had an inventive mind, and his contributions when the postwar CX-type was designed were very important. Oskar Stürzinger, my first employee when I began my activities in Switzerland, developed my original ideas for the Telecrypto machines. He also began to experiment with electronic ciphering devices, at first against my wishes, as this new technology was alien to me. To achieve success, however, it is not enough to have an excellent product but also superior management. Here, Sture Nyberg, who was first the manager and then the director of Crypto AG until his retirement in 1976, proved to be the ideal choice. He also contributed by analyzing the security of the machines which were produced.

In closing this introduction I must pay my humble respect to Emanuel Nobel and to my father, without whose financial and moral support during the long and not always easy years of my work with ciphering machines I would have remained an insignificant engineer.

- B. C. W. H.


I. THE BEGINNING

The use of cryptographic methods of various kinds stretches back some thousands of years. A few hundred years ago inventive minds began to make devices to facilitate the process of enciphering and deciphering messages. In 1891 the Frenchman Etienne Bazeries invented a device, which to the best of my knowledge was still used until World War II. The principle used by Bazeries had been propounded earlier by Thomas Jefferson for the construction of a cipher device, but his writings were not discovered until after World War I.

In Sweden too, interest existed very early in the use of mechanical devices for secure communication. My old friend Sven Wäsström, a Swedish cryptologist, has discovered in the national archives in Stockholm some very interesting documents on this subject. In 1786 Baron Fridric Gripenstierna received permission from King Gustav III to make a cipher device in accordance with Gripenstierna's drawings. In his letter to the King, Gripenstierna indicated that he had gotten the idea for the device from his father-in-law, the celebrated inventor Christopher Polheim. The machine itself, unfortunately, has been lost, but the invoice by the builder, which is dated August 26, 1786, shows that the device was actually built. A remarkably detailed description allowed Crypto AG to produce a replica of this machine.

This device displays certain resemblances to the one developed much later by Bazeries but far surpassed the French device as far as security and method of operation are concerned.

Gripenstierna's device consisted of 57 disks, each of which carried two alphabets. These disks, mounted on an axle, could all be rotated independently of one other. A longitudinal slit with an index guide on each side of the device exposed two sequences of letters. The authorized person who was to perform the enciphering (at the right in Figure 1) adjusted the disks during enciphering so that the plaintext appeared row by row in "his" slit. The assistant (at the left in Figure 1) then copied down from the other slit the ciphertext, again row by row. During the decipherment the assistant adjusted the disks to show the ciphertext row by row in the slit on his side. The authorized person then copied down the plaintext row by row from his slit. Here we have a modern security concept which separates the ciphering process from the transmission of the cipher.

* Fig.
* Fig. Figure 1. Cipher machine of Gripenstierna

In the 19th century, more and more inventors concerned themselves with ciphering devices and machines. In the German patent office, which was founded in 1877, about 150 patents were issued before 1927 for cipher devices.

In this list the name of a Swede appeared in 1915: Arvid Damm. He was able to interest Swedish industrialists in his inventions and A. B. Cryptograph was founded in 1915 to exploit his inventions.

His first fundamentally sound machine is shown in Figure 2. He used a revolvable drum, on which 26 alphabet strips could be fixed in any desired order. The alphabets were formed so that they could be considered to be a scrambled Vigenére square, with the alphabets in reversed order. Close to the alphabet drum a reference alphabet in the normal order (A . . . Z) was mounted.

* Fig.
* Fig. Figure 2. A-21 by Arvid G. Damm

When ciphering, the drum would be advanced one step for each operation, while the reference alphabet could take one of two positions. Through a slit at the top of the machine one could read off the letters to be ciphered from the reference alphabet to one of two drum alphabets. The positions of the reference alphabet were controlled by the low and high links of the chain, which advanced one step for each operation.

The length of the chain and also the position of low and high links could be varied within certain limits. These options as well as the possibility of changing the relative positions of the drum alphabets made it possible to obtain a large number of different cipher series. The use of reciprocal alphabets made it easy to use the machine, as the reading from the reference alphabet could be done both when enciphering and when deciphering. In emergency cases the machine could be opened and the chain disassembled into the individual links, thus destroying an important key element.

Using the same principle, Damm built a number of office machines equipped with keyboards. When a key was pressed, a letter would be exposed below the alphabet strips. During encipherment, it would be a ciphertext letter; during decipherment, the plaintext letter (Figure 3).

* Fig.
* Fig. Figure 3. Office machine by Damm (cover removed)

Here too the chain was used as the controlling element. In operation the machine had a protective cover.

Damm also constructed another purely mechanical machine which printed both the plaintext and the ciphertext. Four of these machines were sold to Japan. Finally, Damm invented a system with so-called rotors, i.e., alphabet permutating wheels, an invention made at almost the same time by the American Edward Hebern, the Dutchman Hugo Koch, and the German Arthur Scherbius.

A normal rotor at that time consisted of a disk that had on each of its two faces 26 metal contacts embedded around the circumference. These contacts were connected in pairs through the inside of the disk in a random fashion, one contact on one face connected to one contact on the other face. The best-known machine using rotors was the German Enigma, equipped with 3 or 4 rotors which were set to a new starting position for each message in accordance with the keying instructions. The machine was provided with a keyboard and a lamp field. Each lamp was arranged to light up one letter. To operate the machine only a flashlight battery was needed.

Damm's rotors were constructed somewhat differently from the Enigma's but produced a similar effect. They were made for machines which were intended for the ciphering of messages transmitted by radio telegraphy, because such messages could easily be intercepted.

Damm aimed to interest primarily the large telegraph companies that handled transatlantic telegraph traffic in particular. With such machines the radio traffic could have been just as secure as the cable traffic.


II. THE FIRST MACHINES

When in 1921 the capital of A. B. Cryptograph was exhausted, none of the stockholders would or could make further funds available. However, at this point Damm, the director of the company, succeeded in interesting Dr. Emanuel Nobel in the A. B. Cryptograph and its possibilities. Because of the Revolution the Nobels had lost their holdings in Russia: a large factory and the largest Russian oil firm, the "Nobel Brothers Oil Production Company".

With regard to the A. B. Cryptograph, Dr. Nobel consulted his colleague and close friend Karl Wilhelm Hagelin, my father, and they came to the conclusion that the possession of good cipher machines for written communications could play a decisive role in business affairs. Based on that conclusion Nobel decided to invest new funds in the A. B. Cryptograph, and my father also took a small part in the company. He was technically interested as he was an engineer himself.

Since Damm had a rather obstinate personality, Nobel and Hagelin needed a trustworthy person to monitor his activities. Because I was then living in Stockholm where I had opened my small engineering office, Nobel asked me to watch over A. B. Cryptograph.

When I joined the company, Damm's interests in cipher machines were concentrated in the field of radio telegraphy. When I first appeared at the company offices a complete layout was on display: a typewriter for the input of text, a cipher machine, and a typewriter equipped with electromagnets for the output of enciphered or deciphered text. Damm had succeeded in arousing the interest of the major radio telegraph companies in his machine and prototypes were to be built by one of these companies in Paris. After I joined the firm Damm moved to Paris, and I had to look after the business side of the firm and its technical development more than I had originally expected.

In the end Damm succeeded in winning the "big four" to his project: Marconi, Telefunken, TSF and Western Union. They financed the construction of four prototypes. But these did not appear to be reliable enough and were too slow, so the project failed.

In the meantime, however, Damm had developed a system with so-called simplified rotors, and some prototypes of these were built for wireless traffic. The machines consisted of three units: The keyboard, the ciphering portion with motor drive, and the output unit - either an electromechanical tape punch (a Creed Morse code punch), which was equipped with a magnetic control or an electric typewriter. Figure 4 shows an electric typewriter, keyboard and ciphering unit. This machine was Damm's Electrocrypto B-18.

* Fig.
* Fig. Figure 4. "Electrocrypto B-18" by A. G. Damm

The principle of the simplified rotor later became the salvation of the faltering business. But this came only after Damm's death in 1927.

In 1925 I was able to make my first positive contribution to the A. B. Cryptograph, and this proved to be of decisive importance for the future of the company. I happened to hear that the Swedish General Staff had received an Enigma machine for study and I rushed to visit the officer concerned with this matter. I explained to him that the A. B. Cryptograph already had ten years of experience in the field of cipher machines and that I would be able to offer something possibly superior to the Enigma.

The General Staff wanted our machine to be the same size as the Enigma and to operate in the same way. Time was short and I had only six months to produce the new machine. At the time we had only Damm's earlier constructions on hand, and they did not fulfill our needs. Nevertheless I promised to deliver. In those days I had no experience whatsoever in cryptography but I had a certain talent for tinkering: I believed I would be able to build a comparatively compact machine on the basis of Damm's "simplified rotors". This design made use of a 5 x 5 grid, whose principle is shown in Figure 5.

* Fig.
* Fig. Figure 5. 5x5 matrix

To build the first model of the machine Nobel allowed me 500 kronor, then about $134.00(!) I succeeded in producing a prototype within the size limitations and in the time authorized, a somewhat primitive model but still adequate for the evaluation. This machine had a keyboard, 2 rotors whose stepping was controlled by two separate pairs of pinwheels (keying-wheels) with different divisions on their periphery, and a display of 25 electric lamps which served to indicate the output letters for encipherment or decipherment (Figure 6). This prototype was analyzed by a mathematician, and approved instead of the Enigma for use by the General Staff.

* Fig.
* Fig. Figure 6. B. C. W. Hagelin'Ls B-21 (prototype)

Because Damm used a 5x5 grid, it was necessary to eliminate one letter of low frequency or to use the same signal for two letters, e.g. for i and j. The keyboard was connected to two groups of five contacts each. By pressing a key, one contact was closed in each of the two groups, forming one of the 25 possible combinations. After the closing of two circuits through the two rotors, and a further circuit through a relay system, one of the 25 electric lamps lit up and indicated the resulting letter. The relay system was necessary because at that time there were as yet no devices like modern diodes, which would have simplified the construction considerably. In order to be able to vary the enciphered alphabets, so-called modificators were added in series with the rotors, i.e., interchangeable leads which could be plugged in as desired. This provided 5! = 120 combinations for each group and for both groups together yielded 14,400 possibilities.


The designation "pinwheel" mentioned above needs to be explained since the pinwheel came to be used later in many other machines. It is a disk that carries on its periphery a number of axial holes or slots in which pins are located. These pins are so arranged that they can protrude on one side or the other of the pinwheel. One side is the "active" side, i.e., pinsthat protrude on this side exert a controlling effect. The pinwheels are stepped with each operation, each wheel moving onepin position during each operation. The cyclic lengths of thewheels have no common factors. Thus as several wheels worktogether, a very long period for the key is obtained.

The prototype had four pinwheels, with cyclic length of 17, 19, 21, and 23, which yielded a combined keying cyclic length of about 1.5 x 105, or 150,000 operations.

By using different combinations of pin settings theoretically 217 x 219 x 221 x 223= 280 or approximately 1024 different period characteristics could be obtained. This meant that an enormously large number of variations was possible. The machine was given the name B-21.

Figure 7 shows the final model, in appearance quite similar to the Enigma. It used a power supply of 110 or 220 volts for a solenoid magnet which operated the stepping of the rotors. For the lamp display a flashlight battery was utilized.

* Fig.
* Fig. Figure 7. B-21 with solenoid magnets (series production model)

For the convenient use of this machine in central offices with extensive operations, it was connected to a Remington electric typewriter instead of the lamp display. Typewriter keys were activated by small electromagnets and the lamp display was shutoff.

III. THE B-211 MACHINE

After Emanuel Nobel died in 1932 his heirs did not want to make any more funds available for A. B. Cryptograph. I myself had been working without pay because the firm's income had been very meager. In spite of the limited possibilities, I was able to make several trips to other countries where I found buyers for the B-21.The decisive success, however, came from the interest of the French Army in our machine. Before the machines could be sold two requirements had to be fulfilled: the machines had to print the text and had to be portable. This called for the implementation of an electromechanical drive. I succeeded in solving this problem ina short time by replacing the lamp display with a special typewheel printing mechanism (Figure 8), which I had designed.

* Fig.
* Fig. Figure 8. B. C. W. Hagelin's B-211

Since the machine, which we called the B-211, was to be built in France, we got as manufacturer the French subsidiary of the Telefon - A. B. L. M. Ericsson, in Colombes just northwest of Paris.

The B-211 was provided with shift keys like those on a typewriter. During encipherment only letters were printed as output, but during decipherment letters, numbers and symbols were printed. The machine also had a crank so it could be operated by hand in case the electric power failed. For the ciphering circuits a flashlight battery was used, as in the B-21 machine.

Before the outbreak of World War II about 500 B-211 machines were built and delivered. It was through the financial support of my father, who lived in Paris at the time, that this business could be carried through. It would not have been possible without his encouragement and funds, and it was not easy for him.

When World War II broke out, my father at the last moment succeeded in transferring the profits from France to Sweden. The amount was large enough to pay his expenses and to install and equip a modern workshop. It was our first workshop since the founding of the A. B. Cryptograph in 1915. The shop was dedicated on New Year's Eve of 1940 by the heirs of the A. B.Cryptograph and was renamed the A. B. Ingeniörsfirman Cryptoteknik.

After the war the French army bought 100 more B-211 machines.

I also might mention that I was obliged before the war to sell"two cipher machines" to the Russian Trade Commission in Stockholm (according to the purchase order). I sold them two B-211 machines which we had in stock in Stockholm. They were copied (with changes) in Russia and used during World War II. They were provided with a 5x6 grid. The Cyrillic alphabet had more than 30 letters but a few of them occurred very seldom, and could be omitted. Also the modificators were plugged in outside of the machine, probably underlocked cover so that the operator would not know which inner settings were in use at any given time.


IV. THE C-MACHINES

In 1934, quite soon after the beginning of our business relationship with the French "Deuxiöme Bureau" I was asked if I could develop a compact pocket cipher machine that printed. As good luck would have it, I had already a few years before received from two Swedes an order to design a coin changing machine.

After I had built a functioning prototype, I got the idea that it should be possible to insert any amount in coins, to enter the invoice amount by a key, and finally to get back the change. This machine was equipped with a unique calculating mechanism but it was never built because we had not been reimbursed for our development effort. We agreed to cancel the debt and retain the rights for our designs.

* Fig.
* Fig. Figure 9. B. C. W. Hagelin's C-35 (closed)

Trying to find a solution for the pocket device for the French,I got the idea of incorporating the calculating mechanism for the money changer into such a device. The "revolutionary invention" (in the opinion of distinguished cryptologists) consisted in a pure association of ideas that were transferred from one field of application to a completely different one.

The calculating mechanism of the money changer consisted of a drum with displaceable axial bars around its circumference which could be affected by operating the keys. Those bars that were affected in an operation were displaced to the left as the drum turned a typewheel meshing with the drum was turned the same number of steps as the number of bars moved to the left. I replaced the keys of the money changer with pinwheels, and the typewheel carried letters now instead of numbers. In this way I obtained a cipher machine. In order to estimate the dimensions desired, I cut out a wooden block to fit snugly into the pocket of a jacket. For the first models I used a drum with 25 bars and five pinwheels of the same construction as those in the B-21. These wheels were provided with pins around their circumferences, which could be placed axially in two different positions, either "active" or"inactive". "Active" pins worked upon the drum by means of a control arm.

The bars of the drum were provided with cams, i.e., raised a reason their edges, of such a design that the first bar would be moved by the control arm of the first pinwheel, the next two bars by the second wheel, the next four by the third, the next eight by the fourth, and the remaining ten bars finally by the fifth wheel(Figure 10).

* Fig.
* Fig. Figure 10. B. C. W. Hagelin's C-35 (open) with Thai alphabet

Depending on the number and the arrangement of active pins in the pinwheel in question, the control levers were pushed against the periphery of the drum, or remained in their inactive positions.When the drum was revolved during an operation, the active levers pushed the corresponding bars (from 0 up to 25 bars) to the left and so engaged the gearwheel connected to the typewheel. The typewheel was therefore displaced a number of steps equal to the number of the active bars. The drum thus acted like a gear wheel with a variable number of teeth. As the numbers of pin divisions on the pinwheels had no common divisor, the length of the key period, i.e., the number of operations before the pinwheels returned to their common starting position, was very long.

For the first machines the divisions, or cyclic wheel lengths selected, were 17, 19, 21, 23, and 25, which resulted in a period length of 3,900,225 operations, a length which until then had never been achieved in any mechanical ciphering machine.

Moreover, since one could set all the pins theoretically in about 1029 different combinations, the number of possible variations was so high that these machines are fully adequate for certain purposes even today.

* Fig.
* Fig. Figure 11. B. C. W. Hagelin's BC-543

Later I made various changes in direct response to the steady progress in cryptanalysis, which included the significant work of the Swedish cryptologist Yves Gyldén. The main improvement consisted of increasing the number of pinwheels from five to six,and to introducing moveable lugs on the drum-bars instead of fixed cams.

When the machine began to be produced in larger quantities, a bottom plate and a protective cover were added. The bottom plate was formed so that the machine could be used in the field strapped over the knee of the operator. If necessary the operator could even march with the machine strapped to his thigh. This new type of apparatus was given the designation C with one letter and a numeral suffix which originally designated the model year but after 1952 defined the specific model of the machine: e.g., CX-52.

The opportunity to establish a modern workshop with the royalties from our French sales occurred just before the outbreak of World War II. It thus became possible for us to make and sella considerable number of machines during the war. Our first large customer was the Swedish defence forces, with whom I had maintained connections since 1925. From France we received an order for more than 5000 machines, which were to be manufactured by Ericsson at Colombes, but this order could not be completed.

* Fig.
* Fig. Figure 12. B. C. W. Hagelin's BC-543 (open)

To promote the C machine I made several trips, first within Europe, but in 1937 to the United States, where the wish was expressed to have an electrically powered machine with a keyboard.In the summer of 1939 1 took a prototype of the so-called BC machine to Washington. Unfortunately the construction needed improvement and I returned to Sweden that autumn after the outbreak of war with matters still unsettled.

* Fig.
* Fig. Figure 13. USA version of the C-36: M-209

I made another trip to the US at the beginning of March 1940 on my own initiative without any request from Washington. I was able to leave, with two machines in my luggage, from Genoa on May 10,1940, on the last ship from Europe before the Italians entered World War II. This trip was to lead to the largest sale of C machines ever made.

The American business began with an initial order for 50 machines which were shipped airfreight from Sweden to Washington.After extensive testing the machine was accepted. The Americans selected it for tactical use, as they did not have any comparable machine at the time. The C machine, designated by the U. S. Army the M-209, was manufactured by the Corona, New York, plant of the L. C. Smith typewriter company, with a daily output of up to 500 units.

I remained in America during the war and had a small shop in my home for servicing the BC machines that were used by an American organization during the war.

When it became possible for me to return to Sweden in 1944, more than 50,000 machines had been built. By the end of the war more than 140,000 units had been made in America.


During my four-year stay in the States, the workshop in Stockholm was kept busy with orders from several foreign countries.One delivery went by an extraordinary way to Japan. The machines were smuggled out by the Japanese military attaché in a night boat passage and were picked up by a U-boat off the coast of Sweden. But very few machines reached their intended destination in Japan.

It seems worth mentioning that the German authorities, who years before the war showed no interest in the demonstrated C machine,began, toward the end of the war, in the Wanderer works at Chemnitz to manufacture a copy of the BC machine for their own use (Figure 14).

* Fig.
* Fig. Figure 14. C-41 (German copy of the BC-543)

When the Third Reich collapsed it had been able to manufacture only about 700 machines. It is even more interesting to note that in the 1950s, a postwar variation of the C machine was built in Germany under license. About 10,000 machines are said to have been produced. France also obtained the manufacturing rights for the C machine for the francophone community.

It should be stressed that the C machine was planned for tactical purposes, i.e., for use at the front. When, because of its simplicity and portability, the machine also proved to be interesting for use in the diplomatic service, improvements were needed to attain a higher degree of security.

I got the idea first of modifying the machine so that the rotation of the pinwheels would be irregular. Other improvements were introduced as well. The new type brought no large changes in the basic structure of the original C machine. This new machine was given the designation CX-52; later, after different pinwheel movement variations were made, new model numbers were used.

Before going into the details of the various improvements, it is important to emphasize that the requirements of our customers led in two opposite directions. One group required very long key periods, while the other wanted a movement of the pinwheels that was as irregular as possible. It required great mathematical competence to compile keys that would assure that sufficiently long periods were obtained. In order to satisfy our customers we also developed hybrid systems where usually one pinwheel was stepped regularly and the others irregularly.

The further development of the C machine, which even today, in the age of electronics, is being used, can now be followed in some detail:

a) The number of pinwheels was increased quite early to six.Instead of the original five wheels with divisions of 17, 19, 21,23 and 25, 12 pinwheels became available with divisions of 25, 26,29, 31, 34, 37, 38, 41, 42, 43, 46 and 47, of which six were to be used at one time, preferably those with no common factors. The wheels were inserted easily without using a tool.
b) The number of slide bars was increased to 32. In place of the fixed cams on the bars which caused the typewheel to turn,movable lugs were installed on each bar so that the movement arrangement for the typewheel could be changed as desired. For other ways of using the machine, some bars with fixed cams which control irregular forward movement of the key wheels were manufactured.
c) Even in the very first C machines, reciprocal alphabets were utilized. With a letter sequence on the typewheel running in the opposite direction to that on the indicating disk, the machine could be used both for enciphering and deciphering without any modifications. In later models a second typewheel was introduced, which was permanently connected to the indicating disk, and which printed the same letters which were set with the indicating disk. They were called "primary" and "secondary" typewheels, where the primary typewheel is connected to the indicating disk. There are also available indicating disks and typewheels, in which the letters can be rearranged at will, and without removing them from the assembly.
One model printed the plaintext in the characters of a non-European language (e.g., Arabic) and the ciphertext in Latin letters, and one text (Arabic) read from right to left and the other text from left to right. This arrangement was indispensable for Arab countries using the international telegraph service.
d) Between the indicating disk with the primary typewheel and the secondary typewheel was a disconnectable clutch. It could either retain the chosen relative positions between the primary and secondary typewheels during a series of messages or change each time at the start of the next letter in the text.
e) For the easy setting of the pins on the pinwheels a special tool was provided. This tool set in a single operation all the pins in one wheel.
f) A special device which was developed at the request of one customer was the mechanization of the one-time pad system, which had been a slow hand procedure.
Near the end of World War I the American Gilbert S. Vernam made the first attempt to mechanize this system for telegraph use. He used for the key tapes punched paper tapes with random sequences of teleprinter signals. The characters of the plaintext in the standard CCITT No. 2 code were combined by means of relays with the key tapes according to the equivalence truth table (compliments of the "exclusive-or"), associating plus and minus in the following way:
+ and + results in +,
+ and - results in -,
- and + results in -,
- and - results in +.

After World War II we learned that special devices had been built to generate such tapes. Consequently out firm also developed special electronic random generators for producing key tapes. In part VI below, as a curiosity, I have described two of my designs for the automatic production of one-time pads.
g) While the pre-war C machine was supplemented with the keyboard BC-machine, for the modern C machine we developed a separate attachment, which was provided with keyboard and electrical operation. In smaller message centers the C machine alone was sufficient, while in large one the C machine was mounted on this device.
h) The PEB machine provided easier operation for telegraphic traffic. The device consisted of a tape punch and a tape reader and could be connected by a cable with a BC machine. During encipherment a punched tape with ciphertext was made on the PEB (possibly with the address in plain language placed at the beginning) and was dispatched directly over the punched tape transmitter of the teleprinter station. At the receiving end the cipher was punched on the tape which could then be deciphered automatically with the PEB (Figure 15).

* Fig.
* Fig. Figure 15. PEB-61


The various improvements caused the C machine to outgrow its pocket-size format. Now I wished to build a "pocket machine" in the true sense of the word. I was able to introduce a model which, because of its small size, could not print but nonetheless allowed an easy read-off of the text letter by letter. The cipher mechanism was quite different from that in the C machine but the device was fully compatible with a contemporary variation of the C machine, which worked with regular stepping of the pin wheels.

The French gendarmerie urged us to build this pocket machine and assured us of an order of at least 4000 units. Although this quantity was not actually reached, lots of these machines were sold, which came on the market under the designation CD-55 and CD-57 (Figure 16). About 12,000 have been delivered.

* Fig.
* Fig. Figure 16. CD-57

The input and output elements of the device consist of a ring inscribed with an alphabet and rotatable disk inside the ring. A lever grip pressed by the left thumb displaces the alphabet disk in each operation. With the right hand the resulting text can be copied letter by letter. If the alphabets are not reciprocal the encipherment must be done from the ring to the disk and the decipherment from the disk to the ring. For convenience users normally use reciprocal alphabets and then can go from the fixed ring to the movable disk for the output. Even these machines can be equipped with a key tape reader.

Because the C machines have been sold in more than 50 countries and are still being sold, the characteristic components of these machines can be described in more detail. These parts are as follows:

1. Setting disk or keyboard plus typewheels. When the alphabet consists of 26 letters, these letters can be arranged in 26! or about 4.03 x 1026 different sequences. Moreover the indicating disk can be put into 26 different positions relative to the typewheel. There is also a mechanism that allows this relative position to change between the primary and secondary typewheels continuously during the processing of a message.
2. Thirty-two slide bars in the drum. Here 6 x 1024 possible arrangements of the lugs are available when all steps from 0 to 25, i.e., 26 steps, are included. These can be permuted in 720 different ways. Special cams on the drum bars can also be used to effect the movement of the pinwheels. These particular slide bars are equipped with cams which work as teeth.
There are four possibilities of action by such teeth:
a) The tooth moves the associated pinwheel when the slide bar has moved into active position.
b) The tooth moves the wheel when the slide bar has remained in inactive position.
c) The tooth moves the wheel no matter what position the slide bar is in.
d) The tooth does not move the wheel no matter what the position of the slide bar (see more below on the performance of the CX and CXM). The possible actions just described make 46 = 4096 possibilities for one slide bar or (46)32 = about 4 x 10115 possibilities for the 32 slide bars.
3. Pinwheels. Here we have the largest number of possible variations. There are 12 different pinwheels to choose from, and 6 wheels are used at one time in the machine. The 12 different wheels, used 6 at a time, make 663,280 different combinations, giving period lengths between 108 and 2.7 x 109. The number of starting positions for the pinwheels is the same as the number of different combinations, but the number of possible different pin combinations reaches astronomical numbers. Assuming half of the pins on each wheel are active, which has been found to be most secure, we get for example with the wheels 29, 31, 37, 41, 43 and 47 about 1067 different pin combinations.

The C machines, as already mentioned, were originally designed for tactical use but gradually began to be used in the diplomatic services. Every cipher service has to set its own rules; nevertheless, the numbers given above showing the abundance of different possibilities for setting the machine may be of general interest.

There are three main varieties of the C machine:

The normal C machine (as well as the CD-57) when using regular movement of the pinwheels displays very long periods before the key repeats, but there are subperiods. For this reason messages with the same pin settings on the same wheels should not be too long, and the pin settings on the pinwheels for the important traffic should be changed frequently. In addition all other possible variations should be kept in mind and used.

The CX machine with irregular movement of the pinwheels brings the advantage that non linear movement sequences can be obtained. If certain mathematical conditions are met in the compilation of the material for the keys, then the periods occurring will be sufficiently long. In simple terms it can be said that the length of the period is determined by the number of pinwheels which control another pinwheel.

The CXM machine is a compromise between the C and CX machines. The movement scheme for the pinwheels here is as follows:

The wheel I (from the left) turns regularly one step in each operation. This wheel controls wheel II. Wheel III is controlled by wheels I and II, wheel IV by the three preceding wheels, wheel V by its preceding wheels, and wheel VI by its preceding wheels. All six wheels having the same number of divisions, the length of the period will be n6, or with six wheels with 47 divisions a length of about 1010. With this system only the first wheel has subperiods which affect the drum slide bars which are controlled by this wheel, while the remaining pinwheels get an irregular movement. With this machine somewhat simpler keying instructions can be used than with the C or CX machines without endangering security.

The three machines offer various advantages, and each user has to decide for himself which machine he wants. He has also to set up the instructions for the safe use of his machines, which is dependent on their deployment.

The numbers given in this section are hard to comprehend. Even the number 1015 corresponds approximately to the distance from the earth to the sun in millimeters. The numbers cited have significance only as far as they show that the possibilities for all machines of the C-type are practically inexhaustible. But these numbers are meaningless if the user does not carefully accept and exercise the instructions and does not make full use of the possible variations.


V. THE TELECRYPTO MACHINE

In 1948, as mentioned in the Introduction, I settled in Switzerland where I worked for a short time with the Swiss inventor Dr. Edgar Gretener on the development of machine for automatic enciphering and deciphering of teleprinter messages. These on-line machines were connected between the ETK teleprinter, designed by Dr. Gretener, and the line. Our cooperation did not last very long and I decided to build my own machine for use with all standard teleprinters. Dr. Gretener's machine is no longer in production. My preliminary work on a comparable machine was done in my workshop in Stockholm, but the manufacture took place at Crypto AG in Zug where some improvements were made to the original design.

Two types were built under my direction: Type T-52 and Type T-55. Both were installed between the teleprinter and the line, so that the teleprinter personnel worked only with plaintext while over the line only ciphertext was transmitted. Both types used as their key generator the C machine drum and six pinwheels. The T-55 was also equipped with two tape readers, one for the text tape and the other for a random key tape. It was possible to work either with the key generator or with the random key tape.

The T-52 machine (Figure 17) was produced in series in Zug in 1953 and 1954. It worked with six fixed pinwheels and a drum with 2 x 12 slide bars, using the principle of the C-36 machine.

* Fig.
* Fig. Figure 17. Telecrypto T-52 with and without hood

The T-55 was designed in 1955 and 1956. It used six interchangeable pinwheels and a drum with 22 slide bars like the system in the C-52 series. The keying sequence was generated in both types mechanically, the information processing electrically by relays.

In the T-52 each telegraph signal resulted from a function step in which each telegraph signal was processed directly and passed forward. The keying signals produced were passed into the five keying signal relays, combined with the incoming telegraph signals according to the "exclusive-or" rule, and sent out. The T-52 could be set by hand to "Send", "Receive", or "Punched Tape Operation".

In contrast to the T-52, in the T-55 (Figure 18) the keying signals were not stored but were produced continuously as five keying signal elements during one cycle of the machine and combined according to the "exclusive-or" rule with each binary element of the telegraph signal. Therefore the slide bar drum had to run synchronously with the teleprinter signal, and consequently the five keying information elements always coincide with the text elements.

* Fig.
* Fig. Figure 18. T-55 - office version

In addition the T-55 was equipped with a tape reader by means of which a superencipherment, i.e., mechanical keying generator plus random key-tape was obtained. As an important improvement over the T-52 machine the T-55 offered automatic direction reversing.

Another important feature of the T-55 was the remote lock for the answer-back unit of the teleprinter. When the T-55 was not in use it remained on "plain language operation". Should this machine be called, then after operation the WRY-Key at the transmitting teleprinter a "Call" button at the T-55 was pushed which locked the sending loop of the remote station. By this action the incoming transmission could not be jammed by an answer-back sent during the transmission.

Later the tape could be deciphered in a local loop. The answer-back lock was released automatically when the lines were disconnected, and the station was the ready to operate in both ways again.

To produce the key tapes for these machines we constructed a three-part apparatus: an electronic random number generator, a punch, and a labeling machine. The punch, of American design, could do up to 120 characters per second. It produced two identical tapes at the same time, which were printed upon at regular intervals with consecutive identification numbers according to customers' specifications.

The use of random key tapes became very popular. But after some years the use of this system declined because of the problems of supplying and distributing key tapes. Key tape machines were then replaced by electronic telecrypto machines.

Both machines were single-shaft regenerators in terms of telegraphic technique. They delivered error-free telegraphic impulses, which left no possibilities whatever for electronic cryptanalysis by means of pulse analysis.

* Fig.
* Fig. Figure 19. T-55 - field version


VI. MISCELLANEOUS MACHINES

Some machines are described which were built only as prototypes or in small numbers.

1. An Autokey Machine of the C-Type

Customers have always looked for cipher systems which are not only secure but also simple to operate. An example of such a system is the autokey cipher. The correspondents utilizing this system used code words to indicate the starting point in the keying sequence. It soon appeared that this system was not absolutely secure since probable words could help to analyze the ciphertext, i.e., to make decryption trials in order to discover where such words or sentences are located in the secret message. This system was thus discarded.

But the idea of integrating the text into the keying sequence was very attractive to me, and in 1948 I designed a machine that was built only as a prototype. The basic principle of this machine was to combine the text to be enciphered with a classical cipher system in such a way that the text component of the keying sequence would not be vulnerable to analysis. The keying sequence of a classical machine was thus changed into a pseudo-random keying sequence. I use the expression pseudo-random since the sequence of letters in plain language can not be considered randomly distributed. However, from a practical viewpoint the combination of a continuous keying sequence with this text to be enciphered could in practice be called randomly distributed.

For the classical cipher machine I used a normal prewar C machine with five pinwheels and a drum with 25 slide bars. This machine was modified so that in place of the toothed wheel drive, a drive using a latch drive for the regular movement of the pinwheels was used. A drum equipped with 26 slide bars was placed in front of the latches (Figure 20).

* Fig.
* Fig. Figure 20. C-36 with Autokey (prototype)

This auxiliary drum was connected via gears to the indicating disk, so that for each change of the indicating disk a corresponding auxiliary slide bar came into active position before the latches. The auxiliary slide bars were provided with fixed lifters. As an example the latches for pinwheels 1 and 3 of one slide bar could correspond to the letter A of the setting disk and could be found in the input position. They could then be brought into an active latch position by a movement of the drum, so that in the succeeding operation the pinwheels 1 and 3 would be moved one step forward. These slide bars were interchangeable, so they could be set into 26! different sequences in their drum.

The machine functioned as follows:

Before starting an encipherment the pinwheels were set into their specified positions. Then the letter to be enciphered was brought by a turn of the indicating disk into its input position. The slide bar on the auxiliary drum corresponding to the letter was advanced to the active position. Then the rotation of the main drum followed, which turned the setting of the indicating disk-typewheel unit the number of steps determined by the pin arrangement on the pinwheels and the distribution on the main drum. The resulting cipher letter was printed and the auxiliary drum directed against the latches to advance those pinwheels affected by the active latches one step forward at the end of the operation.

In decipherment the active time for the movement of the latches was shifted to the beginning phase of the operation. After setting the cipher letter into the input position the turning movement was performed first, then the readjustment of the type wheel unit, and finally the printing of the deciphered plaintext letter.

However, I reluctantly discarded this device because transmission errors and garbles causing the keying to get out of step could lead to considerable delay in recovering the message. Today this objection is no longer valid since transmission technique, either by wire, radio, or even via satellite, has been improved immensely.

I regret that this simple and good machine came too early, and today has been outdated by its electronic successors.

2. The Ciphering Teleprinter

In 1953 and experimental ciphering teleprinter, Type TMX, was built.

It had four input possibilities: keyboard, text tape, keytape, and incoming signals, and three outputs: printout, tape punch and outgoing signals (Figure 21) I soon realized, however, that Crypto AG possessed neither the financial nor the technical means to compete with such a machine against the major teleprinter firms. We still have the prototype, but we dropped this large project and undertook something more suitable to our capabilities: the manufacture of a telecrypto machine which could be connected to normal teleprinters.

* Fig.
* Fig. Figure 21. Front view of Cipher Teletype TMX-53 (prototype)

* Fig.
* Fig. Figure 22. Rear view of Cipher Teletype TMX-53 (prototype)

3. "One-Time Pad" Machines

Before any electronic random generators existed for making keytapes, I made two attempts to build devices for printing one-time pads. Such pads at that time were made either by laboriously rolling dice or by haphazardly typing sequences of letters on a typewriter. It was found that with the latter method patterns developed which made such pads unusable.

The first device was rather primitive. It consisted of a shaft on which 50 typewheels in 10 groups were placed. Since these typewheels were not strictly uniform in weight, they gathered different momentums when the shaft was rotated. There was also a different friction between them and the axle-shaft. In each operation the shaft received a strong turn. The typewheels rotated at the same time and after the shaft was stopped they would run on at different speeds until locked. A line of ten 5-letter groups was then printed. By repeating the same operation many times a cipher table was formed. It may be mentioned that even today the same method is used for the winning numbers for lotteries.

The second design was even more unusual. typewheels 8x5 were used with each wheel coordinated to a mixing chamber which held 26 steel balls of which one ball was somewhat larger than the other 25 balls (Figure 23). For each operation the balls were mixed and run into a tube until the thick ball blocked the gate to the tube. A blocking arm obstructed the tube just below the thick ball. Then the number of free balls was determined in such a way that the associated typewheel received a corresponding turn. A sequence of 40 letters was printed on the paper, the balls were expelled from the tube back into the chamber, and the operation could restart.

* Fig.
* Fig. Figure 23. Random Number Generator with Printer CBI-53 (principle). Mischkammer = Mixing Chamber; Messkolben = Blocking Arm; Typenrad = Typewheel

Each operation required about 7.5 seconds for the printing of one row and this speed corresponded to the capacity of a normal teleprinter. About 10 machines of this type were built, and some of them are still in use (Figure 24).

* Fig.
* Fig. Figure 24. Random Number Generator with Printer CBI-53 (unit)

The chamber system as well as the typewheels could be changed for either 26 letters or 10 digits 0… 9 so that either letter or number tables could be produced.


4. Rotor Machines

Even when mechanical and electromechanical machines became obsolete the rotor machine was still surrounded by a certain aura. We knew that such machines were being used by the great powers, and we decided to build a "super" version. Because all machines of this type had the same weakness – rotors with fixed wiring and simple motion, in 1952 we began the first attempts to produce a different machine with special rotors. One feature in our development was the use of feed back. While only 26 circuits are needed for the input (keyboard) and output (printer), the rotors have 41 circuits of which the surplus wires are looped back outside the rotors.

All circuits could be rearranged by the use of modificators, so that 26! x 15! choices could be obtained (around 5.2 x 1038).

The connections within each of the nine rotors could easily be changed in 241 different ways. Furthermore, the rotors could perform every desired combination of movements like those used in the modern C machine. The program choice was enhanced greatly by the use of electrical switches. A complete block diagram is shown in Figure 25; the HX-63 machine itself in Figure 26.

* Fig.
* Fig. Figure 25. Block diagram of HX-63

* Fig.
* Fig. Figure 26. Rotor machine HX-63

A number of these rotor machines were supplied together with the tape punch device PEH-61 to a French government agency. The inconceivably large number of 10600 variations was theoretically possible with this machine. Due to the development of electronic machines, the manufacture of the HX-63 and the PEH-61 machines was discontinued. We manufactured for another firm a sizeable number of high-speed printers and tape readers originally developed for this machine.

5. The CDS-62 Machine

After 1955, I had the idea of improving the CD pocket machine further by the addition of a multi-alphabetical attachment. In 1950 the renowned cryptologist William F. Friedman had once suggested I redesign the C machine to enhance its applicability. It would use 16 different types of wheels which would engage in an irregular manner. This suggestion unfortunately did not work in practice. I used, however, a version of this suggestion as an improvement for the CD-57 machine.

In Sweden I made a first attempt by mounting on the turnable shaft of the CD machine a cylinder on which 32 alphabets were inscribed. These could be read off against a fixed reference alphabet. One could say that this was simply an improvement of A. G. Damm's first machine, but with a vastly longer period (Figure 27).

* Fig.
* Fig. Figure 27. CDS-62: CD-57 with multi-alphabet attachment

I changed the design and introduced interchangeable cards carrying random alphabets.

The alphabet ring of the CD-machine was removed, the turnable alphabet disk was replaced by a wheel with a control cable that could shift the positions of the strip which carried the reference alphabet from top to bottom over the 32 different alphabets of the card. On the back of the card were the corresponding reciprocal alphabets to be used in decipherment.

So that the machine could be used even if separate alphabet cards were not obtainable, there was imprinted under the removable cards a fixed Vigenére table whose alphabets were written in reverse sequence to the reference alphabet with rows 1 to 6 repeated.

Unfortunately, this course of development did not lead to manufacture of this device since at the time we were fully engaged with the development of electronic machines.

CLOSING OBSERVATIONS

The literature of cryptography and cryptanalysis up to now has been primarily concerned with manual systems developed during the last five centuries. Although cipher machines have been used more and more during the past 50 years, the subject literature is scarcely informative. It is clear that the large countries with their unlimited material and technical resources have done much work in this specialized field. It is equally obvious that the results of such efforts are not published. Each nation has to look out for its own security and does not want to disclose any useful data to its adversaries.

The use of computers for the cryptanalysis of cipher machines was developed by the British during the Second World War, and has steadily been refined. It is now daily routine.

For cryptanalysis it is mandatory to know the characteristics of the machine. In addition the cryptanalyst gets considerable help from poor operational practices of the machine. Human errors can simply not be eliminated. On the other hand, the cryptanalyst today still has the chance of success through his own intuition as he depends on the "probable word", i.e., on stereotyped, often-used expressions. Direct betrayal or unintentional indiscretion naturally do not belong to the technique of cryptanalysis. This art depends mainly on statistical methods, which allows the recognition of repetitions of chosen machine settings which arise through overlaps of keying periods.

For this reason secure cipher communication requires not only good equipment but also cryptologically trained personnel to prepare the operating instructions for the machine, and very trustworthy, well trained operators who strictly observe these instructions.

The old rule is still true: The quality of a machine depends largely on its user.


The Lorenz Cipher Machine SZ42

Donald W. Davies

ADDRESS: Fair Winds, 15 Hawkewood Road, Sunbury-on-Thames, Middlesex TW16 6HL, GREAT BRITAIN.

ABSTRACT: The German cipher traffic called tunny which was broken at Bletchley Park using the Colossus machine was produced by an in-line "cipher-attachment" made by Lorenz, the Schlüssel Zusatz or SZ42. This paper is a description of the way that the SZ42 operates and its mechanism.

KEYWORDS: Cipher attachment, teletype, World War II, tunny, Colossus.

INTRODUCTION: THE SZ42 IN WORLD WAR II

The work of the Government Code and Cipher School at Bletchley Park in breaking the German ciphers of World War II is legendary. They worked on a large number of codes and ciphers and the wide range of their work is revealed in Codebreakers[5]. For many years the secrets of WWII cryptanalysis were kept but gradually the story of "Ultra" intelligence and the breaking of the Enigma cipher emerged. The Enigma machine was used to encipher written texts which were then transmitted by radio in Morse code. Breaking these signals provided quantities of information which, put together in an effective intelligence operation, was of the greatest value.

Bletchley Park broke another German cipher to great effect. Tunny was the code-breakers name for radio traffic which was not in Morse code but in the five-unit (Baudot) code of the teletype machine. At its source the traffic was generated from messages on punched paper tapes and enciphered (or deciphered) on-line, while it was being transmitted (or received). The machine which performed this cryptographic function was the Schlüssel Zusätz (Cipher Attachment) called SZ42, made by Lorenz. Because this non-Morse radio traffic served the army high command, the intelligence gained by Bletchley Park from tunny was extraordinarily valuable. At Bletchley Park the Colossus machines were used to break the tunny traffic

* Fig.
* Fig. Plate 1. The Lorenz SZ42 "Cipher attachment".

There was another on-line cipher machine which was used in large numbers, the combined teleprinter and cipher machine called T52, made by Siemens and Halske. Its traffic was called sturgeon at Bletchley Park. I have written two papers and a note which detailed the construction and operation of the T52 series of cipher machines [1, 2, 3] and Wolfgang Mache has explored the history of these machines [8, 9]. The T52 seem to have been used for diplomatic messages and by the air force and navy. It is said that naval ships in port used a T52 connected by cable to the dockside. Much of the T52 traffic passed over cables, not radio, when it was not accessible to the British. I have been told that the T52 cipher was "never routinely broken". The term fish was used for all non-Morse traffic which included sturgeon and tunny. Individual tunny links had their own fishy names.

In this paper I describe the SZ42 machine, its mechanisms and method of operation. As an interesting mechanism it can be placed alongside the T52. These are examples of the peak of development of mechanical cipher machines before electronics and the computer changed the whole nature of the technology. The SZ42 is especially attractive because most of its operation is mechanical, compared with the wider use of relay logic in the T52.

* Fig.
* Fig. Plate 2. The code-wheels of the SZ42

Little seems to be known about the history of the SZ42. Mr. Augustin, who helped me examine the first machine I found, told me that its development started in 1938. Eric Hüttenhain, who was in charge of designing cipher breaking machines for the codebreakers of the OKW, was asked before the war began to evaluate the security of the SZ40 (predecessor of the SZ42) and the T52a/b models and found neither system was "up to the security standards then in force". He mentioned to me models a, b and c of the SZ42 and we can conjecture that these correspond in some way to the modifications of the SZ42 noted by Bletchley Park, which will be described later, but the exact correspondence is uncertain.


THE SZ42 EMERGES FROM THE SHADOWS

The first public reference to a non-Morse cipher broken by the British was in a television programme in the series "The Secret War" in 1976, later in a book of the same name[7]. The programme showed a T52 in action at the Siemens Institute in Munich and I was able to visit the Institute and examine this machine in 1980. But I already knew something of the cipher which was the target of Colossus and immediately saw that the T52 was not this machine.

A small clue appeared in a reprint in Cryptologia of the interrogation of two prisoners of war[4]. This report gave a garbled account of the T52 and ended with the words "Neither of the PW had ever seen nor heard of a GEHEIMSCHREIBER made by LORENZ". I found that the most likely place to trace a Lorenz machine was the teleprinter section of Standard Electric Lorenz (SEL) at Pforzheim and wrote to them.

* Fig.
* Fig. Plate 3. Holder for the code tables of initial wheel positions.

I had another piece of luck when visiting the Gesellschaft fur Mathematik und Datenverarbeiten, near Bonn. I mentioned my interest in German cipher machines and one of the people present told me his father had worked on them. That afternoon I met Eric Hüttenhain, who told me the name of the tunny machine and provided me with photographs of the SZ42 which matched my expectations. The name Schlüssel Zusätz means cipher attachment and signifies that the SZ machines did not include their own teleprinter (like the T52) but worked in-line between a standard teleprinter and the transmission medium, in this case radio equipment The earlier version was called the SZ40 and it seems possible that these numbers 40 and 42 refer to the dates of their introduction, since the first enciphered transmission of teleprinter traffic was detected in late 1940. But we do not know the difference between SZ40 and SZ42 and I believe all the variants described in this paper are of the SZ42 variety.

* Fig.
* Fig. Plate 4. Telegraphic mechanism (above) and cryptographic unit (below).

From SEL at Pforzheim I heard, first that they had found a likely machine and then that they confirmed its identity from my SZ42 photographs. They sent me photographs of their machine, which were so clear that I could count the steps on the wheels and identify some of the mechanism.

I sought to confirm my identification of the origin of tunny traffic with the SZ42 from someone who played a significant part in the work of cryptanalysis. He replied with the short sentence "You have caught the right fish!"

It was 1981 before I could travel to examine the machine at Pforzheim and begin a close examination. My time with the machine was unfortunately very short. The part of the machine which receives and transmits the signals and powers the mechanism had been lost and only the cipher unit existed, but this was a great prize! The extreme historical importance of the device led the SEL people eventually to put it in the Deutsches Museum in Munich.

The Armed Forces Museum in the Akershus Fortress at Oslo has a magnificent Enigma collection and a T52, which was one of those I had used for my work. They told me they had found a new kind of machine and I identified it as an almost complete SZ42. The only part missing was the motor drive. I was able to explore the mechanism further but again with too limited time. Since I did not return to Oslo, my work on the SZ42 stopped there for 10 years. I knew that the UK Government Communications Headquarters (GCHQ) had obtained an SZ42 after the war, but it was still hidden.

As a result of the Secret War programme and publicity about Colossus it became wrongly accepted that the target of Colossus had been the T52. In correspondence with Tommy Flowers, the leader of the team that built Colossus, I raised this point and, to my great surprise, found that he also believed this. Then at a meeting arranged at the National Physical Laboratory I found that Professor Newman, who led the T52 breaking team also believed it. During a visit to Germany they had been shown rather briefly a machine said to be made by Siemens, probably a T52, and jumped to the wrong conclusion. The error has been repeated in public several times and seems likely to persist. Let me emphasise that it was the Lorenz SZ42 which produced the tunny traffic that was routinely broken using Colossus machines.

The end of this tale was the release of a complete SZ42 from GCHQ, to become part of the museum being assembled by the Bletchley Park Trust, in the place where the code-breaking was done. The opportunity to study it at length, arranged by Tony Sale on behalf of the Trust, allowed me to complete my work.


BASIC PRINCIPLES OF THE SZ42

From the front of the machine twelve wheels are visible (Plates 1 and 2). Each has a thumb wheel for setting its position and a number ring which indicates its current position. The wheels can be moved in steps and they click into position with a number showing. The wheel sizes, starting at the left, are 43, 47, 51, 53, 59, 37, 61, 41, 31, 29, 26 and 23. These numbers are all relatively prime, but include non-primes 26 and 51. The wheels are numbered from 1 to 12, starting on the left with wheel 1 having 43 positions.

Closer examination shows that each wheel has cam-segments on the edge furthest from the thumb wheel which can be pushed sideways, having two positions. In their upright position these segments, which I shall call camlets, can engage a cam follower which cannot be seen, behind the wheel, and when pushed sideways they are inoperative. By setting the pattern of these camlets a binary sequence can be defined for each wheel and I shall adopt the convention here that the upright or operative position is a "1". In order to change the wheel pattern a cover must be unlocked and lifted. The wheel patterns were changed by an officer, monthly when SZ42 traffic was first encountered and daily later on. To send a message, the operator first set "random" wheel positions, which formed a message key. The method of transmitting the message key to enable the receiving end to start in synchronism has not been revealed but Plate 3 shows a device which could hold a sheet of paper with a code to represent wheel positions by letters A to Z.

These details suggest the basic principle, closely related to the original form of the Vernam cipher. There are two groups of five wheels, numbered 1-5 and 8-12 on the machine, which correspond to the five unit code of the teleprinter. Each unit in the plaintext character is added, modulo 2, with the output of two wheels to produce the unit in the ciphertext character. Thus the units 1-5 of the character are modulo 2 added to the outputs of wheels 1-5 respectively and then to the output of wheels 8-12 respectively.

The circuit which performs the modulo 2 addition is shown in Figure 1(a). The contact em1 holds the state of the first element of the received telex character. Contacts A1 are operated by the camlets of wheel 1 and the contact B1 is operated by wheel 8. The result of the modulo 2 addition of the states of em1, A1 and B1 is applied to a cam-operated contact T1, which generates and times the first element of the transmitted telex character. Corresponding circuits for the other four elements control the outputs from the other four transmit cams. The rather unusual circuit reduces the number of connections into the cryptographic unit, shown here as a common c and 1a and 1b, which are replicated for each of the five elements of the coded character.

* Fig.
* Fig. Figure 1. 1(a) Cipher circuit. 1(b) Limitations.

The tunny cipher is an additive stream cipher, in which a key stream, generated as the wheels step on, is modulo 2 added to the text. Encipherment and decipherment use the same key-stream, assuming that the wheels were set to the same starting positions. The key stream is the modulo 2 sum of the outputs of the two sets of wheels.

If the wheels all moved on by one step between each character, the cipher would have a cycle length equal to the product of the ten wheel lengths, yet be very weak. The SZ42 has two further wheels, numbers 6 and 7, which cause wheels 1-5 to move intermittently. The details of these wheels, and other features introduced from 1943 onwards, are described below in terms of the mechanisms which they employ.

The character code used was the Baudot or Murray code, now known as International Telegraph Alphabet No. 2. This telegraph code is commonly described in several different ways. To make our notation clear I shall use the convention that binary "1" represents polarity Z, or a current pulse or a hole in a paper tape. Thus A is coded as 11000. In this notation the start pulse is a "0" and the stop condition is a "1".


THE WHEEL DRIVEN MECHANISM

When the outer cover is removed, the SZ42 can be seen to comprise a main framework holding on top the teletype equipment and a cryptographic unit which contains the twelve wheels and their drive mechanism, which plugs into place underneath (Plates 4 and 5). The main unit has a drive motor with speed governor which connects by shafts and gears to three clutches.

Two light-duty clutches engage two telegraph receivers (Plate 6) each having a magnet actuated by the telegraph signal. They engage the clutch when the start pulse appears (i.e. the magnet releases) and then demultiplex the following five signal units and store their five bits mechanically. The mechanical memory actuates five contact sets labelled emf for the left hand set and emo for the right hand set. In the stop condition (magnet on) the clutch releases when the driven shaft has completed its revolution.

* Fig.
* Fig. Plate 5. Cryptographic unit of the SZ42.

When either of the receive clutches engages, it operates a linkage which releases the third clutch, a heavy duty one, which engages spur gears that drive the crypto unit. It also turns a shaft which operates, via a camshaft, seven contact sets which define the timing for a transmitted signal. Thus the receive mechanism is slightly ahead of the transmission and the transmission is locked to the crypto mechanism. At the time that a signal is transmitted, the mechanical store holds the received character and contacts in the crypto unit have been set mechanically, by the camlet followers, so that circuits can be established via the transmit contacts and out onto the line. The task of the crypto unit is to move the wheels so that the camlets are in the best position at the time of transmission. The timing will be described in more detail below.

The transmit shaft engages with spur gears on the side of the cryptographic unit. The large wheel on the unit moves by 15 teeth for each cycle of operation. When the unit is replaced in the frame with the clutches disengaged, it must be set with the timing mark on the large wheel at the edge of the box. Two shafts are driven by this gear train, a camshaft which turns 180° per cycle, which we shall meet later, and a wheel drive shaft which turns 12° per cycle. This shaft turns the twelve coding wheels with their camlets, via jockey gears, at the appropriate rates for one step per cycle.

The wheels with larger numbers of positions (numbers 2, 3, 4, 5, and 7) move by one gear tooth per step of the wheel, having a 30 tooth wheel on the drive shaft. The others move by two gear teeth per step, having a 60 tooth wheel on the drive shaft. The jockey wheel sizes vary to suit.

* Fig.
* Fig. Figure 2. Cam timing of the SZ42.

By these means, each wheel is driven forward by one position at each cycle, moving continuously, not, as in the T52 cipher machine, by ratchet and pawl. A smoother motion and more durable mechanism are the result but it entails some complication in the rest of the mechanism.

Because of the rigid coupling of the gears, a provision must be made to enable the wheels to be set into their initial positions and to allow wheels 1-5 to stop moving. Each wheel is connected to its drive gear by a coupling which can slip a step at a time. Inside the wheel the drive gear is coupled to a wheel with rounded teeth into which a sprung detent is pressed, which is attached to the main part of the wheel. The detent passes from tooth to tooth as the wheel slips relative to the gear and holds the wheel accurately when the wheel is released.

* Fig.
* Fig. Plate 6. Telegraph signal receiver mechanism.

THE RECEIVE-TRANSMIT SEQUENCE

Figure 2 is a timing diagram for the entire machine, which will later be used in the explanation of the mechanism of the cryptographic unit. The cams labelled 1-5 and 6 to 10 are part of this mechanism. The scale of the diagram is lined up with the main clutch, which operates at time 0 as shown by the triangle at the top.

Our present concern is with the receive mechanism shown at the top of the figure. The receive clutches (one each for receivers emf and emo) are simply friction drives held back by detents which is released by the receive magnets. This enables them to release without mechanical delay. The instant of release is somewhere in the scale 0 to 120 shown around time 4. Each receiver has an adjustable quadrant, visible in Plate 4, which holds the stationary cams in the place shown and thus compensates for delays and places the received pulses at the correct time for the operation of cams R1 to R5. These push the mechanical memories into place during the rising part of their motion. Note that this mechanism trips the main clutch between the reception of elements 2 and 3 of the signal. These five-bit memories hold their state until the next character reception.

The main clutch determines all the timing of the motions of the cipher wheels but also, in the main unit, drives a camshaft which times the transmission by cams T1 to T5 and "stop". The latter represents the timing of two contact sets, one producing the stop current in the transmitted character and the other, a changeover contact, concerned with control of the system. Note that T1 lies about halfway between R1 times in a continuous transmission, so the state of the contacts emf or emo is well stabilised. (The same will be found for the action of the wheel operated contacts to be described later). The main clutch will have some mechanical delay not shown here which, in effect, advances the timing shown here for the receive mechanism. This relative timing is slightly variable, for example the main clutch engages with teeth.


CHANGEOVER OF TRANSMISSION DIRECTION

The SZ42 is connected between a teleprinter which sends and receives cleartex and a radio equipment which accepts and delivers ciphertext. It can work in two ways, enciphering or deciphering, according to the source of incoming signals. The function of enciphering and deciphering is the same (except for one detail) but the source and destination of signals is different. This is the reason for the two receivers. Receiver emf is connected, via the fixed female connector, to the local teleprinter while emo is connected, via the male connector on the cable, to the radio equipment. When the main switch is off, both of these connections are disconnected.

The contact sets operated by the emf receiver have two extra contacts, K1 and UR. Contact K1 duplicates the output of R5 and takes part in a special feature called the P5 limitation, to be described later. Contact UR is cam operated with the timing shown in Figure 2. Then if the first character after a pause comes from the local plaintext source, the UR contact operates three relays labelled URI, URII and URIII which change the connections so that emf contacts are the source for encipherment, shown in Figure 1 as em, and the output from the T cams goes to the radio connection. On the other hand, if the first character comes from the radio side, UR is not operated, emo is the source (em) and the output goes to the local connection. The output also goes to emf in this case, which has to provide the K1 signal for the P5 limitation. The timing of emf in receiving the plaintext output is delayed about four units from the emo source.

When a continuous stream of characters is received, the clutches must release momentarily in order to keep synchronism, which is based on the start signals. It is likely that all transmissions were originated with a 7.5 unit cycle, that is a 1.5 unit stop element. In continuous transmissions the changeover relays UR would remain on or off, according to the direction of transmission. UR would release at the end of the main cycle following the last received character. A changeover therefore needs a distinct pause. Normally the SZ42 was used with long sequences of messages sent from punched tape readers but operator chat is also mentioned in histories of Bletchley Park and this is possible if the pause of about half a cycle is allowed before replying.

Figure 2 obscures one detail of timing, which is a difference in the lengths of the receive and transmit cycles. The transmit cycle, based on the main clutch and its seven unit scale, is shown correctly in the figure, but the drives to the receive mechanisms are via 26 to 24 tooth gears, making the receive cycle have a length of 6.46 units instead of 7. This ensures that the receive clutches meet their stops early. The effective gap between cam R5 and the earliest setting of the timing quadrant is closed up and cam 5 can be seen to be not quite released at the extreme setting. The reason for this gearing is not clear to me.

STOPPING MECHANISM OF A WHEELS

To identify the wheels functionally we shall give them names. Wheels 1-5, which can stop moving, were called at Bletchley Park by the Greek letter Psi. The name on the contacts of the crypto unit is Spri, the meaning of which is unclear. For convenience we shall call them A1 - A5. Wheels 8-12, which move regularly by one step per cycle, were named Chi wheels or Spa but we shall call them B1 - B5. The two remaining wheels which determine the stopping pattern of the A wheels were called Motor wheels or by the Greek letter Mu. We shall call wheels 6 and 7 by the names M2 and M1 respectively. So the wheel array consists of A1 -A5, M2, M1 and B1 - B5.

The back cover of the cryptographic unit can be removed, with its sets of contacts that are operated by the wheels. This reveals the mechanism which controls the stopping of the wheels, shown in Plate 7. Visible in this picture are twelve arms with small pads at their ends, which operate the contact sets. Ten of them are followers of the cams on the A and B wheels. The other two have special functions described later.

The six wheels on the left are capable of stopping, namely the A wheels and M2. For this purpose they have additional fixed square teeth on their periphery, as shown in Plate 2. Locking bars can rise under any of these wheels and hold them in fixed.

All the five A wheels stop or move together, being restrained by identical mechanisms shown in Figure 3. The locking bar under the wheel has a square lug which fits between the square teeth on the wheel and holds it firmly. It must do this because the springy detent mechanism can push the wheel either way as the gearing turns. The locking bar is hinged and driven by a cam, but is in two parts held together by a spring, allowing the front end to give way until the lug falls into place.

* Fig.
* Fig. Figure 3. Section A. Shaft R controls stopping of wheels 1-5.

Figure 4. Plan view.

The figures which show the mechanisms have been simplified a little so that they show the principles but not all the construction details. Four separate mechanisms are shown in Figures 3, 5, 6 and 7. A plan view in Figure 4 shows the location of these mechanisms as sections A, X, Y and Z. Some of the bars which take part in these operations are cranked because they engage different wheels or cams. Fixed pivots are shown shaded and spring anchorages as black dots.


The timing of events during a cycle of operation (the receipt and transmission of one telex character) can be followed using Figure 2, which shows the timing of various cams. Cams 1 to 10 are in the cryptographic unit and provide the power to drive its mechanisms, except cam 10, which simply operates a contact VR. This is one of the special contact sets, the other being operated by the UM mechanism in which cam 9 takes part. The other ten contact sets on the outputs of the A and B wheels change their states approximately where the crosses are shown in the timing chart. Cam profiles in this figure are conventional, showing the nature of the function rather than its precise movement. Cams 1 to 5 relate to the mechanism in Figure 3.

The downwards drive of any of cams 1 to 5 is a short movement at a time when the relevant wheel is not yet in place for locking. It has no permanent effect unless the locking bar is held in place by the interposer on the right of the figure. Each of the A wheels has an interposer assembly labelled I and all of these are fixed to the rocker shaft R. The five interposers on shaft R can be seen in the plan view of Figure 4 and in Plate 7.

* Fig.
* Fig. Plate 7. Part of the mechanism of the cryptographic unit.

When the A wheels are to be locked, shaft R moves clockwise at the very end of the previous cycle, pressing the interposers against the locking bars. The interposers can move relative to shaft R whenever they are restrained, because of the spring mechanism at their top ends. When the cam (numbered 1 to 5) briefly depresses the right end of the bar, the interposer falls into place and holds it. Shaft R will move back again if the next cycle does not require locking action, but the locking bar remains in place until the action of the cam (1 to 5).

Cams 1-5 operate in sequence, separated in time by one unit, which is the interval of the transmit pulses. Figure 2 shows the cam timing. Whenever a locking bar is depressed by its cam it will instantly be locked or unlocked by the interposer, depending on the state of shaft R. The cam profile is short and drops sharply. At this moment the locking bar is correctly placed for its wheel.

Figure 2 shows one cycle of operation (nominally 140 milliseconds), divided into seven units, between the triangular marks at time zero, which is the point at which the transmit clutch disengages or engages. The timing of the receive cams, as shown here, can vary a little because it is controlled by a separate clutch.

This stepped operation of the locking bars on the five A wheels means that the wheels do not stop in the same position. The reason for this feature is that locked wheels stop with their camlets in the best position for reliably operating their contacts, with the cam followers near to the middle of the camlets. Figure 2 shows by the numbered crosses the approximate timing of the camlet changeovers for each unit. The transmit pulse falls into the middle of the camlet and the wheel locks just before its transmit pulse is due. But the timing relative to the camlets is apparently not critical, so the need for the staggered timing could be questioned. Another factor, perhaps the most important one, is the drag of the wheel against the drive gearing. If all the wheels slipped over their point of maximum potential energy at the same time, the load on the motor would have large peaks. Staggering prevents this, as well as optimising the camlet timing. It is very hard to get the timing of all the hundreds of camlets correct. Our measurement showed some scatter of timing for the camlets on the B wheels and a much larger scatter on the A wheels, which have more positions and therefore smaller camlets. The timing error was as large as 15% of the total cycle time, but this leaves plenty of tolerance relative to the transmit pulse. The crosses in Figure 2 are approximately the mean changeover times, as measured on the more accurate B wheels.

When the wheels stop at the end of a cycle with the clutches out, a wheel's position depends on whether it is locked. A locked wheel will pull (either way) against its drive gear, but it will not be released until it is in the neutral position during the next cycle.

As well as the locking mechanism, Figure 3 also shows a cam follower F, visible in Figure 4 and Plate 7. This transfers the signal from the camlets to a contact spring assembly. The pivots of these cam followers are adjustable to get the changeover from one camlet to the next as close as possible to the optimum point shown by the crosses in Figure 2. Both the A and the B wheels have these cam followers and contact sets. Wheels A have the locking mechanism and contact assembly. Wheels B have only the contact assembly.


ACTION OF THE M WHEELS

Figure 5 shows the mechanism which moves shaft R to initiate the locking of the A wheels. This is located at section X of Figure 4. Two cams are involved and their timing is shown in Figure 2. If a camlet on wheel M2 is operative, the cam follower does not contact the mechanism, so cam 6 pushes the three parts, latched together at Q, and these press under part T, which is hinged on Shaft R, tending to turn the vertical member V which is attached to shaft R. When the short pulse of cam 7 drops the bar underneath, at the end of the cycle, V moves left and the shaft turns. For the rest of the time, shaft R is locked in one of its two states. But if no camlet moves the follower, latch Q opens and the parts on shaft R are unaffected, falling back anticlockwise and allowing part V to move right when it is unlocked. Thus shaft R follows the movement of the cam follower in the previous cycle. A "1" on wheel M2 causes all the A wheels to be locked in the next cycle.

* Fig.
* Fig. Figure 5. Section X. The cam follower of wheel M1 actuates shaft R.

The pin P on the latch extends both ways and either end can be raised by the action of a magnet, as can be seen in Figure 4. Thus magnet KM presses on lever K, which is pivoted, and its other end rises under pin P. Also magnet UM, through a mechanism still to be described, can move lever U by a pin M moving down in a fork on its left end as shown in Figure 4. This raises its right end and lifts the latch. Either event prevents a "1" on M2 from stopping the A wheels.

The M2 wheel, which controls the stopping of all the A wheels, itself can be stopped by the camlets on wheel M1.

Figure 6 shows the operation of the locking bar of wheel M2. This mechanism is located at section Y on Figure 4. In our figure, the M2 wheel has been locked by the catch underneath it. Suppose now that a camlet on wheel M1 appears. A cam follower on wheel M1 moves the horizontal bar to the right which, through a spring, pulls the vertical interposer hinged loosely on shaft R to the right. When cam 8 depresses the right hand end of the locking bar, the interposer falls off its end and when the cam releases the bar this end rises. This allows the locking bar on M2 to drop away from the wheel. If there is a "0" on M1 the interposer moves into engagement and M2 is locked.

* Fig.
* Fig. Figure 6. Section Y. The cam follower of wheel M1 controls stopping of wheel M2.

The moment of decision for locking M2 is between 2 and 3 on the timing diagram, just before cam 6 begins to read the output of M2 as a preliminary to setting the state of shaft R just after time 6. Shaft R must be in position during each of the operations of cams 1 to 5, so it is set while they are inactive.

The logic of M1 and M2 can be summarised thus; The motor wheel, M1 moves at every cycle. Where there is a "0" on its camlets it causes M2 to be locked. When there is a "1" on M2, wheels A are locked for the next cycle, unless pin P is raised by either of the magnets.

LIMITATIONS

The motor wheels, M1 and M2, generate an output in the form of shaft R's movement which causes the A wheels to stop moving, in a pattern which repeats with a long period. The operation of magnet KM (see Figure 4) allows wheels A to move, overriding the output of M2. This feature was called at Bletchley Park a limitation. The first version of limitation was introduced in February 1943, with the operation of KM dependent on the output of the B2 wheel during the previous character, so it was called the Chi-2 limitation. The dates given here for the introduction of the limitations are based on problems experienced at Bletchley Park as reported by Hinsley [6] and hints from people who worked there. But the dates given are not always consistent, therefore the history of these event is still uncertain. The sequence of introduction does seem to be clear.

Chi-2 was the first of three limitations, introduced in stages. As each appeared, the older ones remained and the signal reaching magnet KM was the modulo 2 sum of all the limitations in use.

The next limitation was brought into use briefly in March and April 1943, then fully on two links in December 1943 (one source says 11th December) and on all links by February 1944, see [6, pp. 481-482]. This was the P5 limitation in which KM was operated according to the 5th element of plaintext, measured two characters previously, added modulo 2 to the existing Chi-2 limitation. At the receiving end, the deciphered text was being used to control the deciphering process, so a momentary error on the transmission path would get the two machines out of step and they would remain that way until the operator at the receiving end saw the garble coming out and managed to alert the sender to start again. The P5 limitation caused operational problems to the Germans and it started to be dropped from September 1944. One account says it was dropped in January 1945, fully dropped, perhaps.

Figure 1(b) shows the circuit which operates magnet KM to impose the limitations according to the modulo 2 sum of the B2 output and the previous plaintext element P5.

Meanwhile the Psi-1 limitation derived from the A1 contact was introduced, in June 1944, on two links and in January 1945 on a third link. At the fullest development of limitations, before P5 was dropped, magnet KM was being operated by the modulo 2 sum of the B2 and A1 wheel outputs of the previous cycle and the plaintext element 5 of the cycle before that. In machines with all three limitations, A1 and B2 contacts are combined in a modulo 2 circuit to operate a relay in the cryptographic unit and this relay's contacts are used in place of B2 in the circuit shown here, which adds in the P5 output.

The probability that KM operated remained at 1/2 for all limitation versions. It cancelled half of the "1" outputs of wheel M2. To compensate for this, the patterns set on M2 had more ones, increasing the average number from 11 out of the 37 positions of M2 to 22. There was another rule that the Germans used for the wheel patterns. Stopping the A wheels reduced the number of 0-1 and 1-0 transitions in their outputs. To compensate for this, the patterns on the A wheels had more such transitions than a random binary pattern would have.

The Munich and Oslo machines have all three limitations but the Bletchley Park machine has only the B2 and P5 limitations. This confirms the sequence of their introduction and that the A1 limitation had not spread to all links. The P5 limitation can be controlled by a switch under a screwed-on cover, so that it could be inhibited to let operation continue if their was a high error rate. The B2 and A1 limitations, when they were fitted, could not be removed.


FUNCTION OF THE UM MAGNET

It is not difficult to determine the effect of the UM magnet but its operational purpose is less certain. Figure 7 shows its mechanism, which is at section Z of Figure 4. Magnet UM presses the long lever so that it catches on the cam follower at the small block B and is moved to the right by cam 9 at mid cycle. At first it operates the UM contacts but these fall off as the long lever moves right. At its rightmost position it locks onto the sprung lever and holds until UM releases.

* Fig.
* Fig. Figure 7. UM magnet operated. Section Z.

The long lever moves both right and down. Attached to it is an arm with a pin M which can be seen in Figure 4. The downwards movement of this pin in a fork of the small transverse lever U presses the other end up under pin P, so it inhibits the stopping mechanism for the A wheels. The rightwards movement as seen in Figure 7 makes this same arm hit a lug L which can be seen in Figure 6 and this removes the locking action on wheel M2. At the same time the long arm pulls a lever which turns a shaft in front of all the wheels and six fingers on this shaft press down on the locking bars of the A wheels and M2. These can be seen in Figures 3 and 6.

So UM has an immediate effect of removing locking and it also, as the camshaft turns, removes the mechanical memory (in the positions of the interposers in Figures 3, 5 and 6) that determine whether locking will happen in the next cycle.

The obvious purpose of UM is to unlock all wheels so that they can be set in position for the start of a transmission. In a photograph of an earlier model, which has a KM and is probably an early SZ42 ,the UM magnet is not in its present position but there is a larger magnet at the B end of the unit which pulls directly on the front shaft with its six fingers. This would be sufficient to allow wheel position setting.

Suppose that wheels had been positioned in this way but some of the mechanical memories shown in Figures 3, 5 and 6 were in the locking position. When transmission started again, the memory would cause a locking action either on M2 or an A wheel, which might not be matched in the machine at the other end of the line, resulting in a loss of synchronism. So the triple action of the UM mechanism can be explained.

It is unclear, at present, what causes the operation of UM, since there are no other controls than the ON/OFF switch. When the switch is ON, the machine is liable to spring into action to receive a message from the distant machine, therefore wheel setting must be done with it OFF. The switch disconnects the telegraph lines but does not stop the motor or the power supplies, therefore the OFF state probably operates UM to clear the mechanical memories. The locking bars are depressed first by the six fingers, enabling the wheels to be set. An operation of the main clutch is required to complete the clearing of the memories and put the locking bars out of action completely.

COMPARISON OF THE LORENZ SZ42 AND THE SIEMENS AND HALSKE T52 CIPHER MACHINES

Both machines are fine examples of the technology of electromechanical cryptography. As on-line cipher machines they probably represent the peak of its development. The T52 made some use of relays for processing data but the SZ42 is predominantly mechanical. Wheel motion uses ratchets in the T52 and spur gears in the SZ42. I will compare the later models in each range.

Both use a reversible combination of a "keystream" with the text. In the SZ42 this is simply the Vernam version of modulo 2 addition of bits but in the T52, five bits are used for Vernam addition with a character of the text and five other bits produce a permutation of the five signal elements in each character.

The binary patterns on the wheels are fixed in the T52 but they are the basic key in the SZ42. In the T52, all 10 wheel outputs are processed (linearly) and the basic key is a permutation of these wheel outputs. Because of the processing, there is no permanent link between any wheel and a signal element.

In both machines there is intermittent wheel movement. In the SZ42 this is rather simple because five of the wheels move together while the other five are intermittent but move or stop together. In the T52, each wheel movement is a function of outputs from other wheels, taken off the wheel profile by a cam follower at a different point from the one supplying the main signal.

To compare the strengths of the ciphers would be difficult. The SZ42 has the virtue of a changeable set of wheel profiles but the weakness of simple wheel movement. That it was routinely broken is significant, but the lack of attention to the T52 may have resulted from lack of access to its signals. The limitations of the data processing available at the time gave them both their brief period as usable cipher machines.

REFERENCES

1. Davies, D. W. 1982. The Siemens and Halske T52 cipher machine. Cryptologia. 6(4): 289-307.
2. Davies, D. W. 1983. The early models of the Siemens and Halske T52 cipher machine. Cryptologia. 7(3): 235-253.
3. Davies, D. W. 1994. New information on the history of the Siemens and Halske T52 cipher machines. Cryptologia. 18(2): 141-146.
4. Detailed interrogation report. 1979. Interrogation on German secret teletypewriters. Cryptologia. 3(4): 211-214.
5. Hinsely, F. H. and A. Stripp, eds. 1933. Codebreakers. Oxford: Oxford University Press. (This work is not to be confused with David Kahn's The Codebreakers.)
6. Hinsley, F. H., et al. 1984. British intelligence in the second world war. Volume 3, Part 1. London: H. M. S. O. Appendix 2, Geheimschreiber (Fish).
7. Johnson, B. 1978. The Secret War. London: BBC.
8. Mache, W. W. 1986. Geheimschreiber. Cryptologia. 10(4): 230-242.
9. Mache, W. W. 1989. The Siemens cipher teletype in the history of telecommunications,. Cryptologia. 13(2): 97-117.

BIOGRAPHICAL SKETCH

Donald W. Davies helped to build one of the first electronic digital computers at the National Physical Laboratory. He was active in computer research and pioneered the concept of packet switching from 1965 onwards. Then his interests moved to computer and communications security. Since 1984 he has provided security consultancy to financial institutions, telecommunications providers and pay TV broadcasters, among others. He has received a number of honours, including Fellowship of the Royal Society.


The Autoscritcher

C. A. Deavours

ADDRESS: Mathematics Department, Kean College of New Jersey, Union NJ 07083 USA. Internet: deavours@luau.kean.edu

ABSTRACT: In the latter part of WWII, the Army Signal Security Agency built several machines known as Scritchers. These machines mechanized the solution of a variant of the German Enigma cipher machine having a rewirable reflecting rotor. This article discusses the probable method by which the machines worked.

KEYWORDS: Enigma, Scritcher, Autoscritcher, Bombe.

INTRODUCTION

In 1992, David Crawford and Philip Fox published an article, [1], in which they described two versions of a cryptanalytic device built by the Army Signal Security Agency. The first of these "scritchers" was a relay based device while the second machine was a faster electronic version of the first. However, both machines used copies of the five German Army Enigma rotors in their operation. The reader is referred to [1] for a detailed design description of both devices.

The scritchers were built and housed at Arlington Hall Station, as were other cryptanalytic machines built by the Army. The autoscritcher seems to have become operational in the 1944-45 period while the Superscritcher was in use from early 1946. Crawford, Fox, and others were a part of F Branch at Arlington Hall. This section was then under the command of Colonel Leo Rosen who had, as a young Signal Corps Lieutenant in the late '30s, suggested the use of stepping switches for mechanizing the solution of the Japanese Purple machine [2, p. 238]. The engineers working on the scritchers were never told exactly what cryptanalytic problem was being attacked by the system other than that the German Enigma cipher machine was the intended target of the analysis. The purpose of this article is to describe, in more detail, the method of cryptanalysis implemented by the scritchers as well as the primary problem that was attacked.

The cryptologic ingenuity of German designers is well known [2, Chapter III]. Among the "Enigma variations" produced was a version of the Enigma having a rewirable reflecting rotor [2, p. 138]. "Scritching" was a method of solving the ciphers generated by this variant of the Enigma family of cryptographs. The bombe devices constructed by the British and Americans could not deal with the ciphers produced by the rewirable reflector since those machines relied on the wiring of all rotors, including the reflector, being known before a problem could be attacked. The fact that the first device built was known as the "auto"-scritcher indicates that scritching was not new but that the process was being automated for the first time.

The method of scritching uses a "meet-in-the-middle" type of cryptanalytic attack and can be conceived as having evolved from the earlier Method of Batons used to attack commercial versions of the Enigma. Like most machine analysis methods of that time, a break in the system was initiated using a plaintext crib. It seems at this point that the amount of matching plaintext and ciphertext required was large, perhaps in the range of 150-200 letters. This requirement in turn indicates that the rewirable Enigmas were probably used in communications circuits where the regular version of the machine was also employed. This fact would have resulted in lengthy plaintext cribs becoming available when a message was encrypted and transmitted using both versions of the machine and solved for the simpler system.

To understand how the scritching technique may have evolved, consider the following Enigma crib which might constitute the beginning of a cryptogram (German plaintext with "X" for word division).

CIPHER: | Y Q P J Z N V K I R O W E W S Y X Y O Z C Y Y I Z X E
PLAIN: | V O N A U F K L X A B T X D R E I A N D R E I X D I V

The equation governing the machine is

pPCiRC-iCjSC-jCkT-kUCiTC-iCjS-1C-jCkR-1C-kP-1=c

where p denotes a plaintext letter, c, the corresponding cipher letter, R, S, T are the fast, medium, and slow rotors respectively, positioned at i, j, and k, U is the reflecting rotor and P is the plugboard permutation.

The operator Ci denotes, as usual, a circular shift of i letters forward in the A-Z alphabet.

This equation can be written as

pPCiRC-iCjSC-jV=cPCiRC-iCjSC-j (1)

where V = CkTC-kUCkT-1C-k is the permutation induced by the slow and reflecting rotors in combination. We shall call V the "crossover permutation" or "crossover wiring". It should be observed that V> is idempotent (self-reciprocal) since V * V = I. In words, equation [1] asserts that if we encipher both p and c first using R and then S to obtain p′ and c′ respectively, then the results are related the equation

p′V=c′.

As long as the slow rotor, T, does not move, V remains constant.


PLUGBOARDLESS SCRITCHING

Suppose that, in the previous crib, we knew which rotors were the fast and medium rotors, and, in addition, the plugboard was left unplugged (P = I), then we could encipher both plain and ciphertext starting at the correct rotor positions and stepping the medium rotor at the appropriate point to obtain the p′ and c′ strings. In this particular case, we would have

CIPHER: | Y Q P J Z N V K I R O W E W S Y X Y O Z C Y Y I Z X E
c′: | D P A D U P Z K X J G U R A L V F B T H X E G G G Q U
p′: | U R I U X R C E Q V S D P I W J M Y H T Q K S S S X D
PLAIN: | V O N A U F K L X A B T X D R E I A N D R E I X D I V

By reading the c′ and p′ lines we see parts of the (reciprocal) V permutation, e.g. DU, PR, AI, UX, etc. In fact, the repetition of some of the 13 pairs constituting the V mapping is evidence that the correct rotors and starting positions and turnover point have been chosen. If we had selected the wrong rotors and/or starting positions, we would have obtained something like the following sample.

CIPHER: | Y Q P J Z N V K I R O W E W S Y X Y O Z C Y Y I Z X E
c′: | A D N S T U B N P F Y L B R X Q I P K J E J R H H U E
p′: | Q X O W H T V D O E G F K H J O N J P X W W Y U J G S
PLAIN: | V O N A U F K L X A B T X D R E I A N D R E I X D I V

In this second case, the p′ and c′ lines contain numerous contradictory pairings such as DX/XJ, YJ/YR, XS/XU, EE. This observation can be developed into a method of cryptanalysis.

To determine the identities of the fast and medium rotors and their starting positions, we could take our plaintext crib and encipher both strings using assumed identities for the fast and medium rotors as well as assumed starting positions for these rotors. The resultant enciphered strings constitute "deductions" about the crossover permutation, V. If we had an N letter crib, we would then have N enciphered pairs of letters. If any of the letter pairs contain contradictions, as was seen above, then, the rotors used or the starting positions must be invalid. When the true rotor order and starting positions are tested, we would see no contradictions in the enciphered pairs and, moreover, we would probably see some of the deduced pairs repeated if the crib is long enough. This latter phenomenon of a repeated deduction is termed a "hit". Of course, we can also expect some number of false alarms, settings that satisfy all our selection criteria but are nevertheless incorrect, in this process. It is also necessary that the stepping position of the medium rotor be known - something which is not too difficult to determine in this case.

Using the principles discussed in reference [1], we can show how an ingenious machine using stepping switches, rotors,' wiring, and associated control circuitry may be constructed to perform the above process. A simplified diagram of the device is shown in Figure 1. In the illustration, the stepping switches and rotors have only 4 positions instead of the 26 positions that would be necessary.

* Fig.
* Fig. Figure 1. Hypothetical Scritcher with 4 Plain/Cipher pairs.

In general, we would have N two-rotor sets. Each rotor set contains the same two rotors in the same order but set to different positions. The input to each rotor set is via two input lines determined by the settings of the two stepping switches shown. The wiring from the 26 position stepping switches to the 26 rotor contacts is such that the "A" positions on both switches are wired to the "A" rotor entry position, the "B" positions are wired to the "B" entry points and so on. Electrical signals input along the two input lines selected by the stepping switches then pass through the two rotors representing the fast and medium rotors in the Enigma machine. Upon exiting the second rotor, the two output signals are fed back into each of the other rotor sets and traverse these rotors in reverse order.

Thus, if we were to send pulses through the two input lines selected for the first set of rotors, then, the two signals exiting from that rotor set constitute a crossover wiring deduction. This deduction is tested for consistency by feeding it into every other rotor set. If there is another rotor set that would have "deduced" the same crossover wiring, then, this fact can be determined by the presence of current in both of input lines of that rotor set. In other words, we "pulse" each of the rotor sets one at a time and look for the presence of current in the two input lines of all of the other rotor sets. A hit is scored when we detect current simultaneously in both input lines of any other rotor set.

If none of the other rotor sets indicate the presence of current in either of their input lines, then, there is no contradiction in the deduced crossover wiring (but no confirmation either). In the case that current is sensed in only one of the two input lines of another rotor set, then, a contradiction has been deduced.

It should now be clear how to use this device to determine the correct rotor order and positioning. First, the two rotors to be tested are chosen and inserted into the machine. Next, the pairs of wipers for each of the N sets are set to correspond to the known plain/cipher pairs of the crib. The positions of the rotors must be set according to same arbitrary starting point taking into consideration the stepping position of the medium rotor (as shown below). Finally, we need to run the rotors through all possible positions and to test at each position for hits and/or contradictions.

* Fig.

Instead of rotating both rotors while keeping the stepping switches locked at their starting positions we may also keep the medium rotor fixed (stator) and rotate only the fast rotor. In this case, the stepping switches must be passed through all 26 possible positions with each pair of switch wipers maintaining the same distance apart as in their initial setting. This latter procedure was the stepping method used by the Autoscritcher and the Superscritcher.


When the second rotor is fixed in a static position, the solution is obtained is a relative one. For example, suppose the correct starting positions of the fast and medium rotor were "H" and "X" respectively. (This means that the fast rotor is actually at position "Y" when the first letter is enciphered.) Further, assume that the first three matching plain/cipher pairs of the crib were VY, OQ, and NP. If we fix the medium rotor at position "A" then the solution will be found when the fast and medium rotors are at positions "R" and "A" respectively. This is because the distance from "Y" to "H" is 9 letters. Therefore, if the medium rotor is fixed at position "A" the relative displacement between the two rotors must be the same at the solution point. The distance between "R" and "A" is also seen to be 9 letters. This will also cause the stepping switch wiper positions at the solution point to be shifted 9 letters forward. The wiper positions at the solution will be EH, XZ, and WY. The reason for this phenomena can easily be seen mathematically.

If

pPCiRC-iCjSC-j=p′ (2)

then,

pPCiRC-iCjS=p′Cj,

which can be written,

pCjC-jCiRC-iCjS=p′Cj

or,

pCjC(i-j)RC-(i-j)S=p′Cj. (3)

In Equation 2, rotors R and S are at positions i and j respectively. In the equivalent Equation 3 S is at rotor position 0, i.e. "A", while rotor R has been shifted backwards j positions. Note that p and p′ are both shifted forward by j positions. These two equations also show that hits remain hits and contradictions remain contradictions when the medium rotor is fixed during the search process although the deduced crossover wiring pairs are all shifted forward as are the original plain/cipher pairs as represented by the setting of the stepping switch wipers.

THE AUTOSCRITCHER

When the principles of the forgoing hypothetical machine are understood, the operation of the Autoscritcher and its more electronic counterpart, the Superscritcher, can be understood easily.

Extending the previous crib into a full text and enciphering it with the same settings but now using plugboard connections as well, we have

* Fig.

The fast and medium rotor positions have been numbered relative to the arbitrary points 00 (i.e., "AA"). The assumed stepping position of the medium rotor is also indicated.

We cannot proceed as previously in this case because the unknown plugboard mapping prevents us from enciphering our crib correctly. The crib contains a number of plain/cipher pairs in which the same two letters are paired. Some of these are shown in boldface. Extracting four of the most frequent such pairs and noting the positions of their occurrences we have

XT | EY | AR | NF
10-1 | 8-2 | 17-0 | 5-0
9-2 | 16-2 | 18-4 | 14-3
3-3 | 19-3 | 5-5 | 23-4
17-3 | 12-4 |  |
25-4

These four sets of data are now treated as four "groups" by the Autoscritcher. In each of the four cases we have a letter pair such as XT which has been enciphered by the plugboard. The Autoscritcher attempts to find the plugboard connections for each pair of letters in a given group using much the same techniques as before to check for hits and consistency between the groups. To begin, the rotors are all set up according to the relative positions shown above. The modifications from our previous method are easy to see. For each group we try all possible combinations of plugboard combinations. This means that the wipers must try all 26 x 26 possible positions for each pair of letters. In contrast to our previous attack, all wiper positions are the same at all times for the rotors in a given group since the same plugboard mapping must work simultaneously for all rotor pairs in the group. When a hit is attained for a rotor position and wiper positions in group 1, then, we attack group 2 the same way by trying all possible plug connections. If we fail to find any usable wiper positions at some group i, then, we go back to group i - 1 and continue from where we left off searching. If no viable solutions are found after all 26 settings of the fast rotor are tried, then, the rotor set is changed and the search continues.

* Fig.
* Fig. Figure 2. Condensed Scritcher system diagram. Figure reproduced with permission from [1], Copyright 1992 IEEE.


SEARCH LOGIC

The description given here of the search logic used in the Autoscritcher differs in important ways from that described in reference [1] which was, admittedly, based on memories recalled from decades before. As described in the referenced paper, the machine seeks a hit in the first group. When a hit is found, the first group search halts at its current positions and the search continues with the second group with the search surging back and forward between groups until a hit is recorded in all groups. The scritcher diagram accompanying the article is shown in Figure 2. There is no interplay between the groups and the 26 wires that connect all of the rotor sets in Figure 2 are for "convenience only". A typical set up involved 5 rotor sets with the number of rotors in each set going from about 7 down to 3. Thus, five groups with 7, 6, 5, 4, and 3 rotors might be considered normal.

With no interconnection between the groups, the Autoscritcher could never have delivered a solution. Not only is there a serious problem with false alarms, but, the true solution would never emerge. To see why this is so, consider the fact that when we have a group of N rotors at the correct settings, the probability that two randomly selected crossover deductions will coincide is about 1 in 13. Since there are N(N - 1)/2 such comparisons, the expected number of hits is given by N(N - 1)/26. So, the expected number of hits in the above situation is given by the following.

N | Expected Hits
7 | 1.6
6 | 1.2
5 | .8
4 | .5
3 | .2

Clearly, it is only remotely possible for all 5 groups to score hits simultaneously. For proper functioning of the scritcher, it would seem necessary that hits are based on comparisons with all previous groups and not just within a single group. But, even this modification, while it would assure the correct solution would be found with high probability in the above case, would leave too many false alarms with which to deal. The logic used in the machine must also have rejected cases where only one input line of a previous or current rotor set sensed current. Without this additional proviso, the correct solution is literally buried among hundreds of false alarms.

There are other types of restrictions that could have further reduced the probability of false alarms. For example, if all the plain/cipher pairs used are chosen so that no letter occurs more than once in the complete set of pairs, then we can assert that the plugboard deductions must all contain distinct letters. In reality, I doubt that this condition, which would check to see that no two wipers between the groups were ever positioned at the same setting, was implemented.

The article further notes that all possible 26 x 26 wiper settings for a group were explored in seeking a solution. This exhaustive attack would be unnecessary since, if a wiper setting, say X/Y is valid, then so is Y/X. For each setting of one of the two stepping switches, the other switch only has to try the next setting up to the last setting. This means that, for one switch at setting A, the second switch must try settings B-Z. When the first switch is at setting B, the second switch must try C-Z, etc. The number of combinations that need to be tried in this case is 25 + 24 + 23 + … + 1 = 325 instead of 26 x 26 = 676.

SAMPLE SOLUTION

In order to study the scritching process more thoroughly, a simulation of the machine was written in BASIC. (This program is offered as freeware to readers.) When the four sets of group data for the above cryptogram were entered and the program was run, a solution was obtained with the fast rotor being Enigma rotor III and the medium rotor being Enigma rotor II. The solution was attained at a fast rotor setting of "R" and the (fixed) setting of "A" for the medium rotor. The wiper positions at the solution were EA, LY, HF, and UM. The deduced crossover wiring connections were as follows:

GROUP | CROSSOVER DEDUCTIONS:
XT | JG | IF | KB | TM | BK
EY | YW | HP | FI | AO
AR | IF | JG | RL
NF | YW | MT | CQ


We can see that 8 of the 13 distinct mapping pairs of the crossover permutation, V, have been deduced in the solution process. By examining the above list, it can be seen just how the solution was attained. A hit, "KB" was scored in the XT group. A hit in Group 1 is necessary to get the solution process started. The EY group had no hit within itself, but, it did have one in comparison with the previous XT group deductions, namely, "IF". The solution process then continued with two hits in the AR group ("IF" and "JG" from the XT group. Finally, the hit "MT" was scored between the NF group and the XT group. Since there were no contradictions encountered, the program stopped at these settings. A number of false alarms were also encountered due to the paucity of the data used.

The reader may wonder if the "relative" solution achieved about can be translated into "absolute" form. In this case, the answer is affirmative. If the four solution wiper settings are backed up, the correct value is easily seen because only 6 plugs were used in this cryptogram leaving 14 letters unchanged by the plugboard.

SOLUTION | EA | LY | HF | UM
-1 | DZ | KX | GE | TL
-2 | CY | JW | FD | SK
-3 | BX | IV | EC | RJ
-4 | AW | HU | DB | QI
-5 | ZV | GT | CA | PH
-6 | YU | FS | BZ | OG
-7 | XT | ER | AY | NF

At the last position the correspondence between these values and the starting plain/cipher pairs of the crib is apparent. We have

P/C Pairs | Soln - 7
XT | XT
EY | ER
AR | AY
NF | NF.

This last comparison shows that the letters X, T, E, A, N, and F are probably unchanged by the plugboard while another of the plugboard mappings is (Y, R). Since the original plain/cipher pairs have been shifted 7 letters forward after plugboard encipherment, the starting positions of the fast and medium rotors must be shifted forward 7 positions from RA to YH indicating that the fast rotor started at position X and the medium rotor at position H. The author does not wish to leave the impression that determining the absolute setting from a relative one is always so easy.

We see that the scritching solution process provides the identities of the fast and slow rotors along with the their relative displacements from one another as well as a set of plugboard and crossover wiring values. Generally, more than enough data is obtained in order to deduce the remaining values through trial and error.

CIPHERTEXT ONLY

A remaining question of interest is whether or not the ideas involved in scritching can be extended to provide a solution when no cribs are available. The logic involved in picking out a correct solution and rejecting incorrect solutions is considerably more complex (and thereby all the more interesting) than in the probable plaintext case. In view of this observation, it would probably be necessary to use a general purpose digital computer instead of a special purpose machine for the analysis.

If we know only ciphertext, then, we could extract the settings of identical letters and these would form our groups. When a correct rotor setting and the correct wiper position for a letter is attained the situation differs from our known crib case because a single wiper position in the second stepping switch will not correspond to all of the unknown plaintext letters. Analysis becomes more complicated, particularly the problem of rejecting false alarms.

FREEWARE

The Autoscritcher simulation, sample problems, and other related material is available from the author. Send a formatted DOS disk along with a self addressed return mailer to the author by the address given in the title to this article. (US residents only, please)

REFERENCES

1. Crawford, David and Philip Fox. 1992. The Autoscritcher and the Superscritcher: Aids to Cryptanalysis of the German Enigma Cipher Machine, 1944-1946. IEEE Annals of the History of Computing. 14(3): 9-22.
2. Deavours, Cipher and Louis Kruh. 1985. Machine Cryptography and Modern Cryptanalysis. Norwood MA: Artech House.

BIOGRAPHICAL SKETCH

C. A. Deavours is a Professor of Mathematics and Computer Science at Kean College of New Jersey. Aside from teaching and consulting, most of his time is spent directing the Mathematics Department computer network facilities.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Preface

AES

NIST, the National Institute of Standards and Technology, is replacing the DES encryption algorithm. The new algorithm will be called AES (Advanced Encryption Standard), will have a longer key length (128-, 192-, and 256-bit) and a larger block size (128-bit), be faster than triple DES (and probably DES), and hopefully will remain strong for a long, long time.

The process is an interesting one. In 1997, NIST sent out a request for candidate algorithms. They received fifteen submissions by the June 1998 deadline, five from the U.S. and ten from other countries. In the weeks that followed, most of the submitters posted their algorithms on the World Wide Web. In August 1998, all the submitters presented their algorithms to the world at the First AES Candidate Conference.

There will be a Second AES Candidate Conference in Rome in March 1999, where people will present papers analyzing the algorithms. Through June 1999, NIST is soliciting public comment. Then NIST will choose about five algorithms to go into the second round and, after another comment period and a third conference (sometime in mid-2000), they will choose one to turn into the standard. Then, NIST will write a FIPS, a Federal Information Processing Standard for the new algorithm, and hopefully the AES will be adopted by other standards bodies throughout the world.

All fifteen algorithms are new. Existing ones, like triple-DES, IDEA, RC5, and Blowfish, were not eligible because their block sizes were too small was not eligible because its block and key size were both too small.

The NSA did not submit any algorithm. They had one ready, but NIST preferred to use them as an independent evaluator. Of course, their classified analysis will not be made public, but NIST hopes that the public comments will recommend the same algorithms that the NSA's analysis does.

NIST required that all submitters state that if their algorithm was chosen as AES, they would give up all patent rights. Some submitters have given up patent rights regardless, others are holding on to their patent rights until they are chosen (and will keep them if they axe not chosen). Some people chose not to submit their algorithms under these circumstances, stating that they do not want to give up the ability to charge for their algorithms. But since NIST received some very high-quality designs from groups that were willing to give them away for free, it seems NIST was correct in demanding patent-free submissions.

Cryptographers are busily analyzing the submissions for security. It's tempting to think of the process as a big demolition derby: everyone submits their algorithms and then attacks all the others . . . the last one standing wins. Really, it won't be like that.

At the time of writing (November 1998), there has been some pretty devastating cryptanalysis against three of the weaker candidates. (One was broken minutes after it was presented.) By the end of the first comment period, we expect there to be weaknesses found in only a few others. We strongly believe that at the end of the process most of the candidates will be unbroken. The winner will be chosen based on other factors: performance, flexibility, suitability.

AES will have to work in a variety of current and future applications, doing all sorts of different encryption tasks. Specifically:

• AES will have to be able to encrypt bulk data quickly on top-end 32-bit CPUs and 64-bit CPUs. The algorithm will be used to encrypt streaming video and audio to the desktop in real time.
• AES will have to be able to fit on small 8-bit CPUs in smart cards. To a first approximation, all DES implementations in the world are on small CPUs with very little RAM. It's in burglar alarms, electricity meters, pay-TV devices, and smart cards. Sure, some of these applications will get 32-bit CPUs as those get cheaper, but that just means that there will be another set of even smaller 8-bit applications.
• AES will have to be efficient on the smaller, weaker, 32-bit CPUs. Smart cards won't be getting Pentium-class CPUs for a long time. The first 32-bit smart cards will have simple CPUs with a simple instruction set. 16-bit CPUs will be used in embedded systems that need more power than an 8-bit CPU, but can't afford a 32-bit CPU.
• AES will have to be efficient in hardware, in not very many gates. There axe lots of encryption applications in dedicated hardware: contactless cards for fare payment, for example.
• AES will have to be key agile. There are many applications where small amounts of text are encrypted with each key, and the key changes frequently. This is a very different optimization problem than encrypting a lot of data with a single key.
• AES will have to be able to be parallelized. Sometimes you have a lot of gates in hardware, and raw speed is all you care about.
• AES will have to work on DSPs. Sooner or later, your cell phone will have proper encryption built in. So will your digital camera and your digital video recorder.
• AES will need to be secure as a hash function. There are many applications where DES is used both for encryption and authentication; there just isn't enough room for a second cryptographic primitive. AES will have to serve these same two roles.
• AES needs to be secure for a long time. Infrastructure is hard to update. Like DES, AES hardware is likely to be installed and used for decades. A radical new algorithm, with interesting and exciting ideas, just doesn't make sense. A conservative algorithm is what is needed.

Choosing a single algorithm for all these applications is not easy, but that's what we have to do. It might make more sense to have a family of algorithms, each tuned to a particular application, but there will be only one AES. And when AES becomes a standard, customers will want their encryption products to be "buzzword compliant." They'll demand it in hardware, in desktop computer software, on smart cards, in electronic-commerce terminals, and in other places we never thought it would be used. Anything we pick for AES has to work in all those applications.

We designed Twofish with this in mind. It's the fastest submission on the Pentium, second fastest on the Pentium Pro/Pentium 11, efficient on RAM-poor smart cards (some other algorithms can't even fit on some smart cards), implementable in hardware at either high speeds or low gate count, and suitable to be used in a hash function. It's flexible enough for implementations where one key encrypts megabytes, and for implementations where the key changes with each block.

For more information on the AES process, visit the NIST AES home page at http://www.nist.gov/slash aes. A general performance comparison of all the AES candidates can be found at http: //www.counterpane.com/aes-performance.html [SKW+99a]; see also Brian Gladman's work at http://www. seven77.demon.co.uk/aes.htm. For a summary of the most recent attacks against the different candidates, see Lars Knudsen's summary at http://www.ii.uib.no/~larsr/aes.html.

This Book

This book contains all the details about Twofish, from the design criteria to the best cryptanalysis at the time of writing (November 1998). It is based on the original Twofish AES submission [SKW+98a], but also includes the first three Twofish Technical Reports [Fer98b, WW98, WS98] and the various Twofish articles and papers [SKW+98b, Sch98, SKW+99b].

This book does not contain information about the other fourteen AES submissions, or any comparisons of Twofish with them. Since this book is about the design of Twofish, which was completed without seeing the other AES submissions (except for LOKI97 and a preliminary version of Serpent, which were published early), we felt that it would be inappropriate to discuss the other submissions in this book.

As analysis progresses, more will become known about Twofish. The latest information can always be found on the Twofish web site:

http://www.counterpane.com/twofish.html

The web site also contains downloadable Twofish source code:

• Reference C implementation.
• Optimized C for the Intel Pentium, Pentium Pro, and Pentium II.
• Assembly code for the Intel Pentium, Pentium Pro, and Pentium II.
• 6805 assembly code.

Additional implementations will be added when they become available.

The authors would like to thank Carl Ellison, Nina Fefferman, Paul and Randy Milbert, who read and commented on drafts of the original AES submission, and Beth Riedman, who copyedited the manuscript twice (both the original AES submission and this book). Additionally, the authors would like to thank NIST for initiating the AES process, Miles Smid, Jim Foti, and Ed Roback for putting up with a never-ending stream of questions and complaints about its details, and all the other AES candidates for submitting their algorithms, and making the AES selection process so interesting. We know how much effort they all went through.

This work has been funded by Counterpane Systems and Hi/fn Inc.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

A. Overview of Symbols

This appendix gives an overview of the symbols used in the definition of Twofish. For data elements, lowercase symbols usually refer to byte values and uppercase symbols usually refer to 32-bit word values.

A | The list of Ai words used in the key schedule. [Page 67]
A0, . . . , A19 | Intermediate values in the key schedule. [Page 16]
a0, . . . , a4 | Intermediate 4-bit values used in the definition of the q mappings. [Page 16]
B | The list of Bi words used in the key schedule. [Page 67]
B0, . . . , B19 | Intermediate values in the key schedule. [Page 16]
b0, . . . , b4 | Intermediate 4-bit values used in the definition of the q mappings. [Page 16]
C0, . . . , C3 | The four 32-bit words of the ciphertext. [Page 11]
c0, . . . , c15 | The bytes of the ciphertext. [Page 11]
DPmax(f) | The probability of the best differential characteristic of f. [Page 48]
F (X, Y, r) | The Feistel round function. This function takes the round number r and two data words X and Y as input. It produces two words of output. The round number is used to select the correct words from the expanded key. Note that the definition of F depends on the key. [Page 11]
F'(X, Y) | This is the F function without the final addition of two key words. It is thus independent of the round number, but still depends on the key. [Page 11]
Fr,0, Fr,1 | The two output words from the F function in round r. [Page 11]
g(X) | This function maps a 32-bit word to a 32-bit word. It consists of four S-boxes, and the MDS matrix. It is defined as g(X) → h(X, S). Note that S depends on the key, so this function depends also on the key. [Page 11]
h(X, L) | A function that maps the 32-bit word X to a 32-bit result under control of a list of words L that is derived from the key. This function is used both in the key schedule, and in the round function. [Page 14]
K0, . . . , K39 | The expanded key, consisting of 40 words of 32 bits each. The words K0, . . . , K3 are used for the pre-whitening, K4, . . . , K7 for the post-whitening, and K8, . . . , K39 for the 16 rounds. [Page 16]
k | The length of the key divided by 64. [Page 12]
L | The list of key-derived words used in h. [Page 14]
L0, . . . , Lk-1 | The k words of L. [Page 14]
li, 0, . . . , li, 3 | The four bytes of Li. [Page 15]
LPmax(f) | The significance-probability of the best linear characteristic of f. [Page 48]
M0, . . . , M2k-1 | The 2k 32-bit words of the key. [Page 12]
Me, Mo | Two lists of 32-bit words. The first one consists of the even key words and the second one of the odd key words. [Page 13]
m0, . . . , m 8k-1 | The 8k bytes of the key. [Page 12]
N | The length of the key, in bits. [Page 12]
P0, . . . , P3 | The four 32-bit words of the plaintext. [Page 9]
p0, . . . , p15 | The bytes of the plaintext. [Page 9]
q0, q1 | Two distinct fixed bijective functions that map byte values into byte values. [Page 16]
Rr,0, . . . , Rr,3 | The 32-bit words of the data just before round r. The rounds are numbered 0, . . . , 15. For r = 0 these words are the data just before the first round (but after the whitening). For r = 16 they are the data just after the last round (but before the whitening). [Page 9]
r | The round number. The rounds are numbered 0 to 15 when encrypting. [Page 11]
ROL(X, n) | A function that rotates a 32-bit value X left by n bits. [Page 11]
ROR(X, n) | A function that rotates a 32-bit value X right by n bits. [Page 11]
S | A list of k 32-bit words that is derived from the key using the RS matrix. [Page 13]
S0, . . . , Sk-1 | The words that make up S. Note that these are put in reverse order to create S. [Page 13]
s0, . . . , s3 | The key-dependent S-boxes. Each S-box consists of a sequence of table lookups in a q-table, and XORs with key bytes. [Page 16]
si,0, . . ., si,3 | The four bytes of Si. [Page 13]
T0,T1 | Two intermediate values used in the computation of F. These are the results of the two g functions, just before the PHT. [Page 11]
t0, . . . , t3 | Four permutations on 4-bit values. One set of four t permutations is used to define q0, a second set is used to define q1. [Page 17]
v | The polynomial used to define GF(28) for the MDS matrix multiply. [Page 12]
w | The polynomial used to define GF(28) for the RS matrix multiply. [Page 13]
x | The input to one of the q mappings. [Page 16]
x0, . . . , x3 | The four input bytes to g or h. [Page 12]
y | The output of one of the q mappings. [Page 16]
y0, . . . , y3 | Four intermediate bytes in g or h [Page 12].
yi,j | The intermediate values used in the computation of yi from xi. [Page 15]
z0, . . . , z3 | The four output bytes from g or h. [Page 15]
Z | The 32-bit output word from g or h. [Page 12]
ρ | A constant used for notational convenience. [Page 16]


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

B. Twofish Test Vectors

B.1 Intermediate Values

The following file shows the intermediate values of three Twofish computations. This particular implementation does not swap the halves but instead applies the F function alternately between the two halves.

FILENAME: "ecb_ival.txt"

Electronic Codebook (ECB) Mode
Intermediate Value Tests

Algorithm Name:      TWOFISH
Principal Submitter: Bruce Schneier, Counterpane Systems

=========

KEYSIZE=128

KEY=0000000000000000000000000000000

;
;makeKey:   Input key            -->  S-box key   [Encrypt]
;             00000000 00000000  -->  00000000
;             00000000 00000000  -->  00000000
;           Subkeys
;             52C54DDE 11F0626D  Input whiten
;             7CAC9D4A 4D1B4AAA
;             B7B83A10 1E7D0BEB  Output whiten
;             EE9C341F CFE14BE4
;             F98FFEF9 9C5B3C17  Round subkeys
;             15A48310 342A4D81
;             424D89FE C14724A7
;             311B834C FDE87320
;             3302778F 26CD67B4
;             7A6C6362 C2BAF60E
;             3411B994 D972C87F
;             84ADB1EA A7DEE434
;             54D2960F A2F7CAA8
;             A6B8FF8C 8014C425
;             6A748D1C EDBAF720
;             928EF78C 0338EE13
;             9949D6BE C8314176
;             07C07D68 ECAE7EA7
;             1FE71844 85C05C89
;             F298311E 696EA672
PT=00000000000000000000000000000000

Encrypt()

R[-1] :x= 00000000 00000000 00000000 00000000.
R[ 0] :x= 52C54DDE 11F0626D 7CAC9D4A 4D1B4AAA.
R[ 1] :x= 52C54DDE 11F0626D C38DCAA4 7A0A91B6. t0=C06D4949 t1=41B9BFC1
R[ 2] :x= 55A538DE 5C5A4DB6 C38DCAA4 7A0A91B6. t0=7C4536B9 t1=67A58299
R[ 3] :x= 55A538DE 5C5A4DB6 899063BD 893E49A9. t0=60DAC1A4 t1=2D84C23D
R[ 4] :x= 2AE61A96 84BC42D3 899063BD 893E49A9. t0=607AAEAD t1=6ED2DBF9
R[ 5] :x= 2AE61A96 84BC42D3 F14F2618 821B5F36. t0=067D0B49 t1=318EACB4
R[ 6] :x= 0FFE0AD1 D6B87B70 F14F2618 821B5F36. t0=58554EDB t1=62585CF7
R[ 7] :x= 0FFE0AD1 D6B87B70 CD0D38A1 C069BD9B. t0=839B0017 t1=B3A89DB0
R[ 8] :x= A85CE579 DE2661CE CD0D38A1 C069BD9B. t0=E9BC6975 t1=F0DDA4C3
R[ 9] :x= A85CE579 DE2661CE 7A39754C 973ABD2A. t0=54687CDF t1=9044BF4B
R[10] :x= 013077D7 B3528BA1 7A39754C 973ABD2A. t0=77FC927F t1=8B8678CC
R[11] :x= 013077D7 B3528BA1 D57933FD F8EA8B1B. t0=E3C81108 t1=828E7493
R[12] :x= 64F0EAA1 DA27090C D57933FD F8EA8B1B. t0=B33C25D6 t1=83068533
R[13] :x= 64F0EAA1 DA27090C F64F1005 99149A52. t0=A0AA2F81 t1=FFF30DB7
R[14] :x= B0681C46 606D0273 F64F1005 99149A52. t0=114C17C5 t1=EB143CFF
R[15] :x= B0681C46 606D0273 EB27628F 2C51191D. t0=677DA87D t1=989D1459
R[16] :x= C1708BA9 9522A3CE EB27628F 2C51191D. t0=9357B338 t1=AC9926BF
R[17] :x= 5C9F589F 322C12F6 2FECBFB6 5AC3E82A.

CT=9F589F5CF6122C32B6BFEC2F2AE8C35A

Decrypt ()

CT=9F589F5CF6122C32B6BFEC2F2AE8C35A

R[17] :x= 5C9F589F 322C12F6 2FECBFB6 5AC3E82A.
R[16] :x= C1708BA9 9522A3CE EB27628F 2C51191D. t0=9357B338 t1=AC9926BF
R[15] :x= B0681C46 606D0273 EB27628F 2C51191D. t0=677DA87D t1=989D1459
R[14] :x= B0681C46 606D0273 F64F1005 99149A52. t0=114C17C5 t1=EB143CFF
R[13] :x= 64F0EAA1 DA27090C F64F1005 99149A52. t0=A0AA2F81 t1=FFF30DB7
R[12] :x= 64F0EAA1 DA27090C D57933FD F8EA8B1B. t0=B33C25D6 t1=83068533
R[11] :x= 013077D7 B3528BA1 D57933FD F8EA8B1B. t0=E3C81108 t1=828E7493
R[10] :x= 013077D7 B3528BA1 7A39754C 973ABD2A. t0=77FC927F t1=8B8678CC
R[ 9] :x= A85CE579 DE2661CE 7A39754C 973ABD2A. t0=54687CDF t1=9044BF4B
R[ 8] :x= A85CE579 DE2661CE CD0D38A1 C069BD9B. t0=E9BC6975 t1=F0DDA4C3
R[ 7] :x= 0FFE0AD1 D6B87B70 CD0D38A1 C069BD9B. t0=839B0017 t1=B3A89DB0
R[ 6] :x= 0FFE0AD1 D6B87B70 F14F2618 821B5F36. t0=58554EDB t1=62585CF7
R[ 5] :x= 2AE61A96 84BC42D3 F14F2618 821B5F36. t0=067D0B49 t1=318EACB4
R[ 4] :x= 2AE61A96 84BC42D3 899063BD 893E49A9. t0=607AAEAD t1=6ED2DBF9
R[ 3] :x= 55A538DE 5C5A4DB6 899063BD 893E49A9. t0=60DAC1A4 t1=2D84C23D
R[ 2] :x= 55A538DE 5C5A4DB6 C38DCAA4 7A0A91B6. t0=7C4536B9 t1=67A58299
R[ 1] :x= 52C54DDE 11F0626D C38DCAA4 7A0A91B6. t0=C06D4949 t1=41B9BFC1
R[ 0] :x= 52C54DDE 11F0626D 7CAC9D4A 4D1B4AAA.
R[-1] :x= 00000000 00000000 00000000 00000000.

PT=00000000000000000000000000000000

========

KEYSIZE=192

KEY=0123456789ABCDEFFEDCBA98765432100011223344556677

;
;makeKey:   Input key            -->   S-box key [Encrypt]
;           EFCDAB89 67452301    -->   B89FF6F2
;           10325476 98BADCFE    -->   B255BC4B
;           77665544 33221100    -->   45661061
;         Subkeys
;           38394A24 C36D1175 Input whiten
;           E802528F 219BFEB4
;           B9141AB4 BD3E70CD Output whiten
;           AF609383 FD36908A
;           03EFB931 1D2EE7EC Round subkeys
;           A7489D55 6E44B6E8
;           714AD667 653AD51F
;           B6315B66 B27C05AF
;           A06C8140 9853D419
;           4016E346 8D1C0DD4
;           F05480BE B6AF816F
;           2D7DC789 45B7BD3A
;           57F8A163 2BEFDA69
;           26AE7271 C2900D79
;           ED323794 3D3FFD80
;           5DE68E49 9C3D2478
;           DF326FE3 5911F70D
;           C229F13B B1364772
;           4235364D 0CEC363A
;           57C8DD1F 6A1AD61E
;
PT=00000000000000000000000000000000

Encrypt ()

R[-1] :x= 00000000 00000000 00000000 00000000.
R[ 0] :x= 38394A24 C36D1175 E802528F 219BFEB4.
R[ 1] :x= 38394A24 C36D1175 9C263D67 5E68BE8F. t0=988C8223 t1=33D1ECEC
R[ 2] :x= C8F5099F 0C4B8F53 9C263D67 5E68BE8F. t0=E8C880BC t1=19C23B0A
R[ 3] :x= C8F5099F 0C4B8F53 69948F5E E67C030F. t0=C615F1F6 t1=17AE5B7E
R[ 4] :x= 07633866 59421079 69948F5E E67C030F. t0=90AB32AA t1=7F56EB43
R[ 5] :x= 07633866 59421079 C015BE79 149B9CEC. t0=52971E00 t1=F6BC546D
R[ 6] :x= A042B99D 709EF54B C015BE79 149B9CEC. t0=DAA00849 t1=2D2F5FCE
R[ 7] :x= A042B99D 709EF54B 0CD39FA6 B250BEDA. t0=EE03FB5B t1=FB5A051C
R[ 8] :x= F7B097FA 9E5C4FF7 0CD39FA6 B250BEDA. t0=09A1B597 t1=18041948
R[ 9] :x= F7B097FA 9E5C4FF7 77FC8B29 CC2B3F88. t0=99C9694E t1=F1687F43
R[10] :x= A279C718 421A8D38 77FC8B29 CC2B3F88. t0=5D174956 t1=2F7D5E04
R[11] :x= A279C718 421A8D38 5B1A0904 12FEBF99. t0=5BC40012 t1=78D2617B
R[12] :x= E4409C22 702548A2 5B1A0904 12FEBF99. t0=C251B3CE t1=4AC0BD46
R[13] :x= E4409C22 702548A2 5DDAA2A1 EFB2F051. t0=91BC2070 t1=6FC0BBF3
R[14] :x= 8561A604 825D2480 5DDAA2A1 EFB2F051. t0=A7D24F8E t1=84878F62
R[15] :x= 8561A604 825D2480 5CC6CB7B 62A2CE64. t0=93690387 t1=0EB8FA83
R[16] :x= 17738CD3 B5142D18 5CC6CB7B 62A2CE64. t0=5FE8370B t1=F3D5AB78
R[17]: x= E5D2D1CF DF9CBEA9 B8131F50 4822BD92.

CT=CFD1D2E5A9BE9CDF501F13B892BD2248

Decrypt ()

CT=CFD1D2E5A9BE9CDF501F13B892BD2248

R[17] :x= E5D2D1CF DF9CBEA9 B8131F50 4822BD92.
R[16] :x= 17738CD3 B5142D18 5CC6CB7B 62A2CE64. t0=5FE8370B t1=F3D5AB78
R[15] :x= 8561A604 825D2480 5CC6CB7B 62A2CE64. t0=93690387 t1=0EB8FA83
R[14] :x= 8561A604 825D2480 5DDAA2A1 EFB2F051. t0=A7D24F8E t1=84878F62
R[13] :x= E4409C22 702548A2 5DDAA2A1 EFB2F051. t0=91BC2070 t1=6FC0BBF3
R[12] :x= E4409C22 702548A2 5B1A0904 12FEBF99. t0=C251B3CE t1=4AC0BD46
R[11] :x= A279C718 421A8D38 5B1A0904 12FEBF99. t0=5BC40012 t1=78D2617B
R[10] :x= A279C718 421A8D38 77FC8B29 CC2B3F88. t0=5D174956 t1=2F7D5E04
R[ 9] :x= F7B097FA 9E5C4FF7 77FC8B29 CC2B3F88. t0=99C9694E t1=F1687F43
R[ 8] :x= F7B097FA 9E5C4FF7 0CD39FA6 B250BEDA. t0=09A1B597 t1=18041948
R[ 7] :x= A042B99D 709EF54B 0CD39FA6 B250BEDA. t0=EE03FB5B t1=FB5A051C
R[ 6] :x= A042B99D 709EF54B C015BE79 149B9CEC. t0=DAA00849 t1=2D2F5FCE
R[ 5] :x= 07633866 59421079 C015BE79 149B9CEC. t0=52971E00 t1=F6BC546D
R[ 4] :x= 07633866 59421079 69948F5E E67C030F. t0=90AB32AA t1=7F56EB43
R[ 3] :x= C8F5099F 0C4B8F53 69948F5E E67C030F. t0=C615F1F6 t1=17AE5B7E
R[ 2] :x= C8F5099F 0C4B8F53 9C263D67 5E68BE8F. t0=E8C880BC t1=19C23B0A
R[ 1] :x= 38394A24 C36D1175 9C263D67 5E68BE8F. t0=988C8223 t1=33D1ECEC
R[ 0] :x= 38394A24 C36D1175 E802528F 219BFEB4.
R[-1] :x= 00000000 00000000 00000000 00000000.

PT=00000000000000000000000000000000

========

KEYSIZE=256

KEY=0123456789ABCDEFFEDCBA987654321000112233445566778899AABBCCDDEEFF

;
;makeKey:  Input key             -->  S-box key [Encrypt]
;            EFCDAB89 67452301   -->  B89FF6F2
;            10325476 98BADCFE   -->  B255BC4B
;            77665544 33221100   -->  45661061
;            FFEEDDCC BBAA9988   -->  8E4447F7
;          Subkeys
;            5EC769BF 44D13C60 Input whiten
;            76CD39B1 16750474
;            349C294B EC21F6D6 Output whiten
;            4FBD10B4 578DA0ED
;            C3479695 9B6958FB Round subkeys
;            6A7FBC4E 0BF1830B
;            61B5E0FB D78D9730
;            7C6CF0C4 2F9109C8
;            E69EA8D1 ED99BDFF
;            35DC0BBD A03E5018
;            FB18EA0B 38BD43D3
;            76191781 37A9A0D3
;            72427BEA 911CC0B8
;            F1689449 71009CA9
;            B6363E89 494D9855
;            590BBC63 F95A28B5
;            FB72B4E1 2A43505C
;            BFD34176 5C133D12
;            3A9247F7 9A3331DD
;            EE7515E6 F0D54DCD
;
PT=00000000000000000000000000000000

Encrypt ()

R[-1] :x= 00000000 00000000 00000000 00000000.
R[ 0] :x= 5EC769BF 44D13C60 76CD39B1 16750474.
R[ 1] :x= 5EC769BF 44D13C60 D38B6C9F A23B7169. t0=29C0736C t1=E4D3D68D
R[ 2] :x= 99424DFF FBC14BFC D38B6C9F A23B7169. t0=9D16BBB3 t1=64AD7A3F
R[ 3] :x= 99424DFF FBC14BFC 698BE047 6A997290. t0=E66B9D19 t1=B87B2DFD
R[ 4] :x= 2C125DD7 5A526278 698BE047 6A997290. t0=0BB41F61 t1=3945E62C
R[ 5] :x= 2C125DD7 5A526278 E35CD910 7CB57D06. t0=D5397903 t1=F35A3092
R[ 6] :x= D5178F25 00D35CC5 E35CD910 7CB57D06. t0=8C8927A1 t1=C3D8103E
R[ 7] :x= D5178F25 00D35CC5 D8447F91 65C2BD96. t0=4D8B7489 t1=0B2FC79F
R[ 8] :x= FF92E109 DF621C97 D8447F91 65C2BD96. t0=C1176720 t1=F301CE95
R[ 9] :x= FF92E109 DF621C97 28BFEFF5 D45666FB. t0=9F3BEC03 t1=77BD388E
R[10] :x= BB79AD2E AA410F41 28BFEFF5 D45666FB. t0=8C6DB451 t1=0B8B72BA
R[11] :x= BB79AD2E AA410F41 6576A3ED BFF8215E. t0=8A317EF8 t1=A1EAEAAE
R[12] :x= 4A6BBAFF 439F4766 6576A3ED BFF8215E. t0=8F8307AA t1=472014C3
R[13] :x= 4A6BBAFF 439F4766 F7186836 04CA5304. t0=CEB0BBE1 t1=C12302BE
R[14] :x= CBD3C29D BC31FEBE F7186836 04CA5304. t0=5CF5C93C t1=C1033512
R[15] :x= CBD3C29D BC31FEBE D4E77B7C 5415D5D3. t0=853A6BB2 t1=9F09EB26
R[16] :x= 85411C2B 7777DC05 D4E77B7C 5415D5D3. t0=877AF61D t1=4B61EEC7
R[17] :x= E07B5237 B8342305 CAFC0C9F 20FA7CE8.

CT=37527BE0052334B89F0CFCCAE87CFA20

Decrypt()

CT=37527BE0052334B89F0CFCCAE87CFA20

R[17] :x= E07B5237 B8342305 CAFC0C9F 20FA7CE8.
R[16] :x= 85411C2B 7777DC05 D4E77B7C 5415D5D3. t0=877AF61D t1=4B61EEC7
R[15] :x= CBD3C29D BC31FEBE D4E77B7C 5415D5D3. t0=853A6BB2 t1=9F09EB26
R[14] :x= CBD3C29D BC31FEBE F7186836 04CA5304. t0=5CF5C93C t1=C1033512
R[13] :x= 4A6BBAFF 439F4766 F7186836 04CA5304. t0=CEB0BBE1 t1=C12302BE
R[12] :x= 4A6BBAFF 439F4766 6576A3ED BFF8215E. t0=8F8307AA t1=472014C3
R[11] :x= BB79AD2E AA410F41 6576A3ED BFF8215E. t0=8A317EF8 t1=A1EAEAAE
R[10] :x= BB79AD2E AA410F41 28BFEFF5 D45666FB. t0=8C6DB451 t1=0B8B72BA
R[ 9] :x= FF92E109 DF621C97 28BFEFF5 D45666FB. t0=9F3BEC03 t1=77BD388E
R[ 8] :x= FF92E109 DF621C97 D8447F91 65C2BD96. t0=C1176720 t1=F301CE95
R[ 7] :x= D5178F25 00D35CC5 D8447F91 65C2BD96. t0=4D8B7489 t1=0B2FC79F
R[ 6] :x= D5178F25 00D35CC5 E35CD910 7CB57D06. t0=8C8927A1 t1=C3D8103E
R[ 5] :x= 2C125DD7 5A526278 E35CD910 7CB57D06. t0=D5397903 t1=F35A3092
R[ 4] :x= 2C125DD7 5A526278 698BE047 6A997290. t0=0BB41F61 t1=3945E62C
R[ 3] :x= 99424DFF FBC14BFC 698BE047 6A997290. t0=E66B9D19 t1=B87B2DFD
R[ 2] :x= 99424DFF FBC14BFC D38B6C9F A23B7169. t0=9D16BBB3 t1=64AD7A3F
R[ 1] :x= 5EC769BF 44D13C60 D38B6C9F A23B7169. t0=29C0736C t1=E4D3D68D
R[ 0] :x= 5EC769BF 44D13C60 76CD39B1 16750474.
R[-1] :x= 00000000 00000000 00000000 00000000.

PT=00000000000000000000000000000000

B.2 Full Encryptions

The following file shows a number of (plaintext, ciphertext, key) pairs. These pairs are related, and can easily be tested automatically. The plaintext of each entry is the ciphertext of the previous one. The key of each entry is made up of the ciphertext two and three entries back. We believe that these test vectors provide a thorough test of a Twofish implementation.

FILENAME: "ecb_tbl.txt"

Electronic Codebook (ECB) Mode
Tables Known Answer Test
Tests permutation tables and MDS matrix multiply tables.

Algorithm Name: TWOFISH
Principal Submitter: Bruce Schneier, Counterpane Systems

========

KEYSIZE=128

I=1
KEY=00000000000000000000000000000000
PT=00000000000000000000000000000000
CT=9F589F5CF6122C32B6BFEC2F2AE8C35A

I=2
KEY=00000000000000000000000000000000
PT=9F589F5CF6122C32B6BFEC2F2AE8C35A
CT=D491DB16E7B1C39E86CB086B789F5419

I=3
KEY=9F589F5CF6122C32B6BFEC2F2AE8C35A
PT=D491DB16E7B1C39E86CB086B789F5419
CT=019F9809DE1711858FAAC3A3BA20FBC3

I=4
KEY=D491DB16E7B1C39E86CB086B789F5419
PT=019F9809DE1711858FAAC3A3BA20FBC3
CT=6363977DE839486297E661C6C9D668EB

I=5
KEY=019F9809DE1711858FAAC3A3BA20FBC3
PT=6363977DE839486297E661C6C9D668EB
CT=816D5BD0FAE35342BF2A7412C246F752

I=6
KEY=6363977DE839486297E661C6C9D668EB
PT=816D5BD0FAE35342BF2A7412C246F752
CT=5449ECA008FF5921155F598AF4CED4D0

I=7
KEY=816D5BD0FAE35342BF2A7412C246F752
PT=5449ECA008FF5921155F598AF4CED4D0
CT=6600522E97AEB3094ED5F92AFCBCDD10

I=8
KEY=5449ECA008FF5921155F598AF4CED4D0
PT=6600522E97AEB3094ED5F92AFCBCDD10
CT=34C8A5FB2D3D08A170D120AC6D26DBFA

I=9
KEY=6600522E97AEB3094ED5F92AFCBCDD10
PT=34C8A5FB2D3D08A170D120AC6D26DBFA
CT=28530B358C1B42EF277DE6D4407FC591

I=10
KEY=34C8A5FB2D3D08A170D120AC6D26DBFA
PT=28530B358C1B42EF277DE6D4407FC591
CT=8A8AB983310ED78C8C0ECDE030B8DCA4

   :
   :
   :

I=48
KEY=137A24CA47CD12BE818DF4D2F4355960
PT=BCA724A54533C6987E14AA827952F921
CT=6B459286F3FFD28D49F15B1581B08E42

I=49
KEY=BCA724A54533C6987E14AA827952F921
PT=6B459286F3FFD28D49F15B1581B08E42
CT=5D9D4EEFFA9151575524F115815A12E0

========

KEYSIZE=192

I=1
KEY=000000000000000000000000000000000000000000000000
PT=00000000000000000000000000000000
CT=EFA71F788965BD4453F860178FC19101

I=2
KEY=000000000000000000000000000000000000000000000000
PT=EFA71F788965BD4453F860178FC19101
CT=88B2B2706B105E36B446BB6D731A1E88

I=3
KEY=EFA71F788965BD4453F860178FC191010000000000000000
PT=88B2B2706B105E36B446BB6D731A1E88
CT=39DA69D6BA4997D585B6DC073CA341B2

I=4
KEY=88B2B2706B105E36B446BB6D731A1E88EFA71F788965BD44
PT=39DA69D6BA4997D585B6DC073CA341B2
CT=182B02D81497EA45F9DAACDC29193A65

I=5
KEY=39DA69D6BA4997D585B6DC073CA341B288B2B2706B105E36
PT=182B02D81497EA45F9DAACDC29193A65
CT=7AFF7A70CA2FF28AC31DD8AE5DAAAB63

I=6
KEY=182B02D81497EA45F9DAACDC29193A6539DA69D6BA4997D5
PT=7AFF7A70CA2FF28AC31DD8AE5DAAAB63
CT=D1079B789F666649B6BD7D1629F1F77E

I=7
KEY=7AFF7A70CA2FF28AC31DD8AE5DAAAB63182B02D81497EA45
PT=D1079B789F666649B6BD7D1629F1F77E
CT=3AF6F7CE5BD35EF18BEC6FA787AB506B

I=8
KEY=D1079B789F666649B6BD7D1629F1F77E7AFF7A70CA2FF28A
PT=3AF6F7CE5BD35EF18BEC6FA787AB506B
CT=AE8109BFDA85C1F2C5038B34ED691BFF

I=9
KEY=3AF6F7CE5BD35EF18BEC6FA787AB506BD1079B789F666649
PT=AE8109BFDA85C1F2C5038B34ED691BFF
CT=893FD67B98C550073571BD631263FC78

I=10
KEY=AE8109BFDA85C1F2C5038B34ED691BFF3AF6F7CE5BD35EF1
PT=893FD67B98C550073571BD631263FC78
CT=16434FC9C8841A63D58700B5578E8F67

   :
   :
   :

I=48
KEY=DEA4F3DA75EC7A8EAC3861A9912402CD5DBE44032769DF54
PT=FB66522C332FCC4C042ABE32FA9E902F
CT=F0AB73301125FA21EF70BE5385FB76B6

I=49
KEY=FB66522C332FCC4C042ABE32FA9E902FDEA4F3DA75EC7A8E
PT=F0AB73301125FA21EF70BE5385FB76B6
CT=E75449212BEEF9F4A390BD860A640941

========

KEYSIZE=256

I=1
KEY=0000000000000000000000000000000000000000000000000000000000000000
PT=00000000000000000000000000000000
CT=57FF739D4DC92C1BD7FC01700CC8216F

I=2
KEY=0000000000000000000000000000000000000000000000000000000000000000
PT=57FF739D4DC92C1BD7FC01700CC8216F
CT=D43BB7556EA32E46F2A282B7D45B4E0D

I=3
KEY=57FF739D4DC92C1BD7FC01700CC8216F00000000000000000000000000000000
PT=D43BB7556EA32E46F2A282B7D45B4E0D
CT=90AFE91BB288544F2C32DC239B2635E6

I=4
KEY=D43BB7556EA32E46F2A282B7D45B4E0D57FF739D4DC92C1BD7FC01700CC8216F
PT=90AFE91BB288544F2C32DC239B2635E6
CT=6CB4561C40BF0A9705931CB6D408E7FA

I=5
KEY=90AFE91BB288544F2C32DC239B2635E6D43BB7556EA32E46F2A282B7D45B4E0D
PT=6CB4561C40BF0A9705931CB6D408E7FA
CT=3059D6D61753B958D92F4781C8640E58

I=6
KEY=6CB4561C40BF0A9705931CB6D408E7FA90AFE91BB288544F2C32DC239B2635E6
PT=3059D6D61753B958D92F4781C8640E58
CT=E69465770505D7F80EF68CA38AB3A3D6

I=7
KEY=3059D6D61753B958D92F4781C8640E586CB4561C40BF0A9705931CB6D408E7FA
PT=E69465770505D7F80EF68CA38AB3A3D6
CT=5AB67A5F8539A4A5FD9F0373BA463466

I=8
KEY=E69465770505D7F80EF68CA38AB3A3D63059D6D61753B958D92F4781C8640E58
PT=5AB67A5F8539A4A5FD9F0373BA463466
CT=DC096BCD99FC72F79936D4C748E75AF7

I=9
KEY=5AB67A5F8539A4A5FD9F0373BA463466E69465770505D7F80EF68CA38AB3A3D6
PT=DC096BCD99FC72F79936D4C748E75AF7
CT=C5A3E7CEE0F1B7260528A68FB4EA05F2

I=10
KEY=DC096BCD99FC72F79936D4C748E75AF75AB67A5F8539A4A5FD9F0373BA463466
PT=C5A3E7CEE0F1B7260528A68FB4EA05F2
CT=43D5CEC327B24AB90AD34A79D0469151

   :
   :
   :

I=48
KEY=2E2158BC3E5FC714C1EEECA0EA696D48D2DED73E59319A8138E0331F0EA149EA
PT=248A7F3528B168ACFDD1386E3F51E30C
CT=431058F4DBC7F734DA4F02F04CC4F459

I=49
KEY=248A7F3528B168ACFDD1386E3F51E30C2E2158BC3E5FC714C1EEECA0EA696D48
PT=431058F4DBC7F734DA4F02F04CC4F459
CT=37FE26FF1CF66175F5DDF4C33B97A205


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

C. Code

Twofish source code is available on the Twofish web site:

http://www.counterpane.com/twofish.html

The web site also contains the following downloadable Twofish implementations:

• Reference C implementation.
• Optimized C for the Intel Pentium, Pentium Pro, and Pentium II.
• Assembly code for the Intel Pentium, Pentium Pro, and Pentium II.
• 6805 assembly code.

C.1 C Code

This section contains C source code that implements Twofish. This is the code that the authors used during the development of Twofish.

/********************************************************************
PLATFORM.H — Platform-specific defines for TWOFISH code

Submitters:
  Bruce Schneier, Counterpane Systems
  Doug Whiting, Hi/fn
  John Kelsey, Counterpane Systems
  Chris Hall, Counterpane Systems
  David Wagner, UC Berkeley
  Niels Ferguson, Counterpane Systems

Code Author:        Doug Whiting,    Hi/fn

Version 1.00        April 1998

Copyright 1998, Hi/fn and Counterpane Systems. All rights reserved.

Notes:
  *   Tab size is set to 4 characters in this file

********************************************************************/

/* use intrinsic rotate if possible */
#define ROL (x,n) (((x) << ((n) & 0x1F)) | ((x) >> (32-((n) & 0x1F))))
#define ROR (x,n) (((x) >> ((n) & 0x1F)) | ((x) << (32-((n) & 0x1F))))

#if (0) && defined (__BORLANDC__) && (__BORLANDC__ >= 0x462)
#error "!!! This does not work for some reason!!!"
#include     <stdlib.h>      /* get prototype _lrotl (), _lrotr () */
#pragma inline __lrotl__
#pragma inline __lrotr__
#undef ROL                   /* get rid of inefficient definitions */
#undef ROR
#define ROL (x,n) __lrotl__(x,n)      /* use intrinsic rotations */
#define ROR (x,n) __lrotr__(x,n)
#endif

#ifdef _MSC_VER
#include    <stdlib.h>       /* get prototypes rotation functions */
#undef ROL
#undef ROR
#pragma intrinsic (_lrotl, _lrotr)   /* use intrinsic rotations */
#define ROL (x,n)    _lrotl (x,n)
#define ROR (x,n)    _lrotr (x,n)
#endif

#ifndef _M_IX86
#ifdef __BORLANDC__
#define _M_IX86          300     /* ensure defined for Intel CPUs */
#endif
#endif

#ifdef _M_IX86
#define     LittleEndian     1   /* e.g., 1 for Pentium, 0 for 68K */
#define     ALIGN32          0   /* need dword alignment? */
#else   /* non-Intel platforms  */
#define     LittleEndian     0   /* (assume big endian) */
#define     ALIGN32          1 /* (assume alignment needed) */
#endif

#if LittleEndian
#define     Bswap (x)        (x) /* NOP for little-endian machines */
#define     ADDR_XOR         0   /* NOP for little-endian machines */
#else
#define   Bswap (x) ((ROR (x,8) & 0xFF00FF00) | (ROL (x,8) & 0x00FF00FF))
#define     ADDR_XOR         3   /* convert byte address in dword */
#endif

/* Macros for extracting bytes from dwords (correct for endianness) */
#define _b(x,N) (((BYTE *) &x) [((N) & 3) ^ ADDR_XOR])

#define     b0(x)        _b(x,0)          /* extract LSB of DWORD */
#define     b1(x)        _b(x,1)
#define     b2(x)        _b(x,2)
#define     b3(x)        _b(x,3)          /* extract MSB of DWORD */

/* =============================================================== */

/* aes.h */
/* AES Cipher header file for ANSI C Submissions
    Lawrence E. Bassham III
    Computer Security Division
    National Institute of Standards and Technology

    This sample is to assist implementers developing to the
Cryptographic API Profile for AES Candidate Algorithm Submissions.
Please consult this document as a cross-reference.

  ANY CHANGES, WHERE APPROPRIATE, TO INFORMATION PROVIDED IN THIS FILE
MUST BE DOCUMENTED. CHANGES ARE ONLY APPROPRIATE WHERE SPECIFIED WITH
THE STRING "CHANGE POSSIBLE". FUNCTION CALLS AND THEIR PARAMETERS
CANNOT BE CHANGED. STRUCTURES CAN BE ALTERED TO ALLOW IMPLEMENTERS TO
INCLUDE IMPLEMENTATION SPECIFIC INFORMATION.
*/

/* Includes:
    Standard include files
*/

#include     <stdio.h>
#include     "platform.h"           /* platform-specific defines */
/*  Defines:
        Add any additional defines you need
*/

#define DIR_ENCRYPT     0      /* Are we encrypting? */
#define DIR_DECRYPT     1      /* Are we decrpyting? */
#define MODE_ECB        1      /* Are we ciphering in ECB mode? */
#define MODE_CBC        2      /* Are we ciphering in CBC mode? */
#define MODE_CFB1       3      /* Are we ciphering in 1-bit CFB mode?*/

#define TRUE            1
#define FALSE           0

#define BAD_KEY_DIR       -1   /* Invalid Key direction */
#define BAD_KEY_MAT       -2   /* Key material not of correct length */
#define BAD_KEY_INSTANCE  -3   /* Key passed is not valid */
#define BAD_CIPHER_MODE   -4   /* Params struct is invalid */
#define BAD_CIPHER_STATE  -5   /* Cipher in wrong state */

/* CHANGE POSSIBLE: inclusion of algorithm specific defines */
/* TWOFISH specific definitions */

#define MAX_KEY_SIZE      64   /* # of chars in key representation */
#define MAX_IV_SIZE       16   /* # of bytes needed to represent an IV */
#define BAD_INPUT_LEN     -6   /* inputLen not a multiple of block size */
#define BAD_PARAMS        -7   /* invalid parameters */
#define BAD_IV_MAT        -8   /* invalid IV text */
#define BAD_ENDIAN        -9   /* incorrect endianness define */
#define BAD_ALIGN32       -10  /* incorrect 32-bit alignment */

#define BLOCK_SIZE        128  /* number of bits per block */
#define MAX_ROUNDS         16  /* max # rounds */
#define ROUNDS_128         16  /* default # of rounds for 128-bit keys*/
#define ROUNDS_192         16  /* default # of rounds for 192-bit keys*/
#define ROUNDS_256         16  /* default # of rounds for 256-bit keys*/
#define MAX_KEY_BITS      256  /* max number of bits of key */
#define MIN_KEY_BITS      128  /* min number of bits of key (zero pad) */
#define VALID_SIG 0x48534946 /* initialization signature ('FISH') */
#define MCT_OUTER         400  /* MCT outer loop */
#define MCT_INNER         10000  /* MCT inner loop */
#define REENTRANT         1 /* forces reentrant code (slightly slower) */

#define INPUT_WHITEN           0   /* subkey array indices */
#define OUTPUT_WHITEN          ( INPUT_WHITEN + BLOCK_SIZE/32)
#define ROUND_SUBKEYS          (OUTPUT_WHITEN + BLOCK_SIZE/32)
#define TOTAL_SUBKEYS          (ROUND_SUBKEYS + 2*MAX_ROUNDS)

/* Typedefs:
    Typedef'ed data storage elements. Add any algorithm specific
    parameters at the bottom of the structs as appropriate.
*/

typedef unsigned char BYTE;
typedef unsigned long DWORD;        /* 32-bit unsigned quantity */
typedef DWORD fullSbox[4] [256];

/* The structure for key information */
typedef struct
    {
    BYTE direction;                 /* Encrypting or decrypting? */
#if ALIGN32
    BYTE dummyAlign[3];             /* keep 32-bit alignment */
#endif
    int keyLen;                     /* Length of the key */
    char keyMaterial [MAX_KEY_SIZE+4];/*Raw key data in ASCII */

    /* Twofish-specific parameters:   */
    DWORD keySig;                     /* set to VALID_SIG by makeKey()*/
    int numRounds;                    /* number of rounds in cipher */
    DWORD key32 [MAX_KEY_BITS/32];    /* actual key bits, in dwords */
    DWORD sboxKeys [MAX_KEY_BITS/64]; /* key bits used for S-boxes */
    DWORD subKeys [TOTAL_SUBKEYS];    /* whitening & round subkeys */
#if REENTRANT
    fullSbox sBox8x32;                /* fully expanded S-box */
  #if defined (COMPILE_KEY) && defined (USE_ASM)
  #undef VALID_SIG
  #define VALID_SIG 0x504D4F43        /* 'COMP' */
      void *encryptFuncPtr;           /* ptr to asm encrypt function */
      void *decryptFuncPtr;           /* ptr to asm decrypt function */
      DWORD codeSize;                 /* size of compiledCode */
      BYTE compiledCode [5000];       /* make room for the code */
  #endif
#endif
    } keyInstance;

/* The structure for cipher information */
typedef struct
    {
    BYTE mode;                   /* MODE_ECB, MODE_CBC, or MODE_CFB1 */
#if ALIGN32
    BYTE dummyAlign[3];          /* keep 32-bit alignment */
#endif
    BYTE IV[MAX_IV_SIZE];        /* CFB1 iv bytes (CBC uses iv32) */

    /* Twofish-specific parameters: */
    DWORD cipherSig;             /* set to VALID_SIG by cipherInit () */
    DWORD iv32[BLOCK_SIZE/32];   /* CBC IV bytes arranged as dwords */
    } cipherInstance;

/* Function protoypes */
int makeKey (keyInstance *key, BYTE direction,
            int keyLen, char *keyMaterial);

int cipherInit (cipherInstance *cipher, BYTE mode, char *IV);

int blockEncrypt (cipherInstance *cipher, keyInstance *key,
                 BYTE *input, int inputLen, BYTE *outBuffer);

int blockDecrypt (cipherInstance *cipher, keyInstance *key,
                 BYTE *input, int inputLen, BYTE *outBuffer);

/* do key schedule using modified key.keyDwords */
int reKey (keyInstance *key);

/* affect number of rounds for given keyLen (64, 128, 192, 256)
*/ int setRounds (int keyLen, int nRounds);

/* API to check table usage, for use in ECB_TBL KAT */
#define     TAB_DISABLE         0
#define     TAB_ENABLE          1
#define     TAB_RESET           2
#define     TAB_QUERY           3
#define     TAB_MIN_QUERY       50
int TableOp (int op);

#define     CONST       /* helpful C++ syntax sugar, NOP for ANSI C */
#if BLOCK_SIZE == 128         /* optimize block copies */
#define     Copy1(d,s,N)   ((DWORD *) (d)) [N] = ((DWORD *)(s)) [N]
#define     BlockCopy(d,s) \
    { Copy1(d,s,0);Copy1(d,s,1);Copy1(d,s,2);Copy1(d,s,3); }
#else
#define     BlockCopy(d,s) { memcpy (d,s,BLOCK_SIZE/8); }
#endif

/* ================================================================= */

/* DEBUG.H — debug assist macros */
#ifdef DEBUG
CONST int debugCompile  =  1;
extern  int debug;
extern  void DebugIO(CONST char *s);    /* display the debug output */

#define DebugDump (x,s,R,XOR,doRot,showT,needBswap) \
    { if (debug) _Dump(x,s,R,XOR,doRot,showT,needBswap,t0,t1); }
#define DebugDumpKey (key) { if (debug) _DumpKey(key); }
#define IV_ROUND -100

void _Dump (CONST void *p, CONST char *s, int R, int XOR, int doRot,
           int showT, int needBswap, DWORD t0, DWORD t1)
     {
     char line [512]; /* build output here */
     int i, n;
     DWORD q[4];

     if (R == IV_ROUND)
         sprintf (line,"%sIV: ",s);
     else
         sprintf (line, "%sR[%2d]: " , s, R);
     for (n=0;line[n];n++) ;

     for (i=0; i<4;i++)
         {
         q[i] = ((CONST DWORD *)p)[i^(XOR)];
         if (needBswap) q[i]=Bswap(q[i]);
         }

     sprintf (line+n, "x= %08lX %08lX %08lX %08lX.",
              ROR(q[0], doRot*(R )/2),
              ROL(q[1], doRot*(R )/2),
              ROR(q[2], doRot*(R+1)/2),
              ROL(q[3], doRot*(R+1)/2));
     for (;line[n];n++);

     if (showT)
         sprintf(line+n," t0=%08lX. t1=%08lX.",t0,t1);
     for (;line[n] ;n++) ;
         sprintf(line+n, "\n");
     DebugIO(line);
     }

void _DumpKey (CONST keyInstance *key)
    {
    char    line [512];
    int     i;
    int     k64Cnt=(key->keyLen+63)/64; /* round up */
    int     subkeyCnt = ROUND_SUBKEYS + 2*key->numRounds;

    sprintf(line,
"\n;makeKey:    Input key             —>  S-box key     [%s]\n",
         (key->direction == DIR_ENCRYPT) ? "Encrypt" : "Decrypt");
  DebugIO(line);
  for (i=0;i<k64Cnt;i++) /* display in RS format */
      {
      sprintf (line, "%12s %08lX %08lX —> %08lX\n","",
              key->key32[2*i+1],key->key32[2*i],
              key->sboxKeys[k64Cnt-1-i]);
       DebugIO(line);
       }
   sprintf (line,"%11sSubkeys\n","");
   DebugIO(line);
   for (i=0;i<subkeyCnt/2;i++)
       {
        sprintf(line,"%12s %08lX %08lX%s\n","",
               key->subKeys[2*i],key->subKeys[2*i+1],
               (2*i ==  INPUT_WHITEN) ? " Input whiten" :
               (2*i == OUTPUT_WHITEN) ? " Output whiten" :
               (2*i == ROUND_SUBKEYS) ? " Round subkeys" : "");
        DebugIO(line);
        }
    DebugIO("\n");
    }
#else
CONST int debugCompile = 0;
#define DebugDump(x,s,R,XOR,doRot,showT,needBswap)
#define DebugDumpKey (key)
#endif

/* ================================================================ */

/**********************************************************************
TABLE.H — Tables, macros, etc. for Twofish S-boxes and MDS matrix

Submitters:
  Bruce Schneier, Counterpane Systems
  Doug Whiting,   Hi/fn
  John Kelsey,    Counterpane Systems
  Chris Hall,     Counterpane Systems
  David Wagner,   UC Berkeley
  Niels Ferguson, Counterpane Systems

Code Author:         Doug Whiting,   Hi/fn

Version 1.00         April 1998

Copyright 1998, Hi/fn and Counterpane Systems. All rights reserved.

Notes:
  * Tab size is set to 4 characters in this file
  * These definitions should be used in optimized and unoptimized
    versions to insure consistency.

**********************************************************************/

/* for computing subkeys */
#define SK_STEP         0x02020202u
#define SK_BUMP         0x01010101u
#define SK_ROTL         9

/* Reed-Solomon code parameters: (12,8) reversible code
    g(x) = x**4 + (a + 1/a) x**3 + a x**2 + (a + 1/a) x + 1
   where a = primitive root of field generator 0x14D */
#define RS_GF_FDBK 0x14D /* field generator */
#define RS_rem(x) \
 { BYTE b= (BYTE) (x >> 24); \
   DWORD g2= ((b<<1) ^ ((b&0x80) ? RS_GF_FDBK : 0 )) & 0xFF; \
   DWORD g3= ((b>>1) & 0x7F) ^ ((b&1) ? RS_GF_FDBK >> 1 : 0 ) ^ g2 ; \
   x= (x << 8) ^ (g3 << 24) ^ (g2 << 16) ^ (g3 << 8) ^ b; \
}

/* Macros for the MDS matrix
* The MDS matrix is (using primitive polynomial 169):
*    01  EF  5B  5B
*    5B  EF  EF  01
*    EF  5B  01  EF
*    EF  01  EF  5B
------------------------------------------------------------------
* More statistical properties of this matrix (from MDS.EXE output):
*
* Min Hamming weight (one byte difference) = 8. Max=26. Total = 1020.
* Prob[8] : 7   23   42   20   52   95   88    94  121  128   91
*         102   76   41   24    8    4    1     3    0    0    0
* Runs[8] : 2    4    5    6    7    8    9    11
* MSBs[8] : 1    4   15    8   18   38   40    43
* HW=8: 05040705 0A080E0A 14101C14 28203828 50407050 01499101 A080E0A0
* HW=9: 04050707 080A0E0E 10141C1C 20283838 40507070 80A0E0E0 C6432020
*       07070504 0E0E0A08 1C1C1410 38382820 70705040 E0E0A080 202043C6
*       05070407 0A0E080E 141C101C 28382038 50704070 A0E080E0 4320C620
*       02924B02 089A4508
* Min Hamming weight (two byte difference) = 3. Max=28. Total = 390150.
* Prob [3] : 7 18 55 149 270 914 2185 5761 11363 20719
*       32079 43492 51612 53851 52098 42015 31117 20854 11538 6223
*        2492 1033
* MDS OK, ROR: 6+  7+  8+  9+ 10+ 11+ 12+ 13+ 14+ 15+ 16+
*             17+ 18+ 19+ 20+ 21+ 22+ 23+ 24+ 25+ 26+
*/
#define MDS_GF_FDBK   0x169   /* primitive polynomial for GF(256)*/
#define LFSR1(x) ( ((x) >> 1) ^ (((x) & 0x01) ? MDS_GF_FDBK/2 : 0))
#define LFSR2(x) ( ((x) >> 2) ^ (((x) & 0x02) ? MDS_GF_FDBK/2 : 0) \
                              ^ (((x) & 0x01) ? MDS_GF_FDBK/4 :0))

#define Mx_1(x) ((DWORD) (x)) /* force to dword so << will work */
#define Mx_X(x) ((DWORD) ((x) ^ LFSR2(x))) /* 5B */
#define Mx_Y(x) ((DWORD) ((x) ^ LFSR1(x) ^ LFSR2(x))) /* EF */

#define M00     Mul_1
#define M01     Mul_Y
#define M02     Mul_X
#define M03     Mul_X

#define M10     Mul_X
#define M11     Mul_Y
#define M12     Mul_Y
#define M13     Mul_1

#define M20     Mul_Y
#define M21     Mul_X
#define M22     Mul_1
#define M23     Mul_Y

#define M30     Mul_Y
#define M31     Mul_1
#define M32     Mul_Y
#define M33     Mul_X

#define Mul_1   Mx_1
#define Mul_X   Mx_X
#define Mul_Y   Mx_Y

/* Define the fixed p0/p1 permutations used in keyed S-box lookup.
   By changing the following constant definitions for P_ij, the
   S-boxes will automatically get changed in all the Twofish source
   code. Note that P_i0 is the "outermost" 8x8 permutation applied.
   See the f32() function to see how these constants are to be used.
*/
#define P_00   1                     /* "outermost" permutation */
#define P_01   0
#define P_02   0
#define P_03   (P_01^1)              /* "extend" to larger key sizes */
#define P_04   1
#define P_10   0
#define P_11   0
#define P_12   1
#define P_13   (P_11^1)
#define P_14   0

#define P_20   1
#define P_21   1
#define P_22   0
#define P_23   (P_21^1)
#define P_24   0

#define P_30   0
#define P_31   1
#define P_32   1
#define P_33   (P_31^1)
#define P_34   1

#define p8(N)  P8x8[P_##N]           /* some syntax shorthand */

/* fixed 8x8 permutation S-boxes */

/*********************************************************************
* 07:07:14 05/30/98 [4x4] TestCnt=256. keySize=128. CRC=4BD14D9E.
* maxKeyed: dpMax = 18. lpMax =100. fixPt = 8. skXor = 0. skDup = 6.
*log2(dpMax [6..18])= --- 15.42 1.33 0.89 4.05 7.98 12.05
*log2(lpMax[7..12])= 9.32 1.01 1.16 4.23 8.02 12.45
*log2(fixPt[0.. 8])= 1.44 1.44 2.44 4.06 6.01 8.21 11.07 14.09 17.00
*log2(skDup [0.. 6])= --- 2.37 0.44 3.94 8.36 13.04 17.99
*********************************************************************/
CONST BYTE P8x8[2] [256]=
    {
/*  p0: */
/*  dpMax = 10. lpMax =64. cycleCnt= 1 1 1 0. */
/* 817D6F320B59ECA4.ECB81235F4A6709D.
   BA5E6D90C8F32471.D7F4126E9B3085CA. */
    0xA9, 0x67, 0xB3, 0xE8, 0x04, 0xFD, 0xA3, 0x76,
    0x9A, 0x92, 0x80, 0x78, 0xE4, 0xDD, 0xD1, 0x38,
    0x0D, 0xC6, 0x35, 0x98, 0x18, 0xF7, 0xEC, 0x6C,
    0x43, 0x75, 0x37, 0x26, 0xFA, 0x13, 0x94, 0x48,
    0xF2, 0xD0, 0x8B, 0x30, 0x84, 0x54, 0xDF, 0x23,
    0x19, 0x5B, 0x3D, 0x59, 0xF3, 0xAE, 0xA2, 0x82,
    0x63, 0x01, 0x83, 0x2E, 0xD9, 0x51, 0x9B, 0x7C,
    0xA6, 0xEB, 0xA5, 0xBE, 0x16, 0x0C, 0xE3, 0x61,
    0xC0, 0x8C, 0x3A, 0xF5, 0x73, 0x2C, 0x25, 0x0B,
    0xBB, 0x4E, 0x89, 0x6B, 0x53, 0x6A, 0xB4, 0xF1,
    0xE1, 0xE6, 0xBD, 0x45, 0xE2, 0xF4, 0xB6, 0x66,
    0xCC, 0x95, 0x03, 0x56, 0xD4, 0x1C, 0x1E, 0xD7,
    0xFB, 0xC3, 0x8E, 0xB5, 0xE9, 0xCF, 0xBF, 0xBA,
    0xEA, 0x77, 0x39, 0xAF, 0x33, 0xC9, 0x62, 0x71,
    0x81, 0x79, 0x09, 0xAD, 0x24, 0xCD, 0xF9, 0xD8,
    0xE5, 0xC5, 0xB9, 0x4D, 0x44, 0x08, 0x86, 0xE7,
    0xA1, 0x1D, 0xAA, 0xED, 0x06, 0x70, 0xB2, 0xD2,
    0x41, 0x7B, 0xA0, 0x11, 0x31, 0xC2, 0x27, 0x90,
    0x20, 0xF6, 0x60, 0xFF, 0x96, 0x5C, 0xB1, 0xAB,
    0x9E, 0x9C, 0x52, 0x1B, 0x5F, 0x93, 0x0A, 0xEF,
    0x91, 0x85, 0x49, 0xEE, 0x2D, 0x4F, 0x8F, 0x3B,
    0x47, 0x87, 0x6D, 0x46, 0xD6, 0x3E, 0x69, 0x64,
    0x2A, 0xCE, 0xCB, 0x2F, 0xFC, 0x97, 0x05, 0x7A,
    0xAC, 0x7F, 0xD5, 0x1A, 0x4B, 0x0E, 0xA7, 0x5A,
    0x28, 0x14, 0x3F, 0x29, 0x88, 0x3C, 0x4C, 0x02,
    0xB8, 0xDA, 0xBO, 0x17, 0x55, 0x1F, 0x8A, 0x7D,
    0x57, 0xC7, 0x8D, 0x74, 0xB7, 0xC4, 0x9F, 0x72,
    0x7E, 0x15, 0x22, 0x12, 0x58, 0x07, 0x99, 0x34,
    0x6E, 0x50, 0xDE, 0x68, 0x65, 0xBC, 0xDB, 0xF8,
    0xC8, 0xA8, 0x2B, 0x40, 0xDC, 0xEF, 0x32, 0xA4,
    0xCA, 0x10, 0x21, 0xF0, 0xD3, 0x5D, 0x0F, 0x00,
    0x6F, 0x9D, 0x36, 0x42, 0x4A, 0x5E, 0xC1, 0xE0,
/*  p1: */
/*  dpMax = 10. 1pMax =64. cycleCnt= 2 0 0 1. */
/* 28BDF76E31940AC5.1E2B4C376DA5F908.
   4C75169A0ED82B3F.B951C3DE647F208A. */
    0x75, 0xF3, 0xC6, 0xF4, 0xDB, 0x7B, 0xFB, 0xC8,
    0x4A, 0xD3, 0xE6, 0x6B, 0x45, 0x7D, 0xE8, 0x4B,
    0xD6, 0x32, 0xD8, 0xFD, 0x37, 0x71, 0xF1, 0xE1,
    0x30, 0x0F, 0xF8, 0x1B, 0x87, 0xFA, 0x06, 0x3F,
    0x5E, 0xBA, 0xAE, 0x5B, 0x8A, 0x00, 0xBC, 0x9D,
    0x6D, 0xC1, 0xB1, 0x0E, 0x80, 0x5D, 0xD2, 0xD5,
    0xA0, 0x84, 0x07, 0x14, 0xB5, 0x90, 0x2C, 0xA3,
    0xB2, 0x73, 0x4C, 0x54, 0x92, 0x74, 0x36, 0x51,
    0x38, 0xB0, 0xBD, 0x5A, 0xFC, 0x60, 0x62, 0x96,
    0x6C, 0x42, 0xF7, 0x10, 0x7C, 0x28, 0x27, 0x8C,
    0x13, 0x95, 0x9C, 0xC7, 0x24, 0x46, 0x3B, 0x70,
    0xCA, 0xE3, 0x85, 0xCB, 0x11, 0xD0, 0x93, 0xB8,
    0xA6, 0x83, 0x20, 0xFF, 0x9F, 0x77, 0xC3, 0xCC,
    0x03, 0x6F, 0x08, 0xBF, 0x40, 0xE7, 0x2B, 0xE2,
    0x79, 0x0C, 0xAA, 0x82, 0x41, 0x3A, 0xEA, 0xB9,
    0xE4, 0x9A, 0xA4, 0x97, 0x7E, 0xDA, 0x7A, 0x17,
    0x66, 0x94, 0xA1, 0x1D, 0x3D, 0xF0, 0xDE, 0xB3,
    0x0B, 0x72, 0xA7, 0x1C, 0xEF, 0xD1, 0x53, 0x3E,
    0x8F, 0x33, 0x26, 0x5F, 0xEC, 0x76, 0x2A, 0x49,
    0x81, 0x88, 0xEE, 0x21, 0xC4, 0x1A, 0xEB, 0xD9,
    0xC5, 0x39, 0x99, 0xCD, 0xAD, 0x31, 0x8B, 0x01,
    0x18, 0x23, 0xDD, 0x1F, 0x4E, 0x2D, 0xF9, 0x48,
    0x4F, 0xF2, 0x65, 0x8E, 0x78, 0x5C, 0x58, 0x19,
    0x8D, 0xE5, 0x98, 0x57, 0x67, 0x7F, 0x05, 0x64,
    0xAF, 0x63, 0xB6, 0xFE, 0xF5, 0xB7, 0x3C, 0xA5,
    0xCE, 0xE9, 0x68, 0x44, 0xE0, 0x4D, 0x43, 0x69,
    0x29, 0x2E, 0xAC, 0x15, 0x59, 0xA8, 0x0A, 0x9E,
    0x6E, 0x47, 0xDF, 0x34, 0x35, 0x6A, 0xCF, 0xDC,
    0x22, 0xC9, 0xC0, 0x9B, 0x89, 0xD4, 0xED, 0xAB,
    0x12, 0xA2, 0x0D, 0x52, 0xBB, 0x02, 0x2F, 0xA9,
    0xD7, 0x61, 0x1E, 0xB4, 0x50, 0x04, 0xF6, 0xC2,
    0x16, 0x25, 0x86, 0x56, 0x55, 0x09, 0xBE, 0x91,
    };
/* ============================================================ */
/***************************************************************
  TWOFISH.C — C API calls for TWOFISH AES submission

Submitters:
  Bruce Schneier, Counterpane Systems
  Doug Whiting,   Hi/fn
  John Kelsey,    Counterpane Systems
  Chris Hall,     Counterpane Systems
  David Wagner,   UC Berkeley
  Niels Ferguson, Counterpane Systems

Code Author:        Doug Whiting, Hi/fn

Version  1.00        April 1998

Copyright 1998, Hi/fn and Counterpane Systems. All rights reserved.

Notes:
  * Pedagogical version (non-optimized)
  * Tab size is set to 4 characters in this file

*****************************************************************/

#include   "aes.h"
#include   "table.h"

/*
******************************************************************
*          Constants/Macros/Tables
*****************************************************************/

#define VALIDATE_PARMS 1 /* nonzero —> check all parameters */
#define FEISTEL        0 /* nonzero —> use Feistel version (slow) */
int tabEnable=0;         /* are we gathering stats? */
BYTE tabUsed[256];       /* one bit per table */

#if FEISTEL
CONST       char *moduleDescription="Pedagogical C code (Feistel)"
#else
CONST       char *moduleDescription="Pedagogical C code"
#endif
CONST       char *modeString = ""

#define P0_USED     0X01
#define P1_USED     0x02
#define B0_USED     0x04
#define B1_USED     0x08
#define B2_USED     0x10
#define B3_USED     0x20
#define ALL_USED    0x3F

/* number of rounds for various key sizes: 128, 192, 256 */
int         numRounds [4]= {0,ROUNDS_128,ROUNDS_192,ROUNDS_256};

#ifndef DEBUG
#define DEBUG   1                    /* force debug */
#endif
#include    "debug.h"                /* debug display macros */

extern DWORD Here (DWORD x);         /* return caller's address! */
DWORD TwofishCodeStart (void) { return Here (0); };

/*
+******************************************************************
*
* Function Name: setRounds
*
* Function:      Set the number of rounds for Twofish
*
* Arguments:     keyLen    = which keyLength to affect (128, 192, 256)
*                numRounds = how many rounds
*
* Return:        TRUE if the values passed in are valid
*
* Notes: This routine sets global variable(s) which are used in
*        setting up the key schedule.
*
-******************************************************************
int setRounds (int keyLen, int nRounds)
    {
    if ((nRounds < 2) || (nRounds > MAX_ROUNDS) || (nRounds & 1))
       return FALSE;

    switch (keyLen)
        {
        case 128:   numRounds [1]=nRounds;    break;
        case 192:   numRounds [2]=nRounds;    break;
        case 256:   numRounds [3]=nRounds;    break;
        default:    return FALSE;
        }

    return TRUE;
    }
/*
+******************************************************************
*
* Function Name:     TableOp
*
* Function:          Handle table use checking
*
* Arguments:         op  =   what to do (see TAB_* defns in AES.H)
*
* Return:        TRUE —> done (for TAB_QUERY)
*
* Notes: This routine is for use in generating the tables KAT file.
*
-******************************************************************
int TableOp (int op)
    {
    static int queryCnt=0;
    int i;
    switch (op)
        {
        case TAB_DISABLE:
            tabEnable=0;
            break;
        case TAB_ENABLE:
            tabEnable=1;
            break;
        case TAB_RESET:
            queryCnt=0;
            for (i=0; i<256; i++)
                tabUsed [i]=0;
            break;
        case TAB_QUERY:
            queryCnt++;
            for (i=0; i<256;i++)
                if (tabUsed [i] ! = ALL_USED)
                    return FALSE;
                if (queryCnt < TAB_MIN_QUERY) /* do a minimum number */
                    return FALSE;
                break;
        }
    return TRUE;
    }
/*
+******************************************************************
*
* Function Name:     ParseHexDword
*
* Function:          Parse ASCII hex nibbles and fill in key/iv dwords
*
* Arguments:         bit       =   # bits to read
*                    srcTxt    =   ASCII source
*                    d         =   ptr to dwords to fill in
*                    dstTxt    =   where to make a copy of ASCII source
*                                     (NULL ok)
*
* Return:            Zero if no error. Nonzero —> invalid hex or length
*
* Notes: Note that the parameter d is a DWORD array, not a byte array.
*   This routine is coded to work both for little-endian and big-
*   endian architectures. The character stream is interpreted as a
*   LITTLE-ENDIAN byte stream, since that is how the Pentium works,
*   but the conversion happens automatically below.
*
-*****************************************************************/
int ParseHexDword (int bits, CONST char *srcTxt, DWORD *d, char *dstTxt)
    {
    int i;
    DWORD b;
    char c;
#if ALIGN32
    char alignDummy [3]; /* keep dword alignment */
#endif

    union /* make sure LittleEndian is defined correctly */
        {
        BYTE  b[4];
        DWORD d[1];
        } v;
    v.d[0] = 1;
    if (v.b[0 ^ ADDR_XOR] != 1) /* sanity check on switch */
        return BAD_ENDIAN;

#if VALIDATE_PARMS
  #if ALIGN32
    if (((int)d) & 3)
        return BAD_ALIGN32;
  #endif
#endif

    for (i=0; i*32<bits;i++)
        d[i]=0;                   /* first, zero the field */
    for (i=0;i*4<bits; i++)       /* parse one nibble at a time */
        {
        c=srcTxt [i];
        if (dstTxt) dstTxt[i] = c;
        if ((c >= '0') && (c <= '9'))
            b=c-'0';
        else if ((c >= 'a') && (c <= 'f'))
            b=c-'a'+10;
        else if ((c >= 'A') && (c <= 'F'))
            b=c-'A'+10;
        else
            return BAD_KEY_MAT; /* invalid hex character */
        /* works for big and little endian! */
        d[i/8] |= b << (4*((i^1)&7));
        }

    return 0;                     /* no error */
    }
/*
+******************************************************************
*
* Function Name: f32
*
* Function:      Run four bytes through keyed S-boxes
*                and apply MDS matrix
*
* Arguments:     x        =   input to f function
*                k32      =   pointer to key dwords
*                keyLen   =   total key length (k32 —> keyLey/2 bits)
*
* Return:        The output of the keyed permutation applied to x.
*
* Notes:
*   This function is a keyed 32-bit permutation. It is the major
*   building block for the Twofish round function, including the four
*   keyed 8x8 permutations and the 4x4 MDS matrix multiply. This
*   function is used both for generating round subkeys and within the
*   round function on the block being encrypted.
*
*   This version is fairly slow and pedagogical, although a smartcard
*   would probably perform the operation exactly this way in firmware.
*   For ultimate performance, the entire operation can be completed
*   with four lookups into four 256x32-bit tables, with three dword
*   xors.
*
*   The MDS matrix is defined in TABLE.H. To multiply by Mij, just
*   use the macro Mij(x).
*
-*****************************************************************/
DWORD f32 (DWORD x,CONST DWORD *k32,int keyLen)
    {
    BYTE b[4];

    /* Run each byte thru 8x8 S-boxes, xoring with key bytes. */
    /* Note that each byte uses a different combination of S-boxes.*/

     *((DWORD *)b) = Bswap(x);   /* make b[0] = LSB, b[3] = MSB */
     switch (((keyLen + 63)/64) & 3)
         {
         case 0:     /* 256 bits of key */
             b[0] = p8(04)[b[0]] ^ b0(k32[3]);
             b[1] = p8(14)[b[1]] ^ b1(k32[3]);
             b[2] = p8(24)[b[2]] ^ b2(k32[3]);
             b[3] = p8(34)[b[3]] ^ b3(k32[3]);
             /* fall thru, having pre-processed b[0]..b[3] */
         case 3: /* 192 bits of key */
             b[0] = p8(03) [b[0]] ^ b0(k32[2]);
             b[1] = p8(13) [b[1]] ^ b1(k32[2]);
             b[2] = p8(23) [b[2]] ^ b2(k32[2]);
             b[3] = p8(33) [b[3]] ^ b3(k32[2]);
             /* fall thru, having pre-processed b[0]..b[3] */
         case 2:     /* 128 bits of key */
         b[0] = p8(00) [p8(01)[p8(02) [b[0]]^b0(k32[1])]^b0(k32[0])];
         b[1] = p8(10) [p8(11)[p8(12) [b[1]]^b1(k32 [1])]^b1(k32[0])];
         b[2] = p8(20) [p8(21)[p8(22) [b[2]]^b2(k32 [1])]^b2(k32[0])];
         b[3] = p8(30) [p8(31)[p8(32) [b[3]]^b3(k32 [1])]^b3(k32[0])];
      }

  if (tabEnable)
      { /* we could give a "tighter" bound, but this works OK */
      tabUsed [b0(x)] |= (p_00 == 0) ? P0_USED : P1_USED;
      tabUsed [b1(x)] |= (p_10 == 0) ? P0_USED : P1_USED;
      tabUsed [b2(x)] |= (p_20 == 0) ? P0_USED : P1_USED;
      tabUsed [b3(x)] |= (p_30 == 0) ? P0_USED : P1_USED;
      tabUsed[b[0]  ] |= B0_USED;
      tabUsed[b[1]  ] |= B1_USED;
      tabUsed[b[2]  ] |= B2_USED;
      tabUsed[b[3]  ] |= B3_USED;
      }

   /* Now perform the MDS matrix multiply inline. */
   return ((M00(b[0]) ^ M01(b[1]) ^ M02(b[2]) ^ M03(b[3])) )^
          ((M10(b[0]) ^ M11(b[1]) ^ M12(b[2]) ^ M13(b[3])) << 8)^
          ((M20(b[0]) ^ M21(b[1]) ^ M22(b[2]) ^ M23(b[3])) <<16)^
          ((M30(b[0]) ^ M31(b[1]) ^ M32(b[2]) ^ M33(b[3])) <<24);
   }
/*
+******************************************************************
*
* Function Name: RS_MDS_Encode
*
* Function:      Use (12,8) Reed-Solomon code over GF(256) to produce
*                a key S-box dword from two key material dwords.
*
* Arguments:     k0 = 1st dword
*                K1 = 2nd dword
*
* Return:        Remainder polynomial generated using RS code
*
* Notes:
*   Since this computation is done only once per reKey per 64 bits of
*   key, the performance impact of this routine is imperceptible. The
*   RS code chosen has "simple" coefficients to allow smartcard/
*   hardware implementation without lookup tables.
*
-******************************************************************/
DWORD RS_MDS_Encode (DWORD k0, DWORD k1)
    {
    int i, j;
    DWORD r;

    for (i=r=0; i<2;i++)
        {
        r ^=(i) ? k0 : k1;            /* merge in 32 more key bits */
        for (j=0;j<4;j++)             /* shift one byte at a time*/
            RS_rem(r);
        }
    return r;
    }

/*
+*********************************************************************
*
* Function Name:    reKey
*
* Function:         Initialize the Twofish key schedule from key32
*
* Arguments:        key         = ptr to keyInstance to be initialized
*
* Return:           TRUE on success
*
* Notes:
*   Here we precompute all the round subkeys, although that is not
*   actually required. For example, on a smartcard, the round subkeys
*   can be generated on-the-fly using f32()
*
-********************************************************************/
int reKey(keyInstance *key)
    {
    int     i, k64Cnt;
    int     keyLen     = key->keyLen;
    int     subkeyCnt  = ROUND_SUBKEYS + 2*key->numRounds;
    DWORD   A,B;
    /*even/odd key dwords */
    DWORD   k32e[MAX_KEY_BITS/64], k32o [MAX_KEY_BITS/64];

#if VALIDATE_PARMS
  #if ALIGN32
    if ((((int)key) & 3) || (((int)key->key32) & 3))
        return BAD_ALIGN32;
  #endif
    if ((key->keyLen % 64) || (key->keyLen < MIN_KEY_BITS))
        return BAD_KEY_INSTANCE;
    if (subkeyCnt > TOTAL_SUBKEYS)
        return BAD_KEY_INSTANCE;
#endif

    k64Cnt=(keyLen+63)/64;     /* round up to next multiple of 64 bits */
    for (i=0;i<k64Cnt;i++)
        {                         /* split into even/odd key dwords */
         k32e[i]=key->key32[2*i ];
         k32o [i]=key->key32[2*i+1];
         /* compute S-box keys using (12,8) RS-code over GF (256) */
         Key->sboxKeys [k64Cnt-1-i]=RS_MDS_Encode(k32e[i],k32o[i]);
         }
  for (i=0;i<subkeyCnt/2;i++) /* compute round subkeys for PHT */
      {
       A = f32(i*SK_STEP ,k32e, keyLen);
       B = f32 (i*SK_STEP+SK_BUMP, k32o, keyLen);
       B = ROL (B,8);
       key->subKeys [2*i ] = A+ B; /* combine with a PHT */
       key->subKeys [2*i+1] = ROL(A+2*B, SK_ROTL);
       }

   DebugDumpKey(key);

   return TRUE;
   }
/*
+*******************************************************************
*
* Function Name:  makeKey
*
* Function:       Initialize the Twofish key schedule
*
* Arguments:      key         = ptr to keyInstance to be initialized
*                 direction   = DIR_ENCRYPT or DIR_DECRYPT
*                 keyLen      = # bits of key text at *keyMaterial
*                 keyMaterial = ptr to hex ASCII chars representing key
*
* Return:         TRUE on success
*                 else error code (e.g., BAD_KEY_DIR)
*
* Notes:
*   This parses the key bits from keyMaterial. No crypto stuff happens
*   here. The function reKey() is called to actually build the key
*   schedule after the keyMaterial has been parsed.
*
-********************************************************************/
int makeKey(keyInstance *key, BYTE direction,
           int keyLen, CONST char *keyMaterial)
    {
    int i;

#if VALIDATE_PARMS               /* first, check parameters */
    if (key == NULL)
        return BAD_KEY_INSTANCE;  /* must have a keyInstance to init. */
    if ((direction != DIR_ENCRYPT) && (direction !=DIR_DECRYPT))
       return BAD_KEY_DIR;        /* must have valid direction */
    if ((keyLen > MAX_KEY_BITS) || (keyLen < 8))
       return BAD_KEY_MAT;        /* length must be valid */
    if (keyMaterial == NULL)
       return BAD_KEY_MAT;        /* must have some data to work with */
    key->keySig = VALID_SIG;      /* show that we are initialized */
  #if ALIGN32
    if ((((int)key) & 3) || (((int)key->key32) & 3))
        return BAD_ALIGN32;
  #endif
#endif

    key->direction = direction;      /* set our cipher direction */
    key->keyLen    = (keyLen+63) & ~63;            /* round up */
    key->numRounds = numRounds [(keyLen-1)/64];
    for (i=0;i<MAX_KEY_BITS/32;i++) /* zero unused bits */
           key->key32[i]=0;

    if (ParseHexDword(keyLen, keyMaterial, key->key32, key->keyMaterial))
        return BAD_KEY_MAT;

    key->keyMaterial [MAX_KEY_SIZE]=0;    /* terminate ASCII string */

    return reKey(key);    /* generate round subkeys */
    }

/*
+*******************************************************************
*
* Function Name: cipherInit
*
* Function:      Initialize the Twofish cipher in a given mode
*
* Arguments:     cipher = ptr to cipherInstance to be initialized
*                mode   = MODE_ECB, MODE_CBC, or MODE_CFB1
*                IV     = ptr to hex ASCII test representing IV bytes
*
* Return:        TRUE on success
*                else error code (e.g., BAD_CIPHER_MODE)
*
-********************************************************************/
int cipherInit(cipherInstance *cipher, BYTE mode, CONST char *IV)
    {
    int i;
#if VALIDATE_PARMS            /* first, check parameters */
    if (cipher == NULL)
        return BAD_PARAMS;     /* must have a cipherInstance to init */
    if ((mode ! = MODE_ECB) && (mode != MODE_CBC)
       && (mode != MODE_CFB1))
        return BAD_CIPHER_MODE;  /* must have valid cipher mode */
    if ((mode !=MODE_ECB) && (IV == NULL))
        return BAD_PARAMS;   /* must have IV for CBC and CFB modes */
    cipher->cipherSig = VALID_SIG;
  #if ALIGN32
    if ((((int) cipher) & 3) || (((int)cipher->IV) & 3)
       || (((int)cipher->iv32) & 3))
       return BAD_ALIGN32;
  #endif
#endif

    if (mode !=MODE_ECB)          /* parse the IV */
        {
        if (ParseHexDword (BLOCK_SIZE, IV, cipher ->iv32, NULL))
            return BAD_IV_MAT;
        for (i=0;i<BLOCK_SIZE/32;i++) /* byte-oriented copy for CFB1*/
            (DWORD *) cipher->IV) [i] = Bswap (cipher->iv32[i]);
        }
    cipher->mode        =   mode;

    return TRUE;
    }

/*
+*********************************************************************
*
* Function Name: blockEncrypt
*
* Function:      Encrypt block(s) of data using Twofish
*
* Arguments:     cipher    = ptr to already initialized cipherInstance
*                key       = ptr to already initialized keyInstance
*                input     = ptr to data blocks to be encrypted
*                inputLen  = # bits to encrypt (multiple of blockSize)
*                outBuffer = ptr to where to put encrypted blocks
*
* Return:        # bits ciphered (>= 0)
*                else error code (e.g., BAD_CIPHER_STATE, ...)
*
* Notes: The only supported block size for ECB/CBC modes is BLOCK_SIZE
*        bits. If inputLen is not a multiple of BLOCK_SIZE bits in
*        those modes, an error BAD_INPUT_LEN is returned. In CFB1
*        mode, all block sizes can be supported.
*
-*********************************************************************/
int blockEncrypt(cipherInstance *cipher, keyInstance *key,
                 CONST BYTE *input, int inputLen, BYTE *outBuffer)
    {
    int i,n,r;                     /* loop variables */
    DWORD x[BLOCK_SIZE/32];        /* block being encrypted */
    DWORD t0, t1, tmp;             /* temp variables */
    int rounds=key->numRounds;     /* number of rounds */
    BYTE bit, ctBit, carry;        /* temps for CFB */
#if ALIGN32
    BYTE alignDummy;               /* keep 32-bit alignment on stack */
#endif

#if VALIDATE_PARMS
    if ((cipher == NULL) || (cipher->cipherSig != VALID_SIG))
        return BAD_CIPHER_STATE;
    if ((key ==NULL) || (key->keySig != VALID_SIG))
        return BAD_KEY_INSTANCE;
    if ((rounds < 2) || (rounds > MAX_ROUNDS) || (rounds&1))
        return BAD_KEY_INSTANCE;
    if ((cipher->mode != MODE_CFB1) && (inputLen % BLOCK_SIZE))
        return BAD_INPUT_LEN;
  #if ALIGN32
    if ( (((int)cipher) & 3) || (((int)key ) & 3) ||
         (((int)input ) & 3) || (((int)outBuffer) & 3))
        return BAD_ALIGN32;
  #endif
#endif

    if (cipher->mode == MODE_CFB1)
        { /* use recursion here to handle CFB, one block at a time */
         cipher->mode = MODE_ECB;  /* do encryption in ECB */
         for (n=0;n<inputLen;n++)
             {
             blockEncrypt (cipher, key, cipher->IV,BLOCK_SIZE, (BYTE *)x);
             bit = 0x80 >> (n & 7); /* which bit position in byte */
             ctBit = (input[n/8] & bit)
                     ^ (((BYTE *) x) [0] & 0x80) >> (n&7));
             outBuffer[n/8] = (outBuffer[n/8] & ~ bit) || ctBit;
             carry = ctBit >> (7 - (n&7));
             for (i=BLOCK_SIZE/8-1;i>=0;i—)
                 {
                 bit = cipher->IV[i] >> 7; /* save next "carry" */
                 cipher->IV[i] = (cipher->IV[i] << 1) ^ carry;
                 carry = bit;
                 }
             }
         cipher->mode = MODE_CFB1; /* restore mode for next time */
         return inputLen;
         }

     /* here for ECB, CBC modes */
     for (n=0;n<inputLen;n+=BLOCK_SIZE, input+=BLOCK_SIZE/8,
                            outBuffer+=BLOCK_SIZE/8)
        {
#ifdef DEBUG
        DebugDump (input,"\n",-1,0,0,0,1);
        if (cipher->mode == MODE_CBC)
            DebugDump (cipher->iv32, "",IV_ROUND,0,0,0,0);
#endif
        for (i=0;i<BLOCK_SIZE/32;i++) /* copy and add whitening */
            {
            x[i]=Bswap(((DWORD *)input)[i])
                 ^ key->subKeys[INPUT_WHITEN+i];
            if (cipher->mode ==MODE_CBC)
                x[i] ^= cipher->iv32[i];
            }

        DebugDump (x,"",0,0,0,0,0);
        for (r=0;r<rounds;r++)       /* main Twofish encryption loop */
            {
#if FEISTEL
             t0 = f32(ROR(x[0], (r+1)/2), key->sboxKeys, key->keyLen);
             t1 = f32(ROL(x[1],8+(r+1)/2), key->sboxKeys, key->keyLen);
                                       /* PHT, round keys */
             x[2] ^= ROL(t0 + t1 + key->subKeys [ROUND_SUBKEYS+2*r ], r /2);
             x[3] ^= ROR(t0 + 2*t1 + key->subKeys [ROUND_SUBKEYS+2*r+1], (r+2) /2);

             DebugDump(x,"",r+1,2*(r&1),1,1,0);
#else
             t0 = f32( x[0] ,key->sboxKeys,key->keyLen);
             t1 = f32(ROL(x[1],8),key->sboxKeys, key->keyLen);

             x[3] = ROL(x[3],1);
             x[2]^= t0 + t1 + key->subKeys [ROUND_SUBKEYS+2*r ];
             x[3]^= t0 + 2*t1 + key->subKeys       [ROUND_SUBKEYS+2*r+1];
             x[2] = ROR(x[2],1);

             DebugDump(x,"",r+1,2*(r&1),0,1,0); /* use same format */
#endif
             if (r < rounds-1)          /* swap for next round */
                 {
                 tmp = x[0]; x[0]= x[2]; x[2] = tmp;
                 tmp = x[1]; x[1]= x[3]; x[3] = tmp;
                 }
             }
#if FEISTEL
        x[0] = ROR(x[0],8);             /* "final permutation" */
        x[1] = ROR(x[1],8);
        x[2] = ROR(x[2],8);
        x[3] = ROL(x[3],8);
#endif
        for (i=0;i<BLOCK_SIZE/32;i++) /* copy out, with whitening */
            {
            ((DWORD *)outBuffer)[i] =
                Bswap (x[i] ^ key->subKeys[OUTPUT_WHITEN+i]);
            if (cipher->mode == MODE_CBC)
                cipher->iv32[i] = Bswap(((DWORD *)outBuffer)[i]);
            }
#ifdef DEBUG
        DebugDump(outBuffer,"",rounds+1,0,0,0,1);
        if (cipher->mode == MODE_CBC)
            DebugDump(cipher->iv32,"",IV_ROUND,0,0,0,0);
#endif
        }

    return inputLen;
    }

/*
+********************************************************************
*
* Function Name: blockDecrypt
*
* Function:      Decrypt block(s) of data using Twofish
*
* Arguments:     cipher    = ptr to already initialized cipherInstance
*                key       = ptr to already initialized keyInstance
*                input     = ptr to data blocks to be decrypted
*                inputLen  = # bits to encrypt (multiple of blockSize)
*                outBuffer = ptr to where to put decrypted blocks
*
* Return:        #bits ciphered (>= 0)
*                else error code (e.g., BAD_CIPHER_STATE, ...)
*
* Notes: The only supported block size for ECB/CBC modes is BLOCK_SIZE
*        bits. If inputLen is not a multiple of BLOCK_SIZE bits in
*        those modes, an error BAD_INPUT_LEN is returned. In CFB1
*        mode, all block sizes can be supported.
*
-*********************************************************************/
int blockDecrypt(cipherInstance *cipher, keyInstance *key,
                CONST BYTE *input, int inputLen, BYTE *outBuffer)
    {
    int i,n,r;                    /* loop counters */
    DWORD x[BLOCK_SIZE/32];       /* block being encrypted */
    DWORD t0,t1;                  /* temp variables */
    int rounds=key->numRounds;    /* number of rounds*/
    BYTE bit, ctBit, carry;       /* temps for CFB */
#if ALIGN32
    BYTE alignDummy;              /* keep 32-bit alignment on stack */
#endif

#if VALIDATE_PARMS
    if ((cipher == NULL) || (cipher->cipherSig != VALID_SIG))
        return BAD_CIPHER_STATE;
    if ((key == NULL) || (key->keySig != VALID_SIG))
        return BAD_KEY_INSTANCE;
    if ((rounds < 2) || (rounds > MAX_ROUNDS) || (rounds&1))
        return BAD_KEY_INSTANCE;
    if ((cipher->mode != MODE_CFB1) && (inputLen % BLOCK_SIZE))
        return BAD_INPUT_LEN;
#if ALIGN32
  if ( (((int)cipher) & 3) || (((int)key ) & 3) ||
       (((int)input) & 3) || (((int)outBuffer) & 3))
      return BAD_ALIGN32;
 #endif
#endif

    if (cipher->mode == MODE_CFB1)
        { /* use blockEncrypt to handle CFB, one block at a time */
         cipher->mode = MODE_ECB; /* do encryption in ECB */
         for (n=0;n<inputLen;n++)
             {
             blockEncrypt(cipher,key,cipher->IV,BLOCK_SIZE, (BYTE *)x);
             bit = 0x80 >> (n & 7);
             ctBit = input [n/8] & bit;
             outBuffer[n/8] = (outBuffer [n/8] & ~ bit) ||
                         (ctBit ^ (((BYTE *) x) [0] & 0x80) >> (n&7)));
             carry = ctBit >> (7 - (n&7));
             for (i=BLOCK_SIZE/8-1;i>=0;i—)
                 {
                 bit = cipher->IV[i] >> 7; /* save next "carry" */
                 cipher->IV[i] = (cipher->IV[i] << 1) ^ carry;
                 carry = bit;
                 }
         }
     cipher->mode = MODE_CFB1; /* restore mode for next time */
     return inputLen;
     }

/* here for ECB, CBC modes */
for (n=0;n<inputLen;n+=BLOCK_SIZE,input+=BLOCK_SIZE/8,
                    outBuffer+=BLOCK_SIZE/8)
    {
    DebugDump (input,"\n",rounds+1,0,0,0,1);

    for (i=0;i<BLOCK_SIZE/32;i++) /* copy and add whitening */
        x [i]=Bswap (((DWORD *)input)[i])
              ^ key->subKeys[OUTPUT_WHITEN+i];

    for (r=rounds-1;r>=0;r—)            /* main decryption loop */
        {
        t0   = f32 ( x[0] ,key->sboxKeys, key->keyLen);
        t1   = f32(ROL(x[1],8), key->sboxKeys, key->keyLen);

        DebugDump(x,"",r+1,2*(r&1),0,1,0);
        x[2] = ROL(x[2],1);
        x[2]^= t0 + t1 + key->subKeys [ROUND_SUBKEYS+2*r ];
        x[3]^= t0 + 2*t1 + key->subKeys [ROUND_SUBKEYS+2*r+1];
        x[3] = ROR(x[3],1);

        if (r)                /* unswap, except for last round */
            {
            t0 = x[0]; x[0]= x[2]; x[2] = t0;
            t1 = x[1]; x[1]= x[3]; x[3] = t1;
            }
        }
    DebugDump (x,"",0,0,0,0,0);

    for (i=0;i<BLOCK_SIZE/32;i++) /* copy out, with whitening */
        {
        x[i] ^= key->subKeys [INPUT_WHITEN+i];
        if (cipher->mode == MODE_CBC)
            {
            x[i] ^= cipher->iv32[i];
            cipher->iv32[i] = Bswap(((DWORD *)input)[i]);
            }
        ((DWORD *)outBuffer)[i] = Bswap(x[i]);
        }
    DebugDump (outBuffer, "", -1,0,0,0,1);
        }
    return inputLen;
    }

DWORD TwofishCodeSize(void) { return Here(0)-TwofishCodeStart(); };


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

References

[AB96a] | R. Anderson and E. Biham, "Two Practical and Provably Secure Block Ciphers: BEAR and LION," Fast Software Encryption, Third International Workshop Proceedings, Springer-Verlag, 1996, pp. 113-120. [Page 5]
[AB96b] | R. Anderson and E. Biham, "Tiger: A Fast New Hash Function," Fast Software Encryption, Third International Workshop Proceedings, Springer-Verlag, 1996, pp. 89-97. [Pages 38, 46]
[Ada97a] | C. Adams, "Constructing Symmetric Ciphers Using the CAST Design Procedure," Designs, Codes and Cryptography, v.12, n.3, Nov 1997, pp. 71-104. [Pages 5, 35, 38, 39]
[Ada97b] | C. Adams, "DES-80," Workshop on Selected Areas in Cryptography (SAC '97) Workshop Record, School of Computer Science, Carleton University, 1997, pp. 160-171. [Page 37]
[AGMP96] | G. Alvarez, D. De la Guia, F. Montoya, and A. Peinado, "Akelarre: A New Block Cipher Algorithm," Workshop on Selected Areas in Cryptography (SAC '96) Workshop Record, Queens University, 1996, pp. 1-14. [Page 36]
[AK96] | R. Anderson and M. Kuhn, "Tamper Resistance — A Cautionary Note," Proceedings of the Second USENIX Workshop on Electronic Commerce, USENIX Press, 1996, pp. 1-11. [Page 111]
[Anon95] | Anonymous, "this looked like it might be interesting," sci.crypt Usenet posting, 9 Aug 1995. [Pages 41, 104]
[AT90] | C.M. Adams and S.E. Tavares, "The Structured Design on Cryptographically Good S-Boxes," Journal of Cryptology, v. 3,n. 1, 1990, pp. 27-41. [Page 39]
[AT93] | C.M. Adams and S.E. Tavares, "Designing S-boxes for Ciphers Resistant to Differential Cryptanalysis," Proceedings of the 3rd Symposium on State and Progress of Research in Cryptography, Rome, Italy, 15-16 Feb 1993, pp. 181-190. [Pages 35, 39]
[BAK98] | E. Biham, R. Anderson, and L. Knudsen, "Serpent: A New Block Cipher Proposal," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 222-238. [Pages 30, 34, 38, 41]
[Bau93] | F.L. Bauer, Kryptologie, Springer-Verlag, 1993. (In German.) [Page 49]
[Bau97] | F.L. Bauer, Decrypted Secrets, Springer-Verlag, 1997. [Page 49]
[BB93] | I. Ben-Aroya and E. Biham, "Differential Cryptanalysis of Lucifer," Advances in Cryptology — CRYPTO '93 Proceedings, Springer-Verlag, 1994, pp. 187-199. [Page 75]
[BB94] | E. Biham and A. Biryukov, "How to Strengthen DES Using Existing Hardware," Advances in Cryptology — ASIACRYPT '94 Proceedings, Springer-Verlag, 1994, pp. 398-412. [Page 39]
[BB95] | E. Biham and A. Biryukov, "An Improvement of Davies' Attack on DES," Advances in Cryptology — EUROCRYPT '94 Proceedings, Springer-Verlag, 1995, pp. 461-467. [Pages 40, 114]
[BB96] | U. Blumenthal and S. Bellovin, "A Better Key Schedule for DES-Like Ciphers," Pragocrypt '96 Proceedings, 1996, pp. 42-54. [Page 37]
[BCK96] | M. Bellare, R. Canetti, and H. Karwczyk, "Keying Hash Functions for Message Authentication," Advances in Cryptology — CRYPTO '96 Proceedings, Springer-Verlag, 1996, pp. 1-15. [Page 120]
[BDL97] | D. Boneh, R.A. DeMillo, and R.J. Lipton "On the Importance of Checking Cryptographic Protocols for Faults," Advances in Cryptology — EUROCRYPT '97 Proceedings, Springer-Verlag, 1997, pp. 37-51.[Page 111]
[BDR+96] | M. Blaze, W. Diffie, R. Rivest, B. Schneier, T. Shimomura, E. Thompson, and M. Weiner, "Minimal Key Lengths for Symmetric Ciphers to Provide Adequate Commercial Security," Jan 1996. [Page 1]
[BFL96] | M. Blaze, J. Feigenbaum, and F. T. Leighton, Master-Key Cryptosystems, DIMACS Technical Report 96-02, Rutgers University, Piscataway, 1996. [Page 116]
[Bih94] | E. Biham, "New Types of Cryptanalytic Attacks Using Related Keys," Journal of Cryptology, v. 7, n. 4, 1994, pp. 229-246. [Pages 35, 104, 105, 113]
[Bih95] | E. Biham, "On Matsui's Linear Cryptanalysis," Advances in Cryptology — EUROCRYPT '94 Proceedings, Springer-Verlag, 1995, pp. 398-412. [Page 38]
[Bih97] | E. Biham, "A Fast New DES Implementation in Software," Fast Software Encryption, 4th International Workshop Proceedings, Springer-Verlag, 1997, pp. 260-271. [Page 34]
[BK98] | A. Biryukov and E. Kushilevitz, "Improved Cryptanalysis of RC5," Advances in Cryptology — EUROCRYPT '98 Proceedings, Springer-Verlag, 1998, pp. 85-99. [Page 36]
[BKPS93] | L. Brown, M. Kwan, J. Pieprzyk, and J. Seberry, "Improving Resistance to Differential Cryptanalysis and the Redesign of LOKI," Advances in Cryptology — ASIACRYPT '91 Proceedings, Springer-Verlag, 1993, pp. 36-50. [Pages 5, 33]
[BKR94] | M. Bellare, J. Kilian, and P. Rogaway, "The security of the cipher block chaining message authentication code," Advances in Cryptology — CRYPTO '94, Springer-Verlag, 1994. [Page 120]
[BPS90] | L. Brown, J. Pieprzyk, and J. Seberry, "LOKI: A Cryptographic Primitive for Authentication and Secrecy Applications," Advances in Cryptology — AUSCRYPT '90 Proceedings, Springer-Verlag, 1990, pp. 229-236. [Pages 5, 33]
[Bro98] | L. Brown, "Design of LOKI97," draft AES submission, 1998. [Page 39]
[BS92] | E. Biham and A. Shamir, "Differential Cryptanalysis of Snefru, Khafre, REDOC II, LOKI, and Lucifer," Advances in Cryptology — CRYPTO '91 Proceedings, Springer-Verlag, 1992, pp. 156-171. [Page 38]
[BS93] | E. Biham and A. Shamir, Differential Cryptanalysis of the Data Encryption Standard, Springer-Verlag, 1993. [Pages 35, 38]
[BS95] | M. Blaze and B. Schneier, "The MacGuffin Block Cipher Algorithm," Fast Software Encryption, Second International Workshop Proceedings, Springer-Verlag, 1995, pp. 97-110. [Pages 5, 46]
[BS97] | E. Biham and A. Shamir, "Differential Fault Analysis of Secret Key Cryptosystems," Advances in Cryptology — CRYPTO '97 Proceedings, Springer-Verlag, 1997, pp. 513-525. [Page 111]
[CDN95] | G. Carter, E. Dawson, and L. Nielsen, "DESV: A Latin Square Variation of DES," Proceedings of the Workshop on Selected Areas in Cryptography (SAC '95), Ottawa, Canada, 1995, pp. 158-172. [Page 58]
[CDN98] | G. Carter, E. Dawson, and L. Nielsen, "Key Schedules of Iterative Block Ciphers," Third Australian Conference, ACISP '98, Springer-Verlag, to appear. [Page 41]
[Cla97] | C.S.K. Clapp, "Optimizing a Fast Stream Cipher for VLIW, SIMD, and Superscalar Processors," Fast Software Encryption, 4th International Workshop Proceedings, Springer-Verlag, 1997, pp. 273-287. [Page 38]
[Cla98] | C.S.K. Clapp, "Joint Hardware/Software Design of a Fast Stream Cipher," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 75-92. [Page 38]
[CM98] | H. Chabanne and E. Michon, "JEROBOAM," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 49-59. [Page 33]
[Cop94] | D. Coppersmith, "The Data Encryption Standard (DES) and its Strength Against Attacks," IBM Journal of Research and Development, v. 38, n. 3, May 1994, pp. 243-250. [Pages 38, 39]
[Cop98] | D. Coppersmith, personal communication, 1998. [Page 38]
[CW91] | T. Cusick and M.C. Wood, "The REDOC-II Cryptosystem," Advances in Cryptology — CRYPTO '90 Proceedings, Springer-Verlag, 1991, pp. 545-563. [Pages 33, 36, 39]
[CWSK98] | D. Coppersmith, D. Wagner, B. Schneier, and J. Kelsey, "Cryptanalysis of TWOPRIME," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 32-48. [Pages 40, 114]
[Dae95] | J. Daemen, "Cipher and Hash Function Design," Ph.D. thesis, Katholieke Universiteit Leuven, Mar 95. [Page 104]
[DBP96] | H. Dobbertin, A. Bosselaers, and B. Preneel, "RIPEMD-160: A Strengthened Version of RIPEMD," Fast Software Encryption, Third International Workshop Proceedings, Springer-Verlag, 1996, pp. 71-82. [Page 58]
[DC98a] | J. Daemen and C. Clapp, "Fast Hashing and Stream Encryption with PANAMA," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 60-74. [Page 37]
[DC98b] | J. Daemen and C. Clapp, "The Panama Cryptographic Function," Dr. Dobbs Journal, v. 23, n. 12, Dec 1998, pp. 42-49. [Page 37]
[DGV93] | J. Daemen, R. Govaerts, and J. Vandewalle, "Block Ciphers Based on Modular Arithmetic," Proceedings of the 3rd Symposium on: State and Progress of Research in Cryptography, Fondazione Ugo Bordoni, 1993, pp. 80-89. [Page 36]
[DGV94a] | J. Daemen, R. Govaerts, and J. Vandewalle, "Weak Keys for IDEA," Advances in Cryptology — EUROCRYPT '93 Proceedings, Springer-Verlag, 1994, pp. 159-167. [Page 75]
[DGV94b] | J. Daemen, R. Govaerts, and J. Vandewalle, "A New Approach to Block Cipher Design," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer-Verlag, 1994, pp. 18-32. [Pages 35, 104]
[DH76] | W. Diffie and M. Hellman, "New Directions in Cryptography," IEEE Transactions on Information Theory, v. IT-22, n. 6, Nov 1976, pp. 644-654. [Page 116]
[DH79] | W. Diffie and M. Hellman, "Exhaustive Cryptanalysis of the NBS Data Encryption Standard," Computer, v. 10, n. 3, Mar 1979, pp. 74-84. [Page 1]
[DK85] | C. Deavours and L.A. Kruh, Machine Cryptography and Modern Cryptanalysis, Artech House, Dedham MA, 1985. [Page 49]
[DKR97] | J. Daemen, L. Knudsen, and V. Rijmen, "The Block Cipher Square," Fast Software Encryption, 4th International Workshop Proceedings, Springer-Verlag, 1997, pp. 149-165. [Pages 6, 36, 44, 54]
[ElG85] | T. ElGamal, "A Public-Key Cryptosystem and a Signature Scheme Based on Discrete Logarithms," IEEE Transactions on Information Theory, v. IT-31, n. 4, 1985, pp. 469-472. [Page 116]
[Fei73] | H. Feistel, "Cryptography and Computer Privacy," Scientific American, v. 228, n. 5, May 1973, pp. 15-23. [Pages 5, 44]
[Fer96] | N. Ferguson, personal communication, 1996. [Page 7]
[Fer98a] | N. Ferguson, "Bounds on the tail of binomial distributions," research notes, 1998. [Page 93]
[Fer98b] | N. Ferguson, "Upper Bounds on Differential Characteristics in Two-fish," Twofish Technical Report #1, Counterpane Systems, Aug 1998. [Page v]
[FNS75] | H. Feistel, W.A. Notz, and J.L. Smith, "Some Cryptographic Techniques for Machine-to-Machine Data Communications, Proceedings on the IEEE, v. 63, n. 11, 1975, pp. 1545-1554. [Pages 5, 44]
[FS97] | N. Ferguson and B. Schneier, "Cryptanalysis of Akelarre," Workshop on Selected Areas in Cryptography (SAC '97) Workshop Record, School of Computer Science, Carleton University, 1997, pp. 201-212. [Page 36]
[GC94] | H. Gilbert and P. Chauvaud, "A Chosen-Plaintext Attack on the 16-Round Khufu Cryptosystem," Advances in Cryptology — CRYPTO '94 Proceedings, Springer-Verlag, 1994, pp. 359-368. [Page 38]
[GOST89] | GOST, Gosudarstvennyi Standard 28147-89, "Cryptographic Protection for Data Processing Systems," Government Committee of the USSR for Standards, 1989. [Pages 5, 38, 104]
[Gut98] | P. Gutmann, "Software Generation of Random Numbers for Crypto-graphic Purposes," Proceedings of the 1998 USENIX Security Symposium, USENIX Press, 1998, pp. 243-257. [Page 120]
[Har96] | C. Harpes, Cryptanalysis of Iterated Block Ciphers, ETH Series on Information Processing, v. 7, Hartung-Gorre Verlang Konstanz, 1996. [Pages 102, 116]
[Haw98] | P. Hawkes, "Differential-Linear Weak Key Classes of IDEA," Advances in Cryptology — EUROCRYPT '98 Proceedings, Springer-Verlag, 1998, pp. 112-126. [Page 75]
[HKM95] | C. Harpes, G. Kramer, and J. Massey, "A Generalization of Linear Cryptanalysis and the Applicability of Matsui's Piling-up Lemma," Advances in Cryptology — EUROCRYPT '95 Proceedings, Springer-Verlag, 1995, pp. 24-38. [Page 102]
[HKR+98] | C. Hall, J. Kelsey, V. Rijmen, B. Schneier, and D. Wagner, "Cryptanalysis of SPEED," Selected Areas in Cryptography, Springer-Verlag, 1998, to appear. [Page 33]
[HKSW98] | C. Hall, J. Kelsey, B. Schneier, and D. Wagner, "Cryptanalysis of SPEED," Financial Cryptography '98 Proceedings, Springer-Verlag, 1998, pp. 309-310. [Page 33]
[HM97] | C. Harpes and J. Massey, "Partitioning Cryptanalysis," Fast Software Encryption, 4th International Workshop Proceedings, Springer-Verlag, 1997, pp. 13-27. [Page 102]
[HT94a] | H.M. Heys and S.E. Tavares, "On the Security of the CAST Encryption Algorithm," Canadian Conference on Electrical and Computer Engineering, 1994, pp. 332-335. [Page 35]
[HT94b] | H.M. Heys and S.E. Tavares, "The Design of Substitution-Permutation Networks Resistant to Differential and Linear Cryptanalysis," 2nd ACM Conference on Computer and Communications Security, ACM Press, 1994, pp. 148-155. [Page 44]
[HYSK98] | T. Hamade, T. Yokoyama, T. Shimada, and T. Kanedo, "On Partitioning Cryptanalysis of DES," 1998 Symposium on Cryptography and Information Security, 2.2.A. (In Japanese.) [Page 103]
[Jeff+76] | T. Jefferson et al., "Declaration of Independence," Philadelphia PA, 4 Jul 1776. [Page 38]
[JH96] | T. Jakobsen and C. Harpes, "Bounds on Non-Uniformity Measures for Generalized Linear Cryptanalysis and Partitioning Cryptanalysis," Pragocrypt '96 Proceedings, 1996, pp. 467-479. [Page 102]
[JK97] | T. Jakobsen and L. Knudsen, "The Interpolation Attack on Block Ciphers," Fast Software Encryption, 4th International Workshop Proceedings, Springer-Verlag, 1997, pp. 28-40. [Pages 35, 39, 103]
[Kie96] | K. Kiefer, "A New Design Concept for Building Secure Block Ciphers," Proceedings of the 1st International Conference on the Theory and Applications of Cryptography, Pragocrypt '96, CTU Publishing House, 1996, pp. 30-41. [Page 35]
[KKT94] | T. Kaneko, K. Koyama, and R. Terada, "Dynamic Swapping Schemes and Differential Cryptanalysis," IEICE Transactions, v. E77-A, 1994, pp. 1328-1336. [Page 58]
[KLPL95] | K. Kim, S. Lee, S. Park, and D. Lee, "Securing DES S-boxes Against Three Robust Cryptanalysis," Proceedings of the Workshop on Selected Areas in Cryptography (SAC '95), Ottawa, Canada, 1995, pp. 145-157. [Page 38]
[KM97] | L.R. Knudsen and W. Meier, "Differential Cryptanalysis of RC5," European Transactions on Communication, v. 8, n. 5, 1997, pp. 445-454. [Page 36]
[Knu93a] | L.R. Knudsen, "Cryptanalysis of LOKI," Advances in Cryptology — ASIACRYPT '91, Springer-Verlag, 1993, pp. 22-35. [Page 39]
[Knu93b] | L.R. Knudsen, "Cryptanalysis of LOKI91," Advances in Cryptology — AUSCRYPT '92, Springer-Verlag, 1993, pp. 196-208. [Page 39]
[Knu93c] | L.R. Knudsen, "Iterative Characteristics of DES and s2DES," Advances in Cryptology — CRYPTO '92, Springer-Verlag, 1993, pp. 497-511. [Page 38]
[Knu94a] | L.R. Knudsen, "Block Ciphers — Analysis, Design, Applications," Ph.D. dissertation, Aarhus University, Nov 1994. [Page 35]
[Knu94b] | L.R. Knudsen, "Practically Secure Feistel Ciphers," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer-Verlag, 1994, pp. 211-221. [Pages 35, 37, 75]
[Knu95a] | L.R. Knudsen, "New Potentially 'Weak' Keys for DES and LOKI," Advances in Cryptology — EUROCRYPT '94 Proceedings, Springer-Verlag, 1995, pp. 419-424. [Page 40]
[Knu95b] | L.R. Knudsen, "Truncated and Higher Order Differentials," Fast Software Encryption, 2nd International Workshop Proceedings, Springer-Verlag, 1995, pp. 196-211. [Pages 35, 39, 90]
[Knu95c] | L.R. Knudsen, "A Key-Schedule Weakness in SAFER K-64," Advances in Cryptology — CRYPTO '95 Proceedings, Springer-Verlag, 1995, pp. 274-286. [Page 60]
[Koc96] | P. Kocher, "Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and Other Systems," Advances in Cryptology — CRYPTO '96 Proceedings, Springer-Verlag, 1996, pp. 104-113. [Page 111]
[Koc98a] | P. Kocher, "Differential Power Analysis, " available online from http://www.cryptography.com/dpa/. [Page 111]
[Koc98b] | P. Kocher, "DES Key Search Project," available online from http://www.cryptography.com/des/index.html. [Page 1]
[KPL93] | K. Kim, S. Park, and S. Lee, "Reconstruction of s2DES S-Boxes and their Immunity to Differential Cryptanalysis," Proceedings of the 1993 Japan-Korea Workshop on Information Security and Cryptography, Seoul, Korea, 24-26 October 1993, pp. 282-291. [Page 38]
[KR94] | B. Kaliski Jr. and M. Robshaw, "Linear Cryptanalysis Using Multiple Approximations," Advances in Cryptology — CRYPTO '94 Proceedings, Springer-Verlag, 1994, pp. 26-39. [Page 102]
[KR95] | B. Kaliski Jr. and M. Robshaw, "Linear Cryptanalysis Using Multiple Approximations and FEAL," Fast Software Encryption, Second International Workshop Proceedings, Springer-Verlag, 1995, pp. 249-264. [Page 102]
[KR96a] | J. Kilian and P. Rogaway, "How to Protect DES Against Exhaustive Key Search," Advances in Cryptology — CRYPTO '96 Proceedings, Springer-Verlag, 1996, pp. 252-267. [Pages 5, 6]
[KR96b] | L. Knudsen and M. Robshaw, "Non-Linear Approximations in Linear Cryptanalysis," Advances in Cryptology — EUCROCRYPT '96, Springer-Verlag, 1996, pp. 224-236. [Page 102]
[KR97] | L.R. Knudsen and V. Rijmen, "Two Rights Sometimes Make a Wrong," Workshop on Selected Areas in Cryptography (SAC '97) Workshop Record, School of Computer Science, Carleton University, 1997, pp. 213-223. [Page 36]
[KRRR98] | L.R. Knudsen, V. Rijmen, R. Rivest, and M. Robshaw, "On the Design and Security of RC2," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 206-221. [Page 33]
[KSHW98] | J. Kelsey, B. Schneier, C. Hall, and D. Wagner, "Secure Applications of Low-Entropy Keys," Information Security. First International Workshop ISW '97 Proceedings, Springer-Verlag, 1998, 121-134. [Page 42]
[KSW96] | J. Kelsey, B. Schneier, and D. Wagner, "Key-Schedule Cryptanalysis of IDEA, G-DES, GOST, SAFER, and Triple-DES," Advances in Cryptology — CRYPTO '96 Proceedings, Springer-Verlag, 1996, pp. 237-251. [Pages 35, 41, 79, 104]
[KSW97] | J. Kelsey, B. Schneier, and D. Wagner, "Related-Key Cryptanalysis of 3-WAY, Biham-DES, CAST, DES-X, NewDES, RC2, and TEA," Information and Communications Security, First International Conference Proceedings, Springer-Verlag, 1997, pp. 203-207. [Pages 35, 41, 79, 104]
[KSWH98a] | J. Kelsey, B. Schneier, D. Wagner, and C. Hall, "Cryptanalytic Attacks on Pseudorandom Number Generators," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 168-188. [Page 120]
[KSWH98b] | J. Kelsey, B. Schneier, D. Wagner, and C. Hall, "Side Channel Cryptanalysis of Product Ciphers," ESORICS '98 Proceedings, Springer-Verlag, 1998, pp 97-110. [Page 111]
[KSWH98c] | J. Kelsey, B. Schneier, D. Wagner, and C. Hall, "Yarrow: A Pseudo-random Number Generator," in preparation. [Page 120]
[Kwa97] | M. Kwan, "The Design of ICE Encryption Algorithm," Fast Software Encryption, 4th International Workshop Proceedings, Springer-Verlag, 1997, pp. 69-82. [Page 41]
[KY95] | B.S. Kaliski and Y.L. Yin, "On Differential and Linear Cryptanalysis of the RC5 Encryption Algorithm," Advances in Cryptology — CRYPTO '95 Proceedings, Springer-Verlag, 1995, pp. 445-454. [Page 36]
[Lai94] | X. Lai, "Higher Order Derivations and Differential Cryptanalysis," Communications and Cryptography: Two Sides of One Tapestry, Kluwer Academic Publishers, 1994, pp. 227-233. [Page 35, 90]
[LC97] | C.-H. Lee and Y.-T. Cha, "The Block Cipher: SNAKE with Provable Resistance Against DC and LC Attacks, " Proceedings of JW-ISC '97, KIISC and ISEC Group of IEICE, 1997, pp. 3-17. [Page 35]
[Lee96] | M. Leech, "CRISP: A Feistel Network with Hardened Key Scheduling," Workshop on Selected Areas in Cryptography (SAC '96) Workshop Record, Queens University, 1996, pp. 15-29. [Page 37]
[LH94] | S. Langford and M. Hellman, "Differential-Linear Cryptanalysis," Advances in Cryptology — CRYPTO '94 Proceedings, Springer-Verlag, 1994, pp. 17-26. [Page 103]
[LM91] | X. Lai and J. Massey, "A Proposal for a New Block Encryption Standard," Advances in Cryptology — EUROCRYPT '90 Proceedings, Springer-Verlag, 1991, pp. 389-404. [Page 33]
[LMM91] | X. Lai, J. Massey, and S. Murphy, "Markov Ciphers and Differential Cryptanalysis," Advances in Cryptology — CRYPTO '91 Proceedings, Springer-Verlag, 1991, pp. 17-38. [Pages 33, 46]
[MA96] | S. Mister and C. Adams, "Practical S-Box Design," Workshop on Selected Areas in Cryptography (SAC '96) Workshop Record, Queens University, 1996, pp. 61-76. [Page 38]
[Mad84] | W.E. Madryga, "A High Performance Encryption Algorithm," Computer Security: A Global Challenge, Elsevier Science Publishers, 1984, pp. 557-570. [Page 36]
[Mas94] | J.L. Massey, "SAFER K-64: A Byte-Oriented Block-Ciphering Algorithm," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer-Verlag, 1994, pp. 1-17. [Pages 7, 44, 121]
[Mat94] | M. Matsui, "Linear Cryptanalysis Method for DES Cipher," Advances in Cryptology — EUROCRYPT '93 Proceedings, Springer-Verlag, 1994, pp. 386-397. [Page 35]
[Mat95] | M. Matsui, "On Correlation Between the Order of S-Boxes and the Strength of DES," Advances in Cryptology — EUROCRYPT '94 Proceedings, Springer-Verlag, 1995, pp. 366-375. [Page 38]
[Mat96] | M. Matsui, "New Structure of Block Ciphers with Provable Security Against Differential and Linear Cryptanalysis," Fast Software Encryption, 3rd International Workshop Proceedings, Springer-Verlag, 1996, pp. 205-218. [Pages 35, 48]
[Mat97] | M. Matsui, "New Block Encryption Algorithm MISTY," Fast Software Encryption, 4th International Workshop Proceedings, Springer-Verlag, 1997, pp. 54-68. [Page 35]
[McD97] | T.J. McDermott, "NSA comments on criteria for AES," letter to NIST, National Security Agency, 2 Apr 97. [Pages 32, 42]
[Mer91] | R.C. Merkle, "Fast Software Encryption Functions," Advances in Cryptology — CRYPTO '90 Proceedings, Springer-Verlag, 1991, pp. 476-501. [Pages 5, 33, 46]
[Mor98] | S. Moriai, "How to Design Secure S-boxes Against Different Differential, Liear, Higher Order Differential,and Interpolation Attacks," 1998 Symposium on Cryptography and Information Security, 2.2.C. (In Japanese.) [Page 39]
[MS77] | F.J. MacWilliams and N.J.A. Sloane, "The Theory of Error-Correcting Codes," North-Holland, Amsterdam, 1977. [Pages 6, 76]
[MSK98a] | S. Moriai, T. Shimoyama, and T. Kaneko, "Higher Order Differential Attack of a CAST Cipher," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 17-31. [Page 35]
[MSK98b] | S. Moriai, T. Shimoyama, and T. Kaneko, "Interpolation Attacks of the Block Cipher: SNAKE," 1998 Symposium on Cryptography and Information Security, 7.2.C. (In Japanese.) [Pages 35, 103]
[Mur90] | S. Murphy, "The Cryptanalysis of FEAL-4 with 20 Chosen Plaintexts," Journal of Cryptology, v. 2, n. 3, 1990, pp. 145-154. [Page 39]
[NBS77] | National Bureau of Standards, NBS FIPS PUB 46, "Data Encryption Standard," National Bureau of Standards, U.S. Department of Commerce, Jan 1977. [Pages 1, 5, 34]
[NBS80] | National Bureau of Standards, NBS FIPS PUB 46, "DES Modes of Operation," National Bureau of Standards, U.S. Department of Commerce, Dec 1980.[Page 119]
[NIST93] | National Institute of Standards and Technology, "Secure Hash Standard," U.S. Department of Commerce, May 1993. [Pages 39, 46, 58]
[NIST94] | National Institute of Standards and Technologies, NIST FIPS PUB 186, "Digital Signature Standard," U.S. Department of Commerce, May 1994. [Page 116]
[NIST97a] | National Institute of Standards and Technology, "Announcing Development of a Federal Information Standard for Advanced Encryption Standard," Federal Register, v. 62, n. 1, 2 Jan 1997, pp. 93-94. [Page 1]
[NIST97b] | National Institute of Standards and Technology,"Announcing Request for Candidate Algorithm Nominations for the Advanced Encryption Standard (AES)," Federa Register, v. 62, n. 117, 12 Sep 1997, pp. 48051-48058. [Pages 1, 3, 34]
[NK95] | K. Nyberg and L.R. Knudsen, "Provable Security Against Differential Cryptanalysis," Journal of Cryptology, v. 8, n. 1, 1995, pp. 27-37. [Page 35]
[NM97] | J. Nakajima and M. Matsui, "Fast Software Implementation of MISTY on Alpha Processors," Proceedings of JW-ISC '97, KIISC and ISEC Group of IEICE, 1997, pp. 55-63. [Page 34]
[NSA98] | NSA, "Skipjack and KEA Algorithm Specifications," Version 2.0, National Security Agency, 29 May 1998. [Pages 5, 40, 46]
[Nyb91] | K. Nyberg, "Perfect Nonlinear S-boxes," Advances in Cryptology — EUROCRYPT '91 Proceedings, Springer-Verlag, 1991, pp. 378-386. [Page 35]
[Nyb93] | K. Nyberg, "On the Construction of Highly Nonlinear Permutations," Advances in Cryptology — EUROCRYPT '92 Proceedings, Springer-Verlag, 1993, pp. 92-98. [Page 35]
[Nyb94] | K. Nyberg, "Differentially Uniform Mappings for Cryptography," Advances in Cryptology — EUROCRYPT '93 Proceedings, Springer-Verlag, 1994, pp. 55-64. [Page 35]
[Nyb95] | K. Nyberg, "Linear Approximation of Block Ciphers," Advances in Cryptology — EUROCRYPT '94 Proceedings, Springer-Verlag, 1995, pp. 439-444. [Page 35]
[Nyb96] | K. Nyberg, "Generalized Feistel Networks," Advances in Cryptology — ASIACRYPT '96 Proceedings, Springer-Verlag, 1996, pp. 91-104. [Pages 35, 36]
[OCo94a] | L. O'Connor, "Enumerating Nondegenerate Permutations," Advances in Cryptology — EUROCRYPT '93 Proceedings, Springer-Verlag, 1994, pp. 368-377. [Page 35]
[OCo94b] | L. O'Connor, "On the Distribution of Characteristics in Bijective Mappings," Advances in Cryptology — EUROCRYPT '93 Proceedings, Springer-Verlag, 1994, pp. 360-370. [Page 35]
[OCo94c] | L. O'Connor, "On the Distribution of Characteristics in Composite Permutations," Advances in Cryptology — CRYPTO '93 Proceedings, Springer-Verlag, 1994, pp. 403-412. [Page 35]
[PB92] | H. Beker and F. Piper, Cipher Systems, Northwind Books, 1992. [Page 49]
[Plu94] | C. Plumb, "Truly Random Numbers," Dr. Dobbs Journal, v. 19, n. 13, Nov 1994, pp. 113-115. [Page 120]
[PRB98a] | B. Preneel, V. Rijmen, and A. Bosselaers, "Principles and Performance of Cryptographic Algorithms," Dr. Dobbs Journal, v. 23, n. 12, Dec 1998, pp. [Page 121]
[PRB98b] | B. Preneel, V. Rijmen, and A. Bosselaers, "Recent Developments in the Design of Conventional Cryptographic Algorithms," State of the Art and Evolution of Computer Security and Industrial Cryptography, Lecture Notes in Computer Science, B. Preneel, R. Govaerts, J. Vandewalle, Eds., Springer-Verlag, 1998, to appear. [Page 121]
[Pre93] | B. Preneel, Analysis and Design of Cryptographic Hash Functions, Ph.D. dissertation, Katholieke Universiteit Leuven, Jan 1993. [Pages 23, 119]
[QDD86] | J.-J. Quisquater, Y. Desmedt, and M. Davio, "The Importance of 'Good' Key Scheduling Schemes," Advances in Cryptology — CRYPTO '85 Proceedings, Springer-Verlag, 1986, pp. 537-542. [Page 42]
[RAND55] | RAND Corporation, A Million Random Digits with 100,000 Normal Deviates, Glencoe, IL, Free Press Publishers, 1955. [Page 38]
[RC94] | P. Rogaway and D. Coppersmith, "A Software-Optimized Encryption Algorithm," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer-Verlag, 1994, pp. 56-63. [Page 33, 34]
[RC98] | P. Rogaway and D. Coppersmith, "A Software-Optimized Encryption Algorithm," Journal of Cryptology, v. 11, n. 4, 1998, pp. 273-287. [Pages 33, 34]
[RDP+96] | V. Rijmen, B. Preneel, A. Bosselaers, and E. DeWin, "The Cipher SHARK," Fast Software Encryption, 3rd International Workshop Proceedings, Springer-Verlag, 1996, pp. 99-111. [Pages 5, 6, 44]
[Rij97] | V. Rijmen, Cryptanalysis and Design of Iterated Block Ciphers, Ph. D. thesis, Katholieke Universiteit Leuven, Oct 1997. [Page 38]
[RIPE92] | Research and Development in Advanced Communication Technologies in Europe, RIPE Integrity Primitives: Final Report of RACE Integrity Primitives Evaluation (R1040), RACE, June 1992. [Page 58]
[Rit96] | T. Ritter, "The Fenced DES Cipher: Stronger than DES but Made from DES," Ritter Software Engineering, 1996. [Page 46]
[Riv91] | R.L. Rivest, "The MD4 Message Digest Algorithm," Advances in Cryptology — CRYPTO '90 Proceedings, Springer-Verlag, 1991, pp. 303-311. [Pages 46, 58]
[Riv92] | R.L. Rivest, "The MD5 Message Digest Algorithm," RFC 1321, Apr 1992. [Pages 46, 58]
[Riv95] | R.L. Rivest, "The RC5 Encryption Algorithm," Fast Software Encryption, 2nd International Workshop Proceedings, Springer-Verlag, 1995, pp. 86-96. [Pages 5, 34, 121]
[Riv97] | R. Rivest, "A Description of the RC2(r) Encryption Algorithm," Internet-Draft, work in progress, June 1997. [Pages 33, 46]
[RKR98] | B. Van Rompay, L.R. Knudsen, and V. Rijmen, "Differential Cryptanalysis of the ICE Encryption Algorithm," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 270-283. [Page 41]
[Ros98] | G. Rose, "A Stream Cipher Based on Linear Feedback over GF(28)," Third Australian Conference, ACISP '98, Springer-Verlag, to appear. [Page 33]
[RP95a] | V. Rijmen and B. Preneel, "Cryptanalysis of MacGuffin," Fast Software Encryption, Second International Workshop Proceedings, Springer-Verlag, 1995, pp. 353-358. [Page 5]
[RP95b] | V. Rijmen and B. Preneel, "On Weaknesses of Non-surjective Round Functions," Proceedings of the Workshop on Selected Areas in Cryptography (SAC '95), Ottawa, Canada, 1995, pp. 100-106. [Pages 40, 114]
[RPD97] | V. Rijman, B. Preneel, and E. DeWin, "On Weaknesses of Non-surjective Round Functions," Designs, Codes, and Cryptography, v. 12, n. 3, 1997, pp. 253-266. [Pages 40, 114]
[RSA78] | R. Rivest, A. Shamir, and L. Adleman, "A Method for Obtaining Digital Signatures and Public-Key Cryptosystems," Communications of the ACM, v. 21, n. 2, Feb 1978, pp. 120-126. [Page 116]
[SAM97] | T. Shimoyama, S. Amada, and S. Moriai, "Improved Fast Software Implementation of Block Ciphers," Information and Communications Security, First International Conference, ICICS '97 Proceedings, Springer-Verlag, 1997, pp. 203-207. [Page 34]
[Sch94] | B. Schneier, "Description of a New Variable-Length Key, 64-Bit Block Cipher (Blowfish)," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer-Verlag, 1994, pp. 191-204. [Pages 5, 34, 39]
[Sch96] | B. Schneier, Applied Cryptography, Second Edition, John Wiley & Sons, 1996. [Pages 23, 34, 119]
[Sch98] | B. Schneier, "The Twofish Encryption Algorithm," Dr. Dobbs Journal, v . 23, n. 12, Dec 1998, pp. 30-38. [Page v]
[Sco85] | R. Scott, "Wide Open Encryption Design Offers Flexible Implementation," Cryptologia, v. 9, n. 1, Jan 1985, pp. 75-90. [Page 38]
[Sel98] | A.A. Selcuk, "New Results in Linear Cryptanalysis of RC5," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 1-16.[Page 36]
[Seu60] | Dr. Seuss, One Fish, Two Fish, Red Fish, Blue Fish, Beginner Books, 1960. [Page 127]
[Sha49] | C. Shannon, "Communication Theory of Secrecy Systems," Bell Systems Technical Journal, v. 28, n. 4, 1949,pp. 656-715. [Pages 43, 44]
[SK96] | B. Schneier and J. Kelsey, "Unbalanced Feistel Networks and Block Cipher Design," Fast Software Encryption, 3rd International Workshop Proceedings, Springer-Verlag, 1996, pp. 121-144. [Pages 5, 36, 46, 47]
[SK98] | T. Shimoyama and T. Kaneko, "Quadratic Relation of S-box and Its Application to the Linear Attack of Full Round DES," Advances in Cryptology — CRYPTO '98 Proceedings, Springer-Verlag, 1998, pp. 200-211. [Pages 102, 116]
[SKW+98a] | B. Schneier, J. Kelsey, D. Whiting, D. Wagner, C. Hall, and N. Ferguson, "Twofish: A 128-Bit Block Cipher," NIST AES Proposal, 15 June 1998. [Page v]
[SKW+98b] | B. Schneier, J. Kelsey, D. Whiting, D. Wagner, C. Hall, and N. Ferguson, "On the Twofish Key Schedule," Proceedings of the 1998 SAC Conference, Springer-Verlag, 1998, to appear. [Page v]
[SKW+99a] | B. Schneier, J. Kelsey, D. Whiting, D. Wagner, C. Hall, and N. Ferguson, "Performance Comparison of the AES Submissions," submitted to the 2nd AES Candidate Conference, 1999, to appear. [Pages v, 121]
[SKW+99b] | B. Schneier, J. Kelsey, D. Whiting, D. Wagner, C. Hall, and N. Ferguson, "Twofish on Smart Cards," Proceedings of CARDIS 98, Springer-Verlag, to appear. [Pages v, 26]
[SM88] | A. Shimizu and S. Miyaguchi, "Fast Data Encipherment Algorithm FEAL," Advances in Cryptology — EUROCRYPT '87 Proceedings, Springer-Verlag, 1988, pp. 267-278, [Pages 5, 33, 36]
[SMK98] | T. Shimoyama, S. Moriai, and T. Kaneko, "Improving the Higher Order Differential Attack and Cryptanalysis of the KN Cipher," Information Security. First International Workshop ISW '97 Proceedings, Springer-Verlag, 1998, pp. 32-42. [Pages 35, 39]
[SV98] | J. Stern and S. Vaudenay, "CS-Cipher," Fast Software Encryption, 5th International Workshop Proceedings, Springer-Verlag, 1998, pp. 189-205. [Pages 39, 41]
[SW97] | B. Schneier and D. Whiting, "Fast Software Encryption: Designing Encryption Algorithms for Optimal Speed on the Intel Pentium Processor," Fast Software Encryption, 4th International Workshop Proceedings, Springer-Verlag, 1997, pp. 242-259. [Pages 33, 34, 121]
[UTK98] | M. Uemra, H. Tanaka, and T. Kanedo, "On the Weak Keys in SPEED Cipher by Higher Order Differential Attack," 1998 Symposium on Cryptography and Information Security, 1.2.D. (In Japanese.) [Page 33]
[Vau95] | S. Vaudenay, "On the Need for Multipermutations: Cryptanalysis of MD4 and SAFER," Fast Software Encryption, Second International Workshop Proceedings, Springer-Verlag, 1995, pp. 286-297. [Page 6]
[Vau96a] | S. Vaudenay, "On the Weak Keys in Blowfish," Fast Software Encryption, 3rd International Workshop Proceedings, Springer-Verlag, 1996, pp. 27-32. [Pages 39, 114]
[Vau96b] | S. Vaudenay, "An Experiment on DES Statistical Cryptanalysis," 3rd ACM Conference on Computer and Communications Security, ACM Press, 1996, pp. 139-147. [Page 102]
[Wag95a] | D. Wagner, "Cryptanalysis of S-1," sci.crypt Usenet posting, 27 Aug 1995. [Pages 41, 104]
[Wag95b] | D. Wagner, personal communication, 1995. [Page 40]
[WH87] | R. Winternitz and M. Hellman, "Chosen-key Attacks on a Block Cipher," Cryptologia, v. 11, n. 1, Jan 1987, pp. 16-20. [Page 35]
[Whe94] | D. Wheeler, "A Bulk Data Encryption Algorithm," Fast Software Encryption, Cambridge Security Workshop Proceedings, Springer-Verlag, 1994, pp. 127-134. [Page 38]
[Wie94] | M.J. Wiener, "Efficient DES Key Search," TR-244, School of Computer Science, Carleton University, May 1994. [Page 1]
[Win84] | R.S. Winternitz, "Producing One-Way Hash Functions from DES," Advances in Cryptology: Proceedings of Crypto 83, Plenum Press, 1984, pp. 203-207. [Page 119]
[WN95] | D. Wheeler and R. Needham, "TEA, a Tiny Encryption Algorithm," Fast Software Encryption, 2nd International Workshop Proceedings, Springer-Verlag, 1995, pp. 97-110. [Page 36]
[WS98] | D. Whiting and B. Schneier "Improved Twofish Implementations," Twofish Technical Report #3, Counterpane Systems, to appear. [Page v]
[WSK97] | D. Wagner, B. Schneier, and J. Kelsey, "Cryptanalysis of the Cellular Message Encryption Algorithm," Advances in Cryptology — CRYPTO '97 Proceedings, Springer-Verlag, 1997, pp. 526-537. [Page 38]
[WW98] | D. Whiting and D. Wagner, "Empirical Verification of Twofish Key Uniqueness Properties," Twofish Technical Report #2, Counterpane Systems, 22 Sep 1998. [Page v]
[YLCY98] | X. Yi, K.Y. Lam, S.X. Cheng, and X.H. You, "A New Byte-Oriented Block Cipher," Information Security. First International Workshop ISW '97 Proceedings, Springer-Verlag, 1998, 209-220. [Page 39]
[YMT97] | A.M. Youssef, S. Mister, and S.E. Tavares, "On the Design of Linear Transformations for Substitution Permutation Encryption Networks," Workshop on Selected Areas in Cryptography (SAC '97) Workshop Record, School of Computer Science, Carleton University, 1997, pp. 40-48. [Page 7]
[YTH96] | A.M. Youssef, S.E. Tavares, and H.M. Heys, "A New Class of Substitution-Permutation Networks," Workshop on Selected Areas in Cryptography (SAC '96) Workshop Record, Queens University, 1996, pp. 132-147. [Pages 5, 37]
[Yuv97] | G. Yuval, "Reinventing the Travois: Encryption/MAC in 30 ROM Bytes," Fast Software Encryption, 4th International Workshop Proceedings, Springer-Verlag, 1997, pp. 205-209. [Page 104]
[ZG97] | F. Zhu and B.-A. Guo, "A Block-Ciphering Algorithm Based on Addition-Multiplication Structure in GF(2n)," Workshop on Selected Areas in Cryptography (SAC '97) Workshop Record, School of Computer Science,Carleton University, 1997, pp. 145-159. [Page 33]
[Zhe97] | Y. Zheng, "The SPEED Cipher," Financial Cryptography '97 Proceedings, Springer-Verlag, 1997, pp. 71-89. [Pages 33, 121]
[ZMI90] | Y. Zheng, T. Matsumoto, and H. Imai, "On the Construction of Block Ciphers Provably Secure and Not Relying on Any Unproved Hypotheses," Advances in Cryptology — CRYPTO '89 Proceedings, Springer-Verlag, 1990, pp. 461-480. [Pages 5, 36]
[ZPS93] | Y. Zheng, J. Pieprzyk, and J. Seberry, "HAVAL — A One-Way Hashing Algorithm with Variable Length of Output," Advances in Cryptology — AUSCRYPT '92 Proceedings, Springer-Verlag, 1993, pp. 83-104. [Page 58]


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Index

F-function 2, 9-11, 17, 18, 37, 47, 58-60, 63, 64, 72,73, 81-83, 85, 87, 91, 92, 95, 103, 143
F´-function 11, 64, 65, 143
g-function 7, 9-12, 15, 16, 18, 21, 37, 41, 48, 59-61, 65, 71, 72, 76, 83, 85, 88, 104, 125, 143
h-function 14-16, 18, 30-32, 37, 41, 48, 66, 70, 114, 144
q-box 14-16, 21, 22, 29, 31, 48-51, 69, 72, 73, 91, 114, 115, 122, 126, 130, 144, 145
16-bit CPU 3, 33, 34
3-Way 104
32-bit CPU 3, 19, 33, 34, 125
64-bit CPU 3
8-bit CPU 2-4, 19, 26, 33-35, 76, 125
6502 26
6805 2, 26, 28, 157
68040 20
Advanced Encryption Standard see AES
AES 1, 24, 34, 42, 43, 116, 129
candidate 74, 80, 121
design criteria 3, 42
process 1, 117, 130
standard 30
Akelarre 36
algebraic degree 103
Alpha CPU 29
assembly 19, 20, 22-25, 121, 157
attack
brute-force 42, 98
chosen ciphertext 38
chosen key 79
partial 109
chosen plaintext 4, 38, 104
adaptive 104
chosen-key 108, 119
collision-finding 119
differential 40, 52, 60, 75, 79, 90, 97, 99, 100, 104, 105, 112, 114, 115, 121, 130
differential-linear 103
higher-order differential 35, 40
interpolation 35, 103
keysearch 6
known plaintext 104
linear 40, 60, 76, 101, 102, 121, 130
meet-in-the-middle 79-82
non-related-key 61
non-surjective round function 40
partial key guessing 104, 126
partitioning 102
related-key 4, 36, 37, 40, 41, 48, 51, 52, 67, 76, 79, 104-108, 112, 119, 122-124, 126
differential 41, 56, 105-108, 110
slide 105
slide 40, 63, 104, 105, 112, 113, 125
statistical 36, 39, 60, 129
truncated differential 90
unknown 33, 35, 36, 100, 115
weak-key 36, 126
Bear/Lion 5
Biham, Eli 88, 105, 113
Biham-DES 39
birthday paradox 73, 113
block size 1, 7, 44, 46, 119, 120
Blowfish 5, 22, 34, 36-42, 47, 114, 121, 125, 126, 129
Borland C 19, 20, 22, 24
cache 22, 23, 25
CAST 35, 36, 38, 41
CAST-128 5, 35, 121
CBC mode 119
CBC-MAC 120
CFB mode 119, 129
characteristic
differential 48, 49, 52, 75, 76, 83, 90-93, 95-100, 103, 114, 143
2-round 115
2-round iterative 114, 115
iterative 115
trivial 115
truncated 90
iterative 60
key-dependent 75, 76
linear 48, 76, 101-103, 144
cipher
iterated 43, 104
product 5, 43
stream 1, 3, 4, 46, 121, 129
clock cycle 19, 20, 22, 24, 25, 30-32,34, 35, 57, 60
key setup 2, 21-23, 26, 28, 130
per block 2, 19, 23, 24, 26, 28, 34,
130
per byte 2, 21, 24, 34, 121
CMEA 38
code size 2, 19, 22-23, 25, 26, 28-29
complementation property 42, 75
confusion 43, 44, 103
counter mode 31, 119
CRISP 37
cryptanalysis
differential 35, 38, 39, 52, 76, 82-90, 100, 103, 115
differential-linear 103
higher-order differential 39, 90
linear 35, 52, 101-103, 116
generalised 102
non-linear 102
partitioning 102
related-key 35, 79, 104-108
differential 104
side-channel 111
power 111
timing 111
CS-Cipher 39, 41
cycle 5, 61
data-dependent rotation 34, 36
Davies-Meyer 119
Declaration of Independence 38
DES 1, 5-7, 22, 24, 26, 30, 33, 34, 37-43, 47, 61, 102, 103, 105, 116, 121
DES-X 5
DESV 58
difference sequence 52, 71
differential power analysis 111
diffusion 7, 35, 36, 43, 44, 54, 59, 125
global 129
discrete logarithm problem 116
dynamic swapping 58
ECB mode 31, 34
EEPROM 26-28
electronic emanation 111
EPIC 29
equivalent keys 40, 42, 74
factoring 116
family key 4, 38, 122-124
fault analysis Ill
FEAL 5, 33, 36, 39
FEAL-32 121
Feistel
cipher 5, 64, 113
function 113, 114, 143
half 126
network 2, 5, 34, 36, 44-48, 125
balanced 5
generalized 36
incomplete 46-47
source-heavy 46-47
target-heavy 46-47
unbalanced 5, 36
round 6, 34
structure 9, 40, 47, 54, 60, 125
two-dimensional 125
xor 31, 59, 61
Feistel, Horst 5, 44
Fields Medal 116
fixed point 49, 51
gate count 2, 4, 19, 30-32, 58, 76
GF(28) 7, 12, 13, 35, 48, 54-57, 67, 72, 145
GOST 5, 38, 104
Hamming weight 56, 71, 83, 85, 107
hardware 2-4, 6, 19, 26, 30, 35, 125,
129
hash function 1, 3, 4, 23, 41, 42, 79, 119, 120, 130
HAVAL 58
ICE 41
IDEA 33, 36, 37, 39, 41, 42, 46, 61, 75,121
interleave 31, 32, 34
Java 25
Jeroboam 33
key agility 4, 23, 28-29, 42
key schedule 2, 4, 7, 12, 21, 33, 36, 37, 40-42, 61, 63-77, 104, 125, 129
keysetup 19, 20, 22, 23, 28, 30, 40, 42
time 2, 3, 21, 22, 25, 27, 28, 35, 126
key stretching 42
keying
compiled 22-24, 57
full 21-24
minimal 21, 24
option 19, 21, 22
partial 21-24
zero 21, 22, 24
Khafre 5, 33, 38, 39, 46
Khufu 5, 33, 38, 39, 42, 46, 126
Kiefer 35
KN-cipher 35
Latin square 58
LEA opcode 22, 57, 125
little-endian convention 9, 11, 122
LOKI 5, 39, 41
LOK189 33, 39
LOK191 33, 39
LOK197 39
Lucifer 5, 6, 44, 75
MAC see message authentication code
MacGuffin 5, 46
Madryga 36
magic constant 115, 116
Manta 7
master-key cryptosystem 116
Matsui, Mitsuru 48
maximum distance separable see MDS
MD4 46, 58
MD5 46, 58
MDS code 6
MDS matrix 2, 6, 7, 9, 10, 12, 13, 15, 18, 21-23, 27, 28, 31, 32, 34-37, 41, 54-61, 66-68, 70, 72, 83, 90, 91, 93-95, 103, 115, 125, 126, 130, 143, 145
Merced 3, 29
Merkle, Ralph 5
message authentication code 1, 3, 4, 120
Microsoft Visual C 19, 20, 23, 24
MISTY 35
MMB 36
multiple linear approximations 102
NewDES 38
NIST 1, 3, 34, 74 NMR scanning 111
non-bijective S-box 114
nonce 120
NSA 32, 42
OFB mode 119, 129
Panama 37
Pentium 3, 20, 22-25, 30, 34, 35, 49, 60, 67, 73, 74, 121, 125, 130, 157
Pentium family 7, 29, 35, 57, 58, 61
Pentium II 3, 19, 20, 23-25, 29, 30, 130, 157
Pentium MMX 22, 23, 25
Pentium Pro 2, 3, 19, 20, 22-25, 29, 30, 34, 60, 130, 157
Philips 30
PHT 7, 9-11, 16, 18, 22, 31, 32, 36, 37, 54, 55, 57-59, 61, 65, 71, 72, 75, 83, 85, 88, 91, 93, 95, 101, 103, 113, 125, 126
pipeline 31, 32, 59, 61
Power PC 19, 20
primitive polynomial 12, 13, 55, 76
PRNG 1, 4, 41, 120, 130
pseudo-Hadamaxd transform see PHT
public-key
algorithm 116
cryptosystem 116
quasi-weak keys 42
RAM 4, 21, 22, 26-31, 39, 73, 74, 77
RAND tables 38
RC2 33, 46
RC4 34, 37
RC5 5, 34, 36, 41, 121
REDOC-II 33, 36, 39
Reed-Solomon see RS
RIPE-MD 58
RIPEMD-160 58
Rivest, Ron 5
ROM 26-29, 31, 39, 76
rotor machine 44, 49
round function 2, 7, 10, 17, 18, 30, 31, 33-37, 41, 43, 46, 54, 55, 59, 60, 65, 72, 74, 76, 92, 96, 101-104, 114, 125, 126,129, 144
non-surjective 114
RS code 6, 13, 37, 41, 56, 106, 115
RS matrix 13, 26, 49, 76-77, 105-107, 111, 122, 145
S-1 40, 104
SAFER 5, 7, 36, 37, 39, 41, 44, 60, 121, 125
SAFER-K64 121
SEAL 33, 34, 39, 42, 126
self-inverse keys 40, 74
semi-weak keys 42, 75
Serpent 30, 34, 37, 38, 41, 121
session key 120
Seuss, Dr. 127
SHA 39, 58
SHA-1 46
Shannon, C. 44
Shark 5, 6, 41, 44
Skipjack 5, 40, 42, 46, 61
smart card 2, 19, 26-29, 34, 35, 58, 60, 125, 129
SNAKE 35
sn DES 38
SOBER 33
SP network 5, 7, 44, 46-48
SPEED 33, 121
Square 6, 36, 37, 39, 44, 54, 60, 121,125
subkey
permuting the 105
subkey difference
chosen 106
swapping key halves 105
symbols, overview 143
symmetric algorithm 1, 116
TEA 36
TEMPEST 111
test vectors 147
throughput 21, 28, 29, 31, 32, 43
Tiger 38, 46
trap door 115
Travois 104
Triple-DES 1, 121
twist 126
Twofish
definition 9
history 125
name 126
overview 10
performance 19
simplified 112
with non-bijective S-boxes 114
without round keys 112
Twofish-FK 122
Ultra-SPARC 19, 20
uniform transformation structure 5
Vaudenay, Serge 6, 114
VLIW 30
WAKE 38, 39
weak keys 3, 40, 42, 48, 74, 75
web site 157
whitening 4-6, 9-11, 60, 63, 66, 79-82, 87-89, 109, 116, 119, 123, 144
post- 81, 122, 144
pre- 81, 82, 122, 144
YLCY 39
YTH 37
Zhu-Guo 33


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 [Spisak knjiga]
The Twofish Encryption Algorithm-A 128-Bit Block Cipher
Bruce Schneier et al
Preface

Chapter 1—Introduction
Chapter 2—Twofish Design Goals
Chapter 3—Twofish Building Blocks
3.1 Feistel Networks
3.2 Whitening
3.3 S-boxes
3.4 MDS Matrices
3.5 Pseudo-Hadamard Transforms
3.6 Key Schedule
Chapter 4—Twofish
4.1 The Function F
4.2 The Function g
4.3 The Key Schedule
4.3.1 Additional Key Lengths
4.3.2 The Function h
4.3.3 The Key-dependent S-boxes
4.3.4 The Expanded Key Words Kj
4.3.5 The Permutations q0 and q1
4.4 Round Function Overview
Chapter 5—Performance of Twofish
5.1 Performance on Large Microprocessors
5.1.1 Keying Options
5.1.2 Code and Data Size
5.1.3 Large Memory Implementations
5.1.4 Total Encryption Times
5.1.5 Hash Function Performance
5.1.6 Language, Compiler, and Processor Choice
5.2 Performance on Smart Cards
5.2.1 RAM Usage
5.2.2 Encryption Speed and Key Agility
5.2.3 Code Size
5.3 Performance on the Alpha
5.4 Performance on Future Microprocessors
5.5 Hardware Performance
Chapter 6—Twofish Design Philosophy
6.1 Performance-Driven Design
6.1.1 Performance-driven Tradeoffs
6.2 Conservative Design
6.3 Simple Design
6.3.1 Reusing Primitives
6.3.2 Reversibility
6.4 S-boxes
6.4.1 Large S-boxes
6.4.2 Algorithmic S-boxes
6.4.3 Key-dependent S-boxes
6.5 The Key Schedule
6.5.1 Performance Issues
Chapter 7—The Design of Twofish
7.1 The Round Structure
7.1.1 Common Block Cipher Structures
7.2 The Key-dependent S-boxes
7.2.1 The Fixed Permutations q0 and q1
7.2.2 The S-boxes.
7.2.3 Exhaustive and Statistical Analysis
7.3 MDS Matrix
7.3.1 Non-key-dependent Coefficients
7.3.2 Implementation Issues
7.3.3 Preserving Diffusion Properties after Rotation
7.3.4 Rotational Uniqueness of Output Vectors
7.3.5 Maximizing the Minimal Hamming Distance
7.4 PHT
7.4.1 Eliminating the PHT
7.4.2 Diffusion and the Least Significant Bit
7.5 Key Addition
7.6 Feistel Combining Operation
7.7 Use of Different Groups
7.8 Diffusion in the Round Function
7.8.1 Changes Induced by F
7.9 One-bit Rotation
7.9.1 Reason for Rotations
7.9.2 Downsides to Rotations
7.9.3 Converting to a Pure Feistel Structure
7.10 The Number of Rounds
Chapter 8—Design of the Twofish Key Schedule
8.1 Round Subkeys
8.1.1 Equivalence of Round Subkeys
8.1.2 Equivalent keys
8.2 Controlling Changes in Round Subkeys
8.2.1 XOR Difference Sequences in A and B
8.2.2 Byte Sequences with Given Difference
8.2.3 Identical Byte Sequences
8.2.4 The A and B Sequences
8.2.5 The Sequence (K2i, K2i+1)
8.2.6 Difference Sequences in the Subkeys
8.3 The Round Function
8.4 Properties of the Key Schedule and Cipher
8.4.1 Equivalent Keys
8.4.2 Self-Inverse Keys
8.4.3 Pairs of Inverse Keys
8.4.4 Simple Relations
8.5 Key-dependent Characteristics and Weak Keys
8.6 Reed-Solomon Code
Chapter 9—Cryptanalysis of Twofish
9.1 A Meet-in-the-Middle Attack on Twofish
9.1.1 Results of the Attack
9.1.2 Overview of the Attack
9.1.3 Attacking Twofish with Fixed S and no Whitening
9.1.4 Attacking Twofish with Fixed S
9.1.5 Attacking Normal Twofish
9.2 Differential Cryptanalysis
9.2.1 Results of the Attack
9.2.2 Overview of the Attack
9.2.3 Building the Batches
9.2.4 Mounting the Attack
9.2.5 Lessons from the Analysis
9.3 Extensions to Differential Cryptanalysis
9.3.1 Higher-Order Differential Cryptanalysis
9.3.2 Truncated Differentials
9.4 Search for the Best Differential Characteristic
9.4.1 Differentials of the S-boxes
9.4.2 Differentials of F
9.4.3 Differentials of the Round Function
9.4.4 Multi-round Patterns
9.4.5 Results
9.4.6 Other Problems for the Attacker
9.4.7 Best S-box Differential
9.4.8 Other Variants
9.4.9 Further Work
9.4.10 Conclusion
9.5 Linear Cryptanalysis
9.5.1 Multiple Linear Approximations
9.5.2 Non-linear Cryptanalysis
9.5.3 Generalized Linear Cryptanalysis
9.5.4 Partitioning Cryptanalysis
9.5.5 Differential-linear Cryptanalysis
9.6 Interpolation Attack
9.7 Partial Key Guessing Attacks
9.8 Related-key Cryptanalysis
9.8.1 Resistance to Related-key Slide Attacks
9.8.2 Resistance to Related-key Differential Attacks
9.8.3 The Zero Difference Case
9.8.4 Other Difference Sequences
9.8.5 Probability of a Successful Attack with One Related-key Query
9.8.6 Conclusions
9.9 A Chosen-key Attack
9.9.1 Overview of the Attack
9.9.2 Finding a Key Pair
9.9.3 Choosing the Plaintexts to Request
9.9.4 Extracting the Key Material
9.10 Side-Channel Cryptanalysis and Fault Analysis
9.11 Attacking Simplified Twofish
9.11.1 Twofish with Known S-boxes
9.11.2 Twofish without Round Subkeys
9.11.3 Twofish with Non-bijective S-boxes
9.12 Trap Doors in Twofish
Chapter 10—Using Twofish
10.1 Chaining Modes
10.2 One-Way Hash Rinctions
10.3 Message Authentication Codes
10.4 Pseudorandom Number Generators
10.5 Larger Keys
10.6 Additional Block Sizes
10.7 More or Fewer Rounds
10.8 Family Key Variant: Twofish-FK
10.8.1 Analysis
Chapter 11—Historical Remarks
Chapter 12—Conclusions and Further Work
References
Appendix A
Appendix B
Appendix C
Index


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 1
Introduction

In 1972 and 1974, the National Bureau of Standards (now the National Institute of Standards and Technology, or NIST) issued the first public request for an encryption standard. The result was DES [NBS77], arguably the most widely used and successful encryption algorithm in the world.

Despite its popularity, DES has been plagued with controversy. Some cryptographers objected to the "closed-door" design process of the algorithm. The debate about whether DES' key is too short for acceptable commercial security has raged for many years [DH79], but recent advances in distributed key search techniques have left no doubt in anyone's mind that its key is simply too short for today's security applications [Wie94, BDR+96, Koc98b]. Triple-DES has emerged as an interim solution in many high-security applications, such as banking, but it is too slow for some uses. More fundamentally, the 64-bit block length shared by DES and most other well-known ciphers opens it up to attacks when large amounts of data are encrypted under the same key.

In response to a growing desire to replace DES, NIST announced the Advanced Encryption Standard (AES) program in 1997 [NIST97a].

NIST solicited comments from the public on the proposed standard, and eventually issued a call for algorithms to satisfy the standard [NIST97b]. The intention is for NIST to make all submissions public and eventually, through a process of public review and comment, choose a new encryption standard to replace DES.

NIST's call requested a block cipher. Block ciphers can be used to design stream ciphers with a variety of synchronization and error extension properties, one-way hash functions, message authentication codes, and pseudo-random number generators. Because of this flexibility, they are the workhorse of modern cryptography.

NIST specified several other design criteria: a longer key length, larger block size, faster speed, and greater flexibility. While no single algorithm can be optimized for all needs, NIST intends AES to become the standard symmetric algorithm of the next decade.

Twofish is our submission to the AES selection process. It meets all the required NIST criteria—128-bit block; 128-, 192-, and 256-bit keys; efficient on various platforms; etc.—and some strenuous design requirements, performance as well as cryptographic, of our own.

Specifically, Twofish is a 128-bit block cipher that accepts a variable-length key up to 256 bits. The cipher is a 16-round Feistel network with a bijective F function made up of four key-dependent 8-by-8-bit S-boxes, a fixed 4-by-4 maximum distance separable matrix over GF(28), a pseudo-Hadamard transform, bitwise rotations, and a carefully designed key schedule. A fully optimized implementation of Twofish encrypts on a Pentium Pro at 16.1 clock cycles per byte, and an 8-bit smart card implementation encrypts at 1660 clock cycles per byte. Twofish can be implemented in hardware in 8000 gates. The design of both the round function and the key schedule permits a wide variety of tradeoffs between speed, software size, key setup time, gate count, and memory. We have extensively cryptanalyzed Twofish; our best attack breaks six rounds (without the whitening) with 241 chosen plaintexts and 2232 effort.

Twofish can:

• Encrypt data at 258 clock cycles per block on a Pentium Pro, after an 8600 clock-cycle key setup, or after a 6500 clock-cycle key setup if 256 Kbytes of tables are available.
• Encrypt data at 860 clock cycles per block on a Pentium Pro, after a 1250 clock-cycle key setup.
• Encrypt data at 26500 clock cycles per block on a 6805 smart card, after a 1750 clock-cycle key setup, using only 60 bytes of RAM.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 2
Twofish Design Goals

Twofish was designed to meet NIST's design criteria for AES [NIST97b]. Specifically, they are:

• A 128-bit symmetric block cipher.
• Key lengths of 128 bits, 192 bits, and 256 bits.
• No weak keys.
• Efficiency, both on the Intel Pentium Pro and other software and hardware platforms.
• Flexible design: e.g., accept additional key lengths; be implementable on a wide variety of platforms and applications; and be suitable for a stream cipher, hash function, and MAC.
• Simple design, both to facilitate ease of analysis and ease of implementation.

Additionally, we imposed the following performance criteria on our design:

• Encrypt data in less than 500 clock cycles per block on an Intel Pentium, Pentium Pro, and Pentium II for a fully optimized version of the algorithm.
• Be capable of setting up a 128-bit key (for optimal encryption speed) in less than the time required to encrypt 32 blocks on a Pentium, Pentium and Pentium II.
• Encrypt data in less than 5000 clock cycles per block on a Pentium, Pentium Pro, and Pentium II, with no key setup time.
• Not contain any operations that make it inefficient on other 32-bit microprocessors.
• Not contain any operations that make it inefficient on 8-bit and 16-bit microprocessors.
• Not contain any operations that reduce its efficiency on existing and proposed 64-bit microprocessors; e.g., Merced.
• Not include any elements that make it inefficient in hardware.
• Have a variety of performance tradeoffs with respect to the key schedule.
• Encrypt data in less than 10 milliseconds on a commodity 8-bit microprocessor.
• Be implementable on an 8-bit microprocessor with only 64 bytes of RAM.
• Be implementable in hardware using less than 20000 gates.

Our cryptographic goals were as follows:

• 12-round Twofish (without whitening) should have no chosen-plaintext attack requiring fewer than 280 chosen plaintexts and less than 2N time, where N is the key length.
• 12-round Twofish (without whitening) should have no related-key attack requiring fewer than 264 chosen plaintexts, and less than 2N/2 time, where N is the key length.

Finally, we imposed the following flexibility goals:

• Accept any key length up to 256 bits.
• Have variants with a variable number of rounds.
• Have a key schedule that can be precomputed for maximum speed, or computed on the fly for maximum agility and minimum memory requirements. Additionally, it should be suitable for dedicated hardware applications: e.g., no large tables.
• Be suitable as a stream cipher, one-way hash function, MAC, and pseudo-random number generator, using well-understood construction methods.
• Have a family-key variant to allow for different, non-interoperable versions of the cipher.

We feel we have met all of these goals in the design of Twofish.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 3
Twofish Building Blocks

3.1 Feistel Networks

A Feistel network is a general method of transforming any function (usually called the F function) into a permutation. It was invented by Horst Feistel [FNS75] in his design of Lucifer [Fei73], and popularized by DES [NBS77].

All commonly used symmetric block ciphers are product ciphers; they are strong ciphers constructed by many iterations of a round function, which is itself a kind of weak cipher. In a Feistel cipher, the round function consists of taking one part of the data being encrypted, feeding it into some key dependent function F, and then XORing (or otherwise combining) the result into another part of the block. If the part of the data used as input to the F function and the part of the data combined with output from the F function are each half of the block, then the cipher is a "balanced" Feistel network [SK96].

This is the basis of most block ciphers published since then, including FEAL [SM88], GOST [GOST89], Khufu and Khafre [Mer91], LOKI [BPS90, BKPS93], CAST-128 [Ada97a], Blowfish [Sch94], and RC5 [Riv95].

Two rounds of a Feistel network are collectively called a "cycle" [SK96]. In one cycle, every bit of the text block has been modified once.1 Twofish is a 16-round Feistel network with a bijective F function, which corresponds to eight cycles.

1 The notion of a cycle allows balanced Feistel networks to be compared with unbalanced Feistel networks [SK96, ZMI90] such as MacGuffin [BS95] (cryptanalyzed in [RP95a]) and Bear/Lion [AB96a], and with SP-networks (also called uniform transformation structures [Fei73]) such as SAFER and Shark [RDP+96] (see also [YTH96]). Thus, 8-cycle (8-round) SAFER is comparable to 8-cycle (16-round) DES and 8-cycle (32-round) Skipjack [NSA98].

3.2 Whitening

Whitening, the technique of XORing key material before the first round and after the last round, was used by Ralph Merkle in Khufu/Khafre, and independently invented by Ron Rivest for DES-X [KR96a].

In [KR96a], it was shown that whitening substantially increases the difficulty of keysearch attacks against the remainder of the cipher. In our attacks on reduced-round Twofish variants, we discovered that whitening substantially increased the difficulty of attacking the cipher by hiding from an attacker the specific inputs to the first and last rounds' F functions.

Twofish XORs 128 bits of subkey before the first Feistel round, and another 128 bits after the last Feistel round. These subkeys are calculated in the same manner as the round subkeys, but are not used anywhere else in the cipher.

3.3 S-boxes

An S-box is a table-driven non-linear substitution operation used in most block ciphers. S-boxes vary in both input size and output size, and can be created either randomly or algorithmically. S-boxes were first used in Lucifer, then DES, and afterwards in most encryption algorithms.

Twofish uses four different, bijective, key-dependent, 8-by-8-bit S-boxes. Each S-box is constructed using two fixed 8-by-8-bit permutations and several bytes of key material. These S-boxes can either be precomputed for a specific key, or computed on the fly for every required value. This provides a lot of flexibility and tradeoffs both in hardware and software.

3.4 MDS Matrices

A maximum distance separable (MDS) code over a field is a linear mapping from a field elements to b field elements, producing a composite vector of a + b elements, with the property that the minimum number of non-zero elements in any non-zero vector is at least b + 1 [MS77].2 Put another way, the "distance" (i.e., the number of elements that differ) between any two distinct vectors produced by the MDS mapping is at least b + 1. It can easily be shown that no mapping can have a larger minimum distance between two distinct vectors, hence the term maximum distance separable. MDS mappings can be represented by an MDS matrix consisting of a × b elements. Reed-Solomon (RS) error-correcting codes are known to be MDS. A necessary and sufficient condition for an a × b matrix to be MDS is that all possible square submatrices, obtained by discarding rows or columns, are non-singular.

2 We describe only the normal form of a linear MDS code. Any MDS code can be converted to an equivalent code in normal form.

Serge Vaudenay first proposed MDS matrices as a cipher design element [Vau95]. Shark [RDP+96] and Square [DKR97] use MDS matrices (see also [YMT97]), although we first saw the construction used in the unpublished cipher Manta [Fer96].3

3 Manta is a block cipher with a large block size and an emphasis on long-term security rather than speed. It uses an SP-like network with DES as the S-boxes and MDS matrices for the permutations.

MDS matrices are useful building blocks for ciphers because they guarantee a certain degree of diffusion. If one of the input elements is changed, all the output elements must change. If two input elements are changed, all but one of the output elements must change, etc. Twofish uses a single 4-by-4 MDS matrix over GF(28). This is one of the two main diffusion elements of Twofish. (There is also an RS-code with the MDS property used in the key schedule; this doesn't add diffusion to the cipher, but does add diffusion to the key schedule.)

3.5 Pseudo-Hadamard Transforms

A pseudo-Hadamard transform (PHT) is a simple mixing operation that is very efficient in software. Given two inputs, a and b, the 32-bit PHT is defined as:

a´ = a + b mod 232

b´ = a + 2b mod 232

SAFER [Mas94] uses 8-bit PHTs extensively for diffusion.

Twofish uses a 32-bit PHT to mix the outputs from its two parallel 32-bit g functions. This PHT can be executed in two opcodes on most modern microprocessors, including the Pentium family. This is the second main diffusion element in Twofish.

3.6 Key Schedule

The key schedule is the means by which the key bits are turned into round keys that the cipher can use. The requirements call for a variable-length key. The easiest way of using this is to have a key schedule that expands a variable-length key to a fixed set of expanded key values.

Twofish needs a lot of key material, and has a complicated key schedule. To facilitate analysis, the key schedule uses the same primitives as the round function. Except for two additional rotations, each pair of expanded key words is constructed by applying the Twofish round function (with key-dependent S-boxes) to a fixed input.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 4
Twofish

Figure 4.1 shows an overview of the Twofish block cipher. Twofish uses a 16-round Feistel-like structure with additional whitening of the input and output. The only non-Feistel elements are the 1-bit rotates. The rotations can be moved into the F function to create a pure Feistel structure, but this requires an additional rotation of the words just before the output whitening step.

The plaintext is split into four 32-bit words. In the input whitening step, these are XORed with four key words. This is followed by 16 rounds. In each round, the two words on the left are used as input to the g functions. (One of them is rotated by 8 bits first.) The g function consists of four byte-wide key-dependent S-boxes, followed by a linear mixing step based on an MDS matrix. The results of the two g functions are combined using a Pseudo-Hadamard Transform (PHT), and two round key words are added. These two results are then XORed into the words on the right (one of which is rotated left by one bit first, the other is rotated right by one bit afterwards). The left and right halves are then swapped for the next round. After all the rounds, the swap of the last round is reversed, and the four words are XORed with four more key words to produce the ciphertext.

More formally, the 16 bytes of plaintext p0, ... , p15 are first split into four words P0, ..., P3 of 32 bits each using the little-endian convention.

In the input whitening step, these words are XORed with four words of the expanded key.

R0,i = Pi ⊕ Ki i = 0,...,3

In each of the 16 rounds, the first two words are used as input to the function F, which also takes the round number as input. The third word is XORed with the first output of F and then rotated right by one bit. The fourth word is rotated left by one bit and then XORed with the second output word of F. Finally, the two halves are exchanged. Thus,

(Fr,0,Fr,1) = F(Rr,0,Rr,1,r)

*

* Fig. 4.1. Twofish
Rr+1,0 = ROR(Rr,2 ⊕ Fr,0,1)
Rr+1,1 = ROL(Rr,3,1) ⊕ Fr,1)
Rr+1,2 = Rr,0
Rr+1,3 = Rr,1

for r = 0, ..., 15 and where ROR and ROL are functions that rotate their first argument (a 32-bit word) left or right by the number of bits indicated by their second argument.

The output whitening step undoes the "swap" of the last round, and XORs the data words with four words of the expanded key.

Ci = R16,(i+2) mod 4⊕Ki+4 i = 0,...,3

The four words of ciphertext are then written as 16 bytes c0, ..., c15 using the same little-endian conversion used for the plaintext.

4.1 The Function F

The function F is a key-dependent permutation on 64-bit values. It takes three arguments: two input words R0 and R1, and the round number r used to select the appropriate subkeys. R0 is passed through the g function, which yields T0. R1 is rotated left by eight bits and then passed through the g function to yield T1. The results T0 and T1 are then combined using a PHT and two words of the expanded key are added.

T0 = g(R0)
T1 = g(ROL(R1,8))
F0 = (T0 + T1 + K2r+8) mod 232
F1 = (T0 + 2T1 + K2r+9) mod 232

where (F0, F1) is the result of F. We also define the function F´ for use in our analysis. F´ is identical to the F function, except that it does not add any key blocks to the output. (The PHT is still performed.)

4.2 The Function g

The function g forms the heart of Twofish. The input word X is split into four bytes. Each byte is run through its own key-dependent S-box. Each S-box is an 8-bit permutation: it takes eight bits of input and produces eight bits of output. The four results are interpreted as components of a vector of length 4 over GF(28), and multiplied by the 4 × 4 MDS matrix (using the field GF(28) for the computations).

The resulting vector is interpreted as a 32-bit word which is the result of g.

where si are the key-dependent S-boxes and Z is the result of g. For this to be well-defined, we need to specify the correspondence between byte values and the field elements of GF(28). We represent GF(28) as GF(2)[x]/v(x), where v(x) = x8 + x6 + x5 + x3 + 1 is a primitive polynomial of degree 8 over GF(2). The field element a = å7i=0 aixi with ai ∈ GF(2) is identified with the byte value å7i=0 ai2i. This is in some sense the "natural" mapping; addition in GF(28) corresponds to a XOR of the bytes.

The MDS matrix is given by:

where the elements have been written as hexadecimal byte values using the above-defined correspondence.

4.3 The Key Schedule

The key schedule has to provide 40 words of expanded key K0, ..., K39, and the 4 key-dependent S-boxes used in the g function. Twofish is defined for keys of length N = 128, N = 192, and N = 256.

We define k = N/64. The key M consists of 8k bytes m0, ..., m8k-1. The bytes are first converted into 2k words of 32 bits each

and then into two word vectors of length k.

Me = (Mo,M2,...,M2k-2)
M0 = (M1,M3,...,M2k-1)

A third word vector of length k is also derived from the key. This is done by taking the key bytes in groups of eight, interpreting them as a vector over GF(28), and multiplying them by a 4 × 8 matrix derived from an RS code. Each result of four bytes is then interpreted as a 32-bit word. These words make up the third vector.

for i = 0,....k – 1, and

S = (Sk-1, Sk-2,...,S0)

Note that S lists the words in "reverse" order. For the RS matrix multiply, GF(28) is represented by GF(2)[x]/w(x), where w(x) = x8 + x6 + x3 + x 2 + 1 is another primitive polynomial of degree 8 over GF(2). The mapping between byte values and elements of GF(28) uses the same definition as used for the MDS matrix multiply. Using this mapping, the RS matrix is given by:

The three vectors Me, Mo, and S from the basis of the key schedule.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

4.3.1 Additional Key Lengths

Twofish can accept keys of any byte length up to 256 bits. For key sizes that are not defined above, the key is padded at the end with zero bytes to the next larger length that is defined. For example, an 80-bit key m0, ... , m9 would be extended by setting mi = 0 for i = 10, ..., 15 and treating it as a 128-bit key. Every key of non-standard length is thus equivalent to exactly one key of a standard length.

*
* Fig. 4.2. The Function h

We have not defined Twofish for keys whose length is not an integral number of bytes. Although this can be done trivially, we see no reason to do so. Keys that span only part of a byte create a lot of extra software complexity that can easily lead to implementation bugs. There is very little to be gained from the use of such odd-sized keys.

4.3.2 The Function h

Figure 4.2 shows an overview of the function h. This is a function that takes two inputs—a 32-bit word X and a list L = (L0, ..., Lk-1) of 32-bit words of length k—and produces one word of output. This function works in k stages. In each stage, the four bytes are each passed through a fixed S-box and then XORed with a byte derived from the list. Finally, the bytes are once again passed through a fixed S-box, and the four bytes are multiplied by the MDS matrix just as in g. More formally: we split the words into bytes.

li,j = [Li/28j] mod 28
xj = [X/28j] mod 28

for i = 0,..., k – 1 and j = 0, ..., 3. Then the sequence of substitutions and XORs is applied.

yk,j = xj j = 0,...,3

If k = 4 we have

y3,0 = q1[y4,0] ⊕ l3,0
y3,1 = q0[y4,1] ⊕ l3,1
y3,2 = q0[y4,2] ⊕ l3,2
y3,3 = q1[y4,3] ⊕ l3,3

If k ≥ 3 we have

y2,0 = q1[y3,0] ⊕ l2,0
y2,1 = q1[y3,1] ⊕ l2,1
y2,2 = q0[y3,2] ⊕ l2,2
y2,3 = q0[y3,3] ⊕ l2,3

In all cases we have

y0 = q1[q0[q0[y2,0] ⊕ l1,0] ⊕ l0,0]
y1 = q0[q0[q1[y2,1] ⊕ l1,1] ⊕ l0,1]
y2 = q1[q1[q0[y2,2] ⊕ l1,2] ⊕ l0,2]
y3 = q0[q1[q1[y2,3] ⊕ l1,3] ⊕ l0,3]

Here, q0 and q1 are fixed permutations on 8-bit values that we will define shortly. The resulting vector of yi's is multiplied by the MDS matrix, just as in the g function.

where Z is the result of h.

4.3.3 The Key-dependent S-boxes

We can now define the S-boxes in the function g by

g(X) = h(X, S)

That is, for i = 0, ..., 3, the key-dependent S-box si is formed by the mapping from xi to yi in the h function where the list L is equal to the vector S derived from the key.

4.3.4 The Expanded Key Words Kj

The words of the expanded key are defined using the h function. We have

ρ = 224 + 216 + 28 + 20
Ai = h(2iρ, Me)
Bi = ROL(h(2i + 1)ρ, Mo),8)
K2i = (Ai + Bi) mod 232
K2i+1 = ROL((Ai + 2Bi) mod 232,9)

for i = 0, ..., 19. The constant ρ is used here to duplicate bytes; it has the property that for i = 0, ..., 255, the word iρ consists of four equal bytes, each with the value i. The function h is applied to words of this type. For Ai the byte values are 2i, and the second argument of h is Me. Bi is computed similarly using 2i + 1 as the byte value and Mo as the second argument, with an extra rotate over eight bits. The values Ai and Bi are combined in a PHT. One of the results is further rotated by nine bits. The two results form two words of the expanded key.

4.3.5 The Permutations q0 and q1

The permutations q0 and q1 are fixed permutations on 8-bit values. They are constructed from four different 4-bit permutations each. For the input value x, we define the corresponding output value y as follows:

a0, b0 = [x/16], x mod 16
a1 = a0 ⊕ b0
b1 = a0 ⊕ ROR4(b0,1) ⊕ 8a0 mod 16
a2, b2 = t0[a1], t1[b1]
a3 = a2 ⊕ b2
b3 = a2 ⊕ ROR4(b2,1) ⊕ 8a2 mod 16
a4, b4 = t2[a3], t3[b3]
y = 16b4 + a4

where ROR4 is a function similar to ROR that rotates 4-bit values. First, the byte is split into two nibbles (1 nibble = 4 bits). These are combined in a bijective mixing step. Each nibble is then passed through its own 4-bit fixed S-box. This is followed by another mixing step and S-box lookup. Finally, the two nibbles are recombined into a byte. For the permutation q0 the 4-bit S-boxes are given by

t0 = [8 1 7 D 6 F 3 2 0 B 5 9 E C A 4 ]
t1 = [E C B 8 1 2 3 5 F 4 A 6 7 0 9 D ]
t2 = [B A 5 E 6 D 9 0 C 8 F 3 2 4 7 1 ]
t3 = [D 7 F 4 1 2 6 E 9 B 3 0 8 5 C A ]

where each 4-bit S-box is represented by a list of the entries using hexadecimal notation. (The entries for the inputs 0, 1, ..., 15 are listed in order.) Similarly, for q1 the 4-bit S-boxes are given by

t0 = [2 8 B D F 7 6 E 3 1 9 4 0 A C 5 ]
t1 = [1 E 2 B 4 C 3 7 6 D A 5 F 9 0 8 ]
t2 = [4 C 7 5 1 6 9 A 0 E D 8 2 B 3 F ]
t3 = [B 9 5 1 C 3 D E 6 4 7 F 2 0 8 A ]

4.4 Round Function Overview

Figure 4.3 shows a more detailed view of how the function F is computed each round when the key length is 128 bits. Incorporating the S-box and round-subkey generation makes the Twofish round function look more complicated, but is useful for visualizing exactly how the algorithm works.

*
* Fig. 4.3. A View of a Single Round F Function (128-bit Key)


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 5
Performance of Twofish

Twofish has been designed from the start with performance in mind. It is efficient on a variety of platforms: 32-bit CPUs, 8-bit smart cards, and dedicated VLSI hardware. More importantly, though, Twofish has been designed to allow several layers of performance tradeoffs, depending on the relative importance of encryption speed, key setup, memory use, hardware gate count, and other implementation parameters. The result is a highly flexible algorithm that can be implemented efficiently in a variety of cryptographic applications.

All these options are interoperable; these are simply implementation tradeoffs and do not affect the mathematics of Twofish. One end of a communication could use the fastest Pentium II implementation, and the other the cheapest hardware implementation.

5.1 Performance on Large Microprocessors

Table 5.1 gives Twofish's performance, encryption or decryption, for different key scheduling options and on several modern microprocessors using different languages and compilers. This table shows our results for many different implementations. Each implementation is presented on a single line. The first column gives the CPU the implementation was run on (PPro/II = Pentium Pro/Pentium II, U-SPARC = Ultra-SPARC, PPC = Power PC). The second column is the programming language (ASM = assembly language, MS C = Microsoft Visual C++ 4.2, BC = Borland C 5.0, C = standard C compiler). The keying options are explained below. The code size column contains the approximate total code size (in bytes) of the routines for encryption, decryption, and key setup, where available. All remaining numbers in the row are in clock cycles. For each key size we show the number of clock cycles required for the key setup, and the number of clock cycles required to encrypt a single block. The times for encryption and decryption are identical in assembly, and encryption is slightly slower than decryption in C; only the encryption (i.e., the larger) number is given. There is no time required to set up the algorithm except for key setup. The time required to change a key is the same as the time required to set up a key.

For example, on a Pentium Pro a fully optimized assembly-language version of Twofish can encrypt or decrypt data in 258 clock cycles per block, or 16.1 clock cycles per byte, after a 12700-clock key setup (equivalent to encrypting 45 blocks). On a 200 MHz Pentium Pro microprocessor, this translates to a 90 Mbits/sec.

Processor | Lang | Keying Option | Code Size | Clocks to Key |  |  | Clocks to Encrypt
 |  |  |  | 128 | 192 | 256 | 128 | 192 | 256
PPro/II | ASM | Comp. | 9000 | 8600 | 11300 | 14100 | 258 | 258 | 258
PPro/II | ASM | Full | 8500 | 7600 | 10400 | 13200 | 315 | 315 | 315
PPro/II | ASM | Part. | 10700 | 4900 | 7600 | 10500 | 460 | 460 | 460
PPro/II | ASM | Min. | 13600 | 2400 | 5300 | 8200 | 720 | 720 | 720
PPro/II | ASM | Zero | 9100 | 1250 | 1600 | 2000 | 860 | 1130 | 1420
PPro/II | MS C | Full | 11200 | 8000 | 11200 | 15700 | 600 | 600 | 600
PPro/II | MS C | Part. | 13200 | 7100 | 9700 | 14100 | 800 | 800 | 800
PPro/II | MS C | Min. | 16600 | 3000 | 7800 | 12200 | 1130 | 1130 | 1130
PPro/II | MS C | Zero | 10500 | 2450 | 3200 | 4000 | 1310 | 1750 | 2200
PPro/II | BC | Full | 14100 | 10300 | 13600 | 18800 | 640 | 640 | 640
PPro/II | BC | Part. | 14300 | 9500 | 11200 | 16600 | 840 | 840 | 840
PPro/II | BC | Min. | 17300 | 4600 | 10300 | 15300 | 1160 | 1160 | 1160
PPro/II | BC | Zero | 10100 | 3200 | 4200 | 4800 | 1910 | 2670 | 3470
Pentium | ASM | Comp. | 9100 | 12300 | 14600 | 17100 | 290 | 290 | 290
Pentium | ASM | Full | 8200 | 11000 | 13500 | 16200 | 315 | 315 | 315
Pentium | ASM | Part. | 10300 | 5500 | 7800 | 9800 | 430 | 430 | 430
Pentium | ASM | Min. | 12600 | 3700 | 5900 | 7900 | 740 | 740 | 740
Pentium | ASM | Zero | 8700 | 1800 | 2100 | 2600 | 1000 | 1300 | 1600
Pentium | MS C | Full | 11800 | 11900 | 15100 | 21500 | 630 | 630 | 630
Pentium | MS C | Part. | 14100 | 9200 | 13400 | 19800 | 900 | 900 | 900
Pentium | MS C | Min. | 17800 | 3800 | 11100 | 16900 | 1460 | 1460 | 1460
Pentium | MS C | Zero | 11300 | 2800 | 3900 | 4900 | 1740 | 2260 | 2760
Pentium | BC | Full | 12700 | 14200 | 18100 | 26100 | 870 | 870 | 870
Pentium | BC | Part. | 14200 | 11200 | 16500 | 24100 | 1100 | 1100 | 1100
Pentium | BC | Min. | 17500 | 4700 | 12100 | 19200 | 1860 | 1860 | 1860
Pentium | BC | Zero | 11800 | 3700 | 4900 | 6100 | 2150 | 2730 | 3270
U-SPARC | C | Full |  | 16600 | 21600 | 24900 | 750 | 750 | 750
U-SPARC | C | Part. |  | 8300 | 13300 | 19900 | 930 | 930 | 930
U-SPARC | C | Min. |  | 3300 | 11600 | 16600 | 1200 | 1200 | 1200
U-SPARC | C | Zero |  | 1700 | 3300 | 5000 | 1450 | 1680 | 1870
PPC 750 | C | Full |  | 12200 | 17100 | 22200 | 590 | 590 | 590
PPC 750 | C | Part. |  | 7800 | 12200 | 17300 | 780 | 780 | 780
PPC 750 | C | Min. |  | 2900 | 9100 | 14200 | 1280 | 1280 | 1280
PPC 750 | C | Zero |  | 2500 | 3600 | 4900 | 1030 | 1580 | 2040
68040 | C | Full | 16700 | 53000 | 63500 | 96700 | 3500 | 3500 | 3500
68040 | C | Part. | 18100 | 36700 | 47500 | 78500 | 4900 | 4900 | 4900
68040 | C | Min. | 23300 | 11000 | 40000 | 71800 | 8150 | 8150 | 8150
68040 | C | Zero | 16200 | 9800 | 13300 | 17000 | 6800 | 8600 | 10400
Table 5.1. Twofish Performance with Different Key Lengths and Options


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

5.1.1 Keying Options

We implemented four different keying options. All of our keying options pre-compute Ki for i = 0,...,39 and use 160 bytes of RAM to store these constants. The differences occur in the way the function g is implemented. There are several other possible keying options, each with slightly different setup/throughput tradeoffs, but the examples listed below are representative of the range of possibilities.

Full Keying.
This option performs the full key precomputations. Using 4 Kbytes of table space, each S-box is expanded to a 8-by-32-bit table that combines both the S-box lookup and the multiply by the column of the MDS matrix. Using this option, a computation of g consists of four table lookups, and three XORs. Encryption and decryption speeds are constant regardless of key size.
Partial Keying.
For applications where few blocks are encrypted with a single key, it may not make sense to build the complete key schedule. The partial keying option precomputes the four S-boxes in 8-by-8-bit tables, and uses four fixed 8-by-32-bit MDS tables to perform the MDS multiply. This reduces the key-schedule table space to 1 Kbyte. For each byte, the last of the q-box lookups is in fact incorporated into the MDS table, so only k of the q-boxes are incorporated into the 8-by-8-bit S-box table that is built by the key schedule. Encryption and decryption speeds are again constant regardless of key size.
Minimal Keying.
For applications where very few blocks are encrypted with a single key, there is a further possible optimization. Compared to partial keying, one less layer of q-boxes is precomputed into the S-box table, and the remaining q-box is done during the encryption. For the 128-bit key this is particularly efficient, as precomputing the S-boxes now consists of copying the table of the appropriate q-box and XORing it with a constant (which can be done word-by-word instead of byte-by-byte). This option uses a 1 Kbyte table to store the partially precomputed S-boxes. The necessary key bytes from S are of course precomputed, as they are needed in every round.
Zero Keying.
The zero keying option does not precompute any of the S-boxes, and thus needs no extra tables. Instead, every entry is computed on the fly. The key setup time consists purely of computing the Ki values and S. For an application that cannot have any key setup time, the time it takes to encrypt one block is the sum of the key setup time and encryption time for the zero keying option.
Compiled.
In this option, available only in assembly language, the subkey constants are directly embedded into a key-specific copy of the code, saving memory fetches and allowing the use of the Pentium LEA opcode to perform both the PHT and the subkey addition all in a single clock cycle. Some additional setup time is required, as well as about an extra 5000 bytes of memory to hold the "compiled" code, but this option allows the fastest execution time of 258 clocks per block on the Pentium Pro. The setup time for a Pentium more than doubles over the full keying case because of its smaller cache size, but the Pentium MMX setup time is still comparable to the Pentium Pro setup time. However, almost all the extra time required is consumed merely in copying the code; the table does not reflect the fact that, once a single key has been initialized, future keys can be compiled at a cost of only a few hundred more clocks than full key schedule time.

5.1.2 Code and Data Size

As shown in Table 5.1, the code size for a fully optimized Twofish implementation on a Pentium Pro ranges from about 8450 bytes in assembler to 14100 bytes in Borland C. In assembler, the encryption and decryption routines are each about 2250 bytes in size; the remaining 4000 bytes of code are in the key scheduling routines, but about 2200 bytes of that total can be discarded if only 128-bit keys are needed. In Borland C, the encryption and decryption routines are each about 4500 bytes in length, and the key schedule routine is slightly less than 5000 bytes. Note that each routine fits easily within the code cache of a Pentium or a Pentium Pro. These sizes are larger than Blowfish but very similar to a fully optimized assembly language version of DES. Note that, with the exception of the zero keying option, the code sizes in the table are for fully unrolled implementations; in either C or assembler, it is possible to achieve significantly smaller code sizes using loops with round counters, at a cost in performance.

In addition to the code, there are about 4600 bytes of fixed tables for the MDS matrix, q0 and q1 required for key setup, and each key requires about 4300 bytes of key-dependent data tables for the full keying option. During encryption or decryption, these tables fit easily in the Pentium data cache. The other keying options use less data and table space, as discussed above.

5.1.3 Large Memory Implementations

For machines with sufficient RAM and a good memory cache subsystem, large precomputed tables can be used to reduce the key setup time for Twofish even further. For example, in compiled, full, or partial keying modes, the first two levels of q0 and q1 lookups with one key byte can be precomputed for all four S-boxes, requiring 256 Kbytes of table (four tables of 64 Kbytes each). This approach saves roughly 2000 clocks per key setup on the Pentium Pro in assembly language; details are shown in Table 5.2. For instance, the compiled mode key setup for 128-bit keys on a Pentium Pro can be reduced from 8700 clocks to 6500 clocks. Unfortunately, the savings on Pentium and Pentium MMX CPUs seems to depend on the performance of the L2 cache subsystem (which is included in the Pentium Pro and thus is more predictable); the gain seems to range from 500 clocks down to nothing. Implementing this "big table" version in C also leads to savings of about 1000 clocks per key setup on the Pentium Pro, depending on the quality of the compiler; again, Pentium performance gains are minimal.

Processor | Lang | Keying Option | Code Size | Clocks to Key |  |  | Clocks to Encrypt
 |  |  |  | 128 | 192 | 256 | 128 | 192 | 256
PPro/II | ASM | Comp. | 271,200 | 6500 | 9200 | 11900 | 258 | 258 | 258
PPro/II | ASM | Full | 270,600 | 5300 | 8000 | 11000 | 315 | 315 | 315
PPro/II | ASM | Part. | 272,900 | 2600 | 5300 | 8200 | 460 | 460 | 460
PPro/II | MS C | Full | 273,300 | 7300 | 11200 | 15700 | 600 | 600 | 600
Table 5.2. Twofish Performance with Large Fixed Tables


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

For the ultimate in key agility, a full 256 Mbytes of precomputed tables could comprise all four S-boxes for the final two stages of q0, q1, covering all 216 key byte possibilities for the 128-bit key case, and including the MDS matrix multiply. With a good memory subsystem, such a version should cut another 1000 clocks or so out of the above key setup times. Clearly, this is a fairly expensive solution (at least with today's technology), but it illustrates the flexibility of Twofish very nicely.

5.1.4 Total Encryption Times

Any performance measures that do not take key setup into account are only valid for asymptotically large amounts of text. For shorter messages, performance is the sum of key setup and encryption. For very short messages, the key setup time can overwhelm the encryption speed.

Table 5.3 gives Twofish's performance on the Pentium Pro (assembly-language version), both 128-bit key setup and encryption, for a variety of message lengths. This table assumes the best of our implementations (not including the large-memory implementations) for the particular length of text.

5.1.5 Hash Function Performance

Hash functions built from block ciphers generally require the algorithm to be uniquely keyed for each text block [Pre93, Sch96]. Assuming the chaining variables are the plaintext and ciphertext, and the block to be hashed is the key, Twofish can hash text at a rate of 175 clock cycles per byte on a Pentium, assuming a 128-bit key, and 132 clock cycles per byte on a Pentium Pro/II.

Plaintext (bytes) | Keying Option | Clocks to Key | Clocks to Encrypt | Total Clocks per Byte
16 | Zero | 1250 | 860 | 131.9
32 | Zero | 1250 | 1720 | 92.8
64 | Zero | 1250 | 4690 | 73.3
128 | Minimal | 2400 | 6880 | 63.5
256 | Partial | 4900 | 7360 | 47.9
512 | Compiled | 8600 | 8256 | 32.9
1K | Compiled | 8600 | 16512 | 24.5
2K | Compiled | 8600 | 33024 | 20.3
4K | Compiled | 8600 | 66048 | 18.2
8K | Compiled | 8600 | 132096 | 17.2
16K | Compiled | 8600 | 264192 | 16.7
32K | Compiled | 8600 | 528384 | 16.4
64K | Compiled | 8600 | 1056768 | 16.3
1M | Compiled | 8600 | 270532608 | 16.1
Table 5.3. Best Speed to Encrypt a Message with a New 128-bit Key on a Pentium Pro

5.1.6 Language, Compiler, and Processor Choice

As with most algorithms, the choices of language and compiler can have a huge impact on performance. It is clear that the Borland C 5.0 compiler chosen as the standard AES reference is not the best optimizing compiler. For example, the Microsoft Visual C++ 4.2 compiler generates Twofish code that is at least 20 percent faster than Borland on a Pentium computer, with both set to optimize for speed (630 clocks per block for Microsoft Visual C++ 4.2 versus 870 clocks per block for Borland C 5.0), on a Pentium Pro/II, the difference between the compilers is not quite as large (e.g., 600 clocks/block vs. 640 clocks/block), but it is still significant. Part of the difference stems from the inability of the Borland C compiler to generate intrinsic rotate instructions, despite documentation claiming that it is possible. This problem alone accounts for nearly half of the speed difference between Borland and Microsoft. The remaining speed difference comes simply from poorer code generation. The Borland C compiler is uniformly slower than Microsoft's compiler. The encryption speed in Microsoft C of 40 Pentium clocks per byte (i.e., 630 clocks/block at 16 bytes/block) is over ten percent faster than the best known DES assembly language implementation on the same platform. However, coding the Twofish algorithm in assembly language achieves speeds of 258 clocks, achieving a very significant speedup over any of the C implementations.

To make matters even more complicated, the assembly language that optimizes performance on a Pentium (or Pentium MMX) is drastically different from the assembly language required to maximize speed on a Pentium Pro or Pentium II, even though the final code size and speed achieved on each platform are almost identical. For example, the Pentium Pro/II CPUs can perform only one memory read per clock cycle, while the Pentium and Pentium MMX can perform two. However, the Pentium Pro/II can perform two ALU operations per clock in addition to memory accesses, while the Pentium can process only a total of two ALU operations or memory accesses per clock. These (and other) significant architectural differences result in the fact that running the optimal Pentium Twofish encryption code on a Pentium Pro results in a slowdown of nearly 2:1—and vice versa! Fortunately, it is relatively simple to detect the CPU type at run-time and select which version of the assembly code to use. Another anomaly is that the key schedule setup time is considerably faster (43 percent) on the Pentium MMX than on the Pentium, not because the key schedule uses any MMX instructions, but simply because of the larger cache size of the MMX chip.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Empirically, there also seems to be some anomalous behavior of the C compilers. In almost all cases, the encryption and decryption routines in C achieved speeds within a few percent of each other. However, there were cases in the table where the two speeds differed by considerably more than ten percent (we used the larger number in the table), which is very odd because the "inner loop" C code used is virtually identical. We also noted several cases where compiler switches that seemed unrelated to performance optimization sometimes caused very large changes in timings.

It should be noted that performance numbers for Pentium II processors are almost identical in all cases to those for a Pentium Pro, which is not surprising since Intel claims they have the same core. The Pentium and Pentium MMX achieve almost identical speeds for encryption and decryption, although, as noted above, the MMX key setup times are faster, due mainly to the larger cache size.

The bottom line is that, when comparing the relative performance of different algorithms, using the same language and compiler for all implementations helps to make the comparison meaningful, but it does not guarantee a valid measure of the relative speeds. We have listed many different software performance metrics across platforms and languages to facilitate speed comparisons between Twofish and other algorithms. Our belief is that, on any given platform (e.g., Pentium Pro), the assembly-language performance numbers are the best numbers to use to gauge absolute performance, since they are unaffected by the vagaries and limitations of the compiler (e.g., inability to produce rotate opcodes). High-level languages (e.g., C, Java) are also important because of the ease of porting to different platforms, but once an algorithm becomes standardized, it will ultimately be coded in assembly for the most popular platforms.

RAM, ROM, or EEPROM for Key Working RAM Code and Table Size Clocks per Block Time per Block @ 4MHz
24 | 36 | 2200 | 26500 | 6.6 msec
24 | 36 | 2150 | 32900 | 8.2 msec
24 | 36 | 2000 | 35000 | 8.7 msec
24 | 36 | 1750 | 37100 | 9.3 msec
184 | 36 | 1900 | 15300 | 3.8 msec
184 | 36 | 1700 | 18100 | 4.5 msec
184 | 36 | 1450 | 19200 | 4.8 msec
1208 | 36 | 1300 | 12700 | 3.2 msec
1208 | 36 | 1100 | 15500 | 3.9 msec
1208 | 36 | 850 | 16600 | 4.2 msec
3256 | 36 | 1000 | 11900 | 3.0 msec
Table 5.4. Twofish Performance on a 6805 Smart Card

5.2 Performance on Smart Cards

Twofish is ideally suited for smart cards. It can fit on the smallest smart cards while exhibiting reasonable performance, and can take advantage of more RAM or more powerful processors with increased performance. It can operate efficiently in environments that require rapid key changes, and can be implemented in dedicated hardware in only a few gates.

We implemented Twofish on a 6805 CPU, which is a typical smart card processor, with several different space—time tradeoff options [SKW+99b]. Our results are shown in Table 5.4. The code size includes both encryption and decryption.1 The block encryption and decryption times are almost identical. If only encryption is required, minor improvements in code size and speed can be obtained. The only key schedule precomputation time required in this implementation is the Reed-Solomon mapping used to generate the S-box key material S from the key M, which requires slightly over 1750 clocks per key. This setup time can be made considerably shorter at the cost of two additional 256-byte ROM tables. It should also be observed that the lack of a second index register on the 6805 has a significant impact on the code size and performance, so a different CPU with multiple index registers (e.g., 6502) might be a better fit for Twofish.

1For comparison purposes: DES on a 6805 takes about 1K code, 23 bytes of RAM, and 20000 clock cycles per block.

5.2.1 RAM Usage

For any encryption algorithm, memory usage can be divided into two parts: that required to hold the expanded key, and that required as working space to encrypt or decrypt text (including the text block). In applications where a smart card holds a single key for a long period of time, the key can be put into EEPROM or even ROM, greatly reducing RAM requirements. Most applications, however, require the smart card to encrypt using session keys, which change with each transaction. In these situations, the expanded key must be stored in RAM, along with working space to perform the encryption.

Twofish—the 128-bit key version—can be implemented in a smart card in 60 bytes of RAM. This includes the text block, key, and working space. If a slightly expanded key (16 bytes of the key plus another eight bytes of the Reed-Solomon results (S)) can be stored in ROM or EEPROM, then Twofish can be implemented in only 36 bytes of RAM. In either case, there is zero key setup time for the next encryption operation with the same key.2

2All of our implementations leave the key intact so that it can be used again.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Larger key sizes require more RAM to store the larger keys: 36 bytes for 192-bit keys and 48 bytes for 256-bit keys. If these applications can store key material in ROM or EEPROM, then these key lengths can be implemented on smart cards with only 36 bytes of RAM. All of this RAM can be reused for other purposes between block encryption operations.

For smart cards with larger memory to hold key-dependent data, encryption speed can increase considerably. This is because the round keys can be precomputed as part of the expanded key, requiring a total of 184 bytes of key memory. As shown in Table 5.4, this option nearly halves the encryption time. If the smart card has enough additional memory available to hold 1 Kbyte of precomputed S-box in either RAM, ROM, or EEPROM (for a total of 1208 bytes), performance improves further. Finally, as shown in the final row of Table 5.4, if the entire precomputed S-box plus MDS table can be held in memory (3256 bytes), the speed can again be increased slightly more. It should be noted that some of these "large RAM" implementations save 512 bytes of code space by assuming that certain tables are not required in ROM, with the entire precomputation being performed instead on the host that sets the key in the smart card. If the smart card has to perform its own key expansion the code size will increase. This increase has its own space/time tradeoff options.

This flexibility makes Twofish well-suited for both small and large smart-card processors: Twofish works in the most RAM-poor environments, while at the same time it is able to take advantage of both moderate-RAM cards and large-RAM cards.

5.2.2 Encryption Speed and Key Agility

On a 6805 with only 60 bytes of RAM, Twofish encrypts at speeds of 26500 to 37100 clocks per block, depending on the amount of ROM available for the code. On a 4 MHz chip, this translates to 6.6 msec to 9.3 msec per encryption. In these implementations, the key-schedule precomputation time is minimal: slightly over 1750 clocks per key. This setup time could be cut considerably at the cost of two additional 512-byte ROM tables, which would be used during the key schedule.

If ROM is expensive, Twofish can be implemented in less space at slower speeds. The space-speed tradeoffs are of two types: unrolling loops and implementing various lookup tables. By far, the latter has the larger impact on size and speed. For example, Twofish's MDS matrix can be computed in three different ways:

• Full table lookups for the multiplications by EF and 5B. This is the fastest, and requires 512 bytes of ROM for tables.
• Single table lookup for the multiplications by α-1. This is slower, but only requires 256 bytes of ROM for the table.
• No tables, all multiplies done with shifts and XORs. This is the slowest, and the smallest.

Longer keys are slower, but only slightly so. For the small memory versions, Twofish's encryption time per block increases by less than 2600 clocks per block for 192-bit keys, and by about 5200 clocks per block for 256-bit keys. Similarly, the key schedule precomputation increases to 2550 clocks for 192-bit keys, and to 3400 clocks for 256-bit keys.

As shown in Table 5.4, in smart card CPUs with sufficient additional RAM storage to hold the entire set of subkeys, the throughput improves significantly, although the key setup time also increases. The time savings per block is over 11000 clocks, cutting the block encryption time down to about 15000 clocks; i.e., nearly doubling the encryption speed. The key setup time increases by roughly the same number of clocks, thus making the key setup time comparable to a single block encryption. This approach also cuts down the code size by a few hundred bytes. It should be noted further that, in fixed-key environments, the subkeys can be stored along with the key bytes in EEPROM, cutting the total RAM usage down to 36 bytes while maintaining the higher speed.

As another tradeoff, if another 1 Kbyte of RAM or EEPROM is available, all four 8-bit S-boxes can be precomputed. Clearly, this approach has relatively low key agility, but the time required to encrypt a block decreases by roughly 6000 clocks. When combined with precomputed subkeys as discussed in the previous paragraph, the block encryption time drops to about 12000 clocks, nearly three times the best speed for "low RAM" implementations. In most cases, this approach would be used only where the key is fixed, but it does allow for very high throughput. Similarly, if 3 Kbytes of RAM or EEPROM is available for tables, throughput can be further improved slightly.

The wide variety of possible speeds again illustrates Twofish's flexibility in these constrained environments. The algorithm does not have one speed; it has many speeds, depending on available resources.

5.2.3 Code Size

Twofish code is very compact: 1760 to 2200 bytes for minimal RAM footprint, depending on the implementation. The same code base can be used for both encryption and decryption. If only encryption is required, minor improvements in code size can be obtained (on the order of 150 bytes). The extra code required for larger keys is fairly negligible: less than 100 extra bytes for a 192-bit key, and less than 200 bytes for a 256-bit key.

Observe that it is possible to save further ROM space by computing q0 and q1 lookups using the underlying 4-bit construction, as specified in Section 4.3.5. Such a scheme would replace 512 bytes of ROM table with 64 bytes of ROM and a small subroutine to compute the full 8-bit q0 and q1, saving perhaps 350 bytes of ROM; unfortunately, encryption speed would decrease by a factor of ten or more. Thus, this technique is only of interest in smart card applications for which ROM size is extremely critical but performance is not. Nonetheless, such an approach illustrates the implementation flexibility afforded by Twofish.

5.3 Performance on the Alpha

The 64-bit Alpha 21164 CPU can run up to 600 MHz using only a 0.35 micron CMOS process, compared to the 0.25 micron technology used in a Pentium II. The Alpha is widely regarded as the fastest general purpose processor available today. Its architecture and performance are expected to remain at the leading edge of technology for the foreseeable future. It has a 4-way superscalar architecture, which is fairly close in many respects to a Pentium II. Twofish should run on an Alpha in roughly the same number of clocks as on a Pentium Pro (i.e., 300).


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

5.4 Performance on Future Microprocessors

Given the ever-advancing capabilities of CPUs, it is worthwhile to make some observations about how the Twofish algorithm will run on future processors, including Intel's Merced. Not many details are known about Merced, other than that it includes an Explicitly Parallel Instruction Computing (EPIC) architecture, as well the ability to run existing Pentium code. EPIC is related to VLIW architectures that allow many parallel opcodes to be executed at once, while the Pentium allows only two opcodes in parallel, and the Pentium Pro/Pentium II may process up to three opcodes per clock. However, access to memory tables is limited in most VLIW implementations to only a few parallel operations, and we expect similar restrictions to hold for Merced. For example, an existing Philips VLIW CPU can process up to five opcodes in parallel, but only two of the opcodes can read from memory.

Since Twofish relies on 8-bit non-linear S-boxes, it is clear that table access is an integral part of the algorithm. Thus, Twofish might not be able to take advantage of all the parallel execution units available on a VLIW processor. However, there is still plenty of parallelism in Twofish that can be well-utilized in an optimized VLIW software implementation. Equally important, the alternative of not using large S-boxes, while it may allow greater parallelism, also naturally involves less non-linearity and thus generally requires more rounds. For example, Serpent [BAK98], based on "inline" computation of 4-bit S-boxes, may experience a relatively larger speedup than Twofish on a VLIW CPU, but Serpent also requires 32 rounds, and is considerably slower to start with.

It should also be noted that, as with most encryption algorithms, the primitive operations used in Twofish could very easily be added to a CPU instruction set to improve software performance significantly. Future mainstream CPUs may include such support for the new AES standard. However, it is also worthwhile to remember that DES has been a standard for more than twenty years, and no popular CPU has added instruction set support for it, even though DES software performance would benefit greatly from such features.

5.5 Hardware Performance

No actual logic design has been implemented for Twofish, but estimates in terms of gates for each building block have been made. As in software, there are many possible space-time tradeoffs in hardware implementations of Two-fish. Thus, it is not meaningful to give just one figure for the speed and size attributes of Twofish in hardware. Instead, we will try to outline several of the options and give estimates for speed and gate count of several different architectures.

For example, the round subkeys could be precomputed and stored in a RAM, or they could be computed on the fly. If computed on the fly, the h function logic could be time-multiplexed between subkeys and the round function to save size at a cost in speed, or the logic could be duplicated, adding gates but perhaps running twice as fast. If the subkeys were precomputed, the h function logic would be used during a key setup phase to compute the subkeys, saving gates but adding a startup time roughly equal to one block encryption time. Similarly, a single h function logic block could be time-multiplexed between computing T0 and T1, halving throughput but saving even more gates.

As another example of the possible tradeoffs, the S-boxes could be pre-computed and stored in on-chip RAMs, allowing faster operation because there is no need to ripple through several layers of key material XORs and q permutations. The addition of such RAMs (e.g., eight 256-byte RAMs) would perhaps double or triple the size of the logic, and it would also impose a significant startup time on key change to initialize the RAMs. Despite these disadvantages, such an architecture might raise the throughput by a factor of two or more (particularly for the larger key sizes), so for high-performance systems with infrequent re-keying, this option may be attractive.

The construction method specified in Section 4.3.5 for building the 8-bit permutations q0 and q1 from four 4-bit permutations was selected mainly to minimize gate count in many hardware implementations of Twofish. These permutations can be built either directly in logic gates or as full 256-byte ROMs in hardware, but such a ROM is usually several times larger than the direct logic implementation. Since each full h block in hardware (see Figure 4.2) involves six q0 blocks and six q1 blocks (for N = 128), the gate savings mount fairly quickly. The circuit delays in building q0 or q1 using logic gates are typically at least as low as those using ROMs, although this metric is certainly somewhat dependent on the particular silicon technology and circuit library available.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

It should also be noted that the Twofish round structure can be very nicely pipelined to break up the overall function into smaller and much faster blocks (e.g., q's, key XORs, MDS, PHT, subkey addition, Feistel XOR). None of these operations individually is slow, but trying to run all of them in a single clock cycle does affect the cycle time. In ECB mode, counter mode, or an interleaved chaining mode, the throughput can be dramatically increased by pipelining the Twofish round structure. As a very simple example of two-level pipelining, we could compute the qi's, key XORs and MDS multiply for one block during "even" clocks, while the PHT, subkey addition, and Feistel XOR would be computed on the "odd" clocks; a second block is processed in parallel on the alternate clock cycles. Using careful balancing of circuit delays between the two clock phases, this approach allows us to cut the logic delay in half, thus running the clock at twice the speed of an unpipelined approach. Such an approach does not require duplicating the entire Twofish round function logic, but merely the insertion of one extra layer of clocked storage elements (128 bits). Thus, for a very modest increase in gate count, throughput can be doubled, assuming that the application can use one of these cipher modes. It is clear that this general approach can be applied with more levels of pipelining to get higher throughput, although diminishing returns are achieved past a certain point. For even higher levels of performance, multiple independent engines can be used to achieve linear speedups at a linear ear cost in gates. We see no problem meeting NSA's requirement to "be able to encrypt data at a minimum of 1 Gb/s, pipelined if necessary, in existing technology" [McD97].

Gate Count h Blocks Clocks
/Block 		Interleave Levels Clock Speed Throughput (Mbits/sec) Startup Clocks
8000 | 0.25 | 324 | 1 | 80 MHz | 32 | 20
14000 | 1 | 72 | 1 | 40 MHz | 71 | 4
19000 | 1 | 32 | 1 | 40 MHz | 160 | 40
23000 | 2 | 16 | 1 | 40 MHz | 320 | 20
26000 | 2 | 32 | 2 | 80 MHz | 640 | 20
28000 | 2 | 48 | 3 | 120 | MHz 960 | 20
30000 | 2 | 64 | 4 | 150 MHz | 1200 | 20
80000 | 2 | 16 | 1 | 80 MHz | 640 | 300
Table 5.5. Hardware Tradeoffs (128-bit Key)

Table 5.5 gives hardware size and speed estimates for the case of 128-bit keys. The first line of the table is a "byte serial" implementation. It uses one clock per S-box lookup, and four clocks per h function (including the MDS). We allow two clocks for the PHT and key addition. With four h functions per round, each round requires 18 clock cycles.

The next line uses a fully wired h function, but still computes the round keys on the fly. The other versions all precompute the round subkeys. The last line is a version that precomputes the S-boxes in to dedicated RAM instead of computing the S-boxes on the fly. Depending on the architecture, the logic will grow somewhat in size for larger keys, and the clock speed (or startup time) may increase, but it is believed that a 128-bit AES scheme will be acceptable in the market long enough that most vendors will choose to implement that recommended key length.

These estimates are all based on existing 0.35 micron CMOS technology. All the examples in the table are actually quite small in today's technology, except the final (highest performance non-pipeline) instance, but even that is very doable today and will become fairly inexpensive as the next generation silicon technology (0.25 micron) becomes the industry norm.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 6
Twofish Design Philosophy

In the design of Twofish, we tried to stress the following principles:

Performance. When comparing different options, compare them on the basis of relative performance.
Conservativeness. Do not design close to the edge. In other words, leave a margin for error and provide more security than is provably required. Also, try to design against attacks that are not yet known.
Simplicity. Do not include ad hoc design elements without a clear reason or function. Try to design a cipher whose details can be easily kept in one's head.

These principles were applied not only to the overall design of Twofish, but to the design of the S-boxes and the key schedule.

6.1 Performance-Driven Design

The goal of performance-driven design is to build and evaluate ciphers on the basis of performance [SW97]. The early post-DES cipher designs would often compete on the number of rounds in the cipher. The original FEAL paper [SM88], for example, discussed the benefits of a stronger round function and fewer rounds. Other cipher designs of the period—REDOC-II [CW91], LOKI89 [BPS90] and LOKI91 [BKPS93], IDEA [LM91, LMM91]—only considered performance as an afterthought. Khufu/Khafre [Mer91] was the first published algorithm that explicitly used operations that were efficient on 32-bit microprocessors; SEAL [RC94, RC98] is a more recent example. RC2 [Riv97, KRRR98] and Jeroboam [CM98] were designed for 16-bit microprocessors, SOBER [Ros98] for 8-bit ones. Other, more recent designs, do not seem to take performance into account at all. Two 1997 designs, SPEED [Zhe97]1 and Zhu-Guo [ZG97], are significantly slower than alternatives that existed years previous.

1SPEED has been cryptanalyzed in [HKSW98, UTK98, HKR+98].

Arbitrary metrics, such as the number of rounds, are not good measures of performance. What is important is the cipher's speed: the number of clock cycles per byte encrypted. When ciphers are analyzed according to this property, the results can be surprising [SW97]. RC5 might have twice the number of rounds of DES,2 but since its round function is more than twice as fast as DES', RC5 is faster than DES on most microprocessors.

2Here we use the term "round" in the traditional sense: as it was defined by DES [NBS77] and has been used to describe Feistel-network ciphers ever since. The RC5 documentation [Riv95] uses the term "round" differently: one RC5-defined round equals two Feistel rounds.

Even when cryptographers made efforts to use efficient 32-bit operations, they often lacked a full appreciation of low-level software optimization principles associated with high-performance CPUs. Thus, many algorithms are not as efficient as they could be. Minor modifications in the design of Blowfish [Sch94], SEAL [RC94, RC98], and RC4 [Sch96] could improve performance without affecting security [SW97] (or, alternatively, increase the algorithms' complexity without affecting performance). In designing Twofish, we tried to evaluate all design decisions in terms of performance.

Since NIST's platform of choice was the Intel Pentium Pro [NIST97b], we concentrated on that platform. However, we did not ignore performance on other 32-bit CPUs, as well as 8-bit and 16-bit CPUs. If there is any lesson from the past twenty years of microprocessors, it is that the high end gets better and the low end never goes away. Yesterday's top-of-the-line CPUs are currently in smart cards. Today's CPUs will eventually be in smart cards, while the 8-bit microprocessors will move to devices even smaller. The only thing we did not consider in our performance metrics is bitslice implementations [Bih97, SAM97, NM97], since these can only be used in very specialized applications and often require unrealistic implementations; e.g., 32 simultaneous ECB encryptions, or 32 interleaved IVs.3

3One AES submission, Serpent [BAK98], uses ideas from bitslice implementations to create a cipher that is optimized for 32-bit processors while sacrificing performance on 8-bit processors.

6.1.1 Performance-driven Tradeoffs

During our design, we constantly evaluated the relative performance of different modifications to our round function. Twofish's round function encrypts at about 20 clock cycles; 16 rounds translates to about 320 clock cycles per block encrypted. When we contemplated a change to the round function, we evaluated it in terms of increasing or decreasing the number of rounds to keep performance constant. For example:

• We could have added a data-dependent rotation to the output of the two MDS matrices in each round. This would add 10 clock cycles to the round function on the Pentium (two on the Pentium Pro). To keep the performance constant, we would have to reduce the number of rounds to 11. The question to ask is: Are 11 rounds of the modified cipher more or less secure than 16 rounds of the unmodified cipher?
• We could have removed the one-bit rotation. This would have saved clocks equivalent to one Twofish round. Are 17 rounds of this new round function more or less secure than 16 rounds of the old one?
• We could have defined the key-dependent S-boxes using the whole key, instead of half of it. This would have doubled key setup time on high-end machines, and halved encryption speed on memory-poor implementations (where the S-boxes could not be precomputed). On memory-poor machines, we would have to cut the number of rounds in half to be able to afford this. Are 8 rounds of this improved cipher better than 16 rounds of the current design?

This analysis is necessarily dependent on the microprocessor architecture the algorithm is being compared on. While we focused on the Intel Pentium architecture, we also tried to keep 8-bit smart card and hardware implementations in mind. For example, we considered using an 8-by-8 MDS matrix over GF(24) to ensure a finer-grained diffusion, instead of a 4-by-4 MDS matrix over GF(28); the former would have been no slower on a Pentium but at least twice as slow on a low-memory smart card.

6.2 Conservative Design

In the last decade there has been considerable research in designing ciphers to be resistant to known attacks [Nyb91 Nyb93, OCo94a, OCo94b, OCo94c, Knu94a, Knu94b, Nyb94, DGV94b, Nyb95, NK95, Mat96, Nyb96], such as differential [BS93], linear [Mat94], and related-key cryptanalysis [WH87, Bih94, KSW96, KSW97]. This research has culminated in strong cipher designs—CAST-128 [Ada97a] and MISTY [Mat97] are probably the most noteworthy—as well as some excellent cryptanalytic theory.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

However, it is dangerous to rely solely on theory when designing ciphers. Ciphers provably secure against differential cryptanalysis have been attacked with higher-order differentials [Lai94, Knu95b] or the interpolation attack [JK97]: KN-cipher [NK95] was attacked in [JK97, SMK98], Kiefer [Kie96] in [JK97], and a version of CAST in [MSK98a]. The CAST cipher cryptanalyzed in [MSK98a] is not CAST-128, but it does illustrate that while the CAST design procedure [AT93, HT94a] can create ciphers resistant to differential and linear cryptanalysis, it does not create ciphers resistant to whatever form of cryptanalysis comes next. SNAKE [LC97], another cipher provably secure against differential and linear cryptanalysis, was successfully broken using the interpolation attack [MSK98b). When designing a cipher, it is prudent to assume that new attacks will be developed in order to break it.

We took a slightly different approach in our design. Instead of trying to optimize Twofish against known attacks, we tried to make Twofish strong against both known and unknown attacks. While it is impossible to optimize a cipher design for resisting attacks that axe unknown, conservative design and overengineering can instill some confidence.

Many elements of Twofish reflect this philosophy. We used well-studied design elements throughout the algorithm. We started with a Feistel network, probably the most studied block-cipher structure, instead of something newer like an unbalanced Feistel network [SK96, ZMI90] or a generalized Feistel network [Nyb96].

We did not implement multiplication mod 216 + 1 (as in IDEA or MMB [DGV93]) or data-dependent rotations (as in Madryga [Mad84] and RC54 or Akelarre [AGMP96]5) for non-linearity. The most novel design elements we used-MDS matrices and PHTs-are only intended for diffusion (and are used in Square [DKR97] and SAFER, respectively).

4RC5's security is almost wholly based on data-dependent rotations. Although initial cryptanalysis was promising [KY95] (see also [Se198]), subsequent research [KM97, BK98] suggests that there is considerably more to learn about the security properties of data-dependent rotations.
5Akelaxre was severely broken in [FS97, KR97].

We used key-dependent S-boxes because they offer adequate protection against known statistical attacks and are likely to offer protection to any unknown similar attacks. We defined Twofish at 16 rounds, even though our analysis cannot break anywhere near that number. We added one-bit rotations to prevent potential attacks that relied solely on the byte structure. We designed a very thorough key schedule to prevent related-key and weak-key attacks.

6.3 Simple Design

A guiding design principle behind Twofish is that the round function should be simple enough for us to keep in our heads. Anecdotal evidence from algorithms like FEAL [SM88], CAST, and Blowfish indicates that complicated round functions are not always better than simple ones. Also, complicated round functions are harder to analyze and rely on more ad hoc arguments for security (e.g., REDOC-II [CW91]).

However, with enough rounds, even bad round functions can be made to be secure.6 Even a simple round function like TEA's [WN95] or RC5's seems secure after 32 rounds [BK98]. In Twofish, we tried to create a simple round function and then iterate it more than enough times for security.

6Student cryptography projects bear this observation out. At 16 rounds, the typical student cipher fares rather badly against a standard suite of statistical tests. At 32 rounds, it looks better. At 128 rounds, even the worst designs look very good.

6.3.1 Reusing Primitives

One of the ways to simplify a design is to reuse the same primitives in multiple parts of a cipher. Cryptographic design does not lend itself to the adage of not putting all your eggs in one basket. Since any particular "basket" has the potential of breaking the entire cipher, it makes more sense to use as few baskets as possible—and to scrutinize those baskets intensely.

To that end, we used essentially the same construction (8-by-8-bit key-dependent S-boxes consisting of alternating fixed permutations and subkey XORs followed by an MDS matrix followed by a PHT) in both the key schedule and the round function. The differences were in the key material used (the round function's g function uses a list of key-derived words processed by an RS code; the key schedule's h function uses individual key bytes directly) and the rotations. The rotations represent a performance-driven design trade-off: putting the additional rotations into F would have unacceptably slowed down the cipher performance on high-end machines. The use of the RS code to derive the key material for g adds substantial resistance to related-key attacks.

While many algorithms reuse the encryption operation in their key schedule (e.g., Blowfish, Panama [DC98a, DC98b], RC4, CRISP [Lee96], YTH [YTH96]), and several alternative DES key schedules reuse the DES operation [Knu94b, BB96], we are unaware of any that reuse the same primitives in exactly this manner.7 We feel that doing so greatly simplifies the analysis of Twofish, since the same kinds of analysis can apply to the cipher in two different ways.

7The closest idea is an alternate DES key schedule that uses the DES round function, both the 32-bit block input and 48-bit key input, to create round subkeys [Ada97b].

6.3.2 Reversibility

While it is essential that any block cipher be reversible, so that ciphertext can be decrypted back into plaintext, it is not necessary that the identical function be used for encryption and decryption. Some block ciphers are reversible with changes only in the key schedule (e.g., DES, IDEA, Blowfish), while others require different algorithms for encryption and decryption (e.g., SAFER, Serpent, Square).

The Twofish encryption and decryption round functions are slightly different, but are built from the same blocks. That is, it is simple to build a hardware or software module that does both encryption and decryption without duplicating much functionality, but the exact same module cannot both encrypt and decrypt.

Note that having the cipher work essentially the same way in both directions is a nice feature in terms of analysis, since it lets analysts consider chosen-plaintext and chosen-ciphertext attacks at once, rather than considering them as separate attacks with potentially radically different levels of difficulty [Cop98].


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

6.4 S-boxes

The security of a cipher can be very sensitive to the particulars of its S-boxes: size, number, values, usage. Ciphers invented before the public discovery of differential cryptanalysis sometimes used arbitrary sources for their S-box entries.

Randomly constructed known S-boxes are unlikely to be secure. Khafre uses S-boxes taken from the RAND tables [RAND55], and it is vulnerable to differential cryptanalysis [BS92]. NewDES8 [Sco85], with S-boxes derived from the Declaration of Independence [Jeff+76], could be made much stronger with good S-boxes. DES variants with random fixed S-boxes are very likely to be weak [BS93, Mat95], and CMEA was weakened extensively because of a poor S-box choice [WSK97].

8Despite the algorithm name, NewDES is neither a DES variant nor a new algorithm based on DES.

Some cipher designers responded to this threat by carefully crafting S-boxes to resist known attacks—DES [Cop94], sn DES [KPL93, Knu93c, KLPL95], CAST [MA96, Ada97a]—while others relied on random key-dependent S-boxes for security—Khufu, Blowfish, WAKE [Whe94].9 The best existing attack on Khufu breaks 16 rounds [CC94], while the best attack on Blowfish breaks only four [Rij7]. Serpent [BAK98] reused the DES S-boxes.

9The WAKE design has several variants (Cla97, CIa98]; neither the basic algorithm nor its variants have been extensively cryptanalyzed.

GOST [GOST89] navigated a middle course: each application has different fixed S-boxes, turning them into an application-specific family key.

6.4.1 Large S-boxes

S-boxes vary in size, from GOST's 4-by-4-bit S-boxes to Tiger's 8-by-64-bit S-boxes [AB96b]. Large S-boxes are generally assumed to be more secure than smaller ones—a view we share—but at the price of increased storage requirements; DES' eight 6-by-4-bit S-boxes require 256 bytes of storage, while Blowfish's four 8-by-32-bit S-boxes require 4 Kbytes. Certainly input size matters more than output size; an 8-by-64-bit S-box can be stored in 2 Kbytes, while a 16-by-16-bit S-box requires 128 Kbytes. (Note that there is a limit to the advantages of making S-boxes bigger. S-boxes with small input size and very large output size tend to have very good linear approximations; S-boxes with sufficiently large outputs relative to input size axe guaranteed to have at least one perfect linear approximation [Bih95].)

Twofish used the same solution as Square: mid-sized S-boxes (8-by-8-bit) used to construct a large S-box (8-by-32-bit).

6.4.2 Algorithmic S-boxes

S-boxes can either be specified as large tables, like DES, Khufu/Khafre, and YLCY [YLCY98], or derived algebraically, like FEAL, LOKI89/LOKI91 (and LOKI97 [Bro98]), IDEA, and SAFER. The advantage of the former is that there is no mathematical structure that can potentially be used for cryptanalysis. The advantage of the latter is that the S-boxes are more compact, and can be more easily implemented in applications where the ROM or RAM for large tables is not available.

Algebraic S-boxes can result in S-boxes that are vulnerable to differential cryptanalysis: [Mur90] against FEAL, and [Knu93a, Knu93b] against LOKI. Higher-order differential cryptanalysis is especially powerful against algorithms with simple algebraic S-boxes [Knu95b, JK97, SMK98].

Both tabular and algebraic techniques, however, can be used to generate S-boxes with given cryptographic properties simply by testing the results of the generation algorithm. There has been much written about testing S-boxes for their resistance to different statistical attacks, starting from the work done on DES [AT90, AT93, Cop94, Ada97a, Mor98].

In Twofish we tried to do both: we chose to build our 8-by-8-bit S-boxes algorithmically out of 4-by-4-bit S-boxes. However, we chose the 4-by-4-bit S-boxes randomly and then extensively tested the resulting 8-by-8-bit S-boxes against the cryptographic properties we required. This idea is similar to the one used in CS-Cipher [SV98].

6.4.3 Key-dependent S-boxes

S-boxes are either fixed for all keys or key dependent. It is our belief that ciphers with key-dependent S-boxes are, in general, more secure than fixed S-boxes.

There axe two different philosophies regarding key-dependent S-boxes. In some ciphers, the S-box is constructed specifically to ensure that no two entries are identical—Khufu and WAKE—while others simply create the S-box randomly and hope for the best: REDOG-II [CW91] and Blowfish [Sch94]. The latter results in a simpler key schedule, but may result in weaknesses (e.g., a weakness in reduced-round variants of Blowfish [Vau96a]). Another strategy is to generate key-dependent S-boxes from a known secure S-box and a series of strict mathematical rules: e.g., Biham-DES [BB94].

Most key-dependent S-boxes are created by some process completely orthogonal to the underlying cipher. SEAL, for example, uses SHA [NIST93] to create its key-dependent S-boxes. Blowfish uses repeated iterations of it-self The results are S-boxes that axe effectively random, but the cost is an enormous performance penalty in key setup time.10

10For example, setting up a single Blowfish key takes as much time as encrypting 520 blocks, or 4160 bytes, of data.

An alternative is to build the S-boxes using fairly simple key-dependent operations from fixed S-boxes. This results in a much faster key setup, but unless the creation algorithm is extensively cryptanalyzed together with the encryption algorithm, unwanted synergies could lead to attacks on the resulting cipher.

To avoid differential (as well as high-order differential, linear, and related-key) attacks, we made the small S-boxes key dependent. It is our belief that while random key-dependent S-boxes can offer acceptable security if used correctly, the benefits of a surjective S-box are worth the additional complexities that constructing them entails. So, to avoid attacks based on non-surjective round functions [BB95, RP95b, RPD97, CWSK98], we made the 8-by-8-bit S-boxes bijective.

This construction is similar to Skipjack's use of a single fixed 8-by-8-bit S-box, four key bytes, and a 4-round Feistel structure to create five different 16-by-16-bit key-dependent S-boxes [NSA98].

However, there is really no such thing as a key-dependent S-box. Twofish uses a complex multi-stage series of S-boxes and round subkeys that are often precomputed as key-dependent S-boxes for efficiency purposes. (For example, see Figure 4.3 on page 18.) We often used this conceptualization. when carrying out our own cryptanalysis against Twofish.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

6.5 The Key Schedule

An algorithm's key schedule is the mechanism that distributes key material to the different parts of the cipher that need it, expanding the key material in the process. This is necessary for three reasons:

• There are fewer key bits provided as input to the cipher than are needed by the cipher.
• The key bits used in each round must be unique to the round in order to avoid "slide" attacks [Wag95b].
• The cipher must be secure against an attacker with partial knowledge or control over some key bits.

When key schedules are poorly designed, they often lead to strange cipher properties: large classes of equivalent keys, self-inverse keys, and so forth. These properties can often aid an attacker in a real-world attack. For example, the DES weak (self-inverse) keys have been exploited in many attacks on larger cryptographic mechanisms built from DES [Knu95a], and the S-1 [Anon95] cipher was broken due to a bad key-schedule design [Wag95a]. Even worse, they can make attacks on the cipher easier, and some attacks on the cipher will be focused directly at the key schedule, such as related-key differential attacks [KSW96, KSW97]. These attacks can be especially devastating when the cipher is used in a hash function construction.

Key schedules can be divided into several broad categories [CDN98]. In some key schedules, knowledge of a round subkey uniquely specifies bits of other round subkeys. In some ciphers the bits are just reused, as in DES, IDEA, and LOKI; and in others some manipulation of the round subkeys is required to determine the other round subkeys: e.g., CAST and SAFER. Other key schedules are designed so that knowledge of one round subkey does not directly specify bits of other round subkeys. Either the round subkey itself is used to generate the other round subkeys in some cryptographically secure manner, as in RC5 and CS-Cipher [SV98], or a one-way function is used to generate the round subkeys (sometimes the block cipher itself): e.g., Blowfish, Serpent [BAK98], ICE11 [Kwa97], and Shark.

11ICE was cryptanalyzed in [RKR98].

Some simple design principles guided our development of the key schedule for Twofish:

Design the Key Schedule for the Cipher. This is not simply a cryptographic PRNG or hash function grafted onto the cipher; the Twofish key schedule is instead an integral part of the whole cipher design.
Reuse the Same Primitives. The Twofish key schedule's subkey generation mechanism, h, is built from the same primitives as the Twofish round function. This allowed us to apply much of the same analysis to both the round function and the subkey generation. This also makes for a relatively simple picture of the cipher and key schedule together. It is reasonable to consider one round's operations and the derivation of its subkeys at the same time.
Use All Key Bytes the Same Way. All key material goes through h (or g, which is the same function). That is, the only way a key bit can affect the cipher is after it defines a key-dependent S-box. This allows us to analyze the properties of the key schedule in terms of the properties of the byte permutations.
Make It Hard to Attack Both S-box and Subkey Generation. The key material used to derive the key-dependent S-boxes in g is derived from the key using an RS code having properties similar to those of the MDS matrix. Deriving the key material in this way maximizes the difficulties of an attacker's trying to mount any kind of related-key attack on the cipher, by giving him conflicting requirements between controlling the S-box keys and controlling the subkeys.

The key schedule design of some other ciphers has led to various undesirable properties. These properties, such as the existence of equivalent keys; DES-style weak, semi-weak, and quasi-weak keys; and DES-style complementation properties do not necessarily make the cipher weak. However, they tend to make it harder to use the cipher securely. With our key schedule, we can make convincing arguments that none of these properties exists.

6.5.1 Performance Issues

Key schedules vary widely in performance. The DES key schedule can be computed in less than the time required to do one encryption. The Blowfish key schedule requires the time equivalent to 521 encryptions to complete. Most other algorithms fall somewhere in the middle.

For large messages, performance of the key schedule is minor compared to performance of the encryption and decryption functions. For smaller messages, key setup can overwhelm encryption speed. In the design of Twofish, we tried to balance these two items. Our performance criteria included:

• The key schedule must be precomputable for maximal efficiency. This involves trying to minimize the amount of storage required to keep the precomputed key material.
• The key schedule must work "on the fly," deriving each block of subkey material as it is needed, with as little required memory as possible.
• The key schedule must be reasonably efficient for hardware implementations.
• The key schedule must have minimal latency for changing keys.12
12In its comments on the AES criteria, the NSA suggested that "a goal should be that two blocks could be enciphered with different keys in virtually the same time as two blocks could be enciphered with the same key" [McD97]. The cynical reader would immediately conclude that the NSA is concerned with the efficiency of their brute-force keysearch machines. However, there axe implementations where key agility is a valid concern. Key-stretching techniques can always be used to frustrate brute-force attacks [QDD86, KSHW98]. A better defense, of course, is to always use keys too laxge to make a brute-force search practicable, and to generate them randomly.

If security were not an issue, we would design a simple key schedule where the key bits were used in some natural order, like Skipjack, or with some minimal shuffling, like DES and IDEA. However, these key schedules cause weaknesses when the block cipher is used as a one-way hash function.

If performance were not an issue, it would make sense to simply use a one-way hash function to expand the key into the subkeys and S-box entries, as is done in Khufu, Blowfish, and SEAL. However, the AES efficiency requirements make such an approach unacceptable.

Balancing these two requirements led us to design a relatively simple key schedule with a very complicated analysis.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 7
The Design of Twofish

7.1 The Round Structure

An n-bit block cipher with a k-bit key uses its key to select among 2k possible different permutations on the 2n. possible inputs/outputs to the cipher. This can be imagined as an enormous (virtual) codebook; each possible n-bit plaintext block leads to only one possible n-bit ciphertext block under a given key. Data is encrypted in n-bit chunks, with each n-bit chunk's encryption independent of other encryptions. This can be contrasted with a stream cipher, where different parts of the message are treated differently. The goal for a block cipher is to be impossible in practice to distinguish from a random permutation family on n-bit blocks.

DES is a 64-bit cipher, and most block ciphers in the literature are also 64-bit ciphers; AES, on the other hand, requires a 128-bit block cipher.

A product cipher (sometimes called an iterative block cipher) is a block cipher made by iterating a fairly simple round function many times, each time with its own key. Thus, a 4-round product cipher would look like

EK(X) = RK3(RK2(RK1(RK0(X))))

where K0...3 are functions of K.

Each round function, RK(), is actually a weak block cipher. It implements the basic Shannon principles of encryption, confusion and diffusion [Sha49], and can be viewed as a key-dependent transformation of plaintext into ciphertext. It is weak in the sense that if an attacker knows much about the plaintext (such as simple plaintext statistics), he can quickly break the round function and recover its round key.

Even a very weak round function is resistant to attack when the sequence of inputs to the function is random and unknown to the attacker. Iterating the round function many times makes the input to the last round very hard for an attacker to guess; this prevents the attacker from being able to attack the last round, and thus prevents him from attacking the cipher as a whole. An increased number of rounds means a decrease in the throughput of a cipher, but an attack effective against an n-round version of a particular algorithm might be ineffective against an n + 1-round version. Some of the best cipher analysis shows how to break reduced-round variants of the design and why each attack cannot be extended to a greater number of rounds.

*
* Fig. 7.1. A Simple SP-network Construction

7.1.1 Common Block Cipher Structures

All well-known block ciphers use this iterated round structure, and given the number of different algorithms that have been proposed over the years, there are surprisingly few round-function structures.

SP Networks. These were named by Horst Feistel [Fei73, FNS75], although the ideas are much older. "SP" stands for "substitution-permutation," and one round of an SP-network consists of a substitution layer followed by a permutation layer. These two layers are meant to capture C. Shannon's fundamental notions in cipher design of confusion and diffusion [Sha49]. The substitution layer confuses the statistics of the input by breaking it up into smaller pieces and performing substitutions on those pieces. The permutation layer diffuses the statistics of the input by rearranging the bits output from the substitution layer, each of which should be a function of more than one input bit. Figure 7.1 gives a simple SP-network construction (which is not secure).

SP-networks have been well-studied in the cryptographic literature; examples of SP-network designs are SAFER [Mas94], Shark [RDP+96], and Square [DKR97].1 Rotor machines can be viewed as SP networks: the rotor is a substitution layer, and the key-dependent stepping of the rotor is the permutation.

1 See also [HT94b].

Feistel Networks. These were invented by Horst Feistel [FNS75] in his design of Lucifer[Fei73]. The fundamental building block of a Feistel network is the F function, a key-dependent mapping of an input string onto an output string. An F function is always non-linear and possibly non-surjective2

2 A non-surjective F function is one in which not all outputs in the output space can occur.

F : {0,1}n/2 × {0, 1}k → {0, 1}n/2

where n is the block size of the Feistel network, and F is a function taking n/2 bits of the block and k bits of a key as input, and producing an output of length n/2 bits. In each round, the "source block" is the input to F, and the output of F is XORed with the "target block," after which these two blocks swap places for the next round. Feistel networks are the most studied block cipher structure, and are the basis for most of the algorithms proposed in the literature.

*
* Fig. 7.2. A 3-round Feistel Network

Figure 7.2 shows a 3-round Feistel network. Here (PL, PR) is the plaintext, (CL, CR) is the ciphertext, (L1, R1) and (L2, R2) are intermediate values, and K1, K2, and K3 are round-dependent subkeys.

In order to decrypt the ciphertext (CL, CR) one starts at the tail end of the network and works backwards. It is important to understand that data flows the same direction through F even during decryption and that the reversibility of the network comes from the property that α ⊕ α = 0 for any α. This is easy to see if we compute the ciphertext as a function of the plaintext:

(L1, R1) = (PL, PR ⊕ F(PL))
(L2, R2) = (R1, L1 ⊕ F(R1))
(CL, CR) = (R2, L2 ⊕ F(R2))

Then reversing the network we find:

(L2, R2) = (CR ⊕ F(CL), CL)
(L1, R1) = (R2 ⊕ F(L2), L2)
(PL, PR) = (R1 ⊕ F(L1), L1)

*
* Fig. 7.3. An Incomplete 2-round Feistel Network

Incomplete Feistel Networks. This is a variant of the Feistel network where the F-function has an input and output size that is some fraction of n other than n/2. Skipjack, for example, has an F-function whose input and output is one-quarter the block size [NSA98]:

F : {0,1}n/4 × {0, 1}k → {0, 1}n/4

Figure 7.3 shows a 2-round variant of an incomplete Feistel network whose F-function also has an input and output of one-quarter the block size. In each round, the source block is the input into F, and is XORed with the target block; the other bits are unused in the round (hence the name "incomplete Feistel network").3 Then, the subblocks are rotated among each other, so that each subblock becomes the source and target in subsequent rounds. The first incomplete Feistel networks in the literature were Khufu and Khafre [Mer91], although their similarities to shift-register-based stream ciphers (common in military cryptographic hardware) make it likely that there is a significant amount of classified literature on their design and analysis.

3A taxonomy of these Feistel-network variants can be found in [SK96].

Source-heavy Feistel Networks. In this variant of the Feistel network, the source and target blocks are of different size, with the source block larger than the target block. Both RC2 [Riv97] and MacGuffin [BS95] have a 48-by-16-bit source-heavy structure; the F-function takes a 48-bit input and produces a 16-bit output. The underlying block cipher in MD4 [Riv91] and MD5 [Riv92] are 96-by-32-bit structures. SHA-1 [NIST93] is a 128-by-32-bit structure. Figure 7.4 shows a source-heavy 2-round Feistel network.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Target-heavy Feistel Networks. These are the opposite: Feistel-type networks where the F-function has a larger output than input. Tiger [AB96b] uses a 64-by-192-bit target-heavy structure. Figure 7.5 shows a target-heavy 2-round Feistel network.

Other Variants. There are other round function structures, which are variants of the above. IDEA [LMM91] is a variant of an SP-network, for example. In "fenced" constructions, multiple parallel implementations of the same structure are separated by specially constructed mixing operations [Rit96].

*
* Fig. 7.4. A Source-heavy 2-round Feistel Network

*
* Fig. 7.5. A Target-heavy 2-round Feistel Network

Although there has been some analysis of incomplete, source-heavy, and target-heavy Feistel networks [SK96], they are not nearly as well-understood as Feistel and SP networks. Given the current state of analysis, we discarded any of the newer structures.

Our Choice. Twofish was designed as a Feistel network, primarily because it is one of the most studied block-cipher building blocks, but also because the Feistel structure takes care of its own inverse. The structure of a Feistel network means that the F function need only be calculated in one direction.4 This means that we were able to use operations in our F function that are inefficient in the other direction, and make do with tables and constants for one direction only. Contrast this with an SP-network, which must execute its encryption function in both the forward and backward directions.

4In fact, the F function can be non-surjective, as it is in DES or Blowfish.

7.2 The Key-dependent S-boxes

A fundamental component of Twofish is the set of four key-dependent S-boxes. These must have several properties:

• The four different S-boxes actually need to be different.
• Few or no keys may cause the S-boxes used to be "weak," in the sense of having high-probability differential or linear characteristics, or in the sense of having a very simple algebraic representation.
• There should be few or no pairs of keys that define the same S-boxes. That is, changing even one bit of the key used to define an S-box should always lead to a different S-box. In fact, these pairs of keys should lead to extremely different S-boxes.

7.2.1 The Fixed Permutations q0 and q1

The construction method for building q0 and q1 from 4-bit permutations (specified in Section 4.3.5) was chosen because it decreases hardware and memory costs for some implementations, as discussed previously, without adding any apparent weaknesses to the cipher. It is helpful to recall that these individual fixed-byte permutations are used only to construct the key-dependent S-boxes, which, in turn, are used only within the h and g functions. In particular, the individual characteristics of q0 and q1 are not terribly relevant (except perhaps in some related-key attacks), because Twofish always uses at least three of these permutations in series together with at least two XORs with key material bytes.

Consideration was initially given to using random full 8-bit permutations for q0 and q1, as well as algebraically derived permutations (e.g., multiplicative inverses over GF(28)) that have slightly better individual permutation characteristics, but no substantial improvement was found when composite keyed S-boxes were constructed and compared to the q0 and q1 used in Two-fish.

The construction of q0 and q1 is a basically a 2-round SP network. We investigated several possible constructions. The main alternative was a 3- or 4-round Feistel network. We chose the SP network, as it has a lower circuit-depth. This reduces the propagation delay in a hardware implementation of the q-box. As several q-boxes are used in series, an increase in propagation delay has significant effects on the hardware speed of the cipher.

The q0 and q1 permutations were constructed by performing a random search for the 4-bit permutations t0, t1, t2, and t3. Using the notation of Matsui [Mat96], we define

and

where q is the mapping DPmax and LPmax are being computed for, the probabilities are taken over a uniformly distributed X, and the operator computes the overall parity of the bitwise-AND of its two operands. Only fixed permutations with DPmax ≤ 10/256, LPmax ≤ 1/16, and fewer than three fixed points5 were accepted as potential candidates. These criteria alone rejected over 99.8 percent of all randomly chosen permutations of the given construction. Pairs of permutations meeting these criteria were then evaluated as potential (q0, q1) pairs, computing various metrics when combined with key material into Twofish's S-box structure, as described below.

5A fixed point for a function f is a value x such that f(x) = x.

The actual q0 and q1 chosen were one of several pairs with virtually identical statistics that were found with only a few tens of hours of searching on Pentium class computers. Each permutation has DPmax = 10/256 and LPmax = 1/16; q0 has one fixed point, while q1 has two fixed points.

7.2.2 The S-boxes.

Each S-box is defined with two, three, or four bytes of key material, depending on the Twofish key size. This is done as follows for 128-bit Twofish keys:

s0(x) = q1[q0[q0[x] ⊕ s0,0] ⊕ s1,0]
s1(x) = q0[q0[q1[x] ⊕ s0,1] ⊕ s1,1]
s2(x) = q1[q1[q0[x] ⊕ s0,2] ⊕ s1,2]
s3(x) = q0[q1[q1[x] ⊕ s0,3] ⊕ s1,3]

where the si,j are the bytes derived from the key bytes using the RS matrix. Note that if s0,i = s0,j and s1,i = s1,j for i ≠ j, then si(x) ≠ sj(x) for most values of x. In fact, as each S-box uses a unique order of q-boxes, it is extremely unlikely that two different S-boxes could produce the identical mapping.

Note that when all si,j = 0, then s0(x) = q1[s1(q1–1[x])]. Similar relationships hold between other S-boxes. We have not been able to find any weaknesses resulting from this, as long as q0 and q1 have no high-probability differential characteristics.

In some sense this construction is similar to a rotor machine [PB92, DK85, Bau93, Bau97], with two different types of rotor (q0 and q1). The first rotor is fixed, and the fixed offset between two rotors is specified by a key byte. We did not find any useful cryptanalysis from this parallel, but someone else might.

For the 128-bit key, we have experimentally verified that each N/8-bit key used to define a byte permutation results in a distinct permutation. For example, in the case of a 128-bit key, the S-box s0 uses 16 bits of key material. Each of the 216 s0 permutations defined is distinct, as is also the case for s1, s2, and s3. We have not yet exhaustively tested longer key lengths, but we conjecture that all S-boxes generated by our construction are distinct. We also conjecture that this would be the case for almost all choices of q0 and q1 meeting the basic criteria discussed above.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

 |  | – log2(Pr(DPmax = x/256))
Key Size | Max Value | x = | 8 | 10 | 12 | 14 | 16 | 18 | 20 | 22 | 24
128 bits | 18/256 | 15.4 | 1.3 | 0.9 | 4.1 | 8.0 | 12.0 |  |  |
192 bits | 24/256 | 15.2 | 1.3 | 0.9 | 4.1 | 8.0 | 12.2 | 16.5 | 20.8 | 25.0
256 bits* | 22/256 | 15.1 | 1.3 | 0.9 | 4.1 | 8.0. | 12.2 | 16.7 | 22.0 |
Table 7.1. DPmax Over All Keys

7.2.3 Exhaustive and Statistical Analysis

Given the fixed permutations q0 and q1, and the definitions for how to construct s0, s1, s2, and s3 from them, we have performed extensive testing of the characteristics of these key-dependent S-boxes. In the 128-bit key case, all testing has been performed exhaustively, which is feasible because each S-box uses only 16 bits of key material. In many cases, however, only statistical (i.e., Monte Carlo) testing has been possible for the larger key sizes. In this section, we present and discuss these results. It is our hope to complete exhaustive testing over time for the larger key sizes where feasible, but the probability distributions obtained for the statistical tests give us a fairly high degree of confidence that no surprises are in store.

Computing DPmax Over All Keys. Table 7.1 shows the DPmax distribution for the various key sizes. A detailed explanation of the format of this table will also help in understanding the other tables in this section. An asterisk (*) next to a key size in the tables discussed in this section indicates that the entries in that row are a statistical distribution using Monte Carlo sampling of key bits, not an exhaustive test. For example, in Table 7.1, only the 256-bit key row uses statistical sampling; the 128-bit and 192-bit cases involve an exhaustive test. Clearly, the maximum value from a statistical sample is not a guaranteed maximum.

Note that, for 128-bit keys, each S-box has a DPmax value no larger than 18/256. The remaining columns give the distribution of observed DPmax values. Each entry is expressed as the negative base-2 logarithm of the fraction of S-boxes with the given value (or value range), with blank entries indicating that no value in the range was found. For example, in the N = 128 case only one of every 212.0 key-dependent S-boxes has DPmax = 18/256, while over half (1 in 20.9) have DPmax = 12/256. These statistics are taken over all four S-boxes (s0, s1, s2, s3), so a total of 4 × 216 (i.e., 256K) S-boxes were evaluated for the 128-bit key case. Each Monte Carlo sampling involves at least 216 S-boxes, but in many cases the number is considerably larger.

Computing LPmax Over All Keys. Table 7.2 shows the distribution of LPmax for the various key sizes. Observe that the vast majority of Twofish S-boxes have LPmax < (88/256)2, although there is a small fraction of S-boxes with larger values. For 128-bit keys, no Twofish S-box has an LPmax value greater than (100/256)2, while the maximum value is somewhat higher for larger key sizes. Monte Carlo statistics are given for the larger two key sizes, since the computational load for computing LPmax is roughly a factor of 15 higher than for DPmax.

 |  | – log2(Pr(LPmax = (x/256)2))
Key size | Max Value | x = | 56..63 | 64..71 | 72..79 | 80..87 | 88..95 | 96..103 | 104..111
128 bits | (100/256)2 | 9.3 | 1.0 | 1.2 | 4.2 | 8.0 | 12.4 |
192 bits* | (104/256)2 | 9.3 | 1.0 | 1.2 | 4.2 | 8.2 | 12.2 | 17.0
256 bits* | (108/256)2 | 9.4 | 1.0 | 1.2 | 4.2 | 8.1 | 12.5 | 17.4
Table 7.2. LPmax Over All Keys.

 |  | – log2(Pr(# fixed points = x))
Key Size | Max Value | x = | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10
128 bits | 8 | 1.4 | 1.4 | 2.4 | 4.1 | 6.0 | 8.2 | 11.1 | 14.1 | 17.0 |  |
192 bits | 10 | 1.4 | 1.4 | 2.4 | 4.0 | 6.0 | 8.4 | 10.9 | 13.8 | 16.8 | 19.8 | 23.4
256 bits* | 10 | 1.4 | 1.4 | 2.4 | 4.0 | 6.0 | 8.4 | 10.9 | 13.7 | 16.8 | 21.0 | 21.0
random |  | 1.4 | 1.4 | 2.4 | 4.0 | 6.0 | 8.3 | 10.9 | 13.7 | 16.7 | 19.9 | 23.2
Table 7.3. Number of Fixed Points Over All Keys


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Counting Fixed Points Over All Keys. The distribution of the number of permutation fixed points for various key sizes is given in Table 7.3. A fixed point of an S-box si is a value x for which x = si(x). This metric does not necessarily have any direct cryptographic significance. However, it is a useful way to verify that the S-boxes are behaving similarly to random S-boxes, since it is possible to compute the theoretical distribution of fixed points for random S-boxes. The probabilities for random bijective S-boxes is given in the last row. (The probability of n fixed points is approximately e–1/n!.) The Twofish S-box distributions of fixed points over all keys match theory fairly well.

Differences between S-boxes. The metrics discussed so far give us a fair level of confidence that the Twofish method of constructing key-dependent S-boxes does a reasonable job of approximating a random set of S-boxes, when viewed individually. Another possible concern is how different the various Twofish S-boxes are from each other, for a given key length. This issue is of particular interest in dealing with related-key attacks, but it also has an important bearing on the ability of an attacker to model the S-boxes. Despite the fact that the Twofish S-boxes are key dependent, if the entire set (or large subsets) of S-boxes, taken as a black box, are very closely related, the benefit of key dependency is severely weakened. As a pathological example, consider a fixed 8-bit permutation q[x] which has very good DPmax and LPmax values, but which is used as a key-dependent family of S-boxes simply by defining sk(x) = q[x]⊕k. It is true that each sk permutation also has good individual metrics, but the class of s permutations is so closely related that conventional differential and linear cryptanalysis techniques can probably be effectively applied without knowing the key k. The Twofish S-box structure has been carefully designed with this issue in mind.

For example, if an attacker wants to perform a differential attack that is not key dependent, then he can view each S-box as a series of q-boxes and XORs with key bytes. The direct consequence is that he needs to use 3 to 5 differential characteristics of q-boxes per S-box, which makes the probability of any differential characteristic much smaller.

In one class of related-key attacks, an attacker attempts to modify the key bytes in such a way as to minimize the differences in the round subkey values Ai, Bi. Since the Twofish S-box structure is used in computing Ai and Bi, with Me and Mo. as key material, respectively, a measure of the differential characteristics of Ai (or Bi) across keys M will help us understand both how different the S-boxes are from each other and how likely such an attack is to succeed.

To this end, let us first look at how many consecutive values of Ai with a fixed XOR difference can be generated for two different keys. That is, if Ai is the sequence for the key M and A´i is the sequence for the key M´, then we want to examine how many consecutive Ai, A´i satisfy Ai ⊕ A´i = δ for some fixed XOR difference δ. A nearly identical argument holds for the Bi, so we can restrict our attention to the Ai, and furthermore, we can consider only those changes which affect Me, the key material used by the S-boxes when computing Ai.

If we let yi be the output sequence for one S-box used in generating the Ai, and y´i the output sequence for the same S-box when generating the A´i, then we can consider the difference sequence y* = yi ⊕ y´i. For example, in the 128-bit key case, with 16 bits of key material per S-box, there are about 231 distinct pairs of keys for each S-box, so there would be 231 such difference sequences, each of length 20. We can then consider the probability of having a "run" of n consecutive equal values in the sequence. If n can be made to approach 20, then a related-key attack might be able to control the entire sequence of Ai values, and, even worse, our belief in the independence of the key-dependent S-boxes must be called seriously into question. Note that an attacker has exactly 16 bits of freedom for a single S-box in the 128-bit key case, so intuitively it seems unlikely that he should be able to force a given difference sequence that is very long.

Table 7.4 shows the distribution of run lengths of the same XOR difference y*i for consecutive i. For random byte sequences, we would theoretically expect that Pr(XOR run length = n) should be roughly 2–8(n–1), which matches quite nicely with the behavior observed in the table. It can be seen that the probability of injecting a constant difference into more than five consecutive subkey entries is extremely low, which is reassuring.

 |  | – log2(Pr(XOR run length = n))
Key Size | Max Value | n = | 1 | 2 | 3 | 4 | 5
128 bits | 5 | 0.01 | 7.98 | 15.95 | 23.89 | 30.00
192 bits* | 4 | 0.01 | 7.98 | 15.93 | 23.69 |
256 bits* | 5 | 0.01 | 7.98 | 15.93 | 24.02 | 30.29
Table 7.4. Subkey XOR Difference Run Lengths

 |  | – log2(Pr(max # equal XOR differences = n))
Key Size | Max Value | x = | 1 | 2 | 3 | 4 | 4 | 6 | 7
128 bits | 7 | 1.1 | 0.9 | 5.9 | 11.6 | 17.6 | 23.9 | 31.0
192 bits* | 7 | 1.1 | 0.9 | 5.9 | 11.7 | 17.8 | 23.9 | 29.0
256 bits* | 7 | 1.1 | 0.9 | 5.9 | 11.7 | 17.7 | 23.8 | 29.3
Table 7.5. "Mini"-DPmax Subkey Distribution


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Table 7.5 shows the results of measuring this same property in a different way. Instead of requiring a run of n consecutive identical differences, this metric is similar to a "mini"-DPmax, computed over only the 20 input values used in the subkey generation (e.g., 0, 2, 4, . . ., 38). The quantity measured is the maximum number of equal differences out of the 20 values generated, across key pairs. In other words, while it may be difficult to generate a large run, it may be possible to generate equal values in a large (non-consecutive) fraction of the set of differences. However, the results are again very encouraging, showing that it is extremely difficult to force more than five or six differences to be identical. This also shows that it is not possible to influence only a few Ai, as that would require many zero differences.

Comparison to Random S-boxes. For every metric discussed here, a similar distribution using randomly generated 8-bit permutations has been generated for purposes of comparison. For example, DPmax was computed for a set of 216 randomly generated permutations, and the resulting distribution of DPmax values was compared to that of the Twofish key-dependent S-boxes. For each metric, the probability distributions looked virtually identical to those obtained for the Twofish set of key-dependent S-boxes, except for small fluctuations on the tail ends of the distribution, as should be expected. This similarity is comforting, as is the fact that the probability distributions for each metric look quite similar across key sizes. These results help confirm our belief that, from a statistical standpoint, the Twofish S-box sets behave largely like a randomly chosen set of permutations.

7.3 MDS Matrix

The four bytes output from the four S-boxes are multiplied by a 4-by-4 MDS matrix over GF(28). This matrix multiply is the principal diffusion mechanism in Twofish. The MDS property guarantees that the number of changed input bytes plus the number of changed output bytes is at least five. In other words, any change in a single input byte is guaranteed to change all four output bytes; any change in any two input bytes is guaranteed to change at least three output bytes; and so forth. More than 2127 such MDS matrices exist, but the Twofish MDS matrix is also carefully chosen with the property of preserving the number of bytes changed even after the rotation in the round function.

7.3.1 Non-key-dependent Coefficients

The MDS matrix used in Twofish has fixed coefficients. Initially, some thought was given to making the matrix itself key dependent, but such a scheme would require the verification that the key-dependent values in fact formed an MDS matrix, adding a non-trivial computational load to the key selection and key scheduling process. However, it should be noted that there are many acceptable MDS matrices, even with the extended properties discussed below.

7.3.2 Implementation Issues

For software implementation on a modern microprocessor, the MDS matrix multiply is normally implemented using four lookup tables, each consisting of 256 32-bit words, so the particular coefficients used in the matrix do not affect performance. This works because the actual transformation defined by the MDS matrix is purely linear over GF(2). That is, each of the output bits is the XOR of a subset of the input bits and hence we can XOR four table lookups together to compute the entire 32-bit word. However, for smart cards and in hardware, "simple" coefficients, as in Square [DKR97], can make implementations cheaper and faster.

7.3.3 Preserving Diffusion Properties after Rotation

Unlike the MDS matrix used in Square, Twofish does not use the inverse matrix for decryption because of its Feistel structure, nor is there a requirement that the matrix be a circulant matrix. However, because of the rotation applied after the XOR within the round function, it is desirable to select the MDS matrix carefully to preserve the diffusion properties even after the rotation. For both encryption and decryption, a right rotation by one bit occurs after the XOR. This direction of rotation is chosen to preserve the MDS property with respect to the PHT addition a + 2b, since the rotation undoes the shift applied to b as part of the PHT. It is true that the most significant bit of b is still "lost" in this half of the PHT, but the MDS properties for three of the bytes are still fully guaranteed with respect to b, and they are met with probability 254/255 for the fourth byte.

The effect of the rotation on the unshifted PHT additions also needs to be addressed. A single byte input change to the MDS matrix will change all four output bytes, which affect the round function output, but after rotation there is no such guarantee. If a byte value difference of 1 is one output from the matrix multiply, performing a 32-bit rotate on the result will shift in one bit from the next byte in the 32-bit word and shift out the only non-zero bit. The MDS matrix coefficients in Twofish are carefully selected so that, if a byte difference 1 is output from the matrix multiply with a single byte input change, the next highest byte is guaranteed to have its least significant bit changed as well. Thus, if the rotation shifts out the only original flipped bit, it will also shift in a flipped bit from the next byte.

The construction used to preserve this property after rotation is actually very simple. The idea is to choose a small set of non-zero elements of GF(28) with the property that, for each pair x, y in the set, if x*a = 1, then y*a (i.e., y/x) has the least significant bit set. We then use such a set as the coefficients to construct an MDS matrix. Observe that this property is reflexive; i.e., if x = y, then y/x = 1, so the property holds. It is intuitively obvious (and empirically verified) that the probability that two random field elements satisfy this property is roughly one half, so it should not be difficult to find such sets of elements. Also, since over 75 percent of all 4-by-4 matrices over GF(28) are MDS, the hope of finding such a matrix sounds reasonable.

A computer search was executed over all primitive polynomials of degree eight, looking for sets of three "simple" elements x,y,z with the above property. Simple in this context means that x * a for arbitrary a can be easily computed using at most a few shifts and XORs. Several dozen sets of suitable values were found, each of which allowed several MDS matrices with the three values. The primitive polynomial v(x) = x8 + x6 + x5 + x3 + x0 was selected, together with the field elements 01, EF, and 5B (using hexadecimal notation and the field element to byte value correspondence of Chapter 4). The element EF is actually ß–2 + ß–1 + 1, where ß is a root of v(x), and 5B is ß–2 + 1, so multiplication by these elements consists of two LFSR right shifts mod v(x), plus a few byte XORs.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Alternate Constructions. It should also be noted that a very different type of construction could be used to preserve MDS properties on rotation. Use of an 8-by-8 MDS matrix over CF(24) will guarantee eight output nibble changes for every input nibble change. Because the changes now are nibble-based, a 1-bit rotation may shift the only non-zero bit of a nibble out of a byte, but the other nibble remains entirely contained in the byte. In fact, it can easily be seen that this construction preserves the MDS property nicely even for multi-bit rotations. Unfortunately, 8-by-8 MDS matrices over GF(24) are nowhere near as plentiful as 4-by-4 matrices over GF(28), so very little freedom is available to pick simple coefficients. The best construction for such a matrix seems to be an extended RS-(15,7) code over GF(24), which requires considerably more gates in hardware and more tables in smart card firmware than the Twofish matrix. Because of this additional cost, we decided not to use an 8-by-8 MDS matrix over GF(24).

Hamming
Weight 		Number of
Occurrences 		Number with
MSB Set
8 | 7/1020 | 1
9 | 23/1020 | 4
10 | 42/1020 | 15
Table 7.6. Hamming Weight Distribution of MDS Output after 1-byte Input Change

7.3.4 Rotational Uniqueness of Output Vectors

Another constraint imposed on the Twofish MDS matrix is that no row (or column) of the matrix be a rotation of another row (or column) of the matrix. This property guarantees that all single-byte input differences result in unique output differences, even when rotations over eight bits are applied to the output, as is done in generating the round subkeys. This constraint did not seem to limit the pool of candidate matrices significantly. In fact, the Twofish MDS matrix actually exhibits a much stronger property: all 1020 MDS output differences for single-byte input changes are distinct from all others, even under bitwise rotations by each of the rotation values in the range 6..26. The subkey generation routine takes advantage of this property to help thwart related-key differential attacks. For all single-byte input changes, the rotation of the output 32-bit word B by eight bits has an output difference that is guaranteed to be unique from output differences in the unrotated A quantity.

7.3.5 Maximizing the Minimal Hamming Distance

There are many MDS matrices containing only the three elements 01, EF, and 5B. The particular Twofish matrix was also chosen to maximize the minimum binary Hamming weight of the output differences over all single-byte input differences. The Twofish MDS matrix guarantees that any single-byte input change will produce an output Hamming difference of at least eight bits, in addition to the property that all four output bytes will be affected. In fact, as shown in Table 7.6, only seven of the 1020 possible single-byte input differences result in output differences with Hamming weight difference 8; the remaining 1013 differences result in higher Hamming weights. Also, only one of the seven outputs with Hamming weight 8 has its most significant bit set, meaning that at least eight bits will be affected even in the PHT term T0 + 2T1 for 1019 of the 1020 possible single-byte input differences. Input differences in two bytes are guaranteed to affect three output bytes, and they can result in an output Hamming difference of only three bits, with probability 0.000018; the probability that the output difference Hamming weight is less than eight bits for 2-byte input differences is only 0.00125, which is less than the corresponding probability (0.0035) for a totally random binomial distribution.

There are many other MDS matrices, using either the same or another set of simple field elements, that can guarantee the same minimum output Hamming difference, and the particular matrix chosen is representative of the class. No higher minimum Hamming weight was found for any matrices with such simple elements.

7.4 PHT

The PHT operation, including the addition of the round subkeys, was chosen to facilitate very fast operation on the Pentium CPU family using the LEA (load effective address) opcodes. The LEA opcodes allow the addition of one register to a shifted (by 1,2,4,8) version of another register, along with a 32-bit constant, all in a single clock cycle, with the result placed in an arbitrary Pentium register. For best performance, a version of the encryption and decryption code can be "compiled" for a given key, with the round subkeys inserted as constant values in LEA opcodes in the instruction stream. This is what was done for the "compiled" keying option explained in section 5.1.1. This approach requires a full instantiation in memory of the code for each key in use, but it provides a speedup for bulk encryption.

7.4.1 Eliminating the PHT

Instead of using four key-dependent S-boxes, a 4-by-4 MDS matrix, and the PHT, we could have used eight key-dependent S-boxes and an 8-by-8 MDS matrix over GF(28), hence eliminating the PHT. Such a construction would be easier to analyze and would have nicer properties, but it is much slower in virtually all implementations and would not be worth it.

7.4.2 Diffusion and the Least Significant Bit

One disadvantage of the PHT is that it does not provide complete diffusion for the special case of the least significant bit of T0 + 2T1. In particular, the least significant bit of T0 + 2T1 depends only on the least significant bit of T0, and thus depends only on half of the input bits to the Feistel function. This means that F does not exhibit complete diffusion: there is one output bit which is not affected by all input bits. In fact, we use this property in our differential attack in Section 9.2.

We could have replaced the PHT with a variant that exhibits better diffusion; however, this would have incurred a significant performance penalty. Is it better to eliminate this weakness, if the price is that we must reduce the number of rounds to retain the same performance?

We believe that the PHT yields an excellent bang-for-the-buck ratio. In practice, the PHT's incomplete diffusion to the least significant bit of T0 + 2T1 should not be a problem, because full diffusion will be easily achieved in the next two rounds; also, the 1-bit rotation prevents the least significant bits from lining up across multiple rounds, which seems to prevent attacks based on analyzing least significant bits.

7.5 Key Addition

As noted in the previous section, the round subkeys axe combined with the PHT output via addition to enable optimal performance on the Pentium CPU family. From a cryptographic standpoint, an XOR operation could have been used, but it would reduce the best Pentium software performance for bulk encryption. It should be noted that using addition instead of XOR does impose a minor gate count and speed penalty in hardware, but this additional overhead was considered to be well worth the extra performance in software. On a smart card, using addition instead of XOR has virtually no impact on code size or speed.

7.6 Feistel Combining Operation

Twofish uses XOR to combine the output of F with the target block. This is done primarily for simplicity; XOR is the most efficient operation in both hardware and software. We chose not to use addition (used in MD4 [Riv91], MD5 [Riv92], RIPE-MD [RIPE92], HAVAL [ZPS93], RIPEMD-160 [DBP96], and SHA [NIST93]), or a more complicated combining function like Latin squares (used in DESV [CDN95]). We also did not implement dynamic swapping [KKT94] or other additional complexities.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

7.7 Use of Different Groups

By design, the general ordering of operations in Twofish alternates as follows: 8-by-8 S-box, MDS matrix, PHT with subkey addition, and XOR. The underlying algebraic operations thus alternate between non-linear table lookup, a GF(2)-linear combination of the bits by the MDS matrix, integer addition mod 232, and GF(2) addition (XOR). Within the S-boxes, several levels of alternating XOR and 8-by-8 permutations are applied. The goal of this ordering is to help destroy any hope of using a single algebraic structure as the basis of an attack. No two consecutive operations use the same structure, except for the PHT and the key addition that are designed to be merged for faster implementations.

7.8 Diffusion in the Round Function

There are two major mechanisms for providing diffusion in the round function. The first is the MDS matrix multiply, which ensures that each output byte depends on all input bytes. The two outputs of the g functions (T0 and T1) are then combined using a PHT so that both of them will affect both 32-bit Feistel XOR quantities. The half of the PHT involving the quantity T0 + 2T1 will lose the most significant bit of T1 due to the multiply by 2. This bit could be regained using some extra operations, but the software performance would be significantly decreased, with very little apparent cryptographic benefit. In general, the most significant byte of this PHT output will still have a non-zero output difference with probability 254/255 over all 1-byte input differences.

7.8.1 Changes Induced by F

We cannot guarantee that a single byte input change to the F function will change seven or eight of the output bytes of F. The reason is that the carries in the addition of the PHT can remove certain byte differences. For example, an addition with a constant might turn a difference of 0000018016 into 0000008016. The chances of this happening depend on the distance between the two bits that influence each other. A large reduction in the number of changed bytes is very unlikely.

7.9 One-bit Rotation

Within each round, both of the 32-bit words that are XORed with the round function results are also rotated by a single bit. One word is rotated before the XOR, and one after the XOR. This structure provides symmetry for decryption in the sense that the same software pipeline structure can be applied in either direction. By rotating a single bit per round, each 32-bit quantity is used as an input to the round function twice in each of the eight possible bit alignments within the byte.

7.9.1 Reason for Rotations

These rotations in the Twofish round functions were included specifically to help break up the byte-aligned nature of the S-box and MDS matrix operations, which we feared might otherwise permit attacks using statistics based on strict byte alignment. For example, Square uses an 8-by-8-bit permutation and an MDS matrix in a fashion fairly similar to Twofish, but without any rotations. An early draft of the Square paper proposed a very simple and powerful attack, based solely on such byte statistics, that forced the authors to increase the number of rounds from six to eight. Also, an attack against SAFER is based on the cipher's reliance on a byte structure [Knu95c].

Choosing a rotation by an odd number of bits ensures that each of the four 32-bit words are used as input to the g function in each of the eight possible bit positions within a byte. Rotating by only one bit position helps optimize performance on the Pentium CPU (which, unlike the Pentium Pro, has a one-clock penalty for multi-bit rotations) and on smart card CPUs (which generally do not have multi-bit rotate opcodes for 32-bit words). Limiting rotations to single-bit also helps minimize hardware costs, since the wiring overhead of fixed multi-bit rotations is not negligible.

7.9.2 Downsides to Rotations

There are three downsides to the rotations in Twofish. First, there is a minor performance impact (less than 7 percent on the Pentium) in software due to the extra rotate opcodes. Second, the rotations make the cipher non-symmetric in that the encryption and decryption algorithms are slightly different, thus requiring distinct code for encryption and decryption. It is only the rotations that separate Twofish from having a "reversible" Feistel structure. Third, the rotates make it harder to analyze the cipher for security against differential and linear attacks. In particular, they make the simple technique of merely counting active S-boxes quite a bit more complicated. On the other hand, it is much harder for the attacker to analyze the cipher. For instance, it is much harder to find iterative characteristics, since the bits do not line up. The rotates also make it harder to use the same high-probability characteristic several times, because the bits get rotated out of place. On the whole, the advantages were considered to outweigh the disadvantages.

7.9.3 Converting to a Pure Feistel Structure

It is possible to convert Twofish to a pure Feistel structure by incorporating round-dependent rotation amounts in F, and adding some fixed rotations just before the output whitening. This might be a useful view of the cipher for analysis purposes, but we do not expect any implementation to use such a structure.

The rotations were also very carefully selected to work together with the PHT and the MDS matrix to preserve the MDS difference properties for single byte input differences to g. In particular, for both encryption and decryption, the one-bit right rotation occurs after the Feistel XOR with the PHT output. The MDS matrix was chosen to guarantee that a 32-bit word rotate right by one bit will preserve the fact that all four bytes of the 32-bit word are changed for all single input byte differences. Thus, placing the right rotation after the XOR preserves this property. However, during decryption, the rotate right is done after the XOR with the Feistel quantity involving T0 + 2T1. Note that, in this case, the rotate right puts the 2T1 quantity back on its byte boundary, except that the most significant bit has been lost. Therefore, given a single input byte difference that affects only T1, the least significant three bytes of the Feistel XOR output are guaranteed to change after the rotation, and the most significant byte will change with probability 254/255.

The fact that the rotate left by one bit occurs before the Feistel XORs during encryption guarantees that the same relative ordering (i.e., rotate right after XOR) occurs during decryption, preserving the difference-property for both directions. Also, performing one rotation before and one after the Feistel XOR imposes a symmetry between encryption and decryption that helps guarantee very similar software performance for both operations on the Pentium CPU family, which has only one ALU pipeline capable of performing rotations.

7.10 The Number of Rounds

Sixteen rounds corresponds to eight cycles, which seems to be the norm for block ciphers. DES, IDEA, and Skipjack all have eight cycles. Twofish was defined to have eight cycles (16 rounds) primarily out of pessimism. Although our best non-related-key attack only breaks five rounds of the cipher, we cannot be sure that undiscovered cryptanalysis techniques do not exist that can do better. Hence, we consider 16 rounds to be a good balance between our natural skepticism and our desire to optimize performance. Even so, we took pains to ensure that the Twofish key schedule works with a variable number of rounds. It is easy to define Twofish variants with more or fewer rounds.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 8
Design of the Twofish Key Schedule

The key schedule for Twofish is a really a huge piece by itself. While it reuses many of the components of the cipher, it also required a significant amount of independent engineering. This chapter addresses the issues we came across in designing the key schedule.

To understand the design of the key schedule, it is necessary to consider how key material is used in Twofish:

The Whitening Subkeys. A whitening subkey of 128 bits is XORed into the plaintext block before encryption, and another 128 bits subkey after encryption. Since the rest of the encryption is a permutation, this can be seen as selecting among as many as 2256 different (but closely related) 128-bit to 128-bit permutations for the whole cipher. This key material has the effect of making many cryptanalytic attacks a little more difficult at a very low cost. Note that nearly all the added strength against cryptanalytic attack is added by the XOR of subkeys into the input to the first and last rounds' F functions.
The Round Subkeys. Each round, 64 bits of key material are combined into the output of the F function using addition modulo 232. The F function without the round subkey addition is a permutation on 64-bit values; the round subkey selects among one of 264 closely related permutations in each round. These subkeys must be slightly different per round to prevent a slide attack, as will be discussed below.
The Key-dependent S-boxes. To create the S-boxes, the key is mapped down to a block of data half its size, S, and that block of data is used to specify the S-boxes for the whole cipher. As discussed earlier, the key-dependent S-boxes are derived by alternating fixed S-box lookups with XORs of key material. The key-dependent S-boxes define one of 2L/2 possible permutations for the g function, where L is the number of bits in the cipher key. Unlike the other keys used, different S values typically lead to radically different g functions.

8.1 Round Subkeys

8.1.1 Equivalence of Round Subkeys

In this section, we discuss whether different sequences of subkeys can give equivalent encryption functions. Recall that F´ is the F function without the addition of the round subkeys, which takes two 32-bit words as input and produces two 32-bit words as output. For this analysis we keep the function F´ constant. That is, we only vary the subkeys Ki and not S.

The properties of a Feistel cipher ensure that no pair of 2-round subkey sequences can be equivalent for all inputs. It is natural to ask next whether any pairs of three sequential rounds' subkeys can exist that cause exactly the same encryption.

For a pair of subkey sequences, (k0, k1, k2) and (k*0, k*1, k*2), to be equivalent in their effects, every input block (L0, R0) must encrypt to the same output block (L1, R2) for both sequences of subkeys. Note that k0 ≠ k*0 and k2 ≠ k*2, as we would otherwise have two sequences of 2-round keys that would define the same 2-round encryption function. We have the following equalities:

R1 = R0 ⊕ (k0 + F´(L0))

R*1 = R0 ⊕ (k*0 + F´(L0))

L1 = L0 ⊕ (k1 + F´(R1))

L1 = L0 ⊕ (k*1 + F´(R*1))

R2 = R1 ⊕ (k2 + F´(L1))

R2 = R*1 ⊕ (k*2 + F´(L1))

where ⊕ represents bitwise XORing, and + represents 32-bit componentwise addition. Using the two equations for L1 we get

k1 + F´(R1) = k*1 + F´(R*1)

δ1 = F´(R1) - F´(R*1)

where δ1 = k*1 - k1 is fixed. Let T = F´ (L0) + k0 and observe that when L0 goes over all possible values, so does T. We get

δ1 = F´(R0 ⊕ T) - F´(R0 ⊕ (T + δ0)) (8.1)

where δ0 = k*0 - k0. Note that δ1 and δ0 depend only on the round keys, and that the equation must hold for all values of R0 and T. Set T = 0 and look at the cases R0 = 0 and R0 = δ0. We get

F´(0) - F´(δ0) = δ1 = F´(δ0) - F´(0) = - δ1

The subtraction here is modulo 232 for each of the two 32-bit words. That leaves us with the following possible values for δ1:

δ1 ∈ {(0,0), (0,231), (231,0), (231,231)}

These are the possible difference values at the output of F´ in equation 8.1. We can easily convert them to difference values at the input of the PHT of F´. Each of the possible values for δ1 corresponds to exactly one possible value for (δT0, δT1):

(δT0, δT1) ∈ {(0,0), (0,231), (231,0), (231,231)}

We can write down the analogue to equation 8.1 for g:

δT0 = g(R´ ⊕ T´) - g(R´ ⊕ (T´ + δ´0))

for all R´ and T´ and where δ´0 is the appropriate half of δ0. Observe that for the specific values of δT0 that are possible, subtraction and XOR are the same. For T´ = 0 this translates in a simple differential equation

δT0 = g(R´) ⊕ g(R´ ⊕ δ´0)

for all R´. We know that g has only one perfect differential: 0 → 0, so we conclude that δ´0 = 0. Similarly, we can conclude that the other half of δ0 must also be zero, and thus δ0 = 0. This is a contradiction, as k0 ≠ k*0.

We conclude that there are no two sets of 3-round subkey sequences that result in the same encryption function.

A Conjecture about Equivalent Subkeys in Twofish. We believe that, for up to 16 rounds of Twofish, there are no pairs of round subkeys that (with the same F´ function) result in the same encryption function. There simply do not appear to be enough degrees of freedom in choosing the different subkeys to make pairs of equivalent subkey sequences in as few as 16 rounds. However, we have been unable to prove this.

Note that this conjecture does not hold for an arbitrary number of rounds. Any fixed round function is an element of the permutation group on 128-bit values S2128. The order of this group is (2128)! ≈ 22134, and the order of any group element must be a divisor of the order of the group. Thus, any round function when iterated (2128)! times, results in the identity mapping.1 For this number of rounds there are thus many equivalent round subkeys.

1In fact, this property holds for any bijective function, including the complete Twofish encryption and the AES block cipher (irrespective of which candidate is chosen).

8.1.2 Equivalent keys

Can two keys generate the same sequence of round subkeys? We have performed exhaustive tests to show this is not the case. To generate the same round subkeys, the two keys will have to generate the same set of Ai and Bi.

We observe that Ai = h(2ρi, Me), where the h function consists of applying the MDS matrix multiplication to the four values (y0, y1, y2, y3) obtained by running the value 2i through 1 + N/64 levels of q0 and q0, XORing with bytes from Me. Since the MDS matrix is non-singular, it is easily seen that to generate the same sequence of Ai values, the keys must generate the same sequence of values for y0, y1, y2 and y3.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

The sequence of y0 values for 256-bit keys is

q1[q0[q0[q1[q1[i] ⊕ k3] ⊕ k2] ⊕ k1] ⊕ k0]

where the kj are different bytes from Me. Smaller keys result in similar equations with fewer key bytes, so the analysis is almost identical. The question at hand is whether two distinct sets of four key bytes can result in an identical sequence of 20 y0 values. In fact, we exclude the input and output whitening values to concentrate solely on the round subkeys, so only 16 values are included in our test (one per round). Clearly, there are 232 such sequences. However, this number can be reduced for the purposes of our search by noting that, since the "outer" mapping q1 is bijective, it can be removed from the equation, leaving us with

q0[q0[q1[q1[i] ⊕ k3] ⊕ k2] ⊕ k1] ⊕ k0

Now the outer XOR term k0 can be effectively removed by creating a related sequence with only 15 values, where the y0 values for i = 5, ..., 19 are XORed with the i = 4 value. The k0 terms thus cancel out, so there are only 224 equivalence classes of sequences, speeding up the search dramatically. If all these sequences (each with 15*8 = 120 bits) are unique, then the y0 sequence is also guaranteed to be unique.

For 192-bit keys, the y0 sequence is

q1[q0[q0[q1[i] ⊕ k2] ⊕ k1] ⊕ k0] ⊕k0]

and for 128-bit keys, it is

q1[q0[q0[i] ⊕ k1] ⊕ k0]

In a manner identical to that discussed for 256-bit keys, the outer q1 permutation can be removed for these smaller keys, and the outer k0 term can similarly be removed by creating the related XOR sequence. The number of the remaining sequences is 216 and 28, respectively. All these sequences can then be compared across key sizes (with a total of NS = 224 + 216 + 28 sequences) to verify uniqueness.

A computer search has been performed over y0, y1, y2 and y3, for all sequences across all key lengths. It turns out that only 64 bits of each sequence were sufficient to distinguish the sequences. This fact is actually quite encouraging, since there are 120 bits in each sequence, giving some heuristic comfort that the sequences are in fact quite different. Each search requires slightly more than 128 MB of memory (i.e., 8Ns bytes) to hold all the sequences, which are then sorted and compared to adjacent values to guarantee uniqueness. The test was run on a Pentium computer with only 32 MB of memory, so the search was actually performed in multiple passes, running through all the Ns values several times, on each pass selecting only those values falling into certain bins. For example, using approximately 8 MB of memory, there are 16 passes, with the mth pass discarding all sequence values for which a fixed 4-bit field of the sequence does not equal to m.

Similar arguments apply to the Bi sequence, and the same empirical verification has been performed for both sequences. Note that, since the operations used to compute K2i and K2i+1 from Ai and Bi are reversible, proving that the Ai and Bi sequences are distinct is more than sufficient to prove that the Kj sequences are distinct.

The definitive result from these tests is that there are no two distinct keys of any size for which the same sequence of Ai and Bi values is obtained. Thus, the round subkey sequence Kj is unique across keys.

8.2 Controlling Changes in Round Subkeys

In a related-key attack, one is concerned about introducing controlled differences into key material that will produce predictable effects that depend on the pair of keys. The basic intuition is that if one can observe the difference in behavior produced by two keys known to have a particular mathematical difference, then one might be able to discern information about the true value of the two keys. Normally one does not limit the attack to pairs of keys and instead may consider subsets of keys instead. Either way, this is a natural attack in many environments where multiple encryption keys are used which are not truly independent of each other.

One very natural notion of mathematical difference is the XOR difference of two values. In this section we consider the problem of trying to produce two sequences of round subkeys {Ki} and {K´i} that have a desired difference (either additive or XOR). Recall that the Ki (and K´i) are computed as a function of the sequences A and B (respectively A´ and B´). Therefore we consider the problem of trying to produce two sequences A = {Ai} and A´ = {A´i} such that individual elements have a desired difference δi = Ai ⊕ A´i (a similar discussion holds for B = {Bi}). A and B are generated using four 8-bit, key-dependent S-boxes in parallel followed by multiplication by a 4-by-4 MDS matrix over GF(28), where each S-box output is treated as one component of a four-dimensional vector over GF(28). Due to the linearity of the MDS matrix multiply, it is very natural to consider attacks that introduce XOR differences between A and A´, since it is very easy to pass XOR differences through the MDS matrix.

8.2.1 XOR Difference Sequences in A and B

Let A and A´ be sequences for two keys M and M´ respectively. If we have a specific difference sequence, δ = {δi}, we want to see in A (i.e., δi Ai ⊕ A´i), we are faced with an interesting problem: since the MDS matrix multiply is XOR-linear, each desired output XOR from the matrix multiply uniquely determines the input XOR. This means that:

1. Any input difference δIi to the MDS matrix corresponds to an output difference for each of the four S-boxes. Hence a difference sequence δ induces four XOR-difference sequences δj for the four S-boxes. For any difference sequence δj = {δji}, each value is an 8-bit value instead of a 32-bit value. As a result, we will sometimes refer to these sequences as difference byte sequences.
2. The zero sequence δ can occur only when each difference byte sequence δj is the zero sequence.
3. Only 1020 possible output differences (out of the 232) in δi can occur with a single "active" (altered) S-box (i.e., only one δj is a non-zero sequence). Most differences require all four S-boxes to be active and hence all δj to be non-zero sequences.

The above analysis is of course also valid for the B sequence.

Given that any desired difference sequence δ corresponds to four difference byte sequences δj, we need to examine the properties of the S-boxes to determine the difficulty of generating specific δj.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

8.2.2 Byte Sequences with Given Difference

In generating A and A´ with a specific difference δ, one must generate four pairs of byte sequences (σj, σ´j), each with a specific difference δj, where σj = {σji} and σ´j = {σ´ji}. The inputs to the four S-boxes used to produce these values are {0, 2, 4, ..., 38} (whereas they axe 1, 3, 5, ..., 39} for B). Therefore our byte sequences are given by σji = sj [2i] and σ´ji = s´j[2i]. Hence any questions about difference byte sequences reduce to questions about the sequences

{sj[0], sj[2],...,sj[38]}, {s´j[0], s´j[2],...,s´j[38]}

For a given δj we will estimate the chance that there exists a pair (σj, σ´j) with that difference sequence.

We can model each byte sequence generated by a key-dependent S-box as a randomly selected non-repeating byte sequence of length 20. This allows us to make many useful predictions about the likelihood of finding keys or pairs of keys with various interesting properties. Because we will be analyzing the key schedule using this assumption in the remainder of this section, we should discuss how reasonable it is to treat this byte sequence as randomly generated. As discussed in Section 7.2.3, we have not found any statistical deviations between our key-dependent S-boxes and the random model in any of our extensive statistical tests.

Not all byte sequences are possible; due to the bijectivity of the S-boxes we know that all the bytes in the sequence are distinct. There are 256!/236! of those sequences, which is close to 2159.

Each key-dependent S-box uses N/8 bits of key material; hence, if the S-boxes were truly random we would expect two 20-byte sequences to create the desired difference with probability roughly 2N/4-1. 2-160 (There are 2N/4-1 pairs of byte sequences, and each has a 2-160 chance of being a match.) For example, for N = 256 we expect with probability 2-97 that there exists a pair of our S-boxes that produces the desired difference sequence. Note that this probability is for any specific difference sequence. There are many such possible difference sequences, and of course some of those actually occur. Attacks requiring a specific difference sequence seem more likely than attacks requiring any of a class of 297 usable difference sequences.

Most difference sequences in A will require that all four δj are non-zero. For a randomly chosen difference δ in the A sequence we thus expect that there are two keys that generate exactly this difference with probability about 2-388.

8.2.3 Identical Byte Sequences

As discussed in section 8.1.2, we have empirically verified that there are no equivalent S-box keys that generate the same sequence of 20 bytes. That holds for our specific choice of q0 and q1. This section analyzes the probability of two identical sequences occurring for arbitrary q-boxes.

The estimate of section 8.2.2 holds for the special case of δj being the all-zero sequence. If we are a little more careful, then we can actually improve the estimate for this case. While this result shows that the S-boxes are not uniformly distributed throughout the space of all possible 8-bit S-boxes, it shows that the S-boxes display better than average differences in this particular case.

To improve the estimate, we peel off one layer of our q construction and assume the rest of the construction is random. Without loss of generality we look at2

2:For ease of discussion, we number the key bytes as k0,..., k3 going into one S-box. The actual byte ordering for s0 and a 256-bit key's subkey-generating S-box is k0, k8, k16, k24. The numbering of the key bytes has no effect on the security arguments in this section.

s1(x) = q0[q0[q1[q1[q0[x] ⊕ k0] ⊕ k1] ⊕ k2] ⊕ k3]

Identical sequences are possible only if the inputs to the last q0 fixed permutation are identical for both sequences. That means that the task of finding a pair of identical sequences comes down to a simpler task: finding a pair of (k0, k1, k2) byte values that leads to a pair of sequences before the XOR with k3 that have a fixed XOR difference. Then, k3 can be changed to include the XOR difference, and identical sequences of inputs will go into the last q0 S-box.

Let

t[i]:= q0[q1[q1[q0[i] ⊕ k0] ⊕ k1] ⊕ k2]

The goal is to find a pair of t[i] sequences such that

t[i] ⊕ t*[i] = constant

Let us assume that t generates a random sequence (with the restriction that all byte values must be distinct). The chances of any pair of t, t* generating such a constant difference is about 2-151. (The domain in which we are searching for collisions is the set of all 19-byte sequences with non-repeated values. There are 256!/237! of these sequences, which is close to 2151.) This brings the chance of finding a pair with such a constant difference down to 247. 2-151 = 2-104.

8.2.4 The A and B Sequences

From the properties of the byte sequences, we can discuss the properties of the A and B sequences generated by each key M.

Ai = MDS(s0(2i, Me), s1(2i, Me), s2(2i, Me), s3(2i, Me))

Since the MDS matrix multiply is invertible, and since i is different for each round's subkey words generated, we can see that no A or B value can repeat itself.

Similarly, we can see from the construction of h that each key byte affects exactly one S-box used to generate A or B. Changing a single key byte always alters every one of the 20 bytes of output from that S-box; the MDS matrix ensures that every byte of every word in the 20-word A or B sequence to which this key byte contributes is altered.

Consider a single byte of output from one of the S-boxes. If we cycle any one of the key bytes that contributes to that S-box through all 256 possible values, the output of the S-box will also cycle through all 256 possible values. If we take four key bytes that contribute to four different S-boxes, and we cycle those four bytes through all possible values, then the result of h will also cycle through all possible values. This proves that A and B are uniformly distributed for all key lengths, assuming the key M is uniformly distributed.

8.2.5 The Sequence (K2i, K2i+1)

As Ai and Bi are uniformly distributed (over all keys), so are all the Ki. As all pairs (Ai, Bi are distinct, all the pairs (K2i, K2i+1) are distinct, although it might happen that Ki = Kj for any pair of i and j.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

8.2.6 Difference Sequences in the Subkeys

Difference sequences in A and B translate into difference sequences in (K2i, K2i+1). However, while it is natural to consider A and B difference sequences in terms of XOR differences, subkeys can reasonably be considered either as XOR differences or as differences modulo 232. Thus, we may discuss difference sequences:

D[i,M, M*] = Ki,M - Ki,M*

X[i,M, M*] = Ki,M ⊕ Ki,M*

where the difference is computed between the key value M and M*.

XOR Differences in the Subkeys. Each round, the subkeys are added to the results of the PHT of two g functions, and the results of those additions are XORed into half of the cipher block. An XOR difference in the subkeys has a fairly high probability of passing through the addition operation and ending up in the cipher block. (The probability of this is determined by the Hamming weight of the XOR difference, not counting the highest-order bit.) However, to get into the subkeys, an XOR difference must first pass through the first addition.

Consider

x + y = z

(x ⊕ δ0)+ y = z ⊕ δ1

Let k be the number of bits set in δ0 , not counting the highest-order bit. Then, the highest probability value for δ1 is δ0, and the probability that this will hold is 2-k. This is true because addition and XOR are very closely related operations. The only difference between the two is the carry between bit positions. If flipping a given bit changes the carry into the next bit position, this alters the output XOR difference. This happens with probability 1/2 per bit. The situation is more complex for multiple adjacent bits, but the general rule still holds: for every bit in the XOR difference not in the high-order bit position, the probability that the difference will pass through correctly is cut in half.

For the subkey generation, consider an XOR difference, δ0, in A. This affects two subkey words:

K2i = Ai + Bi

K2i+1 = ROL(Ai + 2Bi, 9)

where the additions are modulo 232. If we assume these XOR differences propagate independently in the two subkeys (which appears to be the case), we see that this leads to an XOR difference of δ0 in the even subkey word with probability 2-k, and the XOR difference ROL(δ0, 9) in the odd subkey with the same probability. The most probable XOR difference in the round's subkey block thus occurs with probability 2-2k. A desired XOR difference sequence for all 20 pairs of subkey words is thus quite difficult to get to work when k ≥ 3, assuming the desired XOR difference sequence can be created in the A sequence at all.

When the XOR difference is in B, the result is slightly more complicated; the most probable XOR difference in a round's pair of subkey words may be either 2-(2k-1) or 2-2k, depending on whether or not the XOR difference in B covers the next-to-highest-order bit.

Additive Differences in the Subkeys. An XOR difference in A or B is easy to analyze in terms of additive differences modulo 232: a XOR difference with k active bits has 2k equally likely additive differences. Note that if we have an additive difference in A, we get it in both subkey words, just rotated left nine bits in the odd subkey word. Thus, k-bit XOR differences lead to a given additive difference in a pair of subkey words with probability 2-2k. (The rotation does not really complicate things much for the attacker, who knows where the changed bits are.)

8.3 The Round Function

The round function F takes a 64-bit input and produces a 64-bit output; and F is characterized by the four S-boxes si and the round keys K2r+8, K2r+9. The S-boxes depend on N /2 bits of key material S. We show that all the F functions generated in this way are distinct. Our test for distinctness of the round functions concentrates on only a single bit of the output, which will be sufficient to prove uniqueness.

In particular, consider the value

F1 = (T0 + 2T1 + K2r+9) mod 232

where T0 =g(R0 and T1 =g(ROL(R1, 8)). Because the g function involves an MDS matrix multiply, which uses the XOR operation, it is difficult to analyze the full F1 value due to the interaction between operations of different algebraic groups (i.e., XOR and addition). However, if we examine only the least significant bit (1sb) of F1, we can ignore carries, so the operation can be analyzed entirely Using XOR. Note that this bit does not depend on R1, due to the multiplication by 2 in the PHT. Further, the MDS matrix element that maps the S-box output is a simple linear transformation (i.e., multiplication by a GF(28) field element). Thus, for each S-box, the effect of the final fixed permutation (q0 or q1) of the S-box and the MDS multiply on the lsb of F1 is a simple fixed mapping from eight bits to one bit.

Now, consider two keys with distinct values for S. Since the S values are different, at least one of the four S-boxes must have different key material under the two different keys. To prove uniqueness, we simply fix the inputs to the remaining three S-boxes, so that their XOR "contribution" to the lsb of F1 is fixed. Up to a fixed XOR constant (based on K2r+9 and the other three S-boxes), we are then left with a simple function involving a single S-box that maps eight bits to one bit, with N/8 bits of key material used in the S-box. We can remove dependence on the fixed constant by constructing sequences consisting of the XOR of two bits in the S-box/MDS lsb output sequence, similar to the method discussed in section 8.1.2. If this sequence of bits is unique across the all 2N/8 possible key material values for the S-box, then the F function is unique with respect to that S-box. If all four S-boxes are unique in this way, then the F function is unique for each distinct value of S.

To remove the dependence on the constant bit, we performed our search on a modified sequence in which each bit was the XOR of two lsbs of the S-box/MDS output values. Since the si mapping has 256 inputs, this limits the sequence down to only 255 values, which is still easily sufficient to distinguish between F functions. Let us first consider the 256-bit key case, which obviously involves the longest search. Note, for example,

s0(x) = q1[q0[q0[q1[q1[x] ⊕ k3] ⊕ k2] ⊕ k1] ⊕ k0]

where the kj bytes are a subset of the bytes in S. There are 232 such functions, but, unfortunately, since we are dealing only with the lsb, there is no obvious method to cut down the search time by a factor of 256, as we were able to do in section 8.1.2. Similar sequences were produced for the smaller key sizes, with all sequences for each S-box combined across key sizes in the test, for a total of NF = 232 + 224 + 216 sequences per S-box.

To avoid a "birthday surprise" collision with NF sequences, we require more than 64 bits of each sequence. Even with only 64 bits, however, a total of over 32 GB of memory would be required, a size far beyond the budget of this experiment. Thus, this test was also performed in multiple passes, with each pass generating all NF sequences but discarding those values not falling into the selected bin for the particular pass, as before. It was found empirically that the performance time was roughly proportional to the number of passes; in other words, the sort/compare time for a given pass was considerably shorter than the O(232) time required to generate the "filtered" list. For example, on a 200 MHz Pentium computer with slightly over 256MB of available RAM, 128 passes are required, which empirically were completed for a single S-box in slightly less than three days. Each sequence value in the list consisted of 64 bits, with additional sequence bits used to filter out values not to be used in a given pass. For a 128-pass test, this means that seven extra bits were used to help avoid a birthday surprise collision. The filtering code (in C) was carefully optimized so that most of the values to be filtered on each pass were quickly rejected. Since 127 of every 128 values were filtered out on each pass, this simple optimization sped up performance considerably, without requiring particularly fast generation of the 64-bit sequence values to be included.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

The test was run on several Pentium computers with various amounts of RAM over a period of about ten days. The results showed that, for each of the four S-boxes, all NF mappings have a unique lsb sequence. Thus, each F function is unique for each distinct value of S.

8.4 Properties of the Key Schedule and Cipher

One NIST requirement is that the AES candidates have no weak keys. Here we argue that Twofish has none.

8.4.1 Equivalent Keys

A pair of equivalent keys, M, M*, is a pair of keys that encrypts all plaintexts into the same ciphertexts. We are almost certain that there are no equivalent keys in Twofish. As shown in section 8.1.2, there is no pair of keys, M, M*, that gives the same subkey sequence {Ki}. Section 8.3 showed that for two keys to generate the same round function in any one round, they must generate the same S.

It is conceivable that different sequences of subkeys and different S-boxes in g could end up producing the same encryption function, thus giving equivalent keys. This appears to be extremely unlikely, but we cannot prove that such equivalent keys do not exist.

A Conjecture about Equivalent Keys in Twofish. We conjecture that there are no pairs of Twofish keys that lead to identical encryptions for all inputs.

8.4.2 Self-Inverse Keys

Self-inverse keys are keys for which encrypting a block of data twice with the same key gives back the original data. We do not believe that self-inverse keys exist for Twofish. Keys cannot generate a self-inverse sequence of subkeys, because the same round subkey value can never appear more than once in the cipher. Again, it is conceivable that some keys axe self-inverse despite using different subkeys at different points, but again this is extremely unlikely. Self-inverse DES keys are sometimes referred to as "weak keys."3

3The term "weak key" is used in conflicting ways in the literature. DES' weak keys are fundamentally different from the weak keys of IDEA or Blowfish.

8.4.3 Pairs of Inverse Keys

A pair of inverse keys is a pair of keys M0, M1 such that EM0(EM 1(X)) = X, for all X. We cannot see any way for this to occur. Encryption and decryption are slightly different processes, because of the one-bit rotations, so even if M0, M0, generate a pair of subkey sequences that are reversed, we still do not get pairs of inverse keys.

Consider a Twofish variant without those one-bit rotations. Can we find M0, M1, that lead to reversed subkey sequences? That is, can we find a pair of keys whose subkeys appear in the opposite position—the pre-XOR subkeys for M0 are the post-XOR subkeys for M1, the first round subkeys for M0 are the last round subkeys for M1, etc. To get reversed subkeys, we need to get reversed byte sequences from each of the eight byte-wide key-scheduling S-boxes. Finding a pair of byte sequences that are one another's reverse sequences is the same difficulty as finding a collision in the whole byte sequence, and thus the same difficulty as finding a difference sequence in all 20 output byte positions. By the same arguments as we used above regarding difference sequences, then, this kind of reversed subkey sequence almost certainly doesn't exist for any pair of keys M0, M1. For these reasons, we do not believe that there are pairs of inverse keys for Twofish. Note that pairs of inverse keys in DES are sometimes referred to as "semi-weak keys."

8.4.4 Simple Relations

A key complementation property exists when

EM (P) = C ⇒ EM´ (P´) = C´

where P´, C´, and K´ are the bitwise complement of P, C, and K, respectively. No such property has been observed for Twofish.

More generally, a simple relation [Knu94b] is defined as

EM (P) = C ⇒ Ef(M)´ (g(P,M))= h(C, M)

where f, g, and h are simple functions. We have found no simple relations for Twofish, and strongly doubt that they exist.

8.5 Key-dependent Characteristics and Weak Keys

The concept of a key-dependent characteristic seems to have been introduced in [BB93] in their cryptanalysis of Lucifer, and also appears in [DGV94a] in an analysis of IDEA.4 The idea is that certain iterative properties of the block cipher useful to an attacker become more effective against the cipher for a specific subset of keys.

4See [Haw98] for further cryptanalysis of IDEA weak keys.

A differential attack on Twofish may consider XOR-based differences, additive differences, or both. If an attacker sends XOR differences through the PHT and subkey addition steps, his differential characteristic probabilities will be dependent on the subkey values involved. In general, low-weight sub-keys will give an attacker some advantage, but this advantage is relatively small. (Zero bits in the subkeys improve the probabilities of cleanly getting XOR-based differential characteristics through the subkey addition.) Since there appears to be no special way to choose the key to make the subkey sequence especially low weight, we do not believe this kind of key-dependent differential characteristic will have any relevance in attacking Twofish.

A much more interesting issue in terms of key-dependent characteristics is whether the key-dependent S-boxes are ever generated with especially high-probability differential or high-bias linear characteristics. The statistical analysis presented earlier shows that the best linear and differential characteristics over all possible keys are still quite unlikely.

Note that the structure of both differential and linear attacks in Twofish is such that such attacks appear to generally require good characteristics through at least three of the four key-dependent S-boxes (if not all four), so a single high-probability differential or linear characteristic for one S-box will not create a weakness in the cipher as a whole.

8.6 Reed-Solomon Code

The RS structure helps defend against many possible related-key attacks by diffusing the key material in a direction "orthogonal" to the flow used in computing the 8-by-8-bit S-boxes of Twofish. For example, a single byte change in the key is guaranteed to affect all four key-dependent S-boxes in g. Since RS codes are MDS [MS77], the minimum number of different bytes between distinct 12-byte vectors generated by the RS code is guaranteed to be at least five. Notice that any attempt in a related-key attack to affect only a single byte in the computation of A or B is guaranteed to affect all four bytes in the computation of T0 and Ti. The S-box keys are used in reverse order from the associated key bytes so that related-key material is used in a different order in the round function than in the subkey generation.

The reversible RS code used in Twofish was chosen via computer search to minimize implementation cost. The code generator polynomial is

where α is a root of the primitive polynomial w(x) used to define the field.

Because all of these coefficients are "simple," this RS computation is easily performed with no tables, using only a few shifts and XOR operations; this is particularly attractive for smart cards and hardware implementations. This computation is only performed once per key schedule setup per 64 bits of key, so the concern in choosing an RS code with such simple coefficients was not the performance overhead, but saving ROM space or gates. Precomputing the RS remainders requires only eight bytes of RAM on a smart card for 128-bit keys.

To take advantage of the simple coefficients of the polynomial, the RS matrix multiply should not be implemented as a direct matrix multiply but as a loop over the input bytes, where each iteration uses the coefficients of the polynomial once.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 9
Cryptanalysis of Twofish

We have spent over one thousand man-hours cryptanalyzing Twofish. A summary of our successful attacks is as follows:

• On Twofish with fixed S-boxes, no 1-bit rotations, and no whitening, we have a meet-in-the-middle attack on eleven rounds requiring 2225 memory, 256 known plaintexts, and 2232 work, and a differential attack breaking nine rounds, requiring 241 memory, 241 chosen plaintexts, and 2254 work.
• On standard Twofish, we have a 4-round meet-in-the-middle attack requiring 256 known plaintexts, but 2225 memory and 2232 work. We also have a differential attack which breaks five rounds of full Twofish with 2232 work and 241 chosen-plaintext queries.
• We have a chosen-key attack. This attack involves choosing 160 bits of a pair of keys, K, K*, with the remaining bits to be found. The attack requires 234 work, 232 chosen-plaintext queries, and 212 adaptive chosen-plaintext queries, in order to break 10 rounds without the whitening.
• We have a related-key attack against 10-round Twofish without whitening. This attack requires 2155 related-key queries, 2187 work, and for each of the 2155 keys it requires 232 chosen plaintexts and 212 adaptive chosen plaintexts.

The fact that Twofish seems to resist related-key attacks well is arguably the most interesting result, because related-key attacks give the attacker the most control over the cipher's inputs. Conventional cryptanalysis allows an attacker to control both the plaintext and ciphertext inputs into the cipher. Related-key cryptanalysis gives the attacker an additional way into a cipher: the key schedule. A cipher that is resistant to attacks with related keys is necessarily resistant to simpler techniques that only involve the plaintext and ciphertext.1

1We have discussed the relevance of related-key attacks to practical implementations of a block cipher in [KSW96, KSW97]. Most importantly, related-key attacks affect a cipher's ability to be used as a one-way hash function.

Based on our analysis, we conjecture that there exists no more efficient attack on Twofish than brute force. That is, we conjecture that the most efficient attack against Twofish with a 128-bit key has a complexity of 2128; the most efficient attack against Twofish with a 192-bit key has a complexity of 2192; and the most efficient attack against Twofish with a 256-bit key has a complexity of 2256.

Cipher variant Rounds Work Memory
256-bit key, Fixed S, no whitening, no rotations | 11 | 2232 | 2225
256-bit key, Fixed S, no whitening | 10 | 2232 | 2225
256-bit key, Fixed S | 6 | 2232 | 2225
256-bit key, Full Twofish | 4 | 2232 | 2225

Table 9.1. Attack Results for Our Meet-in-the-Middle Attack

9.1 A Meet-in-the-Middle Attack on Twofish

9.1.1 Results of the Attack

Table 9.1 summarizes our attack results. One notable feature of this attack is that it demonstrates the impact of adding the pre- and post-whitening and the key-dependent S-boxes. Without these two features, we would have an extremely simple attack on 10 out of the 16 rounds of our cipher—enough to raise questions about its ultimate security. With these two features, however, the attack extends to only four rounds. Resistance to meet-in-the-middle attacks comes from quickly getting every bit of the key involved in every bit of the block being encrypted.

9.1.2 Overview of the Attack

In a meet-in-the-middle attack, we try to derive some intermediate state inside the cipher, using only part of the key material from both the plaintext and the ciphertext. For each possible value that the part of the key that we guessed can take, we derive that intermediate value from the plaintext and also from the ciphertext. We get two huge lists of possible intermediate values. We then sort the two lists, and check them for matching values. A matching value of more bits than we would expect to occur randomly indicates that our guesses were correct. This kind of attack is important to consider because the AES candidates are using large keys, which imply a very high security level. A user expecting 256 bits worth of security should be concerned by attacks that involve guessing 200 bits of key in each direction. Also, it is easy for cipher designers used to thinking in terms of 64-bit blocks and 128-bit keys to design ciphers that do not get all 256 bits of key involved in the encryption process quickly enough.

9.1.3 Attacking Twofish with Fixed S and no Whitening

In this Twofish variant, all key material is in the round subkeys. We acquire about 256 plaintext/ciphertext pairs from this Twofish variant with ten rounds.

We label the four words of the data during the encryption as A, B, C, and D. From the plaintext side, we guess the round subkeys for the first three rounds, for a total of 192 bits. We also guess one subkey (used to determine the value XORed into A) in the fourth round. We thus know the value of the low-order bit of D after the fifth round, because (1) we knew the value of D after the third round, (2) we know the value of A going into the fifth round, and (3) we know that the low-order bit of the value XORed into D is dependent only on A and one bit of the fifth round's subkey. For each of the 2225 possibilities for these key values, we compute this bit for all 256 plaintexts we have. We store these in a list, and sort them.

From the ciphertext side, we guess the round keys for the tenth, ninth, and eighth rounds, and we guess the subkey from the seventh round that affects the value XORed into D. We use this to derive the high-order bit from D going into the seventh round, which was rotated from its position as the low-order bit of D after the fifth round. Again, for all 2224 possible key values guessed, we compute this bit for each of the 256 ciphertexts we have. Again, we sort this list.

We now merge the two sorted lists, and thus search for matches. This ought to take on the order of 2232 work total. Thus, we use 2225 blocks of memory, and about 2232 work to break ten rounds of Twofish with fixed S-boxes and no whitening.

Note that without the one-bit rotations in the cipher, we would have needed to know only the low-order bit of the F-function output in the seventh round. This would actually allow us to attack one more round. This is evidence that the one-bit rotations do, in fact, provide some resistance against real attacks.

9.1.4 Attacking Twofish with Fixed S

With the whitening added, the attack becomes much harder. From the plain-text side, we must guess all four pre-whitening subkeys, allowing us to get our known low-order bit only into the output of the third round. Similarly, from the ciphertext side, we must guess all four post-whitening subkeys and the last round's subkeys, and one subkey word from the next-to-last round. We can thus attack only six rounds in this case.

9.1.5 Attacking Normal Twofish

Attacking the full Twofish is still harder. From the plaintext side, we must guess the 128 bits of S, the round subkey that determines the value that is XORed into C in the first round, and the pre-whitening subkey for C. This allows us to predict the low-order bit of B after the second round XORed with an unknown constant (i.e., we predict that this bit will take on a sequence of values or the negation of that sequence). A similar approach is used from the ciphertext side, allowing only four rounds to be attacked.

Key
Length 		Variant Rounds Chosen
Texts 		Work
128 | No whitening, fixed S | 7 | 241 | 2126
192 | No whitening, fixed S | 8 | 241 | 2190
256 | No whitening, fixed S | 9 | 241 | 2254
128 | No whitening, no one-bit rotates | 6 | 241 | 2126
192 | No whitening, no one-bit rotates | 6 | 241 | 2158
256 | No whitening, no one-bit rotates | 7 | 241 | 2254
256 | Normal Twofish | 5 | 241 | 2232

Table 9.2. Attack Results for Our Differential Attack

9.2 Differential Cryptanalysis

9.2.1 Results of the Attack

Table 9.2 summarizes our attack results against several variants of Twofish.

9.2.2 Overview of the Attack

The attack uses two different differential properties of the F function:

1. If the input difference into the F function is (0, X), where X represents any non-zero difference at all, then the output difference of the F function can be expressed additively as (+Y, +2Y), where Y is some other unknown difference. Unfortunately, these additive differences do not always translate cleanly into XOR differences. However, the low-order bit of the second word of F's output is always unchanged, since 2Y always has its low-order bit clear. This happens with probability 1.
2. If the same XOR difference comes out of the two g functions during the F function computation, and this difference has k bits not in the high-order position, then with probability 2-k, the F function will generate an output difference of (0, X), where X is some unknown difference.

Our attack works by first trying to force the second event described above to occur in the second round, in a batch of several pairs of plaintexts. If it occurs, then the first event described above occurs in the next round with probability one, resulting in a single bit whose difference is known in the output of the fourth round. This allows an attack on reduced-round Twofish.

The attack requires that we get a specific, predictable difference sequence. We choose

r | ΔRr,0 | ΔRr,1 | ΔRr,2 | ΔRr,3
0 | 0 | 0 | a′ | b′
1 | a | b | 0 | 0
2 | 0 | ? | a | b
3 | ? | (?,1) | 0 | ?
4 | ? | ? | ? | (?,1)
5 | ? | ? | ? | ?

where the table gives the difference patterns for the round values for each of the 5 rounds, and where a′ = ROL(a, 1), b′ = ROR(b, 1), and (?, 1) represents a 32-bit difference of which one bit is known. To recognize right pairs, we shall (roughly speaking) guess enough of the key to decrypt up one round, checking whether the low-order bit of the difference is as expected.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

The only critical event for this attack occurs at r = 1, where we need the characteristic (a, b) → (0, X) to happen with high probability. Based on the properties of the MDS matrix, we know that there are three single-byte output XORs for s0 that lead to an output XOR of g with Hamming weight of only 8. If the same one of those differences goes into the MDS matrix in both g computations in the second round, then, with probability 2-8, we get an offsetting pair of values in the two g outputs; one g output has some value added to it modulo 232, and the other output has the same value subtracted from it. When the outputs go through the PHT, they lead to only one output word from the whole F being changed, which is what makes this attack possible. Therefore, we choose a = (α, 0, 0, 0) to be a single-byte difference in s0, and b = ROR(a, 8) so that the single-byte difference into the second g computation in round 2 lines up with the single-byte difference in a.

In this attack, we will consider N batches of 256 plaintext pairs, of which at least one is expected to consist of all right pairs. We must choose the inputs to the cipher to allow ourselves to later form such batches. We will then guess some part of the key as needed to check the one known bit of difference going into the fifth round. That key guess will be used to test each batch. Batches that are not made up of right pairs will typically be distinguished after a small number of pairs are checked, because the one-bit difference will be wrong with probability 1/2 in each wrong pair checked with the right key, and in each pair checked with the wrong key. The attack thus consists of the following steps:

1. Request the encryptions of plaintext blocks necessary to form at least one batch of 256 plaintext pairs that has a very high probability of being made up of only right pairs.
2. Guess the smallest amount of key material that will let us see the one bit of difference we know in right pairs.
3. For each such guess, check each batch until one batch turns out to be made of only right pairs.

If there are N batches to be checked, and each takes on average about two trial decryptions' worth of work, then we end up with 2N work we must do for each guess of key material necessary to see whether the known difference is as we expect it to be.

9.2.3 Building the Batches

Overview. We break the problem into three parts, here. First, we describe how to take a single right pair and generate 255 other right pairs from it. Then, we describe how to take a pair that gets the same 8-bit XOR difference out of both g functions, and use it to generate enough pairs that it is likely that at least one is an actual right pair; i.e., the differences from the two g functions cancel out in the first addition of the PHT. Finally, we describe how to choose pairs so that we are very likely to get at least one pair that gets the same 8-bit XOR difference from both g functions.

Let us suppose we must choose n0 pairs before we get a pair such that both g functions get the same 8-bit XOR difference. (That is, we must choose n0 pairs to get our differential through half of the second round.) We have no way to know which pair has the desired differential, so we must treat all of them as potentially right pairs.

Let us also suppose that given a pair whose differential survives through half of the second round, we must choose n1 related pairs before we find one pair that survives through the whole second round. The rest of the differential occurs with probability one, so one of these pairs is a right pair. We must apply this process to all the potential right pairs to be nearly certain of getting a right pair. Thus, we now have n0n1 pairs to deal with.

Finally, let us suppose that, given a right pair, we must expand it into n2 batches of 256 pairs each before we will be able to detect it. We must do this to all potential right pairs, meaning that we end up with n0n1n2 batches total, each of 256 pairs. Thus, we need to request a total of n0n1n2256 pairs of plaintexts for encryption.

This tells us about the chosen text requirements, but not the workload for the attack. Suppose we have to guess k bits, and that given a guessed partial key, we can check a batch with an average of work equivalent to n3 encryptions. Then the total work factor is 2kn0n1n2n3. As we discuss below, n3 ≈ 1, n2 = 216, n1 = 28, and n0 = 28.

Amplifying Right Pairs. The greatest difficulty in this attack is determining a sequence of inputs that gives us at least one batch of 256 pairs of plaintexts that contains all right pairs. This is necessary because we know only that one bit of difference in the output from the fourth round. Essentially, what we need to do is to "magnify" the effects of a right pair so that a single right pair generates 256 right pairs.

To see how this can be done, let us consider a pair of inputs to the F-function,

X = (C, D) = (c0, c1, c2, c3), (d0, d1, d2, d3)

X* = (C*, D*) = (c0, c1*, c2, c3), (d0, d*1, d2, d3)

such that we know that we have a right pair; i.e., that the two g functions have the same XOR difference after processing X and X*, that these XOR differences have Hamming weights of eight bits only, and that the two XOR differences cancel out during the first addition in the PHT, leading to an output difference of (0, ?). We want to find a way to choose related input pairs that will also be right pairs. To do this, we must consider some details of how right pairs really work. When we compute F(C, D), we find

f0 = g(C) + g(D)

f1 = g(C) + 2g(D)

When we apply an XOR difference to a value, this has the effect of flipping k bits, where k is the Hamming weight of the XOR difference. In terms of mod 232 addition, flipping bit j means adding 2j or subtracting it, depending on whether the bit was on or off. If X, X* is a right pair, it means that g(C), g(D) both got the same 8-bit difference, and that each bit of C covered by a bit of that XOR difference was different than the corresponding bit of D covered by that bit of the difference. In other words, the XOR difference must result in subtracting from g(C) whatever it adds to g(D), so that g(C) + g(D) will be unchanged. This means that simply changing g(C) at random won't generally produce more right pairs from this one.

There are two ways to make more right pairs from this right pair:

1. Find a way to change one or more inactive bytes (c0, c2, c3, d0, d2, d3) such that the 8 bits that are changed in g(C), g(D) in the original pair aren't changed. Since we don't know which bits those are, this looks hard to do.
2. Find a way to change those inactive bytes that leads to the same XOR difference being applied to g(C) and g(D). If the same bits are flipped in g(C) and g(D), then the resulting pair will also be a right pair. This is our approach.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Our solution is as follows: If we flip the same bits in g(C) as in g(D), then the sum g(C) + g(D) will be unchanged, and as a result, the effect of the input difference in c1, d1 will also be unchanged. Let x3, y3 be the bytes that actually go into the S-box s3 in the two g functions. We control c3, d3, but their values axe XORed with some unknown bytes before being used as x3, y3. Thus,

x3 = c3 ⊕ z3

y3 = d3 ⊕ z4

We need to be able to choose c3, d3 so that x3 = y3. If we ever did that, then we could generate 255 additional cases where y3 = z3, simply by XORing 1,...,255 into both c3 and d3. Now, if we knew z3 ⊕ z4, we could choose any c3 value we liked, and then set d3 = c3 ⊕ z3 ⊕ z4, giving us x3 = y3. We thus try all 256 possible values for z3 ⊕ z4. One of those values is right; we can use it to take a single right pair, and produce 255 more just like it. This is done by keeping A and B constant, and by choosing

C = (c0, c1, c2, c3 ⊕ i)

D = (d0, d1, d2, d3 ⊕ i ⊕ z3 ⊕ z4)

C* = (c0, c*1, c2, c3 ⊕ i)

D* = (d0, d*1, d2, d3 ⊕ i ⊕ z3 ⊕ z4)

where c1, d1, and c*1, d*1 are the bytes changed in the original right pair. For each potential right pair, we thus generate 256 pairs of text according to the above formula for each guess about z3 ⊕ z4. This means that each potential right pair generates 256 batches of 256 pairs of plaintexts (and corresponding pairs of ciphertexts), or a total of 216 plaintext pairs. Thus, n0 = 256.

Deriving a Right Pair from a Partial Right Pair. Suppose we have a pair of texts, (C, D), (C*, D*), such that g(C) ⊕ g(C*) = g(D) ⊕ g(D*), with that XOR difference having Hamming weight of 8. In most (255/256) cases, this won't be a right pair, because g(C) + g(D) ≠ g(C*) + g(D*). To turn this into a right pair, some of the bits in g(C) which axe covered by the XOR difference need to be flipped. We thus choose the available bytes, c0 and c2, and vary both bytes randomly in different texts. After about 256 different random c0, c2 choices, we expect to derive a real right pair from the partial right pair. Naturally, most of the pairs we generate will not be right pairs, and we won't be able to distinguish them from wrong pairs until we mount the attack.

We thus determine that n1 = 256.

Finding a Single Partial Right Pair. A partial right pair can be found by getting the same XOR difference out from the S-box determined by c1 and also by d1. We don't know the S-box these are going through, but simulations show that if we try i, i ⊕ δ as a pair of inputs into a random 8-bit-wide bijective S-box, s, we have a probability of 1/2 of getting any desired output XOR difference. If there axe three XOR differences that lead, after the MDS matrix is applied, to 8-bit XOR differences, then the probability is thus 7/8 that this approach will work to find a partial right pair.

We have the same complications we did above: we control c1, d1, but we need to control x1 = c1 ⊕ z1,y1 = d1 ⊕ z2, the inputs to the S-boxes in the two g functions. Similarly to the way we dealt with this above, we end up guessing z1 ⊕ z2. The result is that we generate 216 plaintext pairs, of which at least one is almost certainly a partial right pair. Thus, n2 = 216.

Testing a Batch of Plaintext Pairs. We generate n0n1n2 = 2828216 = 232 batches of plaintext pairs. During our attack, we must guess as much of the key material as is required, and for each guess, we must test each batch to see if it is made up of only right pairs. The probability of a randomly-selected key giving us a "false positive"—that is, a batch that appears to be right but is not—is 2-256. This probability is low enough that we never expect a single false positive.

We now consider the difficulty of testing one such batch of plaintext pairs. The question is how many trial partial decryptions we must do for each batch, on average, before determining that the batch is not made up of only right pairs. For a guessed partial key, each batch will yield a sequence of 256 bits; in the batch of right pairs, they will all be the expected bit. However, for each batch, we can stop searching as soon as we find a single bit that is different than the others. The expected number of trial partial decryptions required per batch is thus

2 · 1/2 + 3 · 1/4 + 4 · 1/8 + ··· + (n + 1) · 1/2n ≈ 3

Each partial decryption is less work than a full encryption. We take n3 = 1 for our estimates, which corresponds to each partial decryption being one-third as much work as a full encryption.

9.2.4 Mounting the Attack

To mount the attack, we must determine what key material must be guessed to check our batches. Consider a 5-round Twofish variant without whitening and without the 1-bit rotations. In this case, we need only guess S. Recall that in a right pair, we know the difference in only one bit—the low-order bit of D in this case—at the input to the fifth round. After guessing S, we can compute the value of the g function that uses A as its input in the fifth round. The low-order bit of this is XORed with the low-order bit of the corresponding subkey, and then XORed into the bit whose difference we know. Knowledge of the low-order bit of that g output thus gives us the desired bit difference.

To extend this another round, we must also guess the subkey in the sixth round that affects the value XORed into A. Because we only have to get the low bit of D, we need not compute the B value input into the fifth round's F function at all. This requires 232 additional work.

To extend this one more round, we must guess the whole set of round subkeys in the next round. This requires 264 additional work.

Five-round Twofish consists of ten g computations and a little extra work. To be conservative, we can consider each g computation to be 1/8 of a trial encryption.

Key Length Rounds Work(equiv. trial encs.)
128 | 5 | 230 · 264 = 294
128 | 6 | 230 · 296 = 2126
192 | 5 | 230 · 296 = 2126
192 | 6 | 230 · 2128 = 2158
256 | 5 | 230 · 2128 = 2158
256 | 6 | 230 · 2160 = 2190
256 | 7 | 230 · 2224 = 2254

Table 9.3. Differential Attack Results against a Variant with No Whitening and No Rotations

Key Length Rounds Work(equiv. trial encs.)
128 | 5 | 230 · 274 = 2104 |
128 | 6 | 230 · 2138 = 2166 | (No longer useful.)
192 | 5 | 230 · 2106 = 2136 |
192 | 6 | 230 · 2170 = 2200 | (No longer useful.)
256 | 5 | 230 · 2138 = 2168 |
256 | 6 | 230 · 2202 = 2232 |
256 | 7 | 230 · 2266 = 2296 | (No longer useful.9)

Table 9.4. Differential Attack Results against a Variant with No Whitening

Thus, for five- to seven-round Twofish with no whitening and no 1-bit rotates, we have the results summarized in Table 9.3.2 (Note that all of these results are for 241 chosen plaintexts.)

2We are indebted to Eli Biham for the suggestion of guessing the whole last round's subkeys for the 256-bit key.

When we add back in the 1-bit rotates, the attacks grow substantially harder. (Interestingly, though we put the 1-bit rotates in because of concerns about the byte structure, so far, the 1-bit rotates have complicated several attacks based on exploiting properties of the low bit of one of the PHT words.) We must now learn both g function outputs in the fifth round, as well as guessing the high-order 10 or so bits of round subkey used to alter the value XORed into D in the fifth round. We list the resulting attack complexities in Table 9.4.

The whitening also adds a little difficulty. When combined with the 1-bit rotates, we get the results summarized in Table 9.5. We note that there are straightforward differential attacks on four rounds that work despite the 1-bit rotations and whitening, but they involve using only the (0, X) → (?, (?, 1)) differential.

Key Length Rounds Work(equiv. trial encs.)
128 | 5 | 230 · 2128 = 2158 | (No longer useful.)
192 | 5 | 230 · 2170 = 2200 | (No longer useful.)
256 | 5 | 230 · 2202 = 2232 |

Table 9.5.3 Differential Attack Results against Reduced Rounds

Key Length Rounds Work(equiv. trial encs.)
all | 6 | 230 · 232 = 262
all | 7 | 230 · 296 = 2126
192 & 256 | 8 | 230 · 2160 = 2190
256 | 9 | 230 · 2224 = 2254

Table 9.6. Differential Attack Results on a Variant with Known S and No Rotations

Finally, we can consider the difficulty of this attack on a Twofish variant with a fixed, known S value and no 1-bit rotates. In this case, we have the difficulties results listed in Table 9.6.

Again, this demonstrates how the additional key material in the key-dependent S-boxes adds to the practical difficulty of an attack. Additionally, with fixed S-boxes, it may be possible for the attacker to get a substantial improvement, perhaps by as much as a factor of 256, on the number of plain-text batches he must request. This would reduce the plaintext requirement and the work factors on all these attacks by the same factor.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

9.2.5 Lessons from the Analysis

We have learned three important lessons from this analysis:

1. The 1-bit rotates really do add difficulty to some attacks.
2. The key-dependent S-boxes add a great deal of security to the cipher against some attacks.
3. The whitening adds a fair amount of security to the cipher at essentially no cost.

9.3 Extensions to Differential Cryptanalysis

9.3.1 Higher-Order Differential Cryptanalysis

Higher-order differential cryptanalysis looks at higher order relations (e.g., quadratic) between pairs of plaintext and ciphertexts [Lai94, Knu95b]. These attacks seem to work best against algorithms with simple algebraic functions, only a few rounds, and poor short-term diffusion. In particular, we are not aware of any higher-order differential attack reported in the open literature that is successful against more than six rounds of the target cipher. We cannot find any higher-order differentials that can be exploited in the cryptanalysis of Twofish.

9.3.2 Truncated Differentials

Attacks using truncated differentials apply a differential attack to only a partial block [Knu95b]. We have not found any truncated attacks against Twofish. The almost complete diffusion within a round function makes it very difficult to isolate a portion of the block and ignore the rest of the block. Truncated differential attacks are most often successful when most of the cipher's internal components operate on narrow bit paths; Twofish's 64-bit-wide F function seems to make truncated differential characteristics hard to find. We believe that Twofish is secure against truncated differential attacks. For example, our differential attack, described above, is properly classified as a truncated differential attack, and it can successfully attack only five rounds of the full Twofish.

9.4 Search for the Best Differential Characteristic

One aspect of our analysis work is a search for an upper bound on the probability of a differential characteristic. This section contains our results to date. We expect to continue this work and achieve significant improvements over our current results.

The first choice we have to make in differential cryptanalysis is what type of differences to use. Twofish contains S-boxes, an MDS matrix multiply, addition modulo 232, XORs, and rotations. There are two types of differences that we think could be useful: a XOR difference, and a difference mod 232. When we use a XOR difference we have to use approximations for the S-boxes and the additions modulo 232; when we use a differences modulo 232 we have to use approximations for the S-boxes, the MDS matrix multiply, the XORs, and the rotations.

The XOR and addition operations are fairly closely related, and either operation can be approximated with reasonable success in the group of the other operation. In the comparison we will ignore the S-boxes; we assume they are equally hard to approximate in each of the groups. An XOR differential has to approximate two addition operations in each round. An additive differential has to approximate the MDS matrix, a single XOR, and a rotation. We estimate that it is about as difficult to approximate an addition for an XOR differential as it is to approximate an XOR for an additive differential. The rotations are relatively simple to approximate in both cases, so we ignore them for this comparison. Thus, if we ignore the MDS matrix, it would seem that additive differentials are more attractive.

For our analysis, the MDS matrix multiply is best written as a linear function: each output bit is the XOR of several input bits. This is very easy from the point of view of an XOR difference; no approximation is necessary and any given input difference leads to precisely one output difference. For an additive difference this is much harder. There do not seem to be any good approximations of the MDS matrix for an additive difference. Therefore, we estimate that XOR-based differentials are much more effective than additive differentials. In the rest of this paper we will look only at XOR-based differentials.

We use the following notations: Let β be the set of all possible byte values. Let G be the F-function without the key-dependent S-boxes. Thus G consists of two MDS matrix multiplies, the PHT, and the subkey addition.

9.4.1 Differentials of the S-boxes

In this section we look at differential characteristics of the S-boxes. Each S-box consists of a sequence of q-mappings and XORs with a key byte. For q0 and q1 the probability of each differential can easily be computed by trying all possible pairs of inputs.

We define pi (a, b) to be the probability that qi has an output difference of b, given an input difference of a. In other words,

We now look at the first two stages of an S-box. This consists of a q-mapping, followed by an XOR with a key byte, followed by another q-mapping. As usual, we assume uniform random distributions of the input values and the key bytes. We define pij (a, b) to be the probability that this construction gives an output difference b given an input difference a, where i is the number of the first q-mapping, and j is the number of the second q-mapping. It is easy to see that

for i, j ∈ {0, 1}, and we can extend this definition to arbitrary long chains of q-mappings and key-byte XORs. In general, it holds that

S-box 128-bit Key 192-bit Key 256-bit Key
0 | 1.0649 · 2-8 | 1.0084 · 2-8 | 1.0043 · 2-8
1 | 1.0566 · 2-8 | 1.0087 · 2-8 | 1.0043 · 2-8
2 | 1.0533 · 2-8 | 1.0097 · 2-8 | 1.0045 · 2-8
3 | 1.0538 · 2-8 | 1.0088 · 2-8 | 1.0044 · 2-8

Table 9.7. Best Differential Probabilities of the S-boxes under Random Key

for i, j,..., m ∈ {0, 1}. This allows us to compute the exact probabilities for each of the S-boxes in Twofish. Table 9.7 gives the probabilities of the best differential of each of the S-boxes for each key length. From this point of view the S-boxes are very good; there are no high-probability differentials. (Note that the average differential probability is 1/255 = 1.0039 .2-8, as we know that the S-boxes are permutations, and thus the output differential 0 does not occur in non-trivial cases. The best differential probability must be at least as large as the average.)

Note that the numbers in this table hold only when the key bytes are chosen at random. In our analysis we assumed that the probabilities in each q-stage were independent. If we try a differential many times, each time with random input and key byte values, then we expect to get the numbers in the table. However, for any particular set of key bytes there are differentials with a much higher probability (as shown in section 7.2.3). Our computations are no longer valid, because for any fixed key byte, the differential probabilities of pi and pj in equation 9.1 are no longer independent of each other.

Twofish uses the same S-boxes in each round. When analyzing a multi-round differential characteristic, the differential probabilities of each of the round functions are not independent either. This makes the analysis of the probability of a differential characteristic more difficult.

9.4.2 Differentials of F

The function F takes a 64-bit input and produces a 64-bit output. Thus there are a total of about 2128 possible differentials. It is clearly not possible to compute or list all of them. To alleviate this problem we will group the differentials in sets, and for every set compute upper bounds on the probability of the differentials in that set.

We split the 2128 different differential patterns into a number of subsets. The input difference is classified by the set of input bytes that are nonzero. There are 256 different classifications of input differences. The output difference too is classified by the set of output bytes that are non-zero. We group differentials with the same input and output classification in the same set. There are therefore 216 different sets of differentials, each containing between 1 and 25516 elements.

We will construct differentials of F in two steps. First we use a differential approximation of the S-boxes, and then we use an approximation of the differentials of G.

Differentials of G. The MDS matrix multiply is purely linear, and thus creates no problem for our differential. The PHT and key addition use addition modulo 232 as basic operation. This makes the differentials non-trivial. A theoretical analysis of differential probabilities is difficult, as the probabilities at the result are not independent of each other. We therefore chose to use numerical simulation to establish bounds on the differential probability.

We are trying to derive an upper bound on differential probabilities. Therefore, we are interested in finding good bounds for the most likely differentials of G. We know that for any 128-bit key, the best differential probability of an S-box is 18/256. If we look only at the S-boxes, then the most likely differentials occur when there are a low number of active S-boxes. The most important task is thus to find good bounds on the differential characteristics of G for differentials with a low number of active S-boxes.

We performed numerical simulations of differentials of G. Given an input difference, we generated n random input pairs with that difference and applied the G function (using random keys). We collected the output differences and counted how often each of them appeared. Due to limited resources we could do this analysis only for moderately large n.

From this data we would like to derive a bound on the differential probability. Let us assume a specific differential occurs k times out of n tries. It is obviously not a good idea to use k/n as a bound on the differential probability. Most possible differences occur zero times, but we should not assume that they have a zero probability. If we knew the distribution of the probability we could give some meaningful bound; for example, saying that the probability is less than x with a confidence level of 1 percent. However, in our case we do not know the distribution of the differential probabilities, and it would be dangerous to assume one. We can, however, reverse the process.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Let us assume that a specific differential has a probability p. If we try the input difference n times, we expect to find this differential around p · n times. The number of times this differential is actually observed is binomially distributed. Let X be a stochastic variable that represents the number of times the differential is observed. We have

From this distribution we can derive a bound on the lower tail of the binomial distribution [Fer98a]:

for k ≤ p · n. Given a probability p for the differential, we can say that we have an unlikely event if the differential occurs k times and Pr(X ≤ k) < γ where γ would be a small number. This is a normal test for statistical significance.

We use the following rule to derive a bound on a differential that occurs k out of n times. We use a probability p such that Pr(X ≤ k) < γ for some global parameter γ (typical values for γ are 0.05 or 0.01). Of course, we try to choose p as low as possible given this condition. This will overestimate p for most differentials, but underestimate the actual p in a few cases.

We ran these simulations for all input differences with a low enough number of active S-boxes. For every differential that we tried, we estimated the probability using this rule. For each set of differentials with the same input and output characterization, we computed the maximum estimated probability. For each differential, there is a small chance that we underestimated the probability. However, it is far less likely that we underestimated the maximum probability of a set of differentials. For our maximum to be too low we have to underestimate the probability of the most likely differential. This by itself is rather unlikely. Not only that, we cannot significantly overestimate the probability of any differential with a probability close to the most likely differential. We therefore feel confident that these approximations are reasonable, and that they most likely will result in our overestimating the actual differential probability quite significantly.

For differentials with too many active S-boxes (for which we did not run the simulations), we simply use an upper bound of 1 on the probability of a differential of G.

To improve efficiency, we generate our input differences using a straightforward structure. This improves our performance and allows us to increase the number of samples that we make. However, the differentials that we try are no longer independent of each other. We have observed that the use of structures significantly increases the peaks in the bounds. The smaller the structures that we use, the lower the maximum probability bound tends to be. Therefore, we try to reduce the use of structures. We hope our next software version will allow us to eliminate structures altogether.

Apart from these numerical results, we know that certain differential patterns cannot occur. For example, if the input difference is restricted to the first input word of G, then the output difference must have active bits in both output words. Similarly, if the input difference is restricted to the second input word of G, then the output difference must have active bits in both halves, except when the output of the MDS matrix has a difference of 0x80000000. In this special case, we know that all four S-boxes in this half must be active (otherwise more than one byte in the output of the MDS matrix must change). Our software generates all these impossible differential patterns and sets the differential probabilities of the associated sets to zero.

Prob. Differential
2-20 | x000 0x00 → 0000 xxxx
2-26 | x000 0x00 → 0000 0xxx
2-22 | x000 0x00 → xxxx 0000
2-26 | 000x x000 → 0000 xxxx
2-22 | 000x x000 → xxxx 0000
2-20 | 00x0 000x → 0000 xxxx
2-22 | 00x0 000x → xxxx 0000
2-14 | 0x00 0000 → 0x0x 0x0x

Table 9.8. Experimentally Found Differentials of F

Differentials of F. Given the results from the last section, we can now create a table of upper bounds on the differential probabilities of differentials of F. For each set of differentials we know how many active S-boxes there are. Let σ be the maximum probability of a differential of an S-box. We can now bound the probability of each set of differentials by multiplying the bounds that we found in the previous section on the set by the proper power of σ.

The value σ can be set in various ways. We know that most S-boxes have a best differential probability of 12/256. For the time being we will use this value for σ. Other values, especially larger ones, will be discussed later.

Experimental Results. We ran a test for differential characteristics of the F function with one active byte in each 4-byte g input and at most four active bytes at the 8-byte F output. Table 9.8 shows some interesting differentials that we found. The probabilities are very approximate, as they derive from numerical experiments with not too many samples. Note that in all but the last of these, the two active input bytes are going through the same key-dependent S-box due to the 8-bit rotate at the input of the second g function. The leading cause of all of these differentials seems to be that these two bytes generate the same output differential from the S-box, and thus the same differential at the output of the MDS matrix. With some positive probability, the PHT maps this into an output differential where one half is zero. This probability depends on the bit pattern of the differential just before the PHT.

The last characteristic has a different explanation. The input differential results in a differential at the output of the MDS matrix that has a relatively high chance of being converted into a 0x0x pattern after a constant is added to it. For example, the differential 03f2037a16 has a fair chance of being converted to a 0x0x pattern after a 32-bit addition.

All of these characteristics have a relatively low probability. We have not found any attack that directly relates to this property, but it provides a useful starting point for further research. In particular, upper bounds on the differential characteristics of F of this type could be used to improve our bound on the best differential characteristic for the full cipher.

9.4.3 Differentials of the Round Function

Once we have derived bounds on the differentials of F, we can do the same for the round function. The differential pattern at the start of the round is characterized by 16 bits; each bit indicates whether the differential pattern in the corresponding byte is non-zero. Given the characterization of the differential pattern at the input of the round, we know exactly which S-boxes are active. We can generate a list of all suitable differential patterns of F with their associated probability bound. Each of these differentials is combined with the other half of the input differential using the rotate and XOR operations. Each choice of the F differential set leads to several possible output differential patterns as the rotate and XORs can lead to different output characterizations depending on the exact differential.

For example, let us look at a differential pattern of 0110 in a 32-bit word. This pattern indicates that only the middle two bytes of the 4-byte word contain active differential bits. After a left rotation, the possible output differential patterns are: 0100, 0110, 1010, 1100, and 1110. XORing two differential patterns can similarly lead to a list of possible results. If we XOR two words, one with a differential pattern of 0101 and one with a differential pattern of 0011, then the possible result patterns are 0110 and 0111.

For each input differential pattern to the round, we can go through all possible F differential patterns, and generate all possible output patterns that can arise. For each possible output pattern of the round, we keep track of the largest upper bound that we generate this way. This produces an upper bound for each of the 232 possible input/output differential patterns of the round function.

9.4.4 Multi-round Patterns

The simplest way of generating multi-round patterns would be to use the list of 232 possible round patterns and a standard search algorithm. We use an algorithm that is somewhat more efficient than that. There are 216 possible differential patterns after r rounds, as each of the 16 data bytes can have a zero or non-zero difference. For each of the 216 possible patterns, we store an upper bound on the probability of any characteristic that has this difference pattern after r rounds. Furthermore, we store the list of differential patterns of F, and a precomputed table of how the rotates and XORs can propagate patterns. For each difference pattern after r + 1 rounds, we use this data to compute an upper bound on the probability of a differential characteristic that has this pattern after r + 1 rounds.

Given the output pattern of the round in question, we know the first half of the input pattern. This leaves us with 256 possible differential input patterns, and 256 possible differential patterns of F. Each of the 216 possible combinations is tried to see whether it can yield the required output differential pattern. The process can be speeded up by traversing either the F output patterns or the input patterns in decreasing order of probability and using some simple cut-off logic.

9.4.5 Results

The results depend on the parameters used to estimate the differential probabilities of G, and the values of γ and σ.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Our current results use n = 211 tries for all differentials with one active S-box, and n = 28 tries for all differentials with 2 active S-boxes. The structure size is 8 and 16 respectively. We use γ = 0.05, and σ = 12/256. The full Twofish cipher has 16 rounds. We assume that an adversary can somehow bypass the first round and can mount a 3R-attack. We thus look at the best 12-round differential characteristic.

With these parameters we found an upper bound on the probability of a 12-round differential characteristic of 2-102.8. This puts a differential attack against Twofish well outside the practical realm.

This upper bound is pessimistic in the following areas:

• The best differential pattern used three active S-boxes in four of the 12 rounds. The probability of passing a differential with three active S-boxes through G is currently taken to be 1. This is clearly overly optimistic, especially since the differential pattern used has both a low input and a low output weight. We believe that extending our simulations to all differentials with three active S-boxes will yield a significant further reduction in probability.
• Many of the rounds in the best differential pattern use fancy transformations of the difference pattern by the rotations. This is to be expected of our algorithm, but any non-trivial transformation poses serious restrictions on the actual difference patterns of that word. This makes it much less likely that our upper bound can actually be approached by an actual differential.
• Our estimates are based on the maximum probability of groups of differentials. It is not clear at all that there exists a differential that has a probability that even approaches our upper bound.

9.4.6 Other Problems for the Attacker

To create an attack, the attacker has to choose a specific differential characteristic. That characteristic uses certain specific differences of each of the S-boxes. To get anywhere near the bound, all of these differences need to have a probability close to our σ. We chose σ equal to the probability of the best differential of most S-boxes. However, a specific differential will not have the same probability under all keys. If the S-box keys are not known, the attacker has two options. First, he can guess the S-box key bits, and construct a differential characteristic based on that assumption. To achieve good differential probabilities in enough S-boxes, he will have to guess the keys of at least two S-boxes (between 32 and 64 bits, depending on the key size). Alternatively, he can try to find a differential that works for all keys. As we saw in section 9.4.1, this leads to very low differential probabilities.

9.4.7 Best S-box Differential

We use σ = 12/256 while we know that there are keys for which the best S-box differential has probability 18/256 for a 128-bit key, and even higher for larger key sizes. However, those higher probabilities only occur for a small subset of the keys. We need to address the question of how much we are willing to pay in the size of the keyspace the attack is effective on to get a higher probability. If we have an attack that works for 2-28 the key space, how much more efficient should the attack be before it is a better choice for the attacker?

The most natural way is to look at the expected work for the attacker before recovering a single key. We assume that there are enough keys to attack, and optimize the attacker's strategy to find any one key with the least amount of work. This is a reasonable way of looking at the attacker's problem. After all, we know there is an attack that is effective on a subset of 2-64 of the keys with 264 work: a simple exhaustive search of any subset of that size will do. With such a brute-force attack on a subset of the keys, the expected amount of work before a key is found remains the same.

Let us now look at our Twofish differentials. Suppose we want to use a differential of S-box 0 with probability 14/256; this is possible for about one in eight of all possible keys. We have restricted ourselves to 1/8 of the set of keys, so the workload of our attack should be reduced by at least a factor of 8 for this to be worthwhile. We ran our search for the best differential characteristic pattern again where S-box 0 had a best differential probability of 14/256. The resulting differential probability was 4.6 times higher than the result with σ = 12/256. Is this worth it?

Let us assume a differential has a probability that depends on the key. We have a list of (pi, ki) where the probability of the differential is at most pi for a fraction ki of all keys. The expected workload of the attacker to get a single right pair is 1/pi for a fraction ki of the key space, and thus ς ki/pi when taken over all keys. The workload is at least max ki/pi. This corresponds to the workload of an attack with a differential with probability min pi/ki. In our situation, the minimum occurs when we use the S-box approximation with probability 12/256. As we currently ignore the 1/ki term, the actual "effective" probability of a real differential is lower than the bound that we have derived.

We conclude that using a higher probability than 12/256 for an S-box approximation is not worth the loss in key space on which the approximation holds. Thus the bounds we presented earlier hold, and are in fact pessimistic.

9.4.8 Other Variants

As an experiment, we ran the same analysis for Twofish with the 1-bit rotations removed. This makes our approximations match the behavior of the differential much better. Our results give an upper bound on the probability of a 12-round differential characteristic of 2-104.1

This bound is not much better than we have for full Twofish. However, the sequence of differential patterns that achieves this bound uses far more approximations of F that have three active S-boxes. In all cases it uses differential characteristics of G that have three active input bytes and only a few active output bytes. In practice, such differentials of G will have a far lower probability than the upper bound of 1 that we currently use. Therefore, we expect that our bound can be improved to beyond 2-128.

We have no reason to believe that the 1-bit rotations make Twofish stronger against a differential attack. They were conceived to break up the byte-level structure, but they do not require a separate approximation or increase the avalanche effect of the cipher. We think it is unlikely that the full Twofish has a differential characteristic that is significantly more likely than the version without rotations.

9.4.9 Further Work

We will continue our analysis work to improve the bound and our understanding of the intricacies of Twofish. We have several areas that we plan to improve.

Improved G Differential Estimates. An obvious way to improve our overall bound is to improve our bounds on the differentials of G. We hope to be able to do this in the near future. A larger sample size will improve the accuracy of our estimates. Extending our computations to differentials of G with three active S-boxes should give a great improvement.

More Accurate Patterns. Our current pattern-representation is somewhat coarse. We group differentials only by which bytes contain active bits. Apart from the first and final round, all internal differential patterns in our best result have at most four active bytes (out of 16). A more fine-grained grouping of the differentials could lead to a better upper bound.

For example, there are only a few G differences with one active input byte that have a relatively high probability. Instead of grouping these into the sets, we could treat them separately. This would ensure that our algorithm doesn't magically transform the output of the high-probability difference pattern to one with fewer active bytes by the rotation. The characterization could be extended with special cases for differences that have only a few active nibbles. We expect that this will result in more S-boxes being needed for a full differential characteristic, and thus a lower bound.

Improved Treatment of S-box Differentials. There is still room to improve our approximations of the S-boxes. We can, for example, compute the best differential approximation for each of the output differences separately. This can then be combined with the analysis of G to get a better bound on differentials of F.

The data on the best S-box differentials in section 7.2.3 is merged for all the S-boxes. We plan to test the differentials again and collect information for each S-box separately.

We will also improve our handling of the variation of differential probability over the key-space. This will also result in a better bound.

Additive Differentials. We would like to take a closer look at additive differentials modulo 232. Although we do not expect these to be more useful, it would be nice to derive some bound in that case too.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

9.4.10 Conclusion

For practical purposes, Twofish is immune to differential cryptanalysis. We have shown that any 12-round differential has a probability of at most 2-102.8. This bound is far from tight, and we expect that any real differential has a much smaller probability.

The Twofish structure is not easy to analyze. The mixing of various operations makes it hard to give a clean analysis and forces us to use approximation techniques. Some aspects, such as the rotates, make the analysis a lot harder and force us to use less accurate approximations, while there is no a priori reason to assume that the rotations would have any significant influence on the differential probabilities.

One can argue whether a cipher with a structure that is easier to analyze would be preferable. On the one hand, a structure that allows easier analysis makes it easier to rule out certain attacks. On the other hand, the very structure that makes it easy to analyze might be used in a future attack. Although differential attacks were obviously considered during the design, Twofish was not specifically strengthened against differential attacks, or designed to allow a simple upper bound on differential probabilities to be derived. This is a result of the design philosophy of Twofish. It was not optimized specifically against known attacks; it is a conservative design that tries to resist both known and unknown attacks.

9.5 Linear Cryptanalysis

We perform a similar analysis to that of Section 9.4 in the context of linear cryptanalysis. The approach is the same, but a simpler model was used.

Our model is much as before: it is centered around the pattern of active bytes. (Here a byte position is considered active if any of its bits are active in the linear approximation for that round; i.e., if the mask T selects at least one bit from that byte position.) We used a fairly rough approximation of the effect of the PHT. This approximation fails to accurately treat some low-probability approximations for the PHT where there are fewer active bytes at the output than at the input. This analysis also ignores the 1-bit rotations.

Because of the second limitation, our model should be considered only a heuristic; it may fail to capture some important features of the round function.

A search for the best linear characteristics within this model found that every 12-round characteristic has at least 36 active S-boxes. Here is an example of one characteristic we found:

1  FE->FF (1 active)
2  FF<-FF (2 active)
3  FF->DD (4 active)
4  CC<-DD (6 active)
5  CC->00 (0 active)
6  C<-00  (6 active)
7  CC->77 (4 active)
8  00<-77 (0 active)
9  00->77 (4 active)
10 66<-77 (6 active)
11 66->00 (0 active)
12 66<-00 (3 active)
13 66 07

The first few approximations for the F function are 0116 → FF16, FF16, 2216 → FF16, 3316 → DD16, and so on.

To translate these results into an estimated attack complexity, we turn to the results of Section 7.2.3. We have LPmax ≤ (108/256)2, so this suggests an attacker would need at least (LPmax)-36 ≈ 289.6 known plaintexts; and such an analysis would only work for a very small class of weak keys (representing about 2-49.1 of the keyspace for 128-bit keys, or about 2-68 of the 192-bit keyspace).

It is much more natural to demand that the attack work for a significant percentage of all keys. In this case, LPmax = (80/256)2 is a much more representative value, and thus in this model any linear attack working for a significant fraction of the keyspace would require at least 2120.8 chosen plaintexts.

These results are only heuristic estimates, and probably overestimate the probability of the best linear characteristic. Nonetheless, they present some useful evidence for the security of Twofish against linear cryptanalysis.

A more thorough analysis along the lines of section 9.4 remains to be performed.

9.5.1 Multiple Linear Approximations

Multiple linear approximations [KR94, KR95] allow one to combine the bias of several high-probability linear approximations. However, it only provides a significant advantage over traditional linear cryptanalysis when there are a number of linear approximations whose bias is close to that of the best linear approximation. In practice, this seems to improve linear attacks by a small constant factor. Hence, we do not feel that Twofish is vulnerable to this kind of cryptanalysis.

9.5.2 Non-linear Cryptanalysis

Another generalization of linear cryptanalysis looks at non-linear relations [KR96b], e.g., quadratic relations. While this attack, combined with the technique of multiple approximations [KR94], managed to improve the best linear attack against DES a minute amount [SK98], we do not believe it can be brought to bear against Twofish for the same reasons that it is immune to linear cryptanalysis.

9.5.3 Generalized Linear Cryptanalysis

This generalization of linear cryptanalysis uses the notion of binary I/O sums [HKM95, Har96, JH96]. An attacker attempts to find a statistical imbalance that can be described as the result of some group operation on some function of the plaintext and some function of the ciphertext. We have not found any such statistical imbalances, and believe Twofish to be immune to this kind of analysis.

9.5.4 Partitioning Cryptanalysis

Partitioning cryptanalysis is another generalization of linear cryptanalysis [Har96, JH96, HM97].3. An attacker trying to carry out a partitioning attack is generally trying to find some way of partitioning the input and output spaces of the round function so that knowledge of which partition the input to a round is in gives some information about which partition the output from a round is in. This can be seen as a general form of a failure of the block cipher to get good confusion; an attacker after N rounds can distinguish the output from the Nth round from a random block of bits, because the output is somewhat more likely to be in one specific partition than in any of the others. This can be used in a straightforward way to attack the last round of the cipher.4

3Similar ideas can be found in [Vau96b]
4Basic results against reduced-round DES can be found in [HYSK98].

We have been unable to find any useful way to partition the input and output spaces of the Twofish F function or a Twofish round that works consistently across many keys, because of the key-dependent S-boxes. For the 128-bit key case, there are 264 different F functions, presumably each with its own most useful partitioning. We are not aware of any general way to partition F's inputs and outputs to facilitate such attacks.

9.5.5 Differential-linear Cryptanalysis

Differential-linear cryptanalysis uses a combination of techniques from both differential and linear cryptanalysis [LH94]. Due to the need to cover the last part of the cipher with two copies of a linear characteristic, the bias of the linear characteristic is likely to be extremely small unless the linear portion of the attack is confined to just three or four rounds. (The available linear characteristics for Twofish's round function have a relatively low probability, and are very hard to combine due to the MDS and PHT mappings.) This means that the cryptanalyst would need to cover almost all of the rounds with the differential characteristic, making a differential-linear analysis not much more powerful than a purely differential analysis. Therefore, we feel that differential-linear cryptanalysis is unlikely to be successful against the Twofish structure. In our analysis, we have found no differential-linear attacks that work against Twofish.

9.6 Interpolation Attack

The interpolation attack [JK97, MSK98b] is effective against ciphers that use simple algebraic functions. The principle of the attack is simple: if the ciphertext can be represented as a polynomial or rational expression (with N coefficients) of the plaintext, then the polynomial or rational expression can be reconstructed using N plaintext/ciphertext pairs.

However, interpolation attacks are often only workable against ciphers with a very small number of rounds, or against ciphers whose round functions have very low algebraic degree. Twofish's S-boxes already have relatively large algebraic degree, and the combination of operations from different algebraic groups (including both addition mod 232 and XOR) should help increase the degree even further. Therefore, we believe that Twofish is secure against interpolation attacks after even only a small number of rounds.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

9.7 Partial Key Guessing Attacks

A good key schedule should have the property that, when an attacker guesses some subset of the key bits, he does not learn very much about the subkey sequence or other internal operations in the cipher. The Twofish key schedule has this property.

Consider an attacker who guesses the even words of the key Me. He learns nothing of the key S to g. For each round subkey block, he now knows Ai. If he guesses K0, he can compute the corresponding K1. He can carry this attack out against as many round subkeys as he likes, but each guess takes 32 bits. We can see no way for the attacker to actually test the 96-bit guess that it would take to attack even one round's subkey in this way on the full Twofish.

An alternative is to guess the key input S to g. This is only half the length of the full key M, but provides no information about the round keys Ki. The differential attack described in Section 9.2 is the best way we were able to find to test such a partial key guess. We can see no way to test a guess of S on the full 16-round Twofish.

9.8 Related-key Cryptanalysis

Related-key cryptanalysis [Bih94, KSW96, KSW97] uses a cipher's key schedule to break plaintexts encrypted with related keys. In its most advanced form, differential related-key cryptanalysis, both plaintexts and keys with chosen differentials are used to recover the keys. This type of analysis has had considerable success against ciphers with simplistic key schedules—e.g., GOST [GOST89] and 3-Way [DGV94b, Dae95]—and is a realistic attack in some circumstances. A conventional attack is usually judged in terms of the number of plaintexts or ciphertexts needed for the attack, and the level of access to the cipher needed to get those texts (i.e., known plaintext, chosen plaintext, adaptive chosen plaintext). A related-key attack adds another parameter—number of related keys under which the plaintexts will be encrypted.

9.8.1 Resistance to Related-key Slide Attacks

A "slide" attack occurs in an iterated cipher when the encryption of one block for rounds 1 through n is the same as the encryption of another block for rounds s + 1 to s + n. An attacker can look at two encryptions, and can slide the rounds forward in one of them relative to another. S-1 [Anon95] can be broken with a slide attack [Wag95a]. Travois [Yuv97] has identical round functions, and can also be broken with a slide attack. Conventional slide attacks allow one to break the cipher with only known- or chosen-plaintext queries; however, as we shall see next, there is a generalization to related-key attacks as well.

Related-key slide attacks were first discovered by Biham in his attack on a DES variant [Bih94]. To mount a related-key slide attack on Twofish, an attacker must find a pair of keys M, M* such that the key-dependent S-boxes in g are unchanged, but the subkey sequences slide down one round. This amounts to finding, for each of the eight byte-permutations used for subkey generation, a change in the keys such that:

si(j, M) = si(j + 2s, M*)

for n values of j. In total, this requires 8n of these relations to hold.

Let us look in more detail for a fixed key M. Let m ≥ {5,...,8} be the number of S-boxes used to compute the round keys that are affected by the difference between M and M*. Observe that m ≥ 5 due to the restriction that S cannot change and the properties of the RS matrix that at least five inputs must change to keep the output constant. There are at most (8 m)232m-128 possible choices of M*. We have a total of nm 8-bit relations that need to be satisfied. The expected number of M* that satisfy these relations is thus (8 m).2-8nm+32m-128. For n ≥ 4 this is dominated by the case m = 5; we will ignore the other cases for now. So for each M we can expect about 238-40n keys M* that support a slide attack for n ≥ 4. This means that any specific key is unlikely to support a slide attack with n ≥ 4. Over all possible key pairs, we expect 2293-40n pairs M, M* for which a slide of n ≥ 4 occurs. Thus, it is unlikely that a pair exists at all with n ≥ 8.

Swapping Key Halves. It is worth considering what happens when we swap key halves. That is, we swap the key bytes so that the values of Me and Mo are exchanged. In that case, the sequence of Ai and Bi values totally changes because of the different index values used. We can see no useful attack that could come from this.

Permuting the Subkeys. Although there is no known attack stemming from it, it is interesting to ask whether there exist pairs of keys that give permutations of one another's subkeys. There are 20! ways that the rounds' subkey blocks could be permuted. This is almost as large as 264, and so there may very well be pairs of keys that give permutations of one another's round subkey blocks.

9.8.2 Resistance to Related-key Differential Attacks

A related-key differential attack seeks to mount a differential attack on a block cipher through the key, as well as or instead of through the plain-text/ciphertext port. Against Twofish, such an attack must control the sub-key difference sequence for at least the rounds in the middle. For the sake of simplifying discussions of the attack, let us consider an attacker who wants to put a chosen subkey difference into the middle 12 rounds' subkeys. That is, he wants to change M to M* and control D[i, M, M*] for i = 12,...,35. At the same time, he needs to keep the g function, and thus the key S, from changing. All else being equal, the longer the key, the more freedom an attacker has to mount a related-key differential attack. We thus will assume the use of 256-bit keys for the remainder of this section. Note that a successful related-key attack on 128- or 192-bit keys that gets only zero subkey differences in the rounds whose subkey differences it must control translates directly to an equivalent related-key attack on 256-bit keys.

Consider the position of the attacker if he attempts a related-key differential attack with different S keys. This must result in different g outputs for all inputs, since we know that there are no pairs of S values that lead to identical S-boxes. Assuming the pair of S values does not lead to linearly related S-boxes, it will not be possible to compensate for this change in S with changes in the subkeys in single rounds. If we assume 12 active rounds in the attack, the added difficulty is approximately that of adding 24 active S-boxes to the existing related-key attack. For this reason, we believe that any useful related-key attack will require a pair of keys that keeps S unchanged.

9.8.3 The Zero Difference Case

The simplest related-key attack to analyze is the one that keeps both S and also the middle 12 rounds' subkeys unchanged. It thus seeks to generate identical A and B sequences for 12 rounds, and thus to keep the individual byte sequences used to derive A and B identical.

The RS code used to derive S from M strictly limits the ways an attacker can change M without altering S. The attacker must try to keep the number of active subkey generating S-boxes as low as possible, since each active S-box is another constraint on his attack. The attacker can keep the number of active S-boxes down to five without altering S, so this is what he should do. With only the key bytes affecting these five subkey generation S-boxes active, he can alter between one and four bytes in all five S-boxes; the nature of the RS matrix is that if he needs to alter four bytes in any one of these S-boxes, he must alter bytes in all five. In practice, in order to maximize his control over the byte sequences generated by these S-boxes, he must alter four bytes in all five active S-boxes.

To get zero subkey differences, the attacker must get zero differences in the byte sequences generated by all five active S-boxes. Consider a single such byte sequence: the attacker tries to find a pair of four-byte key inputs such that they lead to identical byte sequences in the middle 12 rounds, which means the middle 12 bytes. There are 263 pairs of key inputs from which to choose, and about 295 possible byte sequences available. If the byte sequences behave more-or-less like random functions of the key inputs, this implies that it is extremely unlikely that an attacker can find a pair of key inputs that will get identical byte sequences in these middle 12 rounds. We discuss this kind of analysis of byte sequences in Section 8.2.3. From this analysis, we would not expect to see a pair of keys for even one S-box with more than eight successive bytes unchanged, and we would expect even eight successive bytes of unchanged byte sequence to require control of all four key bytes into the S-box. We would expect a specific pair of key bytes to be required to generate these similar byte sequences.

To extend this to five active S-boxes, we expect there to be, at best, a single pair of values for the 20 active key bytes that leave the middle eight subkeys unchanged.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

9.8.4 Other Difference Sequences

An attacker who has control of the XOR difference sequences in Ai, Bi does not necessarily have great control over the XOR or modulo 232 difference sequence that appears in the subkeys.

First, we must consider the context of a related-key differential attack. The attacker does not generally know all of the key bytes generating either Ai or Bi. Instead, he knows the XOR difference sequence in Ai and Bi.

Consider an Ai value with an XOR difference of δ. If the Hamming weight of δ is k, not including the high-order bit, then the best estimate for the XOR difference that ends up in the two subkey words for a given round generally has probability of about 2-2k. (Control of the Ai, Bi XOR difference sequence does not make controlling the subkey XOR differences substantially easier.)

Consider an Ai value with an XOR difference of δ. If the Hamming weight of δ is k, then the best estimate for the modulo 232 difference of the two subkey words for a given round has probability of about 2-k.

This points out one of the difficulties in mounting any kind of successful related-key attack with non-zero Ai, Bi difference sequences. If an attacker can find a difference sequence for Ai, Bi that keeps k = 3, and needs to control the subkey differences for 12 rounds, he has a probability of about 2-72 of getting the most likely XOR subkey difference sequence, and about 2-36 of getting the most likely modulo 232 difference sequence. After getting the most likely mod 232 difference into the subkeys, however, the attacker must deal with the fact that the output values are XORed into half of the block. Those relatively high-probability mod 232 differences end up back at 2-72 probability for any given XOR difference sequence actually being XORed into the successive rounds' target blocks.

9.8.5 Probability of a Successful Attack with One Related-key Query

We consider the use of the RS matrix in deriving S from M to be a powerful defense against related-key differential attacks, because it forces an attacker to keep at least five key generation S-boxes active. Our analysis suggests that any useful control of the subkey difference sequence requires that each active S-box in the attack have all four key bytes changed.

Further, our analysis suggests that, for nearly any useful difference sequence, each active S-box in the attack has a specific pair of defining key bytes it needs to work. An attacker specifying his key relation in terms of bytewise XOR has five pairs of sequences of four key bytes each, which he wants to get. This leaves him with a probability of a pair of keys with his desired relation actually leading to the desired attack of about 2-115, which moves the attack totally outside the realm of practical attacks.

So long as an attacker is unable to improve this, either by finding a way to get useful difference sequences into the subkeys without having so many active key bytes, or by finding a way to mount related-key attacks with different S values for the different keys, we do not believe that any kind of related-key differential attack is feasible.

Note the implication of this: clever ways to control a couple extra rounds' subkey differences are not going to make the attacks feasible, unless they also allow the attacker to use far fewer active key bytes. For reference, note that with one altered key byte per active subkey generation S-box, the attacker ends up with a 2-39 probability that a pair of related keys will yield an attack; with two key bytes per active S-box, this increases to 2-78; with three key bytes per active S-box, it increases to 2-117. In practice, this means that any key relation requiring more than one byte of key changed per active S-box appears to be impractical.

9.8.6 Conclusions

Our analysis suggests that related-key differential attacks against the full Twofish are not workable. Note, however, that we have spent less time working on resistance to chosen-key attacks, such as will be available to an attacker if Twofish is used in the straightforward way to define a hash function. For this reason, we recommend that more analysis be done before Twofish is used in the straightforward way as a hash function, and we note that it appears to be much more secure to use Twofish in this way with 128-bit keys than with 256-bit keys, despite the fact that this also slows the speed of a hash function down by a factor of two.

9.9 A Chosen-key Attack

A chosen-key attack is a variant of a related-key attack. Not only are there several keys that are related, the attacker actually gets to choose some parts of these keys. The parts not chosen are unknown to the attacker, and the objective of the attack is to find them.

Attack Type Rounds Work Factor Text Req. Special Req.
Partial Chosen Key | 10 | 234 | 232;212 | 160 bits of key from K, K* chosen by attacker
Related-Key | 10 | 2187 | 232;212 | 2155 related-key queries
Table 9.9. Attack Results for Our Chosen-key Attack

9.9.1 Overview of the Attack

Results. We define a partial chosen key attack on a 10-round Twofish without the pre- or post-whitening. The attack requires us to control 20 selected bytes of the key, and to set them differently for a pair of related keys K, K*. The remaining 12 bytes of the two keys, which are the same for both keys, are unknown to us; recovering them is the objective of the attack. Our attack results are summarized in Table 9.9.

Note that the text requirements are shown as n;m, where n is the number of chosen plaintexts, and m is the number of adaptive chosen plaintexts. Also note that the related-key version of the attack requires this many chosen and adaptive chosen plaintexts for each of the 2155 related-key pairs.

How the Attack Works. The attack works as follows:

1. Choose a pair of keys, K, K*, with the property that the middle eight round subkeys are the same for both keys, and the S key value is the same for both keys. This leaves us with some knowledge about the differences that make it into the subkeys as well.
2. Request one encryption under K, and 232 encryptions under K*, trying to find a way to offset the effect of the change in the first round subkey.
3. When we find a likely right pair (detected because of the zero difference in half the ciphertext block), use it to probe the contents of each S-box in turn. This allows us to recover the S key value. This value, given the RS matrix used for its derivation and our starting partial knowledge of the key, allows us to recover the rest of the key algebraically.

9.9.2 Finding a Key Pair

The first step in the attack is finding a pair of keys, K, K*, with some relationship, such that we get a useful subkey difference sequence, while leaving the S key unchanged. We described above how such keys can exist. Here, we assume the existence of some pair of keys with a subkey XOR difference sequence zero in rounds one through eight but with non-zero XOR subkey differences in rounds zero and nine.

Structure of the Key Pair. Based on the discussion above, we assume that the keys K, K* differ in 20 bytes, with the changed bytes selected in such a way to leave S unchanged between the two keys, and to change all four key bytes used to define five of the S-boxes used for subkey generation. We note that a random pair of keys chosen to have this property has only a 2-155 probability of being the pair of keys that will generate our desired sub-key difference sequence. This makes this attack impractical as a differential related-key attack, though an attacker who is able to control the active 20 bytes of key involved in a pair of keys could mount the attack for a chosen pair of keys. (That is, the attacker could choose the values of the active 20 bytes of both K and K*, while remaining ignorant of the other twelve bytes of K and K*, which must be identical for both keys. The attacker would then attempt to learn those remaining 12 bytes of key.)

Subkey Differences. We actually care only about the subkey difference in the round subkey words. Recall that the subkey words are generated from a pair of 32-bit words, which are generated from the subkey generation S-boxes. Let's call these values U, V and U*, V*. We don't know any of U, V, U*, V*, but we do know U′ = U ⊕ U* and V′ = V ′ V*.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

9.9.3 Choosing the Plaintexts to Request

From U′, V′, it is possible to generate a set of about 232 XOR difference values that we can expect to see result from the changed subkeys when they're used in the F function in the first round. We thus do the following:

1. Request the encryption of some fixed 128-bit block A, B, C, D under key K.
2. For each of the 232 different 64-bit XOR difference values, Xi, Yi, we might expect from U′, V′, request the encryption of 128-bit block (A, B, C ⊕ Xi, D ⊕ Yi).
3. Consider only those ciphertexts from the second step that lead to the same right half of ciphertext as the encryption under K in the first step. The goal is to determine the output XOR from the first round's F function when the input is A, B, under the two keys. (The only difference in the output is caused by the difference in round subkeys.) Note that we expect to have about one false positive occur; we simply run the attack on both potential right pairs; the wrong pair will not allow the attacker to find any S-box offsetting differences.

9.9.4 Extracting the Key Material

At this point, we know the single XOR difference from the F function. We can now learn the S-boxes in the g function. To do this, we replace the high-order byte of A in the right pair with 256 different values. Thus, we request the encryption of Aj, B, C, D under K*. This causes one of 256 possible XOR differences in the g function output. We expect to have to try about 512 requests with different XOR difference values Xi, Yi, where these values are derived from the Xi, Yi values that generated the original right pair based on the possible XOR differences in g. We thus request 512 plaintexts of the form Aj, B, C ⊕ Xi, D ⊕ Yi) to be encrypted under K*. One of these we expect to be a right pair, which we can recognize, again, by the fact that the right half of the ciphertext is the same for Aj, B, C, D encrypted under K, and for Aj, B, C ⊕ Xi, D ⊕ Yi encrypted under K*.

After carrying this out for each of the Aj, we have good information about the XOR differences being generated from g by the changes between A and Aj as input. In particular, we will note the 14 cases where we probably got 8-bit Hamming weights in the output XOR difference of g. This information should be enough to brute-force this S-box, s3. That is, we try all 232 possible S-boxes for s3, and expect one to be the best fit for our data.

We repeat the attack for each of the other bytes in A, so that we recover all four S-boxes in g. When we know these S-boxes, this means we know S. Note that at the beginning of the attack, we already know 20 of the 32 key bytes used. More importantly, due to the structure of the RS matrix used to derive S from the raw key bytes, we know all but three bytes used to derive each 4-byte "row" of values in S. This allows us to guess and check the remaining key bytes three bytes at a time, thus recovering the entire key.

9.10 Side-Channel Cryptanalysis and Fault Analysis

Resistance to these attacks was not part of the AES criteria, and hence not a major concern in this design. However, we do have these comments to make on the design.

Side-channel cryptanalysis [KSWH98b] uses information about the cipher in addition to the plaintext or ciphertext. Examples include timing [Koc96], power consumption (including differential power analysis [Koc98a]), NMR scanning, and electronic emanations.5 With many algorithms it is possible to reconstruct the key from these side channels. While total resistance to side-channel cryptanalysis is probably impossible, we note that Twofish executes in constant time on most processors.

5The NSA refers to this particular side channel as "TEMPEST."

Fault analysis [BDL97, BS97] can be used to successfully cryptanalyze this cipher. Again, we believe that total resistance to fault analysis is an impossible design constraint for a cipher. The resistance to fault analysis of any block cipher can be improved using classical tamper-resistance and fault-tolerance techniques (although see [AK96]).

In most cases, Twofish seems to be inherently resistant to timing analysis. However, it is worth noting that in certain special cases, implementers may need to take extra care to avoid the possibility of timing attacks. On platforms with extremely limited memory resources, it is natural to compute the multiplies over GF(28) on the fly without the aid of additional tables by using a shift-register-based approach. If this approach is to be followed, beware: timing attacks may be possible. The obvious way to clock a shift register in software is to shift it and then XOR in the feedback polynomial if the bit that shifted out was a one—but this algorithm does not execute in constant time, because the XOR operation is performed only if a certain data bit is set. Thus, this implementation pattern should be avoided. A better approach would be to XOR in either zero or the feedback polynomial, depending on the value of the bit just shifted out, thereby guaranteeing constant-time execution and resistance to timing analysis. This is probably only of concern to smart cards and other computing platforms without much memory.

9.11 Attacking Simplified Twofish

9.11.1 Twofish with Known S-boxes

As discussed above, our differential attack on Twofish with fixed S-boxes works for 6-round Twofish, and requires only 267 effort. A 7-round differential attack requires 2131 effort.

Additionally, much of Twofish's related-key attack resistance comes from the derivation of the S-boxes, so a related-key attack is much easier against a known S-box variant.

9.11.2 Twofish without Round Subkeys

We attack a Twofish variant without any subkeys, thus whose whole key material is in the key-dependent S-boxes. The attack requires about 233 chosen plaintexts; it breaks the Twofish variant with about 236 effort, even when the cipher has a 128-bit key. (Note that this is the amount of key material used to define the S-boxes in normal Twofish with a 256-bit key.)

It is obvious that a Twofish variant with fixed S-boxes and no subkeys would be insecure—there would be no key material injected. We develop a slide attack on Twofish with any number of rounds, and a 128-bit key. (The Twofish key would be 256 bits, but since we never use more than 128 bits of key in the key-dependent S-boxes, the effective key size is 128 bits.)

Note that this attack would not work if we used round subkeys that were simply counter values, as would happen if we used the identity permutation for all the key-scheduling S-boxes. We are not currently aware of any attack on such a Twofish variant.

Overview of the Attack. In a slide attack, we attempt to find a pair of plaintexts, (P, P*), such that P* has the same value as the intermediate value after one or more rounds of encrypting P. If we can find and expose this value, we gain direct access to the results of a single round of the cipher; we then attack that single round to recover the key.

Our attack has two parts:

1. We must first find a pair of plaintexts, (P, P*), such that P* is the result of encrypting P with one round.
2. We use this pair to derive four relations on g and thus determine the specific S-boxes used in g. This yields the effective key of the cipher.

Finding Our Related Pair of Texts. Twofish is a Feistel cipher, operating on two 64-bit halves in each block. We use the representation where the left half of the block is the input to the Feistel function, and the halves are swapped after each round. Let (L0, R0) represent a given pair of 64-bit halves input into the cipher, and let (LN, RN) represent the resulting ciphertext. To mount this attack, we need to find some (L0, R0, L1) such that L1 is the value of the left half of the block after the first round. To test whether we have such a pair, we can try encrypting (L0, R0) and (L1, L0). When we have a triple with the desired properties, we get (LN, RN) from the first encryption, and (LN+1 LN) from the second encryption.

We use a trick by Biham [Bih94] to get such a pair of plaintexts with only 233 chosen plaintexts: we choose a fixed L0, and encrypt it with 232 randomly selected R0 values as (L0, R0). We then encrypt the same L0 with 232 random R1, values, as, (R1, L0). By the birthday paradox, we expect to see one pair of values for which R1 = R0 ⊕ F(L0). To find this pair, we sort the ciphertexts from the first batch of encryptions on their left halves, and the ciphertexts from the second batch of encryptions on their right halves. When we find a match, the probability is good that we have a pair with the desired property.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Extracting g Values. Once we have this triple (L0, R0, L1), we also have two relations on F: F(L0) = R0 ⊕ L1 and F(LN) = RN ⊕ LN+1. Note, however, that we do not yet have direct g values. Instead, we have two instances of the results of combining two g outputs with a PHT. Since we have the actual PHT output values, we can simply undo the PHT, yielding relations on g. Our pair has given us two relations on F, and thus four relations on g.

Extracting the S-boxes. The g outputs are the result of applying four key-dependent S-boxes to the input bytes, and then combining those bytes with an MDS matrix multiply. Since the MDS multiply is invertible, we invert it to get back the actual S-box outputs for all four different values. If those values are all different, then we have four bytes of output from each S-box. We can try all 232 possible key input values for each S-box, and see which ones are consistent with the results; for most sets byte values, only one or two S-boxes will match them. We thus learn each key-dependent S-box in g, perhaps with a couple of alternative values. We try all possible alternatives against any plaintext/ciphertext values we have available, and very quickly recover the correct S-boxes. Since this is the only key material in this Twofish variant, the attack is done.

9.11.3 Twofish with Non-bijective S-boxes

We decided early in the design process to use purely bijective S-boxes. One rationale was to ensure that the 64-bit round function is bijective. That way, iterative 2-round differential characteristics cannot exist; when they do exist, they often result in the highest-probability multi-round characteristic, so avoiding them should help to reduce the risk of a successful differential attack. Also, attacks based on non-surjective round functions [BB95, RP95b, RPD97, CWSK98] are sure to fail when the 64-bit Feistel round function is bijective.6

6Vaudenay's attack on Blowfish took advantage of non-bijectivity in the Blowfish round function [Vau96a].

We argue here that this was a good design decision, by showing that a Twofish variant that uses non-bijective S-boxes is likely to be much easier to break.

Observe that when q0 and q1 are non-bijective, their 3-way composition into si is likely to be even more non-surjective than either q0 or q1 on its own. It is easily seen that the expected size of the range of a random function on eight bits (such as q0 or q1) is r1 = 1 - (1 - (1 - 1/256)256 ≈ 1 - e-1 ≈ 0.632. When we compose such a function twice, its expected range becomes r2 = 1 - (1 - 1/256)256r1, ≈ 1-e-r1 ≈ 0.469; and the expected size of the range of a 3-way composition will be rs ≈ 1 - e-r2 ≈ 0.374. In other words, we expect that only about 96 ≈ 0.374 × 256 of all possible 8-bit values can appear as the output of the S-box. Therefore, the output of the h function can attain only about 226.3 ≈ 964 possible values, and the 64-bit Feistel function can attain only about 252.6 of all 264 possible outputs. This is certainly a rather large certificational weakness. (This gets worse when the key size grows, since the number of compositions of the q functions gets larger.)

We point out a serious differential attack when using non-bijective random S-boxes. Consider the probability pΔx that a given input difference Δx yields a collision in the S-box output; i.e., Δx → 0. Let p = ςΔx≠0 pΔx/255 be the average probability over all non-zero input differences, and let m = maxΔx≠0 pΔx be the maximum probability over all non-zero input differences. We have E p = 3/256; also, Pr(p ≥ 2/256) ≈ 0.78, Pr(p ≥ 10/256) ≈ 0.02, and Pr(p ≥ 16/256) ≈ 0.0002. As for the distribution of m, empirically E m ≈ 11.7/256; experiments suggest Pr(m ≥ 10/256) ≈ 0.975, Pr(m ≥ 16/256) ≈ .04, and Pr(m ≥ 22/256) ≈ 0.0002.

Consider a Twofish variant with random non-bijective S-boxes and no rotations. We obtain a 2-round iterative differential characteristic with probability m, and thus a 13-round differential characteristic with probability m6.

We find that, for 97.5 percent of the keys, one can break the variant with about 228 chosen plaintexts; for 4 percent of the keys, it can be broken with 224 chosen plaintexts; and for a class of weak keys consisting of 0.02 percent of the keys, the variant cipher can be broken with roughly 221 chosen plain-texts. (Of course, one can trade off the number of texts needed against the size of the weak key class; the figures given are just examples of points on a smooth curve.)

Next we consider a Twofish variant with random non-bijective S-boxes, but with all rotations left intact. If we look at any 2-round differential characteristic whose first round is the trivial characteristic and whose second round involves just one active S-box, we expect its probability to be about p. One difficulty is that the rotations prevent us from finding an iterative 2-round characteristic. However, we can certainly piece together 6.5 different 2-round differential characteristics (each of the right form so it will have probability about p) to find a 13-round characteristic with expected probability p6. (The latter probability can probably be improved to about 3p6, due to the extra degrees of freedom, but as we are only doing a back-of-the-envelope estimate anyway, we will omit these considerations.) Thus, we can find a differential attack that succeeds with about 239 chosen plaintexts for the majority (about 78 percent) of the keys; also, for 2 percent of the keys, this variant can be broken with 228 chosen plaintexts; and for a class of weak keys consisting of 0.02 percent of the keys, the variant cipher can be broken with roughly 224 chosen plaintexts.

This analysis clearly shows the value of bijective S-boxes in stopping differential cryptanalysis. Also, it helps show the benefit of the rotations: they make short iterative characteristics much harder to find, thereby conferring (we hope) some additional resistance against differential attacks.

9.12 Trap Doors in Twofish

We assert that Twofish has no trap doors. As designers, we have made every effort to make Twofish secure against all known (and unknown) cryptanalysis, and we have made no effort to build in a secret way of breaking Twofish. However, there is no way to prove this, and not much reason for anyone to believe us. We can offer some assurances.

In this book, we have outlined all of the design elements of Twofish and our justifications for including them. We have explained, in great detail, how we chose Twofish's "magic constants": the RS code, q0, q1 and the MDS matrix. There are no mysterious design elements; everything has an explicit purpose. Moreover, we feel that the use of key-dependent S-boxes makes it harder to install a trap door into the cipher. As difficult as it is to create a trap door for a particular set of carefully constructed S-boxes, it is much harder to create one that works with all possible S-boxes or even a reasonably useful subset of them (of relative size 2-20 or so).

Additionally, any trap door would have to survive 16 rounds of Twofish. It would have to work even though there is almost perfect diffusion in each round. It would have to survive the pre- and post-whitening. These design elements have long been known to make any patterns difficult to detect; trap doors would be no different.

None of this constitutes a proof. Any reasonable proof of general security for a block cipher would also prove P ≠ NP. Rather than outlining the proof here, we would likely skip the AES competition and go collect our Fields Medal.

However, we have made headway towards a philosophical proof. Assume for a moment that, despite the difficulties listed above, we did manage to put a trap door into Twofish. This would imply one of two things:

One, that we have invented a powerful new cryptanalytic attack and have carefully crafted Twofish to be resistant to all known attacks but vulnerable to this new one. We cannot prove that this is not true. However, we can point out that as cryptographers we would achieve much more fame and glory by publishing our powerful new cryptanalytic attack. In fact, we would probably publish it along with this paper, making sure Twofish is immune so that we can profitably attack the other AES submissions.

The other possibility is that we have embedded a trap door into the Twofish magic constants and then transformed them by some means so that finding them would be a statistical impossibility (see [Har96] for some discussion of this possibility). The resulting construction would seem immune to current cryptanalytic techniques, but we as designers would know a secret transformation rule that we could apply to facilitate cryptanalysis. Again, we cannot prove that this is not true. However, it has been shown that this type of cipher, called a "master-key cryptosystem," is equivalent to a public-key cryptosystern [BFL96]. Again, as cryptographers we would achieve far greater recognition by publishing a public-key cryptosystem. that is not dependent on factoring [RSA78] or the discrete logarithm problem [DH76, E1G85, NIST94]. And the resulting algorithm's dual capabilities as both a symmetric and public-key algorithm would make it far more flexible than the AES competition requires (and hence an excellent choice for the standard).

There is a large gap between a weakness that is exploitable in theory and one that is exploitable in practice. Even the best attack against DES (a linear-cryptanalysis-like attack combining quadratic approximations and a multiple-approximation method) requires just under 243 plaintext/ciphertext blocks [SK98], which is equivalent to about 64 terabytes of plaintext/ciphertext encrypted under a single key. A useful trap door would need to work with much less plaintext—a few thousand blocks—or it would have to reduce the effective keyspace to something on the order of 272. We believe that, given the quality of the public cryptanalytic research community, it would be impossible to put a weakness of this magnitude into a block cipher and have it remain undetected through the AES process. And we would be foolish to even try.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 10
Using Twofish

10.1 Chaining Modes

All standard block-cipher chaining modes work with Twofish: CBC, CFB, OFB, counter [NBS80]. We axe aware of no problems with using Twofish with any commonly used chaining mode. (See [Sch961 for a detailed comparison of the various modes of operation.) A cryptanalyst considering OFB-, CFB-, or CBC-mode encryption with Twofish may collapse the pre- and post-whitening of key material into a single XOR, but he does not appear to benefit much from this.

10.2 One-Way Hash Rinctions

It is not hard to build a one-way hash function out of Twofish [Win84]. We suggest the following construction, taken from [Pre93]:

Hi = Mi ⊕ EHi-1 (Mi)

This mode seems to be somewhat preferable to Davies-Meyer, because it gives the cryptanalyst less control over the value entering the key port and thus may provide more robust protection in case there are any related-key attacks on Twofish.

We believe that Twofish, with its strong key schedule, can be used securely in any of these formats. Note, however, that the key schedule has been analyzed mainly for related-key attacks, not for the class of chosen-key attack that hash functions must resist. Additionally, the 128-bit block size makes straightforward use of this construction useful only when collision-finding attacks can be expected to be unable to try 264 trial hashes to find a collision.

As keys that are non-standaxd sizes have equivalent keys that are longer, any use of Twofish in a hashing construction must ensure that only a single key length is used. We recommend using 128-bit keys for hashing constructions.

10.3 Message Authentication Codes

Twofish can be easily used to build a very strong message authentication code using the CBC-MAC construction [BKR94]. In fact, [BKR94] gives formal proofs of security for the CBC-MAC approach, assuming that the underlying cipher is secure. We feel that this makes Twofish-CBC-MAC a very strong candidate for use in message authentication applications.

Alternatives include using Twofish as a hash function, and then constructing a MAC out of the hash function [BCK96].

10.4 Pseudorandom Number Generators

Twofish can also be used as a primitive in a pseudorandom number generator suitable for generating session keys, public-key parameters, protocol nonces, and so on [Plu94, KSWH98a, Gut98, KSWH98c].

10.5 Larger Keys

Even though it would be straightforward to extend the Twofish key schedule scheme to larger key sizes, there is currently no definition of Twofish for key lengths greater than 256 bits. We urge caution in trying to extend key lengths; our experience with Twofish has taught us that extending the key length can have important security implications.

10.6 Additional Block Sizes

There is no definition of Twofish for block lengths other than 128 bits. While it may be theoretically possible to extend the construction to larger block sizes, we have not evaluated these constructions at all. We urge caution in trying to extend the block size; many of the constructions we use may not scale well to 256 bits, 512 bits, or larger blocks.

10.7 More or Fewer Rounds

Twofish is defined to have 16 rounds. We designed the key schedule to allow natural extensions to more or fewer rounds if and when required. We strongly advise against reducing the number of rounds. We believe it is safe to increase the number of rounds, although we see no need to do so. Note that the key schedule cannot support more than 124 rounds, as the 8-bit input counter to the key schedule S-boxes overflows.

Algorithm Key Length Width (bits) Rounds Cycles Clocks/ Byte
Twofish | variable | 128 | 16 | 8 | 16.1
Blowfish | variable | 64 | 16 | 8 | 19.8
Square | 128 | 128 | 8 | 8 | 20.3
RC5-32/16 | variable | 64 | 32 | 16 | 24.8
CAST-128 | 128 | 64 | 16 | 8 | 29.5
DES | 56 | 64 | 16 | 8 | 43
Serpent | 128,192, 256 | 128 | 32 | 32 | 45
SAFER (S)K-128 | 128 | 64 | 8 | 8 | 52
FEAL-32 | 64, 128 | 64 | 32 | 16 | 65
IDEA | 128 | 64 | 8 | 8 | 74
Triple-DES | 112 | 64 | 48 | 24 | 116
Table 10.1. Performance of Different Block Ciphers (on a Pentium)

More and more recent ciphers are being defined with a variable number of rounds; e.g., SAFER-K64 [Mas94], RC5 [Riv95], and SPEED [Zhe97]. This means that it is impossible to categorically state that a given cipher construction is insecure: there might always be a number of rounds n for which the cipher is still secure. However, while this might theoretically be true, this is not a useful engineering definition of "secure." After all, the user of the cipher actually has to choose how many rounds to use. In a performance-driven model, it is useful to compare ciphers of equal speed, and compare their security, or compare ciphers of equal security and compare their speeds. For example, FEAL-32 is secure against both differential and linear attacks. Its speed is 65 clock cycles per byte of encryption, which makes it less desirable than faster, also secure, alternatives. As speed is relatively easy to measure and quantify, the simplest way of comparing ciphers in a performance-driven model is to choose the number of rounds in such a way as to make the two ciphers equally fast. The hard question of "which cipher is more secure" then remains, but at least that is a clearly defined problem.

With that in mind, Table 10.1 gives performance metrics for block and stream ciphers on the Pentium processor.1 Similar comparisons are being done for the AES candidates [SKW+99a], and will presumably be one of the criteria in the selection process.

1These metrics are based on theoretical analyses of the algorithms and actual hand-tooled assembly-language implementations [SW97, PRB98a, PRB98b].


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

10.8 Family Key Variant: Twofish-FK

We often see systems that use a proprietary variant of an existing cipher, altered in some hopefully security-neutral way to prevent that system from interoperating with standard implementations of the cipher. A family key is a way of designing this into the algorithm: each different family key is used to define a different variant of the cipher. In some sense, the family key is like an additional key to the cipher, but in general, it is acceptable for the family key to be very computationally expensive to change. We would expect nearly all Twofish implementations that used any family key to use only one family key.

Our goals for the family key algorithm are as follows:

• No family key variant should be substantially weaker than the original cipher.
• Related-key attacks between different unknown but related family keys, or between a known family key and the original cipher, should be hard to mount.
• The family key should not merely reorder the set of 128-by-128-bit permutations provided by the cipher, it should change that set.

A Twofish family key is simply a 256-bit random Twofish key. This key, FK, is used to derive several blocks of bits, as follows:

1. Preprocessing Step:
This step is done once per family key.
a) T0 = FK.
b) T1 = (EFK(0), EFK(1)) ⊕ FK.
c) T2 = EFK(2).
d) T3 = EFK(3).
e) T4 = First 8 bytes of EFK(4).

Note that, using our little-endian convention, the small integers used as plaintext inputs should occur in the first plaintext byte.
2. Key Scheduling Step: This step is done once per key used under this family key.
a) Before subkey generation, T0 is XORed into the key, using as many of the leading bytes of T0 as necessary.
b) After subkey generation, T2 is XORed into the pre-whitening subkeys, T3 is XORed into the post-whitening subkeys, and T4 is XORed into each round's 64-bit subkey block.
c) Before the cipher's S-boxes are derived, T1, is XORed into the key. Once again, we use as many of the leading bytes of T1, as we need. Each byte of the key is then passed through the byte permutation q0, and the result is passed through the RS matrix to get S. Note that the definition of T1, means that the effect of the first XOR of FK into the key is undone.

Note the properties of the alterations made by any family key:

• The keyspace is simply permuted for the initial subkey generation.
• The subkeys are altered in a simple way. However, there is strong evidence that this alteration cannot occur by changing keys, based on the same difference sequence analysis used in discussing related-key attacks on Twofish.
• The S-boxes used in the cipher are altered in a powerful way, but one which does not alter the basic required properties of the key schedule. Getting no changes in the S-boxes used in the cipher still requires changing at least five bytes of key, and those five bytes of key must change five of the S-boxes used for subkey generation.

10.8.1 Analysis

Effects of Family Keys on Cryptanalysis. We are not aware of any substantial difference in the difficulty of cryptanalyzing the family key version of the cipher rather than the regular version. The cipher's operations are unchanged by the family key; only subkey and S-box generation are changed. However, they are changed in simple ways; the S-boxes are generated in exactly the same way as before, but the key material provided to them is processed in a simple way first; the round subkeys are generated in the same way as before (again, with the key material processed in a simple way first), and then have a constant 64-bit value XORed in. Related-key attacks of the kind we have been able to consider are made slightly harder, rather than easier, by this 64-bit value. The new constants XORed into the whitening subkeys simply permute the input and output space of the cipher in a very simple way.

Related Keys across Family Keys. Related-key attacks under the same family key appear, as we said above, to be at least as hard as related-key attacks in normal Twofish. There is still the question, however, of whether there are interesting related keys across different family keys. It is hard to see how such related keys would be used, but the analysis may be worth pursuing anyway.

An interesting question, from the perspective of an attacker, is whether there ar e pairs of keys that give identical subkeys except for the constant values XORed into them, and that also give identical S-boxes. By allowing an attacker to put a constant XOR into all round subkeys, such pairs of keys would provide a useful avenue of attack.

This can be done by finding a pair of related family keys, FK, FK*, that are identical in their first 128 bits, and a pair of 128-bit cipher keys, M, M*, such that M generates the same set of S-boxes with FK that M* does with FK*. For a random pair of M, M* values, this has probability 2-64 of happening, Thus, an attacker given complete control over M, M* and knowledge of FK, FK* can find such a pair of keys and family keys. However, this does not seem to translate into any kind of clean related-key attack; the attacker must actually choose the specific keys used.

We do not consider this to be a valid attack against the system. In general, related-key attacks between family keys seem unrealistic, but one which also requires the attacker to be able to choose specific key values he is trying to recover is also pointless.

A Valid Related-key Attack. An attacker can also try to find quadruples FK, FK*, M, M* such that the subkey generation and the S-box generation both get identical values. This requires that

T0 ⊕ M = T0* ⊕ M*

T1 ⊕ M = T1* ⊕ M*

If FK, FK* have the property that

T0 ⊕ T0* = T1 ⊕ T1* = δ

then related-key pairs M, M ⊕ Ϙ will put a fixed XOR difference into every round subkey, and may allow some kind of related-key attack. Again, we do not think this is an interesting attack; the attacker must force the family keys to be chosen this way, since the probability that any given pair of family keys will work this way is (for 128-bit cipher keys) 2-128. We do not expect such relationships to occur by chance until about 264 family keys are in use.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 11
Historical Remarks

Twofish originated from an attempt to take the original Blowfish design and modify it for a 128-bit block. We wanted to leverage the speed and good diffusion of Blowfish, while also improving it where we could. We wanted the new cipher to have a bijective F function, a much more efficient key schedule, and to be implementable in custom hardware and smart cards in addition to 32-bit processors (i.e., have smaller tables). And we wanted it to be even faster than Blowfish (per byte encrypted), if possible.

Initial thoughts were to have the Blowfish round structure operate on the four 32-bit subblocks in a circular structure (an incomplete Feistel network), but there were problems getting the diffusion to work in both the encryption and decryption directions. Having two parallel Blowfish round functions and letting them interact via a two-dimensional Feistel structure ran into the same problems. Our solution was to have a single Feistel structure with two Blowfish-like 32-bit round functions and to combine them using a PHT (an idea stolen from SAFER). This idea also provided nearly complete avalanche during the round.

Round subkeys are required to avoid slide attacks against identical round functions. We used addition instead of XOR to take advantage of the Pentium LEA opcode and implement them in effectively zero time.

We used 8-by-8-bit S-boxes and an MDS matrix (an idea stolen from Square, although Square uses a single fixed S-box) instead of random 8-by-32-bit S-boxes, both to simplify the key schedule and ensure that the g function is bijective. This construction ensured that Twofish would be efficient on 32-bit processors (by precomputing the S-boxes and MDS matrix into four 8-by-32-bit S-boxes) while still allowing it to be computed on the fly in smart cards. And since our MDS matrix is only ever computed in one direction, we did not have to worry about the matrix's efficiency in the reverse direction (which Square had to consider).

The construction also gave us considerable performance flexibility. We worked hard to keep this flexibility, so implementers would have a choice of how much key pre-processing to do depending on the amount of plaintext to be encrypted. And we tried to maintain these tradeoffs for 32-bit microprocessors, 8-bit microprocessors, and custom hardware.

Since one goal was to be able to keep the complete design in our heads, any complication that did not have a clear purpose was deleted. Additional complications we chose not to introduce were key-dependent MDS matrices or round-dependent variations on the byte ordering and the PHT. We also toyed with the idea of swapping 32-bit words within the Feistel halves (something we called the "twist"), but abandoned it because we saw no need for the additional complexity.

We did keep in the one-bit rotations of the target block, primarily to reduce the vulnerability to any attack based on byte boundaries. The particular manifestation of this one-bit rotation was due to a combination of performance and cryptanalytic concerns. Larger rotations would be slower in some implementations, and simpler constructions were less secure.

We considered using all the key bytes, rather than just half, to define the key-dependent S-boxes. Unfortunately, this made the key setup time for high-end machines unreasonably large, and also made encryption too slow on low-end machines. By dropping this to half the key bits, these performance figures were improved substantially. By carefully selecting how the key bytes were folded down to half their size before being used to generate the cipher's S-boxes, we were able to ensure that pairs of keys with the same S-boxes would have very different subkey sequences.

The key schedule gave us the most trouble. We had to resist the temptation to build a cryptographic key schedule like the ones used by Blowfish, Khufu, and SEAL, because low-end implementations needed to be able to function with minimal additional memory and, if necessary, to compute the subkeys as needed by the cipher. However, a simple key schedule can be an important weak point in a cipher design, leaving the whole cipher vulnerable to partial-key guessing attacks, related-key attacks, or to attacks based on waiting for especially weak keys to be selected by a user. Though our final key schedule is rather complex, it is conceptually much simpler than many of our intermediate designs. Reusing many of the primitives of the cipher (each round's subkeys are generated by a computation nearly identical to the one that goes on inside the round function, except for the specific inputs involved) made it possible to investigate properties of both the key schedule and of the cipher at the same time.

We spent considerable time choosing q0 and q1. Since these provide the primary non-linearity in the cipher, they had to be strong. We wanted to be able to construct them algebraically for applications where storing 512 bytes of fixed tables was not possible. In the end, we built permutations from random parameters, and tested the permutations against our required criteria.

The name "Twofish" reared its head late in the design process. We tried various names beginning with "Blow" or ending with "fish." The name Twofish was chosen for several reasons: it is traditional to name ciphers after sea creatures, and animals in general; no one liked "Blowfish II" the round function originally derived from two Blowfish round functions in parallel; and of course, there is Dr. Seuss [Seu60].1

1For readers who did not grow up in North America: Dr. Seuss was a famous writer of children's books, one of which is called One Fish, Two Fish, Red Fish, Blue Fish. The marketing tie-ins were too lucrative to ignore.

And finally, we cryptanalyzed Twofish. We cryptanalyzed and cryptanalyzed and cryptanalyzed, right up to the morning of the submission deadline. Since then we have cryptanalyzed more, and are continuing to do so. There's no stopping.


The Twofish Encryption Algorithm-A 128-Bit Block Cipher

 The Twofish Encryption Algorithm-A 128-Bit Block Cipher - Bruce Schneier et al

Chapter 12
Conclusions and Further Work

We have presented Twofish, the rationale behind its design, and the results of our initial cryptanalysis. Design and cryptanalysis go hand in hand—it is impossible to do one without the other—and it is only in the analysis that the strength of an algorithm can be demonstrated.

During the design process, we learned several lessons about cipher design:

• The encryption algorithm and key schedule must be designed in tandem; subtle changes in one affect the other. It is not enough to design a strong round function and then to graft a strong key schedule onto it (unless you are satisfied with an inefficient and inelegant construction like Blowfish has); both must work together.
• There is no such thing as a key-dependent S-box, only a complicated multistage non-linear function that is implemented as a key-dependent S-box for efficiency.
• Keys should be as short as possible. It is much harder to design an algorithm with a long key than an algorithm with a short key. Throughout our design process, we found it easier to design and analyze Twofish with a 128-bit key than Twofish with a 192-or 256-bit key.
• Build a cipher with strong local encryption and let the round function handle the global diffusion. Designing Twofish in this manner made it very hard to mount any statistical cryptanalytical attacks.
• Consider performance at every stage of the design. Having a code optimization guru on our team from the beginning drastically changed the way we looked at design tradeoffs, to the ultimate benefit of Twofish.
• Analysis can go on forever. Even after the 15 June 1998 AES submission deadline, we are still cryptanalyzing Twofish.

We believe Twofish to be an ideal algorithm choice for AES. It is efficient on large microprocessors, smart cards, and dedicated hardware, The multiple layers of performance tradeoffs in the key schedule make it suitable for a variety of implementations. And the attention to cryptographic detail in the design—both the encryption function and the key schedule—make it suitable as a codebook, output-feedback and cipher-feedback stream cipher, one-way hash function (using standard techniques for converting block ciphers into hash functions), and pseudorandom number generator.

We welcome any new cryptanalysis from the cryptographic community. We plan on continuing to evaluate Twofish all through the AES selection process and beyond. Specifically:

• Whether the number of rounds can safely be reduced. At this point our best non-related-key attack—a differential attack—can only break five rounds. If no better attacks are found after a few years, it may be safe to reduce the number of rounds to 14 or even 12. 12-round Twofish can encrypt and decrypt data at about 220 clock cycles per block on a Pentium, Pentium Pro, and Pentium II.
• Whether there are alternative fixed tables that increase security. We have chosen both the MDS matrix and the fixed permutations, q0 and q1, to meet our mathematical requirements. In the event we find better constants that make Twofish even harder to cryptanalyze, we may want to revise the algorithm.
• Whether we can define a Twofish variant with fixed S-boxes. This variant would have a faster key setup time than the algorithm presented here—about 1200 clock cycles on a Pentium Pro—and the same encryption and decryption speeds. Although we cannot break Twofish with fixed S-boxes, further research is required on what the fixed S-boxes would look like, and how much data could be safely encrypted with this variant.
• Whether we can improve our lower bounds on the complexity of a differential attack and a linear attack.

Developing Twofish was a richly rewarding experience, and one of our most satisfying cryptographic projects to date. There is an enormous gap between simply creating a cryptographic primitive and completing a full design process, and we have learned much by completing the process.