Thinking About Encryption, Part 50


Approaching Monome-Dinome

ND 2018 and JF 2019 both had Monome-Dinome CONs, and I misremembered having a VBScript solver for M-D. I was going to solve this one last cipher for both issues of the Cm, and then, really, take a break to do my chores. Turned out, though, that the earlier script was for Amsco, which also used the “monome” “dinome” terminology. By then, it was too late. I looked up the description for the cipher-type on the ACA website, and I found an article in the ACA Cm archives on how to solve M-D. The next day, I was at the school I teach at, and during one of the breaks I sat down and tried working on the ND 2018 M-D CON, because I didn’t have anything else to read. A little later, I had a longer break, and I solved ND2018 all the way up to almost getting the key. By the end of the day, I’d extracted the key, too. The following day was a rainy Sunday, so I went to a coffee shop and focused specifically on the JA2019 M-D CON. I had the key for that one 2 hours later, ignoring the fact that I was doing all my work in ink on paper because I couldn’t be bothered to pull my pencil out of my backpack.

Monome-Dinome is a simple monoalphabetic substitution cipher that uses a 3×8 grid to create the substitution table, either to the digits 0-9, or 10 random letters of your choice. Since the grid only holds 24 letters, 2 letters need to be doubled up (generally J with I, and either V with W, or Z with Y, but it could be anything, depending on the author’s discretion). I’ll use numbers, and double up I and J, and Y and Z. First, pick the order of 8 of the 10 digits and write them over the top of the grid. Use the remaining two digits as row selectors for the grid. Also, pick a keyword for the alphabet, write that into the grid and fill in the rest of the grid with the other letters.

. 71029645
----------
.|TEACHRBD
3|FGIKLMNO
8|PQSUVWXY

Write out your plaintext message:

T H I S I S A T E S T I T I S O N L Y A T E S T

Next, use just the single digits for the top row of letters from the grid (the monome), and the row selector plus column number for the letters in the other two rows (the dinome).

7 9 30 80 30 80 0 7 1 80 7 30 7 30 80 35 34 39 85 0 7 1 80 7

Finally, pull out all the spaces, and order the numbers in groups of five (out of tradition, and also because it makes the message harder to crack).

79308 03080 07180 73073 08035 34398 50718 07

Deciphering is the reverse process. First, make the grid, and fill in the key alphabet. Write out the ciphertext, and separate the numbers into 1 and 2-digit pairs. Finally, substitute the numbers for the corresponding letters in the grid.

Cracking M-D first requires that you identify the two digits used for the row selectors. If you look at the grid carefully, you’ll see that a selector can not be combined with itself (i.e. – 33, and 88 are illegal pairs), and it can not combine with the other selector (i.e. – 38 and 83 are illegal pairs).

Now, in my test cipher, the only doubled-up digit is 00, so we need to see what the pairings are. We do have a “93” and “39” combination, “03” and “30”, “08” and “80”, “07”, etc. But, the message is too short to draw other conclusions. So we need to do a frequency count.

0 - 11 (Invalid because of 00 pairing)
1 - 2
2 - 0
3 - 7
4 - 1
5 - 2
6 - 0
7 - 6
8 - 6
9 - 2

Because the two selectors pair generally with the other 8 digits, they should appear more frequently than anything else. Unfortunately, we still have three strong candidates – 3, 7, 8. Of the three, 3 is the most common, so let’s flip a coin and use 3 and 7. Break up the message into 1- and 2-digit units.

79 30 8 0 30 8 0 0 71 8 0 73 0 73 0 8 0 35 34 39 8 5 0 71 8 0 7

Yes! A clear reason to invalidate 7. Actually two clear reasons. First, it’s pairing with 3 (73), and second it appears at the end of the message unpaired. That leaves me with 3 and 8 as the selectors. Let’s try this again.

7 9 30 80 30 80 0 7 1 80 7 30 7 30 80 35 34 39 85 0 7 1 80 7

This is now a Patristocrat (the ACA’s name for newspaper-like Crypto Quips minus the word spacing and punctuation). Let’s supply a crib – “it is”. This gives us a filter of “1_1_”. Or, the same letter has to appear somewhere in the text separated by something that itself isn’t duplicated within the 4-character segment.

Possible candidates are: 30 7 30 80 and 80 30 80 0.

We need to do another frequency count.
7 9 30 80 30 80 0 7 1 80 7 30 7 30 80 35 34 39 85 0 7 1 80 7

0 - 2
1 - 2
7 - 6
9 - 1
30 - 4
34 - 1
35 - 1
39 - 1
80 - 5
85 - 1

Following the ETAOIN SHRDLU pattern, “E” would generally be the most common letter, but since we already know what the plaintext is, we also know that things are kind of out of skew. But, we should still expect “T” to be a little more frequent than “I”. With “30 7 30 80”, 7 outperforms 30 at 6 to 4. while with “80 30 80 0”, it’s 80 for 5 and 30 for 4. We’d also expect “S” to slightly outperform “I”, and we don’t get that with “80 30 80 0” (0 only shows twice). The better bet for our crib right now is “30 7 30 80.”

T 9 I S I S 0 T 1 S T I T I S 35 34 39 85 0 T 1 S T

For the first word, “T9IS”, the first obvious choice would be 9 = “H”. But that only appears once. We do get “0T1ST” twice, and that could be “something-TEST”. “IS something TEST” might be “IS A TEST”, so let’s try that (A = 0, 1 = E).

T H I S I S A T E S T I T I S 35 34 39 85 A T E S T

From here, we don’t have enough information to pin down “35 34 39 85”. So, let’s look at the grid.

. 01245679
----------
.|AE....TH
3|I
8|S

And now we really are at a deadend because we don’t have that many letters in the alphabet to work with. But, we can make maybe one or two observations. First, the last row is going to include RSTUVWXY unless one or more of the letters are in the keyword. We can already see that “T” has to be in the key, while “S” isn’t. The other upper end letters aren’t either because they’re pretty low frequency. That means that the “S” column may be shifted two places to the right. Like so:

. 12045679
----------
.|E.A...TH
3|..I
8|..S

I’d fill in UVWXY, except I don’t know which letters pair with “T” and “H”. Notice, though, that “A” is in the middle of the key. Also, there are 8 + 2 letters until we get to “I”, and “I” is the 8th letter in the alphabet. Therefore, the key has to use 2 letters higher up in the alphabet than “I”. We already have “T”, and we think we’ve placed “S”. We’re not using “J”, and that just leaves K-R. We don’t think “I” is in the key, unless it’s a really long key. There are 7 letters in the grid between “I” and “S”, and normally there are 8 letters there. Anagramming “EATH” + “BCD” + one letter from “KLMNOPQR” will give us “EARTH” and “TEACHER” (drop the second “E” for “TEACHR”). We would like “A” to be the 3rd letter in the key, and we do get that with “TEACHER”. If we rearrange the grid again, we get:

. 71049562
----------
.|TEACHRBD
3|FGIKLMNO
8|PQSUVWXY

Yes, the numbers on the top row are speculative for “C”, “R”, “B” and “D”, but this is the best we can do with such a short plaintext message. Even so, I don’t think I cheated all that much by knowing the original key alphabet. I did make a few leaps, but they were probably justifiable.

BUT… The word we couldn’t place, “35 34 39 85”, uses three letters from “KLMNO” and one from “UVWXY”. Further, 35 and 85 are in the same column. Surprisingly, 39 aligns with “L” (we’ve placed the 9 column because it includes “H”). That means our word could be M?LW, N?LX or O?LY. If we assume that it is “ONLY”, it does fit in with our plaintext message, and maybe we can finish off the number key after all.

. 71049245
----------
.|TEACHRBD
3|FGIKLMNO
8|PQSUVWXY

In any event, Monome-Dinome is solvable with pencil and paper, and I’m not seeing a reason for writing up a solver for it just yet. I may consider a tool for doing the digit counting for me, and for testing for legal row selectors, but it’s not a priority. When the next M-D CON comes out, I’ll try Bion’s solver first, and then see where that takes me.

Now, I have no excuse. Chores. Sigh.

Summary:
1) Monome-Dinome is a simple single-alphabet substitution cipher.
2) Pick a keyword, remove duplicate letters, and then complete the alphabet with the other letters, minus J and maybe V or Z.
3) Pick an 8-digit number (0-9) with no repeating digits and write them above a 3×8 grid. Write the remaining 2 digits to the left of the bottom two rows of the grid, then fill in the grid with the 24-letter alphabet.
4) Write down the plaintext message and encrypt it with the single digits for the top row of the grid, or the row selector and column digit for the bottom two rows. Remove the spaces between numbers and order them in groups of 5.
5) For decrypting, just reverse the process.
6) To solve M-D, do a frequency count on the digits, with the expectation that the 2 highest values will be the row selectors. Of the two selectors, the one with the lower frequency will probably represent the bottom row of the grid (because of the presence of so many low-frequency letters, like Q, V and X.
7) The list of row selectors can be narrowed down by eliminating any digits that appear in pairs in the plaintext (i.e. – 11 or 55). Also, no selector will be paired with the other selector (if you have “83”, and “8” is a selector, “3” can’t be the other selector.)
8) Once you have the selectors identified, solve the CON as a Patristocrat.
9) You can extract the keyword and grid numbering by putting the letters in the lower two rows into ascending order.
10) M-D is not particularly secure, and if you have a crib with repeated letters (i.e. – “letter” or “river,”) it’s pretty easy to solve. It can be hardened by completely randomizing the alphabet, and by applying something like Rail Fence to the finished cipher.

Wiggles, part 46


Just a little ongoing story to give you something to play with until the next blog post.

E MEMG’X HSLOOV OEWS XPS EMSL UT TUOOUZEGR LTXSH XPS XPEGR, DUV, ZPLXSBSH, DFX E PLXSM XPS EMSL UT RUEGR XPHUFRP XPS CLQQLRS SAEX LGM MUUH PLXIP SBSG NUHS. DSQEMSQ, XPSHS ZLQ L TLEGX PUCS XPLX XPSHS NERPX DS NUHS XPLG UGS ZLV UFX UT PSHS. ZPEOS XPS CLQQLRS XU XPS DLH ZLQ IOSLG, LGM IOSLGOV IFX, LGM XPS OLXIP UG XPS MUUH ZLQ EG RUUM IUGMEXEUG, XPS CLZ CHEGXQ EGMEILXSM HSOLXEBSOV IUGQEQXSGX XHLTTEI XPHUFRP PSHS, ZPEIP QUNSUGS EG XPS DLH ZUFOM PLBS QCUXXSM DSTUHS XPEQ ET LOO UT EX PLM DSSG XPHUFRP XPS RHSSG HUUN. E RHEXXSM NV XSSXP, OEXSHLOOV – MLNG QLGM – LGM WSCX IHLZOEGR TUHZLHM.

Cm History, 1932-1940


I bought an Android tablet about a year ago, and I got so irritated with how clunky the interface was, and how difficult it was to write apps in Java, that I tossed it on a stack of stuff and let it gather dust. I have a cell phone and a good pocket camera, and I don’t text or listen to music when I’m outside, so the tablet didn’t really add anything. More recently, though, I’ve been wanting a way to read archived issues of the ACA Cm newsletter while I’m riding the streetcar, or between English classes, and I was wondering if the Android natively supported a PDF reader. By this point I’d forgotten my login password and I didn’t remember where I wrote it down (if I ever did). So, I did a hard reboot, and spent several days trying to find a secure Wifi hotspot site so I could access the net and reset the password (I don’t need wifi at home). When that was over, I downloaded a test archive PDF from the ACA website, and I could read that. So, I downloaded every archive file from 1932 to 2015, and began reading. (Please note, I may not have all of the below information 100% correct, but this is as good as I can get it right now.)

Actually, I’m only up to 1940, but that’s good enough for the moment. For background, the American Cryptogram Association first formed in 1929 as a group of friends that had played card games together, but wanted something else to do when they couldn’t get 4 partners together for Bridge. They investigated various kinds of puzzles, such as crosswords, and hit on newspaper-style crypto quips as their favorite type. They registered the name American Cryptogram Association (ACA) and printed their first newsletter, the Cryptogram (The Cm), in Feb., 1932. Membership was $1 for one year.

The ACA was joining the ranks of other groups such as the National Puzzler’s League (1883-, mostly word games and wordplay; newsletter is The Enigma), Detective Story Magazine (1915-1949, which had a Cipher Secrets corner), and a couple others which seem to have faded into the mists of time. Some of the ACA members came from the NPL, so some of the vocabulary overlapped the two groups (members are called the Krewe, and aliases are “noms”). The ACA leaders felt that crypto quips were “the aristocrats of puzzles,” and were thus dubbed “aristocrats” to differentiate them from other cipher types.

1932

Feb.
– First issue of the Cm released. All ciphers in the issue are Aristocrats.

April
– The term “the Krewe” for the membership is introduced. Martin Gardner joins briefly.

June
– First attempt to introduce rules for how to construct Aristocrats for submission.

Aug.
– First start at trying to explain how to solve Aristos, introduction of the Phillips System, and the Porta Slide.

Oct.
– First mention of “undivided substitutions,” with the process for solving them. First appearance of undivided substitution crypts as an alternative to Aristos.

1933

April
– First appearance of a French Aristo. More foreign-language crypts start showing up after this, in French and Spanish.

June
Herbert Yardley ((1889-1956) former head of the Army Signal Corps intelligence MI8 department, creator of the Black Chamber, and author of The American Black Chamber) joins the ACA and is immediately named Vice President, with the nom “Bozo.”
– Joining at the same time is Helen Fouche Gaines, with the nom “Piccola” (1888-1940). Helen had been working as an author for other puzzle magazines at the time, and became one of the ACA’s most prolific writers in the Cm, introducing a wide variety of cipher types over the next 7 years, and showing how to solve them. Her entry in the Arkansas Biography: A Collection of Notable Lives states that there’s a very good likelihood that she served as a cryptographer/trainer for the U.S. Navy during WW I. She eventually published one of the best books on the subject, Elementary Cryptanalysis to that date in 1939. The book is still recommended by the ACA to members new to the subject. She passed away 1 year later.

Aug.
– Introduction of “bi-literals,” Aristocrats enciphered in such a way that the ciphertext is also readable English words. Introduction of the book review section.

Dec.
– Piccola introduces the first transposition cipher type, which is later called Route Transposition. Transposition CONs start appearing in the Cm after this.

1934

Feb.
– One reader asks for Fletcher Pratt’s address to ask him about Nihilist and Grille ciphers. Piccola writes the first true article on vowel spotting and digraphs in undivided substitutions.

Apr.
– Piccola introduces a method for finding repeated sequences. Transpositions are now a regular column in the CONs section. Foreign language CONs are still mostly Aristos, but now include German and Portuguese.

Aug.
– Italian is added to the foreign CONs.

Oct.
– Piccola introduces Null ciphers.

Dec.
– Piccola introduces 5×5 Checkerboard ciphers using numbers for the row and column identifiers. A user named Patristocrat starts showing up more often in the Cm.

1935

Feb.
– One article introduces the Bazeries cipher. Piccola introduces the Vigenere, and the first set of Vigenere CONs to appear in the Cm. A Challenges column is added to include harder CONs.

April
– One of the members starts writing about Bacon, Shakespeare and ciphers. The problem up to this point is that a certain group of people that felt Bacon wrote Shakespeare’s plays had tended to be borderline crazy in coming up with conspiracy theories, and the ACA members viewed them as nutjobs. Also in this issue, Piccola writes about how to solve Vigenere ciphers.

June
– Piccola introduces the Kasiski method for obtaining the period length of Vigeneres. Bozo writes an article on recovering key alphabets (does not include Aristos. Up to this point, people do not seem to be using key alphabets for encrypting Aristos or “undivided substitutions.”)

Aug.
– We’re given the letter frequencies of different languages.

Oct.
– Piccola introduces the Gronsfeld cipher.

Dec.
– An article appears on the solution of one of Benjamin Franklin’s enciphered messages that had up to that point been unbroken.

1936

Feb.
– Piccola introduces the Saint Cyr slide.
– A new CONs section is specifically dedicated to “undivided substitutions” (Aristos without the punctuation or word breaks), and 2 of the 3 CONs are provided by the user Patristocrat.

Apr.
– First appearance of a cryptarithm (math division problem with letters substituted for the numbers). For the next year or so, these kinds of problems can have multiple solutions, and several members would compete to see who could find the most possible.

Sept.
– This is actually a special cipher contest issue, and just consists of challenge ciphers.

Oct.
– Piccola introduces the Running Key Cipher. At this point, running key is a Vigenere cipher, using a long passage from a book as the key. The current implementation where one half of the plaintext is used to encrypt the other half does not appear until later.

Dec.
– Piccola introduces the Auto Key Cipher.

1937

Feb.
– Occasionally, one or another member will discuss books that have caught their interest, but there’s still no real review section yet. Other times, someone will attempt to compile a comprehensive list of books that either analyze ciphers, or incorporate them in the story.
– In this issue we get one of the “comprehensive lists,” which includes Bram Stoker’s “The Mysteries of the Sea,” with it’s Bacon cipher on pages 465-475.
– Piccola writes more on Auto Key.
– There’s an announcement of the new Cipher Exchange department, to hold the ciphers discussed by Piccola.

Apr.
– Piccola talks about the use of parallel alphabets for periodic ciphers (like Porta and Vigenere), and we get the first appearance of the Cipher Exchange (now a common feature of the modern version of the Cm).

June
– Piccola writes about recovering the primary alphabets from Vigenere and other periodic ciphers.
– Columnar Transposition starts showing up by name in the Cipher Exchange.

Aug.
– Piccola introduces the Nihilist Substitution Cipher.

Oct.
– Piccola writes more on the idea of key phrases for key alphabets.

Dec.
– Piccola delves deep into Columnar Transposition, with Part 1 here, and Part 2 in the Feb., 1938, issue.

1938

Feb.
– The encrypted math problems are now officially renamed to Cryptarithms.
– Up to this point, the user Patristocrat (Webb C. Patterson) had been almost the sole supplier of undivided simple substitution CONs, and the Cm editor claims that because he has a backlog of over 500 of these from Webb, that from here on Undivided Simple Subs will be called “Patristocrats.”

Apr.
– Nap writes an article on Alphabet Slides and their uses. This includes instructions on making cardboard slides with sets of alphabets on them for easy encryption and decryption of periodic ciphers. Good for solving the Vigenere family by hand.

June
– Piccola introduces the Nilihist Cipher.

Aug.
– Piccola writes about the bigram test, which is useful for breaking Columnar and Nihilist Transpositions, and Patristocrats by hand.
– There’s a walkthrough for a solution of a French crypt, with the frequency analysis of the French language.
– There’s also a condensed analysis for identifying unknowns.

Oct.
– First outright suggestion that people construct Aristos using a keyed alphabet. No mention of K-1 through K-4 yet.
– More condensed analysis of unknowns.
– Piccola and RLH discuss the Playfair Cipher, part 1.

Dec.
– The first discussion of using decimation to recover Aristo K-3 type keywords.
– Part 2 of Piccola’s and RLH’s Playfair Cipher article.
– Letter frequencies of other languages.

1939

Feb.
– Part 3 of the Playfair article.
– As the presence of WW II makes itself ever more obvious to America, there are more references to it in the Cm. Several members, specifically those in Canada, are announced as going into military training, and then being posted overseas. The Cm editor suggests that ACA members should contact their local authorities to see if their skills can be used by the government. The ACA votes on whether to use its spare funds for buying War Bonds. And, there is a growing request for members to stop sending ciphers on post cards in the mail, because it’s resulting in the embarrassment of being hauled in for questioning on what exactly is in those secret messages.

Apr.
– Neoteric writes about the difficulties of understanding the German language.
– Piccola writes an introduction on Incomplete Columnar Transposition.

June
– A review of Fletcher Pratt’s book “Secret and Urgent” (1939).

Dec.
– There had been some mention in the Cm up to this point of Piccola’s work on her “Elementary Cryptanalysis” book, with the ACA deciding to publish it and multiple requests for preorders to cover initial printing costs. In this issue, there’s the official announcement that the book is on the shelves. From here, ACA members start sending in CONs based on types described in the book that had not yet been introduced in the Cm. A few articles appear later on analyzing how to break these CONs. There’s also kind of an unofficial contest for who can solve every single CON in the book first. A couple years later, there are still 2 or 3 that remained unsolved.
– There’s an article in this issue on the process for solving Italian crypts.

1940

Feb.
– Ab Struse writes an article on pattern word lists. He basically suggests the usefulness of lists of words with certain repeated letter patterns. It’s good for a pencil and paper approach, and ultimately extends to software tools for breaking Myszkowski ciphers.

Apr.
– The use of partial alphabet tables for Vigenere for determining letter shifts based on frequency counts. This may be a good test for harder Vigs, but it may duplicate the IC test.

June
– Helen Gaines (Piccola) passes away. Her obit appears in this issue.
– A posthumous article by Piccola appears, highlighting Porta.

Oct.
– Use of odd-numbered Grilles, part 1.

Dec.
– A “robot” approach to solving subs is described. This seems to be best suited for Aristos rather than Pats.
– Use of odd-numbered Grilles, part 2.

Thinking About Encryption, Part 49


Approaching Swagman.

Actually, I lied. I wrote last time that there were no more ciphers in the Nov.-Dec. 2018 issue of the Cm that I wanted to tackle, and that I was going to stop for a while to get to chores I’d been putting off. That had been true when I finished working on the Fractionated Morse ciphers, but as I was going through the ND2018 newsletter to count up all the CONs I’d solved to make sure I didn’t miss one when submitting my answers to the ACA for credit, I noticed that both ND2018 and JA2019 had Swagman CONs. I was planning on attacking Swagman in the near future, and I’d looked up the definition for it on the ACA site a few days earlier. So, I knew what it was, and having a Swagman CON in the ND2018 Cm did make me wonder how hard it would be to solve. I ended up cracking both Swagman CONs before writing up the last blog entry, but I still wanted to maintain the illusion I’d crafted for myself when I finished the Fractionated Morse CONs that I was taking a break from cryptanalysis to be productive for a few days.

Swagman is an odd mix of Complete Columnar and something like Cadenus. The idea is to make a square of some size (between side length 4 and 7), and fill in the square with the digits 1 through length such that they only appear once each in each row and column. Example:

1 3 2 4
3 1 4 2
4 2 1 3
2 4 3 1

The plaintext length will need to be a multiple of the square size. Write out the plaintext on the 4 lines (in this example) of equal length, adding padding at the end as necessary. Partial squares at the end are ok.

1324 1324 1324 13
3142 3142 3142 31
4213 4213 4213 42
2431 2431 2431 24
---- ---- ---- --
this isat ests tr
ingi tiso nlya te
stin area lstr in
gwew ould pani cx

Then, going column by column, place the letters in each column in ascending numeric order. Repeat the square as necessary. Then peel the ciphertext off in rows and reorder in groups of five (out of tradition).

tniw iied elti te
gtii orao psta cn
ihen tsla nsnr tr
swgs aust lays ix


tniwi iedel titeg tiior aopst acnih entsl ansnr trswg saust laysi x

Deciphering the message works in reverse. Determine the string length, and divide that by the width of the square. That gives you the length of the rows. Write out the numbered square as many times as you need, and put the ciphertext under that in the rows as calculated.

1324 1324 1324 13
3142 3142 3142 31
4213 4213 4213 42
2431 2431 2431 24
---- ---- ---- --
tniw iied elti te
gtii orao psta cn
ihen tsla nsnr tr
swgs aust lays ix

Go through the square one line at a time, picking the letter off the corresponding row as dictated by the number for that column and row. That is, for row 1, the order will be “1” (t), “3” (h), “2” (i), “4” (s), repeat. For row 2, use “3” (i),”1″ (n), “4” (g), “2” (i), etc. When you’re finished reordering the text, pull it off in rows.

this isat ests tr
ingi tiso nlya te
stin area lstr in
gwew ould pani cx

this is a test string it is only a test in a real string we would panic x (padding)

Cracking Swagman is actually fairly easy. The big weakness is in the repeated use of the key square. Placing one letter on a specific row means that you’re actually placing the character in the same cell of the other squares for that row. In my above example, if the crib were “REAL”, then, because we have 3.5 boxes, we’d be placing 4×3.5 (14) letters all at one time.

If we have the crib, then the first step is to count the ciphertext length: 56 characters. This gives us factors of 2, 28, 4, 14, 8 and 7. 2 and 28 are pretty unlikely. 8^2 is 64, but 56 characters does not give us a complete square. That leaves squares of 4 (3 * 4^2 + 8) and 7 (7^2 + 7). Either of these two are feasible, although the message is a bit short for a 7-square according to the ACA’s guidelines. We can start with a 4-square layout, and if we can’t place the crib there, we can always write out the message for a 7-square and try again.

tniw iied elti te
gtii orao psta cn
ihen tsla nsnr tr
swgs aust lays ix

Next, look for “r”s. They are on row 2, at col. 6; and row 3 at cols. 12 and 14.
For the crib, the next letter will be “e”, which we expect to be in the column to the right, or in column 1 if we have to wrap. We have an “e” in col. 7, row 1; but not in cols. 13 or 1. There’s an “a” in col. 8, row 3; and an “l” in column 9, row 4. Let’s draw a matching blank square set to populate. Right now, we don’t know what row “real” belongs in, so let’s start by just picking row 2 to work with. We can always change the row order later.

tniw iied elti te
gtii orao psta cn
ihen tsla nsnr tr
swgs aust lays ix


---- ---- ---- --
---- ---- ---- --
---- ---- ---- --
---- ---- ---- --

Move the “r” from 2,6 to blank 2,6. When we do this, we also get 2,2, 2,10 and 2,14, because those are the matching spots of the other squares.

tniw iied elti te
g-ii o-ao p-ta c-
ihen tsla nsnr tr
swgs aust lays ix

—- —- —- —
-t– -r– -s– -n
—- —- —- —
—- —- —- —

Because the square was constructed so that each digit only appears once on each row and column, we’ll never get a “1111” situation. That means that the next letter in the crib, “e”, has to come from a different row than for “r”. This is useful for narrowing down choices in cases where a given letter appears more than once in a given column (that’s not a problem right now). Pulling the “e” from 1,7 gives us the matching letters from 1,3 and 1,11.

tn-w ii-d el-i te
g-ii o-ao p-ta c-
ihen tsla nsnr tr
swgs aust lays ix


---- ---- ---- --
-ti- -re- -st- -n
---- ---- ---- --
---- ---- ---- --

Take the “a” from 3,8, and the matching letters from 3,4 and 3,12.

tn-w ii-d el-i te
g-ii o-ao p-ta c-
ihe- tsl- nsn- tr
swgs aust lays ix


---- ---- ---- --
-tin -rea -str -n
---- ---- ---- --
---- ---- ---- --

The “l” is at 4,9, which also gives us 4,1, 4,5 and 4,13.

tn-w ii-d el-i te
g-ii o-ao p-ta c-
ihe- tsl- nsn- tr
-wgs -ust -ays -x


---- ---- ---- --
stin area lstr in
---- ---- ---- --
---- ---- ---- --

If we take what we have, and add spaces between the words, we might get “??st in a real strin??” The last word might be “string”, and there is a loose “g” in column 1. Move that “g” from 2,1, and the other letters from 2,5, 2,9 and 2,13, down to row 3.

tn-w ii-d el-i te
--ii --ao --ta --
ihe- tsl- nsn- tr
-wgs -ust -ays -x


---- ---- ---- --
stin area lstr in
g--- o--- p--- c-
---- ---- ---- --

The current letter placements don’t lend themselves to any other obvious suggestions, so this is where I switch to anagramming. Column 1 contains “t” and “i”. The only thing that pairs with “t” in column 2 is “h”, and column 3 gives us “i” and “e” (the), while column 4 has “s” (this). “i” in column 1 could pair with “n” in column 2 (in?). because our crib doesn’t seem to have been on the line at the beginning of the sentence, I want something that could be the first word of the sentence, and that might be “the” or “this”. I’ll start by putting the “th” (and matching letters from the other squares) on row 1.

-n-w -i-d -l-i -e
--ii --ao --ta --
i-e- t-l- n-n- t-
-wgs -ust -ays -x


th-- is-- es-- tr
stin area lstr in
g--- o--- p--- c-
---- ---- ---- --

Notice now that column 1 only has one letter left, so let’s put that (and the matching letters) on row 4.

-n-w -i-d -l-i -e
--ii --ao --ta --
--e- --l- --n- --
-wgs -ust -ays -x


th-- is-- es-- tr
stin area lstr in
g--- o--- p--- c-
i--- t--- n--- t-

It may be tempting to treat “ust” on the 4th row, in square two as part of one whole word, except that, again, we can’t use consecutive letters from the same row this way. So, building on “th” in the first row, we can have “thiwis”, “thesis” “thgsis”, or any other combination using “ieg” and “wis”. “This” and “Thesis” are the most promising, although few sentences start with “thesis.” Let’s plug in “is” after “th”. If that doesn’t work, we can always go back and try “es”.

-n-w -i-d -l-i -e
---i ---o ---a --
--e- --l- --n- --
-wg- -us- -ay- -x


this isat ests tr
stin area lstr in
g--- o--- p--- c-
i--- t--- n--- t-

Ok, we can pull out full words now, with “this is a test str?” on row 1. The only way to complete “str” is to move row 4 to row 2. But, the current row 2 looks like it continues on row 3, so let’s put row 4 between rows 1 and 2. Anagramming the letters in columns 2 and 3 lets us complete the “str” at the end of row 1 as “string, so I’ll fill in those letters, too.

---w ---d ---i --
---i ---o ---a --
--e- --l- --n- --
-w-- -u-- -a-- -x


this isat ests tr
ing- tis- nly- te
stin area lstr in
g--- o--- p--- c-

The letters on upper rows 3 and 4 can only go on lower row 4.

---w ---d ---i --
---i ---o ---a --
---- ---- ---- --
---- ---- ---- --


this isat ests tr
ing- tis- nly- te
stin area lstr in
gwe- oul- pan- cx

And the remaining two lines almost place themselves.

---- ---- ---- --
---- ---- ---- --
---- ---- ---- --
---- ---- ---- --


this isat ests tr
ingi tiso nlya te
stin area lstr in
gwew ould pani cx

this isat ests tringi tiso nlya testin area lstr ingwew ould pani cx
this is a test string it is only a test in a real string we would panic x

I was thinking about how I could go about writing a VBScript solver for Swagman, and I really didn’t like most of the approaches I’d normally start with. Then I went to Bion’s Gadget Page, and tried using that to solve the ND2018 Swagman CON, and that went pretty quickly. The JF2019 CON went slower because it used a 7×7 square, and it was simply a harder CON. Regardless, I used Bion’s gadget to solve both CONs, and I’m not seeing a lot of reason to write my own solver right now. Maybe later.

Summary:
1) Swagman is a transposition cipher where a sudoku-like square is used to rearrange the letters within individual columns.
2) The size of the square is determined, and the digits 1 through size are filled in so that no digit repeats in any row or column.
3) The plaintext is written out as “size” rows of equal length (pad the message as necessary).
4) Write the square over the message (the last section doesn’t have to form a complete square) repeating the square as necessary.
5) Sort the letters in each column in ascending numeric order, then pull the cipher text off in rows.
6) Deciphering works in reverse order.
7) To solve Swagman, get the factors of the message length, and use them to determine the potential square size.
8) Write the cipher into the squares as rows of “message length”/”square size” length.
9) If you have a crib, look for the columns containing the consecutive letters of the crib, and move them into the plaintext one batch at a time (that is, if you’re moving the letter in row 2, column 1 for a 5-square, then you can move the letters from column 2 + 5n of the same row.
10) Anagram the letters where necessary.
11) Consecutive letters from the same row (i.e. – for a 5 square, in row 1, multiple letters from columns 1-5) are disallowed.
12) Don’t worry about getting the plaintext rows correct right from the start. You can always rearrange them as necessary later.
13) Swagman is a pretty easy cipher type to solve for square sizes of 7 or less (I haven’t seen anything bigger than 7 yet). It’s not secure, but it can be fun to play with.

Wiggles, part 45


Just a little ongoing story to give you something to play with until the next blog post.

J KFORPFKFG UBF CVHSBVJQBU HWG VRRXFG HKRTWG IF LJUB JU HS ITOB HS J ORTVG. J LHS HVRWF. J FDHIJWFG UBF SHWG JW UBF HKFH, HWG J’G GJSUTKYFG JU HVV AKFUUM QRRG – HVV RC UBF VJUUVF HWJIHV AKJWUS LFKF LJAFG OVFHW. FDOFAU UBHU UBFKF LHS RWF OVFHK SFU UBHU AHSSFG KJQBU YM IF HWG GJSHAAFHKFG URLHKG UBF SRTUB FWG RC UBF OKFPJOF. LJUB IM UBJWXJWQ HVV IFSSFG TA WRL, J UKJFG UR KFORWOJVF UBF CHOF UBHU BHG YFFW IFKF OFWUJIFUFKS CKRI IM WRSF, LJUB UBFSF AKJWUS. UBHU BHG YFVRWQFG UR H SIHVV YRM, IHMYF 6 RK 7, YVHOX BHJK JW H YRLV OTU, NFU YHOX FMFS, HWG H VJQBU MFVVRLJSB SXJW. SRIFBRL, BF’G QRUUFW KJQBU TA YFBJWG IF LJUBRTU IM WRUJOJWQ, JW URUHV GHKXWFSS, HWG BHG NTSU YFFW LHJUJWQ CRK IF UR UTKW HKRTWG HWG NTIA RTU RC IM SXJW VJXF UBHU. BF LHSW’U BFKF WRL, HWG UBF AKJWUS JW UBF SHWG LFKFW’U BTIHW. CTKUBFK, LBHUFPFK BHG IHGF UBFI BHG QRUUFW AHSU IF HAAHKFWUVM LJUBRTU UKMJWQ UR GRGQF IM CVHJVJWQ VFQS.

Hotto Motto Doraemon Papercraft


There’s a new Doraemon movie coming out (Doraemon goes to the moon), and the Hotto Motto bento box food shop had a tie-in where they offered papercraft sheets as part of the kid’s happy meals (450 yen – $4 USD). I wanted to make the Dokodemo (Anywhere) Door.

While I was waiting for the bento to be made, I talked to the store manager. He commented on how challenging the Dokodemo Door was to put together. I have to agree. There are a lot of tricky little folds that need to be made to keep the door frame from disassembling by itself over time, but to still let the door fully close and smoothly open. I can’t picture very many young kids being able to make this without a LOT of help from their parents.

The finished toy is about 3″ tall and maybe 3/8″ thick. It’s small and fiddly to work with. I think, in total, it took me an hour to complete (although, I wasn’t timing myself).

The little stand to the right is used to support the door when it’s open.

Doraemon and the Moon Bunnies.

Thinking About Encryption, Part 48


Revisiting Fractionated Morse.

In the last blog entry, I mentioned that my Fractionated Morse solver had crashed on the FM CON from the Nov.-Dec. 2018 issue of the ACA Newsletter, the Cm. After solving all of the other CONs mentioned up to this point, all that remained from that issue was the FM, and the Jan.-Feb. 2019 issue also had a Fractionated Morse CON, so I decided to tackle those two next. They both had really short cribs, which quickly revealed the nature of the bug – I’d hard-coded the size of the array for holding all of the potential placements for the crib, and their associated alphabets, and these two CONs were producing over 90 hits each. I increased the array size, and ran the solver again, focusing first on JA2019 because its crib was 5 letters long (ND2018 was four letters).

The thing is, everything I learned while solving JA2019 I ended up having to also use on ND2018, and the deadline for submitting the solution for that one has passed now. Since it makes more sense to use an example than just wave my hands around, I’ll pick the ND2018 crib for this, but I’m not going to reproduce the CON itself, because that does belong to the ACA. If you want to see the real cipher puzzles, I highly suggest that you become an ACA member yourself.

Anyway, the ND2018 FM crib was “lies,” and there were over 100 potential placements on a ciphertext that was something like 130 characters long. (If you want to refresh yourself on how FM works, check out my earlier blog entry on it.) The key point to remember is that FM uses a 3-digit trinary code to represent the letters 1-26, the digits 0-9, and letter and word breaks.

A = "..."
B = "..-"
C = "..x", etc.

“x” is “end of letter” and “xx” is “end of word.” “xxx” is an invalid string and will never occur in legal plaintext.

Because the combinations of characters could produce “……” and “……..”, while the crib could start anywhere within that, we have to test the crib word (including word break padding) starting with “xx?”, “?xx” and “??x” (since FM is 3-digit, we only need to concern ourselves with these three shifts) throughout the entire message. From this, we build the filter for the placement and see how that fits the ciphertext. With such a short crib (4 letters), all three filters were full wildcards. Sigh.

I wasn’t getting a single, clear placement, so I went to the next step of trying to extend the crib with likely collocations – “his lies”, “her lies”, “the lies,” “lies around”, “lies near,” “their lies”, etc. That went nowhere, and trying to prepend and append individual letters one at a time wasn’t getting me anywhere either. I made the big leap and cloned the solver in order to add a dictionary word appender.

In the Foursquare article, I’d talked about using a “possible word” approach, and matching the resulting alphabets generated to the alphabet for the crib to eliminate false hits. Which was fine then because I had just one place the crib could go in the cipher text. With the current FM CON, there were no clear placements, so I’d be trying to match unknown possible word alphabets against unverified crib alphabets. Instead, it made more sense to prepend, or append, words from the dictionary one at a time to the crib, and see if I could narrow down the placement choices that way. To cut to the chase, I loaded the entire 200,000 word dictionary into an array (which I did program to be dynamic), and I wrote two for-loops to prepend and append the dictionary to the crib simultaneously.

for i = 0 to ubound(wordAry) – 1
for j = 0 to ubound(wordAry) – 1
cribBig = wordAry(i) & ” ” & crib & ” ” & wordAry(j)
if(placedCrib(cribBig) then wscript.echo cribBig
next
next

Ok, granted, I knew that running the crib placer 200,000 * 200,000 times would likely drag down my laptop processor, but I was optimistic (that and I hadn’t debugged the script, so I was fully aware I could have an infinite loop in there somewhere, but, ahhh, such is the folly of youth).

After a couple of hours, no output. I went to bed and let the script run over night. The next morning, still nothing. I killed the script, and did what I should have done at the beginning, and just wrote it to append the dictionary words to the end of the crib.

for i = 0 to ubound(wordAry) – 1
cribBig = crib & ” ” & wordAry(i)
if(placedCrib(cribBig) then wscript.echo cribBig
next

Pretty quickly, I started getting hits out, but it was still taking 10-15 minutes to run through the full dictionary. There’s just so much going on in working with this cipher type. I didn’t want to wait that long to begin wading through the output to look for promising hits. I killed the script again, and added code to store the results to separate files for each initial letter.

letter = “”
for i = 0 to ubound(wordAry) – 1
cribBig = crib & ” ” & wordAry(i)
if(left(wordAry(i, 1) <> letter) then
if(len(letter) > 0) then outFile.close
outFile = set(“outWords_” & left(wordAry(i, 1) & “.txt”)
letter = left(wordAry(i, 1)
if(placedCrib(cribBig) then outFile.writeline cribBig
next
outFile.close

All total, there were hundreds of potential hits, and hundreds of obviously bad alphabets and unlikely word combinations. I needed another way to filter out as many of the bad alphabets as I could. The problem was that with short cribs, there aren’t that many letters that get identified for the alphabet.

Example:
—-V—F-H——R—TI—

FM uses a keyword alphabet. That is, you pick a word and put that at the left, with no duplicated letters. Then you fill in the remaining letters to the right. For “teacher”:

TEACHRBDFGIJKLMNOPQSUVWXYZ

What should be clear is that while the letters at the left could be in any order (given the choice of keyword), the letters in the right half of the string should be ascending. Therefore, my above example can be rejected because “I” follows “T”, and this is far enough to the right of the string to ensure that it’s not part of the key.

FHRTI

I stripped out the dashes, and set up a variable to let me pick the rightmost n letters. Then I built a function to test the segment for whether the letters were in ascending order.

I ran the script, and the number of false positives dropped down to a much more manageable number. But, as I was going through the results, I noticed something else.

----V---F-H------R---T-U--

There were alphabets where consecutive letters had gaps between them. This called for another function.

alphabet = “—-V—F-H——R—T-U–”
failTest = false
segment = “FHRTU”
for i = 1 to width(segment) – 1
testCh1 = mid(segment, i, 1)
testCh2 = mid(segment, i+1, 1)
ch1Pos = instr(alphabet, testCh1)
ch2Pos = instr(alphabet, testCh2)
if((ch2Pos – ch1Pos) > (asc(testCh2) – asc(testCh1))) then
failTest = true
end if
next

I run the script again, and I’m left with maybe 30 hits. One specific hit, on “lies going”, seemed promising. I ran this combination through my regular solver, and the plaintext output included “THERE” as the first word in the sentence. That was VERY reassuring. I modified the script to easily switch between prepending and appending words to the crib with a single flag, and ran the script again, prepending on ” lies going”. This gave me a single crib placement on “OF LIES GOING”, and a few more letters in the alphabet. From here, I could use the regular solver with the new alphabet suggestions to get the full solution to the CON. I took the same approach with the JA2019 CON, and solved that pretty quickly as well.

The thing about most cipher types is that they’re usually easy to solve if you have a long enough crib. You might even be able to crack them with pencil and paper. It’s when the crib gets shorter that I have to become more creative, and I end up relying on an “intelligent brute force method.” In the case of Fractionated Morse, that means extending the crib from the dictionary, appending or prepending one word at a time, testing the proposed placement alphabets to make sure the last 5 or 6 letters are in ascending order, and don’t have inappropriate blanks between consecutive letters. Even with all of this, it’s still possible to get 20-30 false hits if the combined “possible extender word” and crib word are less than 10 letters long. So, it may be necessary to identify each reasonable combination (“broil lies” is not reasonable, but “many lies” and “of lies” are), and test each of them one at a time, running them through the solver to print out whatever plaintext exists, and see which combinations produce garbage (i.e. “many lies” gives “TQX ….L MANY LIES” while “of lies” may give “TH..E ……. OF LIES”.

“LIES GOING” didn’t immediately strike me as a normal collocation, and I would have been perfectly happy to try all the other legal combinations first, but I did have a nagging suspicion at the back of my mind that I should at least test it to see if it could be rejected. So, the next point is “stay opened-minded.” If you don’t know what the plaintext is, it could be anything. Try to eliminate the obviously wrong answers first (illegal keyword alphabets), try to get the false positives down to a manageable number, then start testing what’s left from most reasonable to less reasonable. Don’t reject any crib combinations out of hand just because they’re not what you’re used to. In the case of ND2018, it was a British joke quote going back to the 1940’s, which is why “lies going” felt unnatural to me. In the end, though, that led to “of lies going”, which was all I needed to get the full solution.

I do need to point out an oversight I just picked up on as I’m typing this entry. In my above pseudo code, I have the line:

cribBig = crib & ” ” & wordAry(i)

Assuming that the crib is not the first word in the sentence, and wordAry(i) is not the last word, I really should have bracketed the combination with more word break spaces to extend the crib even further. This may have eliminated that many more false hits. Sigh. I’ll know better next time.

cribBig = ” ” & crib & ” ” & wordAry(i) & ” ”

And with this, I’ve solved all the CONs in the Nov.-Dec. 2018 issue of the Cm that I want to do.

Wiggles, part 44


Just a little ongoing story to give you something to play with until the next blog post.

P ZCMMCKIX YWI QNAIUJPLO KPYW KAPYWPLO UQ P YAPIX YC NAUXMI JB SUX QWCHMXIA KPYW JB OCCX WULX, KWPNW AIQHMYIX PL JB QJUQWPLO JB ZUNI PL YWI QULX CZ YWI ZMCCA CZ YWI NAIGPNI. YWI YIUAQ RCHAPLO CHY CZ JB IBIQ WIMRIX YWI QULX NUVI ULX SMCNV JB GPQPCL. PL KPYW UMM YWPQ, P NCHMX WIUA WPOW-RPYNWIX, WBQYIAPNUM NWPMXPQW MUHOWYIA. P WUGI LC P PXIU WCK MCLO PY YCCV JI YC AINCGIA, CA RCQQPSMB P KUQ EHQY OCPLO PLYC QWCNV. IPYWIA KUB, KWIL P KUQ XCLI, YWI ZMUQWMPOWY WUX SIIL YCQQIX CGIA YC YWI NCALIA CZ YWI QINYPCL P KUQ PL, ULX P WUX QULX UMM CGIA JB ZUNI ULX PL JB WUPA, XCKL JB QWPAY ULX HR PL JB RULYQ, JB SAIUYW KUQ NCJPLO PL QWCAY, EUOOIX OUQRQ, ULX P’X QCJIWCK ILXIX HR CL JB SUNV PL YWPQ MCK, LUAACK QRUNI. P KCAVIX JBQIMZ SUNV YC SIPLO CL “UMM YWAIIQ.” (JB APOWY UAJ MBPLO JCAI MCCQIMB CL YWI OACHLX SIQPXI JI. ULX LC, YWUY XCIQL’Y JIUL PY WUX ZUMMIL CZZ QCJIWCK, LC JUYYIA WCK JHNW P’X KPQWIX PY WUX.)

Thinking About Encryption, Part 47


Revisiting the Foursquare Cipher.

I won’t say that I hate Foursquare, but it is certainly my least favorite of the cipher types I’ve tackled so far. The problem is that even with 30-character long cribs, there may not be enough information to quickly extract the plaintext back out of the CON. The reason is that Foursquare uses two key alphabets; even if you have one of them fully handed to you, you’re going to get nothing out without a significant portion of the second alphabet as well. If you don’t remember how Foursquare works, check out my earlier write-up on it.

Before the Jan.-Feb. 2019 issue of the ACA Cm newsletter arrived, I’d solved about half (50) of the CONs in the Nov.-Dec. 2018 issue, skipping the ND2018 Foursquare CON. After receiving JA2019, I finished 50 of the ones in the new issue, and was preparing to work on the cipher types common between ND2018 and JA2019 (i.e. – Bacon, Cadenus, etc.) Getting those others out of the way, I had two Foursquares from JA2019, and one from ND2018. The two newer CONs had very long cribs, so I started with them first. Neither were all that hard, and I could place the cribs pretty quickly. Even so, it took me maybe 20 minutes each, with my VBScript solver, to fully crack both of them. I was putting in little enhancements and tweaks to the solver along the way, so in the end I’d thought I was ready to tackle the ND2018 CON. The ciphertext was 104 characters long, and the crib was just “the truth.”

Again, I could place the crib pretty quickly (there were two realistic locations and a bunch of obviously wrong ones), and that gave me the two key alphabet strings:

___L_____________STU_____
________E_______P__U_____

The positioning of the U’s implied that both strings were written into their squares horizontally instead of vertically. And, the five blanks after the U’s allowed “VWXYZ” to fit in perfectly. The gap in the second key alphabet between P and U probably held Q, and one letter from RST. So, I plug in the obvious values, holding off on filling in the Q just on general principles, and that gave me:

___L_____________STUVWXYZ
________E_______P__UVWXYZ

I have the solver print out the plaintext so far, and all I get is “THETRUTH”, and almost nothing else. I try plugging in Q and the other RST letters in the second line, and I still get nothing new. Obviously, the majority of the letters used in the CON are from the first half of the key alphabets, and what I’m doing at this stage isn’t helping me much.

From what I’ve picked up so far, with a number of the cipher types, we can either attack the alphabet, attack the key, extend the crib, or switch to a probable word attack. I’m getting nowhere with the alphabet, and the keys could be anything (from 1 to 15 letters each, and I’d have to try every combination of keys as both “key1” in box CT1, key2 in box CT2, and vice versa. That’s an approach that’s not even worth thinking about.

I’d used the “extend the crib” attack in the last few CONs, and it seemed reasonable to try that again. With something like “the truth”, the text could be “is the truth,” “is not the truth,” “can be the truth,” and so on. Again, nothing worked. I switched to prepending one letter at a time, then appending one letter at a time. This just gave me a whole bunch of false positives. After an hour or so of this, I decided to go with “potential word.” The title of the CON was “accidental discovery”, so I tried those two words, “experiment,” “study,” “research,” and whatever else I could think of. Nothing.

Ok, time to get serious, and break out the dictionary word search. But, one thing I’d been noticing with the false hits was that they didn’t have any relationship to the two key alphabets I’d been given by the original crib. So, I clone my solver and start building a second script. This one, called “foursquare_word_search”, would read the dictionary file, try to place each word in the CON, and then compare the proposed new key alphabets against the ones from the crib. Anything where there were mismatched letters in the same string position would be rejected. BUT, unless the new alphabets have at least one letter each in common with the crib alphabets, there’d be no way to tell the potential word really was in the plaintext. Therefore, if the potential word didn’t share at least one letter with the crib alphabet, it would also get rejected.

___L_____________STUVWXYZ;________E_______P__UVWXYZ (from crib)
_DF______________S_______;________E_ARL____________ (good)
_________________________;________Z________________ (bad)
_____JHG_________________;_ROB_____________________ (bad)

I test the new script on the one CON from JA2019 with the longest crib, and it works exactly the way I want it to. What I do figure out is that it’s best to start with the longest possible words first, and slowly go smaller and smaller, building up the crib alphabet along the way. The bigger the alphabet, the easier it is to reject false positives. Everything’s going great, and I return to the ND2018 CON. I start by checking all 15-letter words, and there’s so much processing going on that it takes 20 minutes to display “no hits.” 14 letters, no hits. At 13 letters, I’m getting a whole mass of false positives, and it’s still taking 20-25 minutes per word length. I’m also getting worried about a couple “one-side anomalies” that I’d been seeing all along, but had been been dismissing. I’m getting similar results for the unshifted and shifted crib placements (that is, comparing “AN-OM-OL-Y?” versus “?A-NA-MO-LY”, since Foursquare takes the letters in matched pairs). I dig into the code and discover a bug, which implies that what had been working code before never should have worked at all). And second, sometimes the possible word I’m testing doesn’t appear in the plaintext output, even when the new key alphabet passes the tests. That is:

Testing: AMPHIBIAN
___L________D____STUVWXYZ (CT1)
________E_______PI_UVWXYZ (CT2)
"-----------------THETRUTHBV---QUXK------TX----SX" (plaintext)

gives me a new key alphabet I can work with, but “AMPHIBIAN” isn’t in the plaintext. Largely, the reason for this is that my script placed AMPHIBIAN right underneath THETRUTH, which was something I can’t easily test for. Anyway, I fix the first bug, and I start looking for rules I can use to mechanically weed out false positives.

One of the first rules that I can implement is to reject any word or placement where the same letter appears more than once in the key alphabet. I.e. –

___L___S____E____STUVWXYZ ("S" appears twice)

The second is that the two spaces between P and U shouldn’t have anything other than QRS or T. That’s something I can also easily test for. I put in the latter two rules, and I fix the above bug. Then, I keep testing possible words with shorter lengths. 12 letters still gives me over three hundred false positives, and 10 letters gives me over 1,000. I’m spending a couple hours just wading through every word that gives me a hit, until I get to one (which I’m not going to divulge here) that looks reasonable. Not only is it a good candidate, but it fills in enough of the crib alphabet to suggest a few letters belonging to the key for CT2. A bit more back and forth between the key alphabets and filling in the plaintext and FINALLY, I’ve cracked the CON. That only took, what… four days to solve all total? Sigh.

Now I’ve got the solution, and I go back to see why the other approaches didn’t work. First, the CON is a quote by someone famous, and they didn’t use “the truth” in combination with any of the other words I’d ever have expected. The two words on either side of the crib were four and three letters long, each, and the shorter one had gotten lost in the middle of all the false hits. There was one long word that had been missed during the “possible word” dictionary search, because it didn’t use any of the letters from the crib alphabets. Otherwise, the only other long word was 10 letters, and everything else was shorter. The primary issue that made this such a hard CON to crack was that the crib was just too short to start with.

Ok, my takeaways.
First, the more CONs you try to solve, the more patterns you can pick up.
In the case of Foursquare with a short crib, everything revolves around eliminating obviously wrong answers from the output. That means looking for key alphabets that are either longer or shorter than 25 letters each for CT1 and CT2. Reject words where the same letter appears more than once in the alphabet, or if the “possible word” doesn’t get displayed in the plaintext output (more on that below). Make sure that the possible word alphabets overlap the the crib alphabets, but not 100% completely (there’s no reason to print out “other” if you’ve already placed “another”). Reject possible words if you get obviously wrong alphabets, like “___________________TIQW__”. “I” and “Q” shouldn’t appear after “T” this far away from the left of the alphabet (because none of them are part of the key) and the only thing that should appear between “T” and “W” is “U” and “V”.

Remember, the CT1 and CT2 alphabets are made up by a keyword, followed by all the other letters in ascending order (not including “J”):
TEACHRBDFGIKLMNOPQSUVWXYZ” (This is a clear pattern to look for.)

Oh yeah, one other headache/bug/oversight was in not testing the possible words from the dictionary to see if they contained “J.” That took a bit to figure out why the solver was crashing for “no reason.”

Now, about the test for the possible word not showing up in the plaintext. Recall that Foursquare takes the plaintext in pairs and uses those for creating a rectangle between the PT1 and PT2 alphabet squares. The ciphertext is then pulled off in pairs as well from the CT1 and CT2 squares, and written out in rows. Since we don’t know what comes before the crib or possible word, we could get plaintext that looks like:

awordinedgewise
or,
mywordinedgewise

If the crib is “word,” we can either have “aw” + “or” + “di”
or, “my” + “wo” + “rd” (shortened to “wo” + “rd”).

Meaning the filter will be “?-“+ “–” + “-?”; or “–” + “–“.

When trying to place a possible word, such as “AMPHIBIAN”, the plaintext output could be:

"---THETRUTH-----AMPHIBIA------", or
"---THETRUTH-----MPHIBIAN------"

To ensure that I don’t reject the possible word out of hand, I need to trim the first and last letters, and see if “MPHIBIA” is in the string. If yes, great. Print it out as a possible hit. If not, reject it as an obvious false positive.

Even so, when I’d gotten down to testing the 10-letter words from the dictionary, I’d initially gotten 1,000 hits. After all of my above testing, I could still only get it down to 790 hits. Most of them produced garbage plaintext out, but in ways that were too random to reliably test for in the code. I had to eyeball all the results one at a time, and hope I didn’t overlook something that actually was part of the solution.

But, I now have an idea for how to extend the crib next time. First, place the crib and build up the crib alphabets (CT1 and CT2). Then, extract the ciphertext so that it covers the length of the dictionary word plus the crib. Create the filter for the segment, and for the dictionary word plus crib. Compare the two filters, and if they match, run the above tests to eliminate false positives. Print out anything that passes starting from 1-letter words up to maybe 4 or 5 letters, depending on the CON. In fact, I could go all the way up to 15 letter words, and use crib extension instead of possible word placement.

Example:
CON: GHTYVHGARTFHJSDTSV

Crib: “mighty”

Placement:
GHTYVHGARTFHJSDTSV
.....MIGHTY.......

assume that the the crib extension will be to the right, and that we’ll be testing for 5-letter words:

CON segment:
HGCRTFTJCDS -> "__1_2_2_1__" (CON filter)

Crib + dictionary word to test:
MIGHTYMOUSE -> "__1_2_2_1__" (crib + new word filter)

This would match, and pass the tests for missing/extra letters, word missing from potential plaintext, and new word alphabet conflicting against the confirmed crib alphabet.

It’s a start.
[Edit: “As of when I wrote this”] There’s only one CON left in the ND2018 issue of the Cm that I have any interest in trying to solve now, and that’s a Fractionated Morse that had caused my FM solver to crash. After that, I’ll submit my Nov.-Dec. 2018 solutions to the ACA for credit. I’m up to 63 SOLs now, out of about 105 total for the newsletter. I can live with that. Then I’ll switch over to the JA2019 issue full-time, although there are only 8 or so CONs left in that issue that I want to get to that I haven’t solved yet. Yay, me.

Wiggles, part 43


Just a little ongoing story to give you something to play with until the next blog post.

KAVK SVXK GCKNGY SPJK V RVW KVXKP NY HD HGFKA (GU, HVDRP KAVK TVX BFXK JUGH KAP XPTPU). HD RNMP TVX NY V IGNY CVUMNYZ SGK, VYW NK’W IGXK HP TPSS GEPU 5,000 DPY KG CVD GJJ KAP IAVUZPX. DPVA, N’W RP RPKKPU GJJ UPKFUYNYZ YGT, VYW “JVIP KAP HFXNI,” XG KG XCPVM. N XIVYYPW KAP RSVIMYPXX VAPVW GJ HP, KFUYPW HD APVW KG JVIP RVIM KAP TVD N IVHP, VYW JSNIMPW KAP JSVXASNZAK RVIM GY. KAPY N SPK GFK V DPSS, BPUMPW RVIM VYW BVHHPW HD UPSGIVKPW XAGFSWPU AVUW NYKG KAP ZUGFYW, TANIA HVWP HP XIUPVH XG SGFW KAVK N MYGT XGHPGYP NY KAP RVU AVW KG AVEP APVUW HP.