Cryptanalysis
Frequencies
Also found as frequency analysis · character frequencies · n-gram frequencies
A count begins with four choices
Name the sample, the unit, the filtering rule, and the case rule before reading a frequency table. Counting ASCII letters ignores an accented letter that a Unicode-letter count keeps. Counting non-whitespace characters keeps punctuation and digits. Coordinate-pair mode treats a pair such as 42 as one unit rather than two digits.
fₓ = number of times unit x occursp̂ₓ = fₓ / NRank is only the position after sorting the observed counts. It is useful for comparison, but matching the rank of a reference alphabet does not create a substitution mapping.Military Cryptanalytics, Part IGaines, Cryptanalysis
Groups overlap
For a group width of n, each window starts one unit after the previous window. A five-unit sample therefore contains four digrams and three trigrams. Decipher counts widths 1, 2, and 3.
- Windows
AB BA AB BA- Counts
AB: 2 BA: 2- Proportions
AB: 0.5 BA: 0.5
The denominator is four windows, not five letters.
Filtering happens before those windows are formed. That can create an adjacency that was separated by punctuation in the display. If the boundary itself matters, compare the result with the contact and pair rules.
Reference distributions are comparisons
A language table comes from a particular corpus with its own spelling, punctuation, alphabet, and sample size. It gives a useful baseline, not a universal constant. Compare like with like: letter proportions should be compared with letter proportions prepared under compatible rules.Military Cryptanalytics, Part IGaines, Cryptanalysis