The Mathematics of Changing Your Mind

[rug]

THE THEORY THAT WOULD NOT DIE
How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines and Emerged Triumphant From Two Centuries of Controversy

By Sharon Bertsch McGrayne
320 pp. Yale University Press. $27.50.

Sharon Bertsch McGrayne introduces Bayes’s theorem in her new book with a remark by John Maynard Keynes: “When the facts change, I change my opinion. What do you do, sir?”

Bayes’s theorem, named after the 18th-century Presbyterian minister Thomas Bayes, addresses this selfsame essential task: How should we modify our beliefs in the light of additional information? Do we cling to old assumptions long after they’ve become untenable, or abandon them too readily at the first whisper of doubt? Bayesian reasoning promises to bring our views gradually into line with reality and so has become an invaluable tool for scientists of all sorts and, indeed, for anyone who wants, putting it grandiloquently, to sync up with the universe. If you are not thinking like a Bayesian, perhaps you should be.

At its core, Bayes’s theorem depends upon an ingenious turnabout: If you want to assess the strength of your hypothesis given the evidence, you must also assess the strength of the evidence given your hypothesis. In the face of uncertainty, a Bayesian asks three questions: How confident am I in the truth of my initial belief? On the assumption that my original belief is true, how confident am I that the new evidence is accurate? And whether or not my original belief is true, how confident am I that the new evidence is accurate? One proto-Bayesian, David Hume, underlined the importance of considering evidentiary probability properly when he questioned the authority of religious hearsay: one shouldn’t trust the supposed evidence for a miracle, he argued, unless it would be even more miraculous if the report were untrue.

The theorem has a long and surprisingly convoluted history, and McGrayne chronicles it in detail. It was Bayes’s friend Richard Price, an amateur mathematician, who developed Bayes’s ideas and probably deserves the glory that would have resulted from a Bayes-Price theorem. After Price, however, Bayes’s theorem lapsed into obscurity until the illustrious French mathematician Pierre Simon Laplace extended and applied it in clever, nontrivial ways in the early 19th century. Thereafter it went in and out of fashion, was applied in one field after another only to be later condemned for being vague, subjective or unscientific, and became a bone of contention between rival camps of mathematicians before enjoying a revival in recent years.

http://www.nytimes.com/2011/08/07/books/review/the-theory-that-would-not-die-by-sharon-bertsch-mcgrayne-book-review.html?_r=1

[struggle4progress]

1. The real aids, for cracking the enigma code, were probably not Bayes' rule but various other matters

Polish intelligence had followed enigma machine development before the war. As the Germans invaded in 39, the Poles promptly gathered up everything they had and conveyed it to the British. And what the Poles had was not insubstantial: IIRC, they handed the British an early version of the device, a manual for the current military version (actually detailing the number and operation of the rotors, how to use the device, as well as including a moderate-length example of a plain-text and it's encoded version), together with some mathematical research on how to apply permutation-group theory to the decoding problem

That, of course, was only a start, since the turning of the rotors meant that every new letter in a text was decoded according to a different scheme -- so to decode a text, one needed more info, including the initial rotor setting. But it was a very very good start: it helped Turing to automate, at least partially, the decoding process

The British also hired a number of verbally talented non-expert civilians in their code unit: IIRC, there is a story about somebody staring at a long ciphertext because "something is strange here" and finally pointing out that "L" appeared nowhere in the ciphertext

Cryptographic security lapses by German enigma machine operators also played a role: the British decoders particularly appreciated folk who did things like starting every message with some stereotypical signal such as "AAA"

[rug]

2. Capturing one intact helped.