I made a bad move when I used HJ Hornbeck’s posting on bayesian reasoning about Russian hacking as a jumping-off point for a critique of using bayesian reasoning to attempt to predict events.

In fact, I do think that what goes on in our brains probably resembles bayesian reasoning: it’s the old “past events do not predict future outcomes” on one side versus “things tend to happen the same way, in general” on the other. Since it’s how our brains appear to work, I think we owe ourselves extra caution as we try to consider whether our information leads us to one particular conclusion or another – that tendency to conclude that things are happening more or less the same way, is a form of mistake called “confirmation bias” and you oughtn’t be surprised to learn that it’s a problem with artificial intelligences, too: some classifiers are programmed to weight distant past results as more important, and others weight recent results as more important. If you’ve ever had food poisoning and mentally associated it with a particular dish, you can immediately recall the strength and speed with which these biases are learned. They’re all learned behaviors, and the process of establishing our “priors” and their probabilities is not purely bayesian, but it’s close enough for us to have survived and become a technological species.

Please don’t interpret my earlier posting as dismissing bayesian reasoning, or saying that “The Russians are not hacking the US.” I hope my objection served adequately to illustrate the problem with cherry-picking a bunch of “priors” and assigning them seat-of-the-pants probability, and then massaging the results using a sciency-seeming processes, to reflect (surprise!) our original beliefs. I’m a fan of epistemological simplicity: I’m comfortable saying “I don’t know, but I’d guess…” when I don’t know and am asked to guess.

When two people are arguing about whether or not something is credible, the discussion may look something like this:
Person 1: I believe X
Person 2: I believe Y
Person 1: Well, I believe that fact F1, F2, and F3 support my conclusion
Person 2: I’ll grant you facts F1 and F2 are compelling but F3 has a problem… Besides, have you considered F4?
Person 1: Fact F4 fits in with my assessment of the situation in the following way…
Person 2: Indeed, it does. I was just raising that for your consideration.
Person 1: Then are we agreed that my belief is probably accurate?
Person 2: Seems that way!

OK, that’s a bit of a joke. Really, on the internet, the discussion would go more like:
Person 1: I believe X
Person 2: I believe Y
Person 1: Well, I believe that fact F1, F2, and F3 support my conclusion
Person 2: I’ll grant you facts F1 and F2 BUT YOU SUCK AND I HATE YOU AND YOUR ASS LOOKS BIG IN THOSE JEANS. I KNOW WHERE YOU LIVE…
Person 1: …

In normal discussion, people trying to draw a conclusion pony up their facts, measure them, argue about them, and that’s good enough; it’s been the mode of argument since humans invented arguing. It’s the same thing that happens when you’re arguing using bayesian “priors” except that “measure them” means “argue about their probability” which is really the same thing, again, that we’re doing in a normal discussion.

My concern, then, is that bayesian reasoning is a manipulative hack to sneak a bit of argumentum in percutiendo mensam (I am just doing that to troll cartomancer!) throw a bunch of math and probability in and you sound more authoritative.

Anyhow, HJ Hornbeck generously came back at me with a remise of his previous argument. [hornbeck] I’m not going to try to review it point by point, because Hornbeck is no longer talking about bayesian reasoning. He’s refining his points, supporting his “priors” and doing exactly what I describe above, namely arguing about the supporting facts from which he draws his conclusion.

I’ve already argued about some of those facts in various places, and, as I’ve said before, [stderr] there’s a case that can be made that it was Russia, and it’s a pretty good case. I feel, as I’ve said before, that the US Government has done a mediocre or worse than mediocre job of substantiating that case; it has primarily left making its case to interested third parties and leak-fed media. It’s my opinion, as someone who has worked in the trenches of internet security for 25+ years, now, that the government would be well-served by putting more facts onto the table. I believe that the government has been putting itself in an increasingly weak position because there is a tremendous amount of credible attribution pointing toward the US being involved in hacking on a massive scale, and the US is firing back with, basically, “our experts say that this was done by a group called ‘Fuzzy Bear’ and they’re Russians.” I believe (in fact, I know) that the US has much much better information than that, which they could deploy without disclosing systems and methods – the fact that they aren’t doing it – that’s what’s bothering me.

That’s largely irrelevant, though, I want to stay focused on Hornbeck’s original point, which was that we can use bayesian reasoning to conclude that it was Russians who hacked the US. In my comments against his position, I pointed out that he had, in constructing his “priors” completely biased them by assuming that the hacking was done by a group. If he was doing it right, that option would have been there, and he would have assigned it a suitably low likelihood, which we could then argue about. In fact, he omitted other significant possibilities, thereby further biassing his “priors” – namely: what if it was me? The point remains that, by constraining the “priors” to a list that reflects his biases, it is impossible for him to get an output result that is not in line with his biases.

At that point, were I Hornbeck, I would have hoisted a flag of surrender. But, Hornbeck is made of sterner stuff – he replies:

I’d argue the first two are handled by D, “A skilled independent hacking team did it” but it’s true that I assumed a group was behind the attack. Could the DNC hack be pulled off by an individual? In theory, sure, but in practice the scale suggests more than one person involved. For instance:

thereby proving my point! Not content to have established a biased set of “priors” he then wants to patch and tweak them into the appearance of reasonableness. That’s the algorithm I described above, where we “measure and argue about our facts” and come to conclusions. Hornbeck then continues to hammer home my point, by trying to throw more and more facts and information into the discussion after I have already demonstrated that his “priors” were constructed wrong.

Ranum does say a lot more in that second blog post, but it’s either similar to what Biddle wrote over at The Intercept or amounts to kicking sand at Bayesian statistics.

As I said, my purpose was to kick sand at bayesian reasoning. Which, I did:

With all respect to HJ, I’m going to use his example as an opportunity to critique some of how Bayesian arguments are used in the skeptical community. [stderr]

I left it to Hornbeck, however, to grab a shovel and pile a mountain of sand over it. Remember, Hornbeck’s original argument was that we can convince ourselves it was the Russians by using bayesian reasoning and carefully constructed “priors” – as a shortcut to having a discussion about the facts. I’m comfortable with Hornbeck’s willingness to discuss the facts, which is what I think he should have stuck to doing all along. Leave the bayesian stuff out, you’re doing it all wrong.

There’s a good chance that I taught a lot of those security experts that are producing those reports how to chew cheese, so I’ve got a bit of a dilemma: was I a bad teacher?

By the way, there’s another bit by a real mathematician who takes up Carrier’s bayesianism: [ic] Hornbeck isn’t making the same degree of mistakes as Carrier does (go big, or go home!)  It’s mostly a dissection of Carrier but his conclusion is the same as mine: it’s a way of making your arguments seem more valid than they are, by scientizing them, which is ultimately a dishonest technique.

Oh, and: someone who doesn’t think that it would be possible for an individual to do the level and rate of attacks of something like Fancy Bear does not understand computing, let alone security. That is easily within the purview of an individual, though the people who have those skills tend to have more interesting jobs. I’m rusty like an old tin can and I could do it, if I gave enough shits and weren’t on Verizon’s crappy metered bandwidth. I don’t want to argue this stuff with someone who doesn’t understand metasploit, scripting, and cloud computing, but the idea that you’d need an elaborate team to accomplish what Fancy Bear did is farcical. Back in the days when Lance Spitzner and I were teaching classes on building honeypots, I spent a lot of time watching hackers, and have even been involved in dissecting some hackers’ tool-bags (the guy who was feeding Kevin Mitnick his tools, back in the day) – there’s a lot of automation and macros, when you’re doing bulk attacks; the process of checking for “bites” on 5,000 phishing emails looks like: reading a file listing all the systems you auto-rooted and going, “oooh, look who I got here!” The size of the target-base is irrelevant, what matters is the automation available to the attacker. If you’d ever seen an expert drive a copy of Core Impact through a target network, you’d understand what I’m talking about. Most hackers’ tool-bags don’t have the “hacker ferrari” in it, but if you’re setting up 3 or 4 phishing drive-bys and maybe a trapped PDF that’s going to be sent to 5,000 marks, you’re looking at less than a day for someone who’s got the tools, and a week for someone who doesn’t. It’d take me a month because I’m old and slow. But I’m mean.