I have been reading “Wrong: Why experts* keep failing us–and how to know when not to trust them *Scientists, finance wizards, doctors, relationship gurus, celebrity CEOs, … consultants, health officials and more” by David Freedman. The book introduced me to what may be called the first system combination algorithm: the Condorcet Jury Theorem.
Condorcet lived during the French Revolution and was concerned about the ability of a group of people reaching a correct decision when plurality of votes decides between two choices (e.g. “guilty” or “not guilty”). The theorem has been extended to show that as long as each voter has a greater than 50% probability of voting correctly, the majority decision will be the correct one.
Freedman points out that the problem with Condorcet’s theorem is that if they are likely to choose incorrectly, their majority decision will be incorrect. Therefore, we should not take any comfort on the “wisdom of crowds” since they can lead us astray.
But is being wrong the problem? If we knew that a group of people are likely to choose incorrectly, we could turn them into a good decision maker by doing the opposite of what they decide. For this reason, I would argue that the problem is not that an expert you rely on is wrong but that you are uncertain about their reliability. If you had a way to determine the reliability of collection of judges, you could turn their performance into a much better decider than any of them alone could be. This reminds me of the episode in Seinfeld where George starts doing everything the opposite of what he thinks he should do. Having recognized that he is a bad decision maker, he flips his binary choices. This “George Constanza Algorithm” leads him to immediate success in his life!