This Wired.com article on Nate Silver, one of many dozens now being written about the artful quant, is interesting in that it exactly highlights why it’s not possible to do what it wants to do: opensource Nate Silver. The reason Nate Silver cannot be “open sourced” (meaning, in the context of the article, that his “method” would be written down and publicly scrutinized and modified) is because an important part (the most important part?) of what Nate Silver does is guess. Nate is just a really smart guy that builds sophisticated models that include “weights”. Those “weights” have to be arbitrarily chosen. Nate seems to be pretty good at it, but they’re just guesses. He’ll likely be wrong some day. Nate is just another hedge fund manager and all of Nassim Taleb’s Black Swan criticisms apply.
I’m more inclined to trust Intrade – which is, in a sense “open sourced” by its fundamental design – in the long run than Nate Silver. But this is – it should be noted – essentially a philosophical point: is society deterministic or somehow fundamentally unknowably complex? If deterministic, then Nate Silver-types will likely someday figure out how to scientifically choose the weightings. They will rule the world. We will be (nay, already are, brains in vats). If unknowably complex, then the equation cannot be written without a fudge factor; Intrade-types will always have an edge.
I think one of the dangers of Nate Silvers is that these smart people delude many into thinking that if we just let smart people run the country, then there is hope that a “grand design” can be put into place. But this is folly. Ignore the epistemological issue described in the above paragraph (whether it’s possible or not to even write down the perfect governing equation). There are incentives that people – politicians – respond to that would, I argue, destroy the utility of any Techocratic regime. Bribes, corruption, grift, favoritism, and biases render any government-by-the-smart no better than any other kind of government. It is a dangerous delusion to think that (political) reality could be otherwise.