The Nate Silver Saga: Fundamentals
Or: What's the situation?
Everyone's "favorite" burrito-rater Nate Silver has been pitching a fit over the last few days on Twitter, mostly because of this HuffPost article accusing him of skewing things in Trump's direction. I don't think it's that bad, but Nate's work is pretty subpar in my view. In this and maybe subsequent posts, I'll try to lay out the objections/situation as I see it -- and why I hoot and holler so much about his endless Prevent Defense on something not worth defending.
I think the major issues here are:
538 is held up as a gold standard to the average person as Exactly What All Smart People Do. This is...not great, let's say. We'll get to an example why in a minute.
Silver's assumptions are simultaneously too macro and too micro in a way that isn't particularly effective.That is, the mathematical uncertainty he uses in his projections is at a level that completely ignores other sources of uncertainty that he keeps claiming are functionally bulletproof. The biggest unacknowledged source, of course, is his own understanding of political systems.
Sam Wang's (of Princeton Election Consortium) assumptions are more consistent, but more macro. A common objection to his model, that it changed midstream, doesn't jibe too well with me because the entire point of the change was that he found a forcing effect that was functionally invariant across elections. Namely, the only remaining effect that made his model "volatile" was polling error, which has remained remarkably stable even in this ridiculous shitfest of an election.
Related to the first one: 538 has a lot of writers who revel in this sort of House Certitude, "Forecaster" Enten especially, who as acolyte-defenders of Nate. This rightfully pisses a lot of people off, because it appears to outsiders that there isn't actually a diversity of thought in the 538 universe, just a weird cult devoted to an unlikable dingus.
Speaking of macro: The pseudofield "data science" inexplicably aligns itself with the worst of this pop science bullshit, partially because it's a new field and everyone is desperate to get funding or look cool or whatever the hell. This rightfully pisses off a lot of people in previous fields who were using these techniques for decades. Notable examples include Sam Wang, who used this kind of analysis for investigating problems in neuroscience; and yours truly, who does target tracking & sensor modeling.
These last two are kind of the key to why everyone is going completely bonkers over 538's actual and perceived problems: it looks and smells like a snake oil salesman pushing a revitalizing tonic instead of actual decent science -- all the right buzzwords, roping in a certain group of experts to defend it, blah blah blah. Next time I'll get into the weeds of why I think Nate's model sucks.