begun these nerd wars have

Without Gallup’s Crappy Polls, Nate Silver Is Nothing, Says Gallup

Gallup: Getting stuff wrong since 1948One of the “fun” things about presidential elections is that every four years there’s a new dumb thing about the process for political junkies to yell at each other about despite the disinterest or genuine disgust of normals, and this year it’s polling! Did Nate Silver’s devil-math suck all the fun out of democracy, forever? Were the polls skewed because they didn’t reflect Republican understanding of reality? Were Gallup and Rasmussen “in the tank” for Republicans? Well, Gallup Editor-in-Chief Frank Newport has decided to weigh in on this controversy, and would like you to know that (a) Gallup was not wrong, because it abruptly stopped picking Romney to win by 7 points several days before the election, and (b) Nate Silver is a parasitic remora clinging to the great white shark that is Gallup and if everyone gets into the Nate Silver business the whole polling industry will collapse, and then we’ll have no polling at all, and then we’ll be sorry!

We feel vaguely bad for Frank Newport, truly, because he represents a stolid and storied organization and he’s probably institutionally unable to give vent to his full-on snarky rage against his critics, as he’d like. Instead, he had to jam a bunch of passive aggression into a very serious blog post about how successful Gallup’s polling was this year! First, he’d like you to remember that Gallup does not attempt to “predict” the election, like a common soothsayer. (Yes, he put “predict” in quotes.) But he’d also like to point out that, damn it, their final numbers were within the margin of error of being correct, and if they happened to have Romney a couple points ahead of Obama instead of vice-versa, well, those are the breaks, you know?

You might point out that Gallup actually had Romney ahead way outside the margin of error going into the final week of the campaign, then rather abruptly changed to be more in line with every other reputable polling company. This may have looked like herding, where pollsters start adjusting their methods because they’re afraid they’re wildly wrong, but Frank Newport would like to make it clear that this was not the case and Obama’s numbers just suddenly shifted for the better at the last minute. Was it because of Superstorm Sandy? Hard to say, but the change came after the storm, so yes, almost certainly.

But then he really lets you know what he thinks of math and numbers nerds like Nate Silver, who dare to sift through multiple publicly available yet contradictory sources of information to make sense of them:

But some of this will result from a variant of the venerable “law of the commons.” Individual farmers can each make a perfectly rational decision to graze their cows on the town commons. But all of these rational decisions together mean that the commons became overgrazed and, in the end, there is no grass left for any cow to graze. Many individual rational decisions can end up in a collective mess.

We have a reverse law of the commons with polls. It’s not easy nor cheap to conduct traditional random sample polls. It’s much easier, cheaper, and mostly less risky to focus on aggregating and analyzing others’ polls. Organizations that traditionally go to the expense and effort to conduct individual polls could, in theory, decide to put their efforts into aggregation and statistical analyses of other people’s polls in the next election cycle and cut out their own polling. If many organizations make this seemingly rational decision, we could quickly be in a situation in which there are fewer and fewer polls left to aggregate and put into statistical models. Many individual rational decisions could result in a loss for the collective interest of those interested in public opinion.

This will develop into a significant issue for the industry going forward.

Shorter version: If you people insist on thinking Nate Silver is so great, everyone in the polling business will stop doing expensive polling and just start analyzing other people’s polls instead and then where will you be, huh? Where will reporters get their numbers for their “horse race” stories during presidential campaigns? They might have to write about the issues or something! The horror! This is basically the argument real news organizations have been making for years about how blogs like Wonkette don’t do any expensive reporting but instead just make dick jokes about other people’s news stories and steal their pageviews, and journalism has collapsed over the past eight years, so maybe there’s something to it.

Meanwhile, Dick Morris went on Sean Hannity’s show and pretty much came out and said that his crazy Romney landslide predictions were for Republican morale purposes, despite his assertions at the time that they were all about secret poll information that only he could see.

Sean, I hope people aren’t mad at me about it … I spoke about what I believed and I think that there was a period of time when the Romney campaign was falling apart, people were not optimistic, nobody thought there was a chance of victory and I felt that it was my duty at that point to go out and say what I said. And at the time that I said it, I believe I was right.

Don’t worry, Dick, people won’t be mad at you for lying to them to their face! You will still get paid gigs on conservative media forever and ever, and will have a venue to spout your bullshit numbers when Nate Silver has put well-meaning but incompetent shops like Gallup out of business. [Gallup/HuffPo]

Related

About the author

Josh was born and raised in Buffalo, New York, leaving him with a love of chicken wings and a tendency to say “pop”. He taught ancient Greek and Roman history to undergraduates before fleeing from academia in terror; worked for a failed San Francisco dot-com that neglected to supply him with stock options or an Aeron chair; lived in Berlin, where he mostly ate Indian and Ethiopian food; finished in third place on his sole Jeopardy! appearance (the correct answer was “Golda Meir”); and was named 2007 Blogger of the Year by The Week, for obvious reasons. Josh is the creator/editor of COMICS CURMUDGEON (which you should read) and does geeky editing and writing about geeky things such as "the Java programming industry for JavaWorld." He lives in Baltimore with his wife Amber and his cat Hoagie.

View all articles by Josh Fruhlinger

Hola wonkerados.

To improve site performance, we did a thing. It could be up to three minutes before your comment appears. DON'T KEEP RETRYING, OKAY?

Also, if you are a new commenter, your comment may never appear. This is probably because we hate you.

160 comments

  1. ChillBill

    Nate was able to dissect Gallup and Rasmussen and explain, in detail, why the polls were statistically biased and historically inaccurate. They should be thanking him, not the other way around.

    Poor kid still can't catch a break…for accurately predicting the outcome of the election no less!

    1. PugglesRule

      They should be thanking him, not the other way around.

      That would require admitting they were… oh noooes…. wrong.

    2. BerkeleyBear

      But if the general populace (not wonkie types) ever realizes that Gallup and Rasmussen are generally full of shit, the GOP and media establishment will stop bankrolling their polls (especially the issue polling that pays the bills in the off years). Silver pointed out Gallup's historical problems months ago, and Ras got exposed as a joke in 2008, but as long as they have some level of trust in the public eye, the way it is right now is much better for them – they get to be GOP hacks, paid handsomely to do so, without actually having to worry about pesky little things like accuracy.

  2. sbj1964

    I thought Republicans are suppose to be good at math.What's next are you going to tell me Asians are good drivers?

  3. GunToting[Redacted]

    "we’ll have no polling at all, and then we’ll be sorry!"

    Objection! Assumes facts not in evidence.

  4. memzilla

    Shorter version of Gallup's lame-osity:

    Garbage In, Garbage Out.

    (GIGO, as every CompSci 101 student knows)

  5. Loch_Nessosaur

    I feel like I should say something profound about those who fail to understand the past will never get the future, but fuck it, buttsechs jokes are way funnier.

  6. ManchuCandidate

    Doobie Bros (What a Fool Beiieves)

    He came from somewhere back in her long ago
    The foot licking hooker loving fool don't see
    Tryin' hard to recreate
    What had yet to be created once in the maths
    Sean musters a smile
    For his idiotic tale
    Never coming near what reality was
    Too stupid to realize
    It never really was

    Faux was a place in his life
    He never made them think twice (or once)
    As he spits to make an apology
    Anybody else would surely know
    He's a lying piece of shit

    But what a fool believes he sees
    No wise man has the power to reason away
    What seems to him
    Is always better than nothing
    And nothing at all keeps sending him.

  7. BaldarTFlagass

    "campaign was falling apart, people were not optimistic, nobody thought there was a chance of victory"
    "I believe I was right. "

    Given all those indicators you listed, I don't see anything wrong with the conclusions you drew. Dick.

  8. Lizzietish81

    So….they're saying that they're the Commons and Nate Silver is one of many cows who graze freely off them without contributing to the cost of polling?

    1. Ruhe

      Right. You wouldn't want to take the more responsible and reality based position that, hey, this guy was analyzing our data and came up with a different and more correct picture so should we be learning something?

      1. WootInTarnation

        Bad analogy is bad… But it still manages, while utterly failing to illuminate the matter at hand, to side with the evil, self-interested rich folks who closed the commons to the smaller gentry and poorz in order to preserve timber prices, grain prices, and fox-hunting. Win!

        "We may be increasingly irrelevant, but we will always side with the powerful and the corrupt. Never fear.

        So keep paying our invoices! Pleeeeezee!"

  9. GunToting[Redacted]

    So Rush and Toesucker are now on record as lying to their listeners. Any thoughts as to how this will negatively impact their business model?

  10. actor212

    Meanwhile, Dick Morris went on Sean Hannity’s show and pretty much came out and said that his crazy Romney landslide predictions were for Republican morale purposes

    If I was a conservative¹, I'd be pretty pissed

    ¹This would involve more drinking than I can even imagine, or severe brain trauma.

  11. Terry

    "Don’t worry, Dick, people won’t be mad at you for lying to them to their face! You will still get paid gigs on conservative media forever and ever, "

    Being a regular on Fox News is like being a writer for The Enquirer. It pays well, keeps you busy, but means that you won't really get much worse elsewhere in the industry. Morris peaked years ago but hangs on, fighting to keep his face on TV.

  12. Goonemeritus

    Similarly Benito Mussolini personal pollster felt IL-Duce lost popularity by hanging from a lamp post.

  13. 738838

    I am gonna say it (maybe someone else already has; I haven't read the comments yet) but pollsters are scum. It is all just bullshit. No one I know has ever taken the time to answer them honestly or bothered at all. I just hang up.

  14. DixvilleCrotch

    He simply does not care for Nate Silver's gay math.

    BTW WTF is a parasitic remora? Is that something I was supposed to read a sentence or two about back in some 9th grade biology textbook (and still remember to this day, for this very moment)?

    1. prommie

      Little fish that has a sucker on its head, attaches itself to sharks, mentioned in the Jimmy Buffet song "Fins"
      "But now she feels like a remora
      'cause the school's still close at hand
      Just behind the reef are the big white teeth
      Of the sharks that can swim on the land"

      1. actor212

        Remoras are not parasites, tho. They are commensalists. A parasite works to the detriment of the host, but remoras do not affect their hosts much.

  15. Esteev

    I hear Newport on Marketplace all the time and he says the same thing over and over and over: Americans care about "the economy, jobs, the deficit, fixing government, and healthcare"

    You really need to call people to figure that out?

  16. Estproph

    Shorter Gallup: Waaaaahhh! We don't know what we're doing and Silver showed us up! We hates him forever we does!"

  17. Ruhe

    Newport is obviously assuming that no one is going to come along and bankroll a bigger, more ambitious operation for Silver.

  18. Steverino247

    Gallup gets it right when it doesn't matter to their management. When it does matter to their management, they try to pull the numbers in their desired direction. Here's why:

    "Born in Evanston, Illinois, Gallup received a bachelor's degree in religion from Princeton University in 1953. In 1954, Gallup joined his father's polling company, The Gallup Organization, where he worked until his retirement in 2004."

    Junior very badly wanted a Christian America and would do anything to make that happen, including lie about how many believers there really are.

      1. shelwood46

        Well, it'll make him stop. He died last year.

        Alec and Robyn were always nicer anyway (they all lived near me).

  19. Goonemeritus

    The truth is every Poll aggregator would have been even more accurate if the Gallop and Rasmussen “field” was left un-grazed.

  20. prommie

    Whats this about polling? Is this another Slutvember story? Has Petraus been polling someone else? Who else has been polling Petraus' Slut? Has anyone of flag rank NOT been polling Jill Kelley? Only Nate Silver can make sense of all this Slutvember polling thats been going on.

  21. Lucidamente1

    Shorter Frank Newport:

    But you can't hold all pollsters responsible for the behavior of a few, sick twisted individuals. For if you do, then shouldn't we blame the whole system of polling? And if the whole polling system is guilty, then isn't this an indictment of our educational institutions in general? I put it to you, Nate Silver – isn't this an indictment of our entire American society? Well, you can do whatever you want to us, but we're not going to sit here and listen to you badmouth the United States of America

  22. Defeatably_Joe

    We feel vaguely bad for Frank Newport, truly, because he represents a stolid and storied organization and he’s probably institutionally unable to give vent to his full-on snarky rage against his critics, as he’d like.

    Incidentally, I've felt vaguely bad for Nate Silver for basically the same reason since he's joined the New York Times, with the added fact that he's not a terrible person representing an organization that's spent the last several years degrading its own reputation. (That was only in the Bush years!)

  23. corthylio

    The lack of Gallup polls is probably the only reason Silver didn't do so amazingly well "predicting" Britain's parliamentary elections in 2010…

  24. Defeatably_Joe

    If you people insist on thinking Nate Silver is so great, everyone in the polling business will stop doing expensive polling and just start analyzing other people’s polls instead and then where will you be, huh?

    Here's a fun fact: this is identical to the argument that blogs are killing the news industry and that the Internet, at large, is killing culture, overall. If if any media history nerds are interested, you can almost certainly find similar arguments proffered back in the day, for VHS and Audio Casettes, Television, Radio, and the Printing Press.

  25. Not_So_Much

    Sounds like Frank Newport, like the rest of us, has a huge man-crush on Nate's frontal lobe (which I think we all know is abnormally large and girthy).

  26. Come here a minute

    This Dick can make it all up to the Republican faithful by selflessly contributing his time to a bogus charity.

  27. Defeatably_Joe

    Sean, I hope people aren’t mad at me about it … I spoke about what I believed and I think that there was a period of time when the Romney campaign was falling apart, people were not optimistic, nobody thought there was a chance of victory and I felt that it was my duty at that point to go out and say what I said. And at the time that I said it, I believe I was right.

    You know, appearing as an "analyst" without disclosing a conflict of interest, and making up numbers out of whole cloth in order to "do your duty" and support a political candidate's bid, is the sort of thing that would almost certainly be a MAJOR JOURNALISM SCANDAL at any actual news outlet. Remember when that guy for the New York Times made stuff up, and it was ONLY because he was just really lazy?

  28. BaldarTFlagass

    Hispanic buddy of mine got a tattoo on his back of the Zia sun symbol from the state flag. Says he's a new Mexican.

  29. WIDTAP

    I see. So the options are: polling or aggregating, but not both.

    Well, he is from Gallup, so the conclusion must be true.

  30. vulpes82

    What's so great about this "argument" is that Nate Silver would be the first one to say, "Yes! Totally true!" If there are no polls, he can't do his aggregation model. No polls, no Nate. Of course, it's true only in that it is a tautology and doesn't get into the actual issue, but, eh, whatever.

  31. Biel_ze_Bubba

    So Nate Silver is the Ariana Huffington of the polling industry? I can hardly wait for him to start bussing us to rallies in DC!

  32. Biel_ze_Bubba

    "[T]he Romney campaign was falling apart, people were not optimistic, nobody thought there was a chance of victory and I felt that it was my duty at that point to go out and say what I said."

    Not hard to see where he thought his "duty" lay lied.

  33. SayItWithWookies

    Then again, without Nate Silver, Gallup would't've known how wildly skewed their newly-unskewed polling was and gone back to their previously correct model before the election, would they? And now Mr. Newport would be defending his completely wrong model and its errant predictions instead of defending his swing to rationality at the last minute.

  34. mickeymusing

    Totally cannot fathom how Nate Silver become the focus of so much venom. If you don't like what he does, then ignore him. All this pointing out how evil he is just reinforces the fact that everyone else vilifying him got it wrong. Maybe just let it die now.

  35. Pithaughn

    so Dick admits to lying ( he can prettify that however he wants but he lied ), thus if FOX has him on ever again they prove once and for all that they are ok with liars as long as they tell the correct lies. Did'nt a sitting US senator write a whole book about this?

    1. Defeatably_Joe

      Honestly, it was probably going to be a while before Dick Morris was on again, anyway; with the election season over, Morris has to get back to his regular job, guarding a labyrinth along with a guy who only says things which are true.

  36. nfinitemonkeys

    What Gallup recognizes, to their dismay, is that Nate Silver not only picked the winners and losers in the political horserace, but also those of the polling horserace. Having the ability to compare the success and failure of polling firms over the entire pre-election period and over multiple elections will only strengthen polling. Gallups this ruins everything argument flies in the face of the fact that quality meta-metrics force the underlying providers to improve their methods and models.

    This is, by the way, different from the news agency versus blogger metaphor in that news agencies are not just supplying raw facts and numbers that can be analyzed. Their business model is to sell to consumers, and if bloggers draw from them, then their business may collapse. Polling firms sell to news agencies, political parties, businesses, etc… and will continue to have that market no matter how the meta-analysis goes. They will always have a few days lead over Nate Silver who must wait and aggregate. And in politics, a few days is forever.

  37. barto

    It's better known as the "tragedy of the commons", but I still don't get the analogy when your cows are basically shitting on everybody else's food, Mr. Gallup-poll-person.

  38. joobajooba

    Excuse 1: Obama tortured citizens into voting: "These methods may in the end affect voters who were not certain about voting at the time of a poll interview, but who were brought into the voting pool at the last minute by aggressive get-out-the-vote and late registration methods. "

    Excuse No. 2: (Debunked by Reuters/IPSOS numbers, Nate) – It's Sandy's fault. "He (Romney) held onto at least a marginal lead position in our polling until the week before the election, when Superstorm Sandy hit. Obama gained five points on the gap between our last pre-storm polling and the final poll. It may be that he continued to gain on into Election Day. "

  39. Slim_Pickins

    Do these polls have any importance beyond giving TV talking heads something to talk about? Are voters so clueless that they vote based on the polls, so they can say they voted for the winner? Not even the tea partiers are that dumb.

  40. docterry6973

    Several polling companies had very accurate polling data that predicted the outcome of the election. Gallup did not. Yet Gallup is threatening to deprive us of polling because of Nate Silver? They must be socialists. We must buy their product even if it sucks.

    If all polls ceased to exist today, I suspect my life would be happier for it. So please, be my guest. Stop polling.

  41. zumpie

    Hardly the first time Gallup's been way off the mark: when Puggies talk about "the polls" showing Carter leading Reagan (and losing from one bad debate), "the polls" was actually "Gallup". Every other poll showed Carter losing as early as May.

    But you know, it made for a nice nighty night bedtime story for them to warm themselves by…

  42. ttommyunger

    I've seen Nate on several appearances. He is a really likable kid, unlike most polling "experts" on the tee vee machine. I hope he has a great future.

  43. cousinitt

    OK, so I got out of the boat, went ashore to unskewed polls website looking for some tasty schadenfreude. And what did I find as a headline?

    "We're riding that 'Long Black Train' to economic oblivion"

    Not kidding, not even a little. These people are messed up.

  44. bobbert

    This "hate Nate" shit is amazing. The day after the election, some yoyo on Slate or Salon or something (I'm not gonna bother to re-find it) wrote a quite long piece, the gist of which was "Nate Silver is not a genius. It's the polling companies that do all the work, and they're pretty much right anyhow". (It was much longer than that.) Now this Gallup guy.

    They completely ignore what it is that Nate actually does. The body of work that underlies his his forecasts is this: He has detected evidence of statistical bias in the the published results of various polling organizations, and he has invested a lot of work in quantifying these biases, based on historical data. This is the secret sauce in his models. His model makes adjustments and/or assigns weights to the published polling results, in an attempt to create a less-biased composite result for each election.

    It's worth remarking that "statistical bias" is a mathematical concept. Nate does not presume that any polling organization intentionally skews its results right or left. The numbers speak for themselves — if bias exists, it exists, and it can be measured (historically) and corrected for.

    If Gallup, or any other pollster, wanted to put Nate Silver out of the elections business, all they would have to do is analyze their own results for statistical bias, and then take steps to either eliminate the source of bias or correct for it. In the short run, this would screw Nate up, because he'd be applying history-based corrections that were no longer appropriate. In the long run, it would remove his added value, because anybody can meta-average unbiased polls to get an unbiased composite.

    Based on Gallup-guy's remarks, it doesn't look like they're going to be trying introspection any time soon, so Nate's business model seems secure.

  45. fuflans

    good point frank. no one in the history of ever has ever used 'information' to do 'analyses' and 'draw conclusions' and 'make predictions'.

    man can you IMAGINE how bad THAT would be? why, such laxity might even produce, i don't know, a class of political pundits.

    that would suck.

  46. Tundra Grifter

    First of all, Gallup was off by quite a bit. They suspended polling for about a week and took that opportunity to get back on track. By Nov. 6th they were quite close.

    Second, Nate Silver is no Sludge. He doesn't just aggregate. He has a very complicated model that runs endless (well, not quite – just some 25,000 per day!) iterations of what the data really mean. All the polls are just the first step in his analysis.

    What about Real Clear Politics? They just post the polls and average them. TMC? I thought they were smoking some pretty good shit, but they came pretty close. Yahoo News averaged the various groups like Intrade.

    As others have pointed out, why do they dog Nate Silver? Because he is so good?

Comments are closed.