­
Font Size:
Pin It

This moment of crankiness is brought to you by JZ Analytics, The Zogby poll.

Earlier this week, my colleague and friend John Harwood wrote a piece for the New York Times about problems facing the polling industry. Among these issues: too many people hanging up the telephone, the growing numbers of Americans with cell phones only, and proper representation of voters by party identification. None of these problems is in any way new. None of these are crises. A new industry has been created that aggregates polls, talks inside baseball about polling techniques, and makes daily predictions from available polling data. I suppose all of that is fun; so is playing Soduku. They are both great entertainment but have little social use or make our world a better place. Polls are a measurement of a moment in time and provide important information about voter sentiment, core values that push people's buttons, and measure the salience of issues. Our obligation is to be as accurate as we can. Almost all of us achieve that accuracy on a regular basis. But most of us in this business are working our way through tremendous technological, social and cultural changes that have been accumulating for three decades. My companies that have produced the Zogby Poll have been on top of these trends from the beginning.

I have been a professional pollster for 28 years now and I have seen it all. I go back to the days of 65% response rates in the industry and reports that respondents would try to quiet down family members because "I have a long distance call and it's important; someone is asking me questions". Of course, there was no call waiting and there were very few answering machines back then, either. A long distance call was a major cultural event in many households.

In 1996, when I published my final presidential tracking poll for Reuters and got the results for each candidate within three-tenths of a percent, I shocked the industry when I suggested that this business is "80% science and 20% art". And I recall that by then response rates even in the heat of presidential campaigns had plummeted to 36%. So many things had happened and we were well on the way to a new era. The telephone was pushing us away. By 1997, I was asked by the late publication Public Perspective to peak into the future of polling and I argued that our future would be governed by the internet.

In the election of 2000, another one I got within less than a point of the actual results, response rates were already down to 26% nationally. By that time, my colleagues and I at Zogby International were already vigorously accumulating email addresses, collecting dozens of data points on each, conducting test polls to see if people would actually respond, and utilizing our in-house call center to validate respondents. By 2004, we were doing credible interactive polls nationally and in all 50 states. Our national numbers - both by phone and via the internet --were just about perfect and so were 80% of the states we did. There were critics, just as there were on the shores of Portugal begging Columbus to turn back. "Hey, Galileo, come down from the tower. The Pope is always right."

The cell phone is not new; hence it is not a new problem. There have been colleagues who initially eschewed calling cell phones, then began arbitrarily setting "standards" for the proper numbers of cell phones in each sample, then determining who was a genuine cell phone respondent and who was not. In short, some of my best known competitors were exercising the art of polling, just as I had suggested they would have to do. Those who deny new technologies and methodologies, criticize others for using them, and then arbitrarily embrace them are only causing confusion. But ultimately, a good pollster is one who embraces new means of communicating with consumers and voters can reasonably explain why his methodology works, and can prove that it does work over time. An even better pollster is willing to toss aside outmoded rules that just don't fit anymore.

Today, landline penetration is what it was in 1963. Calling cell phones is vital if we are going to do polls by telephone. But let's not be fooled: polling is not just asking who voters are going to vote for. Some polls are 40, 60, 80, even 100 questions. And those are not going to be done on a cell phone. Some pollsters, and I like to think we at JZ Analytics are again in the forefront, continue to experiment in alternative ways to reach people. In a way, I was pleased to finally see in Harwood's piece a moment of candor from a leader in my industry: "Anyone who claims there's a best practice doesn't know what they're talking about," said Paul J. Lavrakas, president of the American Association for Public Opinion Research. "We as an industry don't know." I appreciate the honesty and even the uncharacteristic humility, but there are indeed best practices. We still aim for random probability sampling - i.e. everyone in the polling universe has to have the same chance of being selected as everyone else. While that is a near impossible goal to achieve, I believe that the internet offers us a greater opportunity toward fulfillment than the telephone. Today, 83% of US households have internet access, including 91% of households with likely voters. When we do an internet poll, we draw a random sampling from a pool of people who are registered and already validated.

In the final analysis, no pollster possesses all the truth all of the time. But the industry is healthy because of the growing number of competitors and new methodologies being introduced. Now what we need is to ensure that consumers of polls - including the media, the bloggers, the aggregators, the pundits - understand that when we do polls we are merely taking a snapshot of a moment in time and are not predicting an outcome. Those who suggest that we are primarily predictors of election outcomes are missing the point of what good polls do. We are measuring where a campaign, a candidate, a message, or an issue stands at a given moment in time. Some of us happen to take better snapshots than others, but let's understand that there is no sudden crisis in polling.

Pin It
­