Remarks to the Retired Faculty Association of UNC-CH, April 14, 2016
The more things change, the more they remain the same. You have probably noticed by now that there are a lot of public opinion polls circulating in the current national election campaign — and a lot of criticism of those polls. Now here’s the irony: This year, 2016, marks the centennial of the first media-sponsored political poll. So where is the celebration?
My old outfit, the American Association for Public Opinion Research, has not scheduled one. Neither, so far as I know, has the present incarnation of the Gallup poll. I think I know
why. That historic media poll in 1916 was quite informal. The Literary Digest, a national weekly magazine, invited its readers to write in with reports on local sentiment regarding the presidential election. It was between Woodrow Wilson and Charles Evans Hughes. Based on the response, the magazine predicted that Wilson would win re-election. And he did.
Encouraged by that result, the magazine made a more formal effort in 1920. The candidates were two Ohio newspaper publishers, Democrat James M. Cox, and Republican Warren G. Harding. It sent out ballots to as many names and addresses as it could collect and predicted a Harding victory. He won, and they figured that validated their methodology. Its main rule was to get as large a sample as possible, and it worked for three more presidential elections. In 1932 it called Roosevelt’s winning margin within a fraction of a percentage point. But it was just lucky – or unlucky as it turned out.
George Gallup was a student at the University of Iowa in that year. He worked on methods for measuring reader interest in different parts of the newspaper. His dissertation in psychology was titled “An Objective Method for Determining Reader Interest in the Content of a Newspaper.” It attracted the attention of a New York advertising agency, which offered him a job. He did his first political poll for his mother-in-law who was running for Iowa secretary of state. It predicted that she would win, and she did.
In 1935, Gallup moved to Princeton and founded the American Institute for Public Opinion. The following year, he got really lucky. He used a technique called quota sampling. Interviewers were free to interview whomever they could conveniently find provided they met specified categories of geography, age, gender, and ethnicity to match the voting-age population. By 1936, the New Deal had created a powerful coalition of Democrat voters that included white southerners and lower-income groups in the great cities—African-Americans, union members, and ethnic and religious minorities, many from recent immigrant groups.
Now here’s the important thing about sampling theory. A biased sample is harmless until the bias correlates with the thing you are trying to measure. In 1932, it did not. The Literary Digest got its names and addresses from the most convenient sources available, telephone listings and automobile registrations. In the 1930s – the time of the Great Depression – these were indicators of wealth. I was a child in the 1930s, and my family had neither a car nor a telephone until after World War II. We were not unusual. For the 1932 election, the one that the Literary Digest got right on the money, wealth did not correlate with presidential choice. The New Deal changed that, and by 1936 the two parties were aligned by social class.
Gallup realized this, and he pulled a hat trick. The Digest survey had 2 million returns to count. While it was doing that, Gallup used the same lists to get about 3,000 returns. And so while the Literary Digest was still tabulating its 2 million ballots, Gallup announced that the magazine would erroneously predict a landslide victory for Alf Landon of Kansas, the Republican candidate.
“The most accurate prediction I ever made,” Gallup recalled when I met him in the 1970s.
Expectations for the Literary Digest were high because of its accuracy in 1932, and it never recovered from its humiliation in 1936. It published its final issue in 1938. Gallup and his peers enjoyed the trust of the public until the next major polling failure in 1948. All of the polls got it wrong that year. You might have seen the classic photo of a smiling Harry Truman holding up a front page of the Chicago Tribune with the headline, “DEWEY DEFEATS TRUMAN.”
And that election marked the end of quota sampling, even though it was not the main cause of the polling errors. Gallup and his peers had assumed that voters made up their minds by mid-October, and so they just stopped polling. They failed to detect a last minute shift by disaffected Democrats who didn’t like Truman, but decided — at the last minute — that they liked Thomas Dewey even less. After that, Gallup kept his poll going right up to as close to election day as he could manage.
His 1952 and later polls were based on a stratified probability sample of housing units in a matrix of 49 cells created from seven geographic categories and seven size categories. Housing units still made a good sampling frame in those days because the population was more stable then and interviews were done in person, in the home. On weekdays, interviewing hours were 4 to 10 p.m. when working people could be found at home. Daytime interviews were held on weekends. Interviewers would ask for the youngest voting-age male present at the time because males and young people were less likely to be found at home. That amounted to a rough quota sample at the block level, but it was easy to execute.
Gallup’s best years spanned the presidential elections 1956 through 1976 – Eisenhower through Carter. Error on the winner was held to less than three percentage points in that period. He had a mild-mannered statistician, Paul Perry, who worked out ingenious ways to erase two barriers to accuracy: determining who in the sample would actually vote and allocating the undecided.
Housing units still make a good sampling frame, but I doubt that a house-to-house survey would work today. We’re no longer used to having strangers knock on our doors. I can remember when milk, ice, dry cleaning, and groceries were routinely delivered to the home. The mail came twice a day. When I was a newspaper carrier, I collected the subscription fees in person. The world was a gentler and more trusting place then, and survey response rates of 80 percent were not uncommon.
It was not until the 1970s that telephone interviewing became the more practical method. Phone service reached 95 percent of households by the middle of that decade. T he telephone company was a regulated monopoly, and that made samples of household numbers easy to get. There was a time, in fact, when the phone company would give us the ranges of working residential telephones for each three-digit prefix. Take a sample of those prefixes, add four digits from a table of random numbers, and you had a pure probability sample. It doesn’t get much better than that.
But then the telephone monopoly was broken up, cell phones started to become popular in the 1980s, and digital cellular networks started growing in the 1990s. And the problem of telephone sampling became seriously complicated. You can reach people easily enough, but knowing the probability of any individual being reached can only be roughly estimated. There are just too many ways to reach us, and we are devolving to the reliability of the Literary Digest.
That doesn’t stop the media from doing polls of course. And the myth still prevails that the quality of a poll is a function of the number of people who respond. The Internet offers many possibilities, none of them very good. There is the poll as click-bait. “Take the poll,” says a pop-up message on your computer screen. We call this an opt-in poll, where the respondents select themselves.
Then there are the Interactive Voice Response or IVR polls. These use robocalls. Your phone rings, a recorded voice invites you to take the poll, and you respond by punching buttons on your phone. They have a strong bias in favor of people who don’t mind talking to machinery.
Some sense is made out of all this confusion by the poll aggregators. The best of these, I think, is Nate Silver with his blog, “538.” He evaluates polls by their track records, their methodology, and their acceptance of certain industry standards. He weights their results accordingly in his model and uses it to estimate a probability that a given candidate will win. It works pretty well – with a few exceptions. For example Silver gave Hilary Clinton a greater than 99 percent chance of winning the Michigan primary on March 8. She lost to Bernie Sanders.
So what are polls for? There must be some higher purpose than predicting an election. We’ll know who the winners are a few hours after the polls close. Isn’t that soon enough?
In the 1990s, Davis Merritt, a Morehead scholar from Hickory, who became the editor of the Wichita Eagle-Beacon, proposed something that he called “public journalism.” He defined it as a conscious effort to report the news in a way that would make public life go well. One of its elements was using polls, not to predict elections, but to find out what issues most concerned voters. Then news coverage could be adjusted to focus on those concerns in the election campaigns, forcing candidates to address them.
The idea caught on. At one point in the 1990s, the New York Times swore off using polls for pre-election predictions. Instead it tried to use them to illuminate the process of coalition formation. Thus it would compare the candidates’ support among specific groups, blacks and whites, for example, or men and women. If you knew the proportion of these groups in the population and remembered tour high school algebra, you could calculate the horse-race standings on the back of an envelope, and I had a little bit of fun with that.
But the idea didn’t stick, and reporting the horse race is still the main application of pre-election polls. I don’t think it’s a bad thing, providing the polls are accurate. It’s a good way to sustain public interest in the campaign. Ånd the election is a good way to assess the validity of the polls.
An even better use of polls is to illuminate the process of coalition formation. The founders made that obscure by separating the executive and legislative powers. No provision was made for political parties, but they arose spontaneously as like-minded citizens banded together to try to gain control of the government. And that is what makes our politics so interesting.
The world changes pretty fast, and the changes often inspire dystopian theories about America’s future. In 1973, I took a year off from my job as a Washington correspondent for the old Knight Newspapers to help write a textbook on the national government of the United States. I was well into the project when the Watergate affair led to the constitutional crisis remembered now as “the Saturday night massacre.” That was in October 1973. We stayed on task, finished the book on schedule, and prayed that the United States would still have the same form of government when the volume finally came off the press. We gave the book a title based on something Benjamin Franklin said after the close of the constitutional convention of 1787. Asked what form of government the convention had designed, he replied, “A republic – if you can keep it.” So we called the book “To Keep the Republic.” A more recent book by Harvard law professor Larry Lessig used the same anecdote in a more pessimistic way. His book is about the effect of unlimited special interest money in political campaigns. Its title: “Republic, Lost.”
But the constitution of 1787 is more resilient than we think. It was written in a spirit of pragmatic compromise, and that set the tone and the precedent for most of the future conflicts. It was also the dawn of the industrial age, which enabled mass production and mass media to sell its products. The first newspapers were the voices of narrow political or business interests. Then publishers discovered the economies of scale that could be obtained by lowering the price from six cents to a penny and getting most of their revenue from advertising. That started around 1830 – mass production for mass audiences. With so many people getting the same information, the spirit of compromise and adjustment could continue.
Newspaper consumption peaked in the 1920s when commercial radio began to compete for the attention of the audience. I’m not talking about the number of newspaper readers but newspaper sales as a percentage of households. The peak was 130 percent. Yes, every day, 130 papers were sold for every 100 households in the USA. Now it is about 40 per 100 households. The decline in the 1920s followed the start of commercial AM radio, which competed for readers’ time. After that came other new technologies: television, FM radio, the Internet. The older technologies survived by finding more specialized audiences. The Internet, of course, supports the most specialized content of all, and that could be one cause of the political polarization that we’re seeing. It’s too tempting to attend only to those messages with which we already agree. We’re losing the moderating effect of mass media.
But more than that is going on in this campaign. I think that what we’re seeing is a realigning election. It happens every 30-40 years or so. Our single-member district (winner-take-all) system discourages third parties, and fairly stable coalitions are formed in the election campaigns. This is why we need polls – to see what groupings of strange bedfellows are being formed. The Republican coalition, forged in the 1964 and 1968 elections, is inherently unstable. It links economic conservatives and social conservatives. When the civil rights act of 1964 was passed, Democrats knew it would cost them their traditional solid support in the south. Republicans talked about “states rights” as code for segregation, and that dog-whistle tactic worked. It led to the “southern strategy” that provides tangible benefits to the economic conservatives but only symbolic reward for the social conservatives. But now we can see it fraying, and we can thank Donald Trump for that. He speaks clearly to the social conservatives and reminds them how little the party has done for them. He is a destabilizing force, and that might be a good thing.
Coalition formation is more stable in a European-style parliamentary system because it encourages multlple parties who can appeal to more specialized groups. The coalitions are formed after the election, in full view of everyone, as the winners negotiate to form a majority and elect a prime minister. In our system, we can’t watch that process without polls, and I wish our news media appreciated that more.
Pollsters today try to sample from a variety of sources, and then they use statistical weights to make their samples match the population. It’s like quota sampling, and there is always under matching. The hidden bias will be revealed only when it correlates with voter choice. Will this be the election that shocks and surprises us like 1948? The more things change, the more they remain the same.