Was the 2004 election stolen?

[Mokele]

::Squints and peers into the distance::

 

Hey, what's that over there?

 

I'm saying it's suspicious that the actual results deviated to such an extreme degree from the exit poll results. In science, we say results are significant if there's only a 1 in 20 chance that they were the product of random variation. Here, we have a level over 100,000 times that, yet some don't want to consider it significant?

 

Hey, look, it's the original topic!

[Phi for All]

OK, barring more statistical skewing by both Dem and Rep sources showing how popular vs electoral college wins would've placed their candidates regardless of the other factors in question, I can still find nothing that refutes the fact that the exit poll information disagreed with the final tallies by a margin that is unprecedented since Mitofsky invented them. This study is one of the only ones I've found that is trying only to access the validity of the process and how it was implemented, without trying to overtly suggest anything was tampered with. I'm really only interested at this point in whether or not my view of exit polls as accurate statistics was well-founded.

 

Given that the exit polls show there might be something wrong with the way the popular vote was counted, is there any other evidence to support this? Remove the aggressive geurilla tactics employed by both sides and the question that begs to be asked is were any of the tactics used in Ohio illegal? Did any cross the line from the aggressive gray area to being against the law?

 

Related to the exit poll question is this: Why didn't the exit poll information coming in all day (in favor of Kerry) skew the vote even more in his favor as undecideds across the country typically figure to back the winner as election day progresses? Bush was able to capitalize on this trend in 2000 when Fox news predicted his Florida win ahead of any other network, apparently biasing the vote in Western states.

 

Also in question are the voting machines used. We've heard the stories about people pushing the "KERRY" button only to see "BUSH" light up, but did the opposite happen just as often? I think we can all agree that the malfunction that tallied several thousand votes for Bush in a precinct where only a fraction of that many showed up to vote is further damning evidence, but were there any malfunctions like that that favored Kerry? Would glitches like that be a "wash" in the long run?

[Pangloss]

I'm not quite following something.... how big was this margin of error? With an election this close, wouldn't the margin of error be way outside of the margin of the election itself?

[Phi for All]

I'm not quite following something.... how big was this margin of error? With an election this close, wouldn't the margin of error be way outside of the margin of the election itself?

The discrepancy was 5.4%, the margin of error for former exit polls was less than +/- 1%, and the sampling for 2004 was six times the normal size (12,219 voters), which should have made it the most accurate ever done.

 

Exit polls had Kerry winning by just under 5 million votes but the count had Bush win by 3.3 million.

[Nevermore]

8.3 million is a lot of people. That's like the entire population of New York City having their vote nixed. Something smells fishy.

[Pangloss]

No, not quite yet it doesn't. There's a key piece missing before a conclusion of "fishy smell" can be reached.

 

First of all, this is interesting, but I don't think it warrants grandiose descriptions like "six times the normal size", etc. It could easily be a very small problem in their polling methods that caused the discrepency.

 

I guess I'm still not seeing a reason to think that their exit polling system is more accurate that the voting system. You're still, in the end, talking about human beings asking other human beings for information and then jotting that information down on a piece of paper. The opportunities for human "skew" in that kind of a system are legion. The voting system, on the other hand, is, assuming it hasn't been tampered with, not subject to ANY kind of skew at the individual level.

 

For example, a small number of poll workers who really wanted to see Kerry win could easily have come up with that degree of skew. They know full well that a certain percentage (far in excess of the margin of victory) of west coast voting is influenced by east coast poll numbers, and IIRC the data was widely dissiminated all that afternoon.

 

(I know you aren't claiming to have answered this question and I appreciate your efforts. This is interesting stuff.)

[Phi for All]

I guess I'm still not seeing a reason to think that their exit polling system is more accurate that the voting system. You're still, in the end, talking about human beings asking other human beings for information and then jotting that information down on a piece of paper. The opportunities for human "skew" in that kind of a system are legion. The voting system, on the other hand, is, assuming it hasn't been tampered with, not subject to ANY kind of skew at the individual level.

I have to disagree with this. One of the things that make the exit polls attractive to me is the uniformity of their methodology as opposed to the voting system. Exit polls are tallied on preprinted cards (you made it sound like they were using crayons on Post-its) before being uploaded to computers by people with lots of training who have done several test runs in the precincts picked for the polls. They were hired by a group of broadcast news services ranging from AP to CBS to Fox. They know the distance they have to maintain away from the voting place, the tone of voice to use for objectivity, and almost every other factor that could apply. Only the weather can affect them to any great degree and they've been trained for inclement conditions as well.

 

The voting system, on the other hand, uses a combination of methods that make it more cumbersome. Optical scans, DREs, punch cards, paper ballots, lever machines, and data-point devices tally votes and then transmit them in a variety of ways to four different *privatized* corporations. This system has numerous holes in it and seems much more susceptible to tampering. I'm still not saying it was tampered with, just that the possibilities seem higher.

 

The other sticking point for me is that the exit polls were accurate to within 3/10 of a percent in precincts where Kerry had an 80% lock on the vote, yet they were off by an average of 10% (!) in precincts where Bush had 80% of the vote. This disparity hasn't been effectively answered for me yet.

[bascule]

I think Diebold fu*ked up. Was it a malicious attempt to skew the results of the election? I don't know... the angry Bush hating liberal in me wants to cry conspiracy over the Diebold contract, but I have no evidence, and even if I did, it's a moot point now. So I've kind of grown apathetic...

[Pangloss]

I have to disagree with this. One of the things that make the exit polls attractive to me is the uniformity of their methodology as opposed to the voting system. Exit polls are tallied on preprinted cards (you made it sound like they were using crayons on Post-its) before being uploaded to computers by people with lots of training who have done several test runs in the precincts picked for the polls.

 

I wasn't trying to characterize it at all' date=' I'm doing the opposite -- pointing out that we (or at least I) don't have enough information TO characterize it.

 

Okay, you don't want to talk about pollsters making mistakes or deliberately skewing numbers. How about the simple fact that [i']they only asked a little over twelve THOUSAND people?[/i] That's a statistical sample of 0.01%! You don't think that getting hung up on a margin of error of 4 percentile over a spread in which each individual respondant counts for an absolutely VAST number of voters is a bad idea? Do you understand that that means that only ~600 respondants have to be in error in order to reach that margin?

 

I don't disagree with the particulars here. I'm just making the point that there's a reason why the old saying goes "there are lies, damned lies, and statistics."

[Phi for All]

I wasn't trying to characterize it at all, I'm doing the opposite -- pointing out that we (or at least I) don't have enough information TO characterize it.

I understand. I'm not a statistics buff either, but the accuracy of exit polls in particular is being touted by those who do understand it. Again, it's mostly because of the nature of these polls. There's nothing speculative about the questions being asked, the voters *just* voted and there is no reason for the pollsters to fake anything.

 

Okay, you don't want to talk about pollsters making mistakes or deliberately skewing numbers.

There is no motive for this. The pollsters are paid for accuracy and since the counted tallies will be coming in within the day it's not in the pollsters best interest to be wrong. Not saying it couldn't happen but add in some of the vote tally irregularities and the exit pollsters go to the bottom of the suspect list.

How about the simple fact that they only asked a little over twelve THOUSAND people?

Here is a Random Sample Calculator that tells you you could have a 95% confidence level in the results from a national poll of all US voters from 2004 (with a 1% error tolerance) using only 9600 respondants. Once again, I'm not a big statistician, but statistical sampling using a pretty black and white question like "Who did you vote for for President today?" seems to be about as accurate as anything you're going to get from paid professionals.

 

I admit that some of my prejudices about the voting system are coming into play here. I've never liked the idea of black box voting, touch screens or any type of computer tallying without a paper back up that can be checked before it drops into the lockbox. I also don't like it that election officials are part of a particular party. I've always thought there should be some way to make the counts more independent. I like Mokele's idea of having a week to vote and a month for counts and recounts. Perhaps the results of partisan counts could be averaged against at least two independent counts.

[Mokele]

How about the simple fact that they only asked a little over twelve THOUSAND people? That's a statistical sample of 0.01%!

 

That's by no means a small sample, and is, in fact, more than sufficient to ensure accuracy. What percentage it is of the population is irrelevant; population size doesn't matter, only sample size. The bigger the sample, the closer you get to the true mean of the population.

 

Think of it like this: say you have a population of bacteria, and you want to count the number of flagella per cell. One cell is a crap sample; it could be the mean, or it could be an extreme individual. If you take ten cells, you're more likely to have your result dominated by individuals closer to normal rather than rare exceptions, even moreso for 100 cells. By the time you reach 10000 cells, you've got a *damn* good idea what the mean is. Whether it's from a single test tube with a million cells in or a 50 gallon drum with a quadrillion doesn't matter; after a certain point, you get diminishing returns in terms of accuracy, with large increases in sample size resulting in only miniscule increases in accuracy. How big the actual sample needs to be depends on the variability in the system. Since this is a very, very simple system (you're reporting past actions that fall into one of 2 choices, maybe 3 for 'other') with a categorical output rather than continuous output, even modest sample sizes can result in great statistical power.

 

Mokele

[john5746]

I think the sample characteristics may come into play here. There are so many variables, race, location, age, etc. Maybe Bush voters were more likely to stand in line longer to vote, etc. In any case, it does appear something unexpected happened statistically. I think it is probably more to do with the sampling process itself, but it is a valid problem. I agree with bascule that it is a done deal, but if some tampering were done, we should try and determine how it was done so it cannot be repeated.

[Pangloss]

That's by no means a small sample' date=' and is, in fact, more than sufficient to ensure accuracy. What percentage it is of the population is irrelevant; population size doesn't matter, only sample size. The bigger the sample, the closer you get to the true mean of the population.

[/quote']

 

My point was not that the sample is too small, but that it may be too small for the degree of accuracy claimed.

[Pangloss]

Also, the accuracy rating of an exit poll doesn't indicate its degree of accuracy with respect to the voting population. It indicates a degree of accuracy with respect to the people who were polled.

 

So the poll could be completely different from what the voters actually did, and still be exactly as accurate as it claims to be.

[Mokele]

But it's not, that's my point. Statistics operates on the assumption that the actual population is effectively infinite. The accuracy of the sample is determined by the sample size. This isn't something debatable; it's the primary law of statistics, the Central Limit Theroem, that as N increases, the sample mean progressively approaches the true mean.

 

The sample size isn't too small for the degree of accuracy claimed because the degree of accuracy is *calculated* from the sample size, simple as that.

 

Those degrees of accuracy are standard error, which is defined as (std deviation / sample size) ^ 0.5, or (variance / N^0.5). The standard deviation and varaince are properties of both the population and sample, as measures of varaibility in each. So, as you can see, for a given variance, increasing N decreases standard error, but the effect of going from N of 10 to 20 is much more than from N of 100 to 200. Essentially, there comes a point where each additional measurement only modestly increases your accuracy.

 

I'd like to see some reasoning as to why the degree of accuracy is claimed, especially since there are *legions* of statisticians perpetually honing these things.

 

Also, the accuracy rating of an exit poll doesn't indicate its degree of accuracy with respect to the voting population.It indicates a degree of accuracy with respect to the people who were polled.

 

Actually, the accuracy with which it indicates the total population's voting is *precisely* what it indicates.

 

Think about it: Why would there be a margin of error for a sample? You know precisely what every point in the sample is. The margin of error is how well the sample reflects the actual population mean.

 

So the poll could be completely different from what the voters actually did, and still be exactly as accurate as it claims to be.

 

And the probability of that happening with random sampling methods is calculable. It's called the margin of error, and is what they give you. Unless there's a methodological flaw (which, as the original link indicates, there's zero evidence for), we can accurately calculate how likely our population mean is to be outside this margin of error.

 

---------------------------

 

Also, is it just me, or is it hypocritical of us to say "Well, if your election differs from the polls, your election is suspect, but if our election differs from the polls, the polls are suspect."

 

Mokele

[Pangloss]

The sample size isn't too small for the degree of accuracy claimed because the degree of accuracy is *calculated* from the sample size, simple as that.

 

But the degree of accuracy was stated at +/- 1%, and the relationship between the sample and the general population was 1/100th of a percent.

 

 

Think about it: Why would there be a margin of error for a sample? You know precisely what every point in the sample is. The margin of error is how well the sample reflects the actual population mean.

 

I don't think that's right, I think the margin of error must be for the polled sample, not for the theoretical sample it's polling against. You DON'T know what every point in the sample is -- people make mistakes. The data *cannot* be perfect -- it has to have some degree of accuracy measured by a statistic.

 

So really if what you say is true then we should have TWO accuracy statistics -- one for the accuracy of the polled data, and one for the accuracy against the general population. (Are you saying they're just leaving out the first statistic because they feel it's too small? If that's true then I need to see some sort of indication of that -- a quote or something.)

 

I've always assumed this is how polls work -- that the accuracy percentage given is for the accuracy of the polled sample. I've heard statements to that effect from talking heads on TV (granted with no more knowledge of how to tie their shoelaces than an understanding of statistics). (grin)

 

If you find anything more definitive on this please pass it on.

[Mokele]

But the degree of accuracy was stated at +/- 1%, and the relationship between the sample and the general population was 1/100th of a percent.

 

As I stated before, not only is the size of the population irrelevant, but statistics actually assumes it to be infinite or effectively so.

 

I don't think that's right, I think the margin of error must be for the polled sample, not for the theoretical sample it's polling against. You DON'T know what every point in the sample is -- people make mistakes. The data *cannot* be perfect -- it has to have some degree of accuracy measured by a statistic.

 

People make mistakes when reporting something they just did? The article specifically mentions exit polls as having great reliability for just that reason: rather than asking people to predict their actions, they're asking people merely to report what action they took in the recent past.

 

Now, there's probably are fancy statistical techniques to deal with mis-reporting or general human stupidity ($10 says there's some sort of official statistical principle they don't publically admit that says 'people are morons' or somesuch). However, when statistics are given for the mean of a population +- x, what that typically indicates (unless they do something totally different in polling from what I learned in Data Analysis this quarter) is that based on the sample (including the sample size) we can conclude that the actual population mean is somewhere in that range.

 

That's actually part of why this caught my eye: I've been having stats drilled into my head for 8 hours a week plus homework. As a result, I got a pretty good feel for sample sizes and such (damn useful course, the TA got hired and most of what they asked in the interview was about this class), and when I saw the numbers in that article, alarm bells went off.

 

If you like, I can try to dig out some more formal stuff from my stats text, but this is pretty elementary stuff; sample sizes, central limit theorem, all that.

 

Mokele

[Pangloss]

People make mistakes when reporting something they just did? The article specifically mentions exit polls as having great reliability for just that reason: rather than asking people to predict their actions' date=' they're asking people merely to report what action they took in the recent past.

[/quote']

 

Absolutely. I do all the time. More to the point, amongst a group of more than twelve THOUSAND people, I would EXPECT some number of them to be in error, or that some error would be made in recording their responses.

 

And so should you. What would be bizarre here would be expecting that number to be perfect. That just defies common sense.

 

 

If you like, I can try to dig out some more formal stuff from my stats text, but this is pretty elementary stuff; sample sizes, central limit theorem, all that.

 

Mokele

 

No, what I'm looking for here would be some sort of statement or instruction manual on polling that indicates what that accuracy rating actually means to the pollsters. What do THEY think that "+/- 1%" means. If you find a source along those lines, I'd be interested in that.

[Pangloss]

Interesting response to Kennedy's article in Salon by Farhad Manjoo:

http://www.salon.com/news/feature/2006/06/03/kennedy/index_np.html

 

It's locked behind a login, but I'll see if I can post a couple of excerpts from it when I get home tonight.

[Pangloss]

Interesting... Salon's articles have generated quite a firestorm in the liberal blogosphere community. Kennedy himself wrote a response to Manjoo's criticism, and Manjoo wrote a response to THAT. And Salon's editorial board posted a position of its own, basically saying "hey it's not our fault, we HATE Republicans!"

 

http://www.salon.com/opinion/feature/2006/06/06/rfk_responds/

http://www.salon.com/opinion/feature/2006/06/06/salon_answers/

[Pangloss]

Mokele, if you take a close look at page three of the first link in the post just above, you'll see an interesting tidbit about the exit polls:

 

I was relying on the very same numbers he cites -- researcher Steven Freeman's exit poll data -- to show that John Kerry's lead was well within the margin of error in Iowa, Nevada, New Mexico and Ohio (the four states in which the exits showed Kerry ahead that he eventually lost).

 

According to the next paragraph, the margin of error was as high as SEVEN PERCENT in some of those states.

 

The way he writes about it seems to support your interpretation of what the margin of error represents (that it refers to the accuracy of the poll against the voting population rather than the accuracy of the poll against internal mistakes). Maybe I'm just completely out in left field on that, I don't know. I still want to see some kind of definitive statement on it, if such can be found.

[Mokele]

According to the next paragraph, the margin of error was as high as SEVEN PERCENT in some of those states.

 

This means, simply, someone is wrong. If the margin of error is 7%, and the results fell within that, all's fair. If it was 1% and the results were outside of that, there's a serious problem. So who's telling the truth, and who's cooking the numbers?

 

The way he writes about it seems to support your interpretation of what the margin of error represents (that it refers to the accuracy of the poll against the voting population rather than the accuracy of the poll against internal mistakes). Maybe I'm just completely out in left field on that, I don't know. I still want to see some kind of definitive statement on it, if such can be found.

 

The way I'm talking about it is the default, used in statistics long before polling ever was born, for things like mean bird beak width and mean number of nematodes per cubic cm of soil. That's probably why it's reported; it's tradition that you report it that way, and deviating from that means you need to spend 10 pages explaining what you did and why, defeating the point of a quick representation.

 

Now, I'm willing to bet there's some sort of correction, calculation or something that goes on to correct for the fact that people are idiots. It may even be factored into that final +- x% answer.

 

Unfortunately, I don't know polling specifically, just stats in general.

 

Mokele

[Pangloss]

Well I guess I'm getting schooled here, but I can't complain -- at least I'm learning something new in the bargain!

[Mokele]

Bah, I didn't really 'get' stats until this class, and even still, I'm no stats major; I've learned enough to design experiments, test them properly, and pick at other people's results, but none of the deep statistical theory.

[swansont]

More on Diebold voting machines

 

http://gizmodo.com/gadgets/gadgets/how-to-steal-an-election-with-a-diebold-machine-200693.php

 

"Some Princeton researchers made a demonstration video of how it's possible to steal an election with a Diebold voting machine in under a minute. Anyone with physical access to the machine can put in malicious software to steal votes—such as election workers who have unsupervised access to the machines before elections."

 

There is a very interesting 10-minute video

 

princeton site: http://itpolicy.princeton.edu/voting/

 

"[i am] committed to helping Ohio deliver its electoral votes to the president next year." Walden O'Dell, CEO, Diebold Inc. Aug 2003