Formula for Judging the Accuracy of Predictions: March Madness

[TreueEckhardt2]

This post deals with a mathematical formula for deciding how well people have made predictions. I would like to receive feedback about this formula and, if possible, test it out with participants in this group in the upcoming March Madness NCAA basketball tournament.

In order to help everyone understand what I'm talking about, I'll start with an example. Suppose three people, Adams, Benson, and Carter are watching a basketball game between Iowa State and Connecticut. Adams thinks that Iowa State has a 60% chance of winning, Benson thinks that Iowa State has a 70% chance of winning, and Carter thinks that Iowa State has an 80% chance of winning. Suppose Iowa State wins. How do we know who has made the best prediction?

The following game that is designed to answer this question. The participants in this game make a series of predictions about the outcome of future events. They predict the percentage chance that certain events will happen. Later, when it is determined whether or not the event actually did happen, each participant gains or loses a certain number of points based on their predicted percentage chance that the thing would happen.

Below is an example of this game followed by the mathematical formulas on which this game is based.

If you predict Then if they win, And if they

that the team you win this lose, you lose

has this chance number of this number of

of winning points points

1% 202 1

2% 408 4

5% 1052 28

10% 2222 123

15% 3529 311

20% 5000 625

25% 6667 1111

33% 9851 2426

40% 13333 4444

45% 16364 6694

50% 20000 10000

55% 24444 14938

60% 30000 22500

66% 40000 40000

75% 60000 90000

80% 80000 160000

 

You should circle no more than one of the percentages in each of the lines below. You should circle the percentage chance that you think each of these teams has to win their game(s).

South

What is the probability that

#1 seed Kentucky will win its first five games

1% 2% 5% 10% 15% 20% 25% 33% 40% 45% 50% 55% 60% 66% 75% 80%

#9 seed Connecticut will win against #8 seed Iowa State

1% 2% 5% 10% 15% 20% 25% 33% 40% 45% 50% 55% 60% 66% 75% 80%

#8 seed Iowa State will win its first two games.

1% 2% 5% 10% 15% 20% 25% 33% 40% 45% 50% 55% 60% 66% 75% 80%

Now let's run through a betting example from beginning to end. First, a person named Adams predicts that there is a 40% chance that Kentucky will win its first five games, a 40% chance that Connecticut will win its game against Iowa state, and a 40% chance that Iowa State will win its first two games.

Second suppose that Kentucky does win its first five games, but Connecticut loses to Iowa State, and Iowa State loses its second game. Since Kentucky won its first five games, Adams gains 13333 points, but since Connecticut lost, he loses 4444 for that bet, and he also loses 4444 points for his bet on Iowa State winning its first two games. For these three bets, Adams gains 13333 points and loses 8888 points, and he has a score of +4445.

 

Now for the formulas:

The formulas are based on the following idea: Suppose I ask you to make a bet. You are betting that X will happen. If X does happen, then you will win some points, but if X does not happen, you will lose that number of points squared. For example suppose I ask you to bet on whether a tossed coin will lands on heads. If you bet two points, then if the coin lands on heads, you get two points, but if it lands on tails you lose four points. Betting two points is clearly not a winning strategy, but you can also bet a number smaller than one. Suppose you bet 0.5 points. Then if the coin lands on heads you get 0.5 points, but if it lands on tails you only lose 0.25 points. That's a winning strategy for a coin toss. We want to find, not just a winning strategy for this game, but the optimal strategy.

To do this, we need to write out a formula for the game. This formula is given below.

 

s=pb-(1-p)b^2

 

In this formula, p is the probability that something will happen, b is the number of points that you bet that the thing will happen, and s is the average score that you would expect to receive if you made a large number of bets with a fixed probability. In this situation, we are assuming that you know the probability that the thing will happen, and you are trying to choose the best number of points to bet in order to get the highest possible score. So p is a constant, b is the independent variable, and s is the dependent variable. We find the optimum score by finding the maximum point of the graph of this equation. Since the graph of this equation is a parabola, the result is straightforward. It will have a maximum where b has the following value:

 

b=p/2(1-p)

 

So, if Adams thinks that Connecticut has a 0.4 chance of winning a basketball game, then he should bet approximately 0.33 points.

 

If Connecticut does win, then he will gain 0.33 points. if they lose, he will lose 0.11 points.

Now look at the table below. This is the table that we have been using for the basketball prediction game that I have just been describing.

1% 202 1

2% 408 4

5% 1052 28

10% 2222 123

15% 3529 311

For example, suppose a team has a 1% chance of winning. Then the amount that you should bet is 1/198, and the amount that you will lose if the team loses is 1/39204. I multiplied both of those numbers by 40,000 so that the numbers in your score would be whole numbers rather than fractions or decimals. I then multiplied all of the other values for 'b' by 40,000 and rounded them to the nearest whole number.

I have tested this prediction game with some friends and family and also several times with my high school math classes. I would like to test it out in this forum and I am interested in receiving feedback on it. Please let me know if you are interested in participating.

[TreueEckhardt2]

Maybe it would help if I talked a bit about what kind of feedback I'm looking for. It seems that this is quite a simple problem and it seems that someone else would have found a solution for it, but I haven't found any other mention of a solution for this problem. If someone has seen a different solution to this problem, I would appreciate if they could tell me where I could find it. Just so we can be clear about what I am looking for, let me restate the problem. Suppose someone makes a series of predictions that certain events have a certain probabilities of happening. The predicted probabilities are not close to one and not close to zero. Some time later, we see which events have happened and which have not happened. How do we judge whether the predictions were good predictions or bad predictions?

I would love to hear comments on this topic.