Error function erf(x) and when to use it

[Dagl1]

Hi there, 

I have just started going through a python programming book and one of the functions presented is the error function. I understand it is not very necessary for me to understand the mathematics behind it, but I would just like to. Now from looking at, reading a bit about it and watching a video how it is derived, it is my current understanding that the function computes the probability that a Random Variable (if assumptions of normal distribution, standard deviation and expected value are all met) can be found within [-x, x]. 

I think two things are unclear for me, firstly: what exactly is a Random Variable, the wiki article and some other websites talk about it as if it just any variable that is determined by chance, such as the rolling of dice (I presume this cannot be used in error functions due to the lack of normal distributions). However I don't understand how this could be used (so most likely I am misinterpreting the explanations on the internet). Let's say I measure how tall some people are, and the people I choose are randomly picked with no bias in selection. That Random Variable's value would be.... what? 

Secondly I don't understand the usage of x in this case (erf(x)). The functions domain is - infinity to infinity, but from values of x that are around -3 or 3, we already have almost a 100% chance of finding our Random Variable. In my imagination those numbers would be arbitrary, and I can't see how one could use it (let's say we measure length in centimetres, why is the chance that the Random Variable will always be present when we do erf(3) = 0.9999978?

I hope the explanation of my thought process and (possibly false) assumptions is enough for someone to point out the faults in my reasoning.
I would like to know how I could apply this function and when this is useful. On Wikipedia is written: "this is useful, for example, in determining the bit error rate of a digital communication system." But that doesn't (yet) make it clear for me;/

Hope someone can help, and please forgive my ignorance on the subject.

-Dagl

[studiot]

Hi Dag ?

How is your applied maths ?

The error function has a much wider application than just in statistics and probability, so I will deal with that first.

As you note it is defined by a definite integral, derived from the definite integral

[math]\int\limits_{ - \infty }^\infty  {{e^{ - {t^2}}}} dt = \sqrt \pi  [/math]

This integral can be linked to the binomial theorem, beta and gamma functions and some other special functions defined by integrals and the fact that it is finite is of great use.

Because it is an integral of a continuous function, (that remains finite) shorter intervals than plus/minus infinity also posses this property and this is employed in defining the error function.

The exponential integral also finds use in deriving inverse laplace transforms, so can be used to complete the solution of partial differential equations. For example in solving the Telegrapher's equation and the heat distribution equation.

So it ops up in widely separated and suprising places. Particularly as some of the variables concerned are discrete,as with the gamma and binomial functions.

Normalising the above integral (setting it to one) yields the probability connection to a random variable, x, since the probability over all x must sum to 1.

 

As to a random variable, this is a variable which can take any of a range of values, which may be continuous or discrete such that relative frequency is interpreted as the probability.

So this is how the error function is connected to both statistical and analytical functions.

 

 

 

 

[timo]

There is only one context in which I ever encountered the error function, and it the same context as here: It is the integral over the (normalized) Gaussian function. The Gaussian, on the other hand, is the most important function in statistics. The reason is the Central Limit Theorem: If you take a variable that is distributed according to some probability distribution, and then take the sum of many of these variables, the probability distribution of the sum becomes increasingly more similar to the Gaussian with an increasing number of numbers added (and the mean value of the sum is the sum of the individual means and the variance of the Gaussian is the sum of the addends' variances).

 

The random walk is a process in which a walker takes a number of independent steps with random length and distance. By the central limit theorem, the resulting total deviation from the original location (the sum of steps) will look like a Gaussian, soon. This Gaussian gives the probability density to find the walker at a certain location. To find the probability to find the walker in a certain region, you sum up the individual probabilities of all locations in this region (i.e. you integrate over this region). When computing this integral, the solution can be expressed in terms of erf().

EDIT: I'm still posting this despite just having received a pop-up saying "someone else posted probably the same thing" 😜

Edited January 9, 2020 by timo

[Huckleberry of Yore]

Also from Wikipedia:

"...error function has the following interpretation: for a random variable Y that is normally distributed with mean 0 and variance 1/2, erf(x) is the probability of Y falling in the range [−x, x]."

I'm no expert but I believe the utility of erf() lies in the fact that using simple arithmetic one can derive probabilities for arbitrary mean and standard deviation.  The mean is fairly obvious, erf(x-3) moves the mean to x=3.  The standard deviation involves a scale factor if I recall correctly.  So let's say you have a measurement with mean=10 and standard deviation of 1 you can use erf to answer questions like: what is the probability of a given measurement is <10.1 and > 9.9.

I wrote some code a while back to generate random numbers given a mean and standard deviation and used erf() to check the results.

[studiot]

3 minutes ago, timo said:

There is only one context in which I ever encountered the error function, and it the same context as here: It is the integral over the (normalized) Gaussian function. The Gaussian, on the other hand, is the most important function in statistics. The reason is the Central Limit Theorem: If you take a variable that is distributed according to some probability distribution, and then take the sum of many of these variables, the probability distribution of the sum becomes increasingly more similar to the Gaussian with an increasing number of numbers added (and the mean value of the sum is the sum of the individual means and the variance of the Gaussian is the sum of the addends' variances).

Yes the moments of area about x = o of the erf(x) are very important in statistics as you note +1.

 

[Sensei]

2 hours ago, Dagl1 said:

I think two things are unclear for me, firstly: what exactly is a Random Variable, the wiki article and some other websites talk about it as if it just any variable that is determined by chance, such as the rolling of dice (I presume this cannot be used in error functions due to the lack of normal distributions).

In C/C++ there are randomization functions

srand() and rand()

http://www.cplusplus.com/reference/cstdlib/srand/

rand() gives unsigned integer in range 0 to RAND_MAX.

So if you want to receive in different range you have to do e.g.

rand() * MAX / RAND_MAX + MIN

 

Python should have equivalent functions of pseudo random number generator (PRNG).

https://docs.python.org/3/library/random.html

Edited January 9, 2020 by Sensei

[Dagl1]

Thanks a lot @studiot!

My (applied) and all other types of math is I suppose just slightly above that of high-school students (that is of course horribly defined as a lot of different school systems exist, here is the curriculum of the type of math that I think I can still all use https://www.examenblad.nl/examenstof/syllabus-2016-wiskunde-b-vwo-nader/2016/f=/wiskunde_B_def_versie_vwo_2016_2_versie_voor_hervaststelling_8-9-2015.pdf (it is in Dutch however, so I presume not very useful, but at least you got an answer haha)). Additionally I use (and to a degree understand) statistical tests that I use and what they are based on (one-way ANOVA's, "post-hoc" corrections and odds ratio tests). I think I also have a pretty good grasp of what exactly P-values are, but that is about all my mathematical skills.

I will read your answer a few times the coming days, as not all of it is making sense to me (but also I don't think all of the information is relevant for the level of understanding I want (especially regarding that "it is finite", and therefore very useful). 

I do have 1 question still, I understand how this function is connected to statistics (as it's derived(?) from a gaussian distribution right?) but I still can't see how I would use it. While I am learning python just because I have this book and want to do coding, I assumed that this specific function would come back a lot if I would apply my python skills in a future biomed PhD (so let's say in interpreting quantifiable results of different treatments), could you give me a tangible example of where and how this function would be used. I am missing some fundamental "aha" moment, so the less abstract the better;p

@Huckleberry of Yore
I understand that you can set whatever x is by changing the value to your mean; but I still don't entirely get the application, could you show the code that does this, and what it does (the function still eludes me).

@timo
Okay so I am answering/reading these a bit in a different order, but what you said made something click, maybe. Now I also understand why we are taking the integral of the distribution (the "sum" stuff you said was made it click). So just to be very sure: This function is useful if I already know the mean/deviation of my standard distribution and with this I can calculate the likelyhood of finding a variable somewhere on that graph (that somewhere being defined by x in erf(x)?)? So it is just like a confidence interval but in reverse (that's a horrible description, ehh: A confidence interval gives me, for example, two points between, with 95% confidence (or whatever number we choose) I will find new values for samples from that same group/population. And this function gives me the chance that a given value is somewhere between 2 points.

@Sensei 
Thanks for the reply, your answer shows how I can use a (pseudo)random number generators to get such a Random Variable, and that it doesn't have to be stuck between 0, and 1 (that is the last line about right?), thanks! But my question, although possibly a bit badly worded, was wondering more what value the Random Variable (as I thought this would be constant) would have when throwing 6 sided dice. Which I may now understand, but I am not entirely sure. 
Oh I just started coding (again), if your code does something else and does answer my question, my apologies but then I don't get this stuff yet.

Thanks everyone for the answers!

[Huckleberry of Yore]

This is from my commented code so someone with more statistics expertise than I can verify but....

Say the mean height of a population of people is 70 inches, and the standard deviation is 4.3 inches.  Compute the probability that an individual of that population has a height greater than 70.5 inches:

L=70;

m=70.5;

s=4.3;

t = (L-m)/sqrt(2)/s

P(x>70.5) = erf(t)/2+0.5

= 0.453716 = 45.37%

BTW, rand/srand has been mentioned.  Keep in mind that those functions emit pseudo random numbers with a flat distribution.  To achieve a Gaussian distribution you would use the Central Value Theorem (already mentioned) by adding a series of rand() return values which in the limit should be normal.

Edited January 9, 2020 by Huckleberry of Yore
Referenced erf() instead of error(), grammar.

[timo]

Ctrl-Z just cost me a long post that I put a lot of time in and will absolutely not type in again just to be fooled by my muscle memory. Key points:

1) Consider erf() just as the number-crunching function that computers provide you to calculate integrals over Gaussians. Not much more. The explicit relation between Gaussians and erf() is https://www.wolframalpha.com/input/?i=integrate+1%2F((2+pi)^(1%2F2)+sigma)+e^(-(x%2Fsigma)^2%2F2)+from+A+to+B

2) Here's some Python code to play around if you want. It plots

a) A single random variable "number of bugs a programmer fixes each day"

b) The resulting "number of bugs fixed per programmer per year", which  is a sum of 200 random variables in a) and itself a random variable. Key observation: The distribution looks very different and very Gaussian.

c) The probability to "fix at most N bugs per year" which is the same as "fix 0 to N bugs per year" which is the same as "the sum of probabilities to fix 0 ... N bugs per year" which indeed is pretty much the same as the integral over [0; N]. The resulting curve, as a function of N, looks ... *surprise* ... like the error function.

import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

# We visualize the  distributions by drawing a large sample of random variables.
SAMPLE_SIZE = 10000

def randomFixesPerDay():
    # Number of bug fixes per day is a random variable that becomes 0, 1 or 2.
    return np.random.randint(3)

def randomFixesPerYear():
    # Number of bug fixes per year is a random variable that is the sum of 
    # 200 (=workddays) random variables (the bugfixes each day)
    return np.random.randint(3, size=200).sum()


# Experimental probability for # bug fixes per day
dailyDistribution = [randomFixesPerDay() for i in range(SAMPLE_SIZE)]

sns.distplot(dailyDistribution, kde=False, norm_hist=True, bins=[0, 1, 2, 3])
plt.title('Probabiliy of Bug-fixes per Day: 1/3')
plt.xlabel('# Bugs')
plt.ylabel('Probability')
plt.show()


# Experimental probability for # bug fixes per year
annualDistribution = [randomFixesPerYear() for i in range(SAMPLE_SIZE)]

sns.distplot(annualDistribution, kde=False, norm_hist=True, bins=np.arange(150, 250))
plt.title('Probabiliy of Bug-fixes per Year\n(note smaller value on y-axis)')
plt.xlabel('# Bugs')
plt.ylabel('Probability')
plt.show()


# Integral [0; x] over annualDistribution looks like error function
xValues = np.arange(150, 250)
yValues = [len( [value for value in annualDistribution if value <= x])/SAMPLE_SIZE for x in xValues ]

plt.plot(xValues, yValues)
plt.title('Integral: Probability of fixing [0; N] bugs per year\n(i.e. "not more than N")')
plt.xlabel('x')
plt.ylabel('Probability')
plt.show()

 

Edited January 9, 2020 by timo

[Dagl1]

Thanks, I get it now, just to re-iterate, in an example of length, I could ask: what is the chance of finding someone x away from the mean? I feel like I may have simplified it too much now, but still think I get it, gonna play around a little with the python as well, thank you a lot!

[taeto]

16 hours ago, Huckleberry of Yore said:

Keep in mind that those functions emit pseudo random numbers with a flat distribution.  To achieve a Gaussian distribution you would use the Central Value Theorem (already mentioned) by adding a series of rand() return values which in the limit should be normal.

There is a classical algorithm, called Box–Muller transform, to convert sampling from a "flat" (uniform) distribution, say uniform over the interval \( [0,1] \), to a Gaussian distribution:

en.wikipedia.org/wiki/Box–Muller_transform

[Ghideon]

13 hours ago, Dagl1 said:

gonna play around a little with the python as well

If you prefer not to code the distributions you could take a look at random, one of the standard libraries in Python: 

Quote

On the real line, there are functions to compute uniform, normal (Gaussian), lognormal, negative exponential, gamma, and beta distributions. For generating distributions of angles, the von Mises distribution is available.

https://docs.python.org/release/2.6.8/library/random.html

Pseudo-code example:

random.gauss(mu, sigma) # Gaussian distribution. mu is the mean, and sigma is the standard deviation. 

 

[timo]

20 hours ago, Dagl1 said:

Thanks, I get it now, just to re-iterate, in an example of length, I could ask: what is the chance of finding someone x away from the mean? I feel like I may have simplified it too much now, but still think I get it, gonna play around a little with the python as well, thank you a lot!

Yes - if the height of people is a normal distribution or at least well-approximated by it (not sure to what extent it is). Very specifically, if you assume a Gaussian with 1.70 m as mean height, and 0.15 as the standard deviation then the chance that a person is 1.90 m or larger (in the region [1.9; infinity)) is roughly 10 %: https://www.wolframalpha.com/input/?i=integrate+1%2F((2+pi)^(1%2F2)+0.15)+e^(-((x+-+1.7)%2F0.15)^2%2F2)+from+1.9+to+infinity (note that I put explicit numbers here - if you replace 1.9 with x you'll see the erf() again in the expression for the solution).

[Dagl1]

2 hours ago, timo said:

Yes - if the height of people is a normal distribution or at least well-approximated by it (not sure to what extent it is). Very specifically, if you assume a Gaussian with 1.70 m as mean height, and 0.15 as the standard deviation then the chance that a person is 1.90 m or larger (in the region [1.9; infinity)) is roughly 10 %: https://www.wolframalpha.com/input/?i=integrate+1%2F((2+pi)^(1%2F2)+0.15)+e^(-((x+-+1.7)%2F0.15)^2%2F2)+from+1.9+to+infinity (note that I put explicit numbers here - if you replace 1.9 with x you'll see the erf() again in the expression for the solution).

One last question, since I see the standard deviation mentioned a lot, if I am just using erf(x), the standard deviation is constant (or does not matter?)?

 

[Ghideon]

51 minutes ago, Dagl1 said:

One last question, since I see the standard deviation mentioned a lot, if I am just using erf(x), the standard deviation is constant (or does not matter?)?

Short answer IIRC: erf(x) is defined for some random variable that is normally distributed with mean 0 and variance 1/2. The variance is the square of the standard deviation.
So erf(x) assumes that the underlaying standard is constant and have this specific value.

 

edit:
If mean is [math]{\mu}[/math] is and deviation [math]{\sigma}[/math] then I think one could use
[math]{erf}\left( \frac{x-\mu}{\sigma} \right) [/math]

Edited January 10, 2020 by Ghideon
added sigma and mu

[Dagl1]

2 minutes ago, Ghideon said:

Short answer IIRC: erf(x) is defined for some random variable that is normally distributed with mean 0 and variance 1/2. The variance is the square of the standard deviation.
So erf(x) assumes that the underlaying standard is constant and have this specific value.

 

Alright thanks!

[studiot]

On 1/9/2020 at 5:37 PM, Dagl1 said:

My (applied) and all other types of math is I suppose just slightly above that of high-school students (that is of course horribly defined as a lot of different school systems exist, here is the curriculum of the type of math that I think I can still all use https://www.examenblad.nl/examenstof/syllabus-2016-wiskunde-b-vwo-nader/2016/f=/wiskunde_B_def_versie_vwo_2016_2_versie_voor_hervaststelling_8-9-2015.pdf (it is in Dutch however, so I presume not very useful, but at least you got an answer haha)). Additionally I use (and to a degree understand) statistical tests that I use and what they are based on (one-way ANOVA's, "post-hoc" corrections and odds ratio tests). I think I also have a pretty good grasp of what exactly P-values are, but that is about all my mathematical skills.

Thanks for the reply.

 

Just to confirm, you have studied that syllabus or something similar, not you are just starting it ?

It seems a pretty standard syllabus to me. The Dutch is better than i feared and i can always ask my relatives in Leiden if I get stuck. My sister-in-law teaches this stuff there.

 

I was going to prepare a sheet with some extensions to outline what you need to know to catch I was talking about in the analysis and leaving the statistics and programming side to other very able members ?
erf(x) and its complement erfc(x) are also very important functions in analysis for Engineering and Physics.

[Dagl1]

I completed that 6 years ago (it was my high school math curriculum), so back then I understood/knew most of it. Now is a different story. I think from the other replies and the videos/sources I found, that I understand it well enough (for now). Thanks though!