The Bayesian Machine

[ydoaPs]

Kristalris has a habit of derailing threads by shouting "BAYES' THEOREM!!!!!!111!1!11!" even when it's utterly irrelevant because no inference is even being made. So, I've decided to make this thread so he doesn't have to derail the others. He's made some hefty claims about Bayes's theorem. Some are true, some are false. Before we get into that, let's see what Bayes's theorem is.

The propositional calculus tells us what is true when other things are true. The problem is, we rarely in a position to tell with the certainty that the propositional calculus demands whether or not propositions are true or false. We can do that with tautologies or contradictions (such as why we know it is necessarily true that naive set theory is false), but not much else. This is where probability theory comes in.

We know from the fact that we can order our beliefs on how likely they are to be true and from the normative claim that we ought not violate the propositional calculus that rational beliefs track the Kolmogorov axioms from which we can derive a simple theorem known as "Bayes's Theorem" (despite the theorem being from Laplace):


[math]P_{f}(h_1)=P_{0}(h_1|e_i)=\frac{P(e_i|h_1){\times}P_{0}(h_1)}{\sum^n_{j=1}{P(e_i|{h_j}){\times}P({h_j})}}[/math]

This theorem lets us take existing beliefs ( P(h), called a "prior probability") and update them as evidence comes in. Krystalris says that "Bayes rules all of science", and he's actually right about that. Bayesian epistemology models both falsification (solving the Duhem problem) and allowing science to not only show which theories are wrong, but which ones are likely to be right.

Where kristalris jumps the shark is the boundaries of Bayesian analysis. The likihood term ( P(e|h) ) should give anyone who thinks that ANY inference can be done by Bayes. Sometimes, there just isn't evidence, or the evidence supports each option equally well. In such cases, it comes down to the intrinsic prior. Since that is the prior before ALL evidence, and Bayes's Theorem is how we add evidence to our bodies of knowledge, it doesn't have a word to say. This, however, is not to say there aren't probabilistic methods for finding intrinsic priors.

Then there's cases where the evidence is in principle impossible. Kristalris seems to think that non-mathematical counterfactual reasoning isn't a thing. Could you have made that put that you missed? Bayesian analysis can only tell you whether or not you probably did make that put, not whether or not you could. Bayesian analysis can only tell you what probably is the case, not what probably isn't, but could be or what must be.

Finally (for this post, anyway), is kristalris's claim that "A truism doesnt exist in the Bayesian formula other than an incorrect garbage in prior odds assumption namely what you (implicitly) imagined".

It's not hard at all to see how "Extraordinary Claims Require Extraordinary Evidence" falls out almost immediately from BT; just set two hypotheses with equal posteriors and unequal priors.

"All else being equal, the simpler hypothesis is probably the correct one" also falls out fairly easily once you add in the easily proven fact that (P(a)>P(a&b))&(P(b)>P(a&b)); just set two posteriors equal with equal likelihoods.

If I were better with words, I'm sure there's a truism in there somewhere about unlikely evidence being more bang for your buck.

[kristalris]

Sorry but the quote box system for some reason or other simply has stopped working for me (worked fairly well before).

Q

This theorem lets us take existing beliefs ( P(h), called a "prior probability") and update them as evidence comes in. Krystalris says that "Bayes rules all of science", and he's actually right about that. Bayesian epistemology models both falsification (solving the Duhem problem) and allowing science to not only show which theories are wrong, but which ones are likely to be right.

EQ

Indeed: I also thus state: if it ain’t Bayes, it ain’t science.

Q

Where kristalris jumps the shark is the boundaries of Bayesian analysis.

EQ

Now here we agree and disagree at the same time. Because I don’t jump outside the boundaries of Bayes and therefore not the boundaries of science or of the site and the rules thereof. Yet you do.

Proof:

Q

The likihood term ( P(e|h) ) should give anyone who thinks that ANY inference can be done by Bayes. Sometimes, there just isn't evidence, or the evidence supports each option equally well. In such cases, it comes down to the intrinsic prior. Since that is the prior before ALL evidence, and Bayes's Theorem is how we add evidence to our bodies of knowledge, it doesn't have a word to say. This, however, is not to say there aren't probabilistic methods for finding intrinsic priors.

EQ

Bayes doesn’t accept you doing anything of the sort. The prior odds is in fact nothing more than Bayes asking you in what sort of a world / universe you (your group) believe you live in. Subsequently Bayes asks you to provide further evidence. If there is none to be had then you are left with the prior THEN AND ONLY THEN becoming the posterior. So Bayes applies as a scientifically to be followed method of reasoning both descriptive but also prescriptive. It is the minimum norm of reasoning in science: a conditio sine qua non. Ergo you go through the process of checking whether or not there is further evidence. No? Well then indeed the prior = posterior.

Q

Then there's cases where the evidence is in principle impossible. Kristalris seems to think that non-mathematical counterfactual reasoning isn't a thing. Could you have made that put that you missed? Bayesian analysis can only tell you whether or not you probably did make that put, not whether or not you could. Bayesian analysis can only tell you what probably is the case, not what probably isn't, but could be or what must be.

EQ

Not true. You are forgetting that Bayes allows / even requires you to provide the norms to which you want to work. I.e. how much risk you want to take? So the probanda you give can most certainly be answered by Bayes. Given A =/= B & A + B = 100% then given you stating it is probably B => Bayes says: A isn’t probable yet possible. (Defined then that probable + possible = 100%)
Could be means via Bayes (and thus science) possible (and using normal language as norm thus not probable);
Must be means via Bayes (and thus science) posterior odds past the given norm.

Q

Finally (for this post, anyway), is kristalris's claim that "A truism doesnt exist in the Bayesian formula other than an incorrect garbage in prior odds assumption namely what you (implicitly) imagined".

EQ

Indeed. An a priori that proves correct after having checked the LR’s giving thus the posterior provides – only after this exercise – a SCIENTIFIC truism. If you – honestly (!) – believe or even feel that a scientific unanimously held truism isn’t true then you are scientifically bound because Bayes dictates this to go through the mental exercise to check whether the truism (that the world is flat for instance) is correct. ABSOLUTELY FUNDAMENTAL TO SCIENCE!!

Q

It's not hard at all to see how "Extraordinary Claims Require Extraordinary Evidence" falls out almost immediately from BT; just set two hypotheses with equal posteriors and unequal priors.

EQ

Oh dear, oh dear, Bayes would have been mighty cross with you. “Posterior” is Latin and means afterwards. So taking it the way you do starting with the posterior constitutes a circular argument and thus a fallacy as Bayes shows. Fallacies are not allowed in science.

Q

"All else being equal, the simpler hypothesis is probably the correct one" also falls out fairly easily once you add in the easily proven fact that (P(a)>P(a&b))&(P(b)>P(a&b)); just set two posteriors equal with equal likelihoods.

EQ

Again you don’t start with posteriors. You start with the a priori. (= prior odds) “Prior” is Latin and means before.

Q


If I were better with words, I'm sure there's a truism in there somewhere about unlikely evidence being more bang for your buck.

EQ

That indeed is Bayes.

So and because if it ain’t Bayes it ain’t science nowhere where I used Bayes and referred to this ultimate arbiter of science was I off topic, thread hijacking or what not because this is a science site. Even this philosophical forum is by your own rules to remain within the boundaries of Bayes.
Per definition it isn’t scientific when you go out of Bayesian bounds.

 

Edit PS: Oh and BTW the illustration depicting a Bayesian versus frequent statistician is besides the point. When used correctly both should render the same result. This is a dictate of logic. Bayes can always be used in science. If it isn't the simplest way of dealing with the problem Bayes itself shows you to best not use Bayes but the simpler method. This by using Occam. It also shows you when not to use mathematics at all in science.

Edited November 23, 2013 by kristalris

[ajb]

It also shows you when not to use mathematics at all in science.

I don't think you can ever really discuss science without mathematics at some level.

 

Anyway, Bayes' theorem is not under any dispute, it is a well founded mathematical statement. It is the use and interpretation of the theorem that can be more controversial. For example poor handling of the prior knowledge is going to lead to poor conclusions.

 

ydoaPs, applied statistics is far from my area of expertise. Do you have some nice examples where Bayes' theorem is useful and examples where it would be the wrong tool to use?

[kristalris]

 

 

I don't think you can ever really discuss science without mathematics at some level.

 

Anyway, Bayes' theorem is not under any dispute, it is a well founded mathematical statement. It is the use and interpretation of the theorem that can be more controversial. For example poor handling of the prior knowledge is going to lead to poor conclusions.

 

ydoaPs, applied statistics is far from my area of expertise. Do you have some nice examples where Bayes' theorem is useful and examples where it would be the wrong tool to use?

 

I fully agree with you.

 

Examples where Bayesian statistics is correctly used is with expert evidence in legal cases concerning DNA matching a swipe of the suspect with a sample found at the crime scene.

 

A case where Bayes shows that Bayesian probabilistic reasoning (=/= exactly the same as Bayesian statistics) had scientifically best not be used is concerning areas where so little or no data is available concerning the probandum especially when time is of the essence: i.e. in courts of law (if you want as you should strive to remain within scientific boundaries). In many daily life problems, in a great part of philosophy (within the realm of science that is.)

 

A case that shows incorrect use of empirical statistics and correct use of Bayesian statistics is the Lucia de B case. http://en.wikipedia.org/wiki/Lucia_de_Berk#Statistical_arguments

In this case there was a long and heated debate between scientists prior to Lucia’s release whether to use Bayes or empirical statistics. It is/ was a nonsensical debate. Correctly used both methods should render the same result (= logical dictate). Why then prefer in this case Bayes before empirical? As a rule of thumb: too little data => use Bayes! In casu had mathematician professor Elfers (teaches statistics to psychologists) used Bayes it would of put him on the right track via the a priori question: in the world you live in do you believe that nurses often, not often or ever kill patients against the rules? (euthanasia (BTW only allowed to be done by doctors if at all in the Netherlands)) I guess most people will assume / be convinced that this if at all will be very unlikely. => Bayes dictates a lot of extra evidence to the contrary on a given norm of proof. => All the more reason to check the data that the police provided, which Elfers didn’t do.

 

Now if we were to run into an Monty Python style argument that in fact my a priori is wrong and there is a Roald Dale scenario in which nurses are in fact witches that regularly kill off patients, based on the evidence on hand and in order to prove that you would have to nearly even resort to a degree of investigation and requiring data via secret camera’s and the such that you indeed better even use frequentist statistics.

 

Another nice area where you shouldn’t even use Bayes or any other form of mathematics is when trying to solve a TOE. You simply don’t have the data needed to solve the – essential and unavoidable question – whether or not the universe is infinite or not. It either is or isn’t a relevant question, and it either is or isn’t infinite. Bayesian probabilistic reasoning as does simple logic will show the use of verbal logic to be the best quickest and most safe way to deal with that problem. And it forbids the only use of mathematics in order to tackle that problem. You get your Escher Institute honoree degree if you don’t grasp that.

 

Ultimately I agree with you in all cases Bayesian mathematics is at the deepest level the thus ultimate arbiter, for clenching any Monty Python style course on argument on ANY scientific problem. Yet, few if any arguments well dealt with using proper verbal argument have to resort to that.

 

Usually the reason not even Bayes or full logical argument will clench the Monty Python issue lies in psychology. Usually a paradigm (or simple bad faith) prevents this and the argument will even go towards a held mathematical truth that is even agreed extremely improbable yet because it has mathematics is widely considered probably best. Which is totally illogical because Bayes shows you’ve gone into a simple or full blown prosecutors or defense attorneys fallacy respectively. Albeit at the heart of it probably DNA talking in the given DNA environment or if you like psychology nurture scenario.

[ajb]

Another nice area where you shouldn’t even use Bayes or any other form of mathematics is when trying to solve a TOE.

I don't think any kind of statistics really has any bearing on the question of a TOE apart from the analysis of data to support or disprove a given TOE. However, I think we have discussed this before and we should not steer the thread back to that.

Edited November 24, 2013 by ajb

[kristalris]

 

I don't think any kind of statistics really has any bearing on the question of a TOE apart from the analysis of data to support or disprove a given TOE. However, I think we have discussed this before and we should not steer the thread back to that.

Hurray it finally works again the quote box. Thanks YodaP.

 

Well we didn’t delve into it fully, but anyway it’s an answer to your question.

 

Still on a different note than TOE it is thus not I guess then contested by you that Bayes is indeed the ultimate arbiter of science, or do I understand you have no position, or object to that?

Because statistics doesn’t as such include - unless used as then as a stipulative definition - Bayesian probalistic reasoning. The latter has baring on everything that science does. Ergo if it doesn’t fit Bayesian probabilistic reasoning it isn’t science. Again, combined with Occam Bayesian probalistic reasoning shows you when to use simple verbal logic instead of say Bayesian nets. The latter is extremely slow and arduous process. Only ultimately in a nice market to be used as the ultimate arbiter on any scientific dispute. It even rules all mathematics, even pure mathematics because there shouldn’t be any conflict between them when used correctly. It is in fact Bayes that intuitively shows the mathematician when to use what sort of mathematics. The algorithm of Bayes is probably according to current science in all our brains as. Bayes describes this algorithm in our brain being thus the mathematics of common sense and all of science.

 

On any scientific question Bayes dictates: take all relevant evidence a priori and subsequent evidence and weigh that becoming the posterior. It even shows you what norms to use on any stated goal. And it shows you what sort of logic or mathematical tools are in order. The fact that everybody always uses shortcuts in deciding what to use doesn’t mean it isn’t governed by Bayes in our brain.

[ydoaPs]

Bayes doesn’t accept you doing anything of the sort. The prior odds is in fact nothing more than Bayes asking you in what sort of a world / universe you (your group) believe you live in. Subsequently Bayes asks you to provide further evidence. If there is none to be had then you are left with the prior THEN AND ONLY THEN becoming the posterior. So Bayes applies as a scientifically to be followed method of reasoning both descriptive but also prescriptive. It is the minimum norm of reasoning in science: a conditio sine qua non. Ergo you go through the process of checking whether or not there is further evidence. No? Well then indeed the prior = posterior.

No, the prior is just the previous posterior. When you add new evidence to your body of knowledge, the updated probability is called the 'posterior probability' and is used as the next prior probability.

 

But you have to worry about your very first prior. That one, though, isn't what you describe either. It is not the case that you can rationally assign any probability you want to the intrinsic prior. There are constraints such as simplicity and coherence. To use the linked example in the OP, we know that the intrinsic prior for theism must be less than 1/2.

 

Not true. You are forgetting that Bayes allows / even requires you to provide the norms to which you want to work. I.e. how much risk you want to take? So the probanda you give can most certainly be answered by Bayes. Given A =/= B & A + B = 100% then given you stating it is probably B => Bayes says: A isn’t probable yet possible. (Defined then that probable + possible = 100%)

Could be means via Bayes (and thus science) possible (and using normal language as norm thus not probable);

Must be means via Bayes (and thus science) posterior odds past the given norm.

That is quite irrelevant. I'm not sure why you quoted that part of my post, said 'not true' then posted irrelevant blather. It's a fact that you cannot have evidence for whether or not that which did not happen could have actually happened. I don't know how many times I'm going to have to correct you on the point that not knowing whether or not something happened is not the same as it being possible.

 

I know that I missed that put, but I could have made it.

 

Oh dear, oh dear, Bayes would have been mighty cross with you. “Posterior” is Latin and means afterwards. So taking it the way you do starting with the posterior constitutes a circular argument and thus a fallacy as Bayes shows. Fallacies are not allowed in science.

Well, no, for several reasons. Bayes's Theorem didn't actually come from Bayes (he used a geometric analog to the method on one specific problem, and derived no theorem. We owe Bayes's Theorem to Laplace). And there's nothing circular at all in setting the end results equal and working backward to see what the inputs must have been.

Again you don’t start with posteriors. You start with the a priori. (= prior odds) “Prior” is Latin and means before.

You really should stop using that word since what it actually refers to is something you think we can't do. That is, make inferences in the complete absence of evidence.

 

When used correctly both should render the same result. This is a dictate of logic.

Not only is that not a 'dictate of logic' (whatever that means), but it's not even true. Bayesian methods and frequentist methods often do come apart. In situations where frequentist methods can even give an answer, the frequentist answer and the Bayesian answer can differ by orders of magnitude.

 

 

For example poor handling of the prior knowledge is going to lead to poor conclusions.

That shouldn't be a point of controversy as it applies to all methods of reasoning. If you have false premises, your conclusions aren't guaranteed. If you model the Earth as a cube, you're going to get nonsense results.

 

 

ydoaPs, applied statistics is far from my area of expertise. Do you have some nice examples where Bayes' theorem is useful and examples where it would be the wrong tool to use?

One of the prime examples of where Bayes was the only tool for the job was nuclear weapon safety. We needed to know the chances of accidental detonation of nuclear weapons, and the frequentists said it was impossible since none had yet accidentally detonated. However, the Bayesian methods allowed such calculations and paved the way for the safety constraints that were eventually used.

 

It was also used to break the Enigma Code(s).

 

As I've told kristalris, it's entirely useless in figuring out what could have happened but didn't, what must happen, or what can never happen.

[kristalris]

No YdoaP, the prior is not only the previous posterior, yet can also be what you or any more or less larger body of people guess it is: such as the question do nurses or don’t they kill their patients? Within the framework of accepted risks and time you want to come up with a scientifically valid way of reasoning (time sometimes prohibiting taking in all possible evidence and thus the previous posterior known to man by checking all data of every hospital in the world and more. Is that what you think it means? Well yes in a way it does, but that is normless. Bayes requires you to define everything if you like, yet also allows you to cut it short. This by Occam. Bayes shows you the way.

 

I don’t know what Bayes would probably of thought by what you think you mean by the very first prior? Adam and Eve? He was a reverend remember. The scope of Bayes is limitless mate. It can if you like start there but I guess it would be a bit too much work to serve any useful scientific purpose.

 

 

Your repeated statement: "It's a fact that you cannot have evidence for whether or not that which did not happen could have actually happened."

 

Get real. If I through a dice and get a six, then what did not happen could of actually happened namely that it became a 1,2,3,4, 5 or indeed again a six. The chance is 1/6. That remains the a priori even taken as absolute (assuming thus a pure chance and perfect die). The evidence for a die under these assumptions being 1/6 is overwhelming. What the hell are you talking about? And the blather you think is blather is blather about something that in practice goes horribly wrong a lot of times. Because there are people who simply can’t grasp its importance. Proof: the Lucia case.

 

This also goes in a court of Law, when you've driven through the town at a 100 km/h. You will be punished more severely - and Justly so - not only for what you did but also for what could of happened yet didn't. Setting therewith the norm and reason for upholding that norm. Reasoning that BTW can be inferred via Bayes.

 

And indeed you cannot make any inference whatsoever if you do not have any evidence. Even in pure mathematics within Bayes you take what you had prior on mathematics in as evidence thereof. The assumed situation of there being no evidence has never occurred to us humans at least anyway, so what do you mean? The situation of no relevant evidence being available is something to be inferred and can not and may not be taken for granted in science. Per bloody definition mate! In science you are not allowed to state "the world is flat as truism" You are not allowed to state any a priori truism whatsoever. This because it simply can't be made to fit the requirements of logic in its broadest sense as described by Bayes.

 

Indeed frequentist and Bayesian approaches on the same problem do often come apart, as in the Lucia case because one or both (that wasn't the case in the Lucia case because Elfers came up against real statisticians whacking him silly.) of the statisticians doesn’t know his business. If they collide then there is something wrong that needs fixing. Both always should render exactly the same results if both are used correctly. This is a dictate of logic for there can only one logic: namely logic. (Now if you indeed don’t know what “logic” means we do then have finally found the problem. Logic always dictates the correct answer, otherwise it is illogical: logic, yet agree it is a pleonasm, used to stress this consequence of logic you keep on missing.) And ALL the correctly used tools of logic such as mathematics should on the same question render the same result. For instance you can use deterministic Rutherford reasoning on a question where most if not all mathematicians would use frequentist statistics. Then there is no conflict as long as you state to accept the larger error that deterministic reasoning then entails. The latter being in fact a Bayesian shortcut. Get it?

 

Bayesian statistics is often called intuitive as opposed to empirical statistics because Bayes allows you contrary to empirical statistics to GUESS. That is also the reason why Bayes always applies ultimately yet not always practically. The latter even again dealt with by Bayes' probabilistic reasoning.

 

And finally: no, Bayes does NOT allow you to start with the posterior other than taking that posterior as prior and subsequently seeing what further relevant evidence is available BEFORE INFERRING a conclusion based on matching the posterior with the NORM you choose, or what not. THAT IS SCIENCE: for if it ain't Bayes it ain't science! Science is always inherently about inferring old boy. By logical definition.

Edited November 25, 2013 by kristalris

[Spyman]

Kristalris has a habit of derailing threads by shouting "BAYES' THEOREM!!!!!!111!1!11!"

Independent of kristalris past behaviour, I strongly dislike threads with a purpose to single out a member and put him/her in the spotlight. It is even more disturbing when it is done by a member of the staff!

 

If someone needs to be corrected it should be done in the thread where the misconduct has been made and if ydoaPs want to discuss Bayes' theorem this thread could have started without pointing fingers and a simple PM could have invited kristalris to the discussion instead.

 

Therefore I will both report the OP and vote negative on it.

[ydoaPs]

Independent of kristalris past behaviour, I strongly dislike threads with a purpose to single out a member and put him/her in the spotlight. It is even more disturbing when it is done by a member of the staff!

If you quotemine like that, it does look bad. However, the thread is for a discussion kristalris wanted to have. It's just a place for him to have the discussion without hijacking other threads.

[Spyman]

If you quotemine like that, it does look bad.

It is the half of your first sentence, quoting it in its whole doesn't make it look much better.

 

However, the thread is for a discussion kristalris wanted to have.

Are not kristalris allowed to start his/her own threads? If he/she asked you to start this thread, then why didn't you say that instead of ranting about personal habits?

 

It's just a place for him to have the discussion without hijacking other threads.

Can't kristalris participate in other threads anymore, is he/she forced/locked to only this thread?

[hypervalent_iodine]

!

Moderator Note

YdoaPs and Spyman,

 

This isn't a thread about kristalris. Please stay on topic.

[ydoaPs]

No YdoaP, the prior is not only the previous posterior, yet can also be what you or any more or less larger body of people guess it is

No, you cannot just set whatever prior you want. Doing so defeats the entire point of using Bayesian analysis in the first place. It's the cumulative addition of information. You can't just pretend all of the updating from the past never happened.

 

The scope of Bayes is limitless mate.

No, it's not. It only works where there is evidence. You cannot use it to answer questions where you in principle cannot have evidence. You cannot use bayesian analysis to tell you whether or not you could have made that put anymore than you can use it to tell you how likely universes with different physical laws than our own are.

 

You can only get evidence from the actual world. You can find out what isn't the case, but that doesn't tell you that it's not the case but could have been.

 

Get real. If I through a dice and get a six, then what did not happen could of actually happened namely that it became a 1,2,3,4, 5 or indeed again a six.

 

Again, that something did not happen does not imply that it could have happened. The universe wasn't created 5 seconds ago by an invisible pink unicorn, but that doesn't mean it could have been.

 

 

If they collide then there is something wrong that needs fixing. Both always should render exactly the same results if both are used correctly.

No, that's not true. They legitimately don't always give the same answer. And it's not just mistakes in applying the methods.

 

 

 

This is a dictate of logic for there can only one logic: namely logic. (Now if you indeed don’t know what “logic” means we do then have finally found the problem. Logic always dictates the correct answer, otherwise it is illogical: logic, yet agree it is a pleonasm, used to stress this consequence of logic you keep on missing.)

Which logic? S? S^*? SL? PL? S4? S5? Bivalent logic? Trivalent logic? There are more logics than you know and none of them say that frequentist methods and bayesian methods must give the same answer.

 

In many cases, the proper frequentist answer is 'It's impossible to tell' while the bayesian method gives a definite answer.

 

no, Bayes does NOT allow you to start with the posterior other than taking that posterior as prior and subsequently seeing what further relevant evidence is available BEFORE INFERRING a conclusion based on matching the posterior with the NORM you choose, or what not. THAT IS SCIENCE: for if it ain't Bayes it ain't science! Science is always inherently about inferring old boy. By logical definition.

You can indeed work backward to find the inputs if you know the outputs. It's basic analysis.

[kristalris]

Okay, YdoaP, we I guess are at odds on what "logic" should entail within a scientific context (i.e. the context of this site). We are running in circles otherwise in a Monty Python style.

 

Now I've scanned this Wikipedia and it seems to me to be in order. Where do you stand in it? I'll scan it again to see where I stand.

 

Beforehand I'd say logic per definition entails inference between assumed absolute truths.

 

So, there is only one logic: logic sec. There further more are only different tools to reach the scientific goal.

 

Science entails the trying to reach the absolute truth (= the goal) on everything as close as possible in the quickest easiest way via the systematic (i.e. logical) inference of observations (i.e. taken in as evidence). In short it's what Bayes given Occam is all about.

 

.http://en.wikipedia.org/wiki/Logic#Philosophical_logic

Edited November 25, 2013 by kristalris

[Cap'n Refsmmat]

In many scientific fields (e.g. psychology, biology, chemistry), there is no clear parametric model for the system in question, and no likelihood function to write down. Alternately, the likelihood may be incredibly complex and involve quantities which are practically impossible to measure.

 

How does one perform Bayesian inference in these fields?

[Gees]

Beforehand I'd say logic per definition entails inference between assumed absolute truths.

 

 

Kristalris;

 

I don't know much about math and nothing about the Bayesian Machine except what I have learned here, but I am pretty good at language. So I am having some problems with your underlined words above.

 

"Assumed absolute truths" are what religion uses to argue that "God" exists. I have read some very good logical arguments based on the "assumed" truth that "God" is real, so I know that logic can be used to infer something that is not real when based on assumed absolute truths.

 

Last time I checked, science bases it's knowledge on facts, and only assumes that those facts are valid because they have been proven, so they will be accepted until proven otherwise. If I am wrong here, I am sure that someone will correct me.

 

It looks to me as though you are still applying logic to guesses and imagination.

 

G

[ajb]

Still on a different note than TOE it is thus not I guess then contested by you that Bayes is indeed the ultimate arbiter of science, or do I understand you have no position, or object to that?

I think it is one tool in a kit box of many tools based on statistics. I don't recall talking much about Bayes' theorem as an undergraduate in physics and I would not say that I am well versed in the analysis of experimental data.

[kristalris]

In reaction to Gees and to Cap'n Refsmmat:

 

Indeed Gees, and that is exactly what Bayes allows you to do: guess by using your imagination and see how your guess taken as a fact (= hypothesis = assumed to be absolutely true) fits all the other facts (= assumed to be absolutely true.) Logic is in fact an empty shell. You put garbage or non-garbage in and compare it to other garbage or non-garbage. Logic in general only tells you if it is consistent with each other. The same goes for all tools of logic such as language and all forms of mathematics. The only tool that covers it all is Bayes.

 

The paradox ( seeming contradiction) is that you don’t practically use Bayes that often, even though current neurology is saying that we probably (= Bayes BTW) have the algorithm described by Bayes in our brains. In fact thus not only the algorithm covering all mathematics but also common sense.

So indeed you can take the probandum “does God exist?”. Well, using the Bayesian what I’d call the enhanced common sense method it goes like this: The applicable norm here is absolute truth. Can I absolutely prove or disprove that he exists? No.

 

Now Bayes lets you take it further than that: Okay let’s assume God exists (taken as an assumed absolute truth) Now I compare this absolute truth with the opposite I take the fact that God doesn’t exist. (In fact a very clever mathematical trick.)

 

A priori I see no evidence that he exists or evidence that he doesn’t exist. So that is 1/1 = 1 or irrelevant.

 

For my likelihood ratio (= on further evidence acquired by further investigation) I thus have no evidence either so I’ll have to guess: I guess the chance of God existing given that there is no evidence either way (extremely far, but he give or take a bit then) less than one in a trillion of being true in general. (For I a priori don’t believe that we can know the absolute truth) I.e. taking something extremely complex to be true in agreed absence (that is then challenged by religious people, but not by most scientists) I’ve already established the fact that I can’t prove it either way on an absolute norm so I’ll prove it for myself on my personal norm. Now I could immediately conclude thus he doesn’t exist, yet I can via Bayes go one further still: The further evidence I have is that I personally have the very strong feeling he doesn’t exist, at least as I wise old man with a beard or the like. This I can give a LR of 1/10 or infinite against because it closes for me the argument: I take as a fact (assumed absolute truth that God doesn’t exist.)

 

Now I could of used the shortcut and simply state: “I don’t believe in God.”

 

If need be my above argument can also be written down in a full blown Bayesian net, and forcing me to answer maybe via guessing certain facts that I’ve missed in the argument. This then via the rigors of mathematics. Like you Gees, I couldn’t cope immediately at least. Yet no problem: you go to your friendly experienced open minded statistician for this. He/she will probably tell you to sod off, because further mathematics won’t prove a thing further for those who don’t believe or disprove a thing for those who do, but anyway Cap’tn it can of course deal with any problem you like, in psychology or what not.

 

Now when I die and indeed find myself standing before Peters Gate I will reconsider my position on this on the basis of the new evidence that has then presented itself. (In fact thus the scientific / Bayesian approach BTW)

 

Now to go one further still: Nuclear physicist Krauss et all believing via mathematics that it all scientifically stems from nothing thus believes in a blatant contradiction. I.e. magic. That is worse than believing in a God because believing in God doesn’t conflict with anything. I.e. as a guess Krauss et all are less than 1 in a trillion to the trillionth of being correct.

 

Now I could even go one further with God. If you guess (like I do BTW) that it fundamentally is something then even a thought is a material thing of fundamental parts bouncing around. So then the collective thought of God exerting a power (for good or bad) by religious people then does exist, as is consistent with what we observe in religion. Taken that way proves / can prove (= dependent on the norm you choose) then that God does exist.

 

There you go: easy Bayes. Bayes is both extremely easy and extremely or even infinitely complex. I.e. you can make it as complex as you like yet keep an eye on Occam => keep it as simple as possible! => Bayes more often than not says: “don’t use me! But says for instance: “use empirical statistics on this problem for it is more simple!”

 

So yes, in science you are allowed and even forced in the fields where you know that you don’t know the answer yet forced to act, or to find out what to test or not to test to use common sense. You may guess using your imagination. As long as you state that that is what you are doing! It then is a scientific fact that you are speculating. The forum thereof is thus correctly part of the science forums.

 

What you’re not allowed to do, yet what happens a lot is claiming science or more science than can be had such as DSM V in psychology. Yet even these too rigid models are not so dangerous in the hands not just of the experienced knowledgeable exert, AS LONG AS HE OR SHE IS OPEN-MINDED! I.e. if it is someone who can make an adult guess because thus having sufficient imagination to fill in what Bayes requires you to fill in, in order to test. Sometimes in the test of actual life BTW.

 

If the one doing the psychological diagnoses is not open-minded - as most are BTW (not open-minded that is) - then the simple observable statistics are that the rigid system like DSM V becomes more and more dominant. Leading to the hilarious consequence that they in fact state that nearly everybody, sometimes even including themselves are mad. Proving that they are Intergalactic Idiots when doing R&D i.e. making diagnoses or providing the "evidence based medicine" thereof like DSM). God / MN would a priori not have nearly everybody mad now would he/ she?

 

(What they should be doing is leaving the R&D diagnoses to the open-minded ones and concentrate on what they are good at: production and / or sales: the therapy that requires conscientiously risk avoiding treatment via the book. If it doesn't work? back to R&D then to figure out what next? Or to provide schooling on what "the book" says. But please no R&D by too narrow minded people on issues where you know that you don't know a lot that needs to be known in order to take a decision!)

 

 

I hope this clarifies Bayes for you.

Edited November 26, 2013 by kristalris

[ydoaPs]

While the purpose of this thread was to stave off an involuntary vacation for kristalris as long as possible and the opening paragraph was intended simply to add background knowledge and context for the kristalris quotes which appear later in the OP, the wording of said paragraph was unnecessary. I should have been more sensitive to the various ways in which it could have been read. I sincerely apologize.

[kristalris]

Although I wasn’t planning on a reaction, I will give one. Insofar apologies also are directed towards me, I accept them, though I don’t deserve them. If you push like I’ve pushed then one shouldn’t moan I believe, yet I accept that others might not, as I do, enjoy a stiff debate. For which I then should apologize, as one should do in Rome as the Romans do, being a guest as I am.

[Popcorn Sutton]

I sincerely enjoyed reading this thread!

Sorry I ended up deleting the original message. The intent was to say that Bayesian inference is applicable in all scenarios that we can think of. Given enough evidence, the likelihood of any hypothesis is going to be infinitesimally small, but there will still be one that is most likely.

Edited November 27, 2013 by Popcorn Sutton

[kristalris]

I sincerely enjoyed reading this thread!

Sorry I ended up deleting the original message. The intent was to say that Bayesian inference is applicable in all scenarios that we can think of. Given enough evidence, the likelihood of any hypothesis is going to be infinitesimally small, but there will still be one that is most likely.

Thank you. yet I don't quite catch your drift concerning the likelihood of any hypothesis only becoming infinitely small, because the opposite is also true dependent on the direction this enough evidence points.

 

Further more only having hypotheses could one be more probable than the other. It is not the likelihood as such but the ratio of likelihood pro versus con that provides the probability. The latter is important the crux so to speak of Bayes. You should compare probabilities and not just likelihoods.

[Popcorn Sutton]

Well, computational linguistics makes use of Bayesian inference in a different way then. The reason I say it gets smaller and smaller is because of language acquisition. Say that knowledge is evidence and output is a hypothesis, if knowledge consists of one billion trillion bits of information, using the relevant knowledge (evidence) could consist of one billion bits. If there is only one hypothesis (output) that is possible (or most probable) then the probability of that output could easily be 0.00000000000000000000000000000000001. But like I said, it still beats 0.000^10, 1. Because of this, all other relevant bits of information are not able to be put in the output because they don't suffice. It's like having subconscious thoughts. It's like ruling out the hypotheses that don't have as much evidence as the output.

Edited November 27, 2013 by Popcorn Sutton

[kristalris]

Well, computational linguistics makes use of Bayesian inference in a different way then. The reason I say it gets smaller and smaller is because of language acquisition. Say that knowledge is evidence and output is a hypothesis, if knowledge consists of one billion trillion bits of information, using the relevant knowledge (evidence) could consist of one billion bits. If there is only one hypothesis (output) that is possible (or most probable) then the probability of that output could easily be 0.00000000000000000000000000000000001. But like I said, it still beats 0.000^10, 1. Because of this, all other relevant bits of information are not able to be put in the output because they don't suffice. It's like having subconscious thoughts. It's like ruling out the hypotheses that don't have as much evidence as the output.

What if the investigation of the hypothesis shows an enormous / probably infinite amount of data?

I think it is one tool in a kit box of many tools based on statistics. I don't recall talking much about Bayes' theorem as an undergraduate in physics and I would not say that I am well versed in the analysis of experimental data.

Well see there the problem then with the education of physicists, they have only just climbed out of the Rutherford tree a few years back by stopping opposition to empirical statistics. It leads to in fact the same sort of a discussion concerning the proper use of Bayes. Bayes spans the extremely simple to the infinitely complex. It can always be applied and is thus the ultimate arbiter of science.

 

Now the problem is as soon as scientists / physicists don't contest that or even accept that, physicists immediately run into a wee problem.

Edited November 28, 2013 by kristalris

[ajb]

It can always be applied and is thus the ultimate arbiter of science.

The philosophy of science is based on the scientific method. This is really a basic guide as we may need a little room in practice and the standards of evidence differ in different branches of science.

 

The question here must be "is Bayes' inference equivalent to the scientific method?"

 

I know that people like to interpret the scientific method in light of Bayes' inference, but I don't think we have a direct equivalence here. One point must be the fact that the scientific method is not a strict mathematical statement, Bayes' theorem is.

 

This is not to take away the fact that Bayes' is useful in science, but it is not the first or final word on science.

Well see there the problem then with the education of physicists, they have only just climbed out of the Rutherford tree a few years back by stopping opposition to empirical statistics.

Based on our previous discussion, I don't think you are in much of a poistion to judge the typical education of undergraduate physicists.