How does an ordinary person know what's mainstream?

[pears]

Is it possible? I can read a research paper but I have no way of validating it (or often even making sense of it). (Incidentally since papers are peer reviewed, where do those peer reviews reside?)

 

Is there a thick black line drawn somewhere within the super-set of hypotheses and research results to give a neat subset of "mainstream" and "accepted". Or is the line fuzzy and blurred?

 

Is it something that only an expert can really know, and only within a narrow field? And is that ultimately only an opinion anyway?

 

What should a lay person trust? Pop science books? Science journalism?

 

How do YOU recognise something as mainstream? Does it just come with experience and education?

[EdEarl]

Education is important. One can take logic and elementary philosophy to learn some things about critical thinking. But, no one class teaches how to observe and understand, because neither observing nor understanding is a simple process. If we want to observe trail signs left by elephant and understand what they mean, we need one teacher, and if we want to observe exploding stars and understand what occurs we need another teacher. To observe either elephants or stars, we need different tools.

 

However, educators could teach a basic class in observation and understanding, to teach the process. Once a person understands how to carefully observe anything, and to tease as much information as possible from that observation, they will have learned a basic skill that has led them to mainstream understanding of that thing. They should realize the same process can be applied to observing and understanding other things, which should help them suspect whether their knowledge is mainstream or otherwise. Understanding the process is experience, and going through the observation and understanding process more than once builds upon experience. In other words, experience and education are important.

Edited September 22, 2013 by EdEarl

[Endy0816]

I look for references, the quality of the publisher and the credentials of the author.

 

This: http://www.nature.com/ja/journal/v58/n1/abs/ja20051a.html

 

is a good example.

 

On peer reviews, they are generally done at the publisher level. There may or may not be an actual "review" written out somewhere. Certainly though any good publisher will have one done prior to publishing the paper.

 

 

 

 

[swansont]

Experience and education are important. You tend to develop a good BS detector after a while. Even outside of your area of expertise, often the style of an argument will tell you if it lacks merit.

[John Cuthber]

It's also worth checking if they are seeking "funding for further work". If they can't get money from orthodox sources, there's often a good reason.

[Tridimity]

Is it possible? I can read a research paper but I have no way of validating it (or often even making sense of it). (Incidentally since papers are peer reviewed, where do those peer reviews reside?)

 

Is there a thick black line drawn somewhere within the super-set of hypotheses and research results to give a neat subset of "mainstream" and "accepted". Or is the line fuzzy and blurred?

 

Is it something that only an expert can really know, and only within a narrow field? And is that ultimately only an opinion anyway?

 

What should a lay person trust? Pop science books? Science journalism?

 

How do YOU recognise something as mainstream? Does it just come with experience and education?

 

Peer reviews are usually only accessible by the authors and fellow reviewers of the paper and by the editors of the journal. Essentially what happens is, once a researcher has published a paper, the editors of the journal in which the researcher has been published may contact them as a prospective reviewer for one or more manuscripts that have been submitted to the journal. Reviewing is not compulsory and usually involves assessing the merit of the manuscript in terms of: originality/significance, quality of Science, and level of interest. The reviewer is also expected to identify any ethical issues with the paper and to determine whether or not the paper is of public interest, beyond the research community. Opportunity is given to provide comments to the authors and confidential comments to the editor. Decisions for the paper are: accept, minor revision, major revision, reject and re-submit, reject. The peer review process is not infallible but it is the most pragmatic approach to quality control of research outputs.

 

There is no concrete division between accepted or mainstream Science versus non-mainstream. One of the main reasons for this is that Science is not a static body of knowledge; many historical scientific theories were accepted at the time of their conception but have subsequently been disproved and so are no longer considered a part of mainstream Science. It is inevitable that a small proportion of scientific concepts that are currently considered outlandish will one day be accepted as a part of mainstream Science. The key to establishing a scientific concept as part of mainstream Science, is to develop a hypothesis that is testable and which consistently fails to be refuted by experiment or observation over long periods of time. This still does not guarantee that the hypothesis will not be disproved at some point in the future - but, alas, that is the nature of Science.

 

It is normally quite easy to tell whether or not a scientific concept is part of mainstream Science. All you need is a lecture hall full of professional scientists, a microphone to present the concept, and the patience to gauge their reactions (snorts of derision versus calm silent cognitive assent). Note that their reactions have no bearing on whether or not the concept is true to reality. Many people, even highly qualified people, can be wrong at the same time, sometimes the lone maverick dissenter is the only one to see the truth.

 

I think that judgments regarding the status of a scientific theory as mainstream or otherwise are available to all who think critically and independently - it is not the preserve of experts or specialists in arbitrary positions of authority (white coats). The lay person should not trust any single source. It is necessary for the novice to rote-learn vast amounts of scientific information, but even this can - and ought to be - accompanied with a kind of mental attitude that recognises that any one or more of those pieces of information may be inaccurate or down-right incorrect (whilst appreciating that this is unlikely to be the case), and with a healthy preparedness to test any such hypotheses experimentally.

 

Nullius in verba

Edited September 22, 2013 by Tridimity

[Phi for All]

You tend to develop a good BS detector after a while.

 

And it's amazing how often it lets you spot things like this:

 

[ajb]

Incidentally since papers are peer reviewed, where do those peer reviews reside?

 

Reviwers for journals are people who have published in that field.

 

Reviwers are people, and people can miss things and mistakes can be made. Peer-review is a safe guard, but it is not an absolute.

 

How do YOU recognise something as mainstream? Does it just come with experience and education?

Experience and education are key.

 

Other general guides include: Are claims that seem too good to be true? Looking at the references also helps, are they modern or all Einstein's original 1905 papers? (Okay, that is an extream example, but you get the point.) Look for fundamental science papers written by people in engineering departments and that are published in obscure journals. Look at editor list of the journal the work is published in. Has anyone you know published in that journal?

 

Depending on how well established the work is, a good rule of thumb is that if it is mainstream then there will be a textbook on the subject published by a good publisher.

[pears]

Thanks everyone for your insightful responses.

 

It's also worth checking if they are seeking "funding for further work". If they can't get money from orthodox sources, there's often a good reason.

 

How would you check? Google search?

 

 

Decisions for the paper are: accept, minor revision, major revision, reject and re-submit, reject. The peer review process is not infallible but it is the most pragmatic approach to quality control of research outputs.

 

 

Is this standard across science papers? Are these ever made public?

I think that judgments regarding the status of a scientific theory as mainstream or otherwise are available to all who think critically and independently - it is not the preserve of experts or specialists in arbitrary positions of authority (white coats)

 

Good to hear

 

The lay person should not trust any single source.

 

I think this is key. Great reply Tridimity.

 

 

And it's amazing how often it lets you spot things like this:

 

 

Haha love this. I've seen similar components in software designs - sometimes in my OWN designs!

[CaptainPanic]

It's also worth checking if they are seeking "funding for further work". If they can't get money from orthodox sources, there's often a good reason.

 

I completely disagree with this. Almost every research group on the planet is in need of more funding.

 

At least in the Netherlands, and perhaps even in all of Europe, researchers are encouraged to seek industrial/commercial participation for their research and especially their funding. Because of the economic crisis, the old-fashioned government-paid research budgets have been slashed everywhere.

 

In addition, a lot of technologies and ideas get stranded in their commercialization phase, when still research money is required, but no more funds are available because the government says that commercial money should fund this, while the commercial parties think it is still too high-risk for them to participate. It's also called the "valley of death" of technology development. Google for it, and you'll find a ton of links.

[Tridimity]

Is this standard across science papers? Are these ever made public?

 

 

This is just my experience of the review process so far. I don't think that they are normally made public but this may be possible.

[swansont]

 

I completely disagree with this. Almost every research group on the planet is in need of more funding.

 

 

I've never seen pleas for funding within the presentation of the science. There's a separate, formal procedure for this for obtaining grants.

[John Cuthber]

 

I completely disagree with this. Almost every research group on the planet is in need of more funding.

 

At least in the Netherlands, and perhaps even in all of Europe, researchers are encouraged to seek industrial/commercial participation for their research and especially their funding. Because of the economic crisis, the old-fashioned government-paid research budgets have been slashed everywhere.

 

In addition, a lot of technologies and ideas get stranded in their commercialization phase, when still research money is required, but no more funds are available because the government says that commercial money should fund this, while the commercial parties think it is still too high-risk for them to participate. It's also called the "valley of death" of technology development. Google for it, and you'll find a ton of links.

OK, I was in a bit of a rush when I wrote that.

What I mean is webpages openly touting for money are a bit of a giveaway.

 

Anyway, Pears, perhaps the next time you see something you are not sure about you could post a link here and we can have a look to see if we think it's legit, and if not, what we think are the tells.

[pears]

Thank you. I'll be sure to.

[CaptainPanic]

 

I've never seen pleas for funding within the presentation of the science. There's a separate, formal procedure for this for obtaining grants.

 

I agree with that. And also John Cuthber already explained his remark a bit better... so it seems we're all on the same line here (and that shouldn't be a surprise to anyone). There are some subtle differences how the mainstream scientific institutes get their money, and how the crackpots attempt to get their money.

 

Let me add this:

 

Crackpot website: black background, with Times new Roman font, and a purple blinking banner

Mainstream science website: white background, with lots of green and blue (colors of nature, showing dedication to sustainability). Banners use flash.

 

Crackpot website: "Give me your donations"

Mainstream science website: "Currently seeking commercial participation"

 

Crackpot website: Put the 'science' right on the front page

Mainstream science website: no concrete science on the front page - at most some recent successes and highlights. No ongoing projects.

 

Crackpot website: No slogan, no mission, no vision, and no strategy

Mainstream science website: Slogan, mission, vision and strategy

 

I'm just joking about the packaging. The main difference obviously between science and crackpottery is that the mainstream science is tested and tested again. That its underlying assumptions are clear and that other people - with sufficient materials and equipment - can reproduce the result, or prove it wrong. Crackpots often miss that approach altogether. That, for me, is the first signal that something weird is going on.

[Tridimity]

 

I agree with that. And also John Cuthber already explained his remark a bit better... so it seems we're all on the same line here (and that shouldn't be a surprise to anyone). There are some subtle differences how the mainstream scientific institutes get their money, and how the crackpots attempt to get their money.

 

Let me add this:

 

Crackpot website: black background, with Times new Roman font, and a purple blinking banner

Mainstream science website: white background, with lots of green and blue (colors of nature, showing dedication to sustainability). Banners use flash.

 

Crackpot website: "Give me your donations"

Mainstream science website: "Currently seeking commercial participation"

 

Crackpot website: Put the 'science' right on the front page

Mainstream science website: no concrete science on the front page - at most some recent successes and highlights. No ongoing projects.

 

Crackpot website: No slogan, no mission, no vision, and no strategy

Mainstream science website: Slogan, mission, vision and strategy

 

I'm just joking about the packaging. The main difference obviously between science and crackpottery is that the mainstream science is tested and tested again. That its underlying assumptions are clear and that other people - with sufficient materials and equipment - can reproduce the result, or prove it wrong. Crackpots often miss that approach altogether. That, for me, is the first signal that something weird is going on.

 

I would agree with most of this, but would add the caveat: do not conflate 'non-mainstream Science' with 'crackpottery'. Granted, most of the non-mainstream material will turn out to be, at best, pseudoscience. However, there are occasions when scientific concepts are presented and initially receive a cold reception (or are blatantly ignored for the next few decades or centuries) until the requisite technology and circumstances are available to confirm the theory experimentally. A scientific concept that initially fails to be accepted as a part of mainstream Science may simply be too revolutionary for the conservative scientific community to whom it is being presented. Paradigm shifts are born in this way.

[Ringer]

Another great way smell test is to look at the ratio of how many times they cite themselves vs anyone else. Also if, say, they're trying to sell you something that is clinically shown to work, but the only clinical trials were in their lab which you have to e-mail the company for the manuscript because they didn't publish it.

[imatfaal]

Another great way smell test is to look at the ratio of how many times they cite themselves vs anyone else. /snipped

 

Definitely - I read an article linked by someone on this website and the vast majority of the references were made up of themselves, superb journals/articles on a different topic, and landmark articles which revolutionised but were now so old that huge swathes of the science had changed again since writing. The meat of the article was not really referenced at all - mainly because it was talking rubbish - but all the ephemera was heavily referenced to no good end except to deceive a lay reader.