On The Mistakes Actual Scientists Make

[Bignose]

I wanted to start a new thread, ancillary to the last one I started here on what amateurs actually have to do to make meaningful contributions (see http://www.scienceforums.net/topic/75040-on-unknowns-making-meaningful-contributions/#entry744966 ), about another very common annoyance I see in the speculations section, which is so very, very often the speculators refuse to admit they make mistakes.

 

Back in 2006, solar scientists announced that the next sunspot cycle was going to be unprecedented. See http://www.ucar.edu/news/releases/2006/sunspot.shtml

 

There are some good quotes in here:

 

The NCAR team predicts the next cycle will be 30-50% more intense than the current cycle.

and

The scientists have confidence in the forecast because, in a series of test runs, the newly developed model simulated the strength of the past eight solar cycles with more than 98% accuracy.

In 2009, this prediction was scaled back some (see http://science1.nasa.gov/science-news/science-at-nasa/2009/29may_noaaprediction) but still predicted to be a very impactful sunspot maximum.

 

$1 to 2 trillion in damages to society's high-tech infrastructure

and

"four to ten years for complete recovery."

Well, here we are today, and virtually nothing. http://www.nytimes.com/2013/09/24/science/space/the-sun-that-did-not-roar.html?_r=2&

 

If theres anyone who has figured it out, I havent heard, thats for sure

said said Douglas Biesecker, a physicist at the Space Weather Prediction Center and the chairman of a panel that had issued predictions about the solar cycle.

 

In other words, the foremost experts in this area are unafraid to admit that when they made a prediction and it did not match well with the actual measured values... they admit that their model it wrong. And needs more work.

 

No digging in of the heels. No decrying that everyone who didn't believe in the [wrong] model belongs to the religion of science. Just acceptance their model obviously didn't work well, and that they need to go back and improve it. I wish we'd see a lot more of this in this Speculations section.

 

I like to write in many of the Speculations threads that the creativity to come up with a new idea is needed. It is craved-for in the sciences. But to actually do science, you need to make predictions with your model and see how it matches up with reality. And then admit when your model doesn't work and needs to be improved.

[Phi for All]

Excellent post.

 

And look at the way the science is phrased. It's about the model predicting this or that (which allows for error), NOT that this or that is going to happen (an assertion which leaves no room for error). The scientists are confident but not absolutely certain. Damages are estimated instead of given as surety. I really wish speculators would learn to do this. Nothing sparks a pile-on like claiming something non-mainstream HAS to be the Truth.

[CaptainPanic]

One of the problems perhaps is that very few scientists publish failed research... this solar cycle research is one of the few exceptions, because the work was used to make predictions for the future that were verifiable in the relative near future. Far enough into the future that they would publish before they could check the results, and near enough that the same researchers were still working and could admit their mistakes.

 

Other fields of research either make predictions which can be verified quicker - and if the predictions are wrong, they will not get published. Or we only find out mistakes decades or centuries later, in which case it is a new generation of scientists who publish the improvements.

[ewmon]

Remember the Italian seismologists prosecuted for the 2009 L'Aquila Earthquake.

[Bignose]

One of the problems perhaps is that very few scientists publish failed research... this solar cycle research is one of the few exceptions, because the work was used to make predictions for the future that were verifiable in the relative near future. Far enough into the future that they would publish before they could check the results, and near enough that the same researchers were still working and could admit their mistakes.

 

Other fields of research either make predictions which can be verified quicker - and if the predictions are wrong, they will not get published. Or we only find out mistakes decades or centuries later, in which case it is a new generation of scientists who publish the improvements.

There is a Journal of Errology http://www.bioflukes.com/JoE that specifically wants you to document what your idea was, what you did to test it, and how the results came out spectacularly wrong from your idea. The concept being, of course, that we can all learn from mistakes, and maybe avoid trying the same flawed mistake in the first place.

[swansont]

One of the problems perhaps is that very few scientists publish failed research

 

That's partly/mostly due a flaw in the publishing system. OTOH, a significant amount of research is later shown to be wrong (also affected by publishing paradigms), so there is failed research out there.