Peer review stifles creativity?

[SkepticLance]

I am not expressing a personal opinion here. Just want to see what others think.

 

The latest New Scientist (Australian printed copy, 23 Feb. 08 page 18) has an editorial by Professor Donald Braben of University College London and author of "Scientific Freedom : The Elixir of Civilisation".

 

Prof. Braben suggests that modern scientific structure, including the requirement for peer review is stifling creativity and scientific innovation. He suggests that, in the 20th Century before the 1970's, the greatest discoveries were made by a small group of top scientists (about 400 - roughly the number of Nobel prize winners) who thrived in a freer research environment. He suggests that their success was to a large extent driven by freedom - the ability to work without the restraints that modern research institutions apply.

 

Prof. Braben suggests that the important basic discoveries are not being made in today's world, because of the stifling effect. He suggests a new small group of top scientists (which he calls the Plank Club) be permitted to work in an environment of much greater freedom, without the need to submit to peer review etc. He accepts that current structures are OK for the majority of researchers, but that the top scientists must be set free.

 

What do others think?

[Realitycheck]

If they didn't make any mistakes in the first place, then there would be no need for peer review. Nobody is perfect. Anyone can overlook anything at any time, especially when you consider the vast amount of detail in all of existence.

[Pangloss]

Any scientist can work without peer review. Write what you want, research what you want, it's a free country. Okay, I guess that's not entirely right -- research methods can violate laws or conventions in inappropriate or illegal ways, and that can have an impact on your career. But there's nothing to stop anybody from saying what they want. It's a free society.

 

I'm curious whether what he's really saying is that they should be taken at face value without peer review -- accepted in what they're saying -- or if this is really about acquiring funding, or if it's something else entirely.

 

Maybe he's just pissed-off because we dumped his anonymous post on WTC Building 7 into Pseudoscience and Speculations.

[bascule]

I personally consider peer review integral to rational, evidence-based thinking. In that regard facts and contrary arguments will certainly get in the way of creativity, but isn't that a good thing?

[Pangloss]

I agree.

 

We have talked about other approaches TO peer review here from time to time, trying to deal with the various cons of traditional approaches without introducing too many new cons. There's one web site (the name eludes me at the moment) that's sometimes used for journal publications -- someone help me out here?

[SkepticLance]

Prof Braben suggested that the major discoveries of science were all made by the 1970's and since then we have been essentially filling in details. His view is that academic freedom for top scientists may lead to new discoveries of critical importance. I suspect this is an over-simplification, but maybe someone else has a slant on this.

 

Reminds me of the 'skunk camp' which was a characteristic of McDonnell Douglas in their designing of jet war aircraft. They set up their top designers as a team - the skunk camp - with essentially no restrictions - just a task. Gave them all the resources they needed, and removed any bureaucratic barriers to success. Apparently the system worked very well.

 

Incidentally, the name 'skunk camp' was bestowed by some of the bureaucrats who were disgusted by their freedom. Suggested that the method stank!

 

Could a 'skunk camp' of top scientists lead to new and vital discoveries?

[CDarwin]

Is Braden's supposition that much more fundamental research was accomplished in this 70 year period of the past really true? We're about the build quantum computers and test string theory with the LHC at Cern, evolutionary biology is being remodeled by "evodevo," both new discoveries and new techniques of analysis are sweeping paleontologists in different directions every few years or so, and we can make stem cells out of skin. I'd say at least 80% of everything we know about human evolution has come to light since 1970.

 

So what's this guy complaining about?

 

Although, I personally would like to apply for future membership and this "Plank Club" that gives me unlimited money to do research which requires no verification as to it's legitimacy.

[timo]

There's one web site (the name eludes me at the moment) that's sometimes used for journal publications -- someone help me out here?

Most (all?) journals are available on web pages. You supposedly meant a page that does not use a peer review system - arxiv ?

 

On topic:

 

Would be interesting to know what field Prof. Braben is in. I think if he really claims to have a good overview over all fields of science (or even just natural sciences) he might be overestimating himself. I'd also not be surprised if people in e.g. atrophysics or genetics would disagree with the statement that all great discoveries were made till ~1970.

[swansont]

Peer-review for publishing isn't the enemy of free inquiry, IMO. All peer-review is supposed to do is have people familiar with the subject matter try and find flaws or weaknesses.

 

One might make the argument that a flaw in the system is with funding, since you need to convince someone that your avenue of investigation is worth a grant, and since grant money is finite, it really is a zero-sum game. Money supporting investigations into tripolar dynamical frammistat fields does actually take money away from something else.

[SkepticLance]

Atheist: Prof. Braben is a physics professor.

 

Just to quote a paragraph from his article.

 

" The key to scientific and technological productivity is to give creativity full rein. The academic research that spawned almost all the major advances of the 20th century, and which in turn fuelled spectacular global economic growth, was largely unmanaged. Yet in the 1970's, things changed. Since then, scientists have had to aim their funding proposals at specific objectives. Peer review, seen as fundamental to scientific progress by too many researchers, has removed all spontaneity from the process of generating ideas. Such policies have led to a glittering profusion of new technologies, but most of them stem from major discoveries made decades ago. We are living on seed corn."

 

 

To CDarwin

 

Braben's Plank Club was to be made up of Nobel prize winners or their equivalent. The best of the best in science. Do you think you are one of them? With the greatest of respect, I doubt it.

 

I recall, several decades back, in the Soviet Union, there was to be set up a science town. A place where scientists could live and work together, with all the supports in place to give them a living and get the resources needed for scientific progress. I do not recall it actually happening. Anyone know more about that?

 

I know this is never going to happen. But I would like to see the equivalent set up today. Imagine a number of top scientists, such as Nobel prize winners, offered the chance to go to such a town and carry out research on whatever project they see as most important. Each would have the chance to offer a similar position to, say, five other top scientists to work with them. The town would provide all the services, and all the technician support they needed. Resources would be made available by a team of specialists whose job would be simply to remove barriers to performance by the scientists.

 

I know it would never happen, even though the USA, Russia, China, and the EEC all have the resources to do it. But if it did happen, what discoveries might be made?

[Cap'n Refsmmat]

Is it appropriate to blame this "lack of creativity" on peer review or on the lack of funding made available to researchers to do what they want?

[swansont]

I know this is never going to happen. But I would like to see the equivalent set up today. Imagine a number of top scientists, such as Nobel prize winners, offered the chance to go to such a town and carry out research on whatever project they see as most important. Each would have the chance to offer a similar position to, say, five other top scientists to work with them. The town would provide all the services, and all the technician support they needed. Resources would be made available by a team of specialists whose job would be simply to remove barriers to performance by the scientists.

 

I know it would never happen, even though the USA, Russia, China, and the EEC all have the resources to do it. But if it did happen, what discoveries might be made?

 

 

Jennifer Ouellette has been blogging about life at the Kavli Institute which sounds a little like that.

[CDarwin]

Braben's Plank Club was to be made up of Nobel prize winners or their equivalent. The best of the best in science. Do you think you are one of them? With the greatest of respect, I doubt it.[/Quote]

 

Just a joke.

 

I recall, several decades back, in the Soviet Union, there was to be set up a science town. A place where scientists could live and work together, with all the supports in place to give them a living and get the resources needed for scientific progress. I do not recall it actually happening. Anyone know more about that?

 

Indeed, Akademgorodok. Khrushchev built it as a haven of intellectual freedom to bolster The Glorious Achievements of the Glorious Soviet Scientific Intelligentsia in the 50s. It's a middle-class suburb now. I don't know that much ever came out of it scientifically.

 

I know this is never going to happen. But I would like to see the equivalent set up today. Imagine a number of top scientists, such as Nobel prize winners, offered the chance to go to such a town and carry out research on whatever project they see as most important. Each would have the chance to offer a similar position to, say, five other top scientists to work with them. The town would provide all the services, and all the technician support they needed. Resources would be made available by a team of specialists whose job would be simply to remove barriers to performance by the scientists.

 

I know it would never happen, even though the USA, Russia, China, and the EEC all have the resources to do it. But if it did happen, what discoveries might be made?

 

I'm far from convinced that winning a Nobel prize should guarantee you a life-long sinecure at the expense of all the rest of the scientific endeavor. You'd just end up with a lot of senile old men (you know how long it takes to win a Nobel?) chasing their pet theories and wasting resources.

[SkepticLance]

To CDarwin

 

Nobel prize winners don't need a sinecure. The value of the prize is enough to retire on, and the reknown enough to make sure you get lots of opportunities to make money easily, such as on lecture circuits.

 

Your point about age is valid, though. Perhaps the Nobel winners should be required to choose younger scientists that they consider exceptional? They become head hunters.

[Pangloss]

This is the site I was thinking of:

 

http://arxiv.org

http://en.wikipedia.org/wiki/ArXiv.org

 

It focuses on an "endorsement" system rather than peer review. I think the idea is that it provides relatively useful information without the high signal-to-noise ratio and non-academic, expository information common to regular web sites and popular magazines, etc.

[iNow]

This is the site I was thinking of:

 

http://arxiv.org

http://en.wikipedia.org/wiki/ArXiv.org

 

It focuses on an "endorsement" system rather than peer review. I think the idea is that it provides relatively useful information without the high signal-to-noise ratio and non-academic, expository information common to regular web sites and popular magazines, etc.

 

The downside of such a system is how easily things like creationism and intelligent design would become "endorsed" if the link to endorse them was just shared with a few congregations here and there... communicated and/or marketed to supportive populations. That wouldn't be so good...

 

Science is not a democracy, and often people really are wrong and need to be told so.

 

 

That's just my "only even prime number" of cents...

[ajb]

A lot of people publish just via the arXiv. The "endorsement" is a good idea as it stops the arXiv being hijacked and full of non-sense. Your papers can be removed from the arXiv if they really are rubbish. I personally know of only one case of this. I won't say who, but he was/is an endorsed author.

 

So, I would take the arXiv to be a reliable source of scientific information, but it is not peer review in the strong sense.

[swansont]

I'm far from convinced that winning a Nobel prize should guarantee you a life-long sinecure at the expense of all the rest of the scientific endeavor. You'd just end up with a lot of senile old men (you know how long it takes to win a Nobel?) chasing their pet theories and wasting resources.

 

The laser cooling, BEC and optical comb physics Nobels went to relatively young physicists. OTOH, I don't think that you have much trouble securing funding when you have been doing that level of research, even before you win the prize.

[Severian]

This is the site I was thinking of:

 

http://arxiv.org

http://en.wikipedia.org/wiki/ArXiv.org

 

It focuses on an "endorsement" system rather than peer review. I think the idea is that it provides relatively useful information without the high signal-to-noise ratio and non-academic, expository information common to regular web sites and popular magazines, etc.

 

A huge amount of rubbish is published on the arXiv, so I would say it has a rather low signal-to-noise ratio (I presume that is the way round you meant).

 

I am a professional scientist working on high energy phenomenology, so in principle hep-ph is my field (http://arxiv.org/list/hep-ph/new), but I am lucky if I find one paper in 30 to be interesting.

[gcol]

Peer review does not really stifle creativity, but it can stifle the dissemination of the fruits of creative thinking. Peer review is, after all, a kind of well-meaning censorship. It is a blunt edged weapon to sort the wheat from the chaff. Creative thinking and new ideas can germinate quite readily within the isolation of a hermit's cave.

 

Fortunately the internet allows non-peer reviewed ideas to be published widely, beyond the control of peer review bodies. Independant and innovative thinkers can sidestep the process, often to the chagrin of the establishment perhaps, but that is the way of the wired-up world.

 

Perhaps a parallel can be seen in the explosion of ideas that accompanied the invention of the printing press.

[CDarwin]

The laser cooling, BEC and optical comb physics Nobels went to relatively young physicists. OTOH, I don't think that you have much trouble securing funding when you have been doing that level of research, even before you win the prize.

 

Then why do we need to build a city for them? It sounds like we already have the sort of set-up that Braden is advocating.

[Pangloss]

A huge amount of rubbish is published on the arXiv, so I would say it has a rather low signal-to-noise ratio (I presume that is the way round you meant).

 

I am a professional scientist working on high energy phenomenology, so in principle hep-ph is my field (http://arxiv.org/list/hep-ph/new), but I am lucky if I find one paper in 30 to be interesting.

 

That's a shame, but I guess the question would be whether 30:1 is a good ratio or a bad ratio, in terms of what it could be compared with. How would you rate it against the old Physics newsgroups, for example?

 

Also, do you think it has any value filling the perceived gap between popular magazines and peer-reviewed journals? It seems to me that it could have some value for people who are not "in the work" and are just trying to follow it out of personal interest, but are able to do so at a higher level than what is typically published in SciAm, for example (which of course also runs several months behind current events).

[Severian]

That's a shame, but I guess the question would be whether 30:1 is a good ratio or a bad ratio, in terms of what it could be compared with. How would you rate it against the old Physics newsgroups, for example?

 

It has become a lot worse in recent years (in my field anyway). It is difficult to tell how much that is my perception though - over the years I have gotten much better at spotting the bullshit.

 

The most worrying trend in modern physics is that papers are now being judged not on their scientific merit, but by the number of citations they have. In my opinion, this is very dangerous.

 

Also, do you think it has any value filling the perceived gap between popular magazines and peer-reviewed journals? It seems to me that it could have some value for people who are not "in the work" and are just trying to follow it out of personal interest, but are able to do so at a higher level than what is typically published in SciAm, for example (which of course also runs several months behind current events).

 

I think it would be of interest to only a very small number of people. In fact, if anything it will only mislead them, especially since they may be unable to tell a good paper from a bad one.

 

I am a referee for several journals, and the last three papers I have reviewed I have rejected, not because I didn't like their subject (or even viewpoint), but because they were actually wrong. They all based significant parts of their work on quantifiable false statements. However, all 3 of these articles are still available on arXiv and I don't believe an interested amateur (or for that matter a careless professional) would spot the mistake.

[CharonY]

Logically, I'd assume (as mentioned earlier) that peer review does not have a stifling effect at least the effect should be neglible (on impact) compared to funding limitations. Of course if you publish more, your chances are better to get a grant, but then the grant proposal is getting peer-reviewed, anyhow. And if both peer-review steps are limited, how should funds be distributed?

 

That being said, especially noble prize winners usually won't have problems in getting grants, as such I cannot see the advantage of club of nobles (they get into important positions anyway).

 

That's just my 2 cents after reading stuff while not sleeping. Hate the weather.

[SkepticLance]

I recall what happened here in NZ, and I assume something similar occurred in other countries.

 

We had an organisation (DSIR) which had a number of divisions carrying out scientific research. Money was dispensed to those divisions by government according to the perceived importance of the division, and supported a wide range of research.

 

Then the government changed the structure. Today, the DSIR no longer exists -replaced by Crown Research Institutes. Within these, any scientist wanting research funds must make a presentation to the appropriate government stooges, who will award money according to what they see as being in the national interest. These are career politicians and bureaucrats who have no feel for the value of pure research.

 

The result has been a whole bunch of scientists who waste a big chunk of their time in the politics of begging for money, and who are constrained within tight boundaries when doing their research. I would imagine that Brabens ideas would be seriously attractive to them.

 

What is it like in your corner of the world?