I want to discuss this danger and the various tendencies that seem to me
to create it, or augment it. I can draw on personal experiences in my
40 years of work on various branches of science and also on many of the
great controversies that have occurred in that same period.
I will start very naively by a definition of what a scientist is. He is a
person who will judge a matter purely by its scientific merits. His
judgment will be unaffected by the evaluation that he makes or the
judgment that others would make. He will be unaffected by the historical
evaluation of the subject. His judgment will depend only on the evidence
as it stands at the present time. The way in which this came about is
irrelevant for the scientific judgment; it is what we now know today that
should determine his judgment. His judgment is unaffected by the
perception of how it will received by his peers and unaffected by how it
will influence his standing, his financial position, his promotion - any
of these personal matters. If the evidence appears to him to allow
several different interpretations at that time, he will carry each on of
those in his mind, and as new evidence comes along, he will submit each
new item of evidence to each of the possible interpretations, until a
definitive decision can be made. That is my naive definition of a
scientist.
I may have reduced the number of those whom you think of as scientists
very considerably by that definition. In fact, I may have reduced it to a
null class. But, of course, we have to be realistic and realize that
people have certain motivations. The motivation of curiosity is an
important one, and I hope it is a very important once in most scientists'
minds. But I doubt that there are many scientists to whom the motivation
of curiosity about nature would suffice to go through a lifetime of hard
struggle to uncover new truths, if they had no other motivation that would
drive them along that same path. If there was no question about appealing
to one's peers to be acknowledged, to have a reasonably comfortable
existence, and so on, if none of this came into the picture, I doubt that
many people would choose a life of science.
When the other motivations come into the act, of course the judgment
becomes cloudy, becomes different from the ideal one, from the scientific
viewpoint, and that is where the main problem lies. What are the
motivations? If there are motivations that vary from individual to
individual, it would not matter all that much because it would not drive
the scientific community as much to some common, and possibly bad,
judgment. But if there are motivations that many share, then of course
that is another matter; then it may drive the whole scientific community
in the field in the wrong direction. So, we must think: what are the
communal judgment-clouding motivations? What is the effect of the
sociological setting? Is our present-day organization of scientific work
favorable or unfavorable in this respect? Are things getting worse, or
are they getting better? That is the kind of thing we would like to know.
The pace of scientific work continues to accelerate, but the question is
whether the pace of discovery will continue to accelerate. If we were
driving in the wrong direction - in the direction where no new ideas can
be accepted - then even if scientific work goes on, the progress would
be stifled. This is not to suggest that we are in quite such a
disastrous position, but on the other hand, I am not going to suggest
that all is well.
What are the many factors that many people might share that go against
the acceptance of scientifically valid new ideas? One obvious factor
that has always been with us is the unwillingness to learn new things.
Too many people think that what they learned in college or in the few
years thereafter is all that there is to be learned in the subject, and
after that they are practitioners not having to learn anymore. Of
course especially in a period of fairly rapid evolution that is very
much the wrong attitude; but unfortunately it is shared by many.
Yes, I have wondered whether one should in fact pursue subjects with a
big wall between two groups that are working in the same field, so that
they absolutely cannot communicate, and see a few years later whether
they come even approximately to the same conclusion. It would then give
some perspective of how much the herd behavior may have been hurting.
But we don't have that. Even with our Soviet colleagues, unfortunately,
we have too much contact to have a display of real independence, to see
where it would have led.
This question of how the support of science - and I don't mean only the
financial support but also the journals, the judgment of referees, the
invitations to conferences, acknowledgments of every kind - how that
interacts with the question of herd behavior, is what I will now
discuss.
It is important to recognize how strong this interaction really is.
Suppose that you have a subject in which there is no clear-cut decision
to be made between a variety of opinions and therefore no clear-cut
decision to be made in which direction you should put money or which
direction you should favor for publications, and so on. No doubt
opinions would need a multidimensional space to be presented, but I will
at the moment just represent them in a one-dimensional situation.
Suppose you have some curve between the extreme of this opinion and
the extreme of that opinion. You have some indefinite, statistically
quite insignificant distribution of opinions. Now in that situation,
suppose that the refereeing procedure has to decide where to put money
in research, which papers to publish, and so on. What would happen?
Well, people would say, "We can't really tell, but surely we shouldn't
take anybody who is out here. Slightly more people believe in this
position than in any other, so we will select our speakers at the next
conference from this position on the opinion curve, and we will judge to
whom to give research funds," because the referees themselves will of
course be included in great numbers in some such curve. "We will select
some region there to supply the funds."
And so, a year later what will have happened? You will have combed out
some of the people who were out there, and you will have put more people
into this region. Each round of decision making has the consequence of
essentially taking the initial curve and multiplying it by itself.
Now we understand the mathematical consequence of taking a shallow curve
and multiplying it by itself a large number of times. What happens? In
the mathematical limit it becomes a delta function at the value of the
initial peak. What does that mean? If you go for long enough, you will
have created the appearance of unanimity. It will look as if you have
solved the problem because all agree, and of course you have got
absolutely nothing. If no new fact has come to light and the subject
has gone on for long enough, - this is what happens. And it does
happen! I am presenting it in its clearest form, and it is by no means
a joke. If many years go by in a field in which no significant new
facts come to light, the field sharpens up the opinions and gives the
appearance that the problem is solved.
I know this very well in one field, which is that of petroleum derivation,
where the case has been argued since the 1880's. At the present time most
people would say the problem is completely solved, though there is
absolutely nothing in the factual situation that would indicate a
solution. It is also very clear there that the holding-in that has taken
place has been an absolute disaster to research. It is now virtually
impossible to do any research outside the widely accepted position. If a
young man with no scientific standing were to attempt this, however
brilliant he might be, the wouldn't have a hope.
I believe that our present way of conducting science is deeply afflicted
by this tendency. The peer review system, which we regard as the only
fair way we know of to distribute money (I don't think it is, but it is
generally thought to be) is an absolute disaster. It is a completely
unstable method. It is completely prone to this tendency; there is no
getting out of it. The more reviews you require for a proposal - now
the NSF requires seven reviewers for a proposal - the more you require,
the more certain it is that you will follow the statistical tendency
dictated by this principle. If you had noise in the situation, it would
be much better. There used to be in the United States many different
agencies, and there was perhaps an odd-ball over here who gave out some
money for one agency, and a funny fellow over there for another. This
was a noisy situation, and it was not driving quite as hard towards
unanimity. But now we have it all streamlined and know exactly to whom
we have to go for a particular subject and, of course, it is an absolute
disaster.
Why is it thought that the peer review system would work for science?
How about trying to make a peer review system work for other forms of
endeavor? Suppose we had a national foundation for the arts, and every
painter had to apply to it to get his canvas and his brushes and his
paints. How do you suppose that would work? I can imagine some of the
consequences, but better than that, we can look them up in historical
examples. If you want to read such, in the book The Experts Speak,
you can do that. There is a long list of them that you can read - it
makes marvelous reading.
Eduard Manet wrote to his colleague Claude Monet, of Renoir: "He has no
talent at all, that boy. Tell him to give up painting."
"Rembrandt was regarded as not comparable with an extraordinarily gifted
artist, Mr. Ripingill."
William Blake spoke of Titian and the Venetians as "such idiots are not
artists."
Degas regarded Toulouse-Lautrec" as merely a painter of a period of no
consequence." One wonders how art would have fared in a peer review
system.
Or would it be different in music? We can read what was said of
Beethoven's compositions by musicians of his time:
"An orgy of vulgar noises" was the verdict of Beethoven's Fifth Symphony
by Mr. Spore, a German violinist and composer.
On Tchaikovsky's appreciation of Brahms, "I played over the music of
that scoundrel Brahms. What a giftless bastard. It annoys me that this
jumping, inflated mediocrity is hailed as a genius." But one could go
on almost endlessly with such quotations. Music would not have fared
any better.
So we see that the herd instinct is a tendency in the human makeup,
which is itself a severe handicap for science. Instead of combating it
as best we can, we have arranged a method of nurturing science which
actually strengthens it enormously - makes it virtually impossible to
depart from the herd and continue to have support, continue to have a
chance of publication, continue to have all the advantages that one
requires to work in a field.
If in a subject there was initially a diversity of opinions, the review
system will assure a very short life for that condition, and soon the
field will be closed to all but those who are in the center.
Once a herd is established, by whatever historical evolution this has
come about, it obtains such firm control that it is extremely difficult
to do anything about it. And even if it were appreciated that that is
the situation, one just doesn't know how to interfere. Where then is
the right to free speech if every journal has to send each article out
to a number of people to review, and the bulk of the people are with the
herd? Usually with just one-third of the reviewers very negative, the
paper does not get published.
So there is no free speech in the sense that you cannot publish diverse
viewpoints. There is also no free speech at conferences because the
same is true there. Would all those who have a divergent opinion be
able to organize their own conference? Very rarely. We (note: meaning
the SSE) represent perhaps an example here showing that it is possible,
but it is pretty rare that one can raise funds to run conferences.
Essentially once the herd is established, it will interfere in any one
of the activities that one would need to further that science.
Would the Dean of a university be willing to promote somebody to tenure
who was outside the pack? He can't, because he has to send out letters
to the leading persons in the field - he may inquire from 20 people
before he gets permission to appoint somebody to tenure - and how can he
get that when the pack is running in another direction than this person?
It is absolutely hopeless! So you establish the situation more and
more.
Once a herd has been established in a subject, it can only be broken by
the most crass confrontation with opposing evidence. There is no
gentle way that I have ever seen in the history of science where a herd
once established has been broken up.
In many subjects such clear evidence is very hard to come by. In the
complex subjects, especially I always think of the earth sciences in
this respect, there are always different ways of interpreting any one
fact; so many complicated things have taken place that any one fact can
have three or four interpretations and the crass confrontation is very
rare.
So then when you have a herd, all the money that you spent on it may be
wasted, or worse than that, it may actually serve to cement further the
bad situation. So it is very likely that money is often spent in
science in a way that is absolutely detrimental to that science.
What does the refereeing procedure really look like? How does it really
go on? If, for example, an application was made in the early 60's or
late 50's suggesting that the person wanted to investigate the
possibility that continents are moving around a little, it would have
been ruled out absolutely instantly without questions. That was crack-
pot stuff, and had long been thought dead. Wegener, of course, was an
absolute crack-pot, and everybody knew that and you wouldn't have any
chance.
Six years later you could not get a paper published that doubted
continental drift. The herd had swung around - but it was still a firm
and arrogant herd.
Shortly after the discovery of pulsars I wished to present an
interpretation of what pulsars were, at this first pulsar conference -
namely that they were rotating neutron stars. The chief organizer of this
conference said to me, "Tommy, if I allow for that crazy an
interpretation, there is no limit to what I would have to allow." I was
not allowed 5 minutes of floor time, although I in fact spoke from the
floor. A few months later, this same organizer started a paper with the
sentence, "It is now generally considered that pulsars are rotating
neutron stars."
I will tell you about a recent application to the Department of Energy
by a colleague of mine and myself for some money to investigate the
chemistry of hydrocarbons at high pressures and high temperatures in the
conditions in which they might be at some depth in the earth. We had
the referee's reports because you are allowed to get them, but not
signed. We got one voluntarily from one of the referees, so we know who
he was. He wrote, "This proposal must be funded. In science every
research project is a risk, but here the risk is negligible because even
if the hypothesis is not correct, this research proposal will contribute
strongly to fundamental science in petroleum engineering, the
thermodynamics of fluids, and geochemistry. If the hypothesis is
correct, the Department of Energy will have hit the jackpot beyond its
wildest imagination." And he continued with the detailed questionnaire
with top marks in every part: the competence of the proposer, the
institution, the test, the facilities, and all that. He gave it top
marks on every point.
There was a second referee who also gave it top marks for all the
questions that are posed on the form. But then the last question is:
"Should this proposal be funded?" and he wrote, "No." And then there
was just a single word after that where it said "If no, why not?" And
he wrote down, "Misguided." It was not funded despite the fact that
most of the referees in fact gave it very high marks, due to the
"misguided," and also similar words were used by two or three other
referees. No reason given; just "don't touch it."
It wasn't the only such that I have submitted over the years now, and
they have all been turned down both at NSF and DOE. It is absolutely
hopeless to get any money in contravention of the opinions that are so
firmly established in the petroleum business now.
That brings me to another problem. If in a subject you have a large
number of people because it has economic applications, that immediately
aggravates the problem. And , of course, in petroleum related matters
there are a huge number of people involved at every step. This means
firstly that a lot of mediocrity is brought into the field and
overpowers the field by sheer numbers; and it also means that much more
commitment to a particular viewpoint has been made by many people. Do
you suppose that the petroleum geologist who has been advising Exxon to
drill for hundreds of millions of dollars for maybe 30 years, will go to
his bosses at Exxon and say, "I am sorry, Sir, but I have been wrong all
those years. We have been finding the petroleum, but if we had searched
for it in another way, we would have found 10 times as much." It is
very unlikely that they will do that. In fact, even if his methods and
his understanding were completely, clearly wrong - even if you had the
crassest confrontation in this case - I don't think that it would be
acknowledged. A very small proportion of people would have that stature
that they would turn around and say, "All my life I have taught or
struggled with these problems on the wrong lines, and now I understand
the right thing." So, in this case, the herd is so firmly established
that one cannot think of converting it. A quotation from Tolstoy comes
to mind:
I know that most men, including those at ease with problems of the
greatest complexity, can seldom accept even the simplest and most
obvious truth, if it be such as would obliged them to admit the falsity
of conclusions which they have delighted in explaining to colleagues,
which they have proudly taught to others, and which they have woven,
thread by thread, into the fabric of their lives.
Another area where it is particularly bad is in the planetary sciences
where NASA made great mistakes in the way in which they set up the
situation. NASA made the grave mistake not only of working with a peer
review system, but one where some of the peers (in fact very influential
ones) were the in-house people doing the same line of work. This
established a community of planetary scientists now which was completely
selected by the leading members of the herd, which was very firmly
controlled, and after quite a short time, the slightest departure from
the herd was absolutely cut down. Money was not there for anybody who
had a slightly diverging viewpoint. The conferences ignored him, and so
on. It became completely impossible to do any independent work. For
all the money that has been spent, the planetary program will one day be
seen to have been extraordinarily poor. The pictures are fine and some
of the facts that have been obtained from the planetary exploration with
spacecraft - those will stand but not much else.
So yes, it is possible to make what is a bad tendency in humans in the
first place (for science at least a bad tendency,) that much worse with
a lack of understanding of how the inward looking effect can be
controlled or at least how it should not be augmented by the method of
nurturing of science.
You may think that what I am saying is that the support for science
poses this intrinsic problem, and that if you want to be fair you have
to go for an unstable system which doesn't work. At first it looks like
that. So should you go for something that's fair - makes people
reasonably happy - but that doesn't work? Or should you go for
something that is not so obviously regarded as fair but does work? It
is a difficult decision to make, but you know there is nothing that says
that things that are fair must also be the things that work. The world
is just not so benign to us. Life is not that easy.
Is there another way of doing it? I suppose that the best that I can
think of is roughly on the lines of what my friend, Arthur Katrowitz
proposed at least for major decisions: The "science court" idea is the
best one. Where a lot is at stake, where a subject has been driven into
an alley, one must set up a science court where the different viewpoints
would be heard, would be argued by the protagonists of each one, with
carefully prepared work. The different viewpoints could be judge, not
by others working in that same field, which would merely take you back
to the herd, but would be judged by a group of very knowledgeable and
very competent scientists distributed over other fields, but with enough
general competence to be able to listen and understand the detailed
arguments of the field in question. I would be much happier to have
subjects surveyed every now and again by a jury of that kind. It has to
be a scientific jury because it would have to understand detailed
scientific arguments, but they do not have to be -and should not be -
from the field in which the decision is to be made.
That is the avenue which I would advise the NSF and such organizations
to pick at this time. I would say that in every field they should set
up such a science court to hear all the different opinions on a
reasonably regular basis. It is true that you cannot do it for every
application that comes along, but it is true that you could do it
sufficiently often for major decisions to break, or at least spoil
somewhat, the herd system. As it is at the moment, the situation seems
not to be understood at all. I have discussed the herd problem with
many people in the funding agencies, and found no understanding of that
problem at all.
I could give you many more examples from my own life of the difficulties
of getting subjects funded. At the present time I am struggling with
the oil and gas business, and after being turned down very firmly by DOE
and NSF, I finally was able to get money from he gas industry itself to
do research which is in good progress now. In this area, which is one
of the worst because no really significant facts have come to light and
everything has been interpreted time and again in the time-honored
fashion, and everyone believes they know in detail now how oil and gas
come to be where they are. And the fact that we find that oil and gas
exist on the other planetary bodies, obviously not due to biology, is
completely ignored. They say there was no oil or gas here, and all that
happened on the Earth was something that was completely specific to
Earth. Of course, it is a peculiar attitude, but that is one that is
widely accepted.
There is one more point I should make. When in a subject a general
attitude or a viewpoint has become established, then it is very easy to
obtain funds to do work in that subject on the bases of what I call
"shoehorn science." I think you will understand what I mean by that.
If you make your proposal which says: "I will demonstrate how this fact
and that fact, that apparently are difficult to see in the accepted
framework, can be figured into that framework," they are all delighted
to give you money. And by the time that has gone on for a long time, so
much work of the shoehorn kind has been diligently done to force the
facts into the pattern that is preordained, that it then looks to many
people as if it all was firmly established. What happens is that they
build a superstructure on what may be no foundation - if I may invent a
"Confucius say" sort of proverb, "Never judge the strength of foundation
by size of building."
In the field of petroleum geology that is really what has happened. The
moment you dare to look a the foundation, you are a scoundrel. I have
made people absolutely wild, shaking their fists at me, when I proposed
in my talks that there was some uncertainty about the origin of
petroleum. One fellow actually wrote a paper that got published, that
there must be life on Jupiter because hydrocarbons have been seen on
Jupiter.
That is my sad story. I believe that we could do something about it,
that we could propose that this kind of a situation be understood in
high quarters - that we could try and have something in the nature of
science courts established, or at any rate some review by independent
persons and not by the herd; but as it is at the moment, I feel that we
are dealing with a large proportion of science funding very firmly in
the wrong hands, and much of it is therefore counterproductive.
LINKS
.