Dutch biologists complain about publishing culture in academia

An interesting article on the current academic climate in the Dutch newspaper NRC Handelsblad: biologists complain about the pressure in the current academic climate not to try to replicate (let alone refute) other scientists' results, but to exaggerate the results of your own research.

"In my field there are articles in Nature, Cell or Science, of which all experienced people know: this can't be right," says Hans Clevers, director of the Hubrecht Instituut in Utrecht and former president of the Dutch Academy of Science, "but rarely does somebody write that explicitly in an article. So every now and again I am approached at a conference by a PhD researcher from a remote university who has been trying for years to replicate that publication. It is very inefficient." [...] Ecologist Raymond Klaassen of Groningen University blames the "short-winded academic climate, that focuses on scoring." "If you find a deviating pattern in one year, then the current practice is to publish that with a lot of ballyhoo in as high-ranking a journal as you can."

The Dutch word used in the original article (which I translated here as "short-winded") is "hijgerig": from hijgen, Dutch for "to pant". It evokes an image of heavy competition and short-termism. It reminds me of the atmosphere at a high-ranking Dutch university that has made quite a name in behavioural economics: scoring was the norm, in the best economics journals, but I saw little of a long-term research agenda. Nevertheless, I don't believe it got as bad there as the biologists describe in this article (knock on wood). I do see it in fisheries science: 2048, anyone?

On interdisciplinarity

Check out the really cool cover of Nature's special feature on interdisciplinarity!

Of course, as an economist I especially like their inclusion of "Invisible Hand" as the sole superhero representing the social sciences in their scientific team of Avengers. But it is also symbolic for the fact that economists have, in my view, gone the furthest in integrating their discipline with the natural sciences. This holds particularly for environmental and resource economists, who by definition deal with problems of the natural environment like pollution and overfishing. The reason is pretty geeky: most economic research is quantitative, and quite a lot involves the development of mathematical models. And whaddayaknow: so do climate science, population biology, hydrology, and a host of other natural sciences. Give me your equations and I'll plug them into my CGE model.

It is actually much, much harder to truly integrate qualitative social sciences like sociology or anthropology with quantitative sciences - even with a social science like economics. Models like IMAGE and DICE describe the global climate as well as the economy; the Gordon-Schaefer fisheries model and Colin Clark's work on renewable resource use, which use basic models from population biology like logistic growth, are part of the standard canon of resource economics since decades; when Daniel Pauly criticizes the limited impact of the "social sciences" on fisheries research, he lumps together economics with biology, not sociology. Meanwhile, it has taken until 2009 that the Nobel committee finally recognized anthropologist Elinor Ostrom for her contributions to the economics of common pool resources, and economists and sociologists share little but contempt for each others' fields. The Indian economist Jagdish Bhagwati is said to have joked that good economists reincarnate as physicists; wicked economists reincarnate as sociologists. But Ostrom's Nobel also shows that things are changing, especially in the field of institutional economics. Let's have more of that in the future.

What is this researcher afraid of?

A thoughtful article in the Dutch newspaper De Volkskrant of 25 September 2013 deals with publishing in Science and Nature and cites a researcher who wishes to remain anonymous:

We try every now and again to get in the top journals, because Science or Nature look good on your cv (...) But it's nonsense. My two most important articles (...) were not even considered by Science or Nature. On the other hand: I once had a Nature publication on what was absolutely the worst experiment I ever did. But that was on a hot and photogenic topic. (...) What matters to the editing boards of those journals is to keep up the status of the journal, by keeping up their impact factor. This automatically leads to a preference for fields and articles that will be cited a lot. (...) The result is also that editors have less expertise in less popular disciplines, and they have failed recently in that respect.

This is very similar to Ray Hilborn's complaints about the quality of these two journals. But then, why does this researcher want to remain anonymous? What is he or she afraid of? Being sued or ostracized by Nature and Science editors? (If he or she has reason to be afraid, why am I putting this on my blog? Oh wait...)

My impressions of the 2013 ICES Annual Science Conference

My impressions after a week of presentations, discussions, and lots of delicious food:

• Of all the interdisciplinary conferences I've been to so far, the ICES meeting was the most scientific (read: least political, notwithstanding ICES's role as advisory body for fisheries policy), and the most constructive in its interaction with social scientists (read: economists). Besides EAERE (which I consider a disciplinary meeting) I was once at an ESEE meeting, and once at the European Congress for Conservation Biology. I had mixed feelings about those for their tendency to bash "mainstream economics" (whatever that may be) and to blur the line between science and activism. Perhaps it's because those communities have the hidden assumption that nature is best left alone by man, whereas fisheries scientists investigate, by definition, a form of interference in nature.
• Is it just me, or is there a major disconnect between textbook fisheries economics and the practice of fisheries management? Concepts we teach (notably maximum economic yield and the role of the discount rate) are nowhere to be seen - in fact, I once heard a fisheries industry representative refer to maximum economic yield as "a plaything for economists". In our teaching we hardly pay attention to the stochastic nature of fish stocks, but these days fisheries science is all about reference points and harvest control rules - which only make sense in a stochastic context.
• Economists can make big contributions to fisheries management by further strengthening how fisheries models describe human behaviour. So far those contributions were largely confined to modelling where fishers fish, but what about investments in gear, or boats? Let alone market structures, global developments (tilapia!), value chains, and policy-makers.
• Iceland is like an extreme version of Norway. Thought the Norwegian landscape was rugged? Iceland has volcanoes, and geysers! And where I thought Norwegians don't give a hoot what the rest of the world thinks of hunting and whaling, only Icelanders can serve raw whale meat and rotten shark to a crowd of foreign scientists. (And it was delicious! The whale, that is.) Neither do Icelandic pubs have qualms with playing the entire Velvet Underground & Nico, including John Cale's ear-piercing viola solo in Heroin.

Dear students, please question my authority

Some more thoughts on the Stapel saga.

In most fraud cases Diederik Stapel told his students that he would perform his experiments, collect the data, do the analysis, and give the students the results. Some students wanted to be at the experiments (a good suggestion, because you can learn a lot from it), but he wouldn't allow them. A few students wanted to see the raw data, but when they pressed him he expressed doubts whether they were good enough to be his PhD students. A filthy intimidation tactic if you ask me. But not only his students were duped: other academics, like the Dutch professor Roos Vonk, co-authored articles that turned out to be based on fake data. In the end the whole fraud came out because a few students finally had the courage to stand up to him (and, possibly, the university board where he had a lot of friends - luckily the board did the right thing and took their complaints seriously). Other students, who allowed themselves to be intimidated, now have flawed dissertations. A few of them have left science because of the affair.

Dear students, learn from this. I promise I won't cook the books, but don't take my word for it - don't take anyone's word for anything. Not just your thesis supervisor. After you graduate you will work with other people, like your boss or your co-authors. They can make mistakes. They can lie. When your name is on a proposal, a thesis, or an article, than you (and your co-authors) are responsible for its contents. Convince yourself that its contents is correct. Yes, I do the same with your contributions.

I know that in some cultures it is impolite to question the advice of your superiors, much like foot soldiers are supposed to follow their sergeant's orders. That may work in the army, but we're not in the army here. The one order I give you is not to take orders from me.

Wise words from the Bird

Originally I planned to include in my PhD thesis the following quote:

Learn all that stuff and then forget it.

At the time I thought it was Miles Davis who said that, but it might as well have been Charlie Parker (sorry Miles, Bird makes for a better blog post title). I'm not really into jazz: the quote tricked me into looking up some Miles Davis and Charlie Parker on Spotify but after a few songs I always decide to turn on Pelican instead. The closest to jazz that I listen to frequently is the Tom Waits. I absolutely love Tom Waits. Oh, and Ethiopians like Mulatu Astatke or Getatchew Mekuria, but only when I'm in the mood.

But this quote - wow, I could tattoo it on my forehead if I were into that kind of thing (don't worry, I'm not). It defines how I view the theories I work with, as well as the musical traditions I play. I consider myself a mainstream economist, but I do have my question marks regarding the behavioral and ethical models applied in my field. I don't think there is any meaningful way of measuring existence value, if it exists at all. I don't believe economic valuation can capture moral or religious considerations. I appreciate the role that social norms, traditions, or downright stupidity can play in human behavior. But I also think that it is too easy to throw your hands in the air and declare the entire body of neoclassical economics as overly abstract nonsense, or worse, to refuse to learn microeconomics because economists did not predict the latest financial crisis.

The bottom line is that you can only criticize a theory if you fully understand it. By the same token, you can only renew a musical tradition if you master its techniques, its rhythms, its harmonies, and its social context. Miles and Bird could only invent and develop bebop because they understood what came before it. Likewise, if you want better economics, well, make sure that you first understand the orthodoxy.

On the other hand, there is also the "forget it" part. Don't let the tradition hinder your creativity. Dare to break its rules, to explore its boundaries. Dare to question a theory's hidden assumptions, don't accept such excuses as "this is how we have always done it." Know the rules, but don't obey them.

The quote, by the way, never made it into my thesis. My professor thought it cynical, as if I recommended the reader to forget all I wrote. I didn't agree, of course, but I also figured that if he interpreted the quote like that, so would many others. So I left it out. I still regret that decision.

Crystal ball gazing, the academic way

The excitement of international science workshops

So I'm in this international research project VECTORS, dealing with many exciting and fascinating issues, working with scientists from all over Europe. But if I tell you I was at a VECTORS workshop in Edinburgh last week, it's not exactly like we were sipping 40 year old Talisker in Edinburgh Castle with Queen Elisabeth. A business hotel near the airport is more like it: easier to reach, lots of meeting rooms, good food, and no distractions.

Scenarios were the main theme this time. A lot of the developments that VECTORS deals with are highly uncertain. How warm (or cold) will our waters be in the coming 50 years? How many sunbathers will visit Mediterranean beaches? How many international transport vessels will visit the ports of Rotterdam and Antwerp?

You may be tempted to address these questions through some sort of sensitivity analysis: do your calculations for different values of the uncertain variables, or rather, combinations of different values. If you know the probability of some outcome of some variable (say, "the probability that temperatures increase by 1 degree or more is 30%") you could estimate the probability of some outcome of your models' output ("the probability that average income increases by €100 per month or more is 50%"). But you will run into a number of problems:

• You don't know probabilities.
• Even if you knew the probabilities they would not be independent. A high nutrient load is more likely if there are many people around spending their money on lots of meat or highly fertilised crops than if the population is small and poor.
• The number of possible combinations grows exponentially with the number of variables. Suppose you want to consider three levels of each variable. Then one variable gives you three possible outcomes; two variables have nine possible outcomes; three variables have 27 possible outcomes; 10 variables have 59,049 possible outcomes. And then the ecologists in your team tell you that running one such outcome on their model takes one month. So if you have 5000 years you can do a sensitivity analysis of all 10 variables!
• There are many other possible developments that are difficult to capture in numbers, such as changes in regulations, customs, technologies, and so on.
• Policy makers have neither the time nor the energy to read your entire sensitivity analysis. They want something you can summarise in one page (which they skim rather than read).

So what do we do? We develop scenarios. But what are scenarios?

A scenario is not a prediction of the future. It is more like a story line that describes how the world might develop in the future, taking into account different possible trends. It should be consistent, and ideally you have a set of scenarios that spans a fairly wide range of possible developments.

A scenario is not a policy option. When you see the different scenarios from, say, the IPCC, you may be tempted to say: "let's go for this one." But although you may certainly like some outlooks more than others, the idea is that you don't know which one will come true - and you have no influence on which one comes true.

At least, scenarios can help us to make sense of the complexity of the different social, economic, political, and biophysical changes that may take place in the future. A few consistent story lines are easier to understand than countless histograms and plots, for scientists as well as policy makers. And they still allow us to explore how different policy choices may work out in the future.

At some point I joked that we should have Philip Pullman write our scenarios, but I was only half joking. It's at least as much an art as it is a science.