Björn Brembs

So this is essentially what happened instead of us sitting down and thinking how we could spend our money in the most technologically savvy way to the benefit of science, scholars and society. A generation later, roughly US$300 billion poorer and none the wiser, it seems.

The ultimate Open Access timeline

tl;dr: So far, I can’t see any principal difference between our three kinds of intellectual output: software, data and texts.   I admit I’m somewhat surprised that there appears to be a need to write this post in 2014. After all, this is not really the dawn of the digital age any more.

What is the difference between text, data and code?

Lately, there has been some public dreaming going on about how one could just switch to open access publishing by converting subscription funds to author processing charges (APCs) and we’d have universal open access and the whole world would rejoice. Given that current average APCs have been found to be somewhat lower than current subscription costs (approx.

What happens to publishers that don’t maximize their profit?

Trial and error is a successful problem-solving strategy not only in humans but throughout evolution. How do nervous systems generate novel, creative trials and how are errors incorporated into already existing experiences in order to improve future trials?

Come and do research with us!

While the first day (day 2, day 3) was dominated by philosophy, mathematics and other abstract discussions of chance, this day of our symposium started with a distinct biological focus.

 Martin Heisenberg, Chance in brain and behavior

First speaker for this second day on the symposium on the role of chance in the living world was my thesis supervisor and mentor, Martin Heisenberg.

Chance in animate nature, day 2

While our own manuscript on journal rank is almost through the peer-review process, this morning I received several messages announcing the DORA (

 San Francisco declaration on research Assessment

), which I signed immediately.

Official call for an end to journal rank

*During my flyfishing vacation last year, pretty much nothing was happening on this blog. Now that I’ve migrated the blog to WordPress, I can actually schedule posts to appear when in fact I’m not even at the computer. I’m using this functionality to re-blog a few posts from the archives during the month of august while I’m away.

Flashback: Programming Free Will – creative robots

For 14 years, the main research funding agency in Germany, the German Research Foundation (DFG) has stated in its guidelines that submitted grant proposals will be assessed primarily on the basis of their

 content

, rather than counting the applicants’ previous publications. However, not all of DFG’s panels seem to be on board.

Research assessment: new panels, new luck?

Notwithstanding the barrage of criticisms and warnings from every corner of the scholarly community, various initiatives, mainly in the Netherlands, Finland, Germany, France and the UK, continue their efforts for a smooth transition from subscriptions to open access without any further disruptions.

Why open access Big Deals are worse than subscriptions

When I finished my PhD 15 years ago, the neurosciences defined the main function of brains in terms of processing input to compute output: “brain function is ultimately best understood in terms of input/output transformations and how they are produced” wrote Mike Mauk in 2000 (DOI: 10.1038/76606). Since then, a lot of things have been discovered that make this stimulus-response concept untenable and potentially based largely on laboratory

Watching a paradigm shift in neuroscience

Over the weekend, I received the following short message from a Hotmail account:  This email is full of signs that the student may not have been all that interested in any information or discourse, but instead trying to make some sort of ‘gotcha’ statement. Curiously, the author first states that their scientific training is lacking and then they disagree with a scientific position.

How not to contact faculty as student

With the start of the new year 2017, about 60 universities and other research institutions in Germany are set to lose subscription access to one of the main STEM publishers, Elsevier. The reason being negotiations of the DEAL consortium (600 institutions in total) with the publisher.

So your institute went cold turkey on publisher X. What now?

In what area of scholarship are repeated replications of always the same experiment every time published and then received with surprise, only to immediately be completely ignored until the next study? Point in case from an area that ought to be relevant to almost every single scientist on the planet: research evaluation.

How scientific are scientists, really?

This is a post written jointly by Nelson Lau from Brandeis and me, Björn Brembs. In contrast to Nelson’s guest post, which focused on the open data aspect of our collaboration, this one describes the science behind our paper and a second one by Nelson, which just appeared in PLoS Genetics.

Laboratories around the world are generating a tsunami of deep-sequencing data from nearly every organism, past and present.

Data Diving for Genomics Treasure

During my flyfishing vacation last year, pretty much nothing was happening on this blog. Now that I’ve migrated the blog to WordPress, I can actually schedule posts to appear when in fact I’m not even at the computer. I want to use this functionality to re-blog a few posts from the archives during the month of august while I’m away. This is the first post in this series, just to see if it works and what needs to be tweaked.

Flashback: The neurobiology of self-learning

Yesterday, Alex Kacelnik published yet another fascinating discovery – one of many over the years out of his lab. This time, they show how birds can pick even five consecutive locks to get to a food reward:  According to the authors, the birds solve this problem by trial and error, i.e., in the operant, goal-directed way, which is the learning mechanism we study in our lab.

…and now for some lock-picking cockatoos

There can be little doubt that the defunding of public academic institutions is a main staple of populist movements today. Whether it is Trump’s budget director directly asking if one really needs publicly funded science at all, or the planned defunding of the endowments of arts and humanities or the initiatives to completely abolish the EPA and other science agencies.

Are we inadvertently supporting the defunding of public science?

This is our first poster at this year’s SfN meeting in San Diego. It’s about decision-making in fruit flies. We find a probabilistic form of decision-making that suggests that without understanding the mechanisms behind this fundamental uncertainty, we will never fully understand decision-making. Clicking on the image will let you download the PDF Version of the poster.

Do flies in groups make individual choices?

I usually don’t write about politics, but there has been one or the other exception to this rule in the last 12 years of this blog. This time, I’ve been missing one particular response to the various terrorist attacks in recent times, perhaps one of the few readers of this obscure blog has found it somewhere and can point me in this direction?

Have you seen this response to terrorism anywhere?

UPDATE

: I’m still not completely sure I understood Churchland accurately, but there are comments suggesting I may just have completely misunderstood her replies to the interviewer. Read the interview and judge for yourself! I really like Patricia Churchland.

[updated]Churchland: “Ignore science and live a make-believe life!”

In the process of migrating content from the old site to WordPress, I’m also moving some articles from there and re-publishing them here as posts. This one is such a case, originally published on December 7, 2006.

Brains as output/input systems

Update, Dec. 4, 2015: With the online discussion moving towards grantsmanship and the decision of what level of expertise to expect from a reviewer, I have written down some thoughts on this angle of the discussion.

With more and more evaluations, assessments and quality control, the peer-review burden has skyrocketed in recent years.

Why cutting down on peer-review will improve it

This is a post loosely based on an article appearing today in the German newspaper “Frankfurter Allgemeine Zeitung” by Axel Brennicke and me. The raw data for our analysis is available. Please do let us know if you find a mistake.


 UPDATE

*, 09/01/2015: A commenter made me aware of data rows in the raw data we had overlooked before.

Booming university administrations

Rejection is one of the unpleasant but inevitable components of life. There are positive components to rejection: they build character, they force you to deal with negativity and sometimes they force you to change your life to avoid future rejections.

The cost of the rejection-resubmission cycle

At seemingly every possibility in a discussion on peer-review, people apparently feel the need to emphasize that in the current model reviewers (or most academic editors handling peer-review) are not being paid.

Peer-review is not free – it’s a subsidy for publishers

On May 23, the Council of the EU adopted a set of conclusions on scholarly publishing that, if followed through, would spell the end for academic publishers and scholarly journals as we know them. On the same day, the adoption was followed by a joint statement of support by the largest and most influential research organizations in Europe.

The beginning of the end for academic publishers?

To my knowledge, no neuroscientist hypothesizes that there is magic in our heads. However, it appears this is a hypothesis that flies by the editors at the New York Times.

Free will:  it’s not what you think it is

This year marks the 12th anniversary of the publication of this legendary letter to the editor in the Journal of systems and Software:  Nothing really has changed in a dozen years, it seems

12 year anniversary of angry letter to scientific journal editor

Barely a fortnight has passed since

 Science Magazine

published the outcomes of a hoax perpetrated by one of their reporters, John Bohannon. Not surprisingly, the news article was widely criticized, not the least on this obscure blog. The content was simple enough: Bohannon picked a swath of largely fake journals, submitted fake manuscripts and boasted that more than 60% of his submissions were accepted.

Not to be outdone, Nature Magazine rejects data, publishes opinion

Just before Christmas 2019, the Washington Post reported, based on “people familiar with the matter”, that the US Justice Department were investigating the Sci-Hub founder Alexandra Elbakyan for potentially “working with Russian intelligence to steal U.S. military secrets from defense contractors”. Besides such a highly unusual connection, the article also reiterated unsubstantiated (but mainly circulated by publishers) allegations that access

Is the SNSI the new PRISM?

Jeffrey “predatory journals” Beall famously catapulted himself out of any serious debate with an article in the journal TripleC, entitled “The Open-Access Movement is Not Really about Open Access”. In it, Beall claimed that OA proponents don’t care about access, but that they form an “anti-corporatist movement that wants to deny the freedom of the press to companies it disagrees with”. The article is so replete with similarly unhinged fairy

Is this Smits’ tripleC moment?

Open Access (OA) pioneer and OA journal eLife founding member and sponsor, the Max Planck Society just released a white paper (PDF) analyzing open access costs in various countries and institutions and comparing them to subscription costs. Such studies are fundamental prerequisites for evidence-based policies and informed decisions on how to proceed with bitterly needed reforms.

A study justifying waste of tax-funds?

Scholarly journals, on the face of it, emerged in the 17

 th

century as a medium to facilitate communication of scientific discoveries among interested scholars. In the 21

 st

century, it’s not all that different: researchers form communities around topics in which they share a common interest: Journal of Neuroscience, Pediatrics and Neonatology, Journal of Economics or the British Educational Research Journal.

Academic publishers and competition

This post was originally published on the London School of Economics “Impact of Social Sciences” blog, on July 30, 2013:

In various fields of scholarship, scholars accrue reputation via the proxy of the containers they publish their articles in. In most if not all fields, scholarly journals are ranked in a hierarchy of prestige.

Solutions to the looming crisis in science

In her recent editorial on Sci-Hub (an initiative I support), editor-in-chief of

 Science Magazine

Marcia McNutt wrote: The editorial is essentially trying to make the somewhat tenuous but not implausible case that using sci-hub may lead to subscription cancellations which, in turn, may lead to scholarly societies (like those of Dr. McNutts employer, AAAS) to miss revenue they need in order to pay for important services (such as paying

In which a Science editorial demonstrates the ineffectiveness of OA activism

Over the years, publishers have left some astonishingly frank remarks over how they see their role in serving the scholarly community with their communication and dissemination needs. This morning, I decided to cherry-pick some of them, take them out of context to create a completely unrealistic caricature of publishers that couldn’t be further from the truth.

How do academic publishers see their role?

You’d be forgiven if after reading the title of this post, you thought scholars have started to revolt against journal rank. Unfortunately, while there is DORA and of course the evidence that journal rank is like homeopathy, most researchers are still fine with ex-scientists rejecting 92% of all submitted articles and charging a grand sum of more than US$30,000 per article that they select on a whim.

Publisher selects the best open access science – authors complain

Earning credibility in post-factual science?

This morning I was reminded of the age of some of the technology we’re using. Hyperlinks were developed at Stanford University and first demonstrated by their inventor Douglas Engelbart (using the first mouse) in 1968:  On Tuesday, Douglas Engelbart died, even before the scholarly literature was able to fully implement the technology he invented, 45 years and counting.

The dysfunctionality of the scholarly literature: hyperlinks

Scientists as social parasites?

Time and time again, academic publishers have managed to create the impression that publishing incurs a lot of costs which justify the outrageous prices they charge, be that US$11M p.a. for an

 Elsevier

Big Deal subscription or an article processing charge (APC) of US$5,200 for a

 Nature Communications

article.

How publishers keep fooling academics

I have now posted the job ads for the PostDoc position in our lab at

 Nature

,

 Science

, SfN and FENS.

PostDoc opportunity in our lab now advertized

As a regular user of the scholarly literature since before the internet (I started reading primary scientific literature for a high-school project around 1989), I have closely followed its digitization.

7 functionalities the scholarly literature should have

See this post with the associated press releases on brembs.net.

The

 F

orkhead B

 ox

P2 (FOXP2) gene is well-known for its involvement in language disorders. We have discovered that a relative of this gene in fruit flies,

 dFoxP

, is necessary for a type of learning called operant self-learning, which resembles some aspects of language learning.

The Drosophila FoxP gene is necessary for operant self-learning

Over the last ten years, scientific funding agencies across the globe have implemented policies which force their grant recipients to behave in a compliant way. For instance, the NIH OA policy mandates that research articles describing research they funded must be available via PubMedCentral within 12 months of publication. Other funders and also some institutions have implemented various policies with similar mandates.

Maybe try another kind of mandate?

The cost of knowledge – in 2003?

It’s the time of the year again where 30,000 neuroscientists head to the US to talk neuroscience. This year the big annual conference is in Washington, DC, one of the three cities able to host this large event. Our first poster will be up on Tuesday morning and shows results from operant self-stimulation experiments using optogenetics and dopaminergic neurons.

Heading for #SfN23 with two posters

There are regular discussions among academics as to who should be the prime mover in infrastructure reform. Some point to the publishers to finally change their business model. Others claim that researchers need to vote with their feet and change how they publish. Again others find that libraries should just stop subscribing to journals and use the saved money for a modern publishing system.

Who’s responsible for the lack of action?

Yesterday, cOAlition S published their updated principles and implementation guidelines for #PlanS, together with the rationale behind the update. This constitutes a very much welcome effort, as evidence of the increasing awareness among funders as to their potential leverage in infrastructure modernization, at a time when institutions have apparently abandoned their faculty completely.

Improved Plan S principles raise hope of more effective policies soon

The recent publication of the “Ten Principles of Plan S” has sparked numerous discussions among which one of several recurring themes was academic freedom. The cause for these discussions is the insistence of the funders supporting Plan S that their grant recipients only publish in certain venues under certain liberal licensing schemes.

Does academic freedom entail exemption from spending rules?

In his fantastic Peters Memorial Lecture on occasion of receiving CNI’s Paul Evan Peters award, Herbert Van de Sompel of Los Alamos National Laboratory described my calls to drop subscriptions as “radical” and “extremist” (starting at about minute 58):  Regardless of what Herbert called my views, this is a must-see presentation in which Herbert essentially presents the technology and standards behind the functionalities I have been asking for

Why academic journals need to go

“an academic career, in which a person is forced to produce scientific writings in great amounts, creates a danger of intellectual superficiality”

Albert Einstein Isaacson W (2008) Einstein (His Life and Universe) (Simon and Schuster, New York), 1st Ed, p 79

On the productivity of scientists

This is the tweetlog covering July 3-5: Interesting! We find something similar in flies: Live fast, die young: Long-lived mice are less active https://feedly.com/k/16RDz21 It smells fishy: Copper prevents fish from avoiding danger https://feedly.com/k/12gw14F @

 biocs

@

 google

Yes!

Tweetlog: neuroscience and #openaccess

In the last “Science Weekly” podcast from the Guardian, the topic was retractions.  At about 20:29 into the episode, Hannah Devlin asked, whether the reason ‘top’ journals retract more articles may be because of increased scrutiny there. The underlying assumption is very reasonable, as many more eyes see each paper in such journals and the motivation to shoot down such high-profile papers might also be higher.

Are more retractions due to more scrutiny?

Until today, I was quite proud of myself for not caving in to SIWOTI syndrome like Mike Taylor did. And then I read his post and caved in as well. What gets us so riled up is Elsevier’s latest in a long list of demonstration of what they think of the intellectual capacities of their customers. It’s precisely because it is only one in a long row that I initially didn’t feel like commenting.

Is this supposed to be the best Elsevier can muster?

For ages I have been planning to collect some of the main aspects I would like to see improved in an upgrade to the disaster we so euphemistically call an academic publishing system. In this post I’ll try to briefly sketch some of the main issues, from several different perspectives.

 As a reader:

I’d like to get a newspaper each morning that tells me about the latest developments, both in terms of general science news (aka.

What should a modern scientific infrastructure look like?

I really loathe reviewing for GlamMagz for two main reasons. For one, it’s hard to remain neutral: publication of a paper in my field in such a journal is beneficial both for the field and for the young people who are authors on this paper. Second, the demands of some of my colleagues so often make my blood boil. At that point I’m very happy these reviews are anonymous and I really don’t want to know the names of these colleagues.

Why this GlamMag will likely not ask for my review again

For our course this year I was planning a standard neurogenetic experiment. I hadn’t ever done this experiment in a course, yet, just two weeks ago I tried it once myself, with an N=1. The students would get two groups of

 Drosophila

fruit fly larvae, rovers and sitters (they wouldn’t know which was which). About ten larvae from each group would be placed on one of two yeast patches on an agar plate.

If only all science were this reproducible

Random Science Video: The value of fly research

Last week, Elizabeth Pennisi asked me to comment on the recent paper from Schreiweis

 et al.

entitled “Humanized FoxP2 accelerates learning by enhancing transitions from declarative to procedural performance”. Since I don’t know how much, if anything, of my answers to her questions will end up in her article, I thought I might expand my answer into a post about this very interesting work.

Humanized FoxP2 and the timing of habits

This is an easy calculation: for each subscription article, we pay on average US$5000. A publicly accessible article in one of SciELO’s 900 journals costs only US$90 on average. Subtracting about 35% in publisher profits, the remaining difference between legacy and SciELO costs amount to US$3160 per article.

Are we paying US$3000 per article just for paywalls?

The data clearly show that publications in

 Cell

,

 Nature

or

 Science

(

 CNS

for short), on average, cannot be distinguished from other publications, be it by methodology, reproducibility or other measures of quality. Even their citation advantage, while statistically significant, is so small that it is practically negligible.

No need to only send your best work to Science Magazine

There is an interesting study out in the journal PNAS: “Evaluating big deal journal bundles”. The study details the disparity in negotiation skills between different US institutions when haggling with publishers about subscription pricing.

Your university is definitely paying too much for journals

The other day I was alerted to an interesting evaluation of international citation data. The author, Curt Rice, mentions a particular aspect of the data:  The context here is that the “bottom” refers to scientific articles that aren’t cited, assuming that no citations mean low scientific quality of that article.

Citation inflation and incompetent scientists

Genetically Separable" href="https://www.slideshare.net/brembs/wcalb14" target="_blank"&gt;Pavlovian and Skinnerian Processes are Genetically Separable

Two evolutionary conserved, fundamental learning mechanisms

Sven Fund, CEO of the German publishing house de Gruyter was recently interviewed by Richard Poynder in his widely read interview series on open access. In the interview, he first avoided answering the question if de Gruyter had ever lobbied against open access.

Is de Gruyter’s lobby-by-proxy not lobbyism?

I just got the nicest decline to review ever:  That’s how you retire from science: you get to work like normal people. Awesome!

Retiring from science?

It is one of these rare events when I can post a video of one of my own talks. This one was in Berlin earlier this year, organized by The Lost Lectures in a very unusual venue, the Stattbad Wedding:  I’m planning to work more closely together with our computing center here to be able to record at least the lectures I give in English here at the university. So maybe there will be some more videos here, soon.

Video: Free will as an evolved brain function

Since the birth of our daughter, we spend our summers in Sweden, just like I used to, when I was little. This not only allows her to get more practice in Swedish then she gets talking with just me or her grandmother, but we also get to really recharge our batteries in the fabulous wilderness up there.

How to recharge in Sweden

A fistful of dollars: why corporate publishers have no place in scholarly communication

Following the example of Glyn Moody, I thought I’d start a log on the tweets I send around. One never knows what’ll happen to Twitter and besides, this provides a neat place to store and find everything. So here are the tweets for July first and second: Where are we, what still needs to be done?

Trying a new feature: Tweetlog

Flashback: All brains possess free will because there is no design in biology

During my flyfishing vacation last year, pretty much nothing was happening on this blog. Now that I’ve migrated the blog to WordPress, I can actually schedule posts to appear when in fact I’m not even at the computer. I’m using this functionality to re-blog a few posts from the archives during the month of august while I’m away.

Science, red in tooth and claw

*During my flyfishing vacation last year, pretty much nothing was happening on this blog. Now that I’ve migrated the blog to WordPress, I can actually schedule posts to appear when in fact I’m not even at the computer. I’m using this functionality to re-blog a few posts from the archives during the month of august while I’m away. This post is from February 9, 2011: *     In this week’s journal club, we talked about an old paper from 1918!

Flashback: Nothing new in science?

During my flyfishing vacation last year, pretty much nothing was happening on this blog. Now that I’ve migrated the blog to WordPress, I can actually schedule posts to appear when in fact I’m not even at the computer. I’m using this functionality to re-blog a few posts from the archives during the month of august while I’m away. This post is from April 19, 2011:

Flashback: Creationists, this is the evidence you have to beat!

*As part of my scheduled re-posts during the summer break, I’ll also post some of the science videos from the archives. I originally posted this one on Agust 31, 2011: *

SummerScienceVideo: science is real

It might just save your life (via Upturned Microscope):    BTW, even if your life is not at stake, someone else’s may be. So you should publish your results if you are sure something definitely will not work, for instance in F1000 Research, where you can publish negative results for free until August 31, 2013. Your colleagues will be grateful.

One more reason to publish negative results

The recently released development draft for SHared Access Research Ecosystem (SHARE), authored by the Association of American Universities (AAU), the Association of Public and Land-grant Universities (APLU) and the Association of Research Libraries (ARL) in response to the OSTP memo on public access to federally funded research in the US sounds a lot like the library-based publishing system I’ve been perpetually arguing for.

SHARE: Library-based publishing becoming a reality?

Academic publishers have been parasitizing the public purse for long enough now. Steffen Böhm, director of the Essex Sustainability Institute, said it best: In his article, he mentions the successful SciELO project in South America. In a lot of ways, SciELO (scielo.org) is quite close to what I have been arguing for: SciELO means “Scientific Electronic

 Library

Online” and is supported by the Brazilian and Chilean government.

Cut out the parasitic middle men!

This year’s Winter Conference on Animal Learning and Behavior (WCALB) will be on one of my oldest and most central research projects, the commonalities and differences between operant and classical conditioning.

Almost 80 years on, progress on operant and classical conditioning

Amazing bead chain experiment

No matter how well-intended (and we all know to which place the road leads that is paved with good intentions!), transformative agreements (such as DEAL in Germany) are generally the wrong tool at the wrong time for making publicly funded science accessible to the public.

No Evilsevier DEAL!

The DFG is a very progressive and modern funding agency. More than two years ago, the main German science funding agency signed the “Declaration on Research Assessment” DORA.

German funder DFG: Why the sudden inconsistency?

tl;dr:
 
 It is a waste to spend more than the equivalent of US$100 in tax funds on a scholarly article.

Collectively, the world’s public purse currently spends the equivalent of US$~10b every year on scholarly journal publishing. Dividing that by the roughly two million articles published annually, you arrive at an average cost per scholarly journal article of about US$5,000.

How much should a scholarly article cost the taxpayer?

Today, our most recent paper got published, before traditional peer-review, at

 F1000 Research

. The research is about how nominally identical fly stocks can behave completely differently even if tested by the same person in the same lab in the same test.

The way academic publishing should be

UPDATE
 
 , 10-02-2015: After a hint from a user on Twitter, I now know that it is possible to open a PDF document in several windows, one for text, one for legends and one for figures. Figures and legends occupy one virtual desktop and the text another. In this way, I can actually review on-screen, but it is one heck of a work-around and by no means convenient.

I cannot take it any more.

Publishers, stop torturing your reviewers!

This is a slightly edited (amended, essentially) version of

my article published today at The Conversation. In cases where a problem within a community is detected and collective action is required to address the problem. one needs to strike a fine line or any efforts to convince the community that action is required will fail.

The Conversation: The looming crisis in science

You may have seen a neutered version of this post over at the LSE blog. This post below, however, puts the tiger in the tank, as it was enhanced by CatGPT:  Maybe scholarly societies have taken “the instruction“follow the money!” a tad too literally?

Scholarly societies: like a cat chasing the laser dot

The recent call for a GlamMag boycott by Nobel laureate Randy Shekman made a lot of headlines, but will likely have no effect whatsoever. For one, the call for boycott isn’t even close in scale to “the cost of knowledge” boycott against Elsevier and even that drew less than 15,000 measly signatures, a drop in the bucket with 970,000 board members, reviewers and authors working for Elsevier largely for free.

In support of subscripton cancellations

Currently, our libraries are paying about US$5000 per peer-reviewed subscription article. What is more difficult to find out is where all that money goes. Which steps are required to make an accepted manuscript public?

What goes into making a scientific manuscript public?

Talk about egg on face!

 Nature

“the world’s best science”

 Magazine

sets out to publish back-to-back papers on – of all topics – stem cell science.

Even the most thorough peer-review at the ‘best’ journals not up to snuff?

Public institutions the world over are required to spend their funds responsibly. Commonly, this is done by requiring them to host bids for purchases or services above a certain threshold. If you work at a public institution and have wondered, e.g., why you are only allowed to buy a computer from your computing facility which only sells one particular brand, then the answer likely is that this brand won the bidding contest.

Should public institutions not be choosing the lowest responsible bidder?

The recently discussed scenario of universal gold open access brought about by simply switching the subscriptions funds at libraries to have the libraries pay for author processing charges instead, seemed like a ghoulish nightmare. One of the few scenarios worse than the desolate state we call the

 status quo

today.

Whither now, Open Access?

Flashback: What can the spine teach us about learning?

Flashback: The neurobiology of operant conditioning

Our lab is looking for a PhD student interested in the molecular mechanisms of operant self-learning, a form of motor learning.

Looking for a PhD student

After Mike Taylor kept complaining about how horrible my old e107 platform was (which obviously it isn’t: I love this CMS), I finally gave in and moved everything over to a new home. The entire brembs.net domain with all its subdomains is no longer hosted in the good old US of A, where its home was since about 2000, when I moved there myself. After almost ten years back in Germany, I’ve now brought my online presence back home as well.

A new home for bjoern.brembs.blog

I recently was sent a report from a university-wide working group on the publishing habits within the Freie Universität Berlin. I don’t think this document is available online, but I think I’m not doing anything illegal if I publish some of the survey results here. The working group polled all faculty members of the university on various questions concerning scholarly publishing.

How important is the Impact Factor?

Think, check, submit: who hasn’t heard of this mantra to help researchers navigate the jungle of commercial publishers? Who isn’t under obligation to publish in certain venues, be it because employers ask for a particular set of journals for hiring, tenure or promotion, or because of funders’ open access mandates?

How academic institutions neglect their duty

The academic journal publishing system sure feels all too often a bit like a sinking boat: we have a reproducibility leak an affordability leak a functionality leak a data leak a code leak an interoperability leak a discoverability leak a peer-review leak a long-term preservation leak a link rot leak an evaluation/assessment leak a data visualization leak … … … and even a tiny access leak still remains even after 30 years of trying to fix it.

Scholarly publishing in three cartoons

bjoern.brembs.blog

**tl;dr:** *Evidence suggests that the prestige signal in our current
journals is noisy, expensive and flags unreliable science. There is a
lack of evidence that the supposed filter function of prestigious
journals is not just a biased random selection of already self-selected
input material. As such, massive improvement along several variables can
be expected from a more modern implementation of the prestige signal.*

Some common responses to the proposal to
[replace](https://doi.org/10.5281/zenodo.5526634) the now more than
35,000 peer-reviewed scholarly journals with more
[modern](https://www.coalition-s.org/blog/creating-a-market-to-replace-publisher-monopolies/)
[solutions](https://researchprofessionalnews.com/rr-news-europe-views-of-europe-2022-1-how-to-reclaim-ownership-of-scholarly-publishing/)
are "Why do you want to get rid of peer review?" or "How should we know
what to read without journals?".

Both, of course, imply insufficient engagement with the proposal and its
consequences. As for the first question regarding peer-review, there is
currently very little evidence as to the cost-effectiveness of
peer-review and most comparisons between un-reviewed 'preprint'
manuscripts and the published articles after peer-review, show [few
substantial
differences](https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3001285).
Given the [huge costs associated with
peer-review](https://researchintegrityjournal.biomedcentral.com/articles/10.1186/s41073-021-00118-2),
one would expect peer-review effectiveness to increase dramatically by
modernizing the way it is organized and deployed. As this conclusion and
expectation is not new, the
[literature](https://www.nature.com/articles/sj.bdj.2016.131) is
[brimming](https://academic.oup.com/ajhp/article/74/24/2080/5102692)
with
[suggestions](https://www.sciencedirect.com/science/article/abs/pii/S1070324116302814)
for [improving peer-review](https://reimaginereview.asapbio.org/), just
[waiting](https://www.tandfonline.com/doi/abs/10.1080/1047840X.2012.692215)
to be
[tested](https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000116).
The proposal to replace journals specifically includes a more effective
way to leverage the power of peer-review.

The second question implies two implicit assumptions, namely that the
prestige inherent in journal rank carries a useful signal and that this
signal is actually being commonly used when making decisions about which
portion of the scholarly literature to read. Let's see if there is
evidence supporting these assumptions.

There is some evidence in the literature
([review](https://www.frontiersin.org/articles/10.3389/fnhum.2013.00291/full)
of a subsection) that there indeed is a signal conveying more 'novel' or
'breakthrough' discoveries in the more prestigious journals. However,
this literature also emphasizes that the signal to noise ratio is very
poor: it contains many false-positives (articles that turn out to be not
particularly novel or ground-breaking) and false-negatives (many novel
and ground-breaking discoveries are first published in lower ranking
journals). Unfortunately, because of the nature of journal publishing,
the full extent to which the false-negatives contribute the noise in the
signal cannot be known, as it is commonly not known which submissions
have been rejected by any given journal (more about that later). Adding
insult to injury, this already noisy signal is then degraded even
further by the reliability of the signal being no higher (in fact,
[slightly
lower](https://www.frontiersin.org/articles/10.3389/fnhum.2018.00037/full))
than in average journals. This means that more of the published articles
in prestigious journals, even those that are novel or signify a
break-through, turn out to be irreproducible or contain more errors than
articles in average journals. Finally, this weak signal is then bought
at a price that exceeds the costs of producing such a scholarly article
by about a [factor of ten](https://f1000research.com/articles/10-20/v2).
Thus, in conclusion, the prestige signal is noisy, the science it flags
is unreliable and the moneys it draws from the scholarly community
exceed the cost of just producing the articles by almost an order of
magnitude.

It may thus be of little surprise that one can find evidence that this
noisy signal that tends to also flag unreliable science does not seem to
be used as commonly by readers as it is sometimes claimed.
Unfortunately, there is no direct evidence as to how readers use the
scholarly literature, in part, again, because our literature is
fragmented into 35,000 different journals. Therefore, currently, we are
forced to restrict ourselves to citations as a proxy for reading: one
can only cite what one has also read. Using the signal of journal rank
(as routinely and reliably measured by impact factor),
[Lozano](https://onlinelibrary.wiley.com/doi/full/10.1002/asi.22731) *et
al*. predicted subsequent citations to published articles. In other
words, the authors tested the assumption that journal rank signals to
readers what they should read and, consequently, cite. When looking at
it historically (figure below), this predictive power of journal rank
starts surprisingly low, but statistically significant, due to the
authors analyzing millions of articles.

::: {.wp-block-image}
<figure class="aligncenter is-resized">
<img
src="https://onlinelibrary.wiley.com/cms/asset/e998d983-dd22-43a8-bc03-099dd26dd42c/asi22731-fig-0001-m.jpg"
decoding="async" height="400" alt="figure" />
</figure>
:::

As the serials crisis starts to hit in the 1960s/70s and libraries are
forced to use the impact factor to cancel subscriptions to low-ranking
journals, this predictive power, not surprisingly, starts to increase,
if ever so slightly: you really can only cite what you can read. The
advent of keyword searches and online access then abolishes this modest
increase and today, the predictive power of journal rank on
citations/reading appears to be as low as it ever was. Given that
citations to articles in prestigious journals are often used by authors
to signal the importance of their topic, even these low numbers are
probably overestimating the true effect of journal rank on reading
decisions. So there seems to be an effect of prestige on
citations/reading, but it is very weak, indicating that either only very
few readers are using it or each reader is not applying it very
stringently. In other words, the low predictive value of journal rank on
future citations can be considered a reflection of users realizing that
the publication venue contains little useful information when making
choices about what to read.

Noticing an apparent disconnect between these citation data and the fact
that journal rank is partly based on citations, Pedro Beltrao [pointed
to](https://www.evocellnet.com/2013/06/doing-away-with-scientific-journals.html)
a different study, where high impact factor journals, on average, tend
to be enriched in papers that will be highly cited in the future,
comparing the slice of 1% most highly cited papers. However, these
results only superficially seem to be in contradiction to the data
presented by Lozano *et al*. In fact, I would argue that the 1% study
actually supports the interpretation that journal rank is only very
rarely used as a signal of what to read or cite: because there are more
"novel" or "break through" papers in these journals (as detailed above),
people will cite these rare 1% papers more often. But that effect hardly
changes the probability of the remaining 99% of papers to be cited.
These papers are 'regular' papers that could have been published
anywhere. This evidence thus suggests that readers decide what they cite
not based on containers, but based on content: the 1% of articles in
prestigious journals that are actually novel and ground-breaking are
cited more, and the other 99% are cited just as much as articles in any
other journal. In fact, this insight is corroborated when comparing
citation distributions between journals.

All of this evidence supports the interpretation that yes, there is a
signal in journal rank, but it mainly comes from a very low percentage
of the papers (that then, in turn, is less reliable than average) and
hardly translates to the vast majority of the papers in prestigious
journals at all: the effect is almost gone when you include all papers.
Therefore, arguably, the little effect that remains and that can be
detected, is likely not due to the selection of the editors or
reviewers, but due to the selection of submission venue by the authors:
they tend to send their most novel and break-through work to the most
prestigious journals. The fact that what we think is our most novel and
groundbreaking work is too often also our worst work in other aspects,
probably explains why the reliability of publications in prestigious
journals is so low: not even the professional editors and their
prestigious reviewers are capable of weeding out the unreliability.

Taken together, despite the best efforts of the professional editors and
best reviewers the planet has to offer, the input material that
prestigious journals have to deal with, appears to be the dominant
factor for any 'novelty' signal in the stream of publications coming
from these journals. Looking at all articles, the effect of all this
expensive editorial and reviewer work amounts to probably not much more
than a slightly biased random selection, dominated largely by the input
and to probably only a very small degree by the filter properties. In
this perspective, editors and reviewers appear helplessly overtaxed,
being tasked with a job that is humanly impossible to perform correctly
in the antiquated way it is organized now.

Unfortunately, given the current implementation of the prestige signal
by antiquated journals, it remains impossible to access the input
material and test the interpretation above. If this interpretation of
the evidence were correct, much would stand to be gained by replacing
the current implementation of the 'prestige signal' with a more modern
way to organize it.

How could a more modern system support a 'prestige signal' that would
actually deserve the moniker? Obviously, if journals were to be replaced
by a modern information infrastructure, only our imagination is the
limit for which filters the scholarly community may want to implement.
Some general ideas may help guide that brainstorming process: If the use
of such 'prestige' not only in career advancement and funding, but also
in the defense of the current system is anything to go by, there should
be massive demand for a prestige signal that was worth its price. Today,
this prestige arises from selectivity based on expertise (Nature's
slogan always was "the world's best science"). This entails an
expert-based filter that selects only very few ('the best') out of of
the roughly 3 million peer-reviewed articles being published each year.
Importantly, there is no *a priori* need to objectively specify and
determine the criteria for this filter in advance. In a scenario after
all journals had been replaced by a modern infrastructure for text, data
and code, such services (maybe multiple services, competing for our
subscriptions?) need only record not just the articles they selected (as
now) but also those they explicitly did not select in addition to the
rest that wasn't even considered. Users (or the services themselves or
both) would then be able to compute track records of such services
according to criteria that are important to them.

Mimicking current implementations, e.g., the number of citations could
be used to determine which service selected the most highly cited
articles, how many it missed and which it falsely didn't even consider.
But why stop at bare citations? A modern infrastructure allows for
plenty of different markers for scholarly quality. One could just as
well use a (already existing) [citation
typology](https://jbiomedsem.biomedcentral.com/articles/10.1186/2041-1480-1-S1-S6)
to differentiate between different types of citations, one could count
replications, media mentions, anything, really, to derive track records
by which these services may be compared. Given the plentiful demand
indicated by the fervent supporters of prestigious journals, services
would compete with each other using their track records for the
subscriptions of individual and institutional users, providing for
innovation at competitive prices, just like any other service market.
Such an efficient, competitive marketplace of services, however, can
ever only arise, if the current monopoly journals are replaced with a
system that allows for such a market to be designed. If demand was not
as high as expected, but such a signal nevertheless desired by some, a
smaller, more basic implementation could be arranged on a non-profit,
open source basis, funded by the vast savings that replacing journals
would entail. One may also opt to hold competitions for such services,
awarding prizes to the service that best serves the needs of the
scholarly community. The possibilities are endless -- but only once the
scholarly community finds a way to put itself into a position where it
has any power over the implementation of its infrastructure.

Perhaps most importantly, such a modern implementation of the prestige
filter offers the promise of actual consequences arising from the
strategies these prestige services employ. Today, publishing articles
that later turn out to be flawed has virtually no tangible consequences
for the already prestigious journals, just as a less prestigious journal
can hardly increase its prestige by publishing stellar work. With power
comes responsibility and our proposal would allow the scholarly
community to hold such services to account, a feat virtually impossible
to reach today.

P.S.: Much like the ideas for replacing journals have been around for
decades, the ideas for replacing prestige signals have been discussed
fro a long time. [There is a post from Peter Suber from
2002](https://dash.harvard.edu/bitstream/handle/1/4727447/suber_thoughts.htm),
and there is [one of my own from
2012](http://blogarchive.brembs.net/comment.php?comment.news.882).

tl;dr:


 Evidence suggests that the prestige signal in our current journals is noisy, expensive and flags unreliable science. There is a lack of evidence that the supposed filter function of prestigious journals is not just a biased random selection of already self-selected input material.

Replacing the prestige signal

In
a [paper](https://www.nature.com/neuro/journal/v17/n2/full/nn.3616.html)
published in *Nature Neuroscience* now over a year ago, the authors
claimed to have found a very surprising feature, which was long thought
to be a bug. In my [blog post covering the hype in the paper and that in
an accompanying news-type
article](https://bjoern.brembs.net/2014/02/hiding-the-shoulders-of-giants/),
I wrote that today, it has become ever rarer that scientists admit to
standing on the shoulders of giants, as we are not rewarded for
referring to giants, but only for being giants ourselves. The blog post
has triggered some email correspondence between a number of colleagues
in or close to this field, at the end of which was the decision that two
of us, [Nicolas
Stergiou](https://coe.unomaha.edu/facstaff/nstergiou.php) and I, would
contact the journal and inform them of the missing references in the two
articles.

The first reply was that we ought to write a 'letter to the editor'
instead of informally notifying the journal that there were some crucial
references missing. So we sat down and wrote our letter (submitted
February 10, 2015):

> Dear Sir,
>
> HHMI Investigator [Michael Eisen recently
> wrote](https://www.michaeleisen.org/blog/?p=1619) that we ought not
> only to come down hard on people who cheat but also on the far great
> number of people who overhype their results. He suggests that hyping
> should be punished just as fraud. This letter to the editor represents
> such an effort as we would like to report on such a case of hyping.
>
> "Standing on the shoulders of giants" is what scientists say to
> acknowledge the work they are building on. It is a statement of
> humility and mostly accompanied by citations to the primary literature
> preceding the current work. In today's competitive scientific
> enterprise, however, such humility appears completely misplaced.
> Instead, what many assume to be required in the struggle to survive is
> to convince everyone that they are the giant, the genius, the prodigy
> who is deserving of the research funds, the next position, tenure. The
> *Nature Neuroscience* article "Temporal structure of motor variability
> is dynamically regulated and predicts motor learning ability" by Wu
> *et al*. (doi:[10.1038/nn.3616](https://dx.doi.org/10.1038/nn.3616))
> with its accompanying news-type article "Motor variability is not
> noise, but grist for the learning mill" by Herzfeld and Shadmehr
> (doi:[10.1038/nn.3633](https://dx.doi.org/10.1038/nn.3633){target="null"})
> from earlier this year clearly fall within this category. Both
> articles claim that the researchers have made the game-changing
> discovery that something long thought to be a bug in our movement
> system is actually a spectacular feature. It is argued that this
> discovery is such a huge surprise, because nobody in their right mind
> would have ever thought this "unwanted characteristic" to actually
> serve some purpose.
>
> The problem with this line of argument is that probably most people in
> the field thought it should be obvious, even to be expected -- and not
> surprising at all. Skinner is largely credited with the analogy of
> operant conditioning and evolution. This analogy entails that reward
> and punishment act on behaviors like selection is acting on mutations
> in evolution: an animal behaves variably and encounters a reward after
> it initiated a particular action. This reward will make the action now
> more likely to occur in the future, just as selection will make
> certain alleles more frequent in a population. Already in 1981,
> Skinner called this "Selection by Consequences" (*Science* Vol. 213
> no. 4507 pp. 501-504, DOI:
> [10.1126/science.7244649](https://dx.doi.org/10.1126/science.7244649)).
> Skinner's analogy sparked wide interest, e.g. an entire journal issue
> (Behavioral and Brain Sciences 7(04), 1984), which later appeared in
> book form (The Selection of Behavior: The Operant Behaviorism of B. F.
> Skinner: Comments and Consequences. A. Charles Catania, Stevan R.
> Harnad, Cambridge University Press). Clearly, the idea that
> reinforcement selects from a variation of different behaviors is not a
> novel concept at all, but more than three decades old and rather
> prominent. This analogy cannot have escaped anybody working on any
> kind of operant/motor learning, except those seriously neglecting the
> most relevant literature. This interaction of variability and
> selection is a well-known and not overly complicated concept, based in
> evolutionary biology and psychology/neuroscience. Consequently,
> numerous laboratories have been studying various aspects of this
> interaction for a long time. Skinner's projection was that increased
> behavioral variability leads to increased operant learning rates, just
> like increased mutations rates lead to increased rates of evolutionary
> change. More than a dozen years ago, Allen Neuringer showed this to be
> the case in rats (Psychonomic Bulletin & Review 2002, 9 (2), 250-258,
> doi: [10.3758/BF03196279](https://dx.doi.org/10.3758/BF03196279)), but
> there are studies in humans as well (Shea, J. B., & Morgan, R. B.
> (1979). Contextual interference effects on the acquisition, retention,
> and transfer of a motor skill. Journal of Experimental Psychology:
> Human Learning and Memory, 5, 179--187). That such variability is
> beneficial, rather than detrimental has been shown even in situations
> where the variability is so high, that the acquisition rate is
> reduced, but post-training performance is enhanced (Schmidt RA, Bjork
> RA (1992): New conceptualizations of practice: Common Principles in
> Three Paradigms Suggest New Concepts for training. Psychological
> Science, 3(4): 207-217).
>
> Wu *et al*. confirm both Skinner's conjecture as well as previously
> published reports (some cited above) that indeed the rate of learning
> in operant conditioning is increased in subjects where the initial
> variability in the behavior is higher. However, instead of citing the
> wealth of earlier work, Wu *et al.* claim that their results were
> surprising: "Surprisingly, we found that higher levels of
> task-relevant motor variability predicted faster learning". Herzfeld
> and Shadmehr were similarly stunned: "These results provide intriguing
> evidence that some of the motor variability commonly attributed to
> unwanted noise is in fact exploration in motor command space."
>
> We regard it as highly unlikely that none of the seven authors in
> total should have never heard of Skinner or the work over the last
> four decades by many human movement scientists that have explored the
> temporal structure of human movement variability and its relationship
> with motor learning. The work by senior scientists such as Karl
> Newell, Michael Turvey, Richard Schmidt, and their students published
> in books and hundreds of journal articles is completely ignored, just
> as the work by several younger mid-career scientists such as Nick
> Stergiou, Jeff Hausdorff, Thurmon Lockhart, Didier Dilignieres, and
> many others. After a thorough review of this literature the authors
> may realize that their results are neither new nor novel. If indeed
> the authors were unaware of this entire section of literature so
> relevant to their own research, it would be an indictment in its own
> right.
>
> Hence, in formal corrections of both articles, we would expect all
> mentions throughout both texts of how surprising these findings were,
> to be replaced with references including, but not limited to the works
> cited above.
>
> Yours sincerely,

In hindsight, we probably ought to have mentioned that the research of
course has merit and that there are a lot of valuable and exciting
results in the paper, but that it is one specific aspect that is really
not surprising at all. However, we wanted to make it brief and focused
just on the hype. Maybe that was a mistake. Be that as it may, after
about two months of peer review, our letter was rejected. Perhaps not so
surprisingly, the peer reviewers apparently were the same ones that
reviewed the original manuscript (at least one of them):

> <div>
>
> This is the basis on which I recommended publication, and I do not
> feel it needs a corrigendum.
>
> </div>

<div>

After all, the [data
indicate](https://journal.frontiersin.org/article/10.3389/fnhum.2013.00291/full)
that such high-ranking journals do not use very rigorous peer-review,
leading to the least reliable science being published there. What was
quite surprising, though, was the implicit support and even endorsement
of the hype:

</div>

> <div>
>
> I agree that the article's tone is a little more breathless than
> strictly required, but this is the style presently in vogue
>
> </div>

<div>

and

</div>

> <div>
>
> The letter complains about "over-hyping" about certain claims made in
> the paper related to previous work. I have some sympathy with the
> letter writers on this front. However, I also have some sympathy for
> the authors, who understandably were trying to emphasize the novelty
> of their work.
>
> </div>

<div>

It thus appears as if at least two of the three reviewers agreed with
us, that the articles were hyping the results, but they don't see
anything wrong with hype. This means that hype is not just a problem
of [journalists](https://www.bmj.com/content/349/bmj.g7015) any more.
Hype is not just a problem of journals
and [GlamMagz](https://www.nature.com/ng/journal/v35/n1/full/ng0903-1.html)
any more. Hype has now arrived in the middle of the scientific community
with explicit endorsement from reviewers. It looks like [Michael Eisen's
call](https://www.michaeleisen.org/blog/?p=1619) for hype-punishment
will go largely unheeded.

</div>

<div>

</div>

<div>

[*You can download the complete reviews of all reviewers [here
(PDF)](https://bjoern.brembs.net/wp/?wpdmdl=817), to make sure none of
the quotes above were taken out of context.*]{style="font-size: 10pt;"}

</div>

In a paper published in

 Nature Neuroscience

now over a year ago, the authors claimed to have found a very surprising feature, which was long thought to be a bug.

Nature reviewers endorse hype

I really loathe reviewing for GlamMagz for two main reasons. For one,
it's hard to remain neutral: publication of a paper in my field in such
a journal is beneficial both for the field and for the young people who
are authors on this paper. Second, the demands of some of my colleagues
so often make my blood boil. At that point I'm very happy these reviews
are anonymous and I really don't want to know the names of these
colleagues. Here are some of the things the reviewers wrote in this most
recent round:

> "the authors have not convinced me of the conceptual novelty of their
> findings to warrant publication in a very top-tier journal"
>
> "appears standard for a top-tier journal"
>
> "However, currently, this manuscript is more suited for a specialized
> journal"

This seems to suggest to me that these colleagues apply different
standards for different journals. So when the journal sent each reviewer
the reviews of the other reviewers and asked for comments, one of my
comments was the following:

> The intuitive notion of journal rank is a figment of our imagination,
> devoid of any empirical support. So-called "top-tier journals" exist
> in the same reality in which homeopathy cures, dowsing rods find water
> and astrologers correctly predict the future. Asking for additional
> 'interesting' or 'curious to know' experiments merely because of the
> venue the authors chose borders on the unethical, IMHO, as there are
> millions of 'curious to know' experiments to be potentially carried
> out. Reviewers ought to try and avoid multiple standards in their
> reviews and restrain additional experiments to the minimum required by
> the statements of the authors. Professional editors are being paid
> precisely because they can predict whether their readers will find the
> statements of the authors 'interesting' or 'curious to know'.

(Where 'curious to know' was a phrase one of the reviewers used whenever
they wanted some tangentially related experiment to be done)

I doubt the editors at GlamMagz are very interested in these sorts of
comments. If they are, I now have a boilerplate from which to draw
future comments.


[[![ResearchBlogging.org](https://www.researchblogging.org/public/citation_icons/rb2_mid.png){decoding="async"
style="border: 0;"}](https://www.researchblogging.org)]{style="float: left; padding: 5px;"}[The
data clearly
show](https://journal.frontiersin.org/Journal/10.3389/fnhum.2013.00291/full)
that publications in *Cell*, *Nature* or *Science* (*CNS* for short), on
average, cannot be distinguished from other publications, be it by
methodology, reproducibility or other measures of quality. Even their
citation advantage, while statistically significant, is so small that it
is practically negligible. Regardless of all the data, individual
examples sometimes serve to illuminate the data and drive some facts
home. There are more or less extreme examples of *CNS* publications not
meeting the expectations to which they are commonly held. It's needless
to mention all the examples that went through the media, but I would
instead [cite a paper from my
field](http://blogarchive.brembs.net/news.php?item.519.3) (where I feel
reasonably competent) as an instance of an extreme case of an obviously
flawed paper making in into the journal *Nature*.

However, one doesn't even need to look at the extreme cases. Let's look
at a [paper in *Science
Magazine*](https://www.sciencemag.org/content/344/6186/901.full) I
actually quite like. It's an interesting topic, a beautiful experimental
paradigm and the results in the wildtype flies are truly exciting. Their
mutant and transgenic results, if correct, are tantalizing as they run
counter to essentially all available literature on the gene in question.
I personally know and like the senior author of the paper, Gero
Miesenböck, and appreciate all of his work, including this publication.
All I'll try to do here is to make the case, that this paper is an
average paper in terms of the level of evidence provided to support the
claims made by the authors. Not obviously worse than most papers in our
field, but definitely nowhere near the level of evidence those
colleagues ignorant of the data would assume to be required for a *CNS*
paper. All papers have strengths and weaknesses, some have more
uncertainty associated with their results, others less. The data suggest
that there isn't much difference between journals, on average, and I'll
try to use some details in this *Science* paper to exemplify the data.

The authors state in their title that the gene *dFoxP* influences the
speed of perceptual decision-making in *Drosophila* fruit flies. They
test the flies by first conditioning them to avoid a given odor using
electric shocks. The choice test is then done in a chamber on individual
flies. Two odors enter the chamber from opposite ends and exit the
chamber in the middle. The fly then walks back and forth between the
ends of the chamber and avoidance is measured by where the fly spends
most of the time: in the half with the odor associated with electric
shock, or in the half with the control odor. In the current study, the
authors made the control odor increasingly similar to the one associated
with shock and found that wildtype flies spent more time in the middle
of the chamber, where they had to decide of whether to keep walking or
make a turn, the more similar the odors were. That is definitely an
exciting finding, suggesting that more similar odors require more
processing time for an avoidance decision than less similar odors.

In order to get a handle on the biological processes underlying this
interesting phenomenon, the authors have conducted a screen of
genetically modified flies, looking for mutants with longer decision
times than the wildtype flies. One of the candidates that showed up in
the screen, is the *Drosophila* orthologue of a gene involved in
language in humans, *FoxP*. It seems hard to understand what a gene
generally thought to be involved in motor learning has to do with this
perceptual task, but data are data, unless there are alternative
explanations. There are three main concerns that raise the suspicion
that the involvement of *dFoxP* in this task may not be as
straightforward as suggested in the title of this paper.

1.  It s common practice to homogenize the genetic background of mutants
    with that of the control strain to which the mutants are compared.
    This is done to exclude variations other than in the gene of
    interest to be responsible for any differences. However, the authors
    do not state whether and if so for how many generations the mutant
    flies were outcrossed to the proper wildtype genetic background.
    Given that the flies came from a screen, it is unlikely that all
    strains tested in the screen were outcrossed. For instance, the
    alleles described in the paper show significant lethality which
    disappears when outcrossed for six generations, suggesting that
    their general level of fitness and health appears to be affected by
    their genetic background. It is thus possible that factors in the
    genetic background and not *dFoxP* may be responsible for the
    phenotype. Given the track record of the Miesenböck lab, it is
    unlikely they did not outcross the flies, but at the very least, the
    number of generations for which the outcrossing has taken place
    would have to be listed in the paper.

2.  A common procedure to test for genetic background effects, in
    addition to outcrossing which can reduce but not eliminate genetic
    background effects, is to use an additional method to manipulate the
    gene in question. The authors used RNA interference (RNAi) to
    attempt to down-regulate the expression of *dFoxP*. It is common
    practice to use quantitative PCR to test the effectiveness of this
    method, as a) RNAi usually does not abolish gene expression
    completely and b) it is known to have off-target effects, i.e., it
    may affect the expression of other genes besides/instead of the
    target gene. The line used in this study comes from a collection
    known to have problems with off-target effects, so testing
    effectiveness is especially important. However, no evidence that
    such a test was performed is described. Below is what such data
    would look like, if you were to order all RNAi constructs currently
    available, drive their expression in all neurons (with nSyb-GAL4)
    and then have an undergraduate student run a whole slew of qPCR
    reactions to compare the expression (in fly heads) of all three
    *dFoxP* isoforms to the respective control strain (many thanks Joel
    and postdoc [Axel](https://lab.brembs.net/author/axel/) for all that
    work!): No effect!

    ::: {#attachment_596 .wp-caption .alignnone style="width: 530px"}
    [![relative expression of dFoxP isoforms in six different RNAi
    lines](https://bjoern.brembs.net/wp/wp-content/uploads/2014/06/foxp_qpcr-1024x429.jpg){.wp-image-596
    .size-large decoding="async"
    aria-describedby="caption-attachment-596" width="520" height="217"
    srcset="https://bjoern.brembs.net/wp/wp-content/uploads/2014/06/foxp_qpcr-1024x429.jpg 1024w, https://bjoern.brembs.net/wp/wp-content/uploads/2014/06/foxp_qpcr-300x125.jpg 300w, https://bjoern.brembs.net/wp/wp-content/uploads/2014/06/foxp_qpcr.jpg 1808w"
    sizes="(max-width: 520px) 100vw, 520px"}](https://bjoern.brembs.net/wp/wp-content/uploads/2014/06/foxp_qpcr.jpg)
    relative expression of dFoxP isoforms in six different RNAi lines
    :::

    I cannot emphasize enough that the graph above represents [very
    preliminary data]{style="text-decoration: underline;"}. It's data
    from one undergraduate student who had prior experience with qPCR
    before coming to our lab, but who did not do this work full-time.
    There definitely need to be more biological replicates and some
    additional drivers need to be used (e.g. an actin driver to drive
    RNAi in all cells). On the other hand, the results match my own qPCR
    results with these lines and results from another laboratory that we
    collaborate with. So while I would not exclude that we may see
    significant reduction in *dFoxP* expression with some technical
    tweak, the currently available data suggest that none of the
    available dFoxP RNAi lines significantly knocks down *dFoxP*
    expression, including the one tested by DasGupta *et al*. (15732V).
    Our work also points to a reason why none of the lines seems to have
    a detrimental effect on *dFoxP* expression: we have localized
    polymorphisms in the *dFoxP* gene which could bias the RNAi process
    towards sequestration, rather than degradation of the mRNA,
    potentially explaining some of the very high values in the plot
    above. Thus, it is possible (and we consider it even very likely)
    that some or all of these lines actually do affect *dFoxP*
    expression in the way intended. However, we currently have little
    possibility to ensure it's not an off-target effect after all.  
    I'd thus tentatively conclude that the phenotype DasGupta *et al.*
    have discovered can indeed be ascribed to *dFoxP* action in this
    task. However, at this point, there is insufficient data to make
    that statement with any reasonable certainty (no matter how likely I
    personally find that to be the case!).

3.  Finally, previous work on *FoxP* both in other animals and in flies
    (including our own paper coming out tomorrow, watch this space after
    5pm EST on June 25) suggests an involvement of *dFoxP* in motor
    learning and perhaps also (consequently?) motor coordination. In
    fact, [our](http://blogarchive.brembs.net/comment-n632.html)
    [work](http://blogarchive.brembs.net/comment-n678.html) shows that
    learning about external stimuli is fine in *FoxP* mutant flies. It
    is thus critical to make sure that any animals showing a deficit in
    this task can perform the required movements (walking, turning,
    starting, stopping) accurately. The authors here attempted to ensure
    this by measuring the time spent in sections of the chamber other
    than the decision section in the middle. However, one would assume
    that flies will mostly turn or sit at the ends of the chamber and in
    the decision sector, with fewer turns and pauses in the sectors
    in-between. Thus, by evaluating sectors in which the flies mainly
    walk straight, the authors may be underestimating the contribution
    of turns and no-movement episodes to the decision-making process. If
    that is correct (and the brief explanations in the methods section
    make it very difficult to be entirely sure about how precisely and
    on which sectors they have performed the calculations),
    *dFoxP*-manipulated flies may either have a problem starting to walk
    from a stop, or have difficulty turning in the chamber and the
    authors would interpret this as an increased decision-time, rather
    than a motor problem.

In summary, it may well be that *dFoxP* entails all the functions
reported by DasGupta *et al*. and this would indeed make for a giant
head-scratcher about the (ancestral) functions of *FoxP*, given all the
mountain of work involving it in motor, rather than perceptual
processes. However, at this point, some more work (at least some more
detailed method explanations!) is required to be reasonably certain
about the existence of such a problem. Personally, I do think that the
phenotype is due to the authors' *FoxP* manipulations, but I may be
wrong, the evidence is not strong enough either way. I do have my
doubts, however, about the phenotype not being due to motor defects, but
I could be wrong there as well, as the data is inconclusive. At this
point, we need more experiments and/or data analysis to show who is
wrong. Again, I still like the paper, it's the kind of work that drives
our field forward.

Speaking more generally: there is no need for any researcher to wait
(perhaps indefinitely?) until they have the perfect data set, with the
unambiguous results and the foolproof conclusions. Just send your plain
decent work to *CNS* magazines as well! If they publish it, you will get
a job and nobody but some lone blogger will ever ask about the content
of that paper ever again.

------------------------------------------------------------------------

[DasGupta, S., Ferreira, C., & Miesenbock, G. (2014). FoxP influences
the speed and accuracy of a perceptual decision in Drosophila [Science,
344]{style="font-style: italic;"} (6186), 901-904 DOI:
[10.1126/science.1252114](https://dx.doi.org/10.1126/science.1252114){rev="review"}]{.Z3988
title="ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.jtitle=Science&rft_id=info%3Adoi%2F10.1126%2Fscience.1252114&rfr_id=info%3Asid%2Fresearchblogging.org&rft.atitle=FoxP+influences+the+speed+and+accuracy+of+a+perceptual+decision+in+Drosophila&rft.issn=0036-8075&rft.date=2014&rft.volume=344&rft.issue=6186&rft.spage=901&rft.epage=904&rft.artnum=http%3A%2F%2Fwww.sciencemag.org%2Fcgi%2Fdoi%2F10.1126%2Fscience.1252114&rft.au=DasGupta%2C+S.&rft.au=Ferreira%2C+C.&rft.au=Miesenbock%2C+G.&rfe_dat=bpr3.included=1;bpr3.tags=Anthropology%2CBiology%2CPsychology%2CNeuroscience%2CBehavioral+Biology%2C+Ecology%2C+Evolutionary+Biology%2C+Genetics%2C+Zoology%2C+Biological+Anthropology%2C+Evolutionary+Anthropology%2C+Behavioral+Neuroscience%2C+Cognitive+Neuroscience%2C+Neurolinguistics%2C+Comparative+Psycholog"}
