open science, open access, meta-science, perception, neuroscience, ...

Alex O. Holcombe

Eleven years of student replication projects provide evidence on the correlates of replicability in psychology

Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study

Many Labs 3: Evaluating participant pool quality across the academic semester via replication

An open investigation of the reproducibility of cancer biology research

Reducing waste from incomplete or unusable reports of biomedical research

Repeatability of published microarray gene expression analyses

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

Investigating Variation in Replicability

To evaluate and build on previous findings, a researcher sometimes needs to know exactly what was done before.

 
 Computational
 



reproducibility is the ability to take the raw data from a study and re-analyze it to reproduce the final results, including the statistics.

An executive summary of science’s replication crisis

Blogged about this web app we created to help researchers document who did what in their journal articles, posted over at Medium for a change.

Tenzing, a web app for contributorship

A brief intro for research students. Good science is cumulative  Scientists seek to create new knowledge, often by conducting an experiment or other research study. But science is much more than doing studies and analyzing the data. Critical to the scientific enterprise is communication of what was done, and what was found.

Introduction to reproducibility

Topics this year: Visual letter processing; role of attention shifts versus buffering (mostly @cludowici, @bradpwyble); reproducibility (@sharoz); visual working memory (mostly @will_ngiam)

 Symposium contribution, 12pm Friday 17 May:

Just a list of our VSS presentations for 2019

Salvaged from an early Feb 2016 Google+ posting: A vision researcher discussion happened on a semi-private email group (CVnet), but you can see some discussion on the visionlist archive (by moving around here: http://visionscience.com/pipermail/visionlist/2016/009312.html ), and below you can see the results of the survey.

survey of vision researchers: 2016 results on open access

A post from 27 Feb 2016 salvaged from Google+:  A discussion about the high author fees charged by some #openaccess journals brought up many other issues, some of which were included in a survey. Nearly 400 vision researchers responded to the survey, of which 93% expressed desire for change.

Psychonomics Society and Perception/i-Perception on open access

An old post salvaged from the dying Google+:  Is red special? There’s a big literature on this, but I haven’t seen any studies investigate this with quality methodology. A number of studies purport to investigate whether the color red is special, like whether “red test materials lower performance”. Two problems: 1) these studies may be impossible to replicate because the authors tend not to report the colors in a device-independent fashion!

Is red special? There’s a big literature on this, but

An old post salvaged from the dying Google+  I used to think diagrams like the above were probably accurate in showing dichromats, like dogs, having a gradient between only two qualitatively distinct colors at most (of course, what those particular colors are qualitatively is unknowable; surely they don’t feel like our blue and yellow). But from Wachtler, Dohrmann, &amp;

Do dogs have color categories?

Several European funders announced planS, which suggests that several European nations will ban grantees form publishing in paywalled journals (including hybrid journals that allow one to make an article open access for a fee), and that includes many society journals that are owned by societies but published by large subscription-based publishers. This is to begin in 2020.

Planning for PlanS

The below, a letter to my department, is a brief primer for researchers on pre/postprint posting, with some comments on the evolving scholarly publication landscape. Dear colleagues, In last week’s School Research Committee meeting, the topic of publishing open access came up, and our Head of School asked me to send you an email about one solution in particular.

Posting manuscripts online

Perhaps the rumors of Anne Treisman’s passing are greatly exaggerated. I hope they are [UPDATE: I’ve gotten confirmation they sadly aren’t]. Regardless, in this era of lists of most-influential psychologists that do not include her, it is a good time to reflect on her influence.

Anne Treisman and feature integration theory

Was that a shift of attention or binding in a buffer?

Charles J. H. Ludowici; Alex O. Holcombe (presented by Alex) 3:50-4:05 PM Friday 10 November, West Meeting Room 118-120   Cueing a stimulus can boost the rate of success reporting it. This is usually thought to reflect a time-consuming attention shift to the stimulus location.

Psychonomics 2017, our presentations

Hoisted from the comments on Scott Alexander’s ethics/IRB nightmare, an insight I hadn’t seen before:    And, to bring home again the absurdity:

Ethics and IRB burden

Mark Wilson and I wrote the below piece for the Australian Open Access Support Group. The principles we lay out guide our vision for working to create an open access future governed by the community of scholars, not publishers. In March 2017 a group of researchers and librarians interested in journal reform formalized the Fair Open Access Principles.

The Fair Open Access principles

Akihiro Munemasa, Christos Athanasiadis, Hugh Thomas, and Hendrik van Maldeghem share the chief editor role at a journal that’s like many others across mathematics and the sciences. The

 Journal of Algebraic Combinatorics

is a subscription journal published by one of the big, highly-profitable publishers (Springer Nature). But they haven’t been happy with the fees Springer charges for people to read their articles.

A major math journal flips to Fair Open Access

Elsevier and other publishers’ ability to detect “self-plagiarism” is an instance of text mining the world’s scientific literature. Over at two vision researcher mailing lists, there is much irritation at being asked to remove sentences that duplicate sentences that one wrote in previous papers, to describe for example the methodology of a study.

Publishers prioritize “self-plagiarism” policing over allowing new discoveries

The below is what we’ll be presenting at EPC 2017 (the Experimental Psychology Conference of Australasia) near Newcastle, Australia. The topics are attention and letter processing, word processing, and visual working memory.

 When do cues work by summoning attention to a target and when do they work by binding to it?

Alex Holcombe &amp;

Our latest work: on attention, letter processing, memory

In science, we frequently see a comparison between two groups of people that differ on multiple demographic variables, say age, IQ, and income, investigating some dependent measures, say body mass index (BMI). Results are often reported as “the groups had substantially different BMIs, after controlling for X and Y” using ANCOVA or multiple regression.

problems with “controlling for” variables: quick notes

It’s the end of the year, and I’m indulging myself by posting these Bayesian jokes. The first three were inspired by the #AcademicNoir hashtag.   Round up the usual suspects, sergeant. But sir, we haven’t a clue! Indeed we don’t. But we do have a prior. #

 Bayes

#

 noir

Bayesian jokes

The headline screams “You’re 45% more likely to be murdered in de Blasio’s Manhattan”.  The evidence? Sixteen people have been killed so far this year in Manhattan, against only eleven over the same period last year. Does this evidence indicate you are more likely to be murdered, as the headline says? To find out, I tested whether a constant murder rate could explain the results.

Yellow journalism and Manhattan murders

A study suggesting that “Sadness impairs color perception” reporting two experiments was recently retracted from

 Psychological Science.

But some colleagues and I don’t think the retraction goes far enough.

When retraction is not enough

Vision researchers recently received an email from ARVO, the publisher of

 Journal of Vision,

that begins: But in the view of most,

 JoV

has been open access since its inception! It’s always been an author-pays, free access journal: all articles are published on its website and can be downloaded by anyone.

What just happened with open access at the Journal of Vision?

Elsevier has a history of unethical behaviour: Elsevier created fake medical journals to promote Merck products. Elsevier sponsored arms fairs for the international sale of weapons. Elsevier sponsored a bill that would have eliminated the NIH mandate that medical research be make freely available within 12 months of publication.

Four Reasons to Oppose the Use of Elsevier’s Services for the Medical Journal of Australia

Evaluating scientists is hard work. Assessing quality requires digging deep into a researcher’s papers, scrutinising methodological details and the numbers behind the narrative.

A “tell” for researcher innumeracy?

My collaborators and I have started using a new behavioural technique to better understand attentional selection from a rapid stream of stimuli.

In our basic task, one or two streams of stimuli are rapidly presented. The target(s) to be reported are highlighted with cues that encircle them. On half of trials, participants are first queried about the left target, and in half they are first queried about the right target. This has no significant effect on the main result- a substantial disadvantage in reporting the right target, if the left target must also be reported.

Reporting items from a stream, and mixture modeling to reveal buffering and a bottleneck

Below is the text of a recent open letter I signed, spearheaded by Jon Tennant. Here he explains why he spearheaded the below letter to the AAAS requesting they make changes to their policy regarding their new open access journal,

 Science Advances.

Dear  AAAS, This is an open letter concerning the recent launch of the new open access journal, Science Advances.

open letter to the AAAS regarding their new open-access journal

Nature Scientific Reports has adopted is piloting fast-tracking for a fee. Four years ago, I noticed that several journals had adopted such a policy. I raised a number of concerns, such as What happens if the fast-tracking period elapses and a reviewer hasn’t gotten their review in yet? Will the decision about the manuscript be made without that review? How is the additional money used? Does any go to reviewers?

Nature Scientific Reports. Fast-tracking fees history and concerns.

The below letter was spearheaded by Erin McKiernan and requests that the Society for Neuroscience bring its new open-access policy in line with emerging best practices for sharing of data and ability to re-use content.

open letter to Society for Neuroscience regarding their new open-access journal

Science is broken; let’s fix it.

This has been my mantra for some years now, and today we are launching an initiative aimed squarely at one of science’s biggest problems. The problem is called publication bias or the file-drawer problem and it’s resulted in what some have called a replicability crisis.

Registered Replication Reports are open for submissions!

I recently published the below chart to document the outrageous profit margins of scholarly publishers in the sciences. This post is to provide the sources for the numbers in the chart.

Scholarly publishers and their high profits

Scientists of all sorts increasingly recognize the existence of systemic problems in science, and that as a consequence of these problems we cannot trust the results we read in journal articles. One of the biggest problems is the file-drawer problem. Indeed, it is mostly as a consequence of the file-drawer problem that in many areas most published findings are false. Consider cancer preclinical bench research, just as an example.

Protect yourself during the replicability crisis of science

Below are research presentations I’m involved in for Vision Sciences Society in May. If you’re attending VSS, don’t forget about the Publishing, Open Access, and Open Science satellite which will be Friday at 11am.

VSS 2012 abstracts, and Open satellite

How many of these laws do

 you

know? The top 15, listed below, are based on psychology journal articles 1900-1999, as calculated by Teigen (2002): LAW  (REFERENCE)

Top 15 most popular laws in psychology journal abstracts

Most academics agree that most scientific articles should be freely available, but we’re stuck in a system where scientific articles still tend to be submitted to journals that one needs a subscription to read. One way we researchers perpetuate this system is by donating our labor to provide “peer review” of manuscripts that will require others to pay hefty subscription fees to read.

Opening access with peer reviewing pledges

Academic publishing is stuck in an outmoded system. Most scientific research is paid for by government and non-profit university funds, but high-profit corporate publishers often control access to the results of the research. In this video, we showcased the absurdity of the situation and also pointed towards how to get ourselves un-stuck.  There are significant costs associated with what journal publishers do, so we need publishers in some form.

Scientist meets Publisher- Explaining the video

We (Hal Pashler, UCSD, hpashler at gmail and Alex Holcombe) are developing web-based software to help people interested in a scientific issue represent and inspect multiple competing hypotheses and the evidence which supports or fails to support each hypothesis.

Job: Part-time evidence-charting in Southern California

My talk for the Sydney University psychology department will be at 4pm in the Education building, room 424. Below, the abstract:  Localizing a single object relative to oneself is fairly easy—ever seen a plant reaching towards the sun? It’s a no-brainer. A less trivial task is determining the position of two objects relative to each other.

“Visual Attention On the Go” seminar on Friday

Our Evidence Charting project has received a commendation as a a Undergraduate Learning, Teaching, and Assessment Resource Commendation.

Evidence Charts receives commendation

We discovered that when an array of colored discs was spun so fast that attention could no longer keep up with it, people could no longer perceive which colors were adjacent.   Together with an additional attentional cueing experiment, this phenomenon suggests that a shift of attention is required to mentally link adjacent elements and apprehend their spatial relationship.  The experiments are described in: Holcombe, A., Linares, D., &amp;

Above attention’s speed limit, blindness for spatial relationships

How can we reduce the ability of publishers to charge exorbitant fees to read articles we give them for free? This situation drains university budgets and prevents public access to scientific information. One of many things we can do is referee manuscripts only for open-access journals. Another is to only publish in open-access journals. I’m trying to organize some kind of group pledge to make a stand on one of these or both.

Pioneering open-access pledges

A few months ago when the 2012 Australian Research Council Discovery Project funding rules came out, I was excited to see them putting money into open access publication:  But you always have to read the fine print. When preparing my grant application, I discovered that one is not allowed to include publication costs as an item in the budget.

Australian Research Council moving backward on open-access, too

Journal Fee Fast-track Service Journal of Internet Medical Research 450 USD “initial decision within 15 working days, publication within 1 month after acceptance” Journal of Digital Content Technology and its Applications (JDCTA), and associated journals* 2000 USD “to review and publish your paper within 6 ~ 8 weeks” Mathematical and Computational Forestry &amp;

Journals that fast-track for a fee

Several academic journals offer fast-tracking for a fee, and I’ve been complaining about it, mainly because it’s hard to see how this policy could be implemented without sometimes giving monied authors an advantage not only in time until publication, but also in likelihood of acceptance. Traditionally, acceptance into a journal was not affected by how much the authors pay, and many of us would like to see things stay that way.

Fast-tracking fees vs. open-access fees

One of the things I love about science is that money can’t buy you admission into a journal. You simply got to do respectable science, as judged by other scientists. Getting accepted by a journal isn’t like getting accepted into a country club or a business deal. Although every system has its biases and science is no exception, at least there’s no actual money being handed over to grease the wheels.

Protest of fast-tracking fees. Two journals respond, and one bows out.

I’ve just sent the fast-track protest letter with our original list of 18 authors to the several journals that charge a fee to fast-track. I believe their policy imperils the fairness of the scientific publication system. I’m sure we’ll need more support to convince some of these journals to discontinue their unsavory policy, so please add your name to the signatories here   I’ll soon be linking to the responses we receive from the journals.

fast-track fee protest letter sent,

Blind justice. A beautiful ideal! That the merits of a case are to be decided without regard to the identities of the parties involved or the size of their bank accounts. This is something science aspires to in evaluating manuscripts for publication.

Keeping science fair

At least seven academic journals now offer preferential treatment for a fee—see the table at bottom. The journals describe this as a “fast-track” service that simply speeds things up. Even if that were all it was, this is an unwelcome development, as it means scientists from poorer countries or with poor funding will fall further behind their rich counterparts, whose publications can now leapfrog over the others.

Fast-track fees imperil journals’ reputation for fairness

I recently learned that a journal called

 Obesity Reviews

has a “Fast Track Facility”: This is a terrible development for academia. It creates a two-tier system, wherein scientists who are well-funded such as those from rich countries now have an unfair advantage over those who don’t. Science traditionally has been a partial refuge from the injustice of rich vs. poor.

Widening the gulf between haves and have-nots: Money fast-tracks journal submissions!

Previously, the Australian Research Council (ARC) expressly forbade use of grant funds to pay publication charges. This prevented many of us from publishing in open-access journals, as they generally charge a fee. Fortunately, the newly-revised funding rules change that, and instead strongly encourage open access, via journals and via depositing one’s research in an institutional repository. Hooray!

finally, Australian Research Council supports open access

Time is harder for humans to understand than is space. Our visual systems abound with machinery for processing extensions of space. A continuum of locations are processed in parallel, their spatial relations apprehended without cognitive effort. But for the most part, the mind represents time poorly. Our perception experience is of a very short duration- the specious present.

metaphors for time in English

Applicants to the lab postdoc / advanced RA position wanna know what they’d be in for if they took the job. Below are some recent conference abstracts from the lab, along the lines of the funded project.

 In Multiple Object Tracking, At High Speeds One May Only Be Able To Track A Single Target—Even If No Crowding Occurs

Alex Holcombe, Wei-Ying Chen

current temporal attention work in the lab

Bianca Hewes is a high-school English teacher and educational technology enthusiast that I was fortunate to connect with. She’s tried a lot of different websites in her search for tools to increase student engagement and improve instruction. With our evidencecharting site, she saw some potential to push her students towards writing better essays.

High school student success with evidence charts

We invite applications for a research fellowship/postdoctoral research fellowship working with Dr. Alex Holcombe in the School of Psychology at the University of Sydney. The research area is visual psychophysics, and the project involves the perception and attentive tracking of moving objects. One line of experiments will investigate the limits on judging the spatial relationship of moving objects.

Position available: postdoc, or highly-qualified research assistant

Quodlibet

is an obscure word that originally referred to a medieval event that included a debate. I haven’t been able to find much information about it, but here is a brief description from Graham (2007): I’m interested in this because I rue the lack of debates in modern science. Perhaps it’s only an accident of history that real debates aren’t happening much nowadays.

Make quodlibets quotidian again! as scientific debates

A new article in the New York Times regarding the allegations against Marc Hauser illustrate how difficult it is to determine whether one is guilty of scientific fraud. A main problem is that record-keeping standards are so lax. This is another reason why open science is important. Open science involves releasing original data and analyses, which is much easier if you have been keeping good records along the way.

Hauser and Baby Einstein cases further call for open science

This is open-access week. Most already know they should be publishing in open-access journals and/or self-archiving their papers. But moving science towards open access has been… slow. I was going to write that things have been moving at a

 glacial

pace, thinking that would be an exaggeration; but nowadays I worry the world’s glaciers may be moving/melting fast enough to be gone before open access is the norm.

open access week, to open code and science

When a scientific article is published, ideally the data behind the reported  results should be made available. Anyone should be able to scrutinize the basis of scientific claims. While this has long been the ideal, it has rarely been practiced. But this has been changing, and momentum is building to actually require the posting of data in circumstances where it’s feasible.

Nudging scientists to share data more

I have argued before that scientists must do more to make available the original data behind their articles. Any scientific claim should ideally be verifiable, via scrutiny of the original evidence. Currently, the vast majority of scientists never share their data.

What should scientists have to say about where the data’s at?

A new version of my 100-minute interactive neural network lesson is available. The lesson webpages guide university-level students through learning and directed play with a connectionist simulator. The outcome is that students gain a sense of how neuron-like processing units can mediate adaptive behavior and memory.

An interactive turnkey tutorial to give students a feel for brain-style computation

If you don’t understand the title of this post, you almost certainly will regret reading further. We’re doing an experiment in which one target is presented along with m distracters.

technical note: d-prime proportion correct in choice experiments (signal detection theory)

Scientific theories are alive. They are debated and actively questioned. Scientists have differing views, strong and informed ones. However, the system of science tends to mask the debate. In the scientific literature, differences are aired, but rarely in a way that most people would recognize as a debate. ‘Debate’ evokes a vision of two parties concisely articulating their positions, disputing points, and rebutting each other.

Where are all the scientific debates? They’re disturbingly hard to find

More and more researchers agree that more access is needed to the original data behind published research articles. Of course, the more general point is that not just the original data, but all materials needed to scrutinize the claims of a manuscript should be available.

Show me the data! A step backwards by the Journal of Neuroscience

First were the Climategate emails. There, Lack of transparency in climate data analyses and climate models contributed to the doubts of skeptics regarding climate change, and made it easier for the skeptics to convince the public that there is good reason for skepticism. Now, the Marc Hauser affair has cast a shadow across another sub-area of science. How can we prevent these scientific fiascos from occurring in the future?

the Marc Hauser affair means we need to require more posting of original data

To what extent should we have confidence in the average published scientific result? The news that Professor Marc Hauser of Harvard University may have been involved in scientific misconduct, yielding irreproducible results, prompts us to reflect on whether irreproducible results are usually caught, or not.

Are most scientific results replicated?

The information so beautifully presented in Saul Griffith’s  lecture is as important as that presented in any talk I’ve seen. Saul has calculated what it’ll take for the world to switch from fossil fuels to renewable energy. And what it takes for an individual to consume an environmentally responsible amount of energy.

lecture on the alternative energy challenge

In writing scientific articles and grants, we find ourselves saying the same type of thing over and over again. We’re almost always talking about theories and evidence. We discuss the relationship of the evidence to the theories, saying whether particular evidence supports the theory or undermines it. To write well, we need to have lots of ways to say these things. So I’m making a list of the ways.

varying the monotony of scientific writing

So I knew neuroscience has exploded over the last few decades, but I didn’t know  its emergence as a more autonomous discipline is “the biggest structural change in scientific citation patterns over the past decade”. In the authors’ words that follow, they are referring to their figure showing neuroscience emerging as a new citation macro-cluster:  “We also highlight the biggest structural change in scientific citation patterns over the past

the rise of neuroscience

We live in an era where students, shift workers, and scientists increasingly consume drugs that modify brain activity in order to enhance cognition. Ethicists are right to fret about this as the number of addictive substances with some ill effects proliferates (DeJong et al. 2008). People will use these things regardless whether or not some condemn the phenomenon, so it is important that information is out there about how best to use them.

optimizing your coffee consumption

Good news from PLoS, they’re posting more metrics indicating the usage and popularity of individual articles.

moving beyond journal impact factor

After several students requested copies, I posted two movies on youtube, one of how visual input to balance can make a baby fall when visual stimulation is perverse.  The other shows how the owl’s vestibular system allows its neck to quickly counterrotate to compensate for the body’s movement. Both videos make people laugh.

Vestibulo-neck-ular reflex and stumbling baby

The current system of publishing scientific papers hampers scientific advancement in particular ways. I’d like to diagnose  the problems so that new publishing efforts (like PLoS ONE) can be shaped to remedy them. I believe that a major flaw in the modern journal system is that it discourages complete treatments of the literature.

scientific journal articles: Too much is left out

Earlier I wrote about the open-source free tools I use to plot and analyze my data—Python and R. One of the most time-consuming and fiddly parts of making graphs for our papers is the need to:  plot multiple subsets of the data (different experimental conditions), sometimes with double axes  make a whole array of plots, one for each of the experimental participants’ data  I’ve been doing this in python with scipy by coding an outer loop

ggplot2 quickly makes beautiful plots in R

In my graduate school days, we had something called ‘The Scathing Wall’, a small bulletin board which provided some outlet for one of our greatest frustrations—dealing with the reviews that came back from a journal after we had submitted a manuscript.

Wall of Shame &amp; Schadenfreude

My scientific workflow includes email between myself and lab members and collaborators, annotations on previously published papers, adding information and ideas to the lab wiki, Python programs to create visual displays and run experiments with them, and Python and R code to plot the results and do the statistical analysis.

open research through automated lab archiving

I’ve transitioned to all open-source software for my science. The Python language and its libraries VisionEgg and Psychopy are more than sufficient to code my perception experiments. For data analysis, I’ve gotten pretty far with the SciPy library for Python, which has probability distributions, function minimization, Fourier transforms, etc. The Matplotlib library makes it easy to make plots in a way familiar for old MATLAB users like me.

Data analysis with Python, SciPy and R

For data analysis, I switched from using MATLAB, partially motivated by a desire to support open source,  to using R. But my experiments nowadays are written in Python, so I decided to try analyzing the data with Python as well. SciPy is an open-source library that helps with this, and duplicates a lot of MATLAB functionality to make it easier to switch from MATLAB.

summarizing data by combinations of variables with python

The conventional encyclopedia: old and unimproved! Here is a preprint of my entry for the

 The Sage Encyclopedia of Perception

with headword “The Binding Problem”. The hardcopy version of the encyclopedia will be a massive 1100-page tome with hundreds of contributors.

The binding problem: A new encyclopedia entry

Nature has published a news item by Declan Butler on the finances of one of its competitors: the open-access PLoS journals, using language that puts the organisation and its journals, especially PLoS ONE, in a negative light.

Nature targets financial weakness of PLoS journals

Free neural network simulation engines, good for understanding simple cognitive-style networks, abstracting away from the actual reality with all those pesky ion channels and membrane potentials and spikes. Emergent is a workhorse, used by serious neural networks researchers but also useful for learning, in conjunction with an associated neural nets textbook, which is probably good for advanced undergraduates.

learning about neural networks: free software

Everything should be made as simple as possible, but no simpler –

Einstein, paraphrased



 KISS- Keep It Simple, Stupid!

– unknown


The principle of parsimony seems obvious, reflexive even. Simpler theories should be favored over more complicated ones. And the idea does seem to have been around for a long time, according to Wikipedia

Parsimony: A newish principle?

The Cauchy distribution is a unimodal distribution with fatter tails than a Gaussian. (Fig 1 at right) Janssen &amp; Shadlen (2005),

 Nature Neuroscience

found that monkey LIP neuron activity followed the subjective hazard function of an objective bimodal probability density function, which goes up, down, then up again.

if it still hasn’t happened yet, it’s likely to take a long time longer!

On twitter I floated this partial solution to the problem of predatory journals,    which I’ll add to here.  If you’ve been in a field for a couple years, then you’re familiar with the journals that most of its research is published in. If you came across a journal that was new to you, you’d probably scrutinise its content and its editorial board before publishing in it, and you’d probably notice if something were a bit dodgy about that journal.

A partial solution to the problem of predatory journals, and a new index of journal quality

Many of the practices associated with modern science emerged in the early days of the Royal Society of London for Improving Natural Knowledge, which was founded in 1660. Today, it is usually referred to as simply “the Royal Society”.  When the Royal Society chose a coat of arms, they included the words

 Nullius in verba

:

A legacy of skepticism and universalism

Today the Gates Foundation announced that they will “cease support for individual article publishing fees, known as APCs, and mandate the use of preprints while advocating for their review”. I am excited by this news because over the last couple decades, it’s been disheartening to see large funders continue to pour money down the throats of high-profit multinational publishers .  In their announcement, the Gates Foundation has recommendations

Gates Foundation, and me: Mandate preprints, support peer review services outside of big publishers

This is a follow-up to my previous post, which was about the APA issuing take-down notices and how you can post preprints to keep your science open.

The APA and publishing costs

When #AcademicNoir trended on Twitter, I had fun making a few memes about science publishing and the Registered Replication Reports that we started at

 Perspectives on Psychological Science

. “Who’s shaking you down?” I asked. “Elsevier,” whispered the librarian. I showed her the door.  I still have to work in this town.

#academicNoir memes

While our retinas process the world around us continuously with little to no interruption, certain mental processes are very limited. One way to show this it to present someone with a rapid stream of letters, one at a time in the center of the screen, at a rate of perhaps 10 letters per second.

The attentional blink and temporal selection

Bradley Voytek spotted a disturbing question in an official “Responsible Conduct of Research” training program:    This defense of the status quo has no place in a “Responsible Conduct of Research” training program. It reads like the old guard self-interestedly maintaining the current system by foisting unjustified beliefs onto young researchers!

Everything’s fine with peer review- if there are any flaws, they’ll be taken care of by evolution?

Seeing slow and seeing fast: two limits on perception

Evidence against the temporal subsampling account of illusory motion reversal

Below is a draft of a chapter I’m writing for

 Subjective Time,

an upcoming book from MIT Press edited by Valtteri Arstila and Dan Lloyd. In a bowling alley, a professional player launches his ball down the lane. As the ball rolls toward the pins, our visual experience of it is smooth and seamless. The ball shifts in position continuously, and this seems to be represented with high fidelity by our brain.

cracks in the edifice of visual time?

Thanks to Efoundations, I see Elsevier has announced an Article 2.0 contest, in which programmers can create new value by harvesting data from 7500 XML-encoded scientific articles.  This is an exciting opportunity for web 2.0 programmers interested in science. But I hope people keep in mind they’d be giving their software ideas to a publisher that charges exorbitant prices for publicly-funded science.

mashing up Elsevier’s journal article database

Richard Feynman, in his 1974 cargo-cult science commencement address:  Unfortunately, the average scientific journal article doesn’t follow this principle. I wouldn’t go so far as to say that the average article is just a sales job, but the emphasis is really on giving the information that favors the author’s theory.

Feynman called for sensible scientific communication; we’re still waiting..

In a clever bit of rhetoric, Professor Dorothy Bishop came up with “the four horsemen of irreproducibility“: publication bias, low statistical power, p-hacking, and HARKing. In an attempt at more complete coverage of the causes of the replication crisis, here I’m expanding on Dorothy’s four horsemen by adding two more causes, and using different wording. This gives me six P’s of the replication crisis!

The Replication Crisis: the Six P’s

I’m releasing an interactive tutorial suitable for either individual learning or in the context of a class wherein each student, or pair of students, has a computer. I used it for my third-year psychology university students. Before beginning the 100-minute class, most had little idea how connectionist networks could store memories or compute visually guided action.

don’t know much ’bout neural networks? An interactive tutorial

I confess that I am an experiment chauvinist – I look down on studies that are purely observational, studies that don’t manipulate anything. Where does my prejudice come from? One factor is that as a perceptual and cognitive psychologist, when I do science, I’m usually interested in the causes, or underlying mechanisms, of a phenomenon.

Alex Holcombe's blog

Evidence Charts receives commendation

High school student success with evidence charts

An interactive turnkey tutorial to give students a feel for brain-style computation