Month: November 2011

tracking replication attempts in psychology–for real this time

I’ve written a few posts on this blog about how the development of better online infrastructure could help address and even solve many of the problems psychologists and other scientists face (e.g., the low reliability of peer review, the ‘fudge factor’ in statistical reporting, the sheer size of the scientific literature, etc.). Actually, that general question–how we can use technology to do better science–occupies a good chunk of my research these days (see e.g., Neurosynth). One question I’ve been interested in for a long time is how to keep track not only of ‘successful’ studies (i.e., those that produce sufficiently interesting effects to make it into the published literature), but also replication failures (or successes of limited interest) that wind up in researchers’ file drawers. A couple of years ago I went so far as to build a prototype website for tracking replication attempts in psychology. Unfortunately, it never went anywhere, partly (okay, mostly) because the site really sucked, and partly because I didn’t really invest much effort in drumming up interest (mostly due to lack of time). But I still think the idea is a valuable one in principle, and a lot of other people have independently had the same idea (which means it must be right, right?).

Anyway, it looks like someone finally had the cleverness, time, and money to get this right. Hal Pashler, Sean Kang*, and colleagues at UCSD have been developing an online database for tracking attempted replications of psychology studies for a while now, and it looks like it’s now in beta. PsychFileDrawer is a very slick, full-featured platform that really should–if there’s any justice in the world–provide the kind of service everyone’s been saying we need for a long time now. If it doesn’t work, I think we’ll have some collective soul-searching to do, because I don’t think it’s going to get any easier than this to add and track attempted replications. So go use it!

 

*Full disclosure: Sean Kang is a good friend of mine, so I’m not completely impartial in plugging this (though I’d do it anyway). Sean also happens to be amazingly smart and in search of a faculty job right now. If I were you, I’d hire him.

naked dense bodies provoke depression (and other tall scientific tales)

I’ve been using Mendeley for about a year now, and while there are plenty of kinks left for the developers iron out (mostly related to the Word plug-in), I have to say I like it a lot overall. I could say more about why I like it a lot, but I won’t, because this isn’t really a post about Mendeley. Rather, it’s a post about one particular group on Mendeley (groups on Mendeley are basically curated sets of thematically related scientific articles). Specifically, the “Creatively named research papers” group.

Since the title of the group is self-explanatory, I’ll just list some of the more noteworthy entries, along with some of the corresponding notes I jotted down (you know, in case I need to refer back to these papers):

 

Naked Dense Bodies Provoke Depression

I don’t think depression is the normative response to this stimulus; this must be a case report.

 

Marvel Universe looks almost like a real social network

“We would like to mention that the actual number of collaborations is 569,770, but this value counts all collaborations in the Marvel Universe history, and while there are 91,040 pairs of characters that have only met once, other pairs have met quite often: for instance, every pair of members of the Fantastic Four has jointly appeared in around 700 comic books (more specifically, this range of collaborations of the members of the Fantastic Four runs between 668 joint appearances of the Thing and the Invisible Woman to 744 joint appearances of the Thing and the Human Torch).” (p. 7)

 

Are Analytic Philosophers Shallow and Stupid?

I’ll leave this one up to the analytic philosophers to mull over. We’ll check back on their progress in another ten or twenty years.

 

Are full or empty beer bottles sturdier and does their fracture-threshold suffice to break the human skull?

Spoiler: the answers are ’empty’ and ‘yes’, respectively.

 

A woman’s history of vaginal orgasm is discernible from her walk

I don’t want to offend anyone, so I’m going to tread very delicately here and just tiptoe away quietly.

 

Traumatic brain injuries in illustrated literature: experience from a series of over 700 head injuries in the Asterix comic books

At some point you kind of start to feel bad for the Romans.

 

Skillful writing of an awful research paper

Pretty sure I already know everything discussed in this article.

 

Chemical processes in the deep interior of Uranus

Obvious joke is obvious.

 

Japan’s Phillips Curve Looks Like Japan

A pretty remarkable article. Gregor Smith isn’t kidding; here’s Japan’s Phillips Curve:

 

Is a jumper angrier than a tree?

Possibly even better than the title of this paper is the set of papers Mendeley thinks are related, which include “The greater-than-g acceleration of a bungee jumper”, “When is a tree more than a tree?”, and my personal favorite, “The Angry, the Angrier, and the Angriest: Relationship Implications”.

 

The Penetration of a Finger into a Viscous Fluid in a Channel and Tube

It’s not often you find your finger stuck in an oil-filled Chinese finger trap, but when it inevitably does happen, you’ll be very glad you read this paper.

 

Executive Decision-Making in the Domestic Sheep

I’m a big fan of studies involving clever sheep.

 

Numerical simulation of fundamental trapped sausage modes

Alternative title: What’s the optimal amount of time to microwave a midnight snack for?

 

Accidental condom inhalation

You’re doing it wrong.

 

On the Effectiveness of Aluminium Foil Helmets: An Empirical Study

Pfft. Like anyone who wears one of these things is going to believe results published by agents of the scientific-industrial complex.

 

Experiments with genitalia : a commentary

Abstract: “There has been a recent burst of studies of the function of genitalia, many of which share several important shortcomings. Given that further studies on this topic are likely (there are probably millions of species showing rapid genital divergence), I discuss the studies critically to promote clear formulation of hypotheses and interpretation of results in the future. I also emphasize some possibly important but neglected variables, including female stimulation, phylogenetic contexts, and the behavior of male genitalia, and outline simple techniques that could improve future studies.”

 

The earth is round (p < . 05)

For shame! This one has no business being in this group! It’s an excellent title to one of the best commentaries on psychological methods ever written!

 

Amusing titles in scientific journals and article citation

Yes, you’re very clever, person who added this self-referential article to the group.

 

The ethics of eating a drug-company donut

It starts with a donut, and before you know it, you’re spending your lunch break stuffing boxes full of Pfizer pens down your shirt pocket.

 

Rectal impalement by pirate ship: A case report

You’re definitely doing it wrong.

 

Anyway, I’m sure this is just a tiny fraction of the creatively-named scientific literature. If you know of (or have authored) any worthy candidates, add them to the Mendeley group–or just indulge me and post them below in the comments. Note that in this context ‘creatively named’ seems to mean humorous rather than clever. There are probably many more clever titles out there than funny ones (a trend abetted by the fact that a clever title is pretty much a prerequisite for publishing in Psychological Science at this point), but for purposes of this thread, we don’t want to hear about your naked dense bodies unless they’re funny-looking!