LLU Live #25: The Fetishization of Peer Review

LLU Live #25: The Fetishization of Peer Review

This entry is part 25 of 25 in the series LLU Live

Peer review isn’t an issue normally covered by the mainstream media, but near the end of last year, an article in Science magazine made a big enough splash to catch the attention of NPR and other national outlets outside the ivory tower. “Who’s Afraid of Peer Review?” highlighted some of the issues with the system of peer-review for journals and other publications, including the egregious failure of the process to weed out a paper submitted with completely fabricated credentials:

Any reviewer with more than a high-school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper’s short-comings immediately. Its experiments are so hopelessly flawed that the results are meaningless. I know because I wrote the paper. Ocorrafoo Cobange does not exist, nor does the Wassee Institute of Medicine.

The whole point of the peer-review process is to make sure that any work in a particular field is reviewed by the people who are best positioned to question the biases, methodologies, results, and conclusions of that work: other people who are experts in that same field. Ideally, it sounds like a great system, but in reality it’s a flawed model for several reasons. For starters: meaningfully culling submissions means reading them carefully — and that takes time many established experts don’t have. So reading either happens more hastily, or it’s done by less experienced members of the field (who usually have even less time). Either way, the ideal situation is already breaking down, and quickly.

This Thursday at 4pm Eastern on LLU Live #25, I’d like to talk about how we fetishize the idea of peer review, and whether the right method moving forward is to kill peer review altogether, or maybe to just reform the model. Regardless of where we individually stand on the question, I’d like to come away with some alternative ideas for ways to encourage high-quality writing, while also taking advantage of new publication methods.

Instructions for joining the conversation can be found on our Live Events page. I hope to see you there!

 

Series Navigation<< LLU Live #24: The Limits of Technology

Ryan has been proudly maintaining and contributing to Language Lab Unleashed since 2005, and is the current President of SWALLT. Since the summer of 2013 he's been causing trouble with his all-star colleagues in the UMW DTLT; when not wrangling websites Ryan can be found doing strange things with heavy objects.

2 Comments

  1. Barbara · April 10, 2014 Reply

    Well you got me at the word “fetishization” :-)

    This sounds fascinating and heck yes I will be there.

    And you know that as soon as something like blind peer review purports itself to be bullet-proof, of COURSE someone will come along and try to hack it. So they did and it was. And here is an example: http://www.ibtimes.com/fake-research-papers-how-did-more-120-gibberish-computer-generated-studies-get-published-1558725
    I just love the fact that students created a paper topic generator (here: http://pdos.csail.mit.edu/scigen/) but that these papers were accepted at a conference? Fabulous.

    Along the same lines: This is a site that a colleague from work gave me: it is a title generator in Spanish for paper or talk topics. Such awesomeness and academic puffiness all in one place…
    http://www.yeray.com/generador/

  2. Barbara · June 6, 2014 Reply

    Just saw this and thought of our conversation about peer review, so I decided to plunk this comment here… https://onarbor.com/ A site to “make, fund and review science and art” Looks like it also has some sort of a reddit-like voting feature too.

    Interesting.

Leave a reply

css.php