What does peer review know?
Listen now
Description
People rag on peer review a lot (including, occasionally, New Things Under the Sun). Yet it remains one of the most common ways to allocate scientific resources, whether those be R&D dollars or slots in journals. Is this all a mistake? Or does peer review help in its purported goal to identify the science most likely to have an impact and hence, perhaps most deserving of some of those limited scientific resources? A simple way to check is to compare peer review scores to other metrics of subsequent scientific impact; does peer review predict eventual impact? A number of studies find it does. This podcast is an audio read through of the (initial version of the) article What does peer review know?, originally published on New Things Under the Sun. Articles mentioned Li, Danielle, and Leila Agha. 2015. Big names or big ideas: Do peer-review panels select the best science proposals? Science 348(6233): 434-438. https://doi.org/10.1126/science.aaa0185 Park, Hyunwoo, Jeongsik (Jay) Lee, and Byung-Cheol Kim. 2015. Project selection in NIH: A natural experiment from ARRA. Research Policy 44(6): 1145-1159. https://doi.org/10.1016/j.respol.2015.03.004. Card, David, and Stefano DellaVigna. 2020. What do Editors Maximize? Evidence from Four Economics Journals. The Review of Economics and Statistics 102(1): 195-217. https://doi.org/10.1162/rest_a_00839 Siler, Kyle, Kirby Lee, and Lisa Bero. 2014. Measuring the effectiveness of scientific gatekeeping. PNAS 112(2): 360-365. https://doi.org/10.1073/pnas.1418218112 Teplitskiy, Misha, and Von Bakanic. 2016. Do Peer Reviews Predict Impact? Evidence from the American Sociological Review, 1978 to 1982. Socius, 2. https://doi.org/10.1177/2378023116640278
More Episodes
Published 04/04/24
Welcome to patents week! I set out to write a post about using patents to measure innovation, but it turned into four. I'm releasing podcasts of each episode, one per day, but if you're too excited to wait, you can read all four here, on New Things Under the Sun. How many inventions are patented?...
Published 04/04/24
Welcome to patents week! I set out to write a post about using patents to measure innovation, but it turned into four. I'm releasing podcasts of each episode, one per day, but if you're too excited to wait, you can read all four here, on New Things Under the Sun. How many inventions are patented?...
Published 04/03/24