Home » News
Category Archives: News
Yesterday in Science, the Transparency and Openness Promotion (TOP) Committee published the TOP Guidelines, referred to by the New York Times as “the the most comprehensive guidelines for the publication of studies in basic science to date” (see here). The guidelines are the output of a November 2014 meeting at the Center for Open Science (COS), co-hosted with BITSS and Science Magazine.
By Garret Christensen (BITSS)
BITSS is proud to announce the publication of the Transparency and Openness Promotion Guidelines in Science. The Guidelines are a set of standards in eight areas of research publication:
- Citation Standards
- Data Transparency
- Analytic Methods (Code) Transparency
- Research Materials Transparency
- Design and Analysis Transparency
- Preregistration of Sudies
- Preregistration of Analysis Plans
Project TIER (Teaching Integrity in Empirical Economics) is pleased to announce its newly-established Advisory Board. The advisors – George Alter (ICPSR), J. Scott Long (Indiana University), Victoria Stodden (University of Illinois at Urbana-Champaign), and Justin Wolfers (Peterson Institute/University of Michigan) – will help project directors Richard Ball and Norm Medeiros consider ways of developing and promoting the TIER protocol for documenting empirical research.
The guiding principle behind the protocol is that the documentation (data, code, and supplementary information) should be complete and transparent enough to allow an interested third party to easily and exactly reproduce all the steps of data management and analysis that led from the original data files to the results reported in the paper. The ultimate goal of Project TIER is to foster development of a national network of educators committed to integrating methods of empirical research documentation, guided by the principle of transparency, into the curricula of the social sciences.
Harsh scrutiny of an influential political science experiment highlights the importance of transparency in research.
The paper, from UCLA graduate student Michael LaCour and Columbia University Professor Donald Green, was published in Science in December 2014. It asserted that short conversations with gay canvassers could not only change people’s minds on a divisive social issue like same-sex marriage, but could also have a contagious effect on the relatives of those in contact with the canvassers. The paper received wide attention in the press.
Yet three days ago, two graduate students from UC Berkeley, David Broockman and Joshua Kalla, published a response to the study, pointing to a number of statistical oddities, and discrepancies between how the experiment was reported and how the authors said it was conducted. Earlier in the year, impressed by the paper findings, Broockman and Kalla had attempted to conduct an extension of the study, building on the original data set. This is when they became aware of irregularities in the study methodology and decided to notify Green.
Reviewing the comments from Broockman and Kalla, Green, who was not involved in the original data collection, quickly became convinced that something was wrong – and on Tuesday, he submitted a letter to Science requesting the retraction of the paper. Green shared his view on the controversy in a recent interview, reflecting on what it meant for the broader practice of social science and highlighting the importance of integrity in research.
New prizes will recognize and reward transparency in social science research.
BERKELEY, CA (May 13, 2015) – Transparent research is integral to the validity of science. Openness is especially important in such social science disciplines as economics, political science and psychology, because this research shapes policy and influences clinical practices that affect millions of lives. To encourage openness in research and the teaching of best practices, the Berkeley Initiative for Transparency in the Social Sciences (BITSS) has established The Leamer-Rosenthal Prizes for Open Social Science. BITSS is an initiative of the Center for Effective Global Action (CEGA) at the University of California, Berkeley. The prizes, which provide recognition, visibility and cash awards to both the next generation of researchers and senior faculty, are generously supported by the John Templeton Foundation.
The competition is open to scholars and educators worldwide.
“In academia, career advances and research funding are usually awarded on the basis of how many journal articles a scientist publishes. This incentive structure can encourage researchers to dramatize their findings in ways that increase the probability of publication, sometimes even at the expense of transparency and integrity,” said Edward Miguel, PhD, Professor of Economics at UC Berkeley and Faculty Director of CEGA. “The Leamer-Rosenthal Prizes will help speed the adoption of transparent practices by recognizing and rewarding researchers and educators whose work and teaching exemplify the best in open social science.”
Garret Christensen–BITSS Project Scientist
BITSS participated in a pair of conferences/workshops recently that we should probably tell you about. First, BITSS was part of a research transparency conference in Washington DC put together by the Laura and John Arnold Foundation. Many of the presentations from the conference can be found here. The idea was to bring together academics, researchers on federal contracts, and federal government research sponsors and policy makers. Just a few things that were new to me or which stuck out were:
The Coalition for Evidence-Based Policy, equivalent to CEGA’s domestic counterpart and a leading force working to institutionalize evidence-based policy making, will merge with one of its funders, the Laura and John Arnold Foundation (LJAF). Also a funder of BITSS, LJAF will integrate the staff of the Coalition into its newly established Evidence-Based Policy and Innovation division. The mission of the new division will be very similar to the one of the Coalition it is replacing which will close down its operations in the next few days and transition its staff to the LJAF in the coming weeks.
According to a LJAF press release, the evidence-based policy subdivision, that will be led by Jon Baron, the former president of the Coalition, will focus on “strategic investments in rigorous evaluations, collaborations with policy officials to advance evidence-based reforms, and evidence reviews to identify promising and proven programs” (LJAF). The innovation subdivision, to be led by Kathy Stack, former adviser for evidence-based innovation at the White House Office of Management and Budget, “will bring policymakers, researchers, and data experts from the public and private sectors together to strengthen the infrastructure and processes needed to support evidence-based decision making” (LJAF).