Table 1 Types of Characteristics Associated with Reproducibility. Sample sizes (N) depends on study type. Protocol for measured characteristics is found online: https://osf.io/x24n3/
From: A review of reproducible and transparent research practices in urology publications from 2014 to2018
Reproducibility markers | Importance of each marker in regard to transparency and reproducibility | |
|---|---|---|
Accessibility | ||
All (N = 300) | Article accessibility (Is the article available to the public without a paywall?) | Accessible research allows for a larger audience to assess and replicate a study’s findings |
Funding | ||
Included studies (N = 294) | Funding statement (Do authors provide a statement to describe if or how the study was funded?) | Including a funding statement provides greater transparency to readers. This increased transparency reveals any signs of bias or influence in the study’s methodology |
Conflict of Interest | ||
Included studies (N = 294) | Conflict of interest statement (Do the authors reveal any conflicts of interest in their manuscript?) | Conflict of interest statements give the authors a chance to be transparent about relationships with entities that may try to influence a study’s findings |
Publication citations | ||
Empirical studiesa (N = 171) | Systematic review/meta-analysis citations (Has the study been cited by data synthesis study designs such as systematic reviews or meta-analyses?) | Systematic reviews and meta-analyses synthesize information in studies that may have been replicated. The synthesis of information reveals a more complete answer to the question being investigated |
Analysis scripts | ||
Empirical studiesb (N = 171) | Availability statement (Is there a statement in the manuscript describing the accessibility of the analysis script?) | Having the analysis script allows raw data to be analyzed exactly as the authors did in the original study, allowing others to replicate the data analysis correctly |
Location of analysis script (Where can the analysis script be found? Supplementary materials?) | ||
Accessibility (Can a reader access the analysis script through the manuscript online or through other methods?) | ||
Materials | ||
Empirical studiesc (N = 162) | Availability statement (Is there a statement in the manuscript describing the accessibility of additional materials to the study?) | Additional materials allows readers to learn what is needed to reproduce the study, enabling the study to be replicated |
Location of additional materials (Where can the additional material be found? Supplementary materials?) | ||
Accessibility (Can a reader access additional material through the manuscript online or through other methods?) | ||
Pre-registration | ||
Empirical studiesb (N = 138) | Availability statement (Is there a statement in the manuscript describing whether the study was pre-registered or not?) | Pre-registering a study prevents any tampering of the study design throughout implementation of the study, increasing the reliability of the study. Pre-registration also can provide components that may aid in replicating a study |
Location of registration(Where was the study registered?) | ||
Accessibility of the registration (Is the registration accessible?) | ||
Components included in registration (What components of the study were found in the registration?) | ||
Protocols | ||
Empirical studiesb (N = 171) | Availability statement (Is there a statement in the manuscript describing whether the study protocol was available or not?) | Access to a detailed protocol allows others to know what, where, why, and how the study was performed, aiding others in the replication of the original study |
Components (What components of the study were found in the protocol?) | ||
Raw data | ||
Empirical studiesb (N = 171) | Availability statement (Is there a statement in the manuscript describing the accessibility of raw data from the study?) | Raw data provide insight into the author’s thoughts and actions throughout implementation of the study, aiding others in replication of the original study. Additionally, raw data provide transparency to what is presented in the study’s findings |
Method of availability (Where can the raw data be found? Supplementary materials?) | ||
Accessibility (Can a reader access raw data through the manuscript online or through other methods?) | ||
Components (Are all the components of raw data that is needed to replicate the study available?) | ||
Clarity (Are the raw data understandable?) | ||