Publication bias

Publication bias

Publication bias is the tendency of researchers, editors, and pharmaceutical companies to handle the reporting of experimental results that are positive (i.e. showing a significant finding) differently from results that are negative (i.e. supporting the null hypothesis) or inconclusive, leading to bias in the overall published literature. Such bias occurs despite the fact that studies with significant results do not appear to be superior to studies with a null result with respect to quality of design.[1] It has been found that statistically significant results are three times more likely to be published than papers affirming a null result.[2] It also has been found that the most common reason for non-publication is an investigator's declining to submit results for publication (because of the investigator's loss of interest in the topic, the investigator's anticipation that others will not be interested in null results, etc.), underlining researchers' role in publication bias phenomena.[1]

In an effort to decrease this problem, some prominent medical journals require registration of a trial before it commences so that unfavorable results are not withheld from publication. Several such registries exist, but researchers are often unaware of them. In addition, attempts to identify unpublished studies have proved very difficult and often unsatisfactory. Another strategy suggested by a meta-analysis is caution in the use of small and non-randomised clinical trials because of their demonstrated high susceptibility to error and bias.[1]

Contents

Definition

Publication bias occurs when the publication of research results depends on their nature and direction.[3]

Positive results bias, a type of publication bias, occurs when authors are more likely to submit, or editors accept, positive than null (negative or inconclusive) results.[4] A related term, "the file drawer problem", refers to the tendency for negative or inconclusive results to remain unpublished by their authors.[5]

Outcome reporting bias occurs when several outcomes within a trial are measured but these are reported selectively depending on the strength and direction of those results. A related term that has been coined is HARKing (Hypothesizing After the Results are Known).[6]

The file drawer effect

The file drawer effect, or file drawer problem, is that many studies in a given area of research may be conducted but never reported, and those that are not reported may on average report different results from those that are reported. An extreme scenario is that a given null hypothesis of interest is in fact true, i.e. the association being studied does not exist, but the 5% of studies that by chance show a statistically significant result are published, while the remaining 95% where the null hypothesis was not rejected languish in researchers' file drawers. Even a small number of studies lost "in the file drawer" can result in a significant bias.[7]

The term was coined by the psychologist Robert Rosenthal in 1979.[8]

Effect on meta-analysis

The effect of this is that published studies may not be truly representative of all valid studies undertaken, and this bias may distort meta-analyses and systematic reviews of large numbers of studies—on which evidence-based medicine, for example, increasingly relies. The problem may be particularly significant when the research is sponsored by entities that may have a financial or ideological interest in achieving favorable results.

Those undertaking meta-analyses and systematic reviews need to take account of publication bias in the methods they use for identifying the studies to include in the review. Among other techniques to minimize the effects of publication bias, they may need to perform a thorough search for unpublished studies, and to use such analytical tools as a Begg's funnel plot or Egger's plot to quantify the potential presence of publication bias. Tests for publications bias rely on the underlying theory that small studies with small sample size (and large variance) would be more prone to publication bias, while large-scale studies would be less likely to escape public knowledge and more likely to be published regardless of significance of findings. Thus, when overall estimates are plotted against the variance (sample size), a symmetrical funnel is usually formed in the absence of publication bias, while a skewed asymmetrical funnel is observed in presence of potential publication bias.

Extending the funnel plot, the "Trim and Fill" method has also been suggested as a method to infer the existence of unpublished hidden studies, as determined from a funnel plot, and subsequently correct the meta-analysis by imputing the presence of missing studies to yield an unbiased pooled estimate.

Examples of publication bias

One study[9] compared Chinese and non-Chinese studies of gene-disease associations and found that "Chinese studies in general reported a stronger gene-disease association and more frequently a statistically significant result".[10] One possible interpretation of this result is selective publication (publication bias).

Risks and remedies

Risks

According to John Ioannidis, negative papers are most likely to be suppressed:[11]

  1. when the studies conducted in a field are smaller
  2. when effect sizes are smaller
  3. when there is a greater number and lesser preselection of tested relationships
  4. where there is greater flexibility in designs, definitions, outcomes, and analytical modes
  5. when there is greater financial and other interest and prejudice
  6. when more teams are involved in a scientific field in chase of statistical significance.

Ioannidis further asserts that "claimed research findings may often be simply accurate measures of the prevailing bias".

Remedies

Ioannidis' remedies include:

  1. Better powered studies
    • Low-bias meta-analysis
    • Large studies where they can be expected to give very definitive results or test major, general concepts
  2. Enhanced research standards including
    • Pre-registration of protocols (as for randomized trials)
    • Registration or networking of data collections within fields (as in fields where researchers are expected to generate hypotheses after collecting data)
    • Adopting from randomized controlled trials the principles of developing and adhering to a protocol.
  3. Considering, before running an experiment, what they believe the chances are that they are testing a true or non-true relationship.
    • Properly assessing the false positive report probability based on the statistical power of the test[12]
    • Reconfirming (whenever ethically acceptable) established findings of "classic" studies, using large studies designed with minimal bias

Study registration

In September 2004, editors of several prominent medical journals (including the New England Journal of Medicine, The Lancet, Annals of Internal Medicine, and JAMA) announced that they would no longer publish results of drug research sponsored by pharmaceutical companies unless that research was registered in a public database from the start.[13] Furthermore, some journals, e.g. Trials, encourage publication of study protocols in their journals.[14]

See also

References

  1. ^ a b c Easterbrook, P. J.; Berlin, J. A.; Gopalan, R.; Matthews, D. R. (1991). "Publication bias in clinical research". Lancet 337 (8746): 867–872. doi:10.1016/0140-6736(91)90201-Y. 
  2. ^ Dickersin, K.; Chan, S.; Chalmers, T. C.; et al. (1987). "Publication bias and clinical trials". Controlled Clin Trials 8 (4): 343–353. doi:10.1016/0197-2456(87)90155-3. 
  3. ^ K. Dickersin (March 1990). "The existence of publication bias and risk factors for its occurrence". JAMA 263 (10): 1385–1359. doi:10.1001/jama.263.10.1385. PMID 2406472. 
  4. ^ D.L. Sackett (1979). "Bias in analytic research". J Chronic Dis 32 (1–2): 51–63. doi:10.1016/0021-9681(79)90012-2. PMID 447779. 
  5. ^ Robert Rosenthal (May 1979). "The file drawer problem and tolerance for null results". Psychological Bulletin 86 (3): 638–641. doi:10.1037/0033-2909.86.3.638. http://content.apa.org/journals/bul/86/3/638. 
  6. ^ N.L. Kerr (1998). "HARKing: Hypothesizing After the Results are Known". Personality and Social Psychology 2 (3): 196–217. doi:10.1207/s15327957pspr0203_4. PMID 15647155. http://direct.bl.uk/bld/PlaceOrder.do?UIN=048343521&ETOC=RN&from=searchengine. 
  7. ^ Jeffrey D. Scargle (2000). "Publication Bias: The "File-Drawer Problem" in Scientific Inference". Journal of Scientific Exploration 14 (2): 94–106. http://www.scientificexploration.org/journal/jse_14_1_scargle.pdf. 
  8. ^ Rosenthal, Robert (1979). "The file drawer problem and tolerance for null results". Psychological Bulletin 86 (3): 638–641. doi:10.1037/0033-2909.86.3.638. 
  9. ^ Zhenglun Pan, Thomas A. Trikalinos, Fotini K. Kavvoura, Joseph Lau, John P.A. Ioannidis, "Local literature bias in genetic epidemiology: An empirical evaluation of the Chinese literature". PLoS Medicine, 2(12):e334, 2005 December.
  10. ^ Jin Ling Tang, "Selection Bias in Meta-Analyses of Gene-Disease Associations", PLoS Medicine, 2(12):e409, 2005 December.
  11. ^ Ioannidis J (2005). "Why most published research findings are false". PLoS Med 2 (8): e124. doi:10.1371/journal.pmed.0020124. PMC 1182327. PMID 16060722. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pmcentrez&artid=1182327. 
  12. ^ Wacholder, S.; Chanock, S; Garcia-Closas, M; El Ghormli, L; Rothman, N (March 2004). "Assessing the Probability That a Positive Report is False: An Approach for Molecular Epidemiology Studies". JNCI 96 (6): 434–42. doi:10.1093/jnci/djh075. PMID 15026468. http://jnci.oxfordjournals.org/cgi/content/full/96/6/434. 
  13. ^ (The Washington Post) (2004-09-10). "Medical journal editors take hard line on drug research". smh.com.au. http://www.smh.com.au/articles/2004/09/09/1094530773888.html. Retrieved 2008-02-03. 
  14. ^ "Instructions for Trials authors - Study protocol". 2009-02-15. http://www.trialsjournal.com/info/instructions/?txt_jou_id=10096&txt_mst_id=61789. 

External links


Wikimedia Foundation. 2010.

Игры ⚽ Поможем написать курсовую

Look at other dictionaries:

  • publication bias — noun The bias to meta analysis resulting from statistical studies with low statistical power tending to remain unpublished and inaccessible to the analyst …   Wiktionary

  • Bias (disambiguation) — Bias is an inclination towards something, or a predisposition, partiality, prejudice, preference, or predilection. Bias may also refer to:In science and statistics: * Bias (statistics), the systematic distortion of a statistic ** A biased sample… …   Wikipedia

  • Bias — This article is about different ways the term bias is used . For other uses, see Bias (disambiguation). Bias is an inclination to present or hold a partial perspective at the expense of (possibly equally valid) alternatives. Bias can come in many …   Wikipedia

  • Bias — In a clinical trial, bias refers to effects that a conclusion that may be incorrect as, for example, when a researcher or patient knows what treatment is being given. To avoid bias, a blinded study may be done. * * * 1. Systematic discrepancy… …   Medical dictionary

  • Publication ban — A publication ban is a court order which prohibits the public or media from disseminating certain details of an otherwise public judicial procedure. In Canada, publication bans are most commonly issued when the safety or reputation of a victim or …   Wikipedia

  • Selection bias — is a statistical bias in which there is an error in choosing the individuals or groups to take part in a scientific study.[1] It is sometimes referred to as the selection effect. The term selection bias most often refers to the distortion of a… …   Wikipedia

  • Confirmation bias — (also called confirmatory bias or myside bias) is a tendency for people to favor information that confirms their preconceptions or hypotheses regardless of whether the information is true.[Note 1][1] As a result, people gather evidence and recall …   Wikipedia

  • Survivorship bias — In finance, Survivorship bias is the tendency for failed companies to be excluded from performance studies because they no longer exist. It often causes the results of studies to skew higher because only companies which were successful enough to… …   Wikipedia

  • Experimenter's bias — In experimental science, experimenter s bias is subjective bias towards a result expected by the human experimenter. David Sackett,[1] in a useful review of biases in clinical studies, states that biases can occur in any one of seven stages of… …   Wikipedia

  • Sampling bias — In statistics, sampling bias is when a sample is collected in such a way that some members of the intended population are less likely to be included than others. It results in a biased sample, a non random sample[1] of a population (or non human… …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”