Ignoring non‐English‐language studies may bias ecological meta‐analyses

Research output: Contribution to journalArticlepeer-review

Electronic versions

DOI

  • Ko Konno
  • Munemitsu Akasaka
    Tokyo University of Agriculture and Technology
  • Chieko Koshida
    Plant Chemical Ecology Technische Universität Darmstadt Darmstadt Germany
  • Naoki Katayama
    Biodiversity Division Institute for Agro‐Environmental Sciences NARO Tsukuba‐shi Japan
  • Noriyuki Osada
    Laboratory of Plant Conservation Science, Faculty of Agriculture Meijo University Nagoya Japan
  • Rebecca Spake
    School of Geography and Environmental Science University of Southampton Southampton UK
  • Tatsuya Amano
    Tokyo University of Agriculture and Technology
Meta-analysis plays a crucial role in syntheses of quantitative evidence in ecology and biodiversity conservation. The reliability of estimates in meta-analyses strongly depends on unbiased sampling of primary studies. Although earlier studies have explored potential biases in ecological meta-analyses, biases in reported statistical results and associated study characteristics published in different languages have never been tested in environmental sciences. We address this knowledge gap by systematically searching published meta-analyses and comparing effect-size estimates between English- and Japanese-language studies included in existing meta-analyses. Of the 40 published ecological meta-analysis articles authored by those affiliated to Japanese institutions, we find that three meta-analysis articles searched for studies in the two languages and involved sufficient numbers of English- and Japanese-language studies, resulting in four eligible meta-analyses (i.e., four meta-analyses conducted in the three meta-analysis articles). In two of the four, effect sizes differ significantly between the English- and Japanese-language studies included in the meta-analyses, causing considerable changes in overall mean effect sizes and even their direction when Japanese-language studies are excluded. The observed differences in effect sizes are likely attributable to systematic differences in reported statistical results and associated study characteristics, particularly taxa and ecosystems, between English- and Japanese-language studies. Despite being based on a small sample size, our findings suggest that ignoring non-English-language studies may bias outcomes of ecological meta-analyses, due to systematic differences in study characteristics and effect-size estimates between English- and non-English languages. We provide a list of actions that meta-analysts could take in the future to reduce the risk of language bias.

Keywords

  • Ecology, Ecology, Evolution, Behavior and Systematics, Nature and Landscape Conservation
Original languageEnglish
Pages (from-to)6373-6384
Number of pages12
JournalEcology and Evolution
Volume10
Issue number13
Early online date29 May 2020
DOIs
Publication statusPublished - 1 Jul 2020
View graph of relations