Electronic versions

  • Heather M Munthe-Kaas
    Norwegian Institute of Public Health, Oslo
  • Andrew Booth
    University of Sheffield
  • Isolde Sommer
    University for Continuing Education Krems
  • Sara Cooper
    Cochrane South Africa
  • Ruth Garside
    University of Exeter
  • Karin Hannes
    KU Leuven
  • Jane Noyes
  • Camelot Development Group
Introduction
Qualitative evidence is increasingly incorporated into decision-making processes. Assessing the methodological limitations of primary studies is critical to making an overall assessment of confidence in findings from qualitative evidence syntheses (QES) using GRADE-CERQual. Current critical appraisal tools were not developed specifically for use in Cochrane reviews or GRADE-CERQual, and few are evidence-based. The aim of CochrAne qualitative Methodological LimitatiOns Tool (CAMELOT) was to address this gap.

Methods
We undertook this project in four stages: (1) systematic literature search to identify existing tools, (2) identification of evidence to support inclusion of potential CAMELOT domains (3) consensus survey to agree on the inclusion and definition of CAMELOT domains, and (4) human-centered design approach to develop and refine CAMELOT by exploring user experience.

Results
CAMELOT is a new evidence-based tool for assessing the methodological strengths and limitations of primary qualitative research studies in a QES. CAMELOT is comprised of 12 domains: four Meta domains that encourage review authors to consider those characteristics of the primary study that are beyond how the study was carried out, but which inform the conduct and design of the study, and eight Method domains which encourage review authors to consider how the study was designed, planned and/or conducted, and how study conduct and design fits with the information provided in the four meta domains. Review authors make an assessment by identifying any concerns regarding the methods used in the study and considering the appropriateness of fit between the Meta and Method domains.

Conclusion
CAMELOT provides review authors with a transparent and systematic method to assess methodological limitations of primary qualitative studies. CAMELOT incorporates qualitative principles and focuses on appropriateness of fit between Meta and Method domains. In line with iterative tool development approach, CAMELOT will continue to be revised over time following extensive user testing and piloting.
Original languageEnglish
Article numbere12058
JournalCochrane Evidence Synthesis and Methods
Volume2
Issue number6
Early online date8 Jun 2024
Publication statusPublished - Jun 2024
View graph of relations