TY - JOUR

T1 - A Framework for Synthesizing Intervention Evidence from Multiple Sources Into a Single Certainty of Evidence Rating: Methodological Developments from a US National Academies of Sciences, Engineering, and Medicine committee

AU - Calonge, B.

AU - Shekelle, G.P.

AU - Owens, D.K.

AU - Teutsch, M.S.

AU - Downey, A.

AU - Brown, L.

AU - Noyes, Jane

N1 - © 2022 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.

PY - 2023/1

Y1 - 2023/1

N2 - BackgroundDespite research investment and a growing body of diverse evidence there has been no comprehensive review and grading of evidence for public health emergency preparedness and response practices comparable to those in medicine and other public health fields.AimsThe National Academies of Sciences, Engineering, and Medicine convened an ad hoc committee to develop and use methods for grading and synthesizing diverse type of evidence to create a single certainty of intervention-related evidence to support recommendations for Public Health Emergency Preparedness and Response Research.MethodsA 13 step consensus building method was used. Experts were first canvassed in public meetings, and a comprehensive review of existing methods was undertaken. Although aspects of existing review methodologies and evidence grading systems were relevant, none adequately covered all requirements for this specific context. Starting with a desire to synthesize diverse sources of evidence not usually included in systematic reviews and using GRADE for assessing certainty and confidence in quantitative and qualitative evidence as the foundation, we developed a mixed-methods synthesis review and grading methodology that drew on (and in some cases adapted) those elements of existing frameworks and methods that were most applicable. Four topics were selected as test cases. The process was operationalized with a suite of method-specific reviews of diverse evidence types for each topic. Further consensus building was undertaken through stakeholder engagement and feedbackConclusionThe NASEM committee’s GRADE adaption for mixed-methods reviews will further evolve over time and has yet to be endorsed by the GRADE working group

AB - BackgroundDespite research investment and a growing body of diverse evidence there has been no comprehensive review and grading of evidence for public health emergency preparedness and response practices comparable to those in medicine and other public health fields.AimsThe National Academies of Sciences, Engineering, and Medicine convened an ad hoc committee to develop and use methods for grading and synthesizing diverse type of evidence to create a single certainty of intervention-related evidence to support recommendations for Public Health Emergency Preparedness and Response Research.MethodsA 13 step consensus building method was used. Experts were first canvassed in public meetings, and a comprehensive review of existing methods was undertaken. Although aspects of existing review methodologies and evidence grading systems were relevant, none adequately covered all requirements for this specific context. Starting with a desire to synthesize diverse sources of evidence not usually included in systematic reviews and using GRADE for assessing certainty and confidence in quantitative and qualitative evidence as the foundation, we developed a mixed-methods synthesis review and grading methodology that drew on (and in some cases adapted) those elements of existing frameworks and methods that were most applicable. Four topics were selected as test cases. The process was operationalized with a suite of method-specific reviews of diverse evidence types for each topic. Further consensus building was undertaken through stakeholder engagement and feedbackConclusionThe NASEM committee’s GRADE adaption for mixed-methods reviews will further evolve over time and has yet to be endorsed by the GRADE working group

KW - GRADE

KW - GRADE CERQual

KW - evidence synthesis

KW - mixed-methods

KW - systematic review

U2 - 10.1002/jrsm.1582

DO - 10.1002/jrsm.1582

M3 - Article

C2 - 35722864

VL - 14

SP - 36

EP - 51

JO - Research synthesis methods

JF - Research synthesis methods

SN - 1759-2879

IS - 1

ER -