Multi-level Intervention Research Methods: Recommendations for Targeting Hard-to-Reach, High-Risk or Vulnerable Populations and Communities

Background

Multilevel interventions address more than one level of influence in the social-ecological model (Figure). Interventions that have multiple components that address only one level of influence are multi-component but not multi-level. Interventions that have multiple components that address a given level, but that also address two or more levels of the socio-ecological model are both multi-component and multi-level.

Multi-level intervention research and systems-based interventions in hard-to-reach or high-risk communities and populations (e.g., rural/urban low socio-economic status populations, elderly, underserved communities, minority populations including American Indians/Alaskan Natives) present unique challenges, including issues related to intervention design, blinding of study staff, recruitment and retention, culturally and linguistically appropriate intervention approaches, built environmental factors, and intervention dose, reach and fidelity at multiple levels of implementation. 

Supporting Slides

Discussions

Workshop participants consisted of experts in multi-level and systems-oriented intervention research, obesity, health promotion and health behavior, nursing, nutrition and physical activity and included pediatricians, family medicine and community health educators, senior research scientists, epidemiologists, biostatisticians, and project coordinators.

The meeting began after a brief overview and introduction and was followed by opening remarks by the Director, NIH Office of Disease Prevention (ODP), who emphasized the importance of Prevention at the NIH, and noted that the workshop supports ODP’s strategic priority III, which is to promote the “use of the best available methods in prevention research and to support the development of better methods.” Multilevel intervention research is relevant to many areas in prevention, including research targeting hard-to-reach, high-risk, or vulnerable populations and communities.

The ODP director’s remarks were followed by presentations and discussions including solution-oriented research approaches to health that emphasize “what works, how and to whom.” Major elements of this approach include the need to address interventions holistically and dynamically, and attend to the interactions, feedback loops, non-linear changes and multiple causal pathways. Further presentations and discussions included multi-level and systems-based intervention implementation (i.e., multiple systems within the ecological framework [see Figure]), including case examples, cultural tailoring of interventions, and approaches for engaging community health workers in multi-level interventions. Multi-level intervention design and challenges, adaptive research designs and their applications in multi-level interventions, statistical approaches, and recruitment and retention issues were emphasized.  Perspectives from project coordinators and agency contacts regarding recruitment of study subjects and coordination of multi-level intervention activities were also discussed. In addition, perspectives were provided on implementation strategies for multi-level interventions that would engage community health workers to prevent cardiovascular disease (Community Preventive Services Task Force recommendations http://www.thecommunityguide.org/cvd/CHW.html). Each presenter provided recommendations for advancing multi-level and systems-oriented interventions in clinical and community trials.

Recommendations

Consistent with established conceptual models of causes and solutions, participants acknowledged that interventions should embrace complexity and systematically apply systems thinking (i.e., Implementation of interventions at multiple levels and systems).  Modeling and simulations are useful in understanding the complexities in multi-level interventions, but directly testing and replicating “solutions” or intervention models will likely be most informative. To move the field of multi-level and systems-oriented interventions forward, and with particular attention to vulnerable populations, the following recommendations were proposed and organized under six research headings:

  1.  Study design and statistical approaches

• Use group randomized trials (GRTs), individual randomized trials, stepped wedge, or regression discontinuity designs to provide the strongest evidence for causal inference.

  • Account for all sources of variation in design and analysis, some interactions within and between levels and the limited degrees of freedom in GRTs.

  • Randomize with stratification on the baseline value of the primary outcome and group size.

  • Blind evaluation staff to the extent possible.

• Distinguish fixed from random factors, and crossed from nested factors. Consider some interactions, within or between levels, especially given the complexities in multi-level interventions.
• Adopt multi-level adaptive designs such as SMART-AR http://www.ncbi.nlm.nih.gov/pubmed/25354029 to maximize the health benefits of study participants and use such designs, whenever appropriate, to maximize the efficiency and information gained from a multi-level intervention study.
• Consider other design and analytical approaches that can provide evidence for causal inference. Note that some approaches may rely on logic as much as statistics and may face more threats to causal inference.

  2.  Intervention implementation

• Use a clearly articulated conceptual model showing the relationships among major intervention components or a program planning model to guide the choice of the intervention components.
• Include persons indigenous to the area or setting in all phases of the planning and implementation process (e.g., Community health workers, local ministers, health care providers and other stakeholders).
• Develop or use validated and reliable measures to assess the effectiveness of interventions.
• Create linkages among the intervention levels and intentionally plan to optimize likelihood of synergy.
• Track intervention participation across settings and pre-specify criteria for success including minimal dose per setting.
• Develop contingency plans to enhance intervention retention; trusting and connected relationship with participants are key to re-engaging participants over time.
• Acknowledge the high level of heterogeneity in participant response to treatment by incorporating an adaptive intervention approach. Use empirical novel statistical tools to find the best tailoring variables for better characterizing patient's heterogeneity.
• Identify useful tailoring variables to design multi-level intervention studies for developing more targeted therapies or better dynamic treatment regimes.
• Use a pilot study (or a series of pilot studies) to determine the adaptation variables (i.e., intervention components being optimized), thresholds, and feasibility of delivery.
• Consider secondary analysis questions regarding the relationship between dose across multiple settings (including synergistic effects) and study outcomes.
• Consider ways to understand the effectiveness of specific intervention levels and components.

  3.  Cultural adaptations of interventions

• Refrain from making assumptions about vulnerable populations.
• Use community engagement and qualitative methods to unravel the complexities of health issues in vulnerable populations.
• Encourage multi-disciplinary approaches and target multiple levels of contextual influence to attain the maximum health outcomes.
• Tailor interventions for targeted populations and respective cultures. Note that intervention efficacy and sustainability in vulnerable populations hinge upon cultural appropriateness of the design and implementation.

  4.  Use of community health workers

• Engage community health workers (CHWs), (whose expertise may range from lay persons up to professionals such as nurses) to prevent cardiovascular disease on the basis of the CDC Task Force Community Preventive Services’ http://www.thecommunityguide.org/cvd/CHW.html finding of sufficient evidence of effectiveness in improving outcomes for (i) blood pressure (ii) cholesterol (iii) physical activity, and (iv) healthful eating habits in patients at increased risk for cardiovascular disease.  Delivery of interventions needs to attend to the prior training of CHWs and scope of practice.  
• Triage and amplify training of CHWs who show aptitude for working with population groups outside the clinical setting to enhance their role in outreach, enrollment, and culturally appropriate health education in groups with increased risk for cardiovascular disease. 
• Further triage and amplify training for those CHWs who show aptitude for work with community organizations in advocating for the mobilization and coordination of community resources.
• Further amplify training of selected CHWs for work with policy makers and academics to represent the needs and priorities of populations at cardiovascular disease risk in policy changes and in conducting community-based participatory research.

  5.  Recruitment and Retention

• Employ a participant centered approach to recruitment and retention by reaching out to participants in their homes, schools, worksites and other familiar venues.
• Provide a team approach to recruitment and encourage caring and trusting relationships that attend to participant needs.
• Prepare and review clear and informative progress reports on a regular basis.
• Provide detailed information at enrollment and ensure full understanding of the study prior to consenting subjects.
• Budget adequately for staffing, incentives and logistics.

  6.  Intervention staff training

• Train and certify staff and boost training periodically.
• Adopt the concept of “management by walking around” at implementation sites to understand and address challenges and tailor approaches that would enhance intervention implementation.
• Apply continuous quality improvement and data driven problem solving approaches to evaluate programs.

Working Group Chair and Planning Group

• June Stevens, PhD, Professor of Nutrition and Epidemiology, AICR/WCRF Distinguished Professor, University of North Carolina at Chapel Hill.
• Josephine Boyington, PhD, MPH, Division of Cardiovascular Sciences, NHLBI.
• Layla Esposito, PhD, Eunice Kennedy Shriver National Institute of Child Health and Human Development.
• Judy Hannah, PhD, National Institute on Aging, NIH.
• Christine Hunter, PhD, National Institute of Diabetes, Digestive and Kidney Diseases, NIH.
• Chitra Krishnamurti, PhD, Center for Translational and Implementation Science, NHLBI.
• Maria Lopez-Class, PhD, MPH, MS. Center for Translation Research and Implementation Science, NHLBI.
• Cheryl Nelson, MSPH, Division of Cardiovascular Sciences, NHLBI.
• Holly Nicastro, PhD, MPH, Division of Cardiovascular Sciences, NHLBI.
• Charlotte A. Pratt, PhD, MS, RD, Division of Cardiovascular Sciences, NHLBI (Contact)
• Marcel Salive, MD, National Institute on Aging, NIH.

Working Group Members

• Clifton Addison, PhD, Senior Research Scientist, Jackson Heart Study Community Outreach Center /Jackson State University.
• Jorge A. Banda, PhD, Postdoctoral Research Fellow, Stanford University.
• Donna Antoine-LaVigne, PhD, Principal Investigator, Jackson Heart Study Community Outreach Center/Jackson State University.
• Shari Barkin, MD, MHS, William K. Warren Family Foundation Chair in Medicine, Vanderbilt University School of Medicine.
• Elaine A. Borawski, PhD, Angela Bowen Williamson Professor of Community Nutrition, Case Western Reserve University, School of Medicine.
• Ying Kuen K. Cheung, PhD, Professor of Biostatistics, Mailman School of Public Health, Columbia University.
• Henry Feldman, PhD, Principal Biostatistician, Boston Children's Hospital, MA 
• Lawrence Green, DrPH, Professor, Department of Epidemiology and Biostatistics, School of Medicine University of California, San Francisco.
• Heather K. Hardin, PhD, RN, Postdoctoral Research Fellow, Case Western Reserve University
• Sarah Jones, MS, RD, LD, Case Western Reserve University.
• Leslie Lytle, PhD, Professor and Department Chair, Health Behavior, University of North Carolina, Chapel Hill.
• Donna Matheson, PhD, Senior Research Scientist, Stanford University.
• Michael Miner, PhD, Professor, Department of Family Medicine and Community Health University of Minnesota.
• Shirley M. Moore, RN, PhD, FAAN, Edward J. and Louise Mellen Professor of Nursing and Associate Dean for Research, Case Western Reserve University.
• Judith Ottoson, PhD, Professor of Health Education, San Francisco State University.
• Thomas Robinson, MD, Irving Schulman, MD Endowed Professor in Child Health and Professor of Pediatrics and of Medicine, Stanford University School of Medicine. 
• Nancy Sherwood, PhD, Senior Investigator and Director of Scientific Development HealthPartners Institute for Education and Research.
• Ana Solano Lopez, RN, Case Western Reserve University.
• Kimberly Truesdale, PhD, Research Assistant Professor, Department of Nutrition, University of North Carolina at Chapel Hill.
• Lu Wang, PhD, John G. Searle Assistant Professor of Biostatistics, University of Michigan
• Dianne Ward, PhD, Professor, Department of Nutrition, University of North Carolina at Chapel Hill.
• Dana Sampson, MS, Public Health Advisor, Office of the Assistant Secretary for Health, DHHS.

NIH Staff

• Sonia Arteaga, PhD, Division of Cardiovascular Sciences, NHLBI.
• Rachel Ballard-Barbash, MD, MPH, Office of Disease Prevention, NIH.
• Susan Czajkowski, PhD, Division of Cardiovascular Sciences, NHLBI.
• Michael Lauer, MD, Director, Division of Cardiovascular Sciences, NHLBI.
• David M. Murray, PhD, Director, Office of Disease Prevention, NIH.
• Emmanuel Peprah, PhD, Center for Translational and Implementation Science, NHLBI.

References

1. Charns MP, Foster MK, Alligood EC, Benzer JK, Burgess JF Jr, Li D, McIntosh NM, Burness A, Partin MR, Clauser SB. Multilevel interventions: measurement and measures. J Natl Cancer Inst Monogr. 2012;2012(44):67-77. doi: 10.1093/jncimonographs/lgs011.
2. Cheung YK, Chakraborty B, Davidson KW. Sequential multiple assignment trial (SMART) with adaptive randomization for quality improvement in depression treatment program.  Biometrics 2015; 71(2):450-459.
3. Clauser SB, Taplin SH, Foster MK, Fagan P, Kaluzny AD. Multilevel intervention research: lessons learned and pathways forward. J Natl Cancer Inst Monogr. 2012 2012(44):127-33. doi: 10.1093/jncimonographs/lgs019.

4. Coday M, Boutin-Foster C, Goldman Sher T, et al. Strategies for retaining study participants in behavioral intervention trials: retention experiences of the NIH Behavior Change Consortium. Ann Behav Med 2005;29 Suppl:55-65.

5. Dunn EC, Milliren CE, Evans CR, Subramanian SV, Richmond TK. Disentangling the relative influence of schools and neighborhoods on adolescents' risk for depressive  symptoms. Am J Public Health. 2015;105(4):732-40. doi: 10.2105/AJPH.2014.302374.
6. Kelder SH, Osganian SK, Feldman HA, Webber LS, Parcel GS, Leupker RV, Wu MC, Nader PR. Tracking of physical and physiological risk variables among ethnic subgroups from third to eighth grade: the Child and Adolescent Trial for Cardiovascular Health cohort study. Prev Med. 2002;34(3):324-33.
7. Murray, D.M.  Design and Analysis of Group-Randomized Trials.  New York:  Oxford University Press, 1998.
8. Murray DM, Pennell M, Rhoda D, Hade E, Paskett ED. Designing studies that would address the multilayered nature of health care. Journal of the National Cancer Institute Monographs, 2010, 40:90-96.
9. Pennell ML, Hade EM, Murray DM, Rhoda DA. Cutoff designs for community-based intervention studies. Statistics in Medicine 2011;30(15):1865-1882.
10. Perry CL Creating Health Behavior Change, Sage Publications, Thousand Oaks CA, 1999.
11. Pratt CA, Arteaga S, Loria C. Forging a future of better cardiovascular health: addressing childhood obesity. J Am Coll Cardiol. 2014, 4;63(4):369-71.doi: 10.1016/j.jacc.2013.07.088
12. Rhoda DA, Murray DM, Andridge RR, Pennell ML, Hade EM. Studies with staggered starts: multiple baseline designs and group-randomized trials. Am J Public Health 2011;101(11):2164-9.
13. Robinson TN, Kemby KM. The Center for Healthy Weight: an academic medical center response to childhood obesity. Int J Obes Suppl. 2012 Jul;2(Suppl1):S33-S38.
14. Schoeppe S, Oliver M,  Badland HM, et al. Recruitment and Retention of Children  Int J. Behav. Med 2014; 21:794-803
15. Stevens J, Taber DR, Murray DM, Ward DS. Advances and controversies in the design of obesity prevention trials. Obesity (Silver Spring). 2007;15(9):2163-70.
16. Story M, Kaphingst KM, Robinson-O'Brien R, Glanz K. Creating healthy food and  eating environments: policy and environmental approaches. Annu Rev Public Health, 2008;29:253-72.

17. Yancey AK, Ortega AN, Kumanyika SK. Effective recruitment and retention of minority research participants. Annu Rev Public Health 2006;27:1-28.