Translating Violence Prevention Programs from Research to Practice: SafERteens Implementation in an Urban Emergency Department

  • Patrick M. Carter
    Reprint Address: Patrick M. Carter, MD, Department of Emergency Medicine, University of Michigan, 2800 Plymouth Road, NCRC 10-G080, Ann Arbor, MI, 48109.
    University of Michigan Injury Prevention Center, Ann Arbor, Michigan

    Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan

    Youth Violence Prevention Center

    Department of Health Behavior/Health Education, University of Michigan School of Public Health, Ann Arbor, Michigan
    Search for articles by this author
  • Rebecca M. Cunningham
    University of Michigan Injury Prevention Center, Ann Arbor, Michigan

    Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan

    Youth Violence Prevention Center

    Department of Health Behavior/Health Education, University of Michigan School of Public Health, Ann Arbor, Michigan

    Department of Emergency Medicine, Hurley Medical Center, Flint, Michigan
    Search for articles by this author
  • Andria B. Eisman
    Youth Violence Prevention Center

    Department of Health Behavior/Health Education, University of Michigan School of Public Health, Ann Arbor, Michigan

    Division of Kinesiology, Health and Sport Studies, College of Education, Wayne State University, Detroit, Michigan
    Search for articles by this author
  • Ken Resnicow
    Department of Health Behavior/Health Education, University of Michigan School of Public Health, Ann Arbor, Michigan
    Search for articles by this author
  • Jessica S. Roche
    University of Michigan Injury Prevention Center, Ann Arbor, Michigan

    Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan
    Search for articles by this author
  • Jennifer Tang Cole
    University of Michigan Injury Prevention Center, Ann Arbor, Michigan

    Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan
    Search for articles by this author
  • Jason Goldstick
    University of Michigan Injury Prevention Center, Ann Arbor, Michigan

    Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan
    Search for articles by this author
  • Amy M. Kilbourne
    Health Services Research and Development Service, Veterans Health Administration, U.S. Department of Veterans Affairs, Washington, DC

    Department of Learning Health Sciences
    Search for articles by this author
  • Maureen A. Walton
    University of Michigan Injury Prevention Center, Ann Arbor, Michigan

    Addiction Center, Department of Psychiatry, University of Michigan Medical School, Ann Arbor, Michigan
    Search for articles by this author



      Youth violence is a leading cause of adolescent mortality, underscoring the need to integrate evidence-based violence prevention programs into routine emergency department (ED) care.


      To examine the translation of the SafERteens program into clinical care.


      Hospital staff provided input on implementation facilitators/barriers to inform toolkit development. Implementation was piloted in a four-arm effectiveness-implementation trial, with youth (ages 14–18 years) screening positive for past 3-month aggression randomized to either SafERteens (delivered remotely or in-person) or enhanced usual care (EUC; remote or in-person), with follow-up at post-test and 3 months. During maintenance, ED staff continued in-person SafERteens delivery and external facilitation was provided. Outcomes were measured using the RE-AIM implementation framework.


      SafERteens completion rates were 77.6% (52/67) for remote and 49.1% (27/55) for in-person delivery. In addition to high acceptability ratings (e.g., helpfulness), post-test data demonstrated increased self-efficacy to avoid fighting among patients receiving remote (incidence rate ratio [IRR] 1.22, 95% confidence interval [CI] 1.09–1.36) and in-person (IRR 1.23, 95% CI 1.12–1.36) SafERteens, as well as decreased pro-violence attitudes among patients receiving remote (IRR 0.83, 95% CI 0.75–0.91) and in-person (IRR 0.87, 95% CI 0.77–0.99) SafERteens when compared with their respective EUC groups. At 3 months, youth receiving remote SafERteens reported less non-partner aggression (IRR 0.52, 95% CI 0.31–0.87, Cohen's d −0.39) and violence consequences (IRR 0.47, 95% CI 0.22–1.00, Cohen's d −0.49) compared with remote EUC; no differences were noted for in-person SafERteens delivery. Barriers to implementation maintenance included limited staff availability and a lack of reimbursement codes.


      Implementing behavioral interventions such as SafERteens into routine ED care is feasible using remote delivery. Policymakers should consider reimbursement for violence prevention services to sustain long-term implementation.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Emergency Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Cunningham RM
        • Walton MA
        • Carter PM.
        The major causes of death in children and adolescents in the United States.
        N Engl J Med. 2018; 379: 2468-2475
      1. Centers for Disease Control and Prevention (CDC). Web-based injury statistics query and reporting system (WISQARS). 2020. Available at: Accessed December 1, 2020.

      2. Centers for Disease Control and Prevention (CDC). 1991–2019 High school youth risk behavior survey data. Available at: Accessed December 1, 2020.

        • Cunningham RM
        • Carter PM
        • Ranney M
        • et al.
        Violent reinjury and mortality among youth seeking emergency department care for assault-related injury: a 2-year prospective cohort study.
        JAMA Pediatr. 2015; 169: 63-70
        • Rowhani-Rahbar A
        • Zatzick D
        • Wang J
        • et al.
        Firearm-related hospitalization and risk for subsequent violent injury, death, or crime perpetration: a cohort study.
        Ann Intern Med. 2015; 162: 492-500
        • DiScala C
        • Sege R.
        Outcomes in children and young adults who are hospitalized for firearms-related injuries.
        Pediatrics. 2004; 113: 1306-1312
        • Carter PM
        • Walton MA
        • Roehler DR
        • et al.
        Firearm violence among high-risk emergency department youth after an assault injury.
        Pediatrics. 2015; 135: 805-815
        • Walton M
        • Epstein-Ngo Q
        • Carter P
        • et al.
        Marijuana use trajectories among drug-using youth presenting to an urban emergency department: violence and social influences.
        Drug Alcohol Depend. 2017; 173: 117-125
        • Garbarino J
        • Bradshaw CP
        • Vorrasi JA.
        Mitigating the effects of gun violence on children and youth.
        Future Child. 2002; 12: 73-86
        • Carter PM
        • Dora-Laskey AD
        • Goldstick JE
        • et al.
        Arrests among high-risk youth following emergency department treatment for an assault injury.
        Am J Prev Med. 2018; 55: 812-821
        • Boynton-Jarrett R
        • Ryan LM
        • Berkman LF
        • Wright RJ.
        Cumulative violence exposure and self-rated health: longitudinal study of adolescents in the United States.
        Pediatrics. 2008; 122: 961-970
        • Stoddard SA
        • Whiteside L
        • Zimmerman MA
        • Cunningham RM
        • Chermack ST
        • Walton MA.
        The relationship between cumulative risk and promotive factors and violent behavior among urban adolescents.
        Am J Community Psychol. 2012; 51: 57-65
        • Pailler ME
        • Kassam-Adams N
        • Datner EM
        • Fein JA.
        Depression, acute stress and behavioral risk factors in violently injured adolescents.
        Gen Hosp Psychiatry. 2007; 29: 357-363
        • Wells K
        • Klap R
        • Koike A
        • Sherbourne C.
        Ethnic disparities in unmet need for alcoholism, drug abuse, and mental health care.
        Am J Psychiatry. 2001; 158: 2027-2032
        • Peterson RD
        • Krivo LJ.
        Macrostructural analyses of race, ethnicity, and violent crime: recent lessons and new directions for research.
        Annu Rev Sociol. 2005; 31: 331-356
        • Peterson RD
        • Krivo LJ.
        Divergent social worlds: neighborhood crime and the racial-spatial divide.
        Russell Sage Foundation, New York2010
        • Phillips JA.
        Variation in African-American homicide rates: an assessment of potential explanations.
        Criminology. 1997; 35: 527-560
        • Corso PS
        • Mercy JA
        • Simon TR
        • Finkelstein EA
        • Miller TR.
        Medical costs and productivity losses due to interpersonal and self-directed violence in the United States.
        Am J Prev Med. 2007; 32: 474-482
        • Carter PM
        • Walton MA
        • Roehler DR
        • et al.
        Firearm violence among high-risk emergency department youth after an assault injury.
        Pediatrics. 2015; 135: 805-815
        • Cunningham R
        • Knox L
        • Fein J
        • et al.
        Before and after the trauma bay: the prevention of violent injury among youth.
        Ann Emerg Med. 2009; 53: 490-500
      3. NIH State-of-the-Science Conference Statement on preventing violence and related health-risking social behaviors in adolescents.
        NIH Consens State Sci Statements. 2004; 21: 1-34
        • Committee on Injury, Violence, and Poison Prevention
        Policy statement–Role of the pediatrician in youth violence prevention.
        Pediatrics. 2009; 124: 393-402
        • Muelleman RL
        • Reuwer J
        • Sanson TG
        • et al.
        An emergency medicine approach to violence throughout the life cycle. SAEM Public Health and Education Committee.
        Acad Emerg Med. 1996; 3: 708-715
        • Fein JA
        • Ginsburg KR
        • McGrath ME
        • Shofer FS
        • Flamma JC
        • Datner EM.
        Violence prevention in the emergency department: clinician attitudes and limitations.
        Arch Pediatr Adolesc Med. 2000; 154: 495-498
        • National Research Council
        Injury in America: a continuing public health problem.
        National Academy Press, Washington, DC1985
        • Walton MA
        • Chermack ST
        • Shope JT
        • et al.
        Effects of a brief intervention for reducing violence and alcohol misuse among adolescents.
        JAMA. 2010; 304: 527-535
        • Cunningham RM
        • Chermack ST
        • Zimmerman MA
        • et al.
        Brief motivational interviewing intervention for peer violence and alcohol use in teens: one-year follow-up.
        Pediatrics. 2012; 129: 1083-1090
        • Cunningham RM
        • Whiteside LK
        • Chermack ST
        • et al.
        Dating violence: outcomes following a brief motivational interviewing intervention among at-risk adolescents in an urban emergency department.
        Acad Emerg Med. 2013; 20: 562-569
        • Ranney ML
        • Goldstick J
        • Eisman A
        • Carter PM
        • Walton M
        • Cunningham RM.
        Effects of a brief ED-based alcohol and violence intervention on depressive symptoms.
        Gen Hosp Psychiatry. 2017; 46: 44-48
        • Carter PM
        • Walton MA
        • Zimmerman MA
        • Chermack ST
        • Roche JS
        • Cunningham RM.
        Efficacy of a universal brief intervention for violence among urban emergency department youth.
        Acad Emerg Med. 2016; 23: 1061-1070
        • Sharp AL
        • Prosser LA
        • Walton MA
        • et al.
        Cost analysis of youth violence prevention.
        Pediatrics. 2014; 133: 448-453
        • Peek-Asa C
        • Butcher B
        • Cavanaugh JE.
        Cost of hospitalization for firearm injuries by firearm type, intent, and payer in the United States.
        Inj Epidemiol. 2017; 4: 20
        • Green L
        • Nasser M.
        Furthering dissemination and implementation research: the need for more attention to external validity.
        in: Brownson RC Colditz GA Proctor EK Dissemination and implementation research in health: translating science to practice. Oxford University Press, Oxford; New York2012: 301-316 (XX–XX)
        • Proctor EK
        • Landsverk J
        • Aarons G
        • Chambers D
        • Glisson C
        • Mittman B.
        Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges.
        Adm Policy Ment Health. 2009; 36: 24-34
        • Neumann MS
        • Sogolow ED.
        Replicating effective programs: HIV/AIDS prevention technology transfer.
        AIDS Educ Prev. 2000; 12: 35-48
        • Kilbourne AM
        • Neumann MS
        • Pincus HA
        • Bauer MS
        • Stall R.
        Implementing evidence-based interventions in health care: application of the replicating effective programs framework.
        Implement Sci. 2007; 2: 42
        • Leeman J
        • Birken SA
        • Powell BJ
        • Rohweder C
        • Shea CM.
        Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice.
        Implement Sci. 2017; 12: 125
        • Beidas RS
        • Edmunds JM
        • Marcus SC
        • Kendall PC.
        Training and consultation to promote implementation of an empirically supported treatment: a randomized trial.
        Psychiatr Serv. 2012; 63: 660-665
        • Kilbourne AM
        • Almirall D
        • Eisenberg D
        • et al.
        Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program.
        Implement Sci. 2014; 9: 132
        • Stetler CB
        • Legro MW
        • Rycroft-Malone J
        • et al.
        Role of "external facilitation" in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration.
        Implement Sci. 2006; 1: 23
        • Kitson A
        • Harvey G
        • McCormack B.
        Enabling the implementation of evidence based practice: a conceptual framework.
        Qual Health Care. 1998; 7: 149-158
        • Glasgow RE
        • Vogt TM
        • Boles SM.
        Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
        Am J Public Health. 1999; 89: 1322-1327
        • Glasgow RE
        • Harden SM
        • Gaglio B
        • et al.
        RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review.
        Front Public Health. 2019; 7: 64
      4. Federal Bureau of Investigation. Uniform crime report. Available at: Accessed December 20, 2020.

        • U.S. Census Bureau
        State and county quick facts.
        U.S. Postal Service, 2020 (Available at:) (Accessed December 20)
        • Feldstein AC
        • Glasgow RE.
        A Practical, Robust Implementation and Sustainability Model (PRISM) for integrating research findings into practice.
        Jt Comm J Qual Patient Saf. 2008; 34: 228-243
        • Squires JE
        • Aloisio LD
        • Grimshaw JM
        • et al.
        Attributes of context relevant to healthcare professionals' use of research evidence in clinical practice: a multi-study analysis.
        Implement Sci. 2019; 14: 52
        • Cunningham R
        • Walton MA
        • Weber JE
        • et al.
        One-year medical outcomes and emergency department recidivism after emergency department observation for cocaine-associated chest pain.
        Ann Emerg Med. 2009; 53: 310-320
        • Fjeldsoe BS
        • Marshall AL
        • Miller YD.
        Behavior change interventions delivered by mobile telephone short-message service.
        Am J Prev Med. 2009; 36: 165-173
        • Ranney ML
        • Freeman JR
        • Connell G
        • et al.
        A depression prevention intervention for adolescents in the emergency department.
        J Adolesc Health. 2016; 59: 401-410
        • Cole-Lewis H
        • Kershaw T.
        Text messaging as a tool for behavior change in disease prevention and management.
        Epidemiol Rev. 2010; 32: 56-69
        • Straus MA.
        Conflict tactics scales.
        in: Jackson NA Encyclopedia of domestic violence. Taylor & Francis Group, New York: Routledge2007: 824
        • Straus MA
        • Hamby SL
        • Boney-McCoy S
        • Sugarman DB.
        The revised conflict tactics scales (CTS2) development and preliminary psychometric data.
        J Fam Issues. 1996; 17: 283-316
        • Walton MA
        • Cunningham RM
        • Chermack ST
        • Maio R
        • Blow FC
        • Weber J.
        Correlates of violence history among injured patients in an urban emergency department: gender, substance use, and depression.
        J Addict Dis. 2007; 26: 61-75
        • Cunningham RM
        • Ranney M
        • Newton M
        • Woodhull W
        • Zimmerman M
        • Walton MA.
        Characteristics of youth seeking emergency care for assault injuries.
        Pediatrics. 2014; 133: e96-105
        • Rollnick S
        • Miller W
        • Butler C.
        Motivational interviewing in health care: helping patients change behavior.
        Guilford, New York2007
        • Resnicow K
        • Rollnick S.
        • W.M. Cox & E Klinger
        Motivational interviewing in health promotion and behavioral medicine.
        Handbook of motivational counseling: goal-based approaches to assessment and intervention with addiction and other problems. John Wiley & Sons, Ltd, Hoboken, NJ2004: 457-476 (XX)
        • Moyers T
        • Martin T
        • Manuel J
        • Miller W
        • Ernst D.
        Revised global scales: motivational interviewing treatment integrity 3.0 (MITI 3.0).
        University of New Mexico, Center on Alcoholism, Substance Abuse and Addictions (CASAA), Albuquerque2007
        • Kauth MR
        • Sullivan G
        • Blevins D
        • et al.
        Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study.
        Implement Sci. 2010; 5: 75
        • Lessard S
        • Bareil C
        • Lalonde L
        • et al.
        External facilitators and interprofessional facilitation teams: a qualitative study of their roles in supporting practice change.
        Implement Sci. 2015; 11: 97
      5. Harris KM, Halpern CT, Whitsel E, et al. The national longitudinal study of adolescent to adult health: research design. 2009. Available at: Accessed December 2020.

        • WHO ASSIST Working Group
        The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): development, reliability and feasibility.
        Addiction. 2002; 97: 1183-1194
        • Humeniuk R
        • Ali R
        • Babor TF
        • et al.
        Validation of the Alcohol, Smoking And Substance Involvement Screening Test (ASSIST).
        Addiction. 2008; 103: 1039-1047
        • National Institute on Drug Abuse
        NIDA-Modified ASSIST—Prescreen V1.0.
        National Institute of Health, 2009 (Available at:) (Accessed December 2020)
        • Smith Jr, GR
        • Babor TF
        • Burnam MA
        • Mosley CL
        • Rost KM
        • Burns B.
        Substance abuse outcomes module: user's manual.
        University of Arkansas for Medical Sciences, Little Rock, AR1996
        • Bosworth K
        • Espelage D.
        Teen conflict survey.
        Center for Adolescent Studies, Indiana University, Bloomington, IN1995
        • Funk JB
        • Elliott R
        • Urman ML
        • Flores GT
        • Mock RM.
        The attitudes towards violence scale: a measure for adolescents.
        J Interpers Violence. 1999; 14: 1123-1136
        • Curran GM
        • Bauer M
        • Mittman B
        • Pyne JM
        • Stetler C.
        Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact.
        Med Care. 2012; 50: 217-226
        • Elliott DS
        • Mihalic S.
        Issues in disseminating and replicating effective prevention programs.
        Prev Sci. 2004; 5: 47-53
        • Meyers DC
        • Durlak JA
        • Wandersman A.
        The quality implementation framework: a synthesis of critical steps in the implementation process.
        Am J Community Psychol. 2012; 50: 462-480
        • Hempel S
        • O'Hanlon C
        • Lim YW
        • Danz M
        • Larkin J
        • Rubenstein L.
        Spread tools: a systematic review of components, uptake, and effectiveness of quality improvement toolkits.
        Implement Sci. 2019; 14: 83
        • Liu N
        • Huang R
        • Baldacchino T
        • et al.
        Telehealth for noncritical patients with chronic diseases during the COVID-19 pandemic.
        J Med Internet Res. 2020; 22: e19493
        • Zhou X
        • Snoswell CL
        • Harding LE
        • et al.
        The role of telehealth in reducing the mental health burden from COVID-19.
        Telemed J E Health. 2020; 26: 377-379
        • Kalin ML
        • Garlow SJ
        • Thertus K
        • Peterson MJ.
        Rapid implementation of telehealth in hospital psychiatry in response to COVID-19.
        Am J Psychiatry. 2020; 177: 636-637
        • Hirko KA
        • Kerver JM
        • Ford S
        • et al.
        Telehealth in response to the COVID-19 pandemic: implications for rural health disparities.
        J Am Med Inform Assoc. 2020; 27: 1816-1818
        • Fanburg JD
        • Walzman JJ.
        Telehealth and the law: the challenge of reimbursement.
        Contemp Ob Gyn. 2018; 63: 59-61
        • Bonar EE
        • Cunningham RM
        • Collins RL
        • et al.
        Feasibility and acceptability of text messaging to assess daily substance use and sexual behaviors among urban emerging adults.
        Addict Res Theory. 2018; 26: 103-113
        • Head KJ
        • Noar SM
        • Iannarino NT
        • Harrington NG.
        Efficacy of text messaging-based interventions for health promotion: a meta-analysis.
        Soc Sci Med. 2013; 97: 41-48
        • Rhodes KV
        • Drum M
        • Anliker E
        • Frankel RM
        • Howes DS
        • Levinson W.
        Lowering the threshold for discussions of domestic violence: a randomized controlled trial of computer screening.
        Arch Intern Med. 2006; 166: 1107-1114
        • Brener ND
        • Billy JO
        • Grady WR.
        Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: evidence from the scientific literature.
        J Adolesc Health. 2003; 33: 436-457
        • Powers EM
        • Shiffman RN
        • Melnick ER
        • Hickner A
        • Sharifi M.
        Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review.
        J Am Med Inform Assoc. 2018; 25: 1556-1566
        • Todahl J
        • Walters E.
        Universal screening for intimate partner violence: a systematic review.
        J Marital Fam Ther. 2011; 37: 355-369
        • Williams JR
        • Halstead V
        • Salani D
        • Koermer N.
        An exploration of screening protocols for intimate partner violence in healthcare facilities: a qualitative study.
        J Clin Nurs. 2017; 26: 2192-2201
      6. Professional Assisted Cessation Therapy. Reimbursement for smoking cessation therapy. Available at: Accessed December 20, 2020.

      7. Centers for Medicare and Medicaid Services. Decision memo for screening and behavioral counseling interventions in primary care to reduce alcohol misuse. Available at: Accessed December 2020.

      8. Substance Abuse and Mental Health Service Administration (SAMHSA). Coding for screening and brief intervention reimbursement. Available at: Accessed December 2020.

        • Gentilello LM.
        Alcohol and injury: American College of Surgeons Committee on trauma requirements for trauma center intervention.
        J Trauma Acute Care Surg. 2007; 62: S44-S45
        • Joint Commission on Accreditation of Healthcare Organizations
        Accreditation manual for Hospitals. Vol 1. Standards.
        Joint Commission of Accreditation of Healthcare Organizations, Oakbrook Terrace, IL1992
      9. Leong FT Austin JT The psychology research handbook: a guide for graduate students and research assistants. Sage, Thousand Oaks, CA2006
        • Thornberry TP
        • Krohn MD.
        The self-report method of measuring delinquency and crime.
        in: Duffee D Measurement and analysis of crime and justice: criminal justice 2000. US Department of Justice, Office of Justice Programs, Washington, DC2000: 33-83
        • Harrison LD
        • Martin SS
        • Enev T
        • Harrington D.
        Comparing drug testing and self-report of drug use among youths and young adults in the general population. Department of Health and Human Services Publication No. SMA 07-4249, Methodology Series M-7.
        Substance Abuse and Mental Health Services Administration, Office of Applied Studies, Rockville, MD2007
        • Ritchie MJ
        • Kirchner JE
        • Parker LE
        • et al.
        Evaluation of an implementation facilitation strategy for settings that experience significant implementation barriers.
        Implement Sci. 2015; 10: A46
        • Rothman KJ.
        No adjustments are needed for multiple comparisons.
        Epidemiology. 1990; 1: 43-46