Top

Published in:

01-10-2012

Design and Implementation of an Integrated, Continuous Evaluation, and Quality Improvement System for a State-Based Home-Visiting Program

Authors: Bridget K. McCabe, Dru Potash, Ellen Omohundro, Cathy R. Taylor

Published in: Maternal and Child Health Journal | Issue 7/2012

Abstract

To describe the design and implementation of an evaluation system to facilitate continuous quality improvement (CQI) and scientific evaluation in a statewide home visiting program, and to provide a summary of the system’s progress in meeting intended outputs and short-term outcomes. Help Us Grow Successfully (HUGS) is a statewide home visiting program that provides services to at-risk pregnant/post-partum women, children (0–5 years), and their families. The program goals are to improve parenting skills and connect families to needed services and thus improve the health of the service population. The evaluation system is designed to: (1) integrate evaluation into daily workflow; (2) utilize standardized screening and evaluation tools; (3) facilitate a culture of CQI in program management; and, (4) facilitate scientifically rigorous evaluations. The review of the system’s design and implementation occurred through a formative evaluation process (reach, dose, and fidelity). Data was collected through electronic and paper surveys, administrative data, and notes from management meetings, and medical chart review. In the design phase, four process and forty outcome measures were selected and are tracked using standardized screening and monitoring tools. During implementation, the reach and dose of training were adequate to successfully launch the evaluation/CQI system. All staff (n = 165) use the system for management of families; the supervisors (n = 18) use the system to track routine program activities. Data quality and availability is sufficient to support periodic program reviews at the region and state level. In the first 7 months, the HUGS evaluation system tracked 3,794 families (7,937 individuals). System use and acceptance is high. A successful implementation of a structured evaluation system with a strong CQI component is feasible in an existing, large statewide program. The evaluation/CQI system is an effective mechanism to drive modest change in management of the program.

Shown in the companion paper.

America’s health rankings ^®—2010 edition. [website] (2010). Accessed June 10, 2011. 2010 Edition. Available from: http://www.americashealthrankings.org/yearcompare/2008/2009/TN.aspx.

KIDS COUNT data center. [website] (2010). Accessed September 22, 2010. Available from: http://www.datacenter.kidscount.org and http://www.aecf.org/cgi-bin/cliks.cgi.

Olds, D. L., Henderson, C. R., Jr, Tatelbaum, R., & Chamberlin, R. (1986). Improving the delivery of prenatal care and outcomes of pregnancy: A randomized trial of nurse home visitation. Pediatrics, 77(1), 16–28.PubMed

Donovan, E. F., Ammerman, R. T., Besl, J., Atherton, H., Khoury, J. C., Altaye, M., et al. (2007). Intensive home visiting is associated with decreased risk of infant death. Pediatrics, 119(6), 1145–1151. doi:10.1542/peds.2006-2411.PubMedCrossRef

Olds, D. L., Eckenrode, J., Henderson, C. R., Jr, Kitzman, H., Powers, J., Cole, R., et al. (1997). Long-term effects of home visitation on maternal life course and child abuse and neglect. Fifteen-year follow-up of a randomized trial. JAMA, 278(8), 637–643.PubMedCrossRef

Eckenrode, J., Ganzel, B., Henderson, C. R., Jr, Smith, E., Olds, D. L., Powers, J., et al. (2000). Preventing child abuse and neglect with a program of nurse home visitation: The limiting effects of domestic violence. JAMA, 284(11), 1385–1391.PubMedCrossRef

DuMont, K., Mitchell-Herzfeld, S., Greene, R., Lee, E., Lowenfels, A., Rodriguez, M., et al. (2008). Healthy families New York (HFNY) randomized trial: Effects on early child abuse and neglect. Child Abuse and Neglect, 32(3), 295–315. doi:10.1016/j.chiabu.2007.07.007.PubMedCrossRef

Olds, D. L., Kitzman, H., Cole, R., Robinson, J., Sidora, K., Luckey, D. W., et al. (2004). Effects of nurse home-visiting on maternal life course and child development: Age 6 follow-up results of a randomized trial. Pediatrics, 114(6), 1550–1559. doi:10.1542/peds.2004-0962.PubMedCrossRef

Olds, D. L., Kitzman, H., Hanks, C., Cole, R., Anson, E., Sidora-Arcoleo, K., et al. (2007). Effects of nurse home visiting on maternal and child functioning: Age-9 follow-up of a randomized trial. Pediatrics, 120(4), e832–e845. doi:10.1542/peds.2006-2111.PubMedCrossRef

10.

Barnet, B., Liu, J., DeVoe, M., Alperovitz-Bichell, K., & Duggan, A. K. (2007). Home visiting for adolescent mothers: Effects on parenting, maternal life course, and primary care linkage. Annals of Family Medicine, 5(3), 224–232. doi:10.1370/afm.629.PubMedCrossRef

11.

El-Kamary, S. S., Higman, S. M., Fuddy, L., McFarlane, E., Sia, C., & Duggan, A. K. (2004). Hawaii’s healthy start home visiting program: Determinants and impact of rapid repeat birth. Pediatrics, 114(3), e317–e326. doi:10.1542/peds.2004-0618.PubMedCrossRef

12.

Lee, E., Mitchell-Herzfeld, S. D., Lowenfels, A. A., Greene, R., Dorabawila, V., & DuMont, K. A. (2009). Reducing low birth weight through home visitation: A randomized controlled trial. American Journal of Preventative Medicine, 36(2), 154–160. doi:10.1016/j.amepre.2008.09.029.CrossRef

13.

Kitzman, H., Olds, D. L., Sidora, K., Henderson, C. R., Jr, Hanks, C., Cole, R., et al. (2000). Enduring effects of nurse home visitation on maternal life course: A 3-year follow-up of a randomized trial. JAMA, 283(15), 1983–1989.PubMedCrossRef

14.

Olds, D., Henderson, C. R., Jr, Cole, R., Eckenrode, J., Kitzman, H., Luckey, D., et al. (1998). Long-term effects of nurse home visitation on children’s criminal and antisocial behavior: 15-year follow-up of a randomized controlled trial. JAMA, 280(14), 1238–1244.PubMedCrossRef

15.

Stoltzfus, E., & Lynch, K. (2009) Home visitation for families with young children. In CRS Report for Congress (pp. 1–55). Congressional Research Service. Accessed September 01, 2010. Available from: www.crs.gov.

16.

Sweet, M. A., & Appelbaum, M. I. (2004). Is home visting an effective strategy? A meta-analytic review of home visiting programs for families with young children. Child Development, 75(5), 1435–1456.PubMedCrossRef

17.

Johnson, K. (2009) State-based home visting: Strengthening programs through state leadership, National Center for Children in Poverty, Mailman School of Public Health, Columbia University: New York, NY. Accessed October 03, 2010. Available from: http://www.nccp.org/publications/pdf/text_862.pdf.

18.

Duggan, A., Windham, A., McFarlane, E., Fuddy, L., Rohde, C., Buchbinder, S., et al. (2000). Hawaii’s healthy start program of home visiting for at-risk families: Evaluation of family identification, family engagement, and service delivery. Pediatrics, 105(1 Pt 3), 250–259.PubMed

19.

Weiss, H., & Klein, L. G. (2006) Changing the conversation about home visiting: scaling up with quality. Harvard Family Research Project, Harvard Graduate School of Education: Cambridge, MA. Accessed September 22, 2010. Available from: http://www.hfrp.org/publications-resources/browse-our-publications/changing-the-conversation-about-home-visiting-scaling-up-with-quality.

20.

The Coalition for Evidence-Based Policy. (2006). Which study designs can produce rigorous evidence of program effectiveness? A brief overview. In Coalition for evidence-based policy working paper. Washington, DC: Coalition for Evidence-Based Policy. Available from http://evidencebasedpolicy.org/docs/RCTs_first_then_match_c-g_studies-FINAL.pdf. Accessed October 04, 2010.

21.

Shortell, S. M., O'Brien, J. L., Carman, J. M., Foster, R. W., Hughes, E. F., Boerstler, H., & O'Connor, E. J. (1995). Assessing the impact of continuous quality improvement/total quality management: Concept versus implementation. Health Services Research, 30(2), 377–401.PubMed

22.

Batalden, P., Smith, S. D., Bovender, J. O., & Hardison, C. D. (1989). Quality improvement: The role and application of research methods. The Journal of Health Administration Education, 7(3), 577–583.PubMed

23.

Berwick, D. M., Godfrey, A. B., & Roessner, J. (1990). Curing health care: New strategies for quality improvement (p. 336). San Francisco, CA: Jossey-Bass.

24.

Deming, W. E. (2000). Out of the crisis (First MIT Press edition ed., p. 507). Cambridge, MA: MIT Center for Advanced Educational Services.

25.

Florida State University. Center for Prevention, Early Intervention Policy. (2004). Partners for a healthy baby (2nd ed.). Tallahassee, FL: Florida State University, Center for Prevention and Early Intervention Policy.

26.

Funds for in-home visitation programs - Emphasis on evidence-based programs - Report on findings, Tennessee Annotated Code. §§ 68-1-125 (2008).

27.

Tennessee Department of Health. (2010). Tennessee home visiting annual report: July 1, 2008–June 30, 2009. Nashville, TN: State of Tennessee. Available from http://health.state.tn.us/Downloads/Home%20Visiting%20Report%20FY%202009.FINAL.pdf. Accessed September 23, 2010.

28.

Hargreave, M. (2010). Evaluating system change: A planning guide, methods Brief. Princeton, NJ: Mathematica Policy Research, Inc. Accessed September 29, 2010. Available from: http://www.mathematica-mpr.com/publications/PDFs/health/eval_system_change_methodbr.pdf.

29.

Langley, G. J., Moen, R. D., Nolan, K. M., Nolan, T. W., Norman, C. L., & Provost, L. P. (2009). The improvement guide: A practical approach to enhancing organizational performance (2nd ed.). San Francisco, CA: Jossey-Bass Publishers.

30.

The breakthrough series: IHI’s collaborative model for achieving breakthrough improvement. IHI innovation series white paper. Boston, MA: Institute for Healthcare Improvement, 2003. (Available on http://www.ihi.org/knowledge/Pages/IHIWhitePapers/TheBreakthroughSeriesIHIsCollaborativeModelforAchievingBreakthroughImprovement.aspx).

31.

Centers for Disease Control and Prevention (CDC). Pregnancy risks assessment monitoring system. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention. Accessed October 15, 2010. Available from: http://www.cdc.gov/prams/index.htm and http://www.cdc.gov/prams/Questionnaire.htm.

32.

Centers for Disease Control and Prevention (CDC). Behavioral risk factor surveillance system (BRFSS) Survey Questionnaire. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Accessed October 15, 2010. Available from: http://www.cdc.gov/brfss/questionnaires/index.htm.

33.

Centers for Disease Control and Prevention (CDC). 2009 youth risk behavior survey. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Accessed October 20, 2010. Available from: www.cdc.gov/yrbss.

34.

Squires, J., Potter, L., & Bricker, D. (1999). Ages & stages questionnaires ^® user's guide for the ages and stages questionnaire (2nd ed., pp. 1–175). Baltimore, MD: Paul H. Brookes Publishing Co., Inc.

35.

Centers for Disease Control and Prevention (CDC). Behavioral risk factor surveillance system survey data. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Accessed October 15, 2010. Available from: http://www.cdc.gov/brfss/technical_infodata/surveydata.htm.

36.

Tennessee Department of Health (TDOH), Vital records system. Nashville, TN: State of Tennessee, Tennessee Department of Health.

37.

Tennessee Department of Health (TDOH) Communicable and Environmental Disease Services (CEDS). (2009). Immunization status survey of 24-Month-old children in Tennessee. Nashville, TN: State of Tennessee. Available from http://health.state.tn.us/Ceds/PDFs/ImmunizationSurvey2009.pdf. Accessed October 20, 2010.

38.

McCabe, B. K., Taylor, C. R., & Cooper, S. R. (2009). Translating clinical quality improvement success to public health. In R. Bialek, G. L. Duffy, & J. W. Moran (Eds.), The public health quality improvement handbook (pp. 385–396). Milwaukee: American Society for Quality, Quality Press.

39.

Dunn, C., Thomas, C., Ward, D., Pegram, L., Webber, K., & Cullitan, C. (2006). Design and implementation of a nutrition and physical activity curriculum for child care settings. Preventing Chronic Disease, 3(2), A58.PubMed

40.

Orfaly, R. A., Frances, J. C., Campbell, P., Whittemore, B., Joly, B., & Koh, H. (2005, November) Train-the-trainer as an educational model in public health preparedness. Journal of Public Health Management and Practice, (Suppl), S123–S127.

41.

Segre, L. S., Brock, R. L., O’Hara, M. W., Gorman, L. L., & Engeldinger, J. (2011). Disseminating perinatal depression screening as a public health initiative: A train-the-trainer approach. Maternal and Child Health Journal, 15(6), 814–821. doi:10.1007/s10995-010-0644-1.

42.

Boonyasai, R. T., & Windish, D. M., Chakraborti, C., Feldman, L. S., Rubin, H. R., & Bass, E. B. (2007). Effectiveness of teaching quality improvement to clinicians: A systematic review. JAMA, 298(9), 1023–1037. doi:10.1001/jama.298.9.1023.PubMedCrossRef

43.

Linnan, L., & Steckler, A. (Eds.). (2002). Process evaluation for public health interventions and research: An overview (pp. 1–23). San Francisco, CA: Jossey-Bass.

44.

Healthy people 2010. With understanding and improving health and objectives for improving health (Volume I and II), U.S. Department of Health and Human Services (Ed.), Nov. 2000. Washington, DC: U.S. Government Printing Office.

45.

U.S. Department of Health and Human Services. Office of Disease Prevention and Health Promotion. Healthy people 2020. (cited August 15, 2011). Available from: http://www.healthypeople.gov.

46.

Hodges, S., Isreal, N., Ferreira, K., & Mazza, J. (2007). System implementation issue brief #2 – Critical Factors in System of Care Implementation. Tampa: University of South Florida, Louis de la Parte Florida. Accessed September 02, 2010. Available from http://rtckids.fmhi.usf.edu.

47.

Hargreave, M. (2010). Evaluating system change: A planning guide, methods brief. Princeton, NJ: Mathematica Policy Research, Inc. Accessed September 29, 2010. Available from http://www.mathematica-mpr.com/publications/PDFs/health/eval_system_change_methodbr.pdf.

Title: Design and Implementation of an Integrated, Continuous Evaluation, and Quality Improvement System for a State-Based Home-Visiting Program
Authors: Bridget K. McCabe
Dru Potash
Ellen Omohundro
Cathy R. Taylor
Publication date: 01-10-2012
Publisher: Springer US
Published in: Maternal and Child Health Journal / Issue 7/2012
Print ISSN: 1092-7875
Electronic ISSN: 1573-6628
DOI: https://doi.org/10.1007/s10995-011-0906-6

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Design and Implementation of an Integrated, Continuous Evaluation, and Quality Improvement System for a State-Based Home-Visiting Program

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 7/2012

Attitudes Toward Birth Spacing Among Low-Income, Postpartum Women: A Qualitative Analysis

Seven-month Pilot of an Integrated, Continuous Evaluation, and Quality Improvement System for a State-Based Home-Visiting Program

Relationship Between Birth Spacing, Child Maltreatment, and Child Behavior and Development Outcomes Among At-Risk Families

Health-Related Quality of Life Among Pregnant Women With and Without Depression in Hubei, China

Relationship Between Prenatal Lead Exposure and Infant Blood Lead Levels

A Conceptual Framework for Future Research on Mode of Delivery