THE NATIONAL EVALUATION

Since 1993, The National Center for Mental Health has been conducting research for a comprehensive, ongoing evaluation of the children's mental health delivery system. The purpose of this evaluation is to guide funding and policy-making at the federal and state levels. Central Minnesota is a proud partner in this evaluation process, referred to as The National Evaluation. Along with over 125 communities across the nation, STARS is asking families receiving services in a system of care to participate in this evaluation.

Purpose of the National Evaluation

Evaluation Research Design

Research Questions

Components of the National Evaluation

• Cross Sectional Descriptive Study

• Longitudinal Child and Family Outcome Study

• Service Experience Study

Benefits of this Research

Administered by the Child, Adolescent and Family Branch (CAFB) of the Center for Mental Health Services (CMHS) of the Substance Abuse and Mental Health Services Administration (SAMHSA), the Comprehensive Community Mental Health Services for Children and Their Families Program (Child Mental Health Initiative) was launched in 1993 and is the largest child mental health services initiative ever funded. Funds from this multi-million dollar program have been allocated across 121 grant communities in the past 13 years. Federal funds have been awarded in 49 states and the District of Columbia, as well as two U.S. territories (Guam and Puerto Rico). The initiative is based on the system of care model of system development and service delivery.

The system of care philosophy asserts that to serve children with serious emotional disturbance, service delivery systems need to offer a wide array of accessible, community-based service options that center on children’s individual needs, include the family in treatment planning and delivery, and are provided in a culturally competent manner. An emphasis is placed on serving children in the least restrictive setting that is clinically appropriate. In addition, because many children with serious emotional disturbance use a variety of services and have contact with several child-serving agencies, service coordination and interagency collaboration are critical. The philosophy holds that if services are provided in this manner, outcomes for children and families will be better than can be achieved in more traditional service delivery systems.

The innovations represented by the system of care approach have been diffused throughout the country. Given the widespread adoption of this approach to serving children with serious emotional disturbance and their families, an examination of its impact is imperative. Evaluation activities for this CMHS grant program are mandated by the same federal law authorizing the services initiative (Public Health Service Act [Section 565(c)]).

Through a contract with CMHS, the evaluation is coordinated by the National Evaluation Team at Walter R. McDonald & Associates, Inc. (WRMA) and ORC Macro and their primary partners:

• Federation of Families for Children’s Mental Health

• University of South Florida Research and Training Center for Children’s Mental Health

• NASMHPD Research Institute and expert consultants

The National Evaluation Team and its partners have coordinated the national evaluation since the inception of the program in 1993. WRMA joined the team in 2005.

All national evaluation protocols and instruments have been approved by the U.S. Office of Management and Budget (OMB).

Return to Top

• Describe the children and families served by the CMHS-funded systems of care.

• Determine clinical and functional outcomes of children and families served by systems of care.

• Describe how children and families experience the service system and how they use services and supports (i.e., utilization patterns).

• Estimate the costs of serving children in systems of care.

• Examine the implementation and development of systems of care.

• Assess the effectiveness of an evidence-based treatment within a system of care.

• Support technical assistance activities to help CMHS best meet program goals.

Return to Top

This evaluation is the most comprehensive study to date of a children’s mental health service delivery program and of a community-based approach to system development. The evaluation design is complex and includes multiple components that involve a variety of methodologies, both quantitative and qualitative. To examine the system of care approach, the national evaluation is designed to answer several overarching questions.

Return to Top

• How do systems of care develop according to system of care principles (e.g., family driven and youth guided, culturally competent, interagency collaboration) over time?

• In what ways does funding accelerate system development?

• What are the service utilization patterns (specific services, treatments, and supports) for children and families in systems of care and what are the associated costs?

• How cost-effective are systems of care over time?

• Who are the children and families served by the program and by the funded communities?

• Does the served population change over time as systems of care mature?

• To what extent do children’s clinical and functional outcomes improve over time?

• How are family outcomes affected?

• How are changes in child, family, and system outcomes associated with efforts to implement and develop systems of care?

• To what extent are child & family experiences consistent with the system-of-care philosophy

• How satisfied are children and families with the services they receive?

• To what extent are family members and youth involved in systems of care?

• To what extent do specific evidence-based interventions enhance positive outcomes among children and families, including prevention of substance abuse among children receiving services in systems of care?

• To what extent are systems of care able to sustain themselves after federal funding has ended?

• What factors facilitate or impede sustainability?

Return to Top

This study describes child and family characteristics of children receiving services in CMHS-supported systems of care.

This longitudinal study, conducted among a sample of children in each community who meet the definition of serious emotional disturbance and are eligible for CMHS-supported system of care case coordination services, examines changes over time in children and families who receive services. Areas of child clinical and functional status assessed include symptomatology, diagnoses, strengths, functioning, development (among young children), substance use, school attendance and performance, delinquency and juvenile justice involvement, and stability of living arrangements. Assessment of families includes family support, communications, resources, and caregiver strain.

This study, conducted locally along with the Longitudinal Child and Family Outcome Study, assesses the services received by children and families, their satisfaction with services, and caregivers’ perceptions of the cultural competence of their primary service providers.

Return to Top

Results obtained from this project will be used to direct policies and procedures at the national level and will also be used by the Minnesota Department of Human Services, counties and local public and private providers of mental health services to improve services to children and families.

Return to Top