KEY PROJECTS National Multi-Site Evaluation Research… As EMT’s prevention research, training and technical assistance capacity matured, we added a significant number of national studies to our evaluation portfolio. Over the last decade our prevention evaluation work has focused on large, multi-site evaluation research projects. We have developed this focus because of our commitment to generating science-based evidence with clear utility for policy and professional practice, and our conviction that multi-site studies bring strong advantages in producing useful and applicable information. We have been prime or sub-contractor on six national, multi-site evaluation research studies for SAMHSA and CSAP, including the National Cross-Site Evaluation of High Risk Youth Programs, the Starting Early Starting Smart early childhood services initiative funded by SAMHSA and the Casey Family Program foundation, and SAMHSA’s Minority Substance Abuse and HIV Prevention Initiative. Information on these projects and the following three projects appears below: • Data Coordinating Center for CSAP’s Children of Substance Abusing Parents and Parenting Adolescents • Parenting Adolescents Cross-Site Study National Data Coordinating Center • Data Coordinating Center for Project Youth Connect CSAP National Evaluation of High-Risk Youth Programs In 2001-2004, EMT was the prime contractor for the CSAP National Evaluation of High-Risk Youth Programs. This national cross-site evaluation was a five-year study that examined selected high risk youth, female adolescent and replication initiatives that were funded in 1994 and 1995. In this study, EMT evaluated 48 programs with approximately 6,000 participants and 4,000 comparison youth. This cross-site evaluation linked a thorough and relevant process evaluation, designed to provide focused information of clear relevance to assessing the effectiveness of intervention strategies and the impacts of program implementation factors, to an outcome study designed to determine how the high-risk youth programs reduced risk and increased the resiliency of youth participants as compared to similar youth who received no high-risk youth services. The outcome analysis contributed to prevention policy and program design by identifying differences in effectiveness between program strategies and service mixes, by testing the relation between prevention dosage and outcomes, and by specifying the role of risk and resiliency factors in producing positive ATOD outcomes. To date, EMT staff have produced four monographs and submitted 13 journal articles related to this study. In 2000, this study received the Outstanding Evaluation of the Year from the American Evaluation Association. Data Coordinating Center of Starting Early, Starting Smart EMT operated the Data Coordinating Center of Starting Early, Starting Smart, a SAMHSA and Casey Family Programs mental health study of family and their children aged birth to 5, during 1997-2004. The project used a multiple site, comparison group design with repeated measures to test two major program hypotheses – 1) the implementation of services integration interventions will increase access and utilization of targeted behavioral health, family, educational and physical health services among participating families at risk compared to similar groups of comparison families; and 2) these increased services will produce improvements in family environment, care giver - child interaction, and child development outcomes for target children and their families as compared to similar comparison children and families. EMT was responsible for organizing, leading, and supporting the Steering Committee in designing and implementing the cross-site study (approximately 3,500 participating families); receiving, cleaning, entering, and maintaining the cross-site data set; designing and conducting site visits to collect data for program level measures, and data explication; and writing interim and final reports on the collaborative process, study products, and cross-site findings and applicable knowledge. CSAP National Cross-Site Evaluation of Minority Substance Abuse and HIV Prevention Programs EMT is currently a subcontractor to ORC Macro for the CSAP National Cross-Site Evaluation of Minority Substance Abuse and HIV Prevention Programs. This national cross-site evaluation is a four-year study examining selected high risk youth and adult initiatives. EMT and ORC Macro are jointly evaluating over 75 programs. This cross-site evaluation links a thorough and relevant process evaluation to an outcome study. The process study is designed to provide focused information of clear relevance to assessing the effectiveness of intervention strategies and the impacts of program implementation factors. This study will also provide program-level measures on service type, intensity and duration for statistical analysis, and on identifying contextual factors important to interpreting study findings. The outcome component of the evaluation study is designed to determine how the programs reduced the use of alcohol, tobacco and other drugs and sexual risk behaviors of youth and adult participants as compared to similar youth and adults who received no prevention services. The outcome analysis will contribute to prevention policy and program design by identifying differences in effectiveness between program strategies and service mixes, by testing the relation between prevention dosage and outcomes, and by specifying the role of risk and resiliency factors in producing positive ATOD and sexual risk behavior outcomes. Data Coordinating Center for CSAP’s Children of Substance Abusing Parents and Parenting Adolescents EMT Associates, Inc. and ORC Macro teamed to operate the Data Coordinating Center for CSAP’s Children of Substance Abusing Parents and Parenting Adolescents, a three-year, multiple site study funded by the Center for Substance Abuse Prevention (CSAP). In 1998, CSAP funded 14 programs through cooperative agreements to test substance abuse prevention strategies for children of substance abusing parents (COSAP’s) and their parents. The multi-site study was guided by a steering committee which included members from all study sites, CSAP, EMT, and ORC Macro. EMT and ORC Macro were jointly responsible for developing the study design; selecting process, outcome, and dosage measures; constructing data collection instruments; coordinating data collections and transmissions from each site; managing study databases; analyzing data; writing reports and making presentations on study findings; and providing evaluation technical assistance to the study sites. Also, staff conducted annual site visits, using a standardized protocol, and prepared site visit reports. A longitudinal quasi-experimental design was used to test program effectiveness. Each program site was responsible for forming an intervention and comparison group of COSAP’s and parents. Outcome data was collected at three points in time (baseline, program exit, and 6-month follow up after exit) using standardized instruments to test for differences between the study groups. Dosage data was collected on each person in the intervention group while they were in the program to determine the effectiveness of specific intervention types and the duration of those interventions. In addition, program level data were systematically collected from each program site so that important program characteristics could be factored into the outcome analysis. Data analysis involved advanced statistical modeling, including multiple regressions, structural equations, and hierarchal linear modeling. Parenting Adolescents Cross-Site Study National Data Coordinating Center EMT and ORC Macro also partnered to be the Parenting Adolescents Cross-Site Study National Data Coordinating Center for a three-year national study on the effects of preventive interventions aimed at parenting teens currently enrolled in, or eligible for, welfare benefits (i.e., TANF) for CSAP. The study included 10 programs located in different regions of the U.S. Because the programs represented a wide range of prevention strategies, they provided a solid basis for the generation of applicable knowledge to the field. Data were collected on approximately 3,000 program and comparison youth over a two-year duration. All programs used a common survey instrument at four points in time, allowing for longitudinal as well as treatment condition comparisons. Data on exposure to services (dosage) was also collected on each program participant, allowing precise assessment of the effects of differential exposure to program activities. Data on program level variables was collected, allowing the identification of program characteristics that contribute to effective prevention and intervention. Data Coordinating Center for Project Youth Connect From 1998 to 2001, EMT served as the Data Coordinating Center for Project Youth Connect, a local and national study, funded by SAMHSA and CSAP, designed to determine the effectiveness of mentoring/advocacy models with youth between 9 to 15 years of age and their families. The purpose of the program was to identify if, how, and why the mentoring/advocacy program helped prevent, reduce or delay substance use among the youth who participated in the program. Additional objectives in support of that goal were to improve 1) school bonding and academic performance, 2) family functioning and bonding, 3) life management skills, and 4) adult relations. Fifteen mentoring/advocacy programs across the nation participated in this study with a total of approximately 1,000 participants in the program group and 800 participants in the control group. Youth Connect was designed as a cross-site study that provided enhanced opportunities to identify and develop applicable knowledge about the service integration for young children. This study design involved the development of common methods across sites to help minimize variance in 1) the nature of the interventions, 2) the formation of program and control groups, 3) the selection of instruments, and 4) process data.