Evaluation research uses the tools of social science to
answer pragmatic questions about the effectiveness of program interventions in achieving
their outcome objectives, or about the effectiveness of management practices in achieving
their program implementation objectives. In either case, the development of instruments
that measure real world events accurately and with relevance is a crucial link between
analysis and usefulness for program activities or policy decisions. EMT has been a leader
in the evaluation field with respect to recognizing the crucial role of measurement in
applied research, and the fact that the adequacy of measures must be established within
the context of each study. Standard published measures are useful for evaluation purposes
only when they clearly reflect the objectives, procedures, and context (e.g., cultural
environment) of the evaluation setting.
Accordingly, EMT has been a leader in adapting existing instrumentation to particular
studies, and to developing new and innovative data collection tools when necessary. The Individual Protective Factors Index (IPFI) is a
prominent example. Other examples include collecting information at both the
individual and program levels concerning outcomes and program processes.
- The CSAP National Youth Survey was developed by EMT under contract to the Center for
Substance Abuse Prevention. This instrument adapts items from existing instruments to
measure outcomes typical of substance use prevention programs.
- EMT has developed a unique field data collection protocol format that supports
development of multiple item measures of program design and implementation. This format
has been used in several multiple site evaluation studies.
- EMT has developed practical formats for measuring program contact (dosage) for
individual participants. These measurement procedures document exposure to different types
of program activity as well as overall contact time.
In summary, EMT recognizes that relevant measurement is the key to drawing practical
and useful lessons concerning program design, management, and effectiveness. Our studies
are characterized by careful development of measures, and often include the application of
multiple measures (triangulation) to improve accuracy.