Evaluation’s Friendly Voice: The Structured Open-Ended Interview

Excerpt:

An interview can be an extremely helpful tool in a programme evaluation. Guidelines for choosing to use a structured open-ended interview are offered, as well as three cases to demonstrate how interviews can be used to gather important evaluative information.

Many programme managers know the importance of conducting program evaluation (C. Bennett, 1976; D. Bennett, 1988/89; Steelquist, 1993; Stevens, 1993). Few organisations have the luxury of operating without knowing if their efforts are successful or how to suggest others replicate their programs. Furthermore, evaluation can provide ideas and directions for improvements that may make the programme more effective. Patton (1986) defined programme evaluation as "the systematic collection of information about the activities, characteristics, and outcomes of programmes for use by specific people to reduce uncertainties, improve effectiveness, and make decisions with regard to what those programs are doing and affecting." Since this definition broadens a traditional ‘did-it-work' evaluation to include a host of questions over the life of the program, the challenge program coordinators face is knowing which questions to ask with which tool. Models, manuals, videos, and consultants abound to help administrators make these determinations, sometimes overwhelming those who need some simple answers to improve these programmes. Though questionnaires provide concrete data, there are good reasons to consider using interviews in a programme evaluation.

A good evaluation would ask many different questions, at appropriate times, with appropriate tools, accommodating the programme development process in three distinct phases (Herman, Morris, and Fitz-Gibbon, 1987; Jacobson, 1991; Jacobson, 1999; Parsons, 1995). First, evaluation can begin with the development of the programme itself, using a needs assessment to better understand the audiences' needs and building the programme's objectives to match. Second, evaluation may include formative strategies that provide checks and balances as the programme is developed and tested. Finally, at the programme's conclusion, a summative evaluation measures student learning, teacher reaction, environmental change, numbers attended, funds raised, or trash barrels filled. This could also include an analysis of observations and staff impressions.

A variety of tools, such as surveys, case studies, observations, records, and tests can be used to gather information for an evaluation. Each has an appropriate place and purpose. If it is helpful to understand what participants already know about the topic, how they perceive the programme, what they like, why they come, what they would change, or how they think, evaluators should consider one particular tool: the structured open-ended interview. This tool can be helpful in all three phases of a programme evaluation (needs assessment, formative, and summative). It can yield important insights from participants for programme improvements, sometimes at a comparatively low cost.

The recent use of structured open-ended interviews to elicit useful data in several different capacities (e.g., needs assessment, formative evaluation) provides concrete examples of the use of this tool to environmental education programme coordinators. This article will first define the structured interview and describe how it can be realistically used by practitioners through these examples...

Click here for the full article from the Applied Environmental Education and Communication site.