The Snyder evaluation process: an
overview
This is a resource file which supports the regular public program "areol" (action research and evaluation on line) offered twice a year beginning in mid-February and mid-July. For details email Bob Dick bdick@scu.edu.au or bd@uq.net.au
... in which a very brief description of the three phases of the Snyder evaluation process is given
Contents
- Introduction
- The
underlying systems model
- Preliminaries
- Process
evaluation
- Outcome
evaluation
- Short
cycle evaluation
- Notes
This document briefly describes the Snyder evaluation process. It is a flexible process which can be used to evaluate an activity (perhaps a program or project) or a unit (for example an organisation or team.
The Snyder process usually:
- is participative,
seeking to involve all of the stakeholders 1
or their representatives in the process; it is usually possible
to involve the stakeholders or their representatives as actual
co-evaluators in the evaluation;
- is conducted using
the methods of action research, for the combination of rigour
and flexibility which that allows; this also provides both
action outcomes in the form of improvement, and research
outcomes in the form of increased understanding;
- uses a combination
of qualitative and quantitative data; and
- encourages critical reflection by all stakeholders within a systematic but flexible process.
The Snyder process uses action research methods to develop an understanding of how the activity or unit operates. It is at its most effective when used in a highly participative manner, and is especially suited to such use. High participation also increases the likelihood that the results of the evaluation will be applied to program improvement.
The following description assumes a participative version will be used; however, an independent evaluator may use the same overall process to conduct a non-participative evaluation if this is desired.
The detailed description of the process may seem complex. In practice it is logical and flexible, and relatively easy to use in moderately experienced hands. As you read the detailed description you will find it useful to remember that the process has three main components. Each of them provides a different form of evaluation. Each of them builds on the previous stages:
- Process
evaluation. 2
This stage enables the evaluator and stakeholders to develop a
better understanding of the functioning of the unit. In
particular, it allows the stakeholders to understand the links
between resource use, unit activities, the intended and
unintended immediate effects of those activity, the
predetermined goals which are pursued, and the contribution of
the unit to some overall and long-term vision.
- Outcome
evaluation. 3
This stage enables the evaluator and stakeholders to apply the
understanding which they develop in the process
evaluation. They are able to assess which of their goals
are achieved, and how well this is done. It also allows
the development of performance indicators which can be used to
set up ongoing feedback and monitoring.
- Short cycle evaluation. This stage uses the understanding from phase 1, and the performance indicators from phase 2, to be applied. The result is a self-improving system which provides regular feedback to the unit about its performance.
The underlying systems model
Each of the three phases also draws upon a systems model of how a unit or project operates. Resources are consumed by activities which produce intended and unintended immediate effects in the pursuit of goals and objectives, which in turn are intended to contribute to some vision of a better world. Briefly:
resources -> activities -> effects -> objectives -> vision
Resources consist of anything which is required if the activities are to be conducted. They include the obvious: money and equipment and other physical materials. They also include the often-overlooked: skills, time, goodwill, and the like.
Activities are the moment-by-moment and day-by-day actions carried out by unit members.
Immediate effects are those outcomes which occur as a result of the activities, usually during or immediately after those activities. They include those immediate outcomes which are intended, and those which are not.
Objectives are the goals, usually pre-set, which unit members pursue over intermediate time frames. They are usually developed through a planning process, explicitly or implicitly. They are usually set or revised each planning cycle. Thus their timeframe is typically a year, though sometimes more or less than this. (In some descriptions the objectives are labelled "targets".)
The vision consists of the ideals held by the unit stakeholders: the views of a better world in general, and of the contribution of the unit to that better world.
A somewhat more detailed description of the process now follows. (A step-by-step description is available as a separate document.)
Preliminaries
Before the actual evaluation begins there are some crucial preparatory tasks to be done. The primary objective of this phase is to create open and effective relationships between all parties, and a clear understanding of their collective and individual roles.
The most important early tasks are:
- Building an
effective and flexible working relationship between the
evaluator and key stakeholders.
- Identifying and
involving other stakeholders in the process, and negotiating
their role and that of the evaluator and key stakeholders.
- If working with a representative group of stakeholders, setting up the means by which they can stay in touch through two-way communication with the stakeholders they are drawn from.
Process evaluation
The purpose of the process evaluation is to develop a better understanding of how the unit operates. This is done by identifying the components corresponding to the elements (resources, activities, and so on) of the systems model, and analysing how well they are linked together.
The main feature of this phase, therefore, is a focus on the links between the elements -- the way resources, activities, immediate effects, objectives and vision are interconnected, and how the early elements contribute to the later elements.
So, for example, there earliest steps proceed as follows:
- identify the
vision;
- identify the
objectives;
- analyse which
objectives contribute most to the vision;
- develop plans to adjust objectives or vision (or both) in the light of this analysis.
In practice, stakeholders often begin to reorder their priorities and change their behaviour almost from the first step of such a process. This tends to occur whether or not formal plans are developed.
The Snyder process encourages and enables improvement almost from the outset. As people come to understand the process of their work better, they change their behaviour to reflect that improved understanding.
By the completion of the process evaluation, stakeholders understand the links between each element and the adjoining elements:
resources -> (activities + effects) -> objective -> vision
Outcome evaluation
The outcome evaluation uses the understanding from the process evaluation to identify performance indicators. These can be used to provide an outcome evaluation, and to set up the feedback systems for ongoing monitoring and improvement.
In practice this is done by taking the important elements of the vision in turn. Measurable indicators of each are then sought in the earlier elements of the system model:
vision -> objectives -> effects -> activities -> resources
Indicators typically contain a mix of qualitative and quantitative measures. Effective indicators are most often to be found within the "resources" and "effects" elements. Desirably, the package of indicators can be viewed as an appropriate sample of the important elements of the vision, and to be drawn primarily from resources, intended immediate effects, and unintended immediate effects.
Short-cycle evaluation
The short-cycle evaluation then seeks to create systems to allow feedback of results and monitoring of effectiveness. This phase contains the following important steps:
- check that the
indicators are an adequate sample of vision, resource use, and
intended and unintended immediate effects;
- identify the sources
of feedback information for each of the indicators;
- set up a process or mechanism to generate that feedback and communicate it to the people most able to make use of it for system improvement.
- Simply put, a
"stakeholder" is anyone who has a "stake" in what they unit
does, or how it does it, or who can affect the operation of the
unit. [ back
]
- The evaluation
literature mostly uses the terms formative evaluation (simply:
evaluation of a project or program in progress) and "summative
evaluation" (simply: after-the-event evaluation). These
are intended, I think, to describe the function rather than the
process used. I've chosen terms which are more
descriptive of the process used. Having said this,
"process evaluation" as used here is almost equivalent to
"formative evaluation"...
[ back ]
- ... and
"outcome evaluation" as used here is almost equivalent to
"summative evaluation" [ back ].
_____
Copyright (c) Bob Dick 1997-2000. This document may be copied if it is not included in documents sold at a profit, and this and the following notice are included.
This document can be cited as follows:
Dick, B. (1997)
The Snyder evaluation process: an overview [On
line].
Available at
http://www.uq.net.au/action_research/arp/snyder-b.html
