Participatory Action Evaluation

By Cindy Banyai originally posted at Participatory Evaluation Forum - Refocus Institute

By: Cindy Banyai, Ph.D.

Here are the theoretical underpinnings of the poster I presented last night at the American Evaluation Association conference.

Participatory Action Evaluation (PAE) incorporates ideas from participatory evaluation, empowerment evaluation, and action research. This combination provides for a useful management tool and a beneficial community intervention (Jackson and Kassam 1998: 9; Small 1995: 949).
PAE takes the same perspective as empowerment evaluation when it comes to implementation (Fetterman 2000: 37). In traditional evaluation there is typically an evaluation practitioner: an external expert that instructs and conducts the evaluation. In PAE, however, the convener organizes the evaluation and facilitates its process, acting when necessary as a ‘critical friend’ or providing technical guidance.
PAE groups move through cycles of action research (Heron and Reason 2006) to come to consensus on how to execute their particular evaluation project. The group decides upon their evaluation framework under their evaluation objective, chooses their preferred mode of communication (i.e. photography, video, logic modeling, dramatic interpretation), and use principles of action research to complete their project. When broader community capacity building is sought a public exhibition is held to continue the evaluation process. Exhibiting PAE outputs includes more participants in the evaluation and expands the community capacity building potential of the project through promoting dialogue between those that attend the exhibition and provides a forum through which the original group participants can demonstrate leadership (Bleiker and Kay, 2007, p. 157; Harper, 2001, p. 16).
The knowledge gained through PAE is not limited to narratives, descriptions or visuals of a particular situation, but includes learning on various levels. The interaction participants have allows for joint learning between them and an exchange of ideas in the re-casting of shared situations and events (Lykes 2006: 273; Mendis-Millard and Reed 2007: 550-551; Vernooy et al. 2003: 24), which provides depth to the evaluation and builds community capacity. Data gathered in PAE can be incorporated into traditional evaluation reports anecdotally, processed using statistical qualitative data analysis techniques, or add visual support to concepts not otherwise easily explained verbally.
PAE contributes to the understanding and contextualization of evaluation objectives and questions, whilst providing benefits and participation incentives to target groups. Data gathered through PAE supports themes found in other evaluation analyses allowing for triangulation and data reassurance. Aside from the supportive quality of this approach, it provides an opportunity to explore community voice and discover hidden truths.
The major themes of PAE are:
the need to recount actual details, experiences, and stories;
emphasis on the process, not the outputs;
providing voice to stakeholders or other groups; and
practical utility of theories and information.
PAE incorporates ideas from participatory evaluation and action research. This combination yields a useful management tool and a beneficial community intervention (Jackson and Kassam 1998: 9; Small 1995: 949). Both action research and participatory evaluation:
gather data (Weiss 1998);
focus on a specific task (Patton 2002: 221; Friedman 2006: 134; Ladkin 2006: 482; Small 1995: 942);
involve discussion and consensus building for outcomes (Friedman 2006: 135; Fults 1993: 86; Small 1995: 946);
promote learning and knowledge sharing (Bogenschnieder,1996: 130; Friedman 2006: 132; Jackson and Kassam 1998: 2; Patton 2002: 179; Thurston, Farrar, Casebeer, and Grossman 2004: 481);
promote ownership of policy initiatives (Jackson and Kassam 1998: 2); and
have the belief that local people have valuable knowledge (Bogenschnieder 1996: 132; Heron and Reason 2006: 144; Razafindrakoto and Roubaud 2002: 127-128; Smith 1999: 12-14).
Evaluation groups of six to twelve participants ensure that the size is manageable, but includes a variety of characters and experiences (Heron and Reason 2006: 151). A PAE meeting would follow this general pattern:
welcome and introductions;
introduction of evaluation objective;
participants discuss evaluation objective in pairs;
floor is opened for questions and discussion, leading to possible modifications of the evaluation objective;
introduction to action evaluation;
discussion in pairs on concept and potential projects;
whole group discussion on action evaluation process and decision on project;
practical discussion: number of cycles, dates, times, venues, financial and other commitments (when, how, and of what will be created, as well as the topic and format);
clarification of criteria for joining the inquiry group (i.e. money, time commitments);
self-assessment exercise in pairs, individuals use the criteria to assess whether they wish to include themselves in project;
participants declare their intention to join;
wrap-up (based on Heron and Reason 2006: 151-152).
The knowledge that is gained through PAE is not limited to narratives, descriptions or visuals of a particular situation, but rather includes learning on various levels. Participants benefit from the process by predicting and setting their own goals (within the project itself), measuring outcomes (of the project and the target of their evaluation), comparing the results with their predictions, and recommending or pursuing a course of action in relation to their findings (Fults 1993: 88; Jackson and Kassam 1998: 3; Rietbergen-McCracken and Narayan 1998: 192). Furthermore, the interaction that participants have allows for joint learning between them and an exchange of ideas in the re-casting of shared situations and events (Lykes 2006: 273; Mendis-Millard and Reed 2007: 550-551; Vernooy et al. 2003: 24).
PAE answer calls to supplement passive evaluation techniques with effective and interesting participatory evaluations (Kaufmann, Recanatini, and Biletsky 2002: vi; Razafindrakoto and Roubaud 2002: 144). The method inherits all of the community capacity building potential that action research and participatory evaluation have and is easy to organize, as well as being unique and interesting to participants.