By Jane Davidson originally posted at Genuine Evaluation
Have you ever felt like you were nearly going cross-eyed, wading through evaluation proposals that all present screeds of information but don’t really tell you what you need to know? Or, writing one of these yourself?
The whole RFP (Request for Proposal) and evaluation contractor selection process frustrates both evaluators and clients (or commissioners) alike.
In a previous post, entitled 9 hot tips for commissioning, managing (and doing!) actionable evaluation, I outlined some alternatives to the traditional “let’s have everyone write a 50-page phone book” approach.
Central to getting the evaluation team selection process right is knowing the right questions to ask.
| Great questions to ask prospective evaluators need to:
help you clearly distinguish among the candidates … |
After lengthy discussions with both evaluator colleagues and clients, here are my …
Top 6 questions that cut to the chase when choosing the right evaluation team
- This is an evaluation of [insert evaluand] as opposed to a research project about it. How would you define the difference? What knowledge and skills will you need to apply to this because it is an evaluation rather than a research project?
- One of our key evaluation questions is, “How valuable are the outcomes for [insert specific recipient group]?” How would you go about answering this question? [- as opposed to a descriptive question such as, "What were the outcomes for this group?"]
- The various different stakeholders with an interest in this program [or evaluand] have some quite different and possibly conflicting perspectives on, for example, which outcomes should be considered most valuable and what “quality service delivery” looks like in this context. How would you manage these differing perspectives and apply them appropriately to drawing your conclusions?
- We anticipate another of the key challenges in this project to be [describe*]. How would you handle this challenge? Please provide examples of how you have done in the past.
- How would you describe yourselves, professionally, as evaluators? What is your ‘brand’? What is your ‘signature approach’, the one you are best known for? What kinds of projects particularly play to your strengths? What kinds of projects do you avoid because they are outside your areas of strength? [Note: Probe specifically about evaluation expertise and approach; don't let them away with an answer about content expertise.]
- Please provide examples of three recent executive summaries for evaluations you have completed. [These speak volumes about how well the team can write, get to the point, be succinct, actually answer important questions evaluatively (not just free associate to them with data), and how well they truly understand intended users and what will make sense for them.]
* “Challenges” to ask about might include the limited budget or timeframe, a lack of baseline data, difficult-to-measure (but important) outcomes, pressure from influential stakeholders to adopt a particular design or approach, anticipated resistance, changing political or economic context, and so on.
Thoughts? Comments? Please click through to the post on the Genuine Evaluation site to chime in with your additions!
Acknowledgements, and more on Hot Tips for Commissioning and Managing Actionable Evaluation:
- Key questions drawn from a presentation at the July 2012 anzea conference (Jane Davidson & Paula White).
- Jane will be speaking on this topic at the Australasian Evaluation Society conference in Adelaide (paper 212.00, Thu Aug 30, 4.30-5.15pm, Room 3)
- Jane and Paula also have an interactive demonstration session accepted at the American Evaluation Association conference (Sat Oct 27 1-2.30pm, room 201B)
- ‘Like’ Jane’s Real Evaluation facebook page to get updates about these and more!
Mark the dates!
To Comment, visit the original post here: Genuine Evaluation
