Who to report to first?


#1

I am about to write up the final evaluation report for a client. It’s an interesting situation where I’m evaluating a curriculum and so there are two separate entities: the organization that created the curriculum and the school (and district) that are implementing the curriculum. All three parties (myself being the external evaluator and the third party) signed an MOU to detail responsibilities of all three parties because the district was wary of its name being shared by the curriculum creator.

I detailed my responsibilities to include providing a report detailing the results to both other parties. However, the curriculum creator wants to see the report first before I send it to the district. This was in response to me sharing the mid-implementation report to both parties at the same time; I got an email from the curriculum creator asking me in the future to send to them first before the district. Upon letting them know approximately when they can expect the final report, they said the same thing back.

In some regards, I’m not too worried about this. I will critically review any requests to change the report before sending to the district, and I will be especially cautious about any requests to change negative or not as positive of findings. However, I’m wondering how you all go about this. Do you just state up front to both parties that they will receive the report at the same time and be given opportunities to provide feedback? Or do you prioritize one side over the other? Should I have written this in my MOU (and write in future MOUs or contracts) or should I have just told the curriculum creator the first time they asked that because we’re all equal parties in the MOU that they will both receive the draft of the report at the same time?

Thanks for your insights!


#2

OOh thats a tough one. What if you say your process is to send the report to both in the same email and call it a draft - asking for thoughts/reflections and if they have any additional “reflections/recommendations/insights” for your consideration as you finalize the report. I would be uncomfortable sending it first to any one party if you are all three on an MOU with no specifics about one party getting that “right.” In the future, in situations like this I would discuss upfront what final reporting might look like and set the expectation there.
Does this help?
-JMC


#3

I agree with Jenny. In the future - or maybe now, if not too late - you might consider holding a “Data Party” with the folks in the MOU - this way they both can react to the data before you write up the report as you reveal some of the findings, and see what they have to say about it. For more on what a data party is, see https://www.betterevaluation.org/en/evaluation-options/data_party (the PDF link at the bottom is very handy). I did this when I knew that the group I was presenting to was going to have a million caveats and a million questions - so midway I did this with them, and included some of their (valid) comments in the final report and presentation. They seemed pleased to have their voices heard, and it did help me with analysis.


#4

How is the evaluation/report structured Dana?

I’d consider a King Solomon approach. Split it in two segments.

A piece where you discuss the curriculum design (delivered first to the design organization for initial review)
A piece where you discuss the curriculum implementation (delivered first to the district for initial review)

So basically the final report is the combo of the two, with conclusions. My guess is that they care first and foremost that you are accurately portraying their efforts. The combined final would then go to all parties.


#5

Thanks. There’s a lot of other issues with why some solutions just wouldn’t work: I am in California, the district is in Colorado, and the curriculum developer is in Florida (meaning we can’t just meet up easily in person!).

It’s a really low-cost evaluation, so there’s no travel and very little examination of implementation.

It’s also being implemented completely by one individual alone, so my “implementation review” would probably consist of just interviewing her or, if we had coordinated it, taping a session to see how it’s being implemented. Therefore, I wouldn’t split the report in that way because none if it is really about what the district did except it’s about their students.

I think I’m going to email back the main client and just let them know that because the MOU has everyone on equal terms that I am going to send the draft report to the two main people of both parties at the same time. They can both provide feedback and then I will finalize for larger dissemination to the district and curriculum development team. Hopefully the curriculum developer doesn’t push back too hard on this, but at the same time I think the MOU and our ethics suggest I shouldn’t withhold the report even for a short period of time.


#6

This is such a good dilemma and I’m weighing in late! I love the data party suggestion (I do these through google drive when we’re working remotely). My main question is if you feel like you have a good understanding of the ‘why’ behind their request. If you haven’t already, would asking some open-ended questions about why they’re pushing for that would help you better understand their concerns and come up with a workable solution? I’m really excited to hear more stories like this in this forum so I can add them to my list of things to anticipate!


#7

That’s definitely a tough one. I’m not sure what I would do in this case and since I don’t know what the MOUs include or the context of the evaluation. However, I would ensure that reporting structure is included in the MOU and evaluation plan in the future (I tend to always include it in the evaluation plans whether formal/informal?).


#8

Good point, @shelley! I am guessing on the why but who knows if it’s correct. And @jrmolle2 I’ll add it to the list of things I need to put in my evaluation plans in the future!

The whole thing is further complicated by the fact that the evaluation didn’t originally have the district. It was supposed to be a different district which ended up just not working out. So I had original relationships with the curriculum developer and then we added on the district. Just an interesting dilemma to work with, so thanks for walking me through some ideas!