If you Google for development coöperation (just over 1.000.000 hits) you will find that some 132,000 of those hits refer to both “evaluation” and “behavio(u)r(s)”. So behaviour is clearly an issue when you look at evaluation of development coöperation.
Out of the one million hits on development coöperation, 4730 give a hit for “evaluation tools”, 2180 for “evaluation tool”. For “evaluation methodology” the result is 5160, and for its plural 2430. If you look within the one million hits for development coöperation for “behavior(u)r(s) of evaluators” you will get exactly two hits.
Evaluation. So much reflection on behaviour, on evaluation methodologies and tools but the behaviour of evaluators is never studied, at least not in the context of development coöperation.
Most evaluations primarily aim to satisfy accountability requirements. Tax money is allocated and channeled down a chain towards public or private actors and finally to the ultimate beneficiaries. Actors in the chain need to inform the actors one step above them. Besides information coming from that actor itself, external audits ensure that resources are handled in accordance with certain standards and not misused for other purposes than they were intended to. External evaluation normally looks as to whether the “use according to the originally intended purposes” actually has any of the originally intended effects on the ultimate beneficiaries.
But do we actually learn from evaluations. Learning and accountability are not the same. Accountability normally has a legal and institutional dimension; it involves negotiation, judgment and possibly punishment. Learning is much more fluid process, there are few rules, it can be an individual or organizational dynamic and it can be quite irrational, involving emotions.
It happened to me again a few months ago: the evaluator comes in, hired by the provider of our project’s financial resources. We are to be reviewed “as part of standard accountability procedures”. After having told us what will happen to us – we and our work are the ‘object’ of evaluation, the evaluator and donor are the subjects in this evaluation grammar – the evaluators cheerfully adds he hopes we will also learn from the process. Intimidated, I nod, off course we hope to learn, it would be actually inappropriate to say you do not intend or expect to learn. But you know you will not. The fact you are treated as an object, and respond that way seems to play a role in does, but how?
Weisbord and Janoff use a four room model for learning: Home base is the room labeled “contentment”. Inevitably at some stage we are in “denial”: reality has changed; our assumptions were wrong, things do not go as well as expected. In a learning process you are able to move on, acknowledge you have a problem and focus on finding a solution: you may linger sometime in the room labeled “confusion”. That uncertainty can lead you to growth, change and move you back into “contentment”, or – if learning does not take place – it hurls us back into the room of denial, waiting for someone or something to draw you back into the uncomfortable room of confusion.
What influences whether the evaluation actually help us overcome denial and confusion and generate growth and learning? When I commission an evaluation or when I am being evaluated in order to be accountable, will I actually also learn? Besides methodologies and tools, the behavior of all those directly involved that will also very much determine or not whether I will actually make my round through those four rooms of learning. But if Google is a measure there seems to be no yet any serious confusion about behaviours of evaluators.