Friday, 17 May 2019

Two or three reasons why two are better than one

Evaluations come in many shapes and sizes. I have led multidisciplinary teams in multi-year assignments, and carried out smaller assignments all by myself. Last year was a lucky year, because most of my work happened in one of my favourite configurations: the tandem or duo - as in two competent persons with complementary or partly overlapping skills and knowledge working together as evaluators on an equal or near-equal footing. Two evaluators working together - even if one of them participates for a shorter spell of time than her colleague - means so much more than the sum of two persons' capacities. 
Obviously, two persons can carry out more work than one, and two pairs of eyes and ears perceive more than one. More importantly, two different persons are likely to interpret data differently, from their different perspectives. In my recent tandem assignments, we - the two evaluators - discussed our findings every day when we worked in the same location. At times we'd split for a few days; in those cases, we'd exchange via the phone or a secure messenger service at least twice a week. The tandem approach forces both evaluators to analyse, distil first findings and develop conclusions throughout the evaluation process. Conversely, when you're on your own, you must keep your impressions to yourself (confidentiality in evaluation!). On lonely evenings in hotels far from home, it can be hard to overcome the fatigue at the end of busy days to study the day's notes - for a tandem, this routine is much more inspiring.

When you evaluate across countries and/or cultures, it makes sense to work in tandems that combine different backgrounds and social identities, so that "insider" and "outsider" perceptions and interpretations can challenge each other and lead to stronger findings. "Objectivity" in evaluation is a lofty goal - a team of two might not attain it, but at least, the inter-subjective setup helps keeping individual bias in check. 

Conversely, when I work as a sole elevator, all I can do is look at my own notes and apply a good dose of self-reflection to question my own findings. I can only be in one place at a time and must juggle interviewing, facilitating group discussions and note-taking. I touch-type while carrying out interviews, a mentally and physically strenuous habit - but a necessary one, because often, resources for transcribing recorded interviews are not part of the evaluation budget. When I write up my conclusions and recommendations, there is no peer to review them. In short, it is a tough, lonely exercise that potentially yields less robust results than an evaluation by a tandem.

Still, my clients appear to be very happy with the evaluations I carry out by myself. But even where resources are tight, I recommend setting up tandems - or at least, some peer review process independent from the client and the evaluand - for the evaluation. Even a couple of extra days with a suitable colleague can turbo-charge the robustness of an evaluation's findings and recommendations.

No comments:

Post a comment