Let's evaluate together

This is the time of the year when I would like to be able to clone myself, to respond to all those requests for evaluation proposals (RFPs) while busily working away on on-going jobs that need to be completed before the Northern hemisphere summer break breaks out. List servers publish new RFPs every day; as July approaches, the deadlines become increasingly adventurous. In late May, RFPs ask for offers to be submitted by the first week of June; the selected evaluation team would start working right away. It seems many of those who publish those last-minute quick-quick RFPs assume evaluation consultants spend their days sitting in their offices, twiddling thumbs, chewing nails or randomly surfing the web, waiting for that one agency to call them up and get them to work right away, tomorrow! Drop everything and work for us!

Work to be done: Ending violence against children

A recent report on the global prevalence of violence against children in the past year has shown that  more than half of the children in 96 countries across the world —1 billion children aged 2–17 years—experienced violence in the past year. Violence against children is a human rights violation. It makes people more likely to fall ill, and to  experience and perpetrate violence in their adult lives. In other words, violence is passed on through generations - even genetically, as it can alter a child's genes.

Sustainable Development Goals (SDG) call for an end to “abuse, exploitation, trafficking and all forms of violence against and torture of children” (SDG 16.2) and to “eliminate all forms of violence against all women and girls in the public and private spheres, including trafficking and sexual and other types of exploitation” (SDG 5.2). SDG 4 on education refers to the importance of promoting non-violence in several sub-goals, e.g. by calling for a non-violent environment for education (SDG 4.a).

With probably more than half of the world's children experiencing violence, major efforts are needed to attain the SDGs. For inspiration, have a look at UNICEF's Six Strategies for Action. If you know of any useful resources to share, please post a comment and share.

3500 evaluation reports for everyone - really everyone?

It is delightful to see that more and more agencies are publishing more and more evaluation reports on-line. Now, UNDP has announced, in a pretty infographic, its revamped Evaluation Resource Centre (ERC), which gives access to more than 3500 reports. A bounty for meta-evaluators!

But one thing that puzzles me: The video spot that explains the ERC, with its suave male speaker and friendly ambient music in the background, suggests that only men - or, say, short-haired trouser-wearing necktie-bearers - make decisions. Look at the visuals near minute 0.57 and 2.18. Little skirt-bearers are only acknowledged as members of "the public". What time and place do we live in? Dear UNDP! We know you can do much better on promoting gender equality, so why not flaunt it and show at least equal numbers of male and female decision-makers in your PR materials?

More basic terminology

Here's another set of concepts that seem to cause a great deal of confusion. They are much used in results-oriented planning (often called results-based management or RBM). I like to explain them as follows:

Output = The direct result of an activity - something that is under your/ your project's control. For instance, I brush and floss my teeth several times a day, and the output is a clean set of teeth.

Outcome = Something that your activity is designed to help produce - but it takes some more factors for that kind of result to come about. For instance, I clean my teeth to avoid getting caries, so healthy teeth are my desired outcome. But my chances to have good teeth are much enhanced if I avoid eating sweets or very acid food, if I have healthy gums, if I have the right kind of genes, and so on. Even people with clean teeth get caries.

Impact = A long-lasting result that can be directly traced to an intervention. For example, if my dentist extracts a tooth, the impact is a gap in my mouth. 

Evaluation terminology

Today a friend has asked me about the difference between findings and conclusions. I put it this way:

Findings:

• Dust has gathered into small woolly clouds in the corner of the room.
• Crumbs are scattered all over the floor.
• There are a couple of spiderwebs in the corners of the ceiling.

Conclusion: This room is dirty.
Recommendation: Clean it.

Also a nice way to explain indicators.

Busy!

Deep into the evaluation of this exciting project, www.womenonthefrontline.eu Will be back by April with new posts...

Writing cultures

A few weeks ago I attended another public discussion on the (potential) role of evaluation in policy making. The brief conference - basically, a panel discussion followed by a question-and-answers session - was hosted in Berlin by the Hanns Seidel Stiftung and CEVAL, the Centre for Evaluation at Saar University. The panel was made up of German speaking evaluation specialists from Austria, Germany and Switzerland.

Policy uptake of evaluation findings has been a main topic of the International Year of Evaluation 2015– for instance at the Paris conference I wrote about in October. Evidence gathered in evaluations and research is supposed to support political decision making.

Virtual Workshopping

Earlier this week I facilitated an internal reflection and planning meeting with evalux, a Berlin-based evaluation firm which celebrated its 10th anniversary this year. One of the workshop participants was based in Beijing. It would have been too onerous to fly her over to Berlin, so we found a way to beam her into the workshop via the internet.

I like highly participatory workshops, where people work in alternating configurations –

Participatory research!

Wow - read this presentation of participatory research by 16-24-year-old girls and young women in Kinshasa. An exciting piece of work supported by the UK Department for International Development (DFID) and Social Development Direct (SDD). The initiative turns the "objects" of research into researchers. I trust it will yield much richer information than what you would get from a "top-down" externally-designed survey on young women in Kinshasa. And the young people who collect and analyse the information will gather skills, knowledge and strength in the process! I would expect their interviewees to benefit from the process, too.

Governments which fund development want to see "evidence-based" approaches, that is, research needs to be built into development. Fortunately, the widespread misconception that only large-scale quantitative surveys and experiments yield reliable evidence appears to be fading.

Good things happen in the short term and bad things happen in the long term

This is a long title but I love that sentence, culled from Elliot Stern’s intervention on the Benefits and Barriers to Evaluation Use at the recent evaluation conference in Paris. The one-day conference, convened jointly by the European Evaluation Society, France’s evaluation society, the United Nations Organisation for Education, Science and Culture (UNESCO) and the Organisation for Economic Cooperation and Development (OECD), took place at the quite extraordinary UNESCO headquarters in Paris.

My data

A couple of days ago a colleague working on an interesting new e-learning tool invited me to test an initial, yet unofficial version of that tool. I clicked on the link they had sent to me. A screen appeared which asked me for my full name, my e-mail address and my company. Every single field was mandatory, that is, I could not move to the subsequent screen without providing my name, my e-mail address and a company name.

That is a threshold. When you open a book or a newspaper, no-one asks you to send your name, your e-mail address or other personal data. You open the thing and you read it. The publisher can track the number of sold books - to some extent - the places where they have been sold, and that's it. Has anyone ever complained about that?

Interesting debate on evaluating human rights work

Who is evaluation of human rights work for? How about "strategic plausibility" as an evaluation criterion? How do we measure success when protecting civilians in conflict? These are the kinds of questions discussed in this web debate on evaluating human rights work. Very commendable!

Workshops that work: Six essential tips for facilitators

It is delightful to get plenty of positive feed-back on the workshops I design and/or facilitate. A few weeks ago one participant even described the workshop I facilitated as a "once in a lifetime experience"! Since I would love all workshops people attend to be useful, I have started asking participants to tell me what exactly they like about "my" workshops, so that I can share it here. Some of the points below have been made in earlier posts on this blog, others have come up in recent conversations.

10 things to know about evaluations

The UK Department for International Development (DFID) has produced this wonderful short guide for everyone who uses evaluations. Have a look at it and spread it around! It'll make evaluations more useful. The guide also refers to the Better Evaluation site, also highly commendable for evaluators and everybode interested in the topic. 

Emerging Evaluators meeting virtually and in real life

My occasional associate Wolfgang Stuppert is part of the Emerging Evaluators' Network of the European Evaluation Society (EES). He has invited me to broadcast this invitation to the First Virtual Conference for Emerging Evaluators. Here it is:

On 19 September 2015, the First Virtual Conference for Emerging Evaluators will take place. On that day, from 3 pm to 8 pm [Berlin time, I presume], more than 100 emerging evaluators will gather on-line to discuss the bright and not-so-bright sides of their profession.