Tuesday, 25 August 2015

What is a sound theory of change?

The term "theory of change" (ToC) has established itself in the development world. Agencies invite consultants (including your blogger) to facilitate workshops which would help them develop the theory of change for a particular programme; donors ask prospective grantees to come with a sound theory of change; evaluators (including your blogger) bemoan the absence thereof. There are companies which have developed theory of change software and dedicated websites that propose to help you build your own ToC. The picture below shows a fraction of what I get when I ask a popular search engine to find images of "theory of change": 

Monday, 20 July 2015

Value for money in training?

In an evaluation of a large initiative designed to help changing social norms on gender-based violence (GBV), I found out that each of the many different organisations involved carried out training workshops. The training participants were mainly for police, staff in health services, religious leaders, social workers and other people who tended to spend no or very little time at universities or other academic venues. That is, the trainees were people who were not used to sitting and listening attentively to complicated presentations. 

Yet, most of the training workshops were organised the "academic" way: The trainer would take the audience, seated in neat rows, through a more or less lengthy set of "power point" slides with plenty of text. Often, the content of the slides was highly theoretical, presenting definitions with plenty of terms people would never use in everyday language. The audience would sit and listen and ask the occasional question, if they dared to. People don't want to look stupid - so if a presenter uses lots of big words the audience are not familiar with, changes are that there won't be many questions.

Arguably, that type of training is a waste of time and money. There is a large and growing body of experience on adult education - and education in general - which shows that one of the most effective ways to acquire new knowledge and skills is learning by doing, by solving problems.

Tuesday, 26 May 2015

Too quick, too dirty

Today I came across the (virtual) file of a "quick and dirty" evaluation I carried out a while ago. That made me feel a bit queasy - because I should have turned down that offer, or advised my clients to go for something radically different. Quick and dirty can be quite wasteful.

Someone had approached me for the job at very short notice - the expectation was that the consultant would start travelling within a couple of weeks. I happened to have time - some other assignment had been postponed - and I found the subject matter interesting. That's why I accepted the job. I was worried about the terms of reference, though. They came with plenty of specific ideas as to how the evaluation would have to be carried out - including requirements to visit three countries, to conduct a survey and to complete the job within some 30 person-days spread across two months. Within that time frame, the relevance, effectiveness, efficiency and sustainability of a project covering several rural areas across three countries was to be assessed.  Oh, impact, too, but I talked them out of that.

Saturday, 21 March 2015

Rape is not about sex

Every so often, I facilitate gender training. One of my favourite ways to get discussions started is a quiz, inspired by the 1994 classic, The Oxfam Gender Training Manual (Suzanne Williams with Janet Seed and Adelina Mwau). I read a set of statements - for instance, recent research findings related to the field the participants work in - about sex and gender. For each statement, the participants are asked to determine whether it is about sex (as in biologically male, female or a bit of both - not sexual activity) or gender (roughly speaking, the behaviour societies expect from people according to their sex).

The discussions are always interesting. In a recent quiz of that type, I read the statement:  "A survey in Botswana revealed that 67% of school girls interviewed had been subjected to sexual harassment by teachers; 10% had consented to sex for fear of reprisals." (That was about a decade ago.)
One participant argued the statement was about sex.

Tuesday, 24 February 2015

Mystery moaners

Today a message arrived in my inbox from an address called DevBalls, with links to a blog called "DevBalls - Exposing the absurdity of the aid industry". The mystery author must be someone who knows people who know my e-mail address, but at some distance - because apparently I have missed out on a whole year of DevBalls. The spring and autumn 2014 issues can be downloaded from that blog.

Everything you find there is harsh criticism of UK government aid policy and its implementation. The style is rather aggressive, but the questions the authors raise are pertinent: for example,

Thursday, 29 January 2015

A written survey with people who don't read and write

Last year - ah, no, in 2013 - my colleague Wolfgang Stuppert and I carried out an evaluation of services for survivors of violence against women and girls in Mozambique. We felt it was important to gather feedback from many women and girls who used the services. We had only little time in Mozambique and no resources to train enumerators who would interview large numbers of service users.

We decided to organise a written survey. But some service users, we were told, could not read and write well enough.

Monday, 12 January 2015

Why Ukraine?

A reader has asked whether I have found out why this blog gets many hits from Ukraine, where I know few people. His blog records many hits from Ukraine, too.
I have no idea!

Could it be that people who experience difficulties accessing websites with such words as "human rights" hide behind a Ukrainian identity to escape censorship in their countries? Or is Ukraine where all those people are located who keep trying to post comments with dubious links?

Will post soon: written surveys with respondents who don't read and write.

Thursday, 25 December 2014

Moral judgment

It is somewhat late in the year to share observations from the 11th Biennial Conference of the European Evaluation Society which took place in early September. Apologies! This year has been a bit breathless for me - more on that in a different, future post.

Meanwhile, one particularly gripping topic at the EES conference was moral judgment, and the question whether evaluators should exercise it even if the evaluation terms of reference were only about, say, value for money. As one speaker put it, “you’re going to crash into [ethics] in the course of your normal trafficking around as an evaluator”. Not only because you need to do your best to avoid hurting anyone’s rights or anyone’s safety, but also because you are bound to bump into things that are bad, or that you might find bad.

Examples abound.

Saturday, 15 November 2014

Nice movies on coffee and gender in Uganda

I have promised posts about the recent European Evaluation Society conference in Dublin. Apologies that'll have to wait for a few weeks! I have been too busy preparing for my next assignment: a multi-country evaluation of a project that promotes gender equity and value chain development. A couple of beautiful short videos illustrate the approach, in two parts: Coffee Value Chain Uganda, Part 1, and part 2. Very commendable! Versions in French and Portuguese are available as well.

Saturday, 18 October 2014

Best Poster Award at EES Conference

Our poster "Paths to Effective Evaluation" won the Best Poster Award at the 11th biennial conference of the European Evaluation Society in Dublin. We - co-researcher Wolfgang Stuppert and I - are delighted, because we did put in much effort to find ways to translate complicated findings from Qualitative Comparative Analysis into something comprehensible to an audience of development practitioners. Find linkes to the poster, as well as to the full documentation of the research, on our dedicated blog www.evawreview.de

Wednesday, 24 September 2014

EES presentation: Evaluation methods and approaches

Next week in Dublin, at the 11th Biennial Conference of the European Evaluation Society (EES), I'll present the findings of our review of methods and approaches in evaluations of interventions on violence against women and girls (written by Michaela Raab and Wolfgang Stuppert, commissioned by the UK Department for International Development DFID).

In case you happen to be at the conference, you are warmly invited to drop by at one of the two sessions where our findings will be presented:

Monday, 22 September 2014

To my readers, especially those in Ukraine

Six years ago, in October 2008, I started this blog. I had planned on developing a website to advertise for my consulting services. I kept pondering what would need to be on that site, what the design would need to look like to attract a wide range of potential clients, and so forth. At the same time I was quite busy with my consultancies, learning exciting things and meeting interesting people. I had started a couple of mailing lists which just kept growing, because I wanted to share so many things with so many professionals around the world. In the end, I decided to stop sending mass mailings and to start blogging instead. It turned out to be very easy, and a bit addictive.

Wednesday, 10 September 2014

A gender-sensitive Human Development Index (HDI)

The latest Human Development Report (HDR) comes with interesting news on gender inequality. This is because, in addition to the gender inequality index (GII), UNDP has calculated male and female human development indices (HDI) separately. Comparing the "female" and the "male" HDI yields a new index called the Gender-related Development Index (GDI).

There is a huge difference between the GII and the GDI.

Tuesday, 12 August 2014

3ie comment on video tutorials

3ie, the International Initiative for Impact Evaluation, has sent a comment explaining why you need to register if you wish to take the useful tests that come with their video tutorials on impact evaluation. For some technical reason their comment cannot be displayed under my original post (below), so I am taking the liberty to post it right here:

Thursday, 7 August 2014

Video tutorials on impact evaluation

The Asian Development Bank (ADB) and the International Initiative for Impact Evaluation (3ie) have published a set of six video lectures on impact evaluation. They are available here. The lectures are presented by different specialists; the slides accompanying the lectures can be downloaded from the same website.