Rigor = being honest about the limits

This is another instalment of my "moans about poor evaluation practice" series, triggered by a recent review of evaluation reports in the complex field of governance and human rights.
One of the reports I read used a "traffic light" system: for each evaluation question, the authors decided whether what they found was good ("green light"), in need of some improvement ("yellow light"), or bad ("red light"). That in itself made me feel a bit queasy. Does a "red light" mean an organisation has to drop everything and stop operating? Does that form of visualisation pay any respect to the efforts people put into their work? Yes, evaluators are there to assess the "value" of what they are supposed to evaluate, but does that entitle us to make pronouncements as to what must stop and what can go on? I am not sure.

Measuring time with a yardstick?

Much of my work on monitoring and evaluation is with organisations that work on human rights and governance. It is harder for them to show they make a difference than for people who build roads or wells. Building a road is complicated, but the builders have reasonable control over the process and at the end they can say, "we have built a 50-kilometre road". Human rights and governance work is more indirect: it is not the intervention of  human rights organisation "X" that frees a prisoner - it is a prison guard, the last piece in a huge puzzle of actors and actions. It is not campaign Y that ends domestic violence in a woman's life - it is herself, when she leaves an abusive relationship, or the abusive partner when he stops battering, for reasons that are far beyond the reach of campaign "Y". And you can't blame, say, Amnesty International if the US government fails to close down the Guantánamo detention camp.

5 reasons why gender mainstreaming remains important

Gender mainstreaming and work to end violence against women (VAW) have been on development agencies' agendas for decades. Why are they still important? Some of us feel that "everyone" in development and human rights organisations are well aware of the issues. But the truth is that in organisations without any explicit focus on gender equality or gender justice, the levels of awareness for gender-based discrimination (and the need to end it) tend to be uneven. Efforts to promote gender equality remain limited and often isolated. Some would prefer to drop "gender" altogether, busy as they feel with all those other issues that must be "mainstreamed" - good governance, environmental protection, HIV/AIDS prevention, "you name it!"

But there are at least five reasons why "gender mainstreaming" must continue:

1. Organisations that are committed to universal human rights have a responsibility to ensure their work respects and promotes human rights. Women’s rights are human rights, enshrined in widely accepted international treaties as the the Convention on the Elimination of All Forms of Discrimination against Women (CEDAW, 1979). Any rights-based approach that neglects women's and girls' rights is inadequate.
2. International movements and campaigns rally large numbers of people. Women and girls make up the majority of the world population (although in India and China, sex-selective abortion and infanticide are skewing sex ratios towards boys). Opportunities are likely to be missed and mistakes will be made if planners, implementers and evaluators ignore women's interests and needs, and refrain from engaging women as interlocutors, collaborators and allies.
3. Many development and human rights agencies are into education and campaigning - i.e., they attempt to spread ideas around, and to mobilise others to join them in their cause. The messages they convey, implicitly or explicitly, influence people's minds: research has shown that campaigning can reinforce or weaken people's value systems - broadly speaking, what they consider to be "good" or "bad", "right" or "wrong". (See for example the report Common Cause - The Case for Working with our Cultural Values. I'll summarise it in a future post!) Hence, it is important to avoid reinforcing values that condone discrimination and other violations against women which would be in stark contradiction with the development and human rights goals most of us defend.
4. Gender-based violence is not only one of the most pervasive human rights violations, it also jeopardises development. For example, large numbers of women and girls (as well as a smaller proportion of men and boys) have experienced sexual extortion in schools, health services and police stations, with dire consequences for their physical well-being, their mental health and their social status. Getting girls to school is right, but if they risk their lives because teachers and classmates are likely to abuse them, something is deeply wrong. Gender-blindness (sometimes euphemistically called "gender neutrality") helps to turn a blind eye on the bleak situation that an estimated one-third of the world's women face.
5. In terms of efficiency, any organisation has an interest in ensuring that staff members and volunteers enjoy equal opportunities to unfold their full potential at work, regardless of their sex (and of the size and form of their households)

These are the main reasons that have come to my mind. Feel free to add more by using the comments function below!

PS: 2012 was my "gender year": I carried out several consultancies linked to mainstreaming gender into evaluations, into development programmes, and into the life of organisations without any particular focus on women's rights. In addition, I wrote a guide for Oxfam International on mainstreaming one aspect of its efforts for gender justice: its commitment to end violence against women (VAW). (Download its English version here; translations are in progress.)

New resource on "Mainstreaming" violence prevention

The Oxfam Guide to End Violence against Women, which I wrote earlier this year with generous input from Oxfam International affiliates around the world, has been published in a colourful version that can now be downloaded from the Oxfam Policy and Practice Blog (click on the title to get there). It can work as a useful guide for any organisation whose objectives are not primarily to end violence against women (hence the term "mainstreaming" in the title).

(Photograph: Oxfam)
Gender-based violence and, in particular, violence against women and girls are such widespread issues that no-one can afford to ignore them. For example, in Germany, one in four women have been violently assaulted by a male intimate partner - and that figure does not include more "subtle" forms of violence that are widespread, such as sexual harassment at work, at educational institutions and in other public places. Please read the guide. I'd also be delighted to take your feed-back, via the comments function below or by e-mail if you know my address ;-) Meanwhile, if you want to find out more about violence against women and what to do about it, there is plenty of information available from the UN WOMEN Virtual Knowledge Centre to End Violence against Women.

Gender Budgeting in Europe

The European Community of Practice on Gender Mainstreaming came together in Berlin in late September to discuss gender budgeting in the European Social Fund. It was interesting to learn about the rich experience - and vast differences - in gender mainstreaming and gender budgeting by public administrations across Europe. The keynote speeches and other conference materials are available on the activities page of the ECP Website (click on the link to get there).

Evidence of what worked, at some point, in some place

A major highlight at this year’s annual conference of the German Evaluation Society (DeGEval) was Prof. Gert Biesta’s keynote speech on research as a provider of evidence for policy. “Evidence-based” is a buzzword in development (as well as other disciplines, such as education); bilateral and multilateral donors have mobilised considerable funding to “building an evidence base”. The basic idea is that interventions should be based on the best possible evidence of what works. In his speech, Biesta unravelled the meanings of “intervention”, “evidence” and “what works” to raise fundamental questions: What role should evidence play in policy making? What kind of evidence are we talking about, anyway, and can it replace professional judgement and wider democratic deliberation?

Take time to prepare evaluations!

Since September, there has been an extraordinary flurry of calls for proposals for external evaluations. This is good news; it suggests that people find it important to evaluate their work. But, upon closer examination, you'll notice that many calls for proposals envisage the evaluations to begin just a couple of weeks after the deadline for offers, and to end within a month or so. That is frustrating for busy consultants, who tend to be fully booked several months ahead. Narrow time-frames may also make it difficult for those who commission the evaluation to identify sufficiently skilled and experienced candidates. If you take evaluation seriously, then surely you want it to be done in the best possible way your resources allow? 

Over the years, I have come to appreciate time as a major element of evaluation quality.

Facilitating workshops in two and more languages

A friend recently asked me about interpreters to support a planning workshop in East Africa, reminding me that I meant to write down a few guiding thoughts on facilitation in multi-lingual contexts. It is complicated, but if it works out well it is truly empowering to those who do not speak the "dominant" language.

Participatory Statistics II

Months ago I promised I would write more about a riveting conference I attended in May 2012, "The Rigour and Potential of Participatory Statistics". (Granted, the title sounds a bit clumsy, but if you don't mention "rigour", you don't get anywhere, nowadays ;-) Unfortunately there seems to be no conference documentation (apart from the invitation) on  the web. But I have my notes!

Embracing Failure

When you have a free moment, watch this inspiring talk: David Damberger: What happens when an NGO admits failure | Video on TED.com
Damberger shares his insights from well-meaning but poorly-designed work in developing countries. Generations of aid providers repeat the same old mistakes, over and over again. Admitting failure and telling others about lessons learned would be a first step out of this vicious cycle. That is rare among development organisations, eager to keep their donors happy. But some have started publishing regular "Failure Reports" - a big bravo to the brave ones.

"Gender" is also about time

Counting women in decision-making positions is a popular way to determine whether gender is "mainstreamed" and women are empowered. That seems reasonable: where women are not or poorly represented, their perspectives and concerns are likely to be ignored.  But women's numbers are a blunt indicator: "token" women, drafted into a committee to fulfil the quota, might have little influence on decisions; and women will not necessarily defend women's gender-specific interests. In rural areas, many women simply don't have time for meaningful participation in those village development committees: they wake up before dawn to fetch water and firewood, prepare food for the day, get their children ready for school, and then spend 8, 9, 10 hours in agricultural work. After that, more water- and firewood-fetching, cooking, other houselhold tasks...  A UNIFEM report estimates that women in Sub-Saharan Africa collectively spend 40,000,000,000 (40 billion) hours a year fetching water. And that is just the water-fetching part of the picture.

Greetings from Myanmar

Where the digital revolution has reached deep into the Ayeyarwady Delta (picture).  And where many people I meet are worried about the "development and democratisation industry" descending upon the country. People fear Myanmar might become "another Cambodia" - i.e. a place with many happy, wealthy expatriates running hundreds of mutually contradictory projects all over the country, recruiting highly skilled local people away from local initiatives and institutions, fostering competition instead of collaboration... If you want to read more about bad examples - as well as avenues for improving the dire record of international aid efforts -, click HERE.

Participatory Statistics and more

Last week I attended a riveting conference on participatory statistics, featuring, among others, Robert Chambers and Carlos Barahona. It'll take a few weeks until I'll find time to write more about this but I am eager to start sharing. Have a look at this video by Carlos Barahona - from an earlier conference, but very much in line with what we discussed last week. Enjoy!

A "must read" on preventing partner violence

The UK Department for International Development (DFID)  has published a study by Lori Heise: "What works to prevent partner violence?" - click on the title to download it. It is a hefty document - don't be discouraged if you can't find time, just read the short and clear executive summary!

The study draws on evidence from all kinds of places, but it is meant to inform chiefly work in low and middle-income countries. It concludes that the following types of interventions work best to prevent partner violence:

Managing for Results?

A group of Swedish researchers and development professionals has published a hefty review of evaluations on Results Based Management in Development Cooperation. The full Vähämäki/ Schmidt/ Molander study is available HERE. The authors conclude that the basic idea behind the "results agenda" - i.e. that you need to know how your development interventions perform so as to make the right decisions - is uncontested. They have found that RBM may indeed improve planning and monitoring of development interventions.