Monday, 14 April 2014

Resources for facilitators

A friend has asked me about resources on facilitation of planning workshops. This makes me realise that my approach in facilitation is fed by many different streams. Some favourite resources in English:
  • One is the "Technology of Participation", developed over a period of about half a century by the Institute of Cultural Affairs. Apparently a recent ToP handbook has been published (late 2013 or early 2014).
  • Another favourite source: "Time to Think" by Nancy Kline. Maybe a bit wordy (pleasant for people who enjoy reading stories about the author's many clients), but definitely worthwhile.
  • On effective low-tech visualisation, "The back of the Napkin" by Dan Roam is full of lovely ideas.
  • Also nice, "Game Storming" by Dave Gray, Sunni Brown and James Macanufo and "Visual Meetings" by David Sibbet.
  • If you're into systems and complexity, "Systems Concepts in Action" by Bob Williams and Richard Hummelbrunner offers inspiration.
  • And then there is that fat compendium of facilitation methods alled "The Change Handbook" compiled by Peggy Holman, Tom Devane and Steven Cady. The descriptions are rather short, but if you don't look for step-by-step guidance, it is quite adequate.
I recommend you start with the first 3 items on the list. More time-tested facilitation tips are on the page "Planning, Strategies, Tools" on this blog. I particularly recommend the posting "Tips for Multi-Everything Facilitation". (Apologies for the messy layout of that page - this blog is an personal unpaid spare time initiative and I find too little time to make it all look neat and smart.)

Friday, 4 April 2014

Review of Evaluations - Inception Report ready

It has been quiet again on this blog - this is because we have been busy producing the Inception Report and pursuing our research on approaches and methods in evaluations of interventions on violence against women and girls. (Apologies I still haven't found a shorter way of saying this!)

You can download our full inception report and find the link to an interesting discussion of our work by Rick Davies on our dedicated review blog

Friday, 7 March 2014

Gentle evaluations for huge projects?

Some weeks ago we – Wolf Stuppert (my associate and co-author of this posting) and I – noticed a call for evaluation proposals that seemed exciting. It was about a 50-million-USDollar initiative in a field where both of us have substantive experience. 

At first sight, the terms of reference (TOR) looked exciting: an ambitious, nationwide programme that would have to be asessed for its replicability in other contexts. But our level of excitement dropped dramatically when we studied the TOR in more detail. Although all DAC criteria were listed – relevance, effectiveness, efficiency, impact, sustainability -, the key questions under those headings seemed suprisingly modest. They focused chiefly on programme process and results among direct programme stakeholders, i.e. the non-governmental organisations that had received grants and free training under the programme. That is, the evaluators would ask those who have obtained those goodies whether they felt the programme was effective. 

The reputed international accounting firm that has run the initiative (and drafted the TOR?) should know that a certain amount of bias might cloud the judgement of people who have drawn such immediate benefits from the programme. 

No mention whatsoever, in the TOR, of the ultimate beneficiaries – the citizens of that country who are expected to enjoy more responsive and accountable governance as a result of the programme. Although the programme has been going on for several years, the only evaluation question about impact is fairly abstract; inviting the evaluators to speculate about the extent to which the outcomes achieved might contribute to longer-term changes.

If the evaluation is supposed to test the replicability of the initiative, it would seem important to scrutinise the theory of change underlying the programme, the way it has been translated into action and the changes it has contributed to. What the TOR calls for falls short of that – by a long way. 

For instance, a special TOR section on potential risks and limitations explicitly rules out quantitative data collection and counterfactuals. Instead, the prospective evaluators are invited to rely primarily on their own judgment, “backed by qualitative evidence” that would be drawn from statements and reports produced by the organisations running the programme. It is unclear what specific risks such restraint is supposed to address – not the risk of discovering the programme has passed unnoticed in the society it is supposed to strengthen, we hope?

We love qualitative research and we feel it is important to gather stakeholders’ views on the programmes they are implementing. And we do not say that every development project needs an impact assessment. In many cases, an exercise that focuses on process and immediate outcomes can be perfectly sufficient. (For instance, many smaller initiatives are so grossly understaffed or underfunded that one can’t expect them to produce any significant results anyway – in such a situation, a combination of in-depth conversations and an experienced evaluator’s own judgment may help to draw attention to necessary adjustments.) 

But if you want to gauge the replicability of a multi-year multi-million-dollar initiative, then you’d better do it with the thoroughness and transparency it takes to produce robust findings. The firm that runs the programme convinces clients around the world to invest massive amounts of money into accountability. It should know how to run the kind of evaluation you need to find out whether a programme works. Are transparency and external scrutiny less important when it comes to one’s "own" programmes? 

Who owns those programmes, anyway? But that opens a different discussion...

Tuesday, 18 February 2014

Tell a story: Evaluations that Make a Difference

The research project "Evaluations that Make a Difference" is looking for stories about evaluations. The idea is to explain what has made evaluations influential or successful in an emotionally more engaging way than scientific publications. A lovely idea! Find more information by clicking on the following link: Call for Stories | Evaluations that Make a Difference

Wednesday, 12 February 2014

Evaluations of interventions related to violence against women and girls

A quick update on our review of evaluations - the full scoping report, a link to dozens of published evaluation reports in the field of violence against women and girls, and more are now available on our dedicated blog
We have started the blog because we use a particularly interesting method in our review - Qualitative Comparative Analysis. Our inception report, to be posted on the blog in March, will provide more detail. Come and have a look!

Thursday, 9 January 2014

A quip on women & maths

Yesterday I bought a block of 20 stamps à €0,02. (The German postal services have increased the postage cost for letters from €0,58 to €0,60.)
While the man behind the cashier started to work his calculator, I said "that'll be €0,40 I suppose". The salesman said, "I always use the calculator... erm... that'll be €0,40 indeed. No surprise our Chancellor is a woman."
The German Chancellor, Angela Merkel, is a woman indeed, and she has a PhD in Sciences. (I shall refrain from discussing her politics here.) 
Germans have realised that women are good at maths. Q.E.D.

Tuesday, 7 January 2014

Happy 2014 - quieter times ahead here

My best wishes for a happy 2014! For me, 'happiness' includes health, peace, a reasonable degree of prosperity and all other good things we wish each other in this season. Plus plenty of sunshine. (Unfortunately I can't insert any sunshine picture with this text today because Google seems to have introduced a bug - or some evil new twist - into its Blogger software.)

Early 2014 will be relatively quiet on this blog, as my attention is quite focused on our project with the UK Department for International Development (DFID), which is accompanied by a dedicated blog: But I will do my best to continue posting at least once a month on

Maybe this year I'll manage to revamp the blog, making it more searchable and updating the design? We'll see.
If you wish to receive new posts by e-mail rather than looking for them on this page, feel free to use the "subscribe" function, to the right of this page.

Saturday, 21 December 2013

Resources on gender, and gender & corruption

In 2012 and 2013, I carried out short series of gender sensitisation and gender mainstreaming training for organisations that work on development, human rights and governance issues. Some of the materials that have inspired the training are in the public domain. My favourites are:

  • The gender training manuals by the UK Department for International Development (DfID) and the Swiss Development Cooperation (SDC). Click on this link to find both resources; the SDC toolkit is available in English, French and Spanish. You will also find a folder with a guide on 'mainstreaming' work to end violence against women into development interventions which I have written for Oxfam International (in Arabic, English, French and Spanish).
  • The OECD DAC Guiding Principles for Aid Effectiveness, Gender Equality and Women's Empowerment, as well as OECD Guidelines for Gender Equality and a handy sourcebook on Concepts and approaches can be found in this folder - as well as on the OECD Gender Equality pages.
  • Basic concepts in gender analysis are nicely explained in this presentation by the Dutch Royal Tropical Institute (KIT). For more KIT resources on gender and development (including a recent course on gender in value chains), see the KIT site. 
  • I have also compiled a set of resources on gender-sensitive communication.
  • Ten studies and other documents on gender and accountability and gender and corruption are available here; I have compiled them from various web sources.
  • I have also built a folder with good gender and development resources in French.That folder includes a few training modules I prepared in 2012, inspired chiefly by the excellent series (in French) by Le Monde Selon les Femmes, a Belgian NGO specialised in gender and development. Unfortunately, most of their materials are available as hard copies only.
One caveat: the vast majority of these resources focus on gender in terms of the familiar male-female dichotomy. Arguably, discrimination and violence against women are gender issues that affect the largest numbers of people. But discrimination and violence against sexual minorities, transgender and intersexual persons are also gender issues. I hope that soon, gender manuals will include references and practical tips around the Yogyakarta Principles - an application of international human rights law in relation to sexual orientation and gender identity. Click on "Yogyakarta Principles" to find the doucment in all six United Nations languages.

Tuesday, 19 November 2013

Scared away by jargon?

Last month I discovered the new IDS-BRIDGE resource pack on Gender and Social Movements (scroll down to my 11 October post for all the links). I was so excited about the new resource that I distributed the handy "in brief bulletin" document to all prospective participants of a gender sensitisation workshop (with NGO staff relatively new to gender issues) - as a preparatory reading.

I should have known better. My workshop day started fine, with getting-to-know-each-other and perspective-changing exercises. When I pulled out the bulletin, the mood changed. Participants became more reserved, more defensive. Yes, one said, he had read the bulletin before the meeting, but he found it rather off-putting. Was it about jargon, I asked. Yes, jargon, "and then, the document is only about women". Another participant joined in, saying that she felt the document was only about women's movements, or about movements that dealt with issues of special interest to women, such as agriculture. 'Her' movement was different, she found. A third one sighed and said that after reading the resource pack, she felt that 'her' movement still had a long, long way to go if it wanted to get anywhere near gender justice.

Some of these perceptions are inaccurate: The resource pack - or the message it contains - is potentially helpful to any movement. I can't think of any social movement that would not be of interest to any women. Actually, I find it hard to imagine any social movement that would reasonably want to exclude women.
The bulletin is not 'only' about women and men: it does refer to 'intersectionality', i.e. the way in which aspects of one's identity reinforce or soften gender inequality or injustice. (An example for the effects of 'inter-sectionality': If you are a female member of a very rich social class in a country where women are confined to their households, you might become the President of that country and no-one will find it strange. That is because your social class enables you to live a life that is different from that of most other women - i.e. the power you hold as a member of your social class compensates for some of the powerlessness associated with womanhood in that country context.)

So what is it with the "in brief bulletin" that makes people believe it doesn't apply to them?


When we spend much of our lives working in a certain field, we become fluent in a specialised language. So fluent that in the end the specialised language infests everything we write and say. I have a statistician colleague; his language is full of frequencies and distributions. That is fine as long as you are with your peers. When you would like others to understand you, you need to remember that what you mean by the words you use is not necessarily what others understand. And when you use jargon with people who are not used to it, you can be pretty sure you'll be misunderstood.

The bulletin is beautifully structured. It includes images, text boxes, summaries - everything you would want for an inviting read. But when you examine the text, you find plenty of jargon. For instance, the box that summarises the characteristics of gender-just social movements in 8 bullet points includes the following line: "[A gender-just social movement] Appreciates the gender dimensions of  backlash and external opposition faced by activists." Alright. If I was new to gender studies, would I know what a gender dimension is? How would I interpret the term "backlash"? And where exactly is "external"?

Studying the bulletin more closely, I realise it hardly contains any sentences which don't imply a good command of jargon. For instance, in "How can we build gender-just social movements?", the section "Support internal activism for change" reads "Getting behind initiatives on women's rights by movement members might involve supporting both women's collective power and individual change-makers, building feminist leadership or developing platforms and caucuses on equality." What is women's collective power (especially in an 'intersectional' world)? What is an individual change-maker (surely this is not about coins)? What does feminist leadership look like?

All this sounds abstract and probably a bit scary to people who don't live in the midst of this particular collection of words. That is potentially counterproductive: the more abstract, the more unreachable things sound.

Women only?

The bulletin does appear to conflate "women's rights" with "gender justice", even though I am convinced that is not its intention. Arguably, women's rights issues are the most prevalent form of gender injustice. The bulletin does refer to other gender issues, for example in the bullet point "A gender-just social movement] Takes into account context-specific gender identities, trans and intersex identities and shifting understandings of gender in life and social activism." But again, this is abstract and jargony - the uninformed reader will have difficulties seeing that this is not necessarily about women 'only', and that 'feminist leadership' is not about oppressing men.

Still commendable reading - for experts

I am still happy the resource pack is "out there". It is a helpful compilation of materials for people with good knowledge of the terminology, for example 'gender specialists' in organisations and consultants working on gender issues. Based on the experience I have had, I would not recommend it for unfiltered sharing with less informed people. Less informed people need materials which use terms that are understandable to 'outsiders', and which contain simple, straightforward examples that anyone can try out 'at home'.
Btw I wonder whether the bulletin is currently undergoing revision - the link on the IDS BRIDGE site is broken.

Sunday, 27 October 2013

Documenting qualitative comparative analysis and process tracing

I am delighted to announce that I have started a dedicated blog on Wolfgang Stuppert's and my work on a DFID-commissioned review of evaluations of interventions related to violence against women and girls. We will use qualitative comparative analysis (QCA) and process tracing to examine the evaluation approaches and methods used in up to 100 evaluations completed since 2008. 

Since QCA and process tracing are still relatively new in evaluation, we have decided to document our work on the new blog, which is supported by the UK Department for International Development (DFID) We will post the main steps of our work, every few weeks. Just like this blog here, it includes a "follow by e-mail" function, if you wish to be alerted when new posts come up.