Monday, 4 November 2019

Proud to be a mentor of Kateryna Kravchuk

Five years ago, the Coady International Institute at St. Francis Xavier University in Nova Scotia (Canada) contacted me with a request to mentor a woman who had just graduated from its Global Change Leaders Programme (GCL). The idea was that we would have a one-hour Skype call every two weeks over a six-month period, to support the graduate in her efforts to translate learning from GCL into practice at home. I loved the idea! My first mentee was Kateryna Kravchuk from Ukraine, very close to my home time zone in Berlin.

Mentorships are great learning processes for everyone involved, I believe. I have happily agreed to mentor further GCL graduates since that first experience. Some mentees have answered a short set of questions to present themselves on my blog. I will post a mini-series of menteesin the coming weeks, starting today with my first GCL mentee, in her own words: 
Kateryna Kravchuk - the economist, cultural researcher, focused in strategy, monitoring and evaluation of the projects aimed at improving the quality of life. Her main areas of expertise are community development, responsible entrepreneurship, cultural and creative industries, and cross-sector cooperation. 

(1) What is the main issue - or are the two or three main issues - that you are currently working on?

Currently I am busy with three projects: Creative Enterprise Ukraine (workshops and mentring for young entrepreneurs; keynote speaker for the Gender equality in culture and creative industries research), the evaluation of the UNDP/ADA project “Sustainable local development in rural areas” (two regions in Ukraine) and strategy development for Ivan Honchar Museum – centre for traditional culture in Kyiv. 

The main issues that are calling me since my childhood and more or less predefine everything I do are:

     Economic empowerment as a tool for building personal and institutional resilience,
     Leadership and power relations in the organizational environment,
     Culture and its impact on sustainable development.

(2) What makes you an excellent person to work on these issues?

Since I do interdisciplinary work, it requires multi-dimension perspective. This I have acquired through my education (both formal and informal) and extensive experiencing of various life situations (mainly through being a Girl Scout in my childhood, and then by continuous volunteering for various causes over the globe). Main skills and knowledge that help me on my professional path are:

     Public speaking
     Facilitation skills
     Embodiment practices
     Monitoring, evaluation, reflection skills
     Mindful relationship building skills
     Strategic planning expertise 
     Prototyping, risk-taking experience
     Experience in human rights activist movement 
     Holistic education management skills
     Education in economics and cultural studies
     Studying and practising the authentic traditions of my land

(3) How do you deal with obstacles and setbacks?

     Take my time to reflect (even if it needs A LOT of time)
     Examine my capacity and leverage my resources (know when to say no)
     Ask for help or advice; do not try to do everything on my own
     Study more, practice more (routine that keeps me going even in low energy mode)
     Challenge myself; take calculated risks (I like to experiment)
     Accept the possibility of failure (I try to enjoy the process as much as the result)
     Celebrate small victories (Rule of 5 things I am grateful for everyday)
     Dream impossible - I really do not care when someone tells me that my dream is                   unrealistic and this is what helps me to achieve it.

(4) What is your advice to younger women who want to drive change?

  1. Take care of yourself 
  2. Know your worth
  3. Look for like-minded people
  4. Be kind (always!)
  5. Take the responsibility over your life choices 
  6. Speak up
  7. Have a plan
  8. Enjoy the road and have fun

(5) What is your advice to people who are in mentorships - in your experience, what makes a mentorship useful?

This is from my experience being a mentee and a mentor:

     Speak of what is essential at the moment
     Don’t be afraid to be weak
     Learn from professional hints and tips
     Ask about the failures
     Ask about the inspiration 
     Learn from the personal journey
     Learn to benefit from the honest feedback 

Mentorship is like a mirror, it helps to clarify something that we cannot really see in ourselves and what we do. And it’s only up to us how much time and efforts we invest into it. The more we do, the more we get.



Sunday, 7 July 2019

Classism in evaluation design

Individual interviews for "important persons", focus groups for "beneficiaries", right? Wrong!

These days I have been reviewing evaluations of projects supporting survivors of traumatising human rights violations in countries that are not quite at peace, or even still at war. One would think that in such circumstances, evaluators would be particularly respectful and careful with their interlocutors, avoiding questions and situations that would make them feel uncomfortable, trigger difficult emotions or cause a resurgence of their trauma. In some cases, the opposite is true:

Monday, 17 June 2019

Small group work - keep it fresh and productive!

It is the early afternoon of the second workshop day; the participants are a bit drowsy from a rich lunch; messages have piled up in their smartphones and some people would prefer to deal with those rather than discussing strategy or whatever the workshop is about. Small group work is on the workshop plan. What can you do to keep it lively and productive?

Friday, 17 May 2019

Two or three reasons why two are better than one

Evaluations come in many shapes and sizes. I have led multidisciplinary teams in multi-year assignments, and carried out smaller assignments all by myself. Last year was a lucky year, because most of my work happened in one of my favourite configurations: the tandem or duo - as in two competent persons with complementary or partly overlapping skills and knowledge working together as evaluators on an equal or near-equal footing. Two evaluators working together - even if one of them participates for a shorter spell of time than her colleague - means so much more than the sum of two persons' capacities. 

Saturday, 13 April 2019

Is there a way to measure efficiency in human rights work?

Jasmin Rocha and I reflect on measuring efficiency in human rights programmes in a new guest post on Zenda Ofir's evaluation blog. Have a look here - and enjoy browsing!

Saturday, 16 March 2019

Why internal evaluations need external perspectives

Internal evaluation can be an excellent way to check the quality of one's work, to track progress (in programmes or projects, for instance) and to gather information for management decisions and longer-term learning. To make the most of such exercises, they should go beyond self-reflection. Especially for small to medium-sized teams or organisations, sitting around a table and contemplating one's strengths and weaknesses, as well as successes and failures, is a good start, but just not enough.

Wednesday, 20 February 2019

Quick evaluation: What a difference a couple of days make!

You know those evaluations that come with dozens of questions on a whole complicated international development programme (or even set of programmes), to be answered within one week of desk work, one week on-site and one week to tie it all up? They are still around and they are not about to disappear. I used to hate them: The time frame makes it near-impossible to draw reasonably rigorous samples of respondents (for interviews or for a survey) and there is so little you can do and see in a week at the programme site. What can an outsider find out in one week that an insider doesn't know yet? After having worked on a couple of "quick evaluations" in recent months, I have adopted a milder stance. They can generate useful insights. But how? Here are a few tips.

Tuesday, 8 January 2019

VEPR Certified

I am delighted to announce that I have been an EES VEPR certified evaluator since autumn 2018. EES stands for European Evaluation Society, VEPR for Voluntary Evaluator Peer Review - a stringent process whereby the evaluator, in cooperation with two experienced international colleagues, identifies own capacities and weaknesses (and ways to deal with the latter) in a documented dialogue that spans over several weeks

Join the EES and try out the process, it really helps to know and hone your assets!

Monday, 1 October 2018

Join us at the EES Biannual Conference

This week, Jasmin Rocha and I will be presenting a paper on stretching the limits of the DAC criteria - in particular, the one on efficiency - to deal with the realities of human rights work. Join us in Thessaloniki on Thursday 4 at 10:30!

For more information on the conference, follow this link.

Wednesday, 30 May 2018

Recent resource on human rights funding

The Foundation Center has published an appealingly designed, informative resource on human rights grant making around the world, mapping tens of thousands of grants by hundreds of foundations. 

You can search the database by region, by issues/rights, by "populations" (meaning, for example, women & girls, human rights defenders, migrants & refugees) and by strategies (for instance "grassroots organising", "arts and culture", "litigation and legal aid"). The resource is reasonably up to date; data on the grants is from 2011 to 2015. More "qualitative" information has been posted up to late 2017, with promises for new updates in 2018. 

Obviously, this resource is very interesting for those who who seek prospective donors. But its structure can also inspire those who wish to gain a structured overview of international human rights work - with the caveat, though, that this resource does not reflect the kind of work that does not get international funding.

Free e-prints of our article on effective evaluation in the field of violence against women and girls

A couple of years ago, Wolfgang Stuppert and I conducted a review on behalf of the UK Department for International Development (DFID) of evaluation methods in the field of violence against women and girls. We used qualitative comparative analysis (QCA), a relatively new method, that has yielded useful insights. You can find plenty of information and links on our dedicated site www.evawreview.de

Meanwhile, Development in Practice has published our article on this review. The article is geared to evaluation practitioners - people who commission, conduct and use evaluations. We can share a few free e-prints; if you wish to have one, use this link. First come, first served. When the free lot is used up, you can still view the article on the Taylor and Francis site here and, of course, in the printed publication. 


Our full report is still available from the UK government, for free.

Tuesday, 13 June 2017

Friendly feed-back on our quasi-experimental evaluation

The German national centre for crime prevention (NZK) has published a synthesis of studies on efforts to prevent violent extremism among youth. Our 2015 study on the HEROES project in Berlin (Raab & Stuppert 2015 - unfortunately Wolfgang Stuppert is misspelled in the article) has been lauded for (1) our  definition of a highly relevant central evaluation question which the evaluation could actually answer with methods that we described transparently, and (2) the strengthening of the internal validity of our findings by the use of propensity score matching in quasi-experimental research.

Those who wish to read the full NZK text (in German only) can find it here. A summary of our study is available from this page of the Berlin Senate (in German, too); just download publication BFG Nr.57/2015.

Tuesday, 6 June 2017

Choosing appropriate evaluation methods

Barbara Befani, a reliable source of high quality information, has held a talk at the Centre for Development Impact on choosing appropriate evaluation methods. Listen to her presentation here; it comes with a slide show that is best watched while listening to the talk.

The spreadsheet which helps choosing appropriate evaluation tools is available from the BOND website here.  Enjoy! And contain your disappointment if you realise that most methods won't work with the weak data and skimpy resources available for your evaluation.

Wednesday, 10 May 2017

Evaluation Norms and Standards - now in six languages!

The United Nations Evaluation Group norms and standards for evaluation are now available in all UN languages - Arabic, Chinese, English, French, Russian and Spanish. 

Read them - spread them - use them! You can find them here.

In recent years I have been involved in large reviews of evaluations in the development sector, studying a couple of hundreds of evaluation reports. Many of the evaluations would have been vastly better if they had complied with, or at least been inspired, by evaluation norms and standards. I don't say that every evaluation must attain the quality of rigorous scientific research - in the opposite, time and resource constraints usually don't permit rigorous research in the context of evaluations, anyway. But you can still try to make it useful. 

At least, evaluators must make sure they respect the rights and safety of people who participate in the evaluation - i.e. ethics. Otherwise, the harm caused by an evaluation might outweigh its benefits.

Tuesday, 9 May 2017

A must-read for evalutors, campaigners and everyone: Beware of confirmation bias

Followers of this blog know that bias is a recurring topic on these virtual pages. Evaluators must be aware of bias to produce robust work. After this year's elections and the disturbing success of "alternative facts" and "post-factual" phenomena, the topic's importance has been acknowledged way beyond research circles, entering public debates.

Bias affects everyone's judgment - and that includes ourselves! A good way to deal with bias is to know about it, to seek out opinions that conflict with ours (i.e. mine, yours, everybody's), and to keep thinking.

A February 2017 issue of the New Yorker aptly summarises a few recent books on confirmation and "myside" bias. Find it here and read it - it may change your way of seeing things, and help you understand why others may sometimes seem terribly obtuse.

And if you like the idea of a song about the types of bias that may occasionally cloud your mind, have a look at this earlier post.