Systemic insight: an alternative to Theory of Change

This blog has first been published on my personal blog site.

I have been blogging quite extensively about the Theory of Change (ToC) approach in recent months. My blog posts reflect a process that I have been going through as part of my different work engagements: adapt ToC approaches to be more sensitive to the complexities development programmes face in their day-to-day work.

Systemic Insight emphases.jpg

Different phases of Systemic Insight

In parallel, with my colleagues at Mesopartner we keep doing research on understanding complex realities and our human reaction to them based on cognitive science, understanding the process of economic change, making decisions under conditions of uncertainty, and managing highly resilient organisations. In these contexts, ToC has limited applicability and a number of drawbacks. Therefore, we have been working on an alternative approach to the ToC approach which we built from the ground up based on our growing understanding of how complex systems work and how involved actors can lead a process of exploration and change. The approach is called Systemic Insight. Continue reading

A new framework for assessing systemic change

Over the last year or so I was hired by a large market systems development programme in Bangladesh to develop a new framework for assessing systemic change for them. We did an initial feasibility study and then a larger pilot study. The report of the pilot study has now been published. Rather than to bore you with the whole report, I would like to share the conceptual thinking behind the framework and the framework itself in this post. In a later post, I will share the methodology. This is not the end of all wisdom and the silver bullet framework everybody has been looking for. For me this is an important step to bring my work and thinking over the last couple of years together into something practically applicable. But this work is not done as I am embarking on a longer research project on systemic change. So there is more learning to come and with it more development of this tool. Please share your thoughts, which would help me to further improve the framework. Continue reading

Day 2: Basics of designing a SenseMaker framework

Fragments of Impact ModulatorsFollowing on the brief recap of day 1 of the Fragments of Impact initiative, here a few insights from day 2. This day was largely about SenseMaker prompts and signification frameworks. Signification frameworks are used to allow people to add meaning and interpret their own stories. The fact that expert interpretation usually is missing the point was the inspiration for the title of yesterday’s post: “Why experts don’t get it, and other lessons“.

So instead of interpreting the stories ourselves, we give the people who tell the stories a semi-constrained framework within which they can interpret their stories. Semi-constrained because we need to give it some boundaries in order to get the things out we are interested in. At the same time, we want to give the people the most liberty to express their own meaning of the story. As both, Tony and Dave Snowden like to mention, it happens very often that members of the research team say that the people mis-interpreted their stories when looking at the signifiers. But actually, the person who mis-interpreted the story was rather the researcher.

A SenseMaker framework starts with a prompt. A prompt is nothing else than a statement or question that provokes people to share an experience. Prompts should be formulated as open as possible while still giving the necessary direction to tell the person what experience we would like to hear (from work, home, on the road, etc.). Thus, prompts should place people in a familiar context (like the work place, the pub, home, etc.). One prompt that is often used as an example goes something like this: “You meet an old friend in a pub. He tells you that he has been offered a job in your company. What experience do you share with him if you want to either encourage or discourage him to join the company?”

After the prompt follow a number of signifiers that allow people to interpret their story. Besides interpretation, signifiers also add a layer of meaning to the story that was not evident in what people shared.

Besides learning about prompts and signifiers, the participants also put serious energy into framing their own projects in a way that can be used to design a SenseMaker framework. The idea of the Fragments of Impact initiative is to develop a handful of frameworks that are then used by all projects. So the over ten projects that were present in the room tried to cluster as good as possible in groups that shared some similar issues. Then they started to collect modulators that influence the people or situation in their projects. Based on these modulators, Cognitive Edge will then develop three signification frameworks.

Now all participants are off to their respective countries and projects. In the following months, we will organise story collection in our projects, test the framework, refine it and eventually start collecting stories. Analysis of the stories will be done in January or February 2016 in another workshop, where we all come together.

Day 1: Why experts don’t get it, and other lessons

This is a brief recap of the first day of the workshop for the Fragments of Impact initiative (I blogged about it here). The aim of the day was to give the participants an introduction to complexity and narrative research with SenseMaker®. Although I have heard many of the things already in earlier events and trainings I participated in, the refresher was useful and there were interesting new aspects I could gather.

Here a few great things I picked up, mainly shared by Tony Quinlan who did the major part of the introduction. I’m paraphrasing here, reflecting the way I perceived the points and why I thought they were.

Many projects in development first pilot their ideas. Then they want to go to scale. Tony borrowed a metaphor from Giulio Quaggiotto (@gquaggiotto) who compared pilots with fish. You put them in a little water and they seem to swim just fine. Then what many projects do is to make a whale. A whale is obviously a totally different beast and will behave in a completely different way. Instead, projects should go for a shoal of fish. They are aligned with the lessons from the pilot but still flexible to adapt to changes in the contexts and new challenges.

In order to show the importance of our history and cultural background, Tony used a simple question: which one is the odd one out from: cow, chicken, grass. The answer largely depends on where you were raised. Psychological research has shown that Westerners choose grass as the odd one out as cows and chicken are both animals. They categorise. East Asians on the other hand choose chicken as the odd one out as cows eat grass. They think much more in relations and networks.

So the above gives already a strong indication that it might be difficult for experts to interpret stories from other people. Especially in multi-cultural settings (which we often find in development). Tony brought three more reasons why experts always get it wrong (i.e. why it is not advisable to interpret stories of others):

  • Metaphors: metaphors are extremely context dependent and difficult to understand if you are not part of the specific culture, region or even village. (Tony made an example related to Cricket and unfortunately I cannot reproduce it.)
  • Sarcasm: how would you detect from a text you read if it was meant the way it was written or not? Tony gave the example of a story he once collected in Pakistan that – reading only the text – would suggest that the person was pretty radical. Using other data collected with SenseMaker, Tony figured out quickly that the text was totally sarcastic.
  • Evolution of language: language changes within generations, if not years. Parents often don’t understand many nuances of the language of their teenage and older children. This makes it difficult for (mostly senior) researcher to interpret stories of others, especially from another generation.

Consequently, the approach used with SenseMaker is to let people interpret their own stories. Today, we will work on a signification framework that we will use over the next six months to collect stories.

SenseMaker® and Systemic Change

Powered by SenseMakerI’m off to Istanbul today. I will participate there in a two-day workshop organised by Cognitive Edge and UNDP. This is part of a new initiative started by the two organisations called Fragments of Impact. The aim of the initiative is to engage with new ways of working on difficult problems in complex and fast-changing situations.

Specifically, the initiative is focused on:

  • generating fast feedback loops
  • capturing data on subtle changes in attitudes, behaviors and perceptions
  • bringing user-centered design methods to development work,
  • implementing projects with the realities of politics and power in mind
  • taking small bets and low-cost prototypes before committing large investments into multi year projects

The initiative will explore the boundaries of using Cognitive Edge’s SenseMaker® software to work on intractable problems in uncertain, culturally-specific environments. SenseMaker® is a research approach and software package that allows us to capture a large number of narratives around a specific issue. Instead of then interpreting the stories ourselves, SenseMaker® gives the power of interpretation to the people who told the stories. They can interpret their own stories within a given framework. The interpretations give us quantitative material that we can analyse statistically. (There is a brief video if you follow the link above by Dave Snowden, one of the brains behind SenseMaker®).

Mesopartner has already successfully used narrative research approaches and SenseMaker® in various projects. For the Fragments of Impact initiative, I have been asked to support a programme of the International Labour Organisation (ILO) called Yapasa. It works with rural youth in Zambia engaged in the soybean and aquaculture markets. Mesopartner also supports EDA, an economic development agency based in Banja Luka, Bosnia. EDA is also participating in the Fragments of Impact initiative.

SenseMaker® is a powerful tool to capture changes in the systems we engage with. It does so from the perspective of the people in those systems, not from the perspective of us as a researcher. Using an open prompt for stories, narrative research allows us to find change in unexpected places and shapes. It is not about confirming a hypothesis but about scanning the context widely. Self-interpretation by the people who tell the story empower them to live the stories.

We are very excited that Mesopartner is participating in the Fragments of Impact initiative. We will keep you posted on how it goes.