Category: Visual working

In government we type so much text. But visually it is all much clearer. And more fun.

Categories
Promoklip Visual working

Reframing leads to new solutions?

Actiefoto! Met prachtig ontworpen werkvormen door mijn designmaat Reinout Tiekstra

When you have been dealing with tough issues for a long time, it is sometimes difficult to find a breakthrough. It can seem like a problem just can’t be solved. These types of “wicked problems” are complex and constantly evolving. There are all sorts of parties involved that often have competing interests.

In government, we have enough of this kind of tough problems. In my research into how we can make government services that are good for people, I come across them regularly. I am currently researching how the government designs services for people in debt.

In my research, I look particularly at government organisations themselves. How do they learn and develop their own skills to better serve citizens? If you want to follow this research, subscribe to my monthly newsletter.

Kees Dorst, a professor in Sydney, wrote the first design book I read 8 years ago as a junior designer in government: Frame Innovation. He describes a number of steps you can take to see new solution directions for tough issues. You do this by thinking differently: by reframing the problem.

In this blog, you will read about how I am using this method with team members I am working with this year. The topic: how does the government seizure someone’s wages when they fail to pay a debt? This month we organized a collaboration day attended by about 40 people. We are still in the middle of the project, so this blog is a reflection on the approach in between. What do I learn about applying this design method in the government context?

The wicked problem

Many people who owe a debt to a government organization fail to pay it. A person can have all kinds of reasons for that. In the process of repayment, or the collection process as it is called on the government side, there are all kinds of steps. One of those steps is for the government to eventually seize your income. That is a major intervention.

To protect citizens, the rule is that such a seizure must never cause you to fall below the subsistence level. Yet it happens. Indeed, the government is legion; different government organizations do not know from each other if and who has already seized. So it may happen that you do get pushed below the subsistence level because the government is competing with itself to collect a debt.

On large boards, all participants wrote or drew the insights from their organization(s).

Solving this problem is proving to be very tough. There are all kinds of different parties involved that have different structures and interests. That’s partly how it was once set up and partly how it grew. Then there are all sorts of other rules that make it difficult to solve this problem, for example the rules about sharing citizens’ data between organizations. And the problem is also constantly changing. In recent years, we as a society have begun to think differently about debt. From “people don’t want to pay” to “people can’t always pay.

How do you look differently?

To understand and address such problems, Kees Dorst devised a series of steps he calls the reframing method.

Steps to arrive at new solution directions for tough problems.

As a team, in the weeks leading up to the collaboration day, we put together as much as we could of what has been done so far to address this issue. To fully understand the problem, we tried to adress everything that makes it so difficult. Often then all sorts of paradoxes come along. Things that at first glance are a contradiction that makes something fail, but don’t have to contradict at all. We also studied the parties involved. What makes them so different, and where is overlap?

On the collaboration day, we set to work naming key themes and seeing if we could come up with some new frames. How might we look at the issue differently?

During the collaboration day, we used this outline of the steps. This stage is the most abstract moment in the process.

Coming up with a new frame like this is always quite a challenge. Once you get it, you think “yeah, makes sense. But just try to think of it. To do this, you must first understand the problem well and examine it from different angles. A good frame helps you to then be able to formulate a good design question. In turn, you can think of solution directions that you can work out.

An example. A paradox in solving this issue is that the people affected are not involved. We assume that they have so little doing power, after all, they can’t pay off their debts either, that they can’t participate in the solution either. Thus we exclude them of the process unfairly. A new insight is that for a good solution, we desperately need the lived experiences of this group. We need to understand much better how things that are conceived work out in practice.

A good frame then is: the experience expertise of people with debt is the key to understanding the issue. A design question that fits with this then is: how can we position these people so that they can contribute their expertise?

Structure of the day

The collaboration day began with a rolling start. Throughout the room we had placed large signs on which the first steps of the method were visualized. Participants could add to them their own expertise. That content was the material we worked on for the rest of the day.

In the morning, each group engaged in a partial perspective. Are you looking through the perspective of the user, society, organization or technology? If you only look through that lens, what do you see? What makes it difficult and what opportunities do you see?

In the afternoon, we continued with those possibilities and came up with ideas to go with them. Again, we worked out those ideas in groups using a work canvas. This is what we will continue to do as a team in the coming weeks.

What can I learn from this for my research?

One of the participants came up with a wonderful quote. “We look through a straw, but with the citizen everything comes together.” Because government organizations are divided into all sorts of separate functions and teams, there are partitions between everything. There are few people who really have an overview of how things work and what the consequences are in practice. That is also what makes such an issue so complex. Where do you put the cut? Do you need to make the issue bigger? So big that it can’t be handled? But then again, many more people need to be involved, because each sub-issue has its own policy staff and product owners. At the same time, we have the luxury of cutting it up; someone in debt does not.

This paradox is not new to me. In previous studies, even before my doctoral research, I also encountered this often. I previously wrote an essay about it“Is there anyone with oversight?

With the group, we discussed all the insights from multiple perspectives.

On top of that, the pressure not to make mistakes is great in government. Failure is not an option because the consequences are dire. For individual citizens, and a social, political reckoning also follows. Yet we regularly fail precisely because we don’t get a shot at this kind of tough problems. This paradox leads to risk avoidance. Experimentation sometimes seems like a dirty word.

In government, you have to think big, but act small. This is a different approach than how government usually operates. The reframing method, and really any design method embraces precisely making, trying out and testing to practice with real people, and doing so as early and as often as possible.

How can you shape this in government? That is what we are going to work on in a small way, in this topic, in the coming time. And on a larger scale, what does this mean for applying design methods in government? Fortunately, I have a few more years to figure that out.

Want to read more?

  • Dorst, K. (2015). Frame innovation: Create new thinking by design. MIT press.
  • Schaminée, A. (2018). Designing with-in public organizations: Building bridges between public sector innovators and designers. Bis Publishers.
  • I summarized the first results from my own research on design in government in this blog: Trusting the process is not enough.
Categories
Promoklip Visual working

The big plan

Foto door Holly Mandarich via Unsplash

It’s always exciting to start something new. Even more exciting to write a first blog about it. So here it is.

In the fall I started a new major study that will keep me entertained for years to come. I will do this research in the open, I hope we will do this together.

This month I will be sharing a few blogs about this big plan, how I got here, how you can get involved, and more. You can also sign up for my newsletter. This way you don’t miss anything and you don’t have to check this blog or Linkedin yourself to see if there’s something new.

In this first blog I will tell you what I want to research in the coming years.

A month of spaghetti

It’s been a while, but in November 2020 I kept track of what I experienced with the Dutch government for a month. On a timeline you could see which things I did at government counters, whether these counters were automated or manned and which implementing organizations and policy departments were involved. I also looked at what legislation and social value was above it.

It turned out to be a big spaghetti and I wrote this blog about it. Later, together with colleague Maureen Hermeling, I did this exercise with an employee of the Money Affairs desk of the Municipality of The Hague with one of their cases.

Some pieces of the spaghetti

I learned a couple things:

  • For the government, laws and services are always bulk. It has these kinds of timelines with all citizens at the same time, but for the citizen it is always tailor-made and personal. It’s my life, my bank account, my house.
  • Stress adds up and citizens easily lose the overview, if they already have it. It was a pain in the ass to make this map anyway. Organizations do not take into account each other’s services and the burden this brings to citizens.
  • There is no joint responsibility with the government. Everything comes together with the citizen, but the government is compartmentalised. Everyone has their own counter. Sometimes even per department, mind you! Each has its own processes, organizational structures and its own funding stream.

On this board you can zoom in on the timeline and view it (in Dutch).

Which perspective do we choose?

In this INNovember 2022,Jasper van Kuijk talks about the four perspectives from which you work as a designer (based on the three lenses of IDEO).

‘Start with the human lens’, of course, but real innovation only takes place when you look from all four lenses.

Collective values are coordinated in a political process and elaborated in laws and policy. When we bring this legislation to the public at the government, we usually first make a business case and then choose the most efficient technical implementation. The usability (the lens ‘human’) is often tested at the end, if we are lucky.

When I place these lenses on my timeline you see them coming back in different layers. In this presentation at CSSDay 2022 I elaborated this with the help of a case from the gas extraction in the Netherlands.

In my previous studies (such as The Compassionate Civil Servant) I came to the conclusion that the government has designed itself as a relay race from law to counter. In doing so, it easily forgets the social purpose (the yellow layer) and finds it difficult to take into account the living environment of citizens (the blue layer).

The green and pink layers become a world in itself, a system world that becomes leading for what and how the relationship between government and citizens is.

In recent years, there has been an increasing call for government policy and services that does take into account the perspective of citizens, how legislation works out for them in their lifeworld (human lens) and whether this will lead to the society we envisioned ( social lens).

At the same time, implementing organizations experience that this is a problem. They are not designed to take this into account. They are driven (and financed) from silos and have to deal with sometimes volatile political wishes and technical debt from the past.

How can this be done? That is what I want to find out in the coming years.

Together with TU Delft and the government’s implementing organizations, I will work on this issue step by step in practice. That means together with you.

How we’re going to do that, I don’t know exactly yet. But it is certain that I want to do it openly and from practice. From now on you can follow every step on this blog and find ways how you can participate, yourself and/or with your organization.

The easiest way to follow this adventure is through my newsletter (Dutch). Here I share updates, summaries, jokes, questions to you every month and I will make sure you don’t miss anything. I will write the first at the end of this month. Subscribe here.

In the next blog I will share my journey so far. A look behind the scenes at how (and with whom) I developed this idea last year.

Categories
How do you do research? Visual working

Visual interviewing

Tijdens een workshop bij DUO over datavisualisatie.

I increasingly use diagrams, drawings and infographics when I interview someone. Since corona, I often interview digitally. At first I found this very inconvenient and impersonal, but I also discovered the benefits. Precisely because it is digital you can very easily draw together and organize information schematically.

In this blog some examples of how I approach visual interviewing and some tips for getting started yourself. I think it would be fun to organize a workshop or meet-up about this sometime, so I’d love to hear if there’s enthusiasm for that.

https://open.spotify.com/episode/5ojiEGVi4QwMZemZEbLrUX?si=4df36a8b08854547

Drawing together

“I can’t draw at all” is the first sentence that rolls out of many people when I introduce this method. That changes when we get down to business and the interviewee loosens up and finds it easier and easier to tell and participate. And at the end, “now it’s much clearer for myself, too.”

I call it visual interviewing, drawing together, schematic conversation, picture talking… Think of a nice name. It means that as we talk, we directly visualize how things fit together. This can be done, for example, by making a mind map together, a timeline or some other useful arrangement. During the conversation, you put all the information directly in the right place, which helps both of you get to the point and immediately see if the story is complete.

And yes, you may also just use text while drawing.

A few examples

Janet Ramesar and I created a timeline together. That one went back ten years to the present. On the timeline, we posted her experiences with child care subsidies. You could see it getting more and more complicated over time. By lining it up like that, you saw cause and effect very clearly. It gave us a foothold to broadly capture the story in a short period of time. We created the entire timeline in a 1-hour zoom session. Janet told, I shared my screen and posted what she told directly on the timeline. If I misplaced it, she corrected me: ‘no, that should be first. That one should be more to the right’. It gave us both something to hold on to.

A snippet from the timeline Janet and I created together about her experiences with the child care subsidy.

I later used the timeline format myself when I charted a month out of my relationship with the government. And I applied it in a project for Werk aan Uitvoering where, together with Maureen Hermeling, I interviewed someone from Loket Geldzaken of the Municipality of The Hague (on my blog I called him Puzzler Patrick). The three of us thus mapped out complex case histories involving debt.

A snippet from the timeline of Puzzler Patrick and Mr. N.

At the Ombudsman, during a series of interviews with colleagues, I used not a timeline but a drawing of the organization. The key question was “how should we organize our own activities to have a greater impact on the government?” In each interview, we drew together a new picture of our ideal organization. While making that drawing, I kept asking why-questions. What I learned from those conversations I described in the blog about a professional listening standard. This is the basic drawing I used as a start for the conversations (i.e., before everyone started shuffling around with it and making a new drawing):

At the beginning of the corona crisis, I got to work for a while with the team that made the corona apps at the Ministry of Health. From Jasper, my husband I had just received an Ipad at that time. I took it along on my visits to the local health organizations (GGD) and outlined with the staff the source and contact tracing and the bottlenecks they experienced. For example, we made these kinds of sketches:

Last example, also with the Ombudsman. In a short brainstorm about signals and complaints as input for research topics, we drew them so you could immediately see the extent of the signals and the overlap with the rest of the topics. This was a first sketch that we could develop later.

Why it works

Drawing together just works nicely. Even in on-site conversations, it’s easy to grab a flipchart: “Shall I draw it out?” and the confusion of speech disappears like snow in the sun.

Digitally, it can be done just as easily. You share your screen, you draw how you envision it, and the other person can immediately respond, “no, you have it all wrong, it should be just like this.”

Creating together means organizing and structuring the story together. It is immediately tangible. You can see it before you. You catch the words in focus before they slip away.

The input is the output. You have the basis for the result and documentation ready immediately after the interview. After talking with Janet, I only needed an extra hour or so myself to make everything neat and write out the keywords. After a check from her, it was finished.

Talking digitally is much less impersonal this way. You are working together. You don’t necessarily have to see each other at all, because you share your story equally and are creative together. That activates even though you are both sitting behind a remote screen. And as an added bonus, you won’t have to type out another stack of post-its or flip charts after the event.

This is what you need

  • a digital drawing tool. I use Miro, but you can also, for example, connect your Ipad to your laptop, draw along and share your screen. Or use another digital tool, there are plenty of them.
  • a basic architecture. Think about how you will set it up in advance. Are you creating a timeline? A schematic of the organization? A mind map of questions and answers? Or something else? In Miro, you have handy ready-made templates that you can use.
  • Skills to sketch visualization. It doesn’t have to be perfect right away. Thinking out loud “hmm how shall I portray this” is fine. The other person then also helps, and you give the other person a chance to make together. In a master class taught by Stefanie Posavec (known for Dear Data), I learned the basic techniques for working visually. I wrote this blog about it and later gave this workshop about it at DUO.
  • program to video call and share your screen

For advanced visual interviewers:

  • also give the other person editing rights in your program. Then take time for a brief intro and explain how the drawing program works. I set aside at least 10 minutes for this and do some practice together. Sometimes it is difficult and it might work easier if I draw and ask questions myself. The other person then thinks with me and gives me drawing instructions. That’s why I call it visual interviewing, because you help the other person tell their story in pictures or diagrams.

Getting Started

Enthusiastic to also draw together with your colleagues or the person you are interviewing? I hope this is of some use to you. It is fun to see how you tackle it, so please share your results with me (and others!) too.

Categories
How do you do research? Not part of a category Visual working

How we can see an automated decision

With increasing talk of algorithm supervision, it is important to know how a third-party study of algorithms can be conducted in government. Preferably in such a way that day-to-day implementation processes are not disrupted but according to the principle that government shows itself.

Since January, I have been working with a very nice group to design a work method for this. We focus mainly on the fixed, ‘dumb’ algorithms and not yet on self-learning algorithms. In this blog, a behind-the-scenes look at our approach, initial observations and two questions for you.

https://open.spotify.com/episode/61MQ5s4ce95L0bMGHiotnF?si=b6a251a59fc040f3

I can just see it

When I had just finished the study The compassionate civil servant, I was sitting with Marlies van Eck on a terrace in Utrecht toasting. She mentioned that she had a follow-up idea to her dissertation (on legal protection in automated chain decisions). Her main conclusion was that, as a lawyer, she could not say whether citizens were sufficiently protected because she could not see the algorithms. The government was a black box, so how can you test whether the decisions it makes are right?

Over my beer I said “but, how weird, because at the Executive Agency of Education, where I worked, I can just see them’. Of course, I immediately began to doubt myself when I said that, but I looked up the photo interview of Cees-Jan talking about decision rules from which the code is then derived. And I mentioned that we were also working to derive the calculation tools on duo.nl from these decision rules so that, as a student, you can simulate how the computer will decide in the future how to pay off your student debt.

Opening and viewing

I am reminded of our conversation frequently because “being able to see the algorithm” is central to this research.

Marleen Stikker writes in her book The Internet is Broken that you only really own something when you can open it up to fix or change it. If you can’t, as with most contemporary technology, the device owns you. Can we open up government’s computers and see how they decide and possibly change that if necessary?

This area is still relatively unknown. The Dutch Tax Service has developed a method for making algorithms testable and explainable. The State Audit Office and accounting firms are preparing for this new task. The General Audit Office has developed a review framework. Recently, the State Council issued a report on automated law enforcement, among other things.

But what is suitable for a financial expert may not work for a researcher who wants to do a legal check. It is also tedious for an implementing organization when different disciplines and organizations consider the use of algorithms at different times and burden the implementation with questions. Both for researchers, and for implementing organizations, a more integrated approach is important.

Say no more. So with a group that I spontaneously get imposter syndrome from, we started in January to experimentally design a method for this. The group: Marlies van Eck, Steven Gort, Abram Klop, Robert van Doesburg, Mariette Lokin, Carlijn Oldeman, Giulia Bössenecker and me. Colleagues from the Executive Agency of Education and the Social Secutiry Bank are cooperating and offering us their automated decisions as trial material to design the method on.

What we want to achieve

Allright. We want to develop a working method for conducting third-party research on a government organization’s use of algorithms. To this end, we have formulated a thought scheme:

  • We study how the use of algorithms can be examined simultaneously legally, financially and model-wise[topic].
  • because we want to know which discipline would like to see which research questions answered[rationale].
  • so that we understand the least burdensome approach to a multidisciplinary and comprehensive assessment[significance].

To keep the research sufficiently concrete but also manageable, we propose that the working method should meet the following four objectives:

  • the working method allows a lawyer to make a judgment on the legality and propriety of the system,
  • the working method allows a data scientist/ information scientist to make a judgment about the quality of the system,
  • the working method enables an accountant or internal controller to make a judgment about …[to be completed, see also help question at the end of this blog],
  • the working method is suitable for repeated use in various public organizations.

Designing the method

In a series of on-site making days, we will work with a government organization. Lucky for me: the first 2 making days are at the Executive Agency of Education in Groningen where I live.

To arrive at a working method that can stand on its own within a few months, we take an embedded and iterative approach. But we must remember to design ourselves out of the method as well, and thus critically examine our bias. We learn by doing: we dive into an algorithm and record our process. Then we reflect on our process and make it explicit. What comes out of this is potentially the method we want to develop.

This is how we shape the creation process:

Outline of the design process divided into the 4 making days and intermediate actions

Together with the Executive Agency of Education, we chose a few automated decisions that are part of the big decision whether or not a student will receive a student loan: the nationality test, the age test and the partner test. Beforehand, we were given the sets of decision rules of these three tests to study firmly.

We decided to look from three perspectives or levels of abstraction. Different questions are important for each perspective.

Perspective 1 is the algorithm itself. How does the decision come about?

  • What are the decision rules dealing with the nationality test?
  • What sections of the law are the decision rules based on?
  • How are the decision rules programmed?
  • How are they included in the work instruction?
  • What data is needed and what are sources?
  • How is the decision explained to the student (user) in personal and general communication?
  • What interaction does a student (user) have with the algorithm and how does this influence the decision?

Perspective 2 is the creators of the algorithm. How does the algorithm come about?

  • Who (what roles and also individuals) are involved in the creation of the algorithm?
  • What considerations were made in the creation of the decision rules?
  • On what basis do these individuals make these trade-offs? What personal bias is there?
  • Is there documentation of this creation, and if so, what is recorded here and for what purpose?
  • What is implicit and difficult to make explicit here? In other words, what do we not know (yet) or cannot know (anymore)? How does this come about?

Perspective 3 is ourselves, the supervisors so to speak. How does the method come about?

  • How did we proceed? What questions did we ask? What worked well and what didn’t?
  • Which questions belong to which discipline (lawyer, accountant and information scientist)? Is there any overlap? Were all disciplines able to get their answers? How do they reinforce each other?
  • What knowledge and expertise do we have that allows us to ask these questions?
  • Can we reproduce our process next time? What would be different?

‘How crazy it is here’

The first day of making was wonderfully chaotic. We were flying in all directions, asking all kinds of questions, and Jean and Cees-Jan from the Executive Agency of Education were infinitely patient to answer all questions. Sometimes we ended up in heated discussion about the way the organization was organized, only to conclude after an hour that that shouldn’t matter at all for the method. Or as Steven so aptly noted, “I couldn’t care less how it is put together.” :’)

After this day of making, Marlies wrote a first blog with her observations and I made a video for the scientific guidance committee and the sounding board group.

Video (in Dutch) showing the experience of the first day of making at the Executive Agency of Education.

The second day of making went a lot more structured. In preparation, I visualized on a digital board all the steps from law to decision. I asked Cees-Jan if he could fill in everything from another decision (the partner test). This was not so simple because the information and docs had to come from all sorts of nooks and crannies of the organization.

I printed what I got (or left a step open) and put the steps on the floor. Underneath I put 3 long sheets of paper for the questions the lawyer, accountant or developer could ask. Perspective 1 and 3 are below each other. We took all afternoon to fill them out and see if we could get answers directly from Cees-Jan and colleagues (perspective 2).

An impression of what that looked like:

In two weeks we will have make day 3 at the Social Security Bank. That will be uncharted territory for me, fun! We are going to look at the age and partner test in the state pension.

In preparation, I digitized the large board we made on the ground at the Executive Agency of Education. It became a version with questions and a blank version for the Social Security Bank colleagues to fill in themselves. We are going to test this prototype and see if this is already a tool for self-assessment to show as an organization how your algorithm comes about and makes a decision.

Through this miro board, you can better zoom in on the prototype. You can also post comments here, and feedback is welcome!

Prototype of the working method we will soon test with the Social Security Bank

Two questions

You can see that the accountant’s perspective has no questions yet. Here we are still searching. Auditors are very busy around February and March because of wrapping up the previous year. So we haven’t been able to include much input from that side. Do you know more about this or know someone who can help, let me know.

And of course: feedback in general. In this blog, I describe our making process and show the prototype. I’m curious what you think of this and how it would work in your organization. After design day 3, we would like to update our prototype and share it with organizations to use without us for self-assessment. So we want to learn from your findings in turn.

If you would like to help with this, please send me a message. You can do so through maike @ klipklaar.nl or the other known ways. Thanks!

Categories
(Un)understood citizens How do you do research? Visual working

The cash flow maze

Tijdens de overstroming vorig jaar is in Valkenburg een kade ingestort en nu tijdelijk gerepareerd door het Waterschap. Het pand rechts staat leeg, de gevel hangt nog altijd gevaarlijk naar beneden.

Almost a year after the floods in Limburg-South, how is the damage compensation going? Have people already returned to their homes and “is everything back to normal”? Or not? In early March, I went with a colleague for three days to Valkenburg aan de Geul, Gulpen-Wittem and Meerssen to see for myself the aftermath of the floods. I took my camera with me.

https://open.spotify.com/episode/2fL6929vlf0epCsUeHPLF4?si=DiPxiIQAT-Kot9aZKwvZ0w

In this blog, you can read about how we approached this visit, about my musings on these types of events in the future and what effect they have on citizens’ relationship with government.

The trigger

In January, Reinier van Zutphen, the National Ombudsman together with Jan, a colleague, paid a regional visit in Limburg as they often do. Among others, Reinier spoke with the mayor of Valkenburg, Daan Prevoo. He told him that with that aftermath, things did not go well at all. The following week, Jan asked if I wanted to go with him to Limburg on short notice for a bit longer to see exactly what was going on. I went along because I did research on the effects of gas extraction last year and we saw some overlap at first glance.

From Monday to Wednesday, we were guests of the three municipalities most affected during last year’s floods: Valkenburg aan de Geul, Gulpen-Wittem and Meerssen. Staff who are working a lot with the aftermath of the floods had created a program for us.

But first: what happened last year?

During the week of July 14, it rained tremendously in Belgium and Germany. This led to floods that then flowed into South Limburg. I found this clip from the youth news that shows well what residents had to deal with:

On July 16, after a crisis council, the cabinet formally declared the flood in South Limburg a national disaster. On BNR that day: “This means that the cabinet will put into effect the Damage Compensation Act. That way, victims will quickly get clarity on whether their damage will be compensated by the government, if their insurance does not cover it.”

But so it’s not going well. We went out for three days with employees of the three Limburg municipalities to get a picture of this ourselves.

Beforehand, Jan and I looked for all kinds of things. I visualized this on a timeline. Approximately what has happened and what bottlenecks do we already see based on what we find online?

Timeline with initial assumptions of the problems – click for larger

We used this timeline as a conversation starter on Monday. Our questions in this regard were:

  • Who are the key parties in the aftermath and how do they relate to each other? How do the most affected areas differ?
  • What are causes and effects that have happened in recent months? Are there chain reactions?
  • What are money flows and who has access to what?
  • What bottlenecks are there and do they have relationships with each other?
  • What do residents and business owners notice about this? How do consequences impact them and those around them?

On large flip charts we mapped out the answers. By the end of the afternoon, my head was full of all the money flows and dead ends in the arrangements. How do we make sense of this?

Real stories from real people

The following days we visited residents and business owners. I had my camera with me to immediately capture what we encountered.

For example, with a man my age in Valkenburg who will probably be living in a cottage with his girlfriend and baby until next year. He is caught between the insurer and the Netherlands Enterprise Agency (RVO), no one wants to get burned on the likely problems with his foundation. He is now receiving an allowance from a private fund to find out what the problem is so that things can move forward again.

Man points to the Geul behind his house and explains how in 7 minutes the water forcefully rushed into the house and ripped out the facade.

At a couple in Gulpen-Wittem who are still doing a lot of work in their home. After last June’s rising water, they again placed sandbags at the doors, the now sweetly babbling brook is flowing directly behind their home. They would like organizations to help them how to proactively protect their homes from future high water but there are no arrangements for that. They are for damage afterwards though.

Husband leans against the garden table as he talks about the interventions he wants to do around the house. The sandbags are still up against the house.

A gentleman in Meerssen who still lives in the garage because the house has not yet been renovated. He is left with a gap between damages and compensation of 80,000 euros that he has nowhere to invoice.

Man shows how he is temporarily living in the garage because the house is not yet habitable again after the floods last year.

All indicated that they had different expectations, perhaps naively, when it was said last year that the floods were a disaster and that the government was going to help them. A year later, that turns out not to be so simple.

How do we move forward?

I see two routes in front of me.

In the short term, I think we need to make the maze understandable. The maze of money flows and schemes that are there for residents and business owners but have dead ends where citizens get stuck. The stories we encountered are examples of this, these people are at the end of such a dead end and cannot go on.

How can the government design such a maze? Probably with good intentions, I know few rogues in government. But still: the government promises something, which is positive, only to design squishy bureaucracy. Why are we doing this? And how come it turned out this way? What can the National Ombudsman do in this regard?

Street in Meerssen where there is still a lot of tinkering and bulky trash and building materials on the street.

In the long run, I think it would be interesting to delve into future events related to this kind of climate conflicts. The people we spoke to, both resident and mayor, are all concerned with the next time this will happen. Because there is no question in their minds that this is not an incident. Something needs to be done in the Geul Valley.

Who is in charge? In part, if still possible, to prevent such major floods from happening again, to warn when they happen and to deal with the consequences quickly and appropriately?

In the hedge along the Geul, you can still see the washed-up grass hanging which was carried away with the flood last year.

Last year, I read many books about climate change and what lies ahead. What should citizens expect from government in this type of climate conflict? And what does this mean for how government designs itself?

This is still a new area for me. So I would love to hear who I can talk to further about that and what to read or listen to about it. Let me know.

Categories
How do you do research? Visual working

Documenting your user research well

Kladblaadjes met aantekeningen uit het interview

For some time I have wanted to write about documenting research. Whew, you’re thinking now, never mind, that’s not a sexy topic. Important though, so in this blog I show very specifically how to properly capture research results. I work for the public service as a civil servant, so I might as well give away my secrets 🙂

Earlier I wrote about why good documentation is important. If you want to work openly, getting your documentation right is a must. At the Executive Agency of Education, where I work, it comes up every year in our good research intentions. And it’s important if you want to make decisions as an organization not on gut feelings, but based on an understanding of your user.

What is good documentation?

The same rules apply to recording your research as to any information. It should be:

  • accessible: anyone in your organization can access it (and if you work openly, anyone outside)
  • findable: the insights are searchable and the information reaches the right people
  • understandable: enough context for every reader, even if you are new to the team
  • actionable: clear conclusions and recommendations that you can deploy and they are well substantiated

Most research at organizations goes hand in hand with development or policy teams who then work on it. That means working in short iterations. The easiest way is to document each step directly. Your research file will naturally grow with you. I do that in 4 steps:

  1. I begin with the research question. This is also usually the reason for the research.
  2. I write and show how I handled it. What is the method, who were involved?
  3. What did I learn? Based on what?
  4. How do I proceed?

It works best when all the research is together and not scattered among departmental disks. Different teams sometimes work for the same user. Standing all together, they can easily use each other’s insights. And when something contradicts a previous study, you can see that. This is how you learn as an organization (without endless meetings, hehe!). I wrote this blog about it earlier: Everything we know about the customer, we all know.

From audio to drafts to blog to essay

An example. For the portrait series The Compassionate Civil Servant, I also tackled it this way. I began with an interview. I recorded that in audio and photographs. Later I listened back to everything and sometimes wrote 10 A4 sheets of paper. I also used Happyscribe for this on occasion, but writing on paper works nicer for me. In my draft, I marked the main points I learned. I looked for any additional context to that (desk research). I summarized everything in a blog with recommendations for myself on how to proceed in the research. After 17 interviews, I summarized all the insights in essays on The Compassionate Civil Servant.

From raw data to wisdom

When I document a research moment, I not only tell what I discovered, I show it. I provide examples and substantiate conclusions with quotes, in text or audio, and visuals.

You should not believe it because I write something, but for the evidence that comes with it.

In my interview with Henk as a compassionate civil servant, this process looked like this (from the “behind-the-scenes” video made by Aljan Scholtens):

The photo interview with Henk and documenting it

From Henk I no longer have my drafts, but from Johan I still do. Especially for you here they are on the blog.

  • Draft sheets with notes from the interview
  • Format of the blog with key insights

Another example: CoronaMelder

Over the summer, I walked along with staff from the Public Health Service in their work to fight the coronavirus. I was not allowed to make recordings, so I typed along live with every conversation. On my laptop, I made folders with a document per visit, or sometimes per person I spoke to.

After the visit, I went through that again. First to correct all the typos of fast typing. Then to discover patterns that I underlined. Different subjects I gave their own color. For each topic, I worked it into an insight. We shared those insights, along with the context of the research and the observations on which it was based, on Github.

What themes were the interviews about? What did I see while walking along? How did employees work with all the systems and with each other? Did something come back regularly, how?

When listening in with a source and contact tracer, for example, it looked like this. Green highlighted is about how the Covid measures are told, orange how people respond to them, yellow how contacts are mapped and blue is about the systems used. We found green and orange information especially important when designing the notification CoronaMelder sends via your phone. For comparison, here is the documented research from that day.

It’s not just about what people say, but what you see, what people do and what happens around you during a research moment. In one of my first visits, several employees told me that they have to work differently all the time. “At every press conference another change is announced and we have to do our work differently.” In the corridors I saw flip charts always showing the new approach, the old sheets lying on the floor in the corner. This follow-along day then led to this insight“The process changes every week.

In all this documentation, you can see exactly how I go from“what did I see” to“what do I know now” to“next step. At CoronaMelder, all research and insights are published on Github, and searchable in Sticktail (the program the researchers work in). That way, the whole research file is together and our wisdom as a team and as an organization grows “naturally” as we do more research.

What to record and what not to record

Of course, there are limits to what information you keep, how you do it and for how long. Consider the following

  • information about individuals. Of course you ask permission for the research, but you usually don’t have to tell in your documentation exactly who it’s about. Sometimes video footage is fine, do ask extra permission for this if you want to share it. (I usually send it too: ‘look, this is how I used it in the report’).
  • information that is irrelevant or outdated.
  • information that should not be shared. This is especially important if you work in the open. Where is the cut and what should not “go out”? Consult on this and discuss who determines this and on what terms.

And make sure your information is easy to find. I use categories and tags on this blog. At the Executive Agency of Education, like at CoronaMelder, we work with Sticktail which also makes tag-based searching in all your research easy. Give documents and folders good names that everyone can understand (not just you). That way you never lose track.

Documenting will never be completely sexy, but hopefully this will get you a long way. Good luck!