Category: How do you do research?

At the Executive Agency of Education I did research into the target group: students, people integrating and employees of schools. I devised new methods for this, brought their stories into the organization and trained colleagues to do the same. Now I’m learning at the National Ombudsman how you can conduct government-wide research.

You will find out how to approach this in this line of research. I share methods and material. I share success stories and also failures.

Categories
How do you do research? Not part of a category

New work = new questions

https://open.spotify.com/episode/0FT7atOoYsnVNqjEQuhzE7?si=9mr9_64xQO-IXHsJHWaQJg&dl_branch=1

It has been quiet on this blog for a while, as I began a new adventure in May: working at the National Ombudsman. In this blog, I share what prompted me to make the switch and what new questions I will be working on in the near future.

If you’ve been following this blog for any length of time, I’m sure the switch will come as no surprise and you’ll also recognize that I invariably begin each new project with a bundle of questions. So a great first blog about a new adventure.

In April, I was a guest on the podcast Astrid and Vasilis Talk Easy (listen tip!). It was my last week at DUO, and in the podcast I talk at length about why I love working in government and why I’m continuing to work at the National Ombudsman.

Beautiful drawing by Astrid Poot about the podcast

In January, I told my manager at DUO that I wanted to work with issues that are government-wide. Problems for citizens with the government don’t stop at that one counter; that’s what I wanted to learn more about.

We agreed that I would look for the next step and coach colleagues to take over my role. I still mapped out as much as I could, for example, our research methodology, which I could very usefully use in my application to the No (new acronym yay).

But before it came to that, I started looking. After all, what exactly did I want to learn?

Government-wide, citizen-based

During that conversation with my manager, I had been working one day a week for a while for the Work in Progress Program where I was co-writing a government-wide vision for shared service delivery. It struck me that it is difficult for government organizations to take responsibility beyond their own boundaries for problems that citizens experience, even though this is necessary to really improve the relationship between citizens and government.

I also noticed that I had blinders on myself. I had been working at the Executive Agency of Education for 7 years and was mostly concerned with digital services, cash flows and education. I sometimes got comments on this blog “yes, you work for the national government for sure, with the municipality things are very different you know”.

Your relationship with government is broader than digital (financial) services. And I don’t really know much about that yet. So I wanted to engage with topics about citizens’ environments, their homes, their lives, the goals they themselves have and not necessarily just the (digital) contact with the government.

Last November, I tracked my own relationship with the government for a month. I visualized my contact (however small) with a government counter and what departments, processes, organizations, ministries and laws were behind it. It was some work, but I learned a lot.

An experience with one counter intervenes with the experience with another process. Some things came to me all at once, but the organizations didn’t know about them. Small encounters, sometimes unconscious, didn’t seem like much, but piled up, I had a stressful month.

The first sketch I made when I wanted to chart a month out of my relationship with the government

I met Janet Ramesar and together we mapped out her experiences with the government around the benefits scandal. On a timeline you can see how one turns into the other: you have one relationship with the government and interaction with one organization is not isolated but builds (or breaks) your entire government relationship.

I decided that over the next few years I wanted to look at the dynamics between government and citizens from different angles, in order to gain better insight, fine-tune my vision and iteratively improve that relationship between government and citizens in the places where I work. Spending time looking through the glasses of the National Ombudsman fits perfectly with that goal.

What does the Ombudsman do?

You can go to the National Ombudsman if things go wrong between you and the government. We will help you get started if you call, email, send a message through the website, mail, or smoke signal. Complaints from citizens can also become patterns. In this case, we examine structural problems and trends in government. With this, we want to help government learn to be ever better at being there for its citizens and acting properly.

The National Ombudsman is a person, Reinier van Zutphen. He was appointed by the House of Representatives. But the National Ombudsman is also an institution, an organization with 200+ colleagues who support the person the National Ombudsman is working for.

As project manager, I am responsible for one of the research topics on the ombuds agenda, the one on Livability. This includes studies such as the energy transition, environmental law and the effects of gas extraction in Groningen and its surroundings. In addition, I will be working on our own research structures and the effect we are having with them to help the government learn.

First weeks of work in pictures

The collection is expanding

About what I learn and do, and how I approach it, I continue to write. Following the example of the wonderful comic A day at the park, I am adding the following questions to my growing collection:

  • How do your experiences with the government affect the life you lead and the confidence you have in your future? What causes trust to be lost and how can government restore trust? For example,with residents in the earthquake area.
  • What patterns interact government-wide, and how can you organize responsibility (and thus solutions!) for this in a chain of organizations?
  • How can we tell citizens’ stories that do justice to their perceptions and complexities, when they do not fit into the estimated boxes of government?
  • How do you structure and organize ombudsman research to help government learn? The National Ombudsman has the feedback loop of citizen – policy – counter, what can I learn from this how to organize that kind of feedback loop government-wide in the government itself?
  • How can the government bring citizens along in major changes such as the energy transition so that it is fair? What does this mean for citizen participation, the way you interact with government and how it organizes its services?
  • If the government withdraws itself more and leaves more to the market or to citizens themselves, how can the government still remain available to citizens to help them when things go wrong or support them in their new civic role (for example, with the new Dutch environmental law)?

Oh, and with each book or article I read, this list gets longer. So I’m sure there will be some blogs coming out of that in the near future :). If you have any tips, on the above questions or new questions, let me know, great!

Categories
How do you do research? Visual working

Documenting your user research well

Kladblaadjes met aantekeningen uit het interview

For some time I have wanted to write about documenting research. Whew, you’re thinking now, never mind, that’s not a sexy topic. Important though, so in this blog I show very specifically how to properly capture research results. I work for the public service as a civil servant, so I might as well give away my secrets 🙂

Earlier I wrote about why good documentation is important. If you want to work openly, getting your documentation right is a must. At the Executive Agency of Education, where I work, it comes up every year in our good research intentions. And it’s important if you want to make decisions as an organization not on gut feelings, but based on an understanding of your user.

What is good documentation?

The same rules apply to recording your research as to any information. It should be:

  • accessible: anyone in your organization can access it (and if you work openly, anyone outside)
  • findable: the insights are searchable and the information reaches the right people
  • understandable: enough context for every reader, even if you are new to the team
  • actionable: clear conclusions and recommendations that you can deploy and they are well substantiated

Most research at organizations goes hand in hand with development or policy teams who then work on it. That means working in short iterations. The easiest way is to document each step directly. Your research file will naturally grow with you. I do that in 4 steps:

  1. I begin with the research question. This is also usually the reason for the research.
  2. I write and show how I handled it. What is the method, who were involved?
  3. What did I learn? Based on what?
  4. How do I proceed?

It works best when all the research is together and not scattered among departmental disks. Different teams sometimes work for the same user. Standing all together, they can easily use each other’s insights. And when something contradicts a previous study, you can see that. This is how you learn as an organization (without endless meetings, hehe!). I wrote this blog about it earlier: Everything we know about the customer, we all know.

From audio to drafts to blog to essay

An example. For the portrait series The Compassionate Civil Servant, I also tackled it this way. I began with an interview. I recorded that in audio and photographs. Later I listened back to everything and sometimes wrote 10 A4 sheets of paper. I also used Happyscribe for this on occasion, but writing on paper works nicer for me. In my draft, I marked the main points I learned. I looked for any additional context to that (desk research). I summarized everything in a blog with recommendations for myself on how to proceed in the research. After 17 interviews, I summarized all the insights in essays on The Compassionate Civil Servant.

From raw data to wisdom

When I document a research moment, I not only tell what I discovered, I show it. I provide examples and substantiate conclusions with quotes, in text or audio, and visuals.

You should not believe it because I write something, but for the evidence that comes with it.

In my interview with Henk as a compassionate civil servant, this process looked like this (from the “behind-the-scenes” video made by Aljan Scholtens):

The photo interview with Henk and documenting it

From Henk I no longer have my drafts, but from Johan I still do. Especially for you here they are on the blog.

  • Draft sheets with notes from the interview
  • Format of the blog with key insights

Another example: CoronaMelder

Over the summer, I walked along with staff from the Public Health Service in their work to fight the coronavirus. I was not allowed to make recordings, so I typed along live with every conversation. On my laptop, I made folders with a document per visit, or sometimes per person I spoke to.

After the visit, I went through that again. First to correct all the typos of fast typing. Then to discover patterns that I underlined. Different subjects I gave their own color. For each topic, I worked it into an insight. We shared those insights, along with the context of the research and the observations on which it was based, on Github.

What themes were the interviews about? What did I see while walking along? How did employees work with all the systems and with each other? Did something come back regularly, how?

When listening in with a source and contact tracer, for example, it looked like this. Green highlighted is about how the Covid measures are told, orange how people respond to them, yellow how contacts are mapped and blue is about the systems used. We found green and orange information especially important when designing the notification CoronaMelder sends via your phone. For comparison, here is the documented research from that day.

It’s not just about what people say, but what you see, what people do and what happens around you during a research moment. In one of my first visits, several employees told me that they have to work differently all the time. “At every press conference another change is announced and we have to do our work differently.” In the corridors I saw flip charts always showing the new approach, the old sheets lying on the floor in the corner. This follow-along day then led to this insight“The process changes every week.

In all this documentation, you can see exactly how I go from“what did I see” to“what do I know now” to“next step. At CoronaMelder, all research and insights are published on Github, and searchable in Sticktail (the program the researchers work in). That way, the whole research file is together and our wisdom as a team and as an organization grows “naturally” as we do more research.

What to record and what not to record

Of course, there are limits to what information you keep, how you do it and for how long. Consider the following

  • information about individuals. Of course you ask permission for the research, but you usually don’t have to tell in your documentation exactly who it’s about. Sometimes video footage is fine, do ask extra permission for this if you want to share it. (I usually send it too: ‘look, this is how I used it in the report’).
  • information that is irrelevant or outdated.
  • information that should not be shared. This is especially important if you work in the open. Where is the cut and what should not “go out”? Consult on this and discuss who determines this and on what terms.

And make sure your information is easy to find. I use categories and tags on this blog. At the Executive Agency of Education, like at CoronaMelder, we work with Sticktail which also makes tag-based searching in all your research easy. Give documents and folders good names that everyone can understand (not just you). That way you never lose track.

Documenting will never be completely sexy, but hopefully this will get you a long way. Good luck!