Edit page Add new page

Choosing and using analysis and synthesis methods

Our second research principle is to find the truth, and tell the truth. And getting to that truth depends on the strength of our analysis of research data and synthesis to produce findings.

At dxw, we don’t have a fixed set of approved ways to do analysis and synthesis. But we do need to choose approaches that are appropriate to our research methods, and then apply them well.

We encourage our teams to get involved in analysis and synthesis, so that they understand, trust and can act on the findings we produce together.

This guide lists our favourite books, articles and videos on the analysis and synthesis methods and techniques we use most often. In future we aim to add links to any tools and templates we have created, along with good examples from previous projects.

Principles and stages of analysis and synthesis #

The purpose of analysis and synthesis (sometimes also called sensemaking) is to produce findings (sometimes also called insights) that help our team, and sometimes the wider organisation, make informed decisions, and design and build good services.

Principles #

We do at least an initial analysis and synthesis at the end of each research activity, so that we can quickly learn, share and adapt.

Later we may triangulate different kinds of data and findings from different research activities to strengthen our findings, identify contradictions and gaps, and create a coherent picture of everything we have learned.

And we plan the pace of our research to avoid building up analysis and synthesis debt.

Our rule of thumb is “one part research to two parts communication”. For a day spent doing interviews, for example, we expect to spend two days on analysis and synthesis, and sharing findings.

In doing analysis and synthesis we want:

Stages #

Whatever approach we take, our analysis and synthesis goes through three broad stages:

  1. Extract observations, answers and numbers

    Cleaning up our notes, recordings, survey responses, and other research data to produce usable sets of observations (what we saw, what people said), answers (response to questions) and numbers (counts, measures, etc).

  2. Identify findings (insights)

    Reviewing our observations, answers and numbers against our research questions to see what meaningful and relevant things we can learn.

  3. Decide next steps

    Working with our colleagues to agree what to do in response to the findings, including adapting our research activities, creating new design ideas, or recording issues to resolve.

Rich data and big data #

On most of our projects, we expect to do many different research activities, with different groups of participants. Through these activities we produce a rich collection of different kinds of data, such as notes or transcripts from interviews, videos of usability tests, completed boards or worksheets from workshops, responses from questionnaires. Along with any initial findings we created and shared after each activity.

An important part of our work is to triangulate this rich collection to:

On some projects we have access to large, numerical data sets - big data. This might be performance data or operational data from a service, data collected by social scientists in government research projects, or government statistics produced by the ONS (Office for National Statistics).

Using this data effectively, and combining it with our other sources, can need significant skills in statistics and numerical analysis. So we may look for support from colleagues with specialist data analysis skills.

Affinity sorting #

The method we use most often is affinity sorting (or diagramming). It works well with many research methods, including interviews and observation, and in producing profiles and personas.

We typically do two-level sorts, where we:

  1. First group observations into broad categories

    In an application process, for example, we might have a group of observations about providing a photo and another about providing an address.

  2. Then break those larger groups into smaller groups of matching observations.

    In the application example, several participants may not want important deliveries sent to their home address, because they share a front door and post often goes missing, so we group those together.

Once we have created the two level grouping, we review them to identify and add statements of our findings.

For collaborative affinity sorting to work well, it’s important that we help colleagues to extract good observations from their notes and other sources.

Annotating design artefacts #

This is a common variant of affinity sorting that we use in workshops, concept testing, content research and usability testing. In this method we use things like the steps in a process map, a set of design concepts, pieces of content or a set of interface screenshots to define the top level groups.

In this approach we:

  1. First place observations against the relevant part of an artefact

    In a workshop reviewing a process map, for example, we place all our observations against the appropriate step.

  2. Then review the observations to create smaller groups of matching observations

    In the process map review example, several participants may use the same workaround at the same step.

Once we have created these smaller groups, we review them to identify and add statements of our findings.

We often have observations that seem relevant to our research questions but don’t relate directly to an artefact. So we keep a place to create and review additional groups.

This method works well with less experienced colleagues, and when we want to involve participants in analysis and synthesis, as they can find it difficult to create good first level groups. In that case, it’s particularly important that we help colleagues and participants to attach clear and relevant observations, rather than jump to conclusions and next steps.

Reviewing existing information #

One of our research principles is to build on existing evidence, combining existing knowledge, poorly understood data and new research into a coherent picture.

So an important source of our findings, particularly in discovery projects, is existing information, whether from inside or outside the client organisation. These sources might be previous discovery reports, service performance investigations, government and industry statistics, and government and third-sector social research.

It can be hard to find and access all the relevant previous and current work that might be useful. We collect and review information that our clients and other stakeholders provide. And we actively seek other sources, including asking stakeholders for the reports, analyses and other evidence that have informed their views.

When reviewing existing information, we are guided by our research questions, and apply the 3 stages of analysis and synthesis (from observations, to findings and actions).

We use our research questions to help us choose information sources, and to choose which parts to review in detail.

Some sources may include meaningful and relevant findings that we can incorporate directly into ours. From others we may need to extract useful observations, answers and numbers that we can bring into our analysis and synthesis. Some previous research may not have been done or presented in ways we would expect, but we work through any limitations to learn what we can.

And we make sure we can trace relevant findings back to the source material.

Quantitative data analysis #

Some of our research methods produce numbers, or data that we can turn into numbers. In quantitative data analysis we explore these numbers to help us produce meaningful and relevant findings.

The numbers most often come from surveys and service performance measures. But they can also come from unmoderated usability testing and content research.

Our quantitative data analyses are usually relatively simple, because we rarely do research that generates large amounts of quantitative data.

We typically look at things like:

For more complicated analysis, particularly where we have access to bigger and more complicated data sets, we can call on colleagues with specialist data analysis skills for support.

With quantitative analysis, we always remember that numbers themselves are not findings. We work out what the numbers mean for us, and produce meaningful and relevant findings, supported by the numbers, that others can understand and act on.

Coding text and media #

Coding is an analysis method we use when we have lots of notes or recordings, and need a systematic way to break down the data. We most often use coding following interviews, observation and contextual research, and large scale surveys.

We first go through your notes or recordings labelling lines or text or parts of the recordings with ‘codes’, and then use the codes to explore patterns or themes.

There are a number of ways we can create the codes. We can start with a set of predefined codes derived from our research questions, or create the codes as we review the data.

Last updated: 10 January 2024 (history)