Keep testing & learning from users until you've achieved the results you're looking for

The Challenge

This stage is primarily about testing on and learning directly from users. Effective learning is based on strong objectives, test design and how lessons results in better informed decisions and enhanced empathy.

There are a few challenges you need to be mindful of at this stage:

Stay focused on learning:

  • Test to learn, not to prove, hence the mantra learn and fail fast; lessons learnt inform decisions, so early failure is team savings
  • People naturally become invested in their ideas and often want to prove value too early; continually engage people with the learning goals, rather than iterative designs (which are more likely to elicit attachment!)

Only test what you need to test:

  • Create tools to support testing and only include features you want to test, no more; anything that doesn't inform learning is waste so what you show target users only need to be minimally viable, not finished good.

My Role

Researcher | Facilitator | Coach

The Process

Form a dedicated team and align around the prioritised user problem - the BET!

Introduce continuous feedback loops. Note, this is a continuous loop of test, measure and learn and informs iterative design until it is ready for market releases (or proven not to be worthy of ongoing investment).

Each loop is focused on defined learning objectives with clear key success outcomes.

Regular team practices and easy to understand metrics help to instil discipline and keep people engaged with learning. A high-touch approach, little and often, with a dedicated team is effective at this point. I recommend distinguishing between setting learning objectives and track results with learning:

  • Weekly co-ordination practices to focus on setting objectives and key results; celebrate successes linked to learning objectives that drive good design and investment decisions, including early failures
  • Weekly learning sessions to review and interpret findings; coach teams to elicit and exploit insights and steer roadmaps.

Learning objectives early in the continuous cycle are likely to be broad, for example to test if the overall concept is a solution-fit. Once this is established you can work towards gradually refining the learning objectives.

A range of artefacts or tools support test and learn. Digital services benefit from a range of Minimally Viable Products (or MVPs) at different learning stages. For direct testing, facilitated by a Researcher you may consider the following:

  • Low-fidelity features; this could be paper and post-its
  • Wizard of Oz; produce a front-end interface and support back-end functionality with a developer for facilitated testing
  • Fully interactive prototype

If you'd like to quantify demand for a digital services consider using a fake-door; a link to a feature on the website but it doesn't go anywhere i.e. the user clicks on the link and they get a message "We are exploring this option, thank you for indicating your interest."

When I'm working with teams who need to develop complex, integrated or multi-channel services, multiple embed concurrent feedback loops to support learning.

As you progress through the learning cycles start to develop a service blueprint to guide wider-scale rollout and delivery.

The key at this stage is to facilitate ongoing learning and deliver measured and quantifiable outcomes to inform decisions. As indicated above, the challenge is to define clear learning objectives and key outcomes. The sharper these are, the best informed experiments, outcomes and subsequent go, pivot or no-go decisions.

The Impact

  • Bottom-up innovation driven by self-managing teams
  • Transparency and easy to understand team performance metrics
  • Baseline metrics that demonstrate user value - sometimes called North Star metrics
  • Strengthened learning capability and how to apply to design and development decisions
  • Service blueprint to support scale of delivery

Client Feedback

"Strategic, pragmatic, collaborative and insight driven, Maire was instrumental in delivering a number of complex B2B projects we worked on together in the information services and analytics space." Product Designer, Lexis Nexis - worked on the same team while I was engaged as a User Experience Researcher & Innovation Partner, 2018 - 2020.

"Máire has a rare skill of being able to help her clients build the capabilities so they can continually deliver new innovations to their customers." Head of Media Consulting, Cognizant Consulting - recruited as an Associate Consultant to work on BBC Monitoring's Digital Transformation Programme, 2014 - 2016.