How I helped grow loyalty from the user-base for a leading media intell service provider (2018 - 2020)
posted 1st March 2024
Lead User Researcher for a global information services provider on their media intelligence subscription service, 2018 to 2020.
Tools is a global B2B subscription media intelligence information service.
Below is a summary of one of three projects I worked on during this engagement, which lasted 10 months from discovery to market delivery.
THE CHALLENGE
To introduce a more human-centred and Design Thinking approach to product development and up-skill the team, so that they could focus on driving loyalty across the existing user-base and uncover organic growth opportunities.
THE SOLUTION
Introduced user frameworks and journey mapping to help teams deepen their understanding of user problems, including root causes and ways to alleviate pain points.
Informed and validated a new Quick Search journey focused on a new user segment with growth potential.
THE IMPACT
Increased retention across target users, particularly early adopters of the service: an easy to use service for technically novice users.
Frameworks and approach I introduced were adopted across the global product portfolios and the new search journey went on to win CODiE Best Search & Delivery, 2020.
APPROACH FROM DISCOVERY TO DELIVERY OF A NEW SERVICE TO MARKET
I approached the problem from a human-perspective, starting with gaining a deeper understanding of users, the jobs they were trying to do and their journeys. Throughout I collaborated closely with the Product Director and their team to support skills transfer and transition to new ways of working (from a very technical to a more human/user-centred orientation).
Below is an outline of the overall approach from early discovery to launching the new Quick Search to the market.
UNDERSTANDING DIFFERENT USER GROUPS - GETTING TO THE ROOT CAUSE OF THEIR PROBLEMS
Existing quantitative user research and satisfaction scores showed that there was a wide range of sentiment across the user-base:
- Loyal advocates who were raving fans of the service
- Active detractors who stopped using the service after very limited use (high churn rates of early users).
At the start of the programme the user churn rate was over 70%, mostly after 1 or 2 log-ons to the service.
The data was telling an inconclusive picture so I needed to understand the root cause of this dynamic to be able to influence priorities and design decisions.
I led a qualitative research study to develop user profiles, map as-is journeys and explore ideal journeys. I led in-depth interviews with 12 users across the spectrum, from long-standing users to newer users, with different professional profiles.
The as-is journeys uncovered a pattern:
- Technical users, with roles in Analysis or Data Science, could build Boolean search queries, and valued the service very highly because of the quality and accuracy of the intel; they tended to be loyal advocates of the service.
- Subject Matter Experts like PR or Comms professionals initially found the search easy to use but the results delivered too much noise and they didn't know how to refine the results. They did not want to invest time learning how to use the service. Rather, they expected the service to work a little like Google. They valued the insights, but relied on professional analysts to find insights.
Ideal journeys were also different for each of the segments:
- Technical users wanted to mine deep rich insights, without algorithms (used by Google) as this helped them trust the results. However, they were frustrated with having to do simple searches which were time-consuming and 'users should be able to do it independently'.
- PR or Comms professionals wanted an easy to use search that delivered reliable results. They still wanted to work with Analysts, but only for very complex data needs.
I introduced a framework to segment users by their needs, abilities and mental models (how they solve problems). The framework below helped me and the team to formulate hypotheses about users, their needs and attitudes towards the service.
Working with the team, we formulated new hypotheses for testing:
- The service is valued by users who can build Boolean search queries; these users are loyal advocates of the service.
- Users who may require deep media insights, but who cannot build complex Boolean searches are less likely to be satisfied with the service or be loyal advocates.
I validated these hypotheses with some further research, which clearly indicated that PR / Comms / Marketing professionals did not adopt the service because it was too difficult to learn. There was an opportunity to grow usage with this cohort if we could deliver an easy to use and reliable service.
As I progressed with developing insights and facilitating Design Thinking activities, this framework was very useful as I was able to map divergent needs, ways of working (mental models) and ideal experiences along technical and domain/content expertise (from novice to expert) spectrums.
This framework was subsequently adopted by other global product teams to inform design strategies.
REFINING THE PROBLEMS TO BE SOLVED
Initially the design problem was to improve usability of the service generally to increase retention and loyalty. However, this needed to be more focused as only some users - technically novice users (e.g. PR / Comms professionals) - were not adopting the service. Furthermore, different user groups had different interpretations of easy to use. The same solution would not work for all user segments.
Working with the team, we redefined the design challenge to focus on a specific user cohort:
Design an easy to use and intuitive search journey for professionals who seek trusted and actionable media intelligence, but lack technical skills to build complex search queries (technically novice users).
Ideally this would be a new search journey in addition to the existing Boolean Search, as different users valued different journeys and outcomes.
DEFINING PRIORITIES & BETS FOR ONGOING LEARNING
Insights clearly pointed to technically novice users, e.g. PR / Comms professionals as a potential growth opportunity. I led a series of workshops with the team to define priority bets and corresponding success metrics for a new search journey and components aimed at alleviating pain and driving loyalty with this cohort.
- Each bet was an idea to be tested directly on users and measured against defined success criteria to determine go, no-go or pivot decisions.
- A bet is by implication not confirmed deliverables. Rather it is a viable solution that you can test against a set of outcomes you are trying to achieve. This helps to embed a more learning and iterative approach to the design process.
(Click here for a 2 minute read on how language can help shift mindsets and attitudes to change).
- At each stage of user testing the outcomes for success were clearly defined - sometimes called Objectives & Key Results or OKRs. Go, no-go or pivot decisions were based on the outcomes or success metrics agreed with the team.
(Click here for a 3 minute read on how objectives and success metrics can help engage team members with test and learn cycles.)
IDEATING & TESTING PROPOSALS DIRECTLY WITH TARGET USERS
Working with the team and we started with ideating conceptual solutions that focused on an intuitive simple search journey for technically novice users so they could find media intelligence they needed without compromising on insight quality.
Insights from the journey mapping exercise demonstrated that while users wanted a service that was intuitive, they would not compromise on results, as the results were the moment of truth for most users, regardless of their technical or media expertise.
So the design challenge, led by the UX Designer, was to design a search journey that was easy to use while also delivering quality search results that the users trusted and valued. As the Lead Researcher, the insights I developed infused the design process and go/no-go decisions after every test cycle.
There were three broad test cycles with the target users:
- Test cycle one focused on validating the problem and overall desirability of the proposed service
- Test cycle two focused on testing the query (inputs) such as selecting content sources and refining results
- Test cycle three focused on details around language to ensure design nudges were as intuitive as possible.
(Watch this video explainer on why a test and learn approach supports innovation.)
TEST CYCLE ONE - VALIDATING THE SOLUTION-FIT OF THE PROPOSED SIMPLE SEARCH
Using a low fidelity interface and backend Wizard of Oz prototype approach, I discovered that the users liked the Quick (simple) Search concept and found it easy to use, but would not compromise on search results. Some said they would try the service up to three times only before quitting as they expected services to be easy to use, just like a Google search journey. The results still delivered 'too much noise', particularly for those starting with a broad search question.
Note, a Wizard of Oz prototype relies on a backend person to produce results; there is no technical development investment at this point of the development cycle.
TEST CYCLE TWO - TESTING THE SEARCH & REFINE JOURNEY
Using a high-fidelity prototype, with more search filtering features, I was able to test how intuitive the whole journey was for target users.
- Participants found the journey very appealing, but again would not compromise on results
- Participants tended to find the order of steps along the journey confusing as they expected to be able to select content sources before 'removing noise'
- Participants also wanted more control and transparency on the content sources they wanted to include in their search results, including date ranges.
TEST CYCLE THREE - ENSURING THE WHOLE JOURNEY MADE SENSE
As I progressed with testing I worked with the UX Designer and focused on more details around what design nudges would be intuitive at each step of the journey so that the target users retrieved the results they expected, ideally from the first try, even if they chose to refine the results at a later stage, or before saving the search.
Insights into divergent users' mental models informed the design all the way through development, including:
- As the Newsdesk search was originally designed with Boolean in mind, we needed to make sure the Quick Search was aligned with the conversational style of other target users.
- Users who were familiar with Boolean interpreted words or terms differently. "AND" in a Boolean query reduces results, while for most other people it expands the results, as illustrated below.
- The design needed to give clear nudges to the user as to what would expand or reduce their search. Users were happy to refine their search once they reviewed the results, but they expected the results to align with their mental models, i.e. how they solved problems. Again they did not want to waste time learning a new system, they expected it to be intuitive.
QUICK SEARCH JOURNEY SUCCESSFULLY DELIVERED TO MARKET
The team successfully delivered a new search journey targeted to PR, Comms and Market workers, after 10 months, starting with discovery.
To support ongoing growth, I defined and introduced new North Star (growth) metrics. The North Star metrics where based on value indicators:
- Number of saved searches
- Number of shared searches
The team was able to track this activity and get early signals of problems with the service, particularly with the target growth users.
This service proved to appeal to users across the spectrum. The advocacy of the service increased in the two quarters following launch and retention of new users increased in double-digits, positioning Newsdesk well for growth from the existing user base:
- More experienced and technical users liked to use Quick Search for broad search queries and then mind deeper insights with Boolean queries
- Newer and technically novice users found the Quick Search intuitive and easy to learn were able to manage their daily media intell requirements, relying on experts for more complex or specialist needs.
Note the Newsdesk model is based on individual user licences so increased usage at an individual level was important to demonstrate value and support ongoing sales.
