Jamie Ferguson

Process

Phase 1: Discover and define

This project spun out of a broader project where we were looking at what a more personalised experience could mean for our users.

Our first concept that led us down this path was an augmented search experience which we tested with some young people. While we got positive feedback around the idea, the actual implementation was not found to be ideal — young people were familiar with search working in a certain way and a generated answer was not seen as fitting with the established mental model at the time (this was before Google introduced their 'AI Overview'!).

This experiment also coincided with us working with a data scientist who was able to create a working proof-of-concept of a question and answer based tool using a RAG (Retreval Augmented Generation) based approach. Impressed by the answers that were generated, this gave us the confidence that the idea was viable and and enabled us to focus on ideating the concept further.

Phase 2: Ideation and testing

While we refined the proof-of-concept I ran some rounds of research looking into what a trustworthy, useful answer looked like for a young person. This provided us with guidance on how to best present the answers and what kind of supporting information we would need.

We were also able to quickly spin off a version to test with our Peer Workers as part of being able to provide personalised support links in their chats.

Phase 3: Refine and release

While the organisation was generally exciting by the idea it was clear that there was understandable concern around how something like hallucination could damage the brand.

Even though the use of the RAG concept meant that hallucination was extremely unlikely we decided to avoid generated answers altogether for the initial release and instead use an LLM to 'match' the user query to a pregenerated question and answer pair.

Outcome

As the main purpose of the first release was to try and understand how young people would interact with a tool such as this, I set up a dashboard in Heap to monitor how it was being used, and we also kept a log of all the queries that had been asked.

We have continued to iterate on the concept and have learned a lot about the kinds of things a young person asks a tool such as this.

Currently we are to combining these learnings with those from some co-design sessions with young people, helping us define how we transition to something more truly generative which will allow ReachOut to provide better and more personalised pathways to support.

Process

Phase 1: Discover and define

After doing a full review of the current platform, I conducted some interviews with SMEs in the organisation to try to understand the pain points and requirements.

Rather than build another bespoke solution, I suggested we consider buying and integrating a 3rd party plaform. Having previously worked at a SAAS company I knew how much effort it takes to build and maintain a platform and it didn't make sense with the kind of team we have at ReachOut.

Once the business had agreed to this approach, the Product Manager and I began researching potential platforms, eventually decided on two different options - each with it's own set of compromises. This also coincided with the hiring of the Coaching Team Leader who was able to work with us to find solutions to those compromises.

I ran some user research activities to better understand the needs of parents trying to support their teens through tough times, as well as testing the performance of the current landing page.

This highlighted the challenge that the concept of 'coaching' was not well understood in this context, so we looked for ways to improve how we communicated the service.

Phase 2: Ideation and testing

While narrowing down platforms I worked with a Content Designer to design a prototype onboarding flow.

We then tested that flow with some parents to get a sense of any remaining friction, along with some revised landing page concepts. We refined the prototype as we went and ended up with a flow that was significantly improved from the current state.

Phase 3: Design and develop

With the platform decided on and a clearer idea of how best to communicate the programme to parents, I was able to finalise the design of the landing page as well as start to build out the final onboarding flow in the new platform.

As we had opted to go with a 3rd party platform, the actual UI design requirement of the coaching experience was limited to the landing page itself, however I was heavily involved in the experience design within the platform itself as well as helping to support the coaches in transitioning to this new platform.

Outcome

We now have a much improved booking experience for parents, and more importantly one that can be updated as needed with no engineering effort. We also ended up with a landing page that better communicates what is a difficult to understand proposition, and that better reflects the ReachOut brand.

I also set up a dashboard in Heap to see what effects the changes have made.

The change did not come without some challenges. As well as the inevitable difficulties that came with the coaches needing to become familiar with an entirely new system, there were also some functional issues with the platform we chose that, despite it being the best option available, will never be as fit-for-purpose as a bespoke solution. While this is frustrating, overall the flexibility afforded by a third-party platform has seen the team able to iterate and improve things rapidly.

This has meant that the coaching team has been able to more easily make changes to their process, improve their workflows and more easily meet their session targets.