Experimentation

Driving experimentation success through data and insights

The background

Each year Shelter helps millions of people struggling with bad housing or homelessness through expert advice, support and legal services. Digital is a vital channel for efficient acquisition due to its strong return on investment. However, increasing media spend was starting to see diminishing returns, and so on-site optimisation was the most logical step for increased growth moving forward. Shelter had run some on-site optimisation projects in the past but their hypotheses were often based on assumptions and hadn’t demonstrated much significant improvement.

The ambition

Shelter sought to improve acquisition performance by increasing landing page and funnel conversion. The charity has a history and culture of delivering solutions using their in-house team, meaning it was crucial to identify opportunities that they could deliver with our specialist help. We looked for quick conversion rate wins alongside long-term ‘change’ projects.

Our approach

Discovery 1 Analytics audit: A deep dive to understand traffic, audiences and paths to conversion which would identify conversion drop off points, alongside measurement gaps.

Discovery 2 – Research: Conducting in-depth user interaction studies, remote usability testing, alongside on-site surveys to uncover visitor behaviour and attitudes including audience motivation, content gaps and anxiety factors.

By combining the analytics and research findings we found many actionable insights from which we could generate A/B test hypotheses.

How we decided what to test?

  • Attaching potential benefits to each test hypotheses and ranking priority of these benefits to Shelter.
  • Prioritisation and scheduling of tests were also informed by an understanding of the capabilities and complexity of the current tech stack.
  • As well as an ongoing program of experimentation we were able to identify quick fixes requiring relatively little effort driving immediate conversion performance uplifts.
“Daydot not only provided us with some fantastic insights into our own customers, but also enabled me to substantiate a number of UX sticking points I had always suspected were impacting conversion, but could never prove. The performance uplift from the quick fixes alone meant that the project paid for itself many times over.“
Eoghan Beecher, Digital Fundraising Manager, Shelter UK
The results

Quick wins & bug fixes: The research and data analysis drove over 20 actionable insights around bugs, errors, and quick fixes which could be implemented immediately by the Shelter team.

  • 14% uplift in donations
  • +12% earnings per click
  • 726% ROI

Longer term experimentation roadmap: The first experiment in the roadmap focused on the core donation landing page, seeking to address value proposition, anxiety and content gaps highlighted through our research. This would drive increases in average donation value. Some of the specifics that we looked at included positioning of donation amounts, the mechanics of the donation form, and customising the description of impact for each donation amount.

After successful completion of this first experiment, Shelter we able to report:

  • 45% uplift in donation value
  • 6.43% uplift in conversion rate
  • +1,000% ROI