Experimentation

Driving experimentation success through data and insights

The background

Shelter helps millions of people struggling with inadequate housing and homelessness each year through expert advice, support, and legal services.

The ambition
  • Increase donations
  • Drive donation amount uplift
  • Measurably prove the value and ROI of testing to create a business case
  • Deliver and actionable roadmap of tests which will increase support acquisition performance
Our approach
Data and research:
  • Analytics audit: A deep dive to understand traffic, audiences and paths to conversion which would identify conversion drop off points, alongside measurement gaps.
  • Research: In-depth user interaction studies, remote usability testing, alongside on-site surveys to uncover visitor behaviour and attitudes including audience motivation, content gaps and anxiety factors.

By comparing the analytics and research findings we found many actionable insights to help us generate A/B test hypothesis. These ideas formed the basis for the experimentation roadmap.

Roadmap testing ideas were prioritised by:
  • Identifying quick wine that would be done without testing.
  • Understanding the capabilities and complexity of Shelter's current tech stack.
  • Attaching potential benefits to each test hypothesis and ranking priority of these benefits to Shelter.
The first experiment in the roadmap:

We focused on the core donation landing page, seeking to address value proposition, anxiety and content gaps highlighted through our research. This would drive increases in average donation value.

Some of the specifics that we looked at included positioning of donation amounts, the mechanics of the donation form, and customising the description of impact for each donation amount.

The results

After successful completion of this first experiment, Shelter were able to report:

  • +45% uplift in donation value
  • +6.43% uplift in conversion rate
  • +1,000% ROI
  • A clear rationale for budget to deliver more testing
“Daydot not only provided us with some fantastic insights into our own customers, but also enabled me to substantiate a number of UX sticking points I had always suspected were impacting conversion, but could never prove. The performance uplift from the quick fixes alone meant that the project paid for itself many times over.“
Eoghan Beecher, Digital Fundraising Manager, Shelter UK
Latest case studies
Experimentation
Vox Media
We achieved in 57% uplift in subscription conversion rate by making the subscription acquisition flow personal for each user
Experimentation
The Wall Street Journal
We achieved a 240% increase in new member onboarding actions leading to 18% uplift in customer retention rate
Experimentation
RSPB
Increasing conversion rate by +5.6% and generating £1.5m incremental revenue from our first test