If you have a digital product, you must instill within the customer the habit of using that product, by meeting their needs, you create a closed loop of use, satisfaction, and continued engagement and purchase. A subscription layer can also be built into many businesses and becomes an important driver of growth and recurring revenue.
The first step when considering habit formation is customer research – analysing subscriber data to identify habits which drive the most engagement. However, this cannot be done only once and forgotten. Subscriber mindset, purchasing priorities, and behaviours are constantly evolving, as illustrated by the impact of the Covid pandemic on many sectors (just two examples being the increase in publishing subscriptions and food recipe boxes). It’s not enough to implement based on assumptions, to achieve long-term sustainable revenue growth brands must identify customer habits and then continue to experiment, optimise, and re-analyse the data as part of an ongoing product testing program.
Habits extend into weeks, months, and even years and we cannot run individual tests for this long. Doing so would stifle the required speed to gather insights and the ROI of an optimisation program would plummet. So how do we test to promote habits, through active UX (user experience) changes and the opaque world of algorithms?
Testing for habit formation requires multi-session KPIs and identifying actions which relate to habit formation and these concepts require statistical analysis that no A/B testing platforms offer out-of-the-box.
In my experience it is essential to always have a clear answer to the question: ‘What does a winner look like?’ Be clear what KPIs you will measure that indicate or correlate with habit formation.
Let me explain further, each time a user visits your site is an opportunity to form a deeper, long-lasting relationship with them, engaging them with features that promote habit formation. In order to identify a winner in an A/B test – with the objective of creating habits for users –we must make casual users look like habitual customers who will return to your product.
Of course habit formation is complex, and goes beyond a single session, however, there are insights you can gather within a session to discover whether that particular session is helping a user create a habit – and crucially a relationship with your product.
If a user is opening new channels of communication with you, by subscribing to a newsletter or a podcast, this is typically a good sign, and an increase in signup is a simple, measurable outcome for an A/B test.
Do certain products or content increase loyalty? Do you have a writer whose articles generate comments, or a product that users purchase repeatedly? You can discover insights about habit formation by measuring page visits and CTR on key elements leading to or on these pages.
At the most complete level, identifying habit-forming actions is done by tracking all actions that users take on-site and measuring their correlation with user propensity to churn. Once this level of data science infrastructure is in place, craft a testing strategy which focuses on your site’s actions that reduce user churn.
If we define a user with a habit as someone who returns regularly to your website, then the ultimate measure of whether a user has formed a habit is whether they are returning to and using your website more frequently.
For example: Google Analytics provides us with a Frequency and Recency report which allows you to work out the average number of visits to a site within a given timeframe.
If we integrate our A/B testing platform with our site analytics platform then we can create segments for users in different test variants and compare the average Frequency and Recency for users who are being tested.
Now we’re getting somewhere!
If we are able to show that a difference in average Frequency and Recency is statistically significant then we can claim to have increased a user’s habits! However, out of the box A/B testing platforms are not equipped to handle this kind of statistical test. Indeed, you’ll run into trouble trying to enter this data into any standard A/B testing calculator!
We’ll have to draw on our wider statistical knowledge and a little bit of data science know-how.
A Student’s T-Test is a statistical hypothesis test widely used in the world of finance and valued for its generality. It was originally developed to aid in the production of Guinness!
To use the T-Test we need the average Frequency and Recency for a set timeframe, and the variance. This is where the data science capabilities enter the frame, as it requires a user-level view of the data. This means granular tracking of users in test groups and the ability to export the Frequency and Recency for each user.
Once this capability is in place the analysis is simple: statistical software such as R and Python have built in libraries that will crunch the numbers for you.
Changing habits can be slow and subtle, challenging of course, but ultimately rewarding, and there’s no time like the present. So get out there, identify what actions on your site correspond to habitual users, and track how individual users respond to the bold changes you start to make – success is just around the corner.
Optimisation Manager, Daydot