- What’s the likely impact on A/B testing and optimisation?
- Is human interaction still important?
- Where can AI bring efficiencies and advancement in strategy and implementation?
When you search for answers to these questions there is a lot of AI scaremongering out there so this article aims to answer these questions and give my point-of-view on the topic.
To properly understand the ongoing importance of AB testing, it is first important to recognise what things AI driven optimisation is valuable for:
- 1:1 personalisation of messaging for different audiences
No matter how mathematical an individual’s brain, there will always be a limit to how many segments can be managed before becoming too complicated. Especially when factoring in contextual data such as user activity, affinities, geography, etc. And with a host of permutations and combinations, picking a winning variation in the face of a constantly changing customer base becomes impossible. AI driven personalisation makes this hyper-personalisation a possibility that is manageable.
- Accelerated roll out of optimal variants during time sensitive periods
For instance, when it comes to experimenting with short-lived events for say, a holiday or back-to-school event, instead of running an AB test and trying to optimise on the fly, machine learning algorithms are able to predict positive outcomes for each individual and thus maximize revenue over the duration of the entire campaign.
Although AI driven personalisation is great at managing hyper-personalised copy and imagery, there are many things that AB testing is needed for:
- Establishing the optimal design foundations for key components
In order to establish the foundations for ongoing AI optimisations we first need to design, develop and test the base foundational page assets. These assets (i.e. layout, imagery, copy etc.) can then be further optimised though AI automation. A well-thought-out AB testing program is crucial to design the base page assets effectively.
- Protecting against risk
In many instances a business will want to develop products and components through user testing and UX best practice. However, even when this process is well-thought-out users can behave very differently in a real-life environment.
AB testing helps mitigate risks associated with launching new features or designs by allowing for iterative improvements and identifying potential issues before a full-scale rollout. By incorporating AB testing into the product development process, companies can enhance user satisfaction, increase conversions, and ultimately drive business growth.
- Learning valuable lessons
Testing and optimisation isn’t as simple as winning or losing. In fact, there are times when losing result can lead to big insights. In these cases, the difference between learning something new and a failed test lies in the analysis—in refusing to take a result at face value. Through AB testing we can gain meaningful insights from failure that help us design better future solutions.
- Experimental design and control
It is important not to underestimate the value of good experimental design. Thinking in advance about what you’d like to learn and having the underlying observations in data at your disposal is an important primer for developing successful products. Understanding what has worked well, and what hasn’t, is required to successfully iterate and develop features that will be meaningful to your users
AB testing strikes a balance between exploring different options and exploiting the best-performing variations. It allows for controlled experimentation with multiple variants simultaneously, ensuring that valuable insights are gained while optimising performance.
As you can see from the article so far, AB testing and AI should to be combined for the best outcome. Incorporating AI and hyper-personalisation will allow you to progress and scale your optimisation program at speed.
However, it is also worth noting the dangers of using algorithm assumptions.
- AI-driven optimisation relies on assumptions about user behaviour, which may not always hold true. AI is as good as the data it learns from, and so particularly in the early stages there is a risk that it won't be tuned for your particular company and customers.
- AI also doesn't have the opportunity to learn from qualitative insights - actually talking to customers and understanding how they feel. AB testing can and does incorporate qualitative insights and know best practice through the processes of human ideation and prioritisation, potentially making it more robust and reliable in real-world scenarios. This process also stands a better chance of picking up more extreme changes of context and commercial environment than an AI process which is limited to incremental changes.
- There is danger that AI optimisation has potential to enhance and amplify the implementation of dark patterns, manipulative design techniques that deceive or coerce users into taking actions they may not want to, leading to unethical and harmful user experiences.
We hope you found this article useful. If you would like to talk further please feel free to drop us a line at hello@daydot.agency and as a Senior Optimisation Manager I would be happy to set up a call, or start an email exchange.
Senior Optimisation Manager, Daydot