Focusing on the people, processes and politics of an experimentation program is a good start, but a common mistake is forgetting to invest in the right tools. Many organisations spend a lot of time and effort hiring and training the right people to run their experimentation program, only to select the wrong technology to support their objectives.
Imagine having a Formula 1 driver on your team, only to have them drive a Mini-Minor - it's a waste of talent. Put them in a Ferrari and it's a different race.
But what do we mean by the 'right tools'? Let's start with some of the gaps teams might encounter with their current set up, before we delve into what they should be looking for in their tech.
Albert Einstein said if he had one hour to solve a problem, he would spend up to two-thirds of that hour attempting to define the problem.
"If I had only one hour to save the world, I would spend fifty-five minutes defining the problem, and only five minutes finding the solution.“ - Albert Einstein
A lot of organisations don't have the right insights to understand customer problems. Most use analytics data and their own heuristics to identify problems, but by doing so, apply their own assumptions to bridge the insights gap.
For example, the team might identify one web page as having a high exit rate, and run a bunch of experiments in an attempt to reduce that exit rate. What they should do first is find out why the page has a high exit rate. Trying different tactics to reduce the exit rate assumes that the exit rate is bad; what if people were leaving the page because they found what they wanted? Maybe they weren't ready to buy, but wanted to save the information for an in-store visit. Or maybe they're exiting because of a bug; perhaps fixing the bug will save you dozens of subsequent experiments. Establishing the context of the problem is essential.
So which tools should you consider as part of your experimentation suite? Just looking at analytics can show you potential issues, or areas where you may need to take a closer look, but without an amalgamation of technologies you're not going to get the full spectrum. For that you need more data points, from customer research to session recordings, or call centre data to panels.
A lot of organisations reach for what's available rather than first identifying their needs. A really good experimentation tool has a mix of technologies including DOM scraping tech like Contentsquare, and voice of customer capacity, such as Askable.
If you're an omni-channel operator, you should also consider integrating knowledge from your other channels. For example, are people contacting your call centre because your chat function is broken? Are visitors frequently researching your products online before buying them in-store? Are customers more likely to have a higher cart value when shopping through your website, or if they interact with a salesperson face-to-face?
Anything that can give you a wider picture and more context as to what's happening on your site should be in your toolbox. Importantly, this tech needs to integrate with your existing setup.
One roadblock you may come up against in your bid to acquire the right tools is buy-in to invest. Heavy investment in martech in recent years means there's a lot of tech debt to answer for, so adding more to the stack can prompt difficult budget conversations.
The language you need to employ should focus on having the right stack to find the right problems, and then giving it to the right practitioners to deliver the business outcomes you are after. At the moment, we're using analytics to try to understand problems beyond its capabilities. As psychologist Abraham Maslow wrote, "It is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail." We need more tools and the right tools to attain the right insights, instead of running 20 different experiments to get close to what we want to understand.
Talk to tech vendors about your needs. A lot of them are willing to run proof of concept, so you can start to build around your existing experimentation tools with tools that integrate and provide the right context. From there, your experiments will become more meaningful and better connected to your business outcomes.
Curious to understand how your experimentation program compares? Deloitte Digital's 2022 Experimentation Maturity index features insights and learnings about corporate Australia's adoption of a 'culture of experimentation'.
This article was originally written by Nima Yassini, Partner at Deloitte Digital.
