With the launch of the 2022 Experimentation Maturity index, we are excited to share some great insights and learnings about corporate Australia’s adoption of a 'culture of experimentation'.
Using the 5 stage of maturity model (Initiating, Building, Scaling, Maturing, Driving), we have reviewed over 250 brands and their use of experimentation as a means of 10-xing their business.
Since 2019 (our first report), Experimentation is continuing to show solid double digital growth. 2022, have as seen a sharp increase with 50% increase in organisations investing in and scaling up their experimentation programs.
Like previous years, experimentation is still finding challenges moving from Building to Scaling. Most of this is due to poor set-up in previous stages, lack of executive buy-in, availability of resources and knowledge on how to scale. All of these mean that many brands are struggling to demonstrate the full value of what experimentation can bring to an organisation.
One notable anomaly in the 2022 report, is the stagnation of maturing organisations. 59% identified challenges with resourcing constraints, more so than respondents in the initial stages of their experimentation journey (42%). These respondents also indicated that ‘having reliable measurements infrastructure’ was a challenge (46%) over the newcomers (38%), which actually signals to me that they are not as 'well established' as they’d like to think.
Either way, this indicates established practitioners are not getting the buy-in to secure resources that we might expect of programs at that phase of maturity, which could limit future growth of experimentation within organisations and overall maturity across the board.
For those who’ve watched the uptick of experimentation in a global context, it will be no surprise to discover that there is clear growth in the category in Australia. More organisations, and a greater number of organisations of different sizes across different industries, are choosing experimentation as a means to improve their offering, from optimising customer experience and gaining a better insight of customer behaviour to using experimentation in product and web development to make improvements.
BMW, General Pants, P&O Cruises, MAIA Financial
One of the biggest indicators that experimentation is growing, is the rise in use of low-cost testing tools such as Google Optimize. The increase in accessibility of testing tools is evidence of the practice moving from early adoption to an early majority stage, giving new entrants the ability to start a low-risk, low-outlay, basic experimentation program. This is a good sign, as experimentation is all about testing and making data-led decisions so being able to prove its value is one way to strengthen the mindset and culture in the organisation, before investing in a more comprehensive program and tools like Optimizely, Adobe etc.
Just over two in five respondents use an external team to run their experimentation program and 32% have a single-member team. However, organisations further along the maturity curve tend to operate across teams rather than in silos, which indicates both a sharing culture and broader opportunities for experimentation to evolve in the organisation.
Organisations at the beginning of their experimentation journey struggle with educating the team to run suitable experiments (40%) and having the expertise on hand to help set up experimentation in the organisation (28%) compared to more mature organisations. Much of that can be, and is, addressed with a hybrid approach – bringing in specialists to create a good foundation for experimentation practices while training and embedding experimentation culture in the organisation to build capability from within.
The second area that new entrants struggle with more than advanced organisations – 18% versus 7% for the most mature – is inspiration, or knowing what to test. This is an area that also gets easier with experience and specialist help.
By comparison, those that are scaling, see resourcing constraints (59%) as a bigger issue and they are seeking reliable measurement infrastructure (46%) and knowledge to run experiments (39%).
This is in line with practitioners who have outgrown the initial phase and are starting to better define what they want from their program. This may be the point at which they seek different tools and restructure the program to better align with business KPIs. It is also the point at which many programs seek more team members as they grow or wean themselves off external assistance.
The most mature organisations experience similar challenges to those in the scaling phase: resourcing constraints (59%) and reliable measurement infrastructure (46%) are issues, as is managing potential knowledge loss when team members leave.
Interestingly, respondents in the scaling phase were more likely to have a sharing culture than the most mature organisations (63% to 51%); improved sharing and good knowledge and data management may be the key to reduce the impact of talent loss.
With the uptake of experimentation on the rise, it’s clear that simply having an experimentation program is not going to offer the competitive edge that it once did when the practice was in its early adoption stage.
As the practice grows, we need to tread carefully on three accounts:
The report shows the greatest flaw in most programs is a lack of executive buy-in and connecting the program back to the metrics that matter to the business. Better alignment with business KPIs and using results data in key business decisions will go some way to connecting the practice with the organisation’s goals and strategy.
The most successful programs focus on learning rather than winning and losing. This is a cultural shift many organisations need to make from within.
What’s needed now is a clearer picture of what ‘best practice’ is. A report like this gives organisations a benchmark and an overview of challenges but it looks like we still have a way to go before we can call Australian experimentation “mature”. One thing’s for sure, the next phase will reveal a lot about where we are and where we’re going.
Contact us for a FREE copy of this years experimentation maturity index.
A version of this article was first published on LinkedIn 2022. This article was originally written in collaboration with Stacey Isaac and Nima Yassini, experimentation leads and Partners at Deloitte Digital.
Leora leads our national Customer Strategy & Design team and Melbourne Deloitte Digital practice. She has 20 years+ of advisory experience specialising in Customer Experience (CX), Customer Growth and Value Management, Product and Audience Experience, particularly in the Entertainment, Media, Telecommunications and Retail sectors. Leora's background incorporates consulting, operations management, product management and screen advisory roles in both Australia and the UK.
