Multivariate testing technology is helping an online sportswear retailer provide the optimal customer experience and stay ahead of the competition
is an online retailer of sportswear, equipment and accessories, focusing on providing specialist sports products that are not available on the High Street.
Although the operation began in 1987 with a store in North London, today more than 99% of revenues are generated online. Millet Sports launched its website in 2000, and, as online revenues began to grow, redeveloped it in 2009 with the aim of transforming the company into a pure e-commerce business. The relaunch quickly generated an increase in conversion rates of around 30% and led to a view that any future changes to the website should be tested in order to ensure they did not damage the conversion uplifts that the redesign had enabled.
Millet Sports partnered with Maxymiser
, specialists in multivariate (MVT) testing, personalisation and multichannel optimisation, in 2009 to develop a customer experience optimisation strategy. “As an online-only business, it is vital that we continuously improve our customer understanding and are able to measure how people used our site,” said Mike Thornhill, chief executive at Millet Sports.
“Critically, we wanted to know how we could enhance the customer journey to drive an increase in revenues. We had lots of ideas of how we wanted to test, but they were relatively unstructured. We chose Maxymiser not only because they were leaders in the field and had strong testing methodologies, vast experience in the retail sector and a powerful testing platform – but also because they were quickly able to structure our ideas and develop a comprehensive roadmap of what we needed to test and how it should be prioritised.”
Optimising the e-commerce basics
Millet Sports’ early campaigns concentrated on the basket, checkout and product pages – with an objective to ensure that there were no conversion barriers in the way as customers made their journey through the purchase funnel. These tests had a positive impact on conversion and profitability, and underlined the value of multivariate testing. But, as the campaign roadmap has evolved, campaigns have become much more sophisticated. “For example, a couple of years ago we ran a few campaigns whereby we compared our own product recommendations algorithm with Maxymiser’s,” said Thornhill. “The results were really interesting. More recently we’ve begun testing our mobile platform – trying to improve the navigation and customer experience.”
He continued: “In the near future, we’ll be using existing and historical customer data gathered from our in-house CRM [customer relationship management] system to test dynamic merchandising behaviour and exploring how behavioural targeting on the mobile site compares to the standard website. Bringing in the CRM data will inform our targeting tactics and make our messaging and offers more relevant. The long-term strategy is to improve our ability to personalise the customer’s online experience and be better able to deliver the most relevant offers to the right customers across the right channel. And testing will play a huge part in how we implement that.”
As Millet Sports endeavours to optimise – and integrate – all of its communication channels, it’s currently testing a range of areas to identify barriers to consumers that could potentially undermine its multichannel approach. “We are striving to develop a single view of the customer, and how individuals interact with us across all of the channels,” Thornhill reported. “Do they shop with us directly, or click on an affiliate banner that takes them to our site? Which devices are they accessing us from, or what route have they taken to arrive at the website? We are constantly working to develop a rich view of the customer, to understand how they are behaving – so that we can create more targeted offers and drive brand loyalty. Our work with Maxymiser is absolutely critical to this.”
The company is currently testing a range of areas. These include ‘abandoned basket’ retargeting, post-purchase lifecycle targeting and examining the general newsletter cycle, as well as how it integrates with the customer journey.
Gaining greater customer insight
Millet Sports believes its partnership with Maxymiser is providing deep insight into how customers – groups or individuals – are using different channels. “Whether users are accessing via the mobile or desktop site, the time of day and the device they’re using, can all have an impact on their propensity to purchase,” added Thornhill. “Understanding that multichannel experience is therefore massively important in today’s marketplace. And testing is the best way to get it.”
The campaigns that have had the biggest impact have generally been those that have generated conversion uplifts. For example, an early campaign around the shopping basket led to a double-digit rise in conversion. “This was a great result and it really reinforced the value of developing a testing culture - but increasing conversion is not the only metric,” said Thornhill. “For example, we conducted a series of campaigns around the checkout; we needed to extend its international capabilities and to test changes we’d made in line with PCI [Payment Card Industry] compliance requirements. This was a relatively advanced test and while it did lead to a slight conversion uplift, more importantly, it meant that we were able to get instant feedback to show that the necessary legislative modifications we had made were still positively affecting the customer experience.”
Some of the most valuable campaigns that Millet Sports has conducted have led to no changes being made at all. “We’ve done a lot of work around the layout of our product pages – all intent on ensuring that information is presented in a way that gives customers what they need and the confidence to click through to the basket to make a purchase. Interestingly, through all of these campaigns, the default layout – which we designed five years ago – has always proved to be the winning experience. A lot of people think that campaigns always need to generate a conversion uplift. But we’ve learned that a neutral response can help you understand that there are parts of your site that are optimised – and provides a clear metric to help you establish that.”
Assessing the value of change
A great example of this was a recent campaign that focused on the mobile basket. The campaign set out to determine the optimal level of content – specifically non-essential items in the basket – to be displayed on the mobile basket page to encourage visitors to click through to the checkout. It explored whether existing items of content were distracting users and whether explanatory messaging had any influence on customers’ propensity to move to the checkout. The campaign showed conclusively that the existing page was well optimised. In the process, it mitigated risk and prevented Millet Sports from implementing changes that would have had a detrimental impact on the performance of a key part of the mobile sales funnel.
So is optimisation the way forward for online retailers? Thornhill believes that it is. “It always surprises me when I hear that some people are still debating whether to do optimisation campaigns,” he said. “In truth, I think anyone who wants to make big changes to the customer experience without testing their impact is mad! If you don’t know the impact of the changes you’re making, you can never be confident in the decisions you take. That’s inevitably going to hit your forecast. A couple of percentage points damage to conversion rates can do serious harm to a business. But more strategically, without optimisation you can never be sure how your business is really performing against customer expectation.
“Our experience with Maxymiser has been great. Every year we have more than justified the investment we’ve made – and we’re really happy with the approach they take and their vertical specialism. The kind of campaigns we’ve been running and the roadmap we have got in place are really working. Not only are we really pleased with the results but, more crucially, our optimisation work is actively guiding both ongoing business decisions and our strategic business direction.”