The secret of successful A/B tests: How we reduced the bounce rate of RapidUsertests by 31%.

Let’s face it, haven’t we all made gut changes to our website or app without first analyzing the exact problem? We are guilty of this too. If you check these adjustments in an A/B test, it often shows no changes or even a deterioration and you are no smarter afterwards than before.

But there is another way – through structured analysis and sound hypotheses we were able to reduce the bounce rate of our website of our tool RapidUsertests by 31% and increase the conversion rate by 47%.

How did we do that?

The challenge: Bounce rate and conversion rate with potential for optimization

RapidUsertests.com is our tool for crowd usability testing. After a redesign in early 2017, we had made no more changes to the home page, but the quantitative analytics data showed us a comparatively high bounce rate and potential for increases in conversion rate.

Quantitative vs. qualitative data in conversion optimization

Without question, quantitative data is important in generating A/B test hypotheses.
As in our example, they can show you where problems are occurring on your website through KPIs like abandonment rates and dwell time. But what they don’t show you is the underlying why. This already starts with the dwell time: Do users stay on a page for a long time because the content is relevant to them or because they don’t find the crucial information? Do your users call up many different individual pages because they find your offer so exciting or because you are not well managed?

You can only obtain this information through qualitative data.
This can be a classic UX test, but an expert review by conversion specialists also provides important information. Do you already have insights about your target group from user research and perhaps even use personas and scenarios in your company? Wonderful – they also contribute to being able to analyze what information your users need, where they need it, and what challenges they want to be picked up on.

The process: With conversion analysis to well-founded A/B test hypotheses

Our conversion specialists therefore combined quantitative and qualitative methods to uncover conversion killers:

• In a
  Usability Test
  we were able to understand the motivations and emotions of our customers and prospective customers. How do they understand our offer? What convinces them? What are their concerns?
• A conversion expert review revealed further optimization potential for the website. Unlike our classic UX analyses, we focus here particularly on conversion potential and less on user experience.
• We then combined these findings and existing knowledge about our target group in the form of personas in a motivational analysis: What bothers users about alternative solutions? Where do other solutions have advantages? What appeals to them about our offer? What concerns do they have? etc.

The main findings of these analyses: the communication on the homepage could be more convincing and the user guidance had potential for improvement.

Generate hypotheses that consider the WHY

From the results of these different analyses, we were able to formulate well-founded hypotheses that follow the following scheme:

If
we make that change, then that metric will improve,
because
this problem will no longer exist. – share on Twitter

For example, one of our hypotheses was:

If we address the needs of our customers even more clearly above the fold in our advantage communication, thenthe bounce rate will drop because users will immediately understand the relevance of our offer.

Important when formulating the hypotheses: Both the problem and its causes, as well as the planned solution and the expected result must be included. If one of these components is missing, your analyses were probably insufficient and the risk of an unsuccessful A/B test increases.

Based on these hypotheses, our UX conceptors and designers designed a new home page stage, which we tested against the original variant in a split test.

The result: 31% fewer bounces and 47% more conversions.

The result of the A/B test was impressive: the bounce rate decreased by 31% compared to the control variant with a significance of 100%, which is why we made the new variant live for all users.
Since then, we’ve also seen a significant increase in conversion rates of over 40%.

Conclusion: Sound hypotheses and successful A/B tests through UX analyses

Combining A/B testing with conversion and UX analytics means you:

• You’ll uncover the biggest conversion killers while getting to know your users better.
• They generate well-founded hypotheses and thus increase the chances of success of the A/B tests.
• They continuously optimize conversion rate or other KPIs based on a variety of prioritized hypotheses.

Can we support you with the analysis, hypothesis generation and execution of the A/B tests?