A/B testing: CTA
When creating an actionable A/B test, I set out to find the following answers to these questions:
What do I want to test?
Why do I want to test it?
How much time is it going to take me to run this test?
What am I going to do with the results?
And with the above answered, I can feel confident to move forward with a solid testing strategy knowing that I’m not wasting time.
Introducing, A/B testing THE CALL TO ACTION.
What do I want to test?
In SaaS product announcements, brands frequently announce new features by telling me to do something. Rarely, do they show me the value of why doing something would be of benefit to me.
I want to test a call to action (CTA) within my email to determine if instead leveraging it as a call to value, will result in higher engagement.
Variant A: a call to action focused new feature announcement - highlights what you can DO with the new update
Variant B: a call to value new feature announcement email - highlights the BENEFIT of the new feature
Why do I want to test it?
I want to know how the way I describe a new feature affects engagement and conversion.
How much time is it going to take me to run this test?
This is a simple test, but it will take me some time to convert the action statements briefed in from Product Marketing, to value statements. I would say in total this test is about an hour of work to create and execute.
What am I going to do with the results?
I would like to take my results and implement them in all future product announcements or emails that reference features that subscribers might not fully understand to ensure they know what they might be getting out of the tool.
My hypothesis
My hypothesis is that variant B is going to see higher engagement, and bottom of funnel conversion to feature adoption due to:
Understanding the value of the feature
More awareness of the benefits
The results
This is a test I have run multiple times now, and each time-variant B comes back with statistical significance for both email engagement (CTR) and conversion. By removing the guesswork in the email copy that might cause a user to hesitate towards taking an action, and instead outlining benefits, more subscribers seem to engage with the message.
