The basics of email testing

The Basics of Email Testing

Every Email Marketer you talk to will tell you about the importance of measuring the success of your emails. 

In fact, we’ve written a great blog post about getting the most out of your email metrics here. 

However, most email success is measured after the email has been sent. Did you know that there is a way to incorporate some testing elements while the email is in the writing/design stage? 

When it comes to testing emails, it’s important to know which metrics are relevant and most effective to look at. How will your email look in multiple inbox clients? Will it pass the spam filter? Do the graphics render correctly? 

All of these things can be looked into before sending, but there are also ways to test the success of your email by using A/B testing methods. 

How to use A/B testing?

A/B testing is the process of sending out two emails that vary slightly, and comparing their success. It’s important to make sure that they don’t vary too much, because otherwise you might lose sight of the metric you want to optimize. 

To set up an A/B test, identify the email you want to send. Let’s say as an example that you want to send a reminder email to someone who hasn’t visited your website in a few weeks. 

Net, identify the variable that you want to change. Make sure to only test ONE variable at a time! 

Here is an example of how to send the same email, but with two different subject lines to test for. 

Email A:

Subject line A: Just updated the shop!

Body: Hi [first name,] Did you know we refreshed the items in our shop based on a mix of everything that makes our style unique?

Explore what’s new today! I’m sure you’re going to have one of those “love it when you see it” moments.

CTA: Shop Now

Email B:

Subject line B: Fresh winter styles are here!

Body: Hi [first name,] Did you know we refreshed the items in our shop based on a mix of everything that makes our style unique?

Explore what’s new today! I’m sure you’re going to have one of those “love it when you see it” moments.

CTA: Shop Now

These two emails have exactly the same body copy, but different subject lines. Note that the subject lines express different statements. The first is more general, and says that the whole shop is updated. The second get’s specific, saying that “fresh winter styles” have been added. Which will do better? 

If we send these out, we would send 50% of our contacts in the campaign Email A, and the other 50% Email B. Let’s say we did it. Now, we can look back and see which email performed better. In our example, Email B had a CTR of 10%, whereas Email A only had a CTA of 3%. Email B was the clear winner!  

Since Email B did so well, we can use this data going forward to design subject lines more like the successful ones, and experience higher CTR’s overall. 

Useful things to test: 

Our example focused on testing the subject line, but there are many other useful things to track in A/B tests. You can test for the following:

  • subject line
  • preview text
  • body copy – long emails or short?
  • time of day
  • content length
  • use of graphics – heavy or light?
  • CTA’s – link or button?

Improve based on results!

The reason why we test emails is to maximize their effectiveness at getting the desired results, whether that means CTRs, open rates, or conversions. It’s important to listen to the metrics you test for. If you get good results, replicate the strategy that works! 

Get into the habit of creating multiple versions of every email. This way testing will always be on your mind, and you can always be looking to improve. 

Don’t test randomly, create a process. Using the list of useful things to test for above, create a pre-launch email design checklist based on the metrics you want to test for. This might mean leaving space for two subject lines, different graphics, no graphics, different CTAs, etc.

 Go through the checklist to make sure you have two versions of every email, and are keeping consistent track of the success of the metrics. This will help you improve based on real results!

Other ways to increase the relevancy of the message

If you’re still experiencing very low engagement with your email, despite all your hard work, you can use segmentation or dynamic content to increase the relevancy of the message 

Segmentation is when you put your subscribers into smaller groups based on similarities such as interests, preferences or behaviors, and send out “personalized” emails to those groups.  

You can segment based on demographic, age, gender, past purchases, position in the sales funnel, professional relationship to you, and much more. 

It’s all about sending the right email to the right person at the right time. Try modifying your message content based on interest, gender, subscriber status, job title, and more.

Develop an ongoing process

A great way to keep your email testing on track is to run 4-6 week testing plans every quarter. If you need any help setting this up, I’d love to get in touch and see what I can do to help. 

Share this post

Share on facebook
Share on twitter
Share on linkedin
Share on pinterest
Share on email