Trial and error: How to improve your email marketing with testing

Trial and error: How to improve your email marketing with testing

We know how intimidating email marketing can be, which is why we produced a beginner’s guide to it a few weeks ago! There’s far more to email marketing than we could ever put in that blog, though, which is why we wanted to continue our series and tell you a bit about how to improve your emails by using testing. This kind of testing is commonly referred to as ‘A/B Testing’.

Testing your emails involves using a variety of different techniques to find out what works for your customers and what doesn’t. As an example, you might find through A/B testing that a shorter subject line leads to better open rates from your list than a long subject line. (Don’t worry we’ll explain terms like ‘open rate’ as we go!)

Need some support with your email marketing? redPepper Marketing are always here to help! Feel free to give us a call at 01787 223850 or book a meeting with us. Our in-house email experts can help you diagnose problems and give your email marketing a breath of fresh air!

How are emails A/B tested?

Before we start talking about specific parts of your emails that you can test, it’s important to define how A/B testing works and some of the key statistics that are used within it.

Most email marketing platforms, such as MailChimp, offer A/B testing campaigns that allow you to do this with ease. They work by having you select an aspect of the email, such as the subject line or main content. You then write 2 or 3 different versions of this.

In MailChimp’s case, it sends out each of these versions to a portion of your recipients. Depending on the size of your list, this may be done with half or less of the recipients first, followed by the rest being sent the “winner”.

The “winner” is decided by MailChimp tracking and is which version worked the best. Typically, the main statistics used in this decision are open rate (the number of people who received the email that opened it e.g. 1%) and the clickthrough rate (the number of people who, once they opened the email, actually clicked on a link within it).

By using A/B testing on loads of different aspects of your emails over a long period of time, you can narrow down what works for a particular set of recipients. This can help to make your email marketing far more effective in the long run!

A/B testing is a great way to improve the effectiveness of your emails. Don’t just rely on one set of tests, though! Try and acquire data over an extended period of time and use this data to make smart decisions on your overall strategy. [Photo by Jason Dent on Unsplash]

Try different subject lines

There are a few different schools of thought on subject lines and everyone has their opinion on how they should be written. Don’t limit yourself to a certain convention, though! Every person is unique, so that means that every list is unique, yours included.

When you’re next writing a subject line for your email, consider trying 2 or 3 different versions. This allows you to compare the results from each different iteration and find out what works best for a particular list of people.

How do you do in this practice, though? Here’s an example using our favoured email marketing platform, Mailchimp:

Dave decides that he wants to try a couple of different subject lines for his next email campaign on widgets. His normal approach hasn’t been working and it’s time that he tried something new! Using Mailchimp, Dave sets up an A/B Testing Campaign. This kind of email campaign allows him to test any aspect of his email. In this case, he’s focusing on the subject line.

The 2 different subject lines that Dave wants to test are: “50% off widgets this weekend” and “This is the perfect time to replace your widgets!”. The subject lines are very different, which means he’ll get some meaningful data about which is more effective.

Once he’s finished creating the email, he sends it to his list of 5000 people. For testing purposes, the list is split in 2. To start, 1,250 are sent the ‘50% off’ subject line and the other 1,250 are sent the ‘perfect time’ subject line. Mailchimp then measures how these each perform over the next few hours. The remaining 2,500 people are then sent the subject line that performs the best!

By using this approach, you can gradually identify the kinds of subject line that resonate with your recipients. It’s important to gather a lot of data, though! Don’t just judge it off of one email, send out 3 to 5 emails testing the same hypothesis. Once you have a good amount of data, you can start making informed decisions.

Finally, as Dave did in our example, make sure the subject lines you’re testing are significantly different from each other. A good way of starting is writing one subject line that is clear and one that is more abstract. A clear subject line would be similar to Dave’s example of “50% off widgets this weekend”, as it makes the offer obvious from the outset. An abstract, or clever, subject line would be similar to Dave’s example of “This is the perfect time to replace your widgets!” as it doesn’t overtly say what the email is about but may make recipients curious enough to open it.

The right subject line can convince people to open your email or even take further action. As with different genres of book, though, each email marketing campaign has a different audience. Test and tailor your subject line to your audience! [Photo by on Unsplash]

Experiment with content and layout

Similar to subject lines, it’s important to test different forms of content and layouts with your recipients. You might have a layout that you’ve used for the past few months or even years, but how do you know if that layout is performing better than others? The simple answer is: you don’t.

Let’s go through another example to explain this better:

Dave has been using the same email layout for the past 3 months. It’s a large, header image followed by a few paragraphs of text. He decides that he wants to find out whether changing this could let to better results.

He starts by reducing the amount of copy he’s using and making his message more focused. After a few tests, he sees a small improvement. He then tries adding in some ‘featured products’ at the bottom of his email. Once again, this leads to another small improvement.

After a few tests of this new layout, he decides to give his recipients some time to adjust before testing more layouts. Over the next few weeks, he continues to monitor the new layout’s performance.

We’ve used a simple example here, and it’s probable that your email contains a lot more variables than Dave’s original email did. The lesson behind it still applies, though. Becoming stagnant in terms of email development can lead to your recipients becoming bored. This can cause lower open and clickthrough rates.

Once you’ve decided to start testing some new layouts, it’s important to take it one step at a time. When changing a layout, it’s likely that you will be removing and adding a few variables. For example, you might reduce the amount of copy, add more images, or change the position of certain variables.

For testing purposes, it’s important that you don’t change all of these variables at once for your first few tests. If you do this, the data you receive back won’t be that useful. This is because you won’t be able to identify which variables you changed actually made a difference!

Start by changing 1 thing and testing that for a few emails. Once you have some solid data on that continue to change other variables. Eventually, you’ll arrive at a layout that is a conglomeration of loads of variables that worked well in individual testing. Once you’re at this stage, see how well your new layout is performing against previous ones.

We’ve only really scratched the surface here in terms of testing. Unfortunately, there wasn’t enough room in this blog to talk about testing different send times or the intricacies of using different sending names. Those will have to wait for another day!

Email marketing is complex, especially as you move into testing, but we hope that this has been helpful to you! Still have some questions? Want to chat to an email marketing expert about how to take the next step with your emails? Get in touch with us today by visiting our website, giving us a call on 01787 223850, or sending us a message on Facebook, Twitter, or Instagram!


Content Marketing Manager