The Value of A/B Testing to Improve Benchmarks

As marketers we know that sending emails to your subscribers is a critical part of your communication strategy to remain top of mind. There are many purposes for sending emails. Whether your goal is to build credibility, build your brand, or promote a product or service, it is important to understand your audience and what engages them. This can be done by identifying who your audience is, setting benchmarks, and A/B testing.

How to Set Benchmarks

In every area of your business it’s important to set key performance indicators (KPIs) and benchmarks specific to your goals. Email is no different. These benchmarks can change over time and should take into consideration industry metrics and standards. Benchmarks can be calculated based on past performance and if you’re just starting on your email journey, looking at industry publications for standard benchmarks for your segment is a good place to start. While there’s many metrics to track depending on your goals, some basic ones  include click rate, open rate, bounce rate, unsubscribe rate, and click-to-open (effective rate).

While all email metrics are equally important for different reasons, each metric provides its own purpose that will help to shed light on different areas for improvement in your email campaigns. It is important to familiarize yourself with each and how they are calculated. There are many metrics you can use to track the success of your emails. 

When tracking metrics, it’s important to note the difference between total opens and unique opens. Total opens are calculated using the total number of times an email is opened. Unique opens only take into consideration the number of individual subscribers who opened the email. For example, total opens are calculated by tracking the total number of opens / number of emails delivered * 100. For unique opens, this is calculated by only counting the first time a single subscriber opens the email. The unique open rate is the total number of subscribers who opened the email / number of delivered email * 100. 

Here are some other important metrics:

What is A/B Testing?

A/B testing is creating a variation between two or more versions of an email, sent to two segments or more of your audience. Segmentation is the dividing of your subscribers into smaller groups based on a variety of set criteria to create your marketing segments. The goal of A/B testing is to determine which version of the email provided the best results based on what you were testing and your goals. If your goal is to improve your open rate, consider testing a variable that would affect opens such as the subject line. On the other hand, if your goal is to increase the click-through rate, consider trying different CTA copy.

How to Improve Open Rate

As mentioned above, open rates are one of the key metrics you can track. If you are finding your open rates on your emails low, it generally can be broken down into one of three things. 1. Your subject line is not interesting or relevant enough 2. Your audience is made up of many different segments with wide interests, or 3. Your frequency of email deployment may be either too high or too low.

To determine which may be affecting your open rate, you can incorporate A/B testing to see which variable makes the largest difference. One solution is to test your subject line copy by length, tone, or incorporating questions. For example, if you were selling tires, you could say “Is your vehicle ready for winter?” as opposed to “Get your vehicle ready for winter”. Another option is to adjust your segmentation and the audience group that is receiving your email by the type of content. Often, depending on the size of your database, not all content is relevant for all audiences. An example of this would be sending decision-making product information to those who are still new to the product offerings compared to a more mature audience who knows what they are interested in, in order to categorize them for future specific product messaging. Lastly, you can review your past and current email deployment schedule and adjust your deployment frequency accordingly based on your type of content and results.

How to A/B Test

We mentioned above a few different ways to A/B, however these are just a few of the many variables you can test. Before you decide what to test, you should consider your goal of testing and what information would be helpful for you to make future decisions. What metric do you want to review? Where would you like to see improvement? Once you determine this, it will help you decide where the variable needs to be. Some examples of A/B testing could be:

• Variable content type

• Email length

• Subject line

• Personalized vs generic

• Length

• Tone

• Use of emojis

• Incorporating questions

• Open-ended statement

• Generic vs specific copy

• Promotional offer messaging vs slogans

• Punctuation marks

• Super subject line copy

• CTA copy

• CTA box shape and size, if applicable 

• Variable images (types of images)

• Deployment time (day, time, weekday vs weekend)

For example, if you are seeing a high open rate, but a low click rate throughout, consider trying different copy lengths to determine an appropriate length for the type of content in your email.

Email Best Practices

Understanding what works for you and your brand will take some trial and error. However, when you test, it is important to stay up to date on email best practices so you can get the most out of your communications.

Some tips to keep in mind is that when it comes to subject lines, less is usually more. When testing variable subject lines, you should include no more than three punctuation marks, and no more than one emoji so it doesn't get mistaken for spam. 

Another tip would be to define your segment. It is very important to send the right information to the right audience, at the right time, to further move them down the sales funnel. Not every email will be appropriate for the entire subscriber list. You should continue to try to understand the content each specific segment would like to see without being too salesy.

Lastly, be consistent. Once you have your audience segments, your content and benchmarks, the last main component you need is timing. Through research, best practices and past results, determine when you would like to send your emails and how often. If you have too many hard bounces, it is a sign your subscriber list is out of date. We recommend you email your subscriber list regularly, no less than every 3 months so that your list is current.

Now that you have your segments, your benchmarks, your content, and your deployment cadence, you are ready to deploy and test. With many A/B test variables to try and different benchmark goals, you should aim for consistent improvement and try to stay up to date with the industry standards and best practices. To learn more about these, feel free to reach out to us. We would love to talk about all things email.

Sources:

https://mailchimp.com/resources/email-marketing-benchmarks/

 https://blog.hubspot.com/marketing/metrics-email-marketers-should-be-tracking

https://phrasee.co/news/unique-opens-vs-total-opens-whats-the-difference-and-why-does-it-matter/