Testing: Take It One Variable At A Time

Whether you are thinking back to your high school days wrestling with trigonometry or a visit to the doctor and the latex gloves, the concept of “test” is apt to fill you with dread. However, testing an email campaign is essential to ensure success and generate leads. It’s tedious, it sucks and you have to do it.

There are two types of testing for email campaigns, both equally important:

1. Proofing. This involves testing the technology side of things to make sure your email is delivered the way you intended.
2. Variable testing. This assesses the impact that various components of the campaign have on the success of your transmission. You test for three main components: list, offer and creative.

Proofing

Although proofing isn’t sexy, it’s the foundation of quality marketing communications. If neglected, some of your audience may not be able to read your email, and you can actually hear the dollars being flushed down the toilet. Technical aspects that you should verify include:

• Browser compatibility. Not just how it performs in Internet Explorer, Firefox, Safari and Chrome, but how does it perform in various versions of each.
• IP address variation. Sure, it looks good on your box. Send it to your Aunt Edna, though, and she might see nothing at all. Get out of the office and check the email on a different IP address.
• Email client compatibility. Whether you are using a desktop client like Outlook or a browser-based client like Gmail, make sure your content and images show up the way you want.

Proofing is boring, but if you overlook simple little things like whether or not your images will load, you’ve wasted time and resources and potentially diminished your company’s brand image.

Variable Testing

Think back to seventh grade biology. Remember when you learned that any scientific test requires a control and a variable? Bet you didn’t think you’d need that little nugget again. But that’s at the heart of what is referred to in marketing as “A/B testing” in marketing. As we stated before, when testing the variables of an email campaign, there are three elements you should test: list, offer and creative.

Variable testing requires adherence to one cardinal rule: Only test one element at a time. You can test list or offer or creative. Testing more than one element at a time produces inconclusive results, and you can’t be sure which element is responsible for the new outcome. If you send an email to a new list and also change the subject line, which variable is responsible for any improvement? You won’t know. (You can actually test multiple variables simultaneously using “split-cell” testing if your list is large enough and you have a sophisticated grasp of variable testing.)

Now let’s discuss each variable. The percentage indicates how much each element contributes to the success of a marketing campaign.

• The List (40 percent): Most people get list testing wrong because they send a brand new email to a brand new list they’re trying to test. They have no idea how the list would perform compared with their house list, which would be the control in this scenario, and therefore make decisions on the quality of the list without a point of reference. To test a list correctly:
  • Choose an email you’ve sent to your house list before that performed on par with most emails sent to your house list.
  • Change nothing about the email you’re testing with the new list. Keep the same offer, content and creative to both lists.

If you’d like to test multiple lists at once, make sure you’re testing on the same day, at relatively the same time and using the same suppression lists (a list of people you don’t want the test sent to because they’re already in your house list or they’ve opted-out of your communications before). Do your best to keep the different list tests identical so you don’t have an extra variable running around. For more about different types of lists, see Chapter 21.

• The Offer (40 percent): The offer is a simple test, and it’s as important as list selection. The offer will drive people to act on the email, whether you want them to fill out a form, schedule a demo or download a paper. To get the highest possible response rate, test different offers.

When testing offers, don’t change the list, the content or the creative. Try out a Starbucks gift card or a highly sought-after industry analysis. Keep in mind the ROI of different offers, and remember that just because one performs better, that doesn’t mean it’s a winner. You could give away BMWs to get sales meetings and get a blockbuster response rate. You’d also go broke real quick. Cost per lead matters.

• The Creative (20 percent): Even though creative elements have the least impact on the success of a marketing campaign, this is where most marketers spend most of their time tweaking and tuning. There are countless things that fall under creative that can be tested. The truth is you probably won’t be able to test all of them unless you’re sending the same email to the same list with the same offer for a full year. Creative elements include:
  • Subject line: Do you want to make it witty or straight­to-the-point?
  • Header graphic: Test whether your audience responds better to a small header or a big, bold header.
  • Image: More or less? Pictures or graphics?
  • Color: Does green mean go? Does the purple font look professional?
  • From line: Does your audience respond better to an email from generic sender (marketing@acme.com) or from the CEO of your company?
  • HTML vs. plain text emails: Choose wisely based on your audience and how well they know you. If they are familiar with you, plain text might work well. If they don’t know you, you might need to create an HTML email to attract attention.
  • Layout: Does your email perform better if your callout boxes are on the right or left? Should your offer be centered or left justified?

All of these elements contribute to the success of a campaign, but remember, the cumulative impact only makes up one-fifth of the total results. In short, don’t kill yourself over all the little details that go around content and creative. Test your lists and offer first. If you feel you have those two locked, test the most important aspects of creative and content – usually subject and layout.

Read More…