Multivariate testing and A/B testing both involve learning from your subscribers by testing engagement within your database. Therein the similarities end, as A/B testing involves just one variable whereas multivariate testing compares multiple variables revealing more data on how each variable behaves against each another. Clever, right!?
These campaigns are used to reveal how small changes can have a big impact on your engagement. You can choose not only what you want to test, such as the subject line or content, but also compare results to see what works and what doesn’t.
Top 5 Multivariate Campaign Ideas
• Does including your company name in your subject line and “From” field increase engagement?
• Are subscribers more likely to click a linked image or linked text?
• Will a different template increase click through rates, even if text content is the same in all the campaign versions you test?
• Do your subscribers prefer a campaign that contains a GIF or one with static images?
• Are subscribers more likely to click a link that is just coloured text, or one that is styled as a button?
Once the data is collected from each combination, it is compared to not only determine the most successful campaign but also which elements have the most significant impact on your overall marketing campaign aims. Include all the data available in your report so you’ll know which elements failed dismally and which were a roaring success.
So what else do we need to know about multivariate campaigns?
A combination is described as each version of your campaign that is created from your chosen variables. If you want to test two “From” fields and two subject lines, marketers would create four different combinations of each campaign. In the test phase these are called test combinations (no surprises there then!).
This is the period of time after the combinations are sent out and we compare the results. Data collected during the test phase can be used to determine the campaign’s winning combination through automatic or manual analysis.
Each version of the variable is called a variation (say that six times quickly!) The variation is which element of your campaign you want to test. With a multivariate campaign, you can test a combination of four variables including subject lines, from fields, content and send time.
Winner or Winning Combination
The combination that performs the best is the ‘Winner’, this may be automatically determined by click rate, open rate, total revenue, or can be manually chosen based on the combined reporting data you find the most valuable.
Try different phrasing, sales offers or emojis to see which gets the most attention. While you’re thinking of it, download our free guide “Email marketing best practice for subject lines” here.
See if your subscribers are more responsive to emails coming from a person’s name or from the name of your company or organisation. You’ll provide both the From field name and From email address you want to use for each combination, which will determine whether your subscribers prefer a personal touch or formal contact.
Create different versions of your content to see which gets a better response and use this variable to test small content changes or completely different templates. When you test content, you may want to better understand the efficacy of calls to action, links or buttons. Top tip: Use our Link Comparison tool in the campaign report to see how your links performed!
Learn when your subscribers are most likely to open your campaigns. Since this option tests specific days of the week, date and times, you must send your combinations to all your recipients at once. Use this data to inform when to send or schedule future campaigns.
Based on our experience, the most common use for multivariate testing in the email campaign sector is in analysing elements that are up for debate. The differences between A/B testing and multivariate testing should not lead marketers to think of them as opposites, instead they should be thought of as two powerful optimisation tools that complement one another… After all, multivariate and A/B testing are two sides of the the “email split testing” coin.