Email marketing case: how UPC won an Emma award using multivariate testing


Contributor Pieter Wuyts from 8Seconds, who landed quite some new deals and partnerships since he wrote about getting more people to the landing page (a post that was really liked by many of you), has a new great example of improving your email campaigns, this time using multivariate testing.

Since his last post, Pieter’s Belgian company signed agreements with ReturnPath, Silverpop, RapidSugar and many more – I say this because Pieter admitted that posting here quite helped him so if there are candidates for good posts out there 😉

Anyway, one of the partners of 8Seconds is 22Times. This company works among others for UPC that won an Emma award using multivariate testing. In this post, Pieter explains, including all the details (buttons, process, results,…), just how UPC did that. A great case.

UPC, a market leading Dutch provider of Internet, television and telephony sent an email to their most loyal clients with a personalized offer.

22Times, the agency of UPC created 40 different buttons to be tested: 4 different colors were tested, in combination with 10 different button texts.

4 different colors 


10 different button texts

Activeer_nu_b1 Activate now  

Bestel_nu_b1 Order now 

Meer_info_b1 More info 

Meer_informatie_b1 More information 

Mijn_aanbiedingen_b1 My offers 

Mijn_abonneescan_b1 My subscription scan 

Stap_over_b1 Switch over now 

Persoonlijke_abonneescan_b1 Personal subscription scan 

Mijn_persoonlijke_abonneescan_b1 My personal subscription scan 

Upgrade_nu_b1 Upgrade now 


The Process:

22Times used the real time optimization service of 8Seconds to optimize this emailcampaign. When the entire emailcampaign has been sent out, 8Seconds showed these different buttons to the customers opening their email and measured which button worked best. The best converting button was found very early in the campaign thanks to the very consistent conversion behavior of the recipients.

The Results:

The 2 buttons with the lowest CTR are respectively:




Both have a CTR of 0%. The readers of UPC apparently cannot be motivated to click by means of the pushy ‘order now’ text.

The button with the highest CTR (61.90% !) is:


The Averages:

Let’s now have a look at some averages:

Your email recipients don’t want commands!

Looking at the chart above, it’s very clear that the purple button works better than the orange button, although many other campaigns indicated that orange is supposed to be a better colour for buttons.

But if we look at the performance of the texts on the buttons, then the differences are even sharper.
What can we learn from the chart above? 

‘Order now’, ‘upgrade now’, ‘switch now’,… are commands, and the customers of UPC do not want to get commands! They want to be addressed personally and they clearly expressed their wish to be invited to have a look at the personal offer that has been selected for them. 

Giving the impression that UPC has done a personal scan to select the best offer without being too pushy increased conversion enormously.

This campaign was repeated several times, with very similar and consistent results. No wonder this email campaign is nominatid for the prestigious EMMA-award! Congrats to 22Times !

The author of this post, Pieter Wuyts, is a no-nonsense entrepreneur
and conversion freak. He’s one of the founders of 8 Seconds, that very
quickly became the market leader is real time email testing. Pieter
inspires people to grow, email campaigns to perform better and plants to
flower. 8Seconds is market leader in real time multivariate testing in
email marketing campaigns. Clients use the email optimization services
to find out in real time what converts well and what does not, based on
statistical algorithms. You can follow Pieter on Twitter here.

Bookmark and Share

Join me on Twitter

Join me on Facebook

Join me in the Social Marketing Forum

Subscribe to the newsletter

3 thoughts on “Email marketing case: how UPC won an Emma award using multivariate testing”

  1. Very interesting results, although I don’t think the only possible outcome is that people don’t like commands. The short commandative texts are also very unspecific, which could well be at least part of the reason the CTR was much lower.
    I wonder how “View my offers now” would have scored for instance, or “View my personal subscription offers”. I think the positive influence of making something personal, by using “my” and “personal” is stronger than the negative influence of the text being in commandative form.
    One other thing: UPC knows Pieter’s first and last name, but they don’t know if he’s a man or a woman? “Geachte heer/mevrouw van de Wuyts” Ahhhh come on!

  2. Hi Michel, you have a point there although I have to admit I emphasized the fact that people don’t like commands because I know from experience they don’t. Not in email, not in any other channel for that matter. If the commands are packaged as incentives and play on the psychological needs and sense of urgency, they work. But you certainly have a point about the fact that there are other reasons. In fact, there always are, you know as I do that marketing results depend from a variety of elements, some of which we even can’t control. Still it’s a nice case I think. And regarding the screenshot with the “Geachte” etc.: you have a sharp eye for details, I missed it. Of course the screenshot can come from a testmail, I’ll ask Pieter. Thanks for your comment and feel free to send us some cases as well! Always good to have thinkers on board 😉

  3. Hi Michel,
    The preview was indeed a testmail.
    I would like to point out to the fact that it is always easy to start analysing results and explain why version A was better than version B AFTER the campaign.
    The challenge is to find the right version BEFORE you send out the campaign. Just scroll through your emails and see how many buttons you find with “buy now!” or “More info”.
    The only way to make sure you show the right message is to let your readers decide on the best content, and that is something that UPC and 22Times understood very well.

Comments are closed.