Free CRO Audit
Why Copy Testing and Optimization Isn't as Easy as It Looks

Why Copy Testing and Optimization Isn't as Easy as It Looks


Twitter Share Link Facebook Share Link Linkedin Share Link

Copy tests that optimise micro-elements can make a powerful difference, but trying to test the relative performance of different language styles is much trickier.
Testing language style or tone with a view to optimising conversions is tricky. Sure, we can often see how adding a word denoting urgency to a call to action (CTA) button can lead to an uplift in conversions, or show that a benefit-led headline on a landing page plays better than a feature-led one.
Many such copy tests have a strong element of common sense. But when it comes to testing the flavour of language to use in, say, your product descriptions, things get harder.
Copy style or tone is notoriously difficult to define and replicate. For one thing, it’s hard to be sure that two completely different sets of copy doing the same job are each written to the same standard, on their own terms.
For another, ensuring that you’re testing the effect of language alone and not also other elements such as design is tricky. And there are reasons to wonder whether the entire enterprise is even the right thing to be testing.
For me, the following two case studies, while interesting and useful, highlight some of these issues…

Conversion Xl: Hedonic v Utility Copy

I’m a big fan of CXL, but this piece on testing utility copy v hedonic copy gave me pause.
The piece claimed to test two different modes or approaches to copywriting product descriptions: hedonic – focused on the pleasure that can be derived from the product; and utilitarian – focused on the product’s utility. Which approach would drive a greater perception of value?
It sounds like an interesting question to test, and the initial example – ‘Do consumers find more value in a blender that makes creamy smoothies and shakes, or a blender with 750 watts of power?’ – initially seemed to illustrate the contrast neatly.
But when we look at the descriptions that are actually tested, the distinctions are not so clear. Three products were tested – a blender, a vacuum cleaner, and a coffee maker.
Here are the two versions for the blender – see if you can guess which is which:
Version A:

  • Stainless steel blade system
  • Four variable speed settings plus pulse & smoothie mode
  • 6 cup contoured pitcher

Version B:

  • Stainless steel blades blend a smooth consistency every time
  • Multiple settings for perfect smoothies, cocktails, shakes & more
  • Contoured 6 cup pitcher for big batches that pour easily

You might have guessed that Version A is the utilitarian one, but perhaps only because it’s terser and less informative.

And the other version is actually full of useful information (utility) – scenarios that turn features of the blender into benefits the user can relate to. None of the latter version is exactly hedonic either – there’s no talk of attractive design or deliciously healthy drinks or crazy cocktail parties.

Sure, there’s talk of ease and practicality, but in the context of a blender these feel more like utilitarian attributes.
We see the same issue with the copy for the other two products. The utilitarian version is balder and feature focused; the hedonic version is basically better copy, adopts a friendlier tone but still talks more about practical benefits rather than aesthetics and pleasure.
Another issue to consider is whether the hedonic/utilitarian relationship is really a binary opposition at all, or more a spectrum. Some products are inherently hedonic – such as a cashmere sweater, perhaps; others – such as a vacuum cleaner – clearly veer far more towards the utilitarian.
And if we accept that, we can also see that really useful features of inherently utilitarian products will – in the right person – generate feelings of pleasure.
I have a friend who’s obsessed with garden machinery; a phrase like ‘features STIHL’s long-life air filter system with pre-separation and professional anti-vibration technology’ would have him drooling with pleasure. Thus features can become benefits, the utilitarian becomes hedonic.
It’s complicated, is all I guess I’m saying.

Copyhackers: Clever v Clear Copy

Here’s another example. This test from Copyhackers aims to answer the question: Should copy be clever or clear? It tests three variants of headline against a control version from JCD Repair, which offers while-you-wait iPhone screen repair services in the States.
Certainly, at first glance, the headline of the winning version, ‘Did Your iPhone Screen Have a Rough Night Out?’ could be described as ‘cleverer’ copy than the control, ‘iPhone 4 or 4S Screen Repair’.
But then the control copy doesn’t feel like a headline at all, more like a clunky label. And in fact it’s not that clear either really; it’s a clunky noun phrase that’s not immediately obvious to parse, whereas the allegedly ‘clever’ version engages me directly, asks a funny question, and sets up a scenario in a way that the target audience can easily (and enjoyably) relate to.
Is this cleverer copy, in other words, or just better written copy? It’s not a keyword-tastic functional description but it’s not exactly unclear either, especially positioned as it is alongside the image of a cracked iPhone screen.
Look further, and we see other differences in the supporting copy used by each version.
The ‘clever’ version includes specificity about turnaround time (‘less than 1 hour’) and a compelling price promise (‘for half of what the other guys charge’).
This is all good – and very clear – stuff, exactly what you’d want from your copywriter; but again, it just feels like better written – rather than ‘cleverer’ – marketing copy.
Control version:

Winning variant version:

Copyhackers know all this, of course, and they make clear in their commentary that they believe that clever v clear is a bogus distinction. ‘
The Copy Hackers team doesn’t believe that clear and clever are mutually exclusive,’ they write.
Later on, they add: ‘Your landing page visitors need to understand (1) where they are, (2) what they can do on your site, and (3) why they should stick around. […] But once you have the essential messages in place, it’s okay to have some fun and let the creative juices flow. Chances are that if your message makes your target audience smile, you’re more likely to be remembered.’
Well, sure. But if clever v clear is a bogus distinction, what are we really testing here? And if clever really means ‘being a bit more creative, adding a touch of humour, prioritising messages better, understanding the audience better, and doing all this without sacrificing clarity’, then we have ended up testing something rather fuzzy.
In short, we have only really proved that well-written copy trumps less well-written copy. And it’s not good because it encapsulates one single, and conceptually problematic, attribute like ‘cleverness’, its good because it’s doing many things at once better, as a good copywriter will.
It’s complicated, is all I guess I’m saying again.

In Summary

What then can we conclude from these examples about how to go about testing language styles? Well, for me, I wonder if this is the right sort of thing to be testing in the first place.
Good copy is doing several things at once, so it’s hard to single one of those things out and test it, or to reduce the way it works to a single attribute.
It’s rather like that vain Holy Grail of working out the optimum length for a blog post or a Facebook update or a landing page.

Sure, shorter things look easier to read than longer ones, and more complex products may need more explaining than simpler ones. And yes, the longer you write, the more interesting you will need to be – and/or the more vital the topic – to hold people’s interest.
But just as the right language style emerges from a whole series of objective inputs and subjective judgements, so does the optimum length of a piece of good copy – not from sweating to get your word count as close as possible to an average value derived from testing a million pieces of content.
CRO people take justified pleasure in proving copywriters wrong, and the latter have much to learn from the former. But I think the learnings can cut both ways. Ultimately, ‘best practices’ and data will only get you so far; craft and intuition have to play their part too.
Dan Brotzel is content director at Sticky Content, a Press Association company. 

Like what you’ve read? Sign up for more!


⬛️ 2023 Black Friday Ecommerce Strategy & Stats Report

This FREE ebook gives you the methods, solutions & trends for building a high-converting remarketing strategy for Black Friday

2023 Black Friday Ecommerce Strategy & Stats Report


Learn how to convert your online audience into revenue with our experts.