We’ve been rotating some headlines and subheads on the Highrise signup page to see if they have an effect on signups. Answer: They do, sometimes significantly.
The test
Here’s how the test works. We used Google Website Optimizer to randomly rotate five different headline and subhead combinations on the signup page. We’re measuring the number of clicks on any green “Sign Up” button. We’re not measuring any specific plan, just that “someone picked a paying plan.” We ran the test for 4000 page views. Why 4000? The numbers didn’t change much after about 3000 page views, so we stopped at 4000.
Note: We recognize that switching both the headline and the subhead isn’t quite as informative or scientific as just switching the headline or the subhead. We’re OK with this. This experiment was part learning how to use Google Website Optimzer, part curiosity, and part conversion research. More detailed tests will follow.
The original: Worst performer
This is the headline we launched with. The headline asked people to “Start a Highrise Account.” “30-day free trial” was centered bold in the subhead. The rest of the subhead highlighted that Highrise is a pay-as-you-go service and that there are no hidden fees.
The winner: 30% better conversion than the original
This combo put the emphasis on the 30-day free trial by making that the headline. The subhead let people know that signup was quick (less than 60 seconds). The second part of the subhead asked someone to “pick a plan.” This was also the only combo to feature an exclamation mark. Would be interesting to run this headline against itself — one with a period and one with an exclamation mark.
Second place: 27% better conversion than the original
This one also promoted the “30-day Free Trial” in the headline, but instead of highlighting signup speed, we highlighted other benefits: Pay as you go, no long term contracts, no hidden fees, no surprises.
Third place: 15% better conversion than the original
This combo went back to the original “Start a Highrise Account” headline, but tacked on “Today” at the end. The subhead was the same as the second place finisher: Pay as you go, no long term contracts, no hidden fees, no surprises.
Fourth place: 7% better conversion than the original
This combo featured the winning “30-day…” headline, but replaced plan information in the subhead with quick customer testimonials plus a link to the buzz page. Even though this was the only design with a link away from the signup page, it still performed better than the original design.
What did we learn
We have some theories, but we’re curious to hear from you. Why do you think these combinations finished the way they did? What other combinations would you like to see us try? What other tests would you like to see run on this page? How else do you think we could increase conversion?
Aaron
on 13 Jan 09The winning one seems to be the most concise with not a lot of distractions/noise.
Adrian Bye
on 13 Jan 09The winning headlines are much stronger from a direct response copywriting perspective. I believe we talked about this over breakfast in Chicago. If you do this across all your sites you can significantly grow your business.
Ryan
on 13 Jan 09I think the combination of Aaron and Adrian’s answers is correct. The winning result is both the strongest call to action, and the most concise. Short punchy phrases beat out longer, flowing verbiage.
k00k
on 13 Jan 09Yep, people just want to glance at something (or at least think that’s all they’re doing) and understand the whole picture.
Good stuff. Thanks for sharing.
James
on 13 Jan 09“30 day free trial on all accounts” translates (in my mind) to “try it, if you don’t like it just forget it”. That’s simple, low-risk and low-pressure. “Start a Highrise Account”, on the other hand, will sound more like an order to some people—maybe I’m stubborn but when I read it, my first reaction was “stop telling me what to do”!
W. Alexy
on 13 Jan 09The winner seems like the most obvious choice, because all it says is free. Option 2 doesn’t complicate the “Free” message with any Pay As You Go language. Free is the strongest word in the English language I’ve heard.
I’m surprised that the testimonials didn’t fare better.
Matt Brown
on 13 Jan 09The last one has too much to digest at just a glance. The other 3 that did’t do so good seem too QVC-ish or infomercial-ish.
Gavin
on 13 Jan 09The next question I’m curious to see an answer to is: what is your retention of those 30% that signed up.
jrome
on 13 Jan 09Maybe you could try the sub-heading: Pick a plan and get started in under 60 seconds!
Michael Hagstrom
on 13 Jan 09I’ve noticed this more and more with you guys over the years. Your straying away from what is making you a unique brand. Why the hell do you care about conversion rates and metrics?
This post alone shows that your going back on some of the previous statements you’ve made.
Look at this post here (http://www.37signals.com/svn/posts/154-a-little-37signals-redesign) and in the comments JF says “Josh: We don’t have metrics. It’s all gut.”
I think times have changed from what the orignal 37signals was and what it is becoming now.
Greg Robertson
on 13 Jan 09I’m in agreement with @W.Alexy about the word “free”. To go a step further I would drop the ” 30 day” and “Trial” To me “Trial” almost means “lite” version. They are getting the full version of the software correct? The “30 days” is more like a terms, less marketing copy. I’d like to see the headline changed to:
Start A Free Highrise Account Sign-up takes less than 60 seconds. Pick a plan to get started!
Let’s start taking bets!
Isaac Weinhausen
on 13 Jan 09First Thought: People do not want to commit
Second Thought: Lower hanging fruit always gets picked first
@Gavin: Good question. Lots of responses aren’t necessarily better if retention stays low. Sometimes the right verbage is needed to filter the good responders from the bad responders. However, there should probably be a lot of evidence to justify that kind of text before implementing.
JF
on 13 Jan 09Why the hell do you care about conversion rates and metrics?
We run a business so we care about conversion rates. Always have. We’ve experimented a lot over the years – always trying to move the numbers. Now we’re experimenting with using a more sophisticated tool to measure those changes a bit more accurately.
Ryan
on 13 Jan 09I don’t want to feel like I am making a pact for some long-term account, so the 30-day free trial title makes me feel at ease.
The less successful subheads seem wordy and are too much to take in – one is four sentences long (albeit short) and the other is just a blue blob when you scan over it due to all the quotation marks.
Jeffrey Gardner
on 13 Jan 09I agree with Aaron, it also seems that people only care about two things. They won’t be charged for 30 days, so they can think about it later. And they won’t have their time wasted with a sixty question signup form. Very informative stuff. Thanks for sharing Jason.
Dustin
on 13 Jan 09It’s FREE and it takes less than 1 minute to sign up??? I’ll try ABSOLUTELY ANYTHING that sounds like that.
That leaves room to try 479 other things in an 8 hour day.
SurviveStyle
on 13 Jan 09One of the major draws for all of your applications is that they increase the efficiency of something. You know, do more in less time.
Knowing that “Free” anything is the #1 biggest draw on the web (and in most other places), the 30 Free trial headline make the most obvious sense.
Next, you let your customers who are already searching for something that will simplify their life, that it only takes 60 seconds to sign up. That speaks to efficient-minded folk as well.
Nice work!
Richard
on 13 Jan 09It’s interesting that adding the word, “Today,” to the original headline resulted in a 15% increase in click-through.
Craft a way to add “Today” or other instant gratification cue to “30-Day Free Trial” and maybe you’re on your way to 45%!
Peter
on 13 Jan 09“30-day trial” – clear risk reversal.
“Sign up takes less than 60 seconds.” – no effort required. Clear benefit.
The second ones doesn’t do as well because of the subconscious cues about money “pay” and “fees”.
“Start a highrise account” is kind of meaningless with no CLEAR action. Plus it’s not obvious there’s a 30-day trial on all accounts.
The last one did the worst because of the testimonials. Boasting doesn’t appeal to anyone and in the customer’s mind, they are probably fake, thereby making you look like a weasel. Doesn’t matter if they are true or not. Plus “Our customers” is totally non-specific. “Our customers” could be your mom.
Tiffani Jones
on 13 Jan 09The last one has too much going on – cluttered visually with few actionable items. The winner has the most concise value proposition (free trial, takes virtually no time) and ends in an actionable item (get started!). Second place winner is the same.
The weirdest part is that that the top two performers don’t mention Highrise at all, though obviously Highrise is noted in the logo and below.
:D Tiffani Jones
ceejayoz
on 13 Jan 09Because higher conversion rates = higher revenue = staying in business? What were you thinking when you typed that question out?
Dante
on 13 Jan 09That +7% version was very weird, the “Our customers” quotes repeated under the Special Offer.
Evgeny
on 13 Jan 09I think the second place was such a bad performer because of the word “no”, the statements were supposed to be positive – so instead of using a positive statement, you used a negative one with a “no” prefix.
Many “experts” would tell you that it is a really bad approach to people, when a person does read the text – the brain just skips the “no” so to speak and imagines the things that are written. So basically your headline could have said :
“Pay long term contracts with hidden fees and surprises.”
And most people’s brains probably read it that way anyways.
StartBreakingFree.com
on 13 Jan 09Great post – reminds me that I need to do more Website Optimizer.
Also – to try an figure out WHY one worked better than another is the miss the point I think. There are so many variables it would be very difficult to guess correctly the future – luckily we don’t have to because we can test! That was the biggest take away for me – test!
Evgeny
on 13 Jan 09There is quite a lot of “research” put into this field, not official academic research, but more metric-measuring like you guys did.
This research is being made in the online internet marketing sphere. They have some really effective techniques to maximize conversion.
The information is there if you just google for “squeeze page”.
Has psychology, copy writing, layout, everything. Worth taking a look if you are trying to maximize registrations.
GeeIWonder
on 13 Jan 09Is the difference between 1st and 2nd significant or within the std. dev?
These things are always informative/cool/useful studies.
Patrick Koppula
on 13 Jan 09What I learned is that Google Website Optimizer exists. What a great tool!
Joshua Paine
on 13 Jan 09@Evgeny, second place wasn’t a bad performer at all. It was almost as good as the first place and much more successful than the original. I’d guess, rather than reading over the “No”s, people understood it fine, but weren’t quite as enticed as by the promise that signing up would take 60 sec or less.
GeeIWonder
on 13 Jan 09I’d say with a sample size of 4000 and a confidence level of 95% the 30% is probably 30+-1.5 and the 27 is probably 27+-1.5 so they’re both equally winners (or second), no?
Evgeny
on 13 Jan 09@Joshua What I meant was drawn from this: http://en.wikipedia.org/wiki/Positive_and_negative_(NLP)
Douglas Jarquin
on 13 Jan 09Just like Gavin, I too would like to see the retention numbers for comparison. In my mind while a conversion is great, they are signing-up for a free trial period.
Tanner Christensen
on 13 Jan 09It’s interesting that the highest performer included a direct call to action “Pick a plan to get started!” while the others didn’t.
JF
on 13 Jan 09GeeIWonder: Yeah, #1 and #2 were effectively the same, but I prefer the winner so the winner wins.
GeeIWonder
on 13 Jan 09Lol. Classic.
Thanks.
Michael
on 13 Jan 09These days people like quick sign-up. “60 seconds” is powerful. That kind of language also helps those of us adding users to our 37S product accounts, and since more users mean upgrades and more money for you, it’s the same sort of thing.
Jeremy
on 13 Jan 09It would be interesting to try a variant that talks up the number of people already using Highrise. Perhaps something like “Pick a plan and join our community of 10,000+ satisfied users” (or whatever the actual number might be).
Adam
on 13 Jan 09I’d be curious to see this same test done again, but tracking which plan they sign up for.
Jason Klug
on 13 Jan 09Why the winner was:
Offer Eliminated Risk (Free Trial) While Staying Realistic (30 Days)
Effort Barriers Minimized (Signup in 60 Seconds)
Avoided Over-Explanation (“No hidden fees” and “No Contract” are implied when you’re dealing with 37 Signals)
BTW-Thanks for opening my eyes to Google Website Optimizer… definitely going to put that to use!
erik
on 13 Jan 09There is a way to do A/B testing with Rails directly but it hasn’t been launched yet. http://railsanalysis.com
Joel
on 13 Jan 09You call attention and provide the answer to the strongest obstacle to your users – “what’s it gonna cost me?” (30-day free trial).
Very glad you posted this. I’m in the middle of writing some copy now and this will definitely help me refine it better, which is always the hardest part. Thanks again Jason.
Harvey Motulsky
on 13 Jan 09Could the differences just be random chance? To assess that possibility, statistical calculations are needed. To do these, I’d need to know the number of clicks (as well as the number of page views, which you state).
John
on 13 Jan 09I don’t think it was bad practice to vary headline and subhead at the same time – these are related elements that are seen by the user at once, and must work together. If you varied, say, headline and page color or type style or something like that, you’d want to do multivariate testing.
I’m not surprised the testimonials underperformed – they don’t speak to a visitor benefit, just tell you that somebody else loves the product. Just not as strong as the other options.
If you want to be sure you’re outside of statistical uncertainty, you need to leave it running for lots more than 4000 page views, but I’d have ended the test at that point with those results too!
Chuck McKinnon
on 13 Jan 09It’s fun watching people supply “obvious” explanations for the better headline after the fact. If you’d tried this beforehand, you’d have had a lot of passionate arguments about which one users would prefer and why. A good friend calls this “reckless opinionation”—what most people use as a substitute for evidence. (I want to note in passing that this phrase didn’t appear in print anywhere on the web before today according to Google, as it’s sure to be indexed here in short order.)
Bryan Eisenberg at FutureNow has been banging on this drum for awhile, and a year ago wrote an article where he noted that almost 77% of the Top 500 retailers attending the Internet Retailer conference didn’t do A/B or multivariate testing. He was baffled at the size of that number, given that tools like GWO are available for free.
The reasons aren’t that hard to understand: first, it’s bruising to the ego to subject our precious work to the cold, unflinching light of competitive data-gathering like you have. Sometimes we’re afraid the data will show (publicly) just how bad our underlying assumptions were.
A second reason is inertia: if you’ve got a publishing process in place, it can involve a lot of coordination and effort to change that process to let people tweak things on the live site. Who gets to change things? What can they change without involving developers? What gets tested? Who decides? Etc… 37S doesn’t have as big a problem here due to its small size, but as companies get bigger this issue looms larger.
Jakob wrote a good Alertbox three years ago on the pros and cons of A/B testing that I’ve found very useful.
Thanks for posting the results of your first experiment, Jason. I’ll be interested to read the followups on this subject.
Chad Garrett
on 13 Jan 09How many abandoned the signup page when they realized that the “30-Day Free Trial on All Accounts” still requires a credit card?
Peter
on 13 Jan 09The other option would be to not have 30 day trials at all but offer a 30 day, 60 day or 90 day unconditional money back guarantee. Typically the 60 day offers the highest conversion, all things being equal.
The problem with free trials is that it’s kind of like saying “we don’t really believe in our product, so we’re giving it away for free, even though we say we want money for it.” Then you hit them with a credit card singup anyway.. which seems a bit “scammy.”
On very large sites, changes become more and more unpredictable and counterintuitive. One change can cause a chain reaction of weird interactions.
Colin Ross
on 13 Jan 09Interesting – wonder what the variance in the “kind” of sign-up might be. Perhaps those designs that highlight the “free” aspects will encourage a lot of try-outs that then bail after the 30 days; while the designs that don’t will see more longer term sign-ups.
I think it is defnitely an interesting study, but if possible a follow-up after 30 days to see how many stay could also be enlightening. Any chance of this?
Justin Mc
on 13 Jan 09I think it’s the numbers, it tells a directly how much money+time it will cost the user.
Have you tried reversing the headlines of the winning entry? (I don’t know for sure that it would work better.)
“Just 60 seconds to sign up!” on top “30 day free trial” underneath
I’d also try other numbers, like, “as low as $24/month!”
The customer testimonials are a bit fluffy, they read like non-specific and uncredited book blurbs. It’d be better as, “Steve Jobs says, ‘Easy as your iPod!’”
Julian
on 13 Jan 09Didn’t you do a multivariate testing? Should be perfectly normal to test content variations in multiple locations on a page…
Esteban Gonzalez
on 13 Jan 09It’d be interesting to to learn about retention rates. Also, would love to know more about your current sign-up spread on the different plans you provide. Did putting the 49 buck plan in the middle and highlighting it worked? (meaning increased sign-up rate for that particular plan) are you going to perform any type of a-b testing on the way you layout the plans?
Nivi
on 13 Jan 09See Andrew Chen’s How to generate awesome test candidates for A/B testing.
My favorite suggestion from Andrew: “make sure you try to go for extremely polarizing, high-variance approaches.”
Ramesh
on 13 Jan 09The “Pay as you go” is the killer. In the last one, there’s just too much to grasp.
Which is also why those “Get a FREE iPod” links get so many hits. They don’t tell you what the catch is, so people sign up for a shot at it (hoping its easy to accomplish)
Crystal
on 13 Jan 09Personally, I don’t like it when ads say “no hidden fees, lock-ins” etc. I’m not thinking about hidden fees when I first see something, so I get wary when they’re the first to bring it up.
I like to read the terms for myself and find out what I am signing up for, rather than what I’m not signing up for.
Jason
on 13 Jan 09The second ad’s subhead is instructive rather informative. I belief people respond better to clear instructions.
zephyr
on 14 Jan 09Thanks for sharing. Good stuff!
Luke Stevens
on 14 Jan 09Yes! It’s so exciting to see you guys using GWO and dipping your toe into testing & (formalized) conversion rate optimization.
I’m a designer & total conversion optimization nerd – this kind of thing is the next layer of web design, as the results are undeniable. Running a test, changing a headline, improving conversions by 30%? That’s as close to “Here, have a bunch of free money” as you’re going to get.
Marketers have been doing this for ages, but we really need design savvy people getting on board.
One quick tip: I’m not sure how quickly you got through 4000 impressions, but the GWO guys usually suggest you run the test for a minimum of 2 weeks to account for day to day fluctuations and get a reasonable cross-section of your audience.
Even if you run the test for longer and the results stay the same, that’s a useful thing to know – ie is there much day-to-day variation on the performance of your variations? Only one way to find out ;)
And of course, statistical significance (+ time) is your friend :)
Dave Martin
on 14 Jan 09Just a heads up, I noticed that you’re loading ga.js once and urchin.js 3 times at the bottom of your signup page. You can actually combine all four of these calls into one. See http://tinyurl.com/7kxd3t for more. Will save you 7kb – not huge, but…
Luke Stevens
on 14 Jan 09And another tip because I can’t help myself… :)
People will (and have) say “Oh you should multivariate test all those different variations”.
I totally understand this, as on the face of it you get the most optimal combination of all variations, and that’s what you want, right?
Well, sometimes. But lets say you have two variations of five elements on the page. To multivariate test those in GWO you have 2×2 x 2×2 x 2 = 32 variations. Got say three headlines in one instance, instead of two? That’s 48 variations. What about three button styles, not two? 72 variations.
It can get pretty crazy, and it can take a long time to get statistically significant results for 32, or 72, or whatever variations, let alone making sure all combinations behave across browsers etc.
In my opinion, it’s better to test broad strokes first (if that’s where you’re at), and then things like headlines, button variations etc. It might not be as ‘pure’ in an experimental sense, but so long as conversion rates keep improving, who cares?
Like most things in life, better to keep it simple, manageable and fast so you get results and can keep moving, rather than agonizing over combo 71 vs combo 72.
David Martin
on 14 Jan 09pwb
on 14 Jan 09I’m surprised the last “Free Trial…” version performed so differently from the other “Free Trial…” versions.
Eric Seijo
on 14 Jan 09Very good post. The winner won IMO because: 1. It had a free trial option. 2. Didn’t talk about money. 3. Made the signup seem fast and easy to start and 4. Gave the user a specific direction / command, i.e., “Pick a plan to get started”.
Great stuff!
Mukul Gupta
on 14 Jan 09I think the headline worked better simply because:
Firstly, It presented an offer within the headline – “30 days free trial” Secondly, It stressed on the ease of sign up – “sign up takes 60 seconds”.
You might also want to try a combination like:
“Start Your 30-day Free, No-Obligation Trial Today Choose a plan and get started in less than 60 seconds. “
Jenna K
on 14 Jan 09Could you please learn the difference between less and fewer?
Use “fewer” when you refer to countable objects.
Use “less” when you are refering to non-countable entities.
“There were 50 fewer candies in my jar.” “I had less time to eat them.”
37Signals seems to love “less” and it earns them fewer readers.
So get started in fewer than 60 seconds already.
Mark
on 14 Jan 09@JF “Now we’re experimenting with using a more sophisticated tool to measure those changes a bit more accurately.”
What’s that then? :)
Ken
on 14 Jan 09Have you ever considered using Glyphius or other headline scoring / conversion products?
Jesse
on 14 Jan 09Agreed – 30-day Free Trial does come off like a low risk option – so why not try? BUT – here’s the thing, it only works in the context of 37S’s OVERALL branding and design.
I’ve seen all sorts of “30 Day Trial” signups on the web that I know are bogus – where I know I’ll start getting charged automatically on day 31. In fact the 30-day trial concept has been so polluted by creeps that it’s almost void of impact…
... unless you establish your trustworthiness through other means – ease of flow, design clarity, reputation, etc.
Walt Kania
on 14 Jan 09It’s always instructive to see hard results like these. But I’m with Chuck McKinnon: after-the-fact jawing is easy.
The hard part is deciding up front which way to go, especially in situations where A/B testing isn’t practical, such as in presentations and speeches or more routine marketing communications.
Oh, and a free suggestion. I’ve had more luck lately by avoiding words like Sign Up, Buy, Pay, and Order.
Try a version that talks about ‘owning’, ‘using’, and ‘having.’
“Have your Own Highrise Account in 60 seconds or so. . . You can be using Highrise just one minute from now . . .” Try Out Highrise Free, right now.”
Having and owning is what they’re after. Not ‘buying’.
Peter
on 14 Jan 09Guys, why do you rip Apple so much? Are you actually aware of this?
http://img178.imageshack.us/img178/7122/37signalsripsappledo4.png
This is a joke. And please fix your code. Inline CSS and tables for layout purposes are not really “cool” nowadays.
Jamis Charles
on 14 Jan 09Thanks for sharing this guys. Showing how to use Google Optimizer to measure conversion rates is invaluable for any designer dealing with a business.
After all, it’s the business that drives everything. If you as their designer can show them metrics like these they’ll love you.
Matt
on 14 Jan 09I guess this is why I’m not in marketing. I can’t believe there would be such a huge difference when they’re all saying pretty much the same thing.
Dr. Pete
on 14 Jan 09I’m with Chuck – it’s always fun to see people’s post-hoc explanations. I’d love to be able to go back in time, make everyone guess, and see just how many of us are wrong. If testing has helped me with anything, it’s definitely humility.
Have you considered doing a multivariate test after this (pairing all the heading and sub-heading options – 2×5 or so)? Sometimes those results can be suprising – the interaction between heading and sub-heading can be fairly strong at times.
Mark Terpstra
on 14 Jan 09Great article, Jason…
Where did the 4,000 clicks come from? The “Pricing & Sign Up” link in the header, the “Start a Highrise Account” button on the home page, etc?
It may be interesting to look at the conversion rates from each referring link on it’s own because the wording of those links/buttons may affect the mindset of the user and therefore what headlines will be most appealing.
If I click on “Pricing & Sign Up” link, I’m probably more concerned about the price/cost. In that case, the “30 Day Free Trial on All Accounts” headline and the “Pay as you go, no long term contracts, no hidden fees, no surprises.” subhead quickly addresses my concerns and removes the risk of a decision.
Ryan Graves
on 14 Jan 09I haven’t read the other comments but it seems fairly obvious that your only measuring click throughs to the sign-up botton and not product plan purchases. I think that with the winning choice you’ll get many more sign-ups but you have a large increase in accounts that cancel after the 30 day trial.
It seems that it would be more effective to measure which header/sub-header combo led to the most paying accounts not just test sign-ups.
Billy Shih
on 14 Jan 09The statistics side of me is sad that you stopped at 4000 page views? Total or per page?! I ran some hypothetical numbers with 800 page views per page using a 20% base conversion rate for the original to check the statistical significance of your results.
There is high confidence (99%+) that the top performers beat the original, however you have a lot of overlapping confidence intervals with the top 4 winners so meaning it is very likely not in that order.
If you have a higher conversion rate then you have better certainty, but if you have lower then you should seriously consider re-running this test for a longer period of time.
As others have mentioned, this would be better structured as a multivariate test if you wanted to test both headlines and sub headlines separately.
Anyway, I believe the winner is a simple case of adding value. Too many companies use their brand name in headlines when it does little to help the average visitor, especially if they are at the purchase point. Mentioning negatives can bring up anxiety so that’s always a good thing to test and as seen by this test, they should not be mentioned to your audience (or not mentioned in that way.)
I would be really interested to see testing on the button text, using “Get started” rather than “Sign Up” as well as testing more advanced concepts, such as different pricing options (less or more options), offers (30 day vs 15 day) and methods of comparing pricing.
Glad to see 37signals is taking advantage of what I do for a living :)
CJ Curtis
on 15 Jan 09Pretty straightforward, really. Although hindsight is 20/20.
The “worst performer” says do something now and pay for it now. No choices…no benefits.
The “winner” is pretty much the opposite…it’s quick, it’s free, and you don’t have to buy anything right now.
Second Place: You introduced “pay” again, but you added several benefits.
Third Place: No free trial statement, so you’re definitely paying now, but the benefits sort of ease the blow.
Fourth Place: That’s just a mess.
Mike Ford
on 15 Jan 09As you know A/B and multivariate testing is really time consuming and a bear to manage. You could read the comments on this post and make decision based on crowd sourcing.
I prefer click sourcing—letting real visitors decide based on a measurable click through on the actions you want
My friend Eric at Sitespect has done a remarkable job of optimizing landing pages for home pages and search campaigns. http://www.sitespect.com
Affinova does this for Fortune 1000 consumer packaged goods and manufacturer product designs.
Cheers!
Liza Cunningham
on 15 Jan 09Thanks for sharing this information. It’s a good reminder that we can continually think, try and learn (regardless of desired outcome).
MC
on 15 Jan 09As others have mentioned, the winning page was the most concise. However, both the headline and subhead include one simple quantity (“30-day” and “60 seconds”) which gives the user something tangible to digest. If you read a bunch of text, you never know until you’ve finished reading it, whether or not it’s just “fluff”. The numeric quantity actually stands out from the text and conveys useful information faster.
Before I read a book, especially non-fiction, I like to scan through the pages looking very quickly for how many capitalized words (names of people and places, acronyms, etc.) as well as numbers (percentages, dollar amounts, dates). If I see a lot of those, I know it’s going to be a slower read for me, and I’ll need a lot more concentration to truly digest it.
For sign-up pages, I want to know quickly what I get. Your winning page does that.
Brandon Durham
on 15 Jan 09What are you guys tracking as a conversion? Clicking the button or completing the sign up form?
Hendrik-Jan Francke
on 16 Jan 09The first headline ‘Start a Highrise Account’ repeats what the visuals below says: ‘Sign Up’. (note that there are give buttons saying ‘sign up’. So it can’t work as well as a headline that provides information as that can compel someone to signup.
I wonder how well the worst performing headline would compare to no headline at all. Are you willing to run that test?
Danny
on 16 Jan 09Hi:
I found this blog very interesting and informative. Now the page has been redesigned and the text and layout is different. Did your tests confirm an even greater conversion with this latest layout?
Joseph Rueter
on 18 Jan 09People don’t seem to care that it is a High Rise account. They seem to care that they can trial that account in a simple way with minimal sacrifice in even the time to set it up.
Ognjen Todic
on 19 Jan 09Ted Rheingold from Dogster gave a talk at Startonomics back in Oct 2008 and touched on this topic. They had a huge up-tick in signup but it turned out these were not valuable users. Slide presentation at: http://www.slideshare.net/Startonomics/startonomics-08-ted-rheingold-presentation
Greg
on 19 Jan 09Great post. We looked into AB testing last year but $10K a month was way out of our price range.
This discussion is closed.