Can You Use A/B Testing with Landing Page Builder Apps?

Learn how to do A/B testing to improve the conversion of your landing page.

Kate Neuer
October 12, 2021

Simple Ideas to Test and Improve Landing Pages Like a Pro


What is A/B testing?

A/B testing (often called Split testing), is a technique that allows you to compare two versions of a page on your website in order to determine which version performs the best.

It involves comparing two slightly different versions of the same page to determine which variation yields the highest conversion rate for the metric you are focused on.

This is how landing pages are routinely edited and improved until their ROI is satisfactory - or the page results stop growing.


Table of Contents:


How Does A/B Testing Work?

It's actually very simple.

You first create a modified landing page that contains a single change, the "variable", for which you believe can improve your conversion rate. It could be the location of your hero image or your headline copy or font style. It could be the wording of the Call to Action (CTA).

You then randomly assign half of your landing page visitors the original version and half the modified version. After the test is completed, compare the results of both pages to determine if the modified landing page had the impact on conversion rates you hypothesized.

If the change is positive, you can adopt the modified page as your new landing page.


Please Note:

It is important to remember that an A/B test isn't finished in a single session. It’s not a one-and-done type of activity.

A/B testing is an ongoing process of making small changes to your website in order to optimize your campaign for maximum conversions. Each test builds on the one before it.

Even a negative test can be useful because it will provide you an idea of what might work - and just as important - what might not.

Multivariate testing is often mentioned alongside A/B testing. Although the two methods may be similar, there are important differences.

A/B testing focuses on changing ONE variable at a time. Multivariate tests multiple variables simultaneously.


Split Testing Landing Page Terminology

Before you begin your testing journey, you should be aware of - and start using - the basic jargon.

Here are the most basic of terms for your testing vocabulary.



A Variant refers to any new versions of your landing pages you add to your A/B test. You can include as many pages in your A/B testing as you like.



You could think of an A/B test as a duel between your pages on your website. There are two (or more) possible variants, but only page should be left standing. The champion variant is the page that converts the most, or improves your target metric(s) by the highest amount..



You can create new versions (variants), to challenge your champion page when you start an experiment. These pages are called challengers. If a challenger outperforms all other variants on your website, it's considered the new champion.


Traffic Weight

In an A/B test, traffic will be randomly assigned to each variant of a page based on a predetermined weighting. If you have two page variations, for example, you might split traffic 50/50 or 60/40. Visitors should see the same variant every time they visit the test to maintain integrity, even if the visitor returns later.

Timing is the main factor in determining how much weight you assign to page variants during a test. It depends on whether you're starting the test with multiple variants simultaneously, or testing new ideas against an existing page.

When starting a new campaign, and you have several ideas about which direction to take with your page layout and content, it's recommended to create a variant for each idea.

In this scenario, you’d most likely assign equal weight to each version of the landing page. For two variants, that’d be 50/50. For three pages, it would be 34/33/33 etc. 

As best practice, treat each page equally on your website, and pick a champion as soon as possible. Since you have no conversion data on any of the pages yet, it's best to begin your experiment from a position of equality.


Key Take-away for You:  

You need to drive a certain amount of traffic through test pages before the results are statistically significant. 
If you need guidance, Google search calculators online like this one from VWO: 


Why Do We Do A/B Testing?

In life, you can't predict the outcome of your plan until it is actually put into action. You simply need to adapt your strategy to the current environment when the unexpected happens.

This logic is very applicable to optimizing the results of landing pages.

Be humble enough to realize that even your best hypothesis of what will make website visitors convert is just a guess. 

It's highly possible you have not considered all the factors, and what worked in theory could generate a different result in practice. 

It's not always easy to predict what will drive conversions for any given target audience. In fact, you'll often find that the catalyst for landing page results are unexpected.

A/B testing offers the advantage of tying your success to hard data and statistics, instead of using guesswork. By continually testing your landing page design hypotheses, you can discover exactly what drives your audience act in ways that take them closer to a profitable conversion. 

With that said, here are the benefits of Split testing your pages:


 - Impact to the Bottom Line

At TruVISIBILITY, we routinely see the impact of small edits to customers' page on their website. 
Sometimes a simple restating of the headline can produce 2-3 digit improvements in sales and revenue.
It takes time and money to generate high-quality traffic to a website. Smart businesses discovered a long time ago that the key is to make the most of what you are already getting.
A higher conversion rate will lead to greater revenues, regardless of whether you are using your landing page for leads generation or sales.

- Improved User Experience

Anyone visiting your our landing page will likely have a specific goal.
Perhaps they are just curious about your product.
Perhaps they have already decided to buy from your company.
Or, maybe they are just curious to learn whether you're the most trustworthy of all the brands they are considering.

It doesn't matter what their goal is, it's critical to make the user's experience on your page as easy - or as frictionless - as possible. 

Confusing content, visually shocking color schemes, and difficult-to-find buttons are all potential stumbling blocks for visitors who want to find what they are searching for. 
These elements, when not weeded out and improved upon, could detract from the user experience and destroy your conversion rate. 

Split testing helps you pinpoint these problems so you can create a more embraceable and pleasant user experiences.  

- High Reward from Minimal Effort

A/B testing is an incremental process, with only one change at each stage. This means that there is very little chance of your conversion rate dropping off the cliff during any test. Even if you find that changing a specific variable causes a significant dip in conversions, you can simply discard the change. If the change increases your conversion rate, you are one step closer to a page that is performing at it's best. 

- More Testing = a Culture of Constant Improvement

You'll gain a better, deeper understanding of your target audience's behavior - and what drives them - by running A/B tests. Uncovering these drivers will help you create better experiments, and hopefully result in better results continually.

This process can become like an upward spiral of successes, culminating in massive value for your company's bottom line. 


How to Decide What to Test on Your Page

Now that you’re convinced that A/B testing is an important part of growing your business, let's get into how you can start running experiments on your landing pages. 

But, where do you start?

It would be foolish to fire up your A B testing software and start changing everything at random. A better approach is to choose hypotheses of what might work based on the current performance (or the lack of results) of your landing pages.

There are several ways to determine how well your page is performing: 
 - You can analyze the quantitative data found in your analytics.
 - You can review the qualitative feedback you get from test-driving the page with friends, family and co-workers. 

In general, you should tap all the data sources at your disposal to figure out where your page could be doing better. 

For example, if your Google analytics reveal that 60% of visitors leave while they’re filling out the sign-up form, you could test reducing the number of fields on your form. 

If a heatmap shows that users struggle to understand what the page is selling, edit your headline to make the value clearer. 

Now, when you've got a list of good tests to run, follow the I.C.E. prioritization framework to help you decide which experiments to prioritize.

The method simply asks you to consider three questions:

  1. How impactful do I expect this test to be?   (Impact)
  2. How sure am I that this test will prove my hypothesis?   (Confidence)
  3. How easily can I launch this test?   (Ease

Each question should be answered with a score between 1 and 10. These three scores are combined to determine your overall I.C.E. score.

For example, if you assigned a test the following scores below:

  • Impact: 5
  • Confidence: 4
  • Ease: 8. 

This test's overall ICE score would be 5.6. 
It would be wiser to prioritize any tests that have a greatest ICE scores first
Then, when they are all done, start this one. 


How to Use a Smart A/B Testing Approach with Your Landing Pages

Every landing page is different. Depending on the conversion goal(s) of your page, you'll have several elements you can test. 
Some of the most common elements of landing pages that you should experiment modifying are shown below. 
And the extent to which you test them, your results could jump up or down significantly. 
The point is, test carefully with a focus on one of the following elements at a time. 


- Headline




It's important to make a good first impression. Your headline will be the first thing visitors see. It’s critical for you to get this right. 

Uninteresting, confusing, or boring headlines can deter visitors from continuing reading. It could also result in a decrease in conversion rates, or at best, poor results. 

If you think your headline could use some work, try testing different versions of:

  • The title lengths
  • The wording
  • Benefit statements
  • Font sizes
  • Term emphasis
  • Wrapping
  • Etc. 

Crafting the perfect headline is often a difficult task. However, getting visitors to stop bouncing and engage your content long enough to convert is always worth the effort. 


- Hero Image 

The hero shot is the main image that appears above the fold, and it supports the most important value proposition of the page. 

It should showcase your product or service in a real-life context. 
But how do you determine which hero shot will be the highest converting for your landing page?
Do you go with the typical smiling couple everyone loves to use on their page? 
What about a close up shot of the product itself or an app screenshot? 
The best answer is: experiment and find out.

Key Take-away for You: 

The hero shot you choose should match your message closely. Please don't show puppy dogs on your hero shot when you're offering cat beds on your page. Doing this will disrupt the visitor's interest in your page, and it could also cause you to pay more for the ads you are buying. 


- Copy




Your landing page body copy should be crystal-clear, just like your headline. 
It should clearly state the value your visitors will receive by submitting their payment information or contact details.

Your copy should address any objections or obvious questions. Also, try to strike the right tone with your audience.

You should not overload your page with text-heavy content blocks. Testing has shown this causes visitors to leave the page.

In extreme cases, consider breaking down large amounts of text on your page into smaller chunks or bullet points to make it easier to read.


- Opt-in Forms




When using opt-in forms, the number of required fields should be kept as few as possible.

If users feel you are asking for too many details, they may not be inclined to enter their information.

Most lead capture forms only require a name and an email address. Anything else is usually considered excessive.


- CTA Buttons




The CTA button is always the focal point of a sound landing page. This is where you encourage your visitors to take the next step. 

Understanding their importance, it’s critical that your CTA buttons are:

  • Instantly clear
  • Meaningful to the visitor
  • Visually available at all times. 

Any aspect of your CTA button could have an impact on your conversion rate, from placement to wording to color. 

You can test different combinations to find the one that works best.


- Price Levels




For decades, pricing experts have been able to see the power of numbers.

Customers' impression of whether something is cheap, expensive or a good value can be swayed by how the numbers are displayed. 

For instance, in the minds of restaurant consumers it appears that:
 - $4.99 seems a lot cheaper than $5
 - The dish listed as “mozzarella sticks: $5” seems more expensive than the one that appears as “mozzarella: 5.” 

These subtle differences can be used to your advantage by changing the way you present numbers on your page.


- Countdown Timers




Countdown timers can be a great way to create a sense of scarcity for visitors when they land on your page.

The visual cue informs visitors that you will not be offering the same value for very long. They should act quickly if they don’t want to miss the limited-time offer.

In this situation, why not test a countdown timer to see if it boosts your conversion rate? 


- Social Proof (or User-Generated Content) 




People buy from who they trust. 

By placing testimonials or credentials on your landing page, you can increase conversions and improve the trustworthiness and perceived value of your product or services. 

Just be aware that sometimes, these elements can be distracting from your CTA, and end up cluttering your page.

Which is the right thing do for your page? Test it and find out!


3 Mistakes to Avoid When Running A/B Tests on Your Website

Although the principles behind A/B testing are easy to understand, it can be difficult to put them into practice.

Here are some common mistakes you should avoid when executing your own testing campaigns.


1. Not Running Your Tests for Long Enough

How long is "enough to run an A/B test? The answer to this question varies from one case to another. It really depends on how much traffic you get to your landing page.

Your sample size must be large enough for it to produce useful results. 

Again, the lower your traffic, the longer it makes sense to run your tests before declaring a winner on any experiment.


 2. Split Testing More Than One Variable at a Time

A/B split tests work best when you limit yourself to one or two variables at the maximum (unlike multivariate testing, where many variables are tested concurrently). 

This makes it much easier to determine which changes had a positive impact on your conversions.

Again, use the I.C.E. framework mentioned above to dictate your testing plan.

 3. Inadequately Accounting for Past Results in Your Testing

Before you begin any split testing program, you should take an inventory of past findings with regard to design and copy changes on your landing page. 

In fact, even if you're not doing testing, it's still a good idea to keep an updated record of site changes and updates.
This way, if noticeable shifts in traffic or conversion rates take place, you can retrace your actions to uncover the causes. 

As it relates to setting up tests, you’ll have a much better chance of generating data-driven hypotheses to start off with and avoid running the same test twice.


Well-Known AB Testing Apps for Split Testing Landing Pages

We've talked about the theory of split testing, the elements on a page you should be testing, and some mistakes to steer clear of.
But, how do you actually run the tests? 

This is where AB testing software comes into play. 
These are the apps that actually control which of your landing page variants are shown to each visitor to your site. 
And there are several A/B testing tools available to use in your experimentation. 

Evaluating these tools can be difficult when trying to sift through the marketing messages and identify the right tool for your business. 
Our own research revealed a list of 8 apps that are a good place for you to start your investigation. 

These are the apps most trusted by full-time CRO experts – people whose businesses depend completely on the results they are able to deliver to their clients.

Click on the links to do your own due diligence. 

We’ve listed these in order of frequency with which they were mentioned by our experts. This is not to be confused as a ranking by quality.

  1.  Optimizely
  2.  VWO
  3.  Convert Experiences
  4.  SiteSpect
  5.  AB Tasty
  6.  Google Optimize
  7.  Qubit
  8.  Adobe Target

Time to Start Using A/B Testing on Your Website!

In this post, we’ve discussed how A/B testing can be a methodical strategy for your improving landing page conversion rates.

A/B testing helps you identify what works and what does not. This allows you to create more revenue-generating campaigns, and with greater certainty that you know your audience at a meaningful level.

Now it’s your turn to put what you've learned into practice. Use this post as your guide, and be fearless in making lots of small mistakes on the way to big successes.

The most important part of A/B testing involves clearly identifying what works and what doesn't. 


Get started building your landing pages to A/B test

by signing up for a freemium account today!