Episode 180 | Posted on

How To Win More Conversions with Khalid Saleh

Conversion is something we all want more of yet increasing your conversions can be tricky. There’s no magic bullet. No one size fits all solution to the conversion problem. One man who knows conversion optimization inside and out is Khalid Saleh. He’s the author of Conversion Optimization published by O’Reilly. Khalid is also Cofounder and CEO of Invesp, a company that has conversion optimized such brands as eBay and the Discovery Channel. If Khalid’s name seems familiar, it may be because he’s been on the show before, specifically episode number 44. On this episode, Khalid and I will be getting into the nitty-gritty of conversion rate optimization or CRO from A/B testing to conversion audits, copywriting and more. Even if you think you’re doing everything right and your conversion rate is amazing, I guarantee you’re going to learn some valuable stuff. This conversation is packed with practical advice that will help even the most advanced marketers win more conversions.


Khalid, it’s great to have you on the show.

I’m glad to be back.

You’re one of the few who has been back. I don’t normally have people come back because what are you going to talk about the second time? Most guests could go on and on talking about different things that they had spoken about on each episode, but I like to keep things fresh and mix things up with different guests. The topic is important. We haven’t had enough conversion experts on the show. You’re the guy who wrote one of the books like the O’Reilly book, Conversion Optimization, specifically. You’re just cool, I’ve got to have you back. Thanks for coming back.

I appreciate the invitation and I love talking about conversion optimization as well. It’s always good to be on a great show talking about my favorite topic.

It was several years ago that you spoke on the show and a lot has changed in the last several years. Let’s talk about what is hot and new in the conversion optimization space or CRO space.

Companies need to set the right expectation and rethink their approach on how to increase their website conversion rates.


What has been interesting is the amount of focus that conversion optimization has received in the last few years where more and more companies are focused on it. You can measure the interest in conversion optimization by looking at the number of conferences that are dedicated to the topic. Back in 2009, we’ve had a single conference. Fast forward to 2018, you’re talking about 50 to 20 different conferences that are dedicated to conversion optimization. What we’re seeing is that the idea is gaining traction. There’s a lot of research that’s getting published. There are lots of interest in conversion optimization.

At the same time, the promise that conversion optimization will be that boat that’s going to save businesses and help deliver massive increases in revenue is failing to some extent. The companies are putting too much hope in conversion optimization and learning a hard lesson that they need to set the right expectation. They need to think their approach into how to increase their website conversion rates. That it’s a slow and painful process. It delivers. You’ve got to believe in the process. You’ve got to go through it, but it’s not going to be an overnight success. That’s the latest and greatest with conversion optimization.

Is that pain for the client or is that pain for you, the CRO agency, or both?

One of the things I wrote about and I’ve spoken about in different conferences, lots of times people come in and we got used to the idea of this instant gratification like, “I’m going to come in. I’m going to do a couple of quick things. I’m going to see amazing results.” I always say, “Let’s move away from this concept of the low-hanging fruit because something might look like a low-hanging fruit because you have not done it before. It looks easy because you have not tried it. You get into it and you discover that there’s more to the process.” As a conversion optimization company that focuses on that, our task is to try and figure out, “What is the job that our clients are hiring us to do? How do we improve that experience for them? How do we make it as painless as possible? How do we set their expectations correctly? More importantly, how do we deliver the results that they’re looking for?”

That has been a major transformation for us over the last couple of years where typically we’ve had to go to some clients and say, “You have the wrong expectations. We’re not going to be able to deliver those.” We went to other clients and we said, “The way you’re thinking about conversion optimization is not correct. Here’s a better approach.” We’ve seen our typical clients used to stay with us for about a year-and-a-half. Now, we’re looking at clients that are staying with us anywhere between two-and-a-half years to three-and-a-half years. We’ve had some clients with us since 2011 and it’s always the same challenge. How do you increase conversion rates? Imagine a website that starts with a 1% conversion rate and jumps off to close to 10% a few years later. You’re talking about 10x the revenue.

Our best clients are the ones who challenge us. Click To Tweet

We’ve had that with a couple of our clients. Our longest client had been with us for a long time. I remember the first time they came to us it was a two-man show, the CEO and one-person marketing department. He did everything. You mentioned that he did it. We worked with them, focused on conversion optimization. Nowadays, they have dedicated conversion optimization departments. Last I spoke with him, they had about eleven people working in their conversion optimization department. It’s amazing to see the journey because we worked together. We push each other. That’s what’s amazing because it’s extremely rewarding. Both teams have grown through this process, and it’s one of those things I’m proud to say, “Our best clients are the ones who challenge us to be better. They will stand by us when we make mistakes and we will grow together.”

I can relate to the length of time that a client stays on and how dedicated and how the ROI happens for those that are more willing to stay longer. They spend more on the services but they also get way more ROI. I’ll have clients who stay for multiple years and they’re the ones who have incredible hockey stick-like growth curves on their SEMrush reports and Ahrefs reports and so forth. We even have some great case studies like that at the StephanSpencer.com website. You also get clients who are coming in for a one-off project. They want an SEO audit. In your case, it might be a CRO audit. They’re like, “Give me all the juice. Give me the nuggets. I’ll go off and implement it. I need a one or two-month assignment project done. Can you do that?” I hate those.

We do projects like these where somebody comes in. I always give them the same speech. Some of those clients are interesting because we will do a discovery where we will uncover the issues. We will do an audit. The challenge typically is not in uncovering the issues. At least 50% of the issues are easy to uncover. The challenge is, “Here is the list of items that you need to fix. Here’s a list of research of things you need to focus on. Here is a different way that you think about your customers and then their journey.” The challenge is in implementing all those recommendations. When everybody comes in and they sign up, they’re excited, “We’re going to do a CRO,” or in your case, “We’re going to do SEO.” A couple of months later after you deliver your report, the priorities shifted and changed. They have this other interesting project they’ve jumped on. We have this one client who would come to us basically around March of every year and they would ask for a CRO audit. I’m like, “It’s almost the same CRO audit that you received for the few years.” It’s something that they need to put a check mark next to and they haven’t implemented any of our recommendations in the last few reports. It’s finding that balance of trying to provide the right service and at the same time advise the client of what’s going to work for them.

Do you keep doing the same audit basically every year for that client? Do you finally tell them, “You’re wasting your money here, if you’re not going to implement a new thing, there’s no ROI here?” There’s no ROI from an audit or any strategy deliverable that doesn’t get implemented.

That’s what I told him the last time and he was extremely offended. He said, “I am willing to pay money and you’re telling me no. This is not a way to keep our relationship.” I don’t think he appreciated what I had to tell him because basically what I told him is, “Me telling you no is the way to keep our relationship.” What will happen is he will take these reports and then eventually somebody will look at them and they will say, “Those guys have been giving us the same recommendations. They just took our money,” and then they will get angrier with us. We’re better off then. It doesn’t make sense. We appreciate the business, but you’re better off implementing the previous year’s report. Not a whole lot has changed since then.

Move away from the concept of the low-hanging fruit. Something might just look like a low-hanging fruit and looks really easy because you have not tried it before.


I can remember one case where I dealt out some tough love and the client took me up on it and had this huge trajectory that he was not expecting at all. He’s a past guest on the show, Steve Spangler. He was a client of mine for many years. Do you remember him speaking at Shop.org and in that retailer?


He’s in the National Speakers Association Hall of Fame. He’s a big deal and he’s won some Emmy awards.

It’s one of those things I’ve learned over the years we’ve been doing conversion optimization is to ask awkward questions like, “Why haven’t you implemented anything in the last three or four?” Awkward questions deserve awkward answers and people answers. Everyone is better off by doing that as opposed to pretending that there is no problem and continued slugging along.

Here’s what happened with Steve Spangler. He comes to me in 2004 when we’d already done an audit and it had generated some nice ROI for him and his company. He wanted us to do a Flash-based thing. I don’t even remember what it was. I refused. I said, “That is a waste of money. I’m not going to do it and my team’s not going to do it. I want you to start a blog instead.” He wasn’t blogging yet and he agreed. He was surprised that I was being tough. It was a good call. The whole Diet Coke Mentos thing, he created that science experiment that blew up all over on YouTube and everybody was doing it. I met with one of these viral sensations. If he didn’t have a blog as a platform to rein that back end to give him credit for that whole thing. Eepy Bird was probably the most famous example of the Diet Coke-Mentos experiment. They were getting a lot of credit for it, but Steve was the inventor, not those Eepy Bird guys. Eventually, he got credit and the Wikipedia article about Diet Coke-Mentos experiment and so forth. If he didn’t have that platform of a blog, things could have gone sideways.

We’re all consumers; everything has been commoditized. Click To Tweet

Nowadays, everybody takes it for granted. At that point in time, blogging was picking up. I’m one of those guys who tries to avoid, “Here’s the latest, flashiest thing that you need to do.” At the same time, I always wonder what the next big thing is we need to focus on and invest wisely to deliver value for our customers. It’s always the question that’s on my mind.

What’s the answer?

That’s the $1 million question. I have to say over the last several months, I’ve been thinking about our approach to conversion optimization, our approach to marketing and thinking through. We go from being one of the first companies to do conversion optimization in the US back in 2006 to nowadays where there are a plethora of conversion optimization companies. The challenge I have as a CRO firm is how do I distinguish myself? The bigger question and that every business needs to figure out is – how do you distinguish yourself from any other competing company that offers a similar service or a similar product? In 2017, this large company came in and said, “We need to do conversion rate optimization. We’ve contacted the top five firms in the world to do CRO. This is a fairly large project. You guys are one of those firms. We need to understand your process.”

I prepared the process and we go through it. I’m talking to their CMO and we established some personal friendship. I said, “Michael, what do you think of our process compared to the other companies that he talked to?” He said, “Khalid, can I be honest with you?” I said, “Sure. That’s exactly the reason I asked you.” He said, “All of you sound exactly the same. You guys are the top firms, the amazing guys who deliver increases in conversion rates. When I forget who I’m talking to, you sound the same.” We were all different but in the same way.

That comes to me and I think about our clients. In our case, if you’re an eCommerce company you’re competing with Amazon, you’re competing with eBay, you’re competing with the top companies. How do you distinguish yourself? For consumers, everything has been commoditized. I can get the same products or services from walking down the street from a brick and mortar store. I can go online and buy it from a different place. How do you compete in an age where everything has been commoditized? One way to compete is creating an amazing experience for your customers. Figure out what exactly they want to hire your product or services to do with the job that they hire you to do. It’s architecting and crafting every little detail and that experience. Architect it in a way where the interaction is to deliver amazing value to the customer.

50% of issues are easy to uncover. The challenge is identifying the items that you need to fix and actually implementing all those recommendations.


I recall this learning I got from somewhere, I forget where, but companies that you work with at most you get two out of the following three; high quality, cheap and fast. Which two would be your two?

When I look back at 2006, we were talking with conversion optimization and trying to learn it in all honesty at that point. We knew usability, but we confused usability with conversion optimization. It took us years to figure out the difference between the two. Many companies are still confused about that. I’m like, “Should I call the first project than what we charged on that project versus nowadays?” The quality had improved tremendously. You cannot even compare the two. Let me give you an example of the experience. For the longest time, whenever we started a project, we did a kickoff meeting with our client. Basically, we go to a meeting, go to a webinar and we talk to the client and then we’ll do all these virtual meetings.

Thinking about the experience and how you can craft an amazing experience. We thought, “We can do something a bit different.” Nowadays, we ask the clients to fly to our location, which is different because the client is not expecting that and we tell them, “If you want to hire us, you have to fly to us.” They say, “If you want us to start, we can fly to your location.” We’ll do the kickoff. It’s about one week worth of activities, “If you want to start before May, if you want to start in March, fly to our location.” By doing this, we have a client for a week dedicated to us. We’re removing them from their environment. By merely bringing the client to our location, having them disconnect from their office, remove their cell phones, the amount of knowledge sharing that the client team members do with each other is incredible. It’s incredible the number of new ideas they come up with. We’re sitting there and watching and that has been tremendous.

That’s what I mean by crafting that amazing experience for the clients where initially there was resistance, but then our clients look forward. We did the kickoff with a client and he calls me up and he said, “When are we coming to your location again?” I’m like, “Why?” He said, “I sat with my team away from being busy. It has been incredible.” They come out of their environment, bring them to your location. Sometimes we resist. They’re not used to that. Imagine, in all honesty at this time, having somebody spend two to three days with you. How much value will they get out of that? It’s not only about SEO. You have an incredible experience when it comes to marketing insights, running a business and your clients will get to see that. All other SEO guys will probably do a virtual kickoff. Next time, you do that.

I’m afraid of the pushback that, “I’m not going to because I’m too busy. I can’t do that.” What do you do? Are you going to play the heavy there and say, “I’m not working with you if you don’t fly out for the kickoff meeting?”

Architecting and crafting every little detail of the experience delivers amazing value to the customer. Click To Tweet

We’ll tell them, “We will start with you. In that case, we will fly. We cannot because we’re booked for the next 30, 45 days. We will do the kickoff.” What we found is that it’s about establishing a relationship. It’s about crafting that experience with our clients where they’re staying with us for two or three years. Every little detail matters. I skipped doing this where we said, “We’re going to fly to the client’s sites.” My partner, Ayat, she called me up, “This was a horrible idea.” I’m like, “Why?” She was like, “I cannot have the client focus with me for more than an hour or more than 45 minutes. People are in and out. They’re removed from the environment. They’re not engaged.” It’s that fine balance of trying to do what’s right for the client. At the same time, you don’t want to lose the business.

I get the process. You give them the option but they have to wait. There’s some pain involved if they say, “We’re not flying to you, you fly to us.” In the early days, you confused usability with the conversion. I’d love to hear more about that because you get people touting usability experts like Jakob Nielsen and Jared Spool as if it’s gospel truth. Sometimes, the horribly designed websites are the ones run by usability experts. I remember looking at Jakob Nielson’s website years ago, it was horrible. I had Jared Spool on this podcast and we did not talk that much about usability. We talked about analytics and metrics. The metrics that don’t move the needle or make a difference for me make any sense. He was railing on NPS score, Net Promoter Score, for example. I thought that was a good metric but he was like, “This is a terrible metric and here’s why.” What are your thoughts about usability and conversion now knowing what you know after all these years of deep diving into conversion and usability? There is an aspect of what you do that’s about usability

The way I think of it is like a pyramid. Imagine a pyramid with three different levels of things you want to focus on. At the lowest level, you have bugs. Things on a website or an application where the website is broke. We don’t have to think about usability or conversion. You’re trying to do something and the screen crashes. That’s simple. Everybody agrees on that. On the second level is usability. Usability focuses on the functional aspects of a product or service that make it easy to use. When I focus on our usability audit or say that this is a user-friendly website, the interaction and the mind of the customer who’s dealing with the website are thinking, “I can buy from this website.” Lots of times, you’re dealing with top of mind issues. That’s where usability comes in and websites until nowadays struggle with usability issues making the website functional, but that’s only one dimension of a product or a service or a website.

A deeper level is the emotional and social aspects of a website. That’s what conversion optimization is about. It’s the top level of that pyramid where when you have a conversion-focused website, the attitude of the visitor is not, “I can buy from this website.” It shifts completely to, “I want to buy from this website.” When you’re talking about conversion optimization, what you’re talking about is trying to increase the website conversion rate. You’ll definitely want to have the first two levels figured out. No bugs on the website, usability issues, but those are exactly where you expect them to be when you click on them, the right action is happening, the user’s not surprised by something. That prepares the journey to increase conversion rates.

You want to go to that next level where somebody converts on your website. Not only that, when they go home or when they go to the office the next day they look at their colleague and they say, “This was cool. I bought from this website.” When was the last time you bought from a website and you looked at a friend or your wife and told them, “This is a cool website, you must check it out?” It rarely happens because we also as humans, there is something called the keno effect where if a website or a business provides an amazing experience to us, the first time we’re amazed by it like, “This is really nice.” By the third, fourth, fifth or tenth time, we get used to it and it becomes expected. Think of Amazon Prime. It’s a simple concept. Two days you pay for membership, you get two days free shipping. It was cool when it came out. Nowadays, we expect it. There are many other concepts. That’s in general how the difference between focusing on usability, focusing on conversion is that social and emotional dimension that conversion optimization pays close attention to.

The bigger question that every business needs to figure out is how do you distinguish yourself from any other competing company that offers a similar service or a similar product?


Can you give me an example where a usability expert comes in and breaks the CRO? They don’t understand the emotional aspects of the top of the pyramid and they do stuff that makes the site supposedly easier to use and more intuitive. You lose some of that emotion or some of the juice that makes people want to do business with you and then the conversions go down, the sales go down.

I’ll give you an example. Lots of times what I find is not that the usability experts are causing a problem for conversion. Rather many of the usability guys and even the conversion guys focus on usability thinking that it is going to be this huge amazing thing that’s going to deliver double-digit increases in conversion rates. They’re focusing on usability. Usability is typically easy to handle. It doesn’t require a lot of effort and it’s not going to deliver amazing increases in conversion rates. It might increase conversion rates by 10%, 15% but that’s it. If you’re looking to double your conversion rate, you need to take a different tactic. Let me give you a story from a client that we’re working with. They sell their clothing and apparel website, a fairly large website. They started running out of stock of some of their items because they have new looks that come out every quarter. They ran out of some of the items. What we did was instead of saying out of stock, we start telling people that first, we had delays in shipping some of those items. We said, “Let’s tell people about those delays. Let’s tell them that due to heavy demand, there’s a three-day delay in shipping.”

Imagine we live in an age where Amazon ships your order. If you place an order now, we’ll ship it now. You will have it in two days. Here we come and we tell people, “Your order’s going to be delayed two or three days.” The usability guy we’re working with is like, “This is horrible.” I’m like, “This is the reality that we have in the business that you are not able to ship your orders right away. Let’s use it to our advantage. Let’s make the conversion process harder where people have to stop.” By doing the small trick, we saw conversions increase tremendously. Our client comes and says, “We want to do that all across the website all the time.” I’m like, “This worked well because it was true.” If you use it as another trick on people to convert on your website, that’s when you start losing. You understand the situation. You understand the emotional aspects of buying a product from you and then utilize that to increase your conversion rates.”

It’s got to be real urgency or scarcity, not fake. You have to be in integrity here and not try and fool or lie to your clientele.

We’ve had this one client and then we departed on one note. Basically, they had used urgency as scarcity to the point where they killed the experience. You’ve got to the point where like, “There are only three items left on stock. There are ten more minutes for you to place an order.” I’m like, “You might be able to trick a customer once, fool him once but in the long run it doesn’t work.” They had this counter. You have this special offer on the cart page where they give you ten minutes to place an order. I look at it then I’m like, “What happens when the counter runs out?” He says, “The counter restarts.” I’m like, “You think you’re fooling people by doing this?” We parted ways because we had different ways of thinking of how you should do conversion optimization. That same client reached out a few months ago to say, “We’re struggling with our conversion rates,” and I looked and it’s the same tricks. I told them, “You’ve got to think about the experience. Trying to trick people does not work. It’s not working for you.”

You might be able to trick a customer once and fool him once, but in the long run it doesn't work. Click To Tweet

You can use influence and persuasion techniques for good or you could use it for evil. I believe in business karma that what goes around comes around. If you’re going to be disingenuous, you’re going to lose in the long-term.

Thinking short-term, I would say 99% of the times it does not work.

This bottom of the pyramid where something is broken, it’s important to emphasize that not everything deserves a test. You have things that don’t make any sense. The button or whatever is in the wrong place where no one would ever guess to click on it or they’re clicking on something that should be clickable, but it’s not. There’s a term for that or people get irate. They keep clicking and it’s called a rage click. They’re clicking and they want it to go somewhere and it doesn’t. That’s something you can measure. Not everything deserves a test. Let’s talk about that a little bit more.

We’re a conversion optimization company, but one of the things we’ve done in the last several months is where we would deliver recommendations to clients and they’re mixed recommendations. Sometimes we’d tell them there is a set of recommendations, what we call to stop the bleeding reports. Things that you need to fix on your website. It’s broke. You don’t need to think through it. I’ll give you an example. We had this one client. They complain about the checkout abandonment rates, about 93%. It’s horrendous. Usually, cart abandonment is about 65%, checkout abandonment is about 40%. When you look at 93%, there’s a problem. We sit there and I’m sitting with the team. I’m like, “93%, that’s unusual. You have to really have to screw this one up to get to that point.”

I sit there and go through the process. I can see the cart go through the checkout. The first thing that asks you, “Create an account.” I’m like, “I’m not a big fan but I’ll create an account.” I put in my email address. I start typing. I have a few passwords that I regularly use, especially on sites that I don’t shop there regularly. My first password complains that it’s too weak. The second password doesn’t work. Combine the two passwords, it doesn’t work. Randomly put numbers and letters and it doesn’t work. I have to download LastPass, put the password up to fifteen characters, put a whole bunch of symbols and it accepts my password. I call the client back and I’m like, “It doesn’t need a whole lot of analysis. What’s going on over here? This is silly.” Their development lead answered me and said, “The site was hacked a couple of weeks ago. We find that this is extremely important.” I’m like, “You can have it but then don’t complain about the 93% abandonment rates.” Those items you don’t have to test those. Go ahead and fix them.

Running A/B tests, the more variations you introduce, the higher your chances are of finding a false positive.


There are two types of A/B tests that you want to conduct on a project. There is what we call strategic A/B test and there are tactical A/B tests. The tactical A/B test might be a quick analysis. You have some customer insights. I always say I’m trying different headlines on a page, something that resonates better with customers. Those are quick. Then there are some more strategic A/B tests where you’re spending the time to do the analysis, pre-tests, trying to figure out, “What’s motivating our customers to come over to our website? What’s the trigger that’s going to encourage them to convert? How do we emphasize that and a new design or a new process flow?” We conduct that as an A/B test.

This is within the A/B testing realm, but then what we’re doing is about 25% of our recommendations to clients are more of what we call strategic projects. We’re not thinking even about one test or two tests. We’re talking about much longer term, larger impact, more about positioning the company in the market place in a tough competition. How do you stand out? Those are tough ones. You cannot implement them as an A/B test. These require you to sit back and think about your positioning. How do you stand out? How do you craft and orchestrate that amazing experience for your customers? It takes quite a bit of thinking, but that’s how we stand out in a crowded market space.

What would be an example of a strategic A/B test that was successful for one of your clients?

We’re doing something of a mix between strategic and also a strategic project. We have a client that is a large furniture retailer. They’re doing some amazing work. Although they have their traditional stores all across the country, they’re trying to grow the online channel and trying to figure out, “How do we grow that channel?” It’s about 27%, 28% in their sales. They get to 33% thinking about the integration points between their online stores and offline stores, running those tests. If you’ve added items to your cart and you’re walking into a store or you flipped through some items. How do we inform the salesperson, “This person had looked at those particular items. We already have them in stock?” How do you prepare that profile for that salesperson so they provide relevant information to the customer? This is not an easy A/B test. This requires quite a bit of integration. At some point, I was talking to the client CMO and what I told him is this is more of the direction that the business needs to go into. Is this important? Is that omnichannel integration extremely important? How do we bet on something like that?

I’ll give you another example where we have a client that sells skin care for men. I take care of my skin but I didn’t know about the monthly subscription where I get some creams and lotions and things of that nature. When we did the initial analysis for them, about 47% of their clientele came back and said, “I love the product. It’s amazing. I need something for my acne,” because of lots of their clientele were basically teenagers. We went back to the client and we said, “Acne is something that you want to think about because 47% of the customers that we surveyed pointed out that this is something important.” I cannot tell you the amount of resistance that I got when we had that call with a client where they’re like, “We don’t need to do that. This is not our core product.” Needless to say, they’ve launched the acne products. We’ve done quite a bit of optimization for them. They launched it a few months ago and it’s one of their most successful products.

An amazing tool is a tool that does not require any help guide of Frequently Asked Questions because it's so clear and so easy to use. Click To Tweet

Do you get any percentage or is he just getting your flat fee?

We get a flat fee. I remember a few years ago where we tried to do the percentage and I’m like, “There’s some good work that we present and it all sounds good until the client starts thinking about the percentage.” It’s not during the sales process when they’re signing up the contract with you and they’re thinking, “We can save money.” It’s when an idea works out and now the client is thinking, “I have to pay this much money.” We’ve decided to move away from that and say, “Here’s a flat fee.” This way, it works for our clients and it works for us.

You can make so much more money by charging a percentage.

What I discovered is that then the clients who did not know anything about statistics or data, the minute they start looking at the amount of money they have to pay, they become a Ph.D. in statistics. I’m like, “We don’t need to do that. We’ll save you the headache or the heartburn.”

What about multivariate tests? Do you do a lot of those or is it mostly A/B test? There are some issues with multivariate tests. For example, you have to have a statistically significant enough sample size, and that means a large number of conversions to get that. What are you doing in terms of multivariate testing?

What analytics reports and what A/B testing platforms report are two different things because they measure different things.


We have not done a multivariate test since 2015. It is not for the lack of clients. Some of our clients are large, some of the largest companies out there. Some of them are on the small side like startups. Even for our larger clients, we’ve shied away from multivariate testing for many reasons. Even if you have the traffic and the conversions and you’re able to figure out statistically how to run a solid multivariate test. The problem in a multivariate test you’ve changed too many factors. Even when you have a winning design, you sit there and you say, “What was the insight? Why did this one headline, with this one image, with this one call to action, with this one product feature and offer work together?” In all honesty, what we find people are randomly guessing. When you’re randomly guessing, you lose the value of an A/B testing exercise because you’re not driving any insights. What we find is that even for our larger clients, we launch A/B tests that include three to five variations against an original. We’re splitting the traffic that the side gets between the original website design and three or five other challengers.

We even moved away from using the traditional statistical analysis or what’s called Frequentist, which is what you find with most A/B testing software to something called Bayesian. Whenever you’re on A/B test and we use statistical significance, the biggest challenge is you want to avoid the cases where you find a false positive. You think, “This variation increased our conversion rates by 15%.” You take it, you deploy it on your website and nothing happens. Even worse, your conversion rates go down. What most people don’t know are the more variations you introduce and at best, the higher your chances are of finding a false positive. At five variations, you have a 50% chance of finding a false positive. 50% is horrendous. You run a test, you invest so much time. Because you have five variations, you end up with a 50% chance of having a false positive. There are lots of statistical analysis that we do to avoid falling into traps like that.

Why don’t we jump into a lightning round because I want to find out some of your favorite tools, books, resources and things like that? Let’s start with what are some of your most favorite metrics that are not necessarily in wide use, that are maybe even counterintuitive or unusual. For example, rage clicks, not every tool measures that. I don’t even know if that’s a metric that you consider important or not. Let’s talk about your favorite metrics.

Being a conversion optimization guy, I look at the conversion rates closely. We look at also customer retention rates, customer churn rates even for eCommerce websites. We spend quite a bit of time trying to figure out how you optimize the time to improve that lifetime value for our customer. There are many other metrics, but those are the ones that I look at quite closely.

What are your favorite tools for CRO?

There are many different tools out there and they’re all are about the same regardless. When you peel the onion, they all provide the same services, the same numbers, and different methods. For example, for surveys, we use Hotjar. For testing, we have our own internal testing tool that we use that’s available in private data, FigPii.com. Google Analytics is a wealth of information, but we have to make sure that it’s configured correctly. About 80% of the Google Analytics implementations that we see are junk. Somebody threw a script on a website and thought, “It should be working. Garbage in, garbage out.”

Google has an optimization platform. Would you recommend that?

It’s okay at best and all honesty. Investing in some of the other platforms, more like there’s VWO and there’s Optimizely. I like those more and it goes back to also how much information you want to give Google. Even Google Analytics is plenty. Some of those other tools, even Microsoft is coming up with their own optimization tool that has lots of the CRO project management built into it. That will be interesting to see how that plays out. The challenges you want to keep in mind, that’s what analytics reports and what A/B testing platforms report are two different things because they measure different things and people struggle with that quite a bit.

Between VWO or Visual Website Optimizer and Optimizely, which do you prefer?

This Is Marketing: You Can’t Be Seen Until You Learn to See

I like VWO.

It’s also cheaper.

I’m a CRO guy. I do use both all the time. I come from a technical background. I was trying to launch an A/B test on Optimizely and I struggled. To me, an amazing tool is a tool that does not require any help guide. It does not require frequently asked questions because it’s clear, it’s easy to use. If I cannot figure out how to launch on Optimizely, then you have a problem. At the same time, VWO over the years how they added more and more, definitely they think about usability. Not even conversion optimization on their site and make it a tool a lot easier to use.

What about your favorite resources and books? Online resources too like TestOne is a cool resource for example that I quite like.

In all honesty, what I’ve been going back to is a different set of books where I’m rethinking about the marketing. How traditional marketing and traditional marketing concepts need to be adapted to the online environment. A book that I’m going through is This Is Marketing by Seth Godin.

I’ve had him on this show. We talked about the book.

It’s like you’re talking to a fountain of knowledge. A book that’s helped me rethink my business and how we conduct the business is Competing Against Luck. It is really awesome. Bob Moesta, the consultant who is mentioned in the book again and again, is from Detroit our hometown. It’s been amazing meeting Bob and he’s the one who referred us to the book. I’ve read that book and made me change the way we conduct our business, how we run things, how we do conversion optimization even for our clients. I’ll give you the last book that I read, It Does Not Have To Be Crazy At Work, from the guys at 37signals. If you follow lots of the concepts regularly, they’re there but it’s nice to have them, especially if you work with a remote team and you have distributed teams.

If somebody wanted to work with you, if they wanted to take the next step and embark on a CRO journey, what would you suggest? How would they get in touch?

We get lots of people asking about their websites and how they compare to other websites. One thing that we’ve done is we’ve made this report where we customize for each site. It’s basically a benchmark report that will show you how your website performs against twenty different metrics when it comes to conversion rates. We’ll give you an overall score. It’s customized for each website and we’ll tell you, “Here’s what you need to focus on, more usability, no more conversion issues, and psychological issues. You have bugs and issues like that.” That’s a good starting point because it allows us to make sure that we’re on the same level. We agree on the problems on the website and that usually is a nice way to at least get started thinking about conversion optimization. The link to that is MarketingSpeak.com/crobenchmark. It’s a quick form. They fill it out. We’ll have one of our conversion optimization specialists reach out. The report takes about a few days to complete. We try and turn them quickly, but it’s a report that’s completed by multiple people. We make sure that the data that people receive is of value.

Thank you so much for that generous gift for our audience. To our audience, take some action and take advantage of that amazing conversion benchmark report. Start making some changes to your website so that you get uplift in your conversion rate. We’ll catch you on the next episode of Marketing Speak.

Important Links:

Your Checklist of Actions to Take

Implement a conversion rate optimization strategy to my website, especially when I am selling or offering something of value to my target viewers.
☑ Be patient with the conversion optimization process. It can be slow and take time to deliver.
☑ Find the right balance between providing my clients with service and recommendations on what’s going to work for them.
☑ Take a closer look at how visitors navigate through my website. Reflect on whether I am giving them the utmost convenience to achieve a desired outcome.
☑ Create a fantastic customer experience. Figure out what my customers want to purchase and then construct it in a way that delivers them amazing value.
☑ Focus on the emotional dimension of conversion. I should be able to understand and utilize the emotional aspects of buying a product to increase my conversion rates.
☑ Learn more about the two types of A/B tests, which are the strategic A/B test and the tactical A/B test, and find out which one is more implementable for my conversion goals.
☑ Invest in Conversion Rate Optimization tools like Hotjar, VWO or Optimizely. Keep in mind that each tool basically provides the same services and numbers, but using different methods.
☑ Grab a copy of Khalid’s book, Conversion Optimization, to gain a deeper understanding of conversion and to convince my website visitors to make a buying decision that finalizes into a sale.
☑ Take advantage of Invesp’s customized report that will show me how my website performs against twenty different conversion rate metrics by going to MarketingSpeak.com/crobenchmark.

About Khalid Saleh

Khalid Saleh is the CEO of Invesp, a global provider of conversion optimization services. He is a recognized expert of marketing strategy and author of Amazon.com bestseller “Conversion Optimization: The Art and Science of Converting Visitors into Customer.” Khalid has been presenting at marketing conferences across the globe since 2006.

Leave a Reply

Your email address will not be published. Required fields are marked *