I recently listened to a podcast that talked about a behavior known as “survivorship bias.”
Survivorship bias is the act of focusing on successful people, businesses, and processes while ignoring the failures. Focusing on just the successes blinds us to other important data points that help tell the whole story and can make the difference between a good decision and a detrimental decision.
In World War II the airforce conducted a study to identify which areas of their airplanes were susceptible to damage. The goal was to build airplanes strong enough to endure combat and bring the pilots safely back to home base. The airplanes still had to be light enough to fly and since materials were in short supply, the airforce had to be selective about the areas on the airplanes to reinforce.
When planes returned from combat, engineers were instructed to map out the bullet holes so it could be determined where airplanes were taking the most gun fire. The bullet maps showed clearly defined clusters that had taken a lot of fire from the enemy. Armed with this information, the airforce was eager to reinforce those areas of the planes.
*Wikipedia image: The damaged portions of returning planes show locations where they can sustain damage and still return home; those hit in other places do not survive. (Image shows hypothetical data.)
*Wikipedia image: The damaged portions of returning planes show locations where they can sustain damage and still return home; those hit in other places do not survive. (Image shows hypothetical data.)
BUT the airforce was NOT instructed to reinforce the most heavily damaged areas.
Why? The map overlooks one important point.
The data only represented planes that could sustain damage and return home – meaning the bulletted-areas were strong enough to take gunfire and continue to fly. The areas without bullet holes represented the vulnerable areas. Planes hit there (engines and cockpit) went down. Had this error in interpretting the data not been brought to light, the airforce would have reinforced areas that didn’t need it.
The planes demonstrate a mindset known as Survivorship Bias. (focusing on only the successes)
We all experience Survivorship Bias and the key is to recognize it.
I’ll be the first to demonstrate. I suffer from food sensitivity issues. As soon as I hear about how somebody overcame food issues by eating cardamom, or doing yoga, I’m the first to try it. Not surprisingly adding one single thing doesn’t work. Just because someone, or a few people, had success with something, we all like to jump on board and think that applies to us. If we just drop out of college, like Steve Jobs, we will create the next Apple, etc.
Survivorship Bias creeps into the business world a lot. Looking only at successes (or failures) leads to bad decision making.
Here are some scenarios for you, the magic dealer or magician, to be aware of:
Website Redesign
If you’re considering upgrading your website think about what’s working but also be aware of what isn’t working. You may not even know. When I do a review of a client’s website I often run into survivorship bias. I hear this all too often: “Our customers really love our website; we haven’t had any complaints.”
But what about all of the website visitors who didn’t hang around long enough to turn into a customer? Or all of the customers who are too nice to say a negative word? Don’t rely on satisfied customers to give you the real story.
A/B Testing
You’ve probably heard of A/B testing in which two different marketing ads or emails are sent out to compare which is more effective. Maybe you’re testing the promotional text of a new magic trick, a Call-To-Action button placement or the effectiveness of your images. Either way, you are looking for the option that performed better.
I’ve seen plenty of website owners conduct A/B testing and make bad decisions because they are relying on only a small number of data points, or not taking seasonality into account.
Customer Feedback
Positive feedback from our customers is always wonderful to receive but don’t become complacent and assume every customer is satisfied. Seek out problem areas in your business before your customers complain because by then it might be too late. Your customers may have already moved on.
Check out these tips on how to create happy and loyal customers.
Heatmaps
A heatmap is a tool/service that can be added to your website to show where visitors are clicking on a page. When interpretting a heatmap you only see the clicks; you don’t know what the visitor is actually reading or thinking.
Case in point, one website owner rearranged his home page based on heatmap results. The Call-To-Action (CTA) was moved up higher on the page so visitors would be more likely to see it and click on it. It was relocated to an area of the page that received the majority of clicks.
However, while the new CTA location resulted in more clicks, the end conversion rate fell because visitors who clicked on the button were less qualified/interested. It turns out that the visitors who scrolled lower on the page, read more and were more likely to complete the purchase. The “winning” location on the home page was the only data considered in the decision and resulted in less revenue.
Similar to A/B testing you need to consider other factors and have enough data to make a sound decision.
It’s so easy to be swayed by survivorship bias, which equates to reading the data points we have and calling them truths.
We’re all guilty of this human bias, but once you become aware you can operate from a more enlightened mind-set and make informed decisions.
When you partner with Premium Magic Network, we help you grow your business the right way with the right information. We’re into magic and will never steer you wrong.
How can we support you? Contact us today for a free 15 minute consultation. We’ll give you 3 things you can do today to improve your business.