Leonardo Matos, Fullstack Engineer
Of all the tools and techniques at your disposal to help you improve your product, my favorite is, by far, the concept of A/B testing.
We all want to build the perfect technology product, things that impact people, and high quality in all its components, processes, interfaces, designs, etc.
Unfortunately, there is no way to do this without user feedback effectively. Some even advocate that there is no limit to the benefit provided by user feedback; the more user feedback your product receives, the better the product will be.
One of the most popular ways to capture user feedback is through a well-tailored survey. While this is a viable option, maintaining a survey can be tedious, and some users don’t have the time or interest to fill out surveys.
Wouldn’t obtaining their feedback by analyzing their behavior and interaction with our product would be better? As opposed to bothering the user with surveys, they might not want to answer at all.
That is precisely what we can accomplish with A/B testing.
As you probably know, A/B testing is an iterative, user-centered, and data-driven method to obtain user feedback that we can use to make decisions regarding our products and services. It is entirely transparent to the user, and they don’t know they are providing this data while they are (aside from mentioning this in your privacy policy and disclaimers).
You are presented with one random variation of a human-facing component from two (or more) options (Version “A”, the control version Version “B,” the variation. Multiple variations can be included)
We can identify the highest-performing variation with proper experiment design and modify our product accordingly. This process can be performed as many times as needed.
A complete framework for an A/B test follows these steps:
- Define a research question
- Refine the question with user interviews
- Formulate a hypothesis, identify appropriate tools, and define test metrics
- Set up and run an experiment
- Collect data and analyze results
- Share results and make decision
Should we put that optional second form in a modal or right next to the first one? A/B test it. It seems users rarely click on our products. Should we show a large image with minimal information or a smaller image with more product information? A/B test it.
If you think about it, A/B testing is pretty much like natural selection, where the more effective version of your product survives.
Even if you take a component of your product to a desirable point of effectiveness, it’s not a bad idea to try the experiment again a year or two later. Trends in product design are ever-changing, so what is very effective to your users today might not be effective several years later.
Also, people’s way of thinking and tech-savvy vary depending on the audience your product is aimed at, so if your audience changes over time, you might want to run more A/B tests.
Since A/B testing requires effort and resources that are not infinite, it should be done in the most important parts of the product or the ones that generate more value. But when used wisely and correctly, A/B testing can help make a great product!
