Thursday, September 10, 2015

Use Split-Tests to Optimize Product


When developing and improving a product, start-ups have to distinguish between value and waste: they must find out which features are valuable for their customers and which aren’t.

Valuable features are those that help the company attract more customers or increase its revenue.

Features that don’t do either are wasteful – even if the founders or engineers think they’re the greatest thing ever.

A clever way of distinguishing between value and waste is split-testing. Whenever you consider adding a feature or changing an existing one, create two versions of your product: one with the new feature and one without it. By testing both versions, you’ll soon see which one is more appealing to customers.

The first companies that used this technique were mail-order businesses. For example, to find out whether a new catalogue layout would increase orders, they printed two versions of it: 50% of their customers got the old design, and 50% got a new one. The catalogues were identical in every other way and the customers were split randomly, so the companies simply had to compare how many orders were placed by each group. These data answered the question of whether the new design was an improvement or not.

In the same spirit, any start-up can test every possible change before actually implementing it. Want to know whether your website works better in red than in blue? Why not create two test versions of it and track customer click-rates for a couple of days?

Any change you wish to make to your product should be tested with this semi-scientific approach before you actually implement it.

No comments: