Within the tech cycles, the term growth hacking is becoming a buzz. But for you and me, it means growing something, be it a product and user base, with an end goal of achieving rapid growth. Sure, every organization would love to grow their revenue and user base. The faster the better.
But, most organizations get it wrong because they base their experiments on speculations and opinions rather than hard evidence. If that sound familiar to you, continue reading.
Many leading organizations such as Twitter, Uber, Walmart, Amazon, and Netflix leverage on data driven experiments to improve customer experience and run lean marketing systems. For instance, Twitter embraces the 1 Percent Experiment philosophy. Twitter allows its product developers, marketers, engineers, and designers to test new innovations on 1 percent of user sessions. The 1 percent process goes like this.
- Build a hypothesis. What is the specific idea being tested? What do you consider as successful?
- Test success metrics. These are the specific metrics that will help you define success.
- Test hypothesis. Build a working implementation to test ideas and measure results.
- Learn. Brace yourself for unexpected outcomes. Your experiments may lead to no major change in metrics. You may need to iterate experiment many times.
- Ship. If you are lucky, present your ideas to the organization to release to them to the world.
- Iterate the process. Create new ideas while taking into account lessons learned.
The data for decision making is derived from a series of experiments that test the hypothesis against a sample of the bigger population. In most cases, the experiment will try to find answers for the following questions.
Can you try the offering?
Would you really use the offering?
Would you pay for the offering if there is an alternative?
The result of the above will lead to any of these conclusions.
The hypothesis is correct, so go ahead with the idea.
There is positive momentum, but it not enough to make a conclusion yet. You should continue testing, and if possible, increase the sample.
• The experiment disproved the hypothesis, so you should abandon the idea altogether.
Most experiments usually end up in the second category. Often, this leaves many organizations in limbo. Most of them are always searching for that one thing (hack) that will speed up growth. But what they don’t know is that the process is often repetitive, and most ideas won’t work. You are likely to fail more while you figure what works.
What does it mean to be data-driven? It is easy for an organization to claim to be data-driven. But in reality, data-driven science goes beyond a buzzword. Being data-driven is all about building abilities, tools, and most important, developing a culture that relies on data and testing.
Here are what should be the foundation of your experiments.
- Collect data Data is the main ingredient in data-driven experiments, so organizations need to collect the right data. It has to be accurate, unbiased, clean, timely, and perhaps most crucial is trustworthy. Unfortunately, data is always dirtier than you expect. And, as you know, any subtle biases can drastically sway your conclusion. Massaging and cleaning data is not enjoyable either.
- Make data accessible Even when you have trustworthy data, you are still limited on what you can achieve with it. You are not data-driven yet.
Data need to be:- Joinable: Data should be available in a format that can combine with other enterprise data. To make this a reality, you need tools such as relational databases, Hadoop, or NoSQL stores.
- Shareable: Let’s look at this scenario. A patient is admitted to a health facility, receive treatment, then released with instruction to go for check-ups and additional treatment in an outpatient clinic. If the clinic and the hospital don’t share data, the patient will surely receive substandard services. The clinic will have a hard time finding out the reason for admission, where he was admitted, and what treatment he received.
- Queryable: Any successful analysis and reporting require aggregation, grouping, and filtering of data to reduce the huge volume of data into subsets that can generate insights.
- Reporting Now we have accessible data, is it sufficient to run data driven experiments? No, it is not. There have to be humans in the loop to extract the right information from the data. Besides the above requirements, here are more takeaways that will help as you conduct your experiments.
- Set up control case. If you are carrying out an A/B test, you should be able to know what is happening if you had chosen option A over B or C. This way, you can have a solid ground to disprove or support your hypothesis.
- Plan for the experiment in advance. Plan for the exact data you will be working with after the experiment and how this data will help you to support or disprove your hypothesis. Needless to say, but you should double check your plan with scientific models. You will catch any flaws in the experiment before it starts.
- Start with simple experiments. Before embarking on more complex strategic questions, hone your skills with simpler experiments. This approach will let you build a foundation of experience when you expand your tests.