How Important Are Experiments to App Growth?

In previous posts, we talked about the importance of organic mobile app acquisition. We’ve also taken a deep dive into what technology your Organic Mobile Growth Platform must utilize in order to accurately attribute and measure your app installs.

But amassing data isn’t going to make your app grow. You need to attach meaning to the data.

The secret to discovering meaning from your app acquisition data and unlocking new quality users is to run tests (or growth experiments) and then use the results to optimize your acquisition channels and campaigns.

Applying Optimization and Experimentation Best Practices from the Web

Website optimization, experimentation and testing has its origins in science, in particular, The Scientific Method. The Oxford English Dictionary definition of the Scientific Method is

“It is now commonly represented as ideally comprising some or all of (a) systematic observation, measurement, and experimentation, (b) induction and the formulation of hypotheses, (c) the making of deductions from the hypotheses, (d) the experimental testing of the deductions, and (if necessary) (e) the modification of the hypothesis. “

Just like the scientific method, web optimization starts with measurement, hypothesis formulation, and experimentation. Cookies, tracking and URLS make data collection for website testing and experiments possible.

Using the web optimization playbook, we know we want to measure, develop a hypothesis, validate it and finally optimize based on the results.

Website best practices have given us some great ideas to start testing. For example, you can test copy, offer or CTAs. There are countless examples of optimizing websites online and many of these can apply to optimizing mobile app acquisition and engagement.

This is one of 100 Optimization examples from Kissmetrics

Mobile Analytics and Web Analytics Are Not The Same

While the knowledge and technology for website testing and optimization are plentiful, this is not the case for mobile apps. Website optimization relies on straightforward URLs, cookies and analytics to track users from acquisition source and channel to conversion. Unfortunately, this does not apply to mobile app optimization. The dam in the mobile data stream is the App Stores.

The App Stores are black boxes that prevent or complicate passing of pre-install data to the app. For example, let’s say you are a media site with a lot of articles that are repeatedly shared on social media. On the web you can easily connect the dots from a newly acquired user’s initial click on Facebook to the exact article on your website to the email signup on your site. Furthermore, you can tie all subsequent email open and click rates back to the original acquisition source — in this case, the share on Facebook. This data flow breaks with mobile apps. As a result, mobile app growth managers don’t have the data needed to run intelligent experiments and optimize.

The Great Mobile Shift

It’s no surprise that mobile is the new frontier. Time spent connected to the internet on mobile devices surpassed desktops in 2014. Furthermore, users are choosing to spend more time in native apps than on brands’ mobile websites. The mobile land grab is here and the companies who properly migrate their users to native apps will be successful. Those who don’t will fail.

As of July 2014, the Google Play Store had 1.3 million apps and Apple’s iOS App Store had 1.2 million apps. Needless to say, there is competition for discovery and mindshare. Brands have a leg up in the mobile race. Specifically, you have existing brand loyalists you can convert to app users before they download a different app. But it’s not only about installs; if you don’t create a good app experience, they’ll go elsewhere.

Putting Organic Optimization and Experimentation to Work for You

As mentioned previously, on the web it’s super easy to run experiments; on mobile it’s not. Most companies don’t optimize organic channels at all. Mostly, this is because there have not been good attribution solutions – only ad attribution tools that don’t offer the level of detail needed to collect actionable data for organic growth.

In order to make organic optimization and experimentation work for you, you must use an Organic Mobile Growth Platform, specifically designed to attribute and measure app installs resulting from organic channels and campaigns.

Best Practices for Organic Mobile Optimization and Experimentation

1. Attribute and measure app installs with a platform designed specifically for organic growth. We’re partial to Yozio because it was designed to attribute and measure the full lifecycle of organic growth from source through install to conversion.

2. Identify which campaigns and channels have the highest growth potential to acquire quality users. Examples of channels are SMS, Email, Mobile Web and Social Media. The breadth of your campaigns is limited to your innovation and imagination, but some common campaigns we see are referrals, promotions, SEO driven campaigns, cross-promotion, content and product sharing.

3. Optimize by designing experiments that focus on one aspect of the user lifecycle at a time, such as activation, retention or conversion.

Acquisition example: By using a Platform specifically designed to accurately attribute and measure organic installs, our partners now have insight into what organic channel led to the install of the app. An example of an experiment to drive new user acquisition is to change the default channel for sharing a piece of content. For example, a media and news app might have different options to share a piece of news. A simple experiment is to test if a One-to-many share such as with Social Media attracts more quality users than a one-to-one share, such as email or SMS.

Activation example: Personalization of the activation and onboarding experience has different flavors. Examples include automatically applying a discount code to newly activated shoppers, defer deeplinking the new user to the specific piece of content that led her to install the app or simply creating a custom banner on a mobile website with one CTA that automagically redirects the user to the correct app store. Our customers have seen a 30% increase in activation rates simply by personalizing the onboarding experience by connecting the intent of the install with the first thing the user sees inside the app.

4. Learn and Iterate. The goal of experimentation and optimization is to learn and grow. Sometimes, your hypotheses are wrong. That’s okay. Learn from your mistakes and test new hypotheses. When your experiments validate your hypothesis, then try to apply your hypothesis and experiments to other aspects of the user lifecycle. Once you see diminishing returns on your experiments, chose another optimization path and rinse and repeat.

Times Are a Changin’

It’s not only the times that are changing. Your users are shifting their preferences. Your app is evolving. Devices are multiplying and marketing channels are developing. So your experimentation and optimization efforts are never static either. You need to continuously test and iterate to continue to grow your app user base and drive engagement and conversions.

(Image from 312analytics )

How Important Are Experiments to App Growth?