Why traction is overrated: retention & engagement hacking and product analytics

Why traction is overrated: retention & engagement hacking and product analytics

The Delta Perspective

Nuno Gonçalves Pedro - Partner
Daniele Pe - Principal
Aditi Asthana - Senior Consultant

We live in incredible times: the digital revolution, and in particular the “softwarisation” of everything, has brought many changes to our world. Whether we are users, consumers or workers, new enterprise services and mobile & TV applications have radically changed our end-user experiences.

It is at times almost overwhelming to think how communication has completely transformed itself in the last five years. For example, this has changed from classic person-to-person (voice, SMS) to inherently social (Facebook, Snapchat), from text/voice based to content based (Musical.ly, YouNow), and from geo-agnostic to location based (YikYak, Tinder). In addition, even the point-to-point transportation industry has shifted with the advent of companies like Uber, ZipCar, Postmates and alike.

This revolution has however, not only led to the creation of amazing new products, but it has also transformed the way those products are created. This can be seen when it comes to the move of software development practices from the previously prevalent waterfall model to what we now often call agile methodologies1. This has created both a significant shift in the software development approach, and a philosophical shift in product and marketing approaches.

This article aims to cover this last point, looking specifically at how a data driven, analytical approach is revolutionising companies’ product and marketing approaches, and specifically focusing on introducing a taxonomy that encompasses the core underlying activities that drive fundamental and sustainable growth.

What do we define as Agile?

Agile development is heralded more as a movement rather than a methodology. A way to address some of the more glaring issues with sequential development, Agile calls for a change in mindset that implies iterative, incremental software development, empirical feedback at every stage of development, as well as a collaborative approach, tackling coding problems in modules to be built in parallel. As there has been much discussion about whether agile product development is alive or dead, or has been good or bad, for the purposes of this article, agile software development encompasses several methodologies of rapid product development, with iteration based on stories / features / use-cases to be implemented and prioritised as viewed by clients, end-users and their proxies in the organisation – e.g. product managers, marketing managers, etc.

This is generally recognised as a step improvement vs. the previous waterfall model, due to the intrinsic benefits it brings (faster time to market and higher user centricity among others). This can be seen in the number of companies adopting this methodology (see exhibit 1). That said, it is also recognised that one size does not fit all and that, for example, key characteristics of the waterfall model, such as careful architecture design and planning, are well desired.

Exhibit 1: Agile methodologies uptake

Why does it matter?

Beyond the approach to software development, Agile has led to a dramatic shift in other parts of the organisation, in particular around product and marketing. On the product side, the most recognisable shift lies in the interactions between product management, engineering and design, which become closer, collaborative, even – ideally – symbiotic. In terms of marketing, there has been an even more profound shift, moving from a marketing based on basic quantitative information (such as market surveys, basic analytics on usage patterns) in addition to a lot of qualitative views, (such as focus groups, instinct, etc), to a marketing that is intrinsically more and more quantitative in nature, even scientific at times. One can say marketing is finally becoming what it was always supposed to be - the core fact-based bridge between markets and product.

Underlying all this is what many refer to as data science, analytics and other nomenclature. As data science is more than this, we will call this “Product Analytics” or elaborating on it, the “ability to make data-driven decisions when they refer to product and/or marketing”.

This is a significant change for product development companies, but more importantly to any company that has IT development. Without any analytics infrastructure and without any view on what is going on in the user base, companies are faced with the equivalent of sailing a boat without any instruments. Achieving product-market fit - even for internal IT tools - becomes challenging, because there is no way to know which parts of the product are not working and which parts are creating barriers that users are simply not overcoming. If the products fail, the root cause for the failure is unclear. If they are hugely successful (millions or tens of millions of downloads or page views), yet it is not understood why, the loss of users at a later point in time will come as a shock and will necessarily lead to numerous questions, rather than focus on what things to specifically test. If the product and marketing team starts asking questions like, “maybe it was those little changes we made to our UX last week?”, “Maybe it is the redesign?”, “Maybe it is that new feature we just launched that confused users?”, then yes, there is something wrong.

Product Analytics brings a new perspective and new capabilities to product managers and marketers, who are finally able to deeply understand customer preferences, interactions with the products and product performance, and make decisions based on that.

A Product Analytics taxonomy

Many associate Product Analytics with Growth Hacking, but the reality is that it goes well beyond that. Growth Hacking is one of the core components of Product Analytics, but in our view, it focuses more around the ability to generate traction and less around the ability to retain, engage and monetise users. The concepts sometimes get mixed, but for the purpose of this article, we will try to separate them as clearly as we can.

Elaborated below is a taxonomy, which we believe provides a more end-to-end view of Product Analytics and is applicable for all developers from small start-ups to growth companies to large corporations:

Analytics operations

To make data-driven decisions when they refer to product and/or marketing, one needs to build up the required analytical capabilities and systems. Analytics form the basis of several critical decisions right across the product management / marketing cycle: for instance, segmenting the users as per their needs and behaviours, measuring usage performance in terms of visits/downloads/sessions/etc., determining effectiveness of marketing channels and identifying product bugs, amongst others.

To successfully manage Product Analytics, it is important to predetermine what needs to be measured2 (what information would be required, what KPIs should be tracked), as well as identify the right type of analytical companies that can help build these capabilities and systems. There is a large variety of companies that can be leveraged for this purpose, which are more or less specialised (see exhibit 2). While this framework still clearly overlaps, we believe it provides a relatively exhaustive view on types of activities and relevant players.

Exhibit 2: Analytics providers landscape

The analytics operations set-up process is sometimes iterative and, over time, it should not be surprising that requirements will change and there will be a need for new tools, or at times, even switch tools. While this is painful, it is part of the normal evolution when trying to be more and more fact-based in the analysis. Along the way, companies realise that new metrics, new types of funnels and new types of cohorts are just what is needed.

Rather than waiting for the evolving analytical needs to stabilise, companies should embrace the process right from the beginning. If users’ activity is not measured as soon as possible, valuable historic data may be lost. There is always a lag of information from when one decides to measure something, to actually measuring, to finally deriving changes to product and marketing. The message is simple: start measuring early what is possible and over time gain more clarity on targets and needs to create pertinent data sets.

Activities in this area include: tool selection, custom development / integration, instrumentalisation, measurement, reporting, user interviews, user surveys.


This is the function in which companies define a segmentation of their user base and target segments. It could initially be linked to other segmentations available in the specific sub-industry, e.g. in a gaming studio, this could mean killer whales, whales, dolphins, minnows and non-players. As shown in the previous example, a skew towards usage patterns is normally a good one to start from, but adding a demographic overlay to a usage-based segmentation can be valuable. That said, having just a usage-based segmentation may be enough, but having a demographic only segmentation is, in our experience, rarely enough. We would recommend that patterns of use, intents and preferences are normally more important indicators of what change needs to be made to products and marketing, than gender, age and other demographic indicators. Fitbit for instance, segments its target market based on fitness levels and intensity of usage, dividing its fitness users into: “Everyday users”, “Active users” and “Performance users”. This is used to create devices with characteristics and price points appealing to each of these segments (basic devices like Zip and One to Everyday users, Charge HR and Blaze to Active and Surge to Performance users), as well as building a path to upgrade fitness level and hence encouraging upgrades to the device3.

The segmentation can go beyond usage and demographic attributes to include geolocation metrics and other behavioural preferences. The data inputs would vary by the type of product as well as stage of development, market dynamics and competitor actions.

It is important to clarify that segmentations are not static, but rather change over time: better understanding of the target users and better analytical capabilities should drive more granularity to achieve micro-segmentation, and, who knows, maybe even the holy grail of “segments of one”. In our experience, when segmentation is not treated dynamically, companies at some point tend to lose track of who the users are and what they do with the service. Bayt.com, a career website operating out of the Middle East, segments its customers based on a combination of static data from user profiles and dynamic behavioural data to cater to the uniqueness of their customers, while still clustering them in categories based on commonalities. As a result, they create mini “marketing universes”, to understand road maps for almost every alternate user story across all touchpoints - web, mobile and email.

Activities in this area will include: user-type definition, usage-driven segmentation, demographic segmentation, brand strategy & definition, value profiling/customer lifetime value analyses, brand management, and marketing ROI analyses.

Growth hacking

Product-focused companies live or die by their ability to gain users. Gaining growth (measured as number of downloads, installs, registered users) early in the life-cycle is paramount for developers and can be achieved organically or via paid, marketing efforts. While paid acquisition can be highly effective, we believe it is unadvisable to resort to them too early in the product lifecycle due to potential “leaky funnel” issues that early stage companies tend to experience. Paid acquisitions might be successful in attracting first time users, but as retention capabilities are still under-developed, too many of them could fall out of the funnel, resulting in wasted marketing dollars. Thus, we would recommend companies to focus on organic growth until a (more) stable, robust funnel is achieved.

There are several ways of achieving organic growth and virality is often a good point to start. Dropbox used referrals as a growth hacking mechanism, incentivising users to recruit additional members by offering free space to both the recommender and the new joiner. Through this approach, Dropbox managed to increase signups by 60%4 while managing to keep SAC spending at far less than a Google AdWords buy5. Another indirect recommendation method, heavily employed by Facebook, is the embedding of widgets in relevant external blogs and websites to redirect traffic to the product. Leveraging this approach, Facebook was able to gain over 300 million users in its first three years after opening the platform beyond selected colleges6. Other creative organic growth hacking techniques such as targeting influencers to get on board first, has helped Facebook reach close to two billion users today with a minimal spending on advertising. Another interesting example is SoundGecko, a platform created to help convert articles/blogs into audio files to listen on the go instead of reading. To improve visibility of their product, SoundGecko created a radio station for the immensely popular news page, HackerNews, and submitted it as a post on HackerNews itself creating a tremendous response from HackerNews users, who are primarily entrepreneurs and innovators – the ideal demographic for SoundGecko. As such, they were then able to double their base within one month of the post.

While sometimes organic approaches suffice in achieving tremendous growth, like in the case of Facebook, in most cases companies do need to invest in marketing to get the word out about a product. However, new market entrants typically have extremely constrained resources with little budget to dedicate to marketing and the traditional marketing channels are dominated by the more entrenched players, which provide little to no visibility for young companies. Players consequently need to resort to more innovative, cost-effective means of marketing. LinkedIn’s early success partially came from investing in SEO where they made LinkedIn profiles publicly searchable – a solution arrived at after testing several different marketing means from direct mailing to invite-only joiners. As people began to discover that a LinkedIn profile was one of the first search results for any person’s name, more began to sign up helping LinkedIn grow from two million to 200 million users.

There are a variety of channels for the dissemination of digital product marketing. Growth hackers should spend a significant amount of their time in running pilots on the different channels to determine the best approach. Conceptually, preferred channels will be those with the best access to the target segment. Hence whilst knowing the potential end-user and where they spend their time is the precondition, the channel to reach them and convert them might not be so obvious. There is no single solution. The method would vary depending on the type of market, the nature of the product and the segment. Sometimes this may vary from company to company in the same industry, hence why testing different channels is unavoidable.

Activities in this area include: paid acquisition, low-cost/rev share distribution channel partnerships/business development, SEO, App Store SEO, cross-promotion.

Retention and engagement hacking

As presented before, we believe this is the key to sustainable growth and maybe one of the most misunderstood areas. Logic dictates that once critical mass has been achieved in terms of acquisition and a significant base has been built up, the natural next phase is engagement and retention. However, as mentioned before, we believe that engagement and retention efforts need to be the focus right from the initial phases. YouTube is an example of how these two elements can be leveraged to create a virtuous circle. YouTube moved from focusing on driving growth through virality (embedding code to share a video just viewed on another forum), to focusing on driving customer engagement by suggesting other videos to the user based on their viewing history, later asking the user to share all of those. As such, this helps to create a “viral loop” as the user “passes on” the suggested videos to their own network.

In the long run, engagement and retention have a better return than acquisition as attracting a new user can cost five times more than keeping an existing one, while existing customers spend 31% more than new ones7.

Exhibit 3: Benefits of retention vs. acquisition

Engagement and retention are slightly different animals: while engagement means extending the user interaction time and actions with the product (typically measured as time per session, number of activities per user, etc.), retention means ensuring the user returns to the product over time (typically measured through MAU, weekly unique visitors, etc.). However, the way to drive both engagement and retention is the same: keep improving the product, so it continuously delights end-users and drives them to come back and use it more and more. Snapchat is an example of this: by introducing the functionality that content needs to be immediately consumed or it would disappear, it was able to capture the full attention of the user, distancing itself from other messaging /social apps in terms of customer engagement (25-30 mins average daily usage time, which is higher than Facebook Messenger and WhatsApp8)

However, achieving this stage is not obvious. The way to do this - a method where agility plays a big role - is to constantly adjust the product on the basis of a deep analytical understanding of the end-user flow and leakage points. Many product developers opt for multivariate analysis (or A/B testing, as it has been bastardised), with different combinations of variables to be tested such as pricing and aesthetics, to determine the user experience, interface and functionalities that work best for customers. There are several case studies available that help illustrate this: Twitter, for instance, on realising that it was unable to retain its growing base, conducted in-depth testing on both the user experience and the interface, and found that the users that selected five-10 accounts to follow in their first day on Twitter would be much more likely to stay longer. Hence it made changes in the product to help users immediately connect with friends, which quickly saw a dramatic rise in success.

Even this step is generally not enough to achieve high engagement and retention metrics. In fact, it is vital to continuously re-engage users through product notifications. The typical channels leveraged here are the OS notification tray (the internal app notification tray), as well as emails and SMS when those details are registered by users. Considering the importance of engagement and retention, the trade-off of making end-users register themselves upon on-boarding (or pushing them to do so forcefully) versus the low-bar of “just use it”, is one of the more critical elements to test and decide upon. Viral game, Candy Crush, is a clear leader when it comes to retention notifications, elegantly combining social channels that require registration (Facebook notifications from friends) with automatic push notifications (informing users who haven’t played in a while of new features or freebies to bring them back on board) as well as in-game messages for added engagement (positive feedback that keeps users playing, statistics on the loading screen that get users more intrigued). It is no surprise that the game developer has been able to maintain more than 400 million MAUs for the last two years.

Activities in this area include: analytics/data-driven product modification/evolution, multivariate (A/B etc.) testing, multi-channel re-engagement (email, SMS, other messaging, OS notification tray, in-app notification tray, among others).

Pricing & Monetisation

This is the holy grail of modern day digital businesses. While most businesses are caught up in trying to solve the end-user problem, growing and engaging the base, as well as creating a business model that can assure recurrent revenues, seems, at times a near unsurmountable problem. A survey by Vision Mobile showed that top 2% of apps are responsible for 54% of revenues9, indicating that most fail in this endeavour.

Eager to generate revenues, but afraid to charge customers, many companies opt for indirect monetisation, leveraging the user base, either by packaging user data for analytics or pushing advertising. The latter is highly common: for example, social networks and news websites are generally based on this model, and if/when a certain scale is achieved, it can be a veritable gold mine: Facebook, for instance, has been able to build a $370bn+ empire based almost entirely on advertising10.

At the other end of the spectrum, monetisation translates directly to having a paid product, i.e. a subscription, or a cost per transaction. This makes sense when there are clear costs that need to be sustained to provide the service and is typically used by video on demand (VOD) players like Netflix and Hulu, education apps (content cost) as well as service providers like Uber (driver salary), or business apps (not necessarily a direct cost, but less price sensitivity). The challenge here is around pricing the product, particularly when the solution is innovative and peerless. In this scenario, the factor that comes into play is the customers’ willingness to pay, determined by testing several price points. Multivariate testing is once again relevant, but there are other tools that can be used for this purpose too, from advanced pricing software to direct customer feedback sourced via email.

Most companies take a more hybrid approach to monetisation. One model that has become a household name is freemium, where the base services are free, thus “hooking” the users, but advanced features require a subscription payment. Music streaming services are largely built on this model.

Others, take a similar free base service approach but instead monetise through in-app purchases. Many navigation and travel apps like Worldmate have this feature built-in, with users paying extra for specific functionalities. Gaming companies are particularly fond of this model in order to stimulate the sale of virtual items at carefully chosen points in time. Freemium can boost user engagement if it is used wisely. But freemium can also be tricky. The key is being able to crack the free to paid conversion rate, which requires sophisticated analytical capabilities. An evolution of this model is paydium, where users are charged a nominal fee for the product with extra charges for purchasing additional features. This is becoming the new staple for gaming apps, particularly those running on gaming consoles.

Exhibit 4: Pricing models proliferation among app categories

The type of product offered tends to influence the type of monetisation model. As shown in the examples above, product categories tend to have similar pricing approaches which are accepted by the user. Sticking to that is definitely a safe choice, but innovating can actually be a source of differentiation and competitive advantage. FreedomPop is an example of a company that adopted a differentiated pricing approach to a very standardised industry (mobile connectivity), introducing a freemium model instead of the traditional subscription / pay per usage approach which mobile operators adopt. This meant that FreedomPop was able to generate significant market traction in the USA with over one million users, with almost half of them paying11.

When it comes to choosing the pricing strategy, our suggestion is to pick the pricing model as early as possible. It is important however, not to force-fit the product into a particular strategy that is thought to be the best way to make money. Instead, understand what the market is doing, which model the end-users seems to be more comfortable with, and what they are willing to pay for. While the “one day we will monetise” strategy seems to still be prevalent in consumer products, very few are able to capitalise on it much later on. Rather than testing pricing before the launch, we suggest that it should be tested early on to ensure the pricing model is fair to the market.

Activities in this area include: analytics/data-driven product & pricing modification, multivariate testing, benchmarking/competitive intelligence.

So how does this differ from core Product Management or Product Marketing?

While Product Management, Product Marketing and Product Analytics are strictly interrelated, there are some intrinsic differences between the three.

Product Management is about defining the “why”, “what” and “when” of the product that the engineering team will build. Product managers are responsible for strategy, roadmap, feature and user-flow definition for that product, including forecasting, and potentially revenue generation.

Product Marketing, on the other hand, is about promoting and selling that product to an audience. Product Marketing, as opposed to Product Management, deals with more outbound marketing or customer-facing tasks (in the older sense of the phrase).

Product Analytics can be seen as the enabler layer that makes the product and marketing decision making process quantitative, even “scientific” rather than qualitative. It helps decision makers take fact- based decisions across the product / marketing cycle.


As the first truly global market for digital goods, the digital economy offers important lessons for how digital markets are evolving. Despite the lower barriers to entry and increased accessibility afforded by digital production and distribution, gaining traction with customers beyond initial acquisition and driving meaningful user engagement and retention, are well-established challenges that can make or break the product-focused or software-enabled business. Long-term engagement means thinking about the community of users that exists in and around the product, and then driving product development and marketing efforts around it. In order to do that, Product Analytics capabilities need to be built, to allow product / marketing teams to take data-driven decisions that can steer the future of the company in the right direction, moving from growth to engagement to monetisation. Last, but definitely not least, while growth hacking is vital, more attention needs to be given to deep and fact-based retention and engagement hacking activities. Without it, one might never know what went wrong… or right for that matter.

1 This was called the iterative model in software development, but the authors recognise that XP – Extreme Programming – and other flavours have gone well beyond that model One of the authors has a particular “pet peeve” around the use of the expression “Big Data” and believes the need for “Right Data” has been grossly underestimated
2 One of the authors has a particular “pet peeve” around the use of the expression “Big Data” and believes the need for “Right Data” has been grossly underestimated
3 Source: Fitbit SEC filing available here
4 Source; Drew Houston, Start-up Lessons Learned
5 Source: Zach Bulygo, KISSmetric blo
6 While Facebook launched in 2004, it only opened to the public (everybody above 13 with an email address in September 2006). In Q3 2009 it had reached 305 million users
7 Source: Leadplum study
8 Source: Bloomberg, USA Today, Statista, Business Insider and Graphiq
9 Source: Vision Mobile
10 Source: Yahoo Finance (market cap)
11 Source: FreedomPop