The most important SaaS Activation KPIs: T2O, T2HW and T2V

Part 2: Key KPIs for Product Managers

Photo by Norbert Braun on Unsplash

Recall a time when you downloaded a new mobile app, perhaps a new budgeting tool to help you make sense of your daily spending. After signing up, you were immediately overwhelmed by a cluttered dashboard, couldn’t find a simple tutorial, and weren’t even sure where to begin.

Not surprisingly, you closed the app and forgot all about it, never opening it again. That initial post-signup phase, where an app should ideally captivate and guide new users, ensuring they see its value and benefits, is known as “activation”. When done right, it can turn a novice user into a regular or even a dedicated one.

In the previous article, we discussed how KPIs can help us measure and drive growth targets in terms of market reach and our ability to retain users and generate revenue. In this part, we will specifically discuss activation KPIs, which can help with all three targets.

Let’s have a look at some of the most important activation KPIs, especially in the context of SaaS products:

Time to Onboard (T2O)

The duration it takes for a user to move from the initial sign-up stage to the point where they’re familiar with and can navigate the most relevant functionalities of the platform without significant challenges.

For example, T2O could be the time it takes for the user to go through all the steps of an onboarding walkthrough guide that pops up as soon as you sign up for an app. This kind of walkthrough can significantly help users become “Active” as efficiently as possible.

Walkthroughs like these are implemented in many ways, with some allowing the user to browse through the most essential parts of the app quickly. In contrast, others guide the user through a more hands-on, step-by-step process. Those walkthroughs that wait for the user to take a real action app before allowing them to move to the next step have an advantage because they ensure that users get hands-on experience using the app instead of simply breezing through it.

Time to Hello World (T2HW)

This KPI measures the duration it takes for a new user to achieve their first successful output or interaction. It’s more common with technical products, but in principle, it can still be helpful regardless. In software development, ‘Hello World’ typically refers to the time it takes to get the most basic version of something running, i.e. a proof of concept.

Time to Value (T2V)

This is the duration between a user’s initial interaction with the product and the moment they first experience or perceive significant value from it. This “value” is subjective and might vary based on the product’s purpose/strategy and the user’s goals.

The ideal way for these KPIs to be tracked is programmatically based on real usage or behaviour in our app. For example, there must be specific events we can name inside our application that indicate the user may be receiving value from our product.

For example, for a CRM platform, The finish line for measuring Time To Value might be the moment a customer of the CRM has added a lead or sent out a campaign.

You are not alone if you are already wondering how the three above KPIs differ in principle. Product teams tend to interpret these slightly differently. As with any useful KPI, the first step is to ensure that everyone shares the same understanding of each measure and why we care about it. Let’s take a closer look:

Time To Onboard is all about getting the user familiar with the platform and its features quickly and effectively. In many apps, however, this doesn’t necessarily entail that the user has managed to find the product useful within that time. Typical onboarding flows tend to focus on navigating to and around high-level features. Still, in practice, users might need a lot more time and effort to extract value from the product they have been onboarded.

Similarly, Time To Hello World can show us how fast we can get to a proof of concept, but the thing about PoCs is they might be far from becoming practically “Valuable” to the user. In heavy, technical B2B products, running a PoC might not even be possible within the product as they might involve setting up and taking actions outside the main app — such as calling an API or embedding software and going through documentation.

Why you should measure your customers’ Time to Value

From all three KPIs, Time to Value is “The” Activation KPI all Product Managers should track. In “What Product Managers Really Do”, I discuss how the Product Manager’s job is ultimately to manage the value generated by the product and for the customer. Time to Value gives us an explicit look into how fast our customers can expect to materialise real business value from our product’s most valuable offerings. This distinction makes this KPI more strategic and more critical than both T2O and T2HW.

In most cases in B2B SaaS, value is generated long after the user has been onboarded or completed their initial PoCs. This sounds reasonable in most cases. However, suppose you do intend to pursue growth by extending your product’s and company’s reach to more customers. In that case, maintaining a long T2V will block you from doing so efficiently at scale, no matter how fast your “Time to Onboard” or “Time to Hello World” are.

Now, let’s look at the distinction between TTV and T20 and T2HW in practice. Consider a technical SaaS platform such as a cloud data store or a hosting provider offering a freemium tier. Their PMs have measured a very fast T2O and T2HW. But is T2V equally short? Let’s answer that through a fictional scenario:

DataSky is a fictional new startup in the cloud data storage space. Developers and businesses flocked to their platform, attracted by its high performance and scalability promise.

The Product Management team at DataSky was proactive, ensuring users had a smooth start. Two of the primary KPIs they had been actively improving were:

1. Time to Onboard (T2O): The duration from when a user signs up to the time they successfully set up their environment. Thanks to the intuitive interface and clear step-by-step guidelines, T2O was very short.
2. Time to Hello World (T2HW): This captured a user’s time to achieve their first meaningful and successful task, like creating new rows in an SQL table in their environment. Comprehensive documentation and community support ensured this was consistently fast.

However, a deeper dive into the free trial’s user engagement revealed a puzzling trend. While users were quickly onboarded and started with basic operations, many seemed to plateau after these initial tasks. They weren’t making full use of the platform’s advanced features.

When the Product team defined and tracked a new KPI, Time to Value (T2V), they shed light on the mystery. T2V measured how long it took users to truly harness the platform’s extensive capabilities and see significant improvements in their projects or workflows. The PMs had to select broader features and events that indicated T2V compared to T2O and T2HW. When they did, they discovered that, despite the easy onboarding and initial setup, T2V was 9 times slower.

Upon further investigation, it became clear that while DataSky’s advanced features were powerful, they were also intricate and often required a steeper learning curve.

To address this, the product teams created a series of ‘Advanced User Workshops’ and ‘Deep Dive’ webinars. At that point, only a few users had proactively discovered this more specialised tutorial, most of which had been prompted by DataSky’s sales and account executives. As the number of new users signing up to the product grew, many still remained oblivious or seemed reluctant to try this material. While TTV could now be measured, it was still quite long. It needed to accommodate the time it took for customers to join scheduled workshops, which meant that, for the most part, they were waiting around for one or still needed to put the effort to rally people together to attend them. Unsurprisingly, engagement rates took a further hit.

Intensifying their efforts, the product teams at DataSky decided to simplify their tutorials even more into bite-sized pieces that users could go through as soon as they signed up and in their own time. The tutorials would become part of the main in-app onboarding walkthrough.

They also introduced an on-demand ‘Expert Connect’ feature where users could chat in-app with platform specialists to understand the intricate features and better solve any pressing concerns.

They even created a downloadable sample backend code fully integrated with DataSky’s API that users could download as the final onboarding step and embed into their backend. They incorporated the moment a new user called their API from their own app as the new cutoff time for measuring Time to Hello World.

After all these, a new side effect was created: T2O increased. However, TTV and Engagement rates improved heavily.

As DataSky worked on shortening the T2V by making its advanced capabilities more accessible and proactively encouraged users to try straight away despite their perceived difficulty, they saw a notable increase in platform activation metrics. They also realised that their T2O, T2HW and T2V measurements were roughly at the same level. Not surprisingly, conversions to more premium plans started to rise, too.

The takeaway?

• An easy start is essential, but for users to deeply engage with a platform, they must experience its most valuable parts swiftly and effortlessly, even if those parts are the hardest to understand.
• While they might overlap, always start by measuring all 3 KPIs (T2O, T2V and T2HW), as they can show you where your gaps might be and how to expand your concepts of what it means to fully onboard or complete a Hello World at your company or product.
• If you think something is “too technical” or “too complex” to be part of the onboarding, you might never have the chance to simplify it enough. If you don’t, a competitor might.
• The hardest problem will be defining what valuable is in the first place. Value has to converge for both your users and your organisation.