CONTENT
  thinking about it, please hold on..

Totango core metrics: Engagement score, usage frequency, time spent, active users and license utilization

A core function of Totango is calculating key metrics on product-usage, measuring the level of engagement end-users have with your web-application.

Each metric measure a different dimension of product engagement and, as a whole, paint a simple yet powerful picture of user engagement with the application. 

Totango core metrics are computed automatically, using data-points collected by the Totango Javascript collector. 

Engagement score

Engagement Score is a relative score attached to each Account in Totango. Its a number between 0 - 100, measuring the intensity of product usage by the account end-users. 

  • 0 means there is no engagement - the users are not logging in and using the web-app
  • 1 - 10 means users from this account are at the bottom 10% in terms of usage
  •  …
  • 99 means users from this account are the top 1% most active

Engagement score is calculated daily and is measured on a 14 full-day sliding window of usage. The average amount of user-actions registered daily  on the account over this period is counted and segmented according to account types to produce a the relative score. Note that any account with less than 1 user-action/day on average will be granted an engagement score of 0 in any case

Account-type segmentation
Because engagement score is a relative metric, it is important to measure it separately for different segments of accounts. The level of use of an Personal account, for example, should not be compared to a large Enterprise implementation  with many dozens of users.

Totango does that segmentation automatically, by looking at each account’s Status attribute, and comparing it to others with the same value.

For example, if Totango is setup with these Status values:

  • 30-day Trial
  • Freemium
  • Pro Plan
  • Enterprise Plan

The Engagement score will be normalized separately for Trials, Freemium, Pro Plan and Enterprise Plan customers.

So an Engagement of 100 for a Free Trial user means they are among the most engaged trials. A similar score for a Pro Plan compares the engagement to other customers on this plan. Accounts in different status are not compared amongst each other.

Note that the classification of a Status to a status type (Free, Paying or Cancelled) does not have bearing on engagement score calculation. So in the above example, Freemium and 30-day Trials would typically be classified as "Free status type" where as the Pro/Enterprise plans as "Paying". However, engagement score is still calculated separately for  each status value (30-day trial, Freemium, Pro & Enterprise) 

Usage frequency

Usage Frequency is an at-a-glance metric that helps you determine how often an account is used. 

Totango looks at the past 14days of usage of an account and ranks their Usage Frequency into one of four possible values:

  • Daily (9+ days from the last 14): Account is used almost every day
  • Weekly (5 - 8 days): Account is in use once or twice every week 
  • Occasionally (2 - 4 days): Account is being used, but not on a regular basis
  • Inactive(1 or less) : The account is inactive, with either no or one day of usage in the last 14 days.

Time spent

Time spent is an approximation of the overall time that the users in the account have spent online in the application.

Totango automatically filters idle time out of the overall time spent metric, only counting minutes in which there was end-user activity. This means that time in which the application was opened on a browser tab but registered no user interaction is not counted in the time estimation. The chart below demonstrates this concept. 

Note that because Totango's Time spent metric estimates actual engagement time, it should not be confused with the total elapsed time from login to logout as in most cases that value will not match. 

Overall time spent is measured in minutes and is available in a daily, 3-day, 7-day, 14-day, 30-day & 90-day time-frame. 

 

Active users

Active users counts the total number of active users in an account. A user is considered "active" if they had at least 1 activity registered during the time period. 

The metric is available in a daily, 3-day, 7-day, 14-day, 30-day & 90-day time frame. 

License utilization

License utilization measures the rate between the number of active-users in an account and the total number of licenses purchased. It's a great metric for applications that have a seat-based pricing model or can set an expected license level for different customers.

For example, consider two customers: both have 10 active users in the past month. The first has purchased 100 licenses (seats), the second 12. Obviously the former (10% license utilization) is well below their expected usage, where the later (83% license utilization) is almost maxing out of their licenses.  

License utilization is available in a daily, 3-day, 7-day, 14-day, 30-day & 90-day time-frame. It is computed as the rate between the Active Users and the Licenses attribute associated with the account. 

 

Have more questions? Submit a request

Comments

Follow
  • Avatar
    Pavel Hardak

    Excellent explanation of Totango core metrics. What is missing - how Engagement score and Usage frequency are calculated for recently created (less than 14 days) accounts?

  • Avatar
    Oren Raboy

    Hi Pavel,

    First appologize for the late reply to your question. Our new knowledgebase is still in soft launch and we missed the incoming comment.

    To answer your question: Totango looks at the avg. number of user-actions registered daily on the account up to the past 14days. If the account is new (for example, created 3 days ago), only the days since its the past 3 days is considered and the daily average is compared against that of other accounts (which may include a longer time-span, of up to 14days, if they were created earlier).

    Also worth noting that accounts with less than 1 activities/day on average will be assigned 0 as their engagement score. We've found this improves the algorithm by weeding out accounts with a very small amount of activity and declaring them as inactive.

  • Avatar
    Pavel Hardak

    Another comment - I see the engagement score is calculated across the accounts per account type, i.e. free, paying and cancelled. Engagement score for cancelled accounts is not relevant. Regarding free vs. paying - in typical SaaS app with freemium model the percentage of paying customers usually is about 3-5%. It is not clear where to put customers with 'Trial' status - probably into 'free', as they do not pay. However, it means they would get artificially high score compared with "real" free customers, as majority of them is less active. It might be good idea to have "trial"customers as the separate status type, so the engagement score for them would be computed relative to other trial accounts. This way, SCMs can focus on accounts with high engagement score, as those might be good candidates to upgrade to "premium" tier, and on low engagement score to check for problems. Currently, there is no such differentiation and therefore engagement score for the customers with the "free" status type is not as usable as it could be.

  • Avatar
    Oren Raboy

    Hi Pavel,

    For the purpose of Engagement score calculation, the account's Status attribute is used (not the Status type). So if you define a Status value of -Freemium- and a different one -Trial-, these groups will be treated and normalized separately

    For apps with a freemium plan and a trial program (typically time-based trials which 'fall-back' to a basic freemium plan once they expire), we recommend placing the Freemium plan in the Free status-type.

    We updated the article to reflect this example to hopefully create more clarity on the topic.

    Let us know if you have any additional questions or concerns on this.

  • Avatar
    Stewart Fernandez

    good

Powered by Zendesk