Content uploaded by Simo Suurkuukka
Author content
All content in this area was uploaded by Simo Suurkuukka on Jan 14, 2021
Content may be subject to copyright.
1
LAPPEENRANTA-LAHTI UNIVERSITY OF TECHNOLOGY LUT
School of Engineering Science
Software Engineering
Impact and Benefits of Digitalization
Assoc. Prof. Ari Happonen
PROJECT RESEARCH REPORT:
Customer Value Creation Indicator for Products –
The North Star Metric
Written by:
Aino Viljakainen, Essi Lehikoinen, Julia Waldén, Simo Suurkuukka
Date: 2020.12.31
2
ABSTRACT
Lappeenranta-Lahti University of Technology
School of Engineering Science
Software Engineering
PROJECT RESEARCH REPORT:
Customer Value Creation Indicator for Products – The North Star Metric
Keywords: Performance measurement, KPI, North Star Metric, Customer value
The aim of the project is to get acquainted with performance measurement and the North
Star Framework and to provide a ready-made process for all types of companies that makes
it possible to find the North Star Metric. The project is made in collaboration with Eficode
and it aims to show how Eficode can find its own North Star Metric using the created process
and what a possible North Star Metric could be. At the beginning of the project, the literature
related to performance measurement and North Star Framework is reviewed. After that a
process for finding a North Star Metric is created, few examples are presented, and the
process created is used to find the North Star Metric for Eficode. Also, a suggestion of a
possible North Star Metric is given to the company. In the implementation of the work, the
related theory and interviews are utilized.
The results of the project show that there are many benefits to be gained from using North
Star Metric. These include, for example, better alignment, clearer prioritization, less wasted
work and valuable conversations. The process by which North Star Metric can be found
consists of seven parts. This process was also implemented for Eficode and the number of
customers using DevOps successfully became a possible North Star Metric for the company.
Of course, there are also challenges associated with the North Star Framework, but if the
impact of the benefits outweighs the challenges, the company should definitely consider
getting to know and implementing the North Star Framework.
3
TABLE OF CONTENTS
1INTRODUCTION ............................................................................................ 5
1.1 BACKGROUND .................................................................................................... 5
1.2 COMPANY INTRODUCTION AND ASSIGNMENT ....................................................... 7
1.3 PROJECT SCOPING AND REPORT STRUCTURE......................................................... 7
2PERFORMANCE MEASUREMENT ............................................................... 9
2.1 KEY PERFORMANCE INDICATORS ...................................................................... 10
3NORTH STAR METRIC ................................................................................ 12
3.1 BENEFITS AND CHALLENGES OF NORTH STAR FRAMEWORK ............................... 13
3.2 NSM PROCESS.................................................................................................. 14
3.3 CASE EXAMPLES ............................................................................................... 20
4NSM FOR EFICODE ..................................................................................... 23
5DISCUSSION AND CONCLUSIONS ............................................................. 27
REFERENCES ...................................................................................................... 28
4
LIST OF ABBREVIATIONS
IT Information Technology
KPI Key Performance Indicator
MA Management Accounting
NSM North Star Metric
DevOps A set of practices that combines software development (Dev) and IT
operations (Ops)
SMART Specific, Measurable, Achievable, Realistic and Timely
5
1INTRODUCTION
In this project we aim to outline a generalizable process that enables the discovery, creation,
implementation, and effective use of a North Star Metric (NSM) for executive, division,
team, and product use (Chen et al. 2017). These different levels of North Star Metric use
could seem very different on the surface but the process of creating them is very similar. The
process can be generalized and applied to almost all different kinds of companies and
products with a data-driven plan for the current or future. As the result of this project, we
hope to understand how, why and when to implement a NSM, and to be able to communicate
it so that companies can implement the process and use the created metric in their
management accounting (MA) and decision making. Additionally, we hope to be able to
present an example case, which we create in collaboration with Eficode. The example case
is created to illustrate the applicability of the North Star Metric creation process and work
as a proof of concept.
1.1 Background
Management accounting has expanded considerably in recent years (Laitinen 2012). Data
driven decision making is the in the core of business decisions in today’s leading companies.
Quality metrics are becoming increasingly relevant in management accounting (Laitinen
2012). According to Neilimo (2015), a company exists for the customer. A company or a
product needs to be able to serve the customer as he/she wants to be served (Neilimo 2015).
The North Star Metric is tied heavily into the idea of customer centric business. What the
perfect NSM does is to measure the value added to the customer in the best possible way
and communicate it to the decision-making entities in the simplest, fastest and most
comprehensive way. Essentially, if a North Star Metric is implemented correctly, it is like
catching lightning in a bottle. Not as hard, but as in how powerful tool it is to steer your
product related activities in the correct direction when it comes to long term growth.
What quality metrics in MA are trying to illuminate are things that cannot be measured and
revealed through classical financial metrics such as gross profit margin and inventory
turnover. Companies’ executives generally have no problem obtaining and utilizing this
6
classical historical financial data from accounting (Neilimo 2015). The competitive edge that
can be gained in MA is in whether your company can accurately measure and communicate
relevant quality metrics to the decision makers. Classical financial accounting is about the
past when quality metrics, like North Star Metric, are about the future. Quality metrics are
for making relevant predictions about the course of the company, and how different decision
would affect the most essential output of the company, customer value. The assumption is
that the more value a customer receives, the longer he stays, the more he buys and the more
people he refers about the product, service, or platform. North Star Metric is a better metric
for the long-term growth of the company because instead of measuring how much value can
be extracted from the customer (e.g. Net profit margin), it focuses on how much value can
be given to the customer and how this value can be increased. NSM also has the advantage
of being a ‘leader’ metric, meaning that it measures revenue before actual revenue can be
measured.
The North Star Metric concept comes from Silicon Valley and it is being used by the biggest
and fastest growing tech companies in the world (Ellis 2017). Here are a few examples of
the kinds of North Star Metrics that big tech companies use (van Gasteren 2019):
·Amazon uses ‘Number of purchases per month’
·Airbnb uses ‘Number of nights booked’
·Facebook uses ‘Monthly active users’
Why North Star is a relevant metric for customer value and decision-making. Let us think
for a second about the C-suite of most medium to large size companies. There are a handful
of executives that make decisions that guide the direction of the whole organization. These
executives are always in demand from every direction and their time and attention is their
most valuable resource. What a properly implemented NSM provides is a simple, intuitive,
singular number that does not take more than a few minutes to find, look at and understand.
The customer value created is instantly visible and decisions can be made accordingly.
7
1.2 Company introduction and assignment
The assignment for this Main Project is from company called Eficode. Eficode is a Finnish-
based software company founded in 2005. It has over 300 employees in seven countries. In
Finland Eficode has offices in Helsinki and Tampere and others are located in Sweden,
Norway, Denmark, Netherlands, Germany and Poland. Eficode is the leading DevOps
company in Europe. With the methods of DevOps, Eficode provides help for their customers
to develop software and other digital solutions. Customers of Eficode are from wide range
of industries such as finance, IT, retail and transport. Eficode wants to share awareness of
the DevOps movement with others by training companies and sharing their own experiences
of DevOps. (Eficode 2020)
We chose Eficode as our company for this Main Project because after the presentations of
different companies, Eficode’s assignment sounded the most interesting and the most
suitable for the skills-background of our group members. We started the project right away
by contacting Eficode because the assignment was not so clear and required clarification.
The Eficode contact person did not give direct instructions on what to do in the task but
guided us to think about what the most important key performance indicators (KPI) for
executive management in different companies would be, and what kind of dashboard could
be made of them. Based on this idea, we set out to work on the project and look for related
information.
After doing some research our group decided that the goal of our project is to create a process
that will allow companies to find their North Star Metric. We presented our idea to Eficode
and discussed about it. The concept of North Star Metric was new for them, but they liked
our idea. Therefore, our final assignment is to find a process that allows Eficode to help its
customers to find their North Star Metric.
1.3 Project scoping and report structure
The purpose of this project is to create a process for creating a North Star Metric that can be
utilized in a company in any industry. To make the process more understandable we find out
8
through a literature review what performance measurement means and what are key
performance indicators. With a literature review we also find out what the North Star Metric
means generally and clarify that with case examples. The case examples are of companies
which have created their own NSM.
This document consists of five chapters, of which introduction is the first one. Chapter two
presents literature-based theory of performance measurement and key performance
indicators. Chapter three explains what the North Star Framework means, and the benefits
and challenges related to it. Also, the creation process for NSM is presented and two case
examples of companies’ NSMs. In chapter four the created process is illustrated by creating
a NSM for Eficode. Last chapter five presents the discussion and conclusions of the project.
9
2PERFORMANCE MEASUREMENT
Performance measurement is the process of ongoing monitoring and reporting of the
performance of an activity or a function. It uses performance measures or indicators to
quantify the efficiency, effectiveness, and customer satisfaction of the monitored activity.
With performance measurement, companies are able to understand how well they are doing
and whether they are reaching their goals, if customers are satisfied and if and where
improvements are necessary. (Franchescini et al. 2019, p.133-134; Neely et al. 2005).
Performance measurement is closely linked to performance management, which is a
continuous process of identifying, measuring, and developing performance and aligning it
with the strategy of an organization (Schleiher et al. 2018).
Performance measurement plays an important role in focusing the organization and its
personnel to certain aspects of the business. Often measured things are considered important
and things that are not measured are left with significantly less attention. A well-designed
performance measurement with a balanced set of indicators can lead to improved business
performance. (Waggoner et al. 1999).
Performance measurement systems are comprised of a set of metrics or indicators that are
used for measuring the performance of multiple actions (Neely et al. 2005). Along with
monitoring performance, they allow for improving communication, enhancing motivation,
strengthening accountability and the identification of problem areas in the organization and
its processes (Waggoner et al. 1999). A performance measurement system presents a
structural approach for focusing on an activity’s objectives and performance and linking it
to the strategic plan of the organization (Franchescini et al. 2019, p. 136).
When establishing a performance measurement system, strategy, key processes, and
stakeholder needs should be considered. The challenges with designing performance
measurement are mostly related to the performance indicators and their selection. The main
issue is in deciding what to measure and finding the right data and metrics that can indicate
the performance and causality clearly. Also, while the amount of data and metrics should be
extensive in terms of the information they provide, the number of indicators should be kept
low. Too many indicators lead to losing sight of their impact, neglecting possible correlations
10
between the indicators and distracting management. Ideally, the objectives of the
organization should be indicated with a critical set of metrics, that are balanced in terms of
covering a broad range of organizational dimensions and minimizing conflicts between the
different metrics used (Cho et al. 2016). Common examples of balanced performance
measurements systems are the balanced scorecard and the critical few method. Both focus
on finding the most critical indicators for performance, derived from different dimensions
of the business. For example, the balanced scorecard method uses indicators for finances,
customers, internal processes and learning in order to provide a comprehensive image of the
organization’s performance. (Franchescini et al. 2019, p. 133-140; 144-147).
2.1 Key Performance Indicators
KPIs (Key Performance Indicators) are used very widely in different areas of business.
According to Eckerson (2006, p. 294) “A KPI is a metric measuring how well the
organization or an individual performs an operational, tactical or strategic activity that is
critical for the current and future success of the organization.” It is like a high-level
snapshot that shows progress of a project and tells if it is going in the direction of targets.
Data from KPIs can be used in decision-making for the future (Kerzner 2013, p. 118). When
tracking the results of KPIs, people get an overview of whether the system is performing the
tasks assigned to it or not and how well (Krause et al. 2019, p. 10). When using KPIs it is
very important to know who is responsible for which area and what is going to happen if the
results are above or below the defined edge values (Kerzner 2013, p. 119).
It is important to define for each project beforehand what is and is not a KPI. This demands
multiple different decisions. KPIs for one industry cannot be used directly in another
industry. Even within the same industry, KPIs can be used in different ways. (Parmenter
2015, p. 3-7; Samsonowa 2012, p. 98) KPI measurement methods need to be explained to
stakeholders regularly in order to KPIs to be accepted, understanded and used correctly.
Before providing KPI-related data to stakeholders, it should be clarified whether the data is
already collected or needs to be collected in the future. The costs, complexity and the
schedule should also be defined. All the possible risks must be taken into consideration.
These can affect KPI data collection. (Kerzner 2013, p. 121)
11
Most companies use KPIs, but sometimes it seems that they don’t have a good understanding
of how to use them. KPIs need to be set in advance and they will screen the project’s critical
success factors. KPIs show how close company is to achieve its project goals (Parmenter
2015, p. 3-7). The underlying purpose of a KPI is to measure things that are influencing to
efficiency. KPIs give information about factors that are suitable for decision making and that
will lead to a positive result. Good KPIs drive change, but don’t tell companies how to do it.
KPIs help to create objectives that should add value to the project or achieve the given
values. (Kerzner 2013, p. 121) There are twelve characteristics of effective KPIs which are
represented in the Figure 1.
Figure 1. Twelve Characteristics of Effective KPIs (Kerzner 2013, p. 125)
KPIs should always be in line with the company's strategy and goals and KPIs should be
owned by someone, who will be responsible for its results. KPIs measure the business value
and they are leading performance indicators for companies. That is why they should be
predictive. The results of KPIs should be up to date as well as practical and KPIs should
focus on a few valuable tasks. They should be simple, understandable, in balance and support
each other. Measuring KPIs should start a chain reaction of positive change. KPIs should
also be based on uniform definitions, rules and calculations. KPIs should also be context
driven and reinforced with incentives. With incentives, the impact of KPIs can be magnified.
Finally, KPIs must be updated regularly so that they are always relevant. (Kerzner 2013, p.
125)
12
3NORTH STAR METRIC
North Star Framework is a model that is strongly associated with performance measurement
and Key Performance Indicators. It helps companies to manage their products by finding
one important metric (North Star Metric) that best describes the core value that their product
is giving to customers (Busch 2020). North Star Framework consists of North Star Metric,
business results and customer value, inputs and the work. Inputs are number of impressive
factors which affect the NSM. (Bauer et al. 2020, p. 10-13) North Star Framework can be
used to help in many different things:
1. Prioritize and increase informed and decentralized decision-making
2. Team communication
3. Enable teams to concentrate on impact and resistant, product-led growth
North Star Metric and KPIs can be thought to be the same thing. However, instead of being
just another metric among other KPIs, North Star Metric combines multiple input metrics
into one leading indicator of customer value. It is one critic value that is describing the
company’s product strategy. It is also a leading indicator of sustainable growth. It works as
a connection between the product and the business. (Bauer et al. 2020, p. 10)
It is important that one product development department, product, or product portfolio
serving a single customer base should have only one NSM. However, in a large company
there might be many divisions, product development departments, products or product
portfolios serving multiple customer bases. In this case, company should consider
introducing more than one NSM. (Bauer et al. 2020, p. 12)
NSM should meet certain conditions. It should indicate value, vision, and strategy. It tells a
company what kind of things are important to customers in product. (Bauer et al. 2020, p.
18) It should be a leading indicator of successful business that is predicting what is
happening in the future (Busch 2020). It also should be actionable and easy to understand
and measure. Company should be able to influence NSM. It should be a metric that describes
13
product’s success in a long term. The North Star Framework is shown in the Figure 2. (Bauer
et al. 2020, p. 18)
Figure 2. North Star Framework (Bauer et al. 2020, p. 10)
3.1 Benefits and challenges of North Star Framework
North Star Framework offers multiple benefits for different companies. North Star Metric is
a leading indicator of sustainable growth. It is like a bond between the product and customer
value and business. It represents value, company vision and strategy. (Bauer et al. 2020, p.
97) North Star Framework allows for better alignment and clearer prioritization (Busch
2020). With the help of North Star Framework companies have less wasted work.
Identifying, designing and implementing North Star Framework inspires new valuable
conversations while employees share their ideas and learn new things from each other.
(Bauer et al. 2020, p. 97)
However, North Star Framework has some challenges. One of the most common challenges
is that people usually think immediately if something is measurable or not. Companies spend
too much time too early on thinking how something can be calculated. People must
remember to focus on important decisions and try to reduce uncertainty. (Bauer et al. 2020,
p. 67-70) Other challenge is that companies usually focus more on the North Star Metric
while focus should be on inputs. Inputs are the basis of the North Star Metric so they should
be considered properly. (Busch 2020) One main challenge is also that companies want to
have many different North Star Metrics. However, the fewer metrics, the more efficient and
14
clearer it will usually work and the better the benefits. Last challenge is that employees
usually let company malfunction beat their ability to progress and improve. (Bauer et al.
2020, p. 67-70)
3.2 NSM Process
In this section we illustrate the process that can be used to create the NSM. The process
consists of seven phases, which are listed in Figure 3.
Figure 3. The NSM process
Define company strategy and vision
The first step of the North Start Metric process is to clarify the company’s strategy and vision
for all the people who participate in the process. The purpose is not to create a strategy or
vision for the company, but they should already exist. Going through the strategy among all
the participants is important because the NSM should represent it and be in line with it.
When the strategy and vision are clear to everyone then the participants identify in which
‘game’ area the business or products belongs. This helps to think what kind of the NSM
should also be. There are three identified games: attention, transaction and productivity
game. In attention game the company wants that customers spend a lot of time with the
product and they benefit of it. In transaction game the business is largely based on selling
products and they benefit when customers buy their products. In productivity game the
company creates products which help other people/companies to accomplish something. The
company benefits when customers complete tasks efficiently with using their product.
(Bauer et al. 2020, p. 30-33) The second step of the process can be moved once the game is
defined.
15
Start designing your input metrics
The aim of the second step is to define potential input metrics for the NSM. The inputs are
metrics that you believe will affect the most to the North Star Metric. In this part you try to
create as many input metrics as possible because some of them will be reduced later. Good
way to start designing the input metrics is to consider them through four different fields:
breadth, depth, frequency, efficiency (Bauer et al. 2020, p. 35). A good number of inputs at
this point would be at least three metrics for each field.
Input metrics belonging to breadth describe how many users are taking the action, for
example number of customers placing orders each month. Depth describes what is the depth
of customer’s engagement, for example number of items within an order. Frequency
describes how often customer takes the action, for example number of orders completed per
customer each month. Efficiency describes how efficient the business is for customer, for
example percentage of orders delivered on time. (Bauer et al. 2020, p. 35) When there are
enough input metrics for each field, you can move on to the next step of the process.
Formulating NSM options
In the third phase of the process, the goal is to start forming the NSM by coming up with
different candidates for it. The first step is limiting the input metrics developed in the
previous phase to a few best and most important ones. Three to five input metrics are a good
amount for the NSM, for example one or two in each category.
Once the input metrics are clear, it is possible to start formulating the candidates for the
NSM. This is best done in a brainstorming session where ideas are formed from multiple
different perspectives. The NSM options should be derived from the chosen input metrics.
In order to help with developing ideas for the NSM, the NSM checklist can be used which
states the characteristics of a good NSM. The criteria for a good NSM are listed in Table 1.
After the input metrics and options for the NSM are ready they should be documented clearly
for the next phase of the process.
16
Table 1. The NSM checklist (Bauer et al. 2020, p.18-21)
Expresses value The NSM should represent what customers value about
the product or service
Represents strategy and
vision Strategy and vision should be visible in the NSM
Leading indicator The NSM should predict what will happen in the future
rather than tell what has already happened
Actionable There should be a possibility to influence the metric
Understandable The metric should not require technical knowledge to
understand
Measurable There should data available to measure the NSM in
order to be able to track and communicate it
Testing
Once the input metrics and possible NSMs have been formulated, the next phase is testing
them in order to make sure they are measuring what is needed. This includes two parts,
principle testing and implementation testing. In principle testing, the idea is to go through
each input metric and NSM and compare them to the characteristics of a good metric. For
the input metrics, a good way to do this is to use the SMART criteria: are the metrics specific,
measurable, actionable, relevant and time bound. For the NSMs, the metrics should be
evaluated against the NSM checklist in Table 1, which contains criteria for a good NSM.
The testing should be done one metric at a time and if some metric does not meet the criteria,
it should be either discarded or reformulated by going back to previous phases.
Once the input metrics and at least one of the NSMs have passed the principle testing, they
should go through the implementation testing. The point of this testing phase is to make sure
that the metrics can be implemented and that the needed data and tools exist or can be
acquired. If it turns out that it cannot be done, the metrics should be reformulated.
17
Deciding the final NSM
Time to get specific about your inputs and North Star Metric. Namely in this part of the
process we are going to decide the final NSM and give a clear and concise name and
definition to it. Select one NSM that you feed is the most descriptive and at the same time
the most intuitive. Make sure the NSM selected conforms to the NSM checklist and therefore
is valid. Remember that your NSM will evolve and doesn’t need to be perfect right away.
The only thing you really need to worry about when selecting an NSM is that it is aiming in
the right direction, so that when company personnel pursue affecting this number, they
simultaneously fulfil the strategy of the company. The North Star Metric will adapt as you
learn. The most important is that the NSM reflects the combination of customers behaviors
and in line with company strategy. (Bauer et al. 2020, p. 48)
After this you need to name and define this metric that has been chosen. Formulate a name
and description that is precise, and it is clear how to measure it. You must be able to explain
your metric to other people so make it as simple as possible. Your growth product roadmap
will be focus on moving this metric up, so your employees need to be able to understand it.
An example of a good NSM name and definition is as follows (Bauer et al. 2020, p. 47-48):
·North Star Metric Name: Frequent Content Shares (FCS)
·Definition: The number of unique subscribers who share an average of two or more
articles per week during the previous 12-week period.
Example of a vague definition for NSM:
·Tracking subscribers who share our content.
Make sure the definition of your NSM covers the product’s market, functionality,
performance and future. To make sure this happens you can use the following questions as
a tool to find out if all of them have been covered by your definition (Bauer et al. 2020, p.
48-49):
·How will your decision-making be impacted by the metric?
18
·What is the metric not telling? Are those areas of significance?
·Would the metric be influenced positively by stopping all development activities? Why?
·How does your chosen metric account for survivor bias?
·Will the evolution and generation of this metric be possible with minimal effort?
·How do different cycles of time (years, seasons, weeks etc.) effect the metric?
·What will be your trigger to review this metric? If reviewing the metric periodically, how
will you determine its relevancy and usefulness?
Implementation
Now we have defined our NSM and the inputs it comes from. How do we make sure the
NSM is utilized to its maximal capacity in the teams’ day to day activities? We must make
sure the NSM framework is integrated to the work, planning, prioritization of tasks. The key
to making NSM work for your organization is to maintain a clear connection between your
work and your NSM (Bauer et al. 2020, p.).
Connecting the work with the NSM framework can be done by using “bets” as in investments
of team members’ work in the product. Bet means your essentially betting that investing
work on something that influences an NSM input has a positive effect on the input. The word
“bet” describes risk, impact, assumption and uncertainty. The more distant the bet is from
the ultimate goal of the business results, the higher the level. Zero is the highest level of bet
which is connected to the NSM and has a time horizon of multiple years. Level 4 bet is an
intervention that is conducted on a short timeframe and is very tangible. (Bauer et al. 2020,
p. 80) Table 2 shows an example of how levels of bets can be mapped out on a working
board.
19
Table 2. The bet levels on a working board (Bauer et al. 2020, p. 84)
Follow-up and possible reformulation
Once a North Star Metric is implemented, it is important to check regularly how successfully
it is working. If it works well and there is nothing to improve, North Star Metric can be used
as it is. However, the metric should be checked every few weeks and update if necessary. If
North Star Metrics does not work as desired and there is room for improvement, it can be
changed.
20
3.3 Case Examples
In this section we present two examples of how the NSM has been implemented in other
companies. These examples provide more insight on how the NSM can look like and how
the process of creating one works in practice.
Spotify
An audio streaming service Spotify has created their own North Star Metric and it is called
‘Time spent listening’ (van Gasteren 2019). Spotify has defined their mission as ‘Unlock the
potential of human creativity – by giving a million creative artists the opportunity to live off
their art and billions of fans the opportunity to enjoy and be inspired by it’ (Spotify 2020).
Spotify wants to offer a platform where music artists can easily provide their music to
listeners. Spotify benefits the most when the customers use a lot of time with their product,
so they belong to the ‘Attention Game’. Spotify wants that their customers listen to music
from their service as much as possible.
Figure 4 shows the input metrics and the output metric of Spotify’s NSM process. First
Spotify has created four different input metrics which relate to their mission and the final
output metric. With notifying a user of new artists or giving recommendations of songs that
are appropriate for the user, Spotify strives to get customers back to the service as soon as
possible. Users are also able to create own playlists in the software and discover new songs,
which increase the time that user spends in Spotify in one session. The metric ‘Bring users
back more often’ is related to the field of frequency and the metric ‘Increase time spent per
session’ is related to the field of depth.
When the best few input metrics were decided Spotify defined the final North Star Metric
which is presented as output metric in the picture. This is a good NSM for Spotify because
it measures the value that customer is able to get from the service and it is related to the
mission of Spotify.
21
Figure 4. North Star Metric of Spotify (Adapted: van Gasteren 2019)
Amplitude
Amplitude is a product intelligence platform company. Their north Star Metric is called
Weekly Learning Users, WLU for short. The definition of this NSM is “the count of active
Amplitude users who have shared a learning that is consumed by at least two other people
in the previous seven days.” This NSM has been selected because Amplitude feels this
represents the most valuable user type, they call the “the advocate”. This ties back to
Amplitudes product strategy which is “to be learning-driven, providing to cross-functional
teams.” and the selected NSM represents this strategy. Amplitude has a core belief that
collaborative learning makes it compound, and hence the North Star Metric selection.
Figure 4 shows the input metrics and output metric of Amplitude’s NSM process. The
explanations of the inputs Amplitude selected for their NSM are (Seitz 2020):
-Activated Organizations (AO). Definition: Organizations that have exceeded five
WLUs.
22
-Broadcasted Learnings (BL). Definition: Things like charts, dashboards, and notebooks
consumed by two or more people in a seven-day period
-Consumption of Learnings (CoL). Definition: Describe the total reach of the BLs in an
organization.
These inputs were selected to better be able to evaluate and prioritize which customer request
to react to in order to satisfy customers better and hence increase customer value. After this
North Star Metric was established, the product team at Amplitude was able to frame the
customer requests better by looking at the requests and comparing them to how they would
positively impact the inputs of the NSM. This in essence helps the product team get a better
grip of what the customers actually need in order for us to deliver them more value. Customer
is always right but doesn’t always know how to frame their requests so that the product team
know what the best solution is. In this case the NSM allowed the teams to contextualize the
need of the customer and create a solution that would lead to increased customer value and
therefore, long term growth. (Seitz 2020)
Figure 4. Amplitude’s North Star Metric
23
4NSM FOR EFICODE
In this section, we illustrate the process of creating an NSM for the case company Eficode.
We followed the steps of the NSM process described in the previous section, excluding the
implementation and follow-up phases. The NSM created can help Eficode better follow the
realization of their strategy and integrate it to the daily work and decisions.
The process starts by defining the vision and strategy of the company. Eficode’s vision and
strategy is to build the future of software development and to be the leading DevOps
consulting company in Europe. As a consulting company, Eficode’s product is the expertise
that their consultants provide to customers. It plays the productivity game as the business
benefits directly if customers become more efficient with the use of their services.
The strategy acts as a basis for building the NSM. The next step is to start formulating the
input metrics for the NSM. We used the four categories, which were breadth, depth,
frequency and efficiency, and came up with possible input metrics that would be relevant
for Eficode’s business. From those options, we limited them to the few most important ones
based on which of them correlate with the strategy and goals of Eficode the most and follow
the SMART criteria. The input metrics are listed in Table 2, and the chosen ones are bolded.
Table 2. Input metrics for Eficode
Breadth Number of customer cases per month
Number of DevOps cases per month
Number of DevOps trainings per employee
Depth Number of customer processes improved each month
Number of DevOps practices initiated in customers operations
each month
Frequency New customers/cases each month
Number of returning customers each month
Efficiency Percentage of satisfied customers out of all customers
Average time spent on projects
Number of recommendations by customers
24
Based on the chosen input metrics, we started to develop possible options for the NSM. The
options were formulated in a way that they are derived from the input metrics and also
correlate with the company’s strategy and vision. After we had some candidates ready, we
moved to the testing phase to limit the options. In the principle testing, we went through the
NSM options and compared them against the criteria for a good NSM or the NSM checklist,
which was presented in Table 1. The NSM options and the faults found for each are listed in
Table 3.
Table 3. NSM options and their evaluation
NSM options Faults in the NSM
Number of DevOps practices successfully
implemented in customer companies Not measurable
Average time spent solving DevOps
problems for customers Does not directly contribute to the vision
and strategy
Number of customers using DevOps
successfully in their operations
Amount of positive feedback from DevOps
cases Not entirely actionable as it is dependent
on customers leaving feedback
Number of times support is needed in
DevOps cases after deployment has been
done
Does not express the value delivered to
customers
Based on the principle testing we identified the best candidate for the NSM, which was the
one that had no fault when compared against the NSM checklist. After this best option was
identified, we started to formulate the final NSM based on it. For the final NSM, we specified
a simpler name and a clear definition, to make it easy to understand, implement and follow
the metric. The name and definition are specified below:
North star metric: Customers using DevOps successfully
Definition: Monthly number of satisfied customers using DevOps in their operations
25
The final NSM correlates with the strategy by measuring the customer cases in which the
company’s core value proposition, successful DevOps consultancy, has been delivered
successfully. If the amount keeps growing, the company is moving toward the right direction
in terms of their strategy and delivering customer value. If the value of the metric were to
decrease, the reason behind it should be identified by drilling down to the input metrics. For
example, a decrease on one month in the successful DevOps implementations may be due to
decreased customer satisfaction (the cases have not been completely successful), a decrease
in cases or an increase in the time spent on each case which decreases the amount of
completed cases. If the NSM is successfully implemented and followed, it can have positive
impacts on increasing the number of times customers recommend Eficode for DevOps
related cases, which would bring the goal of leading DevOps company closer with the
increase of DevOps cases. It can also have benefits on customer satisfaction as more
customers are being delivered the value that they expect. The final input metrics, the NSM
and the impacts are illustrated in Figure 5 below.
Figure 5. The input metrics and NSM for Eficode
The chosen NSM should help Eficode to align their decisions with their strategy of being
the leading DevOps consulting company in Europe. Even though the company will most
likely make profit even if does not get the most successful DevOps cases, it will not be able
26
to provide its core value proposition to customers and will not reach their strategic goals. By
following the NSM, it is possible to see whether the company is heading for the right
direction on a higher, strategic level.
27
5DISCUSSION AND CONCLUSIONS
The North Star Metric can help companies focus on customer value and align daily
operations with their strategy. Instead of being just another metric among other KPIs, it
combines multiple input metrics into one leading indicator of customer value. The input
metrics are an important part of the NSM, as they provide the possibility to drill down from
the metric and further analyze the company’s performance if needed. The NSM does not
replace the company’s existing roadmaps or revenue measures but adds to and supports the
other performance measurement systems by providing a way to measure strategic
performance.
The NSM can be created by following the process presented in this report. The process has
seven phases, and it can be used iteratively to redefine and reformulate the input metrics and
NSM in agile way. Based on the case examples and the experiences gained from creating
the NSM for the case company Eficode, it is clear that the NSM can be beneficial for and
that the process can be used in many different kinds of companies.
The North Star Metric is an interesting research area also in the future because not much
scientific research has been made of it yet. More knowledge is needed about how the NSM
has operated in different companies and how companies have benefitted of their North Star
Metrics. It would also be interesting to find out what kind of North Star Metrics different
companies create with the process presented in this paper and how successfully the NSMs
have worked. With creating more company case examples of North Star Metric in the future
it would be easier to identify the strengths and weaknesses that NSM has brought to
businesses.
REFERENCES
Bauer, J., Hedge, S., Kouzmanoff, A., Levine, D., Misry, P. & Madhavan, A. 2020. The
North Star Playbook – The guide to diset alcovering your product’s North Star. Amplitude.
104 p.
Busch, M. 2020. 7 things Every Product Leader Should know About the North Star Metric.
[WWW-document]. [Accessed 29.12.2020]. Available: https://blog.amplitude.com/north-
star-metric-product-leader-advice
Chen, A., Fu, X. 2017. Data + Intuition: A Hybrid Approach to Developing Product North
Star Metrics. In Proceedings of the 26th International Conference on World Wide Web
Companion (WWW '17 Companion). International World Wide Web Conferences Steering
Committee. 617–625 p.
Cho, P., Hurley, M., & Xu, S. 2016. Metrics and measurement of trustworthy systems.
MILCOM 2016 - 2016 IEEE Military Communications Conference.1237-1242 p.
Eckerson, W. 2006. Performance Dashboards: Measuring, Monitoring and Managing Your
Business. In Performance Dashboards (1. Aufl.). Wiley. 294 p.
Ellis, S. 2017. What is a North Star Metric?. [WWW-document]. [Accessed 26.12.2020].
Available: https://blog.growthhackers.com/what-is-a-north-star-metric-b31a8512923f
Franceschini, F., Galetto, M., Maisano, D. 2019. Designing Performance Measurement
Systems Theory and Practice of Key Performance Indicators. 1st ed. Springer International
Publishing. 211 p.
Kerzner, H. 2013. Project Management Metrics, KPIs, and Dashboards - A Guide to
Measuring and Monitoring Project Performance. 2nd ed. John Wiley & Sons. 429 p.
Krause, H. & Arora, D. 2019. Key Performance Indicators for Sustainable Management: A
Compendium Based on the “Balanced Scorecard Approach”. In Key Performance Indicators
for Sustainable Management. 372 p.
Laitinen, E. Kova ja pehmeä informaatio johtamisessa. Tilisanomat 5/2012. 36 p.
Neely, A., Gregory, M., Platts, K. 2005. Performance measurement system design: a
literature review and research agenda. International journal of operations & production
management. Vol. 25, no. 12, 1228-1263 p.
Neilimo, K. Menestystekijät eivät näy taseessa. Tilisanomat 5/2015. 14 p.
Parmenter, D. 2015. Key Performance Indicators: Developing, Implementing and Using
Winning KPIs. New Jersey. John Wiley & Sons.
Samsonowa, T. 2012. Industrial Research Performance Management Key Performance
Indicators in the ICT Industry. Walldorf. Springer. 461 p.
Schleicher, D., Baumann, H., Sullivan, D., Levy, P., Hargrove, D., Barros-Rivera, B. 2018.
Putting the System Into Performance Management Systems: A Review and Agenda for
Performance Management Research. Journal of management. Vol. 44, no. 6. 2209–2245 p.
Seitz, B. 2020. North Star Playbook. [WWW-document]. [Accessed 29.12.2020]. Available:
http://webseitz.fluxent.com/wiki/NorthStarPlaybook
Spotify. 2020. Company Info. [WWW-document]. [Accessed 28.12.2020]. Available:
https://newsroom.spotify.com/company-info/
Van Gasteren, W. 2019. What is a ‘North Star Metric’? (+8 steps how you can discover your
NSM). [WWW-document]. [Accessed 21.12.2020]. Available:
https://growwithward.com/north-star-metric/#north-star-metric-examples
Waggoner, D., Neely, A., Kennerley, M. 1999. The forces that shape organisational
performance measurement systems: An interdisciplinary review. International journal of
production economics. Vol. 60, no. 1. 53–60 p.