Score model for technology appraisal of Bulgarian start-ups

Abstract

Lack of liquidity in the early stages is one of the leading factors for failure of start-ups. Covering the financial needs requires guarantees for the lender, which have the form of a tangible collateral. However, technology start-ups in early stages have mainly intangible assets, which are difficult to valuate. A robust technology appraisal method would make risky investments in risky ventures and at the same time will increase the return on investment in new technology companies. In this paper we present an overview of the traditional methods for valuations, discuss one successful approach used in Korea and propose to re-engineer and apply it in local environment.

Full text

Score model for technology appraisal of Bulgarian start-ups
Stoyan Panov1 and Petko Ruskov2
1 e-mail: stpanov@gmail.com
2 Sofia University “St. Kliment Ohridski”, Sofia 1113, 125 “Tsarigradsko shose” Blvd., bl.2, Bulgaria, e-
mail: petkor@fmi.uni-sofia.bg
Abstract: Lack of liquidity in the early stages is one of the leading factors for failure of start-ups. Covering the financial needs requires guarantees for
the lender, which have the form of a tangible collateral. However, technology start-ups in early stages have mainly intangible assets, which are difficult
to valuate. A robust technology appraisal method would make risky investments in risky ventures and at the same time will inc rease the return on
investment in new technology companies. In this paper we present an overview of the traditional methods for valuations, discuss one successful approach
used in Korea and propose to re-engineer and apply it in local environment.
Key words: start-ups, technology appraisal, technology assessment
If you haven’t measured something, you really don’t know very much about it.
— Karl Pearson
INTRODUCTION
The world is experiencing a boom in interest around start-ups.
Now days the Lean Start-up methodology is the norm for many
start-ups and spin-offs, starting in at university or alternative
institution [1, 2]. Frequently the key to success for a technology
start-up is to validate the technology and the needs of the
targeted customer. Technology appraisal of start-ups is vital for
all stakeholders, if they want to increase their chances of
success. This essential validation data cannot be found in
participating at different type of workshops. Entrepreneurial
teams must use a proven scientific method or system to assess
currently growth potential of the technology and measure risk
of product development hypotheses of their potential
customers. There is a big risk to possibly get stuck in the
“Valley of Death” area – figure 1 [3]. It refers to the time when
the start-up runs out of funds and create a negative cash flow
before a steady stream of revenues can be established.
Figure 1. Start-ups Cash flow
The financing valley of death commonly tests the commitment,
determination, and problem solving ability of every
entrepreneurial team. During that time when the start-up team
produce remarkable value out of nothing. Efforts at that time
really separate the true entrepreneurs from the imitators.
Usually entrepreneurial team needs outside financial venture
support from angels, incubator, accelerators or government
funds to cross the “death valley”. Today new, agile and
competent venture funds use new business models. They
change the exit strategy from an IPO (Initial public offering) to
acquire (mostly technology disruptive start-ups) – figure 2
Figure 2: Startup Lifecycle Today
In startup financing, the term valley of death relates to the
period of time with negative cash flows, usually from the seed
stage to the generation of positive net income. As companies
with negative cash flows need external financing to cover
expenses, failing to obtain such leads to inevitable failure.
Figure 3a

Figure 3b
To bridge the valley of death, various financial supporters step
in. In the initial stage, these are likely to be friends and family,
followed by angel investors, why at the later stages venture
capitalists step forward.
Often startups with good chances fail to get access to external
capital. On the other side start-ups with less prospects obtain
financing, but fall through afterwards. This suggests that the
financing decisions may well not asses the commercialization
potential of technology: (i) friends and family are emotionally
driven (ii) grants, foundations and state programs have flawed
selection criteria (iii) angel investors and venture capitalists
often delegate the analysis to the entrepreneurs, partly because
they cannot evaluate the implications of the technology.
A closer look into decision making mechanism reveals that the
blame is on the founders. They fail to come up with a sound
business plan that outlines the profitability and the viability of
the undertaking. At the same time, entrepreneurs are often
engineers with little experiencing in assessing technology and
the economic way of thinking.
A holistic model that could assess the profitability and the
marketability of technology would fill in the gap and contribute
to the more efficient allocation of capital. Such method would
reduce the burden on entrepreneurs and provide robust
decision-making foundation for angel investors and venture
capitalists.
Methodology and research design. Starting point of our
research is a literature review and an Internet key-word search
in order to locate possible best practices and case studies in
Europe, Asia and North America. After that the authors
synthesize a methodology successfully applied in South Korea.
Then will continue to mix methods and observation techniques
to transform them into tangible processes, metrics and results.
The goal of the paper is to present a technology assessment
and validation current state and process and to describe our
reverse engineering model for technology assessment system.
NEED FOR TECHNOLOGY ASSESSMENT AND
TECHNOLOGY APPRAISING MODEL
Definition and interpretations. Technology assessment is
generally defined as the examination of the short-, middle- and
long-term implications of applying technology. A search in
Internet and academic databases Springer Link, JSTOR,
Emerald and so on has shown that it may be understood in a
variety of ways, depending on the context it is used. We have
identified two types of assessment:
i. Monetary assessment
ii. Non-monetary assessment: social, economic and
political assessment
The former focuses on methods and practices for examining the
profitability of technology companies. The subject of
evaluation is company and the output is usually a price or a
price range. The latter type focuses on the social and political
aspects of applying new technology. The subject of evaluation
is the technology itself and the output is a forecast of the
economic, political and social shifts.
We have used four different terms in our analysis to address
both categories, which often can be used interchangeably, but
have subtle nuances:
i. “technology assessment”: points mainly to the social
and political aspects.
ii. “technology appraisal”: yields similar results as
“technology assessment”, but less popular.
iii. “technology valuation”: refers to the methodologies
used for estimating the monetary value of a company
iv. “technology evaluation”: refers to both categories
An on-line Google study (August 2016) on the results has
shown the following results – Table 1.
Table 1. Summary of the search results by search term (in
Millions)
Technology
assessment
Technology
appraisal
Technology
valuation
Technology
evaluation
Google
search
23
2.1
65.5
352.0
Google
scholar
3.6
1.4
4.6
4.6
Challenges assessing new technologies. When assessing new
technology, one should start from the future. Methodologies
that rely on past performance and present performance are of
little to no use.
Firstly, the value of a technology is visible after its
commercialization [4]. Estimates and future projections are
highly inaccurate as suggested by the poor performance of
many VC funds. This is further amplified by the lack of track
record and stable cash flows. Secondly, new technologies can
rarely be benchmarked to other technologies. Often no
comparable technologies exist and even if they exist, the data is
not readily available. So the value of the technology cannot be
inferred from the value of comparable technologies. Finally,
technology is an intellectual property and as such human capital
should also be factored in the assessment. This factor is ignored
by the conventional valuation and assessment methods.
State of the art of technology assessment. In terms of
monetary value, the most popular methods for valuating
technology start-ups fall into two categories: discounted cash
flows and comparables [5].
Discounted cash flows projects the future cash flows in the
future and then calculates (discounts) their present value using
risk adjusted discount rate. Often different scenarios are used
(e.g. best, normal and worst case) and a weighted average is
calculated. It is arguably the most reliable method for valuing
high tech companies [6].
Comparables method figures out the value of a company by
comparing its characteristics to those of comparable companies,
which have been already valued. Essentially, this method
doesn’t assess technology, but rather it strives to discover the
consensus price for that technology.
In terms of non-monetary value, the understanding of
technology assessment varies widely. The majority of the
methods are either too general with vague output or tailored to
specific industry. For instance, the technology appraisal
practice used by The National Institute for Health and Care
Excellence (NICE) is based on a clinical and economic
evaluation, which are not transferable to other industries [7]. On
the other side the widely used in Europe PTA method aims at
the social, ethical and political aspects and less at the economic
ones [8].
We have found only one generic method that systematically
assess the economic and technical aspects of technology. The
model has been developed by the Korean government funded
organization The Korea Technology Credit Guarantee Fund
(KOTEC) [9]. In 2005 KOTEC introduced KTRS (KOTEC
Technology Rating System). It apply a scoring model to assess
technology in four categories: management competence,

Analysis
Output: grade
technical feasibility, marketability and business feasibility and
profitability.
KOTEC TECHNOLOGY RATING SYSTEM
The KTRS system is used for assessing growth potential and
measuring risk. The system assigns a rating on an alphabetical
scale from ‘AAA’ to ‘D’. This in line with alphabetical scale
used by Fitch, Moody’s and S&P.[10]
The new KTRS system is forward looking, considering
prospects of commercialization and future potential of
technology with focus on intangible assets, such human
resources and know-how. In contrast, conventional rating
methods (S&P, Fitch, Moody’s) center on the present. The
rating is result from assessing the business and financial risks.
KTRS system formed of two parts: technology assessment and
risk assessment. The combination of both gives results in rating
matrix (tech level) x (risk revel)  (rating) – figure 4.
Figure 4: Rating mechanism of KTRS[10]
Technology assessment: technology grade. For the assessment
of technology level KOTEC analyses four group of key factors:
(i) management, (ii) technology, (iii) marketability and (iv)
feasibility. The result of the analysis is a grade on the scale of
V1 to V10. The components of the analysis as well as the
number of factors analysed vary with the technology level.
Table 2. KTRS modules and key factors [11]
Module
Medium items
Management
competence
1.Technological experience level
2. Managerial capability
3. Human Resources of the
Management
Technical
Feasibility
4.Ability for R&D
5.Output of Tech. Development
6.Level of Technological innovation
(leadership)
7.Level of Technology Completeness
and Expandability
Marketability
8.Market Competitive status
9.Product competitiveness
Business
Feasibility and
Profitability
10.Ability of Products and Mass
production
11.Operational capability
12.Profit forecast
The technology grade is calculated using the appraisal items
from table 2 by applying an Analytic Hierarchy Process (AHP)
weighted score model. The result is mapped on to as scale of
V1 to V10
Risk assessment: risk grade. The risk assessment conducted
using Logit regression model. The results are mapped on a ten
grades scale of “aaa” to “d”.
Technology rating matrix: technology appraisal grade. The
technology appraisal grade is calculated as a matrix of
“technology” grade and “risk grade”. Figure 2 illustrates the
process.
Figure 5: System of technology appraisal model [10]
REVERSE ENGINEERING THE MODEL
We use a reverse innovation and engineering approach to
discovery and learn from KOTEC model and product and hope
to design a new, specific for Bulgarian ecosystem technology
appraisal model and advances in know-how [12,13]. KOTEC
technology rating system provides a systematic approach to
appraising technology. Since it proved successful in Korea, it is
worth trying testing it in local environment. Despite the
available academic research, the model is proprietory and only
little is known about its mechanism. Most of the public
information gives an abstract overview, but the specifics remain
hidden. Although the KOTEC’s model is hidden, we will use
the available information to reverse engineer it. The
classification hierarchy of the grading system outlined in the
previous section is a good starting point for our endeavour and
design.
The Bulgarian model is made up also of the parts: technology
grading and financial risk assessment.
The first part is illustrated in figure 6. The model will assess
companies on 34 factors. The factors will be grouped in four
major categories as outlined in the previous section:
management capability, technology, marketability and
profitability.
The collected information will be then validaded and its
relevance will be evaulated. Finally the technology grade will
be calculated as a wegited averge of the input factors.
We will also use AHP method to calculate the technology
grade.
Figure 6. Technology grading assessment
The second part relates to assessing the current financial
situation. While the first part evaluates the commercialization
Management
Technology
Marketability
Profitability
Validate
Assess
Calculate
Technology
grade
V1 – V10
Input: 34 factors
factors

and profitbility aspects in future, the first/second? part assesses
the financial state the company and the business environment.
Figure 7 Figure 6. Financial risk assessment [14]
CONNECTING THE DOTS
A reliable technology appraisal system will improve the
allocation of capital.
Figure 8 illustrates the authors view of the present structure of
the ecosystem in Bulgaria. Sofia Tech Park serves as a hub
between the academics, entrepreneurs and capital providers. A
system that is able to tell apart successful from unsuccessful
startps will be of interest of all parties invovled in the
ecosystem.
Among the venture funds that invest in bulgaria are Eleven and
LaunchHub. Some of the universities that could benefit from
the technology appraisal system are Sofia University, Technical
University Sofia, Trakia Universty, Technical University of
Varna and the University of Ruse.
Figure 8. Ecosystem in Bulgaria
Universities could focus better its research toward technologies
with good prospects of commercialization. Incubators and
accelerators will pick companies at seed stage, without any
track record and collateral, but having good chances of being
noticed by venture funds. The technology appraisal system may
also find application in technology transfer, investments, credit
and Mergers and Acquisitions (M&A). One interesting use of
technology appraisal that has proven itself in Korea is for
technology appraisal guarantees [15]. Startups are risky
investments and as such they scare off many of the traditional
investors. The technology appraisal guarantees provide
insurance that a company won’t fail in certain period of time. It
takes part of the risk and makes the risky technology a more
attractive investment.
The system could also be used in combination with traditional
valuation methodologies, such as income market and cost
approaches. These methods are used to calculate the monetary
value of the technology. The technology appraisal system will
could assess the management, commercialization, technology
and profitability aspects and validate the results of the
conventional methods.
CONCLUSION
A technology appraisal system will have positive implications
on the Bulgarian ecosystem. Increasing the return on
investment in risky ventures, will trigger even more
investments, which are necessary to foster innovation. This will
create a vicious cycle of investments, from which all
stakeholders involved in the process will benefit.
REFERENCES
1. Steve Blank and Bob Dorf, The Startup Owner's Manual:
The Step-by-Step Guide for Building a Great Company, 2014.
2. Steve Blank, Welcome to the Lost Decade (for
Entrepreneurs, IPO’s and VC’s), 2010
(https://steveblank.com/2010/07/15/welcome-to-the-lost-
decade-for-entrepreneurs-ipos-and-vcs/).
3. Martin Zwilling, 10 Ways For Startups To Survive The
Valley Of Death, The Little Black Book of Billionaire Secrets,
Forbs, Feb 18, 2013
(http://www.forbes.com/sites/martinzwilling/2013/02/18/10-
ways-for-startups-to-survive-the-valley-of-
death/#4a95fb7b5e40).
4. Sohn, S., Moon, T., & Kim, S. (2005). Imporved technology
scoring model for credit guarantee fund. Expert Systems with
Applications, 327-331
5. Krattiger, A., Mahoney, R., Nelsen, L., Thomson, J.,
Bennett, B., Satyanarayana, K. Kowalski, S. (2007).
Intellectual Property Management in Health and Agricultural
Innovation. 978-1-4243-2026-4
6. Goedhart, M., Koller, T., & Wessels, D. (2016, 02). Valuing
high-tech companies. Retrieved from McKinsey&Company:
http://www.mckinsey.com/business-functions/strategy-and-
corporate-finance/our-insights/valuing-high-tech-companies.
7. National Institute for healthcare and excellence (NICE).
https://www.nice.org.uk/about/what-we-do/our-
programmes/nice-guidance/nice-technology-appraisal-
guidance
8. Bellucci, S., Bütschi, D., Klüver, L., & et. al. (2015).
European Participatory Technology Assessment.
9. KOTEC. (2014). Kibo Annual Report. (pp. 1-93). Korea
Technology Finance Corporation.
10. Hyung, K. J. (n.d.). Flying beyond Conventional Practice to
Better Serve the Innovation in a Maturing Economy - KOTEC
Case. 121-150.
11. Chung, M. (2015). KOTEC's Good Practeice: Technology
Appraisal Guarantee. FELAC's Financing Innovation and
Dynamic Entrepreneurship Forum.
12. Vijay Govindarajan; Chris Trimble, Reverse Innovation:
Create Far From Home, Win Everywhere, : Harvard Business,
2012.
13. Amos Winter; Vijay Govindarajan, What Engineering a
Reverse Innovation Looks Like, HBR, No 4, 2015.
14. Katz, L. F. (2008). Rating Agencies and Their
Methodologies. (pp. 1-64). Washington.
15. Hong, J.-K. (n.d.). Supporting Technology Innovation
Companie through Technology Appraisal Guarnatee Schemes
of Korea. 89-1090
Sofia
TechPark
Venture
Funds
Accelerators
Incubators
Universities