Regional Perspective on R&D Policies for SMEs: Does Success Breed Success?

Abstract

In this paper, we examine the effects of EU policy schemes that support innovation in small and medium-sized enterprises. Since the effectiveness of innovation schemes can be expected to differ across Europe as entrepreneurship and innovation tend to be more intense in more developed regions, we postulate that the effect of EU instruments on additionality increases with the level of development. We offer a multi-country perspective using two waves of Community Innovation Survey data (CIS 2008 and CIS 2012). We find that the impact of EU funding depends on the level of country’s innovativeness: both national and EU public schemes exhibit smaller additionality in less developed countries, while crowding-out is observed only in recently joined EU members.

Full text

Sustainability 2020, 12, 3846; doi:10.3390/su12093846 www.mdpi.com/journal/sustainability
Article
Regional Perspective on R&D Policies for SMEs:
Does Success Breed Success?
Sonja Radas 1, Andrea Mervar 2,* and Bruno Škrinjarić 1
1 Department for Innovations, Business Economics and Business Sectors, The Institute of Economics, Zagreb,
Trg J.F Kennedyja 7, 10000 Zagreb, Croatia; sradas@eizg.hr (S.R.); bskrinjaric@eizg.hr (B.S.)
2 Department for Macroeconomics and International Economics, The Institute of Economics, Zagreb, Trg J.F.
Kennedyja 7, 10000 Zagreb, Croatia
* Correspondence: amervar@eizg.hr
Received: 10 April 2020; Accepted: 2 May 2020; Published: 8 May 2020
Abstract: In this paper, we examine the effects of EU policy schemes that support innovation in
small and medium-sized enterprises. Since the effectiveness of innovation schemes can be expected
to differ across Europe as entrepreneurship and innovation tend to be more intense in more
developed regions, we postulate that the effect of EU instruments on additionality increases with
the level of development. We offer a multi-country perspective using two waves of Community
Innovation Survey data (CIS 2008 and CIS 2012). We find that the impact of EU funding depends on
the level of country’s innovativeness: both national and EU public schemes exhibit smaller
additionality in less developed countries, while crowding-out is observed only in recently joined
EU members.
Keywords: R&D; innovation; EU funding; public subsidies; additionality; SMEs; European Union
1. Introduction
All countries have national public policy schemes to help companies innovate. General
justification for public policy interventions is that it is a corrective mechanism for market failure that
is created by firms’ inability to appropriate the returns to research and development (R&D) resulting
from knowledge spillover mechanisms [1,2]. The public policy programs are costly, which makes it
important to investigate their achieved impacts. While a non-efficient outcome is when firms
substitute the public money for their own investments, the desired outcome is that recipients invest
even more of their own resources in innovation development, thus augmenting the effect of the
public instrument and creating input additionality.
Our focus in this paper is on EU instruments. Namely, for EU countries, in addition to a variety
of national policies, there are also EU instruments operated by the EU agencies. While national
funding schemes differ from country to country, the EU mechanisms are either the same or very
similar in form and implementation across the whole EU. Effectiveness of any innovation scheme can
be expected to differ across countries and regions, as entrepreneurship and innovation are linked to
overall level of development [3,4]. There is a lack of studies on cross-country comparison of public
instruments in general, and those existing ones make comparisons on the basis of a small number of
countries [5–9].
In this paper, we focus on small and medium-sized companies (SMEs) and examine the effects
of EU public policy schemes compared to national policies. To our knowledge, there are only few
papers to date which address this issue. Czarnitzki and Lopes-Bento [10] analyze the effects of a
national versus EU policy mix on innovation input and output of firms in Germany while Radicic
and Pugh [11] examine the same for SMEs using pulled data from 28 European countries.
Additionally, Orlic et al. [9] explore the case of six Western Balkan countries, while Mulligan et al.

Sustainability 2020, 12, 3846 2 of 25
[12] study the case of five old EU members. We attempt to broaden the view on the same issue by
offering a multi-country perspective using the Community Innovation Survey data. We group the
EU countries in four clusters according to their geographical location, which, according to European
Innovation Scoreboard, coincides with their economic strength and innovative prowess: northern
countries, southern countries, new member states that joined in 2004, and the three most recent
member states (Bulgaria, Romania and Croatia). EU instruments were envisioned as a tool that will,
among other purposes, aid the convergence by helping to close the gap in innovation output between
old and new member states, which is still persisting after decades [13–15]. However, little is known
about the impact of EU innovation policies. The academic literature that explores the impact of EU
public instruments is scarce and focuses on collaborative programs [16–18]. Even less is known about
the impact of EU instruments on new member states [6,19].
In this paper, we investigate to which degree the EU instruments produce additionality
compared to the national policies, specifically focusing on new member states. We recognize that the
firms that use EU policy incentives are very likely to use the national instruments as well, which
raises the question how “valuable” the EU instrument is by itself, i.e., whether the addition of the EU
instruments produces a significant impact over the impact produced by national policies only. To
generate deeper insight, we add temporal dimension, because the impact of the instruments likely
depends on the overall economic situation that may change with time. This is why we examine
additionality in the period from 2006 to 2008 right before the global financial crisis and the period of
2010 to 2012 after most of the EU countries had entered recovery.
We single out SMEs because of their significance for every EU economy and their vulnerability
to market failure. Innovation policy instruments are very important for SMEs, because such firms
rely on innovation to an even greater extent than large firms. At the same time, SMEs find it more
difficult to extract rents associated with innovation [20,21], as the asymmetric information prevents
them from securing external capital for innovation. Thus public money is of larger significance for
SMEs than for large firms. An overview of empirical studies evaluating the effectiveness of SMEs
grants is given in Dvouletý et al. [22].
In summary, we seek to contribute to the body of knowledge in the following way:
 We conduct a cross-country analysis of input additionality in SMEs, which received national
and EU support for innovation; in particular, we explore the added effect of EU instrument over
national instrument in firms that use both;
 We examine the effects of EU instruments in new member states (our data includes all new
member states since 2004 except for Cyprus and Malta),
The paper is organized as follows. The next section provides a review of the existing literature
linking public subsidies and firm-financed R&D. Out of the large body of empirical literature, it
primarily highlights the studies that relate to SMEs, studies that include cross-country comparisons
as well as those that take into account the effects of economic recessions and/or crisis. Following the
development of the hypothesis, a description of the model is given in the third and the description of
data in the fourth section. The fifth section brings the discussion of results while verification of the
hypothesis follows in the sixth section. The paper ends with a discussion on the limitations of the
analysis along with the concluding remarks.
2. Literature Background and Hypotheses Development
2.1. Literature Background
Economic theory does not give us a clear answer on the effect of public subsidies on the level of
company-financed R&D. Counteracting effects are working in the direction of “crowding-out” and
“crowding-in”, and the final net effect depends on a number of factors. Testing the additionality
hypothesis is, therefore, a policy-relevant issue for evaluating the use of public funds used to
subsidize private R&D activities.

Sustainability 2020, 12, 3846 3 of 25
Over time, this has induced a large number of empirical studies. In surveys of literature, David
et al. [23] and Cerulli [24] emphasized the problem of endogeneity, present in the empirical studies
up to the end of the 1990s due to the use of linear regression models. In the 2000s, improved datasets
along with the advances in the applied micro-econometrics have allowed for controlling many of the
previous deficiencies. The empirical studies undertaken since the beginning of the 2000s have relied
primarily on matching methods, instrumental variables methods, and selection models. Despite the
heterogeneity of empirical results, that cannot be explained only by methodological differences, the
majority of empirical studies find that R&D subsidies lead to the additionality effects [25].
Despite a large number of studies, those that investigate the impact of public subsidies on R&D
activities within a multi-country context have remained scarce as the majority, due to data
availability, focus on individual countries. There are several exceptions. Hashi and Stojčić [6]
investigated the link between innovation activities and firm performance in two different
institutional settings: Western European countries and EU newcomers from Central and Eastern
Europe. In respect to national and EU subsidies, they concluded that recipients spent more on
innovation activities but had not achieved additional innovation output. A study by Charnitzki and
Lopes-Bento [5] on five developed countries of rather different industrial structures identified
positive effects of public subsidies on R&D activity just as Carboni [8] on the sample of developed
European countries. However, by using the firm-level data for the largest EU states during the
2008/09 financial crisis, Aristei et al. [7] could not confirm additionality effects of R&D subsidies.
Zúñiga-Vicente et al. [25] noted that the majority of studies focused on developed countries with
very little evidence on the impact of public subsidies in developing and emerging economies.
Exceptions are studies for Eastern Germany where additionality in public R&D grants was confirmed
[26,27]. Additionally, Franco and Gussoni [19] included transition countries, Romania and the Czech
Republic, in their sample while analyzing the effects of subsidies on cooperation.
In this paper we focus on SMEs, as they are vulnerable when it comes to innovation activity due
to their size and financial constraints. The R&D subsidy programs for SMEs have been scarcely
evaluated within the applied micro-econometric literature. The exceptions include the Small Business
Innovation Program in the US evaluated by Wallsten [28] who finds no effects of subsidies on R&D
expenditures. Lee and Cin [29] and Cin et al. [30] for the case of South Korea and Charnitzki and
Lopes-Bento [31] for the case of Flemish firms in Belgium find that R&D incentive programs for SMEs
had positive effects on at least some of the input indicators, such as R&D expenditures and/or R&D
investment and/or R&D employees. Hud and Hussner [32] also found positive results for SMEs in
Germany. In the case of Finnish SMEs, Karhunen and Huovari [33] got ambiguous results as positive
results after receiving the public grant change to negative in the succeeding years. More recently,
Mariani and Mealli [34] evaluated the small-business program implemented in Tuscany (Italy) and
found, at least, some positive effects such as the increase of R&D activity for former non-performers
and upskilling. However, the program has not had positive effects on innovation or commercial
output of SMEs. Avellar and Botelho [35] confirmed additionality in the case of Brazilian SMEs,
where support programs for the acquisition of machinery and equipment were more effective than
other programs.
Within the EU, the R&D activities are underpinned through national support schemes and a
range of EU instruments and programs, out of which Framework Programmes (FPs) are the most
important. In general, the literature suggests that publicly funded collaborative research projects
have positive effects on innovative activities of its participants (see, for example, the literature survey
in Hagedoorn [36]). However, the results of studies investigating EU framework programs are not so
straightforward [16,17,37]. Maggioni et al. [38] have used data on FP5 and found that networks affect
innovation but geographical proximity still plays a major role. Additionally, Di Cagno et al. [39]
concluded that for countries with high levels of R&D expenditure R&D spillovers contribute to the
generation of new knowledge, while for low R&D spenders, knowledge spillovers facilitate
technological imitation and catching up. Wanzenböck et al. [40] emphasized how distinct regional
factors affect positioning in the European Union-funded R&D network, while Koišova et al. [41]
confirmed differences between regions in the use of EU funds to finance innovation in SMEs.

Sustainability 2020, 12, 3846 4 of 25
The periods of recessions or economic crisis are particularly harmful to R&D activities. Either
due to financial constraints [42,43] or as a response to the overall weaker demand [44,45] firms
commonly reduce R&D investments in such times. Bloom et al. [46] have shown that the response of
firms to policy actions, including subsidies, is much lower in times of high uncertainty (such as
recessions and crisis) than otherwise. The global 2008/09 financial crisis has severely affected the
world economy accompanied by a drop in overall investment as well as R&D investment.
Policymakers in a number of EU countries reacted by increasing national public R&D budgets (for
example, Austria, Belgium, Sweden, and Germany), while in other countries (particularly southern
Europe) that was not possible due to strong fiscal consolidation [14,15]. Studies that investigate the
impact of these actions have so far been rare. For example, Hud and Hussinger [32] investigate the
impact of public R&D subsidies on R&D investment of SMEs in the case of Germany and find an
overall positive effect of subsidies. In 2009, at the height of the crisis a crowding-out effect was present
while in the pre-crisis period as well as in 2010, when the recovery started, the subsidy effect was
positive but smaller than before the crisis. Aristei et al. [7] analyzed firm-level data for the largest EU
states over the period 2007–2009 and tested whether manufacturing firms receiving public subsidies
spent more on R&D. Although firms did not replace their own resources with public grants, they did
not allocate additional funds to R&D and consequently additionality effects of R&D subsidies could
not be confirmed.
2.2. Hypotheses Development
Based on the above review, we formulate two hypotheses. Earlier studies that examine the
additional effect of EU instruments versus the national ones [10,11] find that when national subsidies
are combined with EU money, the recipients invest more than in the counterfactual situation of only
getting a national subsidy. Due to the nature of our data, we cannot differentiate among different EU
programs, so we adopt the approach outlined in [10,11]. We expand this discussion using temporal
and multi-country dimension, asking whether this is true regardless of the country’s development
level or whether these effects change with time.
To generate better insight into input additionality, we consider investment in internal and
external R&D. Adopting Community Innovation Survey (CIS) definitions, by internal R&D we mean
research and development activities undertaken by the firm to create new knowledge or to solve
scientific or technical problems (including software development in-house that meets this
requirement). By external R&D we mean R&D that the firm contracted-out to other enterprises
(including enterprises in its own group) or to public or private research organizations. These are
defined in Table 3. In line with previous studies, we expect that EU instrument will generate input
additionality for internal R&D. Since EU programs foster collaboration, which may induce firms to
outsource some R&D activities to partners, we expect that additionality will be created in external
R&D as well.
Radicic and Pugh [11] have shown that compared to national support, EU support is more
effective in raising R&D expenditures. However, we expect that the benefit from the EU instruments
will depend on the overall level of development of the country/region where it is applied. It is known
that EU was not able to stimulate sufficient R&D spending in new members from Central and Eastern
Europe, and consequently these groups of countries seem to be permanently lagging behind [15].
Therefore, we expect that this lag will be reflected in the impact generated by the public instruments;
namely, we expect that new member states will show smaller additionality compared to more
developed older EU states, and we expect this effect to persist over time. To get a more detailed
picture, we explore these effects in two time periods, before and after the economic crises of 2008.
Economic crisis can induce firms to lower their R&D investments, but after the crisis is over, things
should be returning to normal with time [32].
We formulate the following hypotheses:
The input additionality effects of EU funding over national funding are larger in more developed
countries. In particular, new member states are less successful in generating input. This is true for:

Sustainability 2020, 12, 3846 5 of 25
a) internal R&D and
b) external R&D.
3. Model
3.1. Matching Method
In this paper, we evaluate effects of public policy instruments via the nearest neighbor matching
method using Mahalanobis distance as matching metric, which includes the propensity score as one
of the covariates. The goal of this exercise is to estimate what would have happened with the firms
that used public instruments if they had not used them. Since this counterfactual scenario has never
existed in reality, we need to find matches to the treated firms which can serve as an approximation
of their counterfactual self. To do this, we follow Rubin [47] who introduced the conditional
independence assumption (CIA). CIA states that potential outcomes are independent of treatment
assignment if the selection into treatment is based on a set of observable covariates , which are
themselves not affected by the treatment. Consequently, this allows for the firms that used
instruments to be paired with those that did not, where the later are as similar as possible on pre-
treatment characteristics. These non-participant “twins” can be used to estimate the counterfactual
scenario, and are usually referred to as “control observations”.
We match treated-control pairs using the Mahalanobis distance metric [48]; we match on a
number of covariates including propensity score. Propensity score, (), is defined as the conditional
probability of receiving a treatment (in this context treatment refers to use of public policy
instruments) given a vector of covariates  [49]. In line with CIA assumption, after conditioning on
these covariates, we expect outcomes to be independent of the treatment condition.
Let  ∈ {0,1} be an indicator which shows whether the firm participated in the program (1
stands for participation, 0 otherwise). Then for a firm i, propensity score can be written as ()=
(= 1|)= (ℎ()), where F is a cumulative distribution function (either normal or logistic) and
ℎ() is a linear function of covariates and their interactions. Propensity score matching means
finding controls that are closest to the treated firms in terms of propensity score. To improve the
quality of matching, propensity scores are usually restricted to the area of common support, which
is the intersection of the intervals of propensity scores for treated and control observations.
As for Mahalanobis, the distance metric measures dissimilarity between observations based on
the vector of covariates . The Mahalanobis distance between observations  and  is defined by
the formula:
where  is the vector of covariates, and  is the variance-covariance matrix of  in the control
sample.
3.2. Modeling Joint Use of National and EU Instruments
We study the situation where firms can choose to use national policy instruments, EU
instruments, both, or none. The national instruments may include financial support via tax credits or
deductions, grants, subsidized loans, and loan guarantees. EU instruments come in the form of grants
and participation in collaborative programs such as FPs. So instead of a simple selection into
treatment that can be modeled by a logit or a probit function, we have two simultaneous and
correlated decisions that we model using seemingly unrelated bivariate probit. Here we follow the
approach of Radas et al. [50], Czarnitzki and Lopes-Bento [10], and Radicic and Pugh [11], who used
the Mahalanobis distance metric augmented with bivariate propensity scores to model the use of two
different non-exclusive policy instruments.
 = (− 
)′Σ(− 
)
(1)

Sustainability 2020, 12, 3846 6 of 25
We specify the seemingly unrelated probit where the two dependent variables are as follows:
= 1 if national instrument was used, = 1 if EU instrument was used. Let 
∗ and 
∗ denote
latent variables underlying binary variables  and , then we can write the model as:

∗= + 
,
= 1
if

∗> 0 ,
0 otherwise;
~(0,1)
(2)

∗= + 
,
= 1
if

∗> 0 ,
0 otherwise;
~(0,1)
. (3)
The correlation between the disturbances is denoted as (, |, ) = . In general, it is
possible that participation in each of the two instruments depends on different set of variables, but
in our study we assume = = . This model produces four conditional probabilities, these are
(= 1, = 0|), (= 0, = 1|), (= 1, = 1|), and (= 0, = 0|). In this paper,
we consider only three of the above four possibilities, because the firms that use only EU instruments
without national instruments are almost nonexistent in our dataset.
We consider each of the three comparisons: (1) firms that used national instrument only
compared to firms that did not participate in any programs, (2) firms that used both national and EU
instruments compared to firms that did not participate in any programs, and (3) firms that used both
national and EU instruments compared to firms that used a national instrument only. In each of the
models (1), (2), and (3), we use the vector of selected covariates together with the relevant propensity
score. The scores are (= 1, = 0|) for model (1) and (= 1, = 1|) for model (2) and (3).
Although the propensity score is the same variable (= 1, = 1|) in both models, the difference
between models (2) and (3) is in the set of firms on which we perform matching. In model (2) we take
the firms that used both instruments and firms that did not use any, while in (3) we take the firms
that used both instruments and firms that used only a national instrument.
Our procedure goes as follows: we determine the appropriate propensity score and add it to a
smaller set of chosen covariates used in Mahalanobis distance computation. We keep only those
observations that are on common support. Then we perform nearest neighbor matching where for
every treated firm we choose the non-treated one with the minimal Mahalanobis distance from the
treated observation. Nearest neighbor matching is done with a replacement, which means that once
a control observation is used as a match, it is returned to the pool and can be reused.
The effects on the output are measured using which is defined as  = 
∑(
− 
)
∈ ,
where the sum goes over all the treated units and  is the number of treated units, 
 is the value
of the outcome variable in the matched pair i, while 
 is the value of the outcome variable for the
control observation in the matched pair i. Analytical standard errors are computed using Abadie–
Imbens formula [51], which takes replacement into account. The matching is performed using Stata
package PSMATCH2 [52].
Although in this paper we follow the established methodology for policy evaluation in
economics, we would like to point out that recently there have been some development in new
methodological approaches to policy evaluation based on machine learning. For more information,
please see [53–55].
3.3. Robustness Tests
A limitation of matching methods is the assumption that the covariates in the model completely
determine the selection into treatment. If some unobserved variables simultaneously affect
assignment to treatment and the outcome variable, matching methods can suffer from “hidden bias”.
The sensitivity of our results to hidden bias is tested with Rosenbaum’s bounding approach [56],
which has been increasingly used as an alternative to the instrumental variables (IV) estimation,
which typically relies on additional untestable assumptions [57,58]. The Rosenbaum bounds method
relies on the sensitivity parameter Γ that estimates which magnitude of the hidden bias would render
the test statistics of the study inference insignificant. The larger magnitude of Γ implies larger
robustness of the outcome.

Sustainability 2020, 12, 3846 7 of 25
4. Data
4.1. Data Source
We use the data collected by CIS, which is routinely conducted in two-year intervals in the EU
and some associated countries. The analysis was conducted in the Eurostat Safe Room, where we had
access to anonymized data from 21 EU countries. We divided them in 4 groups: North (Germany,
Finland, France, Norway, Sweden, Netherlands, Luxemburg), South (Portugal, Spain, Italy), EU2004
(Slovenia, Slovakia, Hungary, Czech Republic, Poland, Estonia, Lithuania, Latvia), and NewEU
(Bulgaria, Romania, Croatia).
To elicit information about public support for innovation, the CIS questionnaire asks whether in
the prior two-year period the enterprise had received any financial support for innovation activities
from: (1) national governmental sources which involve local or regional authorities, or central
government (including central government agencies or ministries), and (2) The European Union
(EU). The financial support could have been in the form of tax credits or deductions, grants,
subsidized loans, and loan guarantees.
To explore the effects of EU instruments before and after recession, we analyzed two waves of
CIS data. The first was for the period 2006–2008, just before the recession. The second is for the period
2010–2012, after the recession was over in most of the EU countries.
4.2. Variables
We define three variables that are used for measuring the type of instrument that was used:
national, EU instrument, or both (Table 1). They represent treatment variables in the matching
exercise. We do not have information about different programs that constitute treatment in each
country; however, in this study we are interested only in the general effect of EU versus national
policies.
Table 1. Treatment variables.
natfun 1 if the firm received any public funding from local or regional authorities, or central
government, 0 otherwise
eufun 1 if the firm received any public funding from the EU (including FPs), 0 otherwise
nateufun 1 if the firm received public funding from both national bodies and the EU), 0
otherwise
The list of covariates used in the first stage model is presented in the Table 2. To avoid
simultaneity, the covariates should be either (1) measured at the beginning of the period covered by
CIS survey, or (2) they must be constant throughout this period. Covariates measured at the
beginning of the CIS survey period are turnover and firm size: these two variables give rise to the
variables micro, small, lnturn_b, lnturn_bsq. Since industry sector variables have large inertia, the
covariates constant throughout each two-year long CIS period are the industry sector variables kis,
lkis, mht, mlt. Group membership is also not likely to change, so we also consider it a constant.
Other variables from the Table 2 in CIS survey refer to questions about the entire two-year
period, but here we use them as proxies for the firm capabilities that existed in the beginning of the
period. For example, the variables inno_good, inno_pro, inno_serv and inno_org specify whether a firm
brought an innovation to market, but in this study we use them as indicators for the capacity to
innovate. Our rational is that the resources and capabilities required for innovation cannot be
developed overnight, and therefore must be present in a company even before they become
embodied in a specific outcome. So in order to be able to bring an innovation to market, the company
must have had the competency to do so. We use exporting variable in the same vein, as an indicator
that at the beginning of the two-year period the company has required resources and capabilities to
become an exporter [59].

Sustainability 2020, 12, 3846 8 of 25
Table 2. Covariates used for matching.
kis Knowledge intensive service according to NACE Rev.2: 1 yes, 0 no
lkis Less knowledge intensive service according to NACE Rev.2: 1 yes, 0 no
mht Belonging to high or medium high technology manufacturing sector according to
NACE Rev.2: 1 yes, 0 no
mlt Belonging to low or medium low technology manufacturing sector according to
NACE Rev.2: 1 yes, 0 no
micro If the firm is a micro enterprise: 1 yes, 0 no
small If the firm is a small enterprise: 1 yes, 0 no
lnturn_b Logarithm of total turnover in EUR (microaggregated) measured at beginning of
each CIS round
lnturn_bsq Squared logarithm of total turnover in EUR (microaggregated) measured at
beginning of each CIS round
gp Is the firm part of an enterprise group: 1 yes, 0 no
inngood Introduced onto the market a new or significantly improved good: 1 yes, 0 no
innserv Introduced onto the market a new or significantly improved service: 1 yes, 0 no
innorg
Did the enterprise introduce any organizational innovation: 1 yes, 0 no
(This includes: New business practices for organizing procedures, new methods
of organizing work responsibilities and decision making, new methods of
organizing external relations)
innpro
Did the enterprise introduce any process innovation: 1 yes, 0 no
(This includes: introducing a new or significantly improved method of
production, introducing a new or significantly improved logistic, delivery or
distribution system, introducing a new or significantly improved supporting
activities)
exp If the enterprise is an exporter: 1 yes, 0 no
INTERACTIONS
medium_ kis Medium size and knowledge intensive services
medium_ mht
Medium size and high or medium high technology manufacturing
medium_
lnturn Medium size and ln(turnover)
medium_
lnturnsq Medium size and square of ln(turnover)
medium_ gp Medium size and group membership
medium_ inn
Medium size and innovator (either of good, process, service or organization)
medium_ exp
Medium size and exporter
lnturn_exp ln(turnover) and exporter
lnturnsq_exp Squared ln(turnover) and exporter
inn_exp Innovator and exporter
Within covariates we also include interactions, following the advice by Caliendo and Kopeining
[60], which is that higher order terms and interactions in the propensity score specification can
improve the matching. In addition, these interactions capture the synergistic effects of size and sector
(medium_ kis, medium_ mht), size and earnings (medium_ lnturn, medium_lnturnsq), size and group
membership (medium_ gp) and size and innovative capability (medium_ inn). While the variable
medium measures size in number of employees, we add the interaction of turnover and exporting to
capture financially strong companies with exporter capabilities. Finally, we include an interaction
between exporting and innovation, capturing propulsive global SMEs.
To measure the effects of public instruments, we look at companies’ internal and external R&D
investments as percent of their turnover measured at the end of the observed period. The CIS
definitions of the output variables are presented in Table 3.

Sustainability 2020, 12, 3846 9 of 25
Table 3. Output variables.
rdin_turn
Internal or in-house R&D
Creative work undertaken within the enterprise to increase the stock of knowledge
for developing new and improved products and processes (including software
development in-house that meets this requirement)
rdex_turn
External R&D
Same activities as for in-house R&D, but performed by other enterprises (including
other enterprises or subsidiaries within firm’s group) or by public or private research
organizations, and purchased by the firm
5. Empirical Results
The seemingly unrelated probit estimation results are presented in Table A1 in Appendix for the
period before recession, and in Table A2 for the period after the recession. In both periods, we observe
that firms that have capabilities for more than one type of innovation are more likely to obtain both
national and EU funding. As for the other firm characteristics, being classified as operating in higher
technology or knowledge sector also improved the probability of getting both types of funding. Firms
that belong to a group are less likely to get both types of funding. These findings are true both before
and after the crisis. So we can say that the factors that determine funding award remain the same
regardless of the economic conditions. As for differences among country groups, we observe that in
the case of NewEU, these effects are much smaller compared with more developed northern and
southern states. Also, considering national and EU instruments, we find that almost the same
variables are significant for both outcomes, which leads to the conclusion that the European and
national agencies have similar selection criteria, as Czarnitzki and Lopes-Bento [10] found for
Germany.
The effects of the funding on internal and external financing are presented in Table 4, and the
findings will be interpreted in the next section. The matching quality is shown by balancing in Table
A3, Table A4, and Table A5, which confirm that after matching there is no significant difference in
covariates between the treated observations and their matched pairs.

Sustainability 2020, 12, 3846 10 of 25
Table 4. Results of matching: Average treatment effect on the treated (ATT) estimations.
2008 2012
National
support only vs.
No treatment
National
support and EU
support vs. No
treatment
National
support and EU
support vs.
National
support only
National
support only vs.
No treatment
National
support and EU
support vs. No
treatment
National support and EU
support vs. National
support only
Output ATT (S.e.) ATT (S.e.) ATT (S.e.) ATT (S.e.) ATT (S.e.) ATT (S.e.)
NORTH
N = 1341 N = 299 N = 298 N = 427 N = 94 N =94
rdin_turn 2.10 (0.31)*** 7.08 (1.09)*** 4.14c (1.21)*** 3.24 (0.45)*** 10.41 (0.99)*** 6.77 (1.09)***
rdex_turn 0.40 (0.11)*** 0.81 (0.16)*** 0.15 (0.20) 0.23 (0.14)** 1.27 (0.32)*** 0.93 (0.32)***
SOUTH
N = 5984 N= 463 N= 465 N = 2983 N = 472 N= 475
rdin_turn 3.42 (0.15)*** 11.94 (1.04)*** 8.56c (1.08)*** 4.29 (0.26)*** 12.01 (1.02)*** 5.47 (1.29)***
rdex_turn 0.54 (0.05)*** 2.07 (0.33)*** 1.33 (0.38)*** 0.6 (0.07)*** 1.72 (0.22)*** 0.77b (0.44)**
EU2004
N = 466 N = 221 N = 221 N = 291 N = 161 N = 161
rdin_turn 2.27 (0.39)*** 5.43 (1.07)*** 2.65c (1.12)*** 2.66a (0.58)*** 5.42 (1.28)*** 1.96b (1.13)**
rdex_turn 0.26 (0.11)** 1.05 (0.40)*** 0.95b (0.41)*** 0.20 (0.19) 0.26a (0.27) −0.01 (0.21)
NewEU
N = 75 N = 50 N = 47 N = 179 N = 52 N = 51
rdin_turn 0.08 (3.38) −6.04 (3.16)** −8.34 (3.4)*** 1.35 (0.58)** 1.15 (1.38) −2.56 (2.10)
rdex_turn 0.46 (1.65) 0.77 (0.31)*** 0.80 (0.31)*** 0.27 (0.17) 0.09 (0.21) −0.07 (0.41)
Notes: a denotes Rosenbaum delta up to 0.1 (10% hidden bias). b denotes Rosenbaum delta up to 0.2 (20% hidden bias). c denotes Rosenbaum delta up to 0.3 (30% hidden
bias). No letter denotes Rosenbaum delta greater than 0.3 (more than 30% hidden bias). * denotes one-tailed t-test significance to 10%, ** denotes one-tailed t-test significance
to 5%, *** denotes one-tailed t-test significance to 1%. S.e. represents standard errors.

Sustainability 2020, 12, 3846 11 of 25
6. Hypotheses Verification
The national public policies are shown to produce significant additionality in internal R&D
investments in all the clusters, except for NewEU in the period before recession. Adding EU
instruments to the national ones considerably improves internal R&D spending, significantly
increasing additionality. That effect is robust and persists over time. The only exception is NewEU
countries, where EU instruments used in conjunction with national ones seem to generate no effect.
Considering the added effect of EU funding on top of national policies, the results suggest that in
NewEU, they may be used as substitutes for own funding (here caution is needed as those effects are
not confirmed by Rosenbaum tests). For the remaining three groups of countries, these added effects
of EU instruments are all positive and significant (although robustness tests do not support the results
for northern countries in 2012).
Regarding the external R&D investments, the effects are much smaller compared to internal
R&D. However, the same pattern exists, except that some of the effects are not confirmed in
robustness tests. As a general trend, we observe that public instruments generate larger additionality
effects on development of internal capabilities than on external R&D links (which include purchasing
or outsourcing of R&D).
Examining the differences among the country clusters, we observe that northern and southern
countries are very similar in the produced effects. Interestingly, there is one difference between
northern and southern countries: the input additionality for the joint use of national and EU
instruments is stronger for southern countries, indicating that there EU instruments are more
effective in motivating SMEs to increase their own efforts. The EU2004 member states show much
lower effects, while the NewEU group is the weakest of all. This supports the expectation that benefits
of EU instruments are stronger when used by more developed economies, which confirms our
Hypothesis.
Considering the temporal dimension, in 2010–2012 in northern countries we observe increase of
internal R&D additionality for both instruments, although some effects are not robust. One
explanation is that in developed economies less efficient firms disappeared after crisis of 2008, and
those remaining ones are even more motivated to use EU instruments to bust their innovation powers
and in that effort they increase their own R&D investments to a larger extent. Comparison of northern
and southern countries shows that the additionalities for EU instruments are larger in southern
countries in 2006–2008. In 2010–2012, the effect of joint instruments remains almost the same, but the
differential effect of EU instrument decreases. This is true for internal as well as external R&D. So we
can conclude that by depleting internal resources, crisis of 2008 may have weakened internal “power”
of SMEs to invest in R&D, except for the very best performers (northern economies). As is known,
developed EU members invested more in R&D during the crisis, while southern and less developed
countries did not due to fiscal contraction measures.
In EU2004 countries, the situation for internal additionality is similar to the one in southern
countries, but the effects are much smaller. For the external R&D, we observe absence of
additionality, although the result is not robust to Rosenbaum bounds test. The only region where we
observe crowding-out for internal R&D in both time periods is NewEU. There is a small improvement
in the period 2010-2012: the effect of joint use of national and EU instruments on internal R&D
investment seem to improve from negative one to an effect statistically indistinguishable from zero.
The effect of EU instrument itself is still negative, but smaller, suggesting that some improvement
may be expected with time.
7. Conclusions
In this paper we examine the effects of EU public policy schemes for innovation compared to
national ones in four clusters of EU countries. We consider old member states separately from the
more recently joined ones. The older states are divided into northern and southern ones, to recognize
the fact that EU innovation is concentrated mainly in northern and western Europe. The more

Sustainability 2020, 12, 3846 12 of 25
recently joined states are divided into those that joined in the first wave (EU2004), and the three states
that joined last (NewEU).
We observe that the extent to which EU instruments create additionality depends on the level of
the development, and that effect persists through time. All groups of countries except for NewEU
increase their own R&D investments when using EU funding, meaning that programs are effective.
However, both national and EU public instruments exhibit smaller effects in less developed regions.
Joint use of instruments creates larger effect than the use of national policies alone for all groups
except for NewEU. As a rule, the effects on development of internal capabilities are larger than effects
on external R&D investments, which include activities such as purchasing or outsourcing of R&D.
If we consider temporal dimension, we observe that the input additionality of the best
innovators is consistently high in northern countries. In southern countries and EU2004 it starts high
before 2008, but then it shrinks somewhat in the period 2010–2012. The only group where we observe
consistent crowding-out for internal R&D is NewEU, suggesting that SMEs in those countries use EU
funding to substitute for their own investments (interestingly this substitution is not detected for the
use of national programs). These countries also show lowest effectiveness of national policies
compared to the other groups. It seems that for them, EU instruments do not bring any contribution
to additionality of internal R&D expenditure, which suggests that in NewEU, in the observed time
periods SMEs countries may not have been strong enough to extract full potential of EU instruments.
As for limitations, in this paper we define treatment as being a national or an EU instrument,
without going into details about the programs. It is a usual practice in policy evaluation to aggregate
several types of programs into one treatment, although this has been done for public policies within
one country. We are aware that we are dealing with a higher level of aggregation here, and that a
more detailed picture of EU policy effectiveness could be obtained from specifically constructed
databases that contain specificities of national programs. We consider this paper as the first step in
the exploration of this important topic.
Author Contributions: Conceptualization, S.R., A.M. and B.Š.; Methodology, S.R., A.M. and B.Š; Software, B.Š.;
Validation, S.R.; Formal Analysis, B.Š.; Data Curation, B.Š.; Writing—Original Draft Preparation, S.R. and A.M.
and B.Š.; Writing—Review & Editing, S.R. and A.M.; Project Administration, A.M.; Funding Acquisition, A.M.
All authors have read and agreed to the published version of the manuscript.
Funding: This work has been supported by the European Union’s Horizon 2020 research and innovation
programme under Grant agreement No 645884. The views expressed are those of the authors and do not
necessarily reflect the views of REA (European Commission).
Acknowledgments: The authors would like to thank participants of the UCL-SSEES workshop on Companies,
Innovation and Productivity held on 28th June 2017 at UCL, London for their comments and suggestions on
earlier draft of this paper. The authors also acknowledge the use of Community Innovation Survey (CIS) dataset
by Eurostat. We hereby state that the results and conclusions are ours and not those of Eurostat, the European
Commission, or any of the national authorities whose data have been used.
Conflicts of Interest: The authors declare no conflict of interest.

Sustainability 2020, 12, 3846 13 of 25
Appendix A
Table A1. Seemingly unrelated probit regression estimation results for 2008.
North South EU2004 NewEU
National funding
kis
0.126* (0.0525) 0.407*** (0.0308) −0.116 (0.0790) 0.00475 (0.161)
lkis
−0.322*** (0.0518) −0.168*** (0.0271) −0.490*** (0.0736) −0.456*** (0.133)
mht
0.338*** (0.0591) 0.238*** (0.0384) −0.263* (0.106) 0.132 (0.166)
mlt
0.0761 (0.0454) 0.0659** (0.0249) −0.153* (0.0599) −0.380** (0.122)
micro
14.47 (7.632) 1.521 (3.616) 4.823 (7.906) −21.02* (10.57)
small
14.67 (7.637) 1.162 (3.620) 5.030 (7.914) −20.81* (10.59)
medium
0 (.) 0 (.) 0 (.) −32.48** (10.25)
lnturn_b
−1.359** (0.497) 0.553* (0.259) 0.314 (0.772) 2.809 (1.529)
lnturn_b_sq
0.0446** (0.0166) −0.0183* (0.00878) −0.0107 (0.0266) −0.107 (0.0551)
gp
−0.191*** (0.0358) −0.0865** (0.0270) −0.295*** (0.0713) −0.0206 (0.141)
inngood
0.619*** (0.0308) 0.426*** (0.0199) 0.501*** (0.0442) 0.548*** (0.0668)
innserv
0.185*** (0.0332) 0.200*** (0.0216) 0.237*** (0.0469) 0.256*** (0.0736)
innpro
0.496*** (0.0301) 0.637*** (0.0191) 0.680*** (0.0475) 0.505*** (0.0704)
innorg
0.0761** (0.0292) 0.124*** (0.0180) 0.160*** (0.0423) 0.302*** (0.0674)
exp
−5.529 (4.333) −2.440 (2.405) −6.726 (5.815) 3.523 (10.03)
medium_kis
−0.220** (0.0755) −0.172*** (0.0471) 0.135 (0.100) 0.356* (0.164)
medium_mht
−0.152* (0.0759) 0.0702 (0.0559) 0.405*** (0.121) −0.150 (0.172)
medium_lnturn
1.891* (0.955) 0.0904 (0.463) 0.551 (1.049) 1.243 (1.892)
medium_lnturnsq
−0.0605* (0.0299) −0.00133 (0.0148) −0.0162 (0.0348) −0.0307 (0.0665)
medium_gp
−0.0946 (0.0603) 0.0104 (0.0411) 0.0553 (0.0920) −0.199 (0.177)
medium_inn
0.304*** (0.0819) 0.249*** (0.0417) 0.567*** (0.122) 0.669*** (0.182)
medium_exp
−0.148 (0.0837) −0.192*** (0.0471) −0.229* (0.106) −0.360* (0.156)
lnturn_exp
0.722 (0.571) 0.235 (0.319) 0.840 (0.782) −0.595 (1.390)
lnturnsq_exp
−0.0230 (0.0188) −0.00429 (0.0105) −0.0264 (0.0262) 0.0244 (0.0479)
inn_exp
0.408*** (0.0608) 0.230*** (0.0352) 0.414*** (0.103) 0.328 (0.205)
_cons
−6.606 (7.797) −7.207* (3.532) −9.377 (7.406)
EU funding
kis
0.152 (0.0788) 0.305*** (0.0587) −−0.0799 (0.0904) 0.655** (0.241)

Sustainability 2020, 12, 3846 14 of 25
lkis
−0.248** (0.0809) −0.152** (0.0556) −0.603*** (0.0888) 0.112 (0.220)
mht
0.125 (0.0902) −0.0106 (0.0786) −0.0400 (0.111) 0.664** (0.249)
mlt
0.0404 (0.0699) −0.0819 (0.0504) −0.168* (0.0661) 0.520* (0.205)
micro
−2.790 (10.44) 8.500 (6.922) 2.690 (9.099) −11.80 (10.38)
small
−2.732 (10.45) 8.511 (6.931) 2.792 (9.111) −11.61 (10.39)
medium
0 (.) 0 (.) 0 (.) −30.18** (10.97)
lnturn_b
0.138 (0.814) −0.556 (0.517) 0.470 (0.943) 1.261 (1.490)
lnturn_b_sq
−0.00516 (0.0272) 0.0194 (0.0174) -0.0181 (0.0326) −0.0477 (0.0533)
gp
−0.0905 (0.0524) −0.0400 (0.0536) -0.151 (0.0801) 0.0565 (0.153)
inngood
0.396*** (0.0445) 0.298*** (0.0370) 0.465*** (0.0483) 0.569*** (0.0708)
innserv
0.242*** (0.0466) 0.251*** (0.0377) 0.167** (0.0517) 0.155* (0.0785)
innpro
0.402*** (0.0445) 0.346*** (0.0379) 0.695*** (0.0542) 0.603*** (0.0772)
innorg
0.110* (0.0429) 0.144*** (0.0353) 0.134** (0.0467) 0.438*** (0.0741)
exp
−2.660 (6.619) −9.428* (4.569) −13.74* (6.841) −1.380 (1117.0)
medium_kis
−0.0178 (0.103) 0.0106 (0.0792) −0.0176 (0.112) 0.367* (0.179)
medium_mht
0.0605 (0.112) 0.0213 (0.104) −0.0591 (0.127) −0.338 (0.204)
medium_lnturn
−0.346 (1.318) 1.081 (0.890) 0.264 (1.216) 2.209 (1.921)
medium_lnturnsq
0.0116 (0.0415) −0.0342 (0.0286) −0.00549 (0.0406) −0.0658 (0.0670)
medium_gp
−0.278** (0.0871) 0.0134 (0.0760) −0.0987 (0.101) −0.159 (0.188)
medium_inn
0.308* (0.134) 0.114 (0.0836) 0.449*** (0.127) 0.676* (0.309)
medium_exp
−0.213 (0.124) 0.159 (0.0876) −0.132 (0.121) −0.00749 (0.173)
lnturn_exp
0.399 (0.873) 1.331* (0.606) 1.798 (0.920) −0.268 (1.486)
lnturnsq_exp
−0.0139 (0.0287) −0.0461* (0.0200) −0.0585 (0.0308) 0.00773 (0.0510)
inn_exp
0.299** (0.0959) 0.112 (0.0753) 0.368** (0.118) 3.770 (1116.9)
_cons
−0.981 (10.64) −7.275 (6.607) −8.152 (8.411)
_cons 0.865*** (0.0327) 0.452*** (0.0212) 0.542*** (0.0328) 0.794*** (0.0593)
N
31,983 52,604 15,698 13,576
Notes: Standard errors in parentheses. * p < 0.05, ** p < 0.01, *** p < 0.001.

Sustainability 2020, 12, 3846 15 of 25
Table A2. Seemingly unrelated probit regression estimation results for 2012.
North South EU2004 NewEU
National funding
kis
0.248*** (0.0564) 0.478*** (0.0329) 0.0355 (0.0785) −0.0297 (0.129)
lkis
−0.290*** (0.0638) −0.239*** (0.0311) −0.311*** (0.0770) −0.640*** (0.113)
mht
0.473*** (0.0619) 0.261*** (0.0425) 0.175 (0.0934) 0.216 (0.135)
mlt
0.165** (0.0511) 0.0859** (0.0294) −0.0699 (0.0642) −0.0800 (0.0934)
micro
3.706 (8.052) 7.408 (4.284) 0.379 (7.599) −2.443 (9.978)
small
4.093 (8.057) 7.267 (4.287) 0.371 (7.607) −2.437 (9.996)
medium
0 (.) 0 (.) 0 (.) 0 (.)
lnturn_b
−0.465 (0.545) 0.272 (0.316) 2.261* (0.886) 2.225 (1.303)
lnturn_b_sq
0.0134 (0.0183) −0.00800 (0.0106) −0.0791** (0.0302) −0.0799 (0.0458)
gp
−0.309*** (0.0330) 0.00743 (0.0270) −0.0558 (0.0619) −0.313** (0.117)
inngood
0.695*** (0.0294) 0.469*** (0.0223) 0.544*** (0.0441) 0.643*** (0.0626)
innserv
0.255*** (0.0325) 0.288*** (0.0238) 0.319*** (0.0480) 0.242** (0.0785)
innpro
0.448*** (0.0284) 0.531*** (0.0213) 0.657*** (0.0442) 0.777*** (0.0639)
innorg
0.164*** (0.0279) 0.114*** (0.0206) −0.0291 (0.0420) 0.0331 (0.0623)
exp
1.112 (4.628) −0.958 (2.740) 9.982 (6.382) 9.774 (8.835)
medium_kis
−0.0895 (0.0710) −0.223*** (0.0482) −0.0820 (0.100) −0.113 (0.153)
medium_mht
−0.0804 (0.0738) 0.0513 (0.0605) −0.176 (0.107) −0.341* (0.151)
medium_lnturn
0.550 (1.006) 0.908 (0.543) −0.0880 (1.000) −0.427 (1.384)
medium_lnturnsq
−0.0178 (0.0314) −0.0285 (0.0172) 0.00768 (0.0329) 0.0174 (0.0479)
medium_gp
−0.0749 (0.0613) −0.0355 (0.0433) −0.135 (0.0861) 0.247 (0.144)
medium_inn
0.137 (0.0776) 0.235*** (0.0454) 0.353*** (0.101) 0.323** (0.110)
medium_exp
0.0900 (0.0899) −0.0323 (0.0534) −0.181 (0.119) 0.0166 (0.135)
lnturn_exp
−0.0654 (0.614) 0.124 (0.362) −1.439 (0.857) −1.357 (1.219)
lnturnsq_exp
0.0000148 (0.0203) −0.00340 (0.0119) 0.0507 (0.0287) 0.0473 (0.0419)
inn_exp
0.347*** (0.0542) 0.319*** (0.0359) 0.611*** (0.0819) 0.217* (0.103)
_cons
−2.296 (8.468) −11.53** (4.300) −18.66* (8.060) −15.29 (9.582)
EU funding
kis
0.271*** (0.0770) 0.405*** (0.0592) 0.142 (0.0745) 0.0267 (0.150)
lkis
−0.428*** (0.0978) −0.174** (0.0583) −0.349*** (0.0750) −0.476*** (0.135)
mht
0.252** (0.0854) 0.114 (0.0743) 0.0653 (0.0916) 0.306* (0.154)
mlt
0.0569 (0.0708) −0.0420 (0.0535) −0.147* (0.0620) 0.172 (0.111)

Sustainability 2020, 12, 3846 16 of 25
micro
4.730 (10.39) 10.70 (7.060) 15.47* (7.735) 9.536 (11.72)
small
4.840 (10.40) 10.58 (7.067) 15.77* (7.742) 9.877 (11.73)
medium
0 (.) 0 (.) 0 (.) 0 (.)
lnturn_b
−1.666* (0.741) −0.817 (0.593) 2.004* (0.915) 0.642 (1.435)
lnturn_b_sq
0.0552* (0.0248) 0.0293 (0.0198) −0.0731* (0.0314) −0.0273 (0.0509)
gp
−0.167*** (0.0452) −0.0195 (0.0476) −0.204** (0.0624) −0.258* (0.126)
inngood
0.473*** (0.0400) 0.369*** (0.0356) 0.482*** (0.0430) 0.437*** (0.0674)
innserv
0.196*** (0.0429) 0.221*** (0.0369) 0.228*** (0.0471) 0.332*** (0.0838)
innpro
0.406*** (0.0390) 0.542*** (0.0375) 0.703*** (0.0428) 0.774*** (0.0692)
innorg
0.108** (0.0382) 0.180*** (0.0352) −0.0268 (0.0409) −0.135* (0.0675)
exp
−9.156 (6.302) −4.741 (4.765) 8.431 (6.509) −5.907 (9.557)
medium_kis
−0.0836 (0.0939) −0.162* (0.0759) −0.141 (0.0947) 0.142 (0.165)
medium_mht
−0.0642 (0.100) −0.104 (0.0979) −0.134 (0.105) −0.00217 (0.159)
medium_lnturn
0.715 (1.301) 1.452 (0.901) 1.938 (1.023) 1.191 (1.636)
medium_lnturnsq
−0.0258 (0.0407) −0.0488 (0.0287) −0.0583 (0.0338) −0.0354 (0.0570)
medium_gp
−0.120 (0.0824) −0.0693 (0.0712) −0.0447 (0.0849) 0.104 (0.157)
medium_inn
0.240* (0.122) 0.0923 (0.0849) 0.502*** (0.0953) 0.235* (0.116)
medium_exp
0.0743 (0.125) 0.103 (0.0907) −0.213 (0.117) 0.0504 (0.147)
lnturn_exp
1.281 (0.836) 0.787 (0.629) −1.198 (0.880) 0.852 (1.324)
lnturnsq_exp
−0.0435 (0.0277) −0.0298 (0.0207) 0.0422 (0.0296) −0.0306 (0.0457)
inn_exp
0.229** (0.0789) 0.112 (0.0680) 0.608*** (0.0761) 0.377*** (0.110)
_cons
5.048 (11.07) −7.867 (7.027) −31.56*** (8.333) −15.98 (10.49)
_cons
0.873*** (0.0289) 0.522*** (0.0217) 0.501*** (0.0282) 0.693*** (0.0452)
N
26,823 45,475 16,294 14,994
Notes: Standard errors in parentheses. * p < 0.05, ** p < 0.01, *** p < 0.001.

Sustainability 2020, 12, 3846 17 of 25
Table A3. National funding only vs. No funding: Verification of balancing, common support included.
North South EU2004 NewEU
2008 2012 2008 2012 2008 2012 2008 2012
Treated = 1539 Treated = 1874 Treated = 5988 Treated = 4372 Treated = 623 Treated = 576 Treated = 183 Treated = 290

a

b

c





















kis 0.3 0.3 1 0.29 0.29 1 0.2 0.2 1 0.25 0.25 1 0.21 0.2 0.9 0.22 0.22 1 0.24 0.24 1 0.17 0.17 1
lkis 0.47 0.47 1 0.05 0.05 1 0.14 0.14 0.98 0.14 0.14 1 0.1 0.1 1 0.11 0.11 1 0.12 0.12 1 0.07 0.07 1
mht
0.23 0.22 0.9 0.25 0.25 0.9 0.14 0.14 1 0.17 0.17 1 0.17 0.17 1 0.19 0.19 1 0.27 0.27 1 0.17 0.17 0.9
mlt
0.34 0.34 0.9 0.35 0.35 1 0.38 0.38 1 0.3 0.3 1 0.37 0.37 0.9 0.39 0.39 1 0.31 0.31 1 0.48 0.49 0.9
micro
0.06 0.06 1 0.02 0.02 1 0.06 0.06 1 0.05 0.05 1 0.01 0.01 1 0.03 0.03 1 0.4 0.4 1 0.39 0.39 1
small 0.54 0.54 1 0.69 0.69 0.9 0.6 0.6 0.91 0.57 0.57 1 0.44 0.45 0.7 0.48 0.48 1 0.37 0.39 0.9 0.39 0.41 0.7
medium 0.4 0.4 1 0.29 0.29 0.9 0.34 0.34 0.91 0.38 0.38 1 0.55 0.53 0.7 0.49 0.49 1 0.58 0.57 0.9 0.57 0.55 0.7
lnturn_b 15.3 15.3 1 15.26 15.27 0.8 15.19 15.19 0.73 15.35 15.35 0.9 14.98 14.92 0.4 15.05 15.05 0.9 14.29 14.34 0.8 14.72 14.66 0.6
lnturn_b_sq 236.29 236.11 0.9 234.24 234.55 0.8 232.42 232.15 0.69 237.18 237.2 1 225.82 223.78 0.3 228.24 227.92 0.9 205.56 206.71 0.8 218.1 216.07 0.6
gp 0.36 0.37 0.7 0.38 0.38 0.8 0.25 0.25 0.7 0.39 0.39 0.9 0.33 0.33 1 0.4 0.41 0.7 0.24 0.24 1 0.28 0.26 0.7
inngood
0.67 0.66 0.9 0.8 0.8 0.8 0.5 0.5 0.93 0.71 0.71 0.9 0.66 0.66 0.9 0.7 0.68 0.6 0.95 0.95 1 0.85 0.85 1
innserv
0.45 0.44 0.9 0.41 0.41 0.9 0.31 0.31 0.97 0.47 0.47 0.96 0.42 0.42 1 0.41 0.4 0.8 0.52 0.51 0.9 0.32 0.32 1
innpro 0.7 0.7 1 0.58 0.57 0.9 0.7 0.7 1 0.67 0.67 1 0.79 0.79 1 0.68 0.68 0.8 0.73 0.75 0.9 0.74 0.72 0.6
innorg 0.68 0.68 1 0.58 0.58 0.9 0.56 0.56 0.87 0.61 0.61 1 0.64 0.64 0.9 0.48 0.48 1 0.79 0.8 0.8 0.64 0.61 0.5
exp 0.8 0.8 1 0.78 0.78 1 0.59 0.59 0.99 0.71 0.71 0.9 0.68 0.69 0.9 0.81 0.82 0.9 0.57 0.59 0.9 0.78 0.76 0.7
Notes: a denotes means of treated observations. b denotes means of control observations. c denotes p-value of two-sided t-test for difference in means.

Sustainability 2020, 12, 3846 18 of 25
Table A4. National and EU funding vs. No funding: Verification of balancing, common support included.
North South EU2004 NewEU
2008 2012 2008 2012 2008 2012 2008 2012
Treated = 1539 Treated = 1874 Treated = 5988 Treated = 4372 Treated = 623 Treated = 576 Treated = 183 Treated = 290

a

b

c





















kis 0.2 0.2 1 0.4 0.4 1 0.41 0.41 1 0.4 0.4 1 0.24 0.24 1 0.25 0.25 1 0.32 0.32 1 0.29 0.29 1
lkis 0.14 0.14 1 0.02 0.02 1 0.12 0.12 1 0.11 0.11 1 0.04 0.04 1 0.04 0.04 1 0.08 0.08 1 0.02 0.02 1
mht
0.14 0.14 1 0.24 0.24 1 0.12 0.12 1 0.15 0.15 1 0.18 0.18 1 0.23 0.23 1 0.2 0.2 1 0.14 0.14 1
mlt
0.38 0.38 1 0.28 0.28 1 0.25 0.25 1 0.24 0.24 1 0.36 0.36 0.9 0.36 0.36 1 0.4 0.4 1 0.48 0.48 1
micro
0.06 0.06 1 0.01 0.01 1 0.05 0.05 1 0.05 0.05 1 0.01 0.01 1 0.02 0.02 1 0.04 0.04 1
small 0.6 0.6 1 0.67 0.67 0.9 0.51 0.51 0.9 0.53 0.53 1 0.37 0.37 1 0.4 0.41 0.8 0.32 0.3 0.8 0.42 0.38 0.7
medium 0.34 0.34 0.9 0.32 0.31 0.9 0.44 0.44 0.9 0.41 0.41 1 0.62 0.62 1 0.58 0.57 0.8 0.64 0.66 0.8 0.58 0.62 0.7
lnturn_b 15.2 15.2 0.7 15.28 15.29 0.9 15.2 15.2 0.9 15.25 15.24 0.9 15.05 15.06 1 15.05 15.04 1 14.34 14.27 0.7 14.55 14.57 0.9
lnturn_b_sq 232.42 232.15 0.7 234.97 235.21 0.9 232.57 232.27 0.9 233.72 233.92 0.9 227.78 227.78 1 227.68 227.39 0.9 206.69 204.53 0.7 212.5 213.13 0.9
gp 0.25 0.25 0.7 0.4 0.4 0.9 0.33 0.33 1 0.38 0.38 1 0.34 0.35 0.8 0.35 0.37 0.6 0.16 0.12 0.6 0.23 0.27 0.6
inngood
0.5 0.5 0.9 0.77 0.77 1 0.55 0.55 0.9 0.77 0.77 1 0.71 0.71 1 0.74 0.74 1 0.92 0.9 0.7 0.85 0.85 1
innserv
0.31 0.31 1 0.5 0.49 0.8 0.49 0.49 1 0.59 0.59 1 0.46 0.45 0.9 0.42 0.41 0.9 0.52 0.52 1 0.4 0.4 1
innpro 0.7 0.7 1 0.63 0.62 0.9 0.7 0.7 0.9 0.77 0.77 1 0.83 0.83 1 0.71 0.71 1 0.84 0.86 0.8 0.77 0.77 1
innorg 0.56 0.56 0.9 0.62 0.62 1 0.66 0.66 1 0.76 0.76 0.9 0.63 0.63 1 0.51 0.53 0.7 0.92 0.92 1 0.48 0.5 0.8
exp 0.59 0.59 0.9 0.87 0.86 0.9 0.65 0.65 1 0.78 0.78 0.9 0.81 0.82 0.8 0.87 0.87 0.9 0.52 0.52 1 0.85 0.81 0.6
Notes: a denotes means of treated observations. b denotes means of control observations. c denotes p-value of two-sided t-test for difference in means.

Sustainability 2020, 12, 3846 19 of 25
Table A5. National and EU funding vs. National funding only: Verification of balancing, common support included.
North South EU2004 NewEU
2008 2012 2008 2012 2008 2012 2008 2012
Treated = 298 Treated = 94 Treated = 465 Treated = 472 Treated = 221 Treated = 161 Treated = 47 Treated = 51

a

b

c





















kis 0.3 0.31 0.9 0.4 0.4 1 0.41 0.41 1 0.4 0.4 1 0.24 0.24 1 0.25 0.26 0.8 0.28 0.26 0.8 0.29 0.27 0.8
lkis 0.05 0.05 1 0.03 0.03 1 0.12 0.12 1 0.11 0.11 1 0.04 0.04 1 0.04 0.04 0.8 0.09 0.09 1 0.02 0.02 1
mht
0.22 0.22 1 0.24 0.24 1 0.12 0.12 0.9 0.15 0.15 1 0.18 0.17 0.9 0.22 0.22 0.9 0.21 0.23 0.8 0.12 0.1 0.7
mlt
0.34 0.34 0.9 0.28 0.28 1 0.25 0.26 0.9 0.24 0.24 1 0.36 0.37 0.7 0.36 0.36 1 0.43 0.43 1 0.49 0.53 0.7
micro
0.06 0.06 1 0.01 0.01 1 0.05 0.05 1 0.05 0.05 1 0.01 0.01 1 0.02 0.02 1 0.04 0.04 1 0 0
small 0.54 0.56 0.7 0.67 0.67 0.9 0.51 0.51 0.9 0.53 0.54 0.8 0.37 0.36 0.8 0.4 0.41 0.8 0.34 0.32 0.8 0.41 0.35 0.6
medium
lnturn_b 15.33 15.36 0.7 15.28 15.27 0.8 15.2 15.19 0.9 15.24 15.23 0.8 15.05 15.14 0.4 15.04 15.12 0.4 14.3 14.25 0.8 14.58 14.76 0.3
lnturn_b_sq 236.2 237.13 0.7 234.97 234.27 0.7 232.62 232.12 0.8 233.68 233.12 0.8 227.78 230.14 0.4 227.58 229.82 0.4 205.39 203.98 0.8 213.39 218.82 0.9
gp 0.36 0.35 0.8 0.4 0.39 0.7 0.33 0.32 0.9 0.38 0.37 0.7 0.34 0.33 0.8 0.35 0.34 0.8 0.17 0.15 0.8 0.24 0.24 1
inngood
0.66 0.68 0.6 0.77 0.78 0.7 0.55 0.55 0.9 0.77 0.77 0.9 0.71 0.71 1 0.74 0.74 0.8 0.91 0.94 0.7 0.84 0.9 0.4
innserv
0.45 0.45 1 0.5 0.49 0.8 0.49 0.48 0.9 0.59 0.59 0.9 0.46 0.44 0.7 0.41 0.36 0.2 0.49 0.49 1 0.39 0.33 0.5
innpro 0.7 0.7 1 0.63 0.62 0.9 0.7 0.7 0.9 0.77 0.77 1 0.83 0.84 0.8 0.71 0.7 0.9 0.83 0.87 0.5 0.76 0.8 0.6
innorg 0.68 0.66 0.7 0.62 0.63 0.7 0.66 0.67 0.8 0.76 0.77 0.8 0.63 0.64 0.7 0.51 0.49 0.6 0.91 0.94 0.7 0.47 0.57 0.3
exp 0.8 0.8 0.9 0.87 0.87 0.7 0.65 0.65 1 0.78 0.78 0.9 0.81 0.82 0.8 0.87 0.88 0.6 0.55 0.55 1 0.84 0.86 0.8
Notes: a denotes means of treated observations. b denotes means of control observations. c denotes p-value of two-sided t-test for difference in means.

Sustainability 2020, 12, 3846 20 of 25
Table A6. Rosenbaum bounds for the comparison on National funding only vs. No funding.
North South EU2004 NewEU
2008 2012 2008 2012 2008 2012 2008 2012
n = 1,341 n = 427 n = 5,984 n = 2,983 n = 466 n = 291 n = 75 n = 179
del_rdin_turn
Gamma sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig-
1.0 0 0 0 0 0 0 0 0 0 0 0.02 0.02 0 0 0 0
1.1 0 0 0 0 0 0 0 0 0 0 0.07 0 0 0 0 0
1.2 0 0 0 0 0 0 0 0 0 0 0.17 0 0 0 0 0
1.3 0 0 0 0 0 0 0 0 0 0 0.33 0 0 0 0 0
1.4 0 0 0 0 0 0 0 0 0 0 0.52 0 0 0 0.01 0
1.5 0 0 0.01 0 0 0 0 0 0 0 0.69 0 0 0 0.02 0
1.6 0 0 0.03 0 0 0 0 0 0.01 0 0.82 0 0.01 0 0.03 0
1.7 0 0 0.08 0 0 0 0 0 0.05 0 0.91 0 0.01 0 0.06 0
1.8 0 0 0.18 0 0 0 0 0 0.12 0 0.95 0 0.01 0 0.1 0
1.9 0.01 0 0.32 0 0 0 0 0 0.23 0 0.98 0 0.02 0 0.14 0
2.0 0.06 0 0.48 0 0 0 0 0 0.37 0 0.99 0 0.03 0 0.19 0
del_rdex_turn
Gamma sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig-
1.0 0 0 0 0 0 0 0 0 0 0 0 0 0.85 0.85 0 0
1.1 0 0 0 0 0 0 0 0 0 0 0.01 0 0.88 0.8 0 0
1.2 0 0 0 0 0 0 0 0 0 0 0.02 0 0.91 0.75 0 0
1.3 0 0 0 0 0 0 0 0 0 0 0.04 0 0.93 0.7 0 0
1.4 0 0 0 0 0 0 0 0 0 0 0.08 0 0.95 0.66 0 0
1.5 0 0 0 0 0 0 0 0 0 0 0.13 0 0.96 0.61 0 0
1.6 0 0 0 0 0 0 0 0 0.01 0 0.2 0 0.97 0.56 0 0
1.7 0 0 0 0 0 0 0 0 0.02 0 0.29 0 0.98 0.52 0.01 0
1.8 0.01 0 0 0 0 0 0 0 0.04 0 0.38 0 0.98 0.48 0.01 0
1.9 0.03 0 0 0 0 0 0 0 0.08 0 0.48 0 0.99 0.44 0.02 0
2.0 0.11 0 0 0 0 0 0 0 0.13 0 0.57 0 0.99 0.4 0.03 0

Sustainability 2020, 12, 3846 21 of 25
Table A7. Rosenbaum bounds for the comparison on National and EU funding vs. No funding.
North South EU2004 NewEU
2008 2012 2008 2012 2008 2012 2008 2012
n = 299 n = 94 n = 463 n = 472 n = 221 n = 161 n = 50 n = 52
del_rdin_turn
Gamma sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig-
1.0 0 0 0 0 0 0 0 0 0 0 0 0 0.8 0.8 0.15 0.15
1.1 0 0 0 0 0 0 0 0 0 0 0 0 0.72 0.87 0.21 0.10
1.2 0 0 0 0 0 0 0 0 0 0 0 0 0.64 0.91 0.26 0.07
1.3 0 0 0 0 0 0 0 0 0 0 0 0 0.56 0.94 0.33 0.05
1.4 0 0 0 0 0 0 0 0 0 0 0 0 0.48 0.96 0.39 0.03
1.5 0 0 0 0 0 0 0 0 0 0 0 0 0.41 0.98 0.45 0.02
1.6 0 0 0 0 0 0 0 0 0 0 0 0 0.34 0.98 0.50 0.02
1.7 0 0 0 0 0 0 0 0 0 0 0 0 0.29 0.99 0.56 0.01
1.8 0 0 0 0 0 0 0 0 0 0 0 0 0.24 0.99 0.61 0.01
1.9 0 0 0 0 0 0 0 0 0 0 0.01 0 0.19 1 0.65 0
2.0 0 0 0.01 0 0 0 0 0 0 0 0.02 0 0.16 1 0.69 0
del_rdex_turn
Gamma sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig-
1.0 0 0 0 0 0 0 0 0 0 0 0.02 0.02 0.01 0.01 0.49 0.49
1.1 0 0 0 0 0 0 0 0 0 0 0.05 0.01 0.01 0 0.55 0.43
1.2 0 0 0 0 0 0 0 0 0 0 0.09 0 0.01 0 0.61 0.38
1.3 0 0 0 0 0 0 0 0 0 0 0.16 0 0.02 0 0.66 0.33
1.4 0 0 0 0 0 0 0 0 0 0 0.23 0 0.02 0 0.7 0.29
1.5 0 0 0 0 0 0 0 0 0 0 0.32 0 0.02 0 0.73 0.25
1.6 0 0 0 0 0 0 0 0 0.01 0 0.41 0 0.03 0 0.77 0.22
1.7 0 0 0.01 0 0 0 0 0 0.02 0 0.50 0 0.03 0 0.80 0.19
1.8 0 0 0.01 0 0 0 0 0 0.04 0 0.58 0 0.03 0 0.82 0.17
1.9 0 0 0.02 0 0 0 0 0 0.06 0 0.66 0 0.04 0 0.84 0.15
2.0 0 0 0.03 0 0 0 0 0 0.09 0 0.73 0 0.04 0 0.86 0.13

Sustainability 2020, 12, 3846 22 of 25
Table A8. Rosenbaum bounds for the comparison on National and EU funding vs. National funding only.
North South EU2004 NewEU
2008 2012 2008 2012 2008 2012 2008 2012
n = 298 n = 94 n = 465 n = 472 n = 221 n = 161 n = 47 n = 51
del_rdin_turn
Gamma sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig-
1.0 0 0 0.07 0.07 0 0 0 0 0 0 0.01 0.01 0.33 0.33 0.01 0.01
1.1 0 0 0.15 0.04 0 0 0 0 0.01 0 0.02 0 0.23 0.43 0 0.02
1.2 0.01 0 0.25 0.01 0 0 0.01 0 0.04 0 0.06 0 0.16 0.53 0 0.03
1.3 0.05 0 0.37 0.01 0 0 0.06 0 0.11 0 0.12 0 0.11 0.62 0 0.04
1.4 0.14 0 0.49 0 0 0 0.19 0 0.22 0 0.22 0 0.07 0.70 0 0.06
1.5 0.28 0 0.60 0 0 0 0.40 0 0.36 0 0.34 0 0.05 0.77 0 0.09
1.6 0.46 0 0.70 0 0 0 0.63 0 0.52 0 0.47 0 0.03 0.82 0 0.12
1.7 0.63 0 0.78 0 0.01 0 0.81 0 0.66 0 0.59 0 0.02 0.87 0 0.15
1.8 0.77 0 0.85 0 0.04 0 0.92 0 0.78 0 0.70 0 0.01 0.90 0 0.19
1.9 0.87 0 0.89 0 0.09 0 0.97 0 0.86 0 0.79 0 0.01 0.92 0 0.23
2.0 0.94 0 0.93 0 0.20 0 0.99 0 0.92 0 0.86 0 0.01 0.94 0 0.27
del_rdex_turn
Gamma sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig- sig+ sig-
1.0 0.07 0.07 0.08 0.08 0 0 0 0 0.02 0.01 0.39 0.39 0.21 0.21 0.11 0.11
1.1 0.21 0.02 0.15 0.04 0.01 0 0.01 0 0.06 0 0.55 0.24 0.25 0.17 0.09 0.14
1.2 0.41 0 0.24 0.01 0.04 0 0.07 0 0.13 0 0.69 0.14 0.30 0.14 0.07 0.18
1.3 0.63 0 0.34 0.01 0.13 0 0.20 0 0.24 0 0.80 0.08 0.34 0.11 0.05 0.21
1.4 0.80 0 0.45 0 0.29 0 0.41 0 0.37 0 0.88 0.04 0.39 0.09 0.04 0.24
1.5 0.90 0 0.55 0 0.50 0 0.63 0 0.51 0 0.93 0.02 0.43 0.07 0.03 0.28
1.6 0.96 0 0.65 0 0.69 0 0.81 0 0.65 0 0.96 0.01 0.47 0.06 0.2 0.31
1.7 0.98 0 0.73 0 0.84 0 0.92 0 0.76 0 0.98 0 0.50 0.05 0.02 0.34
1.8 0.99 0 0.79 0 0.93 0 0.97 0 0.84 0 0.99 0 0.54 0.04 0.01 0.37
1.9 1 0 0.85 0 0.97 0 0.99 0 0.90 0 0.99 0 0.57 0.03 0.01 0.40
2.0
1
0
0.89
0
0.99
0
1
0
0.94
0
1
0
0.60
0.02
0.01
0.43

Sustainability 2020, 12, 3846 23 of 25
References
1. Nelson, R.R. The Simple Economics of Basic Scientific Research. J. Polit. Econ. 1959, 67, 297–306.
2. Arrow, K.J. Economic Welfare and the Allocation of Resources for Invention. In The Rate and Direction of
Inventive Activity: Economic and Social Factors; Nelson, R.R., Ed.; Princeton University Press: Princeton, NJ,
USA, 1962; pp. 609–626.
3. McCann, P.; Acs, Z.J. Globalization: Countries, Cities and Multinationals. Reg. Stud. 2011, 45, 17–32.
4. McCann, P.; Ortega-Argilés, R. Smart Specialization, Regional Growth and Applications to European
Union Cohesion Policy. Reg. Stud. 2015, 49, 1291–1302.
5. Czarnitzki, D.; Lopes Bento, C. Evaluation of Public R&D Policies: A Cross-Country Comparison. World
Rev. Sci. Tech. Sustain. Dev. 2012, 9, 254–282.
6. Hashi, I.; Stojčić, N. The Impact of Innovation Activities on Firm Performance Using a Multi-Stage Model:
Evidence from the Community Innovation Survey 4. Res. Policy 2013, 42, 353–366.
7. Aristei, D.; Sterlacchini, A.; Venturini, F. Effectiveness of R&D Subsidies During the Crisis: Firm-Level
Evidence Across EU Countries. Econ. Innov. N. Technol. 2017, 26, 554–573.
8. Carboni, O.A. The Effect of Public Support on Investment and R&D: An Empirical Evaluation on European
Manufacturing Firms. Technol. Forecast. Soc. Change 2017, 117, 282–295.
9. Orlic, E.; Radicic, D.; Balavac, M. R&d and Innovation Policy in the Western Balkans: Are There
Additionality Effects? Sci. Public Policy 2019, 46, 876–894.
10. Czarnitzki, D.; Lopes-Bento, C. Innovation Subsidies: Does the Funding Source Matter for Innovation
Intensity and Performance? Empirical Evidence from Germany. Ind. Innov. 2014, 21, 380–409.
11. Radicic, D.; Pugh, G. R&D Programmes, Policy Mix, and the ‘European Paradox’: Evidence from European
SMEs. Sci. Public Policy 2016, 44, 497–512.
12. Mulligan, K.; Lenihan, H.; Doran, J. More Subsidies, More Innovation? Evaluating Whether a Mix of
Subsidies from Regional, National and EU Sources Crowds Out Firm-Level Innovation. Reg. Stud. Reg. Sci.
2019, 6, 130–138.
13. Radosevic, S.; Auriol, L. Patterns of Restructuring in Research, Development and Innovation Activities in
Central and Eastern European Countries: An Analysis Based on S&T Indicators. Res. Policy 1999, 28, 351–
376.
14. Veugelers, R. The European Union’s Growing Innovation Divide; Policy Contributions Bruegel: Brussels,
Belgium, 2016.
15. Izsak, K.; Radošević, S. EU Research and Innovation Policies as Factors of Convergence or Divergence after
the Crisis. Sci. Public Policy 2016, 44, 274–283.
16. Spanos, Y.E.; Vonortas, N.S.; Voudouris, I. Antecedents of Innovation Impacts in Publicly Funded
Collaborative R&D Projects. Technovation 2015, 36, 53–64.
17. Luukkonen, T. Additionality of EU Framework Programmes. Res. Policy 2000, 29, 711–724.
18. Pavitt, K. The Inevitable Limits of EU R&D Funding. Res. Policy 1998, 27, 559–568.
19. Franco, C.; Gussoni, M. The Role of Firm and National Level Factors in Fostering R&D Cooperation: A
Cross Country Comparison. J. Technol. Transf. 2014, 39, 945–976.
20. Fritz, W. Determinants of Product Innovation Activities. Eur. J. Mark. 1989, 23, 32–43.
21. Sweeney, G.P. New Entrepreneurship and the Smaller Firm; Campus: New York, NY, USA, 1983.
22. Dvouletý, O.; Srhoj, S.; Pantea, S. Public SME Grants and Firm Performance in European Union: A
Systematic Review of Empirical Evidence. Small Bus. Econ. 2020, doi:10.1007/s11187-019-00306-x.
23. David, P.A.; Hall, B.H.; Toole, A.A. Is public R&D a complement or substitute for private R&D? A review
of the econometric evidence. Res. Policy 2000, 29, 497–529.
24. Cerulli, G. Modelling and Measuring the Effect of Public Subsidies on Business R&D: A Critical Review of
the Econometric Literature. Econ. Rec. 2010, 86, 421–449.
25. Zúñiga-Vicente, J.Á.; Alonso-Borrego, C.; Forcadell, F.J.; Galán, J.I. Assessing the Effect of Public Subsidies
on Firm R&D Investment: A Survey. J. Econ. Surv. 2014, 28, 36–67.
26. Almus, M.; Czarnitzki, D. The Effects of Public R&D Subsidies on Firms’ Innovation Activities: The Case
of Eastern Germany. J. Bus. Econ. Stat. 2003, 21, 226–236.
27. Czarnitzki, D.; Licht, G. Additionality of Public R&D Grants in a Transition Economy: The Case of Eastern
Germany. Econ. Transit. 2006, 14, 101–131.

Sustainability 2020, 12, 3846 24 of 25
28. Wallsten, S.J. The Effects of Government-Industry R&D Programs on Private R&D: The Case of the Small
Business Innovation Research Program. RAND J. Econ. 2000, 31, 82.
29. Lee, E.Y.; Cin, B.C. The Effect of Risk-Sharing Government Subsidy on Corporate R&D Investment:
Empirical Evidence from Korea. Technol. Forecast. Soc. Change 2010, 77, 881–890.
30. Cin, B.C.; Kim, Y.J.; Vonortas, N.S. The Impact of Public R&D Subsidy on Small Firm Productivity:
Evidence from Korean SMEs. Small Bus. Econ. 2017, 48, 345–360.
31. Czarnitzki, D.; Lopes-Bento, C. Value for Money? New Microeconometric Evidence on Public R&D Grants
in Flanders. Res. Policy 2013, 42, 76–89.
32. Hud, M.; Hussinger, K. The Impact of R&D Subsidies During the Crisis. Res. Policy 2015, 44, 1844–1855.
33. Karhunen, H.; Huovari, J. R&D Subsidies and Productivity in SMEs. Small Bus. Econ. 2015, 45, 805–823.
34. Mariani, M.; Mealli, F. The Effects of R&D Subsidies to Small and Medium-Sized Enterprises. Evidence
from a Regional Program. Ital. Econ. J. 2018, 4, 249–281.
35. Avellar, A.P.M. de; Botelho, M. dos R.A. Impact of Innovation Policies on Small, Medium and Large
Brazilian Firms. Appl. Econ. 2018, 50, 5979–5995.
36. Hagedoorn, J. Inter-Firm R&D Partnerships: An Overview of Major Trends and Patterns Since 1960. Res.
Policy 2002, 31, 477–492.
37. Aguiar, L.; Gagnepain, P. European Cooperative R&D and Firm Performance: Evidence Based on Funding
Differences in Key Actions. Int. J. Ind. Organ. 2017, 53, 1–31.
38. Maggioni, M.A.; Nosvelli, M.; Uberti, T.E. Space Versus Networks in the Geography of Innovation: A
European Analysis. J. Reg. Sci. 2007, 86, 471–493.
39. Di Cagno, D.; Fabrizi, A.; Meliciani, V. The Impact of Participation in European Joint Research Projects on
Knowledge Creation and Economic Growth. J. Technol. Transf. 2014, 39, 836–858.
40. Wanzenböck, I.; Scherngell, T.; Lata, R. Embeddedness of European Regions in European Union-Funded
Research and Development (R&D) Networks: A Spatial Econometric Perspective. Reg. Stud. 2015, 49, 1685–
1705.
41. Koišová, E.; Grmanová, E.; Habánik, J. Regional Disparities in Financing Innovations in Small and
Medium-Sized Enterprises. J. Int. Stud. 2018, 11, 124–136.
42. Schumpeter, J.A. Business Cycles: A Theoretical, Historical, and Statistical Analysis of the Capitalist Process;
McGraw-Hill Book Company Inc.: New York, NY, USA, 1939.
43. Freeman, C.; Clark, J.; Soete, L. Unemployment and Technical Innovation: A Study of Long Waves and Economic
Development; Pinter: London, UK, 1982.
44. Stiglitz, J. Endogenous Growth and Cycles; National Bureau of Economic Research: Cambridge, MA, USA,
1993.
45. Aghion, P.; Saint-Paul, G. Uncovering Some Causal Relationships Between Productivity Growth and the
Structure of Economic Fluctuations: A Tentative Survey. Labour 1998, 12, 279–303.
46. Bloom, N.; Bond, S.; Van Reenen, J. Uncertainty and Investment Dynamics. Rev. Econ. Stud. 2007, 74, 391–
415.
47. Rubin, D.B. Assignment to Treatment Group on the Basis of a Covariate. J. Educ. Stat. 1977, 2, 1–26.
48. Stuart, E.A.; Rubin, D.B. Best Practices in Quasi-Experimental Designs: Matching Methods for Causal
Inference. In Best Practices in Quantitative Social Science; Osborne, J., Ed.; Sage Publications: Thousand Oaks,
CA, USA, 2007; pp. 155–176.
49. Rosenbaum, P.R.; Rubin, D.B. The Central Role of the Propensity Score in Observational Studies for Causal
Effects. Biometrika 1983, 70, 41–55.
50. Radas, S.; Anić, I.-D.; Tafro, A.; Wagner, V. The Effects of Public Support Schemes on Small and Medium
Enterprises. Technovation 2015, 38, 15–30.
51. Abadie, A.; Imbens, G.W. Large Sample Properties of Matching Estimators for Average Treatment Effects.
Econometrica 2006, 74, 235–267.
52. Leuven, E.; Sianesi, B. Psmatch2: Stata Module to Perform Full Mahalanobis and Propensity Score Matching,
Common Support Graphing, and Covariate Imbalance Testing; Boston College Department of Economics:
Chestnut Hill, MA, USA, 2018.
53. Athey, S. Beyond prediction: Using big data for policy problems. Science 2017, 355, 483–485.
54. Weinberger, K.Q.; Saul, L.K. Distance metric learning for large margin nearest neighbor classification. J.
Mach. Learn. Res. 2009, 10, 207–244.

Sustainability 2020, 12, 3846 25 of 25
55. Ballestar, M.T.; Doncel, L.M.; Sainz, J.; Ortigosa-Blanch, A. A novel machine learning approach for
evaluation of public policies: An application in relation to the performance of university researchers.
Technol. Forecast. Soc. Change 2019, 149, 119756.
56. Rosenbaum, P.R.; Angrist, J.; Imbens, G.; Hill, J.; Robins, J.M.; Rosenbaum, P.R. Covariance Adjustment in
Randomized Experiments and Observational Studies. Stat. Sci. 2002, 17, 286–327.
57. DiPrete, T.A.; Gangl, M. Assessing Bias in the Estimation of Causal Effects: Rosenbaum Bounds on
Matching Estimators and Instrumental Variables Estimation with Imperfect Instruments. Sociol. Methodol.
2004, 34, 271–310.
58. Michalek, J.; Ciaian, P.; Kancs, D. Capitalization of the Single Payment Scheme into Land Value:
Generalized Propensity Score Evidence from the European Union. Land Econ. 2014, 90, 260–289.
59. Díaz-Chao, Á.; Sainz-González, J.; Torrent-Sellens, J. ICT, innovation, and firm productivity: New evidence
from small local firms. J. Bus. Res. 2015, 68, 1439–1444.
60. Caliendo, M.; Kopeinig, S. Some Practical Guidance for the Implementation of Propensity Score Matching.
J. Econ. Surv. 2008, 22, 31–72.
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).