Figure 2 - uploaded by Tomasz Kupiec
Content may be subject to copyright.
In our department/team we learn about implementation PROCESS from (% of answers "strongly agree" and "agree")*
Source publication
Public managers require different types of knowledge to run programs successfully. This includes knowledge about the context, operational know-how, knowledge about the effects, and causal mechanisms. This knowledge comes from different sources, and evaluation studies are just one of them.
This article takes the perspective of knowledge users. It e...
Context in source publication
Citations
... Policy evaluation is an applied process of inquiry for collecting and synthesizing evidence that results in conclusions about a policy's value and merit (Mathison, 2004). Remarkably, evaluation is "but one source of evidence" (Weiss et al., 2005) and does not often imply a reliable judgment (Olejniczak et al., 2017). This limitation must be recognized. ...
... First, they are essential for carrying out evaluation studies. Second, they stimulate policy learning via information sharing with relevant experts and stakeholders (Maybin, 2015;Olejniczak et al., 2016Olejniczak et al., , 2017. The internal capacity to use knowledge in the policy process interacts with the external pressure on policymakers to be accountable, especially for "relevant" policy under scrutiny (Raudla et al., 2018;Rimkutė, 2015;Schrefler, 2010). ...
... Therefore, the picture is quite mixed. The OECD appraised the methodological quality of evaluations (not restricted to CP) in different EU countries through its "Regulatory Policy Outlook" (OECD, 2021). ...
The European Union (EU), especially in the context of Cohesion Policy (CP), has played a crucial role in developing and promoting policy evaluation practices across its Member States. Evaluation systems across the Member States have been established to assess CP investments. Remarkably, the use of evaluation research and its contribution to stimulating policy learning has remained a “black box.” To address this issue, this article aims to develop a novel framework centered around four conditions for evaluation‐based policy learning, namely: (1) policy relevance, (2) resources and organizational settings, (3) quality of evaluation, and (4) evaluation culture. These conditions are retrieved from the existing literature on policy evaluation and applied to the six‐country cases across the EU. The findings suggest how loosening the formal EU evaluation requirements could affect policy learning in the Member States.
... This has not been a faster process because of the diversity of practices, skills, and expectations of evaluation providers and evaluators (Head, 2016). Moreover, political willingness also plays an important role (Olejniczak et al., 2017). Furthermore, if the quality of evaluations is low (e.g., Mastenbroek et al., 2016), politicians may be unwilling to consider the evaluation recommendations. ...
... This has not been a faster process because of the diversity of practices, skills, and expectations of evaluation providers and evaluators (Head, 2016). Moreover, political willingness also plays an important role (Olejniczak et al., 2017). Furthermore, if the quality of evaluations is low (e.g., Mastenbroek et al., 2016), politicians may be unwilling to consider the evaluation recommendations. ...
This engaging and topical book comprehensively explores the complexities surrounding the EU Cohesion Policy, which has been addressing regional and urban development across Europe since the 1980s. Adopting a multidisciplinary approach, it not only considers the goals of this long-term investment policy, which is to reduce territorial disparities between Member States and their regions, but also considers the role it plays in the European integration process and the challenges the EU will face in its future.
... The findings revealed a positive association between evaluation and project outcomes. Olejniczak, Kupiec and Newcomer (2017) also indicated that learning from evaluation results considerably increased the efficacy and efficiency of project performance. They argued that evaluation practices may aid organizations in learning from past mistakes and enhancing the success of upcoming projects. ...
... Implying that the best way to assessed whether a particular project has met its intended purpose is through evaluation, hence the need for managers to ensure that evaluation activities are done at the right time with the appropriate tools to prevent the project activities from deviating from it intended purpose. The findings are also consistent with the findings of Zhang and Yang (2018), who discovered that incorporating evaluation practices into the project management process increased project success rates, Blackwood et al. (2018), also revealed a positive association between evaluation and project outcomes, likewise, Olejniczak, Kupiec and Newcomer (2017) who indicated that learning from evaluation results considerably increased the efficacy and efficiency of project performance Also, the study simple slope analysis reveal an interaction between business environment, monitoring practices and project outcome, however, it was not statistically significant. This could be justified by the fact that irrespective of the actors in the business environment, tech start-ups continue to practice their monitoring activities. ...
Issues relating to Monitoring and Evaluation (M&E) have been established as a key and fundamental tool for the successful implementation of projects regardless of the industry. The study therefore sought to address the following questions: what effect do monitoring and evaluation practices have on tech start-ups project outcomes, as well as the role that business environment play in the relationship between M&E and project outcomes. The study followed a positivist mind-set, relying only on quantitative methods and an explanatory research design. Primary data via structured questionnaire was obtained from 317 respondents in managerial positions in the tech industry and analysed using inferential and descriptive tools. The study found that monitoring practices had a positive significant effect on project outcome. Evaluation practices also had a positive significant effect on project outcome. Business environment was found to have a dampening significant moderating effect in the relationship between evaluation practices and project outcome. However, business environment did not have any significant effect in the relationship between monitoring practice and project outcome. These findings will enable project practitioners understand the dynamics of monitoring and evaluation and the business environment when it comes to project execution. It will further enable project managers, personnel, and donors recognize how significant M&E tools are when creating policies and managing performance. Moreover, tech start-ups should create policies that recognize the integration of M&E in their operations and business functions.
... Four potential functions of the evaluation system. Source: own elaboration based onOlejniczak et al. (2017). ...
Evaluation practice is vital for the accountability and learning of administrations implementing complex policies. This article explores the relationships between the structures of the evaluation systems and their functions. The findings are based on a comparative analysis of six national systems executing evaluation of the European Union Cohesion Policy. The study identifies three types of evaluation system structure: centralized with a single evaluation unit, decentralized with a coordinating body and decentralized without a coordinating body. These systems differ in terms of the thematic focus of evaluations and the targeted users. Decentralized systems focus on internal users of knowledge and produce mostly operational studies; their primary function is inward-oriented learning about smooth programme implementation. Centralized systems fulfil a more strategic function, recognizing the external audience and external accountability for effects.
Points for practitioners
Practitioners who design multi-organizational evaluation systems should bear in mind that their structure and functions are interrelated. If both accountability and learning are desired, the evaluation system needs at least a minimum degree of decentralization on the one hand and the presence of an active and independent coordination body on the other.
La pratique de l’évaluation est essentielle pour la responsabilisation et l’apprentissage des administrations qui mettent en œuvre des politiques complexes. Cet article explore les relations entre les structures des systèmes d’évaluation et leurs fonctions. Les conclusions sont basées sur une analyse comparative de six systèmes nationaux chargés d’évaluer la politique de cohésion de l’Union européenne. L’étude identifie trois types de structures de système d’évaluation : centralisées avec une seule unité d’évaluation, décentralisées avec un organe de coordination et décentralisées sans organe de coordination. Ces systèmes diffèrent en termes d’orientation thématique des évaluations et d’utilisateurs ciblés. Les systèmes décentralisés se concentrent sur les utilisateurs internes des connaissances et produisent principalement des études opérationnelles ; leur fonction principale est l’apprentissage orienté vers l’intérieur pour une mise en œuvre harmonieuse du programme. Les systèmes centralisés remplissent une fonction plus stratégique, et tiennent compte du public externe et de la responsabilité externe des effets. Remarques à l’intention des praticiens Les praticiens qui conçoivent des systèmes d’évaluation multi-organisationnels doivent garder à l’esprit que leur structure et leurs fonctions sont interdépendantes. Si l’on vise à la fois l’imputabilité et l’apprentissage, le système d’évaluation a besoin d’un degré minimum de décentralisation d’une part, et de la présence d’un organe de coordination actif et indépendant d’autre part.
