ArticlePDF Available

On the Issue of the Social Effectiveness of Expert Assessments of Large-Scale Regional Transport Projects

Authors:
490
ISSN 2079-9705, Regional Research of Russia, 2021, Vol. 11, No. 4, pp. 490–501. © Pleiades Publishing, Ltd., 2021.
Russian Text © The Author(s), 2021, published in Region: Ekonomika i Sotsiologiya, 2021, No. 1 (109), pp. 72–96.
On the Issue of the Social Effectiveness of Expert Assessments
of Large-Scale Regional Transport Projects
S. A. Bykadorova, * and E. B. Kibalovb, **
a Siberian Transport University, Novosibirsk, 630049 Russia
b Institute of Economics and Industrial Engineering, Siberian Branch, Russian Academy of Sciences,
Novosibirsk, 630090 Russia
*e-mail: byser@ngs.ru
**e-mail: kibalovE@mail.ru
Received May 10, 2021; revised June 29, 2021; accepted July 22, 2021
Abstract—The article shows how game theory methods and expert technologies make it possible, within Kor-
nai’s systemic paradigm, use a specially developed toolkit for comparing the social effectiveness of large-scale
projects in a situation of radical (nonstochastic) uncertainty. The proposed approach combines the ideas of
the well-known PATTERN methods, “cost-effectiveness,” analysis of Saaty hierarchies with some original
solutions by the authors of the article. Computer products developed on this basis are proposed in support of
making complex investment decisions. As an example of practical application, experimental calculations are
given to assess the comparative social effectiveness of the Transpolar and Lena–Kamchatka railways.
Keywords: systemic anlysis, Kornai’s paradigm, large-scale project, public efficiency, decision making the-
ory, regional transport projects, Lena–Kamchatka railway mainline, Transpolar railway mainline, project
design, JSC Russian Railways
DOI: 10.1134/S2079970521040031
INTRODUCTION
Large-scale investment projects (megaprojects)
categorized as singular and unique in the 20th century
are at the beginning of the 21st century becoming a
phenomenon symbolizing the advent of the “era of
megaprojects.”1 As a consequence, a question arises
about their social (socioeconomic) effectiveness. In
standard methods (Russian and Western) based on the
discounted cash flow model, social effectiveness is
defined as the “costs and results associated with
implementation of the project” that go “beyond the
direct financial interests of the participants in the
investment project.”2 This refers to assessment of the
social and environmental impact of the project and the
costs associated with “social activity and environmen-
tal protection … [which] should be minimized.”
In the case of large-scale projects, the problem-
based situation with methods for assessing their social
effectiveness becomes more complicated. The fact is
that these projects, as demonstrated back in the 20th
century in works by Russian economists, A.L. Lurye
(1965) and V.N. Bogachev (1999), by the fact of their
implementation influence the rates and proportions of
the host economy. Consequently, they also influence
the system of parametric prices and estimates, which
are used to substantiate the social effectiveness of these
projects. The indicated “vicious circle” does not even
generate probabilistic, but fundamental (radical)
uncertainty, which the social sciences cannot yet
unlock.
DISCUSSION OF THE PROBLEM
We consider large-scale railway projects (LRP) at
the stage of structuring their design concepts3 as
investment and construction, aimed at creating tangi-
ble assets for the country’s basic infrastructure indus-
try. As such projects, we analyze two LRP, the Trans-
polar (TPM) and Lena–Kamchatka (LKM) railway
mainlines, which are planned for Siberia and the Far
East in the long term.
The difficulty of the problem lies in its systemic
complexity: the problem has not yet been formulated
with the necessary rigor, even theoretically. The lack
of a corresponding theory leads the “owners of the
problem” to use formula methodologies when making
difficult investment decisions. For example, corpora-
tions (Gazprom, Russian Railways, etc.) create their
1The era of megaprojects. https://www.ukrrudprom.com/
digest/Epoha_megaproektov.html.
2The concept of commercial, budgetary, and public effectiveness
of investment projects. https://port-u.ru/titul/2555-ponyatie-
effektivnosti-investitsionnykh-proektov.
SPATIAL FEATURES OF SECTORAL DEVELOPMENT
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
ON THE ISSUE OF THE SOCIAL EFFECTIVENESS OF EXPERT ASSESSMENTS 491
own methods, which represent exotic combinations of
computerized logical–heuristic, descriptive and nor-
mative economic and mathematical models for assess-
ing large-scale projects, which discard all projects dis-
advantageous for the corporation without due regard
to their social effectiveness (Kibalov and Kin, 2016).
To understand, as R.L. Ackoff said, this “mess of
problems,” today interdisciplinary systemic anlysis no
longer suffices: a traditional tool for revealing the
uncertainty in the assessment of large-scale projects in
general and railway projects in particular. As shown by
(Kornai, 1998), a new systems paradigm is required
that focuses on unique events associated with major
changes and profound transformations of differently
arranged social systems. The multidimensional conse-
quences of such impacts should be analyzed with
interparadigm positions ranging from the Marxist sys-
temic paradigm to the liberal paradigm of the modern
“mainstream.” We project Kornai’s systemic para-
digm onto the problem of interest to us, and the pro-
jection itself, following (Kleiner, 2004), we call the
neosystemic approach.
Analysis of literary sources on this issue did not
reveal any single feature that distinguishes a special
class of large-scale investment projects from a variety
of investment projects. Nevertheless, the most com-
mon, key characteristic in international comparisons
(Kibalov and Nekhoroshkov, 2018) is considered to be
the volume of capital costs for their implementation. It
is clear that such an assessment is relative, since a
large-scale project, e.g., in Spain will be small-scale by
China’s standards. For this reason, we consider a
large-scale (railroad) project in Russia to be one
whose implementation significantly affects the rates
and proportions of development of the Russian econ-
omy and, consequently, the cost system for products
and resources, including investment ones, that devel-
oped in the economic system before its introduction
into the new project. We begin to determine the signif-
icance of the influence based on well-known domestic
precedents. Thus, the costs for construction of the
Baikal–Amur Mainline (BAM) for the launch com-
plex, determined in prices of the corresponding years,
were approximately RUB 17.7 bln, which was 1.77% of
3For more details on what we mean in the article by the design
intent in relation to LRP, see the article “Putin approved the
start of design and survey work for the Power of Siberia-2 gas
pipeline” (https://tass.ru/ekonomika/8097265?utm_source=
yxnews&utm_medium=desktop&utm_referrer=https%3A%2-
F%2Fyandex.ru%2Fnews). In this article, the stage of develop-
ment of the design concept of the large-scale Gazprom project
“Power of Siberia-2” is determined: “A preliminary feasibility
study has been carried out, which gave a positive answer that it is
expedient, and cost-effective. Accordingly, please assign the
transition to the preinvestment stage: the development of a fea-
sibility study and the start of design and survey work for the
Power of Siberia-2 gas pipeline.” Our term “stage of structuring
of the design concept” corresponds to the term “preliminary
technical and economic analysis” (used by decision makers in
relation to LRP).
the level of the national income of the USSR in 1990.4
According to the latest version of Russian Railways,
the cost of the Materik–Sakhalin railway project will
amount to RUB 0.541 tln, which is equal to 0.49% of
Russia’s GDP in 2019.5 It it falls within the range of
0.50–2.0% of GDP of the corresponding years char-
acterizing the cost of a particular railway project, we
will consider it a criterion for classification as large-
scale.
The empirically proposed classification imparts
structure to the problem-based situation, but it does
not solve the problem of assessing the social effective-
ness of the LRP in full. As mentioned above, when
evaluating large-scale projects, a certain vicious circle
arises: The problem owner must assess the social
effectiveness of the LRP, the level of which is inf lu-
enced by the LRP itself. There is a so-called “veil of
uncertainty,” with no other ways to push past it and
perform the main function of “normal” science
(Kuhn, 1970)—a reliable prediction of the future. The
veil can only be lifted with the help of experts, and
although their assessments are, strictly speaking, sub-
jective and unverifiable, expert judgments contain
plausible information (Larichev and Moshkovich,
1996). V.N. Bogachev (1999) pointed out this possibil-
ity for the Baikal–Amur Mainline, a classic large-
scale railway project. It was also emphasized that it is
not purely economic and requires policy decisions.
It follows from the abovesaid that the problem-
based situation with assessing the social effectiveness
of large-scale investment projects, including railroad
projects, is poorly structured (in the sense defined by
G. Simon (1958)) and tends to become unstructured
under the influence of various kinds of turbulence
shaking the modern world: geopolitical, political,
technological, economic, and social. Despite the con-
ceptual (theoretical) difficulties, one is forced to eval-
uate large-scale long-term projects in order to form
survival strategies in permanently occuring develop-
ment crises.
From the authors’ viewpoint, an adequate response
to the external threats that have arisen is the strategic
necessity to construct the TPM and LKM as dual-
purpose railways: not only socioeconomic—the devel-
opment of the resource potential of the Arctic shelf
and Kamchatka—but also military-strategic. In the
latter case, these railways are considered land-based
lateral lines of communications parallel to potential
aquatorial theaters of operations. Lateral lines of com-
munications, their construction and operation in
inhospitable climatic conditions in sparsely populated
areas are extremely expensive measures. Therefore, for
4Russia’s GDP by years: 1991–2021. http://global-finances.ru
vvp-rossii-po-godam/.
5Kommersant has specif ied the cost of the Sakhalin Bridge based
on Russian Railways estimates. https://f inovosti.ru/news/poli-
tika/kommersant-utochnil-stoimost-sakhalinskogo-mos ta-na-
osnove-otsenok-rzhd.
492
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
BYKADOROV, KIBALOV
all importance of the TPM and LKM in ensuring
national security in conditions with limited invest-
ment resources, the projects cannot be implemented
simultaneously, although such a response to Russia’s
geopolitical opponents would be most desirable. It is
necessary, based on the criterion of comparative social
effectiveness, to choose one project—the most prefer-
able— that guarantees the reliability of strategic posi-
tions in potential theaters of operations.
In so doing, a qualitatively different problem arises
in assessing the social effectiveness of TPM and LKM
LRPs than one solved using the traditional methods
mentioned above. These methods solve the problem of
one person (the problem owner) choosing the best
investment project (project option): the owner knows
the objective function, instrumental variables, and
constraints (Intriligator, 1975), given deterministically
or probabilistically. Our problem is formulated as the
choice of a socially effective solution by two partici-
pants: the state corporation Russian Railways (acting
as the Player-customer and future operator of the con-
structed lines) and the Player-investor (the Russian
economy), whose behavior (development scenarios) is
identified by the behavior of nature, which is uninten-
tional, but poorly predictable. Players make interde-
pendent decisions based on expert information, and
the objective function in this case is called the pay-
ment function.
This is, in general outline, the meaningful formula-
tion of the problem considered in the article. Below,
the proposed solution process is described step by step,
starting from the stage of quantifying the assessment
problem.
QUANTIFICATION OF THE PROBLEM
OF ASSESSING TPM AND LKM
We understand quantification at the stage of form-
ing the design concept of these projects in strict accor-
dance with the theory as a measurement, using expert
technologies (including visualization), of the qualita-
tive characteristics of competing LRPs with their sub-
sequent placement into quantitative categories, i.e.,
with allocation of numerical values to the qualitative
characteristics.6 The same requirements apply to the
quantification of scenarios for the development of the
external environment of the projects (the Russian
economy in the global context). Institutional theory in
both cases prescribes the inclusion of a long history of
the discussion and implementation of the projects and
scenarios.
CASE PROJECTS: TRANSPOLAR AND
LENSKO-KAMCHATSKAYA RAILWAYS
Figure 1 shows the sections of the Stalin’s con-
struction project of the TPM railway in different states
6Quantification. https://btimes.ru/dictionary/kvantifikatsiya.
(1946–1953). Today, these sections are included in
the North Latitude Route-1 (NLR-1) project with
NLR-2 added to it, understood as the development of
the ObskayaBovanenkovo meridional (operating)
railway line during its completion to the ports of Kha-
rasavey and Sabetta (Fig. 2).7
The LKM (Fig. 3) is less renowned than the TPM
railway. However, in the 1930s, the surveyors and
designers of the Bamzheldorproekt (BaikalAmur
Mainline Railway Project) considered the LKM a
strategic competitor to the BAM. The LKM, if built,
would potentially connect the potentially largest ice-
free port in the East of the USSR—Petropavlovsk-
Kamchatsky—with the general Russian rail network,
thereby improving transport support for northern
deliveries and beyond. There would be one more out-
let to the Pacific coast and farther, to the main ports of
the western coast of the United States and Canada, as
well as of Southeast Asia and, as a result, the capacious
Asian markets. The implementation of the LKM
today will provide a reliable transport connection for
the currently hard-to-reach areas of the Far East. This
is especially important for Kamchatka, potentially the
richest region of the country, which today has mostly
only sea and air connections with the mainland. The
LKM could ultimately generate modern industrial and
agricultural output around the railway and signifi-
cantly increase the economic potential of northeastern
Russia8.
In particular, according to one of the options, the
LKM passes along the coast of Penzhin Bay of the Sea
of Okhotsk, where there are high sea tides. A project is
being developed for what would be the world’s largest
tidal hydroelectric power plant (HPP)—the Penzhin—
with a capacity of 87 GW and electricity generation of
about 190 TW t h–1 yr–1.9 For construction of the sta-
tion, the northern part of Penzhin Bay is proposed to
be cordoned from the sea by a dam. In this variant, the
railroad can pass along this dam.
There are also more optimistic forecasts.10 Special-
ists of HPP project believe that in Penzhin Bay, not
just one, but two large tidal power plants can be built.
In aggregate, the maximum capacity of the Penzhin
HPP in this case may be up to 135 GW, 60% of the
installed capacity of all power plants in Russia in 2012.
Implementation of the project will make it possible
to organize economically efficient hydrogen produc-
7“We are reformatting our business support system.” Deputy
Governor of the Yamalo-Nenets Autonomous Okrug on the
implementation of infrastructure projects and development of
small business. https://www.kommersant.ru/doc/4134542.
8http://www.ilovekamchatka.ru/topic/308-zheleznaya-doroga-
na-kamchatku/.
9Penzhin tidal HPP: state of the project and prospects.
https://fb.ru/article/385430/penjinskaya-pes-sostoyanie-
proekta-i-perspektivyi.
10About the most powerful power plant in the world.
https://is2006.livejournal.com/134245.html.
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
ON THE ISSUE OF THE SOCIAL EFFECTIVENESS OF EXPERT ASSESSMENTS 493
tion in Kamchatka. The hydrogen will then be bound
by a carbon-containing substance in order to obtain
liquid fuel, and in the future, advances in hydrogen
energy technologies will make it possible to use hydro-
gen in its pure form. Nonenergy use of Kamchatka
hydrogen may also be of great importance: ammonia
synthesis (about half the world’s hydrogen produc-
tion), hydrogenation and hydrotreatment, hydroc-
racking, methanol synthesis, petrochemical synthesis,
various chemical industries, metallurgy, etc. The vol-
ume of hydrogen produced in Kamchatka may reach
30 bln cm3 per year, from which a further 10 mln t of
liquid fuel can be obtained. This would provide artifi-
cial fuel for more than 8 mln passenger cars in the Far
East and neighboring countries. The total income
from the sale of environmentally friendly synthetic fuel
at today’s world prices would be at least USD 10 bln
per year.
Scenarios for the development of the external envi-
ronment of projects. As follows from Figs. 1–3, assess-
ing the effectiveness of the projects under consider-
ation requires account of interactions with other large-
scale projects of the eastern range of the Russian rail-
way network. The requirement makes the assessment
model situational, sensitive to poorly predicted fluctu-
ations of the external environment, the development
scenarios of which are uncertain and multilayered,
since they are included not only in the domestic Rus-
sian, but also in the global economic context.
Indeed, the “last miles” (Bykadorov and Kibalov,
2018) of the TPM and LKM enter the global transport
market, the behavior of which, especially in recent
years, is at best spontaneous, and at worst discrimina-
tory for Russia. This means that when assessing the
comparative social effectiveness of the TPM and
LKM, there is endogenous, largely improbable, uncer-
tainty.
The internal Russian environment of projects con-
trolled by the state regulator is more stable. Unfortu-
nately, since 2003, when JSC Russian Railways was
created instead of the Ministry of Railways of the Rus-
sian Federation, its regulations often create additional
uncertainty. Here we are faced with the institutional
aspect of the problem of choosing and implementing
the most preferable project by the problem owner,
Russian Railways, a state corporation and a natural
monopoly. The key here are two indicators that char-
acterize the scenarios for the development of the exter-
nal environment of LRP throughout their life cycle:
railway fees and loading volumes. These scenarios can
be contrasting, and below we consider the classic
three: optimistic, most probable, and pessimistic sce-
narios. In their makeup, administratively established
fees are what the railway transport “communicates”
with other sectors of the economy and what is
included in the input–output balance in the System of
National Accounts of the Russian Federation. The
current balance of revenues and expenses, ensuring
Fig. 1. Sections of Transpolar Mainline from Chum station to Igarka.
Source: https://fishki.net/1567645-salehard-igarka—doroga-smerti.html.
65°70°
70°
68°
66°
70°
68°
66°
64°
2006 Mar 6 14:33:43 OMC—Mertin Weinelt
0 50 100
km
GMT
75°80°85°
65°70°75°80°
Operating sections
Abandoned sections
Unfinished sections
Other active roads
Dudinka
Igarka
Ermakovo
Yamburg
Chum
Polui
Nadym
Pangody Novy Urengoy
Urengoy Dolgy Yakov Stan
Labytnangi
Salekhard
Kharp
Polar Circle
Yenisei River
Taz River
Nadym River
Pur River
Norilsk
494
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
BYKADOROV, KIBALOV
break-even and financial stability of Russian Rail-
ways, strongly depends not only on the total volume of
traffic, but also on the structure of the turnover of
transported goods. The change in the structure of
freight turnover towards low-profit cargoes leads, in
conditions requiring that a breakeven monopoly be
maintained, to a necessary increase in fees outstrip-
ping inf lation. This, in turn, increases the unpredict-
ability of changes in carriage fees for individual cargo
due to the dependence on these changes on traffic vol-
umes of other goods (other shippers). The redistribu-
tion of the fee load and risks of underutilization of the
newly built infrastructure for all transported goods
leads to a decrease in the responsibility of individual
shippers who are directly interested in the develop-
ment of railway infrastructure.
Thus, pairing of the current rail fee system and cur-
rent traffic volume creates artificial incentives for
Fig. 2. Chum–Igarka segment of the TPM as part of Northern Latitudinal Railway project.
Source: https://yandex.ru/images/search?text=Участок%20Чум%20-%20Игарка&stype=image&lr=65&source=wiz&p=
10&p os=320&rp t=s ima ge&img_url=ht tps %3A %2F%2Fwww.securitymedia.ru%2Fp ic%2Finnovation%2 FIn et_01.png .
Product pipelines
Bridges
Combined bridge Ore fields
Oil and gas
fields
Electricity grid
crossing over
Ob River
Electric
power plants
Combined bridge
crossing over
Nadym River
Geological objects
Planned
Railways Highways
Planned
JSC Russian
Railways
Planned
Active
JSC Gazprom
Northern Sea Route
transport corridor
Kharasavey
Bovanenkovo
Vorkuta
Labytnangi
Cape Kamenny
Yamburg
Nadym
Novy Urengoy
Korotchaevo
Yekaterinburg
Tyumen
Surgut
Samotlor
West Siberian railway
Konosha–Chum–Labytnangi
Polyarnaya CCGT TPP 268 MW
Noyabrskaya CHPP 124 MW
Terelskaya TPP 1200 MW
Novourengoyskaya regional
electric power plant 450 MW
Nyaganskaya regional electric
power station 1200 MW
Pripolyarnaya regional electric
power plant 122 MW
Tarko-Salinskaya TPP 1200 MW
Nadym GTU TPP 90 MW
Severo-Sosvinskaya regional electric
power plant 1200 MW
Salekhard
Sabetta
Lekyn–Talbeyskaya
area (copper-molybdenum
ores)
Voishorskoe
deposit
(barite ores)
Southern ore field
Southwest ore field
(chrome ores)
Yany-Turinsky area
(iron ores)
Otorinsky area
(brown coal)
Severo-Sosvinsky
area
(copper-zinc ores)
Laptapai
ore field
(chrome ores)
Yamal
Peninsula
Murmansk
Ports of Northwest America
and Southeast Asia
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
ON THE ISSUE OF THE SOCIAL EFFECTIVENESS OF EXPERT ASSESSMENTS 495
unjustified investment, in particular, investments in
large-scale projects for the construction of new lines
that are interregional and intersectoral in their eco-
nomic and social purpose.
In terms of aggregate risks characteristic of the
external environment of projects within Russia, the
uncertainty of obtaining the desired effects from an
LRP is fundamental. Neither modern economic the-
ory nor the applied economics of railway transport at
present can “reveal” such uncertainty to the end. In
accordance with Kornai’s systemic paradigm, we
must, in intellectual honesty, admit this and perform
experiments, if impossible at full scale, then at least
computational. We understand that the fundamental
uncertainty in evaluating large-scale projects in the
current state of science can be revealed mainly using
Fig. 3. LenaKamchatka Mainline as part of Strategy 2030.
Source: ru / images / search? pos = 56. https://yandex.ru/images/search?pos=568&p=18&img_url=https%3A%2F%2Flo-
cal.kam24.ru%2Fuploads%2Fblog%2F164.jpg&text=Plot%20Plag % 20as% 20part% 20project% 20North% 20Latitudinal%
20walk & lr = 65 & rpt = simage & source = wiz.
Construction of new railway lines until 2030
Construction of new railway lines after 2030
Proposed option of LKM
Strategic Cargo-generating
and technological
Strategic
Stations between which distances are indicated in kilometers
Railways of JSC Russian Railways
Railways of other departments
Rail ferry crossing
LKM
Cargo-generating
Socially significant
100
450
50
103
87
50
305
90
246
311
498
258
385
370
428
570
450
600
517
221
563
269
58
19
373
354
105
42
249
785
40
30
110 0
373
22
156
110
19
65
313
Uelen
Provideniya
Egvekinot
Markovo
Ust-Oloy
Nizhneyansk Ugolnoye Zyryanka
Verkhnekolymsk Palana
Petropavlovsk-
Kamchatsky
Moma
Ust-Nera
Megino-Aldan Khandyga
Yakutsk
Nizhny Bestyakh
Kerdem
Mirny Kempendyai
Lensk Olekminsk
Tommot
Elginskoe
Nepa Bodaibo Khani BerkakitUpak
Apsatskoe
field
Pravaya Lena
Dzhebarinkaya
Kangalasy
Susuman
Magadan
Klyuchi
Ust-Kamchatsk
Ossora
Omsukchan
Mouth of Belaya
River
Chokurdakh
Chersky
Dachny
Anyui
Anadyr
Pevek
Amguema
Ionyweem
Tura
Khatanga
Tiksi
496
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
BYKADOROV, KIBALOV
logical–heuristic models based on expert judgments.
Mathematical models are also important, but in rela-
tion to the former, they should play the role of main-
tenance and servicing. It is this position of Kornai’s
paradigm that we rely on, proposing below a method-
ology for assessing the comparative social effectiveness
of the TPM and LKM LRPs at the stage of discussing
their design concepts.
Hybrid pricing model. Having explained at the ver-
bal level in the previous scenarios the nature of the
projects for the construction and subsequent opera-
tion of the TPM and LKM, we now formulate at the
stage of “preliminary feasibility studies” a structural
model for assessing their comparative social effective-
ness under the following prerequisites:
(a) the projects have no competing investment
alternatives;
(b) the costs for the implementation of competing
projects are approximately the same;
(c) the level of the funding limit for projects is such
that only one of them can be implemented;
(d) uncertainty in assessing the situation and
choice of the most preferable project arises due to the
actualization of contrasting scenarios of development
of the external environment of the assessed projects.
All other assumptions, conditions, and features of
projects and scenarios are either formulated in the pre-
vious sections of the article or will be introduced
below.
The model for assessing the comparative social
effectiveness of competing LRPs, together with system
of disciplining assumptions (a–d) is understood by us
as a tool for revealing uncertainty at the stage of dis-
cussing the project aims and investment aims of the
problem owners. The novelty of the tool is, of course,
relative. Methodologically, the tool assumes, as in tra-
ditional systemic analysis, the use of the LRP goal
tree—a semiformal model—as a basic tool for reveal-
ing the target assessment uncertainty. It is the semifor-
mality that allows the proposed assessment model to
be called hybrid, combining logical–heuristic and for-
mal–mathematical approaches in its development and
testing on examples close as possible to reality.
A model of this type is Saaty’s model, comparison
with which will demonstrate the features of our
approach. Figure 4 shows Saaty’s (1980) approach,
known as the Hierarchy Analysis method. The
method is relatively simple when used by decision
makers, and allows, as a first approximation, identifi-
cation of the most preferable project from among the
competitors. However, the method has several disad-
vantages (Podinovskii and Podinovskaya, 2012). Let
us dwell on one of them, in our opinion, the main,
conceptual one, but one critics of the method seem to
overlook: Hierarchy Analysis in explicit form does not
take into account the uncertainty of the external envi-
ronment of the project; the uncertainty, as it were,
remains behind the scenes. It is assumed that outside
the model, the experts considered all relevant scenar-
ios of the external environment of the project in which
it will be carried out and chose the one called the basic
scenario in the Russian terminology. For what reasons
and with the help of what procedures he chose it as the
basic one, Saaty does not explain. Thus, the situation
is artificially determined.
Figure 4 and Table 1 show the full cycle of calcula-
tions for assessing the comparative social effectiveness
by the Saaty method of the two LRPs of interest to us.
The calculation results in Fig. 4 suggest that the proj-
ect for construction of the LKM using a deterministic
approach to assessment, i.e., with a single (basic) sce-
nario for development of the external environment of
the project, is preferable to the TPM project.
However, the theory of decision making in a situation
of uncertainty from which Saaty abstracts his method,
introduces the concept of nonuniqueness of scenarios of
the external environment of projects. The theory assumes
that either, when making decisions, one takes into account
either the probabilistic uncertainty (objective or subjective)
of actualizing different scenarios, or radical uncertainty,
when the problem owner—the operating player—plays
with nature, which generates unpredictable scenarios. In
either case, given the invariability in the composition of the
system of goals of the problem owner in different scenarios,
the projects placed in them yield an unequal degree of goal
achievement (GA) of the slicing criterion.
Tabl e 1. Calculation according to criteria of decision mak-
ing theory “embedded” in GlobalD software product with
constants of relative importance of goal achievement in all
scenarios of external environment of compared projects
Enter required details:
Number of alternatives 2
Number of scenarios 3
Alpha (for Hurwitz test) 0.66
Enter scoring matrix:
Projects
Scenarios
Optimistic Pessimistic Most probable
1. TPM 0.12 0. 30 0. 20
2. LKM 0.43 0.17 0.30
Probability distribution of scenarios (for Bayes test)
Tot al 12 3
10.10.60.3
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
ON THE ISSUE OF THE SOCIAL EFFECTIVENESS OF EXPERT ASSESSMENTS 497
This changes the values of the objective function of
the projects, making it possible to choose the most
preferable. Tables 1 and 2 show the results of group
expert assessment of projects, when the probabilities
of actualization of contrasting scenarios of develop-
ment of the Russian economy—optimistic, pessimis-
tic, and most probable—were taken into account. At
the same time, the experts’ judgments were digitized
using GlobalD11 software,12 where the degree of GA of
the GA slicing criterion was estimated by numbers
from the interval [0, 1].
11GlobalD Shibikin D.D. Certificate of registration of the com-
puter program RU 2018660190, 17.08.2018. Application
No. 2018618087 dated July 16, 2018.
12The software was developed by S.V. Minin and D.D. Shibikin
(Siberian Transport University of Roszheldor, the Federal Rail
Tra nsp ort Ag enc y).
Fig. 4. Determination of most preferable LRP according to Saaty.
(0.19 × 0.71) + (0.10 × 0.71) + (0.19 × 0.29) + (0.10 × 0.35) +
(0.29 × 0.35) + (0.29 × 0.35) = (0.35 × 0.71) +(0.35 × 0.71) =
0.40 + 0.60 = Σ 1.00
0.71
0.71 0.71
0.71
A1A2A3A4
0.29 0.29
0. Public goals
of LRP
implementation
0.1. Noneconomic
0.29
0.1.1. Military-strategic
0.65
0.29 × 0.65 = 0.19
0.1.2. Environmental
0.35
0.29 × 0.35 = 0.10
1. Transpolar
Mainline
2. Lena–
Kamchatka
Mainline
0.2.2. Social
0.50
0.71 × 0.50 = 0.35
0.2.1. Economic
0.50
0.71 × 0.50 = 0.35
0.2. Socioeconomic
0.71
0.29
0.29
498
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
BYKADOROV, KIBALOV
From Table 2 it can be seen that the TPM project is
more preferable than the LKM only according to the
Bayes criterion in the pessimistic scenario with the
probability of its actualization being 0.6 or higher. For
all other criteria, the LKM project is preferable.
Now let us consider the case when, in contrast to
Fig. 4, the relativity coefficients (RC) in the contrast-
ing scenariosare not the same (see Fig. 5), and the
probabilities of actualization of the scenarios are not
known.
Based on (Kibalov and Khutoretskii, 2015), let us
show, first in general form, then numerically, the
results of assessing the TPM and LKM by the method
proposed by A.B. Khutoretskii.
At the first step, just line in the previous case, we
assessed the signif icance (RC) of the level 2 subgoals
to achieve the level 0 goal under the conditions of each
scenario (Fig. 5). The experts ordered the objectives of
the slicing criterion (see GA in Fig. 4) in order of their
nonincreasing significance for achieving the overally
goal of the level 0 project; the resulting orderings were
processed by the ORDEX13 program. Assuming that
the degree of achievement of the level 0 goal is com-
pletely determined by the degrees of GA of the slicing
criterion, using ORDEX for each scenario, he normal-
ized the main eigenvector so that the sum of the RC
was equal to one.
At the second step, degree of GA was assessed for
each combination of scenario and project (hereinafter,
Combination). The degree of achievement of each
goal depends on the Combination, and the experts
ordered six possible Combinations in nonincreasing
degree of GA with different “project + scenario”
Combinations.
If we formulate the idea of the estimation method
in general form, but for our case, then for each subgoal
Ak (k = 1, 2, 3, 4), after ORDEX processing of the
orderings presented by the experts, we obtained the
sixth-dimension vector Vk (k = 1, 2), proportional to
the desired vector of estimates of the degrees of GA of
the slicing criterion for all Combinations.
Now let us assume that the analyzed Combinations
of projects and scenarios exhaust all available possibil-
ities at the time of assessment. Then we can assume
that with the most favorable outcome, the goal in
question is fully achieved, since in no case can it be
achieved to a greater extent. The assessment of the
degree of GA with the most favorable outcome should
be equal to one. This reasoning substantiates the nor-
malization of vector Vk divided by its maximum coor-
dinate. The results are shown in Table 3, which aggre-
gates the data required at the final, third step of the
assessment procedures.
13The author-developer of the computer product is
A.B. Khutoretskii (IEIE SB RAS).
At the third step, we built an Evaluation matrix U =
(uij), where uij is assessment of the degree of achieve-
ment of the overall goal by strategy i in scenario j. We
denote the assessment of the significance of the goal of
the slicing criterion of the GA in scenario j as akj; the
assessment of the degree of achievement of goal A of
project i in scenario j as bijk. The akj values are shown
in Fig. 4; the bijk values, in Table 3. Six elements of
evaluation matrix U (see Table 4) are calculated by the
formula:
Tab le 3 p res ents t he in terme diate res ult s of cal cul a-
tions using this formula, but in numbers. The key indi-
cators for the entire method are the indicators of the
Number of Combinations column in Table 3. To
obtain the column metrics for goals A1, A2, A3, A4 (see
Fig. 4), the experts ranked six possible Combinations
in nonincreasing degree of GA achievement. After
ORDEX processing the orderings presented by the
experts, we obtained a six-dimensional vector Vk, pro-
portional to the sought vector of assessments of the
degrees of achievement of the subgoal in all Combina-
tions. Then, the vector Vk was normalized by dividing
by its maximum coordinate. The resulting figures are
in the columns of Table 3 corresponding to goals A1,
A2, A3, A4.
Let us show how the numbers in Table 3 are
obtained using the example of column A1 in a manner
generally explained above. The objective of slicing cri-
terion A1 is meaningfully formulated as military-stra-
tegic (see Fig. 4). Combinations 1–6 with respect to
this goal are ranked by experts so that the rank vector
has the form (2, 3, 4, 1, 5, 6), where the first position
indicates the number of the Combination most useful
for achieving the military-strategic goal; the second is
less useful; etc. ORDEX processing of the rank vector
yields a vector Vk (0.3, 0.23, 0.17, 0.12, 0.09, 0.07),
which is then normalized as described above in general
form, and the vector obtained after normalization (1,
0.77, 0.56, 0.4, 0, 3, 0.23) is entered in column A1 of
Tab le 3. Th e procedure is repeat ed for goa ls A2, A3, A4
and their corresponding columns.
ij kj ijk
k
uab
=
!
Tabl e 2. Criteria for evaluating projects in a situation of
uncertainty
The * sign means that the TPM project is more preferable than
the LKM project only by Bayes criterion; according to all other
criteria specified in the table, the LKM project is preferable, as
indicated by the * symbol placed in the corresponding cell of the
table.
Wald Maximax Savage Hurwitz Bayes Laplace
1. TPM *
2. LKM * * * * *
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
ON THE ISSUE OF THE SOCIAL EFFECTIVENESS OF EXPERT ASSESSMENTS 499
Next, the Evaluation matrix is formed (see Table 4)
using the example of calculating the element u23 the
degree of GA of level 0 in Fig. 4 project 2 (LKM) in
scenario 3 (most probable). For this, we need a vector
(0.35, 0.17, 0.30, 0.20) from Fig. 3 which can be sca-
larly multiplied by the vector (0.07, 0.22, 0.74, 0.20)
corresponding to Combination 6 in Table 3. As a
result, we find
u23= 0.35 × 0.07 + 0.17 × 0.22 + 0.30 × 0.74
+ 0.20 × 0.20 = 0.2 + 0.15 + 0.22 + 0.04 = 0.61.
Similarly, we calculate all other elements of the
Evaluation matrix (Table 4).
After processing Table 4 by GlobalD identifying
the most preferable option according to the decision
theory criteria (see Table 2), we can conclude that the
TPM project is preferable only according to the Savage
criterion, according to other criteria, the LKM project
is preferable.
CONCLUSIONS AND COMMENTS
(1) A common problem for all types of large-scale
projects, including regional transport, and in all coun-
tries where such projects are being implemented, is the
lack of reliable, scientifically based methods for a pri-
ori assessment of the effectiveness of these expensive
Fig. 5. Slicing criterion of goal achievement with relativity coefficients that are not the same in different scenarios.
Slicing criterion of goal tree
RC
0.19 0.10 0.35 0.35
0.50 0.21 0.20 0.09
0.35 0.17 0.30 0.20
In scenario 1
(optimistic)
In scenario 2
(pessimistic)
In scenario 3
(most probable)
Table 3 . Assessment of degrees of GA of slicing criterion of GA tree
* See fig. 4; Opt—optimistic scenario, pess—pessimistic, nv—the most probable.
Combination
no. Project number Scenario
number
Objectives of GA slicing criterion*
A1A2A3A4
11-PM 1-opt 0.4 1 0.38 1
21-PM 2-pess 1 0.54 0.19 0.54
31-PM 3-nv 0.17 0.29 0.29 0.19
42-LKM 1-opt 0.12 0.74 1 0.74
52-LKM 2-pess 0.09 0.38 0.54 0.39
6 2-LKM 3-nv 0.07 0.22 0.74 0.29
500
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
BYKADOROV, KIBALOV
measures. The problem is especially acute at the early
stage of preliminary (preinvestment) designs, which
we call the stage of discussing the design concepts (and
goals) of investors. It is here, not only due to the lack
of the necessary information, but in most cases due to
incorrect formulation of the problem itself, that cata-
strophic errors occur in the multidimensional assess-
ment of losses and benefits of planned large-scale
projects.
(2) The article proposes a public approach to
assessing the effectiveness of large-scale railway proj-
ects in Siberia and the Far East, based on Kornai’s sys-
temic paradigm projected at revealing a situation of
radical uncertainty in assessing such projects; the par-
adigm involves the use of expert technologies in all
assessment procedures.
(3) The purpose of the proposed assessment toolkit
is federal-level decision making as was done during
construction of the Trans-Siberian Railway, when the
corresponding committee was headed by the Emperor
of the Russian Empire, while the Finance Minister
was responsible for financial support of the project.
Institutionally, modern Russia is ready to take such an
organizational macrodecision, as evidenced by the
creation of a project office under the President of the
Russian Federation. The question is whether Russian
economic science is ready to develop support methods
for the adoption of large-scale investment decisions at
this level. Unfortunately, the question, which is inher-
ently systemic, remains open, and we propose, using
the example of railway transport, a response that at
least answers part of it.
(4) To assess the social effectiveness of large-scale
railway projects, a logical–heuristic, hybrid model has
been developed. The article describes its original ver-
sion, however, allowing the expansion of the function-
ality of the model in different directions. Namely, the
original model can be used in two modes: (1) when the
objects for assessment—large-scale railway projects
and the external environment of their implementa-
tion—are modeled by verbal methods based on the
official scenarios of the Ministry of Economic Devel-
opment and (2) when the external environment—the
Russian economy—is modeled using a specialized
economic and mathematical model of the Russian
economy OMMM-Railway (Suslov and Khutoretskii,
2015). In both cases, the submodels—verbal and eco-
nomic–mathematical—are synthesized using the
game approach: a player interested in implementing a
project, who is also an investor, interacts with the
player “nature,” with which the Russian economy is
identified. The result of the game is evaluated accord-
ing to decision making theory criteria in situations of
risk and uncertainty. This article covers in detail the
approach implemented in mode (1).
(5) The results of experimental calculations to
assess the comparative social effectiveness of the TPM
and LKM in mode (1) indicate that the hybrid model
proposed IEIE SB RAS and implemented at the Sibe-
rian Transport University of Roszheldor (the Federal
Rail Transport Agency) is first, efficient; second,
makes it possible to adequately take into account the
factor of military-strategic effectiveness of competing
projects; and, third, makes it possible to determine
further directions for improving the developed
approach.
FUNDING
The article was prepared as part of research on the proj-
ect “Socioeconomic Development of Asian Russia Based
on Synergy of Transport Accessibility, Systemic Knowledge
of the Natural Resource Potential, the Expansion of the
Interregional Interaction Space” (grant from the Ministry
of Science and Higher Education of Russia,
no. 13.1902.21.0016).
CONFLICT OF INTEREST
The authors declare no conflict of interest.
REFERENCES
Bogachev, V.N., Region BAM: Kontseptsiya razvitiya na no-
vom etape (Region of the Baikal-Amur Mainline: New
Concept of Development), Novosibirsk: Sib. Otd.,
Ross. Akad. Nauk, 1996.
Bykadorov, S.A. and Kibalov, E.B., Improvement of the
management model of railway transport in Russia,
EKO, 2018, no. 3, pp. 121–140.
Intriligator, M.D., Mathematical Optimization and Econom-
ic Theory, Philadelphia, PA: Soc. Ind. Appl. Math.,
1987 .
Kibalov, E.B. and Khutoretskii, A.B., Alternatives to trans-
port support for development of Russia’s Arctic shelf,
Reg. Res. Russ., 2016, vol. 6, no. 1, pp. 53–58.
Kibalov, E.B. and Kin, A.A., Large-scale investment proj-
ects: comparative analysis of efficiency evaluation
methods used by Russia’s natural monopolies, Reg.:
Ekon. Sotsiol., 2016, no. 1, pp. 295–313.
Kibalov, E.B. and Nekhoroshkov, V.P., Large-scale railway
projects in Russia: assessment of social efficiency in un-
certain situation, Vopr. Nov. Ekon., 2018, no. 1, pp. 60–
68.
Kleiner, G.B., Evolyutsiya institutsional’nykh sistem (The
Evolution of Institutional Systems), Moscow: Nauka,
2004.
Tabl e 4. Evaluation matrix
Project
Scenario
123
10.610.700.25
20.760.260.61
REGIONAL RESEARCH OF RUSSIA Vol. 11 No. 4 2021
ON THE ISSUE OF THE SOCIAL EFFECTIVENESS OF EXPERT ASSESSMENTS 501
Kornai, J., The System Paradigm, William Davidson Inst.
Working Pap. vol. 278, Ann Arbor, MI: Univ. Michi-
gan, 1998.
Kuhn, T., The Structure of Scientific Revolutions, Chicago:
Univ. of Chicago Press, 1970.
Larichev, O.I. and Moshkovich, E.M., Kachestvennye meto-
dy prinyatiya reshenii. Verbal’nyi analiz reshenii (Quali-
tative Methods of Decision Making. Verbal Analysis of
Decisions), Moscow: Nauka, 1996.
Lurye, A.L., Economic meaning of the rate of efficiency
and percentage of investments, Ekon. Matem. Metody,
1965 , vol. 1 , no. 1, pp. 137–145.
Podinovskii, V.V. and Podinovskaya, O.V., Once again
about the errors in the hierarchy analysis method,
Probl. Upr., 2012, no. 4, pp. 75–78.
Saaty, T.L., The Analytic Hierarchy Process, New York: Mc-
Graw-Hill, 1980.
Simon, H. and Newell, A., Heuristic problem solving: the
next advance in operations research, Oper. Res., 1958,
vol. 6, no. 1, pp. 1–10.
Suslov, N.I. and Khutoretskii, A.B., Assessment methods
of social efficiency of railway large-scale investment
projects, in Kompleksnyi podkhod k otsenke obshchest-
vennoi effektivnosti krupnomasshtabnykh zhelezno-
dorozhnykh proektov (Complex Approach to Evaluation
of Social Efficiency of Large-Scale Railway Projects),
Komarov, K.L., Ed., Novosibir sk: Sib. Gos. Univ. Pu-
tei Soobshch., 2015, ch. 2, pp. 50–68.
ResearchGate has not been able to resolve any citations for this publication.
Article
Based on a case study of the problem of transport support for resource extraction on Russia’s Arctic shelf, an approach is described and implemented to select an expedient strategy for solving weak structurization problems under the conditions of an uncertain scenario. To structurize the problem, a goal tree is constructed and scenarios and strategies are described. The scenarios indicate possible conditions of the external environment in which the problem is solved. The strategies differ in the ways cargoes are shipped: primarily by sea (Northern Sea Route) or by Siberian rivers. It is shown that the Transpolar Railway is necessary for any strategy. The authors have developed a technique and software for collecting and processing expert information, which is used to construct numerical estimates of the significance of goals. This method also allows us to estimate the degrees of achievement of a general goal as a result of using each strategy under the conditions of each scenario; these estimates are entered into the evaluation matrix. An expedient strategy is chosen according to the evaluation matrix with the help of strategic criteria (Laplace, Wald, Hurwitz, and Savage).
Book
Scitation is the online home of leading journals and conference proceedings from AIP Publishing and AIP Member Societies
Article
Problems of optimization are pervasive in the modern world, appearing in science, social science, engineering, and business. Recent developments in optimization theory, especially those in mathematical programming and control theory, have therefore had many important areas of application and promise to have even wider usage in the future. This book is intended as a self-contained introduction to and survey of static and dynamic optimization techniques and their application to economic theory. It is distinctive in covering both programming and control theory. While book-length studies exist for each topic covered here, it was felt that a book covering all these topics would be useful in showing their important interrelationships and the logic of their development. Because each chapter could have been a book in its own right, it was necessary to be selective. The emphasis is on presenting as clearly as possible the problem to be treated, and the best method of attack to enable the reader to use the techniques in solving problems. Space considerations precluded inclusion of some rigorous proofs, detailed refinements and extensions, and special cases; however, they are indirectly covered in the footnotes, problems, appendices, and bibliographies. While some problems are exercises in manipulating techniques, most are teaching or research problems, suggesting new ideas and offering a challenge to the reader. Most chapters contain a bibliography, and the most important references are indicated in the first footnote of each chapter. The most important equations are numbered in bold face type.
Article
Address at the banquet of the Twelfth National Meeting of the Operations Research Society of America, Pittsburgh, Pennsylvania, November 14, 1957. Mr. Simon presented the paper; its content is a joint product of the authors. In this, they rely on the precedent of Genesis 27:22, “The voice is Jacob's voice, but the hands are the hands of Esau.”
Once again about the errors in the hierarchy analysis method
  • V V Podinovskii
  • O V Podinovskaya
Podinovskii, V.V. and Podinovskaya, O.V., Once again about the errors in the hierarchy analysis method, Probl. Upr., 2012, no. 4, pp. 75-78.
The Analytic Hierarchy Process
  • T L Saaty
  • T.L. Saaty
Saaty, T.L., The Analytic Hierarchy Process, New York: Mc-Graw-Hill, 1980.
Improvement of the management model of railway transport in Russia
  • S A Bykadorov
  • E B Kibalov
Bykadorov, S.A. and Kibalov, E.B., Improvement of the management model of railway transport in Russia, EKO, 2018, no. 3, pp. 121-140.