On studying active radio measurements estimating the mobile network quality of service for the Regulatory Authority's purposes

Abstract

The Regulatory Authority monitors and regulates the telecom market at a national-wide range. One of its main tasks is to put spectrum at the disposal of the Mobile Network Operators (MNOs) for serving the increasing number of users and services. Another task of the Regulator is to demand a minimum quality of service to the network infrastructure. For this scope, the Regulator needs to have an independent estimation of the quality of service of the mobile network. This paper presents a methodology for Regulator-triggered monitoring of 5G radio resources and estimation of the maximum throughput offered to the end users. The requirements of the methodology are: (1) monitoring measurements must be external to the network, (2) all information needed for Quality of Service (QoS) estimation may be obtained only through active measurements in the network, and (3) the number of measurements should be as low as possible (for cost-saving) while offering statistical significance. The presented spot measurements validate the methodology and show the boundary use cases where the methodology would overestimate the transmission quality.

Full text

Computer Networks 235 (2023) 109980
Available online 20 August 2023
1389-1286/© 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
On studying active radio measurements estimating the mobile network
quality of service for the Regulatory Authority’s purposes
Jordi Mongay Batalla
a
,
*
, Sławomir Sujecki
b
,
c
, Jan M. Kelner
c
,
d
, Piotr ´
Sliwka
e
,
Dariusz Zmysłowski
c
a
Institute of Telecommunications, Warsaw University of Technology, 15/19 Nowowiejska Str., Warsaw 00-665, Poland
b
Faculty of Informatics and Telecommunications, Wroclaw University of Science and Technology, 27 Wybrzeze Wyspianskiego Str., Wroclaw 50-370, Poland
c
Institute of Communications Systems, Military University of Technology, 2 Sylwestra Kaliskiego Str., Warsaw 00-908, Poland
d
Ofce of Electronic Communications, 7/9 Gieldowa Str., Warsaw 01-211, Poland
e
Faculty of Mathematics and Sciences, Cardinal Stefan Wyszynski University, 1/3 Woycickiego Str., Warsaw 01-938, Poland
ARTICLE INFO
Keywords:
Network measurements
Regulator
Frequency band
Quality of Service
Spot metering
ABSTRACT
The Regulatory Authority monitors and regulates the telecom market at a national-wide range. One of its main
tasks is to put spectrum at the disposal of the Mobile Network Operators (MNOs) for serving the increasing
number of users and services. Another task of the Regulator is to demand a minimum quality of service to the
network infrastructure. For this scope, the Regulator needs to have an independent estimation of the quality of
service of the mobile network. This paper presents a methodology for Regulator-triggered monitoring of 5G radio
resources and estimation of the maximum throughput offered to the end users. The requirements of the meth-
odology are: (1) monitoring measurements must be external to the network, (2) all information needed for
Quality of Service (QoS) estimation may be obtained only through active measurements in the network, and (3)
the number of measurements should be as low as possible (for cost-saving) while offering statistical signicance.
The presented spot measurements validate the methodology and show the boundary use cases where the
methodology would overestimate the transmission quality.
1. Introduction
The telecom market Regulatory Authority (RA, often called the Of-
ce of Electronic Communications or Regulator) is the ofcial national
entity that concedes the licenses for using frequency bands (licensed)
[1]. The Regulator has many other tasks related to the control of the
networks and the activities of the Fixed and Mobile Network Operators
(MNOs). Among others, the RA shall monitor the Quality of Service
(QoS) provided by the MNOs and shall require a minimum quality
threshold to be experienced by the end users.
The Regulator may provide two kinds of measurements or estima-
tions of the quality of the connectivity service provided by the MNO. The
simplest way is the Grade of Service (GoS), dened as a measurement of
resources dedicated to a Service. In the case of mobile networks, the GoS
mainly refers to the strength of the signal in a given measurement point
and is directly related to the probability that a connection is successful.
Another way to estimate the quality of the connectivity is the so-called
QoS. QoS may be dened in mobile networks as the quality that the end
user may experience using the resources assigned to her/him.
In this paper, we present a practical study of how the RA could es-
timate the QoS and, concretely, how the RA could calculate the
maximum throughput that a given MNO may make available to the end
users in a given geographical point through the Fifth Generation (5G)
network. This estimation is based on active radio measurements external
Abbreviations: 5G, fth generation of mobile network; BLER, block error rate; BW, bandwidth; CA, carrier aggregation; CQI, channel quality indicator; CR, code
rate; ETSI, European Telecommunicaitons Standards Institute; FC, fractional coverage; gNB, gNodeB (i.e. 5G basestation); ITU, International Telecommunicaiton
Union; MCS, modulation and coding scheme; MIMO, multiple-in multiple-out; MNO, mobile network operator; NOC, network operation center; NR, new radio; QAM,
quadrature amplitude modulation; QoE, quality of experience; QoS, quality of service; QPSK, quadrature phase shift keying; RA, regulatory authority; RAN, radio
access network; RB, resource block; RE, resource element; RSRP, reference signal received power; RSRQ, reference signal received quality; SCS, sub-carrier spacing;
SINR, signal-to-interference-and-noise ratio; TDD, time division duplex; UE, user equipment; URLLC, ultra-reliable and low latency communication.
* Corresponding author.
E-mail address: jordi.mongay.batalla@pw.edu.pl (J. Mongay Batalla).
Contents lists available at ScienceDirect
Computer Networks
journal homepage: www.elsevier.com/locate/comnet
https://doi.org/10.1016/j.comnet.2023.109980
Received 30 March 2023; Received in revised form 17 July 2023; Accepted 10 August 2023

Computer Networks 235 (2023) 109980
2
to the network. The selection of the measurement points (spots) must be
representative of the whole territory regulated by the RA. Therefore, our
methodology consists of a punctual measurement method (spot meter-
ing) and a selection of the measurement points. Spot metering explains
how to perform the measurements and how to estimate the QoS in that
geographical point. At the same time, the selection of measurement
points shows which geographical points should be selected for spot
metering so that the RA may decide whether the QoS offered by an
Operator fullls the requirements posed by the RA. The selection of
spots is based on technical information of telecommunications infra-
structure provided regularly by MNOs and information on the
geographic territory covered by the MNOs. The contributions of this
paper are two:
- A comparative study of active and passive radio measurements for
the estimation of the quality of the coverage;
- A complete methodology for the selection of metering spots for
demonstrating the quality of the mobile coverage in a big territory (e.
g., the territory of the country).
The next Section presents the context of this work. Section 3 shows
the methodology for spot metering, including the selection of the
network parameters to be actively (active measurements) measured in
the network, as well as the further estimation of the throughput based on
those measurements. Section 4 presents the process of selecting the
geographical points (where the spot measurements will be done) from
the net of points of the territory regulated by the RA. Section 5 presents
the results of the spot measurements and validates the spot metering
methodology and the estimation of the transmission quality. At last,
Section 6 concludes the paper.
2. QoS background for cellular communication systems
In most countries, the RA is obligated to control the quality of the
network, which is directly related to the end-users’ satisfaction (citi-
zens). Especially, control of the network quality is required in places
where the operators have low concurrency. The user’s satisfaction is the
degree of delight or annoyance of the users of an application or service
[2], so even if there is a relationship between users’ satisfaction and the
quality of the network, the user’s satisfaction (quality of user’s experi-
ence) is more related to the service (application level) and the user
characterization. In conclusion, the relationship between users’ experi-
ence and network parameters is not univocal. Moreover, the current
networks are so complex that it is impossible to identify all the param-
eters impacting users’ satisfaction [3].
The International Telecommunication Union (ITU) has published
several reports, requirements, and informal web information regarding
the control of network quality by the RAs. The Electronic Communica-
tion Committee (Report 341) shows the RA’s possibilities for measuring
the quality in 5G networks [4]. This report states that the measurements
should be relevant to the users, essential to the society, common to all or
many services, independent of the technology, and, last but not least, the
requirements should be minimized so that costs and benets are
balanced. The afore-mentioned features of the measurements are
sometimes contradictory: for example, the most straightforward mea-
surements are dependent on the technology, and costly-efcient mea-
surements usually are less relevant to the users.
The users’ satisfaction will depend on the provided service, such that
the satisfaction of multimedia streaming will be related to the non-
paused service and the fast start and recovery of the transmission. In
contrast, Internet payment satisfaction will be related to the security of
the transactions [5]. The so-called QoS assessment considers just the
impact of network parameters on the user’s satisfaction. QoS assessment
should consider all the parameters impacting the end-to-end service, i.e.,
parameters at the network layer but also at the provider’s layer (e.g.,
cloud, Content Delivery Networks, etc.) [6].
The QoS assessment is based on values of network performance
extracted from both prior knowledge of the technology and measure-
ments made on the network as well as on models that assess the QoS
offered by the network and/or provider. The potential measurements
range from the delay, throughput, noise estimation to parameters
measuring the performance of the network when the User Equipment
(UE) is on the move. The most recognized parameter is the bitrate since
it has a clear impact on almost all the services and is comparable among
different operators [7].
The operator gathers the most reliable information on the quality of
the network since it constantly measures the parameters of the network
and adjusts them by both maximizing the performance efciency and
reducing the costs [8]. The measurements and the management policies
are stored in the Network Operation Centres (NOCs) that may be local,
regional or national-wide. Even if the quality of these measurements
(gathered in NOCs) is evidently good, such measurements could be
considered as non-independent, and the procedure for using them in
auditing tasks would be very complex from a legal point of view.
Therefore, law enforcement regulations should consider campaigns with
drive or walk tests with many measurements and a specic measuring
methodology. Such tests assess the coverage of the radio station and
measure the capacity and QoS and Quality of Experience (QoE,
depending on the tool) of a mobile radio interface from an external in-
dependent tester [9]. They are called “drive tests” if they are performed
on the move; otherwise, they are called walk tests.
The external-to-the-network measurements may be passive or active
[10]. Passive measurements are done without any signaling exchange
with the network [11], while active measurements rely on communi-
cation with the network [12]. The main passive measurement in mobile
networks is the Reference Signal Received Power (RSRP), which is the
power of the reference signal received at the UE [13]. It is normally
measured in dBm. The constellation of active measurements contains,
but is not limited to: (1) Reference Signal Received Quality (RSRQ)
characterizes the quality of the received pilot signals. It is an estimation
of the quality and is normally measured in dB; (2)
Signal-to-interference-and-noise ratio (SINR) characterizes the relation
between the signal power and the interference and noise powers. It is an
estimation of the transmission quality and is normally measured in dB;
(3) Block Error Rate (BLER) (%) is the ratio between the number of
erroneous blocks and the total number of blocks sent in the channel. It is
represented as a percentage (%); (4) Modulation and Coding Scheme
(MCS) determines how many usable bits can be sent per one resource
element (RE), and depends on the quality of the radio link. The better
the quality, the higher the MCS and the more usable data can be
transferred. It is used to simplify decisions on modulation, especially for
URLLC (Ultra-Reliable and Low Latency Communications); (5) Channel
Quality Indicator (CQI) is an indicator of the channel quality sent to the
network, represented by scalar values (range 0–15, but 0 is reserved). It
denes the data rate that the UE can handle in real radio conditions.
MCS is calculated on the basis of the CQI [14].
The range of the external-to-the-network measurements (active or
passive) is wide. In our methodology, the selection of the active or
passive measurements depends on the accuracy of how the measure-
ments assess the QoS. This will be the matter of the analysis of Section 5.
3. Methodology for spot metering and estimation of maximum
throughput
The methodology that we propose consists of two parts: the rst is
the methodology for estimating the network quality in a geographical
spot based on measurements performed at that spot. This is presented in
this Section; the second one is how to select and how many geographical
spots should be considered for measuring the quality of the network in
order that the measurements may show whether the Network Operator
fullls the requirements on the quality of the network transmissions. The
selection of spots is presented in Section 4. The full process is shown in
J. Mongay Batalla et al.

Computer Networks 235 (2023) 109980
3
Fig. 1.
3.1. Principles of radio resource allocation in 5G
When the UE initiates a new session, the gNB (the gNodeB is the 5G
base station) offers a number of resources (number of sub-carriers dur-
ing an established portion of time) for the transmission. Those resources
are organized as Resource Blocks (RBs), which is 12 sub-carriers during
one symbol time [15]. The quantity of the RBs assigned depend on the
quality requirements of the session, but also on the state of the network,
and the policies [16].
It is worth remarking that in normal functioning conditions, the
network allocates resources proportionally to the users considering the
service requested (e.g., high quality video [17,18]) and internal policies
(e.g., users’ contracts). So, when a few users are connected to the Radio
Access Networks (RAN), then a new UE may get high bandwidth for the
requested service, whereas when many users are connected to the RAN,
then the resources allocated to that service will be fewer.
In addition to the number of RBs assigned to one session, the quality
of the signal has an important role in estimating the quality of the
connectivity service. The quality of the signal depends on the environ-
ment (e.g., deep urban, sub-rural, rural [19–21]). The UE measures the
power of reference signals and calculates the
Signal-to-Interference-and-Noise Ratio (SINR) of the transmission (the
interferences are created by other antennas and/or other UEs [22]).
Based on this calculation, the quality of the channel is estimated
(Channel Quality Identier, CQI) and this estimation is used by the base
station to decide the optimal modulation and code rate (Modulation and
Coding Scheme, MCS) for the transmission. The modulations in 5 G are
Quadrature Phase Shift Keying (QPSK) or Quadrature Amplitude Mod-
ulation (QAM) that may contain different bits per symbol: 16QAM (4
bits/symbol), 64QAM (6 bits/symbol) and 256QAM (8 bits/symbol).
The MCS may be changed whenever the quality of the channel varies
through the Adaptive Modulation and Coding mechanism [23].
The reduction of SINR is mitigated with the introduction of Multiple-
In Multiple-Out (MIMO) and with the Massive MIMO [24,25], which
shapes the signal beam sent to the UE, reducing interferences and saving
energy, especially in deep edge situations [26].
3.2. Method for estimating the maximum throughput offered by one MNO
This section presents the estimation of the quality of the transmission
(concretely, the maximum throughput) at a given geographical point
(spot metering) based on active measurements. The parameter that we
selected for estimating the QoS of the network is the throughput. The
throughput is the maximum data bitrate the UE may receive (downlink)
in a given geographical point in the case that the UE receives all the
available RBs (i.e., there are not other UEs connected to the gNB). The
total throughput is the sum of the throughputs of each one of the
bandwidths used by the MNO in that cell [27]. If the antenna emits in
two different bandwidths, then the maximum throughput will be the
sum of the two estimated throughputs. In 5 G, the emission in two
different bandwidths is achieved through Carrier Aggregation (CA). In
fact, the spectrum may be dynamically distributed among different
technologies (4G and 5G) [28–30].
For estimating the maximum throughput, we use the denition of the
throughput of one downlink stream. The throughput of one downlink
stream depends on the number of radio resources allocated to that
stream and on the quality of the channel [31]. In addition, the UE may
limit the throughput that it may receive. Concretely, the throughput Thr
i
of the stream I is dened by formula (1), which is an extension of the
throughput formula presented in the standard [32].
Thri=RBi×l×modi×fi×CRi×sc ×sl ×sym × (1−OH)
Tsfr
TDD (1)
where RB
i
is the number of Resource Blocks allocated to stream i; l is the
number of layers dedicated to the transmission and emitted by the
MIMO antenna; mod
i
is the number of bits per symbol in UE
i
-modulation
for stream i; f is the scaling factor at higher layers and represents the
limitation in the transmission that the UE may require. This is a
parameter of the device and its maximum value is 1, which means that
the device does not put limitations in data transmission; CR
i
is the code
rate of the i-transmission and represents the overhead of the channel
coding of the transmission; sc is the number of subcarriers per RB and is
equal to 12 in 5 G (the same as in 4G); sl is the number of slots per
subframe (numerology) and depends on the Sub-Carrier Spacing (SCS)
[33,34], specically, sl is equal to 2
µ
, where µ is 0,1,2,3,4 for SCS=15,
30,60,120,240 kHz, respectively; sym is the number of symbols in one
slot. This may be 12 or 14, depending on the length of the cyclic prex.
Normally sym is equal to 14 and only with SCS=60 kHz sym may (but not
must) be equal to 12 when the cyclic-prex is longer; OH is the overhead
due to the signaling transmission in the air interface, i.e., the relation
between the amount of signaling information and user’s data in the
transmission. OH is 0.14 for downlink transmission in 3.4–3.8 GHz (and
0.08 for uplink transmission); at last, T
sfr
is the time of a subframe (1
ms), and Time Division Duplex, TDD (Downlink/Uplink multiplex)
parameter. The TDD parameter (≤1) denes the structure of the TDD
frame, and concretely, it denes which part of the subframes is dedi-
cated to the downlink transmission. For example, TDD=0.8 means that,
on average, 80% of the time, the antenna emits in the downlink, and
20% of the time, the antenna emits in the uplink (or other uses).
The maximum throughput of one stream is when all the Resource
Blocks of the antenna are allocated to that transmission and subject to
the fact that the UE does not limit the transmission. Let us remark that
the maximum throughput in geographical point A depends only on the
terrain (distance to the antenna, obstacles, etc.) and the technology used
by the antenna (beamforming, TDD downlink/uplink ratio, etc.), but it
does not depend on the number of users in the cell or the limitations
introduced by the User Equipment (i.e., we consider the maximum
throughput as the bitrate in the transmission sent by all the Resource
Blocks and without limitations on the user’s side). In this case, the an-
tenna with a bandwidth BW with a Sub-Carrier Space equal to SCS, will
be able to stream, at the most, the throughput presented in (2).
Thrmax =
BW
SCS×sc ×l×modi×f×CRi×sc ×sl ×sym × (1−OH)
Tsfr
TDD (2)
where SCS/sl is constant and equal to 15 kHz; the value f is equal to 1 for
the maximum throughput (without limitations in the UE); sym is 14; and
(1-OH) is equal to 0.86, as explained above.
Therefore, the maximum Throughput is:
Fig. 1. QoS assessment methodology.
J. Mongay Batalla et al.

Computer Networks 235 (2023) 109980
4
Thrmax[Mbps] = BW [kHz] × l×modi×CRi×14 ×0.86
15 ×Tsfr
TDD
=BW[MHz] × l×modi×CRi×0.80 ×TDD (3)
where BW, l, mod
i
, CR
i
and TDD are parameters that can be directly or
indirectly declared by the radio base station during signaling with the
UE.
The bandwidth, the TDD subframe composition and the number of
layers in MIMO transmission is sent during connection to the network.
This information is sent in the following messages: New Radio- Sub-
Carrier Spacing (NR-SCS) species the subcarrier splitting, New Radio-
Channel Bandwidth (NR-CHBW) denes the bandwidth, and Carrier
Aggregation (CA) informs about the aggregation of the next carrier.
Active measurements are necessary to read this information.
The parameters mod
i
and CR
i
are sent to the UE when a new session is
initiated and depends on the geographical spot where the UE is located.
They are related with the CQI in this way: the UE estimates the noise and
interference in the channel and labels the quality of the channel with a
value from 0 (really from 1 since 0 is reserved) to 15. This is called CQI.
A value of CQI equal to 15 means that the channel has very low noise
and/or interferences. CQI is sent to the base station via the CSI-RS
channel; and the base station decides on the modulation (mod
i
) and
the code rate (CR
i
) to be used in the downlink transmission. The exact
values of mod
i
and CR
i
for each value of CQI are in the standard Euro-
pean Telecommunications Standards Institute (ETSI) TS 138 214 [33]. It
is worth noticing that the base station may dismiss the CQI sent by the
UE and force other values of modulation and code rate. This case will be
analyzed in the results of Section 5.
Therefore, we propose to provide active measurements of BW, l, CQI
and TDD frame format and to calculate the maximum throughput by
applying formula (3).
Let us remark that this method allows to consider in the value of
maximum throughput the noise and interferences that the UE experi-
ments at that specic geographical point.
4. Measuring areal coverage
This section presents a methodology for selecting the geographical
points where the spot measurements will be provided.
The starting point is the requirement of the RA on the quality of the
connectivity in the territory. The requirement could be such as: “95% of
the territory should be covered by, at minimum, 100 Mbps
connectivity”.
For territory coverage, a common practice is to divide the whole
territory administrated by the RA into 100 m x 100 m squares and one
measurement is provided within this square, which is called spot mea-
surement [35]. The problem is that even a relatively small region of
1000 km
2
consists of approximately 10
5
squares. Making one measure-
ment per one square gives 10
5
measurements that would need to be
carried out, which would require a prohibitively great effort. With
realistic resources, one can usually perform measurements only for a
small subset of all squares for the considered area. Practical consider-
ations typically limit such a subset to approximately 1000 elements. One
therefore needs a methodology for assessing the region coverage, mak-
ing measurements only for a small subset of all squares within the
considered area.
From the legal point of view, it is inconsistent to do measurements in
some squares and calculate the percentage of squares that do not full
the aforementioned requirement. Therefore, we need to demonstrate
(statistically) how many of the measurements should show connectivity
of less than 100 Mbps in order to conclude that the 95% of the squares
are not covered with sufcient high quality (100 Mbps) connections.
Using the statistical terminology, the set of all squares within the
considered area corresponds to the population, about the properties of
which we would like to infer using a small subset of all squares, i.e., a
sample. The number of all squares within the considered region, in
which the signal is sufciently strong is the parameter subject to the
inference procedure whereas all squares within the sample is the sta-
tistic. Our primary aim is to derive a formula, which could be used to
calculate the minimum sample size necessary for a meaningful assess-
ment of the population parameter using a statistic and also to dene the
sample selection procedure. In essence, the considered here statistical
analysis starts with choosing a sample consisting from randomly
selected elements (squares) then making measurements and calculating
the results for the sample elements only. This is a typical scenario in
statistics whereby one tries to infer about a parameter of a population
based on a relatively small sample. Therefore, in the second subsection
we discuss an application of statistical methods to solve this problem. In
the rst subsection we precede the statistical consideration with a
development of a probabilistic model. Even though statistical methods
alone are perfectly suitable and sufcient to address the issue at hand,
we develop rst a probabilistic model to gain an intuitive insight into the
nature of approximations inherent in the statistical handling of the
problem.
4.1. Probabilistic model
In an effort to derive a probabilistic model we initially consider an
experiment as follows. For a given region one randomly selects a set of
squares (in our case 100 m x 100 m squares). Then measurements are
made in all selected squares and the nal result is obtained by dividing
the number of squares where RA requirements are fullled by the
number of all squares. This procedure yields a proportion in a statistical
sense, which gives an estimate of the area coverage by the 5G signal. In
order to build a probabilistic model for so dened experiment one needs
to estimate the probability of success under such scenario. For this
purpose, one would need to know the distribution of antennas in the
given region and their parameters. Thus this approach leads quickly to
quite cumbersome calculations before further conclusions could be
drawn. One can however, consider an alternative scenario whereby we
assume that for the selected region measurements have been carried out
for each square of the reference grid and the measurement results are
xed but have not been revealed. For the squares where the re-
quirements of the RA are fullled we assign 1 (success) and otherwise
we assign 0 (failure). The Fractional Coverage (FC) by 5G signal for the
given area (or proportion using the statistical jargon) we obtain by
adding the ones corresponding to all squares for which RA requirements
were fullled -i and dividing it by the number of all squares -j (Bernoulli
trial). It is noted that calculating an average yields the same result.
The problem is that if we select one square and exclude it from
further selection process, i.e. we select the next square from the
remaining squares then the value of probability p changes each time we
select an element, i.e. for the second element p equals either i-1/j-1 (if
one was chosen rst time) or i/j-1 (if zero was selected rst time). As a
consequence, the subsequent trials are not independent because once we
select randomly a particular square on the reference grid the number of
either zeros or ones for selecting the next square changes by one. Hence,
the probability p of selecting one is equal to the number of squares with
measurement result equal to 1 divided by the number of all left squares,
the numerical value of p changes each time. In fact, the probability
distribution governing such experiment is known, and it is that of the
hypergeometric distribution. Further, when sample size is much smaller
than the population, the hypergeometric distribution can be approxi-
mated by the binomial distribution, which, on its turn, can be approxi-
mated with the normal distribution for a sufciently large sample size n
[36]:
P(k) = 1
σ

2
π
√e−1
2k−
μ
σ
2
(4)
J. Mongay Batalla et al.

Computer Networks 235 (2023) 109980
5
with expected value
μ
and standard deviation
σ
of the binomial distri-
bution:
μ
=np(5a)
σ
=
n p (1−p)
(5b)
where the number of experiments is n, the number of yes answers is
equal to k and the probability of obtaining the yes answer is p.
If we dene a new random variable X: X =K/n, then we may express
the probability distribution directly for the measured fraction of the FC
=k/n. Moreover, the probability distribution of so dened random
variable does not move laterally when n is varied. The probability dis-
tribution of the random variable X can be derived from the probability
distribution of the random variable K since X =K/n. Thus the proba-
bility distribution stays normal however, the expected value and the
standard deviation change. So,
μ
and
σ
respectively, change to:
μ
=p(6a)
σ
=
p(1−p)

n
√(6b)
Fig. 2 shows probability distribution of the random variable X. When
n grows, the distribution becomes narrower as in the previous case but
now the probability distribution does not shift laterally, and peaks when
k/n equals the expected value.
Following the properties of the normal distribution for random
variable X, it is known that if the result of an experiment is within the
interval
μ
±
σ
with probability (condence level) 0.68,
μ
±2
σ
with
condence level 0.95,
μ
±3
σ
with condence level 0.997, etc. From z-
tables one can read that for condence level of 0.9, 0.95, 0.98 and 0.99
the results should be within the interval:
μ
±1.6449
σ
,
μ
±1.96
σ
,
μ
±
2.3263
σ
and
μ
±2.5758
σ
, respectively. For a given value of the con-
dence level we can thus obtain the dependence of absolute error Δ on n.
For instance, for the condence level of 0.9 we obtain an inequality:
Δ≤1.6449
σ
(7)
Substituting (4) for
σ
in (5) yields:
Δ≤1.6449 
p(1−p)

n
√(8)
In the same way one can calculate the dependence of the absolute
error Δ on n for other values of the condence level. Having dened the
probabilistic model governing the experiment at hand, we move on to
the next section to discuss the statistical handling of the problem.
4.2. Statistical analysis
Once we know the procedure for the sample elements selection and
the probability distribution which governs the process of the sample
section we apply standard statistical procedures to the proportion
assessment using a randomly selected sample only. Theoretical funda-
mentals for the proposed statistical handling of the problem, i.e. pro-
portion inference for a given sample are discussed in [36]. We begin the
derivation with providing the formula for an estimation of condence
interval for a given value of the success probability p. Let:
X – binary random variable 0–1 with unknown parameter p,
n – sample size,
k – number of successes in an n-element sample,
k/n – sample proportion,
1-
α
– level of condence (
α
– statistical signicance), u
1−
α
/2
–
quantile of order 1−
α
/2 of the standard normal distribution - N(0,1).
An unbiased estimator of parameter p is 
p=k/n [36]. For suf-
ciently large n 
p∼Np,p(1−p)
n, i.e. 
p may be approximated using the
normal distribution. So, the condence interval is given by [36]:
P
p−u1−
α
2
p(1−p)
n
≤p≤p+u1−
α
2
p(1−p)
n

=1−
α
and its length is I
n
:
In=p+u1−
α
2
p(1−p)
n
−p+u1−
α
2
p(1−p)
n
=2u1−
α
2
p(1−p)
n
(9)
Converting (9) one can express n as:
n=p(1−p)

2u1−
α
2
In



2
(10)
Thus from (10) for a given condence level 1−
α
, absolute error Δ =I
n
/2
and knowing the probability of success p one can calculate the sample
size. We note that (10) is a more general form of inequality (6) derived
intuitively in the preceding subsection. Next, we use an inequality:
k
n1−k
n≤1
22
and obtain an approximation for (10) allowing calculating n even if
parameter p is not known:
n≤

u1−
α
2
In



2
(11)
Using inequality (11) one can calculate the minimal size of the
sample for a given condence level and the absolute error Δ =I
n
/2.
Table 1 gives values of the minimal sample size calculated using (11) for
selected values of absolute error and condence level. The quantile of
order 1−
α
/2 has been calculated numerically using the inverse cumu-
lative density function [36]. For the sake of comparison, in Tables 2 and
3 we present the minimal sample size calculated using formula (10) for p
equal to 0.9 and 0.95, respectively. It can be seen that if the value of p is
Fig. 2. Probability distribution for random variable X for selected values of
parameters n and p.
Table 1
Values of the minimal sample size calculated using (11) for selected values of
absolute error and condence level.
error Δ Condence level
90% 95% 98% 99%
5% 271 385 542 664
2% 1691 2401 3383 4147
J. Mongay Batalla et al.

Computer Networks 235 (2023) 109980
6
known we get signicantly smaller sample sizes. Even if p =0.9 the
sample size obtained using (10) is at least half of that obtained using
formula (11). For p =0.95 the benets of using formula (10) are even
larger. Unfortunately, in an actual experiment the population parameter
p is not known. Hence, the formula (11) comes in very handy in this
context even though it signicantly overestimates the sample size. In
fact, this ensures that the measurements performed by the RA are on the
safe side, i.e. they overestimate the error and underestimate the con-
dence level.
More comprehensive discussion of the accuracy limitations for the
proposed normal distribution approximation is given in [37]. The
analysis presented in [37] gives an outline of a procedure that could be
used to derive a method for more accurate estimation of the condence
interval for the parameter p using the approach proposed by Clopper and
Pearson [38]. Clopper and Pearson proposed using Beta distribution - B
(β
1
,β
2
,t) in the form B(β1,β2,t) = Γ(β1+β2)
Γ(β1)Γ(β2)t
0
xβ1−1(1−x)β2−1dx and an
identity, which allows for calculating the Cumulative Distribution
Function (CDF) of the binomial distribution when parameter p is known:
B(n−k,k+1,1−p) = 
k
j=0n
jpj(1−p)n−j(12)
The Clopper and Pearson idea of condence interval (at the con-
dence level 
α
) construction for fraction 
p can be expressed by formula
(13) using formula (12) [37]:
PB−1Sn,n−Sn+1,1−
α
2<p<B−1Sn+1,n−Sn,1+
α
2=
α
(13)
where B
–1
- quantile of Beta CDF, Sn=n
j=1Xj, and B(n−Sn,Sn+1,γ) =
1−B(Sn+1,n−Sn,1−γ).
Analytical calculation of the minimal sample size n from (13) is not
possible to the authors best knowledge. Therefore, one had to use nu-
merical methods.
Assuming
B−1Sn+1,n−Sn,1+
α
2−B−1Sn,n−Sn+1,1−
α
2≥2Δ,(14)
where S
n
is the total number of expected successes and p=S
n
/n =0.9 and

α
are xed.
For a given Δ, we are looking for the smallest n that simultaneously
meets the formula (14) and the assumptions: set condence level 
α
and
S
n
/n =0.9 (or S
n
/n =0.95). This creates a two dimensional space within
which we can search for an optimal solution. The optimal solution is
determined by the Generalized Reduced Gradient method, which looks
at the gradient (slope) of the objective function as the input values (or
decision variables) change and determines that it has reached an opti-
mum solution when the partial derivatives equal zero [39].
Alternatively, one can recast (14) into the following form:
Bn−kupp,kupp +1,1−p−B(n−klow ,klow +1,1−p) ≥ 
α
,(15)
where, klow =max(0,ceil(p−Δ)n),kupp =min(n,floor(p+Δ)n).
The function max selects the larger value of the two arguments while
min the smaller one whereas ceil and oor functions round off towards
the larger integer and the smaller integer, respectively. By using min and
max functions one prevents the values of k
low
and k
upp
to go below zero
and beyond n, respectively. Using oor and ceil functions results in
obtaining an upper bound for the sample size in the subsequent calcu-
lations. In order to calculate sample size from (15) when p, Δ and
α
are
given one needs to increase n from some low initial value, e.g., zero, up
until inequality (15) is fullled.
The values of the minimal sample size calculated using (13) for
selected values of absolute error and condence level for p =0.9 and p =
0.95 are shown in Tables 4 and5 while those relying on (15) are given in
Tables 6 and 7. One can observe that these values are signicantly less
than those presented in Table 1 and consistently slightly larger than
those shown in Tables 2 and 3, which agrees with what one should
expect considering the nature of the calculation procedure. The results
from Tables 4–7 are consistent with the results presented in Tables 2 and
3 and reafrm their validity. This is especially important for values of p
near to 1 since in this case binomial distribution indicates more pro-
nounced skewness, which cannot be reproduced correctly by the normal
approximation. The formulation of the problem based on (13) and (15)
on the other hand, does not suffer from this limitation and hence can be
used to verify formulae (11) and (10). Again the fact, that the sample
sizes from Tables 4–7 are signicantly lower than those shown in
Table 1, indicates that some prior knowledge about the population
parameter can help to reduce signicantly the sample size.
5. Evaluation of the spot metering methodology
We did a testing campaign to validate the spot metering methodol-
ogy presented in Section 3. The results of the tests allowed us to make
conclusions on the parameters used for estimating the throughput, and
to conrm the decisions taken on the selection of the measurements
within the methodology.
Section 5.1 presents several measurements of active (CQI, SINR,
BLER) and passive (RSRP) measurements and analyses the variability of
the measurements in time. Section 5.2 analyses RSRP vs. SINR and
shows that SINR better approximates the user’s experience than RSRP.
In Section 5.3, we compare SINR and CQI for different user’s terminals.
The objective is to check whether the type of terminal has an impact on
the value of CQI. Section 5.4 shows the values of the selected parameters
in the case that the RAN accepts error in transmission. Normally, the
RAN optimizes the radio interface performance by accepting around
10% frame errors. The objective of Section 5.4 is to show how the
quality of the transmission changes when the RAN accepts errors in
transmission.
In all tests, the baseband of the gNB (Distributed Unit and Cetnral-
ized Unit connecthre through F1 interface) sends only one signal to one
Table 2
Values of the minimal sample size calculated using (10) for selected values of
absolute error and condence level for p =0.9.
error Δ Condence level
90% 95% 98% 99%
5% 98 139 195 239
2% 609 865 1218 1493
Table 3
Values of the minimal sample size calculated using (10) for selected values of
absolute error and condence level for p =0.95.
error Δ Condence level
90% 95% 98% 99%
5% 52 73 103 127
2% 322 457 643 788
Table 4
Values of the minimal sample size calculated using (13) for selected values of
absolute error and condence level for p =0.9.
error Δ Condence level
90% 95% 98% 99%
5% 117 158 214 258
2% 658 914 1267 1542
J. Mongay Batalla et al.

Computer Networks 235 (2023) 109980
7
user with the use of 100 MHz in n78 (3.5–3.6 GHz) band. The numer-
ology SCS equals 30 kHz.
The technical data of the Active Antenna Unit (AAU) are shown in
Table 8.
The tests were done with one UE (tester) in the cell, which was Sony
Xperia X mobile phone with Qualcomm Snapdragon 650 System-on-
Chip (SoC), unless it is written otherwise in the specic test descrip-
tion. The testing UE implements XCAL tool, which automatically records
and deciphers messages from the air interface. XCAL is integrated with
the chipset, so it may read signaling and data messages after decipher-
ing. XCAL is used by operators to troubleshoot, monitor and optimize
wireless voice and data network performance, all in real-time.
5.1. Analysis of network quality indicators
In these rst tests, we measured in the UE the following parameters:
RSRP, SINR, and CQI in different points of the cell; thus, the distance of
the UE to the gNB as well as the ‘Line of Sight’ conditions (i.e., Line of
Sight, LoS, or non-Line of Sight, nLoS) varied from one point to another.
Fig. 3 shows one measurement (in one example geographical point)
performed during 20 s. The measurements are taken in the UE each 100
ms. We may observe that the analyzed parameters are quite unstable, so
expected values in a period of time are required for a fair analysis. This
fact was expected due to high variations in the network, however the
question arising is on the quantity of measurements needed to calculate
the expected value of the parameters.
For responding the previous question, we calculate the expected
values of the three quality indicators (RSRP, SINR and CQI) for different
time scales. The expected values of the different quality indicators (see
Table 9) do not change signicatively in different time scales ≥2 s,
especially if we take a time scale ≥5 s. For example, the mean value of
RSRP parameter is −85.08 between the second 0.1 and the second 5.0
(N =50), whereas it equals −86.37 between the second 0.1 and the
second 10 (N =100). We may conclude that the trafc probably ap-
proximates to self-similar for longer time scales, which means that the
rst ve seconds are enough to take the measure expected values of the
network parameters. For all the other tests, we will take 5 s time scales
for measuring the expected values of the network quality indicators.
5.2. Maximum throughput indicator: RSRP vs. SINR
The following gure shows the modulation selected for different
values of RSRP and SINR taken at different measurement points. The
RSRP is measured by the UE and the SINR is estimated by the UE based
on measurements of the reference signal and the response of the channel
to pilot symbols. For concrete values of RSRP and CQI, we observe the
modulation that is more frequently selected by the antenna (mode value
of the modulation). The selected modulation is read in the signaling
between the base station and the UE (MCS parameter). The values of
RSRP presented in the gure have 5 dBm steps. This means that a value
of RSRP=− 110 dBm includes all the measurements where the RSRP was
in the range from −110 to −105 dBm, and concretely RSRP=
[−110,−105) dBm. The same notation is used for SINR. Fig. 4 shows the
mode of the modulation for each value of RSRP and SINR, i.e., the
modulation points shown in Fig. 4 for a given (RSRP, SINR)is
arg max
modulationi
P(MODULATION =modulationi|{RSRP,SINR}).
As we may observe in Fig. 4, there is a much higher variability of the
selected modulation for a value of RSRP (vertical trend) than for a given
SINR (horizontal trend). For example, for RSRP= − 95 dBm, any of the
modulations are possible depending on the value of SINR; whereas for
SINR=0 dB, only 16QAM modulation is selected and for SINR=20 dB
only two modulations are possible.
This higher variability of modulation for values of RSRP may be
observed clearly in Fig. 5. Fig. 5(a) shows that for several values of
RSRP, the selected modulation in different tests is any of the four po-
tential modulations (QPSK, 16QAM, 64QAM or 256 QAM). For example,
for RSRP= − 90 dBm, the modulation QPSK has been selected in 8% of
the tests, 16QAM was selected in 51% of the tests, 64QAM was selected
in 33% of the tests, and the modulation 256QAM was selected in 8% of
the tests. Whereas, Fig. 5(b) shows that the selected modulations for any
value of SINR may be, at the most, two of them. For example, for SINR=
10 dB, the modulation 16QAM was selected in 22% of the samples,
whereas the modulation 64QAM was selected in 78% of the samples.
Fig. 5 clearly shows that the variability of the modulation for a given
value of RSRP is much higher than in the case of SINR. We may conclude
that the RSRP value is not a good estimator of the modulation used and,
as a consequence, is not a good estimator of the throughput (the
maximum throughput depends on the modulation and code rate used in
the transmission, as explained in Section 3). The reason comes from the
fact that the channel quality is strongly impacted by the noise and in-
terferences that may limit the possibility of sending high amounts of
data in the transmission. On the contrary, the SINR has a higher corre-
lation with modulation and is a better estimator of the throughput.
5.3. Dependence of SINR and CQI values with the user terminal
The estimation of the SINR (and the selection of the CQI) is done by
the UE based on an internal (non-standardized) vendor-locked algo-
rithm. Therefore, one of the main listened drawbacks for using CQI and
SINR as indicators of the quality is that two different terminals may
estimate different values of SINR and CQI.
We performed measurements in several user’s terminals with
different chipsets, all implementing XCAL for the extraction of 5 G
Table 5
Values of the minimal sample size calculated using (13) for selected values of
absolute error and condence level for p =0.95.
error Δ Condence level
90% 95% 98% 99%
5% 72 94 125 149
2% 372 508 695 841
Table 6
Values of the minimal sample size calculated using (15) for selected values of
absolute error and condence level for p =0.9.
error Δ Condence level
90% 95% 98% 99%
5% 100 154 200 254
2% 625 875 1225 1525
Table 7
Values of the minimal sample size calculated using (15) for selected values of
absolute error and condence level for p =0.95.
error Δ Condence level
90% 95% 98% 99%
5% 60 80 120 150
2% 329 486 686 829
Table 8
Technological parameters of the Active Antenna Unit used in the evaluation.
Active Antenna Unit
Massive MIMO (64T64R)
192 antenna elements
64 antenna ports
Horizontal beamforming (40 m height)
Gain: 25 dBi
J. Mongay Batalla et al.

Computer Networks 235 (2023) 109980
8
signaling from the chipset.
Table 10 shows the values of SINR estimated by the UEs jointly to the
value of CQI selected by the UEs. The values presented in the table are
mean values with condence intervals at 0.95-condence level. The
measurements were twelve series (three geographical points and four
UEs), each series of 30 s. We selected the UEs that use different 5 G
chipsets, so that we may observe the potential differences in the esti-
mation of the SINR and CQI by different hardware. The four UEs were:
(1) Sony Xperia X with Qualcomm Snapdragon 650 SoC, (2) Samsung
Galaxy S21 5 G with Qualcomm Snapdragon 888 chipset (3) Samsung
Galaxy A14 5 G with Samsung Exynos 1330, and (4) Xiaomi Mi 11 with
Qualcomm Snapdragon 888 SoC chipset.
In Table 10, we may observe that the variations between UEs are
small and all the UEs estimate the SINR and calculate CQI in a similar
way. The Samsung A14 phone estimates more optimistic parameters
than the others; however, it is difcult to afrm if this variation impacts
the functioning of the UE in the network.
It is worth remarking that the CQI sent by the UE to the antenna may
be dismissed by the RAN in the moment of deciding on the MCS. This
means that the RAN may decide on another modulation and code rate for
the transmission regardless of the channel quality estimation made by
the UE. Such a decision may risk increasing the errors in the trans-
missions (when the RAN decides a higher modulation and/or higher
code rate) or may send the session data in an inefcient way (when the
RAN decides a lower modulation and/or lower code rate). The latter is
much more rare (but not unheard), however, the rst option is often
used by the RAN. The RAN normally sends data with a higher MCS than
estimated by the UE and measures the errors during transmission; the
RAN aims to maintain the BLER <10% (acceptable value in practically
all cases). This behavior is not in accordance with the standard, how-
ever, for many scenarios (where the low loss ratio is not crucial for the
transmission) it may increase the efciency of the usage of radio re-
sources. This is analysed below.
5.4. Analysis of BLER impact on the measurements
In order to understand the impact of the error rate, we read the BLER
for different MCS in XCAL application. XCAL may read the blocks that
arrived correctly and the blocks that were lost in the radio interface. The
BLER is the relation between lost and lost +arrived blocks. Table 11
Fig. 3. Cell radio parameters vs. time; (a) RSRP; (b) SINR; (c) CQI.
Table 9.
Mean values of RSRP, SINR and CQI for different numbers of samples (time
scales).
Time scale Network parameters (mean values)

i
RSRPi/N 
i
SINRi/N 
i
CQIi/N
N =20 (from T =0.1 s to T =2 s) −77.35 12.9 −1.31
N =50 (from T =0.1 s to T =5 s) −85.08 11.5 −2.14
N =100 (from T =0.1 s to T =10 s) −86.37 11.35 −2.7
N =150 (from T =0.1 s to T =15 s) −88.02 11.56 −2.3
N =200 (from T =0.1 s to T =20 s) −86.97 11.31 −2.78
Fig. 4. Selected modulation (mode value) for different RSRP and SINR values.
J. Mongay Batalla et al.

Computer Networks 235 (2023) 109980
9
shows some example measurements of the errors in the transmission
(BLER parameter) when the RAN respects the decision taken by the UE
(CQI) and when the RAN forces a higher MCS. The RAN decides the
modulation and code rate through the parameter MCS. The values pre-
sented in Table 11 are the mean values of 30 s tests. The CQI is the mean
value of all the values estimated by the UE and sent to the gNB. The
column Modulation/ Code rate in the UE – column (2), refers to the
modulation and code rate of the rounded-up CQI. For example, for
CQI=12.2, column (2) corresponds to the modulation and code rate of
CQI=13, following the standard [33], i.e., 256QAM and 797/1024. The
modulation and code rate decided by the RAN – column (3), represents
the decision (modulation and code rate) forced by the RAN for the
transmission, and the BLER – column (4), shows the errors in the
transmission with such a forced modulation and code rate.
The UE used in this test is Sony Xperia X with Qualcomm Snapdragon
650 SoC.
As Table 11 shows, the RAN may try to use the channel more ef-
ciently (to send more data) than proposed by the UE, and, in case the
reports show too many errors (normally, an accepted value is
BLER<10%), then the RAN will use the Adaptive Modulation and
Coding mechanism to change the modulation and code rate (during the
transmission). This is an aggressive mechanism that the RAN may use for
better use of the radio resources, regardless of the estimation (of the
channel quality) algorithm provided by the UE.
In our methodology, we assume that the RAN follows the standard
[33] for transmission, such that the RAN respects the estimation of the
channel quality done by the UE (and sent in the CQI parameter). In that
case, the maximum throughput of the gNB in a given point, where the
UE has estimated a channel quality CQI will follow the formula (3). In
the example of our tests, the bandwidth is 100 MHz, the MIMO is 2T2R
and the TDD (Downlink/Uplink DL/UL multiplex) parameter TDD=0.8
(80% of the frame if dedicated to Downlink). Therefore, the maximum
throughput in a geographical spot where the UE estimates a channel
quality CQI=12 will be: Thrmax[Mbps] = BW[MHz]× l×modi×CRi×
0.80 ×TDD =100 ×2×8×711
1024 ×0.80 ×0.80 =711Mbps.
5.5. Analysis of results
The test results showed that a methodology for spot-metering the
network quality should be done based on active measurements, since
SINR is much more correlated with the modulation than RSRP. We
showed that all the measurements in the network have stochastic
components, so any measurement needs to be treated statistically. We
observed that expected results of network measurements are stable after
5 s, since the difference of the SINR expected values of 20 s sample and 5
s sample is only 0.2%, whereas the difference of the SINR expected
values of 5 s sample and 2 s sample is 8%. This means that measurements
in the network should be at least 5 s duration. The results showed that
user’s terminal has impact on the estimation of the SINR and on the
selection of the CQI (the SINR estimation algorithm is different in
different UEs), with a maximum difference of 12.5% in the estimated
SINR values. However, the main error of the selection of the CQI is
introduced by the acceptance of bit errors by the RAN, which may
change up to 3 values of CQI for errors ≤10%.
6. Conclusions and future work
This paper treated the theme of measurements performed by the
Regulator in 5 G. We presented a methodology for the specication of
the spot metering, including which measurements and how they should
be done, and the selection of the points where the spot metering will be
performed.
The metering is based on active measurements since they are much
closer to the throughput, which is the parameter selected for demon-
strating the quality of the network. In our methodology, we proposed to
use the CQI estimated in the testing UE and to calculate which is the
maximum throughput that the gNB may transmit with such a CQI. In
Fig. 5. Percentage of selected modulation for different (a) RSRP and (b) SINR values.
Table 10
Values of SINR and CQI for different UE at several geographical spots.
Geographical spot Sony Xperia X Samsung S21 Samsung A14 Xiaomi Mi 11
SINR [dB] CQI SINR [dB] CQI SINR [dB] CQI SINR [dB] CQI
52.173620, 21.195407 19.6 ±0.4 12.2 ±0.8 20.0 ±0.3 12.2 ±0.7 22.3 ±1.3 12.4 ±1.2 19.5 ±0.9 11.8 ±0.9
52.172710, 21.192578 33.4 ±0.2 14.8 ±0.2 33.6 ±0.1 14.7 ±0.2 33.9 ±0.6 14.8 ±0.2 32.6 ±0.5 14.8 ±0.2
52.174344, 21.194499 13.7 ±0.7 10.2 ±0.4 13.9 ±0.7 10.3 ±0.5 14.2 ±0.6 11.3 ±0.3 13.2 ±0.7 10.0 ±0.7
Table 11
Block error rate for different values of MCS.
Sony Xperia X estimation RAN decision
CQI
(1)
Modulation – Code
rate (2)
Modulation / Code
rate (3)
BLER in transmission
(4)
12.2 256QAM – 797/1024 256QAM – 797/1024 1.1%
256QAM – 885/1024 6.5%
256QAM – 948/1024 10.3%
10.2 64QAM – 873/1024 64QAM – 873/1024 0.4%
256QAM – 711/1024 5.4%
256QAM – 797/1024 13.7%
J. Mongay Batalla et al.

Computer Networks 235 (2023) 109980
10
order to calculate the throughput, other parameters such as bandwidth,
MIMO composition, and frame description (which part of the frame is
used for downlink and which part is used for uplink) are used.
The methodology is completed with the selection of the geographical
points where the measurements will be done. The whole area under the
supervision of the Regulator is divided in 100 m x 100 m squares grid,
and typically one measurement is done in each square. The methodology
provides a mechanism for selecting only some of the grids.
Future work will be focused on the advances of mobile networks
beyond 5 G and how to adapt our methodology to future generations of
mobile networks [40,41].
Funding
This work has been supported by the National Science Centre in
Poland, Sonata-bis Programme, through the project framework
”Context-Aware Adaptation Framework for eMBB services in 5G net-
works”, under Grant No 2018/30/E/ST7/00413.
CRediT authorship contribution statement
Jordi Mongay Batalla: Methodology, Investigation, Validation,
Writing – original draft. Sławomir Sujecki: Investigation, Writing –
original draft. Jan M. Kelner: Validation. Piotr ´
Sliwka: Investigation,
Writing – original draft. Dariusz Zmysłowski: Validation.
Declaration of Competing Interest
On behalf of the authors I do declare no conict of interest beyond
the current work contracts at:
Warsaw University of Technology, Warsaw, Poland
Ofce of Electronic Communications, Warsaw, Poland,
Wroclaw University of Science and Technology. Wroclaw, Poland
Military University of Technology, Warsaw, Poland
Cardinal Stefan Wyszy´
nski University, Warsaw, Poland
Systemics-PAB, Warsaw, Poland
Data availability
Data will be made available on request.
References
[1] A. Narayanan, et al., A comparative measurement study of commercial 5G
mmwave deployments, in: Proceedings of the IEEE INFOCOM 2022 - IEEE
Conference on Computer Communications, 2022, pp. 800–809, https://doi.org/
10.1109/INFOCOM48880.2022.9796693.
[2] ITU-T Recommendation P. 10/G.100, “Vocabulary for performance, quality of
service and quality of experience”, 11/ 2017.
[3] J. Mongay Batalla, S. Sujecki, J. Oko, J.M. Kelner, Cost-effective measurements of
5G radio resources allocation for telecom market regulator’s monitoring, in:
Proceedings of the 19th ACM International Symposium on Performance Evaluation
of Wireless Ad Hoc, Sensor, & Ubiquitous Networks (PE-WASUN’ 22), New York,
NY, USA, Association for Computing Machinery, 2022, pp. 83–90, https://doi.org/
10.1145/3551663.3558684.
[4] Electronic Communications Committee Report 341, Coverage availability and
performance aspects for 5G NR, July 2022.
[5] J. Mongay Batalla, E. Andrukiewicz, G. Peinado Gomez, P. Sapiecha, C.
X. Mavromoustakis, G. Mastorakis, J. Zurek, M. Imran, Security risk assessment for
5G networks - national perspective, IEEE Wirel. Commun. 27 (4) (2020) 16–22,
https://doi.org/10.1109/MWC.001.1900524. Aug.
[6] Digital Regulation Platform, “The relation between quality of service and quality of
experience”, Aug. 2020, https://digitalregulation.org/the-relation-between-qualit
y-of-service-and-quality-of-experience/(last access: March 2023).
[7] J. Mongay Batalla, M. Moshin, C.X. Mavromoustakis, K. Wesołowski,
G. Mastorakis, K. Krzykowska-Piotrowska, On deploying the internet of energy
with 5G open RAN technology including beamforming mechanism, Energies 15
(2022) 2429, https://doi.org/10.3390/en15072429.
[8] ETSI TS 128 533, “5G, management and orchestration, architecture framework”,
Release 15, Oct. 2018.
[9] J.M. Batalla, On analyzing video transmission over wireless WiFi and 5G C-band in
harsh IIoT environments, IEEE Access 8 (2020) 118534–118541, https://doi.org/
10.1109/ACCESS.2020.3005641.
[10] J. Saxon, N. Feamster, GPS-based geolocation of consumer IP addresses, in:
Proceedings of the Passive and Active Measurement, 23rd International
Conference, PAM 2022, Virtual Event, ISBN, 2022. March 28–30978-3-030-98785-
5.
[11] D. Zhong, A. Boukerche, Trafc signal control using deep reinforcement learning
with multiple resources of rewards, in: Proceedings of the 16th ACM International
Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, & Ubiquitous
Networks (PE-WASUN’ 19), New York, NY, USA, Association for Computing
Machinery, 2019, pp. 23–28, https://doi.org/10.1145/3345860.3361522.
[12] M.D. Foegelle, RF measurements in a 5G world, in: Proceedings of the IEEE
International Instrumentation and Measurement Technology Conference (I2MTC),
Houston, TX, USA, 2018, pp. 1–6, https://doi.org/10.1109/I2MTC.2018.8409874.
[13] A. Morton, “Active and passive metrics and methods (with hybrid types in-
between)”, request for comments: 7799 (RFC7799), ISSN: 2070-1721, 2016,
https://www.rfc-editor.org/rfc/rfc7799 (last access: March 2023).
[14] J. Rischke, P. Sossalla, S. Itting, F.H.P. Fitzek, M. Reisslein, 5G campus networks: a
rst measurement study, IEEE Access 9 (2021) 121786–121803, https://doi.org/
10.1109/ACCESS.2021.3108423.
[15] ETSI TS 123 502, “Procedures for the 5G system”, Release 15, June 2018.
[16] A. Gupta, R.K. Jha, A survey of 5G network: architecture and emerging
technologies, IEEE Access 3 (2015) 1206–1232, https://doi.org/10.1109/
ACCESS.2015.2461602.
[17] J.M. Batalla, Advanced multimedia service provisioning based on efcient
interoperability of adaptive streaming protocol and high efcient video coding,
Springer J. Real Time Image Process. (2016), https://doi.org/10.1007/s11554-
015-0496-4.
[18] T. Raúl Vargas, J.C. Guerri, P. Arce, Study on the impact of DASH streaming
services using energy efcient ethernet, in: Proceedings of the 18th ACM
Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, & Ubiquitous
Networks (PE-WASUN’ 21), New York, NY, USA, Association for Computing
Machinery, 2021, pp. 89–94, https://doi.org/10.1145/3479240.3488527.
[19] C.X. Wang, J. Bian, J. Sun, W. Zhang, M. Zhang, A survey of 5G channel
measurements and models, IEEE Commun.Surv. Tutor. 20 (4) (2018) 3142–3168,
https://doi.org/10.1109/COMST.2018.2862141.
[20] M.M. Ahamed, S. Faruque, 5G network coverage planning and analysis of the
deployment challenges, Sensors 21 (19) (2021) 6608, https://doi.org/10.3390/
s21196608.
[21] A. Tikhomirov, E. Omelyanchuk, A. Semenova, Recommended 5G frequency bands
evaluation, in: Proceedings of the Systems of Signals Generating and Processing in
the Field of on Board Communications, 2018, pp. 1–5, https://doi.org/10.1109/
SOSG.2018.8350639.
[22] G. Liu, Y. Huang, Z. Chen, L. Liu, Q. Wang and N. Li, "5G deployment: standalone
vs. non-standalone from the operator perspective," in Proceedings of the IEEE
Communications Magazine, vol. 58, no. 11, pp. 83–89, November 2020, 10.110
9/MCOM.001.2000230.
[23] ETSI TS 138 314, “5G NR layer 2 measurements”, Release 16, July 2020.
[24] Y. Kim, et al., Full dimension MIMO (FD-MIMO): the next evolution of MIMO in
LTE systems, in: Proceedings of the IEEE Wireless Communications 21, 2014,
pp. 26–33, https://doi.org/10.1109/MWC.2014.6812288. April.
[25] L. Bonati, M. Polese, S. D’Oro, S. Basagni, T. Melodia, Open, programmable, and
virtualized 5G networks: state-of-the-art and the road ahead, Comput. Netw. 182
(2020), 107516, https://doi.org/10.1016/j.comnet.2020.107516. VolumeISSN
1389-1286.
[26] A. Osseiran, J.F. Monserrat, P. Marsch, 5G Mobile and Wireless Communications
Technology, Cambridge University Press, 2016. ISBN-13: 978-1107130098June.
[27] T. Lackner, J. Hermann, F. Dietrich, C. Kuhn, M. Angos, J.L. Jooste, D. Palm,
Measurement and comparison of data rate and time delay of end-devices in
licensed sub-6GHz 5G standalone non-public networks, Proc. CIRP 107 (2022)
1132–1137, https://doi.org/10.1016/j.procir.2022.05.120. VolumePagesISSN
2212-8271.
[28] J. Xin, S. Xu, H. Zhang, S. Xiong, Efcient dynamic spectrum sharing for LTE-NR
networks, in: Proceedings of the 2nd International Conference on Electronics,
Communications and Information Technology (CECIT), Sanya, China, 2021,
pp. 538–543, https://doi.org/10.1109/CECIT53797.2021.00101.
[29] G. So´
os, D. Ficzere, P. Varga, Z. Szalay, Practical 5G KPI measurement results on a
non-standalone architecture, in: Proceedings of the NOMS 2020 - 2020 IEEE/IFIP
Network Operations and Management Symposium, Budapest, Hungary, 2020,
pp. 1–5, https://doi.org/10.1109/NOMS47738.2020.9110457.
[30] M.N. Tahir, M. Katz, Performance evaluation of IEEE 802.11p, LTE and 5G in
connected vehicles for cooperative awareness, Eng. Rep. 4 (4) (2022) e12467,
https://doi.org/10.1002/eng2.12467.
[31] L. Chiaraviglio, et al., Massive measurements of 5G exposure in a town:
methodology and results, IEEE Open J. Commun. Soc. 2 (2021) 2029–2048,
https://doi.org/10.1109/OJCOMS.2021.3107287.
[32] ETSI TS 138 306, “User equipment (UE) radio access capabilities”, version 17.3.0.
Release 17, January 2023.
[33] ETSI TS 138 214, “Physical layer procedures for data”, Release 17. May 2022.
[34] C.X. Wang, J. Bian, J. Sun, W. Zhang, M. Zhang, A survey of 5G channel
measurements and models, IEEE Commun. Surv. Tutor. 20 (4) (2018) 3142–3168,
https://doi.org/10.1109/COMST.2018.2862141. Fourthquarter.
[35] Body of European Regulators for Electronic Communications, BEREC Guidelines on
Geographical Surveys of Network Deployments, BoR, 2020, p. 42. https://www.be
rec.europa.eu/sites/default/files/files/document_register_store/2021/2/BoR%
J. Mongay Batalla et al.

Computer Networks 235 (2023) 109980
11
20%2820%29%2042_Cor.1_Guidelines_BBgeoSurveys_final.pdf (last access: May
2023).
[36] R.B. Shinazi, Probability with Statistical Applications, Springer, 2022. ISBN-13:
978-3030936358.
[37] R. Zieli´
nski, Condence interval for proportion”, Math. Appl. 36 (2008) 76–90,
https://doi.org/10.14708/ma.v36i50/09.1503 (Matematyka Stosowana)No. 50/
09(in Polish).
[38] C.J. Clopper, E.S. Pearson, The use of condence or ducial limits illustrated in the
case of the binomial, Biometrika 26 (1935) 404–413, https://doi.org/10.1093/
biomet/26.4.404. Pp.
[39] L.S. Lasdon, R.L. Fox, M.W. Ratner, Nonlinear optimization using the generalized
reduced gradient method. Revue française d’automatique, informatique, recherche
op´
erationnelle, Rech. Op´
er. 8 (1974) 73–103. Volumeno. V3, http://www.num
dam.org/item/RO_1974__8_3_73_0/ (last access: March 2023).
[40] H. Tataria, M. Sha, A.F. Molisch, M. Dohler, H. Sj¨
oland, F. Tufvesson, 6G wireless
systems: vision, requirements, challenges, insights, and opportunities, Proc. IEEE
109 (7) (2021) 1166–1199, https://doi.org/10.1109/JPROC.2021.3061701. July.
[41] M. Alsabah, et al., 6G wireless communications networks: a comprehensive survey,
IEEE Access 9 (2021) 148191–148243, https://doi.org/10.1109/
ACCESS.2021.3124812.
Dr. Jordi Mongay Batalla is an Associate Professor at the
Warsaw University of Technology. He is a CEI expert of the
European Union Agency for Cybersecurity (ENISA) for 5G
cybersecurity and an expert in the Ofce of Electronic Com-
munications, Poland. He has coordinated more than 10
research projects. He is the editor of four books and author of
more than 200 papers published in books, and international
journals (IEEE ComMag, IEEE WCM, IEEE JSAC, ACM CSUR,
etc.). He is/ has been a guest editor and member of the Editorial
Board in more than 10 international journals.
Prof. Slawomir Sujecki (SMIEEE) received his Ph. D. (1997)
and D. Sc. (2010) degrees from the Warsaw University of
Technology. He joined the University of Nottingham in 2000
and was appointed a Lecturer and an Associate Professor in
2002 and 2012, respectively. Currently, he is with the Wroclaw
University of Science and Technology and the Military Uni-
versity of Technology, Poland. Prof. Sujecki is a Life Member of
OSA, a member of the TPC of NUSOD Conference and Associate
Editor of Optical and Quantum Electronics. Prof. Sujecki’s
research interests focus around design and modeling of pho-
tonic devices and telecom networks.
Dr. Jan M. Kelner is an Associate Professor at the Military
University of Technology (MUT), Poland. Since 2021, he is the
director of the Institute of Communications Systems. He is a
Polish principal voting member of the Information Systems
Technology Panel of the NATO Science and Technology Or-
ganization and expert in the Ofce of Electronic Communica-
tions in Poland. He received his Ph.D. (2011) and D.Sc. (2020)
degrees from the MUT and AGH University of Science and
Technology, respectively. He is author of over 200 papers and
principal investigator or researcher in many research projects
related to wireless and mobile networking.
Dr Piotr ´
Sliwka is an Assistant professor in the Faculty of
Mathematics and Natural Sciences at Cardinal Stefan Wys-
zynski University in Warsaw (Poland). His eld of interest is
statistical analysis and data modeling in insurance, economics,
medicine, biology and computer networks. In those elds, he
has written several papers in the last years.
Dariusz Zmysłowski was a graduate of the Electronics Depart-
ment at the Military University of Technology in Warsaw with
an MSc E.E. He completed postgraduate studies at George
Washington University in Project Management. He has gained
professional experience carrying out projects of research &
implementation for the government and the public sector.
Since the beginning of his professional way, he has been
combining engineering with academic activities- rstly at the
Institute of Telecommunications of the Military University of
Technology in Warsaw. He specialises in QoS/QoE in mobile
networks, GNSS, reliability & security, business development
of new technologies, and integrational projects management.
J. Mongay Batalla et al.