Joint admission control & interference avoidance in self-organized femtocells

Abstract

In this paper, we consider a femtocell deployment scenario in which radio resources are shared among self-organized femtocells. We propose a distributed Admission Control Mechanism (ACM) for traffic load balancing among sub-carriers when there are multiple Quality of Service (QoS) classes. Furthermore, we propose a mechanism based on Reinforcement Learning (RL) for slot allocation to the traffic streams on different sub-carriers, which is employed by each Femto Access Point (FAP) to mitigate interference among femtocells and the underlaid macrocell. Through simulations, the performance of the proposed scheme is evaluated where it is shown that femtocells are able to coexist with the overlay macrocell network with no information exchange and by relying only on local information.

Full text

Page 1

Joint Admission Control & Interference Avoidance
in Self-Organized Femtocells
Suneth Namal∗, Kaveh Ghaboosi∗, Mehdi Bennis∗, Allen B. MacKenzie†, and Matti Latva-aho∗
∗Centre for Wireless Communications, University of Oulu, Finland
†Department of Electrical and Computer Engineering, Virginia Tech, USA
Email: {namal, kaveh.ghaboosi, mehdi.bennis, matti.latva-aho}@ee.oulu., mackenab@vt.edu
Abstract——In this paper, we consider a femtocell deploy-
ment scenario in which radio resources are shared among
self-organized femtocells. We propose a distributed Admission
Control Mechanism (ACM) for trafc load balancing among
sub-carriers when there are multiple Quality of Service (QoS)
classes. Furthermore, we propose a mechanism based on Rein-
forcement Learning (RL) for slot allocation to the trafc streams
on different sub-carriers, which is employed by each Femto
Access Point (FAP) to mitigate interference among femtocells and
the underlaid macrocell. Through simulations, the performance
of the proposed scheme is evaluated where it is shown that
femtocells are able to coexist with the overlay macrocell network
with no information exchange and by relying only on local
information.
I. INTRODUCTION
A two-tier cellular network, consisting of macrocell sites un-
derlaid with short-range femtocells, is a potential candidates to
economically improve the capacity of next generation wireless
systems. Femtocell networks have been extensively studied by
the 3GPP standardization body [1], [2] and their capacity has
been analyzed in [3]––[5] for CDMA based systems.
Recently, a new type of indoor Base Station (BS), called
femtocell, has gained the attention of the industry [1], [3]
. A femtocell is a low-cost and low-power BS deployed by
the end-users, designed to extend indoor coverage. Femtocells
are connected to the network of the operator over a backhaul
connection such as Digital Subscriber Line (DSL) or optical
ber. Meanwhile, femtocells also provide coverage to the end
customers using a cellular network standard, e.g., Universal
Mobile Telecommunication System (UMTS), Wireless Inter-
operability for Microwave Access (WiMAX), and Long-Term
Evolution (LTE).
Femtocells aim to enhance the poor indoor coverage, boost
the overall spectral efciency, and ofoad the overlay macro-
cell trafc [1]. Although femtocells provide signicant benets
for mobile operators, their introduction comes with many
challenges. Among these is the cross-tier interference between
macro- and femtocells, as well as co-tier interference arising
among femtocells. This calls for effective interference manage-
ment strategies, such as distributed power allocation, resource
partitioning, and other interference avoidance techniques. Un-
doubtedly, interference avoidance has never been a trivial task
neither in macrocell deployments nor in femtocell networks.
Due to the selsh nature of femtocells and uncertainty on
their number and locations, operators must use optimal and
dynamic approaches rather than the classical static network
planning and optimization to avoid interference. In order to
successfully react to the changes of the trafc, and minimize
interference in femtocell deployments, the use of sophisticated
self-organization techniques is paramount. Self-organization
will allow femtocells to integrate themselves into the network
of the operator, learn about their environment (such as neigh-
boring femtocells and local interference map) and tune their
parameters (transmit power and carrier frequency) accordingly.
different self-organization strategies for femtocells have
been introduced. In [4], a power control method was proposed
for pilot and data channels in UMTS networks that ensures
a constant coverage femtocell radius. Each femtocell sets
the transmit power such that on average it is equal to the
power received from the closest macrocell at a target femtocell
radius. In [6], a method was presented for coverage adaptation
for UMTS networks using information on mobility events
of outdoor passing and indoor users. Each femtocell sets its
power to a value that on average minimizes the total number of
attempts of outdoor passing users to connect to such femtocell.
In [7], a distributed utility-based signal to interference plus
noise ratio (SINR) adaptation at femtocells was proposed
in order to alleviate cross-tier interference at the macrocell
from cochannel femtocells. In [8], interference avoidance
using a time-hopped Code Division Multiple Access (CDMA)
physical layer and sectorial antennas is investigated. These
approaches are mostly based on wide-band code division
multiple Access (WCDMA) networks, and do not mitigate
interference through sub-channel allocation, which is a very
important feature of current Orthogonal Frequency Division
Multiple Access (OFDMA) systems. Recent works investigat-
ing interference mitigation for OFDM femtocell networks can
be found in [9], [10] for frequency resource partitioning, and
[11] [12] using tools from game theory.
In this paper, we consider a network consisting of stand-
alone femtocells overlaid by a macrocell and propose a dis-
tributed ACM for trafc load balancing among sub-carriers
when there are multiple QoS classes. We also introduce an
algorithm based on RL for slot allocation to the trafc streams
on different sub-carriers to be employed by each FAP in
order to mitigate the interference among femtocells and the
underlaid macrocell. To allocate sub-carriers to an incoming
trafc, a FAP maintains a counter for each sub-carrier and
stochastically counts it down using the outage probability
???? ???????????????????????????????????????????????????

Page 2

perceived on the sub-carrier. The trafc is assigned to a set
of sub-carriers, for which the counter reaches zero within a
predened time interval. Once trafc streams are assigned to
the sub-carriers, each FAP independently runs an algorithm
based on RL to perform slot allocation for the trafc streams
in each sub-carrier, which is meant for further interference
mitigation among femtocells and the underlaid macrocell. The
rest of the paper is organized as follows: Section II presents
the distributed admission control mechanisms. Section III
examines the machine learning approach. Numerical results
are given in Section IV and conclusions are drawn in Section
V.
II. DISTRIBUTED ADMISSION CONTROL MECHANISM
In this section, we rst describe the assumptions made
throughout the paper and then explain the proposed mecha-
nisms in detail.
A. Denitions
We assume that there are nf femtocells sharing nsubsub-
carriers. Time is assumed to be divided into consecutive
frames, each of which contains nslot slots. We denote the
number of QoS classes dened for trafc classication by
nqos. Throughout the paper, to refer to a particular QoS class
we always use index i while index j is only used to represent
a sub-carrier. Similarly, index k is employed to refer to a slot
while index t is particularly used for referring to a certain
frame. Finally, we use index z to represent the FAP of interest.
Thus, {z,i,j,k,t} denotes FAP z’’s slot k of sub-carrier j in
frame t, which carries trafc of type QoS class i.
The average outage probability of QoS class i on sub-
carrier j is denoted by pij = Pr{Γij < γth
denotes the Signal-to-Interference plus Noise Ration (SINR)
and γth
i
denotes the threshold SINR of QoS class i. Note
that depending on perceived interference level per sub-carrier,
distinct QoS classes may have different outage probabilities.
Each FAP, from its own perspective and based on the reports
provided by its Femto User Equipments (FUE), estimates the
outage probabilities for different QoS classes and updates them
every frame.
i}, where Γij
B. Stochastic Admission Control Algorithm
In the context of stand-alone self-organized femtocells, we
now describe the proposed solution, which takes into account
the QoS requirements of each trafc stream, perceived inter-
cell interference, and fairness in utilization of each accessible
sub-carrier. In contrast to [13] where the decision on what
sub-carrier to choose is made by the radio users, here the
FAPs take responsibility for handling sub-carrier allocation.
Moreover, instead of performing the load balancing in a
user-oriented fashion, the proposed ACM is entirely stream-
oriented, which allows users to have more than one simul-
taneous trafc streams with different QoS requirements and
makes the accounted system model be closer to the practical
networks.
Algorithm 1 Stochastic Admission Control
1: Input: Number of sub-carriers j = 1,...,nsub, QoS class i of incoming
trafc at time t, γth
i, Mi, u ∼ U (0,1);
2: Init: m = 0;
3: for all Sub-carriers without MUE do
4:if Xj(t − 1) = 0 then
5:Choose Xj(t) ∼ U {1,2,...,W};
6:else
7:
Xj(t) ← Xj(t − 1);
8: end if
9: end for
10: while m < Mido
11:for all Sub-carriers without MUE do
12:if Xj(t) > 0 then
13:Choose u ∼ U (0,1);
14:if u > pijthen
15:
Xj(t) ← Xj(t) − 1;
16:if Xj(t) = 0 then
17:
m ← m + 1;
18: end if
19:end if
20: end if
21: end for
22: end while
23: if m = Mithen
24:Allocate the trafc to all sub-carriers with Xj(t) = 0;
25: else
26: Allocate the trafc to a subset of Misub-carrier with Xj(t) = 0;
27: end if
Every FAP maintains a set of counters, each associated
with a particular sub-carrier [14]. Upon arrival of an incoming
trafc of type QoS class i, those counters which have reached
zero during the previous allocation attempt for another trafc
stream, are loaded with randomly chosen integers taken uni-
formly from the set {1,2,...,W}, where W is a pre-dened
window size. We denote the initial value of the counter of the
sub-carrier j by Xj. On the other hand, the content of those
counters that have not reached zero and have been paused
by the end of previous sub-carrier allocation phase are not
changed and are counted down from the last operating point.
All counters are driven by a common master clock and by each
clock beat, the content of each counter is either decremented
by one or remains unchanged. The content of sub-carrier j’’s
counter remains unchanged with probability pij; otherwise, its
content is decremented by one (See Fig. 1).
Fig. 1.
cally counted down using the perceived outage probability of QoS class i in
that sub-carrier pij.
The counter associated to sub-carrier j loaded by Xjand stochasti-
Given that the incoming trafc is of type QoS class i, a set of
Misub-carriers will be chosen to be assigned to the incoming
stream if their corresponding counters reach zero before the
counters of the remaining sub-carriers. In this case, counting
down of other sub-carriers’’ counters is stopped and their
????

Page 3

contents remain unchanged until the next incoming stream.
Once the request for another stream is received by the FAP,
the counter of previously chosen sub-carriers will be reloaded
with new randomly selected integer numbers, while other
counters will begin to count down from the point where they
have been stopped. Notice that the outage probability drives
the count down process, which depends on the QoS class
of the new incoming stream. The reason for preserving the
last contents of the sub-carriers’’ counters while reloading the
counters of previously selected sub-carriers with new numbers
is to keep track of the history of the trafc distribution among
all accessible sub-carriers. When a Macro User Equipment
(MUE) appears on a sub-carrier, the counter of that sub-carrier
is disabled and will be re-enabled once the MUE vacates the
sub-carrier. The pseudocode for the proposed ACM is given
by Algorithm 1.
III. DISTRIBUTED REINFORCEMENT BASED
INTERFERENCE MANAGEMENT
A. Q-Learning: An Overview
The Q-learning model consists of a set of states S and ac-
tions A aiming at nding a policy that maximizes the observed
rewards over the interaction time of the agents/players (i.e.,
femtocells). Every FAP explores its environment, observes its
current state s, and takes a subsequent action a, according
to a decision policy π : s → a. With their ability to learn,
the knowledge about other players’’ strategies is not needed.
Instead, a Q-function maintains the knowledge about other
players in the network, based on which decisions are made.
B. Reinforcement based Slot Allocation Algorithm
In this subsection, we describe the proposed algorithm based
on Q-learning, used for rearranging the slot allocations per
sub-carrier during a frame. Once done with ACM, each FAP
independently runs this algorithm to (re)assign the slots of a
frame to the trafc streams allocated in each sub-carrier. Here,
we assume that there are nstrtrafc streams that are indexed
by l and each of which has a target rate ϑl
stream, we dene an indication bit denoted by el
whether sub-carrier j has been assigned to stream l during
ACM phase. The agent, action, state, and reward associated
to the proposed Q-learning based mechanism are dened as
follows:
• Agent z: FAPz, z ∈ {1,...,nf}.
• Action Az
t
]j∈{1,...,nsub},k∈{1,...,nslot}, where
az,j,k
t
∈ Tz,j
t
? {l|el
at frame t, where az,j,k
t
indicates whether FAPz wishes
to transmit part of stream l in slot k of sub-carrier j.
• State Sz
target. For each
j, representing
t = [az,j,k
j= 1} being the action of FAPz
t= [sz,l
t]l∈{1,...,nstr}, where
sz,l
t
=
?
0
1
for
for
ϑl
ϑl
t< ϑl
t? ϑl
target
target
and ϑl
t??
(j,k)∈{(j,k)|az,j,k
t=?nstr
t−1=l}log2(1 + SINRz,j,k
l=1min(2ϑl
t
).
• Reward Rz
of FAPzat frame t.
target−ϑl
t,ϑl
t) is the reward
Algorithm 2 describes the Q-learning process, in which
FAPz performs the exploration step with probability αz
addition, λzand βzare FAPzlearning rate and discount factor,
respectively. Essentially, Q(Sz
current reward and the predicted reward if all future actions are
selected optimally. Note that without ACM, Az
[az,j,k
t
]nsub×nslot, with az,j,k
t
∈ {1,2,...,nstr}, however, the
target state is always [1]1×nstr. The Q-learning algorithm to
achieve the above goal is given as follows:
t. In
t,Az
t) is updated based on the
twould become
Q(Sz
λz[Rz
t,Az
t+1+ βzmax
t) = (1 − λz)Q(Sz
t,Az
t+1,Az
t) +
(1)
Az
t+1
Q(Sz
t+1)].
Algorithm 2 Q-learning Algorithm for Time Slot Allocation
1: Input: Number of sub-carriers nsub, Number of slots per sub-carrier
nslot, Number of trafc streams nstr, el
2: Init: Q(.,.) = 0 initialize Q-value for each FAPz
3: while 1 do
4:if (rand < αz
5: Choose an action at random; [exploration]
6: else
7: Choose Az
t+1= argmaxAz
8:[exploitation]
9:end if
10:Receive immediate reward at frame t + 1: Rz
11: Observe new state Sz
12:Update Q-table as given in (1);
13: end while
j,∀j,l;
t) then
t+1Q(Sz
t+1,Az
t+1);
t+1;
t+1;
IV. NUMERICAL RESULTS
The proposed scheme is simulated and the success probabil-
ity towards the number of femtocells is presented in Figure 2.
The parameters shown in Table I are used for the simulation.
We dene three different QoS classes for data, voice and
video trafc. The success probability is observed for multiple
QoS classes while changing the number of subcarriers in
the system. The results in Figure 2 are obtained assuming
only deployed femtocells in the region without any Macro
Base Station (MBS). Thus, the interferences from the neigh-
boring FAPs are taken into account to calculate the success
probability. It is seen that the success probability for data
trafc is comparatively higher with respect to voice and video
trafc. Ultimately, the success probability of video trafc drops
signicantly with an increasing number of femtocells. For a
general case, the success probability of all trafc classes drops
when the number of subcarriers is increased. In particular,
we observe the increasing number of subcarriers does not
signicantly affect the success rate of the voice trafc unlike
video trafc class.
As observed in Figure 3, the overall capacity signicantly
increases with the number of subcarriers. In other words,
increasing the number of FAPs decreases the overall capacity.
It is found that the rate for video trafc is always high
irrespective of the number of subcarriers in the system because
of the high demand for subcarriers.
Next, the presence of a Macro base station is considered
and the success rate is shown in Figure 4 as a function of the
????

Page 4

TABLE I
SIMULATION PARAMETERS
Parameter
Tile Area
FAP transmit power
MBS transmit power
Shadowing standard deviation
Value
1km2
20 dBm
36 dBm
6 dBm
Fig. 2.
number of sub-carriers in the system.
Success probability for Data, Voice and Video trafc for different
number of femtocells deployed in the region. It is shown that
the data trafc has the highest success rate in all scenarios and
for different number of subcarriers. In particular, we nd an
optimal value for each curve because of the presence of the
macro base station. When the number of deployed femtocells
in the region is quite low, there is a high probability that the
communication may get interrupted by the macro base station.
The success probability converges around a xed value even
051015 20253035 404550
0.93
0.94
0.95
0.96
0.97
0.98
0.99
1
Number of Femto Access Points
Capacity / bits per transmission
Number of subcarriers = 05
0510 15202530354045 50
1.86
1.88
1.9
1.92
1.94
1.96
1.98
2
Number of Femto Access Points
Capacity / bits per transmission
Number of subcarriers = 10
Data Traffic
Voice Traffic
Video Traffic
Data Traffic
Voice Traffic
Video Traffic
Fig. 3.
number of sub-carriers in the system.
The comparison shows the capacity improvement by increasing
with the increased number of FAPs in the region. The graphs
show the optimal number of FAPs can be deployed to improve
the success rate. This optimality can be observed particularly
with the video trafc. Furthermore, the results imply the fact
that, success rate for same trafc class does not signicantly
change with the number of FAPs is high.
05 1015 2025 3035 404550
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Number of Femto Access Points
Success Probability
Number of subcarriers = 05
Number of subcarriers = 10
Number of subcarriers = 15
Data
Voice
Video
Fig. 4.
number of sub-carriers in the system when a macro base station is in the
region.
Success probability for Data, Voice and Video trafc for different
0510 1520 2530354045 50
0.995
0.996
0.997
0.998
0.999
1
1.001
Number of subcarriers = 05
Number of Femto Access Points
Capacity / bits per transmission
05 1015202530 354045 50
1.992
1.994
1.996
1.998
2
2.002
Number of subcarriers = 10
Number of Femto Access Points
Capacity / bits per transmission
Data Traffic
Voice Traffic
Video Traffic
Data Traffic
Voice Traffic
Video Traffic
Fig. 5.
number of sub-carriers in the system when a macro base station is in the
region.
The comparison shows the capacity improvement by increasing
The Figure 5 presents the overall system capacity towards
the number of FAPs in the region. We notice capacity im-
provement by the presence of a macro base station. Though,
we expected to see an optimum capacity level because of the
macro base station it is not signicant enough to be visible in
the graphs. But, a signicant capacity improvement is noticed
????

Page 5

compared to the scenario where macro base station does not
exist (Figure 3). The system capacity for all trafc classes lies
on one another when the number of FAPs are quite low in
the region. The increased number of FAPs shows a signicant
change in the capacity.
V. CONCLUSION
We have introduced and simulated a novel distributed ACM
to assign each incoming trafc stream to a set of sub-carriers
based on a stochastic approach. We also proposed an algorithm
based on Q-learning for rearranging the slot allocations per
sub-carrier during a frame. Using system level simulations,
we have shown the proposed ACM convergence. Moreover,
we have shown that femtocells can be deployed as an underlay
to the macrocell network with no information exchange.
ACKNOWLEDGEMENT
The authors would like to thank the Finnish funding agency
for technology and innovation, Elektrobit, Nokia and Nokia
Siemens Networks for supporting this work. This work has
been performed in the framework of the ICT project ICT-4-
248523 BeFEMTO, which is partly funded by the EU.
REFERENCES
[1] 3GPP, ““3G home node b study item,”” Technical Specication Group
Radio Access Networks, 3GPP TR 25.820 version 8.2.0 release 8,
Technical Report, September 2008.
[2] C. H. M. de Lima and G. T. F. de Abreu, ““Clusterization for robust
geographic routing in wireless sensor networks,”” in IEEE Wireless Com-
munications & Networking Conference (WCNC), Las Vegas, Nevada,
USA, 31 March––3 April 2008.
[3] V. Chandrasekhar, J. G. Andrews, and A. Gatherer, ““Femtocell networks:
a survey,”” IEEE Communications Magazine, vol. 46, no. 9, pp. 59––67,
September 2008.
[4] H. Claussen, L. Ho, and L. Samuel, ““An overview of the femtocell
concept,”” Bell Labs Technical Journal, vol. 13, no. 1, pp. 221––245,
2008.
[5] A. Nguyen, N. Milosavljevic, Q. Fang, J. Gao, and L. J. Guibas,
““Landmark selection and greedy landmark-descent routing for sensor
networks,”” in 26th Annual Joint Conference of the IEEE Computer and
Communications Societies, 2007. INFOCOM 2007.
2007, pp. 661––669.
[6] H. Claussen, L. Ho, and L. Samuel, ““Self-optimization coverage for
femtocell deployments,”” In Wireless Telecommunications Symposium,
vol. 24-26, no. 1, pp. 278U––285, 2008.
[7] V. Chandrasekhar, T. Muharemovic, Z. Shen, J. G. Andrews, and
A. Gatherer, ““Power control in two-tier femtocell networks,”” IEEE
Transactions on Wireless Communications, vol. 8, no. 8, August 2009.
[8] V. Chandrasekhar and J. G. Andrews, ““Uplink capacity and interference
avoidance for two-tier cellular networks,”” in IEEE Global Communica-
tions Conference (GLOBECOM), Washington, DC, November 2009.
[9] Z. Bharucha, H. Haas, G. Auer, and I. Cosovic, ““Femtocell resource
partitioning,”” IEEE Globecom, November 2009.
[10] I. Guvenc, M. R. Jeong, M. E. Sahin, H. Xu, and F. Watanabe,
““Interference avoidance in 3gpp femtocell networks using resource
partitioning and sensing,”” IEEE Indoor Outdoor Femtocells (IOFC)
Workshop (co-located with PIMRC 2010), September 2010.
[11] S. Guruacharya, D. Niyato, E. Hossain, and D. I. Kim, ““Hierarchical
competition in femtocell-based cellular networks,”” IEEE Globecom,
December 2010.
[12] M. Bennis and M. Debbah, ““On spectrum sharing in underlaid femtocell
networks,”” IEEE PIMRC IOFC workshop, September 2010.
[13] K. Ghaboosi, A. B. MacKenzie, L. A. DaSilva, A. S. Abdallah, and
M. Latva-aho, ““A channel selection mechanism based on incumbent
appearance expectation for cognitive networks,”” in IEEE Wireless Com-
munications and Networking Conference, 2009. WCNC 2009, vol. 1,
5––8 April 2009, pp. 1320––1325.
IEEE, 6––12 May
[14] V. Rodriguez and F. Jondral, ““Simple, adaptively-prioritised, spatially-
reusable medium access control through the dutch auction, with decen-
tralised implementation for synchronised terminals,”” in 14th IEEE Sym-
posium on Communications and Vehicular Technology in the Benelux,
2007, 15 November 2007, pp. 1––3.
????