# Javier Zazo

7.19

· Doctor Ingeniero de TelecomunicacionesAbout

14

Research items

831

Reads

34

Citations

Introduction

**Skills and Expertise**

Network

Cited

Followers

Following

Research

Research Items (14)

Multiagent systems where the agents interact among themselves and with an stochastic environment can be formalized as stochastic games. We study a subclass, named Markov potential games (MPGs), that appear often in economic and engineering applications when the agents share some common resource. We consider MPGs with continuous state-action variables, coupled constraints and nonconvex rewards. Previous analysis are only valid for very simple cases (convex rewards, invertible dynamics, and no coupled constraints); or considered deterministic dynamics and provided open-loop (OL) analysis, studying strategies that consist in predefined action sequences. We present a closed-loop (CL) analysis for MPGs and consider parametric policies that depend on the current state and where agents adapt to stochastic transitions. We provide verifiable, sufficient and necessary conditions for a stochastic game to be an MPG, even for complex parametric functions (e.g., deep neural networks); and show that a CL Nash equilibrium (NE) can be found (or at least approximated) by solving a related optimal control problem (OCP). This is useful since solving an OCP---a single-objective problem---is usually much simpler than solving the original set of coupled OCPs that form the game---a multiobjective control problem. This is a considerable improvement over previously standard approach. We illustrate the theoretical contributions with an example by applying our approach to a noncooperative communications engineering game. We then solve the game with a deep reinforcement learning algorithm that learns policies that closely approximates an exact variational NE of the game.

We solve a communication problem between a UAV and a set of receivers, in the presence of a jamming UAV, using differential game theory tools. We propose a new approach in which this kind of games can be approximated as pursuit-evasion games. The problem is posed in terms of optimizing capacity, and it is solved in two ways: firstly, a surrogate function approach is used to approximate it as a pursuit-evasion game; secondly, the game is solved without that approximation. In both cases, Isaacs equations are used to find the solution. Finally, both approaches are compared in terms of relative distance and complexity.

- Aug 2016
- 2016 24th European Signal Processing Conference

We address the issue of large scale network security. It is
known that traditional game theory becomes intractable when
considering a large number of players, which is a realistic situation
in today’s networks where a centralized administration
is not available. We propose a new model, based on mean
field theory, that allows us to obtain optimal decentralised
defence policy for any node in the network and optimal attack
policy for an attacker. In this way we settle a promising
framework for the development of a mean field game theory
of large scale network security. We also present a case study
with experimental results.

- Jun 2016
- 2016 IEEE Statistical Signal Processing Workshop

We solve a communication problem between a UAV and a set of relays, in the presence of a jamming UAV, using differential game theory tools. The standard solution involves a set of coupled Bellman equations which are hard to solve. We propose a new approach in which this kind of games can be approximated as pursuit-evasion games. The problem is posed in terms of optimizing capacity and it is approximated as a zero-sum, pursuit-evasion game. This game is solved using a set of differential equations known as Isaacs equations and simulations are run in order to validate the results.

Demand-side management presents significant benefits in reducing the energy load in smart grids by balancing consumption demands or including energy generation and/or storage devices in the user's side. These techniques coordinate the energy load so that users minimize their monetary expenditure. However, these methods require accurate predictions in the energy consumption profiles, which make them inflexible to real demand variations. In this paper we propose a realistic model that accounts for uncertainty in these variations and calculates a robust price for all users in the smart grid. We analyze the existence of solutions for this novel scenario, propose convergent distributed algorithms to find them, and perform simulations considering energy expenditure. We show that this model can effectively reduce the monetary expenses for all users in a real-time market, while at the same time it provides a reliable production cost estimate to the energy supplier.

We use real measurements of the underwater channel to simulate a whole underwater RF wireless sensor networks, including propagation impairments (e.g., noise, interference), radio hardware (e.g., modulation scheme, bandwidth, transmit power), hardware limitations (e.g., clock drift, transmission buffer) and complete MAC and routing protocols. The results should be useful for designing centralized and distributed algorithms for applications like monitoring, event detection, localization and aid to navigation. We also explain the changes that have to be done to Castalia in order to perform the simulations.

We analyze the problem of localization algorithms for underwater sensor networks. We first characterize the underwater channel for radio communications and adjust a linear model with measurements of real transmissions. We propose an algorithm where the sensor nodes collaboratively estimate their unknown positions in the network. In this setting, we assume low connectivity of the nodes, low data rates, and nonzero probability of lost packets in the transmission. Finally, we consider the problem of a node estimating it's position in underwater navigation. We also provide simulations illustrating the previous proposals.

- Mar 2016
- 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

- Mar 2016
- 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

In a noncooperative dynamic game, multiple agents operating in a changing environment aim to optimize their utilities over an infinite time horizon. Time-varying environments allow to model more realistic scenarios (e.g., mobile devices equipped with batteries, wireless communications over a fading channel, etc.). However, solving a dynamic game is a difficult task that requires dealing with multiple coupled optimal control problems. We focus our analysis on a class of problems, named \textit{dynamic potential games}, whose solution can be found through a single multivariate optimal control problem. Our analysis generalizes previous studies by considering that the set of environment's states and the set of players' actions are constrained, as it is required by most of the applications. And the theoretical results are the natural extension of the analysis for static potential games. We apply the
analysis and provide numerical methods to solve four key example problems, with different features each: energy demand control in a smart-grid network, network flow optimization in which the relays have bounded link capacity and limited battery life, uplink multiple access communication with users that have to optimize the use of their batteries, and two optimal scheduling games with nonstationary channels.

- Apr 2015
- IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP

Optimum scheduling is a key objective in many communications systems where different users have to share a common resource. Typically, centralized implementations are capable of guaranteeing certain fairness. In our approach, we follow a different path modeling the scheduling process as a dynamic infinite horizon discrete-time game. This formulation allows us to include any kind of dynamics and distributed implementations. Despite, these games are very difficult to solve, we are able to show that they are in fact dynamic potential games equivalent to a non-stationary multivariate optimum control problem. The dynamic control problem is solved via an augmented Bellman equation including time as an extra state.

- Apr 2015
- IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP

We combine model-based methods and distributed stochastic approximation to propose a fully distributed algorithm for noncon-vex optimization, with good empirical performance and convergence guarantees. Neither the expression of the objective nor its gradient are known. Instead, the objective is like a " black-box " , in which the agents input candidate solutions and evaluate the output. Without central coordination, the distributed algorithm naturally balances the computational load among the agents. This is especially relevant when many samples are needed (e.g., for high-dimensional objectives) or when evaluating each sample is costly. Numerical experiments over a difficult benchmark show that the networked agents match the performance of a centralized architecture, being able to approach the global optimum, while none of the individual nonco-operative agents could by itself.

- Nov 2014
- European Signal Processing Conference

Cognitive radio represents a promising paradigm to further increase transmission rates in wireless networks, as well as to facilitate the deployment of self-organized networks such as femtocells. Within this framework, secondary users (SU) may exploit the channel under the premise to maintain the quality of service (QoS) on primary users (PU) above a certain level. To achieve this goal, we present a noncooperative game where SU maximize their transmission rates, and may act as well as relays of the PU in order to hold their perceived QoS above the given threshold. In the paper, we analyze the properties of the game within the theory of variational inequalities, and provide an algorithm that converges to one Nash Equilibrium of the game. Finally, we present some simulations and compare the algorithm with another method that does not consider SU acting as relays.

- Jun 2014
- IEEE Sensor Array and Multichannel Signal Processing Workshop

Doubly-stochastic matrices are usually required by consensus-based distributed algorithms. We propose a simple and efficient protocol and present some guidelines for imple-menting doubly-stochastic combination matrices even in noisy, asynchronous and changing topology scenarios. The proposed ideas are validated with the deployment of a wireless sensor network, in which nodes run a distributed algorithm for robust estimation in the presence of nodes with faulty sensors.

Current institution

Co-authors

**Top co-authors**

**All co-authors (17)**