Atilla Eryilmaz

• The Ohio State University

One of the most fundamental, and yet relatively less explored, goals in transfer learning is the efficient means of selecting top candidates from a large number of previously trained models (optimized for various "source" tasks) that would perform the best for a new "target" task with a limited amount of data. In this paper, we undertake this goal...

In many applications, we regularly face the fundamental problem of allocating a common resource (funding, time, energy, etc.) among a network of processes that evolve in a continuous-space according to a birth-death dynamics. The state of each process tends to gradually improve with the resource and gradually degrade without it. Formulated as a Con...

In real-world networks, we regularly face the effect of propagating failures over networks, for example, rumors spread over social networks, outages spread over power networks, viruses spread over communication and biological networks. Often, these failures spread over a network of agents with unknown and potentially diverse degrees of vulnerabilit...

Edge caching has been widely implemented to efficiently serve data requests from end users. Numerous edge caching policies have been proposed to adaptively update the cache contents based on various statistics. One critical statistic is the miss cost, which could measure the latency or the bandwidth/energy consumption to resolve the cache miss. Exi...

We introduce a framework and optimal ‘fresh’ caching for a content distribution network (CDN) comprising a front-end local cache and a back-end database. The data content is dynamically updated at a back-end database and end-users are interested in the most-recent version of that content. We formulate the average cost minimization problem that capt...

The ever-growing end user data demands, and the reductions in memory costs are fueling edge-caching deployments. Caching at the edge is substantially different from that at the core and needs to consider the nature of individualized data demands. For example, an individual user may not be interested in requesting the same data item again, if it has...

With the development of edge networks and mobile computing, the need to serve heterogeneous data sources at the network edge requires the design of new distributed machine learning mechanisms. As a prevalent approach, Federated Learning (FL) employs parameter-sharing and gradient-averaging between clients and a server. Despite its many favorable qu...

In this paper, we study a natural policy gradient method based on recurrent neural networks (RNNs) for partially-observable Markov decision processes, whereby RNNs are used for policy parameterization and policy evaluation to address curse of dimensionality in non-Markovian reinforcement learning. We present finite-time and finite-width analyses fo...

We study the optimal scheduling problem where $n$ source nodes attempt to transmit updates over $L$ shared wireless on/off fading channels to optimize their age performance under energy and age-violation tolerance constraints. Specifically, we provide a generic formulation of age-optimization in the form of a constrained Markov Decision Process...

In this work, we consider the problem of ‘fresh’ caching at distributed (front-end) local caches of content that is subject to ‘dynamic’ updates at the (back-end) database. We first provide new models and analyses of the average operational cost of a network of distributed edge-caches that utilizes wireless multicast to refresh aging content. We at...

In this work, we consider a network, where distributed information sources whose states evolve according to a random process transmit their time-varying states to a remote estimator over a shared wireless channel. Each source generates packets in a decentralized manner and employs a slotted random access mechanism to transmit the packets. In partic...

In a broad class of reinforcement learning applications, stochastic rewards have heavy-tailed distributions, which lead to infinite second-order moments for stochastic (semi)gradients in policy evaluation and direct policy optimization. In such instances, the existing RL methods may fail miserably due to frequent statistical outliers. In this work,...

We study the optimal scheduling problem where n source nodes attempt to transmit updates over L shared wireless on/off fading channels to optimize their age performance under energy and age-violation tolerance constraints. Specifically, we provide a generic formulation of age-optimization in the form of a constrained Markov Decision Processes (CMDP...

We investigate a remote estimation system with communication cost for multiple Internet-of-Things sensors, in which the state of each sensor changes according to a Wiener process. Under sublinear communication cost structure, in which the per-transmission cost decreases with the number of simultaneous transmissions, we address an interesting unexpl...

In this work, we perform a comparative study of centralized and decentralized update strategies for the basic remote tracking problem of many distributed users/devices with randomly evolving states. Our goal is to reveal the impact of the fundamentally different tradeoffs that exist between information accuracy and communication cost under thes...

We introduce a framework and provably-efficient schemes for ‘fresh’ caching at the (front-end) local cache of content that is subject to ‘dynamic’ updates at the (back-end) database. We start by formulating the hard-cache-constrained problem for this setting, which quickly becomes intractable due to the limited cache. To bypass this challenge, we f...

In a wide variety of applications including online advertising, contractual hiring, and wireless scheduling, the controller is constrained by a stringent budget constraint on the available resources, which are consumed in a random amount by each action, and a stochastic feasibility constraint that may impose important operational limitations on dec...

Existing content caching mechanisms are predominantly geared towards easy-access to content that is static once created. However, numerous applications, such as news and dynamic sources with time-varying states, generate 'dynamic' content where new updates replace previous versions. This motivates us in this work to study the freshness-driven cachi...

In a wide variety of applications including online advertising, contractual hiring, and wireless scheduling, the controller is constrained by a stringent budget constraint on the available resources, which are consumed in a random amount by each action, and a stochastic feasibility constraint that may impose important operational limitations on dec...

We consider an information network comprised of nodes that are: rational-information-consumers (RICs) and/or biased-information-providers (BIPs). Making the reasonable abstraction that any external event is reported as an answer to a logical statement, we model each node's information-sharing behavior as a binary channel. For various reasons, malic...

In practice, prefetching data strategically has been used to improve caching performance. The idea is that data items can either be cached upon request (traditional approach) or prefetched into the cache before the requests actually occur. The caching and prefetching operations compete for the limited cache space, whose size is typically much small...

In this work, we provide a comprehensive analysis of stability properties and delay gains that wireless multicasting capabilities, as opposed to more traditional unicast transmissions, can provide for content distribution in mobile networks. In particular, we propose a model and characterize the average queue-length (and hence average delay) perfor...

Strategically prefetching data has been utilized in practice to improve caching performance. Apart from caching data items upon requests, they can be prefetched into the cache before requests actually occur. The caching and prefetching operations compete for the limited cache space, whose size is typically much smaller than the number of data items...

Least-recently-used (LRU) caching and its variants have conventionally been used as a fundamental and critical method to ensure fast and efficient data access in computer and communication systems. Emerging data-intensive applications over unreliable channels, e.g., mobile edge computing and wireless content delivery networks, have imposed new chal...

Time-constrained decision processes have been ubiquitous in many fundamental applications in physics, biology and computer science. Recently, restart strategies have gained significant attention for boosting the efficiency of time-constrained processes by expediting the completion times. In this work, we investigate the bandit problem with controll...

The theory of discrete-time online learning has been successfully applied in many problems that involve sequential decision-making under uncertainty. However, in many applications including contractual hiring in online freelancing platforms and server allocation in cloud computing systems, the outcome of each action is observed only after a random...

We consider information networks whereby multiple biased-information-providers (BIPs), e.g., media outlets/social network users/sensors, share reports of events with rational-information-consumers (RICs). Making the reasonable abstraction that an event can be reported as an answer to a logical statement, we model the input-output behavior of each B...

We consider a budget-constrained bandit problem where each arm pull incurs a random cost, and yields a random reward in return. The objective is to maximize the total expected reward under a budget constraint on the total cost. The model is general in the sense that it allows correlated and potentially heavy-tailed cost-reward pairs that can take o...

significant challenge for future virtual reality (VR) applications is to deliver high quality-of-experience, both in terms of video quality and responsiveness, over wireless networks with limited bandwidth. This paper proposes to address this challenge by leveraging the predictability of user movements in the virtual world. We consider a wireless s...

A significant challenge for future virtual reality (VR) applications is to deliver high quality-of-experience, both in terms of video quality and responsiveness, over wireless networks with limited bandwidth. This paper proposes to address this challenge by leveraging the predictability of user movements in the virtual world. We consider a wireless...

Network traffic for delay-sensitive services has become a dominant part in the network. Proactive caching with the aid of predictive information has been proposed as a promising method to enhance delay performance. In this paper, we analytically investigate the problem of how to efficiently utilize uncertain predictive information to design proacti...

The Least Recently Used (LRU) caching and its variants are used in large-scale data systems in order to provide high-speed data access for a wide class of applications. Nonetheless, a fundamental question still remains: in order to minimize the miss probabilities, how should the cache space be organized to serve multiple data flows? Commonly used s...

We consider a bandit problem with K task types from which the controller activates one task at a time. Each task takes a random and possibly heavy-tailed completion time, and a reward is obtained only after the task is completed. The task types are independent from each other, and have distinct and unknown distributions for completion time and rewa...

The future Internet of Things (IoT) networks are expected to be composed of a large population of low-cost devices communicating dynamically with access points or neighboring devices to communicate small bundles of delay-sensitive data. To support the high-intensity and short-lived demands of these emerging networks, we propose an efficient MAC par...

Network traffic for delay-sensitive services has become a dominant part in the network. Proactive caching with the aid of predictive information has been proposed as a promising method to enhance delay performance. In this paper, we analytically investigate the problem of how to efficiently utilize uncertain predictive information to design proacti...

The Least Recently Used (LRU) caching and its variants are used in large-scale data systems in order to provide high-speed data access for a wide class of applications. Nonetheless, a fundamental question still remains: in order to minimize the miss probabilities, how should the cache space be organized to serve multiple data flows? Commonly used s...

We consider a bandit problem with K task types from which the controller activates one task at a time. Each task takes a random and possibly heavy-tailed completion time, and a reward is obtained only after the task is completed. The task types are independent from each other, and have distinct and unknown distributions for completion time and rewa...

The Least Recently Used (LRU) caching and its variants are used in large-scale data systems in order to provide high-speed data access for a wide class of applications. Nonetheless, a fundamental question still remains open: in order to minimize miss probabilities, how should the cache space be organized to serve multiple data flows? Commonly used...

We consider a bandit problem with K task types from which the controller activates one task at a time. Each task takes a random and possibly heavy-tailed completion time, and a reward is obtained only after the task is completed. The task types are independent from each other, and have distinct and unknown distributions for completion time and rewa...

We are interested in identifying a rumor source on a tree network. We begin with extended star networks under the SI infection model with exponential waiting times. We present and analyze the types center, a highly tractable approximation of the ML source estimate, obtained using the method of types. We empirically show that this approximate ML est...

Network traffic of delay-sensitive services has become a dominant part in the network. Proactive caching with the aid of predictive information has been proposed as a promising method to enhance the delay performance, which is one of the principal concerns of such services. In this paper, we analytically investigate the problem of how to efficientl...

The increase in demand for spectrum-based services forms a bottleneck in wireless networks. Device-to-Device (D2D) caching networks tackle this problem by exploiting user's behavior predictability and the possibility of sharing data between them to alleviate the network congestion. However, capturing mobility statistics allows Service Providers (SP...

The increase in demand for spectrum-based services forms a bottleneck in wireless networks. Device-to-Device (D2D) caching networks tackle this problem by exploiting users behavior predictability and the possibility of sharing data between them to alleviate the network congestion. Usually, network congestion occurs at certain times of the day and i...

The increase in demand for spectrum-based services forms a bottleneck in wireless networks. Device-to-Device (D2D) caching networks tackle this problem by exploiting users behavior predictability and the possibility of sharing data between them to alleviate the network congestion. Usually, network congestion occurs at certain times of the day and i...

The dominant portion of smartphone traffic is generated by apps that involve human interactivity. Particularly, when human users receive information from a server, they spend a few seconds of information processing before taking an action. The user processing time creates an idle communication period during the app session. Moreover, the generation...

The behavior of users in relatively predictable, both in terms of the data they request and the wireless channels they observe. In this paper, we consider the statistics of such predictable patterns of the demand and channel jointly across multiple users, and develop a novel predictive resource allocation method. This method is shown to provide per...

In this paper, we introduce the COmbinatorial Multi-Objective Multi-Armed Bandit (COMO-MAB) problem that captures the challenges of combinatorial and multi-objective online learning simultaneously. In this setting, the goal of the learner is to choose an action at each time, whose reward vector is a linear combination of the reward vectors of the a...

Round robin and its variants are well known scheduling policies that are popular in wireline networks due to their throughput optimality, delay insensitivity to file size distributions and short-term fairness. The latter two properties are also extremely important for emerging wireless applications, such as Internet of Things and cyber-physical sys...

To support the multi-Gigabit per second data rates of 5G wireless networks, there have been significant efforts on the research and development of Massive MIMO (M-MIMO) technologies at the physical layer. So far, however, the understanding of how M-MIMO could affect the performance of network control and optimization algorithms remains rather limit...

We study the stochastic multi-armed bandit (MAB) problem in the presence of side-observations across actions that occur as a result of an underlying network structure. In our model, a bipartite graph captures the relationship between actions and a common set of unknowns such that choosing an action reveals observations for the unknowns that it is c...

We consider a system where a load aggregator (LA) serves a large number of small-sized, economically-driven consumers with deferrable demand, as envisioned in smart electricity grid and data networks. In these systems, consumers can behave opportunistically by deferring their demand in response to the prices, to obtain economic gains. However, if n...

The increase in demand for spectrum-based services forms a bottleneck in wireless networks. Device-to-Device (D2D) caching networks tackle this problem by exploiting users behavior predictability and the possibility of sharing data between them to alleviate network congestion. Usually, network congestion occurs at certain times of the day and in so...

In recent years, there have been significant efforts on the research and development of Massive MIMO (M-MIMO) technologies at the physical layer. So far, however, the understanding of how M-MIMO could affect the performance of network control and optimization algorithms remains rather limited. In this paper, we focus on analyzing the performance of...

In this work, we investigate the profit maximization problem for a wireless network carrier and the payment minimization for end-users. Motivated by recent findings on proactive resource allocation, we focus on the scenario whereby end-users who are equipped with device-to-device (D2D)communication can harness predictable demand in proactive data c...

In this work, we investigate the profit maximization problem for a wireless network carrier and the payment minimization for users. We focus on the scenario whereby users who are equipped with device-to-device (D2D) communication can harness their predictable demand in proactive data contents caching and the possibility of trading their proactive d...

Mobile data users are known to possess predictable characteristics both in their interests and activity patterns. Yet, their service is predominantly performed, especially at the wireless edges, “reactively” at the time of request, typically when the network is under heavy traffic load. This strategy incurs excessive costs to the service providers...

We consider the design of throughput-optimal scheduling policies in multihop wireless networks that also possess good mean delay performance and provide regular service for all links—critical metrics for real-time applications. To that end, we study a parametric class of maximum-weight-type scheduling policies, called Regular Service Guarantee (RSG...

In this paper, we study the possibility of data contents peer-to-peer trading. Motivated by recent findings on proactive resource allocation, we focus on the scenario whereby end-users harness their predictable demands and the possibility of being connected together to pre-cache data contents and resell them in order to minimize their expected paym...

We study opportunistic multiuser scheduling in downlink networks with Markov-modeled outage channels. We consider the scenario that the scheduler does not have full knowledge of the channel state information, but instead estimates the channel state by exploiting the memory inherent in the Markov channels along with Automatic-Repeat-reQues-styled-st...

In this work, we study the design of pricing mechanisms for the efficient and stable service of a large consumer base with deferrable demands, as envisioned in future smart electricity and data networks. When users with flexible demand are introduced into such large-scale dynamic markets with dynamic prices, they exhibit opportunistic behavior to m...

This chapter focuses on demand-side flexibilities and the predictability of consumer activities. It discusses the means of exploiting these aspects of consumer behavior by using smart pricing techniques. The chapter provides an example design of pricing policies for the day-ahead electricity markets with flexible demand and an example optimization...

We study the stochastic multi-armed bandit (MAB) problem in the presence of side-observations across actions. In our model, choosing an action provides additional side observations for a subset of the remaining actions. One example of this model occurs in the problem of targeting users in online social networks where users respond to their friends'...

We study the stochastic multi-armed bandit (MAB) problem in the presence of side-observations across actions. In our model, choosing an action provides additional side observations for a subset of the remaining actions. One example of this model occurs in the problem of targeting users in online social networks where users respond to their friends'...

In this work, we investigate the profit maximization problem for wireless network carriers and payment minimization for end users. Motivated by our recent findings on proactive resource allocation, we focus on the scenario whereby end users harness predictable demand and WiFi connectivity in proactive data downloads, to minimize their expected paym...

Motivated by the regular service requirements of video applications for improving Quality-of-Experience (QoE) of users, we consider the design of scheduling strategies in multi-hop wireless networks that not only maximize system throughput but also provide regular inter-service times for all links. Since the service regularity of links is related t...

In this work, we investigate the profit maximization problem of a network service provider through smart pricing and proactive data services. The demand characteristics of each user are dependent on the price and willingness-to-pay values of each service. By learning these characteristics, the service provider can further improve its profit perform...

In a wireless network, the efficiency of scheduling algorithms over time-varying channels depends heavily on the accuracy of the Channel State Information (CSI), which is usually quite ``costly'' in terms of consuming network resources. Scheduling in such systems is also subject to stringent constraints such as power and bandwidth, which limit the...

In this paper, we study the design of joint flow-rate control and scheduling policies in multihop wireless networks for achieving maximum network utility with provably optimal convergence speed. Fast convergence is especially important in wireless networks that are dominated by the dynamics of incoming and outgoing flows as well as the time-sensiti...

We address the modeling and optimal control problems for load following in power grids in the presence of unconventional battery-powered agents. Compared to the traditional large-scale generators such as thermal and hydroelectric plants, these agents have significantly lower short-term response costs but higher long-term generation costs. Moreover,...

We consider the decision problem of an external agent choosing to execute one of M actions for each user in a social network. We assume that observing a user's actions provides valuable information for a larger set of users since each user's preferences are interrelated with those of her social peers. This falls into the well-known setting of the m...

We consider the problem of routing Bernoulli arrivals to parallel queues, where each queue provides service according to an independent Bernoulli process. We assume that the total arrival rate exceeds the sum of the service rates of the queues. Since such a queueing system is unstable, the vector of queue lengths does not have a well-defined statio...

Motivated by the problem of demand management in smart grids, we study the problem of minimizing a weighted-sum of the mean delay of user demands and the power generation cost, where the latter metric increases with both the mean and the variance of the service demand. The state-of-the-art algorithms for this problem are asymptotically optimal, i.e...

In an unreliable single-hop broadcast network setting, we investigate the throughput and decoding-delay performance of random linear network coding as a function of the coding window size and the network size. Our model consists of a source transmitting packets of a single flow to a set of n users over independent time-correlated erasure channels....

In this paper, we address real-time pricing and control of opportunistic consumers with deferrable demands that are motivated by the envisioned smart electrical grid. In the smart grid, demand-side flexibilities from deferrable loads enable consumers to respond to real-time electricity prices for their own economic benefit. However, the aggregate l...

Multirate multicasting is a generalization of single-rate multicasting to prevent destinations with good connections from being limited by the capacity of bottleneck connections. While multirate multicasting has been traditionally performed over fixed trees, advances in network coding theory have enabled higher throughput and have helped us move be...

We consider the design of throughput-optimal scheduling policies in multi-hop wireless networks that also possess good mean delay performance and provide regular service for all links -- critical metrics for real-time applications. To that end, we study a parametric class of maximum-weight type scheduling policies with parameter α ≥ 0, called Regul...

Recently, low-complexity and distributed Carrier Sense Multiple Access (CSMA)-based scheduling algorithms have attracted extensive interest due to their throughput-optimal characteristics in general network topologies. However, these algorithms are not well-suited for time-varying environments (i.e., serving real-time traffic under time-varying cha...