Jieping Ye

• Ph.D.
• Arizona State University

The rapid accumulation of biomedical cohort data presents opportunities to explore disease mechanisms, risk factors, and prognostic markers. However, current research often has a narrow focus, limiting the exploration of risk factors and inter-disease correlations. Additionally, fragmented processes and time constraints can hinder comprehensive ana...

Ride-pooling optimization typically concerns with matching, repositioning, routing (see e.g., Zheng et al. (2018); Alonso-Mora et al. (2017a, 2017b); Tong et al. (2018)). The RL literature has primarily focused on the first two problems. The ride-pooling matching problem differs from that in Chap. 5 in that a combination of multiple passengers, and...

The problems in ridesharing are highly practice-oriented, and results from toy data sets or environments may present a very different picture from those in reality. Hence, real-world data sets and realistic simulators backed up by them are instrumental to research in RL algorithms for these problems.

The rideshare matching problem and its generalized forms have been investigated extensively in the field of operations research (see e.g., Özkan and Ward 2020; Hu and Zhou 2022; Lowalekar et al. 2018 and the references therein). Typically, both the open trip requests and available drivers are batched within time windows of fixed length as they arri...

Given the state of the current literature, we discuss a few challenges and opportunities that we feel crucial in advancing RL for ridesharing.

We first describe the architecture of a ridesharing system in this section, followed by explanation and clarification on the scopes of the problem associated with each module.

We briefly review the RL basics and the major algorithms, especially those used in the works reviewed or described in this survey. For a complete reference, see, e.g., Sutton and Barto (2018).

There has been increasing interest in investigating the interplay of causality and RL in the recent years. The synergy of the two becomes understandable by treating the task of learning the action values Q(s, a) in RL as estimating the long-term counter-factual effects of applying the different actions to a given current state, to which knowing the...

Routing is an ambiguous terminology in the context of ridesharing. Routing, which we discuss in Sect. 7.1, commonly refers to low-level navigation decisions on a road network, typically with output of matching and repositioning algorithms as input. This is not to be confused with the vehicle routing problem, which is a separate class of macro-level...

Vehicle repositioning from a single-driver perspective (i.e., taxi routing) has a relatively long history of research since taxi service has been in existence long before the emergence of rideshare platforms. Likewise, research on RL-based approaches for this problem (see Table B.4) also appeared earlier than that on system-level vehicle reposition...

The aim of this article is to establish a causal link between the policies implemented by technology companies and the outcomes they yield within intricate temporal and/or spatial dependent experiments. We propose a novel temporal/spatio-temporal Varying Coefficient Decision Process model, capable of effectively capturing the evolving treatment eff...

In reinforcement learning, a promising direction to avoid online trial-and-error costs is learning from an offline dataset. Current offline reinforcement learning methods commonly learn in the policy space constrained to in-support regions by the offline dataset, in order to ensure the robustness of the outcome policies. Such constraints, however,...

Long-term user engagement (LTE) optimization in sequential recommender systems (SRS) is shown to be suited by reinforcement learning (RL) which finds a policy to maximize long-term rewards. Meanwhile, RL has its shortcomings, particularly requiring a large number of online samples for exploration, which is risky in real-world applications. One of t...

The gig economy provides workers with the benefits of autonomy and flexibility but at the expense of work identity and coworker bonds. Among the many reasons why gig workers leave their platforms, one unexplored aspect is the lack of an organization identity. In this study, we develop a team formation and interteam contest field experiment at a rid...

While the gig economy provides flexible jobs for millions of workers globally, a lack of organization identity and coworker bonds contributes to their low engagement and high attrition rates. To test the impact of virtual teams on worker productivity and retention, we conduct a field experiment with 27,790 drivers on a ride-sharing platform. We org...

With millions of people using ride-hailing platforms for daily travel, estimated time of arrival (ETA) has become a significant problem in intelligent transportation systems and attracted considerable attention recently. Deep learning-based ETA methods have achieved promising results using massive spatial-temporal data. However, we find that the pr...

In this paper, we present a comprehensive, in-depth survey of the literature on reinforcement learning approaches to decision optimization problems in a typical ridesharing system. Papers on the topics of rideshare matching, vehicle repositioning, ride-pooling, routing, and dynamic pricing are covered. Most of the literature has appeared in the las...

Policy evaluation based on A/B testing has attracted considerable interest in digital marketing, but such evaluation in ride-sourcing platforms (e.g., Uber and Didi) is not well studied primarily due to the complex structure of their temporal and/or spatial dependent experiments. Motivated by policy evaluation in ride-sourcing platforms, the aim of...

A/B testing, or online experiment is a standard business strategy to compare a new product with an old one in pharmaceutical, technological, and traditional industries. Major challenges arise in online experiments of two-sided marketplace platforms (e.g., Uber) where there is only one unit that receives a sequence of treatments over time. In those...

Hashing has been widely applied to the large-scale approximate nearest neighbor search problem owing to its high efficiency and low storage requirement. Most investigations concentrate on learning hashing methods in a centralized setting. However, in existing big data systems, data is often stored across different nodes. In some situations, data is...

Amyloid-β (Aβ) plaques and tau protein tangles in the brain are now widely recognized as the defining hallmarks of Alzheimer’s disease (AD), followed by structural atrophy detectable on brain magnetic resonance imaging (MRI) scans. One of the particular neurodegenerative regions is the hippocampus to which the influence of Aβ/tau on has been one of...

Generative adversarial networks (GANs) have recently become a hot research topic; however, they have been studied since 2014, and a large number of algorithms have been proposed. However, few comprehensive studies exist explaining the connections among different GANs variants and how they have evolved. In this paper, we attempt to provide a review...

Ride-sourcing services play an increasingly important role in meeting mobility needs in many metropolitan areas. Yet, aside from delivering passengers from their origins to destinations, ride-sourcing vehicles generate a significant number of vacant trips from the end of one customer delivery trip to the start of the next. These vacant trips create...

Reinforcement learning (RL) aims at searching the best policy model for decision making, and has been shown powerful for sequential recommendations. The training of the policy by RL, however, is placed in an environment. In many real-world applications, the policy training in the real environment can cause an unbearable cost due to the exploration....

Ride-sourcing drivers spend a significant portion of their service time being idle, during which they can move freely to search for the next customer. Such customer-searching movements, while not being directly controlled by ride-sourcing platforms, impose great impacts on the service efficiency of ride-sourcing systems and thus need to be better u...

We present a new practical framework based on deep reinforcement learning and decision-time planning for real-world vehicle repositioning on ride-hailing (a type of mobility-on-demand, MoD) platforms. Our approach learns the spatiotemporal state-value function using a batch training algorithm with deep value networks. The optimal repositioning acti...

Amyloid-β (Aβ) plaques and tau protein tangles in the brain are now widely recognized as the defining hallmarks of Alzheimer's disease (AD), followed by structural atrophy detectable on brain magnetic resonance imaging (MRI) scans. One of the particular neurodegenerative regions is the hippocampus to which the influence of Aβ/tau on has been one of...

Biomarker assisted preclinical/early detection and intervention in Alzheimer’s disease (AD) may be the key to therapeutic breakthroughs. One of the presymptomatic hallmarks of AD is the accumulation of beta-amyloid (Aβ) plaques in the human brain. However, current methods to detect Aβ pathology are either invasive (lumbar puncture) or quite costly...

This paper proposes a macroscopic fluid modeling framework to assist with strategic decision making of a platform for operating a large-scale on-demand app-based ride-hailing system. The framework captures the spatiotemporal characteristics of a ride-hailing system, and is flexible in representing control policies that a platform is implementing. I...

The streams where multiple transactions are associated with the same key are prevalent in practice, e.g., a customer has multiple shopping records arriving at different time. Itemset frequency estimation on such streams is very challenging since sampling based methods, such as the popularly used reservoir sampling, cannot be used. In this article,...

Recent works on ride-sharing order dispatching have highlighted the importance of taking into account both the spatial and temporal dynamics in the dispatching process for improving the transportation system efficiency. At the same time, deep reinforcement learning has advanced to the point where it achieves superhuman performance in a number of fi...

Video anomaly detection is commonly used in many applications, such as security surveillance, and is very challenging. A majority of recent video anomaly detection approaches utilize deep reconstruction models, but their performance is often suboptimal because of insufficient reconstruction error differences between normal and abnormal video frames...

Ride-hailing platforms generally provide various service options to customers, such as solo ride services, shared ride services, etc. It is generally expected that demands for different service modes are correlated, and the prediction of demand for one service mode can benefit from historical observations of demands for other service modes. Moreove...

Large ride-hailing platforms, such as DiDi, Uber and Lyft, connect tens of thousands of vehicles in a city to millions of ride demands throughout the day, providing great promises for improving transportation efficiency through the tasks of order dispatching and vehicle repositioning. Existing studies, however, usually consider the two tasks in sim...

In this paper, we present a comprehensive, in-depth survey of the literature on reinforcement learning approaches to decision optimization problems in a typical ridesharing system. Papers on the topics of rideshare matching, vehicle repositioning, ride-pooling, routing, and dynamic pricing are covered. Most of the literature has appeared in the las...

An effective presymptomatic diagnosis and treatment of Alzheimer’s disease (AD) would have enormous public health benefits. Sparse coding (SC) has shown strong potential for longitudinal brain image analysis in preclinical AD research. However, the traditional SC computation is time-consuming and does not explore the feature correlations that are c...

We present a new practical framework based on deep reinforcement learning and decision-time planning for real-world vehicle repositioning on ride-hailing (a type of mobility-on-demand, MoD) platforms. Our approach learns the spatiotemporal state-value function using a batch training algorithm with deep value networks. The optimal repositioning acti...

How to dynamically measure the local-to-global spatio-temporal coherence between demand and supply networks is a fundamental task for ride-sourcing platforms, such as DiDi. Such coherence measurement is critically important for the quantification of the market efficiency and the comparison of different platform policies, such as dispatching. The ai...

By serving two or more passenger requests in each ride in ride-sourcing markets, ride-pooling service is now becoming an important component of shared smart mobility. It is generally expected to improve vehicle utilization rate, and therefore alleviate traffic congestion and reduce carbon dioxide emissions. A few recent theoretical studies are cond...

The aim of this paper is to introduce a novel graph-based equilibrium metric (GEM) to quantify the distance between two discrete measures with possibly different masses on a weighted graph structure. This development is primarily motivated by dynamically measuring the local-to-global spatio-temporal coherence between demand and supply networks obta...

DiDi ridesharing platform provides users with accurate and high-precision travel services in real-time by using data-driven technology, machine learning method, large-scale distributed computing, operations optimization and other technique. The platform has made great breakthroughs in the key technologies of intelligent prediction and dispatch of t...

With the rapid development of mobile-internet technologies, on-demand ride-sourcing services have become increasingly popular and largely reshaped the way people travel. Demand prediction is one of the most fundamental components in supply-demand management systems of ride-sourcing platforms. With an accurate short-term prediction for origin-destin...

Developing electric ride-hailing vehicles (ERVs) is an integral part of promoting green transportation. To understand how ERVs operate, and in turn to improve their battery systems, we have developed a comprehensive, methodological framework to construct their utilization patterns and driving cycles. The methodology is based on a multi-state model...

We present a new practical framework based on deep reinforcement learning and decision-time planning for real-world vehicle repositioning on ride-hailing (also known as mobility-on-demand, MoD) platforms. Our approach learns the driver-perspective state-value function using a batch training algorithm with spa-tiotemproal deep value networks. The op...

Ride-hailing platforms generally provide various service options to customers, such as solo ride services, shared ride services, etc. It is generally expected that demands for different service modes are correlated, and the prediction of demand for one service mode can benefit from historical observations of demands for other service modes. Moreove...

Video anomaly detection is commonly used in many applications such as security surveillance and is very challenging. A majority of recent video anomaly detection approaches utilize deep reconstruction models, but their performance is often suboptimal because of insufficient reconstruction error differences between normal and abnormal video frames i...

Collectively, vast quantities of brain imaging data exist across hospitals and research institutions, providing valuable resources to study brain disorders such as Alzheimer's disease (AD). However, in practice, putting all these distributed datasets into a centralized platform is infeasible due to patient privacy concerns, data restrictions and le...

Biomarker assisted preclinical/early detection and intervention in Alzheimer’s disease (AD) may be the key to therapeutic breakthroughs. One of the presymptomatic hallmarks of AD is the accumulation of beta-amyloid (Aβ) plaques in the human brain. However, current methods to detect Aβ pathology are either invasive (lumbar puncture) or quite costly...

For many applications, predicting the users' intents can help the system provide the solutions or recommendations to the users. It improves the user experience, and brings economic benefits. The main challenge of user intent prediction is that we lack enough labeled data for training, and some intents (labels) are sparse in the training set. This i...

While Deep Reinforcement Learning (DRL) has emerged as a promising approach to many complex tasks, it remains challenging to train a single DRL agent that is capable of undertaking multiple different continuous control tasks. In this paper, we present a Knowledge Transfer based Multi-task Deep Reinforcement Learning framework (KTM-DRL) for continuo...

While Deep Reinforcement Learning (DRL) has emerged as a promising approach to many complex tasks, it remains challenging to train a single DRL agent that is capable of undertaking multiple different continuous control tasks. In this paper, we present a Knowledge Transfer based Multi-task Deep Reinforcement Learning framework (KTM-DRL) for continuo...

Graph neural networks have become an important tool for modeling structured data. In many real-world systems, intricate hidden information may exist, e.g., heterogeneity in nodes/edges, static node/edge attributes, and spatiotemporal node/edge features. However, most existing methods only take part of the information into consideration. In this pap...

Order dispatching is instrumental to the marketplace engine of a large-scale ride-hailing platform, such as the DiDi platform, which continuously matches passenger trip requests to drivers at a scale of tens of millions per day. Because of the dynamic and stochastic nature of supply and demand in this context, the ride-hailing order-dispatching pro...

With the recent rapid growth of technology-enabled mobility services, ride-sourcing platforms, such as Uber and DiDi, have launched commercial on-demand ride-pooling programs that allow drivers to serve more than one passenger request in each ride. Without requiring the prearrangement of trip schedules, these programs match on-demand passenger requ...

Customer response prediction is critical in many industrial applications such as online advertising and recommendations. In particular, the challenge is greater for ride-hailing platforms such as Uber and DiDi, because the response prediction models need to consider historical and real-time event information in the physical environment, such as sur...

The estimated time of arrival (ETA) is a critical task in the intelligent transportation system, which involves the spatiotemporal data. Despite a significant amount of prior efforts have been made to design efficient and accurate systems for ETA task, few of them take structural graph data into account, much less the heterogeneous information netw...

Graph-based semi-supervised learning is very important for many classification tasks, but most existing methods assume that all labelled nodes are randomly sampled. With the presence of nonignorable nonresponse, ignoring all missing nodes can lead to significant estimation bias and handicap the classifiers. To solve this issue, we propose a Graph-b...

Millions of drivers worldwide have enjoyed financial benefits and work schedule flexibility through a ride-sharing economy, but meanwhile they have suffered from the lack of a sense of identity and career achievement. Equipped with social identity and contest theories, financially incentivized team competitions have been an effective instrument to...

Network embedding is an effective way to solve the network analytics problems such as node classification, link prediction, etc. It represents network elements using low dimensional vectors such that the graph structural information and properties are maximumly preserved. Many prior works focused on embeddings for networks with the same type of edg...

Deep learning technique has dramatically boosted the performance of face alignment algorithms. However, due to large variability and lack of samples, the alignment problem in unconstrained situations, \emph{e.g}\onedot large head poses, exaggerated expression, and uneven illumination, is still largely unsolved. In this paper, we explore the instinc...

Background Disease progression prediction based on neuroimaging biomarkers is vital in Alzheimer’s disease (AD) research. Convolutional neural networks (CNN) have been proved to be powerful for various computer vision research by refining reliable and high-level feature maps from image patches. Objective A key challenge in applying CNN to neuroima...

In this paper, we focus on graph representation learning of heterogeneous information network (HIN), in which various types of vertices are connected by various types of relations. Most of the existing methods conducted on HIN revise homogeneous graph embedding models via meta-paths to learn low-dimensional vector space of HIN. In this paper, we pr...

Contextual multi-armed bandit (MAB) achieves cutting-edge performance on a variety of problems. When it comes to real-world scenarios such as recommendation system and online advertising, however, it is essential to consider the resource consumption of exploration. In practice, there is typically non-zero cost associated with executing a recommenda...

The target of human pose estimation is to determine body part or joint locations of each person from an image. This is a challenging problems with wide applications. To address this issue, this paper proposes an augmented parallel-pyramid net with attention partial module and differentiable auto-data augmentation. Technically, a parallel pyramid st...

With the availability of the location information of drivers and passengers, ride-sourcing platforms can now provide increasingly efficient online matching compared with physical searching and meeting performed in the traditional taxi market. The matching time interval (the time interval over which waiting passengers and idle drivers are accumulate...

In this paper, we focus on graph representation learning of heterogeneous information network (HIN), in which various types of vertices are connected by various types of relations. Most of the existing methods conducted on HIN revise homogeneous graph embedding models via meta-paths to learn low-dimensional vector space of HIN. In this paper, we pr...

Computer-aided diagnosis (CAD) systems for medical images are seen as effective tools to improve the efficiency of diagnosis and prognosis of Alzheimer’s disease (AD). The current state-of-the-art models for many images analyzing tasks are based on Convolutional Neural Networks (CNN). However, the lack of training data is a common challenge in appl...

On-demand taxi-calling platforms often ignore the social engagement of individual drivers. The lack of social incentives impairs the work enthusiasms of drivers and will affect the quality of service. In this paper, we propose to form teams among drivers to promote participation. A team consists of a leader and multiple members, which acts as the b...

How to optimally dispatch orders to vehicles and how to trade off between immediate and future returns are fundamental questions for a typical ride-hailing platform. We model ride-hailing as a large-scale parallel ranking problem and study the joint decision-making task of order dispatching and fleet management in online ride-hailing platforms. Thi...

Data-enabled smart transportation has attracted a surge of interest from machine learning and data mining researchers nowadays due to the bloom of online ride-hailing industry and rapid development of autonomous driving. Large-scale high quality route data and trading data (spatiotemporal data) have been generated every day, which makes AI an urgen...

Improving the efficiency of dispatching orders to vehicles is a research hotspot in online ride-hailing systems. Most of the existing solutions for order-dispatching are centralized controlling, which require to consider all possible matches between available orders and vehicles. For large-scale ride-sharing platforms, there are thousands of vehicl...

With the rapid development of mobile-internet technologies, on-demand ride-sourcing services have become increasingly popular and largely reshaped the way people travel. Demand prediction is one of the most fundamental components in supply-demand management systems of ride-sourcing platforms. With accurate short-term prediction for origin-destinati...

Improving the efficiency of dispatching orders to vehicles is a research hotspot in online ride-hailing systems. Most of the existing solutions for order-dispatching are centralized controlling, which require to consider all possible matches between available orders and vehicles. For large-scale ride-sharing platforms, there are thousands of vehicl...

Network embedding is an effective way to solve the network analytics problems such as node classification, link prediction, etc. It represents network elements using low dimensional vectors such that the graph structural information and properties are maximumly preserved. Many prior works focused on embeddings for networks with the same type of edg...

Didi Chuxing is the world's leading mobile transportation platform that offers a full range of app-based transportation options for 550 million users. Every day, DiDi's platform receives over 100TB new data, processes more than 40 billion routing requests, and acquires over 15 billion location points. Machine learning has been used in numerous comp...

Recent works on ride-sharing order dispatching have highlighted the importance of taking into account both the spatial and temporal dynamics in the dispatching process for improving the transportation system efficiency. At the same time, deep reinforcement learning has advanced to the point where it achieves superhuman performance in a number of fi...

This tutorial aims to provide the audience with a guided introduction to deep reinforcement learning (DRL) with specially curated application case studies in transportation. The tutorial covers both theory and practice, with more emphasis on the practical aspects of DRL that are pertinent to tackle transportation challenges. Some core examples incl...

Reinforcement learning aims at searching the best policy model for decision making, and has been shown powerful for sequential recommendations. The training of the policy by reinforcement learning, however, is placed in an environment. In many real-world applications, however, the policy training in the real environment can cause an unbearable cost...