Toshiharu Sugawara

Toshiharu Sugawara
Waseda University | Sōdai · Department of Computer Science and Engineering

Doctor of Engineering

About

240
Publications
7,808
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
1,088
Citations

Publications

Publications (240)
Article
Full-text available
Consumer-generated media (CGM), including social live streaming service (SLSS), often incorporate gamification elements to engage users. Online virtual tips and gifts, such as “bits” in Twitch, exemplify this approach by fostering interactive dynamics between live streamers and their audience. However, issues remain in understanding the impact of (...
Article
Although communication plays a pivotal role in achieving coordinated activities in multi-agent systems, conventional approaches often involve complicated high-dimensional messages generated by deep networks. These messages are typically indecipherable to humans, are relatively costly to transmit, and require intricate encoding and decoding networks...
Article
Full-text available
Decentralized execution is a widely used framework in multi-agent reinforcement learning. However, it has a well-known but neglected shortcoming, redundant computation, that is, the same/similar computation is performed redundantly in different agents owing to their overlapping observations. This study proposes a novel method, the locally centraliz...
Preprint
Full-text available
Consumer-generated media (CGM), including social live streaming service (SLSS), often incorporate gamification elements to engage users. Online virtual tips and gifts, such as ”bits” in Twitch, exemplify this approach by fostering interactive dynamics between live streamers and their audience. However, issues remain in understanding the impact of (...
Chapter
Social live streaming services (SLSS) and other consumer-generated media (CGM) offer gamification to attract people. Virtual gifts/tips such as Twitch’s “bits” are examples of this and construct interactive relationships between live streamers and viewers. However, their impact on user behavior and how the collection rates from the platforms to col...
Article
This study proposes a method to automatically generate paths for multiple autonomous agents to collectively form a sequence of consecutive patterns. Several studies have considered minimizing the total travel distances of all agents for formation transitions in applications with multiple self-driving robots, such as unmanned aerial vehicle shows by...
Article
Full-text available
In this paper, we propose an enhanced version of the distributed attentional actor architecture (eDA3-X) for model-free reinforcement learning. This architecture is designed to facilitate the interpretability of learned coordinated behaviors in multi-agent systems through the use of a saliency vector that captures partial observations of the enviro...
Preprint
Full-text available
Consumer generated media (CGM), such as social networking services rely on the voluntary activity of users to prosper, garnering the psychological rewards of feeling connected with other people through comments and reviews received online. To attract more users, some CGM have introduced monetary rewards (MR) for posting activity and quality article...
Preprint
We propose a model-free reinforcement learning architecture, called distributed attentional actor architecture after conditional attention (DA6-X), to provide better interpretability of conditional coordinated behaviors. The underlying principle involves reusing the saliency vector, which represents the conditional states of the environment, such a...
Preprint
We propose a distributed planning method with asynchronous execution for multi-agent pickup and delivery (MAPD) problems for environments with occasional delays in agents' activities and flexible endpoints. MAPD is a crucial problem framework with many applications; however, most existing studies assume ideal agent behaviors and environments, such...
Chapter
In this study, we propose a negotiation protocol for task handovers in the multi-agent cooperative patrol problem (MACPP) to alleviate temporary performance degradation due to planned suspension. In recent years, thanks to improvements in the performance of computers and the spread of technologies such as AI and IoT, systems with multiple agents su...
Article
Full-text available
We propose a method called path and action planning with orientation (PAPO) that efficiently generates collision-free paths to satisfy environmental constraints, such as restricted path width and node size, for the multi-agent pickup and delivery in non-uniform environment (N-MAPD) problem. The MAPD problem, wherein multiple agents repeatedly pick...
Article
Full-text available
We propose a two-stage reward allocation method with decay using an extension of replay memory to adapt this rewarding method for deep reinforcement learning (DRL), to generate coordinated behaviors for tasks that can be completed by executing a few subtasks sequentially by heterogeneous agents. An independent learner in cooperative multi-agent sys...
Chapter
In this paper, we formulate the material transportation problem as a multi-agent pickup and delivery with time synchronization (MAPD-TS) problem, which is an extension of the well-known multi-agent pickup and delivery (MAPD) problem. In MAPD-TS, we consider the synchronization of the movement of transportation agents with that of external agents, s...
Article
Full-text available
We investigate both the influence of monetary reward schemes on user behaviors and the quality of articles posted by users in consumer-generated media (CGM), such as social networking services (SNSs). Recently, CGM platforms have implemented monetary rewards to incentivize users to post articles and comments. However, the effect of monetary rewards...
Article
This paper proposes a control method for the multi-agent pickup and delivery problem (MAPD problem) by extending the priority inheritance with backtracking (PIBT) method to make it applicable to more general environments. PIBT is an effective algorithm that introduces a priority to each agent, and at each timestep, the agents, in descending order o...
Chapter
We propose a variable reward scheme in decentralized multi-agent deep reinforcement learning for a sequential task consisting of a number of subtasks which can be completed when all subtasks are executed in a certain order before a deadline by agents with different capabilities. Developments in computer science and robotics are drawing attention to...
Preprint
This paper proposes a control method for the multi-agent pickup and delivery problem (MAPD problem) by extending the priority inheritance with backtracking (PIBT) method to make it applicable to more general environments. PIBT is an effective algorithm that introduces a priority to each agent, and at each timestep, the agents, in descending order o...
Preprint
In multi-agent systems, noise reduction techniques are important for improving the overall system reliability as agents are required to rely on limited environmental information to develop cooperative and coordinated behaviors with the surrounding agents. However, previous studies have often applied centralized noise reduction methods to build robu...
Preprint
The multi-agent pickup and delivery (MAPD) problem, in which multiple agents iteratively carry materials without collisions, has received significant attention. However, many conventional MAPD algorithms assume a specifically designed grid-like environment, such as an automated warehouse. Therefore, they have many pickup and delivery locations wher...
Chapter
This paper investigates the impact of monetary rewards on behavioral strategies and the quality of posts in consumer generated media (CGM). In recent years, some CGM platforms have introduced monetary rewards as an incentive to encourage users to post articles. However, the impact of monetary rewards on users has not been sufficiently clarified. Th...
Article
Full-text available
The retweet is a characteristic mechanism of several social network services/social media, such as Facebook, Twitter, and Weibo. By retweeting tweet, users can share an article with their friends and followers. However, it is not clear how retweets affect the dominant behaviors of users. Therefore, this study investigates the impact of retweets on...
Chapter
We propose an interpretable neural network architecture for multi-agent deep reinforcement learning to understand the rationale for learned cooperative behavior of the agents. Although the deep learning technology has contributed significantly to multi-agent systems to build coordination among agents, it is still unclear what information the agents...
Article
We propose a coordinated control method of agents, which are self-driving ridesharing vehicles, by using multi-agent deep reinforcement learning (MADRL) so that they individually determine where they should wait for passengers to provide better services as well as to increase their profits in rideshare services. With the increasing demand for rides...
Chapter
Although the multi-agent pickup and delivery (MAPD) problem, wherein multiple agents iteratively carry materials from some storage areas to the respective destinations without colliding, has received considerable attention, conventional MAPD algorithms use simplified, uniform models without considering constraints, by assuming specially designed en...
Chapter
This paper proposes a method for dynamically forming teams and assigning appropriate tasks to their members to provide services accomplished by groups of agents of different types. Task or resource allocation in multi-agent systems has drawn attention and has been applied in many areas, such as robot rescue, UAV wireless networks, and distributed c...
Article
Full-text available
This paper proposes a method to mitigate the significant performance degradation due to planned suspensions in the multi-agent cooperative patrol problem. In recent years, there has been an increased demand to utilize multiple intelligent agents that control robots. Furthermore, cooperation between multiple agents is required for performing tasks t...
Article
Full-text available
Social networking services (SNSs) are constantly used by a large number of people with various motivations and intentions depending on their social relationships and purposes, and thus, resulting in diverse strategies of posting/consuming content on SNSs. Therefore, it is important to understand the differences of the individual strategies dependin...
Chapter
We compare the coordination structures of agents using different types of inputs for their deep Q-networks (DQNs) by having agents play a distributed task execution game. The efficiency and performance of many multi-agent systems can be significantly affected by the coordination structures formed by agents. One important factor that may affect thes...
Chapter
This paper proposes a method to improve the policies trained with multi-agent deep learning by adding a policy advisory module (PAM) in the testing phase to relax the exploration hindrance problem. Cooperation and coordination are central issues in the study of multi-agent systems, but agents’ policies learned in slightly different contexts may lea...
Article
Full-text available
Cooperation and coordination are major issues in studies on multi-agent systems because the entire performance of such systems is greatly affected by these activities. The issues are challenging however, because appropriate coordinated behaviors depend on not only environmental characteristics but also other agents’ strategies. On the other hand, a...
Chapter
Retweeting is a featured mechanism of some social media platforms such as Twitter, Facebook, and Weibo. Users share articles with friends or followers by reposting a tweet. However, the ways in which retweeting affects the dominant behaviors of users is still unclear. Therefore, we investigate the influence of retweeting on the behaviors of social...
Chapter
Recently, multi-agent deep reinforcement learning (MADRL) has been studied to learn actions to achieve complicated tasks and generate their coordination structure. The reward assignment in MADRL is a crucial factor to guide and produce both their behaviors for their own tasks and coordinated behaviors by agents’ individual learning. However, it has...
Chapter
This paper proposes a method for adaptively assigning service areas to self-driving taxi agents in ride-share services by using a centralized deep Q-network (DQN) and demand prediction data. A number of (taxi) companies have participated in ride-share services with the increase of passengers due to the mutual benefits for taxi companies and custome...
Article
We propose a decentralized multi-agent deep reinforcement learning architecture to investigate pattern formation under the local information provided by the agents' sensors. It consists of tasking a large number of homogeneous agents to move to a set of specified goal locations, addressing both the assignment and trajectory planning sub-problems co...
Chapter
This paper investigates how opinions are polarized by simulating opinion formation with Q-learning in multiplex networks. People sometimes change their opinions to accommodate themselves to the surrounding people in communities, but opinions may still be polarized. To investigate the mechanism of opinion polarization, many studies including studies...
Conference Paper
We investigated whether a group of agents could learn the strategic policy with different sizes of input by deep Q-learning in a simulated takeout platform environment. Agents are often required to cooperate and/or coordinate with each other to achieve their goals, but making appropriate sequential decisions for coordinated behaviors based on dynam...
Article
Full-text available
This paper discusses an adaptive distributed allocation method in which agents individually learn strategies for preferences to decide on the rank of tasks which they want to be allocated by a manager. In a distributed edge-computing environment, multiple managers that control the provision of a variety of services requested from different location...
Chapter
We propose a learning method that decides the period of activity according to environmental characteristics and the behavioral strategies in the multi-agent continuous cooperative patrol problem. With recent advances in computer and sensor technologies, agents, which are intelligent control programs running on computers and robots, obtain high auto...
Chapter
How users of social networking services (SNSs) dynamically identify their own reasonable strategies was investigated by applying a co-evolutionary algorithm to an agent-based game theoretic model of SNSs. We often use SNSs such as Twitter, Facebook, and Instagram, but we can also freeride without providing any content because providing information...
Chapter
This paper proposes a method of autonomous strategy learning for multiple cooperative agents integrated with a series of behavioral strategies aiming at reduction of energy cost on the premise of satisfying quality requirements in continuous patrolling problems. We improved our algorithm of requirement estimation to avoid concentration of agents si...
Chapter
Cooperation and coordination are sophisticated behaviors and are still major issues in studies on multi-agent systems because how to cooperate and coordinate depends on not only environmental characteristics but also the behaviors/strategies that closely affect each other. On the other hand, recently using the multi-agent deep reinforcement learnin...
Chapter
We investigate the coordination structures generated by deep Q-network (DQN) with various types of input by using a distributed task execution game. Although cooperation and coordination are mandatory for efficiency in multi-agent systems (MAS), they require sophisticated structures or regimes for effective behaviors. Recently, deep Q-learning has...
Article
Full-text available
We propose a learning and negotiation method to enhance divisional cooperation and demonstrate its flexibility for adapting to environmental changes in the context of the multi-agent cooperative problem. We now have access to a vast array of information, and everything has become more closely connected. However, this makes tasks/problems in these e...
Conference Paper
We propose a novel method for evolutionary network analysis that uses the genetic algorithm (GA), called the multiple world genetic algorithm, to coevolve appropriate individual behaviors of many agents on complex networks without sacrificing diversity. We conducted the experiments using simulated games of social networking services to evaluate the...
Article
In this work, we focus on an environment where multiple agents with complementary capabilities cooperate to generate non-conflicting joint actions that achieve a specific target. The central problem addressed is how several agents can collectively learn to coordinate their actions such that they complete a given task together without conflicts. How...
Article
Full-text available
We propose a control method for an elevator group control system to allocate elevator cars for all types of passengers, including general passengers and special passengers who are likely to be unfairly treated (e.g., with strollers, wheelchairs, or bulky luggage), in order to achieve fair waiting times as well as efficient transportation. Elevators...
Article
Full-text available
Whereas research of the multi-agent patrolling problem has been widely conducted from different aspects, the issue of energy minimization has not been sufficiently studied. When considering real-world applications with a trade-off between energy efficiency and level of perfection, it is usually more desirable to minimize the energy cost and carry o...
Conference Paper
We present an interaction strategy with reinforcement learning to promote mutual cooperation among agents in complex networks. Networked computerized systems consisting of many agents that are delegates of social entities, such as companies and organizations, are being implemented due to advances in networking and computer technologies. Because the...
Chapter
We examine whether a team of agents can learn geometric and strategic group formations by using deep reinforcement learning in adversarial multi-agent systems. This is a significant point underlying the control and coordination of multiple autonomous and intelligent agents. While there are many possible approaches to solve this problem, we are inte...
Chapter
Multi-agent patrolling problem has received growing attention from many researchers due to its wide range of potential applications. In realistic environment, e.g., security patrolling, each location has different visitation requirement according to the required security level. Therefore, a patrolling system with non-uniform visiting frequency is p...
Conference Paper
We propose a model of a social networking service (SNS) with diminishing marginal utility in the framework of evolutionary computing and present our investigation on the effect of diminishing marginal utility on the dominant structure of strategies in all agents. SNSs such as Twitter and Facebook have been growing rapidly, but why they are prosperi...
Article
Full-text available
Recent advances in computer and network technologies enable the provision of many services combining multiple types of information and different computational capabilities. The tasks for these services are executed by allocating them to appropriate collaborative agents, which are computational entities with specific functionality. However, the numb...
Conference Paper
This paper investigates the effect of direct reciprocity on voluntary participation in social networking services (SNS) by modeling them as a type of public goods (PG) game. Because the fundamental structure of SNS is similar to the PG games, some studies have focused on why voluntary activities in SNS emerge by modifying the PG game. However, thei...
Conference Paper
This paper proposes an interaction strategy called the extended expectation-of-cooperation (EEoC) that is intended to spread cooperative activities in prisoner's dilemma situations over an entire agent network. Recently developed computer and communications applications run on the network and interact with each other as delegates of the owners, so...
Conference Paper
We propose a learning and negotiation method to enhance divisional cooperation and demonstrate its robustness to environmental changes in the context of the multi-agent cooperative problem. With the ongoing advances in information and communication technology, we now have access to a vast array of information, and everything has become more closely...
Article
Full-text available
Background Social networking services (SNSs) are widely used as communicative tools for a variety of purposes. SNSs rely on the users’ individual activities associated with some cost and effort, and thus it is not known why users voluntarily continue to participate in SNSs. Because the structures of SNSs are similar to that of the public goods (PG)...
Conference Paper
This paper proposes a control method for in agents by switching their behavioral strategy between rationality and reciprocity depending on their internal states to achieve efficient team formation. Advances in computer science, telecommunications, and electronic devices have led to proposals of a variety of services on the Internet that are achieve...
Article
We describe a method for decentralized task/area partitioning for coordination in cleaning/sweeping domains with learning to identify the easy-to-dirty areas. Ongoing advances in computer science and robotics have led to applications for covering large areas that require coordinated tasks by multiple control programs including robots. Our study aim...
Article
This paper proposes a behavioral strategy with which agents select rational or reciprocal behavior depending on the past cooperative activities. Rational behavioral strategy lets agents select actions to try to maximize the direct and immediate rewards, while agents with the reciprocal behavioral strategy try to work with cooperative partners for s...
Article
We propose a fair and accurate peer assessment method for group work using a multi-agent trust network. Although group work is an effective educational method, accurately assessing individual students is not easy. Mutual evaluation is often used to assess group work because students can observe the contributions of other students. However, mutual e...
Conference Paper
This paper proposes a task allocation method in which, although social utility is attempted to be maximized, agents also give weight to individual preferences based on their own specifications and capabilities. Due to the recent advances in computer and network technologies, many services can be provided by appropriately combining multiple types of...
Conference Paper
This paper proposes a behavioral strategy called expectation of cooperation with which cooperation in the prisoner’s dilemma game spreads over agent networks by incorporating Q-learning. Recent advances in computer and communication technologies enable intelligent agents to operate in small and handy computers such as mobile PCs, tablet computers,...
Article
We propose an SNS-norms game to model behavioral strategies in social networking services (SNSs) and investigate the conditions required for the evolution of cooperation-dominant situations. SNSs such as Facebook and Google+ are indispensable social media for a variety of social communications ranging from personal chats to business and political c...
Conference Paper
We describe the reciprocal agents that build virtual associations in accordance with past cooperative work in a bottom-up manner and that allocate tasks or resources preferentially to agents in the same associations in busy large-scale distributed environments. Models of multi-agent systems (MAS) are often used to express tasks that are done by tea...
Conference Paper
The coalition structure generation problem is now a big issue in the field of multi-agent systems. With many agents working in the same environment, cooperation may become a key point to complete a mission efficiently. This problem can also be found in many areas such as sensor networks, multi-robot systems, and even e-commerce. However, it has bee...
Conference Paper
In this work, we propose agents that switch their behavioral strategy between rationality and reciprocity depending on their internal states to achieve efficient team formation. With the recent advances in computer science, mechanics, and electronics, there are an increasing number of applications with services/goals that are achieved by teams of d...
Chapter
We formulate an assignment problem-solving framework called single-object resource allocation with preferential order (SORA/PO) to incorporate values of resources and individual preferences into assignment problems. We then devise methods to find semi-optimal solutions for SORA/PO problems. The assignment, or resource allocation, problem is a funda...
Article
This paper proposes a method to estimate malicious domain names from a large scale DNS query response dataset. The key idea of the work is to leverage the use of DNS graph that is a bipartite graph consisting of domain names and corresponding IP addresses. We apply a concept of Probabilistic Threat Propagation (PTP) on the graph with a set of prede...
Article
We propose an SNS-norms game to model behavioral strategies in social networking services (SNSs) and in- vestigate the conditions required for the evolution of cooperation-dominant situations. SNSs such as Facebook and Google+ are indispensable social media for a variety of social communications ranging from personal chats to busi- ness and politic...
Conference Paper
We propose a fair peer assessment method for group work using a multi-agent trust network. Although group work is an effective educational method, accurately assessing individual students is not easy. Mutual evaluation is often used for such assessment, but often presents some potential problems such as irresponsible evaluations and collusion. Our...

Network

Cited By