Content uploaded by Thorsten Joachims
Author content
All content in this area was uploaded by Thorsten Joachims on May 30, 2018
Content may be subject to copyright.
A preview of the PDF is not available
WebWatcher: A tour guide for the World Wide Web
Abstract and Figures
We explore the notion of a tour guide software agent for assisting users browsing the world wide web. A web tour guide agent provides assistance similar to that provided by a human tour guide in a museum -- it guides the user along an appropriate path through the collection, based on its knowledge of the user's interests, of the location and relevance of various items in the collection, and of the way in which others have interacted with the collection in the past. This paper describes a simple but operational tour guide, called WebWatcher, which has given over 5000 tours to people browsing CMU's School of Computer Science web pages. WebWatcher accompanies users from page to page, suggests appropriate hyperlinks, and learns from experience to improve its advice-giving skills. 1 Introduction Browsing the World Wide Web is much like visiting a museum. In a museum the visitor has general areas of interest and wants to see relevant artifacts. But visitors find it difficult to locate rele...
Figures - uploaded by Thorsten Joachims
Author content
All figure content in this area was uploaded by Thorsten Joachims
Content may be subject to copyright.
... While the rst attempt to use RL for RSs, WebWatcher [32], is almost simultaneous with the foundation of RSs, it is just recently that many RLRSs have been proposed [17,3338]. The reason of this recent interest is the foundation of DRL, a combination of deep learning and RL, which made it possible to apply RL in large action and state spaces [39,40]. ...
... Olston and Chi [70] proposed ScentTrails that leverage an interface that combines browsing and searching and highlights potentially relevant hyperlinks. WebWatcher [43], like ScentTrails, underlined the relevant hyperlinks and improved the model based on the implicit feedback collected during previous tours. ...
Current interactive systems with natural language interface lack an ability to understand a complex information-seeking request which expresses several implicit constraints at once, and there is no prior information about user preferences, e.g., "find hiking trails around San Francisco which are accessible with toddlers and have beautiful scenery in summer", where output is a list of possible suggestions for users to start their exploration. In such scenarios, the user requests can be issued at once in the form of a complex and long query, unlike conversational and exploratory search models that require short utterances or queries where they often require to be fed into the system step by step. This advancement provides the final user more flexibility and precision in expressing their intent through the search process. Such systems are inherently helpful for day-today user tasks requiring planning that are usually time-consuming, sometimes tricky, and cognitively taxing. We have designed and deployed a platform to collect the data from approaching such complex interactive systems. In this paper, we propose an Interactive Agent (IA) that allows intricately refined user requests by making it complete, which should lead to better retrieval. To demonstrate the performance of the proposed modeling paradigm, we have adopted various pre-retrieval metrics that capture the extent to which guided interactions with our system yield better retrieval results. Through extensive experimentation, we demonstrated that our method significantly outperforms several robust baselines
... Some examples of hybrid recommender systems include Fab [26], the very first hybrid recommendation system, WebWatcher [27], P-Tango [28], Content-boosted Collaborative Filtering [29], CinemaScreen [30] and the one proposed in [31]. Some of the problems solved by hybrid recommendation systems include: ...
These days, the Internet contains an overwhelming amount of data for users looking for specific information. This is why we use recommender systems to deal with information overload problems. Among several issues, this research focuses primarily on recommending articles to users who belong to a group. The groups are pre-defined based on employees' work roles and can be further divided into subgroups. We use different group recommendation and sub-grouping techniques to decide which one gives optimal results. Three recommendation techniques have been applied to suggest articles to the groups, namely: Before Factorization, After Factorization, and Weighted Before Factorization. In the experiment, Weighted Before Factorization achieves the best results on our dataset, collected from a company's internal content management system. We have also proposed an enhancement to the above group recommendation models using clustering methods to create further subgroups. Compared with the results on the original pre-defined groups, k-means sub-grouping improves the F1@5, F1@10, F1@15 by 35.75%, 19.52% and 1.54% respectively.
... Zou et al. [37] formulate the ranking process as a multi-agent Markov Decision Process, where mutual interactions are incorporated to compute the ranking list. In the 1900s, WebWatcher [38] models the web page recommendation problem as an RL problem and adopts Q-learning to improve its performance. Later, with the development of deep learning, combining deep learning with traditional RL methods is becoming increasingly popular in RS. ...
With the increasingly fierce market competition, offering a free trial has become a potent stimuli strategy to promote products and attract users. By providing users with opportunities to experience goods without charge, a free trial makes adopters know more about products and thus encourages their willingness to buy. However, as the critical point in the promotion process, finding the proper adopters is rarely explored. Empirically winnowing users by their static demographic attributes is feasible but less effective, neglecting their personalized preferences. To dynamically match the products with the best adopters, in this work, we propose a novel free trial user selection model named SMILE, which is based on reinforcement learning (RL) where an agent actively selects specific adopters aiming to maximize the profit after free trials. Specifically, we design a tree structure to reformulate the action space, which allows us to select adopters from massive user space efficiently. The experimental analysis on three datasets demonstrates the proposed model's superiority and elucidates why reinforcement learning and tree structure can improve performance. Our study demonstrates technical feasibility for constructing a more robust and intelligent user selection model and guides for investigating more marketing promotion strategies.
... Joachims, Juhne [4] and Pazzani and Billsus [5] demonstrated the instances of users recommend connections of every client. The visitor visit the web server dependent on user enthusiasm, by whenever the program will demonstrate the user propose site dependent on client interest. ...
The recognition of user's visited set of web pages for the prediction of web page is a key drawback. Thus the work is employed with the web access log files which is stored in the server. To understand the user interest patterns, the web access log files are extracted that depicts the user behavior. Various applications can be employed to predicting user's behavior while serving the web. During this work, the proposed framework analyze the user usage, reinforced the content and the content retrieved with the semantic manner. The semantic information retrieval supported the user access pages are preprocessed and the web log data of the particular user is analyzed to identify the user profile. Then the retrieved information is graded with clustering the semantic content based results. The ranked content is then analyzed with the user profile to produce an optimized search results for the users based on the user classification.
With the rapid advancement of ICT, the digital transformation on education is greatly accelerating in various applications. As a particularly prominent application of digital education, quiz question recommendation is playing a vital role in precision teaching, smart tutoring, and personalized learning. However, the looming challenge of quiz question recommender for students is to satisfy the question diversity demands for students ZPD (the zone of proximal development) stage dynamically online. Therefore, we propose to formalize quiz question recommendation with a novel approach of reinforcement learning based two-sided recommender system. We develop a recommendation framework RTR (Reinforcement-Learning based Two-sided Recommender Systems) for taking into account the interests of both sides of the system, learning and adapting to those interests in real time, and resulting in more satisfactory recommended content. This established recommendation framework captures question characters and student dynamic preferences by considering the emergence of both sides of the system, and it yields a better learning experience in the context of practical quiz question generation.
Recommender systems (RSs) have become an inseparable part of our everyday lives. They help us find our favorite items to purchase, our friends on social networks, and our favorite movies to watch. Traditionally, the recommendation problem was considered to be a classification or prediction problem, but it is now widely agreed that formulating it as a sequential decision problem can better reflect the user-system interaction. Therefore, it can be formulated as a Markov decision process (MDP) and be solved by reinforcement learning (RL) algorithms. Unlike traditional recommendation methods, including collaborative filtering and content-based filtering, RL is able to handle the sequential, dynamic user-system interaction and to take into account the long-term user engagement. Although the idea of using RL for recommendation is not new and has been around for about two decades, it was not very practical, mainly because of scalability problems of traditional RL algorithms. However, a new trend has emerged in the field since the introduction of deep reinforcement learning (DRL), which made it possible to apply RL to the recommendation problem with large state and action spaces. In this paper, a survey on reinforcement learning based recommender systems (RLRSs) is presented. Our aim is to present an outlook on the field and to provide the reader with a fairly complete knowledge of key concepts of the field. We first recognize and illustrate that RLRSs can be generally classified into RL- and DRL-based methods. Then, we propose an RLRS framework with four components, i.e., state representation, policy optimization, reward formulation, and environment building, and survey RLRS algorithms accordingly. We highlight emerging topics and depict important trends using various graphs and tables. Finally, we discuss important aspects and challenges that can be addressed in the future.
An Institution Website is a profile of an institution for the people who directly or indirectly related to the agency. Some problems related to the performance of a web are the speed and accuracy of presentation of the information needed by the community.Technology of adaptive Website is one of technology that attempted to simplify the user to find the information that need from a website. The technology is based on web log. Log is used as a reference for the access patterns are realized in the form of recommendations links to information that is often accessed by people from the website. Log data processing performed by implemented the FWDPTree algorithm to get a particular tree structure that stores information page along with the frequency of occurrence, then performed datamining by algorithm FWDP-mine.This Technology has been able to reach information more quickly There are still weaknesses in this system. The adaptive system has not been able to make the process of adaptation in realtime, this is due to the need for time to process large log data in order to obtain the user's access patterns, while the session that occurred during the offline process does not processed. Keywords— Adaptive web, Data Log, FWDP Algorithm, Association Rule, Realtime
Learning Apprentice Systems are interactive knowledge-based consultant systems that directly confront the knowledge-acquisition bottleneck by learning from their users. This paper discusses the notion of Learning Apprentice Systems in general, and summarizes our efforts to develop a specific Learning Apprentice in the domain of VLSI design.
Traducción de: Vosstanovlenie zavisimosteipo émpiricheskim dannym Incluye bibliografía e índice
Recent developments in the storage, retrieval, and manipulation of large text files are described. The text analysis problem
is examined, and modern approaches leading to the identification and retrieval of selected text items in response to search
requests are discussed.
This paper describes the first implementation of WebWatcher, a Learning Apprentice for the World Wide Web. We also explore the possibility of extracting information from the structure of hypertext. We introduce an algorithm which identifies pages that are related to a given page using only hypertext structure. We motivate the algorithm by using the Minimum Description Length principle. 1 Introduction The World Wide Web is growing quickly and addresses more and more users. Although a lot of information is available in the World Wide Web (WWW), it is difficult to find particular pieces of information efficiently. Many have noted the need for software that helps the user search for information. This paper describes the design of WebWatcher [Armstrong et al., 1995], an agent which assists users in locating information on the WWW or searches autonomously on their behalf. In interactive mode WebWatcher acts as a Learning Apprentice [Mitchell et al., 1985] [Mitchell et. al., 1994]...
Webwatcher: A learning apprentice for the world wide web
- R Armstrong
- D Freitag
- T Joachims
- T Mitchell
R. Armstrong, D. Freitag, T. Joachims, and T. Mitchell. Webwatcher: A learning apprentice for the world wide web. In AAAI
Spring Symposium on Information Gathering from Heterogeneous,
Distributed Environments, March 1995.
Learning information retrieval agents: Experiments with automated web browsing
- M Balabanovic
- Y Shoham
M. Balabanovic and Y. Shoham. Learning information retrieval
agents: Experiments with automated web browsing. In AAAI Spring
Symposium Series on Information Gathering from Distributed, Heterogeneous Environments, Working Notes. AAAI-Press, 1995.
Stable function approximation in dynamic programming
- G Gordon
G. Gordon. Stable function approximation in dynamic programming. In International Conference on Machine Learning, 1995.
Letizia: An agent that assists web browsing
- H Lieberman
H. Lieberman. Letizia: An agent that assists web browsing. In
International Joint Conference on Arti cial Intelligence, Montreal,
August 1995.
Syskill & webert: Identifying interesting web sites
- M Pazzani
- J Muramatsu
- D Billsus
M. Pazzani, J. Muramatsu, and D. Billsus. Syskill & webert: Identifying interesting web sites. In AAAI Conference, Portland, August
1996.