Content uploaded by Stephen E. Robertson
Author content
All content in this area was uploaded by Stephen E. Robertson on Sep 07, 2016
Content may be subject to copyright.
Some ethical and political implications of theoretical research in information science
... Over the last decade, a growing number of research articles in the ML community have focused on defining appropriate notions of fairness and then developing models to ensure fairness in automated decision making (DM). Awareness on fairness and ethics in information retrieval has been raised by Belkin and Robertson already in 1976 [20]. Generally, the current notions of fairness are mainly influenced by the concept of discrimination in social sciences, law and economy [35]. ...
... ItemKNN [50,60,70,90, [5,10,20,30] ular recommendations. Third, ItemKNN and UserKNN exploit item-item and useruser similarities based on interaction data, hence they are not expected to promote a particular type of user or item, unless those are already over-represented in the input recommendation data; however, it is true that researchers have exposed that, depending on their parameters, these algorithms might behave as slightly personalized versions of the MostPopular algorithm, hence replicating the same biased/unfair suggestions [21,56,24]. ...
One common characteristic of research works focused on fairness evaluation (in machine learning) is that they call for some form of parity (equality) either in treatment – meaning they ignore the information about users’ memberships in protected classes during training – or in impact – by enforcing proportional beneficial outcomes to users in different protected classes. In the recommender systems community, fairness has been studied with respect to both users’ and items’ memberships in protected classes defined by some sensitive attributes (e.g., gender or race for users, revenue in a multi-stakeholder setting for items). Again here, the concept has been commonly interpreted as some form of equality– i.e., the degree to which the system is meeting the information needs of all its users in an equal sense. In this work, we propose a probabilistic framework based on Generalized Cross-Entropy (GCE) to measure the fairness of a given recommendation model. The framework comes with a suite of advantages: first, it allows the system designer to define and measure fair-ness for both users and items and can be applied to any classification task; second,it can incorporate various notions of fairness as it does not rely on specific and pre-defined probability distributions and they can be defined at design time; finally, in its design, it uses a gain factor, which can be flexibly defined to contemplate different accuracy-related metrics to measure fairness upon such as decision-support metrics(e.g., precision, recall) or rank-based measures (e.g., NDCG, MAP). Results on four real-world datasets show the nuances captured by our proposed metric regarding fair-ness on different user and item attributes, where nearest-neighbor recommenders tend to obtain good results under equality constraints.
... For all other uses, contact the owner/author(s and engineers to understand how these systems interact with society in general, including the various biases -some benign, some connected to historical patterns of discrimination -in their underlying data and in the responses of their users [6]. Indeed, Belkin and Robertson [1] stress the need for considering social implications of information retrieval research when they write, "the development of theory must depend not only on the internal constraints of the science but also upon its external constraints. " ...
Fairness and related concerns have become of increasing importance in a variety of AI and machine learning contexts. They are also highly relevant to recommender systems and related problems such as information retrieval, as evidenced by the growing literature in RecSys, FAT*, SIGIR, and special sessions such as the FATREC and FACTS-IR workshops and the Fairness track at TREC 2019; however, translating algorithmic fairness constructs from classification, scoring, and even many ranking settings into recommendation and other information access scenarios is not a straightforward task. This tutorial will help orient RecSys researchers to algorithmic fairness, understand how concepts do and do not translate from other settings, and provide an introduction to the growing literature on this topic.
... For all other uses, contact the owner/author(s and engineers to understand how these systems interact with society in general, including the various biases -some benign, some connected to historical patterns of discrimination -in their underlying data and in the responses of their users [6]. Indeed, Belkin and Robertson [1] stress the need for considering social implications of information retrieval research when they write, "the development of theory must depend not only on the internal constraints of the science but also upon its external constraints. " ...
Fairness and related concerns have become of increasing importance in a variety of AI and machine learning contexts. They are also highly relevant to information retrieval and related problems such as recommendation, as evidenced by the growing literature in SIGIR, FAT*, RecSys, and special sessions such as the FATREC workshop and the Fairness track at TREC 2019; however, translating algorithmic fairness constructs from classification, scoring, and even many ranking settings into information retrieval and recommendation scenarios is not a straightforward task. This tutorial will help to orient IR researchers to algorithmic fairness, understand how concepts do and do not translate from other settings, and provide an introduction to the growing literature on this topic.
The First Search Futures Workshop, in conjunction with the Fourty-sixth European Conference on Information Retrieval (ECIR) 2024, looked into the future of search to ask questions such as:
• How can we harness the power of generative AI to enhance, improve and re-imagine Information Retrieval (IR)?
• What are the principles and fundamental rights that the field of Information Retrieval should strive to uphold?
• How can we build trustworthy IR systems in light of Large Language Models and their ability to generate content at super human speeds?
• What new applications and affordances does generative AI offer and enable, and can we go back to the future, and do what we only dreamed of previously?
The workshop started with seventeen lightning talks from a diverse set speakers. Instead of conventional paper presentations, the lightning talks provided a rapid and concise overview of ideas, allowing speakers to share critical points or novel concepts quickly. This format was designed to encourage discussion and introduce a wide range of topics within a short period, thereby maximising the exchange of ideas and ensuring that participants could gain insights into various future search areas without the deep dive typically required in longer presentations. This report, co-authored by the workshop's organisers and its participants, summarises the talks and discussions. This report aims to provide the broader IR community with the insights and ideas discussed and debated during the workshop - and to provide a platform for future discussion.
Date : 24 March 2024.
Website : https://searchfutures.github.io/.
Information Retrieval (IR) systems have a wide range of impacts on consumers. We offer maps to help identify goals IR systems could—or should—strive for, and guide the process of scoping how to gauge a wide range of consumer-side impacts and the possible interventions needed to address these effects. Grounded in prior work on scoping algorithmic impact efforts, our goal is to promote and facilitate research that (1) is grounded in impacts on information consumers, contextualizing these impacts in the broader landscape of positive and negative consumer experience; (2) takes a broad view of the possible means of changing or improving that impact, including non-technical interventions; and (3) uses operationalizations and strategies that are well-matched to the technical, social, ethical, legal, and other dimensions of the specific problem in question.
Providing users with relevant search results has been the primary focus of information retrieval research. However, focusing on relevance alone can lead to undesirable side effects. For example, small differences between the relevance scores of documents that are ranked by relevance alone can result in large differences in the exposure that the authors of relevant documents receive, i.e., the likelihood that the documents will be seen by searchers. Therefore, developing fair ranking techniques to try to ensure that search results are not dominated, for example, by certain information sources is of growing interest, to mitigate against such biases. In this work, we argue that generating fair rankings can be cast as a search results diversification problem across a number of assumed fairness groups, where groups can represent the demographics or other characteristics of information sources. In the context of academic search, as in the TREC Fair Ranking Track, which aims to be fair to unknown groups of authors, we evaluate three well-known search results diversification approaches from the literature to generate rankings that are fair to multiple assumed fairness groups, e.g. early-career researchers vs. highly-experienced authors. Our experiments on the 2019 and 2020 TREC datasets show that explicit search results diversification is a viable approach for generating effective rankings that are fair to information sources. In particular, we show that building on xQuAD diversification as a fairness component can result in a significant (p<0.05) increase (up to 50% in our experiments) in the fairness of exposure that authors from unknown protected groups receive.
Ethical questions in information science research, teach ing and practice are considered. It is recommended that supervisors of research make clear their publication policies before a piece of research is undertaken and if they don't wish to publish, then the junior author should have the right to write up the work. In any case, papers should be in alphabetical order of authors. Certain areas of research should possibly not be attempted because of their possible implications. Bias in teaching is considered. The following problems in information work are considered: the unauthor ised use of work facilities, the confidentiality of inquiries, bias in results presented and many aspects of the social responsibility of an information scientist. It is concluded that inquiries should always remain confidential, but that infor mation scientists should be willing to divulge non-confiden tial matter to bona fide third parties and should be prepared to reveal confidential matter if the public interest warrants it. No information scientist should be asked to present biased or misleading results. There is a need for a code of ethics for information scientists, and information scientists need to be far more aware of ethical questions.
Information workers are increasingly required to involve themselves in activities which may have ethical implica tions: deciding between information sources, evaluating out put, carrying out surveys of usage, etc. There is little guidance as to what constitutes reasonable behaviour and there is, as yet, no adequate code of conduct for the profession. Debates on the usefulness of such a code have often lacked direction and failed to address the major issue of whether information can ever by supphed on a neutral, value-free, basis This article considers the arguments for and against the neutral supply of information services and summanses the main areas of ethical concern in information work and information research before assessing whether a formal code of ethics is really the best solution for the profession.
ResearchGate has not been able to resolve any references for this publication.