no code implementations • 21 Sep 2023 • Ngozi Ihemelandu, Michael D. Ekstrand
The strategy for selecting candidate sets -- the set of items that the recommendation system is expected to rank for each user -- is an important decision in carrying out an offline top-$N$ recommender system evaluation.
no code implementations • 19 Sep 2023 • Amifa Raj, Michael D. Ekstrand
We examine how fairness scores change with different ranking layouts to yield insights into (1) the consistency of fair ranking measurements across layouts; (2) whether rankings optimized for fairness in a linear ranking remain fair when the results are displayed in a grid; and (3) the impact of column reduction approaches to support different device geometries on fairness measurement.
no code implementations • 12 Sep 2023 • Michael D. Ekstrand, Ben Carterette, Fernando Diaz
Current practice for evaluating recommender systems typically focuses on point estimates of user-oriented effectiveness metrics or business metrics, sometimes combined with additional metrics for considerations such as diversity and novelty.
no code implementations • 3 May 2023 • Ngozi Ihemelandu, Michael D. Ekstrand
However, these studies are focused on TREC-style experiments, which typically have fewer than 100 topics.
no code implementations • 25 Apr 2023 • Amifa Raj, Bhaskar Mitra, Nick Craswell, Michael D. Ekstrand
There are many ways a query, the search results, and a demographic attribute such as gender may relate, leading us to hypothesize different causes for these reformulation patterns, such as under-representation on the original result page or based on the linguistic theory of markedness.
no code implementations • 21 Feb 2023 • Michael D. Ekstrand, Graham McDonald, Amifa Raj, Isaac Johnson
The 2021 Fair Ranking track aimed to ensure that documents that are about, or somehow represent, certain protected characteristics receive a fair exposure to the Wikipedia editors, so that the documents have an fair opportunity of being improved and, therefore, be well-represented in Wikipedia.
no code implementations • 11 Feb 2023 • Michael D. Ekstrand, Graham McDonald, Amifa Raj, Isaac Johnson
The 2022 Fair Ranking track aimed to ensure that documents that are about, or somehow represent, certain protected characteristics receive a fair exposure to the Wikipedia editors, so that the documents have an fair opportunity of being improved and, therefore, be well-represented in Wikipedia.
no code implementations • 12 Jan 2023 • Christine Pinney, Amifa Raj, Alex Hanna, Michael D. Ekstrand
Information access research (and development) sometimes makes use of gender, whether to report on the demographics of participants in a user study, as inputs to personalized results or recommendations, or to make systems gender-fair, amongst other purposes.
no code implementations • 6 Sep 2022 • Michael D. Ekstrand, Maria Soledad Pera
The last several years have brought a growing body of work on ensuring that recommender systems are in some sense consumer-fair -- that is, they provide comparable quality of service, accuracy of representation, and other effects to their users.
no code implementations • 28 Jun 2022 • Amifa Raj, Michael D. Ekstrand
Search engines in e-commerce settings allow users to search, browse, and select items from a wide range of products available online including children's items.
no code implementations • 2 Oct 2021 • Michael D. Ekstrand
Recommender systems research is concerned with many aspects of recommender system behavior and effects than simply its effectiveness, and simulation can be a powerful tool for uncovering these effects.
no code implementations • 14 Sep 2021 • Ngozi Ihemelandu, Michael D. Ekstrand
In this paper, we argue that the use of statistical inference is a key component of the evaluation process that has not been given sufficient attention.
no code implementations • 11 Aug 2021 • Asia J. Biega, Fernando Diaz, Michael D. Ekstrand, Sergey Feldman, Sebastian Kohlmeier
This paper provides an overview of the NIST TREC 2020 Fair Ranking track.
no code implementations • 13 May 2021 • Amifa Raj, Ashlee Milton, Michael D. Ekstrand
In this position paper, we argue for the need to investigate if and how gender stereotypes manifest in search and recommender systems. As a starting point, we particularly focus on how these systems may propagate and reinforce gender stereotypes through their results in learning environments, a context where teachers and children in their formative stage regularly interact with these systems.
no code implementations • 12 May 2021 • Michael D. Ekstrand, Anubrata Das, Robin Burke, Fernando Diaz
Recommendation, information retrieval, and other information access systems pose unique challenges for investigating and applying the fairness and non-discrimination concepts that have been developed for studying other machine learning systems.
no code implementations • 2 Sep 2020 • Amifa Raj, Michael D. Ekstrand
Ranked lists are frequently used by information retrieval (IR) systems to present results believed to be relevant to the users information need.
no code implementations • 27 Apr 2020 • Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, Ben Carterette
We introduce the concept of \emph{expected exposure} as the average attention ranked items receive from users over repeated samples of the same query.
no code implementations • 25 Mar 2020 • Asia J. Biega, Fernando Diaz, Michael D. Ekstrand, Sebastian Kohlmeier
The goal of the TREC Fair Ranking track was to develop a benchmark for evaluating retrieval systems in terms of fairness to different content providers in addition to classic notions of relevance.
1 code implementation • 26 Jan 2020 • Mucun Tian, Michael D. Ekstrand
Offline evaluations of recommender systems attempt to estimate users' satisfaction with recommendations using static data from prior user interactions.
no code implementations • 12 Jul 2019 • Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, Michael D. Ekstrand
The proceedings list for the program of FACTS-IR 2019, the Workshop on Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval held at SIGIR 2019.
no code implementations • 4 Feb 2019 • Michael D. Ekstrand, Joseph A. Konstan
In the course of years of teaching and research on recommender systems, we have seen the val-ue in adopting a consistent notation across our work.
2 code implementations • 10 Sep 2018 • Michael D. Ekstrand
LensKit is an open-source toolkit for building, researching, and learning about recommender systems.
2 code implementations • 22 Aug 2018 • Michael D. Ekstrand, Daniel Kluver
Collaborative filtering algorithms find useful patterns in rating and consumption data and exploit these patterns to guide users to good items.