no code implementations • 9 Feb 2024 • Andrew Smart, Ding Wang, Ellis Monk, Mark Díaz, Atoosa Kasirzadeh, Erin Van Liemt, Sonja Schmer-Galunder
Data annotation remains the sine qua non of machine learning and AI.
no code implementations • 8 Dec 2023 • Noam Benkler, Drisana Mosaphir, Scott Friedman, Andrew Smart, Sonja Schmer-Galunder
We apply RVR to the text generated by LLMs to characterize implicit moral values, allowing us to quantify the moral/cultural distance between LLMs and various demographics that have been surveyed using the WVS.
no code implementations • 16 May 2023 • Sanna J. Ali, Angèle Christin, Andrew Smart, Riitta Katila
Based on a qualitative analysis of technology workers tasked with integrating AI ethics into product development, we find that workers experience an environment where policies, practices, and outcomes are decoupled.
no code implementations • 14 Mar 2023 • Jamila Smith-Loud, Andrew Smart, Darlene Neal, Amber Ebinama, Eric Corbett, Paul Nicholas, Qazi Rashid, Anne Peckham, Sarah Murphy-Gray, Nicole Morris, Elisha Smith Arrillaga, Nicole-Marie Cotton, Emnet Almedom, Olivia Araiza, Eliza McCullough, Abbie Langston, Christopher Nellum
This paper reports on our initial evaluation of The Equitable AI Research Roundtable -- a coalition of experts in law, education, community engagement, social justice, and technology.
no code implementations • 6 Oct 2022 • Shalaleh Rismani, Renee Shelby, Andrew Smart, Edgar Jatho, Joshua Kroll, AJung Moon, Negar Rostamzadeh
Inappropriate design and deployment of machine learning (ML) systems leads to negative downstream social and ethical impact -- described here as social and ethical risks -- for users, society and the environment.
1 code implementation • 26 Feb 2022 • Negar Rostamzadeh, Diana Mincu, Subhrajit Roy, Andrew Smart, Lauren Wilcox, Mahima Pushkarna, Jessica Schrouff, Razvan Amironesei, Nyalleng Moorosi, Katherine Heller
Our findings from the interviewee study and case studies show 1) that datasheets should be contextualized for healthcare, 2) that despite incentives to adopt accountability practices such as datasheets, there is a lack of consistency in the broader use of these practices 3) how the ML for health community views datasheets and particularly \textit{Healthsheets} as diagnostic tool to surface the limitations and strength of datasets and 4) the relative importance of different fields in the datasheet to healthcare concerns.
no code implementations • 8 Dec 2020 • Angie Peng, Jeff Naecker, Ben Hutchinson, Andrew Smart, Nyalleng Moorosi
The same group of workers then votes on a bonus payment structure, to elicit preferences.
no code implementations • 23 Oct 2020 • Ben Hutchinson, Andrew Smart, Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, Margaret Mitchell
In this paper, we introduce a rigorous framework for dataset development transparency which supports decision-making and accountability.
no code implementations • 17 Jun 2020 • Donald Martin Jr., Vinodkumar Prabhakaran, Jill Kuhlberg, Andrew Smart, William S. Isaac
Machine learning (ML) fairness research tends to focus primarily on mathematically-based interventions on often opaque algorithms or models and/or their immediate inputs and outputs.
no code implementations • 15 May 2020 • Donald Martin Jr., Vinodkumar Prabhakaran, Jill Kuhlberg, Andrew Smart, William S. Isaac
Recent research on algorithmic fairness has highlighted that the problem formulation phase of ML system development can be a key source of bias that has significant downstream impacts on ML system fairness outcomes.
no code implementations • 3 Jan 2020 • Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, Parker Barnes
Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited for harm by investigators from outside the organizations deploying the algorithms.
Computers and Society