no code implementations • EMNLP (NLPOSS) 2020 • Nitin Madnani, Anastassia Loukina
For the last 5 years, we have developed and maintained RSMTool – an open-source tool for evaluating NLP systems that automatically score written and spoken responses.
no code implementations • WS 2020 • Anastassia Loukina, Nitin Madnani, Aoife Cahill, Lili Yao, Matthew S. Johnson, Brian Riordan, Daniel F. McCaffrey
The effect of noisy labels on the performance of NLP systems has been studied extensively for system training.
no code implementations • ACL 2020 • Beata Beigman Klebanov, Nitin Madnani
In this theme paper, we focus on Automated Writing Evaluation (AWE), using Ellis Page{'}s seminal 1966 paper to frame the presentation.
no code implementations • WS 2019 • Anastassia Loukina, Nitin Madnani, Klaus Zechner
We illustrate that total fairness may not be achievable and that different definitions of fairness may require different solutions.
no code implementations • ACL 2019 • Nitin Madnani, Beata Beigman Klebanov, Anastassia Loukina, Binod Gyawali, Patrick Lange, John Sabatini, Michael Flor
Literacy is crucial for functioning in modern society.
no code implementations • COLING 2018 • Nitin Madnani, Jill Burstein, Norbert Elliot, Beata Beigman Klebanov, Diane Napolitano, Slava Andreyev, Maxwell Schwartz
Writing Mentor is a free Google Docs add-on designed to provide feedback to struggling writers and help them improve their writing in a self-paced and self-regulated fashion.
no code implementations • COLING 2018 • Nitin Madnani, Aoife Cahill
In this position paper, we argue that building operational automated scoring systems is a task that has disciplinary complexity above and beyond standard competitive shared tasks which usually involve applying the latest machine learning techniques to publicly available data in order to obtain the best accuracy.
no code implementations • WS 2018 • Burr Settles, Chris Brust, Erin Gustafson, Masato Hagiwara, Nitin Madnani
We present the task of \textit{second language acquisition (SLA) modeling}.
no code implementations • NAACL 2018 • Su-Youn Yoon, Aoife Cahill, Anastassia Loukina, Klaus Zechner, Brian Riordan, Nitin Madnani
In large-scale educational assessments, the use of automated scoring has recently become quite common.
no code implementations • WS 2017 • Anastassia Loukina, Nitin Madnani, Aoife Cahill
We consider the automatic scoring of a task for which both the content of the response as well its spoken fluency are important.
no code implementations • WS 2017 • Nitin Madnani, Anastassia Loukina, Aoife Cahill
We explore various supervised learning strategies for automated scoring of content knowledge for a large corpus of 130 different content-based questions spanning four subject areas (Science, Math, English Language Arts, and Social Studies) and containing over 230, 000 responses scored by human raters.
no code implementations • WS 2017 • Nitin Madnani, Anastassia Loukina, Alina von Davier, Jill Burstein, Aoife Cahill
Automated scoring of written and spoken responses is an NLP application that can significantly impact lives especially when deployed as part of high-stakes tests such as the GRE® and the TOEFL®.
no code implementations • 4 Mar 2014 • Derrick Higgins, Chris Brew, Michael Heilman, Ramon Ziai, Lei Chen, Aoife Cahill, Michael Flor, Nitin Madnani, Joel Tetreault, Daniel Blanchard, Diane Napolitano, Chong MIn Lee, John Blackmore
Developments in the educational landscape have spurred greater interest in the problem of automatically scoring short answer questions.
no code implementations • TACL 2013 • Beata Beigman Klebanov, Nitin Madnani, Jill Burstein
We demonstrate a method of improving a seed sentiment lexicon developed on essay data by using a pivot-based paraphrasing system for lexical expansion coupled with sentiment profile enrichment using crowdsourcing.