1 code implementation • 9 Nov 2023 • Shuyi Xie, Wenlin Yao, Yong Dai, Shaobo Wang, Donlin Zhou, Lifeng Jin, Xinhua Feng, Pengzhi Wei, Yujie Lin, Zhichao Hu, Dong Yu, Zhengyou Zhang, Jing Nie, Yuhong Liu
We construct a hierarchical task tree encompassing 7 major areas covering over 200 categories and over 800 tasks, which covers diverse capabilities such as question answering, reasoning, multiturn dialogue, and text generation, to evaluate LLMs in a comprehensive and in-depth manner.
no code implementations • SEMEVAL 2021 • Shuyi Xie, Jian Ma, Haiqin Yang, Lianxin Jiang, Yang Mo, Jianping Shen
Second, we construct a new vector on the fine-tuned embeddings from XLM-RoBERTa and feed it to a fully-connected network to output the probability of whether the target word in the context has the same meaning or not.
no code implementations • SEMEVAL 2021 • Jian Ma, Shuyi Xie, Haiqin Yang, Lianxin Jiang, Mengyuan Zhou, Xiaoyi Ruan, Yang Mo
This paper describes MagicPai's system for SemEval 2021 Task 7, HaHackathon: Detecting and Rating Humor and Offense.
no code implementations • SEMEVAL 2020 • Jian Ma, Shuyi Xie, Meizhi Jin, Jiang Lianxin, Mo Yang, Jianping Shen
In this paper, we only report our implement of Subtask 2.
no code implementations • SEMEVAL 2020 • Shuyi Xie, Jian Ma, Haiqin Yang, Jiang Lianxin, Mo Yang, Jianping Shen
The goal of this task is to extract definition, word level BIO tags and relations.