NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Prompts

no code implementations • 7 May 2024 • Shudan Zhang, Hanlin Zhao, Xiao Liu, Qinkai Zheng, Zehan Qi, Xiaotao Gu, Xiaohan Zhang, Yuxiao Dong, Jie Tang

To fill this gap, we propose NaturalCodeBench (NCB), a challenging code benchmark designed to mirror the complexity and variety of scenarios in real coding tasks.

Knowledge Conflicts for LLMs: A Survey

no code implementations • 13 Mar 2024 • Rongwu Xu, Zehan Qi, Cunxiang Wang, Hongru Wang, Yue Zhang, Wei Xu

This survey provides an in-depth analysis of knowledge conflicts for large language models (LLMs), highlighting the complex challenges they encounter when blending contextual and parametric knowledge.

Misinformation

Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity

1 code implementation • 11 Oct 2023 • Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, Yidong Wang, Linyi Yang, Jindong Wang, Xing Xie, Zheng Zhang, Yue Zhang

This survey addresses the crucial issue of factuality in Large Language Models (LLMs).

Retrieval Specificity