1 code implementation • 20 Feb 2024 • Che Zhang, Zhenyang Xiao, Chengcheng Han, Yixin Lian, Yuejian Fang
After integrating the original CoT data and checking-correction data for training, we observe that models could improve their self-checking capabilities, thereby enhancing their self-correction capacity and eliminating the need for external feedback or ground truth labels to ascertain the endpoint of correction.
no code implementations • 16 Jan 2024 • Yumeng Wang, Zhenyang Xiao
We introduce Lossless Compressed Memory Attention (LoMA), a novel approach that enables lossless compression of the KV cache, thereby reducing the memory and computational demands during autoregressive generation.
1 code implementation • 11 Jul 2023 • Che Zhang, Ping'an Liu, Zhenyang Xiao, Haojun Fei
The study of human values is essential in both practical and theoretical domains.