no code implementations • 18 May 2024 • Pai Zeng, Zhenyu Ning, Jieru Zhao, Weihao Cui, Mengwei Xu, Liwei Guo, Xusheng Chen, Yizhou Shan
We survey the large language model (LLM) serving area to understand the intricate dynamics between cost-efficiency and accuracy, which is magnified by the growing need for longer contextual understanding when deploying models at a massive scale.
no code implementations • 28 Aug 2023 • Rongjie Yi, Liwei Guo, Shiyun Wei, Ao Zhou, Shangguang Wang, Mengwei Xu
Large Language Models (LLMs) such as GPTs and LLaMa have ushered in a revolution in machine intelligence, owing to their exceptional capabilities in a wide range of machine learning tasks.
no code implementations • 11 Jul 2022 • Liwei Guo, Wonkyo Choe, Felix Xiaozhu Lin
Yet, the unprecedented size of an NLP model stresses both latency and memory, creating a tension between the two key resources of a mobile device.
no code implementations • 17 Feb 2019 • Liwei Guo, Yiying Zhang, Felix Xiaozhu Lin
To safeguard such data on smart devices, we present a novel storage stack architecture that i) protects file data in a trusted execution environment (TEE); ii) outsources file system logic and metadata out of TEE; iii) running a metadata-only file system replica in the cloud for continuously verifying the on-device file system behaviors.
Cryptography and Security