Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve

no code implementations • 24 Sep 2023 • R. Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, Thomas L. Griffiths

This approach - which we call the teleological approach - leads us to identify three factors that we hypothesize will influence LLM accuracy: the probability of the task to be performed, the probability of the target output, and the probability of the provided input.