Calibration and Correctness of Language Models for Code

no code implementations • 3 Feb 2024 • Claudio Spiess, David Gros, Kunal Suresh Pai, Michael Pradel, Md Rafiqul Islam Rabin, Amin Alipour, Susmit Jha, Prem Devanbu, Toufique Ahmed

Our contributions will lead to better-calibrated decision-making in the current use of code generated by language models, and offers a framework for future research to further improve calibration methods for generative models in Software Engineering.

Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)

no code implementations • 13 Apr 2023 • Toufique Ahmed, Kunal Suresh Pai, Premkumar Devanbu, Earl T. Barr

This approach improves performance in several different settings suggested by prior work, including for two different Large Language Models.

Code Summarization Information Retrieval +3