Traces of Memorisation in Large Language Models for Code

1 code implementation • 18 Dec 2023 • Ali Al-Kaswan, Maliheh Izadi, Arie van Deursen

We find that large language models for code are vulnerable to data extraction attacks, like their natural language counterparts.

Code Completion

The (ab)use of Open Source Code to Train Large Language Models

1 code implementation • 27 Feb 2023 • Ali Al-Kaswan, Maliheh Izadi

In recent years, Large Language Models (LLMs) have gained significant popularity due to their ability to generate human-like text and their potential applications in various fields, such as Software Engineering.

Memorization

STACC: Code Comment Classification using SentenceTransformers

1 code implementation • 25 Feb 2023 • Ali Al-Kaswan, Maliheh Izadi, Arie van Deursen

Code comments are a key resource for information about software artefacts.

Classification

Targeted Attack on GPT-Neo for the SATML Language Model Data Extraction Challenge

no code implementations • 13 Feb 2023 • Ali Al-Kaswan, Maliheh Izadi, Arie van Deursen

In this work, we apply a targeted data extraction attack to the SATML2023 Language Model Training Data Extraction Challenge.

Inference Attack Language Modelling +2

Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries

1 code implementation • 4 Jan 2023 • Ali Al-Kaswan, Toufique Ahmed, Maliheh Izadi, Anand Ashok Sawant, Premkumar Devanbu, Arie van Deursen

While the automated summarisation of decompiled code can help Reverse Engineers understand and analyse binaries, current work mainly focuses on summarising source code, and no suitable dataset exists for this task.