no code implementations • 3 May 2024 • Minh Duc Bui, Katharina von der Wense
Current natural language processing (NLP) research tends to focus on only one or, less frequently, two dimensions - e. g., performance, privacy, fairness, or efficiency - at a time, which may lead to suboptimal conclusions and often overlooking the broader goal of achieving trustworthy NLP.
no code implementations • 30 Apr 2024 • Minh Duc Bui, Fabian David Schmidt, Goran Glavaš, Katharina von der Wense
We further find that KD yields larger gains over pretraining from scratch when the data must be repeated under the fixed computation budget.