Search Results for author:
Found 5 papers, 2 papers with code
Demonstrating EMMA: Embodied MultiModal Agent for Language-guided Action Execution in 3D Simulated Environments
We demonstrate EMMA, an embodied multimodal agent which has been developed for the Alexa Prize SimBot challenge.
Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks
Augmenting Large Language Models (LLMs) with image-understanding capabilities has resulted in a boom of high-performing Vision-Language models (VLMs).
Multitask Multimodal Prompted Training for Interactive Embodied Task Completion
Interactive and embodied tasks pose at least two fundamental challenges to existing Vision & Language (VL) models, including 1) grounding language in trajectories of actions and observations, and 2) referential disambiguation.
Quality-agnostic Image Captioning to Safely Assist People with Vision Impairment
In this paper, we propose a quality-agnostic framework to improve the performance and robustness of image captioning models for visually impaired people.
Going for GOAL: A Resource for Grounded Football Commentaries
As the course of a game is unpredictable, so are commentaries, which makes them a unique resource to investigate dynamic language grounding.
