no code implementations • 18 Feb 2024 • Benedict Quartey, Eric Rosen, Stefanie Tellex, George Konidaris
We propose Language Instruction grounding for Motion Planning (LIMP), a system that leverages foundation models and temporal logics to generate instruction-conditioned semantic maps that enable robots to verifiably follow expressive and long-horizon instructions with open vocabulary referents and complex spatiotemporal constraints.
no code implementations • 13 Sep 2023 • Thao Nguyen, Vladislav Hrosinkov, Eric Rosen, Stefanie Tellex
In this work, we bridge the gap in realistic object search by posing the search problem as a partially observable Markov decision process (POMDP) where the object detector and visual sensor noise in the observation model is determined by a single Deep Neural Network conditioned on complex language descriptions.
no code implementations • 18 May 2023 • Eric Rosen, Devesh K. Jha
We address the problem of teleoperating an industrial robot manipulator via a commercially available Virtual Reality (VR) interface.
no code implementations • 17 Nov 2022 • Shreyas Sundara Raman, Vanya Cohen, Ifrah Idrees, Eric Rosen, Ray Mooney, Stefanie Tellex, David Paulius
Our improvements transfer to a Boston Dynamics Spot robot initialized with a set of skills (specified in language) and associated preconditions, where CAPE improves the correctness metric of the executed task plans by 76. 49% compared to SayCan.
no code implementations • MTSummit 2021 • Paul Soulos, Sudha Rao, Caitlin Smith, Eric Rosen, Asli Celikyilmaz, R. Thomas McCoy, Yichen Jiang, Coleman Haley, Roland Fernandez, Hamid Palangi, Jianfeng Gao, Paul Smolensky
Machine translation has seen rapid progress with the advent of Transformer-based models.
no code implementations • 16 Jul 2022 • Eric Rosen, Doruk Senkal
In this paper, we report a hierarchical deep learning model for classification of complex human activities using motion sensors.
no code implementations • 21 Mar 2022 • Eric Rosen, Sreehari Rammohan, Devesh Jha
Mixed Reality (MR) has recently shown great success as an intuitive interface for enabling end-users to teach robots.
1 code implementation • 24 Oct 2021 • Matthias Lalisse, Eric Rosen, Paul Smolensky
We present Harmonic Memory Networks (HMem), a neural architecture for knowledge base completion that models entities as weighted sums of pairwise bindings between an entity's neighbors and corresponding relations.
no code implementations • 28 Jul 2021 • Sreehari Rammohan, Shangqun Yu, Bowen He, Eric Hsiung, Eric Rosen, Stefanie Tellex, George Konidaris
Learning continuous control in high-dimensional sparse reward settings, such as robotic manipulation, is a challenging problem due to the number of samples often required to obtain accurate optimal value and policy estimates.
1 code implementation • 19 May 2021 • Jacob Russin, Roland Fernandez, Hamid Palangi, Eric Rosen, Nebojsa Jojic, Paul Smolensky, Jianfeng Gao
A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition.
no code implementations • 12 Jan 2021 • Ben Abbatematteo, Eric Rosen, Stefanie Tellex, George Konidaris
We propose using kinematic motion planning as a completely autonomous, sample efficient way to bootstrap motor skill learning for object manipulation.
no code implementations • 13 Mar 2017 • John Goldsmith, Eric Rosen
We explore inflectional morphology as an example of the relationship of the discrete and the continuous in linguistics.