no code implementations • 27 Jun 2023 • Chiori Hori, Puyuan Peng, David Harwath, Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh Jha, Diego Romeres, Jonathan Le Roux
This paper introduces a method for robot action sequence generation from instruction videos using (1) an audio-visual Transformer that converts audio-visual features and instruction speech to a sequence of robot actions called dynamic movement primitives (DMPs) and (2) style-transfer-based training that employs multi-task learning with video captioning and weakly-supervised learning with a semantic classifier to exploit unpaired video-action data.
no code implementations • 21 Mar 2022 • Eric Rosen, Sreehari Rammohan, Devesh Jha
Mixed Reality (MR) has recently shown great success as an intuitive interface for enabling end-users to teach robots.
no code implementations • 26 Dec 2019 • Devesh Jha, Arvind Raghunathan, Diego Romeres
The algorithm has achieved state-of-the-art performance when used in reinforcement learning across a wide range of tasks.
2 code implementations • 23 Oct 2019 • Jonathan Chang, Nishanth Kumar, Sean Hastings, Aaron Gokaslan, Diego Romeres, Devesh Jha, Daniel Nikovski, George Konidaris, Stefanie Tellex
We demonstrate that our model trained on 33% of the possible goals is able to generalize to more than 90% of the targets in the scene for both simulation and robot experiments.
no code implementations • 13 Sep 2018 • Diego Romeres, Devesh Jha, Alberto Dalla Libera, William Yerazunis, Daniel Nikovski
We propose the system presented in the paper as a benchmark problem for reinforcement and robot learning, for its interesting and challenging dynamics and its relative ease of reproducibility.
no code implementations • 13 Sep 2018 • Jeroen van Baar, Alan Sullivan, Radu Cordorel, Devesh Jha, Diego Romeres, Daniel Nikovski
Another advantage when robots are involved, is that the amount of time a robot is occupied learning a task---rather than being productive---can be reduced by transferring the learned task to the real robot.