no code implementations • 25 Sep 2023 • Kei Ota, Devesh K. Jha, Krishna Murthy Jatavallabhula, Asako Kanezaki, Joshua B. Tenenbaum
In particular, we estimate the contact patch between a grasped object and its environment using force and tactile observations to estimate the stability of the object during a contact formation.
no code implementations • 27 Jun 2023 • Chiori Hori, Puyuan Peng, David Harwath, Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh Jha, Diego Romeres, Jonathan Le Roux
This paper introduces a method for robot action sequence generation from instruction videos using (1) an audio-visual Transformer that converts audio-visual features and instruction speech to a sequence of robot actions called dynamic movement primitives (DMPs) and (2) style-transfer-based training that employs multi-task learning with video captioning and weakly-supervised learning with a semantic classifier to exploit unpaired video-action data.
no code implementations • 10 Mar 2023 • Kei Ota, Devesh K. Jha, Hsiao-Yu Tung, Joshua B. Tenenbaum
We evaluate our method on several part-mating tasks with novel objects using a robot equipped with a vision-based tactile sensor.
no code implementations • 22 Oct 2022 • Kei Ota, Hsiao-Yu Tung, Kevin A. Smith, Anoop Cherian, Tim K. Marks, Alan Sullivan, Asako Kanezaki, Joshua B. Tenenbaum
The world is filled with articulated objects that are difficult to determine how to use from vision alone, e. g., a door might open inwards or outwards.
no code implementations • 24 Mar 2022 • Rui Fukushima, Kei Ota, Asako Kanezaki, Yoko SASAKI, Yusuke Yoshiyasu
This paper presents a reinforcement learning method for object goal navigation (ObjNav) where an agent navigates in 3D indoor environments to reach a target object based on long-term observations of objects and scenes.
1 code implementation • 9 Sep 2021 • Hana Hoshino, Kei Ota, Asako Kanezaki, Rio Yokota
Inverse Reinforcement Learning (IRL) is attractive in scenarios where reward engineering can be tedious.
no code implementations • 16 Feb 2021 • Kei Ota, Devesh K. Jha, Asako Kanezaki
Previous work has shown that this is mostly due to instability during training of deep RL agents when using larger networks.
no code implementations • 14 Nov 2020 • Kei Ota, Devesh K. Jha, Diego Romeres, Jeroen van Baar, Kevin A. Smith, Takayuki Semitsu, Tomoaki Oiki, Alan Sullivan, Daniel Nikovski, Joshua B. Tenenbaum
The physics engine augmented with the residual model is then used to control the marble in the maze environment using a model-predictive feedback over a receding horizon.
no code implementations • 31 Oct 2020 • Kei Ota, Devesh K. Jha, Tadashi Onishi, Asako Kanezaki, Yusuke Yoshiyasu, Yoko SASAKI, Toshisada Mariyama, Daniel Nikovski
The main novelty of the proposed approach is that it allows a robot to learn an end-to-end policy which can adapt to changes in the environment during execution.
1 code implementation • ICML 2020 • Kei Ota, Tomoaki Oiki, Devesh K. Jha, Toshisada Mariyama, Daniel Nikovski
We believe that stronger feature propagation together with larger networks (and thus larger search space) allows RL agents to learn more complex functions of states and thus improves the sample efficiency.
no code implementations • 3 Mar 2020 • Kei Ota, Yoko SASAKI, Devesh K. Jha, Yusuke Yoshiyasu, Asako Kanezaki
Specifically, we train a deep convolutional network that can predict collision-free paths based on a map of the environment-- this is then used by a reinforcement learning algorithm to learn to closely follow the path.
no code implementations • 13 Mar 2019 • Kei Ota, Devesh K. Jha, Tomoaki Oiki, Mamoru Miura, Takashi Nammoto, Daniel Nikovski, Toshisada Mariyama
Our experiments show that our RL agent trained with a reference path outperformed a model-free PID controller of the type commonly used on many robotic platforms for trajectory tracking.