1 code implementation • 9 May 2024 • Shiva Sreeram, Tsun-Hsuan Wang, Alaa Maalouf, Guy Rosman, Sertac Karaman, Daniela Rus
We provide a sober look at the application of Multimodal Large Language Models (MLLMs) within the domain of autonomous driving and challenge/verify some common assumptions, focusing on their ability to reason and interpret dynamic driving scenarios through sequences of images/frames in a closed-loop control environment.
no code implementations • 26 Oct 2023 • Tsun-Hsuan Wang, Alaa Maalouf, Wei Xiao, Yutong Ban, Alexander Amini, Guy Rosman, Sertac Karaman, Daniela Rus
As autonomous driving technology matures, end-to-end methodologies have emerged as a leading strategy, promising seamless integration from perception to control via deep learning.
no code implementations • 23 Apr 2023 • Noam Buckman, Sertac Karaman, Daniela Rus
Yet the overall impact on traffic flow for this new class of planners remain to be understood.
no code implementations • 21 Mar 2023 • Noam Buckman, Shiva Sreeram, Mathias Lechner, Yutong Ban, Ramin Hasani, Sertac Karaman, Daniela Rus
FailureNet observes the poses of vehicles as they approach an intersection and detects whether a failure is present in the autonomy stack, warning cross-traffic of potentially dangerous drivers.
no code implementations • 14 Dec 2022 • Charles Richter, Patrick R. Barragán, Sertac Karaman
We present a unified probabilistic model that learns a representative set of discrete vehicle actions and predicts the probability of each action given a particular scenario.
1 code implementation • 18 May 2022 • Ryan Sander, Wilko Schwarting, Tim Seyde, Igor Gilitschenski, Sertac Karaman, Daniela Rus
Experience replay plays a crucial role in improving the sample efficiency of deep reinforcement learning agents.
no code implementations • 23 Nov 2021 • Alexander Amini, Tsun-Hsuan Wang, Igor Gilitschenski, Wilko Schwarting, Zhijian Liu, Song Han, Sertac Karaman, Daniela Rus
Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios.
no code implementations • 23 Nov 2021 • Tsun-Hsuan Wang, Alexander Amini, Wilko Schwarting, Igor Gilitschenski, Sertac Karaman, Daniela Rus
Data-driven simulators promise high data-efficiency for driving policy learning.
1 code implementation • 1 Sep 2021 • Yi-Lun Liao, Sertac Karaman, Vivienne Sze
This naturally raises the question of how the performance of ViT can be advanced with design techniques of CNN.
no code implementations • 20 May 2021 • Zhijian Liu, Alexander Amini, Sibo Zhu, Sertac Karaman, Song Han, Daniela Rus
On the other hand, increasing the robustness of these systems is also critical; however, even estimating the model's uncertainty is very challenging due to the cost of sampling-based methods.
1 code implementation • 19 Mar 2021 • Murad Abu-Khalaf, Sertac Karaman, Daniela Rus
We propose a novel controller synthesis involving feedback from pixels, whereby the measurement is a high dimensional signal representing a pixelated image with Red-Green-Blue (RGB) values.
1 code implementation • 19 Feb 2021 • Wilko Schwarting, Tim Seyde, Igor Gilitschenski, Lucas Liebenwein, Ryan Sander, Sertac Karaman, Daniela Rus
We demonstrate the effectiveness of our algorithm in learning competitive behaviors on a novel multi-agent racing benchmark that requires planning from image observations.
no code implementations • 27 Oct 2020 • Tim Seyde, Wilko Schwarting, Sertac Karaman, Daniela Rus
Learning complex robot behaviors through interaction requires structured exploration.
no code implementations • L4DC 2020 • Tim Seyde, Wilko Schwarting, Sertac Karaman, Daniela Rus
Deep exploration requires coordinated long-term planning.
no code implementations • 28 May 2020 • Igor Spasojevic, Varun Murali, Sertac Karaman
The main contribution of this paper is an efficient time optimal path parametrization algorithm for quadrotors with limited field of view constraints.
1 code implementation • ICLR 2020 • Igor Gilitschenski, Roshni Sahoo, Wilko Schwarting, Alexander Amini, Sertac Karaman, Daniela Rus
Reasoning about uncertain orientations is one of the core problems in many perception tasks such as object pose estimation or motion estimation.
no code implementations • 14 Dec 2019 • Igor Gilitschenski, Guy Rosman, Arjun Gupta, Sertac Karaman, Daniela Rus
Our main contribution is the concept of learning context maps to improve the prediction task.
no code implementations • 24 Sep 2019 • Rajat Talak, Sertac Karaman, Eytan Modiano
Probability theory starts with a distribution function (equivalently a probability measure) as a primitive and builds all other useful concepts, such as law of total probability, Bayes' law, independence, graphical models, point estimate, on it.
1 code implementation • 29 Aug 2019 • Oscar Mickelin, Sertac Karaman
We describe a simple, black-box compression format for tensors with a multiscale structure.
Numerical Analysis Numerical Analysis 65F99, 15A69
6 code implementations • 27 May 2019 • Winter Guerra, Ezra Tal, Varun Murali, Gilhyun Ryou, Sertac Karaman
While a vehicle is in flight in the FlightGoggles virtual reality environment, exteroceptive sensors are rendered synthetically in real time while all complex extrinsic dynamics are generated organically through the natural interactions of the vehicle.
Robotics
1 code implementation • 8 Mar 2019 • Diana Wofk, Fangchang Ma, Tien-Ju Yang, Sertac Karaman, Vivienne Sze
In this paper, we address the problem of fast depth estimation on embedded systems.
1 code implementation • NeurIPS 2018 • Fangchang Ma, Ulas Ayaz, Sertac Karaman
In this work, we present new theoretical results on convolutional generative neural networks, in particular their invertibility (i. e., the recovery of input latent code given the network output).
no code implementations • 25 Nov 2018 • Alexander Amini, Guy Rosman, Sertac Karaman, Daniela Rus
We define a novel variational network capable of learning from raw camera data of the environment as well as higher level roadmaps to predict (1) a full probability distribution over the possible control commands; and (2) a deterministic control command capable of navigating on the route specified within the map.
7 code implementations • 3 Oct 2018 • Amado Antonini, Winter Guerra, Varun Murali, Thomas Sayre-McCord, Sertac Karaman
The Blackbird unmanned aerial vehicle (UAV) dataset is a large-scale, aggressive indoor flight dataset collected using a custom-built quadrotor platform for use in evaluation of agile perception. Inspired by the potential of future high-speed fully-autonomous drone racing, the Blackbird dataset contains over 10 hours of flight data from 168 flights over 17 flight trajectories and 5 environments at velocities up to $7. 0ms^-1$.
1 code implementation • 6 Jul 2018 • Oscar Mickelin, Sertac Karaman
Tensor decompositions such as the canonical format and the tensor train format have been widely utilized to reduce storage costs and operational complexities for high-dimensional data, achieving linear scaling with the input dimension instead of exponential scaling.
Numerical Analysis Numerical Analysis 65F99, 15A23, 15A69
2 code implementations • 1 Jul 2018 • Fangchang Ma, Guilherme Venturelli Cavalheiro, Sertac Karaman
Depth completion, the technique of estimating a dense depth image from sparse depth measurements, has a variety of applications in robotics and autonomous driving.
Ranked #6 on Depth Completion on VOID
no code implementations • 13 May 2018 • Alexander Amini, Ava Soleimany, Sertac Karaman, Daniela Rus
Dropout training in deep NNs approximates Bayesian inference in a deep Gaussian process and can thus be used to estimate model uncertainty.
no code implementations • 12 Feb 2018 • Xinxin Du, Marcelo H. Ang Jr., Sertac Karaman, Daniela Rus
Autonomous driving requires 3D perception of vehicles and other objects in the in environment.
Ranked #17 on 3D Object Detection on KITTI Cars Easy
6 code implementations • 21 Sep 2017 • Fangchang Ma, Sertac Karaman
We consider the problem of dense depth prediction from a sparse set of depth measurements and a single RGB image.
4 code implementations • 2 May 2017 • Corey Walsh, Sertac Karaman
A well-established localization approach combines ray casting with a particle filter, leading to a computationally expensive algorithm that is difficult to run on resource-constrained mobile robots.
Data Structures and Algorithms Robotics
1 code implementation • 4 Mar 2017 • Fangchang Ma, Luca Carlone, Ulas Ayaz, Sertac Karaman
We address the following question: is it possible to reconstruct the geometry of an unknown environment using sparse and incomplete depth measurements?
no code implementations • 26 Jul 2016 • David Miculescu, Sertac Karaman
In this paper, we propose a coordination control algorithm for this problem, assuming stochastic models for the arrival times of the vehicles.
4 code implementations • 5 May 2011 • Sertac Karaman, Emilio Frazzoli
The main contribution of the paper is the introduction of new algorithms, namely, PRM* and RRT*, which are provably asymptotically optimal, i. e., such that the cost of the returned solution converges almost surely to the optimum.
Robotics
3 code implementations • 3 May 2010 • Sertac Karaman, Emilio Frazzoli
Second, a new algorithm is considered, called the Rapidly-exploring Random Graph (RRG), and it is shown that the cost of the best path in the RRG converges to the optimum almost surely.
Robotics 68T40