_{DDIVA 2021}

Modern supervised machine learning models require large amounts of labelled data for timely convergence during training and good generalization during testing. Thus, labelling large amounts of data has become an inevitable bottleneck in the model production pipeline. In this talk, I will present techniques and workflows that could considerably reduce the labelling time and/or effort, and in some cases remove the need to label altogether. To further illustrate this, I will present three case studies where these ideas have been put to use in practice, and discuss the resulting outcomes of our approach. In particular, this talk will cover concepts such as - simultaneous data and label capture, automatic labelling using auxiliary sensors, and task-specific data augmentation schemes. These techniques are meant to be for general-use, and could be applied to or adapted for tasks beyond the ones covered in this talk.

Scene understanding and object tracking play a critical role in enabling autonomous vehicles to navigate in dense urban environments. The last decade has witnessed unprecedented progress in these tasks by exploiting learning techniques to improve performance and robustness. Despite these advances, most techniques today still require a tremendous amount of human-annotated training data and do not generalize well to diverse real-world environments. In this talk, I will discuss some of our efforts targeted at addressing these challenges by leveraging self-supervision and multi-task learning for various tasks ranging from panoptic segmentation to cross-modal object tracking using sound. Finally, I will conclude the talk with a discussion on opportunities for further scaling up the learning of these tasks.

Most research on data-driven perception in Intelligent Vehicles focuses on camera and lidar perception tasks, including object detection and scene segmentation. In this talk, I will present our group's research on data-driven methods for other sensing modalities that traditionally rely on signal processing (radar), or are even completely ignored in current IV research (acoustics).
First, I will demonstrate how the low-level radar data, represented as a 3D azimuth-range-Doppler tensor, provides valuable information to classify radar targets before clustering them for multi-class object detection [Palffy,RA-L’20]. We train our low-level radar detection network using annotations obtained from a visual detector on synchronized radar and camera recordings with our group's research vehicle. Extrinsic multi-modal sensor calibration for our vehicle will also be shortly discussed [Domhof,T-IV’21].
Second, I present novel research [Schulz,RA-L’21] that shows how a vehicle-mounted microphone array can be used to detect occluded traffic around corners 1 second before the camera-based detection system can. For these experiments we collected a new dataset with our research vehicle at various narrow alleys in the Delft inner city.
Overall, we seek to exploit easily collected multi-modal sensor data. Still, as we lack truly huge datasets and annotations needed for end-to-end learning, we rely on a combination of data-driven learning and classic signal processing techniques.

[Palffy,RA-L’20]: “CNN based Road User Detection using the 3D Radar Cube”, A. Palffy et al., IEEE Robotics and Automation Letters (RA-L), 2020
[Domhof,T-IV’21]: “A Joint Extrinsic Calibration Tool for Radar, Camera and Lidar”, J. Domhof et al., IEEE Trans. on Intelligent Vehicles (T-IV), 2021
[Schulz,RA-L’21]: “Hearing What You Cannot See: Acoustic Vehicle Detection Around Corners”, Y. Schulz et al., IEEE Robotics and Automation Letters (RA-L), 2021

Automated vehicles (AVs) are gaining increasing interest and their development is making great progress. However, assuring safe driving operation under all possible road and environment conditions is still an open challenge. In this regard, vehicle simulation is seen as a major corner stone for test and validation. Recorded real world challenges can be rebuild in simulation (e.g. NHTSA pre-crash scenarios) and in addition artificial corner cases can be added on top. Those can then be utilized to test the complex software stack in various configurations to find possible safety or availability issues. For example, the same situation can be tested with different settings of the planning modules (driving policy) or road conditions to ensure that all possibilities result in safe driving. In this talk, we will present how the CARLA vehicle simulator in combination with an open source scenario editor can be used to re-create traffic scenarios, play those under various operating conditions and by that means make one step towards safe autonomous driving.

One of the most important efforts by Prof. Kazuya Takeda has been focused in the field of signal processing technology research, in particular, understanding human behavior through data centric approaches. Faithful to that tradition, in this talk, we will introduce our recent data-driven works in driving behavior at Takeda Laboratory. 
In the first part, we will discuss generation of personalized lane change maneuvers based on subjective risk modeling to analyze differences in individual driving behavior, based on drivers' subjective perception of risk. In the second part, we will cover extraction of driving behaviors from human experts; latent features are clustered into behaviors used to create different velocity profiles, allowing an autonomous driving agent to drive in a human-like manner.  In the last part, we will explain a graph convolutional network approach for predicting potential semantic relationships from object proposals; relationship data provides a human-kind description of the objects' behavior. 

Developing reinforcement learning (RL) algorithms for automated tactical decision making has been an attractive topic in recent years. It is evident that designing RL based autonomous driving systems can help tremendously with handling performance and safety-based issues of alternative planning and decision making approaches. That being said, most RL algorithms are trained in simulators, and require large-amounts of data to converge to good solutions. Thus, designing sample-efficient RL algorithms is important for accelerating design cycles and verifying the safety and robustness of RL solutions. In addition, good performance in simulation does not always imply good performance in real life tests. Thus, additional measure needs to be taken to guarantee that trained RL agents generalize to real-life situations. In this talk, we go over three case studies that deals with these issues; i) how to utilize curriculum RL to boost your autonomous driving agent’s performance using limited amount of data; ii) how to take advantage of offline RL to inject external driving demonstrations for improving sample efficiency, iii) how to use multiple fidelity simulators to transfer simulated performance to real-life.