Computer vision applied to assisted and self driving

PhD course by Douglas Coimbra de Andrade, D. Sc. , principal data scientist at Verizon Connect Italy

Computer vision and artificial intelligence have become pervasive in many fields, generating state-of-the-art results in many applications and improving multiple applications.
One of those applications is driver assistance and self driving, which can save many lives by preventing accidents and make traffic more fluid, among other advantages.
This course will provide an overview of computer vision tools currently used as perception sensors in driver assistance systems and self-driving vehicles. The course outline is as follows:

1. Introduction

1.1 What is the task?
1.2 Road driver-assistance systems
1.3 Driver monitoring systems
1.4 Modular and end to end driving systems
1.5 Basic V2V (vehicle to vehicle) and V2X (vehicle to anything) concepts

2. Sensors overview

2.1 Cameras (single and arrays)
2.2 Proprioceptive sensors (MEMS IMU, GPS, encoders)
2.3 IR, thermal, omnidirectional, multispectral and other cameras
2.4 Radar and Lidar

3. Neural network review

3.1 Convolutional networks
3.2 Attention based networks (e.g Vision transformer (ViT))
3.3 Model training pipeline
3.3.1 Data preparation
3.3.2 Data augmentation
3.3.3 Model input, output and backbone
3.3.4 Losses and metrics

4. Driver monitoring tasks

4.1 Face detection
4.2 Features from faces
4.3 Face ID, FaceNet and the Triplet loss
4.4 Action recognition

5. Road perception

5.1 Road object recognition
5.2 Tracking and track-by-detection
5.3 Detection of road elements
5.4 Architecture design

* Note: Control strategies and algorithms are out of the scope. This would include: dynamic response of actuators; feedback loops; PID and optimal control; reinforcement learning controllers.

References (more to be added):