VEHICLE DRIVER CHARACTERIZATION USING TIME DISTRIBUTED DEEP NETWORKS

Abstract

Every year, thousands of accidents can be attributed to drivers who were either distracted or too tired to pay attention to the road. Observing drivers in their natural environment makes it hard to study how they act. The naturalistic driving study (NDS) has become the most popular method because it let researchers see how drivers act in the real world and help them drive safer by detecting various driver behaviours. In the studies, different types of data are used, such as the driver’s physical condition, audio and visual features, and information about the car. However, the main source of data is images of the driver’s face, and hands on the steering wheel taken by a camera inside the car. In order to increase the system’s capacity for generalisation, it is suggested in this work that sensor data be incorporated into the vision-based distracted driver detection model. In order to achieve this, a new data set was generated that consists of driver face images, road images, and sensor data obtained from the Carla simulator. Later, a model was developed to identify four distracted driving behaviours such as aggressive, distracted by phone, drinking, and normal driving. In the proposed system, CNN extracts the visual characteristics and LSTM processes the temporal data. The collected data is evaluated using a CNN-LSTM model. The performance of the model is improved by combining the sensor data with image data. Driver behaviour detection model using both sensor data and face images achieved a classification accuracy of 97%.