Abstract:
Every year, thousands of accidents can be attributed to drivers who were either distracted
or too tired to pay attention to the road. Observing drivers in their natural environment
makes it hard to study how they act. The naturalistic driving study (NDS) has become the
most popular method because it let researchers see how drivers act in the real world and help
them drive safer by detecting various driver behaviours. In the studies, different types of data
are used, such as the driver’s physical condition, audio and visual features, and information
about the car. However, the main source of data is images of the driver’s face, and hands
on the steering wheel taken by a camera inside the car. In order to increase the system’s
capacity for generalisation, it is suggested in this work that sensor data be incorporated into
the vision-based distracted driver detection model. In order to achieve this, a new data set
was generated that consists of driver face images, road images, and sensor data obtained
from the Carla simulator. Later, a model was developed to identify four distracted driving
behaviours such as aggressive, distracted by phone, drinking, and normal driving. In the
proposed system, CNN extracts the visual characteristics and LSTM processes the temporal
data. The collected data is evaluated using a CNN-LSTM model. The performance of the
model is improved by combining the sensor data with image data. Driver behaviour detection
model using both sensor data and face images achieved a classification accuracy of 97%.
