Journal of Information Technology & Software Engineering
Open Access

Driver Drowsiness Detection System

S. Barathvaj^*, B. Ashwanth, M. Shobana and L. Jabasheela ^*Correspondence: S. Barathvaj, Department of Computer Science and Engineering, Panimalar Engineering College, Chennai, India, Email:

Author info »

Introduction

The driver drowsiness detection system is a state-of-the-art solution that contributes to the enhancement of road safety by dealing with one of the highly contributing causes of accidents, namely driver fatigue. It uses sophisticated techniques of computer vision and machine learning to monitor facial features of the driver, mainly eye movements, in real time. The system makes use of OpenCV for video processing and dlib to detect important facial landmarks and it derives the EAR with the help of these facial landmarks. A decreased EAR corresponds to drowsiness and hence the system automatically presents prompt audio warnings through a text-to-speech engine. This solution can be either in the form of an app-based system designed for personal, smartphone-based applications or as part of commercial fleets as a vehicle system. Its inherently cost-effective design of reusing existing hardware makes it not only accessible but also scalable. A driver drowsiness detection system goes a long way in lowering the rate of accidents caused by drowsiness with this timely feedback to the drivers and keeps them alert thus promoting safer roadways for all users.

Literature Review

Singh et al. had explored the EAR for the detection of driver drowsiness with much reliability. Their approach had been to understand the correlation of eye closure with drowsiness, wherein continuous reduction in the values of EAR showed that drowsiness was setting in. This study utilized facial landmark detection, which culminates in real-time performance with the collaboration of OpenCV and Dlib. It demonstrated the efficiency of EAR in capturing eye behavior and how it may provide early warnings of fatigue [1,2]. Gupta, et al., presented an SVM-based drowsiness detection system using machine learning. The model, which trained on an eye-closure rate and head movement, also claimed high accuracy for the recognition of alert and drowsy states. This led to video data processing through machine learning integration into the system, hence increased accuracy in the detection process and reduced false positives in the complex driving environment [3]. Kumar, et al., emphasized the use of non-intrusive alert systems for drowsy drivers. A text-to-speech alert by pyttsx3 was implemented in order to give an audio alert without distracting the attention of the driver. This alerted the driver in time and prevented accidents caused by inattention due to their detected drowsiness.

Audio alerts are better compared with visual alerts for avoiding inattention [4], Patel, et al. proposed a real-time video processing system for in-vehicle driver monitoring based on OpenCV. This paper discussed the problem of high computational intensity while processing video feed directly and suggested optimizations to speed up facial detection. The authors made use of CPU and GPU resources for prompt video analysis while maintaining smooth-running designs to give timely alerts in case of drowsiness [5]. Verma, et al., came up with a driver monitoring system by means of cameras in mobiles and integrated machine learning algorithms in smartphones. The developed system was flexible as people could download an application on their mobile phones, install it and subsequently track features in real time. The focus of study is accessibility of the system and possible increased numbers by exploiting already existing hardware of mobile phones for detecting drowsiness [6]. In 2021, Jha et al., have come up with a hybrid model wherein facial feature analysis and steering pattern monitoring are combined. The model utilized computer vision for eye closure detection, while TensorFlow was used for the study of vehicle steering patterns. By including multiple data sources, the drowsiness detection system became much more reliable even when detecting over long driving sessions on highways. This approach minimizes false positives based on several indicators of driver fatigue.

Existing system

The current systems on driver drowsiness monitoring include technologies under three heads: Physiological monitoring, behavioral analysis and vehicle-based monitoring. Systems under the head of physiological monitoring utilize wearable technology such as smartwatches and fitness bands that monitor heart rate and skin temperature signals. They can be quite good at measuring indicators of driver drowsiness but prove invasive and interfere with the comfort of users as they continually ask for contact with the driver. In camera-based behavioral analysis, the study primarily relates to facial aspects and eye behavior through the use of cameras. An example would be infrared-based eye monitoring systems like seeing machines, which monitor the movement of the eyes and metrics such as EAR are measured to determine when drowsiness is occurring. Advanced algorithms in computer vision are being applied for the assessment of driver attention based on his eye movement patterns by smart eye.

System types: The instant feedback is obtained, but it is expensive and involves substantial installation in a vehicle. In turn, the vehicle-based monitoring system has indirect monitoring of driver behavior depending on the dynamics of the vehicle. The lane departure systems warn a driver when his or her vehicle leaves their lane, whereas pattern analysis of steering reveals wild movements in steering. These solutions, however are reactive in nature and can only detect fatigue after it would have begun to affect driving performance. Although tremendous advancement has been made in these systems, problems persist, such as invasiveness and cost besides timely detection of fatigue to promote general road safety [7].

Proposed system

An advanced driver drowsiness detection system has recently been proposed to utilize better computer vision and machine learning for the purpose of improving road safety through the real-time detection of driver alertness. It captures live video feeds either with a webcam or with a camera in a smartphone. It uses facial detection algorithms, mainly relying on pre-trained dlib models provided on the task of facial landmark identification. The eyes of the driver close on and for how long with the help of EAR, which determines such by analyzing. Furthermore, the drowsiness will be caught better under adaptive learning in changing environmental conditions consisting of lighting and driver posture with the aid of the system proposed which also integrates threshold-based logic with machine learning classifiers. In the event that drowsiness is detected by the system, then it will activate not only visual alerts on the screen but also audio alerts through text-to-speech technology to alert the driver and urge the appropriate action like resting or stopping the vehicle. Apart from the above, the proposed system will host a human-friendly configuration and monitoring interface for the drivers to set specific alert schemes according to their preference. Other features of data logging will track instances of detected drowsiness to allow analysis of driving patterns over time. In general, it considers not merely alerting the driver in real time but adding to an integral approach toward driver safety by constituting the application of machine learning techniques in the practical implementations of the daily context of driving [8].

Architecture

The driver drowsiness detection system is composed of a multistep process that combines computer vision and machine learning techniques. Starting with a live video feed originating from the camera, capturing in real time the driver's face, the architecture succeeds in processing the frames of a captured video through OpenCV for converting the given image into grayscale. Bringing much efficiency into facial landmark detection, it converts the image into grayscale. The developed facial landmark model by Dlib identifies major features of the face primarily the eyes. After it has identified the eye landmark, the system calculates the Eye Aspect Ratio (EAR) and looks for the event of unresponsiveness such as a long time closure of eyes or slow eye blinking patterns. Whenever the value of EAR stays low for a certain period of time, an alert warning audio is sent via the library pyttsx3 to remind the driver to stay alert. The realtime feedback loop prevents accidents that can be caused due to driver fatigue. It can be designed to expand for various platforms, like in-vehicle systems and even mobile devices, thus making it flexible and affordable. Further expansion can also be done using other modules, such as steering pattern analysis using TensorFlow, for a more complete mechanism of drowsiness detection.

Through real-time analysis and alert features, this architecture is a proactive safety solution to minimize road accidents due to drowsy driving. Your architecture of drowsiness detection application will have several modules collaborating with one another in real time to detect and alert a user. First of all, the Input Module captures live video feed from the camera using OpenCV. Video feed is processed as a stream of frames. Each frame that had passed through Preprocessing Module was first transformed from its natural BGR coloring format to a grayscale version with cv2.cvtColor() for performance acceleration of the facial landmark detection algorithm by a substantial value. Having passed through preprocessing module in the grayscale format, the frame undergoes facial landmark detection module. It employs the dlib library to identify facial landmarks and makes predictions about 68 unique points, of which the ones surrounding the eyes are of prime interest. Following the settlement of landmarks, this system proceeds with extracting the coordinates for the regions of the eyes. To provide real-time feedback, the visualization and display module employs Matplotlib for the presentation of the video frames which are processed with over layed facial landmarks and alerts. Hence, through this module, it ensures that the status of the system is observable to the user [9].

Lastly, the control flow/exit condition governs the overall flow of the system. It continuously checks for user inputs-his or her intention to quit using the system via key press of a particular key for example 'q'. This allows it to timely free the camera feed and close up the display. This modular approach makes for both efficient and scalable system performance, means additional features, perhaps more sophisticated alert mechanisms or performance optimizations can be added on in later stages (Figure 1) [10].

Figure 1: Architecture diagram for driver drowsiness detection system.

Methodology: If we explain the general architecture of the model you will observe that it is very easy to operate this model because here only. We are to capture the video of the face of the driver in the camera so it will measure the scoring of blinking of the eyes and beep the alarm accordingly.

Creation of database: The database is created for the attendance system in which it consists the unique features of the detected images and it will also consist the information about the enrolled users and the record of the attendance.

System components: A drowsiness detection system by the driver, therefore, involves a number of key components that interconnect to track the condition of the driver and issue realtime alarms. These can be hence divided into hardware and software elements:

Camera (input device): The camera is considered one of the most vital parts of the driver drowsiness detection system as this video feed that it captures shows live pictures of drivers' faces, especially focusing on the eyes and facial features.

The processing unit is the central component of the driver drowsiness detection system. It works in real-time to analyze the real-time video feed taken by the camera. It performs machine learning algorithms and computer vision techniques, such as OpenCV and Dlib, on facial landmark detection, in order to calculate the EAR. It processes vast amounts of data within seconds to identify eye movements and facial expressions to determine drowsiness signs. It would most likely be an in-car computer or merely the processor of a smartphone, depending on the setting of the system. The processing unit will guarantee there is real-time monitoring with fluidity and efficiency with an alert to indicate sleepiness. Its velocity and precision will determine whether the processing unit will produce efficiency regarding the time for detection and response so that risks with driver's fatigue are reduced [11].

Computer Vision module (OpenCV): An element of the driver drowsiness detection system is the computer vision module, an OpenCV-powered component. It performs real-time processing and analyzing of video frames from the camera. OpenCV helps the system convert captured frames into a grayscale image, which would be easier to analyze for facial feature detection. This module, together with dlib, detects and tracks some of the critical facial points including eyes and mouth and head position. The primary goal of this module developed in OpenCV is to track the driver's eye movements and facial expressions by mainly calculating the Eye Aspect Ratio (EAR) to identify the indication of drowsiness. The OpenCV module is highly efficient as well as processing-fastly; hence frame processing is done in real time for its immediate detection and response. Due to the fact of the nature of robustness in image processing, it is inevitable for detection of even very minor changes in the behavior of eyes that indicate visual fatigue [12].

Eye Aspect Ratio (EAR) calculation: The system computes the eye aspect ratio, EAR, from the detected eye landmarks. Ear is the mathematical representation of the openness of the eye. When the EAR becomes low, the eyes close, indicating sleep.

EAR=(Visual distance 1+Visual distance 2)/(2 × Horizontal distance)

Drowsiness detection: In case the EAR value goes below a threshold predefined for some frames, the system identifies the drowsiness of a driver. Continuous monitoring will help pattern the drivers' progression into the fatigue levels based upon the patterns, such as an extended eye closure or slow blinks.

Alert mechanism: When sleepiness is detected, the system provides an audio alert utilizing the pyttsx3 text-to-speech engine. The alert is in attention to the driver without any distraction from the road.

Real-time monitoring: The system processes and analyzes video frames continuously in real time, providing immediate feedback to the driver. This is crucial for early detection and preventing accidents.

Software requirements

Python: Developed in Python, makes use of many key libraries.

OpenCV: For real time video capture, image processing and frame conversion.

Dlib: Detects facial structures and picks up face landmarks, with the particular focus on the eyes.

Pyttsx3: This is a library used for converting text to speech for voice alert of the driver in case of drowsiness.

Scipy: Computes the Euclidean distance between the eye landmarks to calculate the Eye Aspect Ratio.

Matplotlib: Render the frame with landmarks for testing and debugging.

Algorithm

• Import libraries: Import cv2, dlib and matplotlib.pyplot.
• Load models: Initialize face detector and landmark predictor with dlib.
• Capture video: Capture the video from the default camera using cv2.VideoCapture(0).
• Set up display: Enable interactive mode for Matplotlib and create a figure and axes for displaying frames.
• Main loop start: Starts an infinite loop for continuous capturing of video frames.
• Read frame: Captures the next frame with cap.read().
• Validate the frame is captured; print error message and break the loop
• Convert the frame to grayscale as the face capture will be good.
• Detecting faces: Detect faces in the grayscale image
• Each face processed: Loop each detected face to predict facial landmarks
• Draw the Landmark by using a circle at each landmark position. Where the color of the landmark must be special for eye landmark.
• Change frame: Change the frame to BGR to RGB for display purposes
• Update display: Clear the axis, then display the current frame, pausing a little for updates.
• Check for exit key: Monitor a 'q' key press to break the loop.
• Release resources: After the loop ends, free the video capture and close the display.
• Initialise libraries: Import the same set of libraries again for the remainder of the code:.
• Set-up camera: Provide camera setup for video capture.
• Define eye aspect ratio function: This function calculates EAR using landmark points.
• Thresholds and counters: Threshold of eye closure is defined; counters are defined to represent eye status.
• Main loop begin: Provided another infinite loop to grab video frames
• Read frame: The next frame read cap.read().
• Verify frame validity: If the frame was not captured, it prints an error and breaks the loop.
• Face detection: Just like part 1, it detects faces in the grayscale image.
• Computes EAR: For each face it detects, it computes the EAR for both eyes and averages them out.
• Check for drowsiness: If the EAR falls below a certain threshold for n successive frames, it shows drowsiness alerts and updates the screen

Discussion

The use case diagram of the system that detects driver drowsiness visually describes the interactions between a system and its users, mainly a driver and optionally a system administrator. Monitor driver, which captures video frames perpetually to monitor facial expressions and eye movements; detect drowsiness, which analyzes the driver's eye aspect ratio in order to detect drowsiness; and alert driver, which gives feedback through audio or visual signals when drowsiness has been detected. Another important feature of the system is the logging of drowsiness events for subsequent analysis with the "log events" use case. There are a couple of use cases for the system administrator, namely "view reports," to view historical data from drowsiness events and "adjust settings" to configure parameters such as alert thresholds and camera settings. A system boundary surrounds the use cases associating stick figures for the driver and system administrator to their respective use cases. This high-level overview therefore allows stakeholders to understand how the system works and whom it will interact with, thereby guiding the design and development process (Figure 2) [13].

Figure 2: Use case diagram for driver drowsiness detection system.

General note, this project presents relevance on how technology will be intertwined with life affairs to achieve safety and wellbeing [14,15].

Conclusion

In conclusion, this sleepiness warning system is a novelty technology meant for road and user safety where complex algorithms coupled with real time monitoring are employed to measure alertness levels. Since facial recognition technology and EAR of a human eye are utilized to approximate the levels of sleepiness, you may get the warning most timely if you exhibit more signals to become drowsy. Performance analysis reveals that the system works with very high detection accuracy and very minimalistic response time and it supports reliable operation under different conditions, so it may be a good tool in the prevention of incidents due to fatigue. Moreover, accounting for user experience and resource utilization ensures the created solution is not only effective but also practical to use daily. Continuous improvement and real world testing will further hone the system and lead it to be used in extremely many applications, not only auto but also occupational. On a general note, this project presents relevance on how technology will be intertwined with life affairs to achieve safety and well-being.

Future Scope

The future scope of the driver drowsiness detection system is quite vast and is primarily promising for raising road safety through various advancements and integrations. Some integral development opportunities may include integration with Advanced Driver Assistance Systems (ADAS), thus allowing collaboration between the system and existing vehicle technologies for provision of real-time feedback and preventive measures like automatic braking or lane-keeping assistance. This can improve detection accuracy and also tailor models to various driving behaviors. Adding multi-modal monitoring, for example, by integrating physiological signals from wearable devices, can enhance the system's comprehensive evaluation of the driver's alertness state. The system may be modified to detect fatigue and stress, thereby giving insights into overall driver behavior. Additional user personalization and context-aware features to consider environmental conditions could make user experience significantly better. As regulations regarding the safety of vehicles heighten, drowsiness detection systems would probably become mandatory for commercial vehicles. This technology also opens up scope for continuous further study and research collaboratively with academic institutes and industry leaders to make it relevant with ever-changing environments and factors. Altogether, these improvements would be transformative in a solution for safety and would significantly reduce the risk of drowsy driving as well, thereby affecting all road users.

References

1. Ibrahim S. Social and contextual taxonomy of cybercrime: Socioeconomic theory of Nigerian cybercriminals. Int J Law Crime Justice. 2016;47:44-57.

   [Google Scholar]

2. Chen H, Li Y, Chen L, Yin J. Understanding employees' adoption of the Bring-Your-Own-Device (BYOD): The roles of information security-related conflict and fatigue. J Enterp Inf Manag. 2021;34(3):770-792.

   [Google Scholar]

3. Verfürth M. Design and validation of a questionnaire to measure the acceptance of telemedicine by healthcare professionals in Germany. 2020.

   [Google Scholar]

4. Redmiles EM, Mazurek ML, Dickerson JP. Dancing pigs or externalities? Measuring the rationality of security decisions. In Proceedings of the 2018 ACM Conference on Economics and Computation. 2018;215-232.

   [Google Scholar]

5. Lazarus S, Button M. Tweets and reactions: revealing the geographies of cybercrime perpetrators and the North-South divide. Cyberpsychol Behav Soc Netw. 2022;25(8):504-511.

   [Crossref] [Google Scholar] [PubMed]

6. Kho J, Gillespie N, Martin-Khan M. A systematic scoping review of change management practices used for telemedicine service implementations. BMC Health Serv Res. 2020;20:1-6.

   [Crossref] [Google Scholar] [PubMed]

7. Ameen N, Tarhini A, Shah MH, Madichie N, Paul J, Choudrie J. Keeping customers' data secure: A cross-cultural study of cybersecurity compliance among the Gen-Mobile workforce. Comput Human Behav. 2021;114:106531.

   [Google Scholar]

8. May C, Harrison R, Finch T, MacFarlane A, Mair F, Wallace P. Understanding the normalization of telemedicine services through qualitative evaluation. J Am Med Inform Assoc. 2003;10(6):596-604.

   [Crossref] [Google Scholar] [PubMed]

9. Ly BA, Labonte R, Bourgeault IL, Niang MN. The individual and contextual determinants of the use of telemedicine: A descriptive study of the perceptions of Senegal's physicians and telemedicine projects managers. PLoS One. 2017;12(7):e0181070.

   [Crossref] [Google Scholar] [PubMed]

10. Hijji M, Alam G. Cybersecurity Awareness and Training (CAT) framework for remote working employees. Sensors. 2022;22(22):8663.

    [Crossref] [Google Scholar] [PubMed]

11. Lee D, Yoon S. Application of artificial intelligence-based technologies in the healthcare industry: Opportunities and challenges. Int J Environ Res Public Health. 2021;18(1):271.

    [Crossref] [Google Scholar] [PubMed]

12. Shah NM, Kaltsakas G. Telemedicine in the management of patients with chronic respiratory failure. Breathe. 2021;17(1):210008.

    [Crossref] [Google Scholar] [PubMed]

13. Maalem Lahcen RA, Caulkins B, Mohapatra R, Kumar M. Review and insight on the behavioral aspects of cybersecurity. Cybersecurity. 2020;3:1-8.

    [Google Scholar]

14. Mbunge E, Muchemwa B, Batani J. Are we there yet? Unbundling the potential adoption and integration of telemedicine to improve virtual healthcare services in African health systems. Sens Int. 2022;3:100152.

    [Crossref] [Google Scholar] [PubMed]

15. Ahmed Y, Naqvi S, Josephs M. Cybersecurity metrics for enhanced protection of healthcare IT systems. In2019 13th International Symposium on Medical Information and Communication Technology (ISMICT) 2019:1-9.

    [Google scholar]