Research on Fatigue Driving Feature Detection Algorithms of drivers based on machine learning

In this paper, aiming at the detection of fatigue driving scene of drivers, a diagnostic model based on machine learning is proposed under the scene of long-time driving. The validity of the model is verified by simulation experiments. The simulation result shows that the model can effectively fit the fatigue condition of drivers under long-time driving, and accurately judge and warn the fatigue state of drivers. At the same time, the model also extends the application of fatigue classification detection.


Introduction
In recent years, with the rapid increase of vehicle ownership per capita in the world, fatigue driving has caused a high proportion of car accidents, and the lack of active safety of automobiles has become a serious problem. It is generally recognized that it is necessary to develop detection methods and techniques for fatigue driving. The automobile is the carrier of the transportation system. It is of great significance to diagnose and warn fatigue driving, protect drivers, and reduce the accident rate of the whole transportation system. At the same time, due to the harmfulness and the timeliness of fatigue driving, it is difficult to analyse and track in the later period of the accident. A fatigue detection algorithm with real-time monitoring, high accuracy, and non-sensitive will become an urgent need for the global development of active safety technology for the fatigue detection of drivers.
At present, driver fatigue detection has always been the frontier field of vehicle active safety. Many scholars are actively working on various aspects of fatigue detection algorithms. The driver fatigue detections of scholars mainly focus on the automobile condition, physiological signals, the use of attitude, and more. As early as 1996, Knipling Wang et al. constructed a model by measuring the physiological characteristics of opening and closing, movement, and eyes to study fatigue detection (Knipling et al., 1996). A real-time vehicle fatigue detection method, PERCLOS evaluation method, was proposed by Dr. Wierwille et al. (1996). In recent years, Simon et (Simon et al., 2011). Buendia et al. used abnormal heart rate data to detect the heart and indirectly judge whether a driver is tired or not. This is similar to that of Patel et al. who used a neural network to fit the heart rate data of drivers to assess their fatigue state (Buendia et al., 2019;Patel et al., 2011). This method can get more accurate results, but for drivers, it has a certain impact on the driving experience, and features are easily disturbed. There are also Gao Zhenhai and Li Zuojin who test the state of the automobile to test whether the driver is tired or not by testing the data of the steering wheel on the way of driving (Zhenhai et al., 2017;Zuojin et al., 2017). Although this method does not affect the driving experience and obtaining driver state indirectly from the vehicle condition is too complex and has too much interfered. Wang et al. used computer vision technology to detect driver fatigue (Wang et al., 2019), which seems to be the most reasonable way at present. This paper studies fatigue driving detection based on machine learning explores the realization of driver fatigue detection algorithm under the condition of long-time driving and solves the problem of driver fatigue driving without sensation detection.

Analysis of driver fatigue state features
Fatigue driving refers to the phenomenon that the drivers cannot get timely information of road conditions due to sensory sensitivity reduction, distraction, and even unconsciousness in the driving process caused by longtime driving, or unsatisfied rest conditions of themselves. Under the fatigue state of drivers, the driving state of the vehicle, the force on the steering wheel, the change of speed, and the force on the pedal will change accordingly. For the physiological characteristics of drivers, their heart rate, pulse, respiratory rate, EEG, and more will also change accordingly. For the facial expression of drivers, there will be blinking, yawning, and other fatigue features. The fatigue state can be classified as shown in Table 1.
For fatigue detection to blinking and yawning, the traditional fatigue detection method mainly uses PERCLOS algorithm (Chunfa et al., 2016;Dewu, 2018;Dong et al., 2011;Fu et al., 2016;George & Routray, 2016;Hendra et al., 2019;Ji et al., 2018;Jun et al., 2016;Knapik & Cyganek, 2019;Shin et al., 2019;Xiaoyan, 2016;Xuwei et al., 2016) for single frame image detection: When a person is in a fatigue state, the time of blinking frequency and blinking time is closely related to the degree of fatigue. Therefore, the fatigue state of drivers can be reflected a certain extent by measuring the time of eyes closure; the driver is considered to reach a fatigue state (Dinges & Grace, 1998;Jiang et al., 2018;Niu & Suen, 2012;Wei et al., 2018;Weijian et al., 2014). When using PERCLOS value to judge fatigue, 40% is traditionally used as the threshold of PERCLOS (Baohu et al., 2013;Qin et al., 2008;Zhu & Yu, 2015). When the detection of fatigue features has exceeded the threshold, the blinking times of 5 times per minute are selected as the detection method of fatigue (Mingchu, 2012;Pei, 2001). Traditionally, it often needs to use a large number of data to carefully analyse the physiological characteristics of the driver under fatigue state to calculate the threshold. This generalizes fatigue algorithm reduce and the robustness is not strong. This design is based on machine learning and does not use the traditional manual threshold setting method. Firstly, no prior condition is used to set a threshold, and the fatigue condition of drivers is analysed and studied for a time S i . The analysis is divided into two parts. For single driver image capture, the main purpose is to determine whether fatigue characteristics appear, and judge the fatigue C 1 (fatigue) or C 0 (wakefulness) in a single photograph. The second part is the time S i learning on the condition obtained from the first part of learning judgment and makes a judgment on the driver situation in the period time of S i .
( 2) where g(x) is the resulting output of single-frame image judgment C 1 or C 0 . f (x) is a judgment of fatigue state over a whole period time.

Data collection
Because the open-source fatigue data set is available at present, the training data set of a classifier is mainly used to collect and extract information from images with fatigue and non-fatigued states. The whole process of information acquisition is to collect image data through a camera, and then face detection is performed after a single frame extraction of a video stream. Sixty-eight feature points are extracted from the detected face data (Figure 1).

Feature detection
To reduce the inconvenience, weak generalization, and low accuracy of the model caused by the threshold setting in traditional algorithms, this paper uses machine learning to optimize the fatigue detection method. For the analog ellipse of a single frame, the short axis of the ellipse is directly selected and divided by the long axis, to get the features of blinking and yawning detection.
where b is short axis; a is long axis; then e can indicate the degree of closure of eyes. 0 is the closed state. For yawning and blinking features, the distribution of data under fatigue and non-fatigued conditions is shown in Figures 2 and 3.
It can be seen that the difference between fatigue and non-fatigue state is large, which can complete the judgment of whether the current picture is fatigue or not.
When collecting data, there are four types of data, that is, mouth and eye fatigue, mouth fatigue and nonfatigue eye, non-fatigue mouth and eye fatigue, nonfatigue mouth, and eye, forming the corresponding four types of data sets for data analysis.

Cascade design
The core of this algorithm is to detect the fatigue state by cascading classifiers. For the first level classifier, the image is recognized by a single frame to determine whether fatigue features exist. The secondary classification ranks the results of the first-level classifier according to the time and carries out the detection for some time.
Secondary classifier: slide-window: where f i is the result of current pictures; f n−i is the result of the remaining previous images of the secondary classifier.

Experimental analysis
Because of the limitations of the traditional PERCLOS thresholding algorithm, there is no uniform and representative algorithm feature. Here we choose the conventional standard-setting to design the comparative experiment: if the number of blinks is less than 5 times in a minute, the driver is considered to fatigue (Zhu & Yu, 2015). A comparative experiment is carried out between the proposed algorithm and the traditional algorithm. The hardware parameters are shown in Table 2.
Using the monocular camera with a resolution of 640 * 320, and selecting SVM as the classifier of this algorithm, the first level SVM is mainly used for fatigue detection and judgment of single-frame image. Subsequently, the fatigue results detected by the first-level SVM are input into the second-level cache queue.
Secondary SVM will extract fatigue features from the secondary buffer queue by using a time-series slidewindow to detect driving fatigue for a long time.

Design of sequential slide-window
For secondary SVM, 100 frames of pictures are used as an eigenvector to train the SVM, and the correct classification of the eigenvector is obtained. The corresponding time sequence slide-window and classifier are constructed as shown in Figure 4.
The training set is used to adjust and optimize the parameters of the classifier. Finally, when the penalty coefficient C = 5, the soft interval is 0.04 and the Gaussian kernel (sigma) is 10 as the model parameter, the classification accuracy of the second-level SVM reaches 98.24%.

Experiment design
A total of 40 groups of training data sets were selected from a continuous video sequence, each with 100 frames of pictures.
To explore the advantages and disadvantages of this algorithm, the author designed three groups of comparative experiments.
(1). Using the original threshold alone, the detection accuracy is shown in Figure 5.
(2). In the case of the same training data set gradient, a threshold processing method and an algorithm proposed in this paper are used. The detection accuracy is shown in Figure 6. (3). Based on experiment 2, two different training data sets are added in both groups, and the change of detection accuracy is shown in Figure 7.
It can be seen that the traditional fatigue detection algorithm cannot effectively learn the rules of data because it depends on the choice of threshold. Preset thresholds require a lot of data and do not have good generalization. On the contrary, the method of machine learning can effectively fit the relationship between data and get a good classifier.
In this design, the sequential slide-window is essentially a continuously updated queue, so a piece of data    will exist in the slide-window for some time after entering the slide-window. Therefore, it will affect the resulting judgment for a certain time. That is to say, the current results will affect not only the current results but also the future results. The same as the memory effect, we can get a result that refers to the previous reasons for the occurrence of fatigue features. And since the length of this ingenious memory effect depends on the sliding step entirely, the more consistent and effective the fatigue detection results are when we select the smaller step size.

Conclusion
From the above data, it can be seen that the relationship between the data can be effectively fitted by learning, and the accuracy of long-term fatigue detection can be improved. In theory, this is because the internal data rules are learned by machine learning. In essence, the two ways of thinking are different.
In this paper, the cascade method of SVM is used to improve the accuracy of fatigue test results for a long time, and there are still many shortcomings in some details. Firstly, the fatigue features adopted in this paper are not complete only in the mouth and eyes, but also in the head deflection angle and time dimension. Secondly, the scale of the data sets used in this paper is not enough. This is also because there is no open-source data set at present. Meanwhile, for fatigue driving, it is difficult to directly collect in real scenes. The SVM cascade used in this algorithm realizes the long-term memory of detection. In the future, LTSM and other deep learning longterm memory neural networks should be developed.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
The work in this paper is supported by the opening fund of Urban rail transit vehicle system integration and control laboratory in Chongqing (CKLURTSIC-KFKT-201810).