CN109977786B - Driver posture detection method based on video and skin color area distance - Google Patents

Driver posture detection method based on video and skin color area distance Download PDF

Info

Publication number
CN109977786B
CN109977786B CN201910156046.5A CN201910156046A CN109977786B CN 109977786 B CN109977786 B CN 109977786B CN 201910156046 A CN201910156046 A CN 201910156046A CN 109977786 B CN109977786 B CN 109977786B
Authority
CN
China
Prior art keywords
video
image
skin color
characteristic
data set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910156046.5A
Other languages
Chinese (zh)
Other versions
CN109977786A (en
Inventor
何杰
汤慧
化丽茹
曦曙
郑有凤
赵池航
周博见
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN201910156046.5A priority Critical patent/CN109977786B/en
Publication of CN109977786A publication Critical patent/CN109977786A/en
Application granted granted Critical
Publication of CN109977786B publication Critical patent/CN109977786B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a driver posture detection method based on a video and skin color area distance, which comprises the steps of extracting skin color areas of sampled images in a plurality of sample videos, calculating the centroid coordinate of the skin color areas, converting the centroid coordinate into a characteristic distance to represent the characteristic value of each image, and fusing the characteristic values of a plurality of images corresponding to one section of video into one characteristic value by adopting a clustering algorithm; constructing a BP neural network, inputting the fused characteristic values and the corresponding driving posture categories into the BP neural network as training samples, and training to obtain a driver posture detection model; and during detection, acquiring a video of a driver to be detected during driving, calculating a characteristic value of the video to be detected according to the method in the step, taking a calculation result as an input of a driver posture detection model, and outputting the calculation result as a driving posture category of the video to be detected. The method can effectively improve the detection rate of the posture of the driver, realize the recognition and classification of the driving behaviors of the driver and finally realize the real-time early warning of the operation driving process.

Description

Driver posture detection method based on video and skin color area distance
Technical Field
The invention belongs to the field of traffic safety, and particularly relates to a driver posture detection method based on a video and a skin color area distance.
Background
The world health organization 'road safety global status report 2015' indicates that road traffic accidents are a main factor of global population death, about 3500 people in the world die due to road traffic collision every day, and serious road traffic accidents not only harm the national economy, but also bring heavy burden to families, so that the improvement of traffic safety becomes one of the primary tasks in the current work of various countries.
The occurrence of traffic accidents is the result of the combined action of human-vehicle-road-environment factors, and researchers generally believe that more than 80% of accidents occur due to wrong driving behaviors of drivers, such as overspeed, fatigue driving and cell phone use. These behaviors all seriously affect the perception and judgment of the driver, and how to detect and identify the behavior of the driver becomes important for traffic safety.
With the increasing requirements of people on traffic safety, the existing method for detecting the posture of the driver limited to the head is difficult to meet the requirements of people on safety; the invasive detection method has obvious limitation and is difficult to popularize; meanwhile, research results based on the head and the left and right hand areas are few, and intensive research and further optimization are urgently needed. In addition, the currently applied skin color detection method extracts a single full image pixel as a feature, the feature data is single and the dimension is too large, and the problem that detection cannot be performed due to instantaneous area overlapping, partial shielding and the like can occur in the detection process, so that the detection accuracy is affected.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the problems in the prior art, the invention provides a driver posture detection method based on a video and skin color area distance, which can effectively improve the detection rate of the posture of a driver, realize the recognition and classification of the driving behaviors of the driver and finally realize the real-time early warning of the operation driving process.
The technical scheme is as follows: the invention adopts the following technical scheme:
a driver posture detection method based on video and skin color area distance comprises the following steps:
(1) collecting N sections of videos of drivers during driving, wherein each section of video only comprises one of three different driving postures, namely a steering wheel gripped by two hands, a control gear and an abnormal driving posture; the duration of the ith video is tiFrame rate friThe driving posture class is pi，i＝1..N；
(2) Sequentially processing N sections of videos collected in the step (1), and intercepting one image of each section of video every F frames to form an image data set J ═ J1,J2,…,JN) Wherein the ith video corresponds to a data set
Figure BDA0001982939700000021
mi is the number of images intercepted by the ith video;
(3) sequentially processing a data set J corresponding to the ith videoiExtracting skin color areas and calculating the centroid coordinates of the first three skin color areas with the largest area; when the data set JiIf one image can not detect three skin color areas, the next image is switched to continue processing, and if the data set J is detectediIf none of the images can be processed, an image is intercepted again for the ith video every F' frame to form a new data set JiRe-inspection if data set JiAll images in the video cannot be processed, and the ith video segment, F ', is abandoned'>F；
(4) Converting the centroid coordinates of the skin color area extracted in the step (3) into characteristic distances to represent the characteristic value of each image, wherein the characteristic distances are represented on a data set JiThe images in (1) are processed in sequence to obtain characteristic values
Figure BDA0001982939700000022
ni is the actual effective image quantity after processing in the image intercepted by the ith video, and ni is less than or equal to mi; using a clustering algorithm to convert lambdaiThe eigenvalue Lambda is fused intoiAnd finally, the result obtained by the N-segment video processing forms a characteristic set of lambda ═ (lambda)1,Λ2,…,ΛW)，W≤N；
(5) Adopting a BP neural network classifier to obtain a characteristic set Lambda and a corresponding driving posture class piAs trainingInputting the sample into a classifier, and training the classifier to obtain a driver posture detection model;
(6) collecting a video of a driver to be detected when driving, wherein the video only comprises one of three different driving postures, namely a steering wheel gripped by two hands, a control gear and an abnormal driving posture; intercepting the image of the collected video according to the method in the step (2) to form an image data set V ═ V1,v2,…,vm) Wherein m is the number of the intercepted images; extracting skin color regions from the images in the data set V according to the method in the step (3) and calculating the centroid coordinates of the first three skin color regions with the largest area; obtaining the characteristic value Lambda of the image data set V according to the method in the step (4)VWill beVAnd (5) as the input of the BP neural network model trained in the step (5), outputting the driving posture category of the video to be detected.
The step (3) of calculating the centroid coordinates of the first three skin color regions with the largest area in the image specifically comprises the following steps:
(3-1) preprocessing an image to be processed by using a reference white method, extracting a characteristic region by using a skin color model based on a normalized RGB color space, and finally removing an interference region in the image to be processed by using a mathematical morphology method and reserving the first three characteristic regions with the largest area;
(3-2) scanning the image pixels processed in the step (3-1) row by row and column by using a bwleael function, identifying three characteristic regions according to pixel arrangement positions in a matrix returned by the function, and then calculating the centroid coordinates of the three characteristic regions, wherein the centroid calculation formula is as follows:
Figure BDA0001982939700000031
wherein x iskAnd ykRespectively a centroid abscissa and a centroid ordinate of the kth region, sum _ x is the sum of abscissa of pixel points in the kth region, sum _ y is the sum of ordinate of pixel points in the kth region, area is the number of pixel points in the kth region, and k is 1,2, and 3;
the region with the upper centroid among the three feature regions is set as a head region of a label H, the rest two regions on the left side are set as a left-hand region of a label L, and the regions on the right side are set as a right-hand region of a label R.
In the step (4), a data set JiCharacteristic value of (A)iThe calculation is specifically as follows:
(4-1) calculation of data set JiEach image of
Figure BDA0001982939700000032
Characteristic value of
Figure BDA0001982939700000033
q＝1,..,ni：
Calculating characteristic distance by using the skin color area centroid coordinate calculated in the step (3) to represent the image
Figure BDA0001982939700000034
The eigenvalue, expressed by the formula:
Figure BDA0001982939700000035
Figure BDA0001982939700000036
Figure BDA0001982939700000037
Figure BDA0001982939700000038
wherein (x)H,yH)、(xL,yL)、(xR,yR) Are respectively images
Figure BDA00019829397000000312
Centroid coordinates of the middle head region, the left hand region and the right hand region;
(4-2) on the data set JiEach image in (a) calculates its feature value to form a set
Figure BDA0001982939700000039
Using clustering algorithm to pair lambdaiCentering, continuously iterating each data point
Figure BDA00019829397000000310
The standard deviation from the clustering center reaches the minimum, and the clustering center is JiCharacteristic value of (A)i。
The step (5) is specifically as follows:
(5-1) vs Λ ═ Λ1,Λ2,…,ΛW) Carrying out normalization processing to obtain a set Lambda' as a training sample; will drive the attitude class piConversion to vectors
Figure BDA00019829397000000311
Is represented by [1,0]Represents the first class, [0,1,0 ]]Represents the second class, [0,0,1 ]]Represents a third class;
(5-2) constructing a BP neural network, wherein an input layer of the BP neural network has 2 input nodes; two hidden layers, each hidden layer having 5 hidden nodes; the output layer has 3 nodes, and the values of the 3 output nodes form a driving posture category represented by the vector;
(5-3) mixing the samples in the lambda'iInputting the constructed BP neural network, and obtaining the actual output r of the neural network after the forward layer-by-layer processing is carried out on the connection condition among all the nodesi(ii) a Calculating riAnd
Figure BDA0001982939700000041
the error is reversely transmitted back to the previous layers layer by layer, and the error is loaded on the connection weight, so that the connection weight of the whole BP neural network is converted to the direction of reducing the error; and repeating the steps for each group of input and output samples in the training set until the error of the whole training sample set is reduced to a preset threshold value.
When the image is intercepted again for the ith video in the step (3), the value of the interval F' is as follows:
Figure BDA0001982939700000042
wherein
Figure BDA0001982939700000043
The rounding-up operator.
The abnormal driving postures comprise that two hands leave a steering wheel, one hand drives, eating and calling.
Has the advantages that: compared with the prior art, the method for detecting the posture of the driver based on the video and the skin color area distance detects the behavior of the driver in real time by using the posture sequence observation information acquired from the driving video, and finally realizes real-time early warning on the operation driving process. Compared with the existing detection method for extracting a single characteristic value by using a single full-pixel image, the method can effectively reduce the problem that the detection cannot be carried out due to instantaneous area overlapping, partial shielding and the like, thereby effectively improving the driving posture detection rate and ensuring the safety of the passenger and freight car operation process.
Drawings
FIG. 1 is a flow chart of a driver gesture detection and recognition method disclosed in the present invention;
FIG. 2 is an original image of different postures of the driver captured from a video;
FIG. 3 is a diagram of reference white processing effect
FIG. 4 is a diagram of skin color detection results;
FIG. 5 is a diagram illustrating the effect of image morphological processing;
FIG. 6 is a schematic diagram of extracting centroid coordinates of a skin color region;
FIG. 7 is a diagram of the training effect of the BP neural network.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention are described below with reference to the accompanying drawings.
In the embodiment, a driver posture data set is established by shooting different driving postures of different drivers in different real driving scenes, so that the detection and the recognition of the driver posture are realized, and a flow chart is shown in fig. 1.
Step 1: the placement position of the camera is determined according to the vehicle type and the cab space during video acquisition, the upper body area of a driver is ensured to be positioned in the center of the lens, and the detection area is shown in fig. 2. The method comprises the steps of collecting a video of a driver during driving within 200 periods of time of 10 seconds and 30 frames per second, wherein the video comprises 50 gestures of holding a steering wheel by two hands, 50 gestures of operating a gear and 100 abnormal driving gestures, and the abnormal driving gestures comprise two hands leaving the steering wheel, one-hand driving, eating, making a call and the like;
step 2: sequentially processing 200 sections of videos collected in the step 1, and intercepting one image of each section of video every 30 frames to form an image data set J ═ J1,J2,…,J200) That is, each video segment intercepts 10 images, wherein the ith video segment corresponds to a data set
Figure BDA0001982939700000051
And step 3: sequentially processing a data set J corresponding to the ith videoiExtracting skin color areas and calculating the centroid coordinates of the first three skin color areas with the largest area; when the data set JiIf one image can not detect three skin color areas, the next image is switched to continue processing, and if the data set J is detectediIf none of the images can be processed, an image is intercepted again for the ith video every 15 frames to form a new data set JiRe-inspection if data set JiDiscarding the ith video if all the images in the video still cannot be processed;
182 effective samples are finally obtained in the embodiment, wherein the effective samples comprise 48 gestures of holding the steering wheel with two hands, 50 gestures of operating gears and 84 abnormal driving gestures (two hands leave the steering wheel, one hand drives, eats things and makes a call);
the steps of extracting the first three skin color areas with the largest area in the image and calculating the centroid coordinate specifically comprise:
(3-1) preprocessing the image by a reference white method before extracting the skin color area, wherein the processing effect is shown in fig. 3, wherein fig. 3(a) is the image before processing, and fig. 3(b) is the image after processing; then, extracting a characteristic region by adopting a skin color model based on a normalized RGB color space, and specifically comprising the following steps:
the RGB color space values (r, g, b) for each pixel are normalized according to equation (1):
Figure BDA0001982939700000052
normalized value (r)0,g0,b0) If the conditions of the formulas (2) and (3) are satisfied, the gray-scale value of the pixel is changed to 255, otherwise, the gray-scale value of the pixel is 0, and the result is shown in fig. 4 (b);
r0>95,g0>45,b0>20,r0>g0+15,r0>b0 (2)
Max{r0,g0,b0}-Min{r0,g0,b0}>15 (3)
removing the interference area in the image by using a mathematical morphology method and a connected region labeling method, and reserving the first three characteristic regions with the largest area, wherein the processing effect is shown in fig. 5, wherein fig. 5(a) and 5(c) are images before removing the interference area, and fig. 5(b) and 5(d) are corresponding images after removing the interference area;
(3-2) scanning the image pixels processed in the step (3-1) row by row and column by using a bwleael function, identifying three characteristic regions according to pixel arrangement positions in a matrix returned by the function, and then calculating the centroid coordinates of the three characteristic regions, wherein the centroid calculation formula is as follows:
Figure BDA0001982939700000061
wherein x iskAnd ykThe centroid abscissa and ordinate of the kth region are respectively, sum _ x is the sum of the abscissa of the pixels in the kth region, sum _ y is the sum of the ordinate of the pixels in the kth region, area is the number of the pixels in the kth region, k is 1,2,3, and fig. 6 shows the mass of 3 regionsThe coordinates of the heart.
The region with the upper centroid among the three feature regions is set as a head region of a label H, the rest two regions on the left side are set as a left-hand region of a label L, and the regions on the right side are set as a right-hand region of a label R.
And 4, step 4: converting the centroid coordinates of the skin color area extracted in the step 3 into characteristic distances to represent the characteristic value of each image, wherein the characteristic distances are represented for a data set JiThe images in (1) are processed in sequence to obtain characteristic values
Figure BDA0001982939700000062
Using a clustering algorithm to convert lambdaiThe eigenvalue Lambda is fused intoiThe final 182 segments of video processing result in the set of feature set Λ ═ (Λ)1,Λ2,…,Λ182)；
Calculating the characteristic distance by using the centroid coordinates of the skin color area and fusing each group of processing results specifically comprises the following steps:
(4-1) calculation of data set JiEach image of
Figure BDA0001982939700000063
Characteristic value of
Figure BDA0001982939700000064
q＝1,..,ni：
Calculating characteristic distance by using the skin color area centroid coordinate calculated in the step (3) to represent the image
Figure BDA0001982939700000065
The eigenvalue, expressed by the formula:
Figure BDA0001982939700000066
Figure BDA0001982939700000067
Figure BDA0001982939700000068
Figure BDA0001982939700000071
wherein (x)H,yH)、(xL,yL)、(xR,yR) Are respectively images
Figure BDA0001982939700000072
Centroid coordinates of the middle head region, the left hand region and the right hand region; l1，l2，l3Representing head-to-left, head-to-right, and left-to-right distances, respectively.
(4-2) on the data set JiEach image in (a) calculates its feature value to form a set
Figure BDA0001982939700000073
Using clustering algorithm to pair lambdaiCentering, continuously iterating each data point
Figure BDA0001982939700000074
The standard deviation from the clustering center reaches the minimum, and the clustering center is JiCharacteristic value of (A)i. The data obtained by processing in this embodiment is shown in table 1, where class 1 is a posture in which both hands grip the steering wheel, class 2 is a posture in which a shift is manipulated, and class 3 is an abnormal driving posture, including a posture in which both hands leave the steering wheel, a posture in which both hands drive, eat, and make a call;
and 5: adopting a BP neural network classifier to obtain a characteristic set Lambda and a corresponding driving posture class piInputting the training sample into a classifier, and training the classifier to obtain a driver posture detection model;
in this example, the results of the detection by the various classifiers are shown in table 2. The BP neural network is a neural network model with wider application, and is mainly used for function approximation, model identification and classification, time series prediction and the like. The method has certain superiority in the aspects of accuracy, function calling convenience and the like, so the embodiment adopts the BP neural network to train and classify the sample data.
The training and calling of the BP neural network specifically comprises the following steps:
(5-1) vs Λ ═ Λ1,Λ2,…,Λ182) Carrying out normalization processing to obtain a set Lambda' as a training sample; will drive the attitude class piConversion to vectors
Figure BDA0001982939700000075
Is represented by [1,0]Representing a first type of two-handed grip of the steering wheel, [0,1,0]Represents the second type of operation gear position, 0,1]Representing a third type of abnormal driving posture;
(5-2) constructing a BP neural network, wherein an input layer of the BP neural network has 2 input nodes; two hidden layers, each hidden layer having 5 hidden nodes; the output layer has 3 nodes, and the values of the 3 output nodes form a driving posture category represented by the vector;
(5-3) mixing the samples in the lambda'iInputting the constructed BP neural network, and obtaining the actual output r of the neural network after the forward layer-by-layer processing is carried out on the connection condition among all the nodesi(ii) a Calculating riAnd
Figure BDA0001982939700000076
the error is reversely transmitted back to the previous layers layer by layer, and the error is loaded on the connection weight, so that the connection weight of the whole BP neural network is converted to the direction of reducing the error; and repeating the steps for each group of input and output samples in the training set until the error of the whole training sample set is reduced to a preset threshold value.
Table 1: characteristic value obtained by processing different driving posture categories
Figure BDA0001982939700000081
Step 6: collecting a video of a driver to be detected when the driver drives, wherein the video only comprises a steering wheel tightly held by two hands and a driver to operateOne of three different driving postures, namely a longitudinal gear posture and an abnormal driving posture; intercepting the image of the collected video according to the method in the step (2) to form an image data set V ═ V1,v2,…,vm) Wherein m is the number of the intercepted images; extracting skin color regions from the images in the data set V according to the method in the step (3) and calculating the centroid coordinates of the first three skin color regions with the largest area; obtaining the characteristic value Lambda of the image data set V according to the method in the step (4)VWill beVAnd (5) as the input of the BP neural network model trained in the step (5), outputting the driving posture category of the video to be detected.
The present embodiment uses videos of known driving gesture categories for testing and verification, and the verification result is shown in fig. 7. The open circles in the figure represent the results of the detection performed by the method of the invention, i.e. the prediction output; the asterisks indicate the known driving attitude category, i.e., the desired output. As can be seen from the figure, 4 expected outputs in 45 samples to be tested do not accord with the predicted output, the expected outputs of the other samples to be tested are completely consistent with the predicted output, and the accuracy reaches 91 percent, which shows that the method has higher detection accuracy.
TABLE 2 evaluation of classification methods
Classification method Rate of accuracy Duration of training
Decision tree 89.0％ 5.3883ms
SVM 86.8％ 11.507ms
KNN 87.4％ 12.542ms
BP neural network of the present invention 93.5％ 6.4973ms

Claims (5)

1. A driver posture detection method based on video and skin color area distance is characterized by comprising the following steps:
(1) collecting N sections of videos of drivers during driving, wherein each section of video only comprises one of three different driving postures, namely a steering wheel gripped by two hands, a control gear and an abnormal driving posture; the duration of the ith video is tiFrame rate friThe driving posture class is pi，i＝1..N；
(2) Sequentially processing N sections of videos collected in the step (1), and intercepting one image of each section of video every F frames to form an image data set J ═ J1,J2,…,JN) Wherein the ith video corresponds to a data set
Figure FDA0002653021810000011
mi is the number of images intercepted by the ith video;
(3) sequentially processing a data set J corresponding to the ith videoiExtracting skin color areas and calculating the centroid coordinates of the first three skin color areas with the largest area; when the data set JiIf one image can not detect three skin color areas, the next image is switched to continue processing, and if the data set J is detectediIf none of the images can be processed, an image is intercepted again for the ith video every F' frame to form a new data set JiRe-inspection if data set JiAll images in the video cannot be processed, and the ith video segment, F ', is abandoned'>F；
(4) Converting the centroid coordinates of the skin color area extracted in the step (3) into characteristic distances to represent the characteristic value of each image, wherein the characteristic distances are represented on a data set JiThe images in (1) are processed in sequence to obtain characteristic values
Figure FDA0002653021810000012
ni is the actual effective image quantity after processing in the image intercepted by the ith video, and ni is less than or equal to mi; using a clustering algorithm to convert lambdaiThe eigenvalue Lambda is fused intoiAnd finally, the result obtained by the N-segment video processing forms a characteristic set of lambda ═ (lambda)1,Λ2,…,ΛW)，W≤N；
(5) Adopting a BP neural network classifier to obtain a characteristic set Lambda and a corresponding driving posture class piInputting the training sample into a classifier, and training the classifier to obtain a driver posture detection model;
(6) collecting a video of a driver to be detected when driving, wherein the video only comprises one of three different driving postures, namely a steering wheel gripped by two hands, a control gear and an abnormal driving posture; intercepting the image of the collected video according to the method in the step (2) to form an image data set V ═ V1,v2,…,vm) Wherein m is the number of the intercepted images; extracting skin color regions from the images in the data set V according to the method in the step (3) and calculating the centroid coordinates of the first three skin color regions with the largest area; obtaining the characteristic value Lambda of the image data set V according to the method in the step (4)VWill beVAs the input of the BP neural network model trained in the step (5), outputting the driving posture category of the video to be detected;
in the step (4), a data set JiCharacteristic value of (A)iThe calculation is specifically as follows:
(4-1) calculation of data set JiEach image of
Figure FDA0002653021810000021
Characteristic value of
Figure FDA0002653021810000022
Calculating characteristic distance by using the skin color area centroid coordinate calculated in the step (3) to represent the image
Figure FDA0002653021810000023
The eigenvalue, expressed by the formula:
Figure FDA0002653021810000024
Figure FDA0002653021810000025
Figure FDA0002653021810000026
Figure FDA0002653021810000027
wherein (x)H,yH)、(xL,yL)、(xR,yR) Are respectively images
Figure FDA0002653021810000028
Centroid coordinates of the middle head region, the left hand region and the right hand region;
(4-2) on the data set JiEach image in (a) calculates its feature value to form a set
Figure FDA0002653021810000029
Using clustering algorithm to pair lambdaiCentering, continuously iterating each data point
Figure FDA00026530218100000210
And clusteringThe standard deviation of the center reaches the minimum, and the clustering center is JiCharacteristic value of (A)i。
2. The method for detecting the posture of the driver based on the video and the distance between the skin color regions as claimed in claim 1, wherein the step (3) of calculating the coordinates of the center of mass of the first three skin color regions with the largest area in the image is specifically as follows:
(3-1) preprocessing an image to be processed by using a reference white method, extracting a characteristic region by using a skin color model based on a normalized RGB color space, and finally removing an interference region in the image to be processed by using a mathematical morphology method and reserving the first three characteristic regions with the largest area;
(3-2) scanning the image pixels processed in the step (3-1) row by row and column by using a bwleael function, identifying three characteristic regions according to pixel arrangement positions in a matrix returned by the function, and then calculating the centroid coordinates of the three characteristic regions, wherein the centroid calculation formula is as follows:
Figure FDA00026530218100000211
wherein x iskAnd ykRespectively a centroid abscissa and a centroid ordinate of the kth region, sum _ x is the sum of abscissa of pixel points in the kth region, sum _ y is the sum of ordinate of pixel points in the kth region, area is the number of pixel points in the kth region, and k is 1,2, and 3;
the region with the upper centroid among the three feature regions is set as a head region of a label H, the rest two regions on the left side are set as a left-hand region of a label L, and the regions on the right side are set as a right-hand region of a label R.
3. The method for detecting the attitude of the driver based on the video and the skin color region distance as claimed in claim 1, wherein the step (5) is specifically as follows:
(5-1) vs Λ ═ Λ1,Λ2,…,ΛW) Carrying out normalization processing to obtain a set Lambda' as a training sample; will drive the postureState class piConversion to vectors
Figure FDA0002653021810000031
Is represented by [1,0]Represents the first class, [0,1,0 ]]Represents the second class, [0,0,1 ]]Represents a third class;
(5-2) constructing a BP neural network, wherein an input layer of the BP neural network has 2 input nodes; two hidden layers, each hidden layer having 5 hidden nodes; the output layer has 3 nodes, and the values of the 3 output nodes form a driving posture category represented by the vector;
(5-3) mixing the samples in the lambda'iInputting the constructed BP neural network, and obtaining the actual output r of the neural network after the forward layer-by-layer processing is carried out on the connection condition among all the nodesi(ii) a Calculating riAnd
Figure FDA0002653021810000032
the error is reversely transmitted back to the previous layers layer by layer, and the error is loaded on the connection weight, so that the connection weight of the whole BP neural network is converted to the direction of reducing the error; and repeating the steps for each group of input and output samples in the training set until the error of the whole training sample set is reduced to a preset threshold value.
4. The method for detecting the posture of the driver based on the video and the skin color area distance as claimed in claim 1, wherein when the image is re-captured in the ith video in the step (3), the value of the interval F' is as follows:
Figure FDA0002653021810000033
wherein
Figure FDA0002653021810000034
The rounding-up operator.
5. The method of claim 1, wherein the abnormal driving gesture comprises two hands off a steering wheel, one hand driving, eating, and making a call.
CN201910156046.5A 2019-03-01 2019-03-01 Driver posture detection method based on video and skin color area distance Active CN109977786B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910156046.5A CN109977786B (en) 2019-03-01 2019-03-01 Driver posture detection method based on video and skin color area distance

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910156046.5A CN109977786B (en) 2019-03-01 2019-03-01 Driver posture detection method based on video and skin color area distance

Publications (2)

Publication Number Publication Date
CN109977786A CN109977786A (en) 2019-07-05
CN109977786B true CN109977786B (en) 2021-02-09

Family

ID=67077606

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910156046.5A Active CN109977786B (en) 2019-03-01 2019-03-01 Driver posture detection method based on video and skin color area distance

Country Status (1)

Country Link
CN (1) CN109977786B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111832446B (en) * 2020-06-30 2022-11-08 东南大学 Driver posture identification method based on double-view-angle video data

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102567743A (en) * 2011-12-20 2012-07-11 东南大学 Automatic identification method of driver gestures based on video images
CN104200199A (en) * 2014-08-27 2014-12-10 合肥工业大学 TOF (Time of Flight) camera based bad driving behavior detection method
CN104463146A (en) * 2014-12-30 2015-03-25 华南师范大学 Posture identification method and device based on near-infrared TOF camera depth information
EP3395237A1 (en) * 2017-04-24 2018-10-31 Oxehealth Limited Improvements in or relating to in-vehicle monitoring
CN109214370A (en) * 2018-10-29 2019-01-15 东南大学 A kind of driver gestures detection method based on arm area of skin color center-of-mass coordinate
CN109284698A (en) * 2018-09-03 2019-01-29 深圳市尼欧科技有限公司 A kind of fatigue driving behavioral value method based on image recognition technology

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102289660B (en) * 2011-07-26 2013-07-03 华南理工大学 Method for detecting illegal driving behavior based on hand gesture tracking
CN106295600A (en) * 2016-08-18 2017-01-04 宁波傲视智绘光电科技有限公司 Driver status real-time detection method and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102567743A (en) * 2011-12-20 2012-07-11 东南大学 Automatic identification method of driver gestures based on video images
CN104200199A (en) * 2014-08-27 2014-12-10 合肥工业大学 TOF (Time of Flight) camera based bad driving behavior detection method
CN104463146A (en) * 2014-12-30 2015-03-25 华南师范大学 Posture identification method and device based on near-infrared TOF camera depth information
EP3395237A1 (en) * 2017-04-24 2018-10-31 Oxehealth Limited Improvements in or relating to in-vehicle monitoring
CN109284698A (en) * 2018-09-03 2019-01-29 深圳市尼欧科技有限公司 A kind of fatigue driving behavioral value method based on image recognition technology
CN109214370A (en) * 2018-10-29 2019-01-15 东南大学 A kind of driver gestures detection method based on arm area of skin color center-of-mass coordinate

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Vision-based Classification of Driving Postures by Efficient Feature Extraction and Bayesian Approach;Chihang Zhao等;《Journal of Intelligent & Robotic Systems》;20121221;第483-495页 *
面向交通卡口图像的驾驶员违章行为视觉感知研究;刘操;《中国博士学位论文全文数据库》;20171215;第C034-36页 *

Also Published As

Publication number Publication date
CN109977786A (en) 2019-07-05

Similar Documents

Publication Publication Date Title
CN102289660B (en) Method for detecting illegal driving behavior based on hand gesture tracking
CN110728241A (en) Driver fatigue detection method based on deep learning multi-feature fusion
CN106169081B (en) A kind of image classification and processing method based on different illumination
CN103714660B (en) System for achieving fatigue driving judgment on basis of image processing and fusion between heart rate characteristic and expression characteristic
CN104123549B (en) Eye positioning method for real-time monitoring of fatigue driving
CN108664947A (en) A kind of fatigue driving method for early warning based on Expression Recognition
CN108596087B (en) Driving fatigue degree detection regression model based on double-network result
CN109685026B (en) Real-time monitoring method for mobile phone call held by driver
CN102122357A (en) Fatigue detection method based on human eye opening and closure state
CN108446586B (en) Method for detecting specific action of train driver
CN104778446A (en) Method for constructing image quality evaluation and face recognition efficiency relation model
CN109506628A (en) Object distance measuring method under a kind of truck environment based on deep learning
CN110879982A (en) Crowd counting system and method
CN107844783A (en) A kind of commerial vehicle abnormal driving behavioral value method and system
CN110348350B (en) Driver state detection method based on facial expressions
CN109977786B (en) Driver posture detection method based on video and skin color area distance
CN105718896A (en) Intelligent robot with target recognition function
CN112052829B (en) Pilot behavior monitoring method based on deep learning
CN111553217A (en) Driver call monitoring method and system
CN109214370B (en) Driver posture detection method based on arm skin color area centroid coordinates
CN115661800A (en) Dangerous driving behavior detection method based on sight direction time relation learning
CN110570469A (en) intelligent identification method for angle position of automobile picture
Chen et al. Vehicle classification and counting system
CN115861982A (en) Real-time driving fatigue detection method and system based on monitoring camera
CN113361452B (en) Driver fatigue driving real-time detection method and system based on deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant