CN110738190A - fatigue driving judgment method, device and equipment - Google Patents
fatigue driving judgment method, device and equipment Download PDFInfo
- Publication number
- CN110738190A CN110738190A CN201911030590.1A CN201911030590A CN110738190A CN 110738190 A CN110738190 A CN 110738190A CN 201911030590 A CN201911030590 A CN 201911030590A CN 110738190 A CN110738190 A CN 110738190A
- Authority
- CN
- China
- Prior art keywords
- driver
- head
- target image
- sequence data
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
Abstract
The invention discloses a method, a device and equipment for judging kinds of fatigue driving, wherein the judging method comprises the steps of firstly obtaining a target image containing a head image of a driver, then determining head posture information of the driver in the target image, then judging whether a feature sequence formed by the head feature information in a preset time period is characterized in that the driver is in an unacknowledged state or not to obtain a judging result, and finally sending out reminding information when the judging result is characterized in that the driver is in the unacknowledged state.
Description
Technical Field
The invention relates to the technical field of machine learning, in particular to a method, a device and equipment for judging types of fatigue driving.
Background
With the rapid development of domestic economy and the gradual improvement of the living standard of people, the holding amount of motor vehicles is continuously increasing.
At the same time, the number of road traffic accidents also shows a rising trend.
In a road traffic accident, the fatigue driving of the driver is which is the main factor causing the traffic accident.
Therefore, it is necessary to give a prompt when the driver is tired to reduce the occurrence of traffic accidents.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present invention provide methods, apparatuses, and devices for determining fatigue driving, so as to provide waking-up in time to prevent traffic accidents as much as possible when a driver is not awake, and the technical solution of the present invention is as follows:
A method for judging fatigue driving, comprising:
obtaining a target image containing a head image of a driver;
determining head posture information of a driver in the target image;
judging whether a characteristic sequence formed by the head characteristic information in a preset time period is characterized in that the driver is in an unconscious state or not to obtain a judgment result;
and sending out reminding information when the judgment result represents that the driver is not awake.
Preferably, the determining the head pose information of the driver in the target image comprises:
and inputting the target image as an input parameter into a deep neural network model to obtain an Euler angle parameter representing the head posture of the driver in the target image, and taking the Euler angle parameter as head posture information.
Preferably, the determining the head pose information of the driver in the target image comprises:
inputting the target image as an input parameter into a deep neural network model to obtain a human face key point in the target image;
mapping the face key points to a head three-dimensional space;
and calculating Euler angle parameters representing the head posture of the driver in the target image in the head three-dimensional space according to the key points of the face, and taking the Euler angle parameters as head posture information.
Preferably, the judging whether the feature sequence formed by the head feature information in the preset time period is characterized in that the driver is in an unconscious state, and obtaining a judgment result includes:
accumulating the head posture information in a preset time period into Euler angle sequence data;
performing sliding sampling on the Euler angle sequence data to obtain characteristic sequence data, wherein the characteristic sequence data is used for representing series actions made by the head of a driver in the preset time period;
and identifying the characteristic sequence data by using a cyclic memory neural network model to obtain a judgment result representing whether the driver is in an unacknowledged state or not.
Preferably, before identifying the feature sequence data by using the recurrent memory neural network model, the method further comprises:
carrying out multilayer convolution coding on the feature sequence data to obtain a feature sequence;
the identifying the feature sequence data using a recurrent memory neural network model comprises: and identifying the characteristic sequence by using a cyclic memory neural network model.
Preferably, the recognizing the sequence data by using the recurrent memory neural network model to obtain a judgment result indicating whether the driver is in the non-awake state includes:
respectively inputting characteristic sequences corresponding to every Euler angle parameter values in the characteristic sequences into a cyclic memory neural network model to obtain independent probability values of a plurality of the drivers in an unacknowledged state;
inputting a vector formed by characteristic sequences corresponding to every Euler angle parameter values in the characteristic sequences into a circular memory neural network model to obtain a related probability value of the driver in an unacknowledged state;
taking a calculation result obtained by carrying out weighted average or maximum value on the independent probability value and the related probability value as an output result;
and if the output result is greater than a preset threshold value, determining that the driver is in a non-waking state.
Another aspect of the present invention provides types of fatigue driving determination devices, including:
obtaining a device; for obtaining a target image containing an image of the driver's head;
a determination device; the head posture information of the driver in the target image is determined;
a judging device; the head feature information is used for judging whether a feature sequence formed by the head feature information in a preset time period is characterized in that a driver is in an unconscious state or not, and a judgment result is obtained;
a reminder device; and sending out reminding information when the judgment result represents that the driver is not awake.
Preferably, the judging module specifically includes:
an accumulation unit configured to accumulate the head posture information within a preset time period into euler angle sequence data;
the sampling unit is used for performing sliding sampling on the Euler angle sequence data to obtain characteristic sequence data, wherein the characteristic sequence data is used for representing series actions made by the head of a driver in the preset time period;
and the determining unit is used for identifying the characteristic sequence data by using a cyclic memory neural network model to obtain a judgment result representing whether the driver is in an unconscious state or not.
Preferably, the method further comprises the encoding unit:
the coding unit is used for carrying out multilayer convolution coding on the feature sequence data to obtain a feature sequence;
the determining unit is specifically configured to: and identifying the characteristic sequence by using a cyclic memory neural network model.
Another aspect of the present invention provides types of fatigue driving determination devices, including:
the system comprises image acquisition equipment, processing equipment and display equipment;
the processing equipment is respectively connected with the image acquisition equipment and the display equipment;
the processing device includes a processor configured to perform:
obtaining a target image containing a head image of a driver;
determining head posture information of a driver in the target image;
judging whether a characteristic sequence formed by the head characteristic information in a preset time period is characterized in that the driver is in an unconscious state or not to obtain a judgment result;
when the judgment result represents that the driver is in a non-waking state, sending out reminding information;
the image acquisition equipment is used for acquiring a target image containing a head image of a driver;
the display device is used for displaying the reminding information.
Compared with the prior art, the technical scheme provided by the embodiment of the invention firstly obtains the target image containing the head image of the driver; then determining the head posture information of the driver in the target image; then judging whether a characteristic sequence formed by the head characteristic information in a preset time period is characterized in that the driver is in an unconscious state or not to obtain a judgment result; and finally, sending out reminding information when the judgment result represents that the driver is not awake. Therefore, in the technical scheme of the invention, the driver can be determined to be in the non-waking state according to the head posture of the driver in the preset time period, so that the reminding information is sent to prevent traffic accidents as much as possible.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic flow chart of methods for determining fatigue driving provided in the embodiment of the present invention;
FIG. 2 is a schematic diagram of the head position of a driver in an unacknowledged state according to an embodiment of the invention;
fig. 3 is a structural diagram of types of fatigue driving determination devices provided by the embodiment of the invention;
fig. 4 is a schematic structural diagram of types of fatigue driving judgment devices provided by the embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only partial embodiments of of the present invention, rather than all embodiments.
The embodiment of the invention can be applied to an automatic driving system, a fatigue driving monitoring system or a vehicle related control system, can identify the behavior of a driver, and can determine whether the driver is in an unacknowledged state or not according to the head posture information of the driver in periods of time, thereby realizing timely reminding when the driver is in fatigue driving and preventing traffic accidents.
The following describes embodiments of the present application in detail.
Referring to fig. 1, fig. 1 is a schematic flow chart of a method for determining types of fatigue driving according to an embodiment of the present invention.
The invention provides a method for judging fatigue driving, which comprises the following steps:
s101, obtaining a target image containing a head image of a driver;
in the embodiment of the invention, a target image is obtained firstly, wherein the target image comprises a head image of a driver.
It will be appreciated that the target image is obtained in real time, and the target image may be a 3D image.
It will be appreciated that the target image may be obtained directly by the image capture device or may be obtained from a location maintained by the image capture device. And are not limited herein.
In the embodiment of the present invention, the driver may be a driver driving a vehicle or an operator controlling a machine, or the like, who needs fatigue driving recognition.
S102, determining head posture information of a driver in the target image;
after the target image is obtained, the head pose information of the driver in the target image is determined.
The head posture information represents the relevant information of each posture of the head of the driver. In the embodiment of the present invention, the head posture information may be euler angle information, and the euler angle information may include a pitch angle pitch, a left-right yaw angle yaw, and a rotation angle roll, which are represented by the three values P, Y, and R. The head posture of the driver can be characterized by the three values.
In the embodiment of the present invention, determining the head pose information of the driver may include at least two specific ways.
In an embodiment of the present invention, the determining the head pose information of the driver in the target image includes:
and inputting the target image as an input parameter into a deep neural network model to obtain an Euler angle parameter representing the head posture of the driver in the target image, and taking the Euler angle parameter as head posture information.
In the embodiment of the invention, the acquisition of the head posture information can be directly realized by utilizing the deep neural network model.
Assuming that the deep neural network model is represented by F, the head pose information O (P, Y, R) is represented by equation (1):
o (P, Y, R) ═ F (I) … … formula (1)
Another ways are described below.
The determining of the head pose information of the driver in the target image comprises:
inputting the target image as an input parameter into a deep neural network model to obtain a human face key point in the target image;
mapping the face key points to a head three-dimensional space;
and calculating Euler angle parameters representing the head posture of the driver in the target image in the head three-dimensional space according to the key points of the face, and taking the Euler angle parameters as head posture information.
In the embodiment of the invention, the acquisition of the head posture information can be realized by adopting a deep learning model.
In the embodiment of the invention, the human face key points in the target image can be obtained through the deep neural network model, wherein the human face key points can be points of positions such as eyes, nose, mouth and the like.
And after the key points of the human face are obtained, mapping the key points of the human face into a head three-dimensional space, wherein the head three-dimensional space is used for calculating Euler angle parameters P, Y and R.
The process can be expressed as follows by using the formulas (2) and (3):
K＝F*(I) … … formula (2)
O (P, Y, R) ═ Ψ (K) … … formula (3)
Wherein the neural network model is represented as F*The input image is represented as I, the face key points are represented as K, and the function for calculating the head pose information O (P, Y, R) using the key points is represented as Ψ.
S103, judging whether a characteristic sequence formed by the head characteristic information in a preset time period is characterized in that a driver is in an unconscious state or not, and obtaining a judgment result;
in the embodiment of the invention, when the driver is not awake, the driver can show series abnormal head postures in fixed time periods.
Referring to fig. 2, fig. 2 is a schematic diagram of the head posture of the driver in the non-awake state according to the embodiment of the present invention.
When the driver is in a fatigue state, an unconscious state such as drowsiness can occur, for example, at the th time t1, the head is still in a normal posture, at the second time t2, the head is slightly lowered due to extreme fatigue, at the third time t3, the head is lower and is kept , at the fourth time t4, the state is similar to that at the t3, and at the fifth time t5, the head is raised suddenly to return to a basically normal posture.
It should be understood that the schematic diagram in fig. 2 is only examples of the driver being in the non-waking state, and similarly, it is also possible to deviate backward or sideways at times t2, t3, and t4, and return to the normal posture at time t5, which is not described herein again.
In the embodiment of the invention, by utilizing the characteristics, whether the non-waking state is represented or not is judged by the characteristic sequence formed by the head characteristic information in the preset time period.
Wherein the characteristic sequence is a data sequence formed by continuously accumulating the head characteristic information according to a time dimension. The Euler angle parameters P, Y, R correspond to a characteristic sequence e.g. [ P ]1,p2,p3,...pn,...],[y1,y2,y3,...,yn,...],[r1,r2,r3,...,rn,...]。
In an embodiment of the present invention, it is determined whether the signature sequence can characterize the not-awake state as represented in fig. 2 above.
And S104, sending out reminding information when the judgment result represents that the driver is not awake.
In the embodiment of the invention, if the judgment result represents that the driver is in the uncleaned state, the reminding information is sent.
The reminding information can be an alarm instruction or a trigger instruction for triggering subsequent processes such as automatic driving and the like. As long as it can play a role in improving safe driving. And is not particularly limited herein.
Therefore, in the technical scheme of the invention, the driver can be determined to be in an unacknowledged state, namely fatigue driving, according to the head posture of the driver in a preset time period, so that the reminding information is sent to prevent traffic accidents as much as possible.
In the above embodiment, the process of determining whether the feature sequence formed by the head feature information in the preset time period is characterized that the driver is in the non-awake state is described, and the detailed description is provided below.
In an embodiment of the present invention, the determining whether the feature sequence formed by the head feature information in the preset time period is characterized in that the driver is in an awake state, and obtaining a determination result includes:
accumulating the head posture information in a preset time period into Euler angle sequence data;
performing sliding sampling on the Euler angle sequence data to obtain characteristic sequence data, wherein the characteristic sequence data is used for representing series actions made by the head of a driver in the preset time period;
and identifying the characteristic sequence data by using a cyclic memory neural network model to obtain a judgment result representing whether the driver is in an unacknowledged state or not.
In the embodiment of the invention, the head posture information is accumulated in the preset time period to obtain Euler angle sequence data.
For example, the euler angle sequence data corresponding to three values of euler angle parameters P, Y, R in a preset time period can be expressed as:
[p1,p2,p3,...pn,...],[y1,y2,y3,...,yn,...],[r1,r2,r3,...,rn,...]。
wherein, [ P1, P2, P3,. pn,. is sequence data corresponding to P in a preset time period, [ Y1, Y2, Y3,. yn,. is sequence data corresponding to Y in a preset time period, [ R1, R2, R3,. rn,. is sequence data corresponding to R in a preset time period.
After the euler angle sequence data are obtained, fixed time window T is set, and the sequence data corresponding to three values of the euler angle sequence data are slide sampled to obtain fixed length feature sequence data, such as [ p1, p2, p3,. pT ], [ y1, y2, y3,. pT ], [ r1, r2, r3,. rT ].
The feature sequence data can reflect the change of the head posture feature of the driver in a preset time period, so as to be used as a basis for judging whether the driver is in the non-waking state.
And then, identifying the characteristic sequence data by adopting a circulating memory neural network model to obtain a judgment result of whether the characteristic sequence data can be characterized as an unacknowledged state. For example, the determination result may be waking or not waking, or the determination result may be a drowsy or not drowsy result that can indicate the state of the driver.
The cyclic memory neural network model may be a Long Short-term memory model LSTM (Long Short-term memory) model, or an -controlled model gru (gated redundant unit) model.
Since there may be variations and instability when the feature sequence data is directly used as the input parameters of the recurrent memory neural network model, for example, when the driver is dozing, the head posture of the driver may not be linearly moved, and there is definite deviation, in the embodiment of the present invention, before the input, a process of performing multilayer convolution encoding on the feature sequence data is further included.
Specifically, before identifying the feature sequence data by using the recurrent memory neural network model, the method further includes:
carrying out multilayer convolution coding on the feature sequence data to obtain a feature sequence;
the identifying the feature sequence data using a recurrent memory neural network model comprises: and identifying the characteristic sequence by using a cyclic memory neural network model.
In the embodiment of the invention, in order to make the final result more accurate, multilayer convolutional coding is also carried out on the feature sequence data to realize local feature extraction, thereby being more beneficial to the output result of the circular memory network model to be more accurate.
The following description will be given taking, as an example, sequence data acquired during a specific time period T for the P value of the euler angle parameter. Is a reaction of [ p1,p2,p3,...pT]Generating a characteristic sequence [ p ] by multilayer convolution1,p2,p3,...pm]。
Carrying out multilayer convolutional coding according to a preset convolutional formula;
the preset convolution formula includes:
(p1,p2,p3,...,pm1)＝K1(p1,p2,p3,...,pT)
(p1,p2,p3,...,pm2)＝K2(p1,p2,p3,...,pm1)
......
(p1,p2,p3,...,pm)＝Ki(p1,p2,p3,...,pmd-1)
wherein, K is1Representing the convolution operation performed by layer for convolutional encoding, K2Represents the convolution operation performed by the second layer for convolutional encoding; kiRepresents the convolution operation performed by the i-th layer for convolutional encoding; said (p)1,p2,p3,...,pm1) Convolution signature sequence output for said th layer, said (p)1,p2,p3,...,pT) A data sequence corresponding to the parameter pith value of the Euler angle;
wherein each element values are calculated by adopting a preset element value formula, and the preset original value formula comprises the following steps:
wherein y is a feature sequence output by a rear layer, y (k) represents the value of the kth element in the feature sequence of the rear layer, u is a convolution kernel (a plurality of elements can be provided), N is the length of the convolution kernel, h is a data sequence of a front layer, and i is a positive integer.
In the embodiment of the invention, multilayer convolutional coding is carried out, so that the final result can be more accurate.
In the embodiment of the invention, the input cyclic memory neural network model does not directly input the sequence data as the input parameters, but the input is two large parts in consideration of independence and correlation, so that the final result is accurate in step .
Based on this, the identifying the sequence data by using the recurrent memory neural network model to obtain a judgment result representing whether the driver is in the state of not waking up includes:
respectively inputting characteristic sequences corresponding to every Euler angle parameter values in the characteristic sequences into a cyclic memory neural network model to obtain independent probability values of a plurality of the drivers in an unacknowledged state;
inputting a vector formed by characteristic sequences corresponding to every Euler angle parameter values in the characteristic sequences into a circular memory neural network model to obtain a related probability value of the driver in an unacknowledged state;
taking a calculation result obtained by carrying out weighted average or maximum value on the independent probability value and the related probability value as an output result;
and if the output result is greater than a preset threshold value, determining that the driver is in a non-waking state.
In the embodiment of the invention, the input parameters in the circular memory network model are divided into two large parts, wherein are considered to be independence, and are considered to be correlation.
In the embodiment of the present invention, the characteristic sequence corresponding to each euler angle parameter value is input as the part of independence, and the output result is an independent probability value corresponding to each euler angle parameter value.
The correlation is taken into consideration by inputs, and a vector [ p1, p2, p3,. till. pm, y1, y2, y3,. till. pm, r1, r2, r3,. till. rm ] composed of feature sequences corresponding to Euler angle parameter values is taken as an input parameter, and a correlation probability value is obtained.
And then carrying out weighted average calculation or maximum value calculation on the four probability values to obtain an output result. And judging whether the output result is larger than a preset threshold value or not, and if so, determining that the driver is in a non-waking state.
It can be understood that the learning of the data by the module circular memory unit can be realized by setting a loss function and updating unit parameters by utilizing a back propagation mechanism inherent in a neural network. The design of the loss function can adopt a common cross entropy loss function, a least square loss function and the like. The process refers to the prior art, and is not described in detail in this application.
According to the embodiment of the invention, the independent and relevant separate learning of the sequence data is realized, so that the final result is more accurate.
Based on the above embodiments, it can be seen that the method and the device can realize the output of whether the driver is in an unconscious state or not according to the obtained head posture information within the preset time period, and can prompt the driver or actively take over driving according to the prediction result, so as to effectively prevent traffic accidents caused by excessive fatigue.
Based on the understanding that the technical solution of the present invention or a part contributing to the prior art can be embodied in the form of a software product stored in storage media and including instructions for causing computer devices (which may be personal computers, servers, or network devices) to execute all or part of the steps of the method according to the embodiments of the present invention, and the foregoing storage media include various media capable of storing program codes, such as Read Only Memories (ROMs), Random Access Memories (RAMs), magnetic disks or optical disks, etc.
In addition, , the invention also discloses a device for judging types of fatigue driving, which corresponds to the embodiment of the method.
Referring to fig. 3, fig. 3 is a structural diagram of an fatigue driving determination device according to an embodiment of the present invention.
In an embodiment of the present invention, an fatigue driving determination device includes:
obtaining a device 1; for obtaining a target image containing an image of the driver's head;
a determination device 2; the head posture information of the driver in the target image is determined;
a judging device 3; the head feature information is used for judging whether a feature sequence formed by the head feature information in a preset time period is characterized in that a driver is in an unconscious state or not, and a judgment result is obtained;
a reminding device 4; and sending out reminding information when the judgment result represents that the driver is not awake.
Optionally, the determining module specifically includes:
an accumulation unit configured to accumulate the head posture information within a preset time period into euler angle sequence data;
the sampling unit is used for performing sliding sampling on the Euler angle sequence data to obtain characteristic sequence data, wherein the characteristic sequence data is used for representing series actions made by the head of a driver in the preset time period;
and the determining unit is used for identifying the characteristic sequence data by using a cyclic memory neural network model to obtain a judgment result representing whether the driver is in an unconscious state or not.
Optionally, the method further includes:
the coding unit is used for carrying out multilayer convolution coding on the feature sequence data to obtain a feature sequence;
the determining unit is specifically configured to: and identifying the characteristic sequence by using a cyclic memory neural network model.
It should be noted that, for implementation of each unit in the fatigue driving determination devices provided in the embodiments of the present invention, reference may be made to each step of the fatigue driving determination methods in the foregoing embodiments, which are not described herein again.
Compared with the prior art, the technical scheme provided by the embodiment of the invention firstly obtains the target image containing the head image of the driver; then determining the head posture information of the driver in the target image; then judging whether a characteristic sequence formed by the head characteristic information in a preset time period is characterized in that the driver is in an unconscious state or not to obtain a judgment result; and finally, sending out reminding information when the judgment result represents that the driver is not awake. Therefore, in the technical scheme of the invention, the driver can be determined to be in the non-waking state according to the head posture of the driver in the preset time period, so that the reminding information is sent to prevent traffic accidents as much as possible.
In addition , the invention also provides devices for judging fatigue driving.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an fatigue driving judgment device according to an embodiment of the present invention.
The invention provides kinds of fatigue driving judgment equipment, which comprises:
an image acquisition device 10, a processing device 20, and a display device 30;
the processing device 20 is respectively connected with the image acquisition device 10 and the display device 30;
the processing device 10 comprises a processor configured to perform:
obtaining a target image containing a head image of a driver;
determining head pose information of the driver 40 in the target image;
judging whether a characteristic sequence formed by the head characteristic information in a preset time period is characterized in that the driver is in an unconscious state or not to obtain a judgment result;
when the judgment result represents that the driver is in a non-waking state, sending out reminding information;
the image acquisition equipment is used for acquiring a target image containing a head image of a driver;
the display device is used for displaying the reminding information.
The image capturing device 10 may be a sensor such as a camera, and the display device 30 may be an on-vehicle display or a display screen.
It should be noted that the processor may include each module in the foregoing device embodiments, and the specific function implementation may refer to the foregoing embodiments. And will not be described in detail herein.
Compared with the prior art, the technical scheme provided by the embodiment of the invention firstly obtains the target image containing the head image of the driver; then determining the head posture information of the driver in the target image; then judging whether a characteristic sequence formed by the head characteristic information in a preset time period is characterized in that the driver is in an unconscious state or not to obtain a judgment result; and finally, sending out reminding information when the judgment result represents that the driver is not awake. Therefore, in the technical scheme of the invention, the driver can be determined to be in the non-waking state according to the head posture of the driver in the preset time period, so that the reminding information is sent to prevent traffic accidents as much as possible.
The above-described apparatus or system embodiments are merely illustrative, wherein the elements described as separate components may or may not be physically separate, and the elements shown as units may or may not be physical units, i.e., may be located in places, or may be distributed over a plurality of network elements.
For example, the division of the cells or sub-cells into logical functional divisions only, and other divisions may be made in practice, e.g., multiple cells or sub-cells in combination with . additionally, multiple cells or components may be or may be integrated into another system, or features may be omitted, or not implemented.
Additionally, the systems, devices and methods described, as well as the schematic illustrations of various embodiments, may be combined or integrated with other systems, modules, techniques or methods without departing from the scope of the present application at point , the mutual couplings or direct couplings or communicative connections shown or discussed may be electrical, mechanical or other forms of coupling or communicative connection through interfaces, devices or elements.
The foregoing is directed to embodiments of the present invention, and it is understood that various modifications and improvements can be made by those skilled in the art without departing from the spirit of the invention.
Claims (10)
1, A method for judging fatigue driving, comprising:
obtaining a target image containing a head image of a driver;
determining head posture information of a driver in the target image;
judging whether a characteristic sequence formed by the head characteristic information in a preset time period is characterized in that the driver is in an unconscious state or not to obtain a judgment result;
and sending out reminding information when the judgment result represents that the driver is not awake.
2. The method according to claim 1, wherein the determining head pose information of the driver in the target image comprises:
and inputting the target image as an input parameter into a deep neural network model to obtain an Euler angle parameter representing the head posture of the driver in the target image, and taking the Euler angle parameter as head posture information.
3. The method according to claim 1, wherein the determining head pose information of the driver in the target image comprises:
inputting the target image as an input parameter into a deep neural network model to obtain a human face key point in the target image;
mapping the face key points to a head three-dimensional space;
and calculating Euler angle parameters representing the head posture of the driver in the target image in the head three-dimensional space according to the key points of the face, and taking the Euler angle parameters as head posture information.
4. The method according to any of claims 1-3, wherein the determining whether the feature sequence constituted by the head feature information within the preset time period is characterized by that the driver is in an wakeful state, and obtaining the determination result comprises:
accumulating the head posture information in a preset time period into Euler angle sequence data;
performing sliding sampling on the Euler angle sequence data to obtain characteristic sequence data, wherein the characteristic sequence data is used for representing series actions made by the head of a driver in the preset time period;
and identifying the characteristic sequence data by using a cyclic memory neural network model to obtain a judgment result representing whether the driver is in an unacknowledged state or not.
5. The method according to claim 4, wherein the identifying the feature sequence data using the recurrent memory neural network model further comprises:
carrying out multilayer convolution coding on the feature sequence data to obtain a feature sequence;
the identifying the feature sequence data using a recurrent memory neural network model comprises: and identifying the characteristic sequence by using a cyclic memory neural network model.
6. The method of claim 5, wherein the identifying the sequence data using a recurrent memory neural network model to obtain a determination indicating whether the driver is not awake comprises:
respectively inputting characteristic sequences corresponding to every Euler angle parameter values in the characteristic sequences into a cyclic memory neural network model to obtain independent probability values of a plurality of the drivers in an unacknowledged state;
inputting a vector formed by characteristic sequences corresponding to every Euler angle parameter values in the characteristic sequences into a circular memory neural network model to obtain a related probability value of the driver in an unacknowledged state;
taking a calculation result obtained by carrying out weighted average or maximum value on the independent probability value and the related probability value as an output result;
and if the output result is greater than a preset threshold value, determining that the driver is in a non-waking state.
7, A fatigue driving determination device, comprising:
obtaining a device; for obtaining a target image containing an image of the driver's head;
a determination device; the head posture information of the driver in the target image is determined;
a judging device; the head feature information is used for judging whether a feature sequence formed by the head feature information in a preset time period is characterized in that a driver is in an unconscious state or not, and a judgment result is obtained;
a reminder device; and sending out reminding information when the judgment result represents that the driver is not awake.
8. The apparatus according to claim 7, wherein the determining module specifically includes:
an accumulation unit configured to accumulate the head posture information within a preset time period into euler angle sequence data;
the sampling unit is used for performing sliding sampling on the Euler angle sequence data to obtain characteristic sequence data, wherein the characteristic sequence data is used for representing series actions made by the head of a driver in the preset time period;
and the determining unit is used for identifying the characteristic sequence data by using a cyclic memory neural network model to obtain a judgment result representing whether the driver is in an unconscious state or not.
9. The apparatus according to claim 8, further comprising an encoding unit that:
the coding unit is used for carrying out multilayer convolution coding on the feature sequence data to obtain a feature sequence;
the determining unit is specifically configured to: and identifying the characteristic sequence by using a cyclic memory neural network model.
10, A fatigue driving judgment device, comprising:
the system comprises image acquisition equipment, processing equipment and display equipment;
the processing equipment is respectively connected with the image acquisition equipment and the display equipment;
the processing device includes a processor configured to perform:
obtaining a target image containing a head image of a driver;
determining head posture information of a driver in the target image;
judging whether a characteristic sequence formed by the head characteristic information in a preset time period is characterized in that the driver is in an unconscious state or not to obtain a judgment result;
when the judgment result represents that the driver is in a non-waking state, sending out reminding information;
the image acquisition equipment is used for acquiring a target image containing a head image of a driver;
the display device is used for displaying the reminding information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911030590.1A CN110738190A (en) | 2019-10-28 | 2019-10-28 | fatigue driving judgment method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911030590.1A CN110738190A (en) | 2019-10-28 | 2019-10-28 | fatigue driving judgment method, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110738190A true CN110738190A (en) | 2020-01-31 |
Family
ID=69271715
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911030590.1A Pending CN110738190A (en) | 2019-10-28 | 2019-10-28 | fatigue driving judgment method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110738190A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113320449A (en) * | 2021-07-19 | 2021-08-31 | 浙江吉利控股集团有限公司 | Driving seat automatic adjustment method, system, medium and electronic terminal |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107697069A (en) * | 2017-10-31 | 2018-02-16 | 上海汽车集团股份有限公司 | Fatigue of automobile driver driving intelligent control method |
| CN108961678A (en) * | 2018-04-26 | 2018-12-07 | 华慧视科技（天津）有限公司 | One kind being based on Face datection Study in Driver Fatigue State Surveillance System and its detection method |
| CN109620269A (en) * | 2019-01-28 | 2019-04-16 | 深圳市赛梅斯凯科技有限公司 | Fatigue detection method, device, equipment and readable storage medium storing program for executing |
| CN109726771A (en) * | 2019-02-27 | 2019-05-07 | 深圳市赛梅斯凯科技有限公司 | Abnormal driving detection model method for building up, device and storage medium |
| CN109770925A (en) * | 2019-02-03 | 2019-05-21 | 闽江学院 | A kind of fatigue detection method based on depth time-space network |
| CN109820525A (en) * | 2019-01-23 | 2019-05-31 | 五邑大学 | A kind of driving fatigue recognition methods based on CNN-LSTM deep learning model |
| CN109875568A (en) * | 2019-03-08 | 2019-06-14 | 北京联合大学 | A kind of head pose detection method for fatigue driving detection |
| CN109886241A (en) * | 2019-03-05 | 2019-06-14 | 天津工业大学 | Driver fatigue detection based on shot and long term memory network |
| CN110151203A (en) * | 2019-06-06 | 2019-08-23 | 常熟理工学院 | Fatigue driving recognition methods based on multistage avalanche type convolution Recursive Networks EEG analysis |
-
2019
- 2019-10-28 CN CN201911030590.1A patent/CN110738190A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107697069A (en) * | 2017-10-31 | 2018-02-16 | 上海汽车集团股份有限公司 | Fatigue of automobile driver driving intelligent control method |
| CN108961678A (en) * | 2018-04-26 | 2018-12-07 | 华慧视科技（天津）有限公司 | One kind being based on Face datection Study in Driver Fatigue State Surveillance System and its detection method |
| CN109820525A (en) * | 2019-01-23 | 2019-05-31 | 五邑大学 | A kind of driving fatigue recognition methods based on CNN-LSTM deep learning model |
| CN109620269A (en) * | 2019-01-28 | 2019-04-16 | 深圳市赛梅斯凯科技有限公司 | Fatigue detection method, device, equipment and readable storage medium storing program for executing |
| CN109770925A (en) * | 2019-02-03 | 2019-05-21 | 闽江学院 | A kind of fatigue detection method based on depth time-space network |
| CN109726771A (en) * | 2019-02-27 | 2019-05-07 | 深圳市赛梅斯凯科技有限公司 | Abnormal driving detection model method for building up, device and storage medium |
| CN109886241A (en) * | 2019-03-05 | 2019-06-14 | 天津工业大学 | Driver fatigue detection based on shot and long term memory network |
| CN109875568A (en) * | 2019-03-08 | 2019-06-14 | 北京联合大学 | A kind of head pose detection method for fatigue driving detection |
| CN110151203A (en) * | 2019-06-06 | 2019-08-23 | 常熟理工学院 | Fatigue driving recognition methods based on multistage avalanche type convolution Recursive Networks EEG analysis |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113320449A (en) * | 2021-07-19 | 2021-08-31 | 浙江吉利控股集团有限公司 | Driving seat automatic adjustment method, system, medium and electronic terminal |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11688203B2 (en) | Systems and methods for providing visual allocation management | |
| JP6264492B1 (en) | Driver monitoring device, driver monitoring method, learning device, and learning method | |
| US9714037B2 (en) | Detection of driver behaviors using in-vehicle systems and methods | |
| CN110291478A (en) | Driver's monitoring and response system | |
| CN110765807B (en) | Driving behavior analysis and processing method, device, equipment and storage medium | |
| CN104574817A (en) | Machine vision-based fatigue driving pre-warning system suitable for smart phone | |
| WO2008133746A1 (en) | Systems and methods for detecting unsafe conditions | |
| CN109190485A (en) | Data processing method, device, computer equipment and storage medium | |
| CN113460062A (en) | Driving behavior analysis system | |
| CN110738190A (en) | fatigue driving judgment method, device and equipment | |
| CN112083795A (en) | Object control method and device, storage medium and electronic equipment | |
| CN114194218A (en) | Optimization method, device, server, vehicle and medium for automatic driving system | |
| Vasudevan et al. | Driver drowsiness monitoring by learning vehicle telemetry data | |
| JP2023542058A (en) | Anomaly Detector for detecting anomalies using complementary classifiers | |
| Poon et al. | Driver distracted behavior detection technology with YOLO-based deep learning networks | |
| CN110837810A (en) | Face attention judging method, device, equipment and storage medium | |
| CN114312580B (en) | Method and device for determining seats of passengers in vehicle and vehicle control method and device | |
| Kouchak et al. | Non-intrusive distraction pattern detection using behavior triangulation method | |
| CN108446644A (en) | A kind of virtual display system for New-energy electric vehicle | |
| CN114760417A (en) | Image shooting method and device, electronic equipment and storage medium | |
| CN112215115A (en) | Method and related device for concentration capability evaluation in driving training | |
| CN115320626B (en) | Danger perception capability prediction method and device based on human-vehicle state and electronic equipment | |
| CN114056425B (en) | Vehicle steering control method and device, vehicle and storage medium | |
| CN111797784B (en) | Driving behavior monitoring method and device, electronic equipment and storage medium | |
| CN113326820B (en) | Driving environment sensing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 4 / F, building 1, No.14 Jiuxianqiao Road, Chaoyang District, Beijing 100020 Applicant after: Beijing Jingwei Hengrun Technology Co.,Ltd. Address before: 8 / F, block B, No. 11, Anxiang Beili, Chaoyang District, Beijing 100101 Applicant before: Beijing Jingwei HiRain Technologies Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200131 |
|
| RJ01 | Rejection of invention patent application after publication |