CN113792663A - Detection method and device for drunk driving and fatigue driving of driver and storage medium - Google Patents
Detection method and device for drunk driving and fatigue driving of driver and storage medium Download PDFInfo
- Publication number
- CN113792663A CN113792663A CN202111082202.1A CN202111082202A CN113792663A CN 113792663 A CN113792663 A CN 113792663A CN 202111082202 A CN202111082202 A CN 202111082202A CN 113792663 A CN113792663 A CN 113792663A
- Authority
- CN
- China
- Prior art keywords
- image
- face
- driver
- frame
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 77
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 claims abstract description 55
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 23
- 210000000697 sensory organ Anatomy 0.000 claims abstract description 23
- 238000005516 engineering process Methods 0.000 claims abstract description 21
- 230000005236 sound signal Effects 0.000 claims abstract description 19
- 238000013135 deep learning Methods 0.000 claims abstract description 14
- 210000001508 eye Anatomy 0.000 claims description 34
- 238000000605 extraction Methods 0.000 claims description 7
- 238000009432 framing Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 210000000887 face Anatomy 0.000 description 7
- 238000012549 training Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 206010039203 Road traffic accident Diseases 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 210000004204 blood vessel Anatomy 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000001815 facial effect Effects 0.000 description 3
- 239000008280 blood Substances 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 230000036772 blood pressure Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000003449 preventive effect Effects 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 1
- 208000003464 asthenopia Diseases 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 210000003296 saliva Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002195 synergetic effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 210000002700 urine Anatomy 0.000 description 1
- 208000029257 vision disease Diseases 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention provides a method, a device and a storage medium for detecting drunk driving and fatigue driving of a driver, wherein the method comprises the following steps: collecting a video image containing the face of a driver; performing frame processing on the video image, intercepting the face in each frame of image, and replacing the five sense organs in each frame of image by adopting a deep learning mode; building a drunk driving and fatigue driving detection model; acquiring physiological characteristics and eye state indexes by adopting an image processing technology; respectively acquiring alcohol concentration and a voice signal by adopting an alcohol sensor and a language sensor; and sending the acquired physiological characteristics, the eye state indexes, the alcohol concentration and the sound signals to a drunk driving and fatigue driving detection model for detection to obtain a detection result. The invention adopts a deep learning mode and combines a face changing technology to carry out privacy protection, introduces a language sensor and an alcohol sensor and constructs a new detection model to carry out auxiliary judgment, uses a sensor array to detect the direction prominence of a driver, and improves the judgment accuracy.
Description
Technical Field
The invention relates to the technical field of traffic safety detection, in particular to a method and a device for detecting drunk driving and fatigue driving of a driver and a storage medium.
Background
Investigations by the world health organization have shown that about 50% to 60% of traffic accidents are associated with drunk driving, which has become the first leading cause of traffic accidents. Research shows that drunk driving can cause the reduction of the touch, judgment and operation abilities of a driver, visual disorder, fatigue and the like, and the proportion of traffic accidents caused by drunk driving can be increased by 16 times compared with non-drunk driving. The method for detecting the alcohol content of the driver commonly used at present detects the alcohol content in exhaled air, saliva, urine and blood of the driver, and the detection of the former two belongs to qualitative detection and is generally used as a precondition for the detection of the latter two. Under most circumstances, wine drives and detects all to be the contact detection, need dispose out dynamic police force, detection efficiency and scope limitation are big, and need be detected the better cooperation of personnel to avoid appearing the condition of cheating, also can not accomplish preventive detection. The technology for detecting the alcohol content of the blood vessel by utilizing infrared rays is used as a non-contact detection technology, and the problem of partial contact detection is solved to a certain extent according to the principle that the alcohol content in blood of blood vessels on the surface of skin can influence the absorption amount of the blood vessels on near infrared light, the cooperation of a driver is still needed, and the identity of the driver cannot be determined under the unsupervised condition.
In addition, people pay great attention to privacy protection at the present stage, means for protecting the privacy of a driver to be tested are not implemented in the prior art, meanwhile, an image processing mode is mostly adopted for prediction in the prior art, and the mode also has the problem of misjudgment or missed judgment.
Disclosure of Invention
In light of the technical problems set forth above, a method, an apparatus, and a storage medium for detecting drunk driving and fatigue driving of a driver are provided. The invention adopts a deep learning mode and combines a face changing technology to carry out privacy protection, introduces a language sensor and an alcohol sensor and constructs a new detection model to carry out auxiliary judgment, uses a sensor array to detect the direction prominence of a driver, and improves the judgment accuracy. When a detection model is constructed, signals such as eyes, mouth shapes, skin colors and the like are processed, so that the fatigue driving degree is detected while drunk driving is detected;
the technical means adopted by the invention are as follows:
a detection method for drunk driving and fatigue driving of a driver comprises the following steps:
collecting a video image containing the face of a driver;
performing frame processing on the collected video images, intercepting the face in each frame of image, and replacing the five sense organs in each frame of image in a deep learning mode;
building a drunk driving and fatigue driving detection model;
acquiring physiological characteristics and eye state indexes based on each frame of image after replacing five sense organs by adopting an image processing technology;
respectively acquiring alcohol concentration and a voice signal by adopting an alcohol sensor and a language sensor;
and sending the obtained physiological characteristics, the eye state indexes, the alcohol concentration and the sound signals to the constructed drunk driving and fatigue driving detection model for detection to obtain a detection result.
Further, the manner of acquiring the video image including the face of the driver is to use an image sensor.
Further, the framing the acquired video image, capturing the face in each frame of image, and replacing the five sense organs in each frame of image by a deep learning method includes:
creating an original video area, detecting and extracting a face area, replacing the face area, storing a model, and outputting the video area;
the original video area is used for storing and recording original videos and images and converting the original videos into frame pictures for storage;
the detection and extraction of the face area adopts a model to extract and cut all face detection existing in each original frame picture into face images for storage;
the face replacing area is used for storing the image information of the face to be replaced;
the model storage area is used for storing the model used by the record and the trained model;
and the output video area replaces the human face through an algorithm and outputs the frame picture as a video for storage.
Further, the acquiring physiological characteristics and eye state indexes based on each frame of image after replacing five sense organs by adopting the image processing technology comprises:
intercepting a human face region of interest from each frame of image after replacing five sense organs, separating color channels of the human face region of interest, extracting an IPPG signal, and preprocessing the IPPG signal;
processing IPPG signals with the same time domain length, and calculating and extracting physiological characteristics;
and selecting the area where the eyes are located from the aligned human face of each frame, and carrying out region segmentation, edge extraction, gray level projection and template matching on the human face to position the human eyes so as to obtain the eye state index.
Further, the respectively acquiring the alcohol concentration and the sound signal by adopting the alcohol sensor and the language sensor comprises:
detecting the alcohol concentration by using an alcohol sensor, and if the alcohol is detected, flashing red and green lights at the same time; after 4 seconds, displaying concentration data, and only flashing a green light if the concentration is between 0.00 and 0.40; if the red light is larger than or equal to 0.50, only the red light flickers;
the voice sensor is adopted to detect whether the voice signal transmitted from the direction of the driver accords with normal logic and language, if not, the red light flickers, and if the voice signal transmitted again does not accord with the normal logic and language, the red light is normally on.
Further, the alcohol sensor employs a sensor array for detecting alcohol concentration of each directional source.
The invention also provides a detection device for drunk driving and fatigue driving of a driver, which is realized based on the detection method for drunk driving and fatigue driving of the driver and comprises the following steps:
the video image acquisition unit is used for acquiring a video image containing the face of the driver;
the privacy and identity protection unit is used for performing framing processing on the collected video images, intercepting the face in each frame of image and replacing the five sense organs in each frame of image in a deep learning mode;
the detection model construction unit is used for constructing drunk driving and fatigue driving detection models;
the characteristic index acquisition unit is used for acquiring physiological characteristics and eye state indexes based on each frame of image after the five sense organs are replaced by adopting an image processing technology;
the alcohol concentration and sound signal acquisition unit is used for respectively acquiring the alcohol concentration and the sound signal by adopting an alcohol sensor and a language sensor;
and the detection unit is used for sending the acquired physiological characteristics, the eye state indexes, the alcohol concentration and the sound signals to the constructed drunk driving and fatigue driving detection model for detection to obtain a detection result.
The present invention also provides a computer-readable storage medium having a set of computer instructions stored therein; the computer instruction set realizes the detection method for drunk driving and fatigue driving of the driver when being executed by the processor.
Compared with the prior art, the invention has the following advantages:
1. according to the detection method for drunk driving and fatigue driving of the driver, privacy protection is performed by combining a deep learning mode with a face changing technology, and the use safety and confidentiality of products are improved.
2. According to the detection method for drunk driving and fatigue driving of the driver, the language sensor and the alcohol sensor are introduced to obtain signals, the signals are transmitted into the model, a brand-new detection model is designed by combining image processing results, the purpose of improving the accuracy of the prior art is achieved, the sensor array is used, the direction of the driver is detected in a protruding mode, and the judgment accuracy is improved.
3. According to the method for detecting drunk driving and fatigue driving of the driver, when the detection model is constructed, signals such as eyes, mouth shapes and skin colors are processed, so that the drunk driving is detected and the fatigue driving degree is detected.
Based on the reasons, the invention can be widely popularized in the fields of traffic safety detection and the like.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flow chart of the method of the present invention.
FIG. 2 is a schematic diagram of an alcohol sensor array of the apparatus of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
The relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise. Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate. Any specific values in all examples shown and discussed herein are to be construed as exemplary only and not as limiting. Thus, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
In the description of the present invention, it is to be understood that the orientation or positional relationship indicated by the directional terms such as "front, rear, upper, lower, left, right", "lateral, vertical, horizontal" and "top, bottom", etc., are generally based on the orientation or positional relationship shown in the drawings, and are used for convenience of description and simplicity of description only, and in the absence of any contrary indication, these directional terms are not intended to indicate and imply that the device or element so referred to must have a particular orientation or be constructed and operated in a particular orientation, and therefore should not be considered as limiting the scope of the present invention: the terms "inner and outer" refer to the inner and outer relative to the profile of the respective component itself.
Spatially relative terms, such as "above … …," "above … …," "above … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial relationship to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, devices described as "above" or "on" other devices or configurations would then be oriented "below" or "under" the other devices or configurations. Thus, the exemplary term "above … …" can include both an orientation of "above … …" and "below … …". The device may be otherwise variously oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.
It should be noted that the terms "first", "second", and the like are used to define the components, and are only used for convenience of distinguishing the corresponding components, and the terms have no special meanings unless otherwise stated, and therefore, the scope of the present invention should not be construed as being limited.
As shown in fig. 1, the invention provides a method for detecting drunk driving and fatigue driving of a driver, comprising the following steps:
collecting a video image containing the face of a driver;
performing frame processing on the acquired video images, intercepting the face in each frame of image, and replacing five sense organs (keeping the original index of eyes) in each frame of image by adopting a deep learning mode for protecting the privacy and identity of the tested person;
building a drunk driving and fatigue driving detection model;
acquiring physiological characteristics and eye state indexes based on each frame of image after replacing five sense organs by adopting an image processing technology;
respectively acquiring alcohol concentration and a voice signal by adopting an alcohol sensor and a language sensor;
and sending the obtained physiological characteristics, the eye state indexes, the alcohol concentration and the sound signals to the constructed drunk driving and fatigue driving detection model for detection to obtain a detection result.
In specific implementation, as a preferred embodiment of the present invention, the manner of acquiring the video image including the face of the driver is to use an image sensor.
In specific implementation, as a preferred embodiment of the present invention, the framing the acquired video images, capturing the face in each frame of image, and replacing five sense organs in each frame of image by using a deep learning method includes:
s101, creating an original video area, detecting and extracting a face area, replacing the face area, storing a model, and outputting the video area;
s102, the original video area is used for storing and recording original videos and images, and converting the original videos into frame pictures for storage;
s103, detecting and extracting a face area, and extracting and cutting all face detection existing in each original frame picture into face images by adopting a model for storage;
s104, the face replacing area is used for storing the picture information of the face to be replaced and is used for secondary training;
s105, the model storage area is used for storing the used models and the trained models;
and S106, replacing the human face by the output video area through an algorithm, and outputting the frame picture as a video for storage.
In this embodiment, the specific implementation processes of the original video area, the detection and extraction of the face area, the replacement of the face area, the model storage area, and the output video area are as follows:
original video zone: the original video is converted into a frame picture by using the opencv-python library of python and the Image library of PIL, and is stored, and a common video is extracted to 24 frames or 30 frames per second.
Detecting and extracting a face area: the face extraction method of the S3FD model is adopted to locate and extract a plurality of faces, and cut the faces at the same time, because the sizes of pictures are different, a plurality of faces or no faces may exist at the same time, only frames with faces can be extracted, and invalid frames are ignored. Since these invalid frames are invalid during the final composition of the video without any effect.
Aligning the human face: the 2DFAN algorithm is adopted to extract facial landmarks of a standard posture, and the PRNet algorithm is adopted to extract facial landmarks of a side face. The information extracted by the face alignment comprises elevation angle and face heat map. The face alignment is better facilitated, and the replacement is more accurate.
Shielding by a mask: for common facial obstructions such as hands, lipsticks, glasses and hair, effective elimination is needed during replacement, and by means of an image segmentation technology, only a mask required for designation is generated for a picture, and the mask and the picture are operated to only reserve an interested area and an area to be replaced. The interested mask parts mainly comprise the information of eyes, nose and mouth, and the information is effectively replaced to remove the shelters through training the effective model, so that a better replacement effect is obtained.
Replacing the face area: the GAN training model is used, and for a classical DF structure, the LIAE structure is used, so that the structure is stronger in adaptability to strong light, and more vivid replacement can be achieved. The LIAE structure is used for generating potential face information of an original image and a target image through the InterAB, and only generating an output target image through the InterB, so that weight information of the target image is shared, illumination information of the original image is kept, and a good replacing effect is still achieved even in a strong light environment.
For such a complex image as a human face, liveliness of eyes is very important for identifying authenticity and health detection information at a later stage. Therefore, in this embodiment, the SSIM technology is adopted, and training weight for the positions of the eyes is increased, so that the reality of the eyes is concerned more in the training process, more vivid and lively eyes are obtained, and a more vivid target image is obtained.
In this embodiment, a Reinhard Color Transfer (RCT) technology and a Poisson blending technology are also used in the face replacement process, so that the face replacement is not so hard. The fuzzification process not only makes the replacement more vivid, but also does not destroy the original picture information.
In specific implementation, as a preferred embodiment of the present invention, the acquiring physiological characteristics and eye state indexes based on each frame of image after replacing five sense organs by using an image processing technology includes:
s201, intercepting a human face region of interest from each frame of image after replacing five sense organs, separating color channels of the human face region of interest, extracting an IPPG signal, and preprocessing the IPPG signal;
s202, processing IPPG signals with the same time domain length, and calculating and extracting physiological characteristics (heart rate, blood pressure, heart rate variation and the like);
s203, selecting an area where the eyes are located from the aligned human faces of each frame, and carrying out human eye positioning by carrying out region segmentation, edge extraction, gray projection and template matching on the human faces to obtain eye state indexes; and analyzing the eye state index to obtain whether the fatigue driving condition exists or not.
In a specific implementation, as a preferred embodiment of the present invention, the acquiring the alcohol concentration and the sound signal by using the alcohol sensor and the speech sensor respectively includes:
s301, detecting the alcohol concentration by using an alcohol sensor, and if the alcohol is detected, flashing red and green lights at the same time; after 4 seconds, displaying concentration data, and only flashing a green light if the concentration is between 0.00 and 0.40; if the red light is larger than or equal to 0.50, only the red light flickers;
s302, a language sensor is adopted to detect whether the sound signal transmitted from the direction of the driver accords with normal logic and language, if not, the red light flickers, and if the sound signal transmitted again does not accord with the normal logic and language, the red light is normally on. In the embodiment, whether the condition belongs to drunk driving or fatigue driving is specifically detected by training the detection sound signal, and the index is added to a drunk driving classification prediction model pre-designed by the system.
In a specific implementation, as a preferred embodiment of the present invention, the alcohol sensor employs a sensor array for detecting alcohol concentrations from various directions, so as to more clearly determine whether a drinker is a driver and avoid misjudgment. The sensor array is shown in fig. 2.
In summary, according to the detection method for drunk driving and fatigue driving of the driver provided by the invention, five sense organs are replaced by a deep learning manner to protect the privacy of the tested person, a face ROI is selected from the aligned face, an IPPG signal is extracted from the ROI for preprocessing, and then the IPPG signals with the same time domain length are processed respectively to obtain physiological characteristics (heart rate, blood pressure, heart rate variation and the like). The invention structurally considers the fusion detection idea of multiple sensors, introduces an alcohol sensor and a language sensor, judges through the alcohol content in the vehicle and the sound transmitted from the direction of the driver, and respectively plays the detection role of the multiple sensors under the synergistic effect, and finally judges whether the driver is drunk or not and the drunk driving level. The accuracy is further ensured by the form of the sensor array. And the safe and sanitary preventive drunk driving detection is realized by combining a non-contact acquisition technology. The method is simple and convenient to operate, safe and sanitary, can effectively improve economic benefits and detection efficiency, and can realize predictive, real-time and retrospective drunk driving detection.
Corresponding to the detection method for drunk driving and fatigue driving of the driver in the application, the application also provides a detection device for drunk driving and fatigue driving of the driver, which comprises the following steps:
the video image acquisition unit is used for acquiring a video image containing the face of the driver;
the privacy and identity protection unit is used for performing framing processing on the collected video images, intercepting the face in each frame of image and replacing the five sense organs in each frame of image in a deep learning mode;
the detection model construction unit is used for constructing drunk driving and fatigue driving detection models;
the characteristic index acquisition unit is used for acquiring physiological characteristics and eye state indexes based on each frame of image after the five sense organs are replaced by adopting an image processing technology;
the alcohol concentration and sound signal acquisition unit is used for respectively acquiring the alcohol concentration and the sound signal by adopting an alcohol sensor and a language sensor;
and the detection unit is used for sending the acquired physiological characteristics, the eye state indexes, the alcohol concentration and the sound signals to the constructed drunk driving and fatigue driving detection model for detection to obtain a detection result.
For the embodiments of the present invention, the description is simple because it corresponds to the above embodiments, and for the related similarities, please refer to the description in the above embodiments, and the detailed description is omitted here.
The embodiment of the application also discloses a computer-readable storage medium, wherein a computer instruction set is stored in the computer-readable storage medium, and when being executed by a processor, the computer instruction set realizes the detection method for drunk driving and fatigue driving of the driver, which is provided by any one of the above embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. A detection method for drunk driving and fatigue driving of a driver is characterized by comprising the following steps:
collecting a video image containing the face of a driver;
performing frame processing on the acquired video images, intercepting the face in each frame of image, and replacing the five sense organs (keeping the original indexes of eyes) in each frame of image in a deep learning mode;
building a drunk driving and fatigue driving detection model;
acquiring physiological characteristics and eye state indexes based on each frame of image after replacing five sense organs by adopting an image processing technology;
respectively acquiring alcohol concentration and a voice signal by adopting an alcohol sensor and a language sensor;
and sending the obtained physiological characteristics, the eye state indexes, the alcohol concentration and the sound signals to the constructed drunk driving and fatigue driving detection model for detection to obtain a detection result.
2. The method as claimed in claim 1, wherein the video image including the face of the driver is captured by an image sensor.
3. The method for detecting drunk driving and fatigue driving of a driver as claimed in claim 1, wherein the step of framing the collected video images and capturing the face in each frame of image and replacing five sense organs in each frame of image by deep learning comprises:
creating an original video area, detecting and extracting a face area, replacing the face area, storing a model, and outputting the video area;
the original video area is used for storing and recording original videos and images and converting the original videos into frame pictures for storage;
the detection and extraction of the face area adopts a model to extract and cut all face detection existing in each original frame picture into face images for storage;
the face replacing area is used for storing the image information of the face to be replaced;
the model storage area is used for storing the model used by the record and the trained model;
and the output video area replaces the human face through an algorithm and outputs the frame picture as a video for storage.
4. The method for detecting drunk driving and fatigue driving of a driver as claimed in claim 1, wherein the acquiring physiological characteristics and eye state indexes based on each frame of image after replacing five sense organs by using an image processing technology comprises:
intercepting a human face region of interest from each frame of image after replacing five sense organs, separating color channels of the human face region of interest, extracting an IPPG signal, and preprocessing the IPPG signal;
processing IPPG signals with the same time domain length, and calculating and extracting physiological characteristics;
and selecting the area where the eyes are located from the aligned human face of each frame, and carrying out region segmentation, edge extraction, gray level projection and template matching on the human face to position the human eyes so as to obtain the eye state index.
5. The method for detecting drunk driving and fatigue driving of a driver as claimed in claim 1, wherein the step of respectively acquiring the alcohol concentration and the sound signal by using the alcohol sensor and the speech sensor comprises:
detecting the alcohol concentration by using an alcohol sensor, and if the alcohol is detected, flashing red and green lights at the same time; after 4 seconds, displaying concentration data, and only flashing a green light if the concentration is between 0.00 and 0.40; if the red light is larger than or equal to 0.50, only the red light flickers;
the voice sensor is adopted to detect whether the voice signal transmitted from the direction of the driver accords with normal logic and language, if not, the red light flickers, and if the voice signal transmitted again does not accord with the normal logic and language, the red light is normally on.
6. The method as claimed in claim 5, wherein the alcohol sensor is a sensor array for detecting alcohol concentration of each direction source.
7. A detection device for drunk driving and fatigue driving of a driver, which is realized based on the detection method for drunk driving and fatigue driving of the driver as claimed in any one of claims 1-6, and comprises the following steps:
the video image acquisition unit is used for acquiring a video image containing the face of the driver;
the privacy and identity protection unit is used for performing framing processing on the collected video images, intercepting the face in each frame of image and replacing the five sense organs in each frame of image in a deep learning mode;
the detection model construction unit is used for constructing drunk driving and fatigue driving detection models;
the characteristic index acquisition unit is used for acquiring physiological characteristics and eye state indexes based on each frame of image after the five sense organs are replaced by adopting an image processing technology;
the alcohol concentration and sound signal acquisition unit is used for respectively acquiring the alcohol concentration and the sound signal by adopting an alcohol sensor and a language sensor;
and the detection unit is used for sending the acquired physiological characteristics, the eye state indexes, the alcohol concentration and the sound signals to the constructed drunk driving and fatigue driving detection model for detection to obtain a detection result.
8. A computer-readable storage medium having a set of computer instructions stored therein; the set of computer instructions, when executed by a processor, implement the method of detecting drunk driving and fatigue driving of a driver as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111082202.1A CN113792663A (en) | 2021-09-15 | 2021-09-15 | Detection method and device for drunk driving and fatigue driving of driver and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111082202.1A CN113792663A (en) | 2021-09-15 | 2021-09-15 | Detection method and device for drunk driving and fatigue driving of driver and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113792663A true CN113792663A (en) | 2021-12-14 |
Family
ID=78878549
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111082202.1A Pending CN113792663A (en) | 2021-09-15 | 2021-09-15 | Detection method and device for drunk driving and fatigue driving of driver and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113792663A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115482933A (en) * | 2022-11-01 | 2022-12-16 | 北京鹰瞳科技发展股份有限公司 | Method for assessing driving risk of driver and related product |
| CN117333927A (en) * | 2023-12-01 | 2024-01-02 | 厦门磁北科技有限公司 | Vehicle-mounted face recognition alcohol detection method and system |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229245A (en) * | 2016-12-14 | 2018-06-29 | 贵港市瑞成科技有限公司 | Method for detecting fatigue driving based on facial video features |
| DE102017201328A1 (en) * | 2017-01-27 | 2018-08-02 | Bayerische Motoren Werke Aktiengesellschaft | Method for determining a facial expression |
| CN110001652A (en) * | 2019-03-26 | 2019-07-12 | 深圳市科思创动科技有限公司 | Monitoring method, device and the terminal device of driver status |
| CN110276273A (en) * | 2019-05-30 | 2019-09-24 | 福建工程学院 | Merge the Driver Fatigue Detection of facial characteristics and the estimation of image pulse heart rate |
| CN110751051A (en) * | 2019-09-23 | 2020-02-04 | 江苏大学 | Abnormal driving behavior detection method based on machine vision |
| CN111281367A (en) * | 2018-12-10 | 2020-06-16 | 绍兴图聚光电科技有限公司 | Anti-interference non-contact heart rate detection method based on face video |
| CN112101103A (en) * | 2020-08-07 | 2020-12-18 | 东南大学 | Video driver fatigue detection method based on deep integration network |
| CN112401857A (en) * | 2020-11-23 | 2021-02-26 | 杭州艺兴科技有限公司 | Driver drunk driving detection method |
| CN112735440A (en) * | 2020-12-30 | 2021-04-30 | 北京瞰瞰科技有限公司 | Vehicle-mounted intelligent robot interaction method, robot and vehicle |
| CN113066263A (en) * | 2020-01-02 | 2021-07-02 | 中车株洲电力机车研究所有限公司 | Method and device for preventing fatigue driving |
| US20210209388A1 (en) * | 2020-01-06 | 2021-07-08 | The Research Foundation For The State University Of New York | Fakecatcher: detection of synthetic portrait videos using biological signals |
| CN113139527A (en) * | 2021-05-28 | 2021-07-20 | 深圳文达智通技术有限公司 | Video privacy protection method, device, equipment and storage medium |
-
2021
- 2021-09-15 CN CN202111082202.1A patent/CN113792663A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229245A (en) * | 2016-12-14 | 2018-06-29 | 贵港市瑞成科技有限公司 | Method for detecting fatigue driving based on facial video features |
| DE102017201328A1 (en) * | 2017-01-27 | 2018-08-02 | Bayerische Motoren Werke Aktiengesellschaft | Method for determining a facial expression |
| CN111281367A (en) * | 2018-12-10 | 2020-06-16 | 绍兴图聚光电科技有限公司 | Anti-interference non-contact heart rate detection method based on face video |
| CN110001652A (en) * | 2019-03-26 | 2019-07-12 | 深圳市科思创动科技有限公司 | Monitoring method, device and the terminal device of driver status |
| CN110276273A (en) * | 2019-05-30 | 2019-09-24 | 福建工程学院 | Merge the Driver Fatigue Detection of facial characteristics and the estimation of image pulse heart rate |
| CN110751051A (en) * | 2019-09-23 | 2020-02-04 | 江苏大学 | Abnormal driving behavior detection method based on machine vision |
| CN113066263A (en) * | 2020-01-02 | 2021-07-02 | 中车株洲电力机车研究所有限公司 | Method and device for preventing fatigue driving |
| US20210209388A1 (en) * | 2020-01-06 | 2021-07-08 | The Research Foundation For The State University Of New York | Fakecatcher: detection of synthetic portrait videos using biological signals |
| CN112101103A (en) * | 2020-08-07 | 2020-12-18 | 东南大学 | Video driver fatigue detection method based on deep integration network |
| CN112401857A (en) * | 2020-11-23 | 2021-02-26 | 杭州艺兴科技有限公司 | Driver drunk driving detection method |
| CN112735440A (en) * | 2020-12-30 | 2021-04-30 | 北京瞰瞰科技有限公司 | Vehicle-mounted intelligent robot interaction method, robot and vehicle |
| CN113139527A (en) * | 2021-05-28 | 2021-07-20 | 深圳文达智通技术有限公司 | Video privacy protection method, device, equipment and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| JAVIER HERNANDEZ-ORTEGA等: "DeepFakesON-Phys: DeepFakes Detection based on Heart Rate Estimation", 《ARXIV》, pages 1 - 8 * |
| PEIPENG YU等: "A Survey on Deepfake Video Detection", 《IET BIOMETRICS》, vol. 10, no. 6, pages 607 - 624, XP006112790, DOI: 10.1049/bme2.12031 * |
| UMUR AYBARS CIFTCI等: "FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals", 《ARXIV》, pages 1 - 17 * |
| 邓雄等: "人脸识别活体检测研究方法综述", 《计算机应用研究》, vol. 37, no. 09, pages 2579 - 2585 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115482933A (en) * | 2022-11-01 | 2022-12-16 | 北京鹰瞳科技发展股份有限公司 | Method for assessing driving risk of driver and related product |
| CN115482933B (en) * | 2022-11-01 | 2023-11-28 | 北京鹰瞳科技发展股份有限公司 | Method for evaluating driving risk of driver and related product thereof |
| CN117333927A (en) * | 2023-12-01 | 2024-01-02 | 厦门磁北科技有限公司 | Vehicle-mounted face recognition alcohol detection method and system |
| CN117333927B (en) * | 2023-12-01 | 2024-04-16 | 厦门磁北科技有限公司 | Vehicle-mounted face recognition alcohol detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113792663A (en) | Detection method and device for drunk driving and fatigue driving of driver and storage medium | |
| CN101593425B (en) | Machine vision based fatigue driving monitoring method and system | |
| Su et al. | Does “lie to me” lie to you? An evaluation of facial clues to high-stakes deception | |
| JP5064413B2 (en) | Method and apparatus for automatically detecting spectacles using nasal muscle mask | |
| CN108334848A (en) | A kind of small face identification method based on generation confrontation network | |
| CN108764071A (en) | It is a kind of based on infrared and visible images real human face detection method and device | |
| JP2001167110A (en) | Picture retrieving method and its device | |
| CN109190535B (en) | Face complexion analysis method and system based on deep learning | |
| CN106372629A (en) | Living body detection method and device | |
| CN111126366B (en) | Method, device, equipment and storage medium for distinguishing living human face | |
| Jiang et al. | Rdfnet: A fast caries detection method incorporating transformer mechanism | |
| CN114973412A (en) | Lip language identification method and system | |
| CN109359577A (en) | A kind of Complex Background number detection system based on machine learning | |
| CN112401857A (en) | Driver drunk driving detection method | |
| EP1820158B1 (en) | Album creating apparatus, album creating method, and album creating program | |
| CN114093501A (en) | Intelligent auxiliary analysis method for children's motor epilepsy based on synchronous video and electroencephalogram | |
| Yaccob et al. | Recognizing facial expressions by spatio-temporal analysis | |
| CN116129129A (en) | Character interaction detection model and detection method | |
| CN110110606A (en) | The fusion method of visible light neural network based and infrared face image | |
| CN112861588B (en) | Living body detection method and device | |
| CN106951897A (en) | A kind of inclined degree determination methods of medical document photograph | |
| CN114764944A (en) | Pupil positioning method and device based on angular point detection | |
| CN111339972A (en) | Face image detection method and device, electronic equipment and storage medium | |
| KR20210099489A (en) | Method and Apparatus for Driver Drowsiness Detection with optimized pre-processing for eyelid-closure classification using SVM | |
| CN113536843A (en) | Anti-counterfeiting face recognition system based on multi-mode fusion convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |