CN113792663B - Method, device and storage medium for detecting drunk driving and fatigue driving of driver - Google Patents
Method, device and storage medium for detecting drunk driving and fatigue driving of driver Download PDFInfo
- Publication number
- CN113792663B CN113792663B CN202111082202.1A CN202111082202A CN113792663B CN 113792663 B CN113792663 B CN 113792663B CN 202111082202 A CN202111082202 A CN 202111082202A CN 113792663 B CN113792663 B CN 113792663B
- Authority
- CN
- China
- Prior art keywords
- face
- detection
- driver
- frame
- sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000001514 detection method Methods 0.000 claims abstract description 68
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 claims abstract description 55
- 230000005236 sound signal Effects 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 20
- 210000000697 sensory organ Anatomy 0.000 claims abstract description 20
- 238000005516 engineering process Methods 0.000 claims abstract description 19
- 238000013135 deep learning Methods 0.000 claims abstract description 13
- 238000009432 framing Methods 0.000 claims abstract description 5
- 210000001508 eye Anatomy 0.000 claims description 32
- 210000000887 face Anatomy 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 6
- 230000036651 mood Effects 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000012549 training Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 206010039203 Road traffic accident Diseases 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 239000008280 blood Substances 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 230000036772 blood pressure Effects 0.000 description 2
- 210000004204 blood vessel Anatomy 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000003449 preventive effect Effects 0.000 description 2
- 206010047571 Visual impairment Diseases 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 208000003464 asthenopia Diseases 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 210000003296 saliva Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 210000002700 urine Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a method, a device and a storage medium for detecting drunk driving and fatigue driving of a driver, wherein the method comprises the following steps: collecting video images containing faces of drivers; carrying out framing treatment on the video image, intercepting a human face in each frame of image, and replacing five sense organs in each frame of image by adopting a deep learning mode; constructing a drunk driving and fatigue driving detection model; acquiring physiological characteristics and eye state indexes by adopting an image processing technology; respectively acquiring alcohol concentration and sound signals by adopting an alcohol sensor and a language sensor; and sending the acquired physiological characteristics, eye state indexes, alcohol concentration and sound signals to a drunk driving and fatigue driving detection model for detection, and obtaining a detection result. The invention adopts a deep learning mode and combines a face changing technology to carry out privacy protection, introduces a language sensor and an alcohol sensor, constructs a new detection model to carry out auxiliary judgment, uses a sensor array, and improves judgment accuracy aiming at the prominent detection of the direction of a driver.
Description
Technical Field
The invention relates to the technical field of traffic safety detection, in particular to a method, a device and a storage medium for detecting drunk driving and fatigue driving of a driver.
Background
Investigation by the world health organization shows that about 50% to 60% of traffic accidents are related to drunk driving, and drunk driving has become the first cause of traffic accidents. Research shows that drunk driving can cause the touch, judgment and operation capability of a driver to be reduced, and the situations of visual disturbance, fatigue and the like appear, and the proportion of traffic accidents caused by drunk driving can be increased by 16 times compared with non-drunk driving. The current common method for detecting the alcohol content of a driver is to detect the alcohol content in the expired air, saliva, urine and blood, wherein the detection of the former two belongs to qualitative detection, and is generally used as a precondition for the detection of the latter two. Under most conditions, drunk driving detection is contact type detection, power police needs to be deployed, the detection efficiency and the range are high in limitation, and detection personnel need to be matched well so as to avoid cheating, and preventive detection cannot be achieved. The technology for detecting the alcohol content of the blood vessel by utilizing the infrared rays is used as a non-contact detection technology, the problem of partial contact detection is solved to a certain extent according to the principle that the alcohol content in the blood of the blood vessel on the surface of the skin can influence the absorption amount of the near infrared light, the driver is still required to cooperate, and the identity of the driver cannot be determined under the condition of no supervision.
In addition, people attach great importance to privacy protection at present, no means for protecting the privacy of the tested driver is implemented in the prior art, and meanwhile, most of the prior art adopts an image processing mode for prediction, and the mode also has the problem of misjudgment or missed judgment.
Disclosure of Invention
According to the technical problems, the method, the device and the storage medium for detecting drunk driving and fatigue driving of a driver are provided. The invention adopts a deep learning mode and combines a face changing technology to carry out privacy protection, introduces a language sensor and an alcohol sensor, constructs a new detection model to carry out auxiliary judgment, uses a sensor array, and improves judgment accuracy aiming at the prominent detection of the direction of a driver. When a detection model is constructed, signals such as eyes, mouth shapes, skin colors and the like are processed, so that the fatigue driving degree is detected while drunk driving is detected;
The invention adopts the following technical means:
a detection method for drunk driving and fatigue driving of a driver comprises the following steps:
collecting video images containing faces of drivers;
Carrying out framing treatment on the acquired video images, intercepting human faces in each frame of images, and replacing five sense organs in each frame of images in a deep learning mode;
constructing a drunk driving and fatigue driving detection model;
Acquiring physiological characteristics and eye state indexes based on each frame of image after the five sense organs are replaced by adopting an image processing technology;
Respectively acquiring alcohol concentration and sound signals by adopting an alcohol sensor and a language sensor;
and sending the obtained physiological characteristics, eye state indexes, alcohol concentration and sound signals to a constructed drunk driving and fatigue driving detection model for detection, and obtaining a detection result.
Further, the method for acquiring the video image containing the face of the driver adopts an image sensor.
Further, the frame processing is performed on the collected video images, faces in each frame of images are intercepted, and the five sense organs in each frame of images are replaced by adopting a deep learning mode, including:
creating an original video area, detecting and extracting a face area, replacing the face area, storing a model, and outputting the video area;
the original video area is used for storing and recording original videos and images, and converting the original videos into frame pictures for storage;
the face area is detected and extracted, and all face detection and extraction of each original frame picture are cut into face images by adopting a model to store the face images;
The face replacement area is used for storing picture information of the face to be replaced;
the model storage area is used for storing a model used for recording and a trained model;
And the output video area replaces the human face through an algorithm, and outputs the frame picture as a video for storage.
Further, the acquiring the physiological characteristic and the eye state index based on each frame of image after the five sense organs are replaced by adopting the image processing technology comprises the following steps:
Intercepting a human face region of interest in each frame of image after the five sense organs are replaced, separating color channels of the human face region of interest, extracting IPPG signals, and preprocessing;
processing IPPG signals with the same time domain length, and calculating and extracting physiological characteristics;
And selecting an area where eyes are located from the face after each frame alignment, and performing eye positioning by performing area segmentation, edge extraction, gray level projection and template matching on the face to obtain an eye state index.
Further, the method for acquiring the alcohol concentration and the sound signal by using the alcohol sensor and the language sensor respectively includes:
Detecting the alcohol concentration by adopting an alcohol sensor, and flashing a traffic light and a green light at the same time if alcohol is detected; after 4 seconds, displaying concentration data, and only flashing green light if the concentration is between 0.00 and 0.40; if the light quantity is more than or equal to 0.50, only the red light flashes;
The voice sensor is used for detecting whether the voice signal transmitted from the direction of the driver accords with normal logic and mood, if the voice signal does not accord with the normal logic and mood, the red light flashes, and if the voice signal transmitted again does not accord with the voice signal, the red light is always on.
Further, the alcohol sensor adopts a sensor array for detecting the alcohol concentration of the source in each direction.
The invention also provides a device for detecting drunk driving and fatigue driving of the driver, which is realized based on the method for detecting drunk driving and fatigue driving of the driver, and comprises the following steps:
The video image acquisition unit is used for acquiring video images containing the faces of the driver;
The privacy and identity protection unit is used for carrying out framing treatment on the acquired video images, intercepting the face in each frame of image and replacing the five sense organs in each frame of image in a deep learning mode;
the detection model construction unit is used for constructing drunk driving and fatigue driving detection models;
The feature index obtaining unit is used for obtaining physiological features and eye state indexes based on each frame of images after the five sense organs are replaced by adopting an image processing technology;
An alcohol concentration and sound signal acquisition unit for acquiring alcohol concentration and sound signals respectively by using an alcohol sensor and a language sensor;
the detection unit is used for sending the acquired physiological characteristics, eye state indexes, alcohol concentration and sound signals to the constructed drunk driving and fatigue driving detection model to detect, so as to obtain a detection result.
The present invention also provides a computer-readable storage medium having a set of computer instructions stored therein; and when the computer instruction set is executed by the processor, the detection method for drunk driving and fatigue driving of the driver is realized.
Compared with the prior art, the invention has the following advantages:
1. according to the detection method for drunk driving and fatigue driving of the driver, privacy protection is carried out by combining a deep learning mode with a face changing technology, and the safety and confidentiality of product use are improved.
2. According to the detection method for drunk driving and fatigue driving of the driver, provided by the invention, the language sensor and the alcohol sensor are introduced to acquire signals, the signals are transmitted into the model, and a brand new detection model is designed by combining an image processing result, so that the purpose of improving the accuracy of the prior art is achieved, and the sensor array is used for detecting the direction prominence of the driver, so that the judgment accuracy is improved.
3. According to the detection method for drunk driving and fatigue driving of the driver, when the detection model is constructed, signals such as eyes, mouth, skin color and the like are processed, so that the fatigue driving degree is detected while drunk driving is detected.
Based on the reasons, the invention can be widely popularized in the fields of traffic safety detection and the like.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort to a person skilled in the art.
FIG. 1 is a flow chart of the method of the present invention.
FIG. 2 is a schematic diagram of an alcohol sensor array of the device of the present invention.
Detailed Description
It should be noted that, without conflict, the embodiments of the present invention and features of the embodiments may be combined with each other. The invention will be described in detail below with reference to the drawings in connection with embodiments.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
The relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless it is specifically stated otherwise. Meanwhile, it should be clear that the dimensions of the respective parts shown in the drawings are not drawn in actual scale for convenience of description. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate. In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.
In the description of the present invention, it should be understood that the azimuth or positional relationships indicated by the azimuth terms such as "front, rear, upper, lower, left, right", "lateral, vertical, horizontal", and "top, bottom", etc., are generally based on the azimuth or positional relationships shown in the drawings, merely to facilitate description of the present invention and simplify the description, and these azimuth terms do not indicate and imply that the apparatus or elements referred to must have a specific azimuth or be constructed and operated in a specific azimuth, and thus should not be construed as limiting the scope of protection of the present invention: the orientation word "inner and outer" refers to inner and outer relative to the contour of the respective component itself.
Spatially relative terms, such as "above … …," "above … …," "upper surface on … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial location relative to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as "above" or "over" other devices or structures would then be oriented "below" or "beneath" the other devices or structures. Thus, the exemplary term "above … …" may include both orientations "above … …" and "below … …". The device may also be positioned in other different ways (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.
In addition, the terms "first", "second", etc. are used to define the components, and are only for convenience of distinguishing the corresponding components, and the terms have no special meaning unless otherwise stated, and therefore should not be construed as limiting the scope of the present invention.
As shown in fig. 1, the invention provides a method for detecting drunk driving and fatigue driving of a driver, which comprises the following steps:
collecting video images containing faces of drivers;
Carrying out frame division processing on the acquired video images, intercepting the face in each frame of image, and replacing the five sense organs (retaining the original indexes of eyes) in each frame of image in a deep learning mode for protecting the privacy and the identity of a tested person;
constructing a drunk driving and fatigue driving detection model;
Acquiring physiological characteristics and eye state indexes based on each frame of image after the five sense organs are replaced by adopting an image processing technology;
Respectively acquiring alcohol concentration and sound signals by adopting an alcohol sensor and a language sensor;
and sending the obtained physiological characteristics, eye state indexes, alcohol concentration and sound signals to a constructed drunk driving and fatigue driving detection model for detection, and obtaining a detection result.
In a preferred embodiment of the present invention, the method for capturing the video image including the face of the driver uses an image sensor.
In a specific implementation, as a preferred embodiment of the present invention, the steps of performing frame processing on the collected video image, intercepting a face in each frame of image, and replacing the five sense organs in each frame of image by adopting a deep learning mode include:
S101, creating an original video area, detecting and extracting a face area, replacing the face area, storing a model, and outputting the video area;
s102, the original video area is used for storing and recording original videos and images, and converting the original videos into frame pictures for storage;
s103, detecting and extracting face areas, and adopting a model to detect, extract and cut all face images existing in each original frame picture into face images for storage;
S104, the face area is used for storing the picture information of the face to be replaced, and is used for secondary training;
s105, the model storage area is used for storing the used models and the trained models;
s106, the output video area replaces the human face through an algorithm, and the frame picture is output as a video to be stored.
In this embodiment, the specific implementation process of detecting and extracting the face region, replacing the face region, storing the model and outputting the video region in the original video region is as follows:
original video area: the original video is converted into a frame picture and saved using the python's opencv-python library and the PIL's Image library, and the normal video will be extracted to 24 frames per second or 30 frames per second.
Detecting and extracting a face area: the face extraction method of the S3FD model is adopted to extract a plurality of faces in a positioning way, and simultaneously cut off the faces, and as the sizes of pictures are different, a plurality of faces or no faces can exist at the same time, only frames with faces can be extracted, and invalid frames are ignored. As these inactive frames are inactive in the final composition of the video.
Face alignment: the face landmarks of the standard pose are extracted by adopting a 2DFAN algorithm, and the face landmarks of the side face are extracted by adopting a PRNet algorithm. The information extracted from the face alignment includes elevation angle and facial heat map. Helping to better align the face and making the replacement more accurate.
The mask is shielded: for common face masks such as hands, lipsticks, glasses, hair, which need to be effectively removed at the time of replacement, only the mask required for the specification is generated for the picture by using the image segmentation technology, and the mask is operated with the picture, so that only the region of interest and to be replaced is reserved. The mask part of interest mainly comprises information of eyes, nose and mouth, and the effective replacement information is used for removing the shielding object by training an effective model, so that a better replacement effect is obtained.
Replacement face area: the GAN training model is used, and for a classical DF structure, a LIAE structure is used, so that the structure has stronger adaptability to strong light and can be replaced more realistically. The LIAE structure is used for generating potential face information of the original image and the target image through InterAB, and generating only the output target image through InterB, so that weight information of the target image is shared, illumination information of the original image is reserved, and even in a strong light environment, a good replacement effect is achieved.
For such complex images of the face, the liveness of the eyes is important for identifying authenticity and later health detection information. Therefore, in this embodiment, by adopting the SSIM technique, the reality of the eyes is more focused in the training process by increasing the training weight of the eye positions, so that more vivid and lively eyes are obtained, and further more vivid target images are obtained.
In this embodiment, reinhard Color Transfer (RCT) technology and Poisson blending technology are also used in the face replacement process at the same time, so that the face replacement is not so hard. The blurring process not only makes the replacement more realistic, but also does not destroy the original picture information.
In a specific implementation, as a preferred embodiment of the present invention, the acquiring physiological features and eye state indexes based on each frame of image after the five sense organs are replaced by using an image processing technology includes:
S201, intercepting a human face region of interest in each frame of image after the facial features are replaced, separating color channels of the human face region of interest, extracting IPPG signals, and preprocessing;
S202, processing IPPG signals with the same time domain length, and calculating and extracting physiological characteristics (heart rate, blood pressure, heart rate variation and the like);
S203, selecting an area where eyes are located from the face aligned with each frame, and performing eye positioning by performing area segmentation, edge extraction, gray level projection and template matching on the face to obtain an eye state index; and analyzing the eye state indexes to obtain whether the fatigue driving condition exists.
In a specific implementation, as a preferred embodiment of the present invention, the method for acquiring the alcohol concentration and the sound signal by using the alcohol sensor and the language sensor respectively includes:
S301, detecting the concentration of alcohol by adopting an alcohol sensor, and flashing a traffic light and a green light at the same time if alcohol is detected; after 4 seconds, displaying concentration data, and only flashing green light if the concentration is between 0.00 and 0.40; if the light quantity is more than or equal to 0.50, only the red light flashes;
S302, detecting whether the sound signal transmitted from the direction of the driver accords with normal logic and mood by adopting a language sensor, if not, flashing a red light, and if not, still transmitting the sound signal again, the red light is always on. In this embodiment, the detection condition is specifically detected as whether it belongs to drunk driving or fatigue driving by training the detection sound signal, and the index is added to a drunk driving classification prediction model pre-designed by the system.
In specific implementation, as a preferred embodiment of the present invention, the alcohol sensor adopts a sensor array, so as to detect the alcohol concentration of the source in each direction, and further determine whether the drinker is a driver to avoid misjudgment. The sensor array is shown in fig. 2.
In summary, according to the method for detecting drunk driving and fatigue driving of a driver provided by the invention, the privacy protection of a detected person is performed by replacing five sense organs in a deep learning mode, a face ROI (region of interest) is selected from the aligned faces, IPPG signals are extracted from the ROI, preprocessing is performed on the signals, and physiological characteristics (heart rate, blood pressure, heart rate variation and the like) are obtained by processing IPPG signals with the same time domain length. The invention structurally considers the multi-sensor fusion detection thought, introduces an alcohol sensor and a language sensor, judges through the alcohol content in the vehicle and the sound transmitted by the direction of the driver, and the multi-sensors cooperate to respectively exert own detection function and finally judge whether the driver drives drunk or not and the drunk driving grade. Accuracy is further ensured by the form of the sensor array. And by combining a non-contact acquisition technology, the safe and sanitary preventive drunk driving detection is realized. The operation is simple, safe and sanitary, the economic benefit and the detection efficiency can be effectively improved, and the predictive, real-time and traceable drunk driving detection can be realized.
The application also provides a detection device for drunk driving and fatigue driving of a driver, which comprises:
The video image acquisition unit is used for acquiring video images containing the faces of the driver;
The privacy and identity protection unit is used for carrying out framing treatment on the acquired video images, intercepting the face in each frame of image and replacing the five sense organs in each frame of image in a deep learning mode;
the detection model construction unit is used for constructing drunk driving and fatigue driving detection models;
The feature index obtaining unit is used for obtaining physiological features and eye state indexes based on each frame of images after the five sense organs are replaced by adopting an image processing technology;
An alcohol concentration and sound signal acquisition unit for acquiring alcohol concentration and sound signals respectively by using an alcohol sensor and a language sensor;
the detection unit is used for sending the acquired physiological characteristics, eye state indexes, alcohol concentration and sound signals to the constructed drunk driving and fatigue driving detection model to detect, so as to obtain a detection result.
For the embodiments of the present invention, since they correspond to those in the above embodiments, the description is relatively simple, and the relevant similarities will be found in the description of the above embodiments, and will not be described in detail herein.
The embodiment of the application also discloses a computer readable storage medium, wherein a computer instruction set is stored in the computer readable storage medium, and when the computer instruction set is executed by a processor, the method for detecting drunk driving and fatigue driving of a driver, provided by any embodiment, is realized.
In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (6)
1. The method for detecting drunk driving and fatigue driving of the driver is characterized by comprising the following steps of:
collecting video images containing faces of drivers;
carrying out frame processing on the acquired video images, intercepting human faces in each frame of images, and replacing five sense organs in each frame of images in a deep learning mode, wherein the method comprises the following steps of:
creating an original video area, detecting and extracting a face area, replacing the face area, storing a model, and outputting the video area;
the original video area is used for storing and recording original videos and images, and converting the original videos into frame pictures for storage;
the face area is detected and extracted, and all face detection and extraction of each original frame picture are cut into face images by adopting a model to store the face images;
The face replacement area is used for storing picture information of the face to be replaced;
the model storage area is used for storing a model used for recording and a trained model;
The output video area replaces the human face through an algorithm, and outputs the frame picture as a video for storage;
constructing a drunk driving and fatigue driving detection model;
the method comprises the steps of acquiring physiological characteristics and eye state indexes based on each frame of image after the five sense organs are replaced by adopting an image processing technology, and comprises the following steps:
Intercepting a human face region of interest in each frame of image after the five sense organs are replaced, separating color channels of the human face region of interest, extracting IPPG signals, and preprocessing;
processing IPPG signals with the same time domain length, and calculating and extracting physiological characteristics;
selecting an area where eyes are located from the face aligned with each frame, and performing eye positioning by performing area segmentation, edge extraction, gray level projection and template matching on the face to obtain an eye state index;
Respectively acquiring alcohol concentration and sound signals by adopting an alcohol sensor and a language sensor;
and sending the obtained physiological characteristics, eye state indexes, alcohol concentration and sound signals to a constructed drunk driving and fatigue driving detection model for detection, and obtaining a detection result.
2. The method for detecting drunk driving and fatigue driving of a driver according to claim 1, wherein the method for collecting the video image including the face of the driver is to use an image sensor.
3. The method for detecting drunk driving and fatigue driving of a driver according to claim 1, wherein the step of acquiring the alcohol concentration and the sound signal using the alcohol sensor and the language sensor, respectively, comprises:
Detecting the alcohol concentration by adopting an alcohol sensor, and flashing a traffic light and a green light at the same time if alcohol is detected; after 4 seconds, displaying concentration data, and only flashing green light if the concentration is between 0.00 and 0.40; if the light quantity is more than or equal to 0.50, only the red light flashes;
The voice sensor is used for detecting whether the voice signal transmitted from the direction of the driver accords with normal logic and mood, if the voice signal does not accord with the normal logic and mood, the red light flashes, and if the voice signal transmitted again does not accord with the voice signal, the red light is always on.
4. The method for detecting drunk driving and fatigue driving of a driver according to claim 3, wherein the alcohol sensor is a sensor array for detecting alcohol concentration from various directions.
5. A driver drunk driving and fatigue driving detection device, based on the driver drunk driving and fatigue driving detection method according to any one of claims 1-4, comprising:
The video image acquisition unit is used for acquiring video images containing the faces of the driver;
The privacy and identity protection unit is used for carrying out framing treatment on the acquired video images, intercepting the face in each frame of image and replacing the five sense organs in each frame of image in a deep learning mode;
the detection model construction unit is used for constructing drunk driving and fatigue driving detection models;
The feature index obtaining unit is used for obtaining physiological features and eye state indexes based on each frame of images after the five sense organs are replaced by adopting an image processing technology;
An alcohol concentration and sound signal acquisition unit for acquiring alcohol concentration and sound signals respectively by using an alcohol sensor and a language sensor;
the detection unit is used for sending the acquired physiological characteristics, eye state indexes, alcohol concentration and sound signals to the constructed drunk driving and fatigue driving detection model to detect, so as to obtain a detection result.
6. A computer-readable storage medium having a set of computer instructions stored therein; the set of computer instructions, when executed by a processor, implements a method for detecting drunk driving and fatigue driving of a driver as defined in any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111082202.1A CN113792663B (en) | 2021-09-15 | 2021-09-15 | Method, device and storage medium for detecting drunk driving and fatigue driving of driver |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111082202.1A CN113792663B (en) | 2021-09-15 | 2021-09-15 | Method, device and storage medium for detecting drunk driving and fatigue driving of driver |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113792663A CN113792663A (en) | 2021-12-14 |
| CN113792663B true CN113792663B (en) | 2024-05-14 |
Family
ID=78878549
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111082202.1A Active CN113792663B (en) | 2021-09-15 | 2021-09-15 | Method, device and storage medium for detecting drunk driving and fatigue driving of driver |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113792663B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115482933B (en) * | 2022-11-01 | 2023-11-28 | 北京鹰瞳科技发展股份有限公司 | Method for evaluating driving risk of driver and related product thereof |
| CN117333927B (en) * | 2023-12-01 | 2024-04-16 | 厦门磁北科技有限公司 | Vehicle-mounted face recognition alcohol detection method and system |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104574820A (en) * | 2015-01-09 | 2015-04-29 | 安徽清新互联信息科技有限公司 | Fatigue drive detecting method based on eye features |
| CN108229245A (en) * | 2016-12-14 | 2018-06-29 | 贵港市瑞成科技有限公司 | Method for detecting fatigue driving based on facial video features |
| DE102017201328A1 (en) * | 2017-01-27 | 2018-08-02 | Bayerische Motoren Werke Aktiengesellschaft | Method for determining a facial expression |
| CN109965889A (en) * | 2019-04-01 | 2019-07-05 | 北京理工大学 | A kind of method for detecting fatigue driving using smart phone loudspeaker and microphone |
| CN110001652A (en) * | 2019-03-26 | 2019-07-12 | 深圳市科思创动科技有限公司 | Monitoring method, device and the terminal device of driver status |
| CN110276273A (en) * | 2019-05-30 | 2019-09-24 | 福建工程学院 | Merge the Driver Fatigue Detection of facial characteristics and the estimation of image pulse heart rate |
| CN110751051A (en) * | 2019-09-23 | 2020-02-04 | 江苏大学 | Abnormal driving behavior detection method based on machine vision |
| CN111281367A (en) * | 2018-12-10 | 2020-06-16 | 绍兴图聚光电科技有限公司 | Anti-interference non-contact heart rate detection method based on face video |
| CN111914646A (en) * | 2020-07-01 | 2020-11-10 | 天津中科智能识别产业技术研究院有限公司 | Double-current fusion network iris in-vivo detection method based on light field image sequence |
| CN112101103A (en) * | 2020-08-07 | 2020-12-18 | 东南大学 | Video driver fatigue detection method based on deep integration network |
| CN112401857A (en) * | 2020-11-23 | 2021-02-26 | 杭州艺兴科技有限公司 | Driver drunk driving detection method |
| CN112735440A (en) * | 2020-12-30 | 2021-04-30 | 北京瞰瞰科技有限公司 | Vehicle-mounted intelligent robot interaction method, robot and vehicle |
| CN113066263A (en) * | 2020-01-02 | 2021-07-02 | 中车株洲电力机车研究所有限公司 | Method and device for preventing fatigue driving |
| CN113139527A (en) * | 2021-05-28 | 2021-07-20 | 深圳文达智通技术有限公司 | Video privacy protection method, device, equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10867195B2 (en) * | 2018-03-12 | 2020-12-15 | Microsoft Technology Licensing, Llc | Systems and methods for monitoring driver state |
| US11687778B2 (en) * | 2020-01-06 | 2023-06-27 | The Research Foundation For The State University Of New York | Fakecatcher: detection of synthetic portrait videos using biological signals |
-
2021
- 2021-09-15 CN CN202111082202.1A patent/CN113792663B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104574820A (en) * | 2015-01-09 | 2015-04-29 | 安徽清新互联信息科技有限公司 | Fatigue drive detecting method based on eye features |
| CN108229245A (en) * | 2016-12-14 | 2018-06-29 | 贵港市瑞成科技有限公司 | Method for detecting fatigue driving based on facial video features |
| DE102017201328A1 (en) * | 2017-01-27 | 2018-08-02 | Bayerische Motoren Werke Aktiengesellschaft | Method for determining a facial expression |
| CN111281367A (en) * | 2018-12-10 | 2020-06-16 | 绍兴图聚光电科技有限公司 | Anti-interference non-contact heart rate detection method based on face video |
| CN110001652A (en) * | 2019-03-26 | 2019-07-12 | 深圳市科思创动科技有限公司 | Monitoring method, device and the terminal device of driver status |
| CN109965889A (en) * | 2019-04-01 | 2019-07-05 | 北京理工大学 | A kind of method for detecting fatigue driving using smart phone loudspeaker and microphone |
| CN110276273A (en) * | 2019-05-30 | 2019-09-24 | 福建工程学院 | Merge the Driver Fatigue Detection of facial characteristics and the estimation of image pulse heart rate |
| CN110751051A (en) * | 2019-09-23 | 2020-02-04 | 江苏大学 | Abnormal driving behavior detection method based on machine vision |
| CN113066263A (en) * | 2020-01-02 | 2021-07-02 | 中车株洲电力机车研究所有限公司 | Method and device for preventing fatigue driving |
| CN111914646A (en) * | 2020-07-01 | 2020-11-10 | 天津中科智能识别产业技术研究院有限公司 | Double-current fusion network iris in-vivo detection method based on light field image sequence |
| CN112101103A (en) * | 2020-08-07 | 2020-12-18 | 东南大学 | Video driver fatigue detection method based on deep integration network |
| CN112401857A (en) * | 2020-11-23 | 2021-02-26 | 杭州艺兴科技有限公司 | Driver drunk driving detection method |
| CN112735440A (en) * | 2020-12-30 | 2021-04-30 | 北京瞰瞰科技有限公司 | Vehicle-mounted intelligent robot interaction method, robot and vehicle |
| CN113139527A (en) * | 2021-05-28 | 2021-07-20 | 深圳文达智通技术有限公司 | Video privacy protection method, device, equipment and storage medium |
Non-Patent Citations (8)
| Title |
|---|
| A Survey on Deepfake Video Detection;Peipeng Yu等;《IET Biometrics》;第10卷(第6期);607-624 * |
| Asleep at the automated wheel—Sleepiness and fatigue during highly automated driving;Tobias Vogelpohl等;《Accident Analysis and Prevention》;20180321;第126卷;70–84 * |
| DeepFakesON-Phys: DeepFakes Detection based on Heart Rate Estimation;Javier Hernandez-Ortega等;《arxiv》;1-8 * |
| FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals;Umur Aybars Ciftci等;《arxiv》;1-17 * |
| Xingjian Zhang等.A study on the effects of fatigue driving and drunk driving on drivers' physical characteristics.《Traffic injury prevention》.2014,第15卷(第8期),801-808. * |
| 人脸识别活体检测研究方法综述;邓雄等;《计算机应用研究》;第37卷(第09期);2579-2585 * |
| 基于头部特征和姿态估计的疲劳驾驶检测技术的研究;刘敏;《中国优秀硕士学位论文全文数据库：工程科技Ⅱ辑》;20200315(第2020年第03期);C035-397 * |
| 基于视频序列的驾驶员疲劳检测算法研究;彭玉玺;《中国优秀硕士学位论文全文数据库：工程科技Ⅱ辑》;20190915(第2019年第09期);C035-183 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113792663A (en) | 2021-12-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113792663B (en) | Method, device and storage medium for detecting drunk driving and fatigue driving of driver | |
| CN102708383B (en) | System and method for detecting living face with multi-mode contrast function | |
| CN108764071A (en) | It is a kind of based on infrared and visible images real human face detection method and device | |
| CN106372629A (en) | Living body detection method and device | |
| CN110363129B (en) | Early autism screening system based on smiling paradigm and audio-video behavior analysis | |
| Jiang et al. | Rdfnet: A fast caries detection method incorporating transformer mechanism | |
| CN108229493A (en) | Banknote validation method, apparatus and electronic equipment | |
| CN114333046A (en) | Dance action scoring method, device, equipment and storage medium | |
| CN110874572B (en) | Information detection method and device and storage medium | |
| CN117095128A (en) | Priori-free multi-view human body clothes editing method | |
| CN113064490B (en) | Eye movement track-based virtual enhancement equipment identification method | |
| CN111914925B (en) | Patient behavior multi-modal perception and analysis system based on deep learning | |
| Javed et al. | An intelligent alarm based visual eye tracking algorithm for cheating free examination system | |
| CN111145082A (en) | Face image processing method and device, electronic equipment and storage medium | |
| CN111986157A (en) | Digital pathological image quality evaluation system | |
| CN112861588A (en) | Living body detection method and device | |
| WO2014204126A2 (en) | Apparatus for capturing 3d ultrasound images and method for operating same | |
| CN111274854B (en) | Human body action recognition method and vision enhancement processing system | |
| CN115171189A (en) | Fatigue detection method, device, equipment and storage medium | |
| CN113989886A (en) | Crew identity verification method based on face recognition | |
| CN114330624A (en) | New coronavirus detection result two-dimensional code and two-dimensional code acquisition method | |
| CN113627255A (en) | Mouse behavior quantitative analysis method, device, equipment and readable storage medium | |
| CN111008569A (en) | Glasses detection method based on face semantic feature constraint convolutional network | |
| CN106780420B (en) | Color Image Fusion based on image wave filter | |
| Hu et al. | Using eye gaze to play mind card-game using neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |