CN113421402A - Passenger body temperature and fatigue driving behavior detection system and method based on infrared camera - Google Patents
Passenger body temperature and fatigue driving behavior detection system and method based on infrared camera Download PDFInfo
- Publication number
- CN113421402A CN113421402A CN202110630213.2A CN202110630213A CN113421402A CN 113421402 A CN113421402 A CN 113421402A CN 202110630213 A CN202110630213 A CN 202110630213A CN 113421402 A CN113421402 A CN 113421402A
- Authority
- CN
- China
- Prior art keywords
- infrared camera
- vehicle
- driver
- fatigue driving
- temperature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000036760 body temperature Effects 0.000 title claims abstract description 60
- 238000001514 detection method Methods 0.000 title claims abstract description 54
- 238000000034 method Methods 0.000 title abstract description 11
- 238000009529 body temperature measurement Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 12
- 238000004891 communication Methods 0.000 claims abstract description 10
- 230000002159 abnormal effect Effects 0.000 claims description 28
- 230000006399 behavior Effects 0.000 claims description 25
- 241001282135 Poromitra oscitans Species 0.000 claims description 13
- 206010048232 Yawning Diseases 0.000 claims description 13
- 210000003128 head Anatomy 0.000 claims description 8
- 206010000117 Abnormal behaviour Diseases 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 claims description 3
- 210000000744 eyelid Anatomy 0.000 claims description 3
- 210000001747 pupil Anatomy 0.000 claims description 3
- 230000006855 networking Effects 0.000 claims description 2
- 230000005856 abnormality Effects 0.000 abstract description 4
- 238000012544 monitoring process Methods 0.000 abstract 1
- 206010016256 fatigue Diseases 0.000 description 60
- 238000010586 diagram Methods 0.000 description 7
- 238000013135 deep learning Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 229920000535 Tan II Polymers 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 206010011409 Cross infection Diseases 0.000 description 1
- 206010029803 Nosocomial infection Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- ZRHANBBTXQZFSP-UHFFFAOYSA-M potassium;4-amino-3,5,6-trichloropyridine-2-carboxylate Chemical compound [K+].NC1=C(Cl)C(Cl)=NC(C([O-])=O)=C1Cl ZRHANBBTXQZFSP-UHFFFAOYSA-M 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/0022—Radiation pyrometry, e.g. infrared or optical thermometry for sensing the radiation of moving bodies
- G01J5/0025—Living bodies
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B3/00—Audible signalling systems; Audible personal calling systems
- G08B3/10—Audible signalling systems; Audible personal calling systems using electric transmission; using electromagnetic transmission
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Emergency Management (AREA)
- Software Systems (AREA)
- Business, Economics & Management (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Medical Informatics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Electromagnetism (AREA)
- Traffic Control Systems (AREA)
- Emergency Alarm Devices (AREA)
Abstract
The invention discloses a passenger body temperature and fatigue driving behavior detection system and method based on an infrared camera, wherein the system comprises: no. 1 machine of infrared camera, temperature measurement module, central processing unit, No. 2 machines of infrared camera, driver fatigue detection module, communication module, on-vehicle loudspeaker, display and cloud platform. The method comprises the following steps: s1, carrying out infrared temperature measurement on the passengers by using the No. 1 infrared camera, and giving out prompts to all members when an abnormality occurs; s2, aligning the face of the driver by using an infrared camera No. 2 machine, and adjusting a proper camera angle; s3, detecting the human face by using an infrared camera No. 2 machine, comparing the human face with a preset fatigue driving model, judging whether a driver has fatigue driving, and if so, receiving a warning; and S4, establishing a cloud platform, and monitoring the vehicle in real time. The invention can effectively detect in the day and at night, has high efficiency, reduces the danger coefficient and can monitor the vehicle in real time.
Description
Technical Field
The invention relates to the field of vehicle safety, in particular to a passenger body temperature and fatigue driving behavior detection system and method based on an infrared camera.
Background
At present, the body temperature detection of personnel at each checkpoint on a road is mainly manual, and the personnel in a passing vehicle are detected by a worker holding the body temperature detection equipment, so that the efficiency is low, and cross infection is easily caused; although the outdoor infrared thermal imager can complete non-contact temperature measurement, the outdoor infrared thermal imager is susceptible to the influence of factors such as environment temperature and humidity, and the accuracy cannot be guaranteed.
The existing fatigue driving detection comprises the following steps: firstly, fatigue driving is judged by detecting the behavior of a driver, namely whether the driver is fatigue driving is judged by detecting the operation of an accelerator pedal, a brake pedal and a steering wheel and the driving speed of the driver through various sensors, but the operation habits of each person are possibly greatly different, the vehicle is excessively complicated to modify, and a scheme which can be popularized cannot be found at present; secondly, fatigue driving is judged based on physiological feature detection of a driver, the face of the driver is detected, and eyes and a mouth are monitored in a focused mode, but due to the fact that the actual environment is variable and the limitation of a common camera is achieved, the fatigue driving is judged only through the eyes or the mouth, and the accuracy is not objective and high. And thirdly, a breakpoint is set on the road, whether the driver is in fatigue driving is judged by calculating the driving time of the driver, and the defects that too high manpower and material resources are needed, and a unified fatigue driving standard cannot be provided even if the physical conditions of each person are different. Fourthly, the eye state of a single visual feature such as Zhang F and the like is detected by deep learning, and the hand region of the Chao Yan and the like is detected to judge whether the driving is fatigue driving, so that the running speed is too slow and the real-time performance cannot be achieved; and the method is single, and the error rate is too high under the influence of practical factors.
Therefore, a system and a method for detecting the body temperature and the fatigue behavior of the passenger based on the infrared camera are developed, so that the body temperature is detected, whether the driver is in fatigue driving or not can be judged, and warning intervention is performed on the fatigue driving, which is a technical problem to be solved urgently.
Disclosure of Invention
The invention aims to provide a passenger body temperature and fatigue driving behavior detection system, a passenger body temperature and fatigue driving behavior detection method, a passenger body temperature and fatigue driving behavior detection system and a passenger fatigue driving behavior detection method based on an infrared camera, so that the problems in the prior art are solved, the detection efficiency and accuracy are improved, the risk of fatigue driving of a driver is prompted, and the vehicle can be detected in real time.
In order to achieve the purpose, the invention provides the following scheme: the invention discloses a passenger body temperature and fatigue driving behavior detection system based on an infrared camera, which comprises:
the temperature measurement module is used for judging whether the body temperature is abnormal or not;
the fatigue driving detection module is used for judging whether a driver has fatigue driving;
the central processing unit is used for receiving the in-vehicle temperature output by the in-vehicle temperature sensor;
the prompting device is used for broadcasting abnormal behaviors in the vehicle;
the communication module is used for transmitting the abnormal behavior in the vehicle to a network;
the cloud platform receives information and can alarm surrounding vehicles;
the central processing unit is respectively connected with the temperature measuring module, the fatigue driving detection module, the prompting device and the communication module, and the communication module is further connected with the cloud platform.
Preferably, the temperature measurement module comprises an infrared camera which can measure the temperature of passengers in an infrared manner at the top of the vehicle, and the infrared camera is set as a No. 1 infrared camera.
Preferably, the fatigue driving detection module comprises an infrared camera 2, and is used for acquiring face information.
Preferably, the prompting device is a horn and a display in the vehicle; the loudspeaker is used for judging that the driver state is fatigue and prompting the driver, and the display is used for displaying the current body temperature and the position of the abnormal body temperature member.
A passenger body temperature and fatigue driving behavior detection method based on an infrared camera is characterized by comprising the following steps:
s1, measuring the temperature of passengers once by using an infrared camera No. 1 at the top of a vehicle, judging whether the body temperature of people in the vehicle is abnormal or not by combining the temperature in the vehicle, and detecting fatigue driving of a driver if the body temperature is not abnormal;
s2, aligning the face of the driver by the infrared camera No. 2 in front of the driver, and shooting 2/3 or more of the face of the driver by the infrared camera No. 2 to finish alignment;
s3, acquiring face information of a driver by using an infrared camera No. 2 to obtain a continuous frame of the face information of the driver, extracting features of the continuous frame of the face based on MTCNN, detecting closing time of eyes and yawning duration of a mouth in the face features by using YOLOv3 to construct a fatigue driving model, and judging whether the driver is in fatigue driving based on the fatigue driving model, the closing time of the eyes and the yawning duration of the mouth;
s4, establishing a cloud platform, storing vehicle information in advance, transmitting the body temperature of passengers and the driving behavior of a driver to the cloud platform through a vehicle networking technology after the vehicle is started each time, and uploading place and time information to the cloud platform if the vehicle sends a warning; at the moment, the cloud platform sends a prompt to the vehicles with the system in the local area.
Preferably, in S1, if there is an abnormal passenger at the first temperature measurement, the abnormal passenger needs to be recorded and all the members in the vehicle are prompted that the passenger' S body temperature may be abnormal; then the body temperature of the passenger is detected again at intervals, whether the passenger is abnormal or not is judged for the second time by combining the temperature in the vehicle, and if the body temperature is abnormal again, a prompt is sent to all the members; if the body temperature of the passenger returns to normal, the warning is eliminated.
Preferably, the infrared camera 2 in S2 does not shoot 2/3 and above of the face of the driver, prompts the driver to adjust the angle of the infrared camera until 2/3 and above of the face of the driver are shot, and turns off the prompt.
Preferably, the fatigue driving model constructed in S3 adopts P80 criterion of PERCLO for calculating the closing time of the eyes within 5 seconds: if the area of the pupil covered by the eyelid exceeds 80%, the eye is considered to be closed, if the closing time is more than or equal to 30%, the eye is considered to be tired, and if the eye is closed for two continuous seconds, the eye is considered to be tired; the length-width ratio of the mouth part is more than or equal to 50 percent, namely yawning is considered, the yawning time is more than or equal to 15 percent within 5 seconds, and the eye closing time is more than or equal to 15 percent, namely fatigue is considered; if the time for which the head deviation angle is equal to or greater than 60 degrees is equal to or greater than 30% within 5 seconds, fatigue is determined.
The invention discloses the following technical effects:
the fatigue driving detection system does not depend on vehicle body modification, only needs one infrared camera arranged on the front side of a driver, and is low in cost; the invention can effectively detect in the daytime and at night, and is more effective than a common camera especially at night; and the frequent time period of fatigue driving is the night; the invention combines a plurality of appearance characteristics of the corners of eyes, a mouth and a head, and the neural network detection combined with deep learning is more accurate, thereby improving the accuracy of judging fatigue driving; the prompting device adopts a vehicle-mounted horn for warning, the method is convenient and efficient, and meanwhile, the risk of fatigue driving of other drivers is prompted; the cloud platform is further adopted to collect real-time information of each vehicle, and a real-time reporting function is provided for each vehicle.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic diagram of the system of the present invention;
FIG. 2 is a schematic flow diagram of the system of the present invention;
FIG. 3 is a schematic diagram of 68 feature points detected by a human face in fatigue driving according to the present invention;
FIG. 4 is a schematic diagram illustrating MTCNN network parameter transmission;
fig. 5 is a schematic diagram of the YOLOv3 network parameters.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
The invention provides a passenger body temperature and fatigue driving behavior detection system and method based on an infrared camera, and the passenger body temperature and fatigue driving behavior detection system comprises an infrared camera No. 1 machine, a temperature measurement module, a central processing unit, an infrared camera No. 2 machine, a fatigue driving detection module, a communication module, a vehicle-mounted loudspeaker, a display and a cloud platform, wherein the central processing unit is respectively connected with the temperature measurement module, the fatigue driving detection module, a prompting device and the communication module, the communication module is also connected with the cloud platform, the prompting device comprises the vehicle-mounted loudspeaker and the display, the fatigue driving detection module comprises an infrared camera No. 2 machine and is used for collecting face information, the temperature measurement module comprises an infrared camera which can carry out infrared temperature measurement on passengers at the vehicle roof and is set as the infrared camera No. 1 machine, and the infrared camera is shown in figure 1.
(1) Temperature measuring module
As shown in fig. 2, an infrared camera capable of photographing all persons is provided on the roof of the vehicle, and the infrared camera is a number 1 infrared camera. The No. 1 machine of infrared camera can carry out infrared temperature measurement to the passenger. When the vehicle is started, the No. 1 infrared camera measures the temperature of passengers for the first time, then judges whether the body temperatures of the members in the vehicle are abnormal or not by combining the temperature in the vehicle, and needs to record the passengers with abnormal body temperatures and prompt the possible abnormality of the body temperatures of the members in the vehicle, wherein the abnormal body temperatures of the members in the vehicle are abnormal; and detecting the body temperature of the member again at intervals, judging whether the body temperature is abnormal for the second time by combining the temperature in the vehicle, clearly warning if the body temperature is recovered to be normal, and sending a prompt to all the members if the body temperature is abnormal again.
Adopting a central processing unit to receive the in-car temperature output by the in-car temperature sensor, and if the in-car temperature belongs to [15 ℃ and 26 ℃), setting the normal body temperature of members in the car to [36 ℃ and 37 ℃; if the temperature in the vehicle is lower than 15 ℃, the normal body temperature of the members in the vehicle is set as (35.8 ℃, 37 ℃); if the in-vehicle temperature is higher than 26 ℃, the normal body temperature of the occupant in the vehicle is set to [36 ℃, 37.2 ℃). When the body temperature of a member is not in a normal body temperature range, the position and the body temperature of the member can be displayed on an operation display of a driver, and all the members are prompted through a vehicle-mounted loudspeaker: "there may be an abnormality in the body temperature of the member at the location". And for the member with abnormal temperature measurement for the first time, the central processing unit judges whether the member is still the abnormal body temperature for the second time after 30 minutes, and the member body temperature is determined to be abnormal. All the members are prompted through the vehicle-mounted loudspeaker again: "there is an abnormality in the body temperature of the member at the location". After the temperature of all the members is measured for the first time, for the member with abnormal body temperature, the driver can input the information of whether to know the member in advance to the central processing unit, and if the information is input, the temperature is not measured for the second time.
(2) Fatigue driving detection module
When the number 1 infrared camera is started, the infrared camera in front of the driver is started immediately, and the number 2 infrared camera is set as the infrared camera. Firstly aligning the face of a driver, wherein the infrared camera No. 2 machine can ensure that most of the face of the driver is shot, and prompting the driver to adjust the angle of the camera if the visual angle is not good; the infrared camera No. 2 machine firstly judges the face of a driver, a central processing unit is adopted to judge whether the range shot by the face is larger than 2/3 of the face, and if the range is smaller than 2/3 of the face, the driver is prompted to adjust the angle of the infrared camera; if the angle is greater than or equal to 2/3 of the human face, the driver is not prompted to adjust the angle of the infrared camera.
(3) Preset fatigue driving model
And the infrared camera 2 carries out face detection on the driver to judge whether the driver has fatigue driving. The system is based on MTCNN (Multi-task shielded connected Networks) + YOLOv3 network. The MTCNN detects the face of a driver, collects the characteristics of continuous frames of each part, and the Yolov3 finishes the detection of whether the eyes are closed and the mouth is yawned. Compared with a preset fatigue driving model, if the model is matched, the driver is determined to be not attentive enough in the period of time, the possibility of fatigue driving exists, the driver is prompted to pay attention to rest through the vehicle-mounted loudspeaker, the risk brought by fatigue driving is vigilant, and the attention is improved; the infrared camera has good detection capability at night, and fatigue driving is more frequent at night, so the infrared camera has a specific advantage in fatigue driving detection. And the neural network detection combined with deep learning can be more accurate. The MTCNN + YOLOv3 network structure has the advantages of fatigue driving detection: compared with the traditional method, the method utilizes the deep learning neural network, so that the detection rate can be greatly improved.
MTCNN networks are, as shown in FIG. 4, Proposal Network (P-Net), Refine Network (R-Net), Output Network (O-Net), respectively. And continuously resetting the input picture to obtain a picture pyramid. And inputting the picture pyramid into the P-Net to obtain a large number of candidate frames. Then fine-tuning is carried out through R-Net. And finally, inputting the coordinates into O-Net to obtain the final bbox coordinates of the face, as shown in FIG. 3. After the face coordinates are obtained, the human eyes and the mouth area are input into a YOLOv3 network. And establishing a YOLOv3 human eye opening and closing and yawning detection model. And (3) detecting human eye and mouth areas of the driver by using YOLOv3, and acquiring coordinates of the block diagram and confidence of a detection result. And traversing all categories, drawing frames for all categories with confidence degrees larger than a threshold value according to the coordinates of the block diagram, and labeling the categories. In order to further optimize the technical scheme, YOLOv3 is selected in advance as a network structure, and particularly, training sets of eyes open, eyes closed, yawning open, mouth closed and the like of a fatigue driving model data set are trained. And (4) labeling the anchor points, wherein the anchor points with the intersection ratio of the Box of the real label being more than 0.7 are positive labels, and the anchor points with the intersection ratio being less than 0.3 are negative labels. And outputting detection frames with confidence coefficient higher than 0.8, wherein each frame outputs detection types, namely four types of opening eyes, closing eyes, yawning and closing mouth, if the detection types are matched, the detection type is set to be 1, and if the detection types are not matched, the detection type is not set to be 0. And accumulating the detection types of all frames within 5 seconds, and if the detection types are matched with a preset model, determining fatigue driving. The head posture judgment fatigue or inattentive driving is through the rotation angle of the head posture in real time. As shown in fig. 5, key points of a 2D face are detected. And 3D face model matching. And solving the conversion relation from the 3D point to the corresponding 2D point through the relation of a world coordinate system, a camera coordinate system, an image center coordinate system and a pixel coordinate system. Solving the Euler angle according to the rotation matrix, specifically:
wherein:
is a three-dimensional coordinate, and is,
is a matrix of rotations of the optical system,
is a translation vector, fx，fyIs the focal length, cx，cyIs the optical center of the light beam,
are two-dimensional coordinates.
And solving the Euler angle according to the rotation matrix. The concrete formula is as follows:
rotating in turn by means of a raw-pitch-roll, the matrix is described as:
the euler angle can be solved as follows:
θy＝a tan2(-rxz，rzz)
θz＝a tan2(-ryx，ryy)
thereby obtaining three parameters yaw (yaw angle), pitch (pitch angle) and roll (rotation angle) of the head rotation angle, and when the time of the absolute value of the three angles being more than or equal to 30 degrees in 5 seconds is more than or equal to 30%, the head rotation angle is determined to be fatigue or careless driving. Presetting the standard of fatigue identification: calculating the closing time of the eyes within 5 seconds (using the P80 criterion of PERCLO, namely the eyes are considered to be closed if the pupil is covered by the eyelid for more than 80 percent), and considering the eyes to be fatigue if the closing time is more than or equal to 30 percent, and considering the eyes to be fatigue if the eyes are closed for two continuous seconds; the length-width ratio of the mouth part is more than or equal to 50 percent, namely yawning is considered, the yawning time is more than or equal to 15 percent within 5 seconds, and the eye closing time is more than or equal to 15 percent, namely fatigue is considered; if the time of the head deviation angle being more than or equal to 60 degrees within 5 seconds is more than or equal to 30 percent, the driver is determined to be tired or careless driving.
(4) Establishing a cloud platform
The cloud platform can save vehicle information in advance, and after the vehicle starts at every turn, the body temperature of the members in the vehicle and the driving behavior of the driver can be transmitted to the cloud platform through the internet of vehicles technology. If the vehicle sends a warning, the location and time information can be uploaded to the cloud platform; at the moment, the cloud platform sends a prompt to the vehicles with the system in the region. When the temperature of the members in the vehicle is abnormally measured for the second time and the driver has fatigue driving behavior, the central processing unit sends the position information of the vehicle to the cloud platform through the internet of vehicles. After receiving the information, the cloud platform broadcasts the information and vehicle information such as license plates to an area near the vehicle, and all vehicles connected with the cloud platform can receive the information and prompt other vehicles to pay attention to safety. All vehicles connected to the cloud platform are provided with a database on the cloud platform, the license plate, the vehicle body and the vehicle condition information of the vehicle and the information of all the vehicles are recorded, and the cloud platform can collect the latest traffic behaviors of the vehicle.
The above-described embodiments are merely illustrative of the preferred embodiments of the present invention, and do not limit the scope of the present invention, and various modifications and improvements of the technical solutions of the present invention can be made by those skilled in the art without departing from the spirit of the present invention, and the technical solutions of the present invention are within the scope of the present invention defined by the claims.
Claims (8)
1. The utility model provides a passenger's body temperature and driver fatigue behavior detecting system based on infrared camera which characterized in that includes:
the temperature measurement module is used for judging whether the body temperature is abnormal or not;
the fatigue driving detection module is used for judging whether a driver has fatigue driving;
the central processing unit is used for receiving the in-vehicle temperature output by the in-vehicle temperature sensor;
the prompting device is used for broadcasting abnormal behaviors in the vehicle;
the communication module is used for transmitting the abnormal behavior in the vehicle to a network;
the cloud platform receives information and can alarm surrounding vehicles;
the central processing unit is respectively connected with the temperature measuring module, the fatigue driving detection module, the prompting device and the communication module, and the communication module is further connected with the cloud platform.
2. The passenger temperature and fatigue driving behavior detection system based on an infrared camera as claimed in claim 1, wherein the temperature measurement module comprises an infrared camera capable of performing infrared temperature measurement on the passenger at the roof of the vehicle, and is set as a number 1 infrared camera.
3. The infrared camera-based passenger body temperature and fatigue driving behavior detection system as claimed in claim 1, wherein the fatigue driving detection module comprises an infrared camera number 2 machine for collecting face information.
4. The infrared camera-based passenger temperature and fatigue driving behavior detection system according to claim 1, wherein the prompting device is an in-vehicle speaker and a display; the loudspeaker is used for judging that the driver state is fatigue and prompting the driver, and the display is used for displaying the current body temperature and the position of the abnormal body temperature member.
5. A passenger body temperature and fatigue driving behavior detection method based on an infrared camera is characterized by comprising the following steps:
s1, measuring the temperature of passengers once by using an infrared camera No. 1 at the top of a vehicle, judging whether the body temperature of people in the vehicle is abnormal or not by combining the temperature in the vehicle, and detecting fatigue driving of a driver if the body temperature is not abnormal;
s2, aligning the face of the driver by the infrared camera No. 2 in front of the driver, and shooting 2/3 or more of the face of the driver by the infrared camera No. 2 to finish alignment;
s3, acquiring face information of a driver by using an infrared camera No. 2 to obtain a continuous frame of the face information of the driver, extracting features of the continuous frame of the face based on MTCNN, detecting closing time of eyes and yawning duration of a mouth in the face features by using YOLOv3 to construct a fatigue driving model, and judging whether the driver is in fatigue driving based on the fatigue driving model, the closing time of the eyes and the yawning duration of the mouth;
s4, establishing a cloud platform, storing vehicle information in advance, transmitting the body temperature of passengers and the driving behavior of a driver to the cloud platform through a vehicle networking technology after the vehicle is started each time, and uploading place and time information to the cloud platform if the vehicle sends a warning; at the moment, the cloud platform sends a prompt to the vehicles with the system in the local area.
6. The passenger temperature and fatigue driving behavior detection method based on the infrared camera as claimed in claim 5, wherein: in S1, specifically, if there is a passenger with abnormal body temperature during the first temperature measurement, the passenger with abnormal body temperature needs to be recorded and all the members in the vehicle are prompted that the body temperature of the passenger may be abnormal; then the body temperature of the passenger is detected again at intervals, whether the passenger is abnormal or not is judged for the second time by combining the temperature in the vehicle, and if the body temperature is abnormal again, a prompt is sent to all the members; if the body temperature of the passenger returns to normal, the warning is eliminated.
7. The passenger temperature and fatigue driving behavior detection method based on the infrared camera as claimed in claim 5, wherein: the infrared camera 2 in the S2 does not shoot 2/3 and above of the face of the driver, prompts the driver to adjust the angle of the infrared camera until 2/3 and above of the face of the driver are shot, and closes the prompt.
8. The passenger temperature and fatigue driving behavior detection method based on the infrared camera as claimed in claim 5, wherein: the fatigue driving model constructed in S3 adopts P80 criterion of PERCLO for calculating the closing time of the eyes within 5 seconds: if the area of the pupil covered by the eyelid exceeds 80%, the eye is considered to be closed, if the closing time is more than or equal to 30%, the eye is considered to be tired, and if the eye is closed for two continuous seconds, the eye is considered to be tired; the length-width ratio of the mouth part is more than or equal to 50 percent, namely yawning is considered, the yawning time is more than or equal to 15 percent within 5 seconds, and the eye closing time is more than or equal to 15 percent, namely fatigue is considered; if the time for which the head deviation angle is equal to or greater than 60 degrees is equal to or greater than 30% within 5 seconds, fatigue is determined.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110630213.2A CN113421402A (en) | 2021-06-07 | 2021-06-07 | Passenger body temperature and fatigue driving behavior detection system and method based on infrared camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110630213.2A CN113421402A (en) | 2021-06-07 | 2021-06-07 | Passenger body temperature and fatigue driving behavior detection system and method based on infrared camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113421402A true CN113421402A (en) | 2021-09-21 |
Family
ID=77713920
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110630213.2A Pending CN113421402A (en) | 2021-06-07 | 2021-06-07 | Passenger body temperature and fatigue driving behavior detection system and method based on infrared camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113421402A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115311819A (en) * | 2022-10-10 | 2022-11-08 | 南京智慧交通信息股份有限公司 | Intelligent bus intelligent vehicle-mounted real-time warning system and method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107719065A (en) * | 2017-10-17 | 2018-02-23 | 黄晶 | A kind of dangerous driving prevention and intelligent driving environment adjustment method and system |
| CN108216029A (en) * | 2018-01-26 | 2018-06-29 | 奇瑞汽车股份有限公司 | Vehicle early warning system and method |
| CN210428777U (en) * | 2019-10-09 | 2020-04-28 | 广西科技大学 | Anti-fatigue driving detection system |
| CN111415480A (en) * | 2020-04-27 | 2020-07-14 | 名商科技有限公司 | Passenger car intelligent security identification system |
| CN112258790A (en) * | 2020-10-26 | 2021-01-22 | 上海工程技术大学 | Rail transit train driver on-duty fatigue level and body state detection system |
-
2021
- 2021-06-07 CN CN202110630213.2A patent/CN113421402A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107719065A (en) * | 2017-10-17 | 2018-02-23 | 黄晶 | A kind of dangerous driving prevention and intelligent driving environment adjustment method and system |
| CN108216029A (en) * | 2018-01-26 | 2018-06-29 | 奇瑞汽车股份有限公司 | Vehicle early warning system and method |
| CN210428777U (en) * | 2019-10-09 | 2020-04-28 | 广西科技大学 | Anti-fatigue driving detection system |
| CN111415480A (en) * | 2020-04-27 | 2020-07-14 | 名商科技有限公司 | Passenger car intelligent security identification system |
| CN112258790A (en) * | 2020-10-26 | 2021-01-22 | 上海工程技术大学 | Rail transit train driver on-duty fatigue level and body state detection system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115311819A (en) * | 2022-10-10 | 2022-11-08 | 南京智慧交通信息股份有限公司 | Intelligent bus intelligent vehicle-mounted real-time warning system and method thereof |
| CN115311819B (en) * | 2022-10-10 | 2023-01-20 | 南京智慧交通信息股份有限公司 | Intelligent bus intelligent vehicle-mounted real-time warning system and method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103770780B (en) | A kind of active safety systems of vehicles alarm shield device | |
| CN107092881B (en) | A kind of human pilot changes system and method | |
| US9180887B2 (en) | Driver identification based on face data | |
| WO2018058958A1 (en) | Road vehicle traffic alarm system and method therefor | |
| CN103568992B (en) | Equipment for the driver's location sound image for vehicle and method | |
| WO2020029444A1 (en) | Method and system for detecting attention of driver while driving | |
| CN106571015A (en) | Driving behavior data collection method based on Internet | |
| CN105151049A (en) | Early warning system based on driver face features and lane departure detection | |
| CN104599443A (en) | Vehicle-mounted forewarning terminal for driving behaviors based on information fusion and forewarning method thereof | |
| CN108926352B (en) | Driving fatigue detection method and system | |
| CN104408878A (en) | Vehicle fleet fatigue driving early warning monitoring system and method | |
| CN103700220A (en) | Fatigue driving monitoring device | |
| CN106564501A (en) | Intelligent automobile assisting system and method based on cloud technology | |
| CN113370786B (en) | Vehicle-mounted drunk driving comprehensive detection system for unit vehicle based on multi-source information fusion | |
| CN104299363A (en) | Fatigue driving pre-warning system based on multi-feature fusion | |
| CN109664889A (en) | A kind of control method for vehicle, device and system and storage medium | |
| CN107403541A (en) | The system of real-time eye recognition monitoring fatigue driving | |
| CN103854539A (en) | Electronic road test monitoring system based on video analysis | |
| CN113022445A (en) | Vehicle environment information monitoring method and system | |
| CN113421402A (en) | Passenger body temperature and fatigue driving behavior detection system and method based on infrared camera | |
| CN206961331U (en) | A kind of high threat vehicle monitoring early warning system | |
| CN108537138A (en) | A kind of eyes closed degree computational methods based on machine vision | |
| CN107330389A (en) | A kind of specification for being used for intelligent Driving Test is got on the bus starting determination methods | |
| CN113562005A (en) | Intelligent monitoring and obstacle intrusion limit detecting system for driver behaviors of rail train | |
| CN106447828A (en) | Automobile data recorder with iris recognition function |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210921 |
|
| RJ01 | Rejection of invention patent application after publication |