CN111513697A - Intelligent control system and method for driving safety - Google Patents

Intelligent control system and method for driving safety Download PDF

Info

Publication number
CN111513697A
CN111513697A CN202010330170.1A CN202010330170A CN111513697A CN 111513697 A CN111513697 A CN 111513697A CN 202010330170 A CN202010330170 A CN 202010330170A CN 111513697 A CN111513697 A CN 111513697A
Authority
CN
China
Prior art keywords
driver
heart rate
danger
monitoring
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202010330170.1A
Other languages
Chinese (zh)
Inventor
李晓华
潘容
潘国晨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Shengyingda Technology Development Co ltd
Original Assignee
Chongqing Shengyingda Technology Development Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Shengyingda Technology Development Co ltd filed Critical Chongqing Shengyingda Technology Development Co ltd
Priority to CN202010330170.1A priority Critical patent/CN111513697A/en
Publication of CN111513697A publication Critical patent/CN111513697A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024—Detecting, measuring or recording pulse rate or heart rate
    • A61B5/02416—Detecting, measuring or recording pulse rate or heart rate using photoplethysmograph signals, e.g. generated by infrared radiation
    • A61B5/02427—Details of sensor
    • A61B5/02433—Details of sensor for infrared radiation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6887—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient mounted on external non-worn devices, e.g. non-medical devices
    • A61B5/6893—Cars
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient ; user input means
    • A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Physiology (AREA)
  • Cardiology (AREA)
  • Evolutionary Computation (AREA)
  • Signal Processing (AREA)
  • Psychiatry (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Fuzzy Systems (AREA)
  • Traffic Control Systems (AREA)

Abstract

A driving safety intelligent control system and method comprises a driver heart rate monitoring method, a driver's cab environment monitoring method and a danger alarming method; the method comprises the steps that a driver heart rate monitoring method and a driver's cabin environment monitoring method respectively complete driver heart rate monitoring and driver's cabin environment monitoring, the heart rate monitoring result and the environment monitoring result are sent to a danger alarming method, and a danger alarming result is obtained through comprehensive judgment; the heart rate of the driver can be automatically monitored with high precision and the driving environment of the driver can be automatically monitored; the danger alarm judgment in the vehicle running process is completed, the driver is further alarmed and reminded, and danger alarm data and vehicle positioning data are uploaded to a vehicle monitoring platform; the safety driving control method and the safety driving control system realize the guarantee of the safety driving of the road vehicles and the guarantee of the health of the driver, and avoid major traffic accidents caused by the health of the driver.

Description

Intelligent control system and method for driving safety
Technical Field
The invention relates to a road safety system, in particular to a driving safety intelligent control system and a driving safety intelligent control method.
Background
With the development of science and technology, the living standard of people is continuously improved, the quantity of private cars is increased day by day, the pressure of urban traffic is increased more and more, and safe driving becomes a serious problem gradually. For urban people, a great amount of time is spent on the automobile every day, and more people use the automobile as one of learning and working places.
However, at present, traffic accidents caused by physical discomfort and sudden illness of drivers during driving or caused by fatigue driving are frequently reported. In the prior art, a general vehicle is not provided with a device capable of monitoring the health condition of a driver. Therefore, it is impossible to prevent the risk of traffic accidents caused by fatigue, physical discomfort, sudden illness when the driver operates the vehicle to travel. Therefore, the real-time and accurate monitoring of the physiological state of the driver is of great significance to avoid traffic accidents and protect the life and property safety of the driver and passengers.
Disclosure of Invention
The invention aims to provide an intelligent control method for driving safety, which can finish intelligent and automatic monitoring on whether a driver is in safe driving behavior in the driving process, and provides timely and accurate alarming and data uploading or a method for taking corresponding measure basis and decision.
Another objective of the present invention is to provide an intelligent control system for driving safety, which can complete the prediction and alarm of safe driving behavior by intelligently and automatically monitoring the heart rate of the driver and the driving environment, and further take other corresponding measures to prevent the occurrence of traffic accidents, ensure the normal operation of the traffic network, and ensure the physical safety of the driver and the passengers on the vehicle.
The purpose of the invention is realized according to the following technical scheme:
an intelligent control method for driving safety comprises a driver heart rate monitoring method, a driver's cab environment monitoring method and a danger alarming method; the driver heart rate monitoring method and the driver's cab environment monitoring method respectively complete the driver heart rate monitoring and the driver's cab environment monitoring, and send the heart rate monitoring result and the environment monitoring result to the danger alarming method, and comprehensively judge to obtain a danger alarming result; the method for monitoring the heart rate of the driver comprises the steps of analyzing a near-infrared image of a specific area of the face of the driver to obtain a heart rate curve of the driver; the method comprises the following specific steps: s001, periodically sampling a near-infrared video containing a front image of a driver; the sampling period is T; s002, performing face detection on each frame of image; s003, performing face positioning and ROI interception according to a face detection result; s004, calculating and intercepting a near infrared intensity evaluation value of the ROI; s005, calculating the near-infrared intensity variation of the near-infrared intensity evaluation value of each frame of image compared with the near-infrared intensity of the previous frame; s006, forming a variation sequence by the near infrared intensity variations of all frame images in the video segment; the time interval is a sampling period T; s007, carrying out Fourier transform on the variable quantity sequence to obtain a frequency domain variable modulus; s008, selecting a module value peak value to obtain a driver heart rate value;
the driver's cabin environment monitoring method adopts visible light video analysis of the driver's cabin environment to obtain whether behaviors harmful to or possibly influencing the safe driving of the driver occur or not and judge whether the driver carries out fatigue driving or not; the method comprises the following specific steps:
s101, calibration: under various common road conditions, a driver driving a vehicle normally is shot and a camera picture is stored as a calibration picture; s102, data acquisition: periodically sampling visible light video of a vehicle cab environment in real-time operation to obtain a picture frame; s103, inputting the calibrated picture and the picture frame into a trained driver' S cab environment danger early warning network at the same time; s104, monitoring an environmental risk early warning network output result in real time;
the danger alarm method integrates the heart rate monitoring of a driver and the environment monitoring of a driving cab, and comprehensively judges and provides a danger alarm prompt; the method comprises the following specific steps: s201, calculating the heart rate value change rate of a driver in real time; the heart rate value change rate calculation formula is as follows: Δ pt＝pt-pt-TT is a near-infrared video sampling period; s202, calculating the mean value of the heart rate value change rate intensity of the driver, wherein the calculation formula is as follows:
Figure BDA0002464666080000011
s203, calculating an inhibition factor lambda according to an output result of the environmental hazard early warning network:
Figure BDA0002464666080000021
s204, counting the number of times whether the ratio of the intensity of the heart rate value change rate of the driver to the mean value of the intensity of the heart rate value change rate of the driver in the MT time is greater than a threshold value:
Figure BDA0002464666080000022
wherein μ (x) is a step function,
Figure BDA0002464666080000023
r is a threshold value used for judging that the intensity of the change rate of the heart rate value of the driver is larger than the threshold value; s205, only when n is larger than 0.5M, giving out a danger alarm prompt;
s206, confirming the danger alarm prompt by the driver, and uploading the danger alarm, the vehicle position information and other related information to the vehicle monitoring platform if the driver directly confirms the danger or does not select the danger confirmation within a certain time; if the driver cancels the danger alarm prompt, the danger alarm prompt is not uploaded; and if the driver cancels the danger alarm prompt and continues to carry out the danger alarm prompt in a short time, the danger alarm prompt is directly uploaded.
Specifically, the face positioning and the ROI cutting are performed according to the face detection result, the face detection result is 9 key points of the face, which are respectively a human middle (p1), a left eye corner (p2), a left eye right eye corner (p3), a right eye left eye corner (p4), a right eye corner (p5), a left mouth corner (p6), a right mouth corner (p7), an upper mouth skin center (p8) and a lower mouth skin center (p 9);
the ROI is left forehead and right forehead, and the coordinates are respectively
Figure BDA0002464666080000024
And
Figure BDA0002464666080000025
center and width of
Figure BDA0002464666080000026
Is high as
Figure BDA0002464666080000027
In the formula xi(i ═ 1.., 9), and yi(i ═ 1.., 9.) is p, respectivelyiCoordinate values of (i ═ 1.., 9).
More specifically, the near infrared intensity evaluation value of the intercepted ROI area is calculated by firstly dividing the ROI areaDividing the domain into N equal parts according to the y direction, and calculating the infrared intensity value of each equal part:
Figure BDA0002464666080000028
wherein pix (x, y) is
Figure BDA0002464666080000029
Pixel values of the inner points (x, y);
Figure BDA00024646660800000210
is C1The k-th aliquot, which is the central ROI;
Figure BDA00024646660800000211
is C2The k-th aliquot, which is the central ROI;
Figure BDA00024646660800000212
is composed of
Figure BDA00024646660800000213
The number of pixel points of the region; second, calculate the mean:
Figure BDA00024646660800000214
then removing equal parts of 0.8-1.2 which exceed the mean value, and recalculating the mean value:
Figure BDA00024646660800000215
and finally, solving a near infrared intensity evaluation value of the intercepted ROI:
Figure BDA0002464666080000031
more preferably, the driver's cabin environment danger early warning network is a deep neural network, two standard VGG16 network structures are selected as processing feature sub-networks for calibrating pictures and picture frames, a BN layer is connected behind the two sub-networks, and a self-defined DIFF layer is output and accessed and combined into an enhancement feature with the same dimension; sequentially inputting the enhanced features into the three conv modules, obtaining output features through a GlobalPooling layer, and then connecting a judgment network consisting of a BN layer and a full link layer to obtain an output result; the self-defined DIFF layer mainly completes the operation of obtaining a difference value of two output characteristic pixels, and the input dimension and the output dimension are the same; the method has the effects that the influence of fixed background information is removed from the output characteristics obtained by the picture frame, and the change information different from the fixed background information is more prominently displayed; the three conv modules are connected by a maxpool layer; 3 × 3 convolution kernels are adopted in the maxpool layers, 2-time down-sampling is guaranteed, and the number of output and input characteristic channels is kept unchanged; the three conv modules are formed by sequentially connecting a conv layer, a BN layer and a ReLU layer; three const layers respectively adopt three groups of parameters of 3 x 256, 3 x 128 and 3 x 64, and the width and height dimensions of input and output features are unchanged.
A driving safety intelligent control system is composed of a near-infrared camera, a visible light camera, a monitoring host, a confirmation key, a storage device, a sound box, a wireless communication terminal and a vehicle monitoring platform, wherein the near-infrared camera, the visible light camera, the monitoring host, the confirmation key, the storage device, the sound box, the wireless communication terminal and the vehicle monitoring platform are installed in a vehicle cab; the near-infrared camera is arranged on the upper part right in front of the vehicle cab, and samples the near-infrared video frames monitored to the front of the driver and sends the sampled near-infrared video frames to the monitoring host; the visible light camera is arranged on the upper part of the position, deviated from the right, of the vehicle cab, and is used for sampling and sending the monitored video frames of the environment of the cab to the monitoring host; the monitoring host makes a danger alarm judgment according to an input near-infrared video frame and a vehicle driving platform environment video frame based on an intelligent behavior monitoring vehicle danger alarm method; the confirmation key is arranged on a vehicle control panel and used for confirming the danger alarm judgment of a driver; the storage device is connected with the monitoring host, and stores the current near-infrared video frame and the vehicle driving platform environment video frame when the danger alarm is judged to be true, namely the vehicle has potential safety driving hazards; the wireless communication terminal is connected with the monitoring host, and when the danger alarm judgment is true, the wireless communication terminal sends danger alarm notification information to the vehicle monitoring platform; the danger alarm notification information comprises a wireless terminal unique identifier, a vehicle unique identifier, a danger alarm judgment result, a current near-infrared video frame, a vehicle driving platform environment video frame and vehicle position information; the vehicle monitoring platform receives danger alarm notification information sent by all vehicle wireless communication terminals in a control range; manually intervene, study and judge according to the danger alarm notification information to confirm danger, send safety warning information and take corresponding measures; the safety warning information comprises a target wireless terminal unique identifier, a target vehicle unique identifier and a safety warning recording; the wireless communication terminal installed on the vehicle receives the safety warning information, verifies the unique identification of the target wireless terminal and the unique identification of the target vehicle, and inputs the safety warning record into the sound if the safety warning record is confirmed to be correct; the sound equipment is connected with the wireless communication terminal and used for receiving and playing the safety warning record; the corresponding measures comprise alarming, contacting a driver with the unique vehicle identification contained in the danger alarm notification information, recording and notifying the driver of safe parking nearby, and contacting a nearby traffic management department or hospital.
The invention has the following beneficial effects:
the invention provides an intelligent control system and method for driving safety, which can automatically monitor the heart rate of a vehicle driver and the driving environment of the driver; finishing vehicle danger alarm judgment and synchronizing to a vehicle monitoring platform; the safety driving of the vehicle is guaranteed, and meanwhile, the occurrence of traffic accidents is reduced; in the automatic detection of the heart rate of the vehicle driver, the ROI is equally divided, and the multi-region discrete value elimination and averaging method reduces noise interference and improves the heart rate calculation precision. The driving safety intelligent control system has wide application range and is suitable for various private cars, operation taxies, network appointment cars, buses, large, medium and small buses, trucks and the like.
Drawings
Fig. 1 is a schematic diagram of outputting 9-point key points by a face detection result.
Detailed Description
The present invention is described in detail below by way of examples, it should be noted that the following examples are only for illustrating the present invention and should not be construed as limiting the scope of the present invention, and those skilled in the art can make some insubstantial modifications and adaptations of the present invention based on the above-described disclosure.
Example 1
An intelligent control system for driving safety is composed of a near-infrared camera, a visible light camera, a monitoring host, a confirmation key, a storage device, a sound box, a wireless communication terminal and a vehicle monitoring platform; the system comprises a near-infrared camera, a monitoring host computer, a front-side video monitoring host computer and a front-side video monitoring host computer, wherein the near-infrared camera is arranged at the upper part right in front of a vehicle driving platform, and is used for sampling and sending a monitored front-side near-infrared video frame of; the visible light camera is arranged on the upper part of the position, deviated from the right, of the vehicle cab, and samples and sends the monitored environmental video frames of the vehicle cab to the monitoring host; the monitoring host has computing capability, and the vehicle danger alarm method based on intelligent behavior monitoring makes danger alarm judgment according to the step of finishing computation of the input near-infrared video frame and the vehicle driving platform environment video frame; the confirmation key is arranged on the vehicle control panel and used for confirming the danger alarm judgment of the driver; the storage device is connected with the monitoring host, and stores the current near-infrared video frame and the vehicle driving platform environment video frame when the danger alarm is judged to be true, namely the vehicle has potential safety driving hazards;
the wireless communication terminal is connected with the monitoring host, and when the danger alarm judgment is true and confirmed, the wireless communication terminal sends danger alarm notification information to the vehicle monitoring platform; the danger alarm notification information comprises a wireless terminal unique identifier, a vehicle unique identifier (which can be a vehicle engine number or a license plate number), a danger alarm judgment result, a current near-infrared video frame and a vehicle driving platform environment video frame; the vehicle monitoring platform supervises vehicle wireless communication terminals in all control ranges, when receiving danger alarm notification information, manually intervenes, studies and judges to confirm danger according to the danger alarm notification information, and further ensures the accuracy of the whole judgment by a small amount of manual work under the condition that the system automatically judges and eliminates a large amount of interference, sends safety warning information and takes corresponding measures; the safety warning information comprises a target wireless terminal unique identifier, a target vehicle unique identifier and a safety warning recording; the wireless communication terminal installed on the vehicle receives the safety warning information and verifies the unique identification of the target wireless terminal and the unique identification of the target vehicle (which can be the engine number or the license plate number of the vehicle), if the safety warning information is identical to the local information, the safety warning record is input to the sound equipment connected with the safety warning record, and the safety warning record is played;
meanwhile, the driver of the vehicle can be manually contacted, the driver is informed of safe parking nearby, and the driver can also be contacted with a nearby traffic management department or hospital to alarm the serious plot; and can report the information record.
Example 2
An intelligent control method for driving safety is divided into a driver heart rate monitoring method, a driving platform environment monitoring method and a danger alarming method according to different actions;
the method comprises the steps that a driver heart rate monitoring method and a driver's cabin environment monitoring method respectively complete driver heart rate monitoring and driver's cabin environment monitoring, the heart rate monitoring result and the environment monitoring result are sent to a danger alarming method, and a danger alarming result is obtained through comprehensive judgment; the method for monitoring the heart rate of the driver mainly comprises the steps of analyzing a near-infrared image of a specific area of the face of the driver to obtain a heart rate curve of the driver; s001, periodically sampling a near-infrared video containing a front image of a driver; the sampling period is T; the video selection can adopt a window function to slide and intercept a video segment on the video of the whole time axis; s002, performing face detection on each frame of image; wherein, the adopted human face detection model is finished by offline training and adopts a deep neural network; the output result is 9 key point coordinates of the face, as shown in fig. 1, which are respectively a human middle (p1), a left eye corner (p2), a left eye corner (p3), a right eye corner (p4), a right eye corner (p5), a left mouth corner (p6), a right mouth corner (p7), an upper mouth skin center (p8) and a lower mouth skin center (p 9);
s003, performing face positioning and ROI interception according to a face detection result; wherein ROI is left forehead and right forehead, and its coordinates are respectively
Figure BDA0002464666080000051
And
Figure BDA0002464666080000052
is a center and a width of
Figure BDA0002464666080000053
Is high as
Figure BDA0002464666080000054
In the formula xi(i ═ 1.., 9), and yi(i ═ 1.., 9.) is p, respectivelyiCoordinate values of (i ═ 1.., 9).
S004, calculating and intercepting a near infrared intensity evaluation value of the ROI; the more detailed process is that the ROI area is divided into N equal parts according to the y direction, and the infrared intensity value of each equal part is calculated:
Figure BDA0002464666080000055
wherein pix (x, y) is
Figure BDA0002464666080000056
Pixel values of the inner points (x, y);
Figure BDA0002464666080000057
is C1The k-th aliquot, which is the central ROI;
Figure BDA0002464666080000058
is C2The k-th aliquot, which is the central ROI;
Figure BDA0002464666080000059
is composed of
Figure BDA00024646660800000510
The number of pixel points of the region;
second, calculate the mean:
Figure BDA00024646660800000511
then removing equal parts of 0.8-1.2 which exceed the mean value, and recalculating the mean value:
Figure BDA00024646660800000512
and finally, solving a near infrared intensity evaluation value of the intercepted ROI:
Figure BDA00024646660800000513
s005, calculating the near-infrared intensity variation of the near-infrared intensity evaluation value of each frame of image compared with the near-infrared intensity of the previous frame; s006, forming a variation sequence by the near infrared intensity variations of all frame images in the video segment; the time interval is a sampling period T; s007, carrying out Fourier transform on the variable quantity sequence to obtain a frequency domain variable modulus; s008, selecting a module value peak value, and solving a driver heart rate value by adopting Fourier inverse transformation;
for the method for monitoring the environment of the driver's cab, the visible light video analysis of the environment of the driver's cab is adopted to obtain whether behaviors harmful to the driver or possibly influencing the safe driving of the driver occur or not and judge whether the driver carries out fatigue driving or not; the method comprises the following specific steps: s101, calibration: under various common road conditions, a driver driving a vehicle normally is shot and a camera picture is stored as a calibration picture; s102, data acquisition: periodically sampling visible light video of a vehicle cab environment in real-time operation to obtain a picture frame; s103, inputting the calibrated picture and the picture frame into a trained driver' S cab environment danger early warning network at the same time; s104, monitoring an environmental risk early warning network output result in real time;
in order to ensure the performance, wherein the driver's cabin environment danger early warning network is a deep neural network, two standard VGG16 network structures are selected as processing feature sub-networks for calibrating pictures and picture frames, a BN layer is connected behind the two sub-networks, and a self-defined DIFF layer is output and accessed and combined into an enhancement feature with the same dimension; sequentially inputting the enhanced features into the three conv modules, obtaining output features through a GlobalPooling layer, and then connecting a judgment network consisting of a BN layer and a full link layer to obtain an output result;
the user-defined DIFF layer mainly completes the operation of obtaining a difference value of two output characteristic pixels, and the input dimension and the output dimension are the same; the effect of which is to make the output characteristic of the picture frame availableThe influence of fixed background information is eliminated, and change information different from the fixed background information is more prominently displayed; connecting the three conv modules by a maxpool layer; 3 × 3 convolution kernels are adopted in the maxpool layers, 2-time down-sampling is guaranteed, and the number of output and input characteristic channels is kept unchanged; the three conv modules are formed by sequentially connecting a conv layer, a BN layer and a ReLU layer; three const layers respectively adopt three groups of parameters of 3 × 256, 3 × 128 and 3 × 64, and the input and output characteristic width and height dimensions are unchanged; the danger alarm method integrates the heart rate monitoring of the driver and the environment monitoring of the driving platform, and comprehensively judges and provides a danger alarm prompt; the method comprises the following specific steps: s201, calculating the heart rate value change rate of a driver in real time; the heart rate value change rate calculation formula is as follows: Δ pt＝pt-pt-TT is a near-infrared video sampling period; s202, calculating the mean value of the heart rate value change rate intensity of the driver, wherein the calculation formula is as follows:
Figure BDA0002464666080000061
s203, calculating an inhibition factor lambda according to an output result of the environmental hazard early warning network:
Figure BDA0002464666080000062
s204, counting the number of times whether the ratio of the intensity of the heart rate value change rate of the driver to the mean value of the intensity of the heart rate value change rate of the driver in the MT time is greater than a threshold value:
Figure BDA0002464666080000063
wherein μ (x) is a step function,
Figure BDA0002464666080000064
r is a threshold value used for judging that the intensity of the change rate of the heart rate value of the driver is larger than the threshold value; s205, only when n is larger than 0.5M, giving out a danger alarm prompt;
s206, confirming the danger alarm prompt by the driver, and if the driver directly confirms the danger or does not confirm and select the danger after a certain time (if the danger does not confirm after 5 seconds, the driver sets the danger according to specific conditions), uploading the danger alarm, the vehicle position information and other related information to a vehicle monitoring platform; if the driver cancels the danger alarm prompt, the danger alarm prompt is not uploaded; and if the driver continues to perform the danger alarm prompt within a short time after canceling the danger alarm prompt (for example, within 15s or 30s after canceling the danger alarm prompt, the setting can be performed according to specific conditions), the danger alarm prompt is directly uploaded.

Claims (3)

1. A driving safety intelligent control method is characterized in that: the method comprises a driver heart rate monitoring method, a driver cab environment monitoring method and a danger alarming method; the driver heart rate monitoring method and the driver's cab environment monitoring method respectively complete the driver heart rate monitoring and the driver's cab environment monitoring, and send the heart rate monitoring result and the environment monitoring result to the danger alarming method, and comprehensively judge to obtain a danger alarming result; the method for monitoring the heart rate of the driver comprises the steps of analyzing a near-infrared image of a specific area of the face of the driver to obtain a heart rate curve of the driver; the method comprises the following specific steps: s001, periodically sampling a near-infrared video containing a front image of a driver; the sampling period may be T; s002, performing face detection on each frame of image; s003, performing face positioning and ROI interception according to a face detection result; s004, calculating and intercepting a near infrared intensity evaluation value of the ROI; s005, calculating the near-infrared intensity variation of the near-infrared intensity evaluation value of each frame of image compared with the near-infrared intensity of the previous frame; s006, forming a variation sequence by the near infrared intensity variations of all frame images in the video segment; the time interval may be a sampling period T; s007, carrying out Fourier transform on the variable quantity sequence to obtain a frequency domain variable modulus; s008, selecting a module value peak value to obtain a driver heart rate value;
the driver's cabin environment monitoring method adopts visible light video analysis of the driver's cabin environment to obtain whether behaviors harmful to or possibly influencing the safe driving of the driver occur or not and judge whether the driver carries out fatigue driving or not; the method comprises the following specific steps: s101, calibration: under various common road conditions, a driver driving a vehicle normally is shot and a camera picture is stored as a calibration picture; s102, data acquisition: periodically sampling visible light video of a vehicle cab environment in real-time operation to obtain a picture frame; s103, inputting the calibrated picture and the picture frame into a trained driver' S cab environment danger early warning network at the same time; s104, monitoring an environmental risk early warning network output result in real time;
the danger alarm method integrates the heart rate monitoring of a driver and the environment monitoring of a driving cab, and comprehensively judges and provides a danger alarm prompt; the method comprises the following specific steps: s201, calculating the heart rate value change rate of a driver in real time; the heart rate value change rate calculation formula is as follows: Δ pt＝pt-pt-TT is a near-infrared video sampling period; s202, calculating the mean value of the heart rate value change rate intensity of the driver, wherein the calculation formula is as follows:
Figure FDA0002464666070000011
s203, calculating an inhibition factor lambda according to an output result of the environmental hazard early warning network:
Figure FDA0002464666070000012
s204, counting the number of times whether the ratio of the intensity of the heart rate value change rate of the driver to the mean value of the intensity of the heart rate value change rate of the driver in the MT time is greater than a threshold value:
Figure FDA0002464666070000013
wherein μ (x) is a step function,
Figure FDA0002464666070000014
r is a threshold value used for judging that the intensity of the change rate of the heart rate value of the driver is larger than the threshold value; s205, only when n is larger than 0.5M, giving out a danger alarm prompt;
s206, confirming the danger alarm prompt by the driver, and uploading the danger alarm, the vehicle position information and other related information to the vehicle monitoring platform if the driver directly confirms the danger or does not select the danger confirmation within a certain time; if the driver cancels the danger alarm prompt, the danger alarm prompt is not uploaded; and if the driver cancels the danger alarm prompt and continues to carry out the danger alarm prompt in a short time, the danger alarm prompt is directly uploaded.
2. The intelligent control method for the driving safety according to claim 1, characterized in that: the face positioning and ROI interception are performed according to a face detection result, the face detection result is 9 key points of the face, namely a human middle (p1), a left eye corner (p2), a left eye right eye corner (p3), a right eye left eye corner (p4), a right eye corner (p5), a left mouth corner (p6), a right mouth corner (p7), an upper mouth skin center (p8) and a lower mouth skin center (p 9); the ROI is left forehead and right forehead, and the coordinates are respectively
Figure FDA0002464666070000015
And
Figure FDA0002464666070000016
is a center and a width of
Figure FDA0002464666070000017
Is high as
Figure FDA0002464666070000021
In the formula xi(i ═ 1.., 9), and yi(i ═ 1.., 9.) is p, respectivelyiCoordinate values of (i ═ 1.., 9).
3. The intelligent control method for driving safety as claimed in claim 2, wherein: the calculation of the near-infrared intensity evaluation value of the intercepted ROI area firstly equally divides the ROI area into N equal parts according to the y direction, and calculates the infrared intensity value of each equal part:
Figure FDA0002464666070000022
wherein pix (x, y) is
Figure FDA0002464666070000023
Pixel values of the inner points (x, y);
Figure FDA0002464666070000024
is C1The k-th aliquot, which is the central ROI;
Figure FDA0002464666070000025
is C2The k-th aliquot, which is the central ROI;
Figure FDA0002464666070000026
is composed of
Figure FDA0002464666070000027
The number of pixel points of the region; second, calculate the mean:
Figure FDA0002464666070000028
then removing equal parts of 0.8-1.2 which exceed the mean value, and recalculating the mean value:
Figure FDA0002464666070000029
and finally, solving a near infrared intensity evaluation value of the intercepted ROI:
Figure FDA00024646660700000210
CN202010330170.1A 2020-04-24 2020-04-24 Intelligent control system and method for driving safety Withdrawn CN111513697A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010330170.1A CN111513697A (en) 2020-04-24 2020-04-24 Intelligent control system and method for driving safety

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010330170.1A CN111513697A (en) 2020-04-24 2020-04-24 Intelligent control system and method for driving safety

Publications (1)

Publication Number Publication Date
CN111513697A true CN111513697A (en) 2020-08-11

Family

ID=71910471

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010330170.1A Withdrawn CN111513697A (en) 2020-04-24 2020-04-24 Intelligent control system and method for driving safety

Country Status (1)

Country Link
CN (1) CN111513697A (en)

Similar Documents

Publication Publication Date Title
US20210150390A1 (en) Systems and methods for providing visual allocation management
JP6307629B2 (en) Method and apparatus for detecting safe driving state of driver
US10525984B2 (en) Systems and methods for using an attention buffer to improve resource allocation management
CN110889351B (en) Video detection method, device, terminal equipment and readable storage medium
CN113378771B (en) Driver state determination method and device, driver monitoring system and vehicle
DE102020113422A1 (en) DEVICE AND METHOD FOR AUTONOMOUS DRIVING
CN111753674A (en) Fatigue driving detection and identification method based on deep learning
CN107844783A (en) A kind of commerial vehicle abnormal driving behavioral value method and system
CN109409259A (en) Drive monitoring method, device, equipment and computer-readable medium
DE102020113404A1 (en) DEVICE AND METHOD FOR AUTONOMOUS DRIVING
Cheng et al. Image-based EEG signal processing for driving fatigue prediction
CN109739234B (en) Vehicle real-time image tracking method based on GPS track data
CN110929606A (en) Vehicle blind area pedestrian monitoring method and device
CN111513697A (en) Intelligent control system and method for driving safety
CN113602287A (en) Man-machine driving system for female drivers with low driving ages
CN116453345B (en) Bus driving safety early warning method and system based on driving risk feedback
CN115299948A (en) Driver fatigue detection method and detection system
CN108454514A (en) A kind of traffic safety early warning system based on mobile Internet
CN111540169A (en) Bus danger alarm method and system based on intelligent behavior monitoring
CN115049992A (en) Logistics monitoring system and method based on big data
CN114241452A (en) Image recognition-based driver multi-index fatigue driving detection method
CN112660121A (en) Hidden danger vehicle identification early warning method and device, vehicle-mounted terminal and storage medium
JP2021019940A (en) Biological information extraction device
US11999368B2 (en) Systems and methods for automated vehicle fleet management according to dynamic pedagogical behavior reinforcement
JP7295790B2 (en) In-vehicle device, processing device and program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20200811

WW01 Invention patent application withdrawn after publication