CN111540169A - Bus danger alarm method and system based on intelligent behavior monitoring - Google Patents
Bus danger alarm method and system based on intelligent behavior monitoring Download PDFInfo
- Publication number
- CN111540169A CN111540169A CN202010331548.XA CN202010331548A CN111540169A CN 111540169 A CN111540169 A CN 111540169A CN 202010331548 A CN202010331548 A CN 202010331548A CN 111540169 A CN111540169 A CN 111540169A
- Authority
- CN
- China
- Prior art keywords
- bus
- monitoring
- driver
- danger alarm
- heart rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0438—Sensor means for detecting
- G08B21/0453—Sensor means for detecting worn on the body to detect health condition by physiological monitoring, e.g. electrocardiogram, temperature, breathing
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0438—Sensor means for detecting
- G08B21/0492—Sensor dual technology, i.e. two or more technologies collaborate to extract unsafe condition, e.g. video tracking and RFID tracking
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
- G08B25/01—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems characterised by the transmission medium
- G08B25/10—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems characterised by the transmission medium using wireless transmission systems
Landscapes
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Gerontology & Geriatric Medicine (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Cardiology (AREA)
- Heart & Thoracic Surgery (AREA)
- Physical Education & Sports Medicine (AREA)
- Physiology (AREA)
- Pulmonology (AREA)
- Emergency Alarm Devices (AREA)
Abstract
A bus danger alarm method and system based on intelligent behavior monitoring comprises a bus driver heart rate monitoring method, a bus driving platform environment monitoring method and a danger alarm method; the method comprises the steps that a bus driver heart rate monitoring method and a bus driving platform environment monitoring method respectively complete the monitoring of the heart rate of the driver and the monitoring of the environment of the driving platform, the heart rate monitoring result and the environment monitoring result are sent to a danger alarming method, and a danger alarming result is obtained through comprehensive judgment; the heart rate of the bus driver is automatically monitored with high precision and the driving environment of the bus driver is automatically monitored; the bus danger alarm judgment is completed and is synchronized to the center of the remote hub; the bus safety operation is guaranteed, and meanwhile, the labor cost is reduced.
Description
Technical Field
The invention relates to a public safety system, in particular to a bus danger alarm method and system based on intelligent behavior monitoring.
Background
The bus is used as a general vehicle and is distributed in each area of a city, and when the bus goes out, the bus also advocates green outgoing guarantee while maintaining the daily life outgoing demand of people; the number of buses is large, and the passenger carrying capacity is large, so that the safety problem of the buses is increasingly highlighted; in recent years, bus driver driving errors, bus passengers robbing a steering wheel, bus passengers beating the driver and other events affecting the safe operation of the bus occur, and heavy loss is also caused; the existing equipment lacks of monitoring the state of a bus driver and monitoring the driving environment of the bus driver, so that automatic judgment and processing under the condition that dangerous operation of the bus is possibly caused when certain dangerous behaviors occur are carried out.
Disclosure of Invention
The invention aims to provide a bus danger alarm method for intelligently monitoring behaviors, which is used for intelligently and automatically monitoring whether behaviors influencing safe driving of a driver exist in the bus driving process or not, and providing timely and accurate alarm or adopting a method of corresponding measure basis and decision;
the invention also aims to provide a bus danger alarm system based on intelligent behavior monitoring, which completes intelligent and automatic monitoring of external behaviors influencing safe driving of a driver by a bus danger alarm method based on intelligent behavior monitoring, and adopts alarm or other corresponding measures to prevent the occurrence of bus dangerous conditions;
the purpose of the invention is realized according to the following technical scheme:
a bus danger alarm method based on intelligent behavior monitoring comprises a bus driver heart rate monitoring method, a bus driver's platform environment monitoring method and a danger alarm method;
the method for monitoring the heart rate of the bus driver and the method for monitoring the environment of the bus driving platform respectively complete the heart rate monitoring of the driver and the environment monitoring of the driving platform, send the heart rate monitoring result and the environment monitoring result to a danger alarm method, and comprehensively judge to obtain a danger alarm result;
the method for monitoring the heart rate of the bus driver comprises the steps of analyzing a near-infrared image of a specific area of the face of the driver to obtain a heart rate curve of the bus driver; the method comprises the following specific steps:
s001, periodically sampling a near-infrared video containing a front image of a bus driver; the sampling period is T;
s002, performing face detection on each frame of image;
s003, performing face positioning and ROI interception according to a face detection result;
s004, calculating and intercepting a near infrared intensity evaluation value of the ROI;
s005, calculating the near-infrared intensity variation of the near-infrared intensity evaluation value of each frame of image compared with the near-infrared intensity of the previous frame;
s006, forming a variation sequence by the near infrared intensity variations of all frame images in the video segment; the time interval is a sampling period T;
s007, carrying out Fourier transform on the variable quantity sequence to obtain a frequency domain variable modulus;
s008, selecting a module value peak value to obtain a bus driver heart rate value;
the method for monitoring the environment of the bus driver's seat adopts visible light video analysis of the environment of the bus driver's seat to obtain whether behaviors harmful to the bus driver or possibly influencing the safe driving of the bus occur or not; the method comprises the following specific steps:
s101, calibration: when only a bus driver drives the bus and normal bus driving action is performed, the camera picture is stored and used as a calibration picture;
s102, data acquisition: periodically sampling visible light video of the environment of a bus driving platform in real-time running of a bus to obtain picture frames;
s103, inputting the calibrated picture and the picture frame into a trained driver' S cab environment danger early warning network at the same time;
s104, monitoring an environmental risk early warning network output result in real time;
the danger alarm method integrates the heart rate monitoring of a driver and the environment monitoring of a driving cab, and comprehensively judges and provides a danger alarm prompt; the method comprises the following specific steps:
s201, calculating the change rate of the heart rate value of the bus driver in real time; the heart rate value change rate calculation formula is as follows:
Δpt＝pt-pt-T；
t is a near-infrared video sampling period;
s202, calculating the average value of the heart rate value change rate intensity of the bus driver, wherein the calculation formula is as follows:
s203, calculating an inhibition factor lambda according to an output result of the environmental hazard early warning network:
s204, counting the number of times whether the ratio of the intensity of the heart rate value change rate of the bus driver to the mean value of the intensity of the heart rate value change rate of the bus driver is greater than a threshold value within MT time:
wherein μ (x) is a step function,
r is a threshold value used for judging that the intensity of the change rate of the heart rate value of the driver is larger than the threshold value;
and S205, only when n is larger than 0.5M, giving a danger alarm prompt.
Furthermore, the face positioning and the ROI interception are performed according to the face detection result, the face detection result is 9 key points of the face, and the key points are respectively a middle eye (p1), a left eye corner (p2), a right eye corner (p3), a left eye corner (p4), a right eye corner (p5), a left mouth corner (p6), a right mouth corner (p7), an upper mouth skin center (p8) and a lower mouth skin center (p 9);
the ROI is left forehead and right forehead, and the coordinates are respectively
And
is a center and a width of
Is high as
In the formula xi(i ═ 1.., 9), and yi(i ═ 1.., 9.) is p, respectivelyiCoordinate values of (i ═ 1.., 9).
Further, the calculating of the near-infrared intensity evaluation value of the intercepted ROI region first equally divides the ROI region into N equal parts in the y direction, and calculates the infrared intensity value of each equal part:
wherein pix (x, y) is
Pixel values of the inner points (x, y);
is C1The k-th aliquot, which is the central ROI;
is C2The k-th aliquot, which is the central ROI;
is composed of
The number of pixel points of the region;
second, calculate the mean:
then removing equal parts of 0.8-1.2 which exceed the mean value, and recalculating the mean value:
and finally, solving a near infrared intensity evaluation value of the intercepted ROI:
furthermore, the driver's cabin environment danger early warning network is a deep neural network, two standard VGG16 network structures are selected as processing feature sub-networks for calibrating pictures and picture frames, a BN layer is connected behind the two sub-networks, a user-defined DIFF layer is output and accessed, and the two sub-networks are combined into enhanced features with the same dimensionality; sequentially inputting the enhanced features into the three conv modules, obtaining output features through a GlobalPooling layer, and then connecting a judgment network consisting of a BN layer and a full link layer to obtain an output result;
the self-defined DIFF layer mainly completes the operation of obtaining a difference value of two output characteristic pixels, and the input dimension and the output dimension are the same; the method has the effects that the influence of fixed background information is removed from the output characteristics obtained by the picture frame, and the change information different from the fixed background information is more prominently displayed;
the three conv modules are connected by a maxpool layer; 3 × 3 convolution kernels are adopted in the maxpool layers, 2-time down-sampling is guaranteed, and the number of output and input characteristic channels is kept unchanged;
the three conv modules are formed by sequentially connecting a conv layer, a BN layer and a ReLU layer; three const layers respectively adopt three groups of parameters of 3 x 256, 3 x 128 and 3 x 64, and the width and height dimensions of input and output features are unchanged.
A system of a bus danger alarm method based on intelligent behavior monitoring is composed of a near-infrared camera, a visible light camera, a monitoring host, a storage device, a sound box, a wireless communication terminal and a remote hub center, wherein the near-infrared camera, the visible light camera, the monitoring host, the storage device, the sound box, the wireless communication terminal and the remote hub center are installed on a bus;
the near-infrared camera is arranged on the upper part right in front of the bus driver's seat, and samples the near-infrared video frames monitored on the front of the bus driver and sends the sampled near-infrared video frames to the monitoring host;
the visible light camera is arranged on the upper part of the right position of the bus driving platform, and is used for sampling and sending the monitored video frames of the bus driving platform environment to the monitoring host;
the monitoring host makes a danger alarm judgment according to an input near-infrared video frame and a bus driving platform environment video frame based on the intelligent behavior monitoring bus danger alarm method;
the storage device is connected with the monitoring host, and stores the current near-infrared video frame and the bus driving platform environment video frame when the danger alarm judgment is true, namely the bus has potential safety operation hazards;
the wireless communication terminal is connected with the monitoring host, and when the danger alarm judgment is true, the wireless communication terminal sends danger alarm early warning notification information to the remote hub center;
the danger alarm early warning notification information comprises a wireless terminal unique identifier, a bus unique identifier, a danger alarm judgment result, a current near-infrared video frame and a bus driving platform environment video frame;
the remote hub center receives danger alarm early warning notification information sent by wireless communication terminals of buses in all control ranges; manually intervene, study and judge to confirm the danger according to the danger alarm early warning notification information, send safety warning information and take corresponding measures;
the safety warning information comprises a target wireless terminal unique identifier, a target bus unique identifier and a safety warning record;
the wireless communication terminal installed on the bus receives the safety warning information and verifies the unique identification of the target wireless terminal and the unique identification of the target bus, and if the safety warning information is confirmed to be correct, the safety warning record is input into the sound box;
the sound equipment is connected with the wireless communication terminal and used for receiving and playing the safety warning record;
the corresponding measures comprise alarming, contacting with a bus driver with the unique bus identification contained in the danger alarm early warning notification information, recording and reporting to the superior.
The invention has the following beneficial effects:
the invention provides a bus danger alarm method and system based on intelligent behavior monitoring, which can automatically monitor the heart rate of a bus driver and the driving environment of the bus driver; the bus danger alarm judgment is completed and is synchronized to the center of the remote hub; the bus safety operation is guaranteed, and meanwhile, the manual work cost is reduced; in the automatic detection of the heart rate of the bus driver, the ROI is equally divided, and the multi-region discrete value elimination and averaging method reduces noise interference and improves the heart rate calculation precision.
Drawings
FIG. 1 is a flow chart of steps of a bus driver heart rate monitoring method.
Fig. 2 is a schematic diagram of outputting 9-point key points by a face detection result.
Detailed Description
The present invention is described in detail below by way of examples, it should be noted that the following examples are only for illustrating the present invention and should not be construed as limiting the scope of the present invention, and those skilled in the art can make some insubstantial modifications and adaptations of the present invention based on the above-described disclosure.
Example 1
A system of a bus danger alarm method based on intelligent behavior monitoring is composed of a near-infrared camera, a visible light camera, a monitoring host, a storage device, a sound, a wireless communication terminal and a remote hub center;
the system comprises a near-infrared camera, a monitoring host and a monitoring host, wherein the near-infrared camera is arranged at the upper part right in front of a bus driving platform, and samples and sends the monitored near-infrared video frames on the front of a bus driver to the monitoring host, so that the bus driver can be properly required not to shield the area of the face including the forehead in order to ensure the performance; the visible light camera is arranged on the upper part of the right position of the bus driving platform, and the monitored video frames of the bus driving platform environment are sampled and sent to the monitoring host;
the monitoring host has computing capability, and the bus danger alarm method based on intelligent behavior monitoring makes danger alarm judgment according to the input near-infrared video frame and bus driving platform environment video frame finishing operation;
the storage device is connected with the monitoring host, and stores the current near-infrared video frame and the bus driving platform environment video frame when the danger alarm judgment is true, namely the bus has potential safety operation hazards;
the wireless communication terminal is connected with the monitoring host, and when the danger alarm judgment is true, the wireless communication terminal sends danger alarm early warning notification information to the remote hub center; the danger alarm early warning notification information comprises a wireless terminal unique identifier, a bus unique identifier, a danger alarm judgment result, a current near-infrared video frame and a bus driving platform environment video frame;
the remote hub center supervises the bus wireless communication terminals in all control ranges, when receiving the danger alarm early warning notification information, the system manually intervenes, studies, judges and confirms danger according to the danger alarm early warning notification information, further ensures the accuracy of the whole judgment by a small amount of manual work under the condition that the system automatically judges and eliminates a large amount of interference, sends safety warning information and takes corresponding measures; the safety warning information comprises a target wireless terminal unique identifier, a target bus unique identifier and a safety warning record; the wireless communication terminal installed on the bus receives the safety warning information, verifies the unique identification of the target wireless terminal and the unique identification of the target bus, and if the safety warning information is identical to the local information, the wireless communication terminal inputs the safety warning record into a sound box connected with the wireless communication terminal to play the safety warning record;
meanwhile, the bus driver can be manually contacted, or the alarm processing can be carried out in serious scenes; and can report the information record.
Example 2
A bus danger alarm method and process based on intelligent behavior monitoring is subdivided into a bus driver heart rate monitoring method, a bus driver's platform environment monitoring method and a danger alarm method according to different actions;
the method comprises the steps that a bus driver heart rate monitoring method and a bus driving platform environment monitoring method respectively complete the monitoring of the heart rate of the driver and the monitoring of the environment of the driving platform, the heart rate monitoring result and the environment monitoring result are sent to a danger alarming method, and a danger alarming result is obtained through comprehensive judgment;
the method for monitoring the heart rate of the bus driver mainly comprises the steps of analyzing a near-infrared image of a specific area of the face of the driver to obtain a heart rate curve of the bus driver; the more specific process is shown in figure 1:
s001, periodically sampling a near-infrared video containing a front image of a bus driver; the sampling period is T; the video selection can adopt a window function to slide and intercept a video segment on the video of the whole time axis;
s002, performing face detection on each frame of image; wherein, the adopted human face detection model is finished by offline training and adopts a deep neural network; the output result is 9 key point coordinates of the face, as shown in fig. 2, which are respectively a human middle (p1), a left eye corner (p2), a left eye corner (p3), a right eye corner (p4), a right eye corner (p5), a left mouth corner (p6), a right mouth corner (p7), an upper mouth skin center (p8) and a lower mouth skin center (p 9);
s003, performing face positioning and ROI interception according to a face detection result; wherein ROI is left forehead and right forehead, and its coordinates are respectively
And
is a center and a width of
Is high as
In the formula xi(i ═ 1.., 9), and yi(i ═ 1.., 9.) is p, respectivelyiCoordinate values of (i ═ 1.., 9).
S004, calculating and intercepting a near infrared intensity evaluation value of the ROI; the more detailed process is that the ROI area is divided into N equal parts according to the y direction, and the infrared intensity value of each equal part is calculated:
wherein pix (x, y) is
Pixel values of the inner points (x, y);
is C1The k-th aliquot, which is the central ROI;
is C2The k-th aliquot, which is the central ROI;
is composed of
The number of pixel points of the region;
second, calculate the mean:
then removing equal parts of 0.8-1.2 which exceed the mean value, and recalculating the mean value:
and finally, solving a near infrared intensity evaluation value of the intercepted ROI:
s005, calculating the near-infrared intensity variation of the near-infrared intensity evaluation value of each frame of image compared with the near-infrared intensity of the previous frame;
s006, forming a variation sequence by the near infrared intensity variations of all frame images in the video segment; the time interval is a sampling period T;
s007, carrying out Fourier transform on the variable quantity sequence to obtain a frequency domain variable modulus;
s008, selecting a module value peak value, and solving a heart rate value of a bus driver by adopting Fourier inverse transformation;
for the bus driving platform environment monitoring method, the visible light video analysis of the bus driving platform environment is adopted to obtain whether the behavior harmful to the bus driver or possibly influencing the safe driving of the bus occurs; the method comprises the following specific steps:
s101, calibration: when only a bus driver drives the bus and normal bus driving action is performed, the camera picture is stored and used as a calibration picture;
s102, data acquisition: periodically sampling visible light video of the environment of a bus driving platform in real-time running of a bus to obtain picture frames;
s103, inputting the calibrated picture and the picture frame into a trained driver' S cab environment danger early warning network at the same time;
s104, monitoring an environmental risk early warning network output result in real time;
in order to ensure the performance, wherein the driver's cabin environment danger early warning network is a deep neural network, two standard VGG16 network structures are selected as processing feature sub-networks for calibrating pictures and picture frames, a BN layer is connected behind the two sub-networks, and a self-defined DIFF layer is output and accessed and combined into an enhancement feature with the same dimension; sequentially inputting the enhanced features into the three conv modules, obtaining output features through a GlobalPooling layer, and then connecting a judgment network consisting of a BN layer and a full link layer to obtain an output result;
the user-defined DIFF layer mainly completes the operation of obtaining a difference value of two output characteristic pixels, and the input dimension and the output dimension are the same; the method has the effects that the influence of fixed background information is removed from the output characteristics obtained by the picture frame, and the change information different from the fixed background information is more prominently displayed;
connecting the three conv modules by a maxpool layer; 3 × 3 convolution kernels are adopted in the maxpool layers, 2-time down-sampling is guaranteed, and the number of output and input characteristic channels is kept unchanged; the three conv modules are formed by sequentially connecting a conv layer, a BN layer and a ReLU layer; three conv layers adopt three groups of parameters of 3 × 256, 3 × 128 and 3 × 64 respectively, and the width and height dimensions of input and output features are unchanged
The danger alarm method integrates the heart rate monitoring of the driver and the environment monitoring of the driving platform, and comprehensively judges and provides a danger alarm prompt; the method comprises the following specific steps:
s201, calculating the change rate of the heart rate value of the bus driver in real time; the heart rate value change rate calculation formula is as follows:
Δpt＝pt-pt-T；
t is a near-infrared video sampling period;
s202, calculating the average value of the heart rate value change rate intensity of the bus driver, wherein the calculation formula is as follows:
s203, calculating an inhibition factor lambda according to an output result of the environmental hazard early warning network:
s204, counting the number of times whether the ratio of the intensity of the heart rate value change rate of the bus driver to the mean value of the intensity of the heart rate value change rate of the bus driver is greater than a threshold value within MT time:
wherein μ (x) is a step function,
r is a threshold value used for judging that the intensity of the change rate of the heart rate value of the driver is larger than the threshold value;
and S205, only when n is larger than 0.5M, giving a danger alarm prompt.
Claims (5)
1. A bus danger alarm method based on intelligent behavior monitoring is characterized in that: the method comprises a bus driver heart rate monitoring method, a bus driving platform environment monitoring method and a danger alarming method;
the method for monitoring the heart rate of the bus driver and the method for monitoring the environment of the bus driving platform respectively complete the heart rate monitoring of the driver and the environment monitoring of the driving platform, send the heart rate monitoring result and the environment monitoring result to a danger alarm method, and comprehensively judge to obtain a danger alarm result;
the method for monitoring the heart rate of the bus driver comprises the steps of analyzing a near-infrared image of a specific area of the face of the driver to obtain a heart rate curve of the bus driver; the method comprises the following specific steps:
s001, periodically sampling a near-infrared video containing a front image of a bus driver; the sampling period is preferably T;
s002, performing face detection on each frame of image;
s003, performing face positioning and ROI interception according to a face detection result;
s004, calculating and intercepting a near infrared intensity evaluation value of the ROI;
s005, calculating the near-infrared intensity variation of the near-infrared intensity evaluation value of each frame of image compared with the near-infrared intensity of the previous frame;
s006, forming a variation sequence by the near infrared intensity variations of all frame images in the video segment; the time interval is a sampling period T;
s007, carrying out Fourier transform on the variable quantity sequence to obtain a frequency domain variable modulus;
s008, selecting a module value peak value to obtain a bus driver heart rate value;
the method for monitoring the environment of the bus driver's seat adopts visible light video analysis of the environment of the bus driver's seat to obtain whether behaviors harmful to the bus driver or possibly influencing the safe driving of the bus occur or not; the method comprises the following specific steps:
s101, calibration: when only a bus driver drives the bus and normal bus driving action is performed, the camera picture is stored and used as a calibration picture;
s102, data acquisition: periodically sampling visible light video of the environment of a bus driving platform in real-time running of a bus to obtain picture frames;
s103, inputting the calibrated picture and the picture frame into a trained driver' S cab environment danger early warning network at the same time;
s104, monitoring an environmental risk early warning network output result in real time;
the danger alarm method integrates the heart rate monitoring of a driver and the environment monitoring of a driving cab, and comprehensively judges and provides a danger alarm prompt; the method comprises the following specific steps:
s201, calculating the change rate of the heart rate value of the bus driver in real time; the heart rate value change rate calculation formula is as follows:
Δpt＝pt-pt-T；
t is a near-infrared video sampling period;
s202, calculating the average value of the heart rate value change rate intensity of the bus driver, wherein the calculation formula is as follows:
s203, calculating an inhibition factor lambda according to an output result of the environmental hazard early warning network:
s204, counting the number of times whether the ratio of the intensity of the heart rate value change rate of the bus driver to the mean value of the intensity of the heart rate value change rate of the bus driver is greater than a threshold value within MT time:
wherein μ (x) is a step function,
r is a threshold value used for judging that the intensity of the change rate of the heart rate value of the driver is larger than the threshold value;
and S205, only when n is larger than 0.5M, giving a danger alarm prompt.
2. The bus danger alarm method based on intelligent behavior monitoring as claimed in claim 1, characterized in that: the face positioning and ROI interception are performed according to a face detection result, the face detection result is 9 key points of the face, namely a human middle (p1), a left eye corner (p2), a left eye right eye corner (p3), a right eye left eye corner (p4), a right eye corner (p5), a left mouth corner (p6), a right mouth corner (p7), an upper mouth skin center (p8) and a lower mouth skin center (p 9);
the ROI is left forehead and right forehead, and the coordinates are respectively
And
is a center and a width of
Is high as
In the formula xi(i ═ 1.., 9), and yi(i ═ 1.., 9.) is p, respectivelyiCoordinate values of (i ═ 1.., 9).
3. The bus danger alarm method based on intelligent behavior monitoring as claimed in claim 2, characterized in that: the calculation of the near-infrared intensity evaluation value of the intercepted ROI area firstly equally divides the ROI area into N equal parts according to the y direction, and calculates the infrared intensity value of each equal part:
wherein pix (x, y) is
Pixel values of the inner points (x, y);
is C1The k-th aliquot, which is the central ROI;
is C2The k-th aliquot, which is the central ROI;
is composed of
The number of pixel points of the region;
second, calculate the mean:
then removing equal parts of 0.8-1.2 which exceed the mean value, and recalculating the mean value:
and finally, solving a near infrared intensity evaluation value of the intercepted ROI:
4. the bus danger alarm method based on intelligent behavior monitoring as claimed in claim 3, characterized in that: the driver's cabin environment danger early warning network is a deep neural network, two standard VGG16 network structures are selected as processing feature sub-networks for calibrating pictures and picture frames, a BN layer is connected behind the two sub-networks, a self-defined DIFF layer is output and accessed, and the two sub-networks are combined into enhancement features with the same dimension; sequentially inputting the enhanced features into the three conv modules, obtaining output features through a GlobalPooling layer, and then connecting a judgment network consisting of a BN layer and a full link layer to obtain an output result;
the self-defined DIFF layer mainly completes the operation of obtaining a difference value of two output characteristic pixels, and the input dimension and the output dimension are the same;
the three conv modules are connected by a maxpool layer; 3 × 3 convolution kernels are adopted in the maxpool layers, 2-time down-sampling is guaranteed, and the number of output and input characteristic channels is kept unchanged;
the three conv modules are formed by sequentially connecting a conv layer, a BN layer and a ReLU layer; three const layers respectively adopt three groups of parameters of 3 x 256, 3 x 128 and 3 x 64, and the width and height dimensions of input and output features are unchanged.
5. A system of a bus danger alarm method based on intelligent behavior monitoring is characterized in that: the system comprises a near-infrared camera, a visible light camera, a monitoring host, a storage device, a sound box, a wireless communication terminal and a remote hub center, wherein the near-infrared camera, the visible light camera, the monitoring host, the storage device, the sound box, the wireless communication terminal and the remote hub center are installed on a bus;
the near-infrared camera is arranged on the upper part right in front of the bus driver's seat, and samples the near-infrared video frames monitored on the front of the bus driver and sends the sampled near-infrared video frames to the monitoring host;
the visible light camera is arranged on the upper part of the right position of the bus driving platform, and is used for sampling and sending the monitored video frames of the bus driving platform environment to the monitoring host;
the monitoring host makes a danger alarm judgment according to an input near-infrared video frame and a bus driving platform environment video frame based on the intelligent behavior monitoring bus danger alarm method;
the storage device is connected with the monitoring host, and stores the current near-infrared video frame and the environmental video frame of the bus driving platform when the danger alarm is judged to be true;
the wireless communication terminal is connected with the monitoring host, and when the danger alarm judgment is true, the wireless communication terminal sends danger alarm early warning notification information to the remote hub center;
the danger alarm early warning notification information comprises a wireless terminal unique identifier, a bus unique identifier, a danger alarm judgment result, a current near-infrared video frame and a bus driving platform environment video frame;
the remote hub center receives danger alarm early warning notification information sent by wireless communication terminals of buses in all control ranges; manually intervene, study and judge to confirm the danger according to the danger alarm early warning notification information, send safety warning information and take corresponding measures;
the safety warning information comprises a target wireless terminal unique identifier, a target bus unique identifier and a safety warning record;
the wireless communication terminal installed on the bus receives the safety warning information and verifies the unique identification of the target wireless terminal and the unique identification of the target bus, and if the safety warning information is confirmed to be correct, the safety warning record is input into the sound box;
the sound equipment is connected with the wireless communication terminal and used for receiving and playing the safety warning record;
the corresponding measures comprise alarming, contacting with a bus driver with the unique bus identification contained in the danger alarm early warning notification information, recording and reporting to the superior.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010331548.XA CN111540169A (en) | 2020-04-24 | 2020-04-24 | Bus danger alarm method and system based on intelligent behavior monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010331548.XA CN111540169A (en) | 2020-04-24 | 2020-04-24 | Bus danger alarm method and system based on intelligent behavior monitoring |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111540169A true CN111540169A (en) | 2020-08-14 |
Family
ID=71977128
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010331548.XA Pending CN111540169A (en) | 2020-04-24 | 2020-04-24 | Bus danger alarm method and system based on intelligent behavior monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111540169A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103456146A (en) * | 2012-05-28 | 2013-12-18 | 哈尔滨工业大学深圳研究生院 | Intelligent vehicle-mounted cloud alarm service system and intelligent vehicle-mounted cloud alarm service method |
| CN105769222A (en) * | 2016-02-16 | 2016-07-20 | 北京博研智通科技有限公司 | Method, device and wearable device for detecting drive state based on heart rate variability |
| CN106778695A (en) * | 2017-01-19 | 2017-05-31 | 北京理工大学 | A kind of many people's examing heartbeat fastly methods based on video |
| CN107235045A (en) * | 2017-06-29 | 2017-10-10 | 吉林大学 | Consider physiology and the vehicle-mounted identification interactive system of driver road anger state of manipulation information |
| CN108805284A (en) * | 2018-05-23 | 2018-11-13 | 哈尔滨工业大学深圳研究生院 | A kind of method and its application of convolutional neural networks receptive field quantization |
| CN110236515A (en) * | 2019-07-19 | 2019-09-17 | 合肥工业大学 | A kind of contactless heart rate detection method based on near-infrared video |
| US20200085366A1 (en) * | 2018-09-14 | 2020-03-19 | Warsaw Orthopedic, Inc. | Wearable sensor device and analysis platform for objective outcome assessment in spinal diseases |
-
2020
- 2020-04-24 CN CN202010331548.XA patent/CN111540169A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103456146A (en) * | 2012-05-28 | 2013-12-18 | 哈尔滨工业大学深圳研究生院 | Intelligent vehicle-mounted cloud alarm service system and intelligent vehicle-mounted cloud alarm service method |
| CN105769222A (en) * | 2016-02-16 | 2016-07-20 | 北京博研智通科技有限公司 | Method, device and wearable device for detecting drive state based on heart rate variability |
| CN106778695A (en) * | 2017-01-19 | 2017-05-31 | 北京理工大学 | A kind of many people's examing heartbeat fastly methods based on video |
| CN107235045A (en) * | 2017-06-29 | 2017-10-10 | 吉林大学 | Consider physiology and the vehicle-mounted identification interactive system of driver road anger state of manipulation information |
| CN108805284A (en) * | 2018-05-23 | 2018-11-13 | 哈尔滨工业大学深圳研究生院 | A kind of method and its application of convolutional neural networks receptive field quantization |
| US20200085366A1 (en) * | 2018-09-14 | 2020-03-19 | Warsaw Orthopedic, Inc. | Wearable sensor device and analysis platform for objective outcome assessment in spinal diseases |
| CN110236515A (en) * | 2019-07-19 | 2019-09-17 | 合肥工业大学 | A kind of contactless heart rate detection method based on near-infrared video |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109034036B (en) | Video analysis method, teaching quality assessment method and system and computer-readable storage medium | |
| CN112016457A (en) | Driver distraction and dangerous driving behavior recognition method, device and storage medium | |
| CN111402247B (en) | Machine vision-based method for detecting defects of suspension clamp on power transmission line | |
| CN112102409B (en) | Target detection method, device, equipment and storage medium | |
| EP3073440A1 (en) | Method and device for correcting recovery and analyzing alarm of distorted image | |
| CN105744232A (en) | Method for preventing power transmission line from being externally broken through video based on behaviour analysis technology | |
| CN108596087B (en) | Driving fatigue degree detection regression model based on double-network result | |
| CN111661059B (en) | Method and system for monitoring distracted driving and electronic equipment | |
| CN111854620B (en) | Monocular camera-based actual pupil distance measuring method, device and equipment | |
| CN110543848B (en) | Driver action recognition method and device based on three-dimensional convolutional neural network | |
| CN116416577A (en) | Abnormality identification method for construction monitoring system | |
| CN112703532B (en) | Image processing method, device, equipment and storage medium | |
| CN110677763A (en) | State detection device and earphone | |
| CN115345774A (en) | Method and system for fusing infrared image and visible light image | |
| CN111444837A (en) | Temperature measurement method and temperature measurement system for improving face detection availability in extreme environment | |
| CN111540169A (en) | Bus danger alarm method and system based on intelligent behavior monitoring | |
| CN112714308B (en) | Method and device for detecting video rolling stripes | |
| CN116342891B (en) | Structured teaching monitoring data management system suitable for autism children | |
| CN109255793B (en) | A kind of monitoring early-warning system of view-based access control model feature | |
| JP2018160128A (en) | Image processing apparatus, image processing system, and image processing method | |
| CN109460812B (en) | Intermediate information analysis device, optimization device, and feature visualization device for neural network | |
| CN111513697A (en) | Intelligent control system and method for driving safety | |
| CN112036217B (en) | Person state detection method and device based on mouth information | |
| CN114627434A (en) | Automobile sales exhibition room passenger flow identification system based on big data | |
| CN114298189A (en) | Fatigue driving detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200814 |