CN111166302A - Device and method for monitoring physiological indexes in real time in non-contact manner - Google Patents
Device and method for monitoring physiological indexes in real time in non-contact manner Download PDFInfo
- Publication number
- CN111166302A CN111166302A CN201911357395.XA CN201911357395A CN111166302A CN 111166302 A CN111166302 A CN 111166302A CN 201911357395 A CN201911357395 A CN 201911357395A CN 111166302 A CN111166302 A CN 111166302A
- Authority
- CN
- China
- Prior art keywords
- physiological
- face
- human face
- blood spectrum
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 12
- 238000000034 method Methods 0.000 title claims description 20
- 239000008280 blood Substances 0.000 claims abstract description 94
- 210000004369 blood Anatomy 0.000 claims abstract description 94
- 238000001228 spectrum Methods 0.000 claims abstract description 90
- 238000012545 processing Methods 0.000 claims abstract description 22
- 238000010183 spectrum analysis Methods 0.000 claims abstract description 11
- 230000003287 optical effect Effects 0.000 claims abstract description 10
- 230000036772 blood pressure Effects 0.000 claims description 19
- 238000001514 detection method Methods 0.000 claims description 17
- 230000029058 respiratory gaseous exchange Effects 0.000 claims description 15
- 230000006835 compression Effects 0.000 claims description 9
- 238000007906 compression Methods 0.000 claims description 9
- 238000012856 packing Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 6
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 230000001815 facial effect Effects 0.000 claims 1
- 230000002159 abnormal effect Effects 0.000 abstract description 5
- 230000008447 perception Effects 0.000 abstract description 2
- 230000004044 response Effects 0.000 abstract description 2
- 238000005070 sampling Methods 0.000 description 15
- 102000001554 Hemoglobins Human genes 0.000 description 4
- 108010054147 Hemoglobins Proteins 0.000 description 4
- 210000001367 artery Anatomy 0.000 description 3
- 210000004204 blood vessel Anatomy 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000036651 mood Effects 0.000 description 2
- 230000000241 respiratory effect Effects 0.000 description 2
- 206010010071 Coma Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 210000001061 forehead Anatomy 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 210000000088 lip Anatomy 0.000 description 1
- 210000001331 nose Anatomy 0.000 description 1
- 210000004279 orbit Anatomy 0.000 description 1
- 230000036387 respiratory rate Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/117—Identification of persons
- A61B5/1171—Identification of persons based on the shapes or appearances of their bodies or parts thereof
- A61B5/1176—Recognition of faces
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7225—Details of analog processing, e.g. isolation amplifier, gain or sensitivity adjustment, filtering, baseline or drift compensation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Pathology (AREA)
- Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Physiology (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Veterinary Medicine (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Cardiology (AREA)
- Power Engineering (AREA)
- Pulmonology (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
A device for monitoring physiological indexes in a real-time non-contact manner comprises a video acquisition device, a face acquisition device, a blood spectrum analysis device, an early warning device and a display device, wherein the video acquisition device is electrically connected with the face acquisition device and the display device; the blood spectrum physiological analysis device is electrically connected with the face acquisition device, the early warning device and the display device; the early warning device is also electrically connected with the display device; the video acquisition device comprises a camera; the camera is provided with an infrared device and comprises a lens, an optical sensor, an image processing device and an encoder; by adopting the invention, the physiological fluctuation condition of the audited personnel can be known under the condition that the audited personnel has no physiological perception, and the response can be timely made to the abnormal physiological fluctuation of the audited personnel.
Description
Technical Field
The invention relates to the field of image recognition, in particular to a device and a method for monitoring physiological indexes in a real-time non-contact manner.
Background
In the interrogation process, the auditor can ask corresponding problems according to the audited personnel to achieve the purpose of inquiry. However, in the actual interrogation process, the presented problems may cause the excited mood of the interrogated personnel, which may cause rapid fluctuation of the heart rate, blood pressure and the like of the interrogated personnel, and may cause the excited mood of the interrogated personnel if the situation is mild, and may cause the interrogated personnel to have accidents such as coma, stroke and the like if the situation is severe, so that the wanted information is not obtained, and the physical health of the interrogated personnel is endangered and is not paid. Therefore, it is necessary to find out the abnormal physiological change of the person under interrogation in time during the interrogation. But may cause conflict with the audited personnel if a contact-based physiological sensing device is used.
Because human skin is semitransparent, light can be transmitted to an arterial blood vessel layer below the skin, hemoglobin in the arterial blood vessel can absorb part of the transmitted light, and the rest light can form reflected light at the arterial blood vessel layer; the hemoglobin density of the artery blood vessel fluctuates along with the heart beat along with the contraction and the relaxation of the heart beat of the human body, when the heart contracts, the hemoglobin density in the artery blood vessel becomes high, more transmitted light is absorbed, and the intensity of reflected light becomes weak, and when the heart dilates, the hemoglobin density in the artery blood vessel becomes low, less transmitted light is absorbed, and the intensity of reflected light becomes strong.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a device and a method for monitoring physiological indexes in a real-time non-contact manner, which have simple structure and convenient use.
A device for monitoring physiological indexes in a real-time non-contact manner comprises a video acquisition device, a face acquisition device, a blood spectrum analysis device, an early warning device and a display device, wherein the video acquisition device is electrically connected with the face acquisition device and the display device; the blood spectrum physiological analysis device is electrically connected with the face acquisition device, the early warning device and the display device; the early warning device is also electrically connected with the display device; the video acquisition device comprises a camera; the camera is provided with an infrared device and comprises a lens, an optical sensor, an image processing device and an encoder.
A real-time non-contact physiological index monitoring method based on the device comprises the following steps:
step 1: the video acquisition device acquires a non-compressed video stream and transmits the non-compressed video stream to the face acquisition device; the video acquisition device acquires the coded video stream and transmits the coded video stream to the display device;
step 2: the human face detection device receives the uncompressed video stream, performs primary processing to obtain human face blood spectrum information, packs the human face blood spectrum information with set time length and sends the packed human face blood spectrum information to the blood spectrum physiological analysis device;
and step 3: the blood spectrum physiological analysis device receives the human face blood spectrum information compression packet, decompresses the human face blood spectrum information to obtain human face blood spectrum information, analyzes physiological indexes to obtain a physiological index analysis result, and transmits the physiological index analysis result to the display device and the early warning device;
and 4, step 4: the early warning device receives the physiological index analysis result, performs index comparison and predicts an index trend line, transmits the index trend line to the display device, and judges whether to send warning information to the display device according to the index comparison result and the index trend line;
and 5: and (3) the display device receives the coded video stream, the physiological index analysis result and the index trend line, displays the result in real time and returns to the step 1.
Further, the step of acquiring, by the video acquiring apparatus in step 1, the uncompressed video stream and the encoded video stream includes:
step 1.1: the optical sensor captures light rays input into the lens, converts optical signals into electric signals and transmits the electric signals to the image processing device;
step 1.2: the image processing device receives the electric signal, converts the electric signal into frame images, arranges the frame images according to time sequence to obtain a real-time non-compressed video stream with a set frame rate, and transmits the non-compressed video stream to the face acquisition device and the encoder;
step 1.3: the encoder receives the uncompressed video stream, performs h.264 encoding to obtain an encoded video stream, and transmits the encoded video stream to the display device.
Further, in step 2, the step of performing preliminary processing by the face detection device includes:
step 2.1: selecting a frame of image according to the time sequence, and carrying out face tracking detection on the image to obtain face characteristic points;
step 2.2: positioning a face block according to the distribution of the face feature points;
step 2.3: extracting the block information of each face block to obtain the face blood spectrum information of the frame image;
step 2.4: processing the rest of each frame image in the step 2.1-2.3 to obtain face blood spectrum information for a period of time and packaging;
step 2.5: and transmitting the packed human face blood spectrum information to a blood spectrum analysis device.
Further, in the step 2.4, the packing of the human face blood spectrum information includes the following steps:
step 2.41: obtaining the average value sequence of each specific block in each frame { (Block number, frame number i, R)i,Gi,Bi) I is not less than 1 and not more than s v, s represents the set time length of the packed face blood spectrum information, v represents the number of frame images contained in each second of the video, and R, G, B represents RGB information of the face blocks;
step 2.42: sorting the average value sequence of the specific blocks obtained in the step 1.41 according to the block numbers to obtain a frame image sequence { (k1,1, R)1,G1,B1),…,(k1,s*v,RS*V,GS*V,BS*V) }; sequencing s x v frame image sequences in time to obtain a sequence of k1 blocks;
step 2.43: respectively obtaining sequences of k2 and … kj blocks; wherein j represents the number of the face characteristic points in the frame image;
step 2.44: acquiring a face number and a timestamp of a first frame image; the human face number is the number of human face data acquired by the image acquisition device;
step 2.45: packing the human face blood spectrum information with the time length of s seconds according to the sequence of { human face number, the time stamp of the 1 st frame, the packing frame number, the k1 block sequence and the k2 block sequence … kj block sequence }; the timestamp of the 1 st frame represents the timestamp of the 1 st frame image in the s duration range.
Further, in the step 3, the analysis of the physiological indexes includes the analysis of heart rate, respiration and blood pressure;
the analysis process of the blood spectrum analysis device comprises the following steps:
step 3.1: receiving a human face blood spectrum information compression packet, and decompressing to obtain human face blood spectrum information;
step 3.2: combining the human face blood spectrum information according to time sequence for the specific blocks respectively; the combined human face blood spectrum information is obtained by splicing the contents of one or more human face blood spectrum information compression packets according to the analysis requirements of physiological indexes in a time sequence to form human face blood spectrum information with set time duration; the specific block is determined according to a detection target;
step 3.3: performing physiological index analysis, wherein the physiological index analysis comprises the following steps:
step 3.31: performing wavelet function filtering processing on the human face blood spectrum information with the set time length obtained in the step 3.2;
step 3.32: slicing the filtered human face blood spectrum information according to a set time length;
step 3.33: fourier transform is carried out on the face blood spectrum information after each slice to obtain a target spectrum corresponding to the slice;
step 3.34: averaging each target frequency spectrum, and sequencing according to a time sequence to obtain a target variation curve;
step 3.4: selecting the target variation curve of the specific block as a final physiological index analysis result with the best signal quality;
step 3.5: and (4) sending the physiological index analysis result obtained in the step (3.4) to an early warning device and a display device.
Further, in the step 4, the early warning device is provided with the highest and lowest warning lines of heart rate, respiration and blood pressure; the steps of the early warning device for comparing indexes and predicting the index trend line comprise:
step 4.1: receiving a physiological index analysis result output by a section of blood spectrum physiological analysis device;
step 4.2: comparing and judging whether the heart rate, respiration and blood pressure indexes touch the highest and lowest warning lines, and if the highest and lowest warning lines touch the warning lines, sending warning information to a display device; if the warning line is not touched, turning to the step 4.3;
step 4.3, obtaining an index trend line according to real-time heart rate, respiration and blood pressure indexes, transmitting the index trend line to a display device, judging whether the index trend line touches a highest warning line or a lowest warning line, and if the index trend line touches the warning line, sending trend warning information to the display device; otherwise, go to step 4.1.
Further, in the step 5, the display device comprises a face video area, a physiological curve area and an alarm area; the display device receives the real-time face video stream output by the face acquisition device and displays the real-time face video stream in the face video area; receiving a real-time physiological index analysis result output by the blood spectrum physiological analysis device, and displaying the real-time physiological index analysis result in a physiological curve area in real time; receiving an index trend line output by the early warning device, displaying the index trend line in a physiological curve area, and overlapping the index trend line on a real-time physiological index analysis result; and receiving the alarm and trend alarm information output by the early warning device, and displaying the alarm and trend alarm information in the alarm area.
Further, the physiological indexes comprise heart rate, respiration and blood pressure indexes; the physiological indexes are displayed in a coordinate system or one physiological index is displayed corresponding to one coordinate system.
The invention has the beneficial effects that:
by adopting the invention, the physiological fluctuation condition of the audited personnel can be known under the condition that the audited personnel has no physiological perception, and the response can be timely made to the abnormal physiological fluctuation of the audited personnel;
the early warning device provided by the invention can obtain real-time abnormal physiological fluctuation and give an alarm to the abnormal physiological fluctuation, and can also obtain an index trend line of a physiological index, predict the trend of the physiological index and perform early warning in time;
the invention is provided with a display device, which can display the coded video stream, the physiological index analysis result, the index trend line and the alarm information in real time, and is convenient for visually observing the physiological fluctuation condition of the person under investigation.
Drawings
FIG. 1 is a block diagram of the present invention;
FIG. 2 is a schematic view of a display device according to the present invention;
FIG. 3 is a block diagram of a face detection apparatus according to the present invention;
FIG. 4 is a block diagram of a flow chart of a blood spectrum analyzer according to the present invention;
FIG. 5 is a heart rate trend line of the present invention;
FIG. 6 is a respiratory trend line of the present invention;
FIG. 7 is a blood pressure trend line of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
Example one
A device for monitoring physiological indexes in a non-contact mode in real time comprises a video acquisition device, a human face acquisition device, a blood spectrum analysis device, an early warning device and a display device.
The video acquisition device is electrically connected with the face acquisition device and the display device; the blood spectrum physiological analysis device is electrically connected with the face acquisition device, the early warning device and the display device; the early warning device is also electrically connected with the display device.
The video acquisition device comprises a camera, the camera is provided with an infrared device, and an infrared mode or a non-infrared mode can be selected according to setting. The frame rate of the real-time video captured by the camera can be adjusted as required, in this embodiment 30 frames per second. The camera includes a lens, an optical sensor, an image processing device, and an encoder.
As shown in fig. 1, a method for monitoring a physiological index device includes the following steps:
step 1: the video acquisition device acquires a non-compressed video stream and transmits the non-compressed video stream to the face acquisition device; the video acquisition device acquires the coded video stream and transmits the coded video stream to the display device;
step 2: the human face detection device receives the uncompressed video stream, performs primary processing to obtain human face blood spectrum information, packs the human face blood spectrum information with set time length and sends the packed human face blood spectrum information to the blood spectrum physiological analysis device;
and step 3: the blood spectrum physiological analysis device receives the human face blood spectrum information compression packet, decompresses the human face blood spectrum information to obtain human face blood spectrum information, analyzes physiological indexes to obtain a physiological index analysis result, and transmits the physiological index analysis result to the display device and the early warning device;
and 4, step 4: the early warning device receives the physiological index analysis result, performs index comparison and predicts an index trend line, transmits the index trend line to the display device, and judges whether to send warning information to the display device according to the index comparison result and the index trend line;
and 5: and (3) the display device receives the coded video stream, the physiological index analysis result and the index trend line, displays the result in real time and returns to the step 1.
In step 1, the video acquisition device is responsible for acquiring a real-time video of the face of the audited person, and the real-time video needs the face information of the audited person. The video acquisition device acquires the uncompressed video stream and the coded video stream, and comprises the following steps:
step 1.1: the optical sensor captures light rays input into the lens, converts optical signals into electric signals and transmits the electric signals to the image processing device;
step 1.2: the image processing device receives the electric signal, converts the electric signal into frame images, arranges the frame images according to time sequence to obtain a real-time non-compressed video stream with a set frame rate, and transmits the non-compressed video stream to the face acquisition device and the encoder;
step 1.3: the encoder receives the uncompressed video stream, performs h.264 encoding to obtain an encoded video stream, and transmits the encoded video stream to the display device.
As shown in fig. 3, in step 2, the step of performing the preliminary processing by the face detection apparatus includes:
step 2.1: selecting a frame of image according to the time sequence, and carrying out face tracking detection on the image to obtain face characteristic points;
step 2.2: positioning a face block according to the distribution of the face feature points;
step 2.3: extracting the block information of each face block to obtain the face blood spectrum information of the frame image;
step 2.4: processing the rest of each frame image in the step 2.1-2.3 to obtain face blood spectrum information for a period of time and packaging;
step 2.5: and transmitting the packed human face blood spectrum information to a blood spectrum analysis device.
In step 2.1, the face tracking detection is to detect and track a face in the image, and obtain a face rotation angle and a face detection frame in the image. In step 2.2, the human face feature points comprise lips, nose, eye sockets, eyebrows, forehead and human face contour. J face characteristic points are obtained, j is larger than or equal to 1, and the face blocks are divided according to the coordinate values of the face characteristic points in the face detection frame. In step 2.3, the block information includes RGB or gray value information of the face block, and different physiological/psychological indexes correspond to the block information of different blocks, wherein for the color image, the RGB information of the face block is extracted; for a black-and-white image, grayscale value information of a face image is extracted, and R ═ G ═ B ═ grayscale value. In this embodiment, in order to improve the accuracy, a plurality of related block information may be collected for the physiological indexes to be detected at the same time. In step 2.4, the time range of the face blood spectrum information to be packed can be adjusted according to the setting. And setting the video to be packed once every s seconds, wherein each second of the video comprises v frames of images, dividing the images into j face blocks according to the number of the face characteristic points in each frame of image, numbering the face blocks, and marking the face blocks as k1, k2 and … kj respectively. The packing of the human face blood spectrum information comprises the following steps:
step 2.41: obtaining the average value sequence of each specific block in each frame { (Block number, frame number i, R)i,Gi,Bi)}，1≤i≤s*v；
Step 2.42: sorting the average value sequence of the specific blocks obtained in the step 1.41 according to the block numbers to obtain a frame image sequence { (k1,1, R)1,G1,B1),…,(k1,s*v,RS*V,GS*V,BS*V) }; sequencing s x v frame image sequences in time to obtain a sequence of k1 blocks;
step 2.43: respectively obtaining sequences of k2 and … kj blocks;
step 2.44: acquiring a face number and a timestamp of a first frame image; the face number is the number of face data acquired by the image acquisition device, for example, the face number of the 1 st s-second-duration face video is 1;
step 2.45: packing the human face blood spectrum information with the time length of s seconds according to the sequence of { human face number, the time stamp of the 1 st frame, the packing frame number, the k1 block sequence and the k2 block sequence … kj block sequence }; the timestamp of the 1 st frame represents the timestamp of the 1 st frame image in the s duration range.
In step 3, the blood spectrum analysis device receives the packed human face blood spectrum information from the human face acquisition device, and performs physiological index analysis on the human face blood spectrum information, wherein the physiological index analysis comprises the analysis of heart rate, respiration and blood pressure.
As shown in fig. 4, the analysis process of the blood spectrum analysis device includes the following steps:
step 3.1: receiving a human face blood spectrum information compression packet, and decompressing to obtain human face blood spectrum information;
step 3.2: combining the human face blood spectrum information according to time sequence for the specific blocks respectively; the combined human face blood spectrum information is obtained by splicing the contents of one or more human face blood spectrum information compression packets according to the analysis requirements of physiological indexes in a time sequence to form human face blood spectrum information with set time duration; the specific block is determined according to a detection target, such as heart rate detection corresponding blocks k1 and k 4;
step 3.3: performing physiological index analysis, wherein the physiological index analysis comprises the following steps:
step 3.31: performing wavelet function filtering processing on the human face blood spectrum information with the set time length obtained in the step 3.2; the purpose is to filter out information of non-target frequency bands, taking heart rate analysis as an example, wavelet function filtering can be carried out to filter out information of non-heart rate frequency bands;
step 3.32: slicing the filtered human face blood spectrum information according to a set time length;
step 3.33: performing Fourier transform on the face blood spectrum information after each slice to obtain a target spectrum corresponding to the slice, such as a heart rate spectrum;
step 3.34: averaging each target frequency spectrum, and sequencing according to a time sequence to obtain a target variation curve, such as a heart rate variation curve;
step 3.4: selecting the target variation curve of the specific block as a final physiological index analysis result with the best signal quality;
step 3.5: and (4) sending the physiological index analysis result obtained in the step (3.4) to an early warning device and a display device.
Wherein, the face blood spectrum information after slicing in step 3.32 can be overlapped or partially overlapped. In step 3.4, the signal quality of the specific block is determined according to the signal-to-noise ratio, and the higher the signal-to-noise ratio, the better the signal quality is considered. In the implementation process of step 3.3, taking the heart rate analysis of the G signal in the RGB information of the k1 block as an example, 30 frames per second of sampling time duration is set asFor 60 seconds, 1800 sampling data are obtained, and the sampling data form a sampling set { G }1,…,Gi,….G1800}; sending the 1800 sampling data into a wavelet transform filter to obtain x data of a frequency information segment with the frequency range of 0.67-2.33Hz, wherein x is less than or equal to 1800, and 0.67-2.33Hz is the heart rate range of a human body under normal conditions; adopting a discrete Fourier transform formula of 150 sampling points, and taking 150 points as the slice length; calculate the average heart rate for each sample point, taking the 5 th second heart rate as an example: the sampling point of the 5 th second is No. 150, 75 sampling points before and after the sampling point is selected, namely No. 76 sampling point to No. 225 sampling point form 150 sampling data, and 150 { frequency points, amplitude } data are obtained by inputting a discrete Fourier transform formula; selecting sampling data with the maximum amplitude from 150 pieces of { frequency point, amplitude } data, and taking a frequency point 60 in the sampling data as an average heart rate of 5 seconds; and forming a heart rate variation curve by the average heart rate of each sampling point according to the time sequence.
As shown in fig. 5-7, in step 4, the early warning device is provided with the highest and lowest warning lines of heart rate, respiration, blood pressure. The steps of the early warning device for comparing indexes and predicting the index trend line comprise:
step 4.1: receiving a physiological index analysis result output by a section of blood spectrum physiological analysis device;
step 4.2: comparing and judging whether the heart rate, respiration and blood pressure indexes touch the highest and lowest warning lines, and if the highest and lowest warning lines touch the warning lines, sending warning information to a display device; if the warning line is not touched, turning to the step 4.3;
step 4.3, obtaining an index trend line according to real-time heart rate, respiration and blood pressure indexes, transmitting the index trend line to a display device, judging whether the index trend line touches a highest warning line or a lowest warning line, and if the index trend line touches the warning line, sending trend warning information to the display device; otherwise, go to step 4.1.
In step 4.3, the index trend line is obtained according to the set index trend model, and the trend prediction duration of the index trend line can be modified by inputting a set value. FIG. 5 shows a heart rate trend line where the ordinate represents heart rate, the abscissa represents time, and time 0 is the current real-time indicator; the negative time in the abscissa represents the predicted trend, and the positive time represents the obtained physiological index analysis result. FIG. 6 shows a respiratory trend line, where the ordinate represents respiratory rate, the abscissa represents time, and time 0 is the current real-time indicator; the negative time in the abscissa represents the predicted trend, and the positive time represents the obtained physiological index analysis result. FIG. 7 is a graph showing a blood pressure trend line, in which the ordinate represents blood pressure, the abscissa represents time, and time 0 is a current real-time indicator
As shown in fig. 2, in step 5, the display device includes a face video area, a physiological curve area and an alarm area. The display device receives the real-time face video stream output by the face acquisition device and displays the real-time face video stream in the face video area; receiving a real-time physiological index analysis result output by the blood spectrum physiological analysis device, and displaying the real-time physiological index analysis result in a physiological curve area in real time; receiving an index trend line output by the early warning device, displaying the index trend line in a physiological curve area, and overlapping the index trend line on a real-time physiological index analysis result; and receiving the alarm and trend alarm information output by the early warning device, and displaying the alarm and trend alarm information in the alarm area.
In this embodiment, the physiological indexes include heart rate, respiration, and blood pressure indexes.
The above description is only an embodiment of the present invention, and is not intended to limit the present invention in any way, and simple modifications, equivalent changes and modifications may be made without departing from the technical solutions of the present invention, and the scope of the present invention is defined by the appended claims.
Claims (9)
1. A device for monitoring physiological indexes in a non-contact manner in real time is characterized by comprising a video acquisition device, a human face acquisition device, a blood spectrum analysis device, an early warning device and a display device, wherein the video acquisition device is electrically connected with the human face acquisition device and the display device; the blood spectrum physiological analysis device is electrically connected with the face acquisition device, the early warning device and the display device; the early warning device is also electrically connected with the display device; the video acquisition device comprises a camera; the camera is provided with an infrared device and comprises a lens, an optical sensor, an image processing device and an encoder.
2. A method for real-time contactless monitoring of physiological indicators based on the device of claim 1, comprising the steps of:
step 1: the video acquisition device acquires a non-compressed video stream and transmits the non-compressed video stream to the face acquisition device; the video acquisition device acquires the coded video stream and transmits the coded video stream to the display device;
step 2: the human face detection device receives the uncompressed video stream, performs primary processing to obtain human face blood spectrum information, packs the human face blood spectrum information with set time length and sends the packed human face blood spectrum information to the blood spectrum physiological analysis device;
and step 3: the blood spectrum physiological analysis device receives the human face blood spectrum information compression packet, decompresses the human face blood spectrum information to obtain human face blood spectrum information, analyzes physiological indexes to obtain a physiological index analysis result, and transmits the physiological index analysis result to the display device and the early warning device;
and 4, step 4: the early warning device receives the physiological index analysis result, performs index comparison and predicts an index trend line, transmits the index trend line to the display device, and judges whether to send warning information to the display device according to the index comparison result and the index trend line;
and 5: and (3) the display device receives the coded video stream, the physiological index analysis result and the index trend line, displays the result in real time and returns to the step 1.
3. The method according to claim 2, wherein the step of the video acquiring device acquiring the uncompressed video stream and the encoded video stream in step 1 comprises:
step 1.1: the optical sensor captures light rays input into the lens, converts optical signals into electric signals and transmits the electric signals to the image processing device;
step 1.2: the image processing device receives the electric signal, converts the electric signal into frame images, arranges the frame images according to time sequence to obtain a real-time non-compressed video stream with a set frame rate, and transmits the non-compressed video stream to the face acquisition device and the encoder;
step 1.3: the encoder receives the uncompressed video stream, performs h.264 encoding to obtain an encoded video stream, and transmits the encoded video stream to the display device.
4. The method according to claim 3, wherein in step 2, the step of performing preliminary processing by the human face detection device comprises:
step 2.1: selecting a frame of image according to the time sequence, and carrying out face tracking detection on the image to obtain face characteristic points;
step 2.2: positioning a face block according to the distribution of the face feature points;
step 2.3: extracting the block information of each face block to obtain the face blood spectrum information of the frame image;
step 2.4: processing the rest of each frame image in the step 2.1-2.3 to obtain face blood spectrum information for a period of time and packaging;
step 2.5: and transmitting the packed human face blood spectrum information to a blood spectrum analysis device.
5. The method of claim 4, wherein in the step 2.4, the packing of the facial blood spectrum information includes the following steps:
step 2.41: obtaining the average value sequence of each specific block in each frame { (Block number, frame number i, R)i,Gi,Bi) I is more than or equal to 1 and less than or equal to s x v, s represents the set time length of the packed face blood spectrum information, v represents the number of frame images contained in each second of the video, and R, G, B represents RGB information of the face blocks;
step 2.42: sorting the average value sequence of the specific blocks obtained in the step 1.41 according to the block numbers to obtain a frame image sequence { (k1,1, R)1,G1,B1),…,(k1,s*v,RS*V,GS*V,BS*V) }; sequencing s x v frame image sequences in time to obtain a sequence of k1 blocks;
step 2.43: respectively obtaining sequences of k2 and … kj blocks; wherein j represents the number of the face characteristic points in the frame image;
step 2.44: acquiring a face number and a timestamp of a first frame image; the human face number is the number of human face data acquired by the image acquisition device;
step 2.45: packing the human face blood spectrum information with the time length of s seconds according to the sequence of { human face number, the time stamp of the 1 st frame, the packing frame number, the k1 block sequence and the k2 block sequence … kj block sequence }; the timestamp of the 1 st frame represents the timestamp of the 1 st frame image in the s duration range.
6. The method according to claim 5, wherein in step 3, the analysis of the physiological index comprises analysis of heart rate, respiration, and blood pressure;
the analysis process of the blood spectrum analysis device comprises the following steps:
step 3.1: receiving a human face blood spectrum information compression packet, and decompressing to obtain human face blood spectrum information;
step 3.2: combining the human face blood spectrum information according to time sequence for the specific blocks respectively; the combined human face blood spectrum information is obtained by splicing the contents of one or more human face blood spectrum information compression packets according to the analysis requirements of physiological indexes in a time sequence to form human face blood spectrum information with set time duration; the specific block is determined according to a detection target;
step 3.3: performing physiological index analysis, wherein the physiological index analysis comprises the following steps:
step 3.31: performing wavelet function filtering processing on the human face blood spectrum information with the set time length obtained in the step 3.2;
step 3.32: slicing the filtered human face blood spectrum information according to a set time length;
step 3.33: fourier transform is carried out on the face blood spectrum information after each slice to obtain a target spectrum corresponding to the slice;
step 3.34: averaging each target frequency spectrum, and sequencing according to a time sequence to obtain a target variation curve;
step 3.4: selecting the target variation curve of the specific block as a final physiological index analysis result with the best signal quality;
step 3.5: and (4) sending the physiological index analysis result obtained in the step (3.4) to an early warning device and a display device.
7. The method for real-time non-contact monitoring of physiological indexes according to claim 6, wherein in the step 4, the warning device is provided with the highest and lowest warning lines of heart rate, respiration and blood pressure; the steps of the early warning device for comparing indexes and predicting the index trend line comprise:
step 4.1: receiving a physiological index analysis result output by a section of blood spectrum physiological analysis device;
step 4.2: comparing and judging whether the heart rate, respiration and blood pressure indexes touch the highest and lowest warning lines, and if the highest and lowest warning lines touch the warning lines, sending warning information to a display device; if the warning line is not touched, turning to the step 4.3;
step 4.3, obtaining an index trend line according to real-time heart rate, respiration and blood pressure indexes, transmitting the index trend line to a display device, judging whether the index trend line touches a highest warning line or a lowest warning line, and if the index trend line touches the warning line, sending trend warning information to the display device; otherwise, go to step 4.1.
8. The method according to claim 7, wherein in step 5, the display device comprises a face video area, a physiological curve area and an alarm area; the display device receives the real-time face video stream output by the face acquisition device and displays the real-time face video stream in the face video area; receiving a real-time physiological index analysis result output by the blood spectrum physiological analysis device, and displaying the real-time physiological index analysis result in a physiological curve area in real time; receiving an index trend line output by the early warning device, displaying the index trend line in a physiological curve area, and overlapping the index trend line on a real-time physiological index analysis result; and receiving the alarm and trend alarm information output by the early warning device, and displaying the alarm and trend alarm information in the alarm area.
9. The method of claim 8, wherein the physiological indicators comprise heart rate, respiration, blood pressure indicators; the physiological indexes are displayed in a coordinate system or one physiological index is displayed corresponding to one coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911357395.XA CN111166302A (en) | 2019-12-25 | 2019-12-25 | Device and method for monitoring physiological indexes in real time in non-contact manner |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911357395.XA CN111166302A (en) | 2019-12-25 | 2019-12-25 | Device and method for monitoring physiological indexes in real time in non-contact manner |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111166302A true CN111166302A (en) | 2020-05-19 |
Family
ID=70618021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911357395.XA Pending CN111166302A (en) | 2019-12-25 | 2019-12-25 | Device and method for monitoring physiological indexes in real time in non-contact manner |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111166302A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115188466A (en) * | 2022-07-08 | 2022-10-14 | 江苏优盾通信实业有限公司 | Feature analysis-based inquired auxiliary method and system |
| CN116030960A (en) * | 2022-12-22 | 2023-04-28 | 中国人民解放军海军特色医学中心 | System and method for analyzing physiological and psychological indexes based on face recognition technology |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104173051A (en) * | 2013-05-28 | 2014-12-03 | 天津点康科技有限公司 | Automatic noncontact respiration assessing system and assessing method |
| CN204246117U (en) * | 2014-08-07 | 2015-04-08 | 何嘉健 | A kind of hurtless measure dynamic glucose concentration early warning system |
| CN105989357A (en) * | 2016-01-18 | 2016-10-05 | 合肥工业大学 | Human face video processing-based heart rate detection method |
| CN107133487A (en) * | 2017-06-21 | 2017-09-05 | 深圳市海威达信息技术有限公司 | A kind of disease prevention and cure method, device and disease prevention and cure system |
| CN107669250A (en) * | 2017-11-02 | 2018-02-09 | 佛山鑫进科技有限公司 | A kind of health degree monitoring method based on big data analysis |
| CN109448862A (en) * | 2018-09-17 | 2019-03-08 | 广州中石科技有限公司 | A kind of health monitoring method for early warning and device |
| CN110365947A (en) * | 2019-08-07 | 2019-10-22 | 杭州泽铭睿股权投资有限公司 | A kind of baby's monitor camera can detecte baby's heartbeat breathing |
-
2019
- 2019-12-25 CN CN201911357395.XA patent/CN111166302A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104173051A (en) * | 2013-05-28 | 2014-12-03 | 天津点康科技有限公司 | Automatic noncontact respiration assessing system and assessing method |
| CN204246117U (en) * | 2014-08-07 | 2015-04-08 | 何嘉健 | A kind of hurtless measure dynamic glucose concentration early warning system |
| CN105989357A (en) * | 2016-01-18 | 2016-10-05 | 合肥工业大学 | Human face video processing-based heart rate detection method |
| CN107133487A (en) * | 2017-06-21 | 2017-09-05 | 深圳市海威达信息技术有限公司 | A kind of disease prevention and cure method, device and disease prevention and cure system |
| CN107669250A (en) * | 2017-11-02 | 2018-02-09 | 佛山鑫进科技有限公司 | A kind of health degree monitoring method based on big data analysis |
| CN109448862A (en) * | 2018-09-17 | 2019-03-08 | 广州中石科技有限公司 | A kind of health monitoring method for early warning and device |
| CN110365947A (en) * | 2019-08-07 | 2019-10-22 | 杭州泽铭睿股权投资有限公司 | A kind of baby's monitor camera can detecte baby's heartbeat breathing |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115188466A (en) * | 2022-07-08 | 2022-10-14 | 江苏优盾通信实业有限公司 | Feature analysis-based inquired auxiliary method and system |
| CN115188466B (en) * | 2022-07-08 | 2023-12-12 | 江苏优盾通信实业有限公司 | Polling auxiliary method and system based on feature analysis |
| CN116030960A (en) * | 2022-12-22 | 2023-04-28 | 中国人民解放军海军特色医学中心 | System and method for analyzing physiological and psychological indexes based on face recognition technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113420624B (en) | Non-contact fatigue detection method and system | |
| CN108701357B (en) | Device, system and method for skin detection | |
| CN106778695B (en) | Multi-person rapid heart rate detection method based on video | |
| Zhang et al. | Webcam-based, non-contact, real-time measurement for the physiological parameters of drivers | |
| Zhang et al. | Heart rate extraction based on near-infrared camera: Towards driver state monitoring | |
| JP6521845B2 (en) | Device and method for measuring periodic fluctuation linked to heart beat | |
| Schrumpf et al. | Assessment of deep learning based blood pressure prediction from PPG and rPPG signals | |
| JP2014502548A (en) | Method and apparatus for detection of dysphagia | |
| SG177703A1 (en) | System and method for patient monitoring | |
| US11490853B2 (en) | Methods and devices using meta-features extracted from accelerometry signals for swallowing impairment detection | |
| CN111166302A (en) | Device and method for monitoring physiological indexes in real time in non-contact manner | |
| US20220322996A1 (en) | Methods and devices using swallowing accelerometry signals for swallowing impairment detection | |
| CN115312195A (en) | Health assessment method for calculating individual psychological abnormality based on emotion data | |
| CN113647950A (en) | Psychological emotion detection method and system | |
| CN113040773A (en) | Data acquisition and processing method | |
| Rundo et al. | Innovative saliency based deep driving scene understanding system for automatic safety assessment in next-generation cars | |
| TWI430776B (en) | Smart video skin test system and method of the same | |
| Slapnicar et al. | Contact-free monitoring of physiological parameters in people with profound intellectual and multiple disabilities | |
| CN117136027A (en) | Method and system for extracting heart rate from RGB image | |
| CA3069622A1 (en) | Methods and devices for determining signal quality for a swallowing impairment classification model | |
| KR102493242B1 (en) | Method and System for Judging Aortic Valve Stenosis Risk and Other Cardiovascular Diseases Risk from Photoplethysmography through Artificial Intelligence Learning | |
| CN111195119A (en) | Glasses device and method for detecting physiology in real time | |
| CN108090410A (en) | A kind of contactless facial rhythm of the heart Spinning and monitoring method | |
| CN111195133A (en) | Device and method capable of detecting fatigue in real time | |
| Rumiński et al. | Evaluation of facial pulse signals using deep neural net models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200519 |
|
| RJ01 | Rejection of invention patent application after publication |