CN111685730A - Non-contact physiological parameter detection method, system, terminal equipment and storage medium - Google Patents
Non-contact physiological parameter detection method, system, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN111685730A CN111685730A CN202010400756.0A CN202010400756A CN111685730A CN 111685730 A CN111685730 A CN 111685730A CN 202010400756 A CN202010400756 A CN 202010400756A CN 111685730 A CN111685730 A CN 111685730A
- Authority
- CN
- China
- Prior art keywords
- pixel
- signal
- filtering
- signals
- carrying
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7225—Details of analog processing, e.g. isolation amplifier, gain or sensitivity adjustment, filtering, baseline or drift compensation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
- A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
Abstract
The application provides a non-contact physiological parameter detection method, a non-contact physiological parameter detection system, terminal equipment and a storage medium, wherein the method comprises the following steps: respectively carrying out super-pixel integration on pixel points of each frame of video frame, wherein each super-pixel at least comprises 2 pixel points; filtering each pixel point by adopting a preset filter; the method comprises the steps of carrying out analog-to-digital conversion on pixel points to obtain a plurality of groups of pixel signals, carrying out signal combination on each group of pixel signals to obtain a plurality of groups of combined signals, carrying out signal combination on the combined signals to obtain physiological signals, and carrying out spectrum analysis on the physiological signals to obtain physiological parameters. This application carries out filtering process's design through every pixel in the super pixel respectively, can carry out the filtering of different modes to the pixel in the video frame to, because every super pixel includes 2 pixels at least, consequently, can obtain 2 at least channel output results after the filtering, and then can carry out effectual physiological parameter's detection based on this 2 at least channel output results.
Description
Technical Field
The present application relates to the field of physiological parameter detection, and in particular, to a non-contact physiological parameter detection method, system, terminal device, and storage medium.
Background
The physiological parameters of human body such as respiratory and heartbeat frequency are important physiological indexes for doctors to diagnose and treat abnormal diseases related to the heart and lung. Common respiratory and heartbeat rate measurement methods usually employ contact-type means, such as an electrocardiogram.
There are a large amount of products such as wearable bracelet, wrist-watch on the market at present, but the measuring mode of contact still is inconvenient. For example, the contact measurement method is relatively difficult for infants to perform, and in addition, since the elderly apartment and hotels are usually monitored remotely, the contact measurement method cannot meet the measurement requirement under remote monitoring. With the development of science and technology, the non-contact measurement method based on the RGB image can make up the defects.
In the existing non-contact measurement technology, non-contact physiological parameter detection is realized by shooting RGB images through an RGB camera (color camera), but under the condition of poor light, the RGB camera cannot work, and at the moment, only an infrared camera can be adopted for image acquisition. However, since the infrared camera has only one channel output, and the non-contact measurement method based on the RGB image performs detection of the physiological parameter based on the output results of three channels (R channel, G channel, and B channel), the detection of the physiological parameter cannot be realized when the infrared camera is used for image acquisition.
Disclosure of Invention
The embodiment of the application provides a non-contact physiological parameter detection method, a non-contact physiological parameter detection system, terminal equipment and a storage medium, and aims to solve the problem that the physiological parameter of a user cannot be detected according to the output of a channel in the existing non-contact physiological parameter detection process.
In a first aspect, an embodiment of the present application provides a method for non-contact physiological parameter detection, where the method includes:
acquiring video frames, and respectively carrying out superpixel integration on pixel points of each frame of the video frames, wherein each superpixel at least comprises 2 pixel points;
filtering each pixel point in the super-pixel by adopting a preset filter, wherein at least 2 filters with different filtering parameters exist, and the filtering parameters comprise a central wavelength, a width and a response amplitude corresponding to each wavelength;
performing analog-to-digital conversion on the filtered pixel points to obtain a plurality of groups of pixel signals, and performing signal combination on each group of pixel signals to obtain a plurality of groups of combined signals;
and carrying out signal combination on the combined signal to obtain a physiological signal, and carrying out spectrum analysis on the physiological signal to obtain a physiological parameter.
Compared with the prior art, the embodiment of the application has the advantages that: through carrying out filtering processing to every pixel in the superpixel in every frame video frame respectively to carry out analog-to-digital conversion's design to the pixel after the filtering, can carry out the filtering of different modes to the pixel in the video frame, and, because every superpixel includes 2 pixel at least, consequently, can obtain 2 at least channel output results after the filtering, and then can carry out effectual physiological parameter's detection based on this 2 at least channel's output result.
Further, the performing super-pixel integration on the pixel points of each frame of the video frame respectively includes:
acquiring the number of pixel points in the video frame, and inquiring the integration number of pixels according to the number;
and carrying out image segmentation on the video frame according to the pixel integration number to obtain segmented images, and integrating all the pixel points in each segmented image into one superpixel.
Further, the respectively performing signal combination on each group of pixel signals to obtain a plurality of groups of combined signals includes:
carrying out linear combination on the pixel signals in the same super pixel according to an orthogonalization rule to obtain a plurality of groups of combined signals;
filtering the multiple groups of combined signals respectively to obtain multiple groups of filtering signals, and calculating the signal-to-noise ratio of the filtering signals respectively;
and carrying out signal combination on the combined signal according to the signal-to-noise ratio.
Further, the combination formula for combining the signals of the combined signal according to the signal-to-noise ratio is as follows:
Shr＝S1*X1 2/[(X1 2+X2 2…+Xn 2)]+S2*X2 2/[(X1 2+X2 2…+Xn 2)]…+Sn*Xn 2/[(X1 2+X2 2…+Xn 2)]；
wherein Shr is the physiological signal, n is the nth combined signal in the same super pixel, and XnAnd Sn is the signal-to-noise ratio of the nth combined signal, and Sn is the filtering signal corresponding to the nth combined signal.
Further, the filtering the multiple groups of combined signals respectively to obtain multiple groups of filtered signals includes:
when the physiological parameter is a heart rate parameter, performing band-pass filtering on the combined signal to obtain a heart rate filtering signal;
when the physiological parameter is a respiratory parameter, low-pass filtering is carried out on the combined signal to obtain a respiratory filtering signal;
correspondingly, the performing spectrum analysis on the physiological signal to obtain a physiological parameter includes:
respectively carrying out Fourier transform on the heart rate filtering signal and the respiration filtering signal to obtain a heart rate curve and a respiration curve;
and respectively acquiring peak values in the heart rate curve and the respiration curve to obtain a heart rate value and a respiration value.
Further, when the physiological parameters further include a blood oxygen parameter, the method further includes:
calculating RR values among different combined signals, and matching the RR values with a pre-stored RR blood oxygen fitting curve to obtain the blood oxygen parameters; the calculation formula adopted for calculating the RR value is as follows:
RR＝[AC(lamda1)/DC(lamda1)]/[AC(lamda2)/DC(lamda2)]；
where AC (lamda1) is the wavelength of the analog signal of the first of the combined signals, DC (lamda1) is the wavelength of the digital signal of the first of the combined signals, AC (lamda2) is the wavelength of the analog signal of the second of the combined signals, and DC (lamda2) is the wavelength of the digital signal of the second of the combined signals.
Further, after the step of acquiring the video frame, the method further comprises:
and carrying out face region identification on the video frame, and carrying out image cutting on the video frame according to an identification result.
In a second aspect, an embodiment of the present application provides a non-contact physiological parameter detection system, including:
the super-pixel integration module is used for acquiring video frames and respectively carrying out super-pixel integration on pixel points of each frame of the video frames, and each super-pixel at least comprises 2 pixel points;
the pixel filtering module is used for respectively carrying out filtering processing on each pixel point in the super-pixel by adopting a preset filter, wherein at least 2 filters with different filtering parameters exist, and the filtering parameters comprise a central wavelength, a width and an amplitude of corresponding response of each wavelength;
the signal-to-noise ratio calculation module is used for performing analog-to-digital conversion on the filtered pixel points to obtain a plurality of groups of pixel signals, and performing signal combination on each group of pixel signals to obtain a plurality of groups of combined signals;
and the spectrum analysis module is used for carrying out signal combination on the combined signal to obtain a physiological signal and carrying out spectrum analysis on the physiological signal to obtain a physiological parameter.
In a third aspect, an embodiment of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the method described above.
In a fourth aspect, the present application provides a storage medium storing a computer program, and when the computer program is executed by a processor, the computer program implements the method as described above.
In a fifth aspect, the present application provides a computer program product, which when run on a terminal device, causes the terminal device to execute the contactless physiological parameter detection method according to any one of the above first aspects.
It is understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description of the first aspect, and are not described herein again.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the embodiments or the description of the prior art will be briefly described below.
FIG. 1 is a flow chart of a method for contactless physiological parameter detection provided by a first embodiment of the present application;
FIG. 2 is a flow chart of a non-contact physiological parameter detection method provided by a second embodiment of the present application;
FIG. 3 is a schematic structural diagram of a non-contact physiological parameter detecting system according to a third embodiment of the present application;
fig. 4 is a schematic structural diagram of a terminal device according to a fourth embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
Example one
Referring to fig. 1, a flowchart of a non-contact physiological parameter detecting method according to a first embodiment of the present application is shown, including the steps of:
step S10, acquiring video frames, and respectively performing superpixel integration on pixel points of each frame of video frames;
the non-contact physiological parameter detection method can be applied to a camera with a gray scale sensor or a black and white sensor, the camera can be an infrared camera, and an image corresponding to a video frame shot by the camera is a gray scale image.
Specifically, each super pixel includes 2 pixel points at least, and further, the number of pixel points in each super pixel can set up as required for the number of pixel points in each super pixel can be inequality.
Step S20, filtering each pixel point in the super-pixel by adopting a preset filter;
in the step, different pixel points (sub-pixels) in the same super pixel are filtered by adopting the filters with different filter parameters, so that the filtering effects of different pixel points in each super pixel are different, the filtering aiming at the pixel points in one super pixel can obtain a plurality of channel outputs, and the phenomenon that the non-contact physiological parameter detection cannot be carried out due to the fact that an infrared camera only outputs a channel output result is prevented.
For example, when the pixels in the video frame are integrated to obtain 3 super pixels, each super pixel includes 4 pixels, 4 different pixels in the same super pixel can be filtered by using 4 filters with different filtering parameters to obtain 4 signal output results, for example, a filter with a central wavelength of 800nm and a width of 10nm can be used for filtering for a first pixel; the second pixel point can be filtered by adopting a filter with the center wavelength of 820nm and the width of 10nm, and it can be understood that the third pixel point and the fourth pixel point are respectively filtered by adopting filters with different center wavelengths and/or widths.
Step S30, performing analog-to-digital conversion on the filtered pixel points to obtain a plurality of groups of pixel signals, and performing signal combination on each group of pixel signals to obtain a plurality of groups of combined signals;
the filtered pixel points may be subjected to analog-to-digital conversion in an analog-to-digital converter manner to convert analog signals into digital signals, and preferably, in this step, signal combination between each group of pixel signals may be performed in an orthogonalization manner, so that pairwise orthogonality between the combined signals is independent.
Step S40, merging the signals of the combination signal to obtain a physiological signal, and carrying out spectrum analysis on the physiological signal to obtain a physiological parameter;
the combined signal may be signal-combined in a Maximum Ratio Combining (MRC) manner to improve the signal-to-noise Ratio of the physiological signal, and when the signal-to-noise Ratio is higher, it indicates that the noise mixed in the physiological signal is smaller, the quality of the signal is higher, and thus the accuracy of the subsequent physiological signal spectrum analysis is improved.
Specifically, in this step, a fourier transform may be used to perform spectrum analysis on the physiological signal, so as to convert the time domain features in the physiological signal into frequency domain features, and calculate each parameter value in the physiological parameter based on the frequency domain features.
In this embodiment, through carrying out filtering processing to every pixel in the superpixel in every frame video frame respectively, and carry out analog-to-digital conversion's design to the pixel after the filtering, can carry out the filtering of different modes to the pixel in the video frame, and, because every superpixel includes 2 pixels at least, consequently, can obtain 2 at least channel output results after the filtering, and then can carry out effectual physiological parameter's detection based on this 2 at least channel output results, and through carrying out the design that the signal combines to the pixel signal according to the signal-to-noise ratio, the effectual SNR that improves physiological signal, and then improved follow-up physiological signal spectral analysis's accuracy.
Example two
Referring to fig. 2, a flowchart of a non-contact physiological parameter detecting method according to a second embodiment of the present application is shown, including the steps of:
step S11, acquiring the number of pixel points in the video frame, and inquiring the integration number of pixels according to the number;
the pixel integration table is stored locally in advance, and the corresponding relation between the individual numerical values of different pixel points and the pixel integration number is stored in the pixel integration table, so that the corresponding pixel integration number can be inquired by matching the acquired individual numerical values of the pixel points with the pixel integration table.
In addition, in this embodiment, after the step of acquiring the video frame, the method further includes:
carrying out face region identification on the video frame, and carrying out image cutting on the video frame according to an identification result;
the video frame is subjected to image cutting according to the recognition result, so that the background image in the video frame is effectively removed, and the accuracy of subsequent video frame analysis is improved.
Step S21, image segmentation is carried out on the video frame according to the pixel integration number to obtain segmented images, and all pixel points in each segmented image are integrated into a super pixel;
for example, assuming that a video frame is a grayscale image with 20x20 pixels, if the number of searched pixels is 25, the video frame is averagely divided into 25 divided images with 4x4 pixels, and then all the pixels (4x4 pixels) in each divided image are integrated into a superpixel, thereby obtaining 25 superpixels.
Step S31, filtering each pixel point in the super-pixel by a preset filter;
wherein, there are the wave filter of 2 different filter parameters at least, this filter parameter includes the center wavelength, the width and the range that each wavelength corresponds the response, it is preferred, the quantity of this wave filter is equal with the number value of the pixel in the super pixel that corresponds filtering, for example, when the number of pixel is 16 in this super pixel, then the quantity of the wave filter that corresponds is 16, and the filter parameter between the wave filter is all inequality, make the filter effect of different pixel in the same super pixel all different, and then the filtering to pixel can obtain a plurality of channel outputs in a super pixel, prevented because infrared camera only when a channel output result, the phenomenon that can't carry out non-contact physiological parameter detection that leads to.
Step S41, performing analog-to-digital conversion on the filtered pixel points to obtain a plurality of groups of pixel signals, and performing linear combination on the pixel signals in the same super pixel according to an orthogonalization rule to obtain a plurality of groups of combined signals;
the analog-to-digital conversion of the pixel point can be performed by adopting an analog-to-digital converter, so that the analog signal is converted into a digital signal to obtain a plurality of groups of pixel signals.
Specifically, in this step, the number of pixel signals in the super pixel is the same as the number of pixel points, for example, when the number of pixel points in the super pixel is 3, the pixel signals are obtained correspondingly after analog-to-digital conversion, preferably, an orthogonalization rule is adopted to perform linear combination between the pixel signals in the same super pixel, for example, each pixel signal may be set to a pixel vector of a three-dimensional space by adopting a schmidt orthogonalization method, and in order to ensure independence between all the pixel vectors, the schmidt orthogonalization method is adopted to perform orthogonality between the pixel vectors, so that the signal-to-noise ratio in the combined signal after linear combination is maximized.
Preferably, in this step, the number of combined signals is equal to the number of pixel signals, which is-1, and when 3 pixel signals are obtained after analog-to-digital conversion, the number of combined signals obtained by linear combination is 2, for example, when 3 pixel signals Y1, Y2, and Y3 are obtained after analog-to-digital conversion, the combined signals obtained by combination are Y1 'and Y2';
further, the formula for linearly combining the pixel signals in the same super pixel is as follows:
y1' is a first vector [ Y1Y 2Y 3 ];
y2' is a second vector [ Y1Y 2Y 3 ];
wherein the first vector and the second vector are orthogonal, for example, the first vector may be [ -211 ], and the second vector may be [ 01-1 ], that is, Y1 ═ 2Y1+ Y2+ Y3, Y2 ═ Y2-Y3;
furthermore, when the 4 pixel signals Y1, Y2, Y3 and Y4 are obtained after the analog-to-digital conversion, the combined signals are Y1 ', Y2 ' and Y3 ';
the formula adopted for linearly combining the pixel signals in the same super pixel is as follows:
y1' is the first vector [ Y1Y 2Y 3Y 4 ];
y2' is a second vector [ Y1Y 2Y 3Y 4 ];
y3' is the third vector [ Y1Y 2Y 3Y 4 ];
the first vector, the second vector and the third vector are orthogonal in pairs, and it can be understood that when the number of the pixel signals obtained after the analog-to-digital conversion is other than the number of the pixel signals, the pixel signals are linearly combined in the manner described above, and all the vectors in the linear combination formula are orthogonal in pairs.
Step S51, filtering the multiple groups of combined signals respectively to obtain multiple groups of filtered signals;
specifically, when the physiological parameter is a heart rate parameter, performing band-pass filtering on the combined signal to obtain a heart rate filtering signal, and when the physiological parameter is a respiratory parameter, performing low-pass filtering on the combined signal to obtain a respiratory filtering signal;
the combined signal may be band-pass filtered by a filter with a bandwidth of 0.8-3Hz to obtain a heart rate filtered signal containing heart rate characteristics, and the combined signal may be low-pass filtered by a filter with a bandwidth of 0-0.6Hz to obtain a respiratory filtered signal containing respiratory characteristics.
Step S61, respectively calculating the signal-to-noise ratio of the heart rate filtering signal and the respiration filtering signal, and respectively carrying out signal combination on the heart rate filtering signal and the respiration filtering signal according to the signal-to-noise ratio to obtain physiological signals;
wherein, the combination formula for combining the signals of the combined signal according to the signal-to-noise ratio is as follows:
Shr＝S1*X1 2/[(X1 2+X2 2…+Xn 2)]+S2*X2 2/[(X1 2+X2 2…+Xn 2)]…+Sn*Xn 2/[(X1 2+X2 2…+Xn 2)]；
specifically, Shr is a physiological signal, n is a positive integer, and is the nth combined signal (heart rate filtered signal or respiration filtered signal) in the same super-pixel, and X isnSn is the signal-to-noise ratio of the nth combined signal (heart rate filtered signal or respiration filtered signal), and Sn is the filtered signal corresponding to the nth combined signal.
Step S71, performing Fourier transform on the heart rate filtering signal and the respiration filtering signal after signal combination to obtain a heart rate curve and a respiration curve;
the heart rate filtering signal is subjected to signal combination to obtain a heart rate signal, the respiration filtering signal is subjected to signal combination to obtain a respiration signal, the heart rate signal and the respiration signal are converted into curves in a Fourier transform mode, so that time domain characteristics in the heart rate signal and the respiration signal are converted into frequency domain characteristics, and subsequent physiological parameters are effectively and conveniently calculated.
Step S81, respectively obtaining peak values in a heart rate curve and a respiration curve to obtain a heart rate value and a respiration value;
and the points with the highest amplitude in the corresponding heart rate curve and the corresponding respiration curve are the current heart rate value and the current respiration value.
Preferably, in this step, when the physiological parameters further include a blood oxygen parameter, the method further includes:
calculating RR values among different combined signals, and matching the RR values with a pre-stored respiration blood oxygen fitting curve to obtain the blood oxygen parameters; the calculation formula adopted for calculating the RR value is as follows:
RR＝[AC(lamda1)/DC(lamda1)]/[AC(lamda2)/DC(lamda2)]；
where AC (lamda1) is the wavelength of the analog signal of the first of the combined signals, DC (lamda1) is the wavelength of the digital signal of the first of the combined signals, AC (lamda2) is the wavelength of the analog signal of the second of the combined signals, and DC (lamda2) is the wavelength of the digital signal of the second of the combined signals.
Specifically, since there is a corresponding relationship between the RR value of the human body and the blood oxygen (SpO2), for example, SpO2 ═ a × RR + b or SpO2 ═ a × RR2+ b RR + c, therefore, the corresponding relationship between RR value and blood oxygen may be calculated by curve fitting to obtain a fitted curve of respiration and blood oxygen.
Furthermore, in this embodiment, because output results of a plurality of channels can be obtained, a plurality of different RR values can be obtained, and then sample data of an RR fitting curve is increased, and then accuracy of curve fitting between RR and blood oxygen is improved, so that accuracy of subsequently detected blood oxygen parameters is higher.
In addition, since the correspondence relationship exists between the time-domain feature in the parameter curve of Shr and the blood pressure, the time-domain feature may be a feature such as the amplitude or the rising slope of the peak of the Shr parameter curve, and further, a plurality of sets of sample data between the blood pressure and the time-domain feature may be obtained, and based on the sample data, model training may be performed by a machine learning or deep learning method to obtain a neural network model between the time-domain feature of the Shr parameter curve and the blood pressure, so that the parameter based on Shr is calculated to obtain the current corresponding blood pressure value.
It should be noted that, in this embodiment, because the output results of a plurality of channels can be obtained, Shr parameters obtained by calculation are more, and then training samples in the model training process are improved, and the accuracy of neural network model training is improved.
In this embodiment, each pixel point in the superpixel in each frame of video frame is filtered, and analog-to-digital conversion is performed on the filtered pixel points, different modes of filtering can be performed on the pixel points in the video frame, so as to obtain a plurality of channel output results, physiological parameters can be effectively detected based on the output results of a plurality of channels, and by designing linear combination of pixel signals in the same superpixel according to an orthogonalization rule, the signal-to-noise ratio in a combined signal is effectively improved, so that the accuracy of subsequent heart rate calculation and respiration calculation is improved, and in this embodiment, the accuracy of curve fitting between a respiration value and blood oxygen is effectively improved based on the results of a plurality of signal outputs.
EXAMPLE III
Fig. 3 shows a schematic structural diagram of a non-contact physiological parameter detecting system 100 provided in the third embodiment of the present application, which corresponds to the non-contact physiological parameter detecting method described in the foregoing embodiments, and only shows the relevant parts of the embodiments of the present application for convenience of description.
Referring to fig. 3, the system includes: super pixel integration module 10, pixel filtering module 11, signal-to-noise ratio calculation module 12 and spectrum analysis module 13, wherein:
the super-pixel integration module 10 is configured to acquire video frames and perform super-pixel integration on pixel points of each frame of the video frames, where each super-pixel includes at least 2 pixel points.
Wherein the super-pixel integrated module 10 is further configured to: acquiring the number of pixel points in the video frame, and inquiring the integration number of pixels according to the number;
and carrying out image segmentation on the video frame according to the pixel integration number to obtain segmented images, and integrating all the pixel points in each segmented image into one superpixel.
Further, the super-pixel integrated module 10 is further configured to: and carrying out face region identification on the video frame, and carrying out image cutting on the video frame according to an identification result.
And the pixel filtering module 11 is configured to perform filtering processing on each pixel point in the super-pixel by using a preset filter, where at least 2 filters with different filtering parameters exist, and the filtering parameters include a center wavelength, a width, and an amplitude of a corresponding response of each wavelength.
And the signal-to-noise ratio calculation module 12 is configured to perform analog-to-digital conversion on the filtered pixel points to obtain multiple groups of pixel signals, and perform signal combination on each group of pixel signals to obtain multiple groups of combined signals.
Wherein the signal-to-noise ratio calculating module 12 is further configured to:
carrying out linear combination on the pixel signals in the same super pixel according to an orthogonalization rule to obtain a plurality of groups of combined signals;
filtering the multiple groups of combined signals respectively to obtain multiple groups of filtering signals, and calculating the signal-to-noise ratio of the filtering signals respectively;
and carrying out signal combination on the combined signal according to the signal-to-noise ratio.
Preferably, the signal-to-noise ratio calculating module 12 is further configured to: when the physiological parameter is a heart rate parameter, performing band-pass filtering on the combined signal to obtain a heart rate filtering signal;
when the physiological parameter is a respiratory parameter, low-pass filtering is carried out on the combined signal to obtain a respiratory filtering signal;
correspondingly, the performing spectrum analysis on the physiological signal to obtain a physiological parameter includes:
respectively carrying out Fourier transform on the heart rate filtering signal and the respiration filtering signal to obtain a heart rate curve and a respiration curve;
and respectively acquiring peak values in the heart rate curve and the respiration curve to obtain a heart rate value and a respiration value.
And the spectrum analysis module 13 is configured to perform signal combination on the combined signal to obtain a physiological signal, and perform spectrum analysis on the physiological signal to obtain a physiological parameter.
Wherein, the combination formula for combining the signals of the combined signal according to the signal-to-noise ratio is as follows:
Shr＝S1*X1 2/[(X1 2+X2 2…+Xn 2)]+S2*X2 2/[(X1 2+X2 2…+Xn 2)]…+Sn*Xn 2/[(X1 2+X2 2…+Xn 2)]；
wherein Shr is the physiological signal, n is a positive integer, is the nth combined signal in the same super pixel, and XnAnd Sn is the signal-to-noise ratio of the nth combined signal, and Sn is the filtering signal corresponding to the nth combined signal.
Further, the audio analysis module 13 is further configured to: calculating RR values among different combined signals, and matching the RR values with a pre-stored RR blood oxygen fitting curve to obtain the blood oxygen parameters; the calculation formula adopted for calculating the RR value is as follows:
RR＝[AC(lamda1)/DC(lamda1)]/[AC(lamda2)/DC(lamda2)]；
where AC (lamda1) is the wavelength of the analog signal of the first of the combined signals, DC (lamda1) is the wavelength of the digital signal of the first of the combined signals, AC (lamda2) is the wavelength of the analog signal of the second of the combined signals, and DC (lamda2) is the wavelength of the digital signal of the second of the combined signals.
Specifically, in this embodiment, when there are 4 different combination signals, four RR values can be obtained, so as to effectively improve the accuracy of curve fitting between the RR values and RR blood oxygen.
In this embodiment, through carrying out filtering processing to every pixel in the superpixel in every frame video frame respectively, and carry out analog-to-digital conversion's design to the pixel after the filtering, can carry out the filtering of different modes to the pixel in the video frame, and, because every superpixel includes 2 pixels at least, consequently, can obtain 2 at least channel output results after the filtering, and then can carry out effectual physiological parameter's detection based on this 2 at least channel output results, and through carrying out the design that the signal combines to the pixel signal according to the signal-to-noise ratio, the effectual SNR that improves physiological signal, and then improved follow-up physiological signal spectral analysis's accuracy.
It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/modules, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and reference may be made to the part of the embodiment of the method specifically, and details are not described here.
Fig. 4 is a schematic structural diagram of a terminal device 2 according to a fourth embodiment of the present application. As shown in fig. 4, the terminal device 2 of this embodiment includes: at least one processor 20 (only one processor is shown in fig. 4), a memory 21, and a computer program 22 stored in the memory 21 and executable on the at least one processor 20, the steps of any of the various method embodiments described above being implemented when the computer program 22 is executed by the processor 20.
The terminal device 2 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor 20, a memory 21. Those skilled in the art will appreciate that fig. 4 is merely an example of the terminal device 2, and does not constitute a limitation of the terminal device 2, and may include more or less components than those shown, or combine some components, or different components, such as an input-output device, a network access device, and the like.
The Processor 20 may be a Central Processing Unit (CPU), and the Processor 20 may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 21 may in some embodiments be an internal storage unit of the terminal device 2, such as a hard disk or a memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 2. Further, the memory 21 may also include both an internal storage unit and an external storage device of the terminal device 2. The memory 21 is used for storing an operating system, an application program, a BootLoader (BootLoader), data, and other programs, such as program codes of the computer program. The memory 21 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
An embodiment of the present application further provides a network device, where the network device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, the processor implementing the steps of any of the various method embodiments described above when executing the computer program.
The embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the above-mentioned method embodiments.
The embodiments of the present application provide a computer program product, which when running on a mobile terminal, enables the mobile terminal to implement the steps in the above method embodiments when executed.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), random-access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other ways. For example, the above-described apparatus/network device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A method of non-contact physiological parameter detection, the method comprising:
acquiring video frames, and respectively carrying out superpixel integration on pixel points of each frame of the video frames, wherein each superpixel at least comprises 2 pixel points;
filtering each pixel point in the super-pixel by adopting a preset filter, wherein at least 2 filters with different filtering parameters exist, and the filtering parameters comprise a central wavelength, a width and a response amplitude corresponding to each wavelength;
performing analog-to-digital conversion on the filtered pixel points to obtain a plurality of groups of pixel signals, and performing signal combination on each group of pixel signals to obtain a plurality of groups of combined signals;
and carrying out signal combination on the combined signal to obtain a physiological signal, and carrying out spectrum analysis on the physiological signal to obtain a physiological parameter.
2. The method for detecting non-contact physiological parameters according to claim 1, wherein the performing super-pixel integration on the pixel points of each frame of the video frame respectively comprises:
acquiring the number of pixel points in the video frame, and inquiring the integration number of pixels according to the number;
and carrying out image segmentation on the video frame according to the pixel integration number to obtain segmented images, and integrating all the pixel points in each segmented image into one superpixel.
3. The method of claim 1, wherein the separately combining the signals for each group of the pixel signals to obtain a plurality of groups of combined signals comprises:
carrying out linear combination on the pixel signals in the same super pixel according to an orthogonalization rule to obtain a plurality of groups of combined signals;
filtering the multiple groups of combined signals respectively to obtain multiple groups of filtering signals, and calculating the signal-to-noise ratio of the filtering signals respectively;
and carrying out signal combination on the combined signal according to the signal-to-noise ratio.
4. The method according to claim 3, wherein the combination formula for combining the combined signal according to the SNR is as follows:
Shr＝S1*X1 2/[(X1 2+X2 2…+Xn 2)]+S2*X2 2/[(X1 2+X2 2…+Xn 2)]…+Sn*Xn 2/[(X1 2+X2 2…+Xn 2)]；
wherein Shr is the physiological signal, n is a positive integer, is the nth combined signal in the same super pixel, and XnAnd Sn is the signal-to-noise ratio of the nth combined signal, and Sn is the filtering signal corresponding to the nth combined signal.
5. The method of claim 3, wherein the filtering the plurality of sets of combined signals to obtain a plurality of sets of filtered signals comprises:
when the physiological parameter is a heart rate parameter, performing band-pass filtering on the combined signal to obtain a heart rate filtering signal;
when the physiological parameter is a respiratory parameter, low-pass filtering is carried out on the combined signal to obtain a respiratory filtering signal;
correspondingly, the performing spectrum analysis on the physiological signal to obtain a physiological parameter includes:
respectively carrying out Fourier transform on the heart rate filtering signal and the respiration filtering signal to obtain a heart rate curve and a respiration curve;
and respectively acquiring peak values in the heart rate curve and the respiration curve to obtain a heart rate value and a respiration value.
6. The method of claim 1, wherein when the physiological parameter further comprises a blood oxygen parameter, the method further comprises:
calculating RR values among different combined signals, and matching the RR values with a pre-stored RR blood oxygen fitting curve to obtain the blood oxygen parameters; the calculation formula adopted for calculating the RR value is as follows:
RR＝[AC(lamda1)/DC(lamda1)]/[AC(lamda2)/DC(lamda2)]；
where AC (lamda1) is the wavelength of the analog signal of the first of the combined signals, DC (lamda1) is the wavelength of the digital signal of the first of the combined signals, AC (lamda2) is the wavelength of the analog signal of the second of the combined signals, and DC (lamda2) is the wavelength of the digital signal of the second of the combined signals.
7. The method of claim 1, wherein the step of acquiring the video frame is followed by the method further comprising:
and carrying out face region identification on the video frame, and carrying out image cutting on the video frame according to an identification result.
8. A non-contact physiological parameter sensing system, comprising:
the super-pixel integration module is used for acquiring video frames and respectively carrying out super-pixel integration on pixel points of each frame of the video frames, and each super-pixel at least comprises 2 pixel points;
the pixel filtering module is used for respectively carrying out filtering processing on each pixel point in the super-pixel by adopting a preset filter, wherein at least 2 filters with different filtering parameters exist, and the filtering parameters comprise a central wavelength, a width and an amplitude of corresponding response of each wavelength;
the signal-to-noise ratio calculation module is used for performing analog-to-digital conversion on the filtered pixel points to obtain a plurality of groups of pixel signals, and performing signal combination on each group of pixel signals to obtain a plurality of groups of combined signals;
and the spectrum analysis module is used for carrying out signal combination on the combined signal to obtain a physiological signal and carrying out spectrum analysis on the physiological signal to obtain a physiological parameter.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the computer program.
10. A storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010400756.0A CN111685730B (en) | 2020-05-13 | 2020-05-13 | Non-contact physiological parameter detection method, system, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010400756.0A CN111685730B (en) | 2020-05-13 | 2020-05-13 | Non-contact physiological parameter detection method, system, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111685730A true CN111685730A (en) | 2020-09-22 |
| CN111685730B CN111685730B (en) | 2022-10-11 |
Family
ID=72477750
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010400756.0A Active CN111685730B (en) | 2020-05-13 | 2020-05-13 | Non-contact physiological parameter detection method, system, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111685730B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113100748A (en) * | 2021-03-30 | 2021-07-13 | 联想(北京)有限公司 | Respiratory frequency determination method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102309318A (en) * | 2011-07-08 | 2012-01-11 | 首都医科大学 | Method for detecting human body physiological parameters on basis of infrared sequence image |
| US20120268643A1 (en) * | 2011-04-22 | 2012-10-25 | Panasonic Corporation | Imaging device, imaging system, and imaging method |
| CN103271743A (en) * | 2012-12-10 | 2013-09-04 | 中国人民解放军第一五二中心医院 | Non-contact oxyhemoglobin saturation measuring device based on imaging device |
| US20160132732A1 (en) * | 2014-11-10 | 2016-05-12 | Utah State University | Remote Heart Rate Estimation |
| CN109171649A (en) * | 2018-08-30 | 2019-01-11 | 合肥工业大学 | Intelligent imaging formula vital signs detecting instrument |
| CN110236515A (en) * | 2019-07-19 | 2019-09-17 | 合肥工业大学 | A kind of contactless heart rate detection method based on near-infrared video |
-
2020
- 2020-05-13 CN CN202010400756.0A patent/CN111685730B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120268643A1 (en) * | 2011-04-22 | 2012-10-25 | Panasonic Corporation | Imaging device, imaging system, and imaging method |
| CN102309318A (en) * | 2011-07-08 | 2012-01-11 | 首都医科大学 | Method for detecting human body physiological parameters on basis of infrared sequence image |
| CN103271743A (en) * | 2012-12-10 | 2013-09-04 | 中国人民解放军第一五二中心医院 | Non-contact oxyhemoglobin saturation measuring device based on imaging device |
| US20160132732A1 (en) * | 2014-11-10 | 2016-05-12 | Utah State University | Remote Heart Rate Estimation |
| CN109171649A (en) * | 2018-08-30 | 2019-01-11 | 合肥工业大学 | Intelligent imaging formula vital signs detecting instrument |
| CN110236515A (en) * | 2019-07-19 | 2019-09-17 | 合肥工业大学 | A kind of contactless heart rate detection method based on near-infrared video |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113100748A (en) * | 2021-03-30 | 2021-07-13 | 联想(北京)有限公司 | Respiratory frequency determination method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111685730B (en) | 2022-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Al-Naji et al. | Remote monitoring of cardiorespiratory signals from a hovering unmanned aerial vehicle | |
| CN110367950B (en) | Non-contact physiological information detection method and system | |
| US9020185B2 (en) | Systems and methods for non-contact heart rate sensing | |
| CN113408508B (en) | Transformer-based non-contact heart rate measurement method | |
| Rahman et al. | Non-contact physiological parameters extraction using facial video considering illumination, motion, movement and vibration | |
| US20130322729A1 (en) | Processing a video for vascular pattern detection and cardiac function analysis | |
| Casado et al. | Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces | |
| Wang et al. | Detail-preserving pulse wave extraction from facial videos using consumer-level camera | |
| CN109589101A (en) | A kind of contactless physiological parameter acquisition methods and device based on video | |
| CN112788200B (en) | Method and device for determining frequency spectrum information, storage medium and electronic device | |
| Hernandez-Ortega et al. | A comparative evaluation of heart rate estimation methods using face videos | |
| CN111685730B (en) | Non-contact physiological parameter detection method, system, terminal equipment and storage medium | |
| CN111671420A (en) | Method for extracting features from resting electroencephalogram data and terminal equipment | |
| Tabei et al. | A novel diversity method for smartphone camera-based heart rhythm signals in the presence of motion and noise artifacts | |
| US10803353B2 (en) | Method and apparatus for acquiring information | |
| Panigrahi et al. | Non-contact HR extraction from different color spaces using RGB camera | |
| CN105844648A (en) | Hair quality detection device, hair quality detection method and mobile terminal | |
| Zheng et al. | Remote measurement of heart rate from facial video in different scenarios | |
| Abdulrahaman | Two-stage motion artifact reduction algorithm for rPPG signals obtained from facial video recordings | |
| US11213256B2 (en) | Biological image processing method and biological information detection device | |
| CN115553777A (en) | Non-contact mental stress detection method and system | |
| CN115376179A (en) | Non-contact heart rate measuring method and system | |
| CN115670480A (en) | Biological characteristic analysis method and device, electronic equipment and storage medium | |
| Geng et al. | Motion resistant facial video based heart rate estimation method using head-mounted camera | |
| Wong et al. | Deep Learning of Sparse Patterns in Medical IoT for Efficient Big Data Harnessing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |