CN113317766A - A311D-based embedded non-contact physiological parameter monitoring system - Google Patents
A311D-based embedded non-contact physiological parameter monitoring system Download PDFInfo
- Publication number
- CN113317766A CN113317766A CN202110518993.1A CN202110518993A CN113317766A CN 113317766 A CN113317766 A CN 113317766A CN 202110518993 A CN202110518993 A CN 202110518993A CN 113317766 A CN113317766 A CN 113317766A
- Authority
- CN
- China
- Prior art keywords
- pulse wave
- signal
- heart rate
- denoising
- embedded
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
- A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
- A61B5/726—Details of waveform analysis characterised by using transforms using Wavelet transforms
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Physiology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Veterinary Medicine (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Cardiology (AREA)
- Pulmonology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Mathematical Physics (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
Abstract
An embedded non-contact physiological parameter monitoring system based on A311D. The device comprises a video acquisition module, an embedded processor module, a display module, a storage module and a network module, wherein the video acquisition module records face and hand videos and transmits the videos to the embedded processor module, the embedded processor module performs single-frame face identification, cheek region extraction, palm region extraction, gray level mean value calculation, ICA denoising, seven-layer wavelet packet denoising, heart rate extraction, heart rate variation calculation and blood pressure calculation on the videos, the display module displays physiological parameters, the storage module records the physiological parameters, and the network module transmits the physiological parameters to a cloud server. The invention reduces the cost of the non-contact physiological parameter monitoring system, reduces the power consumption of the non-contact physiological parameter monitoring system and improves the portability of the auditing parameter monitoring system.
Description
Technical Field
The invention relates to physiological parameter detection, in particular to an embedded non-contact physiological parameter monitoring system based on A311D, which can be used for continuously monitoring human respiration, heart rate variation and blood pressure.
Background
Cardiovascular diseases have recently become the biggest killer of human life health. Has important significance for preventing and monitoring cardiovascular diseases. Conventional measurement techniques have tended to mature, but suffer from the disadvantage of requiring contact measurements, greatly binding the patient, and being only suitable for a single measurement, not allowing twenty-four hours of continuous monitoring. Whereas non-contact measurements can overcome these pain points.
There are many methods for realizing non-contact measurement, and imaging type photoplethysmography is one of typical realization methods. The method mainly captures the tiny change of human skin reflected light along with heartbeat through a color camera, and then obtains physiological parameters such as respiration, heart rate variation, blood pressure, blood oxygen saturation and the like through algorithm processing such as denoising, amplification and the like. The prior system for realizing non-contact physiological parameter detection by using the technology still has the defects, which are mainly reflected in the following three aspects.
The measurement cost is high. The imaging type photoplethysmography technology is that videos of relevant parts of a human body are collected through a camera and calculated to obtain relevant physiological parameters of the human body, so that the selection of the camera is very important, heart rate and respiratory rate measurement can be realized only through a common camera, a high signal-to-noise ratio camera is needed for measuring blood pressure saturation, and a high signal-to-noise ratio high frame rate camera is needed for measuring heart rate variation and blood pressure. Most non-contact physiological parameter monitoring systems currently use expensive industrial cameras. In addition, the high signal-to-noise ratio and the high frame rate cameras acquire a huge amount of video data, and a PC is required for processing, which further increases the cost of the system.
The measurement signal is single. In order to reduce the cost, some non-contact physiological parameter measurement systems adopt a common network camera, and as a result, only two physiological parameters, namely the heart rate and the respiratory rate, can be measured. In addition, the accuracy of the measurement of blood pressure and heart rate variability is still to be improved.
The system has large power consumption and poor portability. Most of non-contact physiological parameter measuring systems adopt a PC to process videos to obtain physiological parameters, so that the power consumption of the whole system is large, and the system is only suitable for single measurement or short-time measurement. If monitoring is continued for a long time, the system power consumption must be taken into account. And the system built by adopting the PC is poor in portability, and even if some systems adopt small hosts, the purpose of portability is still difficult to achieve.
According to the analysis, in the occasion of needing long-time continuous monitoring, the non-contact physiological parameter monitoring system is set up by using the PC, so that the cost is high, the power consumption of the system is large, the portability is poor, and the requirement cannot be met.
The embedded non-contact physiological parameter monitoring system based on A311D can make up the deficiency of the non-contact physiological parameter monitoring system based on PC. In recent years, electronic technology has been rapidly developed, and embedded SOCs are continuously updated and iterated. The computing power of embedded processors is comparable to that of low-end desktop-level CPUs. The semiconductor a311D embedded SOC is a powerful embedded processor. The processor integrates a four-core Cortex A73 with a main frequency of 2.2GHz and a two-core Cortex A53 with a main frequency of 1.8GHz, an independent NPU is arranged in the processor, the processor supports LPDDR4, USB3.0 and NVME, can support 4K decoding, and simultaneously supports common communication protocols such as SDIO, UART, IIC and SPI. The performance of the power-saving switch meets the requirements completely, and meanwhile, the power consumption of the power-saving switch is extremely low and does not exceed 10W even if the power-saving switch is operated at full speed.
On the other hand, a non-contact physiological parameter algorithm is continuously developed, and the influence caused by background and motion artifact can be greatly reduced through face recognition and ROI (region of interest) region selection, so that the dependence of the system on the high signal-to-noise ratio of the camera is reduced to a certain extent; continuous optimization of heart rate variability and blood pressure measurement algorithms can reduce the dependence of the system on the high frame rate of the camera. The feasibility of measuring physiological parameters by using a camera with a low frame rate and a high signal-to-noise ratio is extremely high.
The university of Hefei industry disclosed a non-contact type physiological and psychological health detection system in the patent document "non-contact type physiological and psychological health detection System" (publication No. CN111839489B, application No. 202010456476.1, application date: 26/05/2020). The system comprises: a doctor end, a patient end, and a mobile end. The doctor end comprises a non-contact heart rate monitoring subsystem, a non-contact respiration subsystem, a non-contact surface temperature detection system, a non-contact blood pressure detection subsystem and a non-contact psychological detection subsystem. The human face view screen is collected through the camera, and physiological parameters such as respiratory rate, heart rate, surface temperature and blood pressure are extracted through algorithms such as human face recognition, ROI (region of interest) region selection and denoising. The non-contact blood pressure system calculates the blood pressure by calculating the peak amplitude mean value and the trough mean value of the pulse wave, and the peak-trough value of the pulse wave signal is inevitably fluctuated greatly due to the influence of background noise and the influence of human face motion artifact, so that the accuracy of a measurement result is greatly influenced. And the system has higher requirement on the signal-to-noise ratio of the camera, so that the cost of the system is increased inevitably.
In the patent of Sichuan university, which is applied for, a video-based non-contact physiological parameter acquisition method and device (publication number: CN109589101B, application number: 201910038663.5, application date: 2019, 16.01), a non-contact physiological parameter acquisition method and device are disclosed, wherein a physiological parameter is obtained by extracting a blood volume pulse signal from an ROI region and then performing denoising processing and calculation on the signal. The system has the defects that only three physiological parameters of respiratory rate, heart rate and blood oxygen saturation which are easy to measure can be obtained, and the physiological parameters with higher requirements on the frame rate and the signal to noise ratio of a camera, such as heart rate variation, blood pressure and the like, cannot be measured.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide an embedded non-contact physiological parameter monitoring system based on A311D, which uses a 30-frame USB2.0 camera to reduce the video processing data volume, and uses a cubic spline interpolation algorithm to improve the time resolution so as to reduce the influence of a low-frame-rate camera on the physiological parameter measurement result. Finally, the embedded processor A311D is used for realizing a non-contact physiological parameter monitoring system, thereby greatly reducing the power consumption of the system and reducing the volume of the device.
In order to achieve the purpose, the invention adopts the technical scheme that:
an embedded non-contact physiological parameter monitoring system based on A311D is characterized in that the A311D embedded system is connected with a storage module, a display and a network module; the video acquisition module is connected with the A311D embedded system through a USB, the display module is connected with the A311D embedded system through an RGB interface or an HDMI interface, the storage module is connected with the A311D embedded system through an SD card interface, the network module is connected with the A311D embedded system through an RMII interface protocol, and the video acquisition module is connected with the camera; the network module is connected with the cloud server.
The camera adopts a 30-frame USB2.0 camera.
The A311D embedded system adopts an A311D embedded SOC chip of the semiconductor company of Jingchen.
The display supports an HDMI display and a seven-inch RGB color screen.
An embedded non-contact physiological parameter monitoring method based on A311D comprises the following steps:
step 1, inputting a face video;
recording human face and hand videos through a video acquisition module, transmitting the videos to an embedded processor module through a USB data line for preprocessing, and calculating to obtain the total frame number and the frame rate of the videos; inputting the preprocessed video to the next step;
step 2, face recognition and hand recognition;
performing single-frame processing on the preprocessed video, performing face recognition by using a face recognition classifier in an OpenCV (open source computer vision library), and calculating the face position; identifying the palm by using an autonomously designed hand identification algorithm, and calculating the position of the palm;
step 3, extracting an ROI (region of interest);
because the capillary vessels of the cheek part are rich and are less influenced by expression change, the obtained pulse wave signal quality is better, and the cheek part is intercepted from the face area to be used as an ROI area; the palm part has rich blood vessels in the palm region and better pulse wave signal quality, and the palm region is intercepted as an ROI region; through the steps, the noise caused by the background can be effectively reduced, and meanwhile, the noise caused by the human face and the hand motion artifact can be obviously reduced;
step 4, obtaining an original pulse wave signal;
calculating the RGB three-channel pixel mean value of the ROI area, and performing the same operation on each frame of image in the video, wherein the pixel mean values in all the images form an original pulse wave signal;
step 5, denoising the pulse wave by using an ICA algorithm;
because the original pulse wave signal has larger noise, independent component analysis denoising and seven-layer wavelet packet decomposition denoising are adopted; taking an original pulse wave signal of RGB three channels as an observation signal, and performing blind source separation by using an independent component analysis algorithm to obtain a pure pulse wave signal so as to achieve the purpose of denoising; defining the source signal as s (t), and the hybrid system as a matrix
Defining the output signal as x (t), the unmixing matrix as W, and y (t) as the estimation of the source signal; the observed signal can be obtained by the following equation;
if the observed signal is known, an estimated signal of the source signal can be obtained by the following formula;
taking the RGB three-channel pulse wave signals as observation signals, and solving a solution mixing matrix to obtain denoised pulse wave signals;
step 6, denoising the pulse wave by using seven layers of wavelet packet decomposition;
after denoising in the step 5, an ideal pulse wave signal can be obtained, but a certain noise still exists in the signal, the frame rate of a camera is 30Hz, the Hexi rate and the heart rate of a human body are about 0.1-3.2Hz, after the decomposition of seven layers of wavelet packets, the IPPG signal is divided into 64 frequency bands, the heart rate signal is roughly distributed in the 1 st to 7 th nodes of the seventh layer which are rearranged from small to large according to the frequency, the noise is filtered by using a soft threshold, the signal is reconstructed, and the pulse wave after the denoising of the wavelet packets is obtained;
step 7, calculating the respiration rate and the heart rate;
performing fast Fourier transform on the pulse wave signal obtained in the step 6, and analyzing the peak value of the frequency spectrum of the pulse wave signal, wherein the first peak value is a peak value generated by respiration, and the second peak value is a peak value generated by heartbeat; the frequency corresponding to the first peak value is the respiration rate, and the frequency corresponding to the second peak value is the heart rate;
step 8, cubic spline interpolation processing;
because heart rate variation and blood pressure depend on higher time resolution, the frame rate of a camera is required to be approximately more than 100 frames, and the invention adopts a 30-frame camera to reduce the video data volume, so that the time resolution of the pulse wave is improved by adopting a cubic spline interpolation method; the signal after cubic spline interpolation is resampled, and the sampling frequency is set to be 120 hz;
step 9, self-adaptive threshold method peak value detection;
the invention designs a self-adaptive differential threshold algorithm and adds a time threshold, and limits the time difference between two peak values, thereby further reducing the influence of noise and effectively reducing false detection;
step 10, calculating the accurate heart rate;
although most of noise is removed through a series of denoising operations, the IPPG system is easily interfered by the outside world, and the situation that the noise is large in a certain time period may exist, and the heart rate value obtained by using the spectrum peak value solution under the condition is easily influenced by the noise; but the mutation points caused by noise can be effectively eliminated through peak-to-peak detection, so that the influence of the noise on the measurement result is greatly reduced; obtaining the average time T between peak values by the following steps;
step 11, calculating heart rate variability related parameters;
the heart rate variability time domain parameters were as follows:
respectively representing the average pulse wave time interval and the standard deviation of the five-minute normal pulse wave time interval, and reflecting the components of the HRV with fast change; heart rate variation related parameters can be sequentially obtained from the peak value detected in the step 9;
step 12, measuring the pulse wave conduction time;
calculating the pulse wave time difference PTT of the cheek part and the palm part by using the first-order difference maximum value point when the pulse rises; the pulse wave is transmitted from the cheek part to the palm part, and after a certain time, the pulse wave transmission speed PWV can be calculated according to the transmission distance and the transmission time;
step 13, calculating a blood pressure value;
establishing a function model by using the pulse wave conduction velocity and the blood pressure, and calculating a blood pressure value;
step 14, displaying, storing and uploading the cloud;
and outputting the measured physiological parameter data to a display, storing the measurement result, and uploading the measurement result to a cloud end through a network module so as to record for a long time.
The pulse wave denoising process is RGB three-channel separation, then ICA independent component analysis denoising and seven-layer wavelet packet decomposition denoising.
The heart rate variability calculation process is to perform cubic spline interpolation operation on the denoised pulse wave, improve the signal time resolution and solve heart rate variability related parameters.
The video acquisition module is used for acquiring human face and hand videos, a 30-frame USB2.0 camera is used as the video acquisition module, the cost is reduced by 3 to 5 times compared with a high-frame-rate USB3.0 industrial camera, and the amount of acquired video data is reduced by 2 to 3 times. The operating pressure of the embedded processor can be greatly reduced.
The video embedded processor module adopts a wafer morning semiconductor A311D embedded SOC as an embedded controller. The processor has strong computing power, and mainly completes face recognition, palm recognition, ROI (region of interest) region extraction, gray scale mean value processing, ICA (independent component analysis) denoising, wavelet denoising, heart rate extraction, cubic spline interpolation, heart rate variation calculation and blood pressure calculation on videos collected by the video collection module.
The display module is used for displaying the current heart rate, heart rate variation and blood pressure of the human body and can display the pulse waves of the human body.
Further, the HDMI display and the seven-inch RGB display are simultaneously supported, so that different applicable scenes can be met.
The storage module is used for recording physiological parameters of human respiration, heart rate variation, blood pressure and the like. A person profile may be established for each user.
Further, the SD card can be used for exporting data to a computer and analyzing the data. An assessment is made of the health status of the monitored person.
The network module is used for uploading the human body physiological parameter data to the cloud server. For long-term physiological parameter data recording.
Compared with the prior art, the invention has the following advantages:
the invention adopts the low frame rate USB2.0 camera with 30 frames, so that the cost of the camera of the non-contact physiological parameter monitoring system is reduced by 3 to 5 times, and the video data volume to be processed is reduced by 2 to 4 times. Therefore, the embedded processor can be used as a control core of the non-contact physiological parameter monitoring system instead of a PC.
The invention adopts the wafer semiconductor A311D embedded SOC to replace a PC as the control core of the non-contact physiological parameter monitoring system. The power consumption of the system is reduced by 5-10 times, and the volume of the whole system is reduced by 5-20 times.
According to the invention, the time resolution is improved by adopting a cubic spline interpolation method, so that the defects of a low frame rate camera can be overcome, and continuous monitoring of multiple physiological parameters such as respiration, heart rate variation, blood pressure and the like can be realized.
Drawings
FIG. 1 is a block diagram of the inventive system;
FIG. 2 is a flowchart of a physiological parameter measurement algorithm according to the present invention.
Detailed Description
Embodiments of the present invention are described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, the embedded non-contact physiological parameter monitoring system based on a311D of the present invention includes a video acquisition module, an embedded processor module, a display module, a storage module and a network module. The video acquisition module is connected with the embedded processor module through a USB data line, the display module is connected with the embedded processor module through an RGB interface or an HDMI interface, the storage module realizes data communication with the embedded processor module through an SDIO protocol, and the network module carries out data communication with the embedded processor module through an RMII protocol. Wherein:
the video acquisition module adopts a 30-frame USB2.0 camera to continuously acquire human face and hand videos and transmits the human face and hand videos to the embedded processor module in real time through a USB data line.
The embedded processor module adopts a crystal semiconductor A311D embedded SOC as a controller, and mainly completes video face recognition, palm recognition, ROI (region of interest) extraction, gray level mean processing, ICA (independent component analysis) denoising, wavelet denoising, heart rate extraction, cubic spline interpolation, heart rate variation calculation and blood pressure calculation.
The display module adopts HDMI interface display screen or seven cun display screens of RGB, can show human breathing, rhythm of the heart variation, blood pressure measurement result directly perceived, can also realize the pulse ripples demonstration simultaneously.
The storage module adopts an SD card as a storage medium, realizes continuous recording of physiological parameters, and can export data to a computer for data analysis.
The network module is connected with the internet through a network cable and transmits the measurement result to the cloud server, and the measurement result of the physiological parameter can only be recorded for 24 hours due to the limited capacity of the storage module. The measurement result is uploaded to a cloud security device, so that the physiological parameters of the human body can be recorded for a long time, and the health state of the human body can be conveniently analyzed.
Referring to the attached figure 2, the human physiological parameter monitoring method comprises the following steps:
step 1, inputting a face video.
The video acquisition module records human face and hand videos, the videos are transmitted to the embedded processor module through the USB data line for preprocessing, and the total frame number and the frame rate of the videos are obtained through calculation. The video after the preprocessing is input to the next step.
And 2, recognizing the face and the hand.
And performing single-frame processing on the preprocessed video, performing face recognition by using a face recognition classifier in an OpenCV (open source computer vision library), and calculating the face position. And (4) recognizing the palm by using an autonomously designed hand recognition algorithm, and calculating the position of the palm.
And 3, extracting an ROI (region of interest).
Because the capillary vessels of the cheek part are rich and are less influenced by expression change, the obtained pulse wave signal quality is better, and the cheek part is intercepted from the face area to be used as an ROI area. The palm part, the regional blood vessel of palm is abundant, and pulse wave signal quality is better, with the regional interception of palm region as the ROI area. Through the steps, the noise caused by the background can be effectively reduced, and meanwhile, the noise caused by the human face and hand motion artifact can be obviously reduced.
And 4, obtaining an original pulse wave signal.
And solving the RGB three-channel pixel mean value of the ROI area, and performing the same operation on each frame of image in the video, wherein the pixel mean values in all the images form an original pulse wave signal.
And 5, denoising the pulse wave by using an ICA algorithm.
Because the original pulse wave signal has larger noise, the independent component analysis denoising and the seven-layer wavelet packet decomposition denoising are adopted. The original pulse wave signals of RGB three channels are used as observation signals, blind source separation is carried out by using an independent component analysis algorithm, and pure pulse wave signals are obtained to achieve the purpose of denoising. Defining the source signal as s (t), and the hybrid system as a matrix
Defining the output signal as X (t), the unmixing matrix as W, and y (t) as the source signal estimation. The observed signal can be obtained by the following equation.
If the observed signal is known, an estimated signal of the source signal can be obtained by the following formula.
The RGB three-channel pulse wave signals are used as observation signals, and the solution mixing matrix is solved, so that the de-noised pulse wave signals can be obtained.
And 6, denoising the pulse wave by using seven layers of wavelet packet decomposition.
After denoising in the step 5, a relatively ideal pulse wave signal can be obtained, but a certain noise still exists in the signal, the frame rate of a camera is 30Hz, the Hexi rate and the heart rate of a human body are about 0.1-3.2Hz, after the decomposition of seven layers of wavelet packets, the IPPG signal is divided into 64 frequency bands, the heart rate signal is roughly distributed in the 1 st to 7 th nodes of the seventh layer which are rearranged from small to large according to the frequency, the noise is filtered by using a soft threshold, the signal is reconstructed, and the pulse wave after the denoising of the wavelet packets is obtained.
And 7, calculating the respiration rate and the heart rate.
And 6, performing fast Fourier transform on the pulse wave signal obtained in the step 6, and analyzing the peak values of the frequency spectrum, wherein the first peak value is a peak value generated by respiration, and the second peak value is a peak value generated by heartbeat. The first peak corresponds to a respiration rate and the second peak corresponds to a heart rate.
And 8, carrying out cubic spline interpolation processing.
Because heart rate variation and blood pressure depend on higher time resolution, the frame rate of a camera is required to be about more than 100 frames, and the invention adopts a 30-frame camera to reduce the video data volume, so that the time resolution of the pulse wave is improved by adopting a cubic spline interpolation method. Here, the signal after cubic spline interpolation is resampled, and the sampling frequency is set to 120 hz.
And 9, detecting the peak value by the self-adaptive threshold method.
The invention designs a self-adaptive differential threshold algorithm and adds a time threshold, and limits the time difference between two peak values, thereby further reducing the influence of noise and effectively reducing false detection.
And step 10, calculating the accurate heart rate.
Although most of noise is removed after a series of denoising operations, the IPPG system is very susceptible to external interference, and there may be a case where the noise is large in a certain time period, and the heart rate value obtained by using the spectrum peak value solution under such a condition is very susceptible to the noise. But the mutation point caused by noise can be effectively eliminated through peak-to-peak detection, thereby greatly reducing the influence of the noise on the measurement result. The average time T between peaks was obtained as follows.
And 11, calculating heart rate variability related parameters.
The heart rate variability time domain parameters were as follows:
respectively represents the average pulse wave time interval and the standard deviation of the five-minute normal pulse wave time interval, and reflects the components of the HRV which changes rapidly. The peak-to-peak values detected in step 9 may in turn be used to find heart rate variability i.e. related parameters.
And step 11, measuring the pulse wave conduction time.
And calculating the time difference PTT of the pulse waves of the cheek part and the palm part by using the first difference maximum value point when the pulse rises. The pulse wave is transmitted from the cheek part to the palm part, and after a certain time, the pulse wave transmission speed PWV can be calculated according to the transmission distance and the transmission time.
And step 11, calculating a blood pressure value.
And establishing a function model by using the pulse wave velocity and the blood pressure, and calculating the blood pressure value.
And step 11, displaying, storing and uploading the cloud.
And outputting the measured physiological parameter data to a display, storing the measurement result, and uploading the measurement result to a cloud end through a network module so as to record for a long time.
Claims (7)
1. An embedded non-contact physiological parameter monitoring system based on A311D is characterized in that the A311D embedded system is connected with a storage module, a display and a network module; the video acquisition module is connected with the A311D embedded system through a USB, the display module is connected with the A311D embedded system through an RGB interface or an HDMI interface, the storage module is connected with the A311D embedded system through an SD card interface, the network module is connected with the A311D embedded system through an RMII interface protocol, and the video acquisition module is connected with the camera; the network module is connected with the cloud server.
2. The embedded contactless physiological parameter monitoring system based on a311D as claimed in claim 1, wherein the camera is a 30 frame USB2.0 camera.
3. The embedded non-contact physiological parameter monitoring system based on A311D as claimed in claim 1, wherein the A311D embedded system adopts A311D embedded SOC chip of Chichen semiconductor company.
4. The embedded contactless physiological parameter monitoring system according to claim 1, wherein the display supports HDMI display and seven-inch RGB color screen.
5. An embedded non-contact physiological parameter monitoring method based on A311D is characterized by comprising the following steps:
step 1, inputting a face video;
recording human face and hand videos through a video acquisition module, transmitting the videos to an embedded processor module through a USB data line for preprocessing, and calculating to obtain the total frame number and the frame rate of the videos; inputting the preprocessed video to the next step;
step 2, face recognition and hand recognition;
performing single-frame processing on the preprocessed video, performing face recognition by using a face recognition classifier in an OpenCV (open source computer vision library), and calculating the face position; identifying the palm by using an autonomously designed hand identification algorithm, and calculating the position of the palm;
step 3, extracting an ROI (region of interest);
because the capillary vessels of the cheek part are rich and are less influenced by expression change, the obtained pulse wave signal quality is better, and the cheek part is intercepted from the face area to be used as an ROI area; the palm part has rich blood vessels in the palm region and better pulse wave signal quality, and the palm region is intercepted as an ROI region; through the steps, the noise caused by the background can be effectively reduced, and meanwhile, the noise caused by the human face and the hand motion artifact can be obviously reduced;
step 4, obtaining an original pulse wave signal;
calculating the RGB three-channel pixel mean value of the ROI area, and performing the same operation on each frame of image in the video, wherein the pixel mean values in all the images form an original pulse wave signal;
step 5, denoising the pulse wave by using an ICA algorithm;
because the original pulse wave signal has larger noise, independent component analysis denoising and seven-layer wavelet packet decomposition denoising are adopted; taking an original pulse wave signal of RGB three channels as an observation signal, and performing blind source separation by using an independent component analysis algorithm to obtain a pure pulse wave signal so as to achieve the purpose of denoising; defining a source signal as s (t), a mixing system as a matrix A, an output signal as x (t), a demixing matrix as W, and y (t) as an estimate of the source signal; the observed signal can be obtained by the following equation;
if the observed signal is known, an estimated signal of the source signal can be obtained by the following formula;
taking the RGB three-channel pulse wave signals as observation signals, and solving a solution mixing matrix to obtain denoised pulse wave signals;
step 6, denoising the pulse wave by using seven layers of wavelet packet decomposition;
after denoising in the step 5, an ideal pulse wave signal can be obtained, but a certain noise still exists in the signal, the frame rate of a camera is 30Hz, the Hexi rate and the heart rate of a human body are about 0.1-3.2Hz, after the decomposition of seven layers of wavelet packets, the IPPG signal is divided into 64 frequency bands, the heart rate signal is roughly distributed in the 1 st to 7 th nodes of the seventh layer which are rearranged from small to large according to the frequency, the noise is filtered by using a soft threshold, the signal is reconstructed, and the pulse wave after the denoising of the wavelet packets is obtained;
step 7, calculating the respiration rate and the heart rate;
performing fast Fourier transform on the pulse wave signal obtained in the step 6, and analyzing the peak value of the frequency spectrum of the pulse wave signal, wherein the first peak value is a peak value generated by respiration, and the second peak value is a peak value generated by heartbeat; the frequency corresponding to the first peak value is the respiration rate, and the frequency corresponding to the second peak value is the heart rate;
step 8, cubic spline interpolation processing;
because heart rate variation and blood pressure depend on higher time resolution, the frame rate of a camera is required to be approximately more than 100 frames, and the invention adopts a 30-frame camera to reduce the video data volume, so that the time resolution of the pulse wave is improved by adopting a cubic spline interpolation method; the signal after cubic spline interpolation is resampled, and the sampling frequency is set to be 120 hz;
step 9, self-adaptive threshold method peak value detection;
the invention designs a self-adaptive differential threshold algorithm and adds a time threshold, and limits the time difference between two peak values, thereby further reducing the influence of noise and effectively reducing false detection;
step 10, calculating the accurate heart rate;
although most of noise is removed through a series of denoising operations, the IPPG system is easily interfered by the outside world, and the situation that the noise is large in a certain time period may exist, and the heart rate value obtained by using the spectrum peak value solution under the condition is easily influenced by the noise; but the mutation points caused by noise can be effectively eliminated through peak-to-peak detection, so that the influence of the noise on the measurement result is greatly reduced; obtaining the average time T between peak values by the following steps;
step 11, calculating heart rate variability related parameters;
the heart rate variability time domain parameters were as follows:
respectively representing the average pulse wave time interval and the standard deviation of the five-minute normal pulse wave time interval, and reflecting the components of the HRV with fast change; heart rate variation related parameters can be sequentially obtained from the peak value detected in the step 9;
step 12, measuring the pulse wave conduction time;
calculating the pulse wave time difference PTT of the cheek part and the palm part by using the first-order difference maximum value point when the pulse rises; the pulse wave is transmitted from the cheek part to the palm part, and after a certain time, the pulse wave transmission speed PWV can be calculated according to the transmission distance and the transmission time;
step 13, calculating a blood pressure value;
establishing a function model by using the pulse wave conduction velocity and the blood pressure, and calculating a blood pressure value;
step 14, displaying, storing and uploading the cloud;
and outputting the measured physiological parameter data to a display, storing the measurement result, and uploading the measurement result to a cloud end through a network module so as to record for a long time.
6. The embedded non-contact physiological parameter monitoring method according to claim 1, wherein the pulse wave denoising process is RGB three-channel separation, then ICA independent component analysis denoising, seven-layer wavelet packet decomposition denoising.
7. The embedded non-contact physiological parameter monitoring method according to claim 1, wherein the heart rate variability calculation process is to perform cubic spline interpolation on the denoised pulse wave, improve the signal time resolution, and then solve the heart rate variability related parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110518993.1A CN113317766A (en) | 2021-05-12 | 2021-05-12 | A311D-based embedded non-contact physiological parameter monitoring system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110518993.1A CN113317766A (en) | 2021-05-12 | 2021-05-12 | A311D-based embedded non-contact physiological parameter monitoring system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113317766A true CN113317766A (en) | 2021-08-31 |
Family
ID=77415512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110518993.1A Pending CN113317766A (en) | 2021-05-12 | 2021-05-12 | A311D-based embedded non-contact physiological parameter monitoring system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113317766A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023193711A1 (en) * | 2022-04-07 | 2023-10-12 | Faceheart Corporation | Contactless physiological measurement device and method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108245176A (en) * | 2017-12-07 | 2018-07-06 | 江苏大学 | Based on the interactive contactless psychology detection therapeutic device of Internet of Things, system and method |
| CN109009052A (en) * | 2018-07-02 | 2018-12-18 | 南京工程学院 | The embedded heart rate measurement system and its measurement method of view-based access control model |
| CN109589101A (en) * | 2019-01-16 | 2019-04-09 | 四川大学 | A kind of contactless physiological parameter acquisition methods and device based on video |
| CN111714105A (en) * | 2020-07-24 | 2020-09-29 | 长春理工大学 | Human vital sign perception system based on IPPG |
| CN111839492A (en) * | 2020-04-20 | 2020-10-30 | 合肥工业大学 | Heart rate non-contact type measuring method based on face video sequence |
| CN111839489A (en) * | 2020-05-26 | 2020-10-30 | 合肥工业大学 | Non-contact physiological and psychological health detection system |
| CN112006673A (en) * | 2020-08-26 | 2020-12-01 | 西安电子科技大学 | Human body heart rate detection method and system, storage medium, computer equipment and terminal |
-
2021
- 2021-05-12 CN CN202110518993.1A patent/CN113317766A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108245176A (en) * | 2017-12-07 | 2018-07-06 | 江苏大学 | Based on the interactive contactless psychology detection therapeutic device of Internet of Things, system and method |
| CN109009052A (en) * | 2018-07-02 | 2018-12-18 | 南京工程学院 | The embedded heart rate measurement system and its measurement method of view-based access control model |
| CN109589101A (en) * | 2019-01-16 | 2019-04-09 | 四川大学 | A kind of contactless physiological parameter acquisition methods and device based on video |
| CN111839492A (en) * | 2020-04-20 | 2020-10-30 | 合肥工业大学 | Heart rate non-contact type measuring method based on face video sequence |
| CN111839489A (en) * | 2020-05-26 | 2020-10-30 | 合肥工业大学 | Non-contact physiological and psychological health detection system |
| CN111714105A (en) * | 2020-07-24 | 2020-09-29 | 长春理工大学 | Human vital sign perception system based on IPPG |
| CN112006673A (en) * | 2020-08-26 | 2020-12-01 | 西安电子科技大学 | Human body heart rate detection method and system, storage medium, computer equipment and terminal |
Non-Patent Citations (3)
| Title |
|---|
| HAMIDUR RAHMAN, SHANKER IYER, CAROLINE MEUSBURGER: "SmartMirror: An Embedded Non-contact System for Health Monitoring at Home", 《INTERNET OF THINGS TECHNOLOGIES FOR HEALTHCARE》 * |
| 刘赫: "容积脉搏成像特征提取方法研究及在生理信号检测的应用", 《中国博士学位论文全文数据库》 * |
| 周秦武，隋芳芳，白平，宋亚楠，尹小龙: "嵌入式无接触视频心率检测方法", 《西安交通大学学报》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023193711A1 (en) * | 2022-04-07 | 2023-10-12 | Faceheart Corporation | Contactless physiological measurement device and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | A comparative survey of methods for remote heart rate detection from frontal face videos | |
| CN109977858B (en) | Heart rate detection method and device based on image analysis | |
| Casado et al. | Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces | |
| Gudi et al. | Efficient real-time camera based estimation of heart rate and its variability | |
| CN111728602A (en) | Non-contact blood pressure measuring device based on PPG | |
| Rahman et al. | Non-contact physiological parameters extraction using facial video considering illumination, motion, movement and vibration | |
| CN104138254A (en) | Non-contact type automatic heart rate measurement system and measurement method | |
| CN110647815A (en) | Non-contact heart rate measurement method and system based on face video image | |
| Christinaki et al. | Comparison of blind source separation algorithms for optical heart rate monitoring | |
| CN113693573B (en) | Video-based non-contact multi-physiological-parameter monitoring system and method | |
| CA2882080A1 (en) | Real-time physiological characteristic detection based on reflected components of light | |
| CN112890792A (en) | Cloud computing cardiovascular health monitoring system and method based on network camera | |
| Przybyło | A deep learning approach for remote heart rate estimation | |
| US20230005295A1 (en) | Non-contact facial blood pressure measurement method based on 3d cnn | |
| Yin et al. | Heart rate estimation based on face video under unstable illumination | |
| CN108272449A (en) | Contactless monitoring of blood pressure method and system | |
| Li et al. | An improvement for video-based heart rate variability measurement | |
| Kang et al. | Transppg: Two-stream transformer for remote heart rate estimate | |
| CN112200099A (en) | Video-based dynamic heart rate detection method | |
| Wang et al. | VitaSi: A real-time contactless vital signs estimation system | |
| CN114387479A (en) | Non-contact heart rate measurement method and system based on face video | |
| CN108814543B (en) | High-speed video eye shakes appearance system | |
| Zheng et al. | Remote measurement of heart rate from facial video in different scenarios | |
| Panigrahi et al. | Non-contact HR extraction from different color spaces using RGB camera | |
| CN113317766A (en) | A311D-based embedded non-contact physiological parameter monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210831 |