CN115624322B - Non-contact physiological signal detection method and system based on efficient space-time modeling - Google Patents

Non-contact physiological signal detection method and system based on efficient space-time modeling Download PDF

Info

Publication number
CN115624322B
CN115624322B CN202211451949.4A CN202211451949A CN115624322B CN 115624322 B CN115624322 B CN 115624322B CN 202211451949 A CN202211451949 A CN 202211451949A CN 115624322 B CN115624322 B CN 115624322B
Authority
CN
China
Prior art keywords
neural network
loss function
image sequence
space
loss
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211451949.4A
Other languages
Chinese (zh)
Other versions
CN115624322A (en
Inventor
邹博超
马惠敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology Beijing USTB
Original Assignee
University of Science and Technology Beijing USTB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology Beijing USTB filed Critical University of Science and Technology Beijing USTB
Priority to CN202211451949.4A priority Critical patent/CN115624322B/en
Publication of CN115624322A publication Critical patent/CN115624322A/en
Application granted granted Critical
Publication of CN115624322B publication Critical patent/CN115624322B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024—Detecting, measuring or recording pulse rate or heart rate
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
    • A61B5/0816—Measuring devices for examining respiratory frequency
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Theoretical Computer Science (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Pathology (AREA)
  • Public Health (AREA)
  • Physiology (AREA)
  • Pulmonology (AREA)
  • Cardiology (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention provides a non-contact physiological signal detection method and system based on efficient space-time modeling, and relates to the technical field of intelligent data processing. The method comprises the steps of obtaining an original video stream, preprocessing the original video stream, and obtaining a preprocessed image sequence; acquiring an image sequence, inputting the image sequence into a depth neural network based on a three-dimensional center differential convolution operator, and extracting space-time information by combining a attention mask mechanism of a convolution layer; based on the space-time information and the epsilon-insensitive Huber loss loss function, constructing a multi-task loss function to optimize the deep neural network; and filtering the optimized deep neural network by adopting a second-order Butterworth filter, and outputting the heart rate and the respiratory rate. The three-dimensional center difference convolution operator can be used for extracting pulse wave information by gathering time difference information, and cross-database evaluation and ablation research are carried out, so that the effectiveness and the robustness of the method are proved.

Description

Non-contact physiological signal detection method and system based on efficient space-time modeling
Technical Field
The invention relates to the technical field of intelligent data processing, in particular to a non-contact physiological signal detection method and system based on efficient space-time modeling.
Background
Along with the continuous development of modern society and the continuous improvement of living standard of people, the incidence rate of cardiovascular diseases is also continuously increased, which is probably caused by the increase of working pressure and the increase of living rhythm. The detection of human physiological indexes is very important for sensing human health conditions. Early detection and treatment can effectively prevent and control cardiovascular diseases, and can avoid and reduce sudden death caused by cardiovascular problems. Traditional heart rate measurement methods, such as electrocardiography, are all contact heart rate measurements, and have the following limitations: first, it is not suitable for some specific application scenarios, such as a scenario requiring long-term monitoring: neonates, large area burn patients, sleep monitoring, driver monitoring, etc., and contact measurements require subjective cooperation of the subject. Secondly, when the contact position of the measuring instrument and the skin deviates, a large deviation of the measuring result is easy to occur. Furthermore, although electrocardiographs have the advantage of accurate measurements, they are relatively expensive, require specialized operations, and are not suitable for routine physiological signal measurements. With the initial success of the non-contact remote photoplethysmography prototype method, classical signal processing has demonstrated the feasibility of heart rate measurements based on remote photoplethysmography techniques. However, these methods tend to degrade when subjected to noise such as motion, illumination changes, and skin tone. In practical application, the method based on signal separation is found to be only aimed at specific interference, and coexistence of multiple interferences in a real scene cannot be effectively processed.
For practical applications, recent research is beginning to focus on deep learning based approaches because of their better characterizations. Several deep learning-based methods have been successfully applied to remote photoplethysmography recovery tasks with pulse or respiration as the target signal, but these methods still have difficulty in efficiently modeling spatiotemporal information. Although there are methods of modeling spatiotemporal information by generating feature maps that require preprocessing, including face detection, face keypoint localization, face alignment, skin segmentation, and color space transformation, these are quite complex. Furthermore, performance across database evaluations and practical applications may be degraded due to limitations in supervised learning. The learned spatiotemporal features are still susceptible to lighting conditions and motion, and they do not take full advantage of extensive temporal context information.
Disclosure of Invention
Aiming at the problems that the learned space-time characteristics are still easily influenced by illumination conditions and motions and the learned space-time characteristics cannot fully utilize wide time context information in the prior art, the invention provides a non-contact physiological signal detection method and system based on efficient space-time modeling.
In order to solve the technical problems, the invention provides the following technical scheme:
in one aspect, a method for detecting a non-contact physiological signal based on efficient spatiotemporal modeling is provided, which is applied to an electronic device, and includes the following steps:
s1: the method comprises the steps of obtaining an original video stream, preprocessing the original video stream, and obtaining a preprocessed image sequence;
s2: acquiring an image sequence, inputting the image sequence into a depth neural network based on a three-dimensional center differential convolution operator, and extracting space-time information by combining a attention mask mechanism of a convolution layer;
s3: based on the space-time information and the epsilon-insensitive Huber loss loss function, constructing a multi-task loss function to optimize the deep neural network;
s4: and filtering the optimized deep neural network by adopting a second-order Butterworth filter, and outputting the heart rate and the respiratory rate simultaneously to finish non-contact physiological signal detection based on efficient space-time modeling.
Optionally, in step S1, an original video stream is acquired, and the original video stream is preprocessed to obtain a preprocessed image sequence, including:
respectively carrying out time domain normalization difference value and downsampling treatment on an original video stream to obtain a preprocessed image sequence;
wherein, the calculation of the time domain normalized difference is performed according to the following formula (1):
Figure 880627DEST_PATH_IMAGE001
wherein the method comprises the steps of
Figure 112281DEST_PATH_IMAGE002
Represent the first
Figure 626439DEST_PATH_IMAGE003
Individual skin pixels are in time
Figure 490490DEST_PATH_IMAGE004
Is used for the RGB values of (a),
Figure 226234DEST_PATH_IMAGE005
is a time variation value.
Optionally, the image sequence comprises: a time domain normalized difference image sequence and a downsampled image sequence.
Optionally, in step S2, an image sequence is acquired, the image sequence is input into a deep neural network based on a three-dimensional central differential convolution operator, and the spatio-temporal information is extracted by combining with a attention mask mechanism of a convolution layer, including:
s21: taking the time domain normalized difference image sequence as a motion branch, and inputting the motion branch into a depth neural network based on a three-dimensional center differential convolution operator;
taking the downsampled image sequence as an appearance branch, and inputting the downsampled image sequence into a depth neural network based on a three-dimensional center differential convolution operator;
s22: through a attention mask mechanism, modeling skin interested areas based on appearance branches assist the motion branches to extract space-time information;
s23: repeating the steps S21-S22, extracting the space-time information, and transmitting the space-time information to the full connection layer.
Optionally, in step S21, the method includes:
obtaining a three-dimensional center difference convolution operator according to the following formula (2)
Figure 783117DEST_PATH_IMAGE006
：
Figure 518992DEST_PATH_IMAGE007
Wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure 639264DEST_PATH_IMAGE008
is an input feature map which is used to input a feature map,
Figure 245825DEST_PATH_IMAGE009
representing a local receptive field cube, the shape of which is shown,
Figure 456551DEST_PATH_IMAGE010
is a weight that can be learned and is,
Figure 397831DEST_PATH_IMAGE011
representing the current position on the feature map,
Figure 275788DEST_PATH_IMAGE012
representation of receptive fields
Figure 986124DEST_PATH_IMAGE009
And adjacent time steps
Figure 275023DEST_PATH_IMAGE013
Enumeration of middle position, hyper-parameters
Figure 706529DEST_PATH_IMAGE014
For balancing spatial intensity and gradient.
Optionally, in step S22, it includes:
obtaining a function of the attention mask mechanism according to the following equation (3)
Figure 919335DEST_PATH_IMAGE015
The formula:
Figure 687440DEST_PATH_IMAGE016
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure 678399DEST_PATH_IMAGE017
is the appearance branch
Figure 345003DEST_PATH_IMAGE018
A feature map of a layer convolution layer;
Figure 610769DEST_PATH_IMAGE019
is the branch of motion
Figure 577588DEST_PATH_IMAGE020
A feature map of a layer convolution layer;
Figure 818076DEST_PATH_IMAGE021
and
Figure 958595DEST_PATH_IMAGE022
is the first
Figure 496892DEST_PATH_IMAGE018
The height and width of the layer convolution layer feature map;
Figure 708431DEST_PATH_IMAGE023
the sigmoid function is represented as a function,
Figure 588662DEST_PATH_IMAGE024
is the weight of the convolution kernel,
Figure 682389DEST_PATH_IMAGE025
is a convolution kernel offset which is a function of the convolution kernel,
Figure 837427DEST_PATH_IMAGE026
is the L1 norm of the sample,
Figure 778838DEST_PATH_IMAGE027
representing per-element products.
Optionally, in step S3, constructing a multi-task loss function based on the spatio-temporal information and the epsilon-insensitive Huber loss loss function optimizes the deep neural network, including:
calculating the intensity loss of pulse wave and respiratory wave according to the following equation (4) epsilon-insensitive Huber loss loss function
Figure 816589DEST_PATH_IMAGE028
：
Figure 476240DEST_PATH_IMAGE029
Wherein the method comprises the steps of
Figure 90761DEST_PATH_IMAGE030
A true value is indicated and,
Figure 542471DEST_PATH_IMAGE031
representing input
Figure 498926DEST_PATH_IMAGE008
Through a function of
Figure 895141DEST_PATH_IMAGE032
The predicted value after the mapping is used for mapping,
Figure 391981DEST_PATH_IMAGE033
is a hyper-parameter of Huber loss, default to 1,
Figure 511247DEST_PATH_IMAGE034
is an superparameter of epsilon-intrinsic loss, and the default value is 0.1;
constructing a multiple-task loss function in combination with the following equation (5)
Figure 613502DEST_PATH_IMAGE035
：
Figure 982167DEST_PATH_IMAGE036
Wherein the method comprises the steps of
Figure 282698DEST_PATH_IMAGE037
And
Figure 505738DEST_PATH_IMAGE038
weights for a loss function;
and (3) optimizing the weight of the deep neural network through the back propagation loss function value, and stopping optimizing after the loss function is not reduced any more, namely selecting the deep neural network with the lowest loss function value in the training process.
Optionally, in step S4, the optimized deep neural network is filtered by using a second-order butterworth filter, and the heart rate and the respiration rate are output at the same time, so as to complete non-contact physiological signal detection based on efficient space-time modeling, which includes:
a second-order Butterworth filter deep neural network is adopted, and heart rate and respiratory rate are output simultaneously;
wherein, the cut-off frequency of heart rate is 0.75Hz and 2.5Hz, and the cut-off frequency of respiratory frequency is 0.08Hz and 0.5Hz respectively;
the position of the highest peak value in the power spectrum obtained by filtering the signals is selected as heart rate and respiratory rate output, and non-contact physiological signal detection based on efficient space-time modeling is completed.
In one aspect, a non-contact physiological signal detection system based on efficient spatiotemporal modeling is provided, the system being applied to an electronic device, the system comprising:
the data acquisition module is used for acquiring an original video stream, preprocessing the original video stream and acquiring a preprocessed image sequence;
the space-time information extraction module is used for acquiring an image sequence, inputting the image sequence into a depth neural network based on a three-dimensional center difference convolution operator, and extracting space-time information by combining a attention mask mechanism of a convolution layer;
the model optimization module is used for constructing a multi-task loss function to optimize the deep neural network based on the space-time information and the epsilon-insensitive Huber loss loss function;
and the data output module is used for filtering the optimized deep neural network by adopting a second-order Butterworth filter, outputting heart rate and respiratory rate at the same time, and finishing non-contact physiological signal detection based on efficient space-time modeling.
Optionally, the data acquisition module is further configured to perform time domain normalization difference and downsampling on the original video stream respectively to obtain a preprocessed image sequence;
wherein, the calculation of the time domain normalized difference is performed according to the following formula (1):
Figure 335154DEST_PATH_IMAGE001
wherein the method comprises the steps of
Figure 456693DEST_PATH_IMAGE002
Represent the first
Figure 747866DEST_PATH_IMAGE003
Individual skin pixels are in time
Figure 107304DEST_PATH_IMAGE004
Is used for the RGB values of (a),
Figure 638779DEST_PATH_IMAGE005
is a time variation value.
In one aspect, an electronic device is provided that includes a processor and a memory having at least one instruction stored therein that is loaded and executed by the processor to implement a non-contact physiological signal detection method based on efficient spatiotemporal modeling as described above.
In one aspect, a computer-readable storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement a non-contact physiological signal detection method based on efficient spatiotemporal modeling as described above is provided.
The technical scheme provided by the embodiment of the invention has at least the following beneficial effects:
in the scheme, an accurate non-contact physiological signal measurement method based on a three-dimensional center difference convolution attention network is provided, the method is used for efficient space-time modeling, and the utilized three-dimensional center difference convolution operator can extract pulse wave information by gathering time difference information.
Epsilon-insensitive Huber loss is proposed as a loss function of the non-contact pulse wave measurement network, as it can focus the pulse wave intensity constraint, showing better performance of epsilon-insensitive Huber loss loss function by evaluating different loss functions and combinations thereof.
And further provides a network for combined multitasking measurement of heart and respiratory motion, which has the advantage of sharing information between related physiological signals, can measure heart rate and respiratory rate simultaneously, further improves accuracy and reduces calculation cost. A large number of experiments show that the proposed method has excellent performance on the public database. And cross-database evaluation and ablation studies were performed, demonstrating the effectiveness and robustness of the proposed method.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a non-contact physiological signal detection method based on efficient space-time modeling provided by an embodiment of the invention;
FIG. 2 is a flow chart of a non-contact physiological signal detection method based on efficient spatiotemporal modeling provided by an embodiment of the invention;
FIG. 3 is a block diagram of a non-contact physiological signal detection system based on efficient spatiotemporal modeling provided by an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
The embodiment of the invention provides a non-contact physiological signal detection method based on efficient space-time modeling, which can be realized by electronic equipment, wherein the electronic equipment can be a terminal or a server. The flow chart of the non-contact physiological signal detection method based on the efficient space-time modeling as shown in fig. 1, the processing flow of the method can comprise the following steps:
s101: the method comprises the steps of obtaining an original video stream, preprocessing the original video stream, and obtaining a preprocessed image sequence;
s102: acquiring an image sequence, inputting the image sequence into a depth neural network based on a three-dimensional center differential convolution operator, and extracting space-time information by combining a attention mask mechanism of a convolution layer;
s103: based on the space-time information and the epsilon-insensitive Huber loss loss function, constructing a multi-task loss function to optimize the deep neural network;
s104: and filtering the optimized deep neural network by adopting a second-order Butterworth filter, and outputting the heart rate and the respiratory rate simultaneously to finish non-contact physiological signal detection based on efficient space-time modeling.
Optionally, in step S101, an original video stream is acquired, and the original video stream is preprocessed to obtain a preprocessed image sequence, which includes:
respectively carrying out time domain normalization difference value and downsampling treatment on an original video stream to obtain a preprocessed image sequence:
wherein, the calculation of the time domain normalized difference is performed according to the following formula (1):
Figure 168986DEST_PATH_IMAGE001
wherein the method comprises the steps of
Figure 811320DEST_PATH_IMAGE002
Represent the first
Figure 556422DEST_PATH_IMAGE003
Individual skin pixels are in time
Figure 714259DEST_PATH_IMAGE004
Is used for the RGB values of (a),
Figure 325238DEST_PATH_IMAGE005
is a time variation value.
Optionally, the image sequence comprises: a time domain normalized difference image sequence and a downsampled image sequence.
Optionally, in step S102, an image sequence is acquired, the image sequence is input into a deep neural network based on a three-dimensional central differential convolution operator, and the spatio-temporal information is extracted in combination with an attention mask mechanism of a convolution layer, including:
s121: taking the time domain normalized difference image sequence as a motion branch, and inputting the motion branch into a depth neural network based on a three-dimensional center differential convolution operator;
taking the downsampled image sequence as an appearance branch, and inputting the downsampled image sequence into a depth neural network based on a three-dimensional center differential convolution operator;
s122: through a attention mask mechanism, modeling skin interested areas based on appearance branches assist the motion branches to extract space-time information;
s123: repeating the steps S121-S122, extracting the space-time information, and transmitting the space-time information to the full connection layer.
Optionally, in step S121, it includes:
obtaining a three-dimensional center difference convolution operator according to the following formula (2)
Figure 771262DEST_PATH_IMAGE039
：
Figure 308554DEST_PATH_IMAGE040
Wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure 431100DEST_PATH_IMAGE008
is an input feature map which is used to input a feature map,
Figure 483369DEST_PATH_IMAGE009
representing a local receptive field cube, the shape of which is shown,
Figure 405189DEST_PATH_IMAGE010
is a weight that can be learned and is,
Figure 377081DEST_PATH_IMAGE011
representing the current position on the feature map,
Figure 155681DEST_PATH_IMAGE012
representation of receptive fields
Figure 898509DEST_PATH_IMAGE009
And adjacent time steps
Figure 935604DEST_PATH_IMAGE013
Enumeration of middle position, hyper-parameters
Figure 447488DEST_PATH_IMAGE014
For balancing spatial intensity and gradient.
Optionally, in step S122, the method includes:
obtained according to the following formula (3)Note the function of the mask mechanism
Figure 396989DEST_PATH_IMAGE015
The formula:
Figure 673119DEST_PATH_IMAGE016
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure 202320DEST_PATH_IMAGE041
is the appearance branch
Figure 631027DEST_PATH_IMAGE042
A feature map of a layer convolution layer;
Figure 203960DEST_PATH_IMAGE043
is the branch of motion
Figure 983697DEST_PATH_IMAGE042
A feature map of a layer convolution layer;
Figure 847748DEST_PATH_IMAGE021
and
Figure 320842DEST_PATH_IMAGE022
is the first
Figure 143305DEST_PATH_IMAGE044
The height and width of the layer convolution layer feature map;
Figure 879180DEST_PATH_IMAGE023
the sigmoid function is represented as a function,
Figure 592927DEST_PATH_IMAGE024
is the weight of the convolution kernel,
Figure 652018DEST_PATH_IMAGE045
is a convolution kernel offset which is a function of the convolution kernel,
Figure 645382DEST_PATH_IMAGE026
is the L1 norm of the sample,
Figure 335731DEST_PATH_IMAGE027
representing per-element products.
Optionally, in step S103, constructing a multi-task loss function based on the spatio-temporal information and the epsilon-insensitive Huber loss loss function optimizes the deep neural network, including:
calculating the intensity loss of pulse wave and respiratory wave according to the following equation (4) epsilon-insensitive Huber loss loss function
Figure 807164DEST_PATH_IMAGE046
：
Figure 799391DEST_PATH_IMAGE047
Wherein the method comprises the steps of
Figure 150607DEST_PATH_IMAGE048
A true value is indicated and,
Figure 64336DEST_PATH_IMAGE049
representing input
Figure 791989DEST_PATH_IMAGE008
Through a function of
Figure 373143DEST_PATH_IMAGE050
The predicted value after the mapping is used for mapping,
Figure 442731DEST_PATH_IMAGE033
is a hyper-parameter of Huber loss, default to 1,
Figure 93024DEST_PATH_IMAGE034
is an superparameter of epsilon-intrinsic loss, and the default value is 0.1;
constructing a multiple-task loss function in combination with the following equation (5)
Figure 906259DEST_PATH_IMAGE051
：
Figure 390854DEST_PATH_IMAGE036
Wherein the method comprises the steps of
Figure 631343DEST_PATH_IMAGE037
And
Figure 768932DEST_PATH_IMAGE052
weights are loss functions:
and optimizing the weight of the deep neural network through the back propagation loss function value, stopping optimizing after the loss function is not reduced any more, and selecting the deep neural network with the lowest loss function value in the training process.
Optionally, in step S104, the optimized deep neural network is filtered by using a second-order butterworth filter, and the heart rate and the respiration rate are output at the same time, so as to complete non-contact physiological signal detection based on efficient space-time modeling, which includes:
a second-order Butterworth filter deep neural network is adopted, and heart rate and respiratory rate are output simultaneously;
wherein, the cut-off frequency of heart rate is 0.75Hz and 2.5Hz, and the cut-off frequency of respiratory frequency is 0.08Hz and 0.5Hz respectively;
the position of the highest peak value in the power spectrum obtained by filtering the signals is selected as heart rate and respiratory rate output, and non-contact physiological signal detection based on efficient space-time modeling is completed.
In the embodiment of the invention, a remote photoelectric volume pulse wave recovery method for physiological signal non-contact measurement based on efficient space-time modeling is provided. The effective space-time modeling is realized by combining a three-dimensional center differential convolution operator, a motion and appearance double-branch structure and a soft attention mask. The three-dimensional center differential convolution operator is good at describing the intrinsic mode of the pulse wave by a combination of gradient and intensity information. Deep neural networks based on three-dimensional central differential convolution operators can provide more reliable spatio-temporal information modeling capabilities than traditional three-dimensional convolutions.
In addition, the patent firstly introduces an epsilon-insensitive Huber loss loss function in a remote photoplethysmography task, has the advantages of L1 and L2 loss, and combines epsilon-insensitivity to ensure that the loss function can ignore noise samples in a insensitive domain, thereby increasing robustness and displaying better performance.
The embodiment of the invention provides a non-contact physiological signal detection method based on efficient space-time modeling, which can be realized by electronic equipment, wherein the electronic equipment can be a terminal or a server. The flow chart of the non-contact physiological signal detection method based on the efficient space-time modeling as shown in fig. 2, the processing flow of the method can comprise the following steps:
s201: acquiring an original video stream, preprocessing the original video stream, and acquiring a preprocessed image sequence;
in a possible implementation, the original video stream is subjected to a time-domain normalized difference value and downsampling process, and the resolution after the process is 36×36×3. Obtaining a preprocessed image sequence;
wherein, the calculation of the time domain normalized difference is performed according to the following formula (1):
Figure 385858DEST_PATH_IMAGE001
。
wherein the method comprises the steps of
Figure 659713DEST_PATH_IMAGE002
Represent the first
Figure 336682DEST_PATH_IMAGE003
Individual skin pixels are in time
Figure 243459DEST_PATH_IMAGE004
Is used for the RGB values of (a),
Figure 588377DEST_PATH_IMAGE005
is a time variation value.
In a possible embodiment, the image sequence comprises: a time domain normalized difference image sequence and a downsampled image sequence. In this embodiment the image sequences are 10 frames each.
S202: taking the time domain normalized difference image sequence as a motion branch, and inputting the motion branch into a depth neural network based on a three-dimensional center differential convolution operator;
taking the downsampled image sequence as an appearance branch, and inputting the downsampled image sequence into a depth neural network based on a three-dimensional center differential convolution operator;
s203: by noting the masking mechanism, modeling skin regions of interest based on appearance branches assists the motion branches in extracting spatiotemporal information.
S204: repeating the steps S202-203, extracting the space-time information, and transmitting the space-time information to the full connection layer.
In a possible embodiment, the three-dimensional central differential convolution operator is obtained according to the following formula (2):
Figure 795367DEST_PATH_IMAGE053
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure 112079DEST_PATH_IMAGE055
is an input feature map which is used to input a feature map,
Figure 958681DEST_PATH_IMAGE009
representing a local receptive field cube, the shape of which is shown,
Figure 917410DEST_PATH_IMAGE010
is a weight that can be learned and is,
Figure 713328DEST_PATH_IMAGE057
representing the current position on the feature map,
Figure 919050DEST_PATH_IMAGE012
representation of receptive fields
Figure 65997DEST_PATH_IMAGE009
And adjacent time steps
Figure 297259DEST_PATH_IMAGE013
Enumeration of middle position, hyper-parameters
Figure 400213DEST_PATH_IMAGE014
For balancing spatial intensity and gradient.
In one possible implementation of this embodiment, the convolution kernel size is (3 x 3, 32) the number of convolution layers is 2.
In a possible embodiment, the functional formula of the attention mask mechanism is obtained according to the following formula (2):
Figure 324306DEST_PATH_IMAGE058
。
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure 414010DEST_PATH_IMAGE060
is the appearance branch
Figure 698229DEST_PATH_IMAGE042
A feature map of a layer convolution layer;
Figure 468739DEST_PATH_IMAGE061
is the branch of motion
Figure 32576DEST_PATH_IMAGE062
A feature map of a layer convolution layer;
Figure 137804DEST_PATH_IMAGE021
and
Figure 914130DEST_PATH_IMAGE022
is the first
Figure 804726DEST_PATH_IMAGE042
The height and width of the layer convolution layer feature map;
Figure 514363DEST_PATH_IMAGE023
the sigmoid function is represented as a function,
Figure 60882DEST_PATH_IMAGE024
is the weight of the convolution kernel,
Figure 218062DEST_PATH_IMAGE025
is a convolution kernel offset which is a function of the convolution kernel,
Figure 900848DEST_PATH_IMAGE026
is the L1 norm of the sample,
Figure 603224DEST_PATH_IMAGE063
representing per-element products.
In a possible embodiment, layering is performed using average pooling with a core size of (2 x 2, 32) with a probability of Dropout of 0.5.
In a possible implementation, the full connection layer has input/output dimensions 128 and 10 with a probability of Dropout of 0.5.
S205: based on the space-time information and the epsilon-insensitive Huber loss loss function, constructing a multi-task loss function to optimize the deep neural network;
in a possible embodiment, the intensity losses of the pulse wave and the respiration wave are calculated according to the following equation (4) ε -insensitive Huber loss loss function:
Figure 620728DEST_PATH_IMAGE064
wherein the method comprises the steps of
Figure 535594DEST_PATH_IMAGE048
A true value is indicated and,
Figure 400782DEST_PATH_IMAGE065
representing input
Figure 461011DEST_PATH_IMAGE008
Through a function of
Figure 513280DEST_PATH_IMAGE066
The predicted value after the mapping is used for mapping,
Figure 497417DEST_PATH_IMAGE033
is a hyper-parameter of Huber loss, default to 1,
Figure 406992DEST_PATH_IMAGE034
is an superparameter of epsilon-intrinsic loss, and the default value is 0.1;
as a multitasking loss function, the following equation (5) is combined:
Figure 185592DEST_PATH_IMAGE067
wherein the method comprises the steps of
Figure 990737DEST_PATH_IMAGE069
And
Figure 965515DEST_PATH_IMAGE070
weights for a loss function;
and optimizing the weight of the deep neural network through the back propagation loss function value, stopping optimizing after the loss function is not reduced any more, and selecting the deep neural network with the lowest loss function value in the training process.
In one possible embodiment, the deep neural network model with the lowest loss function value is used as the final model.
In a possible implementation, α=β=1 is adopted in one example, and finally, the neural network weight is optimized through the back propagation loss function value, and the neural network weight stops after the loss function is no longer reduced, and the model with the best effect adopts the recovered pulse wave and the respiratory wave as the final model;
s206: and filtering the optimized deep neural network by adopting a second-order Butterworth filter, and outputting the heart rate and the respiratory rate simultaneously to finish non-contact physiological signal detection based on efficient space-time modeling.
In a possible implementation, a second-order Butterworth filter is adopted to further filter the output heart rate and the respiratory rate of the network, wherein the cut-off frequency of the heart rate is 0.75Hz and 2.5Hz, and the cut-off frequency of the respiratory rate is 0.08Hz and 0.5Hz respectively;
and selecting the position of the highest peak value in the power spectrum obtained by the filtering signal as heart rate and respiratory rate output, and finishing non-contact physiological signal detection based on efficient space-time modeling.
In a possible embodiment, the present invention differs from the previous methods in mainly two ways: one is a spatio-temporal network. Three-dimensional convolution and time-shifted convolution are employed in the conventional art to reduce computational budget but without precision gain. The three-dimensional center differential convolution operator used herein may replace conventional convolution operations without the need for additional parameters. The result is improved because of the enhanced spatiotemporal context modeling capabilities that facilitate the representation of appearance and motion information. Meanwhile, the center difference can be regarded as a regularization term, and overfitting is relieved. The other is a network architecture, such as Dual-GAN, which is a design based on generating an antagonistic network architecture for signal decoupling, with performance at certain metrics (e.g., root mean square error on UBFC data sets) superior to the method of the present invention. But Dual GAN contains preprocessing steps called spatiotemporal map generation, including face detection, face keypoint localization, face alignment, skin segmentation and color space transformation, which are relatively complex. Whereas the method of the invention requires only a simple subtraction between frames as input to the motion branch.
Furthermore, in terms of the loss function, epsilon-insensitive Huber loss is employed, the gradient slowly decreases with Huber loss as the loss between the predicted signal and the real signal approaches a minimum, so the model is more robust in signal prediction. The multi-task network provided by the invention can model internal correlation, has accuracy advantage compared with a single-task version, and simultaneously saves computing resources. In general, a remote photoplethysmography measurement network based on efficient spatiotemporal modeling can capture rich temporal context by aggregating temporal difference information related to the remote photoplethysmography technology to obtain a more robust and accurate non-contact physiological signal measurement result.
In the embodiment of the invention, an accurate non-contact physiological signal measurement method based on a three-dimensional center difference convolution attention network is provided, the method is used for efficient space-time modeling, and pulse wave information can be extracted by utilizing a three-dimensional center difference convolution operator through gathering time difference information.
Epsilon-insensitive Huber loss is proposed as a loss function of the non-contact pulse wave measurement network, as it can focus the pulse wave intensity constraint, showing better performance of epsilon-insensitive Huber loss loss function by evaluating different loss functions and combinations thereof.
The network for combined multitasking measurement of heart rate and respiratory motion is further provided, and has the advantage of sharing information between related physiological signals, so that the heart rate and the respiratory rate can be measured simultaneously, the accuracy is further improved, and the calculation cost is reduced. A large number of experiments show that the proposed method has excellent performance on the public database. And cross-database evaluation and ablation studies were performed, demonstrating the effectiveness and robustness of the proposed method.
FIG. 3 is a block diagram illustrating a non-contact physiological signal detection system based on efficient spatiotemporal modeling, according to an exemplary embodiment. Referring to fig. 3, the system 300 includes:
the data acquisition module 310 is configured to acquire an original video stream, perform preprocessing on the original video stream, and acquire a preprocessed image sequence;
the spatio-temporal information extraction module 320 is configured to obtain an image sequence, input the image sequence into a deep neural network based on a three-dimensional central differential convolution operator, and extract spatio-temporal information by combining a attention mask mechanism of a convolution layer;
the model optimization module 330 is configured to construct a multi-task loss function to optimize the deep neural network based on the spatio-temporal information and the epsilon-insensitive Huber loss loss function;
the data output module 340 is configured to filter the optimized deep neural network by using a second-order butterworth filter, and output the heart rate and the respiration rate at the same time, so as to complete non-contact physiological signal detection based on efficient space-time modeling.
Optionally, the data acquisition module 310 is further configured to perform a time domain normalization difference value and a downsampling process on the original video stream respectively, so as to obtain a preprocessed image sequence;
wherein, the calculation of the time domain normalized difference is performed according to the following formula (1):
Figure 274136DEST_PATH_IMAGE001
wherein the method comprises the steps of
Figure 489217DEST_PATH_IMAGE002
Represent the first
Figure 703030DEST_PATH_IMAGE003
Individual skin pixels are in time
Figure DEST_PATH_IMAGE072A
Is used for the RGB values of (a),
Figure 77904DEST_PATH_IMAGE005
is a time variation value.
Optionally, the image sequence comprises: a time domain normalized difference image sequence and a downsampled image sequence.
Optionally, the spatio-temporal information extraction module 320 is further configured to input the time domain normalized difference image sequence as a motion branch into a deep neural network based on a three-dimensional center differential convolution operator;
taking the downsampled image sequence as an appearance branch, and inputting the downsampled image sequence into a depth neural network based on a three-dimensional center differential convolution operator;
through a attention mask mechanism, modeling skin interested areas based on appearance branches assist the motion branches to extract space-time information;
and repeatedly extracting the space-time information, and transmitting the space-time information to the full-connection layer.
Optionally, the spatio-temporal information extraction module 320 is further configured to
Obtaining a three-dimensional center difference convolution operator according to the following formula (2)
Figure 427983DEST_PATH_IMAGE039
：
Figure 751648DEST_PATH_IMAGE073
Wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure 531385DEST_PATH_IMAGE008
is an input feature map which is used to input a feature map,
Figure 847965DEST_PATH_IMAGE009
representing a local receptive field cube, the shape of which is shown,
Figure 131179DEST_PATH_IMAGE010
is a weight that can be learned and is,
Figure 140592DEST_PATH_IMAGE011
representing the current position on the feature map,
Figure 142047DEST_PATH_IMAGE012
representation of receptive fields
Figure 747471DEST_PATH_IMAGE009
And adjacent time steps
Figure 402968DEST_PATH_IMAGE013
Enumeration of middle position, hyper-parameters
Figure 130753DEST_PATH_IMAGE014
For balancing spatial intensity and gradient.
Optionally, the spatio-temporal information extraction module 320 is further configured to
Obtaining a function of the attention mask mechanism according to the following equation (3)
Figure 337612DEST_PATH_IMAGE015
The formula:
Figure 995995DEST_PATH_IMAGE058
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure 457064DEST_PATH_IMAGE075
is the appearance branch
Figure 26190DEST_PATH_IMAGE044
A feature map of a layer convolution layer;
Figure 454766DEST_PATH_IMAGE076
is the branch of motion
Figure 933152DEST_PATH_IMAGE044
A feature map of a layer convolution layer;
Figure 45465DEST_PATH_IMAGE021
and
Figure 36423DEST_PATH_IMAGE022
is the first
Figure 686716DEST_PATH_IMAGE044
The height and width of the layer convolution layer feature map;
Figure 968793DEST_PATH_IMAGE023
the sigmoid function is represented as a function,
Figure 922230DEST_PATH_IMAGE024
is the weight of the convolution kernel,
Figure 100402DEST_PATH_IMAGE045
is a convolution kernel offset which is a function of the convolution kernel,
Figure 441253DEST_PATH_IMAGE026
is the L1 norm of the sample,
Figure 510709DEST_PATH_IMAGE063
representing per-element products.
Optionally, a model optimization module 330 for calculating the intensity loss of pulse wave and respiratory wave according to the following equation (4) [ epsilon ] -insensitive Huber loss loss function
Figure 987827DEST_PATH_IMAGE077
：
Figure 602479DEST_PATH_IMAGE064
Wherein the method comprises the steps of
Figure 774834DEST_PATH_IMAGE048
A true value is indicated and,
Figure 650911DEST_PATH_IMAGE078
representing input
Figure 716956DEST_PATH_IMAGE008
Through a function of
Figure 564826DEST_PATH_IMAGE032
The predicted value after the mapping is used for mapping,
Figure 411429DEST_PATH_IMAGE033
is a hyper-parameter of Huber loss, default to 1,
Figure 370157DEST_PATH_IMAGE034
is an superparameter of epsilon-intrinsic loss, and the default value is 0.1;
constructing a multiple-task loss function in combination with the following equation (5)L Total ：
Figure 353026DEST_PATH_IMAGE067
Wherein the method comprises the steps of
Figure DEST_PATH_IMAGE079
And
Figure 233781DEST_PATH_IMAGE070
weights for a loss function;
and (3) optimizing the weight of the deep neural network through the back propagation loss function value, and stopping optimizing after the loss function is not reduced any more, namely selecting the deep neural network with the lowest loss function value in the training process.
Optionally, the data output module 340 is configured to use a second order butterworth filter deep neural network to output the heart rate and the respiration rate simultaneously;
wherein, the cut-off frequency of heart rate is 0.75Hz and 2.5Hz, and the cut-off frequency of respiratory frequency is 0.08Hz and 0.5Hz respectively;
the position of the highest peak value in the power spectrum obtained by filtering the signals is selected as heart rate and respiratory rate output, and non-contact physiological signal detection based on efficient space-time modeling is completed.
In the embodiment of the invention, a remote photoelectric volume pulse wave recovery method for physiological signal non-contact measurement based on efficient space-time modeling is provided. The effective space-time modeling is realized by combining a three-dimensional center differential convolution operator, a motion and appearance double-branch structure and a soft attention mask. The three-dimensional center differential convolution operator is good at describing the intrinsic mode of the pulse wave by a combination of gradient and intensity information. Deep neural networks based on three-dimensional central differential convolution operators can provide more reliable spatio-temporal information modeling capabilities than traditional three-dimensional convolutions. In addition, the patent firstly introduces an epsilon-insensitive Huber loss loss function in a remote photoplethysmography task, and simultaneously combines epsilon-insensitivity to ensure that the loss function can ignore noise samples in a insensitive domain, thereby increasing robustness and displaying better performance.
Fig. 4 is a schematic structural diagram of an electronic device 400 according to an embodiment of the present invention, where the electronic device 400 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 401 and one or more memories 402, where at least one instruction is stored in the memories 402, and the at least one instruction is loaded and executed by the processors 401 to implement the following steps of a non-contact physiological signal detection method based on efficient space-time modeling:
s1: acquiring an original video stream, preprocessing the original video stream, and acquiring a preprocessed image sequence;
s2: acquiring the image sequence, inputting the image sequence into a depth neural network based on a three-dimensional center differential convolution operator, and extracting space-time information by combining a notice mask mechanism of a convolution layer;
s3: constructing a multi-task loss function to optimize the deep neural network based on the space-time information and the epsilon-insensitive Huber loss loss function;
s4: and filtering the optimized deep neural network by adopting a second-order Butterworth filter, and outputting the heart rate and the respiratory rate simultaneously to finish non-contact physiological signal detection based on efficient space-time modeling.
In an exemplary embodiment, a computer readable storage medium, e.g. a memory comprising instructions executable by a processor in a terminal to perform the above-described non-contact physiological signal detection method based on efficient spatiotemporal modeling, is also provided. For example, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims (2)

1. The non-contact physiological signal detection method based on the efficient space-time modeling is characterized by comprising the following steps of:
s1: acquiring an original video stream, preprocessing the original video stream, and acquiring a preprocessed image sequence;
s2: acquiring the image sequence, inputting the image sequence into a depth neural network based on a three-dimensional center differential convolution operator, and extracting space-time information by combining a notice mask mechanism of a convolution layer;
s3: constructing a multi-task loss function to optimize the deep neural network based on the space-time information and the epsilon-insensitive Huber loss loss function;
in the step S3, constructing a multi-task loss function to optimize the deep neural network based on the spatio-temporal information and the epsilon-insensitive Huber loss loss function includes:
calculating the intensity loss of pulse wave and respiratory wave according to the following equation (4) epsilon-insensitive Huber loss loss function
Figure QLYQS_1
：
Figure QLYQS_2
（4）
Wherein the method comprises the steps of
Figure QLYQS_3
Indicating true value(s)>
Figure QLYQS_4
Representation input +.>
Figure QLYQS_5
Through a function->
Figure QLYQS_6
Mapped prediction value ∈ ->
Figure QLYQS_7
Is a superparameter of Huber loss, default 1, < >>
Figure QLYQS_8
Is an superparameter of epsilon-intrinsic loss, and the default value is 0.1;
the following formula (5) is combinedBuilding a multitasking loss functionL Total ：
Figure QLYQS_9
（5）
Wherein the method comprises the steps of
Figure QLYQS_10
And->
Figure QLYQS_11
Weights for a loss function;
optimizing the weight of the deep neural network through the back propagation loss function value, stopping optimizing after the loss function is not reduced any more, and selecting a model of the deep neural network with the lowest loss function value in the training process;
s4: and filtering the optimized deep neural network by adopting a second-order Butterworth filter, and outputting the heart rate and the respiratory rate simultaneously to finish non-contact physiological signal detection based on efficient space-time modeling.
2. A non-contact physiological signal detection system based on efficient spatiotemporal modeling, characterized in that the system is adapted for use in the method of claim 1 above, the system comprising:
the data acquisition module is used for acquiring an original video stream, preprocessing the original video stream and acquiring a preprocessed image sequence;
the space-time information extraction module is used for acquiring the image sequence, inputting the image sequence into a deep neural network based on a three-dimensional center difference convolution operator, and extracting space-time information by combining a attention mask mechanism of a convolution layer;
the model optimization module is used for constructing a multi-task loss function to optimize the deep neural network based on the space-time information and the epsilon-insensitive Huber loss loss function;
and the data output module is used for filtering the optimized deep neural network by adopting a second-order Butterworth filter, outputting heart rate and respiratory rate at the same time, and finishing non-contact physiological signal detection based on efficient space-time modeling.
CN202211451949.4A 2022-11-17 2022-11-17 Non-contact physiological signal detection method and system based on efficient space-time modeling Active CN115624322B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211451949.4A CN115624322B (en) 2022-11-17 2022-11-17 Non-contact physiological signal detection method and system based on efficient space-time modeling

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211451949.4A CN115624322B (en) 2022-11-17 2022-11-17 Non-contact physiological signal detection method and system based on efficient space-time modeling

Publications (2)

Publication Number Publication Date
CN115624322A CN115624322A (en) 2023-01-20
CN115624322B true CN115624322B (en) 2023-04-25

Family

ID=84910095

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211451949.4A Active CN115624322B (en) 2022-11-17 2022-11-17 Non-contact physiological signal detection method and system based on efficient space-time modeling

Country Status (1)

Country Link
CN (1) CN115624322B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115840890B (en) * 2023-02-24 2023-05-19 北京科技大学 Emotion recognition method and device based on non-contact physiological signals

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020047750A1 (en) * 2018-09-04 2020-03-12 深圳先进技术研究院 Arrhythmia detection method and apparatus, electronic device, and computer storage medium
CN112998690A (en) * 2021-03-29 2021-06-22 华南理工大学 Pulse wave multi-feature fusion-based respiration rate extraction method

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113343821B (en) * 2021-05-31 2022-08-30 合肥工业大学 Non-contact heart rate measurement method based on space-time attention network and input optimization
CN115024706A (en) * 2022-05-16 2022-09-09 南京邮电大学 Non-contact heart rate measurement method integrating ConvLSTM and CBAM attention mechanism

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020047750A1 (en) * 2018-09-04 2020-03-12 深圳先进技术研究院 Arrhythmia detection method and apparatus, electronic device, and computer storage medium
CN112998690A (en) * 2021-03-29 2021-06-22 华南理工大学 Pulse wave multi-feature fusion-based respiration rate extraction method

Also Published As

Publication number Publication date
CN115624322A (en) 2023-01-20

Similar Documents

Publication Publication Date Title
CN110944577B (en) Method and system for detecting blood oxygen saturation
Meng et al. Enhancing dynamic ECG heartbeat classification with lightweight transformer model
CN110133610B (en) Ultra-wideband radar action identification method based on time-varying distance-Doppler diagram
CN109645989B (en) Anesthesia depth estimation system
Hnoohom et al. An Efficient ResNetSE Architecture for Smoking Activity Recognition from Smartwatch.
CN112465905A (en) Characteristic brain region positioning method of magnetic resonance imaging data based on deep learning
CN112001122A (en) Non-contact physiological signal measuring method based on end-to-end generation countermeasure network
CN115624322B (en) Non-contact physiological signal detection method and system based on efficient space-time modeling
CN115024706A (en) Non-contact heart rate measurement method integrating ConvLSTM and CBAM attention mechanism
CN116343284A (en) Attention mechanism-based multi-feature outdoor environment emotion recognition method
Ma et al. EEG Signal and Feature Interaction Modeling‐Based Eye Behavior Prediction Research
Brophy et al. An interpretable machine vision approach to human activity recognition using photoplethysmograph sensor data
Zhao et al. A CNN based human bowel sound segment recognition algorithm with reduced computation complexity for wearable healthcare system
Asyhar et al. Implementation LSTM Algorithm for Cervical Cancer using Colposcopy Data
CN116012916A (en) Remote photoplethysmograph signal and heart rate detection model construction method and detection method
Gan et al. Human-computer interaction based interface design of intelligent health detection using PCANet and multi-sensor information fusion
Lee et al. Lstc-rppg: Long short-term convolutional network for remote photoplethysmography
He et al. Localization of myocardial infarction using a multi-branch weight sharing network based on 2-D vectorcardiogram
Chen et al. A novel imbalanced dataset mitigation method and ECG classification model based on combined 1D_CBAM-autoencoder and lightweight CNN model
CN111278353A (en) Method and system for detecting vital sign signal noise
CN111461206A (en) Electroencephalogram-based fatigue detection method with steering wheel embedded with electroencephalogram sensor
CN116138756A (en) Non-contact heart rate detection method, system and medium based on face feature point detection
Aruna et al. Accelerating deep convolutional neural network on FPGA for ECG signal classification
CN115813409A (en) Ultra-low-delay moving image electroencephalogram decoding method
Kuang et al. Remote photoplethysmography signals enhancement based on generative adversarial networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant