WO2022221987A1 - 一种基于可穿戴传感信息融合的能量代谢评估方法和系统 - Google Patents

一种基于可穿戴传感信息融合的能量代谢评估方法和系统 Download PDF

Info

Publication number
WO2022221987A1
WO2022221987A1 PCT/CN2021/088096 CN2021088096W WO2022221987A1 WO 2022221987 A1 WO2022221987 A1 WO 2022221987A1 CN 2021088096 W CN2021088096 W CN 2021088096W WO 2022221987 A1 WO2022221987 A1 WO 2022221987A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
acceleration
information
convolutional neural
energy metabolism
Prior art date
Application number
PCT/CN2021/088096
Other languages
English (en)
French (fr)
Inventor
李烨
孙方敏
倪志强
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Priority to PCT/CN2021/088096 priority Critical patent/WO2022221987A1/zh
Publication of WO2022221987A1 publication Critical patent/WO2022221987A1/zh

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/318—Heart-related electrical modalities, e.g. electrocardiography [ECG]

Definitions

  • the invention relates to the technical field of information processing, and more particularly, to a method and system for evaluating energy metabolism based on wearable sensor information fusion.
  • the existing methods for measuring energy consumption evaluation mainly include: direct calorimetry, indirect calorimetry, double-labeled water method and self-report.
  • Direct calorimetry measures the energy consumed by a person over a period of time by building a closed and insulated environment and measuring the heat emitted by the human body to the environment. This method has the highest accuracy.
  • the indirect calorimetry method uses the relationship between energy metabolism and respiratory entropy to indirectly calculate human energy consumption by measuring the amount of oxygen inhaled and carbon dioxide exhaled. In essence, the double-labeled water method also calculates energy consumption by measuring the amount of exhaled carbon dioxide. The amount of carbon dioxide, and finally the energy consumption of the human body is further calculated.
  • the self-report method estimates human energy consumption through physical activity records and questionnaires filled in by subjects. This method is relatively low-cost, simple to operate, and is the most common and practical method.
  • Wearable method is a newly emerging energy consumption monitoring method.
  • commonly used methods include heart rate monitoring method and acceleration sensor method.
  • the heart rate monitoring method calculates energy consumption by monitoring the human heart rate, because the heart rate can reflect the human body's functional activity state and is closely related to energy metabolism.
  • the accelerometer method is to evaluate the body's energy consumption by measuring the duration and intensity of physical activity based on the motion or acceleration information of the limb to which the sensor is attached. Wearable methods have become an important research topic in the field of energy consumption computing due to the advantages of convenient wearing and low cost.
  • the patent application CN201810092947.8 uses piezoelectric energy harvesters instead of traditional accelerometers to evaluate human movement, and establishes four independent random forest regression prediction models according to different movements with different degrees of exercise intensity.
  • the power consumption is only Only 0.2% of the energy metabolism estimation method based on the accelerometer can effectively reduce the power consumption of the system, but the disadvantage is that the data collected by the piezoelectric energy harvester contains less information, which affects the effect of human motion assessment.
  • the patent application CN202011288776.X calculates the comprehensive standard deviation of each axis signal collected by the six-axis inertial sensor as the feature of the instantaneous motion energy consumption of the human body, and establishes a linear model for calculating the motion energy consumption.
  • the calculation of this method is very simple, but it only uses the time domain information of the acceleration signal in essence, and the final model is only a simple linear model, so the accuracy of the calculation of motion energy consumption is not high.
  • the patent application CN202011249044.X designs three linear energy metabolism calculation models based on the heart rate value of human movement and the cumulative total number of steps.
  • the linear models use the combination of heart rate, accumulated acceleration value, movement speed and the wearer's personal information as input.
  • the input of this method contains various information of human motion, but its calculation model is too simple, and the accuracy of exercise energy consumption calculation is not high.
  • the purpose of the present invention is to overcome the above-mentioned defects of the prior art, and to provide an energy metabolism evaluation method and system based on wearable sensor information fusion.
  • an energy metabolism evaluation method based on wearable sensor information fusion includes the following steps:
  • the electrocardiographic feature, the acceleration feature and the physical feature information of the human body structure are fused, and corresponding energy metabolism information is predicted based on the fused feature.
  • an energy metabolism evaluation system based on wearable sensor information fusion includes:
  • Data acquisition unit used to acquire the ECG signal information reflecting the physiological state of the human body, the acceleration signal information reflecting the exercise intensity level of the human body, and the physical feature information of the human body structure;
  • Feature extraction unit used to input the ECG signal information into the first convolutional neural network to extract ECG features, and input the acceleration signal information into the second convolutional neural network to extract acceleration features, wherein the first convolutional neural network
  • the network and the second convolutional neural network are obtained by training, and the first convolutional neural network and the second convolutional neural network are both multi-branch structures to extract multi-scale features;
  • Prediction unit used to fuse the ECG feature, the acceleration feature and the physical feature information of the human body structure, and predict the corresponding energy metabolism information based on the fused feature.
  • the present invention has the advantage that the electrocardiogram signal, acceleration signal and precise energy metabolism value (gold standard) in the process of human body movement are simultaneously collected through the wearable device and the human body energy metabolism instrument based on the breath measurement method. , using the deep learning method to effectively integrate the ECG signal, acceleration signal and personal basic information synchronously collected by the wearable device to establish a regression model for accurate evaluation of energy metabolism, and realize the accurate calculation of energy consumption during exercise.
  • FIG. 1 is a flowchart of an energy metabolism assessment method based on wearable sensor information fusion according to an embodiment of the present invention
  • FIG. 2 is an overall architecture diagram of an energy metabolism assessment method based on wearable sensor information fusion according to an embodiment of the present invention.
  • the present invention designs a multi-branch and multi-scale convolutional neural network, introduces convolution kernels of different scales in different branch networks, and constructs convolution blocks of different scales, which can deeply mine ECG signals and acceleration signals. Traits closely related to energy metabolism in different dimensions.
  • the present invention designs a cascaded regression energy metabolism prediction method, which further improves the accuracy of energy metabolism prediction.
  • the provided energy metabolism assessment method based on wearable sensor information fusion includes the following steps.
  • Step S110 preprocessing the data to construct training samples.
  • the data preprocessing process mainly includes signal denoising, composite acceleration extraction, data segmentation, data enhancement and so on.
  • the acceleration signal was preprocessed using a Butterworth low-pass filter with a cutoff frequency of 10Hz to eliminate the interference of high frequency noise; then the gravitational acceleration was estimated using a Wiener filter with a window size of 1 second, and the accelerometer data were The direction of gravitational acceleration is subtracted from , to exclude the effect of gravitational acceleration on the motion signal.
  • resultant acceleration information is extracted.
  • the magnitude of the resultant acceleration captures the magnitude of the acceleration during motion, independent of the orientation of the sensor.
  • the six-degree-of-freedom inertial sensor can collect the acceleration and angular acceleration of the x, y, and z axes, so the resultant acceleration and the resultant angular acceleration are calculated. The effect of sensor position changes can be eliminated.
  • each window is a sample, and then the calorie data of each window is calculated.
  • the average value is taken as the real energy consumption value of the sample.
  • each window contains 6000x8 acceleration data (8 channels include acceleration and angular acceleration of x, y, z axis, and resultant acceleration and resultant angular acceleration), 12000x1 ECG data and 1 calorie data.
  • a data augmentation technique is used for the training data to effectively expand the data samples and increase the generalization ability of the model.
  • noise data can also be added to improve the robustness of the model.
  • a variety of data augmentation methods can be used, including: multiplying the magnitude of the acceleration data and ECG data by a random scalar that follows a Gaussian distribution with mean 1 and standard deviation 0.1 to randomly change the data magnitude;
  • the axis data is randomly swapped or rotated by a random angle to simulate different placements of the accelerometer.
  • step S120 a model based on deep learning is trained to extract ECG features and acceleration features, and fuse them with the physical feature information of human body structure.
  • two multi-branch convolutional neural networks are designed to extract the features of the acceleration signal and the ECG signal respectively.
  • Each multi-branch convolutional neural network contains three convolutional blocks, and each convolutional block uses convolution kernels of different sizes.
  • the convolution kernel sizes are 3, 5, and 7, and for the ECG signal, the convolution kernel sizes are 5, 7, and 9, respectively.
  • the acceleration signal convolution block contains 8 convolution layers and 5 pooling layers, while the ECG signal convolution block contains 10 convolution layers and 6 pooling layers, and the activation function selects a modified linear unit. Batch normalization is added after each convolutional layer to alleviate the internal covariate shift problem and speed up the training process.
  • the dropout layer is added at the end to prevent the model from falling into overfitting.
  • the specific structure of a single 1xk convolution block is shown in Table 1 below, where k represents the size of the convolution kernel, conv represents the convolution layer, maxpool represents the maximum pooling layer, and global avgpool represents the global average pooling layer.
  • each branch network uses convolution kernels of different sizes, context information of different scales of the signal can be extracted, and the problem of difficulty in feature extraction caused by randomness and self-similarity of acceleration signals and ECG signals is solved.
  • the individual basic information that affects the energy metabolism level of the individual or the physical characteristics information of the human body structure (such as gender, age, waist circumference, height, weight and other parameters) and different scale acceleration characteristics and heart rate.
  • the electrical features are further fused through a fully connected layer with 128 hidden neural units to improve the generalization ability of the model to predict energy metabolism in different populations.
  • the specific value of energy consumption will change due to the difference of each person's body structure characteristics. Therefore, inputting the physical characteristics of the body structure into the consumption prediction model and further adjusting the model can achieve the effect of adapting to different individuals.
  • a multi-branch convolutional neural network that simultaneously extracts multi-scale contextual features of ECG and acceleration signals, which can effectively capture small changes hidden in the signals.
  • a wearable human body energy metabolism assessment scheme based on multi-source information fusion is proposed, using the ECG signal reflecting the physiological state of the human body, the acceleration signal reflecting the human body exercise intensity level and the physical feature information of the human body structure as the model input, mining from multiple dimensions. The key features that affect human energy consumption can be excavated more comprehensive and subtle features that affect the level of energy metabolism.
  • Step S130 performing regression prediction of energy metabolism information based on the fusion feature.
  • a coarse-grained estimate of energy metabolism levels is first performed by an ordered regression method. Then, on the basis of the coarse-grained estimation, the linear regression method is used to further estimate the energy metabolism prediction value in fine-grained manner, thereby significantly improving the accuracy of the energy consumption prediction model.
  • the essence of ordinal regression is to convert the original regression task into multiple classification tasks through label discretization. Therefore, the real value of energy metabolism is firstly discretized, for example, the equal interval discretization method is used. If the energy metabolism value interval [ ⁇ , ⁇ ] is equally divided into K equal parts, the discrete interval becomes [r 1 , r 2 , ..., r K ], where ri is defined as:
  • the next step is to encode the discretized energy metabolism value.
  • Common encoding methods include hard label method and soft label method. Since the hard label encoding method may lead to the loss of valuable information, the processing of soft label considers the real value. Continuity is more conducive to the learning of the model. Therefore, the encoding method of soft labels is preferably used. ri is encoded into a soft label vector yi of dimension 1xK, and the jth element in vector yi is defined as
  • ⁇ (r i , r j ) is the distance metric function, which represents the distance between the discretized energy metabolism value ri and the discrete level r j
  • the distance metric can use Euclidean distance, Manhattan distance, Chebyshev distance Wait.
  • mapping can be decomposed into two steps, that is, the predicted vector corresponding to the real soft label vector y is obtained by the ordered regression calculation And linear regression calculation to get the predicted value corresponding to the true value y
  • the process can be expressed as:
  • W 1 and W 2 are network weight parameters
  • b 1 and b 2 are network bias parameters.
  • two cost functions are defined for a two-step (ie, ordered regression and linear regression) regression task.
  • the cost function of ordered regression uses KL divergence as the loss function, which is used to measure the difference between the real soft label and the predicted motion energy consumption distribution, and control the interval classification accuracy of energy consumption, which is expressed as:
  • N is the total number of samples
  • y ij is the jth element of the real soft label vector of energy metabolism of the ith sample
  • y ij is the jth element of the prediction vector for the ith sample.
  • the cost function of linear regression controls the prediction of the final motion energy consumption, for example, using an absolute value loss function, expressed as:
  • y i is the true value of energy metabolism of the ith sample, is the predicted value of energy metabolism for the ith sample.
  • ⁇ is the hyperparameter used to balance the contributions of the two cost functions during training.
  • a cascaded regression energy metabolism prediction method which integrates ECG features, acceleration features and personal basic information and uses two regression predictions based on soft label-based ordered regression and ordinary linear regression to realize the prediction of exercise. Accurate calculation of energy consumption.
  • Step S140 using the trained model to predict energy metabolism information in real time.
  • the energy metabolism information of an individual can be predicted in real time.
  • combined acceleration extraction, data segmentation, etc. are processed into a series of continuous data, combined with the physical characteristics information of the individual human body structure and input into the trained model (including neural network model and regression prediction model), the corresponding predicted energy metabolism can be obtained in real time. information.
  • the data processing process is similar to step S110, and is not repeated here.
  • the present invention also provides an energy metabolism assessment system based on wearable sensor information fusion, which is used to implement one or more aspects of the above method.
  • the system includes: a data acquisition unit, which is used for acquiring electrocardiographic signal information reflecting the physiological state of the human body, acceleration signal information reflecting the human body exercise intensity level, and physical feature information of the human body structure; a feature extraction unit, which is used for The electrocardiographic signal information is input into the first convolutional neural network to extract electrocardiographic features, and the acceleration signal information is input into the second convolutional neural network to extract acceleration features, wherein the first convolutional neural network and the second convolutional neural network are processed by the first convolutional neural network.
  • both the first convolutional neural network and the second convolutional network are multi-branch structures to extract multi-scale features;
  • a prediction unit which is used to combine the electrocardiographic features, acceleration features and the physical features of the human body structure The information is fused, and the corresponding energy metabolism information is predicted based on the fusion features.
  • experiments were carried out.
  • the experiment recruited 20 physically fit and athletic subjects, 10 males and 10 females, aged 24-43 years. Each subject performed 2 experiments, with a total of 40 experimental data. Before each test, measure and record the subject's basic information such as height, weight, waist circumference, age, gender, etc.
  • the subject wears an exercise heart rate belt, a Shimmer3IMU acceleration sensor is tied to the waist, and an energy metabolism tester is worn.
  • the respirator is tested on the treadmill for incremental load exercise, and the wearable ECG, acceleration signals, and energy metabolism standard values are collected synchronously.
  • the Shimmer3 inertial sensor samples acceleration and gyroscope signals at 100Hz.
  • the experiment adopts leave-one-out cross-validation, that is, the data of 19 subjects is selected from the data of 20 subjects as the training set, and then the data of the remaining 1 subject is used for testing, and 20 rounds of testing are performed iteratively.
  • the test results are shown in Table 2
  • the present invention uses the data (gold standard) collected by the human energy metabolism tester based on the breath measurement method as the model reference real value, which ensures the reliability of the energy metabolism prediction model; the original single-lead ECG signal is used for the first time.
  • the ECG data contains more effective information reflecting the energy metabolism level, which improves the accuracy of the human energy metabolism prediction.
  • Various data enhancement techniques are used to increase the training samples, which improves the accuracy of the human energy metabolism prediction.
  • the model generalization ability eliminates the influence of accelerometer orientation changes and increases the robustness of the model; the designed multi-branch convolutional neural network realizes end-to-end feature learning and simplifies the feature extraction process.
  • the network has multiple branches , each branch introduces a multi-channel and multi-scale convolution kernel. Compared with the features extracted by manual design, the features extracted by this network are more comprehensive and of higher quality; the energy metabolism prediction method of cascade regression is designed to reduce the difficult energy
  • the metabolic regression prediction problem is decomposed into two sub-problems: simple interval classification and small-scale regression, which effectively reduces the difficulty of predicting human energy metabolism and improves the prediction accuracy of the model.
  • the present invention may be a system, method and/or computer program product.
  • the computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present invention.
  • a computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM erasable programmable read only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disk read only memory
  • DVD digital versatile disk
  • memory sticks floppy disks
  • mechanically coded devices such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • Computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.
  • the computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • the computer program instructions for carrying out the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages.
  • Source or object code written in any combination including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect).
  • LAN local area network
  • WAN wide area network
  • custom electronic circuits such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs)
  • FPGAs field programmable gate arrays
  • PDAs programmable logic arrays
  • Computer readable program instructions are executed to implement various aspects of the present invention.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are all equivalent.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Physics & Mathematics (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physiology (AREA)
  • Dentistry (AREA)
  • Cardiology (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

一种基于可穿戴传感信息融合的能量代谢评估方法和系统，方法包括：获取反映人体生理状态的心电信号信息、反映人体运动强度水平的加速度信号信息和人体结构物理特征信息；将心电信号信息输入第一卷积神经网络提取心电特征，并将加速度信号信息输入第二卷积神经网络提取加速度特征，其中，第一卷积神经网络和第二卷积神经网络经训练获得，且第一卷积神经网络和第二卷积神经网络均是多分支结构，以提取多尺度特征；将心电特征、加速度特征和人体结构物理特征信息进行融合，并基于融合特征预测对应的能量代谢信息。实现了运动过程中能量消耗的精准计算，且泛化能力强。

Description

一种基于可穿戴传感信息融合的能量代谢评估方法和系统 技术领域
本发明涉及信息处理技术领域，更具体地，涉及一种基于可穿戴传感信息融合的能量代谢评估方法和系统。
背景技术
随着社会经济的发展和人们生活水平的提高，由人体能量代谢失衡引起的多种代谢性疾病(如脂肪肝、糖尿病、高血脂等)成为高发性疾病。科学合理的控制饮食能量摄入和运动能量消耗是代谢性疾病预防及康复治疗的有效手段，同时也是个体主动健康管理的主要部分。人体是一个复杂的时变、非线性系统，运动能量消耗水平受很多因素影响，例如运动强度、环境温湿度、个体身体物理状态(身高、体重、年龄等参数)及精神状态等，因此，对人体运动过程中的能量消耗的精准评估是当前代谢性疾病防控领域面临的一个关键科学问题。
人体能量消耗分为三个部分：基础能量消耗，食物生热作用和身体活动能量消耗。已有的测量能量消耗评估方法主要有：直接测热法、间接测热法、双标水法和自我报告等。
直接测热法通过搭建密闭隔热的环境，测量人体向环境散发出的热量来测量一个人在一段时间内消耗的能量，这种方法准确度最高。间接测热法则是利用能量代谢与呼吸熵的关系，通过测量人体吸入氧气量和呼出二氧化碳量间接计算人体能量消耗。双标水法本质上也是通过测量呼出二氧化碳量计算能量消耗，原理是受试者口服一定量的同时含有氢和氧稳定同位素的水，然后通过测定这两种同位素浓度在人体中的变化计算呼出二氧化碳量，最后进一步计算人体能量消耗。自我报告法通过受试者填写的身体活动记录和问卷调查估计人体能量消耗，该方法成本相对较低、操作简单，是最普遍、最实用的方法。
在上述方法中，自我报告依赖于参与者记忆，主观性太强导致准确性和可靠性低。而直接测热法、间接测热法、双标水法测量能量消耗的有效性和可靠性很高，但它们都存在测量复杂、成本昂贵、耗费时间长、侵入性等缺点，其应用受到限制。因此，这些方法都不适合在大规模人群和日常生活中推广。
可穿戴方法是最近新兴起来的能量消耗监测方法，目前常用的方法包括心率监测法和加速度传感器法等。心率监测法通过监测人体的心率来计算能量消耗，因为心率能够反应人体机能活动状态，与能量代谢密切相关。加速度传感器法则是根据传感器所依附的肢体的运动或加速度信息，通过测量身体活动的持续时间和强度评估机体能量消耗。可穿戴方法由于佩戴方便、成本低廉等优点，已经成为能量消耗计算领域重要的研究课题。
例如，专利申请CN201810092947.8将压电式能量采集器代替传统的加速度计来进行人体运动评估，并根据运动剧烈程度不同的动作，建立了四个独立的随机森林回归预测模型，其功耗仅仅只有基于加速度计的能量代谢估计方法的0.2％，有效地降低了系统的功耗，但缺点是压电式能量采集器采集的数据包含信息较少，影响人体运动评估的效果。
专利申请CN202011288776.X计算六轴惯性传感器采集的各轴信号的综合标准差作为人体瞬时运动能量消耗的特征，建立了一个计算运动能量消耗的线性模型。该方法计算很简单，但是其本质上只利用了加速度信号的时域信息，且最后构建的模型也只是简单的线性模型，因此运动能量消耗计算的准确性不高。
专利申请CN202011249044.X根据人体运动的心率值和累加总步数，设计了三种线性的能量代谢计算模型，线性模型分别使用心率、加速度累计值、运动速度和佩戴者的个人信息结合作为输入。该方法的输入包含了人体运动的各种信息，但其计算模型过于简单，运动能量消耗计算的准确性也不高。
总之，现有的基于可穿戴的运动能量消耗评估方法大都是基于心率或步数估算的，缺少与能量代谢测量金标准的对标分析，可信性较差。而精准度较高的直接测热法和呼吸测量法又因设备庞大、价格昂贵无法普及使 用。随着微电子技术、MEMS传感器技术及计算机技术的发展，可穿戴设备的感知和计算功能越来越强大。可穿戴设备上集成的多种生理、运动监测传感器能够对人体的生理、运动状态进行多维度，全时段的监测，这使得基于可穿戴传感的人体能量代谢评估成为可能。而目前采用可穿戴设备对动运过程中能量消耗的精准评估尚缺乏行之有效的手段。
发明内容
本发明的目的是克服上述现有技术的缺陷，提供一种基于可穿戴传感信息融合的能量代谢评估方法和系统。
根据本发明的第一方面，提供一种基于可穿戴传感信息融合的能量代谢评估方法。该方法包括以下步骤：
获取反映人体生理状态的心电信号信息、反映人体运动强度水平的加速度信号信息和人体结构物理特征信息；
将所述心电信号信息输入第一卷积神经网络提取心电特征，并将所述加速度信号信息输入第二卷积神经网络提取加速度特征，其中，第一卷积神经网络和第二卷积神经网络经训练获得，且第一卷积神经网络和第二卷积网络均是多分支结构，以提取多尺度特征；
将所述心电特征、加速度特征和所述人体结构物理特征信息进行融合，并基于融合特征预测对应的能量代谢信息。
根据本发明的第二方面，提供一种基于可穿戴传感信息融合的能量代谢评估系统。该系统包括：
数据采集单元：用于获取反映人体生理状态的心电信号信息、反映人体运动强度水平的加速度信号信息和人体结构物理特征信息；
特征提取单元：用于将所述心电信号信息输入第一卷积神经网络提取心电特征，并将所述加速度信号信息输入第二卷积神经网络提取加速度特征，其中，第一卷积神经网络和第二卷积神经网络经训练获得，且第一卷积神经网络和第二卷积网络均是多分支结构，以提取多尺度特征；
预测单元：用于将所述心电特征、加速度特征和所述人体结构物理特征信息进行融合，并基于融合特征预测对应的能量代谢信息。
与现有技术相比，本发明的优点在于，通过可穿戴设备和基于呼吸测量法的人体能量代谢仪，同时采集人体运动过程中的心电信号、加速度信号和精准能量代谢值(金标准)，利用深度学习方法有效融合可穿戴设备同步采集的心电信号、加速度信号及个人基本信息建立能量代谢精准评估回归模型，实现了对运动过程中能量消耗的精准计算。
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其它特征及其优点将会变得清楚。
附图说明
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。
图1是根据本发明一个实施例的基于可穿戴传感信息融合的能量代谢评估方法的流程图；
图2是根据本发明一个实施例的基于可穿戴传感信息融合的能量代谢评估方法的总体架构图。
具体实施方式
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步 讨论。
由于人体加速度信号、心电信号同属人体生理信号中的低频信号，受生理系统的自组织性的影响表现出随机性和自相似性，普通的网络难以将隐藏在信号中的各种微小变化完全挖掘出来。因此本发明设计了一个多分支、多尺度的卷积神经网络，在不同的分支网络中引入了不同尺度的卷积核，构建了不同尺度的卷积块，可以深度挖掘心电信号和加速度信号在不同维度上与能量代谢密切相关的特征。此外，考虑到普通线性回归方法无法一次性精准地估计出能量消耗的值，本发明设计了一种级联回归的能量代谢预测方法，进一步提升了能量代谢预测的准确度。
具体地，参见图1所示，所提供的基于可穿戴传感信息融合的能量代谢评估方法包括以下步骤。
步骤S110，对数据进行预处理，以构建训练样本。
数据预处理过程主要包括信号去噪、合加速度提取、数据分割、数据增强等。
在运动过程中采集的加速度信号难免会存在大量噪声，因此需要去除噪声。首先使用截止频率为10Hz的巴特沃斯低通滤波器对加速度信号进行预处理，以消除高频噪声的干扰；接着使用窗口大小为1秒的维纳滤波器估计重力加速度，并从加速度计数据中减去了重力加速度的方向，以排除重力加速度对运动信号的影响。
在一个实施例中，提取了合加速度信息。合加速度的幅值捕获了运动过程中加速度的幅值，与传感器的方向无关。六自由度惯性传感器可以采集x、y、z轴的加速度和角加速度，因此计算合加速度与合角加速度
Figure PCTCN2021088096-appb-000001
可以消除传感器位置变化影响。
进一步地，为构建训练样本集进行数据分割。例如，能量消耗的估计问题按分钟估计，将加速度信号、心电信号以及能量代谢真实值数据分割为一系列连续的1分钟窗口，每个窗口就是一个样本，然后计算每个窗口的卡路里数据的平均值作为该样本真实的能量消耗值。当采集的加速度信号和心电信号采样率分别为100Hz和200Hz时，每个窗口包含6000x8个加速度数据(8通道包括x、y、z轴的加速度和角加速度以及合加速度、合 角加速度)、12000x1个心电数据以及1个卡路里数据。
为了提高模型训练精度，优选地，对训练数据使用数据增强技术来有效地扩充数据样本，增加模型的泛化能力，另外也可以增加噪声数据，提高模型的鲁棒性。
例如，可使用多种数据增强手段，包括：将加速度数据和心电数据的幅度乘以服从均值1和标准差为0.1的高斯分布的随机标量，用来随机更改数据幅度；对加速度计的三轴数据进行随机交换排列或者对其旋转一个随机角度，用来模拟加速度计不同放置的方式。
步骤S120，训练基于深度学习的模型，以提取心电特征、加速度特征，并与人体结构物理特征信息进行融合。
结合图2所示，在一个实施例中，设计了两个多分支卷积神经网络分别用于提取加速度信号和心电信号的特征。每个多分支卷积神经网络包含三个卷积块，每个卷积块使用不同大小的卷积核。对于加速度信号，卷积核大小分别为3、5、7，对于心电信号，卷积核大小分别为5、7、9。加速度信号卷积块包含8个卷积层和5个池化层，而心电信号卷积块包含10个卷积层和6个池化层，激活函数选择修正线性单元。每个卷积层后添加批量归一化用于缓解内部协变量偏移问题，加速训练过程。最后添加dropout层防止模型陷入过拟合。单个1xk的卷积块具体结构参见下表1，其中k表示卷积核大小，conv表示卷积层，maxpool表示最大池化层，global avgpool表示全局平均池化层。
表1单个1xk的卷积块结构
Figure PCTCN2021088096-appb-000002
Figure PCTCN2021088096-appb-000003
由于各分支网络使用了不同大小卷积核，因此可以提取信号不同尺度的上下文信息，解决了加速度信号、心电信号因随机性和自相似性导致特征提取困难的问题。
优选地，为了提升模型的个体适应特性，将影响个体能量代谢水平的个体基本信息或称人体结构物理特征信息(如性别、年龄、腰围、身高、体重等参数)与不同尺度的加速度特征和心电特征通过一个具有隐藏层神经单元数目为128的全连接层进一步融合，以提升模型对不同人群进行能量代谢预测的泛化能力。由于每个人身体结构特征的不同会导致能量消耗的具体值有所变化，因此将人体结构物理特征信息输入到消耗预测模型，进一步调整模型，能够达到适应不同个体的效果。
综上，在此步骤S120中，提出了同时提取心电和加速度信号多尺度上下文特征的多分支卷积神经网络，能够有效捕捉隐藏在信号中的微小变化。此外，提出了多源信息融合的可穿戴人体能量代谢评估方案，利用反映人体生理状态的心电信号、反映人体运动强度水平的加速度信号和人体结构物理特征信息作为模型输入，从多个维度挖掘影响人体能量消耗的关键特征，能够挖掘出更全面、更细微的影响能量代谢水平的特征。
步骤S130，基于融合特征进行能量代谢信息的回归预测。
在一个实施例中，首先通过有序回归方法对能量代谢水平进行粗粒度的估计。接着在粗粒度估计的基础上，通过线性回归方法进一步对能量代谢预测值进行细粒度的估计，从而显著提高了能量消耗预测模型的准确性。
具体地，有序回归的本质是通过标签离散化将原本的回归任务转换成多个分类任务，因此首先将能量代谢真实值进行离散化，例如采用等间隔离散化方法。若能量代谢值区间[α，β]被均分成K等份时，则离散区间变为 [r 1，r 2，...，r K]，其中r i定义为：
Figure PCTCN2021088096-appb-000004
其中y i为第i个样本的能量代谢真实值，r i为第i个样本的能量代谢真实值的离散化结果，
Figure PCTCN2021088096-appb-000005
表示向下取整函数。
下一步对离散化的能量代谢值进行编码，常用的编码方法包括硬标签法和软标签法，由于硬标签的编码方法可能会导致有价值的信息丢失，而软标签的处理考虑了真实值的连续性，更有利于模型的学习，因此，优选使用软标签的编码方法。r i被编码成一个维度为1xK的软标签向量y i，向量y i中的第j个元素定义为
Figure PCTCN2021088096-appb-000006
其中φ(r i，r j)是距离度量函数，它代表了离散化的能量代谢值r i与离散等级r j之间的距离，距离度量可以使用欧氏距离、曼哈顿距离、切比雪夫距离等。
由图2可以看出，从影响运动能量消耗的特征X到能量代谢最终预测值
Figure PCTCN2021088096-appb-000007
的映射可分解为两个步骤，即有序回归计算得到对应真实软标签向量y的预测向量
Figure PCTCN2021088096-appb-000008
以及线性回归计算得到对应真实值y的预测值
Figure PCTCN2021088096-appb-000009
该过程可以表示为：
Figure PCTCN2021088096-appb-000010
其中W 1、W 2为网络权重参数，b 1、b 2为网络偏置参数。
在一个实施例中，为两个步骤(即有序回归和线性回归)的回归任务定义了两个代价函数。有序回归的代价函数采用KL散度作为损失函数，用于衡量真实软标签与预测运动能量消耗分布之间的差异，控制能量消耗的区间分类准确率，表示为：
Figure PCTCN2021088096-appb-000011
其中N为总样本个数，y ij为第i个样本的能量代谢真实软标签向量的第j个元素；
Figure PCTCN2021088096-appb-000012
为第i个样本的预测向量的第j个元素。
线性回归的代价函数控制最终运动能量消耗的预测，例如采用绝对值损失函数，表示为：
Figure PCTCN2021088096-appb-000013
其中y i为第i个样本的能量代谢真实值，
Figure PCTCN2021088096-appb-000014
为第i个样本的能量代谢预测值。
因此，在训练过程中总代价函数计算为：
J＝λJ rank+J reg  (6)
其中λ是用于平衡两个代价函数在训练期间贡献的超参数。
在此步骤S130中，提出了一种级联回归的能量代谢预测方法，融合心电特征、加速度特征和个人基本信息使用基于软标签的有序回归和普通线性回归两个回归预测，实现对于运动能量消耗的精准计算。
步骤S140，利用经训练的模型实时预测能量代谢信息。
在训练上述用于特征提取的多分支卷积神经网络以及级联回归模型之后，即可实时预测个体的能量代谢信息，例如，采用可穿戴设备采集加速度信号和心电信号，并通过信号去噪、合加速度提取、数据分割等处理为一系列连续的数据，结合个体的人体结构物理特征信息输入到经训练的模型(包括神经网络模型和回归预测模型)，即可实时获得对应的预测能量代谢信息。数据处理过程与步骤S110类似，在此不再赘述。
相应地，本发明还提供一种基于可穿戴传感信息融合的能量代谢评估系统，用于实现上述方法的一个方面或多个方面。例如，该系统包括：数据采集单元，其用于获取反映人体生理状态的心电信号信息、反映人体运动强度水平的加速度信号信息和人体结构物理特征信息；特征提取单元，其用于将所述心电信号信息输入第一卷积神经网络提取心电特征，并将所述加速度信号信息输入第二卷积神经网络提取加速度特征，其中，第一卷积神经网络和第二卷积神经网络经训练获得，且第一卷积神经网络和第二卷积网络均是多分支结构，以提取多尺度特征；预测单元，其用于将所述心电特征、加速度特征和所述人体结构物理特征信息进行融合，并基于融合特征预测对应的能量代谢信息。
为进一步验证本发明的效果，进行了实验。实验招募了20名身体健康的具有运动能力的受试者，其中10名男性，10名女性，年龄在24-43岁之间。每名受试者进行2次实验，总共40次实验数据。每次测试前测量并 记录受试者的身高、体重、腰围、年龄、性别等个体基本信息，实验过程中受试者佩戴运动心率带，腰部绑上Shimmer3IMU加速度传感器，并佩戴好能量代谢测试仪的呼吸面罩在跑台上进行递增负荷运动测试，可穿戴心电、加速度信号，及能量代谢标准值同步采集，能量代谢测试仪采样率为0.2Hz，心率带对心电信号采样率为200Hz，Shimmer3惯性传感器对加速度和陀螺仪信号的采样率为100Hz。
实验采用留一法交叉验证，即从20个受试者数据中选出19个人的数据做训练集，然后用剩下1个人的数据做测试，迭代进行20轮测试。测试结果如表2所示
表2测试结果
Figure PCTCN2021088096-appb-000015
综上所述，本发明采用基于呼吸测量法的人体能量代谢测试仪采集的数据(金标准)作为模型参考真实值，保证了能量代谢预测模型的可靠性；首次采用原始单导联心电信号作为人体能量代谢预测模型的输入信号，心电数据包含了更多的反映能量代谢水平的有效信息，提升了人体能量代谢预测的准确性；使用多种数据增强技术用于增加训练样本，提高了模型泛化能力，消除了加速度计方向变化影响，增加了模型的鲁棒性；设计的多分支卷积神经网络实现了端到端的特征学习，简化了特征提取的流程，该网络拥有多条分支，每条分支引入了多通道、多尺度卷积核，相比手工设计提取的特征，该网络提取的特征更加全面，质量更高；设计了级联回归的能量代谢预测方法，将困难的能量代谢回归预测问题拆解成简单的区间分类和小范围回归两个子问题，有效地降低了人体能量代谢预测的难度，提升了模型的预测准确性。
本发明可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本发明操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++、Python等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执 行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本发明的各个方面。
这里参照根据本发明实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本发明的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本发明的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不 同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本发明的范围由所附权利要求来限定。

Claims (10)

  1. 一种基于可穿戴传感信息融合的能量代谢评估方法，包括以下步骤：
    获取反映人体生理状态的心电信号信息、反映人体运动强度水平的加速度信号信息和人体结构物理特征信息；
    将所述心电信号信息输入第一卷积神经网络提取心电特征，并将所述加速度信号信息输入第二卷积神经网络提取加速度特征，其中，第一卷积神经网络和第二卷积神经网络经训练获得，且第一卷积神经网络和第二卷积网络均是多分支结构，以提取多尺度特征；
    将所述心电特征、加速度特征和所述人体结构物理特征信息进行融合，并基于融合特征预测对应的能量代谢信息。
  2. 根据权利要求1所述的方法，其中，第一神经网络每个分支的卷积核大小不同，第二神经网络每个分支的卷积核大小不同，并且利用全连接层将所述心电特征、加速度特征和所述人体结构物理特征信息进行融合。
  3. 根据权利要求1所述的方法，其中，所述基于融合特征预测对应的能量代谢信息是利用经训练的级联回归模型获得，包括：
    利用基于软标签的有序回归计算得到对应真实软标签向量y的预测向量
    Figure PCTCN2021088096-appb-100001
    表示为：
    Figure PCTCN2021088096-appb-100002
    利用线性回归计算得到对应真实值y的预测值
    Figure PCTCN2021088096-appb-100003
    表示为：
    Figure PCTCN2021088096-appb-100004
    其中W 1、W 2是权重参数，b 1、b 2是偏置参数，X表示融合特征。
  4. 根据权利要求3所述的方法，其中，训练所述级联回归模型的总代价函数表示为：
    J＝λJ rank+J reg
    其中J rank是有序回归的代价函数，J reg是线性回归的代价函数，λ是超参数。
  5. 根据权利要求4所述的方法，其中，代价函数J rank用于衡量真实软标签与预测运动能量消耗分布之间的差异，控制能量消耗的区间分类准 确率，表示为：
    Figure PCTCN2021088096-appb-100005
    其中N是总样本数量，y ij为第i个样本的能量代谢真实软标签向量的第j个元素，
    Figure PCTCN2021088096-appb-100006
    为第i个样本的预测向量的第j个元素。
  6. 根据权利要求4所述的方法，其中，代价函数J reg用于控制最终运动能量消耗的预测，表示为：
    Figure PCTCN2021088096-appb-100007
    其中y i为第i个样本的能量代谢真实值，
    Figure PCTCN2021088096-appb-100008
    为第i个样本的能量代谢预测值，N是总样本数量。
  7. 根据权利要求1所述的方法，其中，用于训练第一神经网络和第二神经网络的训练样本用于表征加速度信号信息、心电信号信号与能量消耗值之间的对应关系，所述加速度信号信息包括加速度计x、y、z轴的加速度、加速度计x、y、z轴的角加速度、合加速度以及合角加速度。
  8. 根据权利要求1所述的方法，其中，用于训练第一神经网络和第二神经网络的训练样本根据以下步骤获得：
    利用可穿戴设备采集加速度信号、心电信号以及对应的能量代谢真实值；
    将加速度信号、心电信号以及能量代谢真实值数据分割为一系列连续的窗口，每个窗口的数据作为一个样本，并计算每个窗口的卡路里数据的平均值作为该样本真实的能量消耗值；
    使用数据增强丰富样本，包括：将加速度信号数据和心电信号数据的幅度乘以高斯分布的随机标量；对加速度信号的三轴数据进行随机交换排列或者对其旋转一个随机角度。
  9. 一种基于可穿戴传感信息融合的能量代谢评估系统，包括：
    数据采集单元：用于获取反映人体生理状态的心电信号信息、反映人体运动强度水平的加速度信号信息和人体结构物理特征信息；
    特征提取单元：用于将所述心电信号信息输入第一卷积神经网络提取心电特征，并将所述加速度信号信息输入第二卷积神经网络提取加速度特 征，其中第一卷积神经网络和第二卷积神经网络经训练获得，且第一卷积神经网络和第二卷积网络均是多分支结构，以提取多尺度特征；
    预测单元：用于将所述心电特征、加速度特征和所述人体结构物理特征信息进行融合，并基于融合特征预测对应的能量代谢信息。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其中，该程序被处理器执行时实现根据权利要求1至8中任一项所述方法的步骤。
PCT/CN2021/088096 2021-04-19 2021-04-19 一种基于可穿戴传感信息融合的能量代谢评估方法和系统 WO2022221987A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/088096 WO2022221987A1 (zh) 2021-04-19 2021-04-19 一种基于可穿戴传感信息融合的能量代谢评估方法和系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/088096 WO2022221987A1 (zh) 2021-04-19 2021-04-19 一种基于可穿戴传感信息融合的能量代谢评估方法和系统

Publications (1)

Publication Number Publication Date
WO2022221987A1 true WO2022221987A1 (zh) 2022-10-27

Family

ID=83723637

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/088096 WO2022221987A1 (zh) 2021-04-19 2021-04-19 一种基于可穿戴传感信息融合的能量代谢评估方法和系统

Country Status (1)

Country Link
WO (1) WO2022221987A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010016689A1 (en) * 2000-02-23 2001-08-23 Ilkka Heikkila Measurement relating to human body
CN102727185A (zh) * 2012-07-18 2012-10-17 重庆邮电大学 一种基于心率和加速度的运动能耗测量仪及测量方法
CN106037738A (zh) * 2016-07-11 2016-10-26 中国人民解放军第三军医大学第三附属医院 可穿戴式体能训练评估系统及装置
CN111160139A (zh) * 2019-12-13 2020-05-15 中国科学院深圳先进技术研究院 心电信号的处理方法、装置及终端设备
CN112263254A (zh) * 2020-06-11 2021-01-26 复旦大学附属华山医院 一种基于表面肌电信号传感器的人体能量消耗预测系统及其预测方法
CN112508110A (zh) * 2020-12-11 2021-03-16 哈尔滨理工大学 一种基于深度学习的心电信号图的分类方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010016689A1 (en) * 2000-02-23 2001-08-23 Ilkka Heikkila Measurement relating to human body
CN102727185A (zh) * 2012-07-18 2012-10-17 重庆邮电大学 一种基于心率和加速度的运动能耗测量仪及测量方法
CN106037738A (zh) * 2016-07-11 2016-10-26 中国人民解放军第三军医大学第三附属医院 可穿戴式体能训练评估系统及装置
CN111160139A (zh) * 2019-12-13 2020-05-15 中国科学院深圳先进技术研究院 心电信号的处理方法、装置及终端设备
CN112263254A (zh) * 2020-06-11 2021-01-26 复旦大学附属华山医院 一种基于表面肌电信号传感器的人体能量消耗预测系统及其预测方法
CN112508110A (zh) * 2020-12-11 2021-03-16 哈尔滨理工大学 一种基于深度学习的心电信号图的分类方法

Similar Documents

Publication Publication Date Title
Staudenmayer et al. An artificial neural network to estimate physical activity energy expenditure and identify physical activity type from an accelerometer
Zhu et al. Using deep learning for energy expenditure estimation with wearable sensors
Juen et al. A natural walking monitor for pulmonary patients using mobile phones
CN110944577B (zh) 一种血氧饱和度的检测方法与系统
CN113171080B (zh) 一种基于可穿戴传感信息融合的能量代谢评估方法和系统
JP5870646B2 (ja) 患者からのムード情報の単純な収集の方法
Troiano Translating accelerometer counts into energy expenditure: advancing the quest
Zhong Internet of things sensors assisted physical activity recognition and health monitoring of college students
Liu et al. The use of a finger-worn accelerometer for monitoring of hand use in ambulatory settings
Long et al. A scoping review on monitoring mental health using smart wearable devices
CN110659677A (zh) 一种基于可移动传感器组合设备的人体跌倒检测方法
Kelly et al. Automatic prediction of health status using smartphone-derived behavior profiles
Wu et al. A novel pedal musculoskeletal response based on differential spatio-temporal LSTM for human activity recognition
Du et al. RETRACTED: Research on the intelligent model of progress in physical education training based on motion sensor
Young et al. Just find it: The Mymo approach to recommend running shoes
Alam et al. Autocognisys: Iot assisted context-aware automatic cognitive health assessment
Koo et al. Contrastive accelerometer–gyroscope embedding model for human activity recognition
Ma et al. A prediction method for transport stress in meat sheep based on GA-BPNN
Haque et al. State-of-the-art of stress prediction from heart rate variability using artificial intelligence
Mallol-Ragolta et al. Outer product-based fusion of smartwatch sensor data for human activity recognition
WO2022221987A1 (zh) 一种基于可穿戴传感信息融合的能量代谢评估方法和系统
Li et al. Predicting Parkinson's Disease with Multimodal Irregularly Collected Longitudinal Smartphone Data
Rachakonda et al. Stress-Lysis: An IoMT-enabled device for automatic stress level detection from physical activities
Petty et al. Evaluating deep learning algorithms for real-time arrhythmia detection
Weippert et al. Tri-axial high-resolution acceleration for oxygen uptake estimation: validation of a multi-sensor device and a novel analysis method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21937238

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21937238

Country of ref document: EP

Kind code of ref document: A1