WO2021258333A1 - 一种步态异常早期识别与风险预警方法和装置 - Google Patents
一种步态异常早期识别与风险预警方法和装置 Download PDFInfo
- Publication number
- WO2021258333A1 WO2021258333A1 PCT/CN2020/098100 CN2020098100W WO2021258333A1 WO 2021258333 A1 WO2021258333 A1 WO 2021258333A1 CN 2020098100 W CN2020098100 W CN 2020098100W WO 2021258333 A1 WO2021258333 A1 WO 2021258333A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- gait
- kinematics
- features
- phase
- recognition
- Prior art date
Links
- 230000005021 gait Effects 0.000 title claims abstract description 203
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000005856 abnormality Effects 0.000 title claims abstract description 24
- 241000282414 Homo sapiens Species 0.000 claims abstract description 41
- 206010017577 Gait disturbance Diseases 0.000 claims abstract description 36
- 230000033001 locomotion Effects 0.000 claims abstract description 22
- 238000004458 analytical method Methods 0.000 claims abstract description 16
- 230000035945 sensitivity Effects 0.000 claims abstract description 16
- 210000001503 joint Anatomy 0.000 claims description 22
- 210000004394 hip joint Anatomy 0.000 claims description 20
- 210000000629 knee joint Anatomy 0.000 claims description 20
- 210000000544 articulatio talocruralis Anatomy 0.000 claims description 19
- 238000012937 correction Methods 0.000 claims description 16
- 230000004927 fusion Effects 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 13
- 230000003044 adaptive effect Effects 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 12
- 230000007246 mechanism Effects 0.000 claims description 12
- 210000002683 foot Anatomy 0.000 claims description 11
- 210000003141 lower extremity Anatomy 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 244000309466 calf Species 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 6
- 238000013506 data mapping Methods 0.000 claims description 6
- 230000002123 temporal effect Effects 0.000 claims description 6
- 210000000689 upper leg Anatomy 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 3
- 230000001953 sensory effect Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 15
- 238000012545 processing Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 8
- 238000005457 optimization Methods 0.000 description 8
- 230000001133 acceleration Effects 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 230000007774 longterm Effects 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 239000000306 component Substances 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 208000024827 Alzheimer disease Diseases 0.000 description 1
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 101000911772 Homo sapiens Hsc70-interacting protein Proteins 0.000 description 1
- 208000007101 Muscle Cramp Diseases 0.000 description 1
- 206010028289 Muscle atrophy Diseases 0.000 description 1
- 208000018737 Parkinson disease Diseases 0.000 description 1
- 206010061363 Skeletal injury Diseases 0.000 description 1
- 208000005392 Spasm Diseases 0.000 description 1
- 208000006011 Stroke Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 210000003423 ankle Anatomy 0.000 description 1
- 206010003246 arthritis Diseases 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000000546 chi-square test Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 208000013409 limited attention Diseases 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000007659 motor function Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000020763 muscle atrophy Effects 0.000 description 1
- 201000000585 muscular atrophy Diseases 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000012567 pattern recognition method Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000035479 physiological effects, processes and functions Effects 0.000 description 1
- 239000002243 precursor Substances 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000001020 rhythmical effect Effects 0.000 description 1
- 238000012502 risk assessment Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
Definitions
- the present invention relates to the technical field of biometric identification, and more specifically, to a method and device for early recognition and risk warning of abnormal gait.
- abnormal gait Compared with normal gait, abnormal gait manifests as lack of phase, chaotic timing, imbalance of proportion, and abnormal parameters such as step frequency, pace, stride length, and joint angle. Timely and accurate diagnosis of abnormal gait is a necessary condition to ensure that patients can receive timely and effective rehabilitation treatment.
- gait abnormalities often cannot be detected in time when they appear at an early stage or when the symptoms are mild.
- early gait abnormalities in the elderly are often mistaken as normal manifestations of physiological aging during clinical examinations, thus delaying recovery The best time for treatment.
- the diagnosis method that relies on the experience of the clinician, due to the lack of analysis of gait variability characteristics on a large time scale, cannot achieve early recognition and early warning of gait abnormalities.
- the gait detection method based on machine vision developed in recent years usually uses a high-speed camera to continuously photograph the walking posture of the human body, and obtain the characteristic parameters of the human gait through computer image processing technology. Although this method can obtain accurate and comprehensive gait parameters, it is still not widely used due to the constraints of equipment costs, deployment sites, and professional operators. At present, wearable health monitoring equipment has been rapidly developed and widely used.
- wearable devices such as inertial sensors, plantar pressure sensors, and surface EMG sensors
- Human body movement posture information provides rich and comprehensive big data information for analyzing the subtle changes of human gait in a multi-dimensional and all-round way from time and space, and for realizing early recognition and early warning of abnormal gait. Therefore, wearable sensor-based gait analysis and early recognition of gait abnormalities are important means to realize early detection, early prevention, early diagnosis and early treatment of diseases related to gait abnormalities.
- the existing gait analysis and gait abnormality detection methods are all based on gait characteristics to classify (for example, patent applications CN201910064053.2 and CN201910813891.5), which can only be used when the gait abnormalities are already obvious.
- patent applications CN201910064053.2 and CN201910813891.5 can only be used when the gait abnormalities are already obvious.
- the purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide a method and device for early recognition and risk warning of gait abnormalities. It is a new technical solution for early recognition and risk warning of gait abnormalities based on multi-source information fusion. Long-term monitoring of the user's gait inertial sensor data, monitoring the variability characteristics of the gait in a large time dimension, so as to realize the early recognition of gait abnormalities.
- a method for early recognition and risk warning of abnormal gait includes the following steps:
- the salient features are selected according to the gait phase sensitivity to perform gait phase recognition, and the gait phase characteristics are obtained;
- the virtual sensor information of the multiple joints is obtained according to the following steps:
- the motion data is collected and fused by sensors worn on different parts of the body, input to the pre-trained neural network model, and mapped to multiple joints reflecting the gait characteristics, and the virtual sensor information of each joint is obtained respectively.
- the multiple joints are ankle joints, knee joints, and hip joints that can significantly reflect gait characteristics.
- the gait phase characteristics are obtained according to the following steps:
- a n relative posterior probability of utilizing Bayesian conditional probability and prior probability P (B i) is calculated gait P (A n / B i) ;
- the gait recognition results after the fusion of the characteristic layers at the ankle joint, the knee joint and the hip joint are calculated, and the gait phase characteristics are obtained.
- the gait kinematics characteristics are obtained according to the following steps:
- the kinematics characteristics of the three loop segments of the human foot, calf, and thigh are respectively calculated as the gait kinematics characteristics.
- the real-time data collected by the sensor is used to generate individualized zero-speed and zero-angular velocity determination thresholds for the current motion mode, and at the same time, the gait phase detection results are merged to realize the individualized threshold change.
- the spatial relationship of the human body structure is used as the space constraint and the timing relationship of the gait phase is used as the time constraint, and the estimated parameters are calibrated in the time and space dimensions respectively.
- the space constraint conditions include the maximum step length, the maximum step width, and the maximum height of the foot from the ground.
- fusing the gait phase characteristics and the gait kinematics characteristics, and selecting the gait salient characteristics according to the sensitivity of gait variability characteristics includes:
- the adaptive incremental learning mechanism is used to realize the feature selection of continuously collected human motion data, and the attention mechanism is introduced to construct the attention correlation between the geometric structure parameters of the human body and the features of real-time inertial sensors to realize the dynamic fusion of multi-modal data.
- an early recognition and risk warning device for abnormal gait includes:
- Sensing data mapping module used to fuse the collected motion data of different parts of the body and map it into virtual sensor information of multiple joints reflecting gait characteristics;
- Temporal feature extraction module based on the virtual sensor information of the multiple joints, select salient features according to the gait phase sensitivity for gait phase recognition, and obtain gait phase features;
- Kinematics feature extraction module used to establish a human structure kinematics model, and extract gait kinematics features based on the virtual sensor information of the multiple joints using the human structure kinematics model;
- Gait saliency feature extraction module used to fuse the gait phase feature and the gait kinematics feature, and select the gait saliency feature according to the sensitivity of gait variability characteristics;
- Gait recognition module used to construct gait variability characteristic analysis models under different time scales, and input the gait salient features to the abnormal gait recognition network to realize automatic recognition and classification of abnormal gait.
- the present invention has the advantage of proposing an environment-adaptive sensor data fusion algorithm to realize accurate detection of the gait phase in a wearable complex and changeable environment; and proposing the inertia under multi-dimensional constraints.
- Guided solution parameter constraint optimization method to achieve accurate extraction of human gait kinematics characteristics; proposes a gait variability analysis method based on attention mechanism and adaptive incremental learning mechanism to achieve gait variability at different time scales Characteristic analysis and early recognition of abnormal gait.
- Fig. 1 is a flowchart of a method for early recognition and risk warning of abnormal gait according to an embodiment of the present invention
- FIG. 2 is a schematic diagram of a method for dynamic fusion of sensor data based on neural network according to an embodiment of the present invention
- Fig. 3 is a schematic diagram of a gait phase recognition process based on Bayesian inference according to an embodiment of the present invention
- Fig. 4 is a schematic diagram of a seven-link rigid body model of a human lower limb according to an embodiment of the present invention
- FIG. 5 is a schematic diagram of an inertial navigation solution and multi-constraint optimization method according to an embodiment of the present invention
- Fig. 6 is a schematic diagram of an early recognition and risk warning model of abnormal gait based on an adaptive incremental learning network according to an embodiment of the present invention.
- the method for early recognition of abnormal gait and early warning of risk includes the following steps:
- step S110 the gait phase is detected and the gait phase characteristics are extracted.
- human body motion information may be collected based on inertial sensors integrated in wearable devices (such as smart phones, watches, bracelets, etc.), for example, sensor signals of multiple parts of the human body.
- wearable devices such as smart phones, watches, bracelets, etc.
- the neural network model (or ANN-based sensor data mapping model)
- the motion data collected by any number of sensors worn on different parts of the body are fused and then mapped to multiple joints that can significantly reflect the gait characteristics.
- select the ankle, knee, and hip joints as The mapped target joint is shown in Figure 2.
- the ankle, knee, and hip joints will be introduced as examples, but it should be understood that, depending on factors such as movement type and data mapping accuracy, you can also choose to map to other joint parts.
- the gait features can be automatically extracted using convolutional neural networks, and the extracted features can be screened using genetic algorithms, and the salient features that are sensitive to the gait phase can be selected for subsequent steps. State and phase recognition.
- the present invention proposes a feature layer data fusion method based on Bayesian inference.
- the posterior probability P(A n /B i ) expressed as:
- a time phase feature containing time sequence information can be obtained.
- the inertial sensor data mapping method based on the ANN network proposed by the embodiment of the present invention solves the influence of the variability of the placement position of the inertial sensor on the accuracy of gait abnormality recognition.
- the ankles, knees, hip joints and other parts that obviously reflect the gait features simplify the subsequent data processing process and improve the accuracy of gait feature extraction.
- Step S120 construct a kinematics model of human body structure, and extract gait kinematics characteristics.
- the embodiment of the present invention implements individualized kinematic modeling by establishing a regression equation of the human body's lower limb structure.
- the lower limbs of the human body are mainly composed of three parts: thigh, calf, and foot, and are connected by hip, knee, and ankle joints to form a kinematic chain with multiple degrees of freedom.
- a binary regression equation of the mass of each ring segment of the foot, calf, and thigh and the position of the center of mass on the height and weight is established, and the geometric parameters of the individual kinematics model are obtained through the body structure parameters input by the individual.
- an individualized multi-link rigid body motion model of the lower limb is constructed based on the structural parameters of the lower limb as a kinematic model of the human body structure.
- a 7-link rigid body inverted pendulum model of the lower limbs is established according to the physiological structure characteristics and movement modes of the human hip, knee, and ankle joints.
- a 1 (A 4 ), A 2 (A 5 ), A 3 (A 6 ) Represents the thigh, calf, and foot, respectively
- H, K (K') and A (A') represent the hip, knee, and ankle joints, respectively.
- the virtual sensor information of the ankle, knee and hip joints are used to calculate the three ring segments of the human body ( That is, the position, speed, and posture angle of the thigh, calf, and foot, as shown in Figure 5.
- the inertial navigation solution process includes: First, use the inertial navigation specific force equation to transform the sensing coordinates into the earth coordinate system.
- the earth coordinate system based on the collected acceleration information, the After the invalid acceleration constituted by the central acceleration and the gravitational acceleration, the speed can be obtained by integrating the acceleration once, and the position can be obtained by integrating again.
- construct the quaternion of the attitude transformation aiming at the possible error of the quaternion, introduce the quaternion correction method based on the gravity vector and the geomagnetic vector, and adjust and oppose it based on the proportional integral (PI) feedback
- the matrix is called to update the posture quaternion.
- the calculation of the speed, step length, step width, stride length, joint angle and other gait space parameters of the gait requires the inertial sensor information to be calculated. Integral or quadratic integration, the accumulation of sensing errors brought about by this process is an important source of error that affects the accuracy of gait space parameter calculation, and it is also a key scientific problem that needs to be addressed.
- the traditional error correction method mainly uses the characteristic that the speed is zero when the foot is in contact with the ground, and uses the speed when the carrier is stationary as the observation to correct other information of the carrier, that is, zero speed correction (ZUPT).
- the zero-speed correction Due to the unobservability of the sky gyro in the zero-speed correction, the zero-speed correction easily introduces heading drift; in addition, the zero-speed correction algorithm is only applicable to the foot position and the zero-speed state, and its application in the time and space domains is relatively limited. .
- the embodiment of the present invention proposes a multi-dimensional, multi-form (equation and inequality) gait kinematics parameter inertial navigation solution based on kinematic constraints, spatial constraints, and timing constraints. Precision optimization method.
- the embodiment of the present invention provides an individualized and adaptive zero-speed determination threshold.
- Speed judgment threshold generation method using real-time sensor data to generate individualized zero speed and zero angular speed judgment thresholds for current motion patterns (different walking speeds, different ground slopes, etc.), and fusion of gait phase detection results to achieve individualization
- the threshold is adjusted adaptively with the phase state of the gait.
- a double-constraint correction of the heading error is carried out by introducing the zero-angular speed correction.
- the space-time constraints for example, the maximum step length, the maximum step width, and the maximum height of the foot from the ground are introduced as the space constraints, and introduced
- the time sequence information of the gait phase is used as the time constraint.
- the constraint conditions are decomposed, and the estimated parameters are calibrated in the time and space dimensions to improve the effectiveness of the constraint conditions.
- step S120 the embodiment of the present invention proposes a method for calculating gait kinematics parameters based on multi-dimensional optimization. Aiming at the influence of sensor error accumulation on the accuracy of kinematics parameter calculation during the process of calculating gait kinematics parameters, it is proposed A multi-dimensional gait parameter optimization method integrating zero speed correction, space correction, and time sequence correction improves the accuracy of gait kinematics parameter calculation.
- Step S130 fusing the gait phase characteristics and gait kinematics characteristics, selecting gait saliency characteristics according to the sensitivity of gait variability characteristics, and constructing analysis models of gait variability characteristics at different time scales to realize abnormal gait Automatic recognition and classification of
- the embodiment of the present invention implements feature selection of continuously collected human motion data through an adaptive incremental learning method.
- the attention mechanism is also introduced to construct the attention association between effective information such as human body geometric structure parameters and real-time inertial sensor features to realize the dynamic fusion of multi-modal data, so as to improve the model’s response to the motion features collected under different sensor configuration conditions.
- Generalization ability and then realize the optimization of the early warning model performance under the constraint of data imbalance.
- the overall process of early recognition of gait abnormalities and risk warning includes:
- Step S131 Obtain multi-modal gait features.
- multi-modal features include temporal features (gait recognition features), gait kinematics features, and/or comprehensive features.
- Step S132 Perform multi-modal gait feature fusion, and select gait salient features according to the sensitivity of gait variability characteristics.
- an incremental learning mechanism is used to implement sample update. Since the inertial sensor information collected continuously for a long time by the wearable is a dynamic continuous signal, the monitoring frequency is high, the time is long, and the amount of data is large. It is required to be able to incrementally learn incrementally to realize the collection of data under different configuration conditions of the wearable sensor. Robustness and adaptability. For this reason, the incremental memory module is designed, and the current period samples and the previous period samples are used to participate in the training model by using the incremental learning method. Through the adaptive threshold, the incremental knowledge is retained, and the module is updated after the training is completed, so as to solve The forgetting problem in incremental learning. Through the incremental learning mechanism, the quantitative analysis of gait variability characteristics under different time scales is studied.
- incremental learning can be described as: whenever new data is added, all the knowledge bases do not need to be rebuilt, but on the basis of the original knowledge base, only the changes caused by the new data are updated.
- the judging mechanism of new knowledge is the core component of incremental learning.
- the salient features of the typical abnormal gait to be identified and the incremental features obtained, learn the target area that needs attention, efficiently allocate the limited attention resources of the wearable device, and obtain more gait details that need attention Information, suppress irrelevant information.
- the robust extraction of gait variability information and the accurate mining of gait abnormal features are realized.
- the embodiment of the present invention further adopts an adaptive model parameter optimization mechanism to improve generalization ability.
- an adaptive parameter optimization module In the wearable complex environment, in order to further adapt the early recognition and early warning model of gait abnormalities to the diversity and time-varying characteristics of wearable sensors in terms of number, wearing position, parameter configuration, etc., by constructing an adaptive parameter optimization module, During model decision-making, the network parameters are adjusted adaptively to reduce the degree of over-fitting of the model and improve the predictive ability of the model. Further, according to the data input mode in the real-time wearable environment, the weight vector of the real neuron and the topology of the network are adaptively and dynamically adjusted to realize the self-organization and incremental learning capabilities of the model, and optimize the expression accuracy of the input data. In addition, by adaptively determining the number of neurons, it is possible to meet certain quantization error constraints and adapt to input patterns that have not been learned before without affecting the previous learning results.
- step S133 a gait variability characteristic analysis model is constructed to realize the risk prediction of abnormal gait.
- the risk prediction of abnormal gait is realized.
- construct analysis models of gait variability characteristics at different time scales, and based on gait variability characteristics establish a variety of typical abnormal gait early recognition and classification networks (or abnormal gait recognition network for short) to realize gait risk Assessment and automatic identification and classification of abnormal gait.
- Early recognition and classification networks can be realized by convolutional neural networks or support vector machines.
- the gait variability characteristic analysis method based on adaptive incremental learning proposed in the present invention aims at the current application requirements for early recognition of gait abnormalities, and uses long-time collected gait inertial sensing data to analyze the large-time dimension of gait Variability characteristics, to achieve early recognition of gait abnormalities.
- the present invention also provides an early recognition and risk warning device for abnormal gait, which is used to implement one or more aspects of the above method.
- the device includes a sensing data mapping module, which is used to fuse and map the collected motion data of different parts of the body into virtual sensing information of multiple joints reflecting gait characteristics; a temporal feature extraction module, which Based on the virtual sensor information of the multiple joints, the salient features are selected according to the gait phase sensitivity for gait phase recognition to obtain gait phase features; a kinematic feature extraction module, which is used to establish a human body The structural kinematics model is used to extract the gait kinematics features based on the virtual sensor information of the multiple joints; the gait saliency feature extraction module is used to combine the gait temporal features with The gait kinematics characteristics are fused, and the gait saliency characteristics are selected according to the sensitivity of the gait variability characteristics; the gait recognition module is used to construct the analysis model of the gait variability characteristics at different time scale
- the abnormal gait recognition method and device is a technical solution for early recognition of gait abnormalities and risk warning based on multi-source information fusion of wearable sensors, which integrates acceleration information worn on the body surface, Gyroscope information and geomagnetic information can accurately extract gait phase characteristics and kinematic characteristics, and further realize early recognition of gait abnormalities and risk warning through wearable big data analysis and pattern recognition methods.
- the present invention proposes an early recognition of gait abnormalities based on adaptive incremental learning. Through long-term monitoring of user gait inertial sensor data, the variability of the gait in a large time dimension is monitored. Features to realize early recognition of abnormal gait.
- the present invention may be a system, a method and/or a computer program product.
- the computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling a processor to implement various aspects of the present invention.
- the computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device.
- the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
- Non-exhaustive list of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon
- RAM random access memory
- ROM read-only memory
- EPROM erasable programmable read-only memory
- flash memory flash memory
- SRAM static random access memory
- CD-ROM compact disk read-only memory
- DVD digital versatile disk
- memory stick floppy disk
- mechanical encoding device such as a printer with instructions stored thereon
- the computer-readable storage medium used here is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or through wires Transmission of electrical signals.
- the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
- the network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
- the network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device .
- the computer program instructions used to perform the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, status setting data, or in one or more programming languages.
- Source code or object code written in any combination, the programming language includes object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as "C" language or similar programming languages.
- Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or entirely on the remote computer or server implement.
- the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect to the user's computer) connect).
- LAN local area network
- WAN wide area network
- an electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be personalized by using the status information of the computer-readable program instructions.
- FPGA field programmable gate array
- PDA programmable logic array
- the computer-readable program instructions are executed to realize various aspects of the present invention.
- These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine that makes these instructions when executed by the processor of the computer or other programmable data processing device , A device that implements the functions/actions specified in one or more blocks in the flowchart and/or block diagram is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make computers, programmable data processing apparatuses, and/or other devices work in a specific manner, so that the computer-readable medium storing the instructions includes An article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.
- each block in the flowchart or block diagram can represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction contains one or more components for realizing the specified logical function.
- Executable instructions can be included in the blocks in the flowchart or block diagram.
- the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved.
- each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation through hardware, implementation through software, and implementation through a combination of software and hardware are all equivalent.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- Dentistry (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Physiology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
一种步态异常早期识别与风险预警方法和装置。其中，方法包括：将所采集的身体不同部位的运动数据进行融合并映射为反映步态特性的多个关节的虚拟传感信息；基于多个关节的虚拟传感信息，根据步态时相敏感度选取显著性特征进行步态时相识别，获得步态时相特征；建立人体结构运动学模型，基于多个关节的虚拟传感信息利用该人体结构运动学模型提取步态运动学特征；将步态时相特征和步态运动学特征进行融合，并根据步态变异性特性敏感度选取步态显著性特征；构建不同时间尺度下的步态变异性特性分析模型，将步态显著性特征作为输入异常步态识别网络，实现步态异常的早期识别与风险预警。
Description
本发明涉及生物特征识别技术领域，更具体地，涉及一种步态异常早期识别与风险预警方法和装置。
行走是人类最基本的运动功能，行走过程的身体姿态和动作(即步态)包含着丰富的运动学、动力学、心理学和生理学信息。正常的步态是人体神经、骨骼、肌肉和感官系统的高度协调活动，具有节律性、周期性和一致性，任一个环节出现异常或失调都将会导致步态异常。同时，步态异常也是多种组织器官发生病变的前兆，许多常见的老年疾病，如帕金森、阿尔茨海默症，骨损伤、关节炎、中风、肌肉萎缩、肌肉痉挛等，在发病的前期都会表现出步态异常。相比于正常步态，异常步态表现为时相缺失、时序混乱、比例失调，且存在步频、步速、步幅和关节角度等参数异常。对异常步态的及时、准确的诊断是保证患者能得到及时有效康复治疗的必要条件。然而，现实生活中步态异常在早期出现或症状轻微时常常无法得到及时检测，尤其是老年人的早期步态异常，在临床检查时常常被误认为是生理性衰老的正常表现，从而延误康复治疗的最佳时间。
在现有技术中，依靠临床医生经验的诊断方法，由于缺少大时间尺度下步态变异性特性的分析，无法做到步态异常的早期识别和预警。而近年发展起来的基于机器视觉的步态检测方法通常是利用高速摄像机连续拍摄人体走路姿态，通过计算机图像处理技术获取人体步态的特征参数。这种方法虽然能够获得准确、全面的步态参数，但由于受设备成本，部署场地、专业操作人员等条件限制，仍然无法普遍使用。目前，可穿戴健康监测设备得到快速发展和普遍使用，可穿戴设备集成的多种类型的传感器，如惯性传感器、足底压力传感器和表面肌电传感器等，可以在任意环境下，长 期、连续采集人体运动姿态信息，为从时间和空间多维度、全方位分析人体步态的细微变化，为实现异常步态的早期识别和预警提供了丰富、全面的大数据信息。因此，基于可穿戴传感器的步态分析及步态异常早期识别技术是实现步态异常相关疾病早发现、早预防、早诊断、早治疗的重要手段。
经统计分析，现有的步态分析和步态异常检测方法都是基于步态特征进行分类(例如，专利申请CN201910064053.2和CN201910813891.5)，只能在步态异常症状已经比较明显的时候才能实现异常步态的识别，而在步态病变发生的初期无法实现早期的识别与检测。
发明内容
本发明的目的是克服上述现有技术的缺陷，提供一种步态异常早期识别与风险预警方法和装置，是基于多源信息融合的步态异常早期识别与风险预警的新技术方案，通过对用户步态惯性传感数据的长期监测，监测步态大时间维度的变异性特性，从而实现对步态异常的早期识别。
根据本发明的第一方面，提供一种步态异常早期识别与风险预警方法。该方法包括以下步骤：
将所采集的身体不同部位的运动数据进行融合并映射为反映步态特性的多个关节的虚拟传感信息；
基于所述多个关节的虚拟传感信息，根据步态时相敏感度选取显著性特征进行步态时相识别，获得步态时相特征；
建立人体结构运动学模型，基于所述多个关节的虚拟传感信息利用该人体结构运动学模型提取步态运动学特征；
将所述步态时相特征和所述步态运动学特征进行融合，并根据步态变异性特性敏感度选取步态显著性特征；
构建不同时间尺度下的步态变异性特性分析模型，并将所述步态显著性特征输入至异常步态识别网络，实现步态异常早期识别与风险预警。
在一个实施例中，根据以下步骤获得所述多个关节的虚拟传感信息：
通过穿戴于身体不同部位的传感器采集运动数据并进行融合，输入至 预训练的神经网络模型，映射到反映步态特性的多个关节，分别获得每个关节的虚拟传感信息。
在一个实施例中，所述多个关节是能够显著反映步态特性的踝关节、膝关节和髋关节。
在一个实施例中，根据以下步骤获得所述步态时相特征：
依据先验知识计算踝关节、膝关节、髋关节处虚拟传感信息的输出特征的先验概率P(B
i)，其中i＝1,2,3，B
i分别表示踝关节、膝关节和髋关节处的虚拟传感信息的输出特征；
计算在观测结果为B
i时，步态时相为A
n的条件概率P(B
i/A
n)，n是步态时相的索引标识；
利用贝叶斯条件概率和先验概率P(B
i)计算步态时相A
n的后验概率P(A
n/B
i)；
基于贝叶斯融合理论，计算踝关节、膝关节和髋关节处特征层融合之后的步态识别结果，获得所述步态时相特征。
在一个实施例中，根据以下步骤获得所述步态运动学特征：
构建下肢七连杆刚体倒立摆模型作为所述人体结构运动学模型；
利用惯性导航算法，基于踝关节、膝关节和髋关节处的虚拟传感信息，分别解算人体足部、小腿、大腿三个环段的运动学特征，作为所述步态运动学特征。
在一个实施例中，在惯性导航解算过程中，利用传感器实时采集的数据生成面向当前运动模式的个体化零速和零角速度判定阈值，同时融合步态时相检测结果，实现个体化阈值随步态时相状态的自适应调整，并通过引入零角速度修正对航向误差进行双重约束修正。
在一个实施例中，在惯性导航解算过程中，将人体结构空间关系作为空间约束条件并将步态时相的时序关系作为时间约束条件，分别在时间和空间维度校准估计参数。
在一个实施例中，所述空间约束条件包括最大步长、最大步宽和足离地的最大高度。
在一个实施例中，将所述步态时相特征和所述步态运动学特征进行融 合，并根据步态变异性特性敏感度选取步态显著性特征包括：
通过自适应增量学习机制实现连续采集的人体运动数据的特征筛选，并引入注意力机制，构建人体几何结构参数与实时惯性传感器特征之间的注意力关联，实现多模态数据的动态融合。
根据本发明的第二方面，提供一种步态异常早期识别与风险预警装置。该装置包括：
传感数据映射模块：用于将所采集的身体不同部位的运动数据进行融合并映射为反映步态特性的多个关节的虚拟传感信息；
时相特征提取模块：用于基于所述多个关节的虚拟传感信息，根据步态时相敏感度选取显著性特征进行步态时相识别，获得步态时相特征；
运动学特征提取模块：用于建立人体结构运动学模型，基于所述多个关节的虚拟传感信息利用该人体结构运动学模型提取步态运动学特征；
步态显著性特征提取模块：用于将所述步态时相特征和所述步态运动学特征进行融合，并根据步态变异性特性敏感度选取步态显著性特征；
步态识别模块：用于构建不同时间尺度下的步态变异性特性分析模型，并将所述步态显著性特征输入至异常步态识别网络，实现异常步态的自动识别与分类。
与现有技术相比，本发明的优点在于，提出了环境自适应的传感器数据融合算法，实现可穿戴复杂、多变环境下步态时相的准确检测；提出了多维度约束条件下的惯导解算参数约束优化方法，实现人体步态运动学特征的准确提取；提出了基于注意力机制和自适应增量学习机制的步态变异性特性分析方法，实现不同时间尺度下步态变异性特性分析和步态异常早期识别。
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其它特征及其优点将会变得清楚。
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。
图1是根据本发明一个实施例的步态异常早期识别与风险预警方法的流程图；
图2是根据本发明一个实施例的基于神经网络的传感数据动态融合方法的示意图；
图3是根据本发明一个实施例的基于贝叶斯推理的步态时相识别过程的示意图；
图4是根据本发明一个实施例的人体下肢七连杆刚体模型的示意图；
图5是根据本发明一个实施例的惯导解算与多约束优化方法的示意图；
图6是根据本发明一个实施例的基于自适应增量学习网络的步态异常早期识别与风险预警模型的示意图。
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
参见图1所示，本发明实施例的步态异常早期识别与风险预警方法包括以下步骤：
步骤S110，检测步态时相并提取步态时相特征。
在本发明实施例中，可基于可穿戴设备(例如智能手机、手表、手环等)集成的惯性传感器采集人体运动信息，例如，采集人体多个部位的传感信号。
具体地，为了对传感器误差进行补偿，针对可穿戴传感在传感器数目、穿戴位置、采样率、采样精度等方面配置的差异性，通过训练的神经网络模型(或称基于ANN的传感器数据映射模型)，将穿戴于身体不同部位的任意数目的传感器采集到的运动数据融合之后再分别映射到能够显著反映步态特性的多个关节部位，例如，选取踝关节、膝关节和髋关节等部位作为映射的目标关节，参见图2所示。在下文描述中，为清楚起见，将以踝关节、膝关节和髋关节为例进行介绍，但应理解的是，根据运动类型和数据映射精度等因素，也可以选择映射到其他的关节部位。
进一步地，基于映射之后的虚拟传感信息，可利用卷积神经网络自动提取步态特征，并利用遗传算法对提取的特征进行筛选，选取步态时相敏感的显著性特征用于后续的步态时相识别。
对于步态时相识别，可采用多种方式，例如，采用现有的基于阈值的步态时相识别方法，而针对现有方法在复杂多样的病理步态时相识别过程中准确率低、适用性差的问题，本发明提出了基于贝叶斯推理的特征层数据融合方法。
参见图3所示，首先，依据先验知识计算踝、膝、髋关节处虚拟传感器输出特征的先验概率P(B
i)，其中i＝1,2,3，B
i分别标示位于踝、膝和髋关节部位的虚拟传感器输出的特征。进一步，依据统计分析，计算在观测结果为B
i时，步态时相为A
n的条件概率P(B
i/A
n)，其中n标示时相，例如，n＝1,2,…,6分别表示支撑相的早期、中期、晚期和摆动相的早期、中期、晚期等；最后利用贝叶斯条件概率公式和虚拟传感器输出特征的先验概率P(B
i)计算步态时相的后验概率P(A
n/B
i)，表示为：
最后，在踝、膝和髋关节传感器识别结果相互独立的假设下，基于贝叶斯融合理论，计算三个部位传感器特征层融合之后的步相识别结果。
在该步骤S110，通过步态时相检测和识别，能够获得包含时序信息的时相特征，或简称为时序特征。本发明实施例所提出的基于ANN网络的惯性传感器数据映射方法，解决了惯性传感器放置位置多变性对步态异常识别准确度的影响，通过将身体不同部位的惯性传感器采集到的数据映射到能够明显反应步态特征的踝、膝、髋关节等部位，简化后续数据处理过程，提高了步态特征提取的准确性。
步骤S120，构建人体结构运动学模型，提取步态运动学特征。
为降低人体下肢形态结构的个体差异性对运动学参数解算准确度的影响，本发明实施例通过建立人体下肢结构回归方程实现个体化的运动学建模。
从生物学的角度来讲，人体下肢主要有大腿、小腿、足三部分组成，并由髋、膝、踝关节连接起来，组成一个具有多个自由度的运动链。在本发明实施例中，建立足部、小腿、大腿各环段质量、质心位置对身高、体重的二元回归方程，进而通过个体输入的体型结构参数获取个体化的运动学模型几何参数。例如，基于下肢结构参数构建个体化的下肢多连杆刚体运动模型作为人体结构运动学模型，运动学模型的建立需在尽可能保证模型准确性的前提下提高模型的可解性，如图4所示，根据人体髋、膝、踝关节的生理结构特点和运动方式，建立下肢七连杆刚体倒立摆模型，其中，A
1(A
4)、A
2(A
5)、A
3(A
6)分别表示大腿、小腿和足部，H、K(K')和A(A')分别表示髋、膝、踝关节。
在下肢多连杆刚体运动学模型的基础上，基于惯性导航算法(简称惯导算法)，利用踝、膝和髋关节三个部位的虚拟传感信息，分别解算人体下肢三个环段(即大腿、小腿和足部)的位置、速度和姿态角等信息，如图5所示。
结合图5所示，惯导解算过程包括：首先，利用惯导比力方程将传感坐标变换到地球坐标系中，在地球坐标系中，基于采集到的加速度信息，除去由对地向心加速度和重力加速度构成的无效加速度后，对加速度积分一次可得速度，再积分一次得位置。在传感器坐标变换的基础上，构建姿态变换的四元数，针对四元数可能存在的误差，引入基于重力向量和地磁 矢量的四元数修正方法，并基于比例积分(PI)反馈调节和反对称矩阵进行姿态四元数更新。
在惯导解算过程中，由于惯导解算的积分原理，计算步态的速度、步长、步宽、跨步长、关节角度等步态空间参数的解算需要对惯性传感信息进行积分或二次积分，此过程带来的传感误差积累是影响步态空间参数解算精度的重要误差来源，也是需要重点解决的一个关键科学问题。传统的误差修正方法主要利用脚与地面接触时速度为零的特征，以载体静止时的速度作为观测量对载体的其它信息进行修正，即零速修正(ZUPT)。由于天向陀螺在零速修正中的不可观测性，零速修正容易引入航向漂移；此外，零速修正算法只适用于足部位置和零速状态，其在时间和空间领域的适用范围比较有限。
针对目前步态参数解算方法中存在的问题，本发明实施例提出了基于运动学约束、空间约束、时序约束的多维度、多形式(等式和不等式)步态运动学参数惯导解算精度优化方法。
具体地，参见图5所示，首先，针对基于单一、固定的零速判定阈值的ZUPT算法对不同用户、不同运动模式的适应性较差的问题，本发明实施例提供个体化自适应的零速判定阈值生成方法，利用传感器实时采集的数据生成面向当前运动模式(不同走速、不同地面坡度等)的个体化零速和零角速度判定阈值，同时融合步态时相检测结果，实现个体化阈值随步态时相状态的自适应调整。进一步，针对零速修正适用范围较窄的问题，在零速修正的基础上，通过引入零角速度修正对航向误差进行双重约束修正。最后，创新性地引入人体结构空间关系和步态时相的时序关系作为时空约束条件，例如分别引入最大步长、最大步宽、和足离地最大高度等空间参数作为空间约束条件，并引入步态时相的时序信息作为时间约束条件，根据约束模型的时空特征，将约束条件进行分解，分别在时间和空间维度校准估计参数，提高约束条件的有效性。
在步骤S120中，本发明实施例提出了基于多维优化的步态运动学参数解算方法，针对步态运动学参数解算过程中传感器误差积累对运动学参数解算准确度的影响，提出了融合零速修正、空间修正、时序修正的多维 步态参数优化方法，提升步态运动学参数解算的准确度。
步骤S130，融合步态时相特征和步态运动学特征，根据步态变异性特性敏感度选取步态显著性特征，并构建不同时间尺度下的步态变异性特性分析模型，实现异常步态的自动识别与分类。
针对可穿戴惯性传感器数据具有高纬度、多模态、高噪声的特点，以及步态异常样本稀缺的挑战，本发明实施例通过自适应增量学习方法实现连续采集的人体运动数据的特征筛选，并引入注意力机制，构建人体几何结构参数等有效信息与实时惯性传感器特征之间的注意力关联，实现多模态数据的动态融合，以提高模型对不同传感器配置条件下采集到的运动特征的泛化能力，进而实现在数据不平衡约束条件下预警模型性能的优化。
参见图6所示，步态异常早期识别与风险预警的整体过程包括：
步骤S131，获取多模态步态特征。
例如，多模态特征包括时序特征(步态识相特征)、步态运动学特征和/或综合特征等。
步骤S132，进行多模态步态特征融合，并根据步态变异性特性敏感度选取步态显著性特征。
例如，构建深度神经网络将数步态时序特征、步态运动学特征和/或综合特征融合到同一特征空间，并基于卡方检验、皮尔逊相关系数分析等消除多模态特征间的冗余特征，选择对步态变异性特性敏感的步态显著性特征。
在一个优选实施例中，采用增量学习机制实现样本更新。由于可穿戴长时间连续采集的惯性传感信息属于动态连续信号，监测频率高、时间长、数据量大，要求能够渐进式的进行增量学习，实现对可穿戴传感器不同配置条件下采集数据的鲁棒性和自适应性。为此设计增量记忆模块，利用增量学习方法将当前时段样本与前一时段样本一同参与训练模型，通过自适应的门限，将增量知识保留，并在训练完成后更新该模块，从而解决增量学习中的遗忘问题。通过增量学习机制，研究不同时间尺度下步态变异性特性的量化分析。增量学习思想可以描述为：每当新增数据时，并不需要重建所有的知识库，而是在原有知识库的基础上，仅对由于新增数据所引 起的变化进行更新。经研究发现，增量学习方法更符合人的思维原理。增量学习框架有很多类型，各框架最核心的内容是处理新数据与已存储知识相似性评价方法。因为该方法决定觉察新知识并增加知识库的方式，它影响着知识的增长。新知识的判定机制是增量学习的核心部件。
优选地，针对要识别的典型异常步态的显著性特征和得到的增量特征，学习需要关注的目标区域，高效分配可穿戴设备有限的注意力资源，并获取更多需要关注的步态细节信息，抑制不相关信息。通过构建注意力关联机制，实现步态变异性信息的鲁棒提取与步态异常特征的准确挖掘。
此外，本发明实施例还进一步采用自适应模型参数优化机制来提高泛化能力。在可穿戴复杂环境下，为了进一步使步态异常早期识别与预警模型适应可穿戴传感器在数目、穿戴位置、参数配置等方面的多样性和时变性特性，通过构建自适应的参数优化模块，在模型决策时自适应的调节网络参数，降低模型的过拟合程度，提高模型的预测能力。进一步，根据实时可穿戴环境下的数据输入模式，自适应动态调整实神经元的权值向量和网络的拓扑结构，实现模型具有自组织和增量学习能力，优化对输入数据的表达精度。此外，通过自适应地确定神经元的数量，可以实现在满足一定的量化误差约束，并在不影响前期学习结果条件下适应之前没有学习过的输入模式。
步骤S133，构建步态变异性特性分析模型，实现步态异常风险预测。
在此步骤中，实现步态异常风险预测。例如，构建不同时间尺度下步态变异性特性分析模型，并基于步态变异性特性，建立多种典型异常步态的早期识别和分类网络(或简称异常步态识别网络)，实现步态风险评估和异常步态的自动识别与分类。早期识别和分类网络可采用卷积神经网络或支持向量机等实现。
本发明提出的基于自适应增量学习的步态变异性特性分析方法，针对当前对步态异常早期识别的应用需求，利用长时间采集的步态惯性传感数据，分析步态大时间维度的变异性特性，实现步态异常的早期识别。
相应地，本发明还提供一种步态异常早期识别与风险预警装置，用于实现上述方法的一个方面或多个方面。例如，该装置包括传感数据映射模 块，其用于将所采集的身体不同部位的运动数据进行融合并映射为反映步态特性的多个关节的虚拟传感信息；时相特征提取模块，其用于基于所述多个关节的虚拟传感信息，根据步态时相敏感度选取显著性特征进行步态时相识别，获得步态时相特征；运动学特征提取模块，其用于建立人体结构运动学模型，基于所述多个关节的虚拟传感信息利用该人体结构运动学模型提取步态运动学特征；步态显著性特征提取模块，其用于将所述步态时相特征和所述步态运动学特征进行融合，并根据步态变异性特性敏感度选取步态显著性特征；步态识别模块，其用于构建不同时间尺度下的步态变异性特性分析模型，以所述步态显著性特征作为输入，实现异常步态的自动识别与分类。该装置中的各模块可采用处理器或专用逻辑器件实现。
综上所述，本发明提供的异常步态识别方法和装置，是基于可穿戴传感多源信息融合的步态异常早期识别与风险预警的技术方案，通过融合佩戴于身体表面的加速度信息、陀螺仪信息、及地磁信息实现步态时相特征与运动学特征准确提取，并进一步通过可穿戴大数据分析与模式识别方法实现步态异常早期识别与风险预警。此外，本发明针对现有技术的不足，提出了一种基于自适应增量学习的步态异常早期识别，通过对用户步态惯性传感数据的长期监测，监测步态大时间维度的变异性特性，实现对步态异常的早期识别。
本发明可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸 起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本发明操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本发明的各个方面。
这里参照根据本发明实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本发明的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本发明的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原 理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本发明的范围由所附权利要求来限定。
Claims (11)
- 一种步态异常早期识别与风险预警方法，包括以下步骤：将所采集的身体不同部位的运动数据进行融合并映射为反映步态特性的多个关节的虚拟传感信息；基于所述多个关节的虚拟传感信息，根据步态时相敏感度选取显著性特征进行步态时相识别，获得步态时相特征；建立人体结构运动学模型，基于所述多个关节的虚拟传感信息利用该人体结构运动学模型提取步态运动学特征；将所述步态时相特征和所述步态运动学特征进行融合，并根据步态变异性特性敏感度选取步态显著性特征；构建不同时间尺度下的步态变异性特性分析模型，并将所述步态显著性特征输入至异常步态识别网络，实现步态异常早期识别和风险预警。
- 根据权利要求1所述的方法，其中，根据以下步骤获得所述多个关节的虚拟传感信息：通过穿戴于身体不同部位的传感器采集运动数据并进行融合，输入至预训练的神经网络模型，映射到反映步态特性的多个关节，分别获得每个关节的虚拟传感信息。
- 根据权利要求1所述的方法，其中，所述多个关节是能够显著反映步态特性的踝关节、膝关节和髋关节。
- 根据权利要求3所述的方法，其中，根据以下步骤获得所述步态时相特征：依据先验知识计算踝关节、膝关节、髋关节处虚拟传感信息的输出特征的先验概率P(B i)，其中i＝1,2,3，B i分别表示踝关节、膝关节和髋关节处的虚拟传感信息的输出特征；计算在观测结果为B i时，步态时相为A n的条件概率P(B i/A n)，n是步态时相的索引标识；利用贝叶斯条件概率和先验概率P(B i)计算步态时相A n的后验概率P(A n/B i)；基于贝叶斯融合理论，计算踝关节、膝关节和髋关节处特征层融合之 后的步态识别结果，获得所述步态时相特征。
- 根据权利要求3所述的方法，其中，根据以下步骤获得所述步态运动学特征：构建下肢七连杆刚体倒立摆模型作为所述人体结构运动学模型；利用惯性导航算法，基于踝关节、膝关节和髋关节处的虚拟传感信息，分别解算人体足部、小腿、大腿三个环段的运动学特征，作为所述步态运动学特征。
- 根据权利要求5所述的方法，其中，在惯性导航解算过程中，利用传感器实时采集的数据生成面向当前运动模式的个体化零速和零角速度判定阈值，同时融合步态时相检测结果，实现个体化阈值随步态时相状态的自适应调整，并通过引入零角速度修正对航向误差进行双重约束修正。
- 根据权利要求5所述的方法，其中，在惯性导航解算过程中，将人体结构空间关系作为空间约束条件并将步态时相的时序关系作为时间约束条件，分别在时间和空间维度校准估计参数。
- 根据权利要求7所述的方法，其中，所述空间约束条件包括最大步长、最大步宽和足离地的最大高度。
- 根据权利要求1所述的方法，其中，将所述步态时相特征和所述步态运动学特征进行融合，并根据步态变异性特性敏感度选取步态显著性特征包括：通过自适应增量学习机制实现连续采集的人体运动数据的特征筛选，并引入注意力机制，构建人体几何结构参数与实时惯性传感器特征之间的注意力关联，实现多模态数据的动态融合。
- 一种步态异常早期识别与风险预警装置，包括：传感数据映射模块：用于将所采集的身体不同部位的运动数据进行融合并映射为反映步态特性的多个关节的虚拟传感信息；时相特征提取模块：用于基于所述多个关节的虚拟传感信息，根据步态时相敏感度选取显著性特征进行步态时相识别，获得步态时相特征；运动学特征提取模块：用于建立人体结构运动学模型，基于所述多个关节的虚拟传感信息利用该人体结构运动学模型提取步态运动学特征；步态显著性特征提取模块：用于将所述步态时相特征和所述步态运动学特征进行融合，并根据步态变异性特性敏感度选取步态显著性特征；步态识别模块：用于构建不同时间尺度下的步态变异性特性分析模型，并将所述步态显著性特征输入至异常步态识别网络，实现异常步态的自动识别与分类。
- 一种计算机可读存储介质，其上存储有计算机程序，其中，该程序被处理器执行时实现根据权利要求1所述的方法的步骤。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/098100 WO2021258333A1 (zh) | 2020-06-24 | 2020-06-24 | 一种步态异常早期识别与风险预警方法和装置 |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/098100 WO2021258333A1 (zh) | 2020-06-24 | 2020-06-24 | 一种步态异常早期识别与风险预警方法和装置 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2021258333A1 true WO2021258333A1 (zh) | 2021-12-30 |
Family
ID=79282498
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2020/098100 WO2021258333A1 (zh) | 2020-06-24 | 2020-06-24 | 一种步态异常早期识别与风险预警方法和装置 |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2021258333A1 (zh) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115281662A (zh) * | 2022-09-26 | 2022-11-04 | 北京科技大学 | 一种慢性踝关节不稳智能辅助诊断系统 |
| CN116187748A (zh) * | 2022-12-16 | 2023-05-30 | 清华大学 | 风险域确定方法、装置、计算机设备、介质和程序产品 |
| CN116458852A (zh) * | 2023-06-16 | 2023-07-21 | 山东协和学院 | 基于云平台及下肢康复机器人的康复训练系统及方法 |
| CN116740811A (zh) * | 2023-06-15 | 2023-09-12 | 东芯泰合(深圳)科技有限公司 | 一种智能手表的步态识别方法、介质及设备 |
| WO2023169465A1 (zh) * | 2022-03-11 | 2023-09-14 | 中国科学院深圳先进技术研究院 | 基于多源信息融合的人体运动监测方法、装置 |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101564328A (zh) * | 2009-05-07 | 2009-10-28 | 杭州电子科技大学 | 基于支持向量数据描述的膝上假肢多运动模式识别方法 |
| US20130324890A1 (en) * | 2010-12-01 | 2013-12-05 | Movea | Method and system for determining the values of parameters representative of a movement of at least two limbs of an entity represented in the form of an articulated line |
| US20140358040A1 (en) * | 2013-06-04 | 2014-12-04 | Electronics And Telecommunications Research Institute | Gait monitoring apparatus and method |
| CN104613963A (zh) * | 2015-01-23 | 2015-05-13 | 南京师范大学 | 基于人体运动学模型的行人导航系统与导航定位方法 |
| CN104757976A (zh) * | 2015-04-16 | 2015-07-08 | 大连理工大学 | 一种基于多传感器融合的人体步态分析方法和系统 |
| CN106017461A (zh) * | 2016-05-19 | 2016-10-12 | 北京理工大学 | 基于人体/环境约束的行人导航系统三维空间定位方法 |
| CN107480651A (zh) * | 2017-08-25 | 2017-12-15 | 清华大学深圳研究生院 | 异常步态检测方法及异常步态检测系统 |
| CN108968918A (zh) * | 2018-06-28 | 2018-12-11 | 北京航空航天大学 | 早期帕金森的可穿戴辅助筛查设备 |
| CN111700620A (zh) * | 2020-06-24 | 2020-09-25 | 中国科学院深圳先进技术研究院 | 一种步态异常早期识别与风险预警方法和装置 |
-
2020
- 2020-06-24 WO PCT/CN2020/098100 patent/WO2021258333A1/zh active Application Filing
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101564328A (zh) * | 2009-05-07 | 2009-10-28 | 杭州电子科技大学 | 基于支持向量数据描述的膝上假肢多运动模式识别方法 |
| US20130324890A1 (en) * | 2010-12-01 | 2013-12-05 | Movea | Method and system for determining the values of parameters representative of a movement of at least two limbs of an entity represented in the form of an articulated line |
| US20140358040A1 (en) * | 2013-06-04 | 2014-12-04 | Electronics And Telecommunications Research Institute | Gait monitoring apparatus and method |
| CN104613963A (zh) * | 2015-01-23 | 2015-05-13 | 南京师范大学 | 基于人体运动学模型的行人导航系统与导航定位方法 |
| CN104757976A (zh) * | 2015-04-16 | 2015-07-08 | 大连理工大学 | 一种基于多传感器融合的人体步态分析方法和系统 |
| CN106017461A (zh) * | 2016-05-19 | 2016-10-12 | 北京理工大学 | 基于人体/环境约束的行人导航系统三维空间定位方法 |
| CN107480651A (zh) * | 2017-08-25 | 2017-12-15 | 清华大学深圳研究生院 | 异常步态检测方法及异常步态检测系统 |
| CN108968918A (zh) * | 2018-06-28 | 2018-12-11 | 北京航空航天大学 | 早期帕金森的可穿戴辅助筛查设备 |
| CN111700620A (zh) * | 2020-06-24 | 2020-09-25 | 中国科学院深圳先进技术研究院 | 一种步态异常早期识别与风险预警方法和装置 |
Non-Patent Citations (1)
| Title |
|---|
| URIEL MARTINEZ-HERNANDEZ ET AL.: "Simultaneous Bayesian Recognition of Locomotion and Gait Phases With Wearable Sensors", IEEE SENSORS JOURNAL, vol. 18, no. 3, 11 December 2017 (2017-12-11), pages 1282 - 1290, XP011675614, ISSN: 1558-1748, DOI: 10.1109/JSEN.2017.2782181 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023169465A1 (zh) * | 2022-03-11 | 2023-09-14 | 中国科学院深圳先进技术研究院 | 基于多源信息融合的人体运动监测方法、装置 |
| CN115281662A (zh) * | 2022-09-26 | 2022-11-04 | 北京科技大学 | 一种慢性踝关节不稳智能辅助诊断系统 |
| CN116187748A (zh) * | 2022-12-16 | 2023-05-30 | 清华大学 | 风险域确定方法、装置、计算机设备、介质和程序产品 |
| CN116187748B (zh) * | 2022-12-16 | 2023-12-29 | 清华大学 | 风险域确定方法、装置、计算机设备、介质和程序产品 |
| CN116740811A (zh) * | 2023-06-15 | 2023-09-12 | 东芯泰合(深圳)科技有限公司 | 一种智能手表的步态识别方法、介质及设备 |
| CN116740811B (zh) * | 2023-06-15 | 2024-05-10 | 东芯泰合(深圳)科技有限公司 | 一种智能手表的步态识别方法、介质及设备 |
| CN116458852A (zh) * | 2023-06-16 | 2023-07-21 | 山东协和学院 | 基于云平台及下肢康复机器人的康复训练系统及方法 |
| CN116458852B (zh) * | 2023-06-16 | 2023-09-01 | 山东协和学院 | 基于云平台及下肢康复机器人的康复训练系统及方法 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021258333A1 (zh) | 一种步态异常早期识别与风险预警方法和装置 | |
| Dorschky et al. | CNN-based estimation of sagittal plane walking and running biomechanics from measured and simulated inertial sensor data | |
| Khokhlova et al. | Normal and pathological gait classification LSTM model | |
| CN111700620B (zh) | 一种步态异常早期识别与风险预警方法和装置 | |
| Sethi et al. | A comprehensive survey on gait analysis: History, parameters, approaches, pose estimation, and future work | |
| Chen | Human motion analysis with wearable inertial sensors | |
| Zhang et al. | Lower-limb joint torque prediction using LSTM neural networks and transfer learning | |
| Molinaro et al. | Subject-independent, biological hip moment estimation during multimodal overground ambulation using deep learning | |
| Achanta et al. | Wearable sensor based acoustic gait analysis using phase transition-based optimization algorithm on IoT | |
| US20220409098A1 (en) | A wearable device for determining motion and/or a physiological state of a wearer | |
| Khodabandelou et al. | A fuzzy convolutional attention-based GRU network for human activity recognition | |
| Chen et al. | IMU-based estimation of lower limb motion trajectory with graph convolution network | |
| Choi et al. | Unsupervised gait phase estimation with domain-adversarial neural network and adaptive window | |
| Rani et al. | Human gait recognition: A systematic review | |
| Zhen et al. | Hybrid deep-learning framework based on Gaussian fusion of multiple spatiotemporal networks for walking gait phase recognition | |
| Hafer et al. | Challenges and advances in the use of wearable sensors for lower extremity biomechanics | |
| Wang et al. | Arbitrary spatial trajectory reconstruction based on a single inertial sensor | |
| Hollinger et al. | The influence of gait phase on predicting lower-limb joint angles | |
| US20230329587A1 (en) | System And Method For Assessing Neuro Muscular Disorder By Generating Biomarkers From The Analysis Of Gait | |
| Napieralski et al. | Classification of subjects with balance disorders using 1D-CNN and inertial sensors | |
| Gaud et al. | Human gait analysis and activity recognition: A review | |
| Fang et al. | A wearable comprehensive data sampling system for gait analysis | |
| Low et al. | Lower extremity kinematics walking speed classification using long short-term memory neural frameworks | |
| García-de-Villa et al. | Inertial sensors for human motion analysis: A comprehensive review | |
| İnanç et al. | Recognition of daily and sports activities |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20941817 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 20941817 Country of ref document: EP Kind code of ref document: A1 |
|
| 32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205N DATED 03.07.2023) |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 20941817 Country of ref document: EP Kind code of ref document: A1 |