CN110755070B - Multi-sensor fusion-based lower limb movement pose rapid prediction system and method - Google Patents

Multi-sensor fusion-based lower limb movement pose rapid prediction system and method Download PDF

Info

Publication number
CN110755070B
CN110755070B CN201910804483.3A CN201910804483A CN110755070B CN 110755070 B CN110755070 B CN 110755070B CN 201910804483 A CN201910804483 A CN 201910804483A CN 110755070 B CN110755070 B CN 110755070B
Authority
CN
China
Prior art keywords
ankle
hip
joint
angle
knee
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910804483.3A
Other languages
Chinese (zh)
Other versions
CN110755070A (en
Inventor
陶云飞
黄玉平
郑继贵
陈靓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Research Institute of Precise Mechatronic Controls
Original Assignee
Beijing Research Institute of Precise Mechatronic Controls
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Research Institute of Precise Mechatronic Controls filed Critical Beijing Research Institute of Precise Mechatronic Controls
Priority to CN201910804483.3A priority Critical patent/CN110755070B/en
Publication of CN110755070A publication Critical patent/CN110755070A/en
Application granted granted Critical
Publication of CN110755070B publication Critical patent/CN110755070B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/389—Electromyography [EMG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1071—Measuring physical dimensions, e.g. size of the entire body or parts thereof measuring angles, e.g. using goniometers
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1116—Determining posture transitions
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques

Abstract

The system for quickly predicting the motion pose of the lower limbs based on multi-sensor fusion comprises a surface electromyography sensor, an inertial sensor, a pose response module, a pose resolving module and an external auxiliary device. The external assisting device is worn on the waist and lower limbs of the wearer; the method comprises the following steps that a surface electromyography sensor acquires electromyography signals of the surfaces of muscles of the lower limb of a wearer in real time; the inertial sensor is fixed on the external auxiliary device; the pose response module determines the motion state corresponding to the electromyographic signals of the surfaces of the muscles of the current lower limb according to the time sequence of the electromyographic signals of the surfaces of the muscles collected; and the pose calculation module performs short-time dynamic weighted data fusion on each joint motion angle acquired by the inertial sensor and the calculated and estimated joint motion angle, and obtains the lower limb motion attitude of the wearer and the position coordinates of the joints in real time. The invention also provides a method for quickly predicting the motion pose of the lower limbs. The invention can quickly predict the motion pose of the lower limbs of the human body and simultaneously improve the estimation precision of the continuous motion amount.

Description

Multi-sensor fusion-based lower limb movement pose rapid prediction system and method
Technical Field
The invention relates to a lower limb movement pose rapid prediction system and method based on multi-sensor fusion, and belongs to the field of exoskeleton robots and man-machine cooperative control.
Background
At present, in the aspects of rehabilitation training and power-assisted walking aid, an exoskeleton robot is increasingly favored by patients as an important device capable of replacing manual assistance. However, the existing exoskeleton robot has poor human-computer synergy effect, mainly because the human motion information prediction mode is mostly carried out by adopting a physical sensor mode, and the physical sensor can acquire the human motion information only under the condition that the body of a wearer has a certain motion amplitude, so that large time delay exists, and the requirement of quick prediction cannot be realized; meanwhile, a single sensor has certain accuracy in predicting a discrete motion, and continuous motion estimation has great defects.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the system and the method for quickly predicting the lower limb movement pose based on multi-sensor fusion can quickly predict the lower limb movement pose of a human body and improve the estimation precision of continuous movement amount.
The technical solution of the invention is as follows:
the lower limb movement pose rapid prediction system based on multi-sensor fusion comprises a surface electromyography sensor, an inertial sensor, a pose response module, a pose resolving module and an external auxiliary device;
the external auxiliary device is worn on the waist and the lower limbs of a wearer and can move along with the left and right lower limbs of the human body;
the surface electromyographic sensors are adhered to the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle of a wearer, and are used for acquiring electromyographic signals of the surfaces of the muscles of the lower limb of the wearer in real time and transmitting the signals to the pose response module;
the inertial sensor is fixed on the external auxiliary device and used for acquiring the hip joint movement angle, the knee joint movement angle and the ankle joint movement angle of the lower limb of the wearer in real time;
the pose response module is fixed on an external auxiliary device, carries out filtering processing on the electromyographic signals of the surface of the lower limb of the wearer, then matches the time sequence of the electromyographic signals of the surface of each muscle collected with the time sequence of the muscle activity of the lower limb of the human body stored in a time sequence database of the muscle activity of the lower limb of the human body in advance, and finds the motion state corresponding to the electromyographic signals of the surface of each muscle of the current lower limb;
the pose calculation module is fixed on an external auxiliary device, and is used for calculating signals acquired by the surface electromyography, estimating the hip joint motion angle, the knee joint motion angle and the ankle joint motion angle of the lower limb of a wearer, carrying out short-time dynamic weighted data fusion on the hip joint motion angle, the knee joint motion angle and the ankle joint motion angle acquired by the inertial sensor and the calculated lower limb hip joint motion angle, the knee joint motion angle and the ankle joint motion angle of the wearer, obtaining the lower limb motion posture of the wearer in real time, and calculating the position coordinates of the hip joint, the knee joint and the ankle joint through the size of the external auxiliary device.
The external auxiliary device comprises a waist coaming, a thigh outer side plate, a thigh fixing bandage, a shank outer side plate, a shank fixing bandage, a pedal plate and a pedal plate bandage, wherein the waist coaming is worn on the waist of a wearer, the thigh outer side plate is worn on the outer side of the thigh of the wearer through the thigh fixing bandage, the shank outer side plate is worn on the outer side of the shank of the wearer through the shank fixing bandage, and the pedal plate is worn under the foot of the wearer through the pedal plate bandage; universal joints are adopted to connect the waist coaming with the thigh outer side plate, the thigh outer side plate with the shank outer side plate and the shank outer side plate with the pedal.
The inertial sensors comprise a waist inertial sensor, a thigh inertial sensor, a shank inertial sensor and a foot inertial sensor;
the waist inertial sensor is fixed on the outer side of the front end of the waist coaming, the thigh inertial sensor is fixed on the outer surface of the thigh outer side plate, the shank inertial sensor is fixed on the outer surface of the shank outer side plate, and the foot inertial sensor is fixed on the outer surface of the pedal.
The acquisition mode of the human body lower limb muscle activity time sequence database is as follows:
s1, a large number of testers wear external auxiliary devices to perform corresponding movement according to use requirements, and surface electromyographic signal training samples of a large number of rectus femoris muscles, medial femoral muscles, lateral femoral muscles, biceps femoris muscles, semitendinosus muscles, tibialis anterior muscles, gastrocnemius muscles and soleus muscles are obtained;
s2, performing endpoint detection on the myoelectric signal training samples on the surfaces of the muscles by adopting a cepstrum method, and extracting effective activity section data;
s3, extracting characteristic parameters of effective activity segment data of the muscle surface electromyogram signal training samples by adopting a characteristic extraction method, wherein the characteristic parameters comprise an absolute value mean value, a root mean square, an average power frequency and a median frequency;
and S4, acquiring 8 muscle activity time sequences corresponding to different motions according to the characteristic parameters to form a motion characteristic matching list, wherein the motion characteristic matching list forms a human body lower limb muscle activity time sequence database.
The method of forming the motion feature matching list in step S4 is as follows:
for a certain movement, carrying out binary coding conversion on four characteristic parameters in a muscle surface electromyographic signal training sample according to increasing and decreasing trends, and sequentially arranging according to the absolute value mean value, the root mean square, the average power frequency and the median frequency to form coding strips;
the effective activity segment data of the surface electromyogram signal training samples of 8 muscles form 8 coding strips, the coding strips are combined and stored into a data matrix, different data matrices are stored for different movements, and finally a movement characteristic matching list for different movements is formed.
The method for carrying out the binary code conversion on four characteristic parameters in a muscle surface electromyogram signal training sample according to the increasing and decreasing trend comprises the following steps:
if the value of a certain characteristic parameter is increased, the value is 01, if the value of the characteristic parameter is decreased, the value is 10, and if the value of the characteristic parameter is kept unchanged, the value is 00.
Let the angle of flexion and extension of hip joint be alphaHip jointThe abduction and adduction angle of the hip joint is betaHip jointThe internal rotation and external rotation angles of the hip joint are gammaHip jointRecording the hip joint motion angle acquired by the inertial sensor as (alpha)Hip 2，βHip 2，γHip 2) And the pose calculation module calculates the estimated hip joint of the lower limb of the wearerThe angle of motion is (alpha)Hip 1，βHip 1，γHip 1) The pose resolving module will (alpha)Hip 2，βHip 2，γHip 2) And (alpha)Hip 1，βHip 1，γHip 1) The method for short-time dynamic weighting data fusion comprises the following steps:
during the time t before the current movement moment, there are n hip joint movement angles (alpha) acquired by the inertial sensorsHip 2，βHip 2，γHip 2) N posture resolving modules resolve the estimated movement angle (alpha) of the hip joint of the lower limb of the wearerHip 1，βHip 1，γHip 1) Then, within time t, the reference value of the hip joint movement angle
Figure BDA0002183237270000041
Satisfies the following conditions:
Figure BDA0002183237270000042
wherein i is 1,2, 3.
Average value of measured deviation of hip joint movement angle
Figure BDA0002183237270000043
Satisfies the following conditions:
Figure BDA0002183237270000044
the mean square error of the measured deviation of the hip joint movement angle:
Figure BDA0002183237270000051
wherein j is 1,2, σ1 hip jMeasuring the mean square deviation, sigma, of the deviation for hip flexion and extension motion angles2 hip jMeasuring the mean square error, sigma, of the deviation for the abduction and adduction angles of the hip joint3 hip jThe mean square error of the measured deviation of the hip external rotation and internal rotation angles (c)σ1 hip 1，σ2 hip 1，σ3 hip 1) Calculating and estimating the mean square error of the hip joint motion angle measurement deviation for a pose calculation module, (sigma)1 hip 2，σ2 hip 2，σ3 hip 2) Measuring the mean square error of the deviation for the inertial sensor hip joint movement angle;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure BDA0002183237270000052
wherein k is 1,2,3, and w is 11 hip jA short-time dynamic weighting fusion coefficient for measuring the flexion and extension movement angles of the hip joint, when k is 2, w2 hip jA short-time dynamic weighting fusion coefficient for measuring abduction and adduction movement angles of the hip joint, when k is 3, w3 hip jShort-time dynamic weighting fusion coefficients for measuring the hip joint external rotation and internal rotation motion angles;
hip joint motion angle after data fusion:
Figure BDA0002183237270000053
let the flexion and extension angle of knee joint be alphaKneeRecording the knee joint motion angle acquired by the inertial sensor as alphaKnee 2And the pose calculation module calculates the estimated knee joint motion angle of the lower limb of the wearer to be alphaKnee 1The pose resolving module will compare alphaKnee 2And alphaKnee 1The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the knee joint movement angle acquired by n inertial sensors is alphaKnee 2The n posture calculation module calculates the estimated knee joint movement angle of the lower limb of the wearer to be alphaKnee 1Then, within the time t, the reference value of the knee joint movement angle
Figure BDA0002183237270000061
Satisfies the following conditions:
Figure BDA0002183237270000062
average value of measured deviation of knee joint movement angle
Figure BDA0002183237270000063
Satisfies the following conditions:
Figure BDA0002183237270000064
the mean square error of the measured deviation of the knee joint movement angle is as follows:
Figure BDA0002183237270000065
wherein j is 1,2, σKnee jMeasuring the mean square error, sigma, of the deviation for knee joint movement anglesKnee 2For measuring the mean square deviation, sigma, of the deviation of the knee joint movement angle of the inertial sensorKnee 1Calculating the mean square error of the measurement deviation of the knee joint motion angle for a pose calculating module;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure BDA0002183237270000066
wherein j is 1,2, wKnee jShort-time dynamic weighting fusion coefficients for knee joint movement angle measurement;
knee joint motion angle after data fusion:
αknee＝wKnee 1αKnee 1+wKnee 2αKnee 2。
Let the angle of plantar flexion and dorsiflexion of the ankle joint be alphaAnkle jointThe angle of the ankle joint varus and valgus is betaAnkle jointThe internal rotation and external rotation angle of the ankle joint is gammaAnkle jointRecording the motion angle of the ankle joint acquired by the inertial sensor as (alpha)Ankle 2，βAnkle 2，γAnkle 2) The pose calculation module calculates the estimated motion angle (alpha) of the lower limb ankle joint of the wearerAnkle 1，βAnkle 1，γAnkle 1) The pose resolving module will (alpha)Ankle 2，βAnkle 2，γAnkle 2) And (alpha)Ankle 1，βAnkle 1，γAnkle 1) The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the ankle joint movement angle acquired by the n inertial sensors is (alpha)Ankle 2，βAnkle 2，γAnkle 2) And the n posture calculation module calculates the estimated motion angle (alpha) of the lower limb ankle joint of the wearerAnkle 1，βAnkle 1，γAnkle 1) Then, within the time t, the reference value of the ankle joint movement angle
Figure BDA0002183237270000071
Satisfies the following conditions:
Figure BDA0002183237270000072
average value of measured deviation of ankle joint movement angle
Figure BDA0002183237270000073
Satisfies the following conditions:
Figure BDA0002183237270000074
the mean square error of the measured deviation of the ankle joint movement angle is:
Figure BDA0002183237270000081
wherein j is 1,2, σ1 ankle jMeasuring the mean square deviation, sigma, of the deviation for the plantar flexion and dorsiflexion motion angles of the ankle joint2 ankle jMeasuring the mean square error, sigma, of the deviation for the ankle inversion and eversion motion angles3 ankle jIs an internal rotation of the ankle joint,Mean square error of the measured deviation of the angle of the outward rotation (σ)1 ankle 1，σ2 ankle 1，σ3 ankle 1) Calculating the mean square error (sigma) of the measurement deviation of the ankle joint motion angle for a pose calculation module1 ankle 2，σ2 ankle 2，σ3 ankle 2) Measuring the mean square error of deviation for the motion angle of the ankle joint of the inertial sensor;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure BDA0002183237270000082
wherein k is 1,2,3, and w is 11 ankle jA short-time dynamic weighting fusion coefficient is measured for the plantar flexion and dorsiflexion motion angles of the ankle joint, when k is 2, w2 ankle jThe short-time dynamic weighting fusion coefficient for measuring the ankle joint inversion and eversion motion angle is that when k is 3, w3 ankle jShort-time dynamic weighting fusion coefficients for ankle internal rotation and external rotation movement angle measurement;
ankle joint motion angle after data fusion:
Figure BDA0002183237270000083
the method for quickly predicting the motion pose of the lower limbs based on multi-sensor fusion comprises the following steps:
(1) the wearer pastes surface electromyography sensors on the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle;
(2) the external auxiliary device is worn by a wearer, wherein the waistline plate is worn at the waist of the wearer, the thigh outer side plate is worn at the outer side of the thigh of the wearer through a thigh fixing bandage, the calf outer side plate is worn at the outer side of the calf of the wearer through a calf fixing bandage, and the pedal plate is worn under the foot of the wearer through a pedal bandage; universal joints are adopted to connect the waist coaming and the thigh outer side plate, the thigh outer side plate and the shank outer side plate, and the shank outer side plate and the pedal plate;
(3) fixing an inertial sensor on the outer side of the front end of the waist coaming, the outer thigh side plate, the outer shank side plate and the outer surface of the pedal;
(4) the lower limbs of a wearer do any movement, and the surface myoelectric sensor and the inertial sensor acquire information;
(5) the surface electromyographic sensor transmits the surface electromyographic signals to the pose response module, and the pose response module matches the acquired time sequence of the electromyographic signals of the surfaces of the muscles with a database of the activity time sequence of the muscles of the lower limbs of the human body to find the motion state at the moment;
(6) the surface electromyographic sensor and the inertial sensor transmit the acquired signals to the pose resolving module, and the pose resolving module estimates the motion angle (alpha) of the hip joint of the lower limb of the wearer through the surface electromyographic signalsHip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint motion angle (alpha)Ankle 1，βAnkle 1，γAnkle 1) The hip joint motion angle (alpha) acquired by the inertial sensor is processed by a short-time dynamic weighting fusion methodHip 2，βHip 2，γHip 2) Knee joint movement angle alphaKnee 2Ankle joint motion angle (alpha)Ankle 2，βAnkle 2，γAnkle 2) Angle of motion with hip joint (alpha)Hip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint motion angle (alpha)Ankle 1，βAnkle 1，γAnkle 1) Data fusion is carried out, and the hip joint motion angle (alpha) of the human body lower limb motion posture is obtained in real timeHip joint，βHip joint，γHip joint) Knee joint movement angle alphaKneeAnkle joint movement angle (alpha)Ankle joint，βAnkle joint，γAnkle joint) The position coordinates (x) of hip, knee and ankle joints are respectively calculated by the installation sizes of the waist coaming, the thigh outer side plate, the shank outer side plate and the foot pedal of the external auxiliary deviceHip joint,yHip joint,zHip joint)、(xKnee,yKnee,zKnee)、(xAnkle joint,yAnkle joint,zAnkle joint)。
Compared with the prior art, the invention has the following beneficial effects:
(1) according to the invention, the surface electromyography sensor is adhered to the surface of the lower limb muscle, so that the electromyography signal of the muscle can be collected in real time, and the quick and real-time judgment on the motion state of the lower limb of a wearer is realized by combining a pre-established database of the activity time sequence of the muscle of the lower limb of the human body.
(2) According to the invention, each joint angle obtained by the estimation of the surface electromyography sensor and each joint angle obtained by the inertia sensor are subjected to data fusion by a short-time dynamic weighting fusion method, so that the accurate estimation of the continuous motion amount is realized.
(3) According to the invention, the surface electromyographic sensor is pasted on the surface of the lower limb muscle, the electromyographic signal of the muscle can be collected in real time, the pose calculation module predicts the motion angle and position coordinate of each joint through a short-time dynamic weighting data fusion method, so that the real-time prediction of the motion pose of the lower limb of the human body is realized, and 7 degrees of freedom of the lower limb of the human body and corresponding joint action combinations can be predicted.
Drawings
FIG. 1 is a flow chart of a method for rapidly predicting a motion pose of a lower limb of a human body based on multi-sensor fusion according to the invention;
FIG. 2 is a schematic diagram of a system for rapidly predicting a motion pose of a lower limb of a human body based on multi-sensor fusion according to the present invention;
FIG. 3 is a schematic view of the external attachment of the present invention (only the reference numerals on one side are shown, and the same are provided on the opposite side);
1-waist coaming; 2-a waist inertial sensor; 3-thigh lateral plate; 4-thigh inertial sensors; 5-thigh fixing bandage; 6-outer shank plate; 7-a calf portion inertial sensor; 8-shank fixing band; 9-foot board binding band; 10-a foot pedal; 11-foot inertial sensor.
Detailed Description
The invention provides a lower limb movement pose rapid prediction system based on multi-sensor fusion, which comprises a surface electromyography sensor, an inertial sensor, a pose response module, a pose resolving module and an external auxiliary device.
FIG. 2 is a schematic diagram of a human body lower limb movement pose rapid prediction system based on multi-sensor fusion.
Specifically, the external assisting device is worn on the waist and lower limbs of the wearer and can move along with the left and right lower limbs of the human body. As shown in fig. 3, the external auxiliary device includes a waist panel 1, a thigh outer panel 3, a thigh fixing band 5, a shank outer panel 6, a shank fixing band 8, a foot board 10 and a foot board band 9, the waist panel 1 is worn on the waist of the wearer, the thigh outer panel 3 is worn on the outer side of the thigh of the wearer through the thigh fixing band 5, the shank outer panel 6 is worn on the outer side of the shank of the wearer through the shank fixing band 8, and the foot board 10 is worn under the foot of the wearer through the foot board band 9; universal joints are adopted to connect the waist boarding 1 and the thigh outer side plate 3, the thigh outer side plate 3 and the shank outer side plate 6, and the shank outer side plate 6 and the pedal 10.
The inertial sensor is used for acquiring the angle signal of the lower limb of the wearer in real time. Specifically, the inertial sensors include a waist inertial sensor 2, a thigh inertial sensor 4, a calf inertial sensor 7, and a foot inertial sensor 11. The waist inertial sensor 2 is fixed on the outer side of the front end of the waist boarding 1, the thigh inertial sensor 4 is fixed on the outer surface of the thigh outer side plate 3, the shank inertial sensor 7 is fixed on the outer surface of the shank outer side plate 6, and the foot inertial sensor 11 is fixed on the outer surface of the foot board 10.
The surface electromyographic sensors are adhered to the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle of a wearer, and are used for acquiring electromyographic signals of the surfaces of the muscles of the lower limb of the wearer in real time and transmitting the signals to the pose response module;
the pose response module is fixed on an external auxiliary device, carries out filtering processing on the electromyographic signals of the lower limb surface of the wearer, then matches the time sequence of the collected electromyographic signals of the muscle surface with the time sequence of the activity of the muscles of the lower limb of the human body, which is stored in a database in advance, and finds the motion state corresponding to the electromyographic signals of the lower limb surface at present.
The acquisition mode of the human body lower limb muscle activity time sequence database is as follows:
a large number of testers wear external auxiliary devices to perform corresponding exercises according to use requirements to obtain a large number of training samples, end point detection is performed on the training samples to extract effective activity section data, a characteristic extraction method is adopted to extract series characteristic parameters, 8 muscle activity time sequences corresponding to different exercises are obtained through arrangement and combination of different characteristic parameters and increasing and decreasing trends of the different characteristic parameters, then a special coding mode is adopted to form an exercise characteristic matching list, the exercise characteristic matching list comprises starting point and end point information of the 8 muscles in the exercise process, and the exercise characteristic matching list forms a human body lower limb muscle activity time sequence database.
For example, a tester performs a certain exercise to obtain surface electromyogram signal training samples of rectus femoris, vastus medialis, vastus lateralis, biceps femoris, semitendinosus, tibialis anterior, gastrocnemius and soleus, performs endpoint detection on the surface electromyogram signal training samples of each muscle by using a cepstrum method to extract effective activity segment data, extracts characteristic parameters of the effective activity segment data of the surface electromyogram signal training samples of each muscle by using a time domain characteristic extraction method and a frequency domain characteristic extraction method, wherein the characteristic parameters comprise an absolute value mean, a root mean square, a mean power frequency and a median frequency, performs binary coding conversion on the characteristic parameters according to the increasing and decreasing trend of the characteristic parameters to form coding strips, the characteristic parameter value is 01 if the characteristic parameter value is increased, the characteristic parameter value is 10 if the characteristic parameter value is decreased, the characteristic parameter value is 00 if the characteristic parameter value is maintained, the effective activity segment data of the surface electromyogram signal training samples of each muscle has 4 characteristic parameters, the coding strip arranges the increasing trend of each characteristic parameter in turn according to the sequence of the absolute value mean, the root mean square, the average power frequency and the median frequency, such as 01001000, the coding strip represents a certain movement, the absolute value mean of a certain muscle is increased, the root mean square is unchanged, the average power frequency is reduced, and the median frequency is unchanged.
For a certain movement, all characteristic parameters of the effective activity segment data of the surface electromyogram signal training samples of 8 muscles are obtained simultaneously, the increasing and decreasing trends of the characteristic parameters are subjected to binary coding conversion to form 8 coding strips, the coding strips are combined and stored into a data matrix, different data matrices are stored for different movements, and finally a movement characteristic matching list for different movements is formed.
Pose resolving module fixed on external auxiliary deviceIn the method, the motion angle (alpha) of the hip joint of the lower limb of the wearer is calculated and estimated by the signal acquired by the surface electromyography through a neural network model based on particle swarm optimizationHip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint movement angle (alpha)Ankle 1，βAnkle 1，γAnkle 1) Angle of hip joint motion (alpha) acquired by inertial sensorHip 2，βHip 2，γHip 2) Knee joint movement angle alphaKnee 2Ankle joint motion angle (alpha)Ankle 2，βAnkle 2，γAnkle 2) Angle of motion with hip joint (alpha)Hip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint motion angle (alpha)Ankle 1，βAnkle 1，γAnkle 1) Dynamic weighted data fusion is carried out, the lower limb movement posture of the wearer is obtained in real time, and the position coordinates (x) of hip, knee and ankle joints are respectively calculated through the mounting sizes of a middle waist coaming, a thigh outer side plate, a shank outer side plate and a pedal of an external auxiliary deviceHip joint,yHip joint,zHip joint)、(xKnee,yKnee,zKnee)、(xAnkle joint,yAnkle joint,zAnkle joint)。
Wherein, the motion poses of the lower limbs of the human body are the positions and the postures of the hip, the knee and the ankle, and the postures of the hip, the knee and the ankle comprise the bending angles and the extending angles of the hip joint, namely alphaHip jointThe angle of abduction and adduction of the hip joint is betaHip jointThe angle of internal rotation of the hip joint and external rotation of the hip joint is gammaHip jointThe angle of knee flexion and knee extension is αKneeThe angle of plantar flexion of the ankle joint and dorsiflexion of the ankle joint is alphaAnkle jointThe angle of ankle inversion and ankle eversion is betaAnkle jointThe angle of the internal rotation of the ankle joint and the external rotation of the ankle joint is gammaAnkle joint。
The pose resolving module will (alpha)Hip 2，βHip 2，γHip 2) And (alpha)Hip 1，βHip 1，γHip 1) The method for short-time dynamic weighting data fusion comprises the following steps:
during a time t before the current movement moment, there are n hip joint movement angles (alpha) acquired by the inertial sensorsHip 2，βHip 2，γHip 2) N posture resolving modules resolve the estimated movement angle (alpha) of the hip joint of the lower limb of the wearerHip 1，βHip 1，γHip 1) Then, within time t, the reference value of the hip joint movement angle
Figure BDA0002183237270000131
Satisfies the following conditions:
Figure BDA0002183237270000132
wherein i is 1,2, 3.
Average value of measured deviation of hip joint movement angle
Figure BDA0002183237270000133
Satisfies the following conditions:
Figure BDA0002183237270000141
the mean square error of the measured deviation of the hip joint movement angle:
Figure BDA0002183237270000142
wherein j is 1,2, σ1 hip jMeasuring the mean square deviation, sigma, of the deviation for hip flexion and extension motion angles2 hip jMeasuring the mean square error, sigma, of the deviation for the abduction and adduction angles of the hip joint3 hip jThe mean square error of the measured deviation of the external rotation and internal rotation angles of the hip joint is (sigma)1 hip 1，σ2 hip 1，σ3 hip 1) Calculating and estimating the mean square error of the hip joint motion angle measurement deviation for a pose calculation module, (sigma)1 hip 2，σ2 hip 2，σ3 hip 2) Measuring the mean square error of the deviation for the inertial sensor hip joint motion angle;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure BDA0002183237270000143
wherein k is 1,2,3, and w is 11 hip jA short-time dynamic weighting fusion coefficient for measuring the flexion and extension movement angles of the hip joint, when k is 2, w2 hip jA short-time dynamic weighting fusion coefficient for measuring abduction and adduction movement angles of the hip joint, when k is 3, w3 hip jShort-time dynamic weighting fusion coefficients for measuring the hip joint external rotation and internal rotation motion angles;
hip joint motion angle after data fusion:
Figure BDA0002183237270000151
the pose resolving module compares alphaKnee 2And alphaKnee 1The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the knee joint movement angle acquired by n inertial sensors is alphaKnee 2The n posture calculation module calculates the estimated knee joint movement angle of the lower limb of the wearer to be alphaKnee 1Then, within the time t, the reference value of the knee joint movement angle
Figure BDA0002183237270000152
Satisfies the following conditions:
Figure BDA0002183237270000153
average value of measured deviation of knee joint movement angle
Figure BDA0002183237270000154
Satisfies the following conditions:
Figure BDA0002183237270000155
the mean square error of the measured deviation of the knee joint movement angle is as follows:
Figure BDA0002183237270000156
wherein j is 1,2, σKnee jMeasuring the mean square error, sigma, of the deviation for knee joint movement anglesKnee 2For measuring the mean square deviation, sigma, of the deviation of the knee joint movement angle of the inertial sensorKnee 1Calculating the mean square error of the measurement deviation of the knee joint motion angle for a pose calculating module;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure BDA0002183237270000157
wherein j is 1,2, wKnee jShort-time dynamic weighting fusion coefficients for knee joint movement angle measurement;
knee joint motion angle after data fusion:
αknee＝wKnee 1αKnee 1+wKnee 2αKnee 2。
The pose resolving module will (alpha)Ankle 2，βAnkle 2，γAnkle 2) And (alpha)Ankle 1，βAnkle 1，γAnkle 1) The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the ankle joint movement angle acquired by the n inertial sensors is (alpha)Ankle 2，βAnkle 2，γAnkle 2) The n posture calculation module calculates the estimated movement angle (alpha) of the lower limb ankle joint of the wearerAnkle 1，βAnkle 1，γAnkle 1) Then, within the time t, the reference value of the ankle joint movement angle
Figure BDA0002183237270000161
Satisfies the following conditions:
Figure BDA0002183237270000162
average value of measured deviation of ankle joint movement angle
Figure BDA0002183237270000163
Satisfies the following conditions:
Figure BDA0002183237270000164
the mean square error of the measured deviation of the ankle joint movement angle is:
Figure BDA0002183237270000171
wherein j is 1,2, σ1 ankle jMeasuring the mean square deviation, sigma, of the deviation for the plantar flexion and dorsiflexion motion angles of the ankle joint2 ankle jMeasuring the mean square error, sigma, of the deviation for the ankle inversion and eversion motion angles3 ankle jThe mean square error of the deviation is measured for the internal rotation and external rotation motion angles of the ankle joint (sigma)1 ankle 1，σ2 ankle 1，σ3 ankle 1) Calculating the mean square error (sigma) of the measurement deviation of the ankle joint motion angle for a pose calculation module1 ankle 2，σ2 ankle 2，σ3 ankle 2) Measuring the mean square deviation of the deviation for the motion angle of the ankle joint of the inertial sensor;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure BDA0002183237270000172
wherein k is 1,2,3, and w is 11 anklej is a short-time dynamic weighting fusion coefficient of the measurement of the motion angles of plantar flexion and dorsiflexion of the ankle joint, and when k is 2, w2 anklej is a short-time dynamic weighting fusion coefficient for measuring the ankle joint inversion and eversion motion angles, and when k is 3, w3 anklej is the ankle jointShort-time dynamic weighting fusion coefficients of the internal rotation and external rotation motion angle measurement;
ankle joint motion angle after data fusion:
Figure BDA0002183237270000173
the pose calculation module calculates signals acquired by the surface electromyography through a joint angle estimation model to estimate the hip joint motion angle (alpha) of the lower limb of the wearerHip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint motion angle (alpha)Ankle 1，βAnkle 1，γAnkle 1). The joint angle estimation model is realized by adopting a neural network model based on particle swarm optimization.
Inputting characteristic parameters [ X ] of effective activity segment data of myoelectric signal training samples on the surface of each muscle based on a neural network model optimized by particle swarm1,X2,X3,...,X32]The output is the angle of the hip joint movement (alpha) of the lower limb of the wearerHip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint movement angle (alpha)Ankle 1，βAnkle 1，γAnkle 1) Adopting a Sigmoid function as an action function of a hidden layer and adopting a linear function as an action function of an output layer;
the mean square error generated by the neural network on the training set is used as an objective function, and the following fitness function is constructed to calculate the individual fitness value:
Figure BDA0002183237270000181
in the formula: y isln′Computing an output for the neural network; t isln′Is a target output; n is the number of training samples; l is the number of output neurons;
in a D-dimensional search space, in each iteration process, the speed and the position of the particles are updated through the individual extremum and the global extremum, and the optimal fitness value is found, so that the neural network parameters are optimized, and the updating formula is as follows:
Figure BDA0002183237270000182
wherein w is the inertial weight; d ═ 1,2, …, D; 1,2, …, m>32; k is the current iteration number; x is the number ofidIs the position of the particle; v. ofidIs the velocity of the particle; c. C1And c2Is an acceleration factor; r is1And r2Is distributed in [0, 1 ]]A random number in between;
Figure BDA0002183237270000183
is an individual extremum of the particle;
Figure BDA0002183237270000184
is the global extreme of the population of particles.
Meanwhile, the invention provides a method for quickly predicting the motion pose of the lower limbs of the human body by using the system for quickly predicting the motion pose of the lower limbs of the human body based on multi-sensor fusion, which comprises the following steps as shown in figure 1:
the method comprises the following steps: the wearer removes hair and cutin from the central position of the lower limb muscle belly, coats alcohol and the like, and sequentially sticks the surface electromyography sensor according to the use requirement;
step two: the external auxiliary device is worn by a wearer, wherein the waistline plate is worn on the waist of the wearer, the thigh outer side plate is worn on the outer side of the thigh of the wearer through a thigh fixing bandage, the shank outer side plate is worn on the outer side of the shank of the wearer through a shank fixing bandage, and the pedal plate is worn under the foot of the wearer through a pedal bandage; universal joints are adopted to connect the waist coaming and the thigh outer side plate, the thigh outer side plate and the shank outer side plate, and the shank outer side plate and the pedal plate; fixing an inertial sensor on the outer side of the front end of the waist coaming, the outer thigh side plate, the outer shank side plate and the outer surface of the pedal;
step three: when the lower limbs of a wearer do any movement, all the surface electromyographic sensors and the inertial sensors are arranged to acquire information, respectively acquire surface electromyographic signals of the rectus femoris muscle, the medial femoral muscle, the lateral femoral muscle, the biceps femoris muscle, the semitendinosus muscle, the tibialis anterior muscle, the gastrocnemius muscle and the soleus muscle of the left and right lower limbs, and acquire angle signals of thighs, calves and feet of the left and right lower limbs;
step four: the surface electromyography sensor transmits 8 measured paths of surface electromyography signals to the pose response module, the pose response module matches the activity time sequence of the 8 paths of surface electromyography signals with a database of activity time sequences of muscles of lower limbs of a human body to find the motion pose at the moment, and the activity time sequences of the group of surface electromyography signals are filled into the database as new samples;
step five: the surface electromyographic sensor and the inertial sensor transmit the acquired signals to the pose resolving module, and the pose resolving module estimates the motion angle (alpha) of the hip joint of the lower limb of the wearer through the surface electromyographic signalsHip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint motion angle (alpha)Ankle 1，βAnkle 1，γAnkle 1) The hip joint motion angle (alpha) acquired by the inertial sensor is processed by a short-time dynamic weighting fusion methodHip 2，βHip 2，γHip 2) Knee joint movement angle alphaKnee 2Ankle joint motion angle (alpha)Ankle 2，βAnkle 2，γAnkle 2) Angle of movement with hip joint (alpha)Hip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint motion angle (alpha)Ankle 1，βAnkle 1，γAnkle 1) Data fusion is carried out, and the hip joint motion angle (alpha) of the human body lower limb motion posture is obtained in real timeHip joint，βHip joint，γHip joint) Knee joint movement angle alphaKneeAnkle joint motion angle (alpha)Ankle joint，βAnkle joint，γAnkle joint) The position coordinates (x) of hip, knee and ankle joints are respectively calculated by the installation sizes of the waist coaming, the thigh outer side plate, the shank outer side plate and the foot pedal of the external auxiliary deviceHip joint,yHip joint,zHip joint)、(xKnee,yKnee,zKnee)、(xAnkle joint,yAnkle joint,zAnkle joint) And the quick prediction of the motion pose of the lower limbs of the human body is realized.
According to the invention, the surface electromyography sensor is adhered to the surface of the lower limb muscle, so that the electromyography signal of the muscle can be collected in real time, and the quick and real-time judgment on the motion state of the lower limb of a wearer is realized by combining a pre-established database of the activity time sequence of the muscle of the lower limb of the human body. On the basis, the pose calculation module predicts the motion angle and position coordinates of each joint through a short-time dynamic weighting data fusion method, and realizes the real-time prediction of the motion pose of the lower limbs of the human body and the accurate estimation of the continuous motion amount.
Those skilled in the art will appreciate that those matters not described in detail in the present specification are well known in the art.

Claims (7)

1. The lower limb movement pose rapid prediction system based on multi-sensor fusion is characterized in that: the system comprises a surface electromyography sensor, an inertial sensor, a pose response module, a pose resolving module and an external auxiliary device;
the external auxiliary device is worn on the waist and the lower limbs of a wearer and can move along with the left and right lower limbs of the human body;
the surface electromyographic sensors are adhered to the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle of a wearer, and are used for acquiring electromyographic signals of the surfaces of the muscles of the lower limb of the wearer in real time and transmitting the signals to the pose response module;
the inertial sensor is fixed on the external auxiliary device and used for acquiring the hip joint movement angle, the knee joint movement angle and the ankle joint movement angle of the lower limb of the wearer in real time;
the pose response module is fixed on an external auxiliary device, carries out filtering processing on the electromyographic signals of the surface of the lower limb of the wearer, then matches the time sequence of the electromyographic signals of the surface of each muscle collected with the time sequence of the muscle activity of the lower limb of the human body stored in a time sequence database of the muscle activity of the lower limb of the human body in advance, and finds the motion state corresponding to the electromyographic signals of the surface of each muscle of the current lower limb;
the pose resolving module is fixed on an external auxiliary device, resolves signals acquired by the surface electromyography, estimates the hip joint motion angle, the knee joint motion angle and the ankle joint motion angle of the lower limb of a wearer, performs short-time dynamic weighting data fusion on the hip joint motion angle, the knee joint motion angle and the ankle joint motion angle acquired by the inertial sensor and the lower limb hip joint motion angle, the knee joint motion angle and the ankle joint motion angle of the wearer, obtains the lower limb motion posture of the wearer in real time, and resolves the position coordinates of the hip joint, the knee joint and the ankle joint through the size of the external auxiliary device;
let the angle of flexion and extension of hip joint be alphaHip jointThe abduction and adduction angle of the hip joint is betaHip jointThe angle of internal rotation and external rotation of the hip joint is gammaHip jointRecording the hip joint motion angle acquired by the inertial sensor as (alpha)Hip 2，βHip 2，γHip 2) The pose calculation module calculates the estimated hip joint movement angle (alpha) of the lower limb of the wearerHip 1，βHip 1，γHip 1) The pose resolving module will (alpha)Hip 2，βHip 2，γHip 2) And (alpha)Hip 1，βHip 1，γHip 1) The method for short-time dynamic weighting data fusion comprises the following steps:
during the time t before the current movement moment, there are n hip joint movement angles (alpha) acquired by the inertial sensorsHip 2，βHip 2，γHip 2) N posture resolving modules resolve the estimated movement angle (alpha) of the hip joint of the lower limb of the wearerHip 1，βHip 1，γHip 1) Then, within time t, the reference value of the hip joint movement angle
Figure FDA0003589837130000021
Satisfies the following conditions:
Figure FDA0003589837130000022
wherein i is 1,2, 3.
Average value of measured deviation of hip joint movement angle
Figure FDA0003589837130000023
Satisfies the following conditions:
Figure FDA0003589837130000024
the mean square error of the measured deviation of the hip joint movement angle:
Figure FDA0003589837130000031
wherein j is 1,2, σ1 hip jMeasuring the mean square deviation, sigma, of the deviation for hip flexion and extension motion angles2 hip jMeasuring the mean square error, sigma, of the deviation for the abduction and adduction angles of the hip joint3 hip jThe mean square error of the measured deviation of the external rotation and internal rotation angles of the hip joint is (sigma)1 hip 1，σ2 hip 1，σ3 hip 1) Calculating and estimating the mean square error of the hip joint motion angle measurement deviation for a pose calculation module, (sigma)1 hip 2，σ2 hip 2，σ3 hip 2) Measuring the mean square error of the deviation for the inertial sensor hip joint movement angle;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure FDA0003589837130000032
wherein k is 1,2,3, and w is 11 hip jA short-time dynamic weighting fusion coefficient for measuring the flexion and extension movement angles of the hip joint, when k is 2, w2 hip jA short-time dynamic weighting fusion coefficient for measuring abduction and adduction movement angles of the hip joint, when k is 3, w3 hip jShort-time dynamic weighting fusion coefficients for measuring the hip joint external rotation and internal rotation motion angles;
hip joint motion angle after data fusion:
Figure FDA0003589837130000033
let the flexion and extension angle of knee joint be alphaKneeRecording the knee joint motion angle acquired by the inertial sensor as alphaKnee 2And the pose calculation module calculates the estimated knee joint motion angle of the lower limb of the wearer to be alphaKnee 1The pose resolving module will compare alphaKnee 2And alphaKnee 1The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the knee joint movement angle acquired by n inertial sensors is alphaKnee 2The n posture calculation module calculates the estimated knee joint movement angle of the lower limb of the wearer to be alphaKnee 1Then, within the time t, the reference value of the knee joint movement angle
Figure FDA0003589837130000041
Satisfies the following conditions:
Figure FDA0003589837130000042
average value of measured deviation of knee joint movement angle
Figure FDA0003589837130000043
Satisfies the following conditions:
Figure FDA0003589837130000044
the mean square error of the measured deviation of the knee joint movement angle is as follows:
Figure FDA0003589837130000045
wherein j is 1,2, σKnee jMeasuring the mean square error, sigma, of the deviation for knee joint movement anglesKnee 2For inertial sensingMean square error, sigma, of measured deviation of knee joint movement angleKnee 1Calculating the mean square error of the measurement deviation of the knee joint motion angle for a pose calculating module;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure FDA0003589837130000046
wherein j is 1,2, wKnee jShort-time dynamic weighting fusion coefficients for knee joint movement angle measurement;
knee joint motion angle after data fusion:
αknee＝wKnee 1αKnee 1+wKnee 2αKnee 2；
Let the angle of plantar flexion and dorsiflexion of the ankle joint be alphaAnkle jointThe angle of the ankle joint varus and valgus is betaAnkle jointThe internal rotation and external rotation angle of the ankle joint is gammaAnkle jointRecording the motion angle of the ankle joint acquired by the inertial sensor as (alpha)Ankle 2，βAnkle 2，γAnkle 2) The pose calculation module calculates the estimated motion angle (alpha) of the lower limb ankle joint of the wearerAnkle 1，βAnkle 1，γAnkle 1) The pose resolving module will (alpha)Ankle 2，βAnkle 2，γAnkle 2) And (alpha)Ankle 1，βAnkle 1，γAnkle 1) The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the ankle joint movement angle acquired by the n inertial sensors is (alpha)Ankle 2，βAnkle 2，γAnkle 2) The n posture calculation module calculates the estimated movement angle (alpha) of the lower limb ankle joint of the wearerAnkle 1，βAnkle 1，γAnkle 1) Then, within the time t, the reference value of the ankle joint movement angle
Figure FDA0003589837130000051
Satisfies the following conditions:
Figure FDA0003589837130000052
average value of measured deviation of ankle joint movement angle
Figure FDA0003589837130000053
Satisfies the following conditions:
Figure FDA0003589837130000054
the mean square error of the measured deviation of the ankle joint movement angle is:
Figure FDA0003589837130000061
wherein j is 1,2, σ1 ankle jMeasuring the mean square deviation, sigma, of the deviation for the plantar flexion and dorsiflexion motion angles of the ankle joint2 ankle jMeasuring the mean square error, sigma, of the deviation for the ankle inversion and eversion motion angles3 ankle jThe mean square error of the deviation is measured for the internal rotation and external rotation motion angles of the ankle joint (sigma)1 ankle 1，σ2 ankle 1，σ3 ankle 1) Calculating the mean square error (sigma) of the measurement deviation of the motion angle of the ankle joint for the pose calculating module1 ankle 2，σ2 ankle 2，σ3 ankle 2) Measuring the mean square error of deviation for the motion angle of the ankle joint of the inertial sensor;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure FDA0003589837130000062
wherein k is 1,2,3, and w is 11 ankle jA short-time dynamic weighting fusion coefficient is measured for the plantar flexion and dorsiflexion motion angles of the ankle joint, when k is 2, w2 ankle jThe short-time dynamic weighting fusion coefficient for measuring the ankle joint inversion and eversion motion angle is that when k is 3, w3 ankle jIs the ankle jointShort-time dynamic weighting fusion coefficients of the internal rotation and external rotation motion angle measurement;
ankle joint motion angle after data fusion:
Figure FDA0003589837130000063
2. the multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 1, characterized in that: the external auxiliary device comprises a waist coaming (1), a thigh outer side plate (3), a thigh fixing bandage (5), a shank outer side plate (6), a shank fixing bandage (8), a pedal plate (10) and a pedal plate bandage (9), wherein the waist coaming (1) is worn on the waist of a wearer, the thigh outer side plate (3) is worn on the outer side of the thigh of the wearer through the thigh fixing bandage (5), the shank outer side plate (6) is worn on the outer side of the shank of the wearer through the shank fixing bandage (8), and the pedal plate (10) is worn under the feet of the wearer through the pedal plate bandage (9); universal joints are adopted to connect the waist coaming (1) and the thigh outer side plate (3), the thigh outer side plate (3) and the shank outer side plate (6), and the shank outer side plate (6) and the pedal (10).
3. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 2, characterized in that: the inertial sensors comprise a waist inertial sensor (2), a thigh inertial sensor (4), a shank inertial sensor (7) and a foot inertial sensor (11);
the waist inertial sensor (2) is fixed on the outer side of the front end of the waist boarding (1), the thigh inertial sensor (4) is fixed on the outer surface of the thigh outer side plate (3), the shank inertial sensor (7) is fixed on the outer surface of the shank outer side plate (6), and the foot inertial sensor (11) is fixed on the outer surface of the pedal (10).
4. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 1, characterized in that: the acquisition mode of the human body lower limb muscle activity time sequence database is as follows:
s1, a large number of testers wear external auxiliary devices to perform corresponding movement according to use requirements, and surface electromyographic signal training samples of a large number of rectus femoris muscles, medial femoral muscles, lateral femoral muscles, biceps femoris muscles, semitendinosus muscles, tibialis anterior muscles, gastrocnemius muscles and soleus muscles are obtained;
s2, performing endpoint detection on the myoelectric signal training samples on the surfaces of the muscles by adopting a cepstrum method, and extracting effective activity section data;
s3, extracting characteristic parameters of effective activity segment data of the muscle surface electromyogram signal training samples by adopting a characteristic extraction method, wherein the characteristic parameters comprise an absolute value mean value, a root mean square, an average power frequency and a median frequency;
and S4, acquiring 8 muscle activity time sequences corresponding to different motions according to the characteristic parameters to form a motion characteristic matching list, wherein the motion characteristic matching list forms a human body lower limb muscle activity time sequence database.
5. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 4, characterized in that: the method of forming the motion feature matching list in step S4 is as follows:
for a certain movement, carrying out binary coding conversion on four characteristic parameters in a muscle surface electromyographic signal training sample according to increasing and decreasing trends, and sequentially arranging according to the absolute value mean value, the root mean square, the average power frequency and the median frequency to form coding strips;
the effective activity segment data of the surface electromyogram signal training samples of 8 muscles form 8 coding strips, the coding strips are combined and stored into a data matrix, different data matrices are stored for different movements, and finally a movement characteristic matching list for different movements is formed.
6. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 5, characterized in that: the method for carrying out the binary code conversion on four characteristic parameters in a muscle surface electromyogram signal training sample according to the increasing and decreasing trend comprises the following steps:
if the value of a certain characteristic parameter is increased, the value is 01, if the value of the characteristic parameter is decreased, the value is 10, and if the value of the characteristic parameter is kept unchanged, the value is 00.
7. The method for quickly predicting the motion pose of the lower limbs based on multi-sensor fusion is characterized by comprising the following steps of:
(1) the wearer pastes surface electromyography sensors on the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle;
(2) the external auxiliary device is worn by a wearer, wherein the waistline plate is worn on the waist of the wearer, the thigh outer side plate is worn on the outer side of the thigh of the wearer through a thigh fixing bandage, the shank outer side plate is worn on the outer side of the shank of the wearer through a shank fixing bandage, and the pedal plate is worn under the foot of the wearer through a pedal bandage; universal joints are adopted to connect the waist coaming and the thigh outer side plate, the thigh outer side plate and the shank outer side plate, and the shank outer side plate and the pedal plate;
(3) fixing an inertial sensor on the outer side of the front end of the waist coaming, the outer thigh side plate, the outer shank side plate and the outer surface of the pedal;
(4) the lower limbs of a wearer do any movement, and the surface myoelectric sensor and the inertial sensor acquire information;
(5) the surface electromyographic sensor transmits the surface electromyographic signals to the pose response module, and the pose response module matches the acquired time sequence of the electromyographic signals of the surfaces of the muscles with a database of the activity time sequence of the muscles of the lower limbs of the human body to find the motion state at the moment;
(6) the surface electromyographic sensor and the inertial sensor transmit the acquired signals to the pose resolving module, and the pose resolving module estimates the movement angle (alpha) of the hip joint of the lower limb of the wearer through the surface electromyographic signalsHip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint motion angle (alpha)Ankle 1，βAnkle 1，γAnkle 1) The hip joint motion angle (alpha) acquired by the inertial sensor is processed by a short-time dynamic weighting fusion methodHip 2，βHip 2，γHip 2) Knee joint movement angle alphaKnee 2Ankle jointPitch motion angle (alpha)Ankle 2，βAnkle 2，γAnkle 2) Angle of motion with hip joint (alpha)Hip 1，βHip 1，γHip 1) Knee joint movement angle alphaKnee 1Ankle joint motion angle (alpha)Ankle 1，βAnkle 1，γAnkle 1) Data fusion is carried out, and the hip joint motion angle (alpha) of the human body lower limb motion posture is obtained in real timeHip joint，βHip joint，γHip joint) Knee joint movement angle alphaKneeAnkle joint motion angle (alpha)Ankle joint，βAnkle joint，γAnkle joint) The position coordinates (x) of hip, knee and ankle joints are respectively calculated by the installation sizes of the waist coaming, the thigh outer side plate, the shank outer side plate and the foot pedal of the external auxiliary deviceHip joint,yHip joint,zHip joint)、(xKnee,yKnee,zKnee)、(xAnkle joint,yAnkle joint,zAnkle joint)。
CN201910804483.3A 2019-08-28 2019-08-28 Multi-sensor fusion-based lower limb movement pose rapid prediction system and method Active CN110755070B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910804483.3A CN110755070B (en) 2019-08-28 2019-08-28 Multi-sensor fusion-based lower limb movement pose rapid prediction system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910804483.3A CN110755070B (en) 2019-08-28 2019-08-28 Multi-sensor fusion-based lower limb movement pose rapid prediction system and method

Publications (2)

Publication Number Publication Date
CN110755070A CN110755070A (en) 2020-02-07
CN110755070B true CN110755070B (en) 2022-07-05

Family

ID=69329233

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910804483.3A Active CN110755070B (en) 2019-08-28 2019-08-28 Multi-sensor fusion-based lower limb movement pose rapid prediction system and method

Country Status (1)

Country Link
CN (1) CN110755070B (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111632361B (en) * 2020-04-16 2021-06-11 北京邮电大学 Foot pronation detection method and device based on sole pressure
CN111643321B (en) * 2020-04-30 2023-05-12 北京精密机电控制设备研究所 Exoskeleton joint angle prediction method and system based on sEMG signals
CN111588376A (en) * 2020-05-07 2020-08-28 北京联合大学 Lower limb gait information extraction equipment based on electromyographic signals and angle signals
CN111803250A (en) * 2020-07-03 2020-10-23 北京联合大学 Knee joint angle prediction method and system based on electromyographic signals and angle signals
CN111993391B (en) * 2020-08-25 2022-02-15 深圳市优必选科技股份有限公司 Robot pose estimation method and device, humanoid robot and storage medium
CN112107397B (en) * 2020-10-19 2021-08-24 中国科学技术大学 Myoelectric signal driven lower limb artificial limb continuous control system
CN112535474B (en) * 2020-11-11 2021-12-28 西安交通大学 Lower limb movement joint angle real-time prediction method based on similar rule search
CN112596534A (en) * 2020-12-04 2021-04-02 杭州未名信科科技有限公司 Gait training method and device for quadruped robot based on deep reinforcement learning, electronic equipment and medium
CN112704491B (en) * 2020-12-28 2022-01-28 华南理工大学 Lower limb gait prediction method based on attitude sensor and dynamic capture template data
CN112936224B (en) * 2021-01-27 2022-11-25 山东大学 Non-binding type easy-to-wear lower limb exoskeleton assistance system and use method
CN113080946A (en) * 2021-04-29 2021-07-09 合肥工业大学 Human body sitting and standing transfer capacity measuring device and method and electronic equipment
CN113520413B (en) * 2021-08-25 2022-10-21 长春工业大学 Lower limb multi-joint angle estimation method based on surface electromyogram signals
CN114983400A (en) * 2022-07-27 2022-09-02 南昌大学 Lower limb joint mobility monitoring system and monitoring method
CN116509380B (en) * 2023-04-14 2023-12-29 江苏泰科博曼医疗器械有限公司 Knee joint coordinate establishment device, method and system
CN116442202B (en) * 2023-06-19 2023-08-18 贵州航天控制技术有限公司 Waist boosting equipment control method based on back posture information
CN117281667B (en) * 2023-11-09 2024-04-09 浙江强脑科技有限公司 Motion pattern recognition method and device, intelligent artificial limb, terminal and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105213153A (en) * 2015-09-14 2016-01-06 西安交通大学 Based on the lower limb rehabilitation robot control method of brain flesh information impedance
CN105597298A (en) * 2016-04-05 2016-05-25 哈尔滨工业大学 Fitness effect evaluation system based on electromyographic signal and body movement detection
CN106821391A (en) * 2017-03-23 2017-06-13 北京精密机电控制设备研究所 Body gait acquisition analysis system and method based on inertial sensor information fusion
CN108283569A (en) * 2017-12-27 2018-07-17 北京精密机电控制设备研究所 A kind of exoskeleton robot control system and control method
CN109222969A (en) * 2018-10-31 2019-01-18 郑州大学 A kind of wearable human upper limb muscular movement fatigue detecting and training system based on Fusion
CN109589496A (en) * 2019-01-18 2019-04-09 刘坤 A kind of wearable bionical rehabilitation system of human motion overall process
CN109793500A (en) * 2019-01-24 2019-05-24 河南省人民医院 Knee joint load forces credit analysis apparatus

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI455705B (en) * 2009-10-07 2014-10-11 Ind Tech Res Inst Method and system for monioring sport related fitness by estimating muscle power and joint force of limbs
US8951192B2 (en) * 2010-06-15 2015-02-10 Flint Hills Scientific, Llc Systems approach to disease state and health assessment
US8679009B2 (en) * 2010-06-15 2014-03-25 Flint Hills Scientific, Llc Systems approach to comorbidity assessment
US10729905B2 (en) * 2017-01-11 2020-08-04 Boston Scientific Neuromodulation Corporation Pain management based on muscle tension measurements

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105213153A (en) * 2015-09-14 2016-01-06 西安交通大学 Based on the lower limb rehabilitation robot control method of brain flesh information impedance
CN105597298A (en) * 2016-04-05 2016-05-25 哈尔滨工业大学 Fitness effect evaluation system based on electromyographic signal and body movement detection
CN106821391A (en) * 2017-03-23 2017-06-13 北京精密机电控制设备研究所 Body gait acquisition analysis system and method based on inertial sensor information fusion
CN108283569A (en) * 2017-12-27 2018-07-17 北京精密机电控制设备研究所 A kind of exoskeleton robot control system and control method
CN109222969A (en) * 2018-10-31 2019-01-18 郑州大学 A kind of wearable human upper limb muscular movement fatigue detecting and training system based on Fusion
CN109589496A (en) * 2019-01-18 2019-04-09 刘坤 A kind of wearable bionical rehabilitation system of human motion overall process
CN109793500A (en) * 2019-01-24 2019-05-24 河南省人民医院 Knee joint load forces credit analysis apparatus

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
《一种基于表面肌电和惯性传感器的上肢运动功能评估方法》;李嫣然;《中国生物医学工程联合学术年会》;20151231;全文 *
《基于多传感器的机器人遥操作人机交互系统》;易润泽;《测控技术》;20181231;全文 *

Also Published As

Publication number Publication date
CN110755070A (en) 2020-02-07

Similar Documents

Publication Publication Date Title
CN110755070B (en) Multi-sensor fusion-based lower limb movement pose rapid prediction system and method
Zhang et al. Ankle joint torque estimation using an EMG-driven neuromusculoskeletal model and an artificial neural network model
Xiong et al. Intelligent prediction of human lower extremity joint moment: an artificial neural network approach
Zheng et al. Gait phase estimation based on noncontact capacitive sensing and adaptive oscillators
Chen et al. Locomotion mode classification using a wearable capacitive sensing system
WO2018050191A1 (en) A human intention detection system for motion assistance
Zhang et al. Lower-limb joint torque prediction using LSTM neural networks and transfer learning
Chinmilli et al. A review on wearable inertial tracking based human gait analysis and control strategies of lower-limb exoskeletons
CN112754468A (en) Human body lower limb movement detection and identification method based on multi-source signals
Triloka et al. Neural computing for walking gait pattern identification based on multi-sensor data fusion of lower limb muscles
Liu et al. sEMG-based continuous estimation of knee joint angle using deep learning with convolutional neural network
Yi et al. Continuous prediction of lower-limb kinematics from multi-modal biomedical signals
Liang et al. Synergy-based knee angle estimation using kinematics of thigh
Wang et al. Prediction of contralateral lower-limb joint angles using vibroarthrography and surface electromyography signals in time-series network
Li et al. A human joint torque estimation method for elbow exoskeleton control
Tao et al. Multi-channel sEMG based human lower limb motion intention recognition method
Chen et al. IMU-based estimation of lower limb motion trajectory with graph convolution network
Yahya et al. Intelligent integrated wearable sensing mechanism for vertical jump height prediction in female netball players
Elbagoury et al. A hybrid real-time EMG intelligent rehabilitation robot motions control based on Kalman Filter, support vector machines and particle swarm optimization
Farhat et al. Biomechanical model of the lower limb based on relevant actions for the control of knee-rehabilitation parallel robots
Li et al. Hip joint trajectory generation based on human limb motion synergy
Guo et al. A novel algorithm OF kinematics parameters measurement for upper limb based ON motion capture
CN110633644A (en) Human body joint angle prediction method based on electromyographic wavelet packet decomposition and GABP
Ascioglu et al. Prediction of lower extremity joint angles using neural networks for exoskeleton robotic leg
Kalani et al. SVM for Decoding the Human Activity Mode from sEMG Signals

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant