CN110755070A - Multi-sensor fusion-based lower limb movement pose rapid prediction system and method - Google Patents

Multi-sensor fusion-based lower limb movement pose rapid prediction system and method Download PDF

Info

Publication number
CN110755070A
CN110755070A CN201910804483.3A CN201910804483A CN110755070A CN 110755070 A CN110755070 A CN 110755070A CN 201910804483 A CN201910804483 A CN 201910804483A CN 110755070 A CN110755070 A CN 110755070A
Authority
CN
China
Prior art keywords
ankle
hip
joint
knee
angle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910804483.3A
Other languages
Chinese (zh)
Other versions
CN110755070B (en
Inventor
陶云飞
黄玉平
郑继贵
陈靓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Research Institute of Precise Mechatronic Controls
Original Assignee
Beijing Research Institute of Precise Mechatronic Controls
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Research Institute of Precise Mechatronic Controls filed Critical Beijing Research Institute of Precise Mechatronic Controls
Priority to CN201910804483.3A priority Critical patent/CN110755070B/en
Publication of CN110755070A publication Critical patent/CN110755070A/en
Application granted granted Critical
Publication of CN110755070B publication Critical patent/CN110755070B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/389—Electromyography [EMG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1071—Measuring physical dimensions, e.g. size of the entire body or parts thereof measuring angles, e.g. using goniometers
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1116—Determining posture transitions
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Veterinary Medicine (AREA)
  • Surgery (AREA)
  • Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Molecular Biology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Physiology (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Dentistry (AREA)
  • Theoretical Computer Science (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Psychiatry (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The system for quickly predicting the motion pose of the lower limb based on multi-sensor fusion comprises a surface electromyography sensor, an inertial sensor, a pose response module, a pose resolving module and an external auxiliary device. The external assisting device is worn on the waist and lower limbs of the wearer; the method comprises the following steps that a surface electromyography sensor acquires electromyography signals of the surfaces of muscles of the lower limb of a wearer in real time; the inertial sensor is fixed on the external auxiliary device; the pose response module determines the motion state corresponding to the electromyographic signals of the surfaces of the muscles of the current lower limb according to the time sequence of the electromyographic signals of the surfaces of the muscles collected; the pose calculation module carries out short-time dynamic weighted data fusion on each joint motion angle acquired by the inertial sensor and the calculated and estimated joint motion angle, and obtains the lower limb motion attitude of the wearer and the position coordinates of the joints in real time. The invention also provides a method for quickly predicting the motion pose of the lower limbs. The invention can quickly predict the motion pose of the lower limbs of the human body and simultaneously improve the estimation precision of the continuous motion amount.

Description

Multi-sensor fusion-based lower limb movement pose rapid prediction system and method
Technical Field
The invention relates to a lower limb movement pose rapid prediction system and method based on multi-sensor fusion, and belongs to the field of exoskeleton robots and man-machine cooperative control.
Background
At present, in the aspects of rehabilitation training and power-assisted walking aid, an exoskeleton robot is increasingly favored by patients as an important device capable of replacing manual assistance. However, the existing exoskeleton robot has poor human-computer synergy effect, mainly because the human motion information prediction mode is mostly carried out by adopting a physical sensor mode, and the physical sensor can acquire the human motion information only under the condition that the body of a wearer has a certain motion amplitude, so that large time delay exists, and the requirement of quick prediction cannot be realized; meanwhile, a single sensor has certain accuracy in predicting a certain discrete motion, and the estimation of continuous motion amount has great defects.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the system and the method for quickly predicting the lower limb movement pose based on multi-sensor fusion can quickly predict the lower limb movement pose of a human body and improve the estimation precision of continuous movement amount.
The technical solution of the invention is as follows:
the lower limb movement pose rapid prediction system based on multi-sensor fusion comprises a surface electromyography sensor, an inertial sensor, a pose response module, a pose resolving module and an external auxiliary device;
the external auxiliary device is worn on the waist and the lower limbs of a wearer and can move along with the left and right lower limbs of the human body;
the surface electromyographic sensors are adhered to the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle of a wearer, and are used for acquiring electromyographic signals of the surfaces of the muscles of the lower limb of the wearer in real time and transmitting the signals to the pose response module;
the inertial sensor is fixed on the external auxiliary device and used for acquiring the hip joint movement angle, the knee joint movement angle and the ankle joint movement angle of the lower limb of the wearer in real time;
the pose response module is fixed on an external auxiliary device, carries out filtering processing on the electromyographic signals of the surface of the lower limb of the wearer, then matches the time sequence of the electromyographic signals of the surface of each muscle collected with the time sequence of the muscle activity of the lower limb of the human body stored in a time sequence database of the muscle activity of the lower limb of the human body in advance, and finds the motion state corresponding to the electromyographic signals of the surface of each muscle of the current lower limb;
the pose calculation module is fixed on an external auxiliary device, and is used for calculating signals acquired by the surface electromyography, estimating the hip joint motion angle, the knee joint motion angle and the ankle joint motion angle of the lower limb of a wearer, carrying out short-time dynamic weighted data fusion on the hip joint motion angle, the knee joint motion angle and the ankle joint motion angle acquired by the inertial sensor and the calculated lower limb hip joint motion angle, the knee joint motion angle and the ankle joint motion angle of the wearer, obtaining the lower limb motion posture of the wearer in real time, and calculating the position coordinates of the hip joint, the knee joint and the ankle joint through the size of the external auxiliary device.
The external auxiliary device comprises a waist coaming, a thigh outer side plate, a thigh fixing bandage, a shank outer side plate, a shank fixing bandage, a pedal plate and a pedal plate bandage, wherein the waist coaming is worn on the waist of a wearer, the thigh outer side plate is worn on the outer side of the thigh of the wearer through the thigh fixing bandage, the shank outer side plate is worn on the outer side of the shank of the wearer through the shank fixing bandage, and the pedal plate is worn under the foot of the wearer through the pedal plate bandage; universal joints are adopted to connect the waist coaming and the thigh outer side plate, the thigh outer side plate and the shank outer side plate, and the shank outer side plate and the pedal plate.
The inertial sensors comprise a waist inertial sensor, a thigh inertial sensor, a shank inertial sensor and a foot inertial sensor;
the waist inertial sensor is fixed on the outer side of the front end of the waist coaming, the thigh inertial sensor is fixed on the outer surface of the thigh outer side plate, the shank inertial sensor is fixed on the outer surface of the shank outer side plate, and the foot inertial sensor is fixed on the outer surface of the pedal.
The acquisition mode of the human body lower limb muscle activity time sequence database is as follows:
s1, a large number of testers wear external auxiliary devices to perform corresponding movement according to use requirements, and surface electromyographic signal training samples of a large number of rectus femoris muscles, medial femoral muscles, lateral femoral muscles, biceps femoris muscles, semitendinosus muscles, tibialis anterior muscles, gastrocnemius muscles and soleus muscles are obtained;
s2, performing endpoint detection on the myoelectric signal training samples on the surfaces of the muscles by adopting a cepstrum method, and extracting effective activity section data;
s3, extracting characteristic parameters of effective activity segment data of the muscle surface electromyogram signal training samples by adopting a characteristic extraction method, wherein the characteristic parameters comprise an absolute value mean value, a root mean square, an average power frequency and a median frequency;
and S4, acquiring 8 muscle activity time sequences corresponding to different motions according to the characteristic parameters to form a motion characteristic matching list, wherein the motion characteristic matching list forms a human body lower limb muscle activity time sequence database.
The method of forming the motion feature matching list in step S4 is as follows:
for a certain movement, carrying out binary coding conversion on four characteristic parameters in a muscle surface electromyographic signal training sample according to increasing and decreasing trends, and sequentially arranging according to the absolute value mean value, the root mean square, the average power frequency and the median frequency to form coding strips;
the effective activity segment data of the surface electromyogram signal training samples of 8 muscles form 8 coding strips, the coding strips are combined and stored into a data matrix, different data matrices are stored for different movements, and finally a movement characteristic matching list for different movements is formed.
The method for carrying out the binary code conversion on four characteristic parameters in a muscle surface electromyogram signal training sample according to the increasing and decreasing trend comprises the following steps:
if the value of a certain characteristic parameter is increased, the value is 01, if the value of the characteristic parameter is decreased, the value is 10, and if the value of the characteristic parameter is kept unchanged, the value is 00.
Provided for flexion and extension of hip jointAngle αHip jointThe abduction and adduction angle of the hip joint is βHip jointThe angle of internal rotation and external rotation of the hip joint is gammaHip jointThe hip joint motion angle acquired by the inertial sensor is recorded as (α)Hip 2，βHip 2，γHip 2) The pose calculation module calculates the estimated hip joint movement angle of the lower limb of the wearer as (α)Hip 1，βHip 1，γHip 1) The pose solution module will (α)Hip 2，βHip 2，γHip 2) And (α)Hip 1，βHip 1，γHip 1) The method for short-time dynamic weighting data fusion comprises the following steps:
during a time t preceding the current movement instant, there are n hip-joint movement angles (α) acquired by the inertial sensorsHip 2，βHip 2，γHip 2) And the n posture resolving modules resolve the estimated hip joint movement angle of the lower limb of the wearer (α)Hip 1，βHip 1，γHip 1) Then, within time t, the reference value of the hip joint movement angle
Figure BDA0002183237270000041
Satisfies the following conditions:
Figure BDA0002183237270000042
wherein i is 1,2, 3.
Average value of measured deviation of hip joint movement angle
Figure BDA0002183237270000043
Satisfies the following conditions:
Figure BDA0002183237270000044
the mean square error of the measured deviation of the hip joint movement angle:
Figure BDA0002183237270000051
wherein j is 1,2, σ1 hip jointjMeasuring the mean square deviation, sigma, of the deviation for hip flexion and extension motion angles2 hip jMeasuring the mean square error, sigma, of the deviation for the abduction and adduction angles of the hip joint3 hip jThe mean square error of the measured deviation of the external rotation and internal rotation angles of the hip joint is (sigma)1 hip 1，σ2 hip 1，σ3 hip 1) Calculating and estimating the mean square error of the hip joint motion angle measurement deviation for a pose calculation module, (sigma)1 hip 2，σ2 hip 2，σ3 hip 2) Measuring the mean square error of the deviation for the inertial sensor hip joint movement angle;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure BDA0002183237270000052
wherein k is 1,2,3, and w is 11 hip jA short-time dynamic weighting fusion coefficient for measuring the flexion and extension movement angles of the hip joint, when k is 2, w2 hip jA short-time dynamic weighting fusion coefficient for measuring abduction and adduction movement angles of the hip joint, when k is 3, w3 hip jShort-time dynamic weighting fusion coefficients for measuring the hip joint external rotation and internal rotation motion angles;
hip joint motion angle after data fusion:
Figure BDA0002183237270000053
let the flexion and extension angle of the knee joint be αKneeThe knee joint motion angle acquired by the inertial sensor is recorded as αKnee 2The pose resolving module resolves the estimated knee joint motion angle of the lower limb of the wearer to αKnee 1The pose resolving module will αKnee 2And αKnee 1The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the knee joint movement angle acquired by the n inertial sensors is αKnee 2The n posture calculation module calculates the estimated knee joint movement angle of the lower limb of the wearer to be αKnee 1Within time tReference value of knee joint movement angle
Figure BDA0002183237270000061
Satisfies the following conditions:
Figure BDA0002183237270000062
average value of measured deviation of knee joint movement angle
Figure BDA0002183237270000063
Satisfies the following conditions:
Figure BDA0002183237270000064
the mean square error of the measured deviation of the knee joint movement angle is as follows:
wherein j is 1,2, σKnee jMeasuring the mean square error, sigma, of the deviation for knee joint movement anglesKnee 2For measuring the mean square deviation, sigma, of the deviation of the knee joint movement angle of the inertial sensorKnee 1Calculating the mean square error of the measurement deviation of the knee joint motion angle for a pose calculating module;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure BDA0002183237270000066
wherein j is 1,2, wKnee jShort-time dynamic weighting fusion coefficients for knee joint movement angle measurement;
knee joint motion angle after data fusion:
αknee＝wKnee 1αKnee 1+wKnee 2αKnee 2。
Let the angle of plantar flexion and dorsiflexion of the ankle joint be αAnkle jointThe angle of the ankle joint varus and valgus is βAnkle jointInward rotation and outward rotation of ankle jointThe angle of rotation is gammaAnkle jointThe ankle joint movement angle acquired by the inertial sensor is recorded as (α)Ankle 2，βAnkle 2，γAnkle 2) The pose calculation module calculates the estimated motion angle of the ankle joint of the lower limb of the wearer to be (α)Ankle 1，βAnkle 1，γAnkle 1) The pose solution module will (α)Ankle 2，βAnkle 2，γAnkle 2) And (α)Ankle 1，βAnkle 1，γAnkle 1) The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the ankle movement angle acquired by the n inertial sensors is (α)Ankle 2，βAnkle 2，γAnkle 2) The n posture resolving module resolves the estimated movement angle of the lower limb ankle joint of the wearer (α)Ankle 1，βAnkle 1，γAnkle 1) Then, within the time t, the reference value of the ankle joint movement angle
Figure BDA0002183237270000071
Satisfies the following conditions:
Figure BDA0002183237270000072
average value of measured deviation of ankle joint movement angle
Figure BDA0002183237270000073
Satisfies the following conditions:
the mean square error of the measured deviation of the ankle joint movement angle is:
Figure BDA0002183237270000081
wherein j is 1,2, σ1 ankle jMeasuring the mean square deviation, sigma, of the deviation for the plantar flexion and dorsiflexion motion angles of the ankle joint2 ankle jMeasuring the mean square error, sigma, of the deviation for the ankle inversion and eversion motion angles3 ankle jThe mean square error of the deviation is measured for the internal rotation and external rotation motion angles of the ankle joint (sigma)1 ankle 1，σ2 ankle 1，σ3 ankle 1) Calculating the mean square error (sigma) of the measurement deviation of the ankle joint motion angle for a pose calculation module1 ankle 2，σ2 ankle 2，σ3 ankle 2) Measuring the mean square error of deviation for the motion angle of the ankle joint of the inertial sensor;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
wherein k is 1,2,3, and w is 11 ankle jA short-time dynamic weighting fusion coefficient is measured for the plantar flexion and dorsiflexion motion angles of the ankle joint, when k is 2, w2 ankle jThe short-time dynamic weighting fusion coefficient for measuring the ankle joint inversion and eversion motion angle is that when k is 3, w3 ankle jShort-time dynamic weighting fusion coefficients for ankle internal rotation and external rotation movement angle measurement;
ankle joint motion angle after data fusion:
Figure BDA0002183237270000083
the method for quickly predicting the motion pose of the lower limbs based on multi-sensor fusion comprises the following steps:
(1) the wearer pastes surface electromyography sensors on the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle;
(2) the external auxiliary device is worn by a wearer, wherein the waistline plate is worn on the waist of the wearer, the thigh outer side plate is worn on the outer side of the thigh of the wearer through a thigh fixing bandage, the shank outer side plate is worn on the outer side of the shank of the wearer through a shank fixing bandage, and the pedal plate is worn under the foot of the wearer through a pedal bandage; universal joints are adopted to connect the waist coaming and the thigh outer side plate, the thigh outer side plate and the shank outer side plate, and the shank outer side plate and the pedal plate;
(3) fixing an inertial sensor on the outer side of the front end of the waist coaming, the outer thigh side plate, the outer shank side plate and the outer surface of the pedal;
(4) the lower limbs of a wearer do any movement, and the surface myoelectric sensor and the inertial sensor acquire information;
(5) the surface electromyographic sensor transmits the surface electromyographic signals to the pose response module, and the pose response module matches the acquired time sequence of the electromyographic signals of the surfaces of the muscles with a database of the activity time sequence of the muscles of the lower limbs of the human body to find the motion state at the moment;
(6) the surface electromyographic sensor and the inertial sensor transmit the acquired signals to the pose resolving module, and the pose resolving module estimates the movement angle of the hip joint of the lower limb of the wearer through the surface electromyographic signals (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angle (α)Ankle 1，βAnkle 1，γAnkle 1) The hip joint motion angle collected by the inertial sensor is fused by short-time dynamic weighting (α)Hip 2，βHip 2，γHip 2) Knee joint movement angle αKnee 2Ankle joint motion angle (α)Ankle 2，βAnkle 2，γAnkle 2) Angle of motion with hip (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angle (α)Ankle 1，βAnkle 1，γAnkle 1) Data fusion is carried out, and the hip joint motion angle (α) of the human body lower limb motion posture is obtained in real timeHip joint，βHip joint，γHip joint) Knee joint movement angle αKneeAnkle joint motion angle (α)Ankle joint，βAnkle joint，γAnkle joint) The position coordinates (x) of hip, knee and ankle joints are respectively calculated by the installation sizes of the waist coaming, the thigh outer side plate, the shank outer side plate and the foot pedal of the external auxiliary deviceHip joint,yHip joint,zHip joint)、(xKnee,yKnee,zKnee)、(xAnkle joint,yAnkle joint,zAnkle joint)。
Compared with the prior art, the invention has the following beneficial effects:
(1) according to the invention, the surface electromyography sensor is adhered to the surface of the lower limb muscle, so that the electromyography signal of the muscle can be collected in real time, and the quick and real-time judgment on the motion state of the lower limb of a wearer is realized by combining a pre-established database of the activity time sequence of the muscle of the lower limb of the human body.
(2) According to the invention, each joint angle obtained by the estimation of the surface electromyography sensor and each joint angle obtained by the inertia sensor are subjected to data fusion by a short-time dynamic weighting fusion method, so that the accurate estimation of the continuous motion amount is realized.
(3) According to the invention, the surface electromyographic sensor is pasted on the surface of the lower limb muscle, the electromyographic signal of the muscle can be collected in real time, the pose calculation module predicts the motion angle and position coordinate of each joint through a short-time dynamic weighting data fusion method, so that the real-time prediction of the motion pose of the lower limb of the human body is realized, and 7 degrees of freedom of the lower limb of the human body and corresponding joint action combinations can be predicted.
Drawings
FIG. 1 is a flow chart of a method for rapidly predicting a motion pose of a lower limb of a human body based on multi-sensor fusion according to the invention;
FIG. 2 is a schematic diagram of a system for rapidly predicting a motion pose of a lower limb of a human body based on multi-sensor fusion according to the present invention;
FIG. 3 is a schematic view of the external attachment of the present invention (only the reference numerals on one side are shown, and the same are provided on the opposite side);
1-waist coaming; 2-a waist inertial sensor; 3-thigh lateral plate; 4-thigh inertial sensors; 5-thigh fixing bandage; 6-outer shank plate; 7-a calf portion inertial sensor; 8-shank fixing band; 9-foot board binding band; 10-a foot pedal; 11-foot inertial sensor.
Detailed Description
The invention provides a lower limb movement pose rapid prediction system based on multi-sensor fusion, which comprises a surface electromyography sensor, an inertial sensor, a pose response module, a pose resolving module and an external auxiliary device.
FIG. 2 is a schematic diagram of a human body lower limb movement pose rapid prediction system based on multi-sensor fusion.
Specifically, the external assisting device is worn on the waist and lower limbs of the wearer and can move along with the left and right lower limbs of the human body. As shown in fig. 3, the external auxiliary device includes a waist panel 1, a thigh outer panel 3, a thigh fixing band 5, a shank outer panel 6, a shank fixing band 8, a foot board 10 and a foot board band 9, the waist panel 1 is worn on the waist of the wearer, the thigh outer panel 3 is worn on the outer side of the thigh of the wearer through the thigh fixing band 5, the shank outer panel 6 is worn on the outer side of the shank of the wearer through the shank fixing band 8, and the foot board 10 is worn under the foot of the wearer through the foot board band 9; universal joints are adopted to connect the waist boarding 1 and the thigh outer side plate 3, the thigh outer side plate 3 and the shank outer side plate 6, and the shank outer side plate 6 and the pedal 10.
The inertial sensor is used for acquiring the angle signal of the lower limb of the wearer in real time. Specifically, the inertial sensors include a waist inertial sensor 2, a thigh inertial sensor 4, a calf inertial sensor 7, and a foot inertial sensor 11. The waist inertial sensor 2 is fixed on the outer side of the front end of the waist boarding 1, the thigh inertial sensor 4 is fixed on the outer surface of the thigh outer side plate 3, the shank inertial sensor 7 is fixed on the outer surface of the shank outer side plate 6, and the foot inertial sensor 11 is fixed on the outer surface of the foot board 10.
The surface electromyographic sensors are adhered to the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle of a wearer, and are used for acquiring electromyographic signals of the surfaces of the muscles of the lower limb of the wearer in real time and transmitting the signals to the pose response module;
the pose response module is fixed on an external auxiliary device, carries out filtering processing on the electromyographic signals of the lower limb surface of the wearer, then matches the time sequence of the collected electromyographic signals of the muscle surface with the time sequence of the activity of the muscles of the lower limb of the human body, which is stored in a database in advance, and finds the motion state corresponding to the electromyographic signals of the lower limb surface at present.
The acquisition mode of the human body lower limb muscle activity time sequence database is as follows:
a large number of testers wear external auxiliary devices to perform corresponding exercises according to use requirements to obtain a large number of training samples, end point detection is performed on the training samples to extract effective activity section data, a characteristic extraction method is adopted to extract series characteristic parameters, 8 muscle activity time sequences corresponding to different exercises are obtained through arrangement and combination of different characteristic parameters and increasing and decreasing trends of the different characteristic parameters, then a special coding mode is adopted to form an exercise characteristic matching list, the exercise characteristic matching list comprises starting point and end point information of the 8 muscles in the exercise process, and the exercise characteristic matching list forms a human body lower limb muscle activity time sequence database.
For example, a tester performs a certain exercise to obtain surface electromyogram signal training samples of rectus femoris, vastus medialis, vastus lateralis, biceps femoris, semitendinosus, tibialis anterior, gastrocnemius and soleus, performs endpoint detection on the surface electromyogram signal training samples of each muscle by using a cepstrum method to extract effective activity segment data, extracts characteristic parameters of the effective activity segment data of the surface electromyogram signal training samples of each muscle by using a time domain characteristic extraction method and a frequency domain characteristic extraction method, wherein the characteristic parameters comprise an absolute value mean, a root mean square, a mean power frequency and a median frequency, performs binary coding conversion on the characteristic parameters according to the increasing and decreasing trend of the characteristic parameters to form coding strips, the characteristic parameter value is 01 if the characteristic parameter value is increased, the characteristic parameter value is 10 if the characteristic parameter value is decreased, the characteristic parameter value is 00 if the characteristic parameter value is maintained, the effective activity segment data of the surface electromyogram signal training samples of each muscle has 4 characteristic parameters, the coding strip arranges the increasing trend of each characteristic parameter in turn according to the sequence of the absolute value mean, the root mean square, the average power frequency and the median frequency, such as 01001000, the coding strip represents a certain movement, the absolute value mean of a certain muscle is increased, the root mean square is unchanged, the average power frequency is reduced, and the median frequency is unchanged.
For a certain movement, all characteristic parameters of the effective activity segment data of the surface electromyogram signal training samples of 8 muscles are obtained simultaneously, the increasing and decreasing trends of the characteristic parameters are subjected to binary coding conversion to form 8 coding strips, the coding strips are combined and stored into a data matrix, different data matrices are stored for different movements, and finally a movement characteristic matching list for different movements is formed.
Pose resolving module fixed on external auxiliaryOn the device, the motion angle of the hip joint of the lower limb of the wearer is calculated and estimated by a signal acquired by a surface electromyogram based on a particle swarm optimization neural network model (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angle (α)Ankle 1，βAnkle 1，γAnkle 1) Hip joint motion angle acquired by inertial sensor (α)Hip 2，βHip 2，γHip 2) Knee joint movement angle αKnee 2Ankle joint motion angle (α)Ankle 2，βAnkle 2，γAnkle 2) Angle of motion with hip (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angle (α)Ankle 1，βAnkle 1，γAnkle 1) Dynamic weighted data fusion is carried out, the lower limb movement posture of the wearer is obtained in real time, and the position coordinates (x) of hip, knee and ankle joints are respectively calculated through the mounting sizes of a middle waist coaming, a thigh outer side plate, a shank outer side plate and a pedal of an external auxiliary deviceHip joint,yHip joint,zHip joint)、(xKnee,yKnee,zKnee)、(xAnkle joint,yAnkle joint,zAnkle joint)。
Wherein, the motion poses of the lower limbs of the human body are the positions and the postures of the hip, the knee and the ankle, and the postures of the hip, the knee and the ankle comprise the bending angles and the extending angles of the hip joint, namely αHip jointThe angle of abduction and adduction of the hip joint is βHip jointThe angle of internal rotation of the hip joint and external rotation of the hip joint is gammaHip jointThe angle of knee flexion and knee extension is αKneeThe angle of plantar flexion of the ankle joint and dorsiflexion of the ankle joint is αAnkle jointThe angle of ankle inversion and ankle eversion is βAnkle jointThe angle of the internal rotation of the ankle joint and the external rotation of the ankle joint is gammaAnkle joint。
The pose resolving module will (α)Hip 2，βHip 2，γHip 2) And (α)Hip 1，βHip 1，γHip 1) The method for short-time dynamic weighting data fusion comprises the following steps:
during a time t preceding the current movement instant, there are n hip-joint movement angles (α) acquired by the inertial sensorsHip 2，βHip 2，γHip 2) And the n posture resolving modules resolve the estimated hip joint movement angle of the lower limb of the wearer (α)Hip 1，βHip 1，γHip 1) Then, within time t, the reference value of the hip joint movement angle
Figure BDA0002183237270000131
Satisfies the following conditions:
Figure BDA0002183237270000132
wherein i is 1,2, 3.
Average value of measured deviation of hip joint movement angle
Figure BDA0002183237270000133
Satisfies the following conditions:
Figure BDA0002183237270000141
the mean square error of the measured deviation of the hip joint movement angle:
Figure BDA0002183237270000142
wherein j is 1,2, σ1 hip jMeasuring the mean square deviation, sigma, of the deviation for hip flexion and extension motion angles2 hip jMeasuring the mean square error, sigma, of the deviation for the abduction and adduction angles of the hip joint3 hip jThe mean square error of the measured deviation of the external rotation and internal rotation angles of the hip joint is (sigma)1 hip 1，σ2 hip 1，σ3 hip 1) Calculating and estimating the mean square error of the hip joint motion angle measurement deviation for a pose calculation module, (sigma)1 hip 2，σ2 hip 2，σ3 hip 2) Measuring the mean square error of the deviation for the inertial sensor hip joint movement angle;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure BDA0002183237270000143
wherein k is 1,2,3, and w is 11 hip jA short-time dynamic weighting fusion coefficient for measuring the flexion and extension movement angles of the hip joint, when k is 2, w2 hip jA short-time dynamic weighting fusion coefficient for measuring abduction and adduction movement angles of the hip joint, when k is 3, w3 hip jShort-time dynamic weighting fusion coefficients for measuring the hip joint external rotation and internal rotation motion angles;
hip joint motion angle after data fusion:
Figure BDA0002183237270000151
the pose resolving module will αKnee 2And αKnee 1The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the knee joint movement angle acquired by the n inertial sensors is αKnee 2The n posture calculation module calculates the estimated knee joint movement angle of the lower limb of the wearer to be αKnee 1Then, within the time t, the reference value of the knee joint movement angle
Figure BDA0002183237270000152
Satisfies the following conditions:
Figure BDA0002183237270000153
average value of measured deviation of knee joint movement angle
Figure BDA0002183237270000154
Satisfies the following conditions:
Figure BDA0002183237270000155
the mean square error of the measured deviation of the knee joint movement angle is as follows:
Figure BDA0002183237270000156
wherein j is 1,2, σKnee jMeasuring the mean square error, sigma, of the deviation for knee joint movement anglesKnee 2For measuring the mean square deviation, sigma, of the deviation of the knee joint movement angle of the inertial sensorKnee 1Calculating the mean square error of the measurement deviation of the knee joint motion angle for a pose calculating module;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
wherein j is 1,2, wKnee jShort-time dynamic weighting fusion coefficients for knee joint movement angle measurement;
knee joint motion angle after data fusion:
αknee＝wKnee 1αKnee 1+wKnee 2αKnee 2。
The pose resolving module will (α)Ankle 2，βAnkle 2，γAnkle 2) And (α)Ankle 1，βAnkle 1，γAnkle 1) The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the ankle movement angle acquired by the n inertial sensors is (α)Ankle 2，βAnkle 2，γAnkle 2) The n posture resolving module resolves the estimated movement angle of the lower limb ankle joint of the wearer (α)Ankle 1，βAnkle 1，γAnkle 1) Then, within the time t, the reference value of the ankle joint movement angleSatisfies the following conditions:
Figure BDA0002183237270000162
then the motion angle of the ankle jointAverage value of measurement deviation
Figure BDA0002183237270000163
Satisfies the following conditions:
Figure BDA0002183237270000164
the mean square error of the measured deviation of the ankle joint movement angle is:
wherein j is 1,2, σ1 ankle jMeasuring the mean square deviation, sigma, of the deviation for the plantar flexion and dorsiflexion motion angles of the ankle joint2 ankle jMeasuring the mean square error, sigma, of the deviation for the ankle inversion and eversion motion angles3 ankle jThe mean square error of the deviation is measured for the internal rotation and external rotation motion angles of the ankle joint (sigma)1 ankle 1，σ2 ankle 1，σ3 ankle 1) Calculating the mean square error (sigma) of the measurement deviation of the ankle joint motion angle for a pose calculation module1 ankle 2，σ2 ankle 2，σ3 ankle 2) Measuring the mean square error of deviation for the motion angle of the ankle joint of the inertial sensor;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
wherein k is 1,2,3, and w is 11 anklej is a short-time dynamic weighting fusion coefficient of the measurement of the motion angles of plantar flexion and dorsiflexion of the ankle joint, and when k is 2, w2 anklej is a short-time dynamic weighting fusion coefficient for measuring the ankle joint inversion and eversion motion angles, and when k is 3, w3 anklej is a short-time dynamic weighting fusion coefficient for measuring the internal rotation and external rotation motion angles of the ankle joint;
ankle joint motion angle after data fusion:
Figure BDA0002183237270000173
the pose calculation module calculates signals acquired by the surface electromyography through a joint angle estimation model to estimate the hip joint motion angle of the lower limb of the wearer (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angle (α)Ankle 1，βAnkle 1，γAnkle 1). The joint angle estimation model is realized by adopting a neural network model based on particle swarm optimization.
Inputting characteristic parameters [ X ] of effective activity segment data of myoelectric signal training samples on the surface of each muscle based on a neural network model optimized by particle swarm1,X2,X3,...,X32]The output is the angle of the wearer's lower extremity hip joint movement (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angle (α)Ankle 1，βAnkle 1，γAnkle 1) Adopting a Sigmoid function as an action function of a hidden layer and adopting a linear function as an action function of an output layer;
the mean square error generated by the neural network on the training set is used as an objective function, and the following fitness function is constructed to calculate the individual fitness value:
Figure BDA0002183237270000181
in the formula: y isln′Computing an output for the neural network; t isln′Outputting for the target; n is the number of training samples; l is the number of output neurons;
in a D-dimensional search space, in each iteration process, the speed and the position of the particles are updated through the individual extremum and the global extremum, and the optimal fitness value is found, so that the neural network parameters are optimized, and the updating formula is as follows:
wherein w is the inertial weight; d ═ 1,2, …, D; 1,2, …,m，m>32, a first step of removing the first layer; k is the current iteration number; x is the number ofidIs the position of the particle; v. ofidIs the velocity of the particle; c. C1And c2Is an acceleration factor; r is1And r2Is distributed in [0, 1 ]]A random number in between;is an individual extremum of the particle;
Figure BDA0002183237270000184
is the global extreme of the population of particles.
Meanwhile, the invention provides a method for quickly predicting the motion pose of the lower limbs of the human body by using the system for quickly predicting the motion pose of the lower limbs of the human body based on multi-sensor fusion, which comprises the following steps as shown in figure 1:
the method comprises the following steps: the wearer removes hair and cutin from the central position of the lower limb muscle belly, coats alcohol and the like, and sequentially sticks the surface electromyography sensor according to the use requirement;
step two: the external auxiliary device is worn by a wearer, wherein the waistline plate is worn on the waist of the wearer, the thigh outer side plate is worn on the outer side of the thigh of the wearer through a thigh fixing bandage, the shank outer side plate is worn on the outer side of the shank of the wearer through a shank fixing bandage, and the pedal plate is worn under the foot of the wearer through a pedal bandage; universal joints are adopted to connect the waist coaming and the thigh outer side plate, the thigh outer side plate and the shank outer side plate, and the shank outer side plate and the pedal plate; fixing an inertial sensor on the outer side of the front end of the waist coaming, the outer thigh side plate, the outer shank side plate and the outer surface of the pedal;
step three: when the lower limbs of a wearer do any movement, all the surface electromyographic sensors and the inertial sensors are arranged to acquire information, respectively acquire surface electromyographic signals of the rectus femoris muscle, the medial femoral muscle, the lateral femoral muscle, the biceps femoris muscle, the semitendinosus muscle, the tibialis anterior muscle, the gastrocnemius muscle and the soleus muscle of the left and right lower limbs, and acquire angle signals of thighs, calves and feet of the left and right lower limbs;
step four: the surface electromyography sensor transmits 8 measured paths of surface electromyography signals to the pose response module, the pose response module matches the activity time sequence of the 8 paths of surface electromyography signals with a database of activity time sequences of muscles of lower limbs of a human body to find the motion pose at the moment, and the activity time sequences of the group of surface electromyography signals are filled into the database as new samples;
fifthly, the surface electromyographic sensor and the inertial sensor transmit the acquired signals to a pose resolving module, and the pose resolving module estimates the hip joint movement angle of the lower limb of the wearer through the surface electromyographic signals (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angle (α)Ankle 1，βAnkle 1，γAnkle 1) The hip joint motion angle collected by the inertial sensor is fused by short-time dynamic weighting (α)Hip 2，βHip 2，γHip 2) Knee joint movement angle αKnee 2Ankle joint motion angle (α)Ankle 2，βAnkle 2，γAnkle 2) Angle of motion with hip (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angle (α)Ankle 1，βAnkle 1，γAnkle 1) Data fusion is carried out, and the hip joint motion angle (α) of the human body lower limb motion posture is obtained in real timeHip joint，βHip joint，γHip joint) Knee joint movement angle αKneeAnkle joint motion angle (α)Ankle joint，βAnkle joint，γAnkle joint) The position coordinates (x) of hip, knee and ankle joints are respectively calculated by the installation sizes of the waist coaming, the thigh outer side plate, the shank outer side plate and the foot pedal of the external auxiliary deviceHip joint,yHip joint,zHip joint)、(xKnee,yKnee,zKnee)、(xAnkle joint,yAnkle joint,zAnkle joint) And the quick prediction of the motion pose of the lower limbs of the human body is realized.
According to the invention, the surface electromyography sensor is adhered to the surface of the lower limb muscle, so that the electromyography signal of the muscle can be collected in real time, and the quick and real-time judgment on the motion state of the lower limb of a wearer is realized by combining a pre-established database of the activity time sequence of the muscle of the lower limb of the human body. On the basis, the pose calculation module predicts the motion angle and position coordinates of each joint through a short-time dynamic weighting data fusion method, and realizes the real-time prediction of the motion pose of the lower limbs of the human body and the accurate estimation of the continuous motion amount.
Those skilled in the art will appreciate that those matters not described in detail in the present specification are well known in the art.

Claims (10)

1. The lower limb movement pose rapid prediction system based on multi-sensor fusion is characterized in that: the system comprises a surface electromyography sensor, an inertial sensor, a pose response module, a pose resolving module and an external auxiliary device;
the external auxiliary device is worn on the waist and the lower limbs of a wearer and can move along with the left and right lower limbs of the human body;
the surface electromyographic sensors are adhered to the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle of a wearer, and are used for acquiring electromyographic signals of the surfaces of the muscles of the lower limb of the wearer in real time and transmitting the signals to the pose response module;
the inertial sensor is fixed on the external auxiliary device and used for acquiring the hip joint movement angle, the knee joint movement angle and the ankle joint movement angle of the lower limb of the wearer in real time;
the pose response module is fixed on an external auxiliary device, carries out filtering processing on the electromyographic signals of the surface of the lower limb of the wearer, then matches the time sequence of the electromyographic signals of the surface of each muscle collected with the time sequence of the muscle activity of the lower limb of the human body stored in a time sequence database of the muscle activity of the lower limb of the human body in advance, and finds the motion state corresponding to the electromyographic signals of the surface of each muscle of the current lower limb;
the pose calculation module is fixed on an external auxiliary device, and is used for calculating signals acquired by the surface electromyography, estimating the hip joint motion angle, the knee joint motion angle and the ankle joint motion angle of the lower limb of a wearer, carrying out short-time dynamic weighted data fusion on the hip joint motion angle, the knee joint motion angle and the ankle joint motion angle acquired by the inertial sensor and the calculated lower limb hip joint motion angle, the knee joint motion angle and the ankle joint motion angle of the wearer, obtaining the lower limb motion posture of the wearer in real time, and calculating the position coordinates of the hip joint, the knee joint and the ankle joint through the size of the external auxiliary device.
2. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 1, characterized in that: the external auxiliary device comprises a waist coaming (1), a thigh outer side plate (3), a thigh fixing bandage (5), a shank outer side plate (6), a shank fixing bandage (8), a pedal plate (10) and a pedal plate bandage (9), wherein the waist coaming (1) is worn on the waist of a wearer, the thigh outer side plate (3) is worn on the outer side of the thigh of the wearer through the thigh fixing bandage (5), the shank outer side plate (6) is worn on the outer side of the shank of the wearer through the shank fixing bandage (8), and the pedal plate (10) is worn under the feet of the wearer through the pedal plate bandage (9); universal joints are adopted to connect the waist coaming (1) and the thigh outer side plate (3), the thigh outer side plate (3) and the shank outer side plate (6), and the shank outer side plate (6) and the pedal (10).
3. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 2, characterized in that: the inertial sensors comprise a waist inertial sensor (2), a thigh inertial sensor (4), a shank inertial sensor (7) and a foot inertial sensor (11);
the waist inertial sensor (2) is fixed on the outer side of the front end of the waist boarding (1), the thigh inertial sensor (4) is fixed on the outer surface of the thigh outer side plate (3), the shank inertial sensor (7) is fixed on the outer surface of the shank outer side plate (6), and the foot inertial sensor (11) is fixed on the outer surface of the pedal (10).
4. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 1, characterized in that: the acquisition mode of the human body lower limb muscle activity time sequence database is as follows:
s1, a large number of testers wear external auxiliary devices to perform corresponding movement according to use requirements, and surface electromyographic signal training samples of a large number of rectus femoris muscles, medial femoral muscles, lateral femoral muscles, biceps femoris muscles, semitendinosus muscles, tibialis anterior muscles, gastrocnemius muscles and soleus muscles are obtained;
s2, performing endpoint detection on the myoelectric signal training samples on the surfaces of the muscles by adopting a cepstrum method, and extracting effective activity section data;
s3, extracting characteristic parameters of effective activity segment data of the muscle surface electromyogram signal training samples by adopting a characteristic extraction method, wherein the characteristic parameters comprise an absolute value mean value, a root mean square, an average power frequency and a median frequency;
and S4, acquiring 8 muscle activity time sequences corresponding to different motions according to the characteristic parameters to form a motion characteristic matching list, wherein the motion characteristic matching list forms a human body lower limb muscle activity time sequence database.
5. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 4, characterized in that: the method of forming the motion feature matching list in step S4 is as follows:
for a certain movement, carrying out binary coding conversion on four characteristic parameters in a muscle surface electromyographic signal training sample according to increasing and decreasing trends, and sequentially arranging according to the absolute value mean value, the root mean square, the average power frequency and the median frequency to form coding strips;
the effective activity segment data of the surface electromyogram signal training samples of 8 muscles form 8 coding strips, the coding strips are combined and stored into a data matrix, different data matrices are stored for different movements, and finally a movement characteristic matching list for different movements is formed.
6. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 5, characterized in that: the method for carrying out the binary code conversion on four characteristic parameters in a muscle surface electromyogram signal training sample according to the increasing and decreasing trend comprises the following steps:
if the value of a certain characteristic parameter is increased, the value is 01, if the value of the characteristic parameter is decreased, the value is 10, and if the value of the characteristic parameter is kept unchanged, the value is 00.
7. The lower limb exercise based on multi-sensor fusion of claim 1The dynamic pose rapid prediction system is characterized in that the flexion and extension angles of the hip joint are αHip jointThe abduction and adduction angle of the hip joint is βHip jointThe angle of internal rotation and external rotation of the hip joint is gammaHip jointThe hip joint motion angle acquired by the inertial sensor is recorded as (α)Hip 2，βHip 2，γHip 2) The pose calculation module calculates the estimated hip joint movement angle of the lower limb of the wearer as (α)Hip 1，βHip 1，γHip 1) The pose solution module will (α)Hip 2，βHip 2，γHip 2) And (α)Hip 1，βHip 1，γHip 1) The method for short-time dynamic weighting data fusion comprises the following steps:
during a time t preceding the current movement instant, there are n hip-joint movement angles (α) acquired by the inertial sensorsHip 2，βHip 2，γHip 2) And the n posture resolving modules resolve the estimated hip joint movement angle of the lower limb of the wearer (α)Hip 1，βHip 1，γHip 1) Then, within time t, the reference value of the hip joint movement angleSatisfies the following conditions:
Figure RE-FDA0002325106050000041
wherein i is 1,2, 3.
Average value of measured deviation of hip joint movement angle
Figure RE-FDA0002325106050000042
Satisfies the following conditions:
Figure RE-FDA0002325106050000043
the mean square error of the measured deviation of the hip joint movement angle:
Figure RE-FDA0002325106050000044
wherein j is 1,2, σ1 hip jMeasuring the mean square deviation, sigma, of the deviation for hip flexion and extension motion angles2 hip jMeasuring the mean square error, sigma, of the deviation for the abduction and adduction angles of the hip joint3 hip jThe mean square error of the measured deviation of the external rotation and internal rotation angles of the hip joint is (sigma)1 hip 1，σ2 hip 1，σ3 hip 1) Calculating and estimating the mean square error of the hip joint motion angle measurement deviation for a pose calculation module, (sigma)1 hip 2，σ2 hip 2，σ3 hip 2) Measuring the mean square error of the deviation for the inertial sensor hip joint movement angle;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure RE-FDA0002325106050000051
wherein k is 1,2,3, and w is 11 hip jA short-time dynamic weighting fusion coefficient for measuring the flexion and extension movement angles of the hip joint, when k is 2, w2 hip jA short-time dynamic weighting fusion coefficient for measuring abduction and adduction movement angles of the hip joint, when k is 3, w3 hip jShort-time dynamic weighting fusion coefficients for measuring the hip joint external rotation and internal rotation motion angles;
hip joint motion angle after data fusion:
Figure RE-FDA0002325106050000052
8. the multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 1, wherein the flexion and extension angle of the knee joint is αKneeThe knee joint motion angle acquired by the inertial sensor is recorded as αKnee 2The pose resolving module resolves the estimated knee joint motion angle of the lower limb of the wearer to αKnee 1The pose resolving module will αKnee 2And αKnee 1The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the knee joint movement angle acquired by the n inertial sensors is αKnee 2The n posture calculation module calculates the estimated knee joint movement angle of the lower limb of the wearer to be αKnee 1Then, within the time t, the reference value of the knee joint movement angle
Figure RE-FDA0002325106050000053
Satisfies the following conditions:
Figure RE-FDA0002325106050000054
average value of measured deviation of knee joint movement angle
Figure RE-FDA0002325106050000055
Satisfies the following conditions:
Figure RE-FDA0002325106050000061
the mean square error of the measured deviation of the knee joint movement angle is as follows:
Figure RE-FDA0002325106050000062
wherein j is 1,2, σKnee jMeasuring the mean square error, sigma, of the deviation for knee joint movement anglesKnee 2For measuring the mean square deviation, sigma, of the deviation of the knee joint movement angle of the inertial sensorKnee 1Calculating the mean square error of the measurement deviation of the knee joint motion angle for a pose calculating module;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure RE-FDA0002325106050000063
wherein j is 1,2, wKnee jFor knee joint movementShort-time dynamic weighting fusion coefficients of angle measurement;
knee joint motion angle after data fusion:
αknee＝wKnee 1αKnee 1+wKnee 2αKnee 2。
9. The multi-sensor fusion-based lower limb movement pose rapid prediction system according to claim 1, wherein the plantar flexion and dorsiflexion angle of the ankle joint is αAnkle jointThe angle of the ankle joint varus and valgus is βAnkle jointThe internal rotation and external rotation angle of the ankle joint is gammaAnkle jointThe ankle joint movement angle acquired by the inertial sensor is recorded as (α)Ankle 2，βAnkle 2，γAnkle 2) The pose calculation module calculates the estimated motion angle of the ankle joint of the lower limb of the wearer to be (α)Ankle 1，βAnkle 1，γAnkle 1) The pose solution module will (α)Ankle 2，βAnkle 2，γAnkle 2) And (α)Ankle 1，βAnkle 1，γAnkle 1) The method for short-time dynamic weighting data fusion comprises the following steps:
in the previous time t of the current movement moment, the ankle movement angle acquired by the n inertial sensors is (α)Ankle 2，βAnkle 2，γAnkle 2) The n posture resolving module resolves the estimated movement angle of the lower limb ankle joint of the wearer (α)Ankle 1，βAnkle 1，γAnkle 1) Then, within the time t, the reference value of the ankle joint movement angle
Figure RE-FDA0002325106050000064
Satisfies the following conditions:
Figure RE-FDA0002325106050000071
average value of measured deviation of ankle joint movement angle
Figure RE-FDA0002325106050000072
Satisfies the following conditions:
Figure RE-FDA0002325106050000073
the mean square error of the measured deviation of the ankle joint movement angle is:
Figure RE-FDA0002325106050000074
wherein j is 1,2, σ1 ankle jMeasuring the mean square deviation, sigma, of the deviation for the plantar flexion and dorsiflexion motion angles of the ankle joint2 ankle jMeasuring the mean square error, sigma, of the deviation for the ankle inversion and eversion motion angles3 ankle jThe mean square error of the deviation is measured for the internal rotation and external rotation motion angles of the ankle joint (sigma)1 ankle 1，σ2 ankle 1，σ3 ankle 1) Calculating the mean square error (sigma) of the measurement deviation of the ankle joint motion angle for a pose calculation module1 ankle 2，σ2 ankle 2，σ3 ankle 2) Measuring the mean square error of deviation for the motion angle of the ankle joint of the inertial sensor;
and solving the short-time dynamic weighting fusion coefficient by using the following formula:
Figure RE-FDA0002325106050000081
wherein k is 1,2,3, and w is 11 ankle jA short-time dynamic weighting fusion coefficient is measured for the plantar flexion and dorsiflexion motion angles of the ankle joint, when k is 2, w2 ankle jThe short-time dynamic weighting fusion coefficient for measuring the ankle joint inversion and eversion motion angle is that when k is 3, w3 ankle jShort-time dynamic weighting fusion coefficients for ankle internal rotation and external rotation movement angle measurement;
ankle joint motion angle after data fusion:
Figure RE-FDA0002325106050000082
10. the method for quickly predicting the motion pose of the lower limbs based on multi-sensor fusion is characterized by comprising the following steps of:
(1) the wearer pastes surface electromyography sensors on the surfaces of lower limb rectus femoris, medial femoral muscle, lateral femoral muscle, biceps femoris muscle, semitendinosus muscle, tibialis anterior muscle, gastrocnemius muscle and soleus muscle;
(2) the external auxiliary device is worn by a wearer, wherein the waistline plate is worn on the waist of the wearer, the thigh outer side plate is worn on the outer side of the thigh of the wearer through a thigh fixing bandage, the shank outer side plate is worn on the outer side of the shank of the wearer through a shank fixing bandage, and the pedal plate is worn under the foot of the wearer through a pedal bandage; universal joints are adopted to connect the waist coaming and the thigh outer side plate, the thigh outer side plate and the shank outer side plate, and the shank outer side plate and the pedal plate;
(3) fixing an inertial sensor on the outer side of the front end of the waist coaming, the outer thigh side plate, the outer shank side plate and the outer surface of the pedal;
(4) the lower limbs of a wearer do any movement, and the surface myoelectric sensor and the inertial sensor acquire information;
(5) the surface electromyographic sensor transmits the surface electromyographic signals to the pose response module, and the pose response module matches the acquired time sequence of the electromyographic signals of the surfaces of the muscles with a database of the activity time sequence of the muscles of the lower limbs of the human body to find the motion state at the moment;
(6) the surface electromyographic sensor and the inertial sensor transmit the acquired signals to the pose resolving module, and the pose resolving module estimates the movement angle of the hip joint of the lower limb of the wearer through the surface electromyographic signals (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angle (α)Ankle 1，βAnkle 1，γAnkle 1) The hip joint motion angle collected by the inertial sensor is fused by short-time dynamic weighting (α)Hip 2，βHip 2，γHip 2) Knee joint movement angle αKnee 2Ankle joint motion angle (α)Ankle 2，βAnkle 2，γAnkle 2) Angle of motion with hip (α)Hip 1，βHip 1，γHip 1) Knee joint movement angle αKnee 1Ankle joint motion angleDegree (α)Ankle 1，βAnkle 1，γAnkle 1) Data fusion is carried out, and the hip joint motion angle (α) of the human body lower limb motion posture is obtained in real timeHip joint，βHip joint，γHip joint) Knee joint movement angle αKneeAnkle joint motion angle (α)Ankle joint，βAnkle joint，γAnkle joint) The position coordinates (x) of hip, knee and ankle joints are respectively calculated by the installation sizes of the waist coaming, the thigh outer side plate, the shank outer side plate and the foot pedal of the external auxiliary deviceHip joint,yHip joint,zHip joint)、(xKnee,yKnee,zKnee)、(xAnkle joint,yAnkle joint,zAnkle joint)。
CN201910804483.3A 2019-08-28 2019-08-28 Multi-sensor fusion-based lower limb movement pose rapid prediction system and method Active CN110755070B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910804483.3A CN110755070B (en) 2019-08-28 2019-08-28 Multi-sensor fusion-based lower limb movement pose rapid prediction system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910804483.3A CN110755070B (en) 2019-08-28 2019-08-28 Multi-sensor fusion-based lower limb movement pose rapid prediction system and method

Publications (2)

Publication Number Publication Date
CN110755070A true CN110755070A (en) 2020-02-07
CN110755070B CN110755070B (en) 2022-07-05

Family

ID=69329233

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910804483.3A Active CN110755070B (en) 2019-08-28 2019-08-28 Multi-sensor fusion-based lower limb movement pose rapid prediction system and method

Country Status (1)

Country Link
CN (1) CN110755070B (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111588376A (en) * 2020-05-07 2020-08-28 北京联合大学 Lower limb gait information extraction equipment based on electromyographic signals and angle signals
CN111632361A (en) * 2020-04-16 2020-09-08 北京邮电大学 Foot pronation detection method and device based on sole pressure
CN111643321A (en) * 2020-04-30 2020-09-11 北京精密机电控制设备研究所 Exoskeleton joint angle prediction method and system based on sEMG signals
CN111803250A (en) * 2020-07-03 2020-10-23 北京联合大学 Knee joint angle prediction method and system based on electromyographic signals and angle signals
CN111993391A (en) * 2020-08-25 2020-11-27 深圳市优必选科技股份有限公司 Robot pose estimation method and device, humanoid robot and storage medium
CN112107397A (en) * 2020-10-19 2020-12-22 中国科学技术大学 Myoelectric signal driven lower limb artificial limb continuous control system
CN112535474A (en) * 2020-11-11 2021-03-23 西安交通大学 Lower limb movement joint angle real-time prediction method based on similar rule search
CN112596534A (en) * 2020-12-04 2021-04-02 杭州未名信科科技有限公司 Gait training method and device for quadruped robot based on deep reinforcement learning, electronic equipment and medium
CN112704491A (en) * 2020-12-28 2021-04-27 华南理工大学 Lower limb gait prediction method based on attitude sensor and dynamic capture template data
CN112936224A (en) * 2021-01-27 2021-06-11 山东大学 Non-binding type easy-to-wear lower limb exoskeleton assistance system and use method
CN113080946A (en) * 2021-04-29 2021-07-09 合肥工业大学 Human body sitting and standing transfer capacity measuring device and method and electronic equipment
CN113520413A (en) * 2021-08-25 2021-10-22 长春工业大学 Lower limb multi-joint angle estimation method based on surface electromyogram signal
CN114983400A (en) * 2022-07-27 2022-09-02 南昌大学 Lower limb joint mobility monitoring system and monitoring method
CN116442202A (en) * 2023-06-19 2023-07-18 贵州航天控制技术有限公司 Waist boosting equipment control method based on back posture information
CN116509380A (en) * 2023-04-14 2023-08-01 江苏泰科博曼医疗器械有限公司 Knee joint coordinate establishment device, method and system
CN117281667A (en) * 2023-11-09 2023-12-26 浙江强脑科技有限公司 Motion pattern recognition method and device, intelligent artificial limb, terminal and storage medium

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110082394A1 (en) * 2009-10-07 2011-04-07 Industrial Technology Research Institute Method and system for monioring sport related fitness by estimating muscle power and joint force of limbs
US20110306846A1 (en) * 2010-06-15 2011-12-15 Ivan Osorio Systems approach to disease state and health assessment
US20140200414A1 (en) * 2010-06-15 2014-07-17 Flint Hills Scientific, L.L.C. Systems approach to comorbidity assessment
CN105213153A (en) * 2015-09-14 2016-01-06 西安交通大学 Based on the lower limb rehabilitation robot control method of brain flesh information impedance
CN105597298A (en) * 2016-04-05 2016-05-25 哈尔滨工业大学 Fitness effect evaluation system based on electromyographic signal and body movement detection
CN106821391A (en) * 2017-03-23 2017-06-13 北京精密机电控制设备研究所 Body gait acquisition analysis system and method based on inertial sensor information fusion
US20180193644A1 (en) * 2017-01-11 2018-07-12 Boston Scientific Neuromodulation Corporation Pain management based on muscle tension measurements
CN108283569A (en) * 2017-12-27 2018-07-17 北京精密机电控制设备研究所 A kind of exoskeleton robot control system and control method
CN109222969A (en) * 2018-10-31 2019-01-18 郑州大学 A kind of wearable human upper limb muscular movement fatigue detecting and training system based on Fusion
CN109589496A (en) * 2019-01-18 2019-04-09 刘坤 A kind of wearable bionical rehabilitation system of human motion overall process
CN109793500A (en) * 2019-01-24 2019-05-24 河南省人民医院 Knee joint load forces credit analysis apparatus

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110082394A1 (en) * 2009-10-07 2011-04-07 Industrial Technology Research Institute Method and system for monioring sport related fitness by estimating muscle power and joint force of limbs
US20110306846A1 (en) * 2010-06-15 2011-12-15 Ivan Osorio Systems approach to disease state and health assessment
US20140200414A1 (en) * 2010-06-15 2014-07-17 Flint Hills Scientific, L.L.C. Systems approach to comorbidity assessment
CN105213153A (en) * 2015-09-14 2016-01-06 西安交通大学 Based on the lower limb rehabilitation robot control method of brain flesh information impedance
CN105597298A (en) * 2016-04-05 2016-05-25 哈尔滨工业大学 Fitness effect evaluation system based on electromyographic signal and body movement detection
US20180193644A1 (en) * 2017-01-11 2018-07-12 Boston Scientific Neuromodulation Corporation Pain management based on muscle tension measurements
CN106821391A (en) * 2017-03-23 2017-06-13 北京精密机电控制设备研究所 Body gait acquisition analysis system and method based on inertial sensor information fusion
CN108283569A (en) * 2017-12-27 2018-07-17 北京精密机电控制设备研究所 A kind of exoskeleton robot control system and control method
CN109222969A (en) * 2018-10-31 2019-01-18 郑州大学 A kind of wearable human upper limb muscular movement fatigue detecting and training system based on Fusion
CN109589496A (en) * 2019-01-18 2019-04-09 刘坤 A kind of wearable bionical rehabilitation system of human motion overall process
CN109793500A (en) * 2019-01-24 2019-05-24 河南省人民医院 Knee joint load forces credit analysis apparatus

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
易润泽: "《基于多传感器的机器人遥操作人机交互系统》", 《测控技术》 *
李嫣然: "《一种基于表面肌电和惯性传感器的上肢运动功能评估方法》", 《中国生物医学工程联合学术年会》 *

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111632361A (en) * 2020-04-16 2020-09-08 北京邮电大学 Foot pronation detection method and device based on sole pressure
CN111643321A (en) * 2020-04-30 2020-09-11 北京精密机电控制设备研究所 Exoskeleton joint angle prediction method and system based on sEMG signals
CN111588376A (en) * 2020-05-07 2020-08-28 北京联合大学 Lower limb gait information extraction equipment based on electromyographic signals and angle signals
CN111803250A (en) * 2020-07-03 2020-10-23 北京联合大学 Knee joint angle prediction method and system based on electromyographic signals and angle signals
CN111993391A (en) * 2020-08-25 2020-11-27 深圳市优必选科技股份有限公司 Robot pose estimation method and device, humanoid robot and storage medium
CN112107397A (en) * 2020-10-19 2020-12-22 中国科学技术大学 Myoelectric signal driven lower limb artificial limb continuous control system
CN112107397B (en) * 2020-10-19 2021-08-24 中国科学技术大学 Myoelectric signal driven lower limb artificial limb continuous control system
CN112535474A (en) * 2020-11-11 2021-03-23 西安交通大学 Lower limb movement joint angle real-time prediction method based on similar rule search
CN112596534A (en) * 2020-12-04 2021-04-02 杭州未名信科科技有限公司 Gait training method and device for quadruped robot based on deep reinforcement learning, electronic equipment and medium
CN112704491A (en) * 2020-12-28 2021-04-27 华南理工大学 Lower limb gait prediction method based on attitude sensor and dynamic capture template data
CN112936224B (en) * 2021-01-27 2022-11-25 山东大学 Non-binding type easy-to-wear lower limb exoskeleton assistance system and use method
CN112936224A (en) * 2021-01-27 2021-06-11 山东大学 Non-binding type easy-to-wear lower limb exoskeleton assistance system and use method
CN113080946A (en) * 2021-04-29 2021-07-09 合肥工业大学 Human body sitting and standing transfer capacity measuring device and method and electronic equipment
CN113520413A (en) * 2021-08-25 2021-10-22 长春工业大学 Lower limb multi-joint angle estimation method based on surface electromyogram signal
CN114983400A (en) * 2022-07-27 2022-09-02 南昌大学 Lower limb joint mobility monitoring system and monitoring method
CN116509380A (en) * 2023-04-14 2023-08-01 江苏泰科博曼医疗器械有限公司 Knee joint coordinate establishment device, method and system
CN116509380B (en) * 2023-04-14 2023-12-29 江苏泰科博曼医疗器械有限公司 Knee joint coordinate establishment device, method and system
CN116442202A (en) * 2023-06-19 2023-07-18 贵州航天控制技术有限公司 Waist boosting equipment control method based on back posture information
CN116442202B (en) * 2023-06-19 2023-08-18 贵州航天控制技术有限公司 Waist boosting equipment control method based on back posture information
CN117281667A (en) * 2023-11-09 2023-12-26 浙江强脑科技有限公司 Motion pattern recognition method and device, intelligent artificial limb, terminal and storage medium
CN117281667B (en) * 2023-11-09 2024-04-09 浙江强脑科技有限公司 Motion pattern recognition method and device, intelligent artificial limb, terminal and storage medium

Also Published As

Publication number Publication date
CN110755070B (en) 2022-07-05

Similar Documents

Publication Publication Date Title
CN110755070B (en) Multi-sensor fusion-based lower limb movement pose rapid prediction system and method
Zhang et al. Ankle joint torque estimation using an EMG-driven neuromusculoskeletal model and an artificial neural network model
Zheng et al. Gait phase estimation based on noncontact capacitive sensing and adaptive oscillators
Xiong et al. Intelligent prediction of human lower extremity joint moment: an artificial neural network approach
Chen et al. Locomotion mode classification using a wearable capacitive sensing system
Wei et al. Synergy-based control of assistive lower-limb exoskeletons by skill transfer
WO2018050191A1 (en) A human intention detection system for motion assistance
Zhang et al. Lower-limb joint torque prediction using LSTM neural networks and transfer learning
Chinmilli et al. A review on wearable inertial tracking based human gait analysis and control strategies of lower-limb exoskeletons
Triloka et al. Neural computing for walking gait pattern identification based on multi-sensor data fusion of lower limb muscles
Liu et al. sEMG-based continuous estimation of knee joint angle using deep learning with convolutional neural network
Yi et al. Continuous prediction of lower-limb kinematics from multi-modal biomedical signals
Liang et al. Synergy-based knee angle estimation using kinematics of thigh
Wang et al. Prediction of contralateral lower-limb joint angles using vibroarthrography and surface electromyography signals in time-series network
Li et al. A human joint torque estimation method for elbow exoskeleton control
Tao et al. Multi-channel sEMG based human lower limb motion intention recognition method
Yahya et al. Intelligent integrated wearable sensing mechanism for vertical jump height prediction in female netball players
Chen et al. An adaptive gait learning strategy for lower limb exoskeleton robot
Farhat et al. Biomechanical model of the lower limb based on relevant actions for the control of knee-rehabilitation parallel robots
Li et al. Hip joint trajectory generation based on human limb motion synergy
Guo et al. A novel algorithm OF kinematics parameters measurement for upper limb based ON motion capture
CN110633644A (en) Human body joint angle prediction method based on electromyographic wavelet packet decomposition and GABP
Shen et al. Simultaneous Gait Event Intention Detection Using Single sEMG Sensor for Lower Limb Exoskeleton
Ascioglu et al. Prediction of lower extremity joint angles using neural networks for exoskeleton robotic leg
Kalani et al. SVM for Decoding the Human Activity Mode from sEMG Signals

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant