WO2010095636A1 - Method and device for estimating muscle tension - Google Patents

Abstract

The scale of optimization calculations for estimating muscle tension is reduced. A multiplicity of muscles is categorized into a multiplicity of muscle groups M_i (i = 1, 2, … n) based on muscle movement directionality or heteronymous muscle facilitation, one or a multiplicity of muscle subgroups is formed from the muscles in each muscle group originating or inserting on the same bone, and the level of muscle activity is treated as the same for muscles affiliated with the same subgroup. One of the aforementioned one or multiplicity of subgroups of some or all of the multiplicity of muscle groups M_i is a first subgroup, which is formed from a single representative muscle, to which an electromyograph has been attached, and the muscles originating or inserting on the same bone as said representative muscle. The number of variables in the optimization calculations is reduced by acquiring the muscle tension of the muscles affiliated with the first subgroup from the myoelectric potential of the aforementioned representative muscle measured during subject activity, or the number of variables in the optimization calculations is reduced by using optimization calculations to estimate the level of muscle activity representing each subgroup.

Description

Muscle tension estimation method and apparatus

The present invention relates to estimation of muscle tension and presentation of activity information inside the body based on the estimated muscle tension.

In order to analyze human movements, it is important to know the muscle tension pattern of muscles acting as actuators in the human body. For example, in order to estimate the muscle activity and the activity of the spinal nerve bundle that controls the muscle from the motion data, it is necessary to obtain the muscle tension from the motion data.
As a conventional muscle tension estimation method, two methods are typically known.

One solution is to model muscle contraction characteristics, such as the Hill-Stroeve muscle model (Non-patent document 1, Non-patent document 2), and calculate the parameters and muscle lengths obtained from physiology and experiments, and muscles calculated from the measured EMG. A biomechanical approach to calculate muscle tension from the degree of activity can be mentioned. Since myoelectric potential, which is a parameter that can be measured relatively easily and best represents muscle tension, is used, it can be said that the estimated value is the most accurate. There is also an advantage that the calculation cost is small and the calculation can be performed at high speed. However, in order to measure muscle tension, it is necessary to measure EMG, and it is necessary to attach electromyographs to all muscles that estimate muscle tension. Therefore, there is a limit to the number of muscles for which muscle tension is estimated from the limitation of the number of electromyographs. Even if countless electromyographs are available, it is impossible to attach them to all the muscles, and this restricts the behavior of the subject.

As another solution, there is a dynamic approach using inverse dynamics calculation and optimization calculation developed in the field of robotics. In this solution, by calculating each joint torque using a musculoskeletal model based on inverse kinematics and inverse dynamics, it is possible to obtain a solution that is dynamically consistent. In Patent Document 1, Non-Patent Document 3, and Non-Patent Document 4, joint dynamics are calculated using joint angles and floor reaction force meter data obtained by motion capture, and joint torque is calculated. The muscle tension is estimated by optimization calculation.
In this approach, the number of muscles that drive them with respect to the degree of freedom of the model is large, and it is not possible to uniquely determine muscle tension from joint torque only by inverse dynamics calculation, but one solution using optimization calculation It is necessary to focus on. In other words, the optimization calculation at the time of muscle tension distribution involves calculations such as linear programming and quadratic programming, and there is a problem that the calculation cost is large and a lot of time is required.
More accurate calculation of muscle tension in the whole body by taking into account the error term with muscle tension obtained from the muscle activity measured with the Hill-Stroeve muscle model and electromyograph in the objective function in the optimization calculation (Patent Document 1, Non-Patent Document 3, Non-Patent Document 4) However, the optimization calculation of muscle tension using inverse dynamics calculation is calculated when real-time estimation of muscle tension is considered. Cost becomes a problem.

Although various muscle tension estimation methods have been proposed in this way, a complete method has not yet been established, and each method has advantages and disadvantages. These muscle tension estimations are expected to be applied to training support and rehabilitation. If muscle tension is estimated in real time and presented in an easy-to-understand form, efficient exercise support can be achieved. Conceivable. However, the biomechanical method cannot effectively estimate the muscle tension of the whole body, and the method using the optimization calculation has a problem that the calculation takes time. Therefore, it is difficult to estimate the muscle tension of the whole body in real time using the conventional method.

International Publication Number WO2005-122900

The object of the present invention is to reduce the scale of the optimization calculation by reducing the variables in the optimization calculation for muscle tension estimation.

Another object of the present invention is to estimate the muscle tension during exercise in real time and to present the activity status inside the body acquired based on the muscle tension and / or muscle tension in real time.

The first technical means adopted by the present invention is:
In the method of estimating the muscle tension of each muscle by calculating each joint torque during exercise of the subject by inverse dynamics calculation using a musculoskeletal model and distributing the joint torque to the muscle tension by optimization calculation,
Classifying a plurality of muscles of a subject into a plurality of muscle groups M _i (i = 1, 2,... N) based on muscle movement directivity or alias muscle facilitation;
In each muscle group of a plurality of muscle groups M _i (i = 1, 2,... N), bones that start and stop form one or more subgroups from the same muscle, and belong to the same subgroup. Are considered to be the same muscle activity,
In some or all of the plurality of muscle groups M _i, at least one, one representative muscle and the representative muscle and bones origin stop electromyograph is mounted in one of said one or more sub-groups Is a first subgroup formed from muscles that are the same,
The muscle tension of the muscles belonging to the first subgroup is obtained from the myoelectric potential of the representative muscle measured during the exercise of the subject without using the optimization calculation, and is excluded from the optimization calculation target, thereby performing the optimization calculation. Reduce the variables in
Or
In the one or a plurality of subgroups, the muscle activity representing each subgroup is estimated by the optimization calculation, thereby reducing variables in the optimization calculation.
This is an estimation method of muscle tension.
In the first technical means, the number n of the number of channels and muscle groups M _i of electromyograph is not necessarily the same.
In the first technical means, in the muscle group in which the first group is formed, typically one first group is formed per one muscle group, but two or more representative muscles in one muscle group. May be selected to form two or more first groups.

In one aspect, the muscle tension of the muscle belonging to the first subgroup is acquired from the myoelectric potential of the representative muscle measured during exercise of the subject,
Optimizing muscle tension of muscles not belonging to the first subgroup so as to realize joint torque that cannot be realized by muscles belonging to the first subgroup in joint torque necessary to realize exercise of the subject; Estimated by

In one aspect, in each muscle group M _i , the one or more subgroups are classified into the first subgroup and the muscles that do not belong to the first subgroup from the same muscle whose start and stop bones are the same. Zero or more subgroups, and
The muscle activity representing each subgroup is estimated by optimization calculation.
Furthermore, in one aspect, for the muscles belonging to the first group, the muscle activity obtained from the myoelectric potential should match the muscle activity calculated by the optimization calculation.
In the optimization calculation, the measured muscle activity of the representative muscle is used as a reference value.

In this specification, “muscle grouping based on muscle movement directivity” means, in other words, “muscle grouping related to each movement direction of each joint” or “based on the direction of torque that the muscle is involved in each joint. It can be called "grouping". These represent the geometrical relationship between movement and muscle placement. Specifically, it means, for example, “muscle related to elbow joint extension” and “muscle related to hip joint flexion”.
That is, the grouping of the muscle group M _i is a classification according to the role of muscles involved in the direction of motion of each joint as such extend the elbow joint (extension) muscle for, bending the elbow (bend) muscle for . By performing the grouping into muscle group M _i, in the grouping of muscle by only bones origin stopped, it is prevented that the muscle to stretch the muscle and joint for bending joints from being classified in the same group The
The muscle groups M _i of a joint contributes to a direction of bending the (contributing joints identical) muscle, a plurality of streaks of bones origin stop is different belongs, by bone to further origin stop this Classify into subgroups. By performing the grouping into muscle group M _i, further it is possible to classify into the lower sub-groups based on the joint to strictly involved.
In addition, the grouping of the muscle group M _i is, the multi-joint muscle of influence ignored the movement directed by muscle classification, grouping of the sub-group is also considering the influence of the multi-joint muscle, to directly exercise when the muscle is contracted It can be said that the muscles of the affected joints are classified into the same group.
“Grouping based on alias muscle facilitation” represents muscles controlled by the same nerve bundle as a group.
When grouping muscles, there are two approaches from the mechanical and neurophysiological aspects. The former is determined from the running of the muscle, and the latter is determined from the connection between the muscles by nerves. It has been shown that there is a relationship between these two in the field of sports science.
It can be said that the grouping into subgroups is “classified into cooperative muscles based on kinematics”. The “cooperative muscle” can be defined as “a muscle that works to bend a certain joint in the same direction as the main muscle”. “Muscle directionality” and “cooperative muscle” are both thoughts from the viewpoint of kinematics, and as mentioned above, “sympathetic muscle facilitation” is an idea from the viewpoint of neurophysiology.
In one aspect, muscle grouping is performed in accordance with “muscle directionality” and “cooperative muscle”. For example, even with respect to flexion and extension, it may be an antagonistic muscle in adduction and abduction, even if it is a cooperative muscle. If necessary, it is desirable to confirm the grouping by “unknown muscle facilitation”.
In addition, the classification of the muscle group M _i of cooperative muscle has been known to change by the displacement of the body posture and joint. Although the classification can be treated as a static one that does not change approximately, the classification can be changed dynamically according to the posture of the body or the displacement of the joint in order to further improve the accuracy.

The second technical means adopted by the present invention is as a method invention,
In the method of estimating the muscle tension of each muscle by calculating each joint torque during exercise of the subject by inverse dynamics calculation using a musculoskeletal model and distributing the joint torque to the muscle tension by optimization calculation,
Classifying a plurality of muscles of a subject into a plurality of muscle groups M _i (i = 1, 2,... N) based on muscle movement directivity or alias muscle facilitation;
In each muscle group of a plurality of muscle groups M _i (i = 1, 2,... N), bones that start and stop form one or more subgroups from the same muscle, and belong to the same subgroup. Are considered to be the same muscle activity,
In the one or a plurality of subgroups, the muscle activity representing each subgroup is estimated by the optimization calculation, thereby reducing variables in the optimization calculation.
Muscle tension estimation method,

The second technical means is an invention of the device,
Each joint torque during exercise of the subject is calculated by inverse dynamics calculation using a musculoskeletal model, and the muscle tension of each muscle is estimated by distributing the joint torque to the muscle tension by optimization calculation. Means,
A grouping means for classifying a plurality of muscles of a subject;
With
The grouping means includes
First grouping means for classifying the plurality of muscles into a plurality of muscle groups M _i (i = 1, 2,... N) based on muscle movement directivity or alias muscle facilitation;
In each muscle group of the plurality of muscle groups M _i (i = 1, 2,... N), second grouping means in which the bones that start and stop form one or more subgroups from the same muscle; Become
The muscle tension acquisition means regards the muscle activity level of muscles belonging to the same subgroup as the same, and estimates the muscle activity level representing each subgroup in the one or more subgroups by optimization calculation.
An apparatus for estimating muscle tension.

In the second technical means, in one aspect, in some or all of the plurality of muscle groups M _i, at least one of the one or more sub-groups, one of electromyograph is mounted A first subgroup formed of a representative muscle and a muscle having the same bone that starts and stops;
For muscles belonging to the first group, it is assumed that the muscle activity obtained from the myoelectric potential should match the muscle activity calculated by the optimization calculation, and the muscle of the representative muscle measured in the optimization calculation Use activity as a reference value.

The third technical means adopted by the present invention is a method invention.
At least a part of a plurality of muscles of the subject's whole body or a part of the body is divided into a plurality of muscle groups M _i (i = 1, 2,... N) based on the movement direction of the muscles or the aliasing muscle facilitation. classifying the electromyograph attached to the representative muscle by selecting one representative muscles from each muscle group M _i,
The plurality of muscles,
A first incision group _{M iEMG} comprising the representative muscle of each muscle group _{M i,}
In each muscle group _{M i,} bones and origin stopped first muscle group _{M iEMG} are the same (contributing joints are exactly the same) and the second incision group _{M Ihigh} consisting muscle,
A first muscle group M _iEMG , a third muscle group consisting of muscles not included in the second muscle group M _high ;
Divided into
_Obtaining the muscle tension of the muscles belonging to the first muscle group _MiEMG and the second muscle group _Mhigh from the myoelectric potential of the representative muscle measured during the exercise of the subject;
The muscle tension of the muscles belonging to the third muscle group is calculated by inverse dynamics calculation to calculate the joint torque necessary to realize the exercise of the subject, and in the joint torque, the first muscle group _MiEMG and Estimating by optimizing to realize joint torque that cannot be realized by muscles belonging to the second muscle group M _high ,
This is an estimation method of muscle tension.

The third technical means is the invention of the device,
Multiple electromyographs,
First muscle tension acquisition means for acquiring muscle tension from myoelectric potential;
Second muscle tension acquisition means for calculating joint torque by inverse dynamics calculation and estimating muscle tension by performing optimization calculation so as to realize the joint torque;
A grouping means for classifying a plurality of muscles of a subject;
With
The grouping means includes a first grouping means and a second grouping means,
The first grouping means classifies the plurality of muscles into a plurality of muscle groups M _i (i = 1, 2,... N) based on muscle movement directivity or alias muscle facilitation. each electromyograph is attached to one representative muscle selected from the muscle group M _i,
The second grouping means includes the plurality of lines,
A first incision group _{M iEMG} comprising the representative muscle of each muscle group _{M i,}
In each muscle group _{M i,} and a second incision group _{M Ihigh} consisting muscle is a bone to the origin stop first muscle group _{M iEMG} the same,
A first muscle group M _iEMG , a third muscle group consisting of muscles not included in the second muscle group M _high ;
Divided into
The first muscle tension acquisition means acquires the muscle tension of the muscles belonging to the first muscle group _MiEMG and the second muscle group _Mhigh from the myoelectric potential of the representative muscle measured during the exercise of the subject,
The second muscle tension acquisition means calculates the muscle torque of the muscles belonging to the third muscle group, and calculates the joint torque necessary for realizing the movement of the subject measured by inverse dynamics calculation. Estimating by optimizing to realize joint torque that cannot be realized by the muscles belonging to the first muscle group _MiEMG and the second muscle group _Mihigh ,
Muscle tension estimation device,
It is.

The third technical means has several modes.
In one embodiment, in the plurality of muscle, the muscle that does not belong to any muscle group _{M i} and muscle groups _{M others,}
Wherein the third muscle group, in each muscle group _{M i,} contained the first muscle group _{M iEMG,} and the second muscle group _M are not included in _ihigh muscle groups _{M Ilow,} and the muscles _{M others,} is .
In one embodiment, a plurality of streaks, all muscle are classified as belonging to one of the muscle groups M _i,
Wherein the third muscle group, in each muscle group _{M i,} the first muscle group _{M iEMG,} include muscle which is not included in the second muscle group _{M Ihigh.}
In one aspect, in each muscle group M _i , a plurality of muscles belonging to the muscle group M _high do not antagonize. In addition, regarding multi-joint muscles, in one aspect, they are grouped so as to include mainly the joints involved.
In one aspect, the muscle tension of the muscles included in the first muscle group _MiEMG and the second muscle group _Mihigh is determined by measuring the muscle activity obtained from myoelectric potential data, muscle parameters obtained by empirical rules, and measurement. It is acquired from the muscle length obtained by the inverse kinematics calculation based on the exercise data and the change speed of the muscle length.
In one aspect, in each muscle group M _i , the myoelectric potential of the muscles included in the second muscle group M _high is determined as a function of the myoelectric potential of the representative muscle.
Typically, the myoelectric potential of the muscle included in the second muscle group M _high is regarded as the same as that of the representative muscle, but the myoelectric potential of the second muscle group M _{high is} the muscle of the first muscle group _Mi EMG. It does not have to be exactly the same as the electric potential, and can be determined by a function obtained from the geometric position, posture, etc. of these muscles.
In one embodiment, the number n of the number of channels and muscle groups M _i of electromyograph are the same.
If the number n of the number of channels and muscle groups M _i of electromyograph do not match, for example, when the number of channels of the myoelectric potential is small, sets the M _iEMG for any group of muscle groups M _i, myoelectric Wear a meter. Regarding the muscle group M _i for which Mi _EMG is not set, all the muscles belonging to the muscle group M _i become M _ilow , and consequently the muscles belonging to the third muscle group. That, not M _iEMG can be determined in all M _i from restrictions such as the number of channels of EMG, all muscle case M _i will belong to the third muscle group.

In one aspect,
The optimization calculation is
Assuming that the plurality of muscle groups drive the skeleton, a first step of estimating for each muscle group the joint torque to be output by each muscle group;
A second step for estimating the muscle tension output by each muscle so as to realize the joint torque estimated in the first step in each muscle group;
Consists of.
The idea of reducing the size of the optimization calculation problem by the number of muscles that can be solved using EMG information is also advantageous when obtaining some muscle tensions. At this time, grouping is used to determine the line that EMG solves, but grouping is also useful for reducing the dimension of the line that is solved by the remaining optimization calculations.
There are various optimization methods, one of which has a high calculation cost and satisfies the constraint condition accurately, and the other of which is a low calculation cost and satisfies the constraint condition only vaguely. The constraint condition here means an inequality constraint condition in which the muscle does not exert a force in the extending direction. However, even in the latter case, an accurate solution can be obtained even if the constraint condition is only ambiguous if the relationship between the muscles considered in the optimization is not included.
Therefore, by first grouping muscles, assuming that a plurality of muscle groups drive the skeleton, a torque to be output by each muscle group is obtained by the former method with high calculation cost.
And the muscle tension which each muscle outputs can be calculated | required with the method with the latter calculation cost small so that those torques may be implement | achieved in each muscle group.
Roughly speaking, the muscle dimension means the number of muscles. More specifically, the muscle dimension refers to the number of independent variables in determining the muscle tension throughout the body. If the tension is determined independently for each muscle, the total number of muscles becomes a dimension, which in this case becomes a large-scale optimization problem. If the muscles are grouped and the distribution rule of the tension within the group is determined, the independent variable in determining the muscle tension of the whole body is the number of groups. The muscle dimension in this case is the number of muscle groups.

In one aspect, muscle tension is estimated in real time as the subject exercises. The muscle tension estimation device is a real-time muscle tension estimation device that estimates muscle tension in real time during exercise of a subject.

The fourth technical means adopted by the present invention is a method invention.
The subject's captured image or / and a composite image based on the captured image are displayed on the display unit, and a musculoskeletal model is overlaid on the displayed subject's image,
The activity information inside the body based on the muscle tension obtained by the above estimation method is reflected in the musculoskeletal model and displayed visually.
It is a method for presenting activity information inside the body.
The fourth technical means is the invention of the device.
A real-time muscle tension estimation device;
Means for photographing the subject during exercise;
Display means for displaying a photographed image of the subject or / and a composite image based on the photographed image;
With
The musculoskeletal model is overlaid on the image of the subject displayed on the display means, and the activity information inside the body based on the muscular tension estimated in real time by the real-time muscular tension estimation device is reflected in the musculoskeletal model to visually Configured to display real-time,
A device for presenting activity information inside the body.
In one aspect, internal body activity information is displayed in real time as the subject exercises.

In one aspect, the activity information inside the body is muscle activity.
In one aspect, muscle activity is visually displayed by a change in muscle color or / and shape of the musculoskeletal model.

In one aspect, the activity information inside the body represents muscle activity as activity of a spinal nerve bundle that governs the muscle activity. The spinal nerves control a plurality of muscles, and the activity of all the muscles that control them is calculated as the activity of the spinal nerve bundle.
In one aspect, spinal nerve bundle activity is displayed visually by changing the color or / and shape of the symbol at each spinal nerve bundle location on the musculoskeletal model.

In the present invention, by grouping a plurality of muscles of the subject, the variables in the optimization calculation for muscle tension estimation are reduced, and the scale of the optimization calculation is reduced.
By reducing the scale of the optimization calculation, muscle tension can be estimated in real time, and muscle activity based on muscle tension can be visualized in real time.
According to the present invention, the scale of optimization calculation can be reduced by EMG information (used together with the Hill-Strove model) and grouping.
Furthermore, while using a limited number of electromyographs, the muscle tension of the whole body can be estimated in real time by combining optimization calculation methods with a comparatively small calculation amount.
Therefore, somatosensory information can be presented during sports training or rehabilitation. By presenting somatosensory information centered on muscle tension and joint load in real time, it is possible to construct a system that can replace the trainer.
By estimating the activity of the human muscle and the spinal nerve bundle that controls it from the movement data at high speed and overlaying it on the person's video in real time, the activity status inside the body is shown. Intuitive understanding. You can intuitively see your own exercise and the activities inside the body at that time, and you can confirm the effect of the exercise.
The effects of exercise can be confirmed in sports training, rehabilitation, medical diagnosis, health management, entertainment, and the like.

It is a figure which shows the relationship between muscle length and the maximum isometric strength. It is a figure which shows the relationship between the change rate of muscle length, and the maximum muscle tension. It is a figure which shows the musculoskeletal model of a human body. It is a figure which shows the flow of muscle tension estimation by the conventional optimization calculation. It is the schematic of a real-time muscle tension visualization system. It is a schematic flow diagram of a real-time muscle tension visualization system. It is the schematic of the delivery of each thread | sled and data in a real-time muscle tension visualization system. FIG. 5 shows a real-time overlay of a musculoskeletal model and estimated muscle tension information on a subject image during exercise of the subject. The upper figure shows squats, and the lower figure shows pitching motion. Actually, when the muscle tension increases, the color of the muscle changes from yellow to red. It is a figure which shows the squat of FIG. 6 in the time series of several frames. It is a figure which shows the pitching operation | movement of FIG. 6 in the time series of several frames. It is a figure which shows a spinal nerve bundle | flux with a symbol on the displayed musculoskeletal model. In the image, spheres arranged in four directions for each spinal cord symbolize the spinal nerve bundle, and each symbol changes to red when the corresponding spinal nerve bundle is activated. It is a figure which shows the grouping of the line | wire which concerns on the 1st Embodiment of this invention. It is a figure which shows the grouping of the line | wire which concerns on the 2nd Embodiment of this invention.

Embodiments of the present invention will be described in detail. First, the concept and technique that are the background of the present invention will be described. These concepts and techniques are not only the background art of the present invention, but also the techniques that can be used to implement the embodiments of the present invention. Next, an embodiment of muscle tension estimation according to the present invention will be described. In addition, about the number of a numerical formula, it assign | provides independently for every section.

[A] Hill-Stroeve Muscle Model In the Stroeve muscle model formulated from Hill and Wilkie muscle models, the relationship between muscle length and maximum isometric muscle strength is expressed as shown in FIG. The relationship between the change rate of the muscle length and the maximum muscle tension is expressed as shown in FIG. At the maximum contraction speed _vmax , the muscle tension is 0, and the maximum muscle tension when the muscle length does not change corresponds to the maximum isometric muscle strength. In addition, when a force greater than the maximum isometric muscle force is applied, the muscle stretches.

The relationship between the muscle activity level a and the muscle tension f ^* is expressed by equation (1).

Here, F _max is the maximum muscle tension, and F _l (l) and F _v (l (dot)) are functions representing the relationship between the normalized muscle tension, the muscle length, and the rate of change of the muscle length, respectively. F _l (l) corresponds to FIG. 1 and is approximated by a Gaussian function of Equation (2).

Here, l ₀ is the natural length of the muscle. F _v (l (dot)) corresponds to FIG. 2 and is approximated by equation (3).

Here, K _l , V _sh , V _shl , and V _ml are constants, and in one embodiment, values shown in Stroeve (Table 1) are used. These values may be identified based on the motion capture data.

Data necessary for estimating muscle tension using the Hill-Stroeve muscle model includes muscle length, muscle length change rate, and muscle activity. When the muscle length, the muscle length change rate, and the muscle activity of all muscles are obtained, the muscle tension f ^* of each muscle is expressed by the following expression.

Here, a _i , l _i , l (dot) _i , and F _max represent the activity, muscle length, muscle activity, and maximum muscle tension of the i-th muscle, respectively, and F _l and F _v are normalized, respectively. It is a function showing the relationship between the made muscle tension, muscle length, and muscle length change speed. Nmus represents the total number of muscles included in the musculoskeletal model. Muscle length l ₁ ,... L _Nmus and muscle length change speed l (dot) ₁ ,... L (dot) _Nmus can all be calculated from motion data obtained from motion capture.

Regarding muscle activity, models for muscle tension estimation using myoelectric potentials have been devised at various parts of the body such as the trunk, elbows, shoulders, knees, and ankles. These models are described in the following documents, for example.

SM
McGill, A myoelectrically based dynamic three-dimensional model to predict
loads on lumbar spine tissues during lateral bending, Journal of Biomechanics,
Vol. 25, pp. 395-414, 1992.

TS Buchanan,
SL Delp, and JA Solbeck, muscular resistance to varus and valgus loads at
the elbow, Journal of Biomechanics, Vol. 120, pp. 634-639, 1998.

B.
Laursen, B. Jenson, G. Nemeth, and G. Sjogaard, A model predicting individual
shoulder muscle forces based on relationship between electromyographic and 3D
external forces in static position, Journal of Biomechanics, Vol. 31, pp.
731-739, 1998.

David G.
Lloyd, Thomas S. Buchanan, and Thor F. Besier, Neuromuscular Biomechanical Modeling
to Understand Knee Ligament Loading, Medicine & Science in Sports & Exercise,
Vol. 37, pp. 1939-1947, 2005.

AL Hof
and JW van den Berg, EMG-to-force processing. II. Estimation of parameters of
the Hill Muscle model for the human triceps surae by means of calf ergometer,
Journal of Biomechanics, Vol. 14, pp. 759770, 1981.

In these models, the myoelectric potential of multiple muscles for each joint was measured with an electromyograph, and the force expressed as% MVC (Maximal Voluntary Contraction (MVC)). Etc.) is used as the activity level of the muscle.

[B] Muscle tension calculation using inverse dynamics / optimization calculation The method of inverse dynamics calculation in the whole body detailed musculoskeletal model is shown (Patent Document 1, Non-Patent Document 3, Non-Patent Document 4).

[B-1] Musculoskeletal model A detailed whole body musculoskeletal model used in the embodiment of the present invention will be described. As shown in FIG. 1, the designed detailed human body model is composed of a skeletal rigid body model grouped with appropriate fineness and a muscle / tendon / ligament wire model stretched on the skeleton. The skeletal model consists of 206 bones throughout the body. Of these, the skull, hand, and toe are treated as a single rigid body, and the model is composed of a total of 53 links. Between each link is a spherical 3 degrees of freedom joint, except for the tarsal bone-toe toe rotation 1 free joint and the first thoracic vertebra 6 breast joint. The skeletal model has a total of 155 degrees of freedom, adding 6 degrees of freedom for the entire translational rotation.

Next, muscles, tendons, and ligaments are placed in the skeletal model. Muscles, tendons, and ligaments are modeled as wires that pass through the start point, end point, and waypoint at each link. Bones, muscles, tendons and ligaments have the following properties.
Bone: A rigid link with mass. Muscle: A wire that actively generates tension.
Tendon: A passively tensioning wire that connects to muscles and transmits muscle tension to bone.
Ligaments: Passive tension wires that connect bones and constrain their relative movement.
Differences in muscle, tendon, and ligament functions are modeled as follows.
A simple part consisting of a series connection of muscles and tendons is represented by a single muscle wire.
If the muscle is caught in a part of the bone or if you want to model the tendon restraint by the tendon sheath, place a via point.
There is a case where tendons such as the upper arm bilateral muscle branch and the branched tendons connect to different bones. Since the start point, end point, and waypoint of the wire are all fixed to the link, a virtual link is placed at this branch point. The virtual link has no mass but transmits tension. The virtual link can move freely so that the force and moment are zero.
Wide muscles such as the great pectoral muscle and latissimus dorsi are expressed by a plurality of parallel muscle wires.
Such a musculoskeletal model is also described in Patent Document 1, for example, and can be referred to.
The above-mentioned musculoskeletal model is merely an example, and the musculoskeletal model that can be applied to the present invention is not limited to these.

[B-2] Acquisition of Muscle Tension Using Musculoskeletal Model Acquisition of muscle tension using the musculoskeletal model will be described. In one aspect, the device for acquiring muscle tension includes a plurality of imaging means (camera) for imaging a subject to which a marker is attached, a floor reaction force measuring means (force plate), and an electromyograph means (myoelectric meter). The computer apparatus includes an arithmetic processing unit that performs various calculations, an input unit, an output unit, a display unit, and a storage unit that stores various data. Here, motion capture data (exercise data), myoelectric potential, and floor reaction force are simultaneously measured and used for optimizing muscle strength, thereby obtaining appropriate muscle strength both mechanically and physiologically.

The muscle tension calculation of the whole body detailed musculoskeletal model will be described.
In the methods disclosed in Patent Document 1, Non-Patent Documents 3 and 4, muscle tension is calculated as follows.
(1) The motion of the subject is measured by the motion capture system, and time-series data of the three-dimensional position of the marker is obtained.
(2) The motion information including the joint angle, the joint angular velocity, and the joint angular acceleration is calculated from the three-dimensional position of the marker by inverse kinematics calculation.
(3) The joint torque required to realize the motion is calculated by inverse dynamics calculation using the Newton oiler method or the like.
(4) The joint torque obtained in (3) is mapped to the floor reaction force and the tension of the muscle, tendon, and ligament using the relationship between the muscle, tendon, ligament length change obtained from the joint angle and each joint angular velocity.

In inverse dynamics, the tension of muscles, tendons, and ligaments that realize the movement is obtained based on the movement data obtained by the movement measurement. The flow of inverse dynamics calculation is as follows: 1. Calculation of joint torque by inverse dynamics of rigid link system; 2. Calculation of Jacobian for wire length joint value; The joint torque is converted into wire tension.
By using the inverse dynamics calculation of the rigid link system, it is possible to calculate the joint torque τ _g necessary for realizing the motion in the skeleton model. Using D'Alembert's principle and virtual work's principle, muscle, tendon, and ligament tension f equivalent to τ _g is obtained by using Jacobian J of muscle, tendon, and ligament length l for joint angle θ _g .

It is expressed.
Since the calculation method of Jacobian J is well known to those skilled in the art, detailed description thereof is omitted here for the purpose of avoiding complicated description. As for the calculation method of Jacobian J, for example, JP 2003-339673 or “DE Orin and WW Schrader. Efficient computation of the jacobian”
for robot manipulators. Inter-national Journal of Robotics Research, Vol. 3,
No. 4, pp. 66.75, 1984 ”can be referred to.

While the joint angle vector theta _G is a 155-dimensional in the present embodiment, the wire tension f Non-patent document 3, in the non-patent document 4 models a 989-dimensional. Therefore, there arises a redundancy problem in which f is not uniquely determined from τ _g . Here, in the inverse dynamics calculation of the musculoskeletal model, there is an undetermined problem that the number of elements of muscles, tendons, and ligaments is very large with respect to the parameters that determine movement, and the force cannot be determined uniquely. It is well known to those skilled in the art, and the joint moment obtained by the inverse dynamics calculation is distributed to the muscle tension of the muscle that drives each joint by the optimization calculation.

A method of setting some evaluation function and constraint conditions in order to determine f and solving using optimization by mathematical programming or the like is disclosed in, for example, Patent Document 1 and Non-Patent Documents 3 and 4. Specific examples of optimization calculation are shown below. However, it is known to those skilled in the art that several methods have been proposed as optimization calculation used for muscle tension calculation, and optimization calculation that can be applied to the present invention. Those skilled in the art will appreciate that is not limited to what is described herein.

One aspect of inverse dynamics calculation in the whole body detailed musculoskeletal model will be described. By solving the following equations, the contact force with the floor and the muscle tension of the whole body are calculated.

here,
τ _G : Generalization force;
J: Jacobian matrix from generalized coordinates to wire length;
f: wire tension;
J _C : Jacobian matrix from the generalized coordinates to the point of contact with the floor;
τ _C : contact force with the floor;
It is.

Equation (4) is solved by the following flow.
The contact force τ _C with the floor is calculated considering only the row corresponding to 6DOF of the hip joint in the equation (4). Here, optimization is performed using quadratic programming.
By removing τ _C from equation (4), the following equation is obtained.

Here, muscle tension is calculated using linear programming or quadratic programming.
The contact force with the floor is a typical example of the contact force with the outside world. For example, the contact force with a wall other than the floor can be used. Such muscle tension estimation by subtracting the contact force with the outside world is disclosed in Patent Document 1, Non-Patent Document 3, and Non-Patent Document 4.

Hereinafter, a method for obtaining the joint torque τ _G ′ considering the optimization of the contact force τ _C will be described. Complex optimization to determine muscle tension can be solved with quadratic programming, or more simply with linear programming. Here, a method using linear programming as a fast solution is considered. However, as a problem specific to linear programming, the resulting muscle tension data is discontinuous in the time and space directions. Spatial discontinuities indicate that muscle tensions differ greatly between muscles that are close to geometry, which is unnatural in the human body.

This problem is solved by smoothing the muscle tension within the cooperative muscle group. Using the following equation, the optimization vector a _tau each element is positive, a _f, with respect to a _m, [delta] _tau minimizes the objective function is the following formula, where [delta] _f, obtaining the [delta] _m, f Solve the problem using linear programming.

Here, the constraint condition can be written as follows.

The third term of equation (6) is added to smooth muscle tension. Equations (6) to (13) will be described below.

The first term of Equation (6), Equation (7), and Equation (8) are intended to minimize the error in Equation (5) and ensure dynamic consistency. Equation (5) can be written in the form of an equation, but the condition is relaxed in consideration of the case where equation (5) has no solution. Since the objective function formula (6) includes a _τ ^T δ _τ , the minimum δ _τ positively constrained by the formula (8) is obtained. On the other hand, by the equation (7) is made smaller than the [delta] _tau error of formula (5). Considering these constraint conditions, the error in equation (5) can be minimized.
The second term of Equation (6), Equation (9), and Equation (10) have the effect of bringing f closer to the given target value f ^* . For example, by giving an appropriate value of f ^* , it is possible to uniquely determine the relationship of the muscular strength between the flexors and extensors. It is conceivable to use a measurement value of an electromyograph as a biomechanical application (Patent Document 1, Non-Patent Document 3, Non-Patent Document 4). When f ^* = 0, the minimum muscle / tendon / ligament tension can be obtained.

Equation (11) is the upper limit and lower limit constraints of muscle / tendon / ligament tension, and f _max ≧ 0 represents the maximum muscle tension. The maximum muscle tension f _max can be determined independently for each muscle. Using the Hill-Stroeve muscle model, f _max can be calculated from the muscle length and its rate of change to give the maximum muscle tension.

Finally, the third term of Equation (6), Equation (12), and Equation (13) have the effect of smoothing the muscle tension as much as possible within the cooperative muscle group. This effect can be realized by adding a term of sum of squares of muscle tension to the objective function in quadratic programming. Consider the n _m made from this muscle m-th cooperative muscles G _m.
The average value of muscle tension in this cooperative muscle group is

Is calculated by Here, f _k represents the muscle tension of the k-th muscle. The difference between the muscle tension of the k (k∈Gm) th muscle and the average muscle tension in the cooperative muscle group that contains it is

Can be expressed as Where _EGmk is the i-th element

Is a row vector. By arranging the E _Gmk (k∈G _m) for all cooperating muscles, E _G shown in Equation (12) is obtained.

In the above example, the optimization method using linear programming has been described. Next, one aspect of muscle tension optimization using quadratic programming will be described. Quadratic programming with inequality constraints `` M. Renouf and P. Alart. Conjugate gradient type algorithms forfricional
multi-contact problems: Applications to granular materials.Vol. 194, pp. 2019.2041,
2005 ", the evaluation function Z is

Then, f that minimizes Z is obtained. K ₁ and K ₂ are weights. τ _G ′ is a generalization force obtained by subtracting J _c ^T τ _c from τ _G.
Thereby, muscle tension can be approximated to a measured value. In addition, (τ _G ′ −J ^T f) is also small, so that a muscle tension that is mechanically reasonable can be calculated.

[C] Real-time whole body muscle tension estimation method [C-1] Real-time muscle tension visualization system FIG. 5 shows a schematic diagram of a real-time muscle tension visualization system. The muscle tension visualization system includes a muscle tension estimation means, a means for acquiring activity information inside the body using the estimated muscle tension, an image of a subject taken during exercise and an estimated muscle tension / acquired body. Means for displaying internal activity information. More specifically, the muscle tension visualization system displays a plurality of imaging means (camera 1) for photographing a subject having a plurality of markers attached to a plurality of predetermined parts of the body, and displays the subject in motion on the display means. Photographing means (DV camera 2) for photographing, ground reaction force measuring means (force plate 3), myoelectric meter means (wireless myoelectric meter 4) such as myoelectric meter, and one or more computer devices 5 and display means (screen 6). The computer device includes an arithmetic processing unit that performs various calculations, an input unit, an output unit, a display unit, and a storage unit that stores various data. By simultaneously measuring motion capture data (exercise data), myoelectric potential, and floor reaction force, and using them in optimizing muscle force, a muscle tension that is appropriate mechanically and physiologically is acquired.

FIG. 5A shows a flow diagram of the real-time muscle tension visualization system. Human body motion data and floor reaction force are measured using an optical motion capture and force plate, and myoelectric potential data is measured using a wireless electromyograph. Each data is acquired in real time in synchronization with the system control PC. Thereafter, by solving inverse kinetic calculation (IK) of the musculoskeletal model from the motion data, the joint angle, the muscle length, and the muscle length change speed can be obtained. In this manner, information necessary for acquiring muscle tension of the muscle based on the Hill-Stroeve muscle model and estimating muscle tension using inverse dynamics calculation (ID) and optimization calculation is acquired. The estimated muscle tension is visualized by a change in the color of the muscles arranged on the musculoskeletal model (for example, the color is changed from yellow to red as the muscle tension increases). Furthermore, the actual experimental scenery is photographed using a DV camera synchronously, and the viewpoints are matched and superimposed on the musculoskeletal model for visualization. FIG. 6 shows a screen shot of the actually presented video.

[C-2] Overall Configuration of Muscle Tension Estimation Muscle tension acquisition according to this embodiment is roughly divided into the following two steps.
First, using EMG data, the muscle tension of a muscle to which an EMG electrode is attached and the muscle tension of a muscle closely related to the muscle are obtained.
Next, the relationship between muscle tension f and joint torque τ _G ′

Is used to estimate the muscle tension of other muscles. In addition to reducing the number of unknowns, the EMG data enables efficient estimation of solutions that satisfy the following constraint (3).

Table 2 shows the number of elements and the degree of freedom of the musculoskeletal model.
The left column is a model for conventional analysis in which all muscles such as the standing spine of the trunk are modeled. The objective function of the optimization calculation is

And inequality constraints

Optimization calculation is performed so as to satisfy (see Non-Patent Document 3).
In Table 2, the simplified model in the right column is a model in which elements of low importance are thinned out from a complex model that is a detailed model in the left column. The models shown in Table 2 are examples, and the present invention is not limited to these models. In the second embodiment to be described later, the number of muscles in the simplified model is increased from 274 to 314.

In the embodiment of the present invention, the scale of the optimization calculation is reduced by reducing the number of elements by reducing the number of elements by further grouping the simplified model for the purpose of reducing the calculation cost for real-time estimation. In fact, in the present embodiment, the muscle tension can be estimated at 16 ms (10 times or more faster than the conventional optimization calculation).

[C-3] Muscle Grouping Here, consider the nerve connection between muscles in the field of neurophysiology. Nerve connection is the connection of muscles via interneurons, and facilitating and inhibitory properties are considered. Muscles with fascinating bonds act as cooperative muscles and muscles with inhibitory bonds act as antagonistic muscles. When considering the functional significance of this nerve connection, it is often classified according to the action of the muscles and divided into muscle groups (cooperative muscles) such as the elbow flexor muscle group and the extensor muscle group in many cases. This is based on the premise of synonymous muscle facilitation and antagonistic muscle suppression, which can be expected to be facilitating within the same muscle group, and suppressive binding between muscle groups that exhibit antagonism. This is because the function of nerve connection can be considered in a simplified manner.

For this reason, in this embodiment, the muscles of the whole body are classified into groups of aliasing muscles, one muscle is selected as the representative muscle in the group, and an electromyograph is provided to measure the myoelectric potential of the representative muscle. The activity level is set as a representative value of the activity levels of all muscles in the group. In this embodiment, muscles of the whole body are measured with 16 channels, which is the number of channels of a general wireless electromyograph.

For muscle division, focus on the movement of the joints of the limbs and classify them into shoulder joint (horizontal) flexion / extension, elbow joint flexion / extension, hip joint flexion / extension, ankle joint dorsiflexion / plantar flexion (for knee joints) Because there are many parts that are linked to the hip joints, they are summarized.) The muscles represented in each group are the anterior deltoid muscle, the posterior deltoid muscle, the long biceps long head, the triceps lateral head, the rectus femoris, the biceps long head, the anterior tibial muscle, the gastrocnemius lateral head and To do. The% MVC of each muscle is measured, and the muscle activity is obtained from the muscle length and the muscle length change rate obtained from the motion capture, and the muscle tension is obtained from the muscle activity levels of all the muscles in the alias muscle promotion group including the muscle.
In this embodiment, attention is paid to the movement of the limbs, and the five joints (10 in total) on the left and right sides of the body are considered. Furthermore, the muscles are divided into 8 groups as shown in Table 3.

Each muscle group _{M i (i = 1, 2} ,..., 8) each muscle is further divided into the following three sets.
1. _MiEMG : Muscle groups of representative muscles whose EMG signals are measured.
2. M _high : A muscle group composed of muscles that have exactly the same joints as the M _iEMG .
3. M _ilow : A muscle group composed of muscles that are the same in part of the joints that contribute to the muscle group _MiEMG .
M _EMG , M _high , and M _low are respectively
M _EMG = M _1EMG ∪M _2EMG ∪... ＥＭM _8EMG
A muscle that does not belong to any of the muscle groups of Table 3, _{M others.}

The muscle tension of these muscles is acquired in the following flow.
First, for the muscles included in M _EMG and M _high , muscle tension is acquired from the myoelectric potential and the Hill-Stroeve muscle model.
Then, the muscle tension of the muscles included in the remaining muscles, that is, the muscles included in M _low and _Others , is estimated by inverse dynamics calculation and optimization calculation.

[C-4] Acquisition of muscle tension using the Hill-Stroeve muscle model Data necessary for estimating muscle tension using the Hill-Stroeve muscle model includes muscle length, muscle length change rate, and muscle activity. . When the muscle length, the muscle length change rate, and the muscle activity of all muscles are obtained, the muscle tension f ^* of each muscle is expressed by the following expression.

Here, a _i , l _i , and F _maxi are the muscle activity, muscle length, muscle length change speed, and maximum isometric strength of muscle i, respectively, and F ₁ (*), F _v (* ) Is a function that represents muscle tension and muscle length, and the rate of change of muscle tension and muscle length. The forward dynamics calculation using the joint angle and velocity gives l _i , l (dot _{) i} .

The evolution of muscle activity a _i is modeled by a first-order differential equation.

Here, T is a time constant, and u _i is an input from a motor nerve calculated from an EMG signal normalized by MVC.
Several methods are known to those skilled in the art for calculating the degree of muscle activity from the EMG signal. For example, the calculation methods described in the following documents can be used.
S.Stroeve. Learning combined feedback and feedforward
control of a musculoskeletal system. Biological Cybernetics, Vol. 75, pp. 73.83,
1996.

The muscle tension of the representative muscle of each group can be calculated directly from Equation (4).
The muscle kεM _high can be estimated from the activity of the representative muscle rεM _iEMG of the same group by the following equation.

Here, E _{r → k (*)} represents the activity of a _i of muscle k included in the group M _i, the relationship between the activity of a _r of the measurement representative muscle. As a function of E _{r → k (*),} a method according to cosine tuning claimed by Georgepoulos et al. Can be considered, but here it is defined by the following equation.

[C-5] Muscle tension estimation by inverse dynamics calculation and optimization calculation Up to this point, the muscle tensions of the first muscle group and the second muscle group have been acquired, and therefore the number of unknowns for optimization is reduced. Can do. However, the computational cost of optimization calculation with inequality constraints is still high. Below, efficient muscle tension estimation which does not use an inequality constraint condition is demonstrated.

First, the muscle tension f and Jacobian J in equation (1) are distributed to each muscle group.

Here, J _EMG , J _high , J _low , J _others are the muscle length Jacobian matrix for each joint angle, M _EMG , M _high , M _low , M _others , f _EMG , f _high , f _low , f _others are , Muscle tension in each group.
τ _G ′ is a generalized force from which the floor reaction force τ _C has already been subtracted.
Further, the deformation is made as follows.

here,

Although the number of unknowns is reduced, it has an inequality constraint f ≦ 0, and therefore it is necessary to perform an optimization calculation with an iterative calculation.
Where singularity-robust
(SR) inverse [NAKAMURA, Y., AND HANAFUSA, H. 1986. “Inverse Kinematics Solutions
with Singularity Robustness for Robot Manipulator Control ”. Journal of Dynamic Systems,
Using Measurement, and Control 108, 163.171.], We propose an optimization calculation without iterative calculation without using inequality constraints.

First, using Equation (6), an initial estimated value of M _low muscle tension is obtained.

_Here, k∈M ilow, is r∈M _iEMG. Since the muscle r and the muscle k belong to the same synonymous muscle promotion group, it can be said that the expression (11) is appropriate as an initial value.

Next, the muscle tension is corrected using the joint torque obtained by the inverse dynamics calculation. The algorithm repeats the following steps for each joint j.

Step 1: Drive joint j and collect all muscles belonging to M _low and M _others to form muscle group M _Jj . Further, a matrix corresponding to the joint j in equation (9) and the muscle in the muscle group M _Jj is extracted,

And

Step 2: For all muscles k∈M _Jj , _obtain an initial estimate of f _jk0 by the following formula:

By correcting f ^* _jk0 for all k, f ^* _j0 is formed.

Step 3: _Subtract J ^T _j f ^* _j0 from equation (12) to obtain the following equation.

Step ^{4: J} _{T j} of ^SR-inverse, using ^{J T} _{* j,} solving Delta] f _j0, obtain _{f j} updated by the following equation.

SR-inverse

, Where t is a positive weight. The element of Δf _j0 is expected to be small enough that f _j1 will not be positive.
If f _j1 ≦ 0 is held, the processing ends with f _j = f _j1 .
Otherwise, go to step 5.

Step 5: _Form a vector f _j1max having the same size as f _j1 and the element being the maximum value (positive) of f _j1 . Computing the second estimated value _{f * j1 = f j1 -f j1max} ≦ 0.

Step 6: _Subtract J ^T _j f ^* _j0 from equation (12) to obtain the following equation.

Step 7: Again, Δf _j1 is solved using SR-inverse of J ^T _j and f _j is updated by the following equation.

Similar to the discussion in step 4 above, many elements of f _j2 are negative and, if positive, are expected to be at least small. Therefore, f _j2 can be used as an approximation of f _j .

The above algorithm uses only J ^T _j SR-inverse, and only the muscles that drive the joint j need to be considered, so the size of J _j is small. Therefore, this algorithm is faster than the optimization calculation by iterative calculation.
According to the muscle tension estimation method according to this embodiment, the time required to estimate muscle tension of the whole body was 16 ms, and the time required to visualize somatosensory information was 68 ms (the computer used was 3.33 GHz Intel Xeon processor (3.25 GB RAM, NVIDIA Quadro FX3700). As a result, a 15fps frame rate visualization system was constructed.

[C-6] Real-time presentation of somatosensory information In the embodiment of the present invention, a photographed image of a subject and / or a composite image based on the photographed image is displayed on a display unit, and the musculoskeletal is displayed on the displayed subject image. The model is overlaid, and the activity information (somatosensory information) inside the body based on the muscle tension acquired by the above estimation method is reflected in the musculoskeletal model and displayed visually.
By putting an image of muscle activity on your live-action video that moves in synchronization with your movement, you can naturally feel the sensation of seeing through your body. A situation where the sexual sense can be seen through can be realized (see FIG. 6).

Here, the flow when the muscles are active will be explained. The α motor neurons in the spinal cord are activated in an excitatory manner by motor command signals from the upper center or signals from muscle proper sensory receptors. This signal goes out of the spinal cord through the spinal nerve bundle. From the α motor neuron to the endplate structure on each muscle, the “spinal nerve bundle” differentiates and signals are transmitted. In the endplate structure, excitatory signals are combined to give action potentials to the muscles. This leads to “muscle activity”. “Muscle tension” is generated when action potential is transmitted on the muscle. Therefore, in order to acquire and present the muscle activity level and spinal nerve bundle activity in real time, it is important to acquire muscle tension in real time.

More specifically, the musculoskeletal model is overlaid and displayed on the video taken by the video camera, and the muscle activity is expressed by the color and shape of the muscle on the musculoskeletal model. The muscle tension / muscle activity level is displayed by continuously changing the color of the muscle, for example, from yellow to red. Accordingly, in the overlay, the portion where the muscle is active is displayed in red, for example. Further, muscle fatigue may be expressed by muscle thickness. In one aspect, a value obtained by integrating the muscle tension over time is used as muscle fatigue.

For the activity of the spinal nerve bundle, a symbol, for example, a sphere, is displayed at the position of each spinal nerve bundle, and the activity is expressed by its color (see FIG. 9). In FIG. 9, for convenience, a sphere is displayed at the position of the spinal nerve bundle in the musculoskeletal model displayed on the display unit, but this can be further overlaid on the image of the subject.
For estimation of spinal nerve bundle activity, for example, the following documents can be referred to.
Murai, A, Yamane, K, and Nakamura,
Y, "Modeling and Identification of Human Neuromusculoskeletal Network Based
on Biomechanical Property of Muscle, "the 30 ^th IEEE EMBS Annual
International Conference, pages 3706-3709, Vancouver, August 2008.

[D] Second Embodiment of Muscle Tension Estimation The overall configuration of the apparatus and the overall processing flow in the second embodiment are the same as in the first embodiment except for muscle grouping and optimization calculation, and FIG. This is as shown in FIG. 5A. First, the joint angle of the subject is obtained by optical motion capture and inverse kinematics calculation. Next, the joint torque is obtained by performing the reverse dynamics calculation from the joint angle obtained from the reverse kinematics and the floor reaction force obtained from the force plate. Finally, optimization calculation is performed based on muscle grouping from EMG and joint torque, and muscle tension is estimated.

One of the reflections in the living body is somatic reflection that contracts skeletal muscle. Muscle proper receptors sense changes in muscle length, rate of change, and muscle tension. Muscle length and the rate of change are sensed by the muscle spindle, and muscle tension is sensed by the Golgi tendon organ. Properceptors contribute to central motor regulation from two aspects. One is the reflex effect, and the other is that information from proper receptors conveys the state of movement and posture to the upper brain. When movement occurs in response to a movement command, the state of the muscle changes, and the proper receptor reflex is inevitably triggered. The reflection effect is reflected in the motion program and contributes to the formation and correction of the motion pattern.
Reflexes derived from muscle spindles include stretch reflex, antagonistic inhibition, α-γ related, etc., but pay attention to stretch reflex. Stretch reflex is the firing of nerve fibers derived from the main spindle of the main muscle, which is the motor neuron that controls the muscle
(Same muscle motor neuron
) And its synergistic motor neurons cause monosynaptic excitement.
When a signal is sent from the upper center to the main muscle required for certain movements due to stretch reflex, the signal returned from the Ia group nerve fiber of that muscle gives excitement effect to the motor neuron of the synergistic muscle and signals to the synergistic muscle Will be sent. As a result, muscle tension is distributed among the cooperative muscles in order to generate joint torque. Therefore, cooperative muscles are expected to have the same degree of muscle activity. Therefore, paying attention to the cooperative muscles, the cooperative muscles have the same degree of muscle activity, and the grouping is performed to reduce the calculation. However, based on anatomy, the synergistic muscle has not been clarified from the connection of group Ia nerve fibers and motor neurons. In this embodiment, cooperative muscles are considered from the viewpoint of kinematics rather than anatomy, and grouping of cooperative muscles is performed.

[D-1] Muscle Grouping In the following description, i and j are used as the muscle group index, and k is used as the muscle index.
First, attention to the movement of the limbs and trunk, group the muscle of the whole body of the nickname muscle facilitation _{M i (i = 1,2, ...} , n G) be classified into. n _G is the number of groups of M _i. Each group is further classified into the following two groups.

1. M _ihigh : Muscle where EMG representing the group is measured, and muscle where EMG is measured and the bone that starts and stops are the same. Muscles belonging to the same group are expected to have similar muscle activity. M _high in the second embodiment corresponds to M _EMG + M _high in the first embodiment.
2. M _ilow : A muscle not belonging to the above Mi _high . Further, it is classified into M _{i, 1low} , ..., M _{i, nilow} according to the bone that has started and stopped. Here, n _i is the number of groups of M _{i, jlow} in M _ilow .

Each group M _i streaks to be measured EMG shall not exist two or more. Furthermore, M _high = M _1high ∪M _2high ∪ ... ∪M _nGhigh , M _low = M _1low ∪M _2low ∪ ... ∪M _nGlow defines M _high and M _low . If the number of muscles measuring myoelectric potential is n _EMG (≦ n _G ), the group containing the muscles measuring myoelectric potential is M ₁ , ..., M _nEMG , M _{(nEMG + 1) high} , ..., M _nghigh is an empty set. In this embodiment, n _G = 36 and n _EMG = 16. Table 4 can be referred to for the muscle that measures EMG and the movement of the joint to which the muscle contributes. The reason why the number of muscles belonging to each group is slightly different from that in Table 3 is that the number of muscles in the musculoskeletal model (simplified model) that is a premise of grouping differs between the first embodiment and the second embodiment.

Details of the muscle group used in the present embodiment will be described below. Among the muscles belonging to M _high , the muscles representing the group and for which the myoelectric potential is measured are shown in bold. In a musculoskeletal model such as the great pectoral muscle or latissimus dorsi muscle, one muscle may be represented by multiple wires, but here it is also represented as a single muscle when represented by multiple wires. Groups 1-8 correspond to M ₁ ,..., M _nEMG (corresponding to groups _{1-8 in} Table 4). In groups 9 to 18, M _high is an empty set. It should be noted that the following grouping is merely an example, and those skilled in the art will understand that groupings that can be applied to the present invention are not limited to those described in this specification.

Group 1: Horizontal adduction of shoulder joint

Group 2: Horizontal abduction of shoulder joint

Group 3: Elbow joint flexion

Group 4: Elbow joint extension

Group 5: Hip joint extension and knee joint flexion

Group 6: Hip flexion and knee extension

Group 7: Ankle joint dorsiflexion

Group 8: Ankle joint plantar flexion

Group 9: Flexion of neck joint

Group 10: Shoulder joint control

Group 11: Raising the shoulder joint

Group 12: wrist flexion

Group 13: Wrist joint extension

Group 14: External rotation of hip joint

Group 15: Flexion of tarsal metatarsal joint

Group 16: Trunk flexion

Group 17: Trunk extension

Group 18: Raising the trunk

[D-2] Muscle tension estimation by quadratic programming method Each muscle group

As for the same group

The muscles belonging to M _* contribute to the movement of the same joint and work as synergistic muscles by facilitating synonymous muscles, so all muscles belonging to M _* have the same degree of muscle activity.

Is expected to have According to the Hill-Stroeve muscle model (Non-patent Documents 1 and 2), the muscle tension f _k of muscle k (∈M _* ) is

It can be expressed as
F _maxk is the maximum muscle tension of muscle k, F _l (l _k ) is the ratio of muscle tension that the muscle can exert when the length of muscle k is l _k , F _v (l (dot) _k ) Is the ratio of muscle tension to maximum tension that can be exerted when the contraction speed of muscle k is l (dot) _k .

Here, since F _l (l _k ), F _v (l (dot) _k ), and F _maxk in equation (1) are obtained by inverse kinematics calculation, the joint torque τ _Gk ′ generated by the muscle k is a constant term. And the variable term

It can be expressed.
However, J _k ∈ R ^{1 × ndof} is the Jacobian matrix of muscle length of muscle k with respect to the joint angle, and H _k ∈ R ^{1 × ndof} is

It is. further

The

Then, from equation (2), the joint torque τ _G ′ is

It becomes.

Based on the above, the muscle activity vector a representing each group is obtained by quadratic programming. Quadratic programming is an objective function,

Is a method for obtaining x that minimizes x under linear equality constraints and inequality constraints on x. However, x and c are n-dimensional vectors, and Q is an n × n matrix.

In the first embodiment, the muscle tension of a muscle belonging to M _high is obtained using the Hill-Stroeve muscle model using the muscle activity obtained from the myoelectric potential as it is, and the muscle tension of a muscle belonging to M _low is reversed. It is calculated by kinematics and quadratic programming.
In the second embodiment, the muscle activity calculated by EMG is given as a reference value, and the muscle tension of all the muscles including the muscle belonging to M _high is obtained by the quadratic programming method.
In the second embodiment, the reduction in the amount of calculation for optimization calculation mainly depends on the grouping of muscles. Further, the calculation speed can be increased by performing parallel processing of IK, ID / muscle tension estimation, and drawing calculation. By assigning a thread to each of these processes, parallel processing is performed as a whole (see FIG. 5B). Since IK, ID / muscle tension estimation, and drawing calculation are simultaneously performed on data at a plurality of times, the throughput of the system is improved. Since the drawing calculation is rate-limited by the parallel calculation, the frame rate of the output image does not change even if the quadratic programming method takes a little time.

The objective function and inequality constraint conditions are determined in consideration of the following four conditions.
Condition 1: Muscle torque generated by joint torque and muscle tension obtained by inverse kinematics is equal. That is, Expression (7) is satisfied.
Condition 2: A muscle activity pattern is generated such that the sum of muscle activity is minimized.
Condition 3: The muscle activity of the muscle whose myoelectric potential is measured is equal to the muscle activity calculated from the myoelectric potential.
Condition 4: Muscle activity a _* is between 0 and 1.

Since the joint torque τ _G ′ obtained by inverse kinematics from condition 1 and the joint torque H ^T a calculated from the degree of muscle activity are desired to coincide with each other, the difference H ^T a−τ _G ′ of 2 Think of power. As a result, an objective function for obtaining a solution satisfying condition 1

Is obtained.
However, W _dyn ∈ R ^ndof is a weight matrix.

Considering the following objective function in order to minimize the total sum of muscle activity from condition 2.

Where W _tot ∈R ^ng is a weight matrix.

For muscle groups whose muscle activity is calculated by Hill-Stroeve model from condition 3, the muscle activity calculated by quadratic programming should match the muscle activity calculated from the measured myoelectric potential. Consider the muscle activity of only the muscle group M _high whose muscle activity is calculated by the Hill-Stroeve model.
Let a _iEMG (i = 1,..., n _EMG ) be the _Mihigh muscle activity calculated from the myoelectric potential, and define a _EMG as follows.

From the above, the objective function of condition 3 is expressed using the weight matrix W _EMG ∈R ^ng

And

From the above conditions 1, 2 and 3, the objective function Z is

And
Here, k _tot and k _EMG (<0) are weighting factors.
Equation (13) has the form of the quadratic programming objective function shown in Equation (8).

Can be considered.

Next, from condition 4, the inequality constraint condition is

It becomes. If a is obtained by quadratic programming under the objective function Z and the inequality constraint condition (17), the muscle tension of the muscle k (εM _* ) can be obtained from the equation (1).

Next, the weight matrix included in the objective function will be described.
W _dyn , W _tot , and W _EMG are diagonal matrices,

It is expressed as
^{_{Incidentally, σ idyn 2 (i = 1}} , ..., n dof), σ jtot 2 (j = 1, ..., n g), σ kEMG 2 (k = 1, ..., n EMG)
_{Are obtained} when W _dyn = E _{ndof × ndof} , W _tot = E _{ng × ng} , and W _EMG = E _{nEMG × nEMG} , respectively (τ _G ′ −H ^T a), a, (a _high −a _EMG ) Is the variance of each element. This makes the objective function Z dimensionless.
Further, it is assumed that the user determines the weighting factors k _tot and k _EMG as necessary.

Finally, the method according to the first embodiment and the method according to the second embodiment will be compared and arranged based on FIGS. 10 and 11. FIG. 10 shows the grouping of the technique of the first embodiment, and FIG. 11 shows the grouping of the technique of the second embodiment. 10 and 11, the number of electromyographs is three for simplicity. In FIG. 11, a broken line is an empty set.

In FIG. 10, some of the plurality of muscles of the subject are classified into the muscle group M _i (i = 1, 2, 3). In the muscle group M _i , one muscle M _iEMG (i = 1 to 3) with an _{electromyograph} and one or a plurality of cooperative muscles that _{perform the} same function as M _iEMG are _defined as M _ihigh . Activity of the muscle of the _M ihigh is uniformly on which were the same as the M _iEMG, put it together than the rest of all of the muscle _(M ilow and M _others). All the remaining streaks are calculated from dynamic force balance using SR-Inverse (like a stable inverse matrix). Each one of all the remaining streaks (about 350 in the first embodiment) is calculated as a variable, and the quadratic programming method is not used. In the previous optimization calculation, quadratic programming or linear programming was used to calculate about 1000 muscle activities.

In FIG. 11, all the plurality of muscles of the subject are classified into muscle groups M _i (i = 1, 2, 3, 4, 5). The muscle group M _i is further classified into a plurality of subgroups by grouping them into cooperative muscle groups having the same activity (muscle groups having common bones that start and stop). There are three subgroups (consisting of _Mihigh ) that contain muscles with electromyographs, and these subgroups have one or the same as one muscle with an electromyograph or Multiple cooperative sources are also included. That is, Mi _high in FIG. 11 corresponds to _MiEMG + _Mihigh in FIG.
In each muscle group M _i, muscle M _Ihigh is all muscle other than M _Ilow, classified into M _Ilow further subgroup by summarized M _Ilow each cooperative muscles (M _i, consists _Jlow) Is done. M _{i, jlow} is a muscle group having common bones that start and stop in M _ilow .
In the second embodiment, the total number of subgroups is about 70. The muscle activity belonging to each subgroup is uniformly expressed by about 70 variables, and this is subjected to optimization calculation of the quadratic programming method. At this time, the muscle activity of the muscle M _iEMG to which myoelectricity is _applied is used as a reference value for the muscle of Mi _high .

The present invention can be used in the fields of sports training, rehabilitation, medical diagnosis, health management, entertainment, and the like.

Claims (18)

1. In the method of estimating the muscle tension of each muscle by calculating each joint torque during exercise of the subject by inverse dynamics calculation using a musculoskeletal model and distributing the joint torque to the muscle tension by optimization calculation,
   Classifying a plurality of muscles of a subject into a plurality of muscle groups M _i (i = 1, 2,... N) based on muscle movement directivity or alias muscle facilitation;
   In each muscle group of a plurality of muscle groups M _i (i = 1, 2,... N), bones that start and stop form one or more subgroups from the same muscle, and belong to the same subgroup. Are considered to be the same muscle activity,
   In some or all of the plurality of muscle groups M _i, at least one, one representative muscle and the representative muscle and bones origin stop electromyograph is mounted in one of said one or more sub-groups Is a first subgroup formed from muscles that are the same,
   The muscle tension of the muscles belonging to the first subgroup is obtained from the myoelectric potential of the representative muscle measured during the exercise of the subject without using the optimization calculation, and is excluded from the optimization calculation target, thereby performing the optimization calculation. Reduce the variables in
   Or
   In the one or a plurality of subgroups, the muscle activity representing each subgroup is estimated by the optimization calculation, thereby reducing variables in the optimization calculation.
   Muscle tension estimation method.
2. Obtaining the muscle tension of the muscle belonging to the first subgroup from the myoelectric potential of the representative muscle measured during the exercise of the subject;
   Optimizing muscle tension of muscles not belonging to the first subgroup so as to realize joint torque that cannot be realized by muscles belonging to the first subgroup in joint torque necessary to realize exercise of the subject; Estimated by
   The method for estimating muscle tension according to claim 1.
3. In each muscle group M _i, wherein one or more subgroups, the first subgroup, the muscles do not belong to the first sub-group, zero or more bone to origin stop is classified from the same muscle And a subgroup of
   Estimate the muscle activity that represents each subgroup with optimization calculations.
   The method for estimating muscle tension according to claim 1.
4. For muscles belonging to the first group, the muscle activity obtained from the myoelectric potential should match the muscle activity calculated by the optimization calculation.
   In the optimization calculation, the measured muscle activity of the representative muscle is used as a reference value.
   The method for estimating muscle tension according to claim 3.
5. The method for estimating muscle tension according to any one of claims 1 to 4, wherein the muscle tension is estimated in real time during exercise of the subject.
6. The subject's captured image or / and a composite image based on the captured image are displayed on the display unit, and a musculoskeletal model is overlaid on the displayed subject's image,
   The activity information inside the body based on the muscle tension acquired by the estimation method according to claim 1 is reflected in a musculoskeletal model and displayed visually.
   Activity information presentation method inside the body.
7. The internal body activity information presentation method according to claim 6, wherein the internal body activity information is displayed in real time during exercise of the subject.
8. The method for presenting internal activity information according to claim 6 or 7, wherein the internal activity information is muscle activity.
9. The method for presenting activity information inside the body according to claim 8, wherein the muscle activity is visually displayed by a change in the color or / and shape of the muscle of the musculoskeletal model.
10. The method for presenting internal activity information according to any one of claims 6 to 9, wherein the activity information inside the body represents muscle activity as activity of a spinal nerve bundle that governs the muscle activity.
11. The activity of the spinal nerve bundle according to claim 10, wherein a symbol is displayed at a position of each spinal nerve bundle on the musculoskeletal model and is visually displayed by a change in the color or / and shape of the symbol. Information presentation method.
12. In the method of estimating the muscle tension of each muscle by calculating each joint torque during exercise of the subject by inverse dynamics calculation using a musculoskeletal model and distributing the joint torque to the muscle tension by optimization calculation,
    Classifying a plurality of muscles of a subject into a plurality of muscle groups M _i (i = 1, 2,... N) based on muscle movement directivity or alias muscle facilitation;
    In each muscle group of a plurality of muscle groups M _i (i = 1, 2,... N), bones that start and stop form one or more subgroups from the same muscle, and belong to the same subgroup. Are considered to be the same muscle activity,
    In the one or a plurality of subgroups, the muscle activity representing each subgroup is estimated by the optimization calculation, thereby reducing variables in the optimization calculation.
    Muscle tension estimation method.
13. In some or all of the plurality of muscle groups M _i, at least one, one representative muscle and the representative muscle and bones origin stop electromyograph is mounted in one of said one or more sub-groups Is a first subgroup formed from muscles that are the same,
    For muscles belonging to the first group, it is assumed that the muscle activity obtained from the myoelectric potential should match the muscle activity calculated by the optimization calculation, and the muscle of the representative muscle measured in the optimization calculation Use activity as a reference value,
    The method for estimating muscle tension according to claim 12.
14. Each joint torque during exercise of the subject is calculated by inverse dynamics calculation using a musculoskeletal model, and muscle tension of each muscle is estimated by distributing the joint torque to muscle tension by optimization calculation Means,
    A grouping means for classifying a plurality of muscles of a subject;
    With
    The grouping means includes
    First grouping means for classifying the plurality of muscles of the subject into a plurality of muscle groups M _i (i = 1, 2,... N) based on muscle movement directivity or alias muscle facilitation;
    In each muscle group of the plurality of muscle groups M _i (i = 1, 2,... N), second grouping means in which the bones that start and stop form one or more subgroups from the same muscle; Become
    The muscle tension acquisition means regards the muscle activity level of muscles belonging to the same subgroup as the same, and estimates the muscle activity level representing each subgroup in the one or more subgroups by optimization calculation.
    Muscle tension estimation device.
15. At least a part of a plurality of muscles of the subject's whole body or a part of the body is divided into a plurality of muscle groups M _i (i = 1, 2,. classifying the electromyograph attached to the representative muscle by selecting one representative muscles from each muscle group M _i,
    The plurality of muscles,
    A first incision group _{M iEMG} comprising the representative muscle of each muscle group _{M i,}
    In each muscle group _{M i,} and a second incision group _{M Ihigh} consisting muscle is a bone to the origin stop first muscle group _{M iEMG} the same,
    A first muscle group M _iEMG , a third muscle group consisting of muscles not included in the second muscle group M _high ;
    Divided into
    _Obtaining the muscle tension of the muscles belonging to the first muscle group _MiEMG and the second muscle group _Mhigh from the myoelectric potential of the representative muscle measured during the exercise of the subject;
    The muscle tension of the muscles belonging to the third muscle group is calculated by inverse dynamics calculation to calculate the joint torque necessary to realize the exercise of the subject, and in the joint torque, the first muscle group _MiEMG and Estimating by optimizing to realize joint torque that cannot be realized by muscles belonging to the second muscle group M _high ,
    Muscle tension estimation method.
16. Multiple electromyographs,
    First muscle tension acquisition means for acquiring muscle tension from myoelectric potential;
    Second muscle tension acquisition means for calculating joint torque by inverse dynamics calculation and estimating muscle tension by performing optimization calculation so as to realize the joint torque;
    A grouping means for classifying a plurality of muscles of a subject;
    With
    The grouping means includes a first grouping means and a second grouping means,
    The first grouping means may include a plurality of muscle groups M _i (i = 1, 2,... N) based on the movement directionality of the muscles or alias muscle promotion. ) in are those classified, each electromyograph is attached to one representative muscle selected from the muscle group M _i,
    The second grouping means includes a plurality of muscles of the subject,
    A first incision group _{M iEMG} comprising the representative muscle of each muscle group _{M i,}
    In each muscle group _{M i,} and a second incision group _{M Ihigh} consisting muscle is a bone to the origin stop first muscle group _{M iEMG} the same,
    A first muscle group M _iEMG , a third muscle group consisting of muscles not included in the second muscle group M _high ;
    Divided into
    The first muscle tension acquisition means acquires the muscle tension of the muscles belonging to the first muscle group _MiEMG and the second muscle group _Mhigh from the myoelectric potential of the representative muscle measured during the exercise of the subject,
    The second muscle tension acquisition means calculates the muscle torque of the muscles belonging to the third muscle group, and calculates the joint torque necessary for realizing the movement of the subject measured by inverse dynamics calculation. Estimating by optimizing to realize joint torque that cannot be realized by the muscles belonging to the first muscle group _MiEMG and the second muscle group _Mihigh ,
    Muscle tension estimation device.
17. The apparatus for estimating muscle tension according to any one of claims 14 and 16, wherein the muscle tension is estimated in real time during exercise of the subject.
18. The real-time muscle tension estimating device according to claim 17,
    Means for photographing the subject during exercise;
    Display means for displaying a photographed image of the subject or / and a composite image based on the photographed image;
    With
    The musculoskeletal model is overlaid on the image of the subject displayed on the display means, and the activity information inside the body based on the muscular tension estimated in real time by the real-time muscular tension estimation device is reflected on the musculoskeletal model to visually Configured to display real-time,
    Activity information presentation device inside the body.

Images