WO2006027869A1

WO2006027869A1 - Movement learning assisting device and method, movement learning assisting program, and recording medium on which the program is recorded

Info

Publication number: WO2006027869A1
Application number: PCT/JP2005/007444
Authority: WO
Inventors: Mihoko Otake; Yoshihiko Nakamura
Original assignee: The University Of Tokyo
Priority date: 2004-09-10
Filing date: 2005-04-19
Publication date: 2006-03-16
Also published as: JP2006075398A; JP4054879B2

Abstract

A technique useful for movement learning assist by detecting nerve information from movement information on muscles and tendons of the whole body. A processing section performs initial setting (S101), reads reference data and object data from a nerve data file (S103), performs a pre-processing (S105), sets the initial time (S107), segments a template of a specific time width from the reference data (S109), computes the correlation with the template over a specific region of the object data (S111), stores about a different trial of a similar nerve the time in the reference data and the maximum correlation value of the object data associated with the time in a similar nerve trial corresponding time data file (S113), advances the time by a unit time until the final time (S115, S117), performs a post-processing, determines the corresponding time and the correlation value about the same trial of a dissimilar nerve, stores them in a dissimilar nerve same trial corresponding time data file (S119), and presents them on a display section (S121).

Description

Specification

Motor learning support apparatus and method, motor learning support program, and recording medium recording the program

Technical field

[0001] The present invention relates to a motor learning support device and method, a motor learning support program, and a recording medium on which the program is recorded.

Background art

[0002] It is known that deep somatosensory sensations occurring in motor organs such as muscles, tendons, and joints are necessary for learning new movements. Somatosensory sensation is thought to act as a teacher signal for evaluating motion results in motor learning that involves repeated trials, not just acting as a feedback signal for a single trial. Therefore, if the evaluation is not performed correctly, the learner will not be proficient in the desired movement, but it is not always clear whether the person can accurately perceive the result of movement, and it is also subjective. Measurement is not easy because it is a sense. The subjective evaluation of movement based on somatosensory sensation may coincide with the objective evaluation based on observations from outside, and the slashing action (swinging the sword while stepping on the foot and swinging it down diagonally) This has become clear through experiments. If the somatosensory information processing mechanism can be implemented on a computer from the viewpoint of how each movement looks when observed from inside the body, it is based on the internal sense of the person. It is possible to make a motor learning support device that works on people. Currently, it is technically possible to measure the global activity state of the brain non-invasively. It is possible to visualize the size of the blood flow that supplies the energy necessary for brain activity and the magnetic field generated when nerve signals are transmitted. It is also possible to directly detect the firing state of a nerve by embedding a minute electrode in the nerve. However, clarifying the state of brain and nerve activity does not directly divide what the person thinks and feels. Rather, by measuring the movements of organs that are controlled by the brain and nerves, such as muscles, it may be possible to know human conscious or potential movement control processes. [0003] Human movement occurs when a plurality of muscles cooperate. Therefore, it is considered that the movement can be evaluated by the coordination state of the muscles distributed throughout the body, not only by the exercise of each muscle. A person can be awkward when in tension This awkward movement can be seen as a result of imbalanced muscle coordination. Bernstein pointed out that motor organs have redundant degrees of freedom and defined cooperation as converting this into a controllable system. In order to overcome redundant degrees of freedom, it is indispensable to adjust movement based on sensory organs, especially muscles, tendons, and information on sensory organ forces inherent in joints. called.

[0004] The goal of the present invention is to clarify a method for quantitatively handling human sensation and movement problems that have been qualitatively discussed by observing movements so far. Through this, we aim to approach motor control processes from the outside and support motor learning. There has been proposed a method for executing forward / reverse dynamics calculation at high speed for a detailed model of a body having bone geometry data and muscle / tendon 'ligament data (Non-patent Document 1). Many studies have been conducted to measure body movements and calculate the length of muscle 'tendon' ligaments and the tensions generated in these motor organs. Delp et al. SIMM and Rasmussen et al. AnyBody have been commercialized as systems for analyzing and simulating the motion of human musculoskeletal models.

[0005] In general, it has been pointed out that perceptual information processing methods for dynamical media such as somatic sensations are delayed compared to visual and auditory senses. The following is an overview of the prior art related to the body model and the nervous system model combined with the body model.

Nakamura et al. Have proposed a method for performing forward and reverse dynamics calculations at high speed for detailed models of the body with bone geometry data and muscle 'tendon' ligament data (Patent Documents 1 and 2). . This method enables us to observe the human force and calculate the changes in the length of the muscle 'tendon' ligament and the tension generated in these moving organs.

Kawahito and others naturally control the virtual body, which is a model of all or part of the body in the world inside the computer, as if it were a part of itself, and not only the position but also the force, speed, and acceleration. We propose a human interface device that can be operated freely (Patent Document 3).

[0006] In addition, Hase et al., A biped model with a three-dimensional musculoskeletal system and a hierarchical nervous system (non-patent document) Sugawara et al. Proposed an actual walking measurement data force walking neural network (Non-patent Document 3).

In general, to support motor learning, it is important to evaluate motor based on motor analysis. This is because exercise adjustment is performed based on the evaluation. Conventionally, when comparing movements based on movement measurements, some parts of the whole body that aggregate the results of coordinated movements, such as the positions of the hands and toes, or joint angles such as elbows and knees, are considered. Has been done. Alternatively, it has been common to compare muscle-by-muscle activities, such as myoelectric potential and muscle length. For example, by comparing the difference in the skill of tennis experts and beginners in the dexterity of the peak time of muscle activity, a simple analysis has been performed (Non-patent Document 4).

Patent Document 1: JP 2003-339673 A

Patent Document 2: JP 2004-013474

Patent Document 3: Japanese Patent Laid-Open No. 07-028592

Non-patent literature 1: Y. Nakamura et. Al. Dynamic computation of musculo-skeletal human model based on efficient algorithm for closed kinematic chains. In Proceedings of the 2nd International Symposium on Adaptive Motion of Animals and Machines, 2003. : Kazunori Hase, Junya Nishiguchi, Nobutoshi Yamazaki Biped model with three-dimensional musculoskeletal system and hierarchical nervous system Biomechanism 15—Exploration of shape and movement, Biomechanism Society, Tokyo University Press , (2000).

Non-Patent Document 3: Naomichi Sugawara, Nobutoshi Yamazaki Estimation of walking neural network from real gait measurement data 2000)

Non-Special Reference 4: Sakurai S. et. Al. Muscle activity and performance accuracy of the smash stroke in badminton with reference to skill and practice. J. Sports Science vol. 18, pp.1— 14 (2000)

Disclosure of the invention

Problems to be solved by the invention

These conventional technologies can be applied to medical or ergonomics or scanning, such as examining the effects of muscle strength when surgically changing the skeletal muscle placement, and evaluating the workspace when a product is used by humans. It is intended for application to Pau science. For this reason, the information generated by the sensory organs inherent in the motor organs has not been fully examined for the route through the nervous system to the motor organs.

[0008] In the conventional technology outlined above, Nakamura et al.'S model does not include the nervous system.

In addition, Kawato et al. Connect a non-linear dynamics model including a neural circuit to a virtual body to realize a human interface device, but the neural circuit does not consider the structure of the peripheral nervous system. Furthermore, Hase et al.'S technologies are all proposing a neuromusculoskeletal model. The α motor neurons that directly control the force muscles and the muscles are directly connected to each other, and are identical from multiple spinal cords. Redundant structures that connect to other muscles are not considered. In addition, branching structures that connect to multiple muscles with a single nerve bundle force are not considered. Therefore, the model includes the concept of muscle groups controlled by the same nerve, or muscle groups controlled by the same spinal nerve!

[0009] In general, motor organs and other organs including muscle 'tendon' ligaments and the central nervous system (brain) are connected via the peripheral nervous system. This peripheral nervous system has a redundant branch structure. It is mechanically or functionally close! The nerves that connect with the heel organs are bundled together to form the common nerve. In addition, nerves that connect with mechanically or functionally related organs connect from the same spinal cord. Since the conventional technology has made great efforts taking such anatomical structures into consideration, there have been the following problems.

Inability to handle the mechanistic or functional relationship between innervated organs

• From the physical quantity (objective quantity) that the motor nerves generate, how humans feel, t cannot detect the sensory quantity! /,

This is thought to be due to the lack of perspective on how the human brain receives signals from motor organs and how they are processed. Human body movements result from the interaction of the nervous system with the musculoskeletal system and the outside world. Body movement information includes internal information of the nervous system. Conversely, body movement information is detected by motor sensory organs such as muscle spindles and Golgi tendon organs, and is input to the nervous system. It is thought that the mechanism of human movement recognition and generation can be clarified by tracing the neural information related to body movement in order from the terminal tree to the center. [0010] In addition, the method of comparing the position and orientation of a part such as a hand or a foot, which is the result of cooperative movement, which is the mainstream of conventional motion analysis methods, is too rough, and a part of the whole is shifted. There was a problem that cannot be detected. In other words, when comparing different exercises, it was necessary to normalize the time because the raw data cannot be compared when the speed of the exercise is different. However, since speed is an important factor in movement, it is not desirable to keep time regular. In fact, there may be a shift in the whole body movement due to differences in speed and timing, such as the upper body being slow and the lower body being fast. In order to evaluate such movement, time must be compared for each part, but it is difficult to separate cleanly because the upper body and lower body are linked. On the other hand, the method of evaluating several muscles one by one was too detailed, and there was a problem that the overall state of cooperation could not be clarified. Muscle tension and other factors have been obtained with electromyographs and are limited by the number of channels of the electromyograph. When evaluating the whole body movement by representing a part of muscle distributed throughout the body, there was a problem that the degree of local cooperation was not clarified. A method to calculate muscle length by mapping motion capture data to a musculoskeletal model has been proposed. However, the number of muscles reached several hundreds, and even if these were processed uniformly, there was a problem that only global information could be obtained. In the end, we had to rely on the method of selecting and comparing the main muscles, and we were able to obtain information on many muscles. That is, it was difficult to perform a global and local analysis of movement.

The present invention has been made in view of the above points, and an object of the present invention is to provide a technique that is useful for motor learning support after detecting neural information from motor information such as muscles and tendons of the whole body. The present invention focuses on the spinal nerve that connects sensation and movement, and models its muscle control structure. Based on this, the neural information flowing through the spinal cord during exercise is mapped to an image obtained by slicing the spinal cord or the neural information flowing through the terminal nerve to a spatiotemporal pattern. Neural information inherent in the spinal cord is obtained from human movement data by mapping muscle movement information to spinal nerve information using the anatomical structure of the nervous system. The present invention further proposes a method for detecting how subtle differences in movement appear in the neural information using the neural information mapped by the movement information force, and the human movement. The objective is to evaluate and compare global and local features. By using the structure of the nervous system, the human body moves the whole body Aiming to get closer to the process of observation. In addition, as a related technology of the present invention, there is a patent application by the present inventors (Japanese Patent Application No. 2004-176455, filed on June 15, 2004), which is incorporated into this specification with reference to the technical content thereof (incorporated). )be able to.

[0012] Considering how the living body processes a lot of muscle strength information distributed throughout the body! /, It is grouped to some extent by the peripheral nerves and then bundled for each spinal cord. The muscle information is converted into a two-dimensional map that holds the body phase information and processed. The present invention can check the degree of cooperation of the distant parts of the whole body by comparing the grouped information. Moreover, this invention can investigate the cooperation degree of proximal muscles by comparing the information in a group. The present invention proposes a method for hierarchically processing global information and local information expressing whole body motion using the structure of the human nervous system. In particular,

'Calculate the spatiotemporal pattern similarity for each nerve and detect the phase difference by matching the time with high similarity

• Compare the pattern and timing of neural information that occurs when trying the same action multiple times for each nerve.

• A method for assessing the activity of the entire spinal cord layer that dominates different muscles, in particular, to determine how co-operation is expressed at the level of the spinal cord, and to determine the strength, degree of coordination and dimension.

• In addition to detecting the peak of time fluctuation and left / right symmetry for each nerve

With the goal. The purpose is to compare and evaluate motors at the neurological level and to support motor learning.

Means for solving the problem

[0013] According to the first solving means of the present invention,

With respect to time, nerve ID, spinal cross-sectional images at each time or nerve information placed in the position space of muscles controlled by nerves at the trunk and terminal at each time, and data for determining trials A neural data file storing neural data including the data ID,

Reference data time and target data time representing the corresponding time of neural information in the same nerve in different trials, correlation values at the corresponding time, reference data ID, target data ID, and nerve ID, different trials for the same type of nerve The first corresponding time data file to be memorized And

A display for displaying the correspondence between nerves or trials;

Reading and Z or writing to the neural data file and the first corresponding time data file, obtaining the correspondence between nerves or trials, displaying the correspondence on the display unit, and the first corresponding time data file And a processing unit for storing

With

The processing unit, as an initial setting, sets a data attribute including a reference data ID and a target data ID for determining nerves and different trials from the input unit or other devices, and

The processing unit reads, from the neural data file, as reference data and target data, the same portion of neural data in different trials according to the data attribute selected in the initial setting.

The processing unit performs preprocessing for the reference data and the target data, including setting an initial time for correlation calculation,

The processing unit creates a template by cutting out data with a length corresponding to the reference data force specified time width, scans the template from the initial time to the final time, which is the calculation target of the target data, and scans the template and the target data. A means of repeatedly calculating the correlation value for each time,

The processing unit sets the time when the correlation value between the reference data and the target data is the maximum for the template as the corresponding time, and the reference data ID, the target data ID, and the nerve ID are different in the first corresponding time data file. Means for storing the reference data time and the target data time indicating the corresponding time of the neural information in the same nerve of the trial and the correlation value at the corresponding time,

The processing unit creates a template by advancing a predetermined time until the final time in the reference data, and repeats the means for calculating and the means for storing;

The processing unit reads data from the first corresponding time data file, and displays a correspondence relationship between the reference data time and the target data time on the display unit,

A motor learning support device is provided. According to the second solution of the present invention,

For the time, the nerve ID, the nerve information arranged in the muscle number space indicating the position of the muscle governed by the nerve to the end of the trunk strength at each time, and the data ID for defining the trial A neural data file storing neural data,

Reference data ID, first nerve ID, target data ID, second nerve ID, nerve data specified by the muscle number controlled by the first nerve, and muscle number controlled by the second nerve A degree-of-cooperation data file that stores the correlation value with the specified nerve data in association with each other; a display unit that displays the correspondence between nerves or trials;

To read and Z or write to the nerve data file and the cooperation degree data file, obtain the correspondence between nerves or trials, display the correspondence on the display unit, and store it in the cooperation degree data file And the processing part

With

The processing unit, as an initial setting, sets, from the input unit or other device, a data attribute including first and second nerve IDs and a data ID for determining a trial, and

The processing unit reads the same trial nerve data in different nerves as reference data and target data from the nerve data file according to the data attribute selected in the initial setting,

The processing unit creates a template by cutting out the neural data for the entire time to be calculated for each muscle number from the reference data, and scans using the template for the entire time to be calculated for each muscle number of the target data. A means for repeatedly calculating a correlation value for each muscle number between the template and the target data,

The processing unit includes, in the cooperation degree data file, a reference data ID, a target data ID, first and second nerve IDs, a muscle number controlled by the first nerve, and a muscle number controlled by the second nerve. Means for storing correlation values on a matrix of

The processing unit creates a template for each muscle number in the reference data, and repeats the means for calculating and the means for storing; The processing unit reads data from the cooperation degree data file, and displays the correlation values representing the degree of movement cooperation of each muscle controlled by the first and second nerves in a matrix on the display unit;

A motor learning support device is provided.

According to the third solution of the present invention,

Reference data time and target data time representing the corresponding time of neural information in the same nerve in different trials, correlation values at the corresponding time, reference data ID, target data ID, and nerve ID, different trials for the same type of nerve The first corresponding time data file to be stored,

A display for displaying the correspondence between nerves or trials;

A motor learning support method using a motor learning support device equipped with a motor learning support program for causing a computer to execute each step, and a computer-readable recording medium storing the program,

The processing unit reads the neural data of the same part in different trials from the neural data file as reference data and target data according to the data attribute selected in the initial setting,

The processing unit cuts out the data for the length of the reference data force specified time width and creates a template A step of scanning using a template from the initial time to the final time for calculation of the target data, and repeatedly calculating a correlation value for each time between the template and the target data;

The processing unit sets the time when the correlation value between the reference data and the target data is the maximum for the template as the corresponding time, and the reference data ID, the target data ID, and the nerve ID are different in the first corresponding time data file. A step of storing the reference data time and the target data time indicating the corresponding time of the neural information in the same nerve of the trial, and the correlation value at the corresponding time correspondingly stored;

The processing unit advances a predetermined time by a predetermined time until the final time in the reference data, creates a template, repeats the calculating step and the storing step, and the processing unit starts from the first corresponding time data file. Reading the data and displaying the correspondence between the reference data time and the target data time on the display unit;

A motor learning support method, a motor learning support program for causing a computer to execute each step, and a computer-readable recording medium storing the program are provided.

According to a fourth solution of the invention,

A motor learning support method using a motor learning support device, a motor learning support program for causing a computer to execute each step, and a combination recording the program In a readable recording medium,

The processing unit sets, as an initial setting, a data attribute including the first and second nerve IDs and a data ID for determining a trial from the input unit or another device; and

The processing unit reads the same trial nerve data in different nerves from the nerve data file as reference data and target data according to the data attribute selected in the initial setting,

The processing unit creates a template by cutting out the neural data for the entire time to be calculated for each muscle number from the reference data, and scans using the template for the entire time to be calculated for each muscle number of the target data. A step of repeatedly calculating a correlation value for each muscle number between the template and the target data,

The processing unit includes, in the cooperation degree data file, a reference data ID, a target data ID, first and second nerve IDs, a muscle number controlled by the first nerve, and a muscle number controlled by the second nerve. Storing correlation values on a matrix of

The processing unit creates a template for each muscle number in the reference data, repeats the calculating step and the storing step,

The processing unit reads data from the cooperation degree data file, and displays the correlation values representing the degree of motor cooperation of each muscle controlled by the first and second nerves in a matrix on the display unit;

The invention's effect

According to the present invention, it is possible to provide a technique that is useful for support of movement learning after detecting nerve information from movement information of muscles and tendons of the whole body. Further, according to the present invention, a method for detecting how a subtle difference in motion appears as a difference in neural information using neural information mapped from this movement information is provided. Global and local movement Evaluation and comparison can be realized.

[0018] According to the present invention, it is possible to examine the degree of cooperation of distant parts of the whole body by comparing the grouped information, and by comparing the information in the group, the proximal muscle The degree of cooperation between each other can be examined. According to the present invention, it is possible to provide a method for hierarchically processing global information expressing local movements and local information using the structure of the human nervous system. In addition, according to the present invention, comparison and evaluation at the nerve level of motor can be realized, which can be used for support of motor learning.

Brief Description of Drawings

[0019] FIG. 1 is a diagram of the human central nervous system composed of the brain and spinal cord.

[Fig. 2] Explanatory drawing of spinal cord cross section and reflection path.

[Fig. 3] Explanatory diagram of spinal gray matter cross section and anterior horn somatic localization.

[Figure 4] Space layout for C5.

[Fig. 5] An illustration of the slashing action of swinging a sword diagonally.

[Fig. 6] Diagram of nerve information image every 10 frames in C5 spinal nerve during slashing action (1

) o

[Fig.7] Diagram of nerve information image every 10 frames in C5 spinal nerve during slashing action (2

) o

[Fig.8] Diagram of nerve information image every 10 frames in C5 spinal nerve during slashing action (3)

) o

FIG. 9 is an explanatory diagram of the classification of muscles governed by the fifth cervical nerve (C5).

FIG. 10 is a schematic configuration diagram showing a connection relationship of the apparatus.

FIG. 11 is a hardware configuration diagram of the motor learning support device 40.

FIG. 12 is an explanatory diagram of a nerve data file 12 (spinal cord image) (input data).

FIG. 13 is an explanatory diagram of another neural data file 13 (spatiotemporal image 1) (input data).

FIG. 14 is an explanatory diagram of another neural data file 14 (spatiotemporal image 2) (input data).

FIG. 15 is an explanatory diagram of an interface when displaying a spinal cord cross-sectional image.

[Fig. 16] Trial-specific trial time data.

[FIG. 17] Time data corresponding to different trials of different nerves. FIG. 18 is a main flowchart.

[Fig. 19] Spatiotemporal patterns of neural information in (a) myocutaneous nerve and (b) obturator nerve during slashing movement.

圆 20] (a) The fifth cervical nerve (C5) and (b) The second lumbar nerve (

L2) Correlation with corresponding time of neural information in (c) myocutaneous nerve and (d) obturator nerve. The corresponding time is plotted as a blue circle and the similarity is plotted as a red diamond.

圆 21] (a) The fifth cervical nerve (C5) and (b) The second lumbar nerve (

L2), (d) Time chart of nerve information in myocutaneous nerve and (e) obturator nerve. The phase difference of nerve information in C5 and L2 is plotted in (c), and the phase difference of nerve information in myocutaneous and obturator nerves is plotted in (f).

[Fig.22] Modified main flowchart.

FIG. 23 is an explanatory diagram of calculation of the time variation.

[24] Temporal variability of spinal nerve information in cervical and lumbar enlargement during slashing action: (a) C4, (b) C5, (c) C6, (d) C7, (e) C8, (f) L2 , (G) L3, (h) L4, (i) L5, (j) Sl 圆 25] Dimensions of spinal nerve information in cervical and lumbar enlargement during slashing motion.

[FIG. 26] An explanatory diagram of the calculation of the degree of cooperation.

[FIG. 27] Patterns representing the degree of intra-nerve coordination and the degree of inter-nerve coordination: (a) C5 neuronal coordination, (b) Total inter-nervous coordination. The greater the brightness, the higher the degree of cooperation.

圆 28] Explanatory drawing of cooperation degree data file 18 (output data).

(29) (a) Explanatory diagram of the time variability file 19, (b) Explanatory diagram of the dimensional data file 23, (c) Explanatory diagram of the left-right symmetry data file 25.

FIG. 30 is an explanatory diagram of dimensions.

FIG. 31 is an explanatory diagram of symmetry.

[FIG. 32] An explanatory diagram of the degree of cooperation.

FIG. 33 is a hardware configuration diagram of the exercise information nerve information conversion device 30.

FIG. 34 is an explanatory diagram of a neurogeometric data file 1 (input data or intermediate data).

圆 35] (A) Diagram of nerve feature data file 2 (input data), (B) Nerve conduction time data Illustration of single file 3 (output data).

[FIG. 36] (A) An explanatory diagram of a neuromuscular data file 4 (intermediate data), and (B) an explanatory diagram of a nerve branch data file 5 (input data).

[Fig.37] (A) Explanatory diagram of muscle rank data file 6 (output data), (B) Explanatory diagram of muscle feature data file 7 (input data), (C) Data file 8 for extensor flexor muscles (input data) )

FIG. 38 (A) Explanatory diagram of muscle movement data file 9 (input data), (B) Explanatory diagram of spinal nerve sectional coordinate data file 11 (output data).

FIG. 39 is a main flowchart.

FIG. 40 is an explanatory diagram showing a state of data at the time of space arrangement.

FIG. 41 is an explanatory diagram of rearrangement 1 for creating a spatiotemporal pattern.

圆 42] Explanatory drawing of motion definition data26.

FIG. 43 is an explanatory diagram of trial evaluation data 28.

FIG. 44 is an explanatory diagram of position-posture-force data 27.

FIG. 45 is an explanatory diagram of inter-nerve coordination data 21.

BEST MODE FOR CARRYING OUT THE INVENTION

[0020] A. Muscle control model of spinal cord (somatic) nervous system

1. the anatomical structure of the spinal cord

The nervous system is functionally classified into a somatic nervous system and an autonomic nervous system. The somatic nervous system organizes conscious perception, voluntary movement and information. The main role of the autonomic nervous system is to constantly maintain the internal environment of the living body according to changes in the external world and regulate the functions of the organs. In the present invention, attention is focused on the somatic nervous system that controls the movement of the body.

[0021] 1. 1 Spinal nervous system

Figure 1 shows a diagram of the human central nervous system composed of the brain and spinal cord.

The nervous system is anatomically composed of a central nervous system and a peripheral nervous system. Generally speaking, the central nervous system reminds me of the brain. In fact, the brain and spinal cord are collectively called the central nervous system. On the other hand, the peripheral nervous system consists of cranial nerves that connect directly to the organs from the brain and spinal cords that connect to the organs. Consists of the spinal nerves that join. Since the organs that govern each nerve are different, organs can be classified according to the nerve that governs them. In humans, 31 pairs of spinal nerves are counted, cervical nerve (C) 8 pairs, thoracic nerve (T) 12 pairs, lumbar nerve (L) 5 pairs, sacral nerve (S) 5 pairs, coccygeal nerve (Coc) 1 Composed of pairs. These nerves go out of the vertebra through the gaps in the vertebra. In particular, the present invention focuses on the structure of the peripheral nerve that connects the spinal cord and the skeletal muscle, and the spinal cord that controls the majority of skeletal muscles throughout the body.

After exiting the vertebrae, the spinal nerves form a bundle called the plexus, branch again, and spread throughout the body. Here, the fibers contained in different spinal nerves are exchanged, and the nerves ahead are composed of nerves derived from multiple spinal cords. Fifth force The eighth cervical nerve (C5 C8) and the first thoracic nerve (T1) join together to form the brachial plexus and control the upper trunk, upper limbs, upper arm, forearm, and hand muscles. Peripheral nerves that branch off from the brachial plexus include myofascial nerves, median nerves, ulnar nerves, axillary nerves, and radial nerves. Similarly, parts of the first to third lumbar nerves (LI-L3) and the fourth lumbar nerve (L4) make up the lumbar plexus and control the muscles in the pelvis and thighs. From the lumbar plexus, the closing nerve and the femoral nerve come out. From the spinal cord, muscles in a relatively close region are bundled along the spinal cord cross-sectional direction. This includes both extensor and flexor muscles. On the other hand, the peripheral nerves are bundled with muscles having relatively close functions such as extensors and flexors. For example, the muscle cutaneous nerve controls the flexor of the upper arm, and the closing nerve controls the motion of the adductor of the thigh.

[0022] In the embodiment of the present invention, the fifth cervical nerve (C5) and the second lumbar nerve (L2) are taken as spinal nerves, and the percutaneous nerve and the obturator nerve are taken as peripheral nerves branched from these nerves. Neural information that passes through is processed. The spinal nerves and peripheral nerves include both efferent fibers that travel from the spinal cord to the motor organs and afferent fibers that move from the motor sensory organs to the spinal cord. In motor sensory organs such as muscle spindles and Golgi tendon organs, somatosensory information such as muscle length, muscle extension speed, and muscle tension is detected and sent to the spinal cord.

[0023] 1.2 Spinal cord cross-section structure and reflex

Figure 2 shows an illustration of the spinal cord cross-section and reflection path.

A cross-sectional view of the spinal cord reveals butterfly-shaped gray matter and surrounding white matter (Fig. 2, Top). The white matter is a nerve pathway that connects the brain and spinal cord. The gray matter is the junction between the peripheral nerve and the central nerve. Gray matter is divided into a rear corner and an anterior corner. The dorsal horn includes centripetal or sensory-euron and the anterior horn includes efferent or motor-euron. Sensory excitement is transmitted to the dorsal horn cells through the afferent nerve, and excitement is transmitted from these cells to the brain. This excitement is also transmitted to the frontal motility-Euron, which causes muscle movement. The latter-induced muscle response is called reflex as is well known. For example, when a muscle is temporarily extended, momentary contraction occurs. This is called the stretch reflex, and is performed via a small number of -eurons in the spinal cord at a certain height. It is the muscle spindle that senses muscle elongation. The muscle spindles are parallel to the muscle fibers and send information about muscle length and extension rate to the spinal cord via afferent nerves.

[0024] 1.3 Somatic localization of the spinal cord

Figure 3 shows a cross-sectional view of spinal cord gray matter and an illustration of somatic localization of the anterior horn.

In the anterior horn of the gray matter that sends commands to the motor organs, there is a constitution corresponding to the part of the body, that is, somatic localization. From the inside to the outside of the anterior horn 1) Trunk, 2) Trunk to extremity, 3) Limb girdle to extremity, 4) Upper arm and thigh, 5) Forearm and lower leg, 6) Neurons that control the muscles of the hands and feet It is said that the proximally controlled neurons are lined up inside and the distally controlled neurons lined up outside. Furthermore, flexor group control-Euron is arranged on the dorsal side of the anterior horn, and extensor control-Euron is arranged on the ventral side.

[0025] 2. Motor sensory organs and innervating nerves

It is the muscle spindle that senses muscle elongation. The muscle spindles are aligned with the muscle fibers (external muscle fibers). It is the Golgi tendon organ that senses the force generated by the muscle. Other kinesthetic organs include joint receptors that respond to joint forces and nociceptors that respond to muscle and joint pain. Here, we will describe the muscle spindle and Golgi tendon organ, which have the function of feeding back muscle movement information, and its innervation.

[0026] 2.1 Muscle spindle Muscle spindles are made up of muscle fibers in the spindle wrapped in a membrane, and there are two types: nucleus pouch fibers with a bulged center and nucleus chain fibers with a constant thickness. The afferent nerves that control the muscle spindle include group la and group II. The former is spirally wrapped around both nuclear bladder fibers and nuclear chain fibers (primary endings), and the latter is attached to the surface of the nuclear chain fibers (secondary endings) and ends. The primary terminal is strongly excited when the length of the muscle changes greatly (dynamic response), and continues to fire at a constant length when the muscle is kept at a constant length (static response). There is almost no dynamic response at the secondary end.

On the other hand, muscle spindles have efferent innervation. The efferent nerve that contracts muscle is called α motor neuron, and the efferent nerve that contracts muscle spindle is called γ motor-euron. The efferent nerve that contracts both muscles and muscle spindles is the β motor-euron. In particular, γ motor neurons regulate the sensitivity of muscle spindles. Sensitivity improves when the intramuscular muscle fiber contracts due to input from the γ motor-euron.

By integrating signals transmitted through afferent nerves (group Ia, group II) and efferent nerves (α, β, γ motor-Euron), information on muscle length and elongation rate can be obtained in the spinal cord. Divide

[0027] 2.2 Golgi tendon organ

The afferent nerve that governs the Golgi tendon organ is called the group lb. Both ends of the muscle become tendons and are attached to the bone, and the Golgi tendon organ exists in the joint between the muscle and the tendon and in the tendon. Among them, the Golgi tendon organ in the tendon detects the force that acts on the entire muscle.

[0028] 3. Mapping from motor information to somatic nerve information

The anatomical structure centered on the somatic nervous system, especially the spinal nerve, that has been clarified up to the previous section, is summarized.

1. The muscle that controls whole body movement is structured by the spinal cord that controls it.

2. Signals are sent to the anterior horn cell muscle of the spinal cord. Feedback signals from muscle spindles and Golgi tendon organs are sent through the dorsal horn of the spinal cord, partly to the brain and partly to anterior horn cells.

3. Body part localization is seen in anterior horn cells. Muscle length, muscle elongation rate, and muscle tension information are bundled for each spinal cord that integrates efferent and afferent signals and affects the activity of the governing muscle. By placing the muscle movement information during exercise along the anterior horn cell array, it can be converted into neural information inherent in the spinal cord. The data structure is defined using the spinal cord of the fifth cervical nerve (C5), which is known to be particularly developed in the spinal cord.

In this figure, the muscles controlled by C5 are arranged in order according to the arrangement rules for anterior horn cells. In the figure, the first column is the nerve number; the second column is the muscle position; the third column is the extensor flexor muscle; the fourth column is the name of the muscle (muscle name).

Then, the method of expanding this in two dimensions is described. In the XY plane, the flexor is placed in the first and second quadrants, and the extensor is placed in the third and fourth quadrants with the origin at the center. The right and left quadrants are placed in the first and fourth quadrants, and the left and right half are placed in the second and third quadrants. The absolute value of the X coordinate is small, the force is large, the force is directed toward the trunk, and the force is also arranged in order. When one muscle is composed of multiple muscles, arrange them in order of the force closer to the trunk from the smallest y-coordinate absolute value to the largest.

[0030] FIG. 4 shows a spatial layout diagram for C5 in which a grid having a side length of 1 is arranged according to the above rules. (Neural information image every 10 frames in somatic localization of C5 spinal nerve: Frame rate is 30 [frameZsec].) The length of muscle obtained by motion measurement and calculation along the placed mesh and Arrange speed information. By the procedure described so far, mapping from motion information to nerve information can be performed.

[0031] Conventionally, models with muscle control systems! /, Models with lotus cells and motor units, gait control models, cerebellar models, etc. have been proposed. The segmental and topological structures of information at the connecting spinal level were taken into consideration. The method proposed in the present invention attempts to approach physical movement viewed from the inside of the body by defining the data structure of sensory information using anatomical structures. In image training, it is thought to be a clue to specifically obtain the motion commands actually recalled by the brain and the images of the sensory signals obtained from them. When performing a motion, it is thought that an efferent copy that recalls the sensory signal obtained by the motion in advance is sent with the motion command. Yes. The image obtained in this experiment is considered to be the image sequence that forms the basis of this efferent copy.

[0032] Bernciutine stated that the spinal cord was simply a relay for signal transmission to the brain force motor organs, and that all of the control of the movement was transferred to the motor center of the brain. It is now known that the spinal cord is a complex integrated device for coordinating motor functions that are not just relays. The brain force command or sensory signal coming to the anterior horn on the output side is not transmitted directly to the motor-iron, but reaches the intervening-euron. This intervening-Euron has a direct effect on exercise-Euron, or acts in an inhibitory or expeditious manner by intervening in reflexes between muscle receptors and exercise-Euron. The spinal cord and brain work together to regulate the movement of the senses.

[0033] 4. Measurement and calculation of spinal nerve information during whole body exercise

The moving image obtained when the present invention was implemented was confirmed by experiments. Demonstrate an example of a slashing action (the action of swinging a sword while stepping one foot and swinging it down diagonally).

[0034] 4.1 Cutting action

Figure 5 shows an illustration of the slashing action of swinging the sword diagonally.

Samurai slashing is an operation of swinging a sword down diagonally from the neck of the heel along the chest, assuming that a person with a heel in front is standing. Swordsman power Upper left force When cutting to the lower right, the following procedure is followed.

1. Keep your line of sight always in front and keep your sword root in front of the midline. Hold your right foot forward, your left foot behind, and your sword diagonally left front (Fig. 5 (a)).

2. Swing the sword up to the upper right, quickly pass over the head, and move the tip of the sword to the upper left. At this time, step on both feet (when slashing on the spot) (Fig. 5 (b)-(d)).

3. Swing the sword straight from the upper left to the lower right so that the speed increases in front (Fig. 5 (d) —

(f)) o At this time, the tip of the sword is accelerated by the gravity acting on the sword with the shoulder force removed as much as possible.

4. When the sword reaches the lower right, quickly stop the sword. When stationary, left foot is front and right foot is It is located behind (Fig. 5 (f)).

As described above, slashing is a typical whole body coordinated action and requires proficiency.

[0035] 4.2 Motion measurement and calculation of muscle movement information

Using the optical motion capture system, we measured the three-dimensional position of the marker placed throughout the body during the slashing motion. We calculated the joint angle of the human musculoskeletal model by inverse kinematics calculation and the muscle tension by inverse dynamics calculation, and obtained time series data every 33 [msec]. The muscle length was divided by the length in the upright posture and normalized. The human musculoskeletal model (Non-patent Document 1) is composed of 366 muscles, 91 tendons, 34 ligaments, 56 cartilage, and 53 bone groups.

[0036] 4.3 Calculation of spinal nerve information

Extract muscle data controlled by the 5th cervical nerve (C5) from whole body muscle length and muscle tension data. And mapped to neural information. The magnitude of the value is expressed in luminance.

[0037] Fig. 6 shows a diagram (1) of the nerve information image every 10 frames in the C5 spinal nerve during the slashing action (frame rate is 30 [frameZsec] and the length of the governing muscle is coded). .

This figure shows changes in the spinal nerve information pattern that represents the length of the muscles to which the dominant muscle strength is fed back to the spinal cord of the C5 part during the slashing action. C5 mainly controls the upper body muscles, especially the chest, shoulders and upper arms. Both are parts that dynamically expand and contract during the slashing operation. Left diagonal downward force When swinging the sword diagonally right upward, the front saw blade (10: Mus.SerratusAnterior in Fig. 9) placed on the left trunk is stretched. For this reason, the brightness of the region of central coordinate force S (x, y) = (-0. 5, 0.5) was high (40 [frame]). Passing over the head with the sword swinging and tilting the left diagonal upward force, the same muscle on the right side is stretched, so the center coordinate is in the region of (X, y) = (0, 5, 0.5) The brightness became high (60 [frame]).

[0038] Figs. 7 and 8 show diagrams (2) and (3), respectively, of the nerve information image every 10 frames in the C5 spinal nerve during the slashing operation (frame rate is 30 [frame / sec], 7 Is encoded for the rate of elongation of the governing muscle, and in Figure 8 the tension for the governing muscle is encoded).

As shown in the figure, an image sequence representing the distribution of nerve information inherent in the spinal cord of the C5 portion was similarly obtained for the muscle extension speed and the muscle tension. Figures 6 to 8 show the slashing operation as an example, and other operations are also applicable.

[0039] B. Motor learning support device

1.Hardware

FIG. 10 is a schematic configuration diagram showing the connection relationship of this apparatus.

The apparatus includes a motion capture device 10, a motion information calculation device 20, a motion information nerve information conversion device 30, a motion learning support device 40, a presentation device 50, and a storage device 60. The storage device 60 stores a three-dimensional position, motion information, nerve information, motion feature information, nerve feature information, and the like.

[0040] The motion capture device 10 measures the three-dimensional position of the human body and stores the three-dimensional position in the storage device 60 (commercially available: VICON, etc.). The motion information calculation device 20 calculates the length and generated force (motion information) of a motion organ such as a muscle 'tendon' ligament from the measurement result of the motion capture device 10, and stores the motion information in the storage device 60 (commercially available: SIMM etc.). The movement information neural information conversion device 30 converts the movement information obtained by the movement information calculation device 20 into nerve information based on the structural function model of the human nervous system, and stores the nerve information in the storage device 60. . The motor learning support device 40 extracts the motor information obtained by the motor information calculation device 20 and the features of the nerve information obtained by the motor information-nerve information conversion device 30 and stores the motor feature information and the nerve feature information in the storage device 60. To remember. In addition, the motor learning support device 40 refers to the motor information, nerve information, motor feature information, and nerve feature information stored in the storage device 60, and is obtained from the motor information calculation device 20 or the motor information nerve information conversion device 30. By processing in combination with information and nerve information, motor feature information and nerve feature information are obtained and stored in the storage device 60. Although the storage device 60 is described as an external device, the storage device 60 may be provided inside each of the devices 10 to 30 to exchange each information. In addition, the presentation device 50 includes a display device and a motor learning support in the motor information nerve information conversion device 30. Use the display device inside the assisting device 40.

In the figure, as an example, the solid line is online, and the dotted line is offline power.

FIG. 11 shows a hardware configuration diagram of the motor learning support device 40.

This device shows, for example, a hardware configuration in the case of offline 'corresponding time display, and includes a display unit 41, an input unit 42, a processing unit (CPU) 43, an interface unit (iZF) 44, and a storage unit 45. Are the neuronal data files 12-14, time data file 15 for the same-type nerve-specific trials, time data file 17 for heterogeneous nerves same trial, cooperation degree data file 18, time variability data file 19, interneuronal cooperation degree file 21 Dimensional data file 23, left / right symmetry data file 25, motion definition data file 26, position-and-posture force data file 27, and trial evaluation data file 28.

[0042] The data file included in the storage unit 45 will be described below.

FIG. 12 shows an explanatory diagram of the nerve data file 12 (spinal cord image) (input data). Neural data is a pair of neural information that transmits time and an arbitrary point of an arbitrary nerve at a certain time. In the illustrated example, the nerve data file 12 stores nerve data including a nerve ID, nerve information arranged in a spinal cord cross-sectional image at each time, and a data ID for determining a trial with respect to time. The nerve information includes, for example, information obtained from the motor sensory organs, such as muscle length, muscle extension speed, and muscle tension information. In particular, at the spinal cord cross-section, the nerve arrangement is arranged while maintaining the phase structure of muscle and nerve. Neural information is represented by an image, and motion is represented as a moving image. Also, in general, nerve information has a time delay in motor information power due to nerve conduction velocity. Spinal cord information Information from distant limbs is slow spinal cord information. For example, there are reports that the cerebrum recognizes it by canceling the time delay. For this reason, the movement information at the time when the movement actually occurs can be handled as the nerve information.

FIG. 13 is an explanatory diagram of another neural data file 13 (spatiotemporal image 1) (input data). The

Neural data is a pair of neural information that transmits time and an arbitrary point of an arbitrary nerve at a certain time. A spatio-temporal image represents this temporal change in neural information as a still image. In this example, for the spinal nerve and the peripheral nerve, the muscle length and the muscle extension speed are stored for each time and for each position (left and right, trunk, periphery, etc.).

FIG. 14 shows an explanatory diagram of another neural data file 14 (spatio-temporal image 2) (input data).

In the example shown in the figure, the nerve data files 13 and 14 perform the trial with the nerve ID, the nerve information arranged in the position space of the muscle controlled by the nerve from the trunk to the terminal for each time, and the trial. Neural data including a data ID for determination is stored.

These neural data files 12 to 14 are also data stored in the storage unit, and the data also indicate images displayed on the display unit.

FIG. 15 is an explanatory diagram of an interface in the case of spinal cord cross-sectional image display.

On the presentation device 50 or the display unit 41, nerve information on an arbitrary spinal cord cross section is presented as a moving image. When the corresponding spinal cord is selected using the input unit 42 etc., the cross section can be switched interactively.

In addition, as a display method, for example,

• Select multiple to display at the same time

• Display the same spinal nerves side by side with different exercises

• Display a series of actions as a still image sequence

and so on.

Note that the configuration of each data file is merely an example, and an appropriate file configuration can be used as necessary. You can combine or divide the files as appropriate, or change the data items included as necessary. The order of the data items may be rearranged. Further, the output of nerve data or the like is only an example, and a display example appropriately changed or a plurality of display examples can be displayed.

FIG. 16 shows an explanatory diagram of the time data file 15 (output data) corresponding to trials for different types of nerves. The trial-specific time data for different types of nerves are stored in correspondence with the corresponding time of neural information in the same nerve of different trials and the correlation value at that time. In the example shown in the figure, the trial-specific time data file (first corresponding time data file) 15 for the same type of nerve includes a reference data time and a target data time representing the corresponding time of neural information in the same nerve of different trials, and a correlation at the corresponding time The value, reference data ID, target data ID, and nerve ID are stored for different trials of the same type of nerve. Various methods are known for calculating the correlation, but as an example, we use similarity that is strong against noise. The similarity is a value between 1 and 1, and the error is the value obtained by subtracting the similarity from 1. The greater the similarity, the smaller the error. The data type indicates the type of data used in the calculation, such as neural data, time variation data, dimension data, and left / right symmetry data.

FIG. 17 shows an explanatory diagram of the time data file 17 (output data) corresponding to different nerves same trial.

The heterogeneous nerve same trial corresponding time data is a corresponding time of neural information in different nerves of the same trial and a correlation value at that time correspondingly stored. In the illustrated example, the heterogeneous nerve same trial corresponding time data file (second corresponding time data file) 17 includes a reference data time and a target data time indicating a corresponding time of nerve information in different nerves of the same trial, and each nerve. Correlation value 1 and correlation value 2, reference data ID and its nerve ID, and target data ID and its nerve ID are stored for the same trial of different types of nerves. This data is obtained by calculating the corresponding times of nerve 1 and nerve 2 in trial 2 based on the time of trial 1 as reference data. It is obtained by converting the time data file corresponding to trials for different types of nerves. The amount of information generated at the same time in Trial 1 and the force that occurs in Trial 2, that is, the phase difference is known.

FIG. 28 is an explanatory diagram of the cooperation degree data file 18 (output data).

The cooperation degree data included in the cooperation degree data file 18 (output data) is stored in association with the reference data file name, the target data file name, and their correlation values. The reference data and target data may be neural data or time variability data obtained at the preprocessing stage. The larger the correlation value, the greater the degree of cooperation.

In the illustrated example, the cooperation degree data file 18 includes the reference data ID, the first nerve ID, and the pair. The correlation value between the elephant data ID, the second nerve ID, and the muscle number controlled by the first nerve and the muscle number controlled by the second nerve are stored in association with each other.

FIG. 45 is an explanatory diagram of the inter-nerve coordination data file 21 (output data).

Interneuronal cooperation data file 21 includes a set of reference data ID, target data ID, and nerve ID.

The average value of correlation values between these neural data is stored correspondingly. Fig. 27

(b) shows a correlation value representing the degree of cooperation between nerves specified by the set of the first and second nerve IDs in a matrix form on the display unit.

FIG. 29 (a) shows the time variability file 19, FIG. 29 (b) shows the dimension data file 23, and FIG. 29 (c) shows the left-right symmetry data file 25.

The time variability data included in the time variability data file 19 (output data) is generated at the pre-processing stage, and is obtained by converting either or both of the reference data and the target data. The time and the degree of time fluctuation at that time are stored in correspondence. The time variability value increases at the time when the internal state of the input data changes significantly.

[0051] The dimension data included in the dimension data file 23 (output data) is generated at the pre-processing stage, and is obtained by converting either or both of the reference data and the target data. Neural data becomes reference data and target data. The neuron data file name, the original dimension of the neuron data, and the dimension obtained by principal component analysis are stored correspondingly. This means that the smaller the dimension obtained by principal component analysis, the greater the degree of cooperation. The dimension data will be described later.

[0052] The left / right symmetry data included in the right / left symmetry data file 25 (output data) is generated at the pre-processing stage, and is obtained by converting either or both of the reference data and the target data. is there. The time and the left-right symmetry at that time are stored correspondingly. It can be obtained by dividing the input data into neural information for the left and right bodies and correlating the corresponding data. The higher the correlation, the higher the symmetry.

FIG. 43 shows an explanatory diagram of the trial evaluation data file 28. Trial Evaluation Data File 28

The trial evaluation data included in (input / output data) is obtained by storing the evaluation results for trials in association with trial IDs and evaluation values for evaluation items. Prepare yourself Advanced learners' trials and learner's trials accumulated during the exercise process. The motion is evaluated by calculating according to the type of trial from the motion information obtained from the motion information calculation device.

FIG. 44 is an explanatory diagram of the position / posture / force data file 27. Position-posture-force data file 27 The position-posture-force data contained in the input data (input data) is any position and Z or posture and Z Or it is stored in association with information representing the generated force, and includes a data ID for defining the body or tool part and a data ID for defining the trial. Position / posture—The data includes the motion capture device 10 and the motion information calculation device 20 and Z or storage device 60 that calculates the length and force (motion information) of the motion organ such as the Z or muscle 'tendon' ligament. It can be obtained by connecting.

FIG. 42 shows an explanatory diagram of the action definition data file 26. Action definition data file 26

The action definition data included in (input data) is stored in association with the trial ID and the action name and Z or the action person. The processing unit 43 presents the action selected based on the action definition data as reference data, processes the learner's action as target data, and presents the comparison result. The learner changes the movement so that they approach.

[0056] Among the above data files, trial-specific time data file 15 for different types of nerves, heterogeneous neuro-same trial-compatible time data file 17, time variability data file 19, dimension data file 23, cooperation degree data file 18, inter-nerve The cooperation degree data file 21 and the left / right symmetry degree data file 25 represent neural information. The correspondence time, the degree of time variation, and the left / right symmetry can be processed even if the input data is motion information in addition to the neural information. In this case, the output data represents motion feature information.

[0057] 2. Software

2.1 Motor learning support processing of the first embodiment

FIG. 18 is a flowchart of the motor learning support process according to the first embodiment. When the processing is started, the processing unit 43 performs initial setting (S101). After that, the processing unit 43

, Read reference data and target data from neural data file 12 or 13 or 14 (S 103), pre-processing is executed (S105). Next, the processing unit 43 sets the initial time (S107), cuts out the template with the specified time width for the reference data force (S109), and calculates the correlation with the template over the specified area of the target data (SI 11). For different trials of the same type of nerves, the correlation between the time in the reference data and the maximum correlation value of the target data and the time is stored in the time data file 15 for trials of different types of homologous nerves (S 113). . The processing unit 43 determines whether it is the final time (S115), advances the unit time by the unit time until the final time (S117), executes post-processing, and obtains the corresponding time and correlation value for the same trial of different types of nerves. Is stored in the time data file 17 corresponding to different nerves and the same trial (S119), and the result is presented (displayed) on the display unit 11 (S121). Steps S109, Sill, and S113 are repeated to complete the process. Details of each step will be described later. As described above, the corresponding time of the reference data and the target data and the set of correlation values at that time are obtained. Information on the local delay or advance of the movement, ie, phase difference, is obtained.

Here, the relationship between data will be described.

The processing unit 43 reads a plurality of neural data in any one of the neural data files 12 to 14, performs a correlation operation based on these data, obtains a corresponding time and a correlation value, and uses the same kind of nerve-specific trial corresponding time data. Store in file 15 and time data file 17 corresponding to different nerves in same trial.

[0059] Details of the processing of each step will be described below.

(Initial setting: S101)

As initial settings, the following should be set in advance when the processing unit performs analysis. These items may be set as default values in advance. The processing unit 43 may input these set values through another input device or IZF, or may read data stored in advance from the storage unit.

(1) Select the reference data and target data attributes. Data attributes include the nerve part (neural ID), the type of trial (data ID), and neural information characteristics. Here, an attribute including multiple nerve parts (nerve IDs) can be set for repeated processing.

• The nerve part (nerve ID) is identification information for identifying the nerve such as the nerve name and nerve number, and the spinal cord name representing the nerve (spinal cord etc.) cross section, for example, cervical nerve C18, thoracic nerve T1 — 12, Lumbar nerve LI-5, sacral nerve SI-5, coccygeal nerve Cocl 31 in total. The nerve site may be a peripheral nerve.

• Trial type (data ID) includes all exercises. For example, for the reference data, select advanced trials of experts, masters and teachers, trials with the best or lowest results. For the target data, a trial in the process of motor learning is selected. In this example, the reference data ID and target data ID are used to define the neural information of different trials or the neural information of different nerves of the same trial.

• Neural information characteristics include muscle length, muscle extension speed, and muscle tension information, which are information obtained from motor sensory organs.

In this example, the reference data and the target data are both nerve data of the same part in different trials. Not limited to this, appropriate neural data may be used.

(2) Select the template time range. If the time width is too short, the corresponding time is likely to go back and forth, and if the time width is too long, a fine change in the time width is detected.

(3) Set the minimum threshold value of the time variability required for preprocessing.

(4) Select the correlation calculation method. Typical examples include similarity and Euclidean distance.

(5) Select the display format. For example, select the data format (cross section, spatiotemporal, etc.) such as the spatial arrangement of the output data, or one or more. For example, select one of the patterns of neural data files 12-14. In the case of spatio-temporal, information at a certain time is arranged in a horizontal row, and the power representing time in the vertical direction or vice versa. In addition, there are formats such as arranging neural information at the same time of multiple movements, and arranging neural information at multiple times of a single movement in parallel.

[0060] (Reading reference data and target data: S 103)

The processing unit 43 reads the reference data having the attribute selected in the initial setting and the target data from the neural data file 12 or 13 or 14. In this example, the reference data and the target data are both nerve data of the same part in different trials.

FIG. 19 shows an example of reference data and target data. This example shows the spatiotemporal pattern of neural information via (a) myocutaneous nerve and (b) obturator nerve when two trials of slashing are performed. It is The muscle length data of the muscles controlled by these nerves were extracted and normalized with the length of the upright posture. The magnitude of the value is expressed in luminance. Trial 1 is the reference data (or reference data) and Trial 2 is the target data (or target data).

[0062] (Pretreatment: S 105)

The processing unit 43 obtains the time variation degree for the reference data and the target data, and obtains the correlation start time and end time. Note that the processing unit 43 may obtain the time variability in advance and store it in the time variability data file 19 so that it can be read and used in this step. The processing unit 43 does not calculate the correlation when the time variation is smaller than the threshold value. By adding this pre-processing, it is possible to obtain an effect of removing a portion where the change in time is small and the corresponding time is not clear. This preprocessing may be omitted. The method for obtaining the time variation will be described later.

[0063] (Initial time setting: S107)

The processing unit 43 sets an initial time. The initial time can be obtained from the preprocessing (S105), or when the preprocessing S105 is omitted, the initial time of the data is used.

[0064] (Cut out template: S109)

The processing unit 43 cuts out data having a length corresponding to the specified time width from the current time from the reference data and uses it as a template. In other words, the current time is t, the specified time width is N, and the data up to time t + N is extracted for the time t force. In this embodiment, N = 9. Note that the specified time width can be initialized or predetermined in step S101.

[0065] (Correlation between template and target data: SI 11)

In the target data, the processing unit 43 correlates the data from an arbitrary time t ′ to t ′ + N between the initial time and the final time to be calculated with the template cut out from the reference data (here: (Similarity) is calculated. The similarity is defined as x for the reference data template pattern and y for the template size pattern of the target data.

f (x, y) = (x-y) / (| x | | y |)

It is represented by The similarity score ranges from 1 to 1, taking 1 for perfect match, 1 for the same size but opposite orientation. This process is repeated from the initial time to the final time for which the target data is calculated. [0066] (Calculation of correspondence time between reference data and target data: SI 13)

The processing unit 43 obtains the similarity to all the times that are the calculation target of the target data for the template, and the time having the maximum similarity is set as the corresponding time. The processing unit 43 stores the nerve, data type (neural data, time variation data, dimension data, left / right symmetry data, etc.), reference data ID, target data ID, different trials in the homologous trial-specific time data file 15. The reference data time indicating the corresponding time of the nerve information in the same nerve, the corresponding time corresponding to the target data, and the same nerve type trial corresponding time data including the correlation value at that time are stored correspondingly. As described above, the processing unit 43 can calculate the difference between the movement pattern and timing at the nerve level.

[0067] (Repeated processing: S115, S117)

The processing unit 43 determines whether the extraction of the template in the reference data is the final time (S115), advances the template start time to the final time by the unit time (S117), steps S109, Slll, Repeat SI 13. The last time is obtained by preprocessing or the last time of data. However, in all cases, the length N of the template is subtracted. (Multiple data attributes, repeated processing: S116, S118)

When the processing unit 43 repeats the processing for a plurality of nerves (S116), the data attribute set in the initial setting (S101) or the data attribute set in the input unit 42 or another device is used for another part. Then, the same processing is performed and the data is stored in the time data file 15 corresponding to the same-type nerve trials.

[0068] (Post-processing: S119)

The processing unit 43 reads a plurality of homogeneous nerve trial-response time data from the homogeneous nerve trial-response time data file 15 and associates them. For example, the processing unit 43 calculates the corresponding times of nerve 1 and nerve 2 in trial 2 based on the time of trial 1 of nerve 1 as reference data. For example, the processing unit 43 generates homologous nerve-specific trial response time data including the corresponding times of nerve 1 trial 1 and trial 2 and homogenous nerve-specific trial response time data including corresponding times of nerve 2 trial 1 and trial 2. In this case, the processing unit 43 uses the same ID for Nerve 1, the reference data ID that is the data for Nerve 1 trial 2, the ID for Nerve 2, the target data ID that is the data for Nerve 2 trial, the data type, and the like. Reference data indicating the corresponding time of neural information in different trial nerves The time corresponding to the target data corresponding time and the correlation value at that time are stored in the heterogeneous nerve same trial corresponding time data file 17 as the different nerve same trial corresponding time data. As a result, it is possible to know how much the data at different parts that occurred at the same time in trial 1 shifts in trial 2, that is, the phase difference.

[0069] (Presentation of processing result: S121)

The processing unit 43 reads the corresponding time data from the homologous nerve-specific trial corresponding time data file 15 and Z or the heterogeneous nerve same trial corresponding time data 16, executes display processing, and displays it on the display unit 41. Examples of procedures for plotting graphs used for presentation (plots (1) to (3)) are shown below. Plot (1)

The correspondence time and the method of presenting the similarity at that time are described. First, the processing unit 43 reads homogenous nerve-specific trial corresponding time data from the homogenous nerve-specific trial corresponding time data file 15, and takes the time of the reference data on the horizontal axis and the time and similarity of the corresponding data on the vertical axis, or A graph with the time of reference data on the horizontal axis and the similarity or error of the corresponding data on the vertical axis is displayed. The corresponding time and similarity may be plotted on the same graph. Since the similarity is extremely high and close to 1, the error value obtained by subtracting the similarity from 1 can actually be plotted. The error indicates the likelihood of the corresponding time. It can be said that the response time at the time when the error is small is accurate.

[0070] Plot (2)

The processing unit 43 reads the homogenous nerve-specific trial corresponding time data from the homogenous nerve-specific trial corresponding time data file 15. In order to clarify the difference in timing, the processing unit 43 may take the time axis horizontally, arrange the reference data and the target data vertically, and connect the corresponding times with a line. As a result, when all the corresponding times are connected, it becomes difficult to see the corresponding times. Therefore, the corresponding times may be connected by a line for each arbitrary time width ΔΤ. As a result, time advance and delay can be visually presented.

[0071] Plot (3)

The processing unit 43 converts the heterogeneous nerve identical trial corresponding time data file to the heterogeneous nerve identical trial pair. Read from the time data 17. Since the processing unit 43 easily presents the phase difference between different parts, the corresponding times of the different parts of the target data may be connected with a line using the time of the reference data. As a result, when all the corresponding times are connected, it becomes difficult to see the corresponding times. Therefore, the corresponding times may be connected by a line between the corresponding time intervals ΔΤ. This makes it possible to visually present the phase difference of each part in different trials.

FIG. 20 shows an example of plot (1).

This is a plot of the similarity of neural information between the same spinal nerve and peripheral nerve and the corresponding time. The degree of similarity is drawn with a diamond, and the corresponding time is drawn with a circle. Figures 20 (a) and (b) correspond to the fifth cervical nerve (C5) and the second lumbar nerve (L2), and Figures 20 (c) and (d) correspond to the myofascial nerve and the obturator nerve. The horizontal axis represents the time of trial 1, the vertical axis represents the corresponding time of trial 2, and the similarity between the times. A degree of similarity is extremely high strength Tsutano, fully taking the difference from when they match (1) and plotted ¹⁰⁴ times the error simultaneously. The larger the value, the greater the pattern difference.

[0073] When the degree of similarity between the spinal nerve and the peripheral nerve is compared, the spinal cord can take time relatively smoothly, but the pattern difference is large. On the other hand, peripheral nerves tend to have smaller errors (higher similarity) than the spinal cord, although the time correspondence is not smooth. The corresponding time plot of the fifth cervical nerve (C5) (Fig. 20 (a)) has a gentle slope of 1 or less in the first half. The slope becomes almost 1 after the latter half 75 [frame]. Trial 2 shows that the rate of change in the first half is larger than Trial 1. On the other hand, in the second lumbar nerve (L2) (Fig. 20 (b)), the slope between 65 [frame] and 70 [frame] is steep, and the slope at other times is gentle. Compared to Trial 1, Trial 2 has a slower rate of change for the steep part. Myoderma (Fig. 20 (c)) has a large gap in the corresponding time around 65 [frame]. Also, before and after that, the corresponding time takes a constant value for a while. The value of the error is higher than before and after that, and the corresponding time between 60 and 70 [frame] is not reliable. The obturator nerve (Fig. 20 (d)) shows a tendency similar to that of the second lumbar nerve (L2), with a steep portion near 65 [frame].

[0074] How to read the corresponding time plot is arranged. The case where the horizontal axis represents the time of reference data and the vertical axis represents the time of target data will be described. If the slope is greater than 1, the same change This shows that the target data occurred in a longer time than the reference data. In other words, the target data is slower than the reference data. Conversely, if the slope is less than 1, it indicates that the same change occurred in the target data, but in a shorter time than the reference data. That is, the target data is faster than the reference data. Such a local speed change is indicated by a slope.

FIG. 21 shows an example of plots (2) and (3).

Figure 21 shows a time chart for the two trials. In Fig. 21 (a), (b), (d), and (e), the upper horizontal axis is the time of trial 1, and the lower horizontal axis is the time of trial 2. Figure 21 (c) is based on the time of trial 1 in Figure 21 (a) and (b), and Figure 21 (c) is based on the time of trial 1 in Figures 21 (a) and (b). Represents the corresponding time between different nerves in trial 2. In Fig. 21 (c), the upper horizontal axis is the time of the fifth cervical nerve (C5), and the lower horizontal axis is the corresponding time of the second lumbar nerve (L2). In Fig. 21 (f), the upper part of the horizontal axis is the time of the myelinated nerve and the lower part of the horizontal axis is the corresponding time of the closed nerve. The corresponding time was taken every 5 [frames]. Muscle skin trial 1, 60-75 [frame] The corresponding time is not reliable, so the line connecting the corresponding times is drawn with a dotted line, and the others are drawn with a solid line.

[0076] The timing shift of the fifth cervical nerve (C5) is concentrated in the first half (FIG. 21 (a)). In trial 1, 20 [frame] force from 50 to 70 [frame] In trial 2, it corresponds to 10 [frame] from 26 to 36 [frame], which is half the time. In other words, a similar pattern appeared twice as fast. On the other hand, in the second half, 36 forces 46 [fra me] correspond to 70 to 80 [frame] in trial 1, and the speed is the same. Deviations in the timing of the second lumbar nerve (L2) can be seen little by little in the first and second half (Fig. 21 (b)). Trial 1 from 65 to 70 [frame] corresponds to Trial 1 from 35 to 40 [frame], and the average speed is the same. Before and after that, it takes ll [frame] for 15 [frame] and 7 [frame] for 10 [frame], and the start and end speeds are slightly faster. Looking at the correspondence time between C5 and L2 in trial 2 (Fig. 21 (c)), the activity of C5 starts later than L2 and proceeds earlier, and the second half progresses later than L2. The myocutaneous nerve (Fig. 21 (d)) shows the same trend as C5, but the correspondence between 60 and 70 [frame] in trial 1 is not clear and unknown. Looking at the correspondence at other times, the overall speed is increasing. When comparing the corresponding time of the closed nerve (Fig. 21 (e)), 60-65 [frame] of trial 1 corresponds to 30-37 [frame] of trial 1, and the speed is a little slow. Other than that, the speed increases little by little as a whole! It ends in time. Looking at the corresponding times of the musculoskeletal and obturator nerves in Trial 2 (Fig. 21 (f)), the average speeds of the first and the last are the same, and the time between them is shorter for the muscular skin as a whole.

As described above, in the trial of this example, it was found that the spatio-temporal patterns for each nerve are similar and the generation timing is different.

[0077] 2.2 Motor learning support processing according to the second embodiment

FIG. 22 is a flowchart of the motor learning support process according to the second embodiment. In the second embodiment, compared to the first embodiment, step S109 for cutting out the template from the reference data is omitted, and the processing unit 43 performs the entire time for which the reference data is not calculated within the predetermined time width. The correlation between the target data and the total time is directly calculated (S112). Then, the processing unit 43 stores a set of reference data, target data, and an overall correlation value (S114). Accompanying this, the initial time setting S107, the determination of whether the final time is S115, and the step S117 of advancing the unit time by the final time are omitted. Note that the calculation of the corresponding time described in FIG. 18 may be performed as preprocessing (S105) in FIG. 22, and the force and correlation may be calculated by matching the corresponding times at a certain time. In other words, in step S101, settings related to the template can be omitted. In step S112, the processing unit 43 executes correlation calculation by a known or known method, and in step S114, the processing unit 43 performs the reference data ID, the first nerve ID, the target data ID, and the second data ID. Correlation values between the muscle number controlled by the nerve ID of the first nerve ID and the muscle number controlled by the nerve of the first nerve ID and the muscle number controlled by the second nerve ID are associated and stored in the cooperation degree data file.

[0078] 2.3 Modified example of preprocessing

The following example of pre-processing (S105) is a variation that can be executed in the second embodiment of the exercise support processing of FIG. 22, and can be selected in advance by, for example, initial setting (S101). Note that even if applied in the exercise support processing of the first embodiment in FIG. Good.

[0079] (Calculation of pre-processing time variability: S105)

FIG. 23 is an explanatory diagram of the time variation calculation method executed in the preprocessing S105. In order to obtain the time variability of each of the reference data and the target data, the processing unit 43 cuts out the reference data or the target data from time t to t + N as a template, and times t + S t of the same data. To the correlation between t + S t + N. This is repeated in order from the initial time to the end time. The processing unit 43 stores the time variability data including the nerve, the reference data ID or the target data ID, and the time variability with respect to time in the time variability file 19.

[0080] A strong similarity to noise is used as a powerful example of the correlation calculation. When using similarity, it takes a value between -1 and 1. In many cases, the correlation of neural information is generally large and close to 1. Therefore, in order to see how the time fluctuates, the value obtained by subtracting the similarity from 1 is calculated to obtain the time fluctuation. The greater the similarity, the smaller the variation. As described above, it is possible to detect the degree of fluctuation from the one hour step. By changing the time width N of the template, the calculation target of the variability can be changed. Changes are detected by taking into account absolute values when N = l, speed when N = 3, acceleration when N = 5, and time differential of acceleration when N = 7.

When investigating which muscles are in particular state transitions, it is only necessary to calculate the temporal variation of movement information for each muscle that is controlled by the nerve. In particular, it is possible to examine the strength of which muscle information changes greatly.

FIG. 24 shows, as an example, the degree of temporal fluctuation of nerve information during the slashing operation.

It shows the time variation of the neural information input near the cervical enormous part (C4 C8) and the lumbar enormous part (L2- L5, SI) of the spinal cord. Figures 24 (a)-(e) correspond to C4-C8, respectively, and Figure 24 (f)-(j) forces SL2-L5, S1 respectively. Here, the time width N = 9. The cervical and lumbar regions of the spinal cord dominate the upper and lower limbs, respectively. Comparing the degree of variability between the cervical and lumbar areas, the cervical areas are more bimodal. In the cervical enormous part, the peak in the second half is larger than the first half in the upper part. On the contrary, in the lumbar region, the lower peak is larger at the lower part. Fluctuation is smoother in the hips than in the necks It is. The peak time in the first half is earlier in the cervical enormous area except for C4, and the peak time in the latter half is earlier in the lumbar enormous area. The muscle activity changes synchronously from spinal cord to spinal cord, and the synchronization timing is slightly different. It can be seen that the peak time and size differ depending on the nerve. It is possible to compare the global differences in whole-body cooperative behavior for each local unit of nerves. The time when the time variation takes the maximum is the state transition time of the neural data.

[0082] (Preprocessing Dimension calculation: S 105)

FIG. 30 shows an explanatory diagram of dimensions.

The pre-processing S 105 shows the dimension calculation method executed.

The processing unit 43 calculates dimensions by performing known or well-known principal component analysis on the neural data (reference data, target data). In this example, specifically, the number of elements whose cumulative contribution rate exceeds the threshold is calculated. The processing unit 43 stores the dimension data corresponding to the nerve ID, the reference data ID or the target data ID, and the nerve ID in the dimension data file 23. The smaller the dimension obtained by principal component analysis with respect to the original number of elements, the greater the degree of cooperation.

In this step, the processing unit 43 obtains a dimension for each neural information, for example. The principal component analysis and the dimension divided by the original number of elements is called the contraction rate. The processing unit 43 calculates this for each nerve. The global degree of cooperation can be expressed and compared in the form of dimensions. If the dimensions of the same movement differ greatly from person to person, the degree of coordination of the movement will vary greatly.

Note that the principal component analysis is a method of summarizing information on many types of variables that are correlated with each other into a small number of comprehensive characteristic values that are uncorrelated with each other. Let n variables (n dimensions) observations be represented by m (m dimensions) comprehensive indices (principal components). As an indicator of how much each principal component represents the features included in the original data, or how many principal components can be used to sufficiently represent the features included in the original data, There is a cumulative contribution rate. The sum of the variances up to the m-th principal component is defined as the proportion of the total variance.

FIG. 25 shows an example of dimension calculation.

Regarding the slashing action, the dimensions of the principal component in the spinal neck enormous area (C4–C8) and the lumbar area (L2–S1) of somatic nerve information were obtained. Number of elements whose cumulative contribution exceeds 0.8 From 3 to 7. The contraction rate obtained by dividing the dimension obtained by principal component analysis by the number of elements was around 0.1 regardless of the position of the spinal cord. Figures 25 (a) and 25 (c) plot the dimensions of each nerve in the form of a line graph and a bar graph. Figure 25 (b) shows the reduction ratio. Similarly, when the upper kicking motion was calculated, the nerve information dimension for each spinal cord was 6-10, and the contraction rate was around 0.2. The results obtained represent the dimensions of the data needed to approximate the time pattern with a linear vector. The information on somatic nerves during whole body exercise was extremely degenerate for each spinal cord, and the degree of coordination was high. In addition, the two types of motions that were tested also showed that the degree of degeneration depends on the position of the spinal cord. This is thought to be because neural information is internally coupled by the spinal cord neural network.

[0086] (Preprocessing symmetric calculation: S105)

FIG. 31 shows an explanatory diagram of symmetry.

In the pre-processing S 105, the calculation method of the left-right symmetry executed is shown.

The input neural data is obtained by dividing the neural information of the left half of the body and the information of the right half of the body, and by correlating the muscle movement data that are symmetrical and controlling the same part, and taking the difference of the neural information. The smaller the absolute value, the higher the symmetry. The processing unit 43 stores the difference corresponding to the nerve ID, reference data ID or target data ID, and nerve ID in the left-right symmetry data file 25 as left-right symmetry data.

[0087] In order to calculate the symmetry of the neural information, the left half data from a certain time t to t + N is cut out as a template, and the right half data from the time t to t + N of the same neural information is extracted. Calculate the correlation with. From t = T (initial time) to t = T (end time)

0 1

repeat. Similarity is used for correlation calculation as well as temporal variation. The result obtained is called the degree of symmetry, and the value obtained by subtracting the similarity from 1 is called the degree of asymmetry. The greater the degree of similarity, the greater the degree of symmetry and the smaller the degree of asymmetry. Here, by changing the time width N of the template, the object of calculation of symmetry and asymmetry can be changed. The symmetry is detected by taking into account the absolute value when N = l, the speed when N = 3, the acceleration when N = 5, and the time derivative of acceleration when N = 7.

[0088] When performing continuous movements, for example, the basics of martial arts, such as breaking the girder, changing body, and cutting the sword In operation, the posture is often reversed left and right in the initial state and the final state. At this time, the timing of transition to the initial state force final state varies depending on the muscle and nerve. In transition, it is encouraged to keep the front facing as short as possible. To achieve this, each line must be switched in a timely manner. The timing of when and when to reach the final state in the shortest time or through the intermediate state depends on the region, but these actions can be improved by being aware of the timing of muscle switching. The correlation between the reference data and the target data is calculated, and it can be said that the different part is the part where the horizontal flip timing is different.

[0089] 2.4 Modification of correlation calculation

The following variation of the correlation calculation is mainly the force which is the step S112 of the motor learning support process of the second embodiment. The same process is performed after the step S 119 of the motor learning support process of the first embodiment. You can do it for processing!

[0090] (Correlation calculation Cooperation degree calculation: S112)

Fig. 32 shows an illustration of the degree of cooperation.

FIG. 26 shows a calculation method of the degree of cooperation performed in the correlation calculation S112 between the reference data and the target data.

The reference data and the target data are in the (spatio-temporal image) format of the neural data file 13 or 14, respectively. The processing unit 43 calculates the correlation between the dominating muscle movement data one by one for the nerve data file. The processing unit 43 sequentially shifts the template cut out from the reference data along the streak number (horizontal direction). The correlation between all the muscle movement information included in the reference data and the target data is calculated. As described above, the degree of coordination of movement information between muscles controlled by a certain nerve is detected. The higher the degree of similarity, the lower the degree of cooperation between the muscles, and the lower the degree of cooperation. This represents a local degree of cooperation within the nerve. The processing unit 43 arranges the obtained cooperation degree in a matrix of each muscle number as shown in the figure, and stores it in the cooperation degree file 18 together with the reference data ID, the first nerve ID, the target data ID, and the second nerve ID. Remember.

[0091] Figure 27 (a) shows the corresponding reference data on the vertical axis and the target data on the horizontal axis. A state where the furniture is arranged on a plane is shown. The number of elements of the neural data is not necessarily the same, and may not be the same nerve. For this reason, the correlation between different nerves can be calculated. Similarity was used for correlation. The time variability may be obtained at the preprocessing stage, and the correlation between the reference data and the time variability of the target data may be calculated. The higher the brightness, the higher the degree of coordination. The brightness of each point represents the degree of coordination of movement information between the same innervating muscles. Note that the time variation of neural data may be used as an input for calculating the degree of cooperation. In addition, a normal value obtained so that peak values are aligned at 1 may be used. By performing the pre-processing as described above, it is possible to examine the degree of cooperation between the state transition time and the transition process regardless of the absolute value of the change amount.

[0092] 2.5 Post-processing and presentation processing conversion examples

(Post-processing Calculation of inter-nerve coordination: S119)

Step S112 of the motor learning support process according to the second embodiment After the calculation of the degree of cooperation S119, a calculation method of the degree of cooperation between nerves executed is shown.

[0093] The processing unit reads the correlation value from the cooperation degree data file 18 specified by the reference data ID and the first nerve ID, the target data ID and the second nerve ID, and stores the correlation value in the cooperation degree data file 18. The average of the correlation values on the matrix is calculated and stored in the interneuronal cooperation data file 21. That is, the processing unit has a correlation value on a matrix of a plurality of first nerve ID sets in a predetermined reference data ID and a plurality of second nerve ID sets in a predetermined target data ID. The average is stored in the interneuronal cooperation data file 21. This average is the average of the correlation values of multiple muscles controlled by the same nerve. For example, the value of one cell in the matrix of nerve coordination (FIG. 27 (b)) is the average value of all the cells in the nerve coordination matrix (FIG. 27 (a)). The processing unit repeats this process for all combinations of the first nerve ID group and the second nerve ID group. That is, the processing unit calculates an average value for each of the other first nerve IDs in the predetermined reference data ID and the other second nerve IDs in the predetermined target data ID. Repeat the memory.

[0094] Next, the processing unit reads data from the inter-neuron cooperation degree data file 21 and displays the correlation values representing the inter-nerve cooperation degree specified by the first and second nerve ID pairs in a matrix form. To display. In this way, by calculating the correlation between nerves, it is possible to obtain a global degree of cooperation between nerves in the whole body movement, and the inter-nerve cooperation degree data file 21 shown in FIG. 45 is obtained.

[0095] FIG. 27 (b) shows the correlation values representing the degree of cooperation between nerves specified by the set of the first and second nerve IDs in a matrix on the display unit. In other words, it is possible to calculate the state of local and global cooperation by calculating the degree of cooperation hierarchically for each nerve.

[0096] (Presentation Image presentation: S121)

Presentation The image presentation method executed in S121 is shown.

In addition to displaying the processing results as line graphs, bar graphs, or plane plots, the reference data and the target data may be displayed in an overlapping manner, displayed side by side, or correlations and differences may be displayed. The learner will practice to approach the advanced neural data, which is the reference data, or to generate a movement that goes beyond that towards the pattern of success.

[0097] (Presentation voice presentation: S121)

In Presentation S121, the voice presentation method executed is shown.

Even if analysis is performed at the muscle or nerve level, it is a challenge to present this in a form that the learner can recognize again. In some cases, it may be difficult to apply force by visual observation. In this case, the processing unit 43 can convert the sound into speech. An example of how to convert neural information into speech information is shown. This makes it possible for learners to process somatosensory sensations in the auditory cortex, which is limited to the somatosensory cortex.

• The processing unit 43 assigns sounds having different frequencies for each nerve or muscle, and converts the magnitude or change of nerve information into sound pressure and volume.

[0098] At this time, the learner exercises to produce the same chord.

• The processing unit 43 pays attention to the main muscles and plays sounds using percussion instruments with slightly different timbre or volume at the time when the temporal fluctuation of neural information including the main muscles and main muscles reaches a peak.

• The processing unit 43 pays attention to the main muscles and plays sounds using percussion instruments with slightly different tones or volumes at times when the left and right symmetry of the nerve information including the main muscles and the main muscles peaks. [0099] Since it takes time to hear the sound and react to the power, the learner listens to the timbre rhythm first and then changes so that the same timbre and rhythm can be produced at the same timing. Let me. This method can convey detailed movements to visually impaired people at the muscle and nerve level. It is expected to be effective in learning dance. Therefore, this method realizes universal design. Even if you are not visually impaired, you can present dynamic exercises to the learners who are unable to see the screen and wear the head-mounted display.

[0100] 2.6 Initial variation on reference data and evaluation value calculation method for setting Here we describe how to select reference data based on the motor learning mechanism. There are two types of kinematics. One is supervised learning that imitates models and learns patterns. The other is unsupervised learning or reinforcement learning in which repeated exercises are used to search for movements that suit your body. The latter, in particular, tries to get closer to the objective while assessing whether the results meet the purpose. There is brain science knowledge that the former is mainly performed in the cerebellum and the latter in the basal ganglia. Therefore, for example, advanced trials, masters, and teachers can be stored in the reference data. In another example, the reference data may store the learner's own trials in the motor learning process as iterative learning progresses, evaluate them, and organize the best or lowest trials. it can. The objective evaluation criteria differ depending on the type of movement. For example, in the case of a slashing action, (a) the start point and (b) the position of the end point of the sword tip, (c) the straightness of the trajectory connecting the start point and the end point, (D) adequate relaxation of the whole body, especially the arm, (e) dynamic stability of the whole body, (f) constant waist height, etc. Supporting the supervised learning process when model trials are selected for reference data and compared with learner trials. Support the unsupervised reinforcement learning process when choosing a self-highest or lowest evaluation trial and comparing it to a learner trial.

The following evaluation value calculation method is applied to the motor learning support processing of the first and second embodiments, and the main change steps will be described below.

Note that the setting of whether to support an unsupervised learning process or a supervised learning process is For example, in the initial setting in step S101 or in an appropriate step, the setting can be made as appropriate from the input unit, the storage unit, or another device.

[0102] (Initial setting operation definition: S101)

When supporting the supervised learning process, the processing unit 43 reads the motion definition data 26. FIG. 42 shows an example of the action definition data 26. The trial ID, action name, and operator are stored in correspondence. When an action name and an operator are set, the processing unit 43 searches for a corresponding trial ID and uses it to identify reference data. As an operator, it is recommended to set a teacher, a master, and an expert who are skilled in operation.

[0103] (Initial setting Evaluation item: S101)

When supporting the unsupervised learning process, the processing unit 43 reads the trial evaluation data 28. Figure 43 shows an example of trial evaluation data28. The trial ID and the evaluation value corresponding to the evaluation item are stored correspondingly. When the evaluation item is set, the processing unit 43 searches for the trial ID having the maximum or minimum evaluation value and uses it for specifying the reference data.

[0104] (Initial setting evaluation function: S101)

When supporting the unsupervised learning process, the processing unit 43 reads the position and orientation power data 27. Fig. 44 shows an example of position / posture force data27. To determine the position of the body or any tool used for body movement, Z or posture, Z or generated force, data ID for defining the body or tool part, and trial The data ID is stored. Establish an evaluation function and specify how to calculate the evaluation value of trial from position / posture force data or neural data.

The position / posture / force data is connected to the motion capture device and the motion information calculation device that calculates the length and force (motion information) of the motion organ such as Z or muscle 'tendon' ligament. You can get it.

[0105] (Post-processing evaluation: S119)

When supporting the unsupervised learning process, the processing unit 43 calculates an evaluation value of the trial using the evaluation function from the position / posture power data or the neural data, and the trial evaluation data file 28 stores the trial ID, the evaluation item, and the evaluation. The values are stored in association with each other. The evaluation value can be calculated, for example, by using an appropriate evaluation function using predetermined data for each item. [0106] As a specific example, description will be given using an example of a slashing operation. It is recommended that the hip height should not change as much as possible when stepping on the slashing action. Beginners tend to float for a moment when they step on their feet, with their hips becoming taller and lower at the beginning and end. In contrast, the skilled person changes his / her foot without greatly changing the height at which the waist is low. For this reason, the whole body is stable at the moment of stepping. Such an evaluation function for the slashing action (f) The evaluation function for the constant waist height is defined as follows. Taking the Z axis from the floor in the direction opposite to gravity, the waist height is expressed by the Z component of the waist position. If the Z component time series of the waist position is expressed as [z (0), _Z (1), _Z (2), z (n)], the total sum of changes in the height direction of the waist height is

[0107] [Equation 1]

^ \ z (i -z (i-l) \. The smaller this is, the smaller the vertical movement of the waist height, so the evaluation function f (z) is, for example,

[0108] [Equation 2] f (z) = -∑ \ z (i) -z (i-\) \

i = l

Or

f (z) ^ \ / ^ \ z (i) -z (i-l) \

(However, when the denominator is 0, is a predetermined value.) That is, in this example, the input data for determining the evaluation value is the time-series Z component of the position data of the position / orientation / force data. The evaluation value is calculated by substituting the time component Z component of the position data into the evaluation function f (z).

In this way, trial data on the process of motor learning can be accumulated while being evaluated each time, and the results can be used in subsequent trials.

[0109] (Presentation evaluation value: S121) When supporting the unsupervised learning process, the processing unit reads data from the trial evaluation data file 28 and displays a combination of trial ID, evaluation item, and evaluation value on the display unit. During repeated trials, the learner will be able to carry out kinematic learning while confirming whether each trial is strong or bad.

[0110] C. Motor information Neural information converter

Exercise Information Regarding the nerve information conversion device, there is a patent application by the present inventors (Japanese Patent Application No. 2004-176455, filed on June 15, 2004). Can be incorporated).

[0111] 1. Hardware

FIG. 33 shows a hardware configuration diagram of the exercise information / neural information conversion apparatus 30.

This device shows, for example, a hardware configuration in the case of offline 'spinal cord cross-sectional image display, and includes a display unit 31, an input unit 32, a processing unit (CPU) 33, an interface unit (I / F) 34, and a storage unit 35. Prepare.

The storage unit 35 includes a nerve geometry data file 1, a nerve feature data file 2, a nerve conduction time data file 3, a nerve one-corresponding data file 4, a nerve branch data file 5, a muscle rank data file 6, a muscle feature data file 7, Extensor flexor muscle data file 8, muscle movement data file 9, spinal nerve cross-section coordinate data file 11, and nerve data files 12-14.

Note that the storage unit 35 or the processing unit 33 or the like may be partly or entirely shared with the storage unit 45 or the processing unit 43 or the like of the motor learning support device.

[0112] The data file included in the storage unit 35 will be described below.

Fig. 34 shows an explanatory diagram of the neurogeometric data file 1 (input data or intermediate data). As shown in Table 1, the neurogeometric data stored in the neurogeometric data file 1 includes the nerve number, the corresponding spinal cord name, muscle Name, spinal cord and muscle, and nerve line names (columns) between them are stored in correspondence. A nerve line name can also define a nerve as a point sequence. In addition, Since nerve lines have characteristics such as conduction velocity and conduction time, nerve lines are defined separately from nerve points. In Table 2, the start point name and the end point name are stored for the nerve line name. The start and end points of nerve lines are collectively called nerve points. Table 2 is used in combination with data that correlates nerve point names and nerve point coordinates as shown in Table 3.

FIG. 35 (A) shows an explanatory diagram of the nerve feature data file 2 (input data).

The neural geometry data stored in the nerve feature data file 2 is the one in which peripheral nerve names and nerve line trains are stored correspondingly as shown in Table 1, and the nerve line name and conduction velocity as shown in Table 2. Are configured correspondingly. The conduction velocity includes centripetal and centrifugal nerve conduction velocities. In this example, the neural geometric data and the neural feature data are separated from each other. However, the neural geometric data and the neural characteristic data are only examples, and may be appropriately configured without being separated. As an example, only the afferent nerve conduction velocity is used here.

FIG. 35 (B) shows an explanatory diagram of nerve conduction time data file 3 (output data).

The nerve conduction time data is stored in correspondence with the nerve number for the nerve number.

FIG. 36 (A) shows an explanatory diagram of the nerve-muscle correspondence data file 4 (intermediate data).

The data corresponding to a single nerve is a combination of the muscle name and information on the spinal nerve (horizontal axis) and peripheral nerve (vertical axis) that control the muscle. As an example, the figure shows neuromuscular data related to spinal nerve (C8). Here, a part of the whole-body neuromuscular correspondence is shown, but in fact it can be defined for the whole body. Such correspondence tables can be created based on specialized anatomy books. In addition, the neurogeometric data force can also be calculated by using each information. Using this data, the processing unit 33 can search for the muscle that is controlled by the spinal nerve of interest and the peripheral nerve that controls the muscle. For example, when focusing on the spinal nerve C8, the search for muscles requires the ulnar carpal flexor corresponding to the vertical arrow, and the search for peripheral nerves requires the ulnar nerve corresponding to the horizontal arrow.

FIG. 36 (B) shows an explanatory diagram of the nerve branch data file 5 (input data).

Nerve bifurcation data is a representation of the spinal nervous system in a tree structure with the spinal cord as the root and the muscle as the leaf. A contact point represents a via point or a start point (spinal cord), an end point (muscle), a branch point, and a branch represents a nerve pathway. Here, the nerve pathway is taken as a branch, but the nerve pathway itself is also expressed as a contact point. There is also a way to do it.

FIG. 37 (A) shows an explanatory diagram of the muscle rank data file 6 (output data).

The muscle ranking data stores information indicating muscle ranking, muscle characteristics (left and right, extensor flexor muscle classification, muscle region classification), and muscle name. Here, the force that shows a part of the muscles of the whole body can actually be defined for the whole body.

FIG. 37 (B) shows an explanatory diagram of the muscle feature data file 7 (input data).

The muscle feature data stores information representing muscle characteristics (left and right, extensor flexor muscle classification, muscle part classification) for the muscle name. Here, the force showing a part of the muscles of the whole body can actually be defined for the whole body. The muscle parts are classified into, for example, the following six types: 1) trunk, 2) trunk to limb, 3) limb band to limb, 4) upper arm, thigh, 5) forearm, lower leg, 6) hand and leg.

FIG. 37 (C) shows an explanatory diagram of the extensor-flexor correspondence data file 8 (input data).

Extensor flexor muscle correspondence data is a pair of muscle names belonging to the corresponding flexor muscle group and extensor muscle group. Corresponding muscles are not necessarily paired just because they have the same force that is considered to correspond to each other at almost the same site. On the other hand, there are cases where the response is made across multiple sites. For this reason, the parts included in the extensor flexor muscle data are combined together.

FIG. 38 (A) shows an explanatory diagram of the muscle movement data file 9 (input data).

Muscle movement data includes the time and length of any muscle at a certain time, length change speed, force, force change speed, etc. All muscle movement information is a pair. There can be a format in which multiple pieces of information (force and length) are placed in the same file, multiple pieces of information are placed, and multiple pieces of information at a certain time are combined. Here, the file name is the name of the line, and by specifying the name of the line, it is read into the file contents cache. In the figure, as an example, the time change of the length of the biceps is shown. The length may be an absolute value or a value that is standardized by the length of the initial posture or standard posture. The same applies to changes in muscle length.

FIG. 38 (B) shows an explanatory diagram of the spinal nerve sectional coordinate data file 11 (output data).

The spinal nerve cross-section coordinate data stores the spatial arrangement of nerves in the spinal cord cross-section with respect to the nerve number. Also, coordinate data on the 2D plane is r The Θ coordinate system may be used. Furthermore, instead of the coordinate data, identification information indicating the position of the spatial arrangement may be used.

[0122] 2. Software

2.1 Main flow

Figure 39 shows the main flowchart.

When the processing is started, the processing unit 33 performs initial setting (S101). Thereafter, the processing unit 33 executes nerve conduction time calculation (S103), muscle rank calculation (S105), and nerve cross section spatial arrangement calculation (S107). Next, the processing unit 33 sets an initial time (S109), executes conversion processing from exercise information to nerve information (S111), and presents (displays) the result on the display unit 11 (S113). The processing unit 33 determines whether it is the final time (S115), advances the unit time by the unit time until the final time (S117), repeats steps S111 and S113, and ends the processing. Details of each step will be described later.

[0123] 2.2 Processing of each step

Details of each step will be described below.

(Initial setting: S101)

As initial settings, the following should be set in advance when the processing unit performs analysis. The processing unit 33 may input these set values via another input device or IZF, and may read the stored data from the prestored data.

(1) Select the spinal cord name representing the nerve (spinal cord, etc.) cross section. For example, the spinal cord is composed of a total of 31 cervical nerves, 12 thoracic nerves, 5 lumbar nerves, 5 sacral nerves, and 1 coccyal nerve.

(2) Select motion data (throw, jump, motion characteristics (speed, force, etc.)).

(3) Select the display format. For example, select the data format (cross section, spatiotemporal, etc.) and the singular or plural of the output spatial arrangement. For example, select the pattern of neural data files 12-14. In the case of space-time, information at a certain time is arranged in a horizontal row, and represents the time in the vertical direction or vice versa. In addition, there is a form in which neural information at the same time of multiple movements and neural information at multiple times of a single movement are arranged in parallel [0124] (Nerve conduction time calculation: S 103)

When the processing is started, the processing unit 33 obtains the spinal cord from the neural geometric data file 1 (Table 1) for each nerve number based on the spinal cord name representing the nerve (spinal cord, etc.) section selected in the initial setting S101. The name and the name of the nerve line (column) are extracted. Depending on the nerve, one or more strings of nerve line names are included. Next, the processing unit 33 obtains the starting point name and the ending point name of the neural line from the neural geometric data file 1 (Table 2) based on the extracted neural line name (column), and further, the neural point of the starting point name and the ending point name. Based on the above, the length of each nerve line (column) is calculated by retrieving the nerve point coordinates from the neural geometric data file 1 (Table 3). The processing unit 33 determines the length of each nerve line (column) and the nerve feature data file 2 (Table 2) force according to each nerve line (column). The centripetal (or efferent) nerve conduction velocity force of the read nerve feature data Calculate the conduction time of the nerve line (column). Further, the processing unit 33 calculates the nerve signal conduction time of the entire nerve path to any arbitrary spinal force represented by one or more nerve lines (rows). In this way, the processing unit 33 stores the nerve signal conduction time in the nerve conduction time data 3 corresponding to the nerve number.

[0125] (Straight ranking calculation: S 105)

In the muscle rank calculation, the processing unit 33 calculates the rank of muscles belonging to the same region after classifying the extensor and flexor muscles and the region, and further calculates the rank within the same muscle. This process is necessary to determine the spatial arrangement.

[0126] When the muscle rank calculation is started, the processing unit 33 controls the spinal cord selected with reference to the single nerve-corresponding data in the single nerve-corresponding data file 4 based on the spinal cord name selected in the initial setting S101. The peripheral nerve corresponding to the muscle name is obtained, and the muscle name is further classified according to peripheral nerves. The processing unit 33 refers to the muscle feature data in the muscle feature data file 7 for the classified muscle names, and determines and classifies the extensor and flexor muscles based on the muscle names. The processing unit 33 refers to the tree branching data of the nerve branch data file 5 and the leaf order in which the root force also branches in the same peripheral nerve (for example, the order close to the root or the order in which there are few contacts) Sort by. The processing unit 33 refers to the nerve conduction time data 3 for the muscle name corresponding to each nerve number, and based on the nerve conduction time, the same part Sort them in order of short conduction time between different peripheral nerves. At this time, for example, between the same types of different peripheral nerves, the shortest conduction times are compared, and the shorter peripheral nerve is arranged first. In this way, the processing unit 33 stores the muscle rank data in the muscle rank data file 6 in association with the rearranged muscle rank, left and right, extensor 'flexor, muscle part number, and muscle name. In addition, the processing unit 33 displays the created spinal cord cross-sectional coordinate data on the display unit or outputs it via the IZF unit as necessary.

[0127] The muscle ranking can be determined in the order of 'deepness close to the trunk, using muscle geometric data. In addition, the muscle rank may be determined in the order of short nerve conduction time determined from the neural geometry data. Not limited to these, it may be determined in the reverse order, or may be determined in an order that is determined according to appropriate arrangement, such as determining the order using the displacement of the geometric data of muscle and nerve. .

[0128] (Spatial layout calculation of nerve section: S107)

FIG. 40 is an explanatory diagram showing an example of a state of data at the time of space arrangement. Based on the muscle rank data read from the muscle rank data file 6, the processing unit 33 classifies each record (corresponding to a muscle name) into left and right, extensor 'flexor muscles, and muscle parts, and sets a predetermined nerve (spinal cord). Etc.) Place in a space related to the cross section. The space to be placed is stored in the memory unit and connected to each nerve (spinal cord, etc.) cross section! Different region shapes may be used, and the same region shape or a plurality of region shapes may be used. In this example, the cells where each muscle name is placed are divided into a matrix in a shape simulating spinal cord gray matter. First, as shown in Fig. 40, the spatial arrangement is classified into 4 types, left and right, up and down (flexor's extensor), and further classified into 6 parts in the horizontal direction, and each muscle name at the corresponding position. Place. Then, the processing unit 33 rearranges and arranges the muscles with high (low, long) and low (long, short) muscles according to the muscle ranking and away from the central axis. This is almost equivalent to the method of rearrangement in the order of muscle classification and muscle placement strength. Further, the processing unit 33 combines the muscle control nerves by the extensor-flexor muscle correspondence data read from the extensor-flexor correspondence data file 8, and rearranges them in the upper (flexor) and lower (extensor). Thus, a plurality of muscles having the same function are associated with a plurality of muscles having opposite functions. If the corresponding muscle group spans two parts, they are stacked vertically.

[0129] Next, the processing unit 33 matches the classified muscle innervating nerve group with a predetermined space. . That is, the processing unit 33 determines whether or not the classified muscle innervating nerve group exceeds a preset number N. This is a device to prevent the height from increasing or decreasing extremely, and the number of N can be determined arbitrarily. If the processing unit 33 exceeds N, the processing unit 33 divides the number so that the maximum number is less than N. At this time, divide equally into heights that do not exceed the set height N. If equal division is not possible, the larger X coordinate absolute value is left as the remainder. If either the extensor or flexor muscle exceeds N, the remaining muscular control nerve group is also divided into the same number. On the other hand, when the height N is less than the set height N, the processing unit 33 combines the heights that do not exceed the X coordinate absolute value in ascending order. At this time, it can be combined so that the maximum number is less than N. In addition, the flexor and extensor muscles may be connected in correspondence, and may be connected when neither exceeds N. Furthermore, if there is a gap, the processing unit 33 packs the X coordinate absolute value in the direction of decreasing toward the y axis that is the symmetry axis. The processing unit 33 creates the spinal cord cross-sectional coordinate data and the spinal cord neurological cross-sectional coordinate data file 11 in the spatial arrangement thus created. In addition, the processing unit 33 displays the generated spinal cord cross-sectional coordinate data on the display unit or outputs it via the IZF unit as necessary. The spinal cord coordinate data may use identification information for identifying the position of the cell in the spatial arrangement.

In addition, as another embodiment of the flowchart of the spatial arrangement calculation, the processing unit 33 may be arranged in a flat manner without dividing by the set number N in the spatial arrangement calculation of the neural section.

FIG. 41 shows an explanatory diagram of rearrangement 1 for creating a spatiotemporal pattern. In rearrangement 1, the processing unit 33 rearranges the extensor and flexor muscles while rearranging them in the order of the muscle parts.

[0131] (Transformation from motor information to neural information: S 111)

The processing unit 33 advances the time by the unit time, executes the conversion process from the exercise information to the nerve information (S111), and presents (displays) the result on the display unit 11 (S113).

When the conversion from the motion information to the nerve information is started, the processing unit 33 extracts the conduction time for the nerve corresponding to each nerve number from the nerve conduction time data read from the nerve conduction time data 3. The processing unit 33 obtains a muscle name for each nerve number with reference to the neurogeometric data file 1 according to the exercise data set in the initial setting S 101, and the muscle name is set in the muscle exercise data file 9. Motion data BJC sticks out.

[0132] Here, the processing unit 33 determines whether or not the force is in consideration of the conduction time delay. When considering the delay, the processing unit 33 extracts data before the conduction time from the obtained time, and calculates motion data by interpolation when there is no data at the corresponding time. On the other hand, the processor 33 extracts the motion data at the time obtained from the muscle motion data file 9 when the delay is not considered.

Next, the processing unit 33 uses the spinal cord cross-section coordinate data read from the spinal nerve nerve cross-section coordinate data file 11 to map the movement data to the nerve data. For example, depending on the value of the muscle movement information determined in the muscle movement data by the muscle name and each time for each nerve number at the position of the nerve of the nerve number determined in the spinal cord cross-section coordinate data, Give changes. The processing unit 33 converts the neural data thus created into the neural data file 12 B 0

Similarly, the neural data file 13 14 can be created by rearranging as appropriate.

D. Conclusion

What has been described in the present embodiment is mainly summarized as follows.

'An overview of the anatomy of the nervous system, especially the spinal cord. Using the geometric structure of the anterior horn cells of the spinal cord and the structure of the peripheral nervous system, we proposed a method for mapping muscle movement information and processing the spinal nerve information and peripheral nerve information obtained.

• We proposed a method to calculate the similarity when body movements are observed from the inner nervous system. We extracted the spatiotemporal pattern of the neural information from the observation data of the whole body movement, and calculated the local pattern similarity of the neural information. We proposed a method to find the corresponding time between different trials from the similarity and compare the difference in the timing of the generation of similar patterns. By processing different trials of the same person and the same action, we found that similar patterns occur at different timings, and showed the effectiveness of the present invention.

• Proposed a method for obtaining temporal changes in neural information. This technique can extract the characteristics of whole body movement observed from the nervous system. An afferent god who conducts experiments and enters the spinal cord A temporal and spatial pattern of trans-information was processed. It was clarified that the difference in peak size and timing is required during the whole body coordinated action, and the effectiveness of the present invention was shown. Here, the corresponding time was obtained by the most basic calculation of similarity, but it would be possible to process more robustly by applying various methods of pattern recognition.

• We proposed a method to classify muscle movement data during whole body coordinated action by dominant spinal cord and calculate the dimension of each nerve.

• We proposed a method to calculate the left / right symmetry for each dominant spinal cord and detect when the left and right are switched.

• We proposed a method to calculate the degree of cooperation between muscles controlled by the same nerve and the degree of cooperation between different nerves.

• We proposed a method to support motor learning by presenting these differences in neural activity pattern levels with images, sounds, etc.

[0135] A feature of the present invention is that, based on knowledge of neuroanatomy, a whole body movement pattern is divided into base patterns for each nerve, and comparison between noturns is performed. Somatosensory information of the whole body is bundled for each peripheral nerve and sent to the spinal cord, and somatosensory information is processed to some extent locally at the nerve level. Comparing the spatio-temporal patterns of information flowing from one nerve to another for different trials of the same action, for example, the following question can be answered: How different are the spatio-temporal patterns? Is the spatiotemporal pattern of each nerve similar, and is the timing of pattern generation different for each nerve?

[0136] In this embodiment, in particular, the ability to compare the similarity of two trials is shown. By comparing a series of actions when the same person performs repetitive practice, the tendency of how to tune the action is shown. You can see it. If you can extract the tuning method of a master with excellent physical skills, it will lead to the development of a training method to approach the master's skill. This method can be extended so that different humans can compare the same actions or different actions.

[0137] In motor learning, the factors that hinder the realization of the target movement can be broadly divided into three stages. First stage Inability to correctly detect the resulting movement Second stage Even if motion is detected, there is no way to correct it

The third stage

By associating a set of objective evaluation and neural data, and comparing at the level of neural data

, Helps to remove the first and second stage factors.

The present invention provides a means for reaching spinal nerve information during exercise by a non-invasive method called exercise measurement. Motion measurement is a classic method with the power of the times that is a powerful means of measuring cranial nerves. However, in combination with the present invention, there is a possibility that it becomes a third cranial nerve measurement technique that is different from direct nerve measurement and brain measurement such as PET and fMRI.

[0139] The motor learning support method or the motor learning support apparatus / system of the present invention includes a motor learning support program for causing a computer to execute each procedure, a computer-readable recording medium storing the motor learning support program, and a motor It can be provided by a program product that includes a learning support program and can be loaded into the internal memory of the computer, or a computer such as a server that includes the program.

Claims

The scope of the claims

A display for displaying the correspondence between nerves or trials;

With

The processing unit sets the time when the correlation value between the reference data and the target data is the maximum for the template as the corresponding time, and the reference data ID, the target data ID, and the nerve ID are different in the first corresponding time data file. Reference data indicating the corresponding time of neural information in the same nerve of the trial Data time and target data time, and means for storing the correlation value at the corresponding time correspondingly,

A motor learning support device.

[2] In the display means,

The motor learning support device according to claim 1, wherein the processing unit displays a plurality of target data times corresponding to a plurality of reference data times and a relationship indicating Z or a correlation value or an error on the display unit.

[3] In the display means,

2. The motor learning support device according to claim 1, wherein the processing unit displays a relationship between corresponding times of the reference data and the target data with respect to time on the time axis.

[4] Reference data time and target data time indicating the corresponding time of nerve information in different nerves of the same trial, first and second correlation values for each nerve at the corresponding time, reference data ID and its nerve ID The second corresponding time data file that stores the target data ID and its nerve ID for the same trial of different types of nerves

Further comprising

In the first nerve, the processing unit obtains the first homologous trial-specific time data obtained using the reference data as the first trial and the target data as the second trial, and the reference data in the second nerve. Means for reading from the first corresponding time data file, the second same-type neurobiological trial corresponding time data obtained as the first trial and the target data as the second trial;

Based on the reference data time of either the first or second trial that is the reference data, the processing unit takes the correspondence of the second trial that is the target data for the first nerve and the second nerve. A means of obtaining the time;

The processing unit obtains the data time of the second trial of the first nerve, the reference data time, and the data time of the second nerve. With the data time of the second trial as the target data time, the first and second nerve IDs, reference data ID, target data ID, and the first and second trials in the first nerve at the corresponding time Means for associating and storing the first correlation value and the second correlation value for the first and second trials in the second nerve in the second corresponding time data file;

The processing unit reads the second corresponding time data file force data, and displays the corresponding time in different nerves of the same trial on the display unit.

The motor learning support device according to claim 1, further comprising:

With

The processing unit creates a template by cutting out the neural data for the entire time to be calculated for each muscle number from the reference data, and scans using the template for the entire time to be calculated for each muscle number of the target data. Correlation between the template and target data for each muscle number Means for repeatedly calculating a value;

The processing unit creates a template for each muscle number in the reference data, and repeats the means for calculating and the means for storing;

The processing unit reads data from the cooperation degree data file, and displays the correlation values representing the degree of movement cooperation of each muscle controlled by the first and second nerves in a matrix on the display unit;

A motor learning support device.

6. The motor learning support device according to claim 1 or 5, wherein the nerve ID includes a spinal cord name or a peripheral nerve name representing a nerve cross section.

7. The motor learning support device according to claim 1, wherein the nerve information is any one of muscle length, muscle extension speed, and muscle tension.

[8] In the preprocessing means, the processing unit extracts a predetermined time width of the reference data or the target data as a template, and uses a correlation value with the reference data or the target data shifted by a predetermined time as the time variability. The nerve ID, the reference data ID or the target data ID, the time variability with respect to the time are stored in the time variability data file, and the processing unit reads the reference data and the target data in the time variability data file. The motor learning support device according to claim 1, wherein force reading is performed and subsequent processing is executed.

[9] In the means for performing the preprocessing, the processing unit performs principal component analysis on the reference data or the target data, and obtains the nerve ID, the reference data ID or the target data ID, and dimension data for the nerve signal. Stored in a dimensional data file,

6. The kinematics learning support device according to claim 1, wherein in the reading means, the processing unit reads the reference data and the target data from the dimensional data dolphins and executes the subsequent processing.

[10] In the means for performing the preprocessing, the processing unit is symmetrical and displays neural information of the same nerve. The difference is taken for the stored nerve data, and the nerve ID, data ID, and difference are stored in the right / left symmetry data file,

6. The motor learning support device according to claim 1 or 5, wherein in the reading means, the processing unit reads the reference data and the target data from the left / right symmetry data file, and executes the subsequent processing.

[11] The processing unit according to claim 1 or 5, wherein the processing unit assigns a different type of sound to each nerve or muscle, converts the magnitude or change of the nerve information into a sound pressure level, and audibly displays the sound on the display unit. The motor learning support device according to the description.

[12] The motor learning support device according to claim 1 or 5, wherein the processing unit displays an audible display by the display unit so that a tone color or a sound volume is different at a time when the temporal variation degree of the nerve or the main muscle becomes a peak. .

[13] The motor learning support device according to claim 1 or 5, wherein the processing unit displays an audible display by the display unit so that a tone color or a sound volume is different at a time when the left-right symmetry of the nerve or the main muscles reaches a peak. .

[14] An action definition data file is further provided for storing the action type and Z or the action person in association with the data ID for defining the trial.

The processing unit, as an initial setting, means for setting the type of operation and Z or the operator from the input unit or other device,

A processing unit that reads the operation definition data file;

The processing unit specifies the data ID of the corresponding trial using the action definition data according to the type of action and Z or operator selected in the initial setting, and uses this to set the reference data ID

The motor learning support device according to claim 1 or 5, further comprising:

[15] The processing unit, as an initial setting, sets a teacher, master, expert, etc. who are skilled in operation as an operator.

15. The motor learning support device according to claim 14, further comprising:

[16] A trial evaluation data file storing a data ID for defining a trial and an evaluation value of the trial for each evaluation item is further provided. The processing unit has means for setting evaluation items from the input unit or other device in the initial setting; and

The processing unit reads a combination of a data ID for determining a trial and a trial evaluation value corresponding to the evaluation item, and

The processing unit identifies the trial data ID that maximizes or minimizes the evaluation value of the set evaluation item, and uses this to set reference data

[17] Information on the arbitrary position and Z or posture and Z or generated force of the tool used for the body or body movement with respect to the time, the data ID for determining the body or tool part, and trial A position-posture-force data file containing a position-posture-force data including a data ID for determining

The processing unit includes means for reading the position-posture-force data into the position-posture-force data file force,

The processing unit is a means for setting an evaluation function based on an evaluation item in an initial setting, and the processing unit is a means for calculating an evaluation value of a trial of position-posture-force data or neural data force using the evaluation function;

The processing unit stores a trial ID, an evaluation item, and an evaluation value in the trial evaluation data file in association with each other, and

The processing unit reads data from the trial evaluation data file, and displays a combination of trial ID, evaluation item, and evaluation value on the display unit.

The motor learning support device according to claim 16, further comprising:

[18] The processing unit is a motion capture device, and a motion information calculation device that calculates motion information of Z or muscle 'tendon' ligaments such as length and generated force 'motion information, and a position and orientation from Z or storage device. A means of acquiring one-off data;

The motor learning support device according to claim 17, further comprising means for storing position / posture first effort data in a position / posture first effort data file.

[19] Among the reference data ID and first nerve ID pair, the target data ID and second nerve ID pair, and the first nerve ID pair, the neural data of any nerve ID and the second nerve ID Any of the pair Interneuronal cooperation data file that stores neural ID neural data and correlation values between them

Further comprising

The processing unit reads a correlation value from the cooperation degree data file specified by the reference data ID and the first nerve ID, the target data ID and the second nerve ID,

The processing unit calculates a mean of correlation values on a matrix stored in the cooperation degree data file;

The processing unit displays correlation values on a matrix of a plurality of first nerve ID sets in a predetermined reference data ID and a plurality of second nerve ID sets in a predetermined target data ID. Means for storing the average in the inter-nerve coordination data file;

The processing unit calculates the means and the means for storing each of the other first nerve IDs in the predetermined reference data ID and each of the other second nerve IDs in the predetermined target data ID. Means to repeat and

The processing unit reads the inter-neuron cooperation degree data file force data, and displays the correlation values representing the inter-nerve cooperation degree specified by the set of the first and second nerve IDs in a matrix on the display unit.

The motor learning support device according to claim 5, further comprising:

A display for displaying the correspondence between nerves or trials;

Reading and Z or writing to the neural data file and the first corresponding time data file, obtaining the correspondence between nerves or trials, displaying the correspondence on the display unit, and the first corresponding time data file And a processing unit for storing A motor learning support method using a motor learning support device equipped with

The processing unit creates a template by cutting out data with a length corresponding to the reference data force specified time width, scans the template from the initial time to the final time, which is the calculation target of the target data, and scans the template and the target data. A step of repeatedly calculating a correlation value for each time of

Motor learning support method including.

The reference data ID, the first nerve ID, the target data ID, the second nerve ID, the nerve data specified by the muscle number controlled by the first nerve, and the muscle number controlled by the second nerve. A degree-of-cooperation data file that stores a correlation value with the specified nerve data, a display unit that displays the correspondence between nerves or trials, and

In a motor learning support method using a motor learning support device,

Motor learning support method including.

A display for displaying the correspondence between nerves or trials;

In a motor learning support program in a motor learning support device equipped with

The processing unit advances the time by a predetermined time until the final time in the reference data, and the template And a step of repeating the calculating step and the storing step, the processing unit reads data from the first corresponding time data file, and determines a correspondence relationship between the reference data time and the target data time. An exercise learning support program for causing a computer to execute the step of displaying on the display unit;

In the motor learning support program in the motor learning support device,

The processing unit includes, in the cooperation degree data file, a reference data ID, a target data ID, first and second nerve IDs, a muscle number controlled by the first nerve, and a muscle number controlled by the second nerve. of Storing correlation values on a matrix;

A motor learning support program to make a computer execute.

A display for displaying the correspondence between nerves or trials;

A computer-readable recording medium that records a motor learning support program in a motor learning support device equipped with

The processing unit performs preprocessing for the reference data and the target data, including setting an initial time for correlation calculation, The processing unit creates a template by cutting out data with a length corresponding to the reference data force specified time width, scans the template from the initial time to the final time, which is the calculation target of the target data, and scans the template and the target data. A step of repeatedly calculating a correlation value for each time of

A computer-readable recording medium on which a motor learning support program for causing a computer to execute is recorded.

A computer-readable recording medium that records a motor learning support program in a motor learning support device,

As an initial setting, the processing unit receives the first and second nerve IDs and the test from the input unit or other devices. Setting a data attribute including a data ID for defining a row;