JP2022018477A - Learning execution device, program, and learning execution method - Google Patents

Abstract

To provide a learning execution device for executing learning for causing a muscle model, which is modeled muscle movement, to perform a target operation, a program, and a learning execution method.SOLUTION: A learning execution unit 100 for communicating with a plurality of communication terminals via a network comprises: an information storage unit which stores a muscle model in which muscle is operated by contracting a muscle fiber connected to a motor neuron included in an exercise unit according to a firing pattern of a plurality of intervening neurons each of which is connected with an exercise unit; an operation setting unit for setting a target operation of a muscle model; and a learning execution unit which learns a firing pattern for achieving a target operation by executing learning for giving a compensation to a firing pattern whose operation of a muscle model is closer to a target operation among a plurality of firing patterns.SELECTED DRAWING: Figure 4

Description

The present invention relates to a learning execution device, a program, and a learning execution method.

In the field of CG (Computer Graphics), a simulation method based on muscle contraction has been known (see, for example, Non-Patent Documents 1 to 6). In the conventional simulation method, a so-called hill type model and a so-called CPG (Central Pattern Generator) or the like have been used.
[Prior Art Document]
[Non-patent literature]
[Non-Patent Document 1] Thomas Geitenbeek, Michiel van de Panne, AF vds Flexible muscle-based locomotion for bipedal creatures. ACM Transactions on Graphics, (206), 2013.
[Non-Patent Document 2] Jack M. Wang, Samuel R.Hmner, SLVK Optimizing locomotion controllers using biologically-based actuators and objectives. ACM Trans. Graph, 31 (4), 2012.
[Non-Patent Document 3] Yoonsang Lee, Moon Seok Park, TKJL Locomotion control for many-muscle humanoids. ACM Transactions on Graphics, 33 (6), 2014.
[Non-Patent Document 4] Sehee Min, Jungdam Won, SLJPJL Softcon: simulation and control of soft-bodied animals with biomimetic actuators. ACM Transactions on Graphics, 38 (6): 208: 1-208: 12, 2019.
[Non-Patent Document 5] Cecila Laschi, Matteo Cianchetti, BML m. MFPD Soft robot arm inspired by the octopus. Advanced Robotics, 26 (7): 709-727, 2012.
[Non-Patent Document 6] Jungdam Won, Jongho Park, KKJL How to train your dragon: Example-guided control of flapping flight. ACM Transactions on Graphics, 36 (4): 1: 1-1: 12, 2017.

According to the first aspect of the present invention, a learning execution device is provided. The learning execution device stores a muscle model that operates a muscle by contracting muscle fibers connected to the motor neurons included in the motor unit according to the firing pattern of a plurality of interneurons to which the motor unit is connected. May have a part. The learning execution device may include a motion setting unit that sets a target motion of the muscle model. The learning execution device is a learning execution unit that learns the firing pattern, and realizes the target motion by executing learning that rewards the firing pattern in which the motion of the muscle model is closer to the target motion among a plurality of firing patterns. It may be provided with a learning execution unit for learning the firing pattern.

The learning execution unit operates the muscle model according to each of the plurality of firing patterns, generates a plurality of firing patterns based on a firing pattern in which the motion of the muscle model is closer to the target motion, and the plurality of firing patterns. By operating the muscle model according to each of the above and repeating the generation of a plurality of firing patterns based on the firing pattern in which the motion of the muscle model is closer to the target motion, the firing pattern that realizes the target motion is learned. It's okay. The learning execution unit operates the muscle model according to each of the plurality of randomly generated firing patterns, and generates a plurality of firing patterns based on a firing pattern in which the movement of the muscle model is closer to the target movement. The target motion is realized by operating the muscle model according to each of the plurality of firing patterns and repeatedly generating a plurality of firing patterns based on the firing patterns in which the motion of the muscle model is closer to the target motion. You may learn the firing pattern. The learning execution unit operates the muscle model according to each of the plurality of firing patterns generated based on the learned firing pattern, and the motion of the muscle model is closer to the target motion based on the firing pattern. By generating an ignition pattern, operating the muscle model according to each of the plurality of ignition patterns, and repeating the operation of the muscle model to generate a plurality of ignition patterns based on the ignition pattern closer to the target motion. You may learn the firing pattern that realizes the above target motion. When the muscle model is operated based on the firing pattern, the learning execution unit may grow the motor unit in which the muscle fibers are contracted. The muscle model may include a fast muscle motor unit and a slow muscle motor unit, and the learning execution unit may exercise the fast muscle when the muscle model is operated based on the firing pattern. The unit and the slow muscle motor unit may be grown according to different criteria. The information storage unit includes a first parameter indicating whether the muscle is a fast muscle or a slow muscle, a second parameter indicating contractile energy, and a maximum value of the second parameter with respect to the motor unit. The third parameter indicating self-recovery power and the maximum value of the third parameter may be stored, and the learning execution unit may store the first parameter, the second parameter, the maximum value of the second parameter, and the second parameter. Learning may be performed using the three parameters and the maximum value of the third parameter. The learning execution unit subtracts a predetermined value from the second parameter each time the motor unit contracts, and while the third parameter is not 0, the second parameter is restored with the passage of time. It's okay. The information storage unit stores a fourth parameter indicating the amount of energy consumed each time the motor unit contracts when the motor unit is a fast muscle, and the learning execution unit stores the first parameter, Learning using the second parameter, the maximum value of the second parameter, the third parameter, the maximum value of the third parameter, and the fourth parameter may be executed. When the motor unit is a fast muscle, the learning execution unit subtracts the value of the fourth parameter from the second parameter every time the motor unit contracts, and when the motor unit is a slow muscle. May be subtracted from the second parameter a value other than the value of the fourth parameter each time the motor unit contracts. When the learning execution unit determines that the muscle fiber has been damaged and then determines that the muscle fiber has recovered, and the motor unit is a fast muscle, the maximum value of the second parameter and the above The value of the fourth parameter may be increased, and if the motor unit is a slow muscle, the maximum value of the third parameter may be increased. When the learning execution unit determines that the muscle fiber is damaged and then the muscle fiber is recovered, and the motor unit is a fast muscle, the maximum value of the third parameter is not increased. It's okay. When the learning execution unit determines that the muscle fiber has been damaged and then the muscle fiber has recovered, and the motor unit is a slow muscle, the maximum value of the second parameter is not increased. It's okay. The learning execution unit may determine that the muscle fiber is damaged when the second parameter becomes 0. The information storage unit may store a fifth parameter related to the use of the motor unit for the motor unit, and the learning execution unit may store the level of the motor unit as the fifth parameter increases. The higher the level of the motor unit, the more difficult it is to increase the maximum value of the second parameter and the value of the fourth parameter when the motor unit is a fast muscle, and the motor unit is a slow muscle. It may be difficult to increase the maximum value of the third parameter in a certain case. After contracting one motor unit, the learning execution unit may learn the firing pattern by preventing the one motor unit from contracting until a predetermined refractory period elapses. The learning execution unit may learn the ignition pattern by shortening the refractory period as the temperature of the motor unit increases. When the movement of the muscle model operated according to the plurality of firing patterns in the time series achieves the target movement, the learning execution unit traces back a predetermined time from the firing pattern in the state where the target movement is achieved. The above learning may be performed by updating the firing pattern in the state of being in the state.

According to the second aspect of the present invention, a learning execution device is provided. The learning execution device has, for each of the plurality of muscle fibers contained in the muscle, a first parameter indicating whether the muscle fiber is a fast muscle or a slow muscle, and a second parameter indicating contractile energy. An information storage unit may be provided for storing the maximum value of the second parameter, the third parameter indicating the self-healing power, and the maximum value of the third parameter. The learning execution device obtains a muscle model by executing learning using the first parameter, the second parameter, the maximum value of the second parameter, the third parameter, and the maximum value of the third parameter. It may be provided with a learning execution unit for learning.

The information storage unit may store a fourth parameter indicating the amount of energy consumed each time the muscle fiber contracts when the muscle fiber is a fast muscle, and the learning execution unit may store the first parameter. Learning may be performed using the parameters, the second parameter, the maximum value of the second parameter, the third parameter, the maximum value of the third parameter, and the fourth parameter. The learning execution unit subtracts a predetermined value from the second parameter each time the muscle fiber contracts, and while the third parameter is not 0, the second parameter is restored with the passage of time. If it is determined that the muscle fiber is damaged and then the muscle fiber is recovered, and the muscle fiber is a fast muscle, the maximum value of the second parameter and the value of the fourth parameter are obtained. If the muscle fiber is a slow muscle, the muscle model may be learned by increasing the maximum value of the third parameter. When the muscle fiber is a fast muscle, the learning execution unit subtracts the value of the fourth parameter from the second parameter every time the muscle fiber contracts, and when the muscle fiber is a slow muscle. May be obtained by subtracting a value other than the value of the fourth parameter from the second parameter each time the muscle fiber contracts. When the learning execution unit determines that the muscle fiber is damaged and then determines that the muscle fiber has recovered, if the muscle fiber is a fast muscle, the maximum value of the third parameter is not increased. It's okay. When the learning execution unit determines that the muscle fiber is damaged and then determines that the muscle fiber has recovered, if the muscle fiber is a slow muscle, the maximum value of the second parameter is not increased. It's okay.

According to the third aspect of the present invention, a program for making a computer function as the learning execution device is provided.

According to the fourth aspect of the present invention, a learning execution method executed by a computer is provided. The learning execution method follows the firing pattern of multiple interneurons to which each motor unit is connected, and the target movement of the muscle model that moves the muscle by contracting the muscle fibers connected to the motor neurons included in the motor unit. It may have an operation setting step to be set. The learning execution method includes a learning execution step of learning the firing pattern that realizes the target motion by executing learning that rewards the firing pattern in which the motion of the muscle model is closer to the target motion among the plurality of firing patterns. good.

According to a fifth aspect of the present invention, there is provided a learning execution method executed by a computer. The learning execution method includes, for each of the plurality of muscle fibers contained in the muscle, a first parameter indicating whether the muscle fiber is a fast muscle or a slow muscle, and a second parameter indicating contractile energy. A storage step may be provided for storing the maximum value of the second parameter, the third parameter indicating the self-healing power, and the maximum value of the third parameter. The learning execution method is a learning execution step of learning a muscle model by executing learning using the first parameter, the second parameter, the maximum value of the second parameter, the third parameter, and the maximum value of the third parameter. May be equipped.

The outline of the above invention does not list all the necessary features of the present invention. A subcombination of these feature groups can also be an invention.

An example of the learning execution device 100 is shown schematically. An example of the muscle model 300 is shown schematically. An example of the ignition pattern 400 is shown schematically. An example of the functional configuration of the learning execution device 100 is schematically shown. A specific example of the muscle model 300 is shown schematically. An example of the hardware configuration of the computer 1200 that functions as the learning execution device 100 is schematically shown.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention to which the claims are made. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention.

FIG. 1 schematically shows an example of a learning execution device 100. The learning execution device 100 executes learning for causing a muscle model that models muscle movement to execute a target movement.

The muscle model corresponds to, for example, some muscles of a person. The muscle model may correspond to all muscles of a person. The muscle model is not limited to humans and may correspond to any organism having muscle. Further, the muscle model may correspond to a CG character or the like.

The learning execution device 100 according to the present embodiment, for example, causes muscles by contracting muscle fibers connected to motor neurons included in the motor unit according to an firing pattern of a plurality of interneurons to which the motor unit is connected. Stores the muscle model to operate. Interneurons are sometimes called interneurons. Motor neurons are sometimes called motor neurons. Motor units are sometimes referred to as motor units.

The learning execution device 100 learns the firing pattern in which the muscle model realizes the target motion. The learning execution device 100 learns the firing pattern by, for example, performing learning in which the motion of the muscle model rewards the firing pattern close to the target motion among a plurality of randomly generated firing patterns.

As a conventional simulation method based on muscle contraction, a hill type model, CPG, and the like are known. In the conventional method, the parameters are set manually to simulate the movement. In the conventional method, when trying to make the muscle model perform different movements, it was necessary to set the parameters manually. On the other hand, according to the learning execution device 100 according to the present embodiment, since the firing pattern that can realize the target motion can be automatically learned, it is possible to eliminate the need to set parameters individually for each type of motion. ..

The learning execution device 100 may grow the muscles of the muscle model when the muscle model is operated based on the firing pattern while the learning is progressing. The learning execution device 100 grows a motor unit in which muscle fibers are contracted, for example, when a muscle model is operated based on an ignition pattern. In the conventional method, depending on the setting of parameters, a movement different from the actual movement of the muscle may be realized. On the other hand, the learning execution device 100 according to the present embodiment can realize more realistic movements by also considering the growth of muscles.

The learning execution device 100 may be applied to various fields. The learning execution device 100 learns, for example, an ignition pattern that causes a CG character to realize an arbitrary operation, and generates a CG animation of the character that executes an arbitrary operation.

Conventionally, it was necessary to create an animation in order to make a character perform an arbitrary movement, but according to the learning execution device 100 according to the present embodiment, for example, while growing the muscles of a muscle model, a target movement is performed. By learning the firing pattern of interneurons to perform, it is possible to automatically generate a CG animation of a character that performs an arbitrary action. For example, if a dance motion is set as the target motion, a CG animation in which the character executes the dance can be automatically generated. According to the learning execution device 100 according to the present embodiment, it is possible to realize a realistic movement by learning the firing pattern from the interneuron and realizing the movement of the same control system as an actual organism.

Further, in the conventional technique, for example, when a three-headed character is made to perform a dance movement of an eight-headed human being, the movement may not be compatible and the movement may become unnatural. rice field. On the other hand, according to the learning execution device 100 according to the present embodiment, a natural movement is realized for the three-headed character by executing learning considering the structure and growth of the muscles of the three-headed character. Can be made to.

The learning execution device 100 displays, for example, the generated CG animation on the display included in the learning execution device 100. Further, the learning execution device 100 may display the generated CG animation on the communication terminal 200 by transmitting it to the communication terminal 200 via the network 20, for example.

The communication terminal 200 may be a PC (Personal Computer), a tablet terminal, a smartphone, or the like. The learning execution device 100 and the communication terminal 200 may communicate with each other via the network 20. The network 20 may include the Internet. The network 20 may include a LAN (Local Area Network). The network 20 may include a mobile communication network. The mobile communication network complies with any of 3G (3rd Generation) communication method, LTE (Long Term Evolution) communication method, 5G (5th Generation) communication method, and 6G (6th Generation) communication method or later. May be good.

Further, the learning execution device 100 may be applied to, for example, the field of rehabilitation. The learning execution device 100 registers, for example, a muscle model of a performer who performs walking rehabilitation, and also registers walking as a target motion. Then, the learning of the firing pattern of the interneuron is advanced, and the movement and the growth of the muscle until the walking becomes possible are recorded. As a result, it is possible to search for an appropriate movement until the person can walk.

Further, the learning execution device 100 may be applied to, for example, the field of sports science. The learning execution device 100 registers, for example, a muscle model of an athlete and also registers an ideal form or the like as a target motion. Then, the learning of the firing pattern of the interneuron is advanced, and the movement and muscle growth until the ideal form is acquired are recorded. This makes it possible to find a training method.

The learning execution device 100 may apply muscle growth to a muscle model other than the muscle model that operates the muscle according to the firing pattern of the interneuron. For example, the learning execution device 100 applies muscle growth to a muscle model based on a hill type model. Further, for example, the learning execution device 100 applies muscle growth to a muscle model using CPG. Further, for example, the learning execution device 100 applies muscle growth to a muscle model using DQN. The learning execution device 100 may apply muscle growth to any other existing model.

FIG. 2 schematically shows an example of the muscle model 300. The muscle model 300 includes a plurality of interneurons 320 within the spinal cord 310 and a plurality of motor units 330 connected to each of the plurality of interneurons 320. One motor unit 330 includes a motor neuron 340 and a muscle fiber 350 connected to the motor neuron 340. A plurality of muscle fibers 350 are connected to one motor neuron 340.

FIG. 3 schematically shows an example of the ignition pattern 400. The firing pattern 400 shows time series on 402 and off 404 of multiple interneurons 320. By applying the firing pattern 400 to the muscle model 300, signals are input from the interneuron 320 to each motor unit 330 in chronological order, and the muscle fibers 350 of the motor unit 330 contract according to the on 402. As a result, various muscle movements are realized.

FIG. 4 schematically shows an example of the functional configuration of the learning execution device 100. The learning execution device 100 includes an information storage unit 102, an input reception unit 104, a data reception unit 106, an operation setting unit 108, a learning execution unit 110, and a display control unit 112.

The information storage unit 102 stores various types of information. The information storage unit 102 may store the muscle model. The information storage unit 102 operates the muscle by contracting the muscle fiber 350 connected to the motor neuron 340 included in the motor unit 330 according to the firing pattern of the plurality of interneurons 320 to which the motor unit 330 is connected. The muscle model may be stored.

The information storage unit 102 may store related parameters for each of the plurality of motor units 330 included in the muscle model 300. The information storage unit 102 may store a type parameter indicating whether the motor unit 330 is a fast muscle or a slow muscle. The type parameter may be an example of the first parameter.

The information storage unit 102 may store HP, which is a parameter indicating contractile energy. HP may be an example of the second parameter. The information storage unit 102 may store MAXHP indicating the maximum value of HP.

The information storage unit 102 may store MP, which is a parameter indicating self-healing power. MP may be an example of the third parameter. The information storage unit 102 may store MAXMP indicating the maximum value of MP.

The information storage unit 102 may store a fourth parameter indicating the amount of energy consumed each time the muscle fiber 350 contracts when the muscle fiber 350 is a fast muscle. In this example, the information storage unit 102 stores a DIAM indicating the diameter of the muscle fiber 350, which is an example of the fourth parameter. The information storage unit 102 may store EXP, which is a parameter related to the use of the motor unit 330. EXP may be, for example, a parameter that increases each time the motor unit 330 is used. EXP may be, for example, a parameter related to the number of times the motor unit 330 has been used. The EXP may be the number of times the motor unit 330 has been used. EXP may be an example of the fifth parameter.

The input receiving unit 104 receives various inputs. The input receiving unit 104 may receive an input via an input device included in the learning execution device 100.

The data receiving unit 106 receives various data via the network 20. The data receiving unit 106 receives, for example, the muscle model 300 from the communication terminal 200 and stores it in the information storage unit 102. Further, the data receiving unit 106 receives, for example, the parameter of the motor unit 330 from the communication terminal 200 and stores it in the information storage unit 102.

The motion setting unit 108 sets the target motion of the muscle model. The motion setting unit 108 may set the target motion of the muscle model 300 according to the input received by the input reception unit 104, for example. The motion setting unit 108 may set the target motion of the muscle model 300 according to the setting instruction received from the communication terminal 200 by the data reception unit 106.

The learning execution unit 110 executes learning. The learning execution unit 110 may learn the firing pattern. The learning execution unit 110 may learn the firing pattern that realizes the target motion by learning that the motion of the muscle model 300 rewards the firing pattern that is closer to the target motion among the plurality of firing patterns. The learning execution unit 110 uses, for example, reinforcement learning. The learning execution unit 110 may use DQN (Deep Q-Network). The learning execution unit 110 may use GA (Genetic Algorithm). The learning execution unit 110 may use any other learning method.

For example, when learning a firing pattern that realizes a certain target motion, the learning execution unit 110 randomly generates a plurality of firing patterns. The learning execution unit 110 operates the muscle model 300 according to each of the plurality of randomly generated firing patterns, and generates a plurality of firing patterns based on the firing patterns in which the movement of the muscle model 300 is closer to the target movement. The learning execution unit 110 operates the muscle model 300 according to each of the generated firing patterns, and generates a plurality of firing patterns based on the firing patterns in which the movement of the muscle model 300 is closer to the target movement. By repeating these, the learning execution unit 110 may learn the firing pattern that realizes the target motion.

The learning execution unit 110 may generate a plurality of ignition patterns based on the learned ignition patterns. For example, the learning execution unit 110 has a firing pattern learned for the target motion of bending and maintaining the knee at 20 degrees, and a firing pattern learned for the target motion of bending and maintaining the knee at 60 degrees. Multiple firing patterns are generated based on the firing patterns learned for the target motion of bending and maintaining the knee at 90 degrees. This makes it possible to easily prepare multiple firing patterns for the target motion of, for example, bending and maintaining the knee at an arbitrary angle, which is required as a whole compared to the case where the firing patterns are randomly generated. The time can be shortened.

The learning execution unit 110 may grow the motor unit 330 in which the muscle fiber 350 is contracted when the muscle model 300 is operated based on the firing pattern while the learning is progressing.

The muscle model 300 may include a fast muscle motor unit 330 and a slow muscle motor unit 330. When the muscle model 300 is operated based on the firing pattern, the learning execution unit 110 may grow the motor unit 330 of the fast muscle and the motor unit 330 of the slow muscle according to different criteria.

The learning execution unit 110 may subtract a predetermined value from the HP each time the motor unit 330 contracts, and may recover the HP with the passage of time while the MP is not 0. When the motor unit 330 is a fast muscle, the learning execution unit 110 may subtract DIAM from the HP each time the motor unit 330 contracts. When the motor unit 330 is a slow muscle, the learning execution unit 110 may subtract 1 from the HP each time the motor unit 330 contracts. Not limited to this, the learning execution unit 110 may subtract a value other than DIAM from the HP every time the motor unit 330 contracts when the motor unit 330 is a fast muscle. Further, the learning execution unit 110 may subtract a value other than 1 from the HP every time the motor unit 330 contracts, for example, the value of DIAM, when the motor unit 330 is a slow muscle. The learning execution unit 110 may recover HP with the passage of time while MP is not 0. The learning execution unit 110 does not have to recover the HP when the MP becomes 0. The learning execution unit 110 may recover MP with the passage of time.

When the learning execution unit 110 determines that the muscle fiber 350 has been damaged and then determines that the muscle fiber 350 has recovered, if the motor unit 330 is a fast muscle, MAXHP and DIAM are increased, and the motor unit is increased. If 330 is slow, MAXMP may be increased.

When the learning execution unit 110 determines that the muscle fiber 350 has been damaged and then determines that the muscle fiber 350 has recovered, the MAXMP does not have to be increased if the motor unit 330 is a fast muscle. When the learning execution unit 110 determines that the muscle fiber 350 has been damaged and then determines that the muscle fiber 350 has recovered, the MAXHP does not have to be increased if the motor unit 330 is a slow muscle. For example, when the HP becomes 0, the learning execution unit 110 may determine that the muscle fiber 350 is damaged, and when the HP becomes MAXHP or the HP becomes higher than a predetermined threshold value, the learning execution unit 110 may determine. It may be determined that the muscle fiber 350 has recovered.

The learning execution unit 110 may improve the level of the motor unit 330 as the EXP increases. For example, the learning execution unit 110 registers an EXP value determined for each level, and improves the level of the motor unit 330 when the EXP value exceeds the EXP value corresponding to the level. More EXP values may be registered for higher levels.

The learning execution unit 110 makes it difficult to increase MAXHP and DIAM when the motor unit 330 is a fast muscle, and makes it difficult to increase MAXMP when the motor unit 330 is a slow muscle, as the level of the motor unit 330 is higher. It's okay.

After contracting the muscle fiber 350 of the motor unit 330, the learning execution unit 110 may prevent the muscle fiber 350 from contracting until a predetermined refractory period elapses. The information storage unit 102 may store the temperature of each of the plurality of motor units 330. The learning execution unit 110 may raise the temperature of the motor unit 330 as the motor unit 330 is used, and if the motor unit 330 is not used, lower the temperature of the motor unit 330 over time. good. The learning execution unit 110 may shorten the refractory period as the temperature of the motor unit 330 increases.

When the movement of the muscle model operated according to a plurality of time-series firing patterns achieves the target movement, the learning execution unit 110 fires in a predetermined time retroactive state from the firing pattern in the state where the target movement is achieved. Learning may be performed by updating the pattern. Since there are various time delays from the generation of the firing pattern to the actual movement of the muscle, such as the refractory period and the law of inertia, it may not be desirable to update the firing pattern at the moment of reward. be. On the other hand, according to the learning execution unit 110, since the firing pattern in the state of going back in a predetermined time from the firing pattern in the state where the target motion is achieved is updated, the learning accuracy can be improved.

The predetermined time may be arbitrarily set or may be changeable. The learning execution unit 110 may use different times for the fast muscle and the slow muscle. For example, when the motor unit 330 is a fast muscle, the learning execution unit 110 updates the firing pattern in the state 20 ms before the firing pattern in the state where the target motion is achieved, and when the motor unit 330 is the slow muscle, the target. The ignition pattern in the state 40 ms before may be updated from the ignition pattern in the state where the operation is achieved.

The learning execution unit 110 may generate display data using the learned firing pattern. The learning execution unit 110 generates, for example, a CG animation in which an arbitrary character is operated according to a firing pattern. The learning execution unit 110 may generate display data for displaying data related to the movement of the muscle model 300 and the growth of the muscle from the start of learning of the muscle model 300 to the realization of the target movement. Even if the learning execution unit 110 generates display data that displays data on the movement and muscle growth of the muscle model 300 from the start of learning of the muscle model 300 until the ideal form can be realized. good.

The display control unit 112 controls various displays related to the learning result by the learning execution unit 110. The display control unit 112 displays, for example, the display data generated by the learning execution unit 110 on the display included in the learning execution device 100. The display control unit 112 may transmit the display data generated by the learning execution unit 110 to the communication terminal 200 via the network 20 and display it on the display provided in the communication terminal 200.

The information storage unit 102 may store a muscle model according to a known model. The information storage unit 102 stores, for example, a muscle model based on a hill type model. The information storage unit 102 may store a muscle model using CPG. The information storage unit 102 may store a muscle model using DQN.

The information storage unit 102 stores the type parameters, HP, MAXHP, MP, MAXMP, and DIAM for each of the plurality of muscle fibers contained in the muscle of the muscle model according to the known model. It's okay.

Each time the muscle fiber contracts, the learning execution unit 110 subtracts DIAM from HP when the muscle fiber is a fast muscle, subtracts 1 from HP when the muscle fiber is a slow muscle, and MP is not 0. In the meantime, HP is restored with the passage of time, and after it is determined that the muscle fiber is damaged, when it is determined that the muscle fiber has recovered, MAXHP and DIAM are increased if the muscle fiber is a fast muscle. If the muscle fibers are slow muscles, MAXMP may be increased.

The learning execution unit 110 does not have to recover the HP when the MP becomes 0. The learning execution unit 110 may recover MP with the passage of time. The learning execution unit 110 does not have to increase MAXMP when the muscle fiber is a fast muscle in the case where the HP is recovered after determining that the muscle fiber is damaged. The learning execution unit 110 does not have to increase MAXHP when the muscle fiber is a slow muscle in the case where the HP is recovered after determining that the muscle fiber is damaged.

The learning execution unit 110 may improve the level of muscle fibers as the EXP increases. The learning execution unit 110 registers, for example, an EXP value determined for each level, and improves the muscle fiber level when the EXP value exceeds the EXP value corresponding to the level. More EXP values may be registered for higher levels. The learning execution unit 110 makes it difficult to increase the values of MAXHP and DIAM when the muscle fiber is a fast muscle, and makes it difficult to increase the MAXMP when the muscle fiber is a slow muscle, as the level of the muscle fiber is high. good.

After contracting the muscle fiber, the learning execution unit 110 may prevent the muscle fiber from contracting until a predetermined refractory period elapses. The information storage unit 102 may store the temperature of each of the plurality of muscle fibers. The learning execution unit 110 may raise the temperature of the muscle fiber as the muscle fiber is used, and may lower the temperature of the muscle fiber with the passage of time if the muscle fiber is not used. The learning execution unit 110 may shorten the refractory period as the temperature of the muscle fiber is higher.

FIG. 5 schematically shows a specific example of the muscle model 300. FIG. 5 illustrates a muscle model 300 of muscles 360 corresponding to human tendons 372, knees 374, and bones 376. As described above, there are a plurality of interneurons 320 in the spinal cord 310. The spinal cord 310 can also be regarded as a learning device. Each of the plurality of interneurons 320 can be in two states, firing and non-firing. Motor unit 330 includes motor neurons 340 and muscle fibers 350 connected to motor neurons 340. A plurality of muscle fibers 350 are connected to one motor neuron 340. There may be two types of motor neurons 340, fast muscles and slow muscles. The motor neuron 340 may be a fast muscle if it is large in size and a slow muscle if it is small in size. The motor neuron 340 is, for example, a fast muscle when the size is larger than the threshold, and a slow muscle when the size is smaller than the threshold. There may be two types of muscle fibers 350, fast muscle fibers and slow muscle fibers. Muscle 360 is an aggregate of muscle fibers 350. In this example, the learning execution unit 110 controls a simple movement with a firing pattern for a knee joint in which two muscles (extensor muscle and flexor muscle) are connected as one model. There may be two types of motor units 330, fast muscles and slow muscles, and the learning execution unit 110 may cause the fast muscles and the slow muscles to grow differently.

In order to control muscles using firing patterns, it is necessary to calculate the action potentials of neurons. When the learning execution unit 110 fires the interneuron 320, for example, the Hodgkin-Huxley model (AL Hodgkin, A. A quantitative description of membrane current and its application to conduction and excitation in nerve, from the physiological laboratory. University of The activity potential may be calculated according to Cambridge, pp. 500-544, 1952.). The calculated action potentials are distributed to the connected motor neurons 340 according to Kirchhoff's law.

The learning execution unit 110 may use the expanded hill type model, and the sum of the action potentials of the firing motor neurons 340 may be used as the input signal of the muscle model. In the muscle model, the contractile force of the muscle is calculated, and according to the laws of physics, the contractile force of the muscle is converted into the torque of the knee joint, and the knee is moved to change the joint angle. The learning execution unit 110 may output the exercise result as a joint angle. When the joint angle reaches the target angle, the learning execution unit 110 may reward the firing pattern.

As described above, the learning execution unit 110 may use the extended hill type model. The conventional leech type model is composed of a muscle contraction element (CE), a parallel elastic element (PEE) arranged in parallel with the CE, and a series elastic element (SEE) arranged in series. In the extended model, a damping coefficient is added to the tendon force calculation in the conventional hill type model in order to reduce the muscle spasm caused by the spring constant. In the hill-type model, muscle fibers take current from motor neurons and use PEE, SEE, and CE to convert it into force.

lopt is the length optimized to obtain the maximum force of CE, A is the muscle activity ratio, and lc is the length of CE. Several equations have been proposed to approximate this function. For example, the Rosen and Kuo model (Deshpande, P.-HK. AD Contribution of passive properties of muscle-tendon units to the metacarpophalangeal joint torque of the index finger. IEEE, pp. 288-294, 2010.) may be applied. ..

Vce is the contraction rate of CE, and Vmax is the maximum contraction rate of CE. The formula of Fpe, which is the force generated by PE, is as follows.

Kpe is the spring constant of PE, lpe is the length of PE, lpe_rest is the equilibrium length of PE, dpe is the damping coefficient of PE, and Vpe is the terminal velocity of PE. The formula of Fse, which is the power of SEE, is as follows.

kse is the spring constant of SEE, lse is the length of SEE, lse_rest is the equilibrium length of SEE, dse is the damping coefficient of SEE, and Vse is the terminal velocity of SEE.

When the motor unit 330 receives an action potential, muscle contraction is triggered. The time until the contraction reaches the peak of force is called the contraction time. The motor unit 330 of the slow muscle has a long contraction time and a small maximum contractile force. The motor unit 330 of the fast muscle has a short contraction time and a high maximum contractile force. One muscle is composed of a plurality of fast muscles and a plurality of slow muscles and a plurality of motor units 330. Therefore, a hill type model consisting of these motor units 330 is adopted.

N is the number of motor units 330 of the fast muscle, M is the number of motor units 330 of the slow muscle, Fce_f_i is the contraction force of the i-th fast muscle, and Fce_s_j and the contraction force of the j-th slow muscle. Is.

The biological properties of the slow muscle motor unit 330 and the fast muscle motor unit 330 are modeled by the muscle model according to the present embodiment. In a growth model of motor unit 330 composed of motor neurons 340 and muscle fibers 350, all muscle fibers 350 have an energy value (HP) that can be used for muscle contraction. With each contraction, the HP value of the fast muscle may be reduced by a value equal to the diameter of the muscle fiber 350. Further, the HP value of the slow muscle may be decreased by 1 each time the contraction occurs. HP decreases due to continuous muscle contraction, and when HP becomes 0, muscle rupture occurs. When a muscle rupture occurs, the muscle fiber 350 cannot be contracted again even if an electric signal is received from the interneuron 320 unless it recovers. On the other hand, if the signal spacing from the interneuron 320 is large enough, the muscle fiber 350 can recover spontaneously. In this model, the muscle fiber 350 recovers over time unless the MP showing self-healing power is 0. As a result, the learning execution unit 110 can learn various firing patterns.

Motor unit 330 acquires EXP each time it is used and promotes growth. In this model, LV is defined to represent the level of growth. Slow muscle motor units and fast muscle motor units have different growth rules. For the fast muscle motor unit 330, the MAXHP and muscle fiber 350 diameter parameters increase. The rules are based on biological growth rules.

In the motor unit 330 of the fast muscle, satellite cells are present around the muscle fiber. When muscle rupture occurs, satellite cells divide and the size of fast muscle fibers 350 increases. The thicker the muscle fiber 350, the higher the strength, but it requires more HP.

Slow muscle fibers 350 do not increase in size but increase self-healing power. According to biological growth rules, the number of capillaries surrounding the slow muscle fiber 350 increases, thus increasing the amount of oxygen transported to the slow muscle fiber 350.

In this model, SP, which is a parameter indicating the fatigue of the muscle fiber 350, may be further included. Only in the slow muscle fiber 350, the SP value decreases with growth. That is, slow muscle fibers can be used longer.

Table 1 shows a description of each parameter, and Table 2 shows an example of the algorithm.

The learning execution unit 110 may use Q-learning to learn the firing pattern of the interneuron 320. The learning process may be based on multi-agent system learning with each interneuron 320 as an agent. Each agent monitors its environment. The environment may be defined as the connectivity between the interneuron 320 and the motor neuron 340, and the parameters of the motor unit 330. During learning, the initial connection settings are not changed, but the motor unit parameters may be changeable.

Each interneuron 320 is connected to a plurality of motor neurons 340 and is connected to either fast or slow muscles. Whether the muscle fiber 350 of the motor unit 330 is a fast muscle or a slow muscle is determined by the size of the connected motor neuron. This principle comes from biology. Agents can share state information between agents. This is equivalent to information sharing via a myelin connection.

In the combination of state and action in Q-learning, Qi = (si: ai), and Si indicates the state of each agent.

M is the sum of the motor neurons 340 connected to the interneuron 320, and O is the sum of the motor neurons 340 connected to the other interneurons 320. Each agent performs an action ai such as firing (1) or not firing (0).

Each interneuron 320 monitors all parameters of each connected motor unit and the total energy of the motor unit held by other interneurons 320 sharing information and determines whether it fires. decide. Based on the firing of the interneuron 320, the Hodgkin Huxley model is used to calculate the electrical signal of the connected motor unit and use it to calculate the input signal. Then, using the expanded hill type model, the angle calculated from the contraction of the muscle is fed back to the agent.

There may be two types of rewards, immediate and delayed. As an immediate reward, you will receive an rgoal each time the knee joint reaches the target angle. As long as the knee joint continues to reach the target angle, the agent will continue to receive rewards.

As a late reward, the sum of the remaining HPs of all agents will be evenly distributed to all agents as a reward at the end of the episode. This contributes to cooperative action that produces efficient movement.

FIG. 6 schematically shows an example of a hardware configuration of a computer 1200 that functions as a learning execution device 100. A program installed on the computer 1200 causes the computer 1200 to function as one or more "parts" of the apparatus according to the present embodiment, or causes the computer 1200 to perform an operation associated with the apparatus according to the present embodiment or the one or the like. A plurality of "parts" can be executed and / or a computer 1200 can be made to execute a process according to the present embodiment or a stage of the process. Such a program may be run by the CPU 1212 to cause the computer 1200 to perform certain operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphic controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via the input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The computer 1200 also includes a legacy input / output unit such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires the image data generated by the CPU 1212 in a frame buffer or the like provided in the RAM 1214 or itself so that the image data is displayed on the display device 1218.

The communication interface 1222 communicates with other electronic devices via the network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads a program or data from a DVD-ROM or the like and provides it to the storage device 1224. The IC card drive reads the program and data from the IC card and / or writes the program and data to the IC card.

The ROM 1230 stores in it a boot program or the like executed by the computer 1200 at the time of activation, and / or a program depending on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

The program is provided by a computer-readable storage medium such as a DVD-ROM or IC card. The program is read from a computer-readable storage medium, installed in a storage device 1224, RAM 1214, or ROM 1230, which is also an example of a computer-readable storage medium, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200 and provides a link between the program and the various types of hardware resources described above. The device or method may be configured to implement the operation or processing of information in accordance with the use of the computer 1200.

For example, when communication is executed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded in the RAM 1214, and performs communication processing with respect to the communication interface 1222 based on the processing described in the communication program. You may order. Under the control of the CPU 1212, the communication interface 1222 reads and reads the transmission data stored in the transmission buffer area provided in the recording medium such as the RAM 1214, the storage device 1224, the DVD-ROM, or the IC card. The data is transmitted to the network, or the received data received from the network is written to the reception buffer area or the like provided on the recording medium.

Further, the CPU 1212 makes the RAM 1214 read all or necessary parts of the file or the database stored in the external recording medium such as the storage device 1224, the DVD drive (DVD-ROM), the IC card, etc., on the RAM 1214. Various types of processing may be performed on the data. The CPU 1212 may then write back the processed data to an external recording medium.

Various types of information such as various types of programs, data, tables, and databases may be stored in recording media and processed. The CPU 1212 describes various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, and information retrieval described in various parts of the present disclosure with respect to the data read from the RAM 1214. Various types of processing may be performed, including / replacement, etc., and the results are written back to the RAM 1214. Further, the CPU 1212 may search for information in a file, database, or the like in the recording medium. For example, when a plurality of entries each having an attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 is the first of the plurality of entries. The attribute value of the attribute of is searched for the entry that matches the specified condition, the attribute value of the second attribute stored in the entry is read, and the attribute value of the second attribute is changed to the first attribute that satisfies the predetermined condition. You may get the attribute value of the associated second attribute.

The program or software module described above may be stored on a computer 1200 or in a computer-readable storage medium near the computer 1200. Further, a recording medium such as a hard disk or RAM provided in a dedicated communication network or a server system connected to the Internet can be used as a computer-readable storage medium, whereby the program can be transferred to the computer 1200 via the network. offer.

The blocks in the flowcharts and block diagrams of this embodiment may represent the stage of the process in which the operation is performed or the "part" of the device responsible for performing the operation. Specific steps and "parts" are supplied with a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or with computer-readable instructions stored on a computer-readable storage medium. It may be implemented by the processor. Dedicated circuits may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuits include logical products, logical sums, exclusive logical sums, negative logical products, negative logical sums, and other logical operations, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like. , Flip-flops, registers, and reconfigurable hardware circuits, including memory elements.

The computer readable storage medium may include any tangible device capable of storing instructions executed by the appropriate device, so that the computer readable storage medium having the instructions stored therein may be in a flow chart or block diagram. It will be equipped with a product that contains instructions that can be executed to create means for performing the specified operation. Examples of the computer-readable storage medium may include an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, and the like. More specific examples of computer-readable storage media include floppy (registered trademark) disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), and erasable programmable read-only memory (EPROM or flash memory). , Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD), Blu-ray® Disc, Memory Stick , Integrated circuit cards and the like may be included.

Computer-readable instructions are assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object-oriented programming such as Smalltalk, JAVA®, C ++, etc. Includes either source code or object code written in any combination of one or more programming languages, including languages and traditional procedural programming languages such as the "C" programming language or similar programming languages. good.

Computer-readable instructions are used to generate means for a general purpose computer, a special purpose computer, or the processor of another programmable data processing device, or a programmable circuit, to perform an operation specified in a flowchart or block diagram. General purpose computers, special purpose computers, or other programmable data processing locally or via a local area network (LAN), a wide area network (WAN) such as the Internet, etc. to execute such computer-readable instructions. It may be provided to the processor of the device or a programmable circuit. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that the form with such changes or improvements may be included in the technical scope of the present invention.

The order of execution of each process such as operation, procedure, step, and step in the apparatus, system, program, and method shown in the claims, specification, and drawings is particularly "before" and "prior to". It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the scope of claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. It's not a thing.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that the form with such changes or improvements may be included in the technical scope of the present invention.

The order of execution of each process such as operation, procedure, step, and step in the apparatus, system, program, and method shown in the claims, specification, and drawings is particularly "before" and "prior to". It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the scope of claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. It's not a thing.

20 network, 100 learning execution device, 102 information storage unit, 104 input reception unit, 106 data reception unit, 108 operation setting unit, 110 learning execution unit, 112 display control unit, 200 communication terminal, 300 muscle model, 310 spinal cord, 320 Intervening neurons, 330 motor units, 340 motor neurons, 350 muscle fibers, 360 muscles, 372 tendons, 374 knees, 376 bones, 400 firing patterns, 402 on, 404 off, 1200 computers, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims (27)

An information storage unit that stores a muscle model that operates a muscle by contracting muscle fibers connected to the motor neurons included in the motor unit according to the firing pattern of a plurality of interneurons connected to each motor unit.
The motion setting unit that sets the target motion of the muscle model,
The learning execution unit that learns the firing pattern realizes the target motion by executing learning that rewards the firing pattern in which the motion of the muscle model is closer to the target motion among the plurality of firing patterns. A learning execution device including a learning execution unit that learns a firing pattern. The learning execution unit operates the muscle model according to each of the plurality of firing patterns, generates a plurality of firing patterns based on a firing pattern in which the motion of the muscle model is closer to the target motion, and the plurality of firing patterns. The muscle model is operated according to each of the above, and the firing pattern that realizes the target motion is learned by repeating that the motion of the muscle model generates a plurality of firing patterns based on the firing pattern closer to the target motion. , The learning execution device according to claim 1. The learning execution unit operates the muscle model according to each of the plurality of randomly generated firing patterns, and generates a plurality of firing patterns based on a firing pattern in which the movement of the muscle model is closer to the target movement. The target motion is realized by operating the muscle model according to each of the plurality of firing patterns and repeatedly generating a plurality of firing patterns based on the firing patterns in which the motion of the muscle model is closer to the target motion. The learning execution device according to claim 2, which learns an ignition pattern. The learning execution unit operates the muscle model according to each of the plurality of firing patterns generated based on the learned firing pattern, and the motion of the muscle model is a plurality of firing patterns based on the firing pattern closer to the target motion. By generating an ignition pattern, operating the muscle model according to each of the plurality of ignition patterns, and repeating the operation of the muscle model to generate a plurality of ignition patterns based on the ignition pattern closer to the target motion. The learning execution device according to claim 2, which learns an ignition pattern that realizes the target operation. The learning according to any one of claims 1 to 4, wherein the learning execution unit grows the motor unit in which the muscle fiber is contracted when the muscle model is operated based on the firing pattern. Execution device. The muscle model includes a fast muscle motor unit and a slow muscle motor unit.
The learning according to claim 5, wherein the learning execution unit grows the motor unit of the fast muscle and the motor unit of the slow muscle according to different criteria when the muscle model is operated based on the firing pattern. Execution device. The information storage unit includes a first parameter indicating whether the muscle is a fast muscle or a slow muscle, a second parameter indicating contractile energy, and a maximum value of the second parameter with respect to the motor unit. The third parameter indicating self-healing power and the maximum value of the third parameter are stored.
The sixth aspect of the present invention, wherein the learning execution unit executes learning using the first parameter, the second parameter, the maximum value of the second parameter, the third parameter, and the maximum value of the third parameter. Learning execution device. The learning execution unit subtracts a predetermined value from the second parameter each time the motor unit contracts, and restores the second parameter with the passage of time while the third parameter is not 0. , The learning execution device according to claim 7. The information storage unit stores a fourth parameter indicating the amount of energy consumed each time the motor unit contracts when the motor unit is a fast muscle.
The learning execution unit executes learning using the first parameter, the second parameter, the maximum value of the second parameter, the third parameter, the maximum value of the third parameter, and the fourth parameter. The learning execution device according to claim 8. When the motor unit is a fast muscle, the learning execution unit subtracts the value of the fourth parameter from the second parameter each time the motor unit contracts, and when the motor unit is a slow muscle. The learning execution device according to claim 9, wherein a value other than the fourth parameter is subtracted from the second parameter each time the motor unit contracts. When the learning execution unit determines that the muscle fiber has been damaged and then determines that the muscle fiber has recovered, and if the motor unit is a fast muscle, the maximum value of the second parameter and the said. The learning execution device according to claim 9 or 10, wherein the value of the fourth parameter is increased, and when the motor unit is a slow muscle, the maximum value of the third parameter is increased. When the learning execution unit determines that the muscle fiber has been damaged and then determines that the muscle fiber has recovered, and the motor unit is a fast muscle, the maximum value of the third parameter is not increased. The learning execution device according to claim 11. When the learning execution unit determines that the muscle fiber is damaged and then determines that the muscle fiber has recovered, and the motor unit is a slow muscle, the maximum value of the second parameter is not increased. The learning execution device according to claim 11 or 12. The learning execution device according to any one of claims 11 to 13, wherein the learning execution unit determines that the muscle fiber is damaged when the second parameter becomes 0. The information storage unit stores a fifth parameter related to the use of the motor unit for the motor unit.
The learning execution unit improves the level of the motor unit as the fifth parameter increases, and the higher the level of the motor unit, the maximum value of the second parameter when the motor unit is a fast muscle. The learning according to any one of claims 11 to 14, which makes it difficult to increase the value of the fourth parameter and makes it difficult to increase the maximum value of the third parameter when the motor unit is a slow muscle. Execution device. From claim 1, the learning execution unit learns the firing pattern by contracting one motor unit and then preventing the one motor unit from contracting until a predetermined refractory period elapses. The learning execution device according to any one of 15. The learning execution device according to claim 16, wherein the learning execution unit learns the ignition pattern by shortening the refractory period as the temperature of the motor unit is higher. When the movement of the muscle model operated according to the plurality of firing patterns in a time series achieves the target movement, the learning execution unit sets a predetermined time from the firing pattern in the state of achieving the target movement. The learning execution device according to any one of claims 1 to 17, wherein the learning is executed by updating the firing pattern in the retroactive state. For each of the plurality of muscle fibers contained in the muscle, a first parameter indicating whether the muscle fiber is a fast muscle or a slow muscle, a second parameter indicating contractile energy, and the second parameter. An information storage unit that stores a maximum value, a third parameter indicating self-healing power, and a maximum value of the third parameter.
Learning execution to learn the muscle model by executing learning using the first parameter, the second parameter, the maximum value of the second parameter, the third parameter, and the maximum value of the third parameter. A learning execution device equipped with a unit. The information storage unit stores a fourth parameter indicating the amount of energy consumed each time the muscle fiber contracts when the muscle fiber is a fast muscle.
The learning execution unit executes learning using the first parameter, the second parameter, the maximum value of the second parameter, the third parameter, the maximum value of the third parameter, and the fourth parameter. The learning execution device according to claim 19. The learning execution unit subtracts a predetermined value from the second parameter each time the muscle fiber contracts, and while the third parameter is not 0, the second parameter is restored with the passage of time. If it is determined that the muscle fiber has been damaged and then the muscle fiber has recovered, and the muscle fiber is a fast muscle, the maximum value of the second parameter and the value of the fourth parameter. 20 is the learning execution device according to claim 20, wherein the muscle fiber is a slow muscle, and the maximum value of the third parameter is increased to learn the model of the muscle. When the muscle fiber is a fast muscle, the learning execution unit subtracts the value of the fourth parameter from the second parameter each time the muscle fiber contracts, and when the muscle fiber is a slow muscle. 21. The learning execution device according to claim 21, wherein a value other than the value of the fourth parameter is subtracted from the second parameter each time the muscle fiber contracts. When the learning execution unit determines that the muscle fiber is damaged and then determines that the muscle fiber has recovered, if the muscle fiber is a fast muscle, the maximum value of the third parameter is not increased. The learning execution device according to claim 21 or 22. When the learning execution unit determines that the muscle fiber is damaged and then determines that the muscle fiber has recovered, if the muscle fiber is a slow muscle, the maximum value of the second parameter is not increased. The learning execution device according to any one of claims 21 to 23. A program for operating a computer as the learning execution device according to any one of claims 1 to 24. A learning method performed by a computer
Motion setting that sets the target motion of the muscle model that operates the muscle by contracting the muscle fibers connected to the motor neurons included in the motor unit according to the firing pattern of a plurality of interneurons to which the motor unit is connected to each. Steps and
Learning execution including a learning execution step for learning the firing pattern that realizes the target motion by executing learning that rewards the firing pattern in which the motion of the muscle model is closer to the target motion among the plurality of firing patterns. Method. A learning method performed by a computer
For each of the plurality of muscle fibers contained in the muscle, a first parameter indicating whether the muscle fiber is a fast muscle or a slow muscle, a second parameter indicating contractile energy, and the second parameter. A storage step for storing the maximum value, the third parameter indicating the self-healing power, and the maximum value of the third parameter.
Learning execution to learn the muscle model by executing learning using the first parameter, the second parameter, the maximum value of the second parameter, the third parameter, and the maximum value of the third parameter. A learning execution method with steps.

Images