KR20240008265A - Method and apparatus for generating motion of character based on musculoskeletal system model - Google Patents

Abstract

A method for generating motion of a character based on a musculoskeletal model includes the steps of receiving, by an image processing device, status information of the character; Inputting, by the image processing device, state information of the character into a reinforcement learning-based model; and controlling, by the image processing device, the state of the character based on the output value of the reinforcement learning-based model.

Description

Method and device for generating motion of a character based on a musculoskeletal model {METHOD AND APPARATUS FOR GENERATING MOTION OF CHARACTER BASED ON MUSCULOSKELETAL SYSTEM MODEL}

The technology described below relates to a method for generating motion of a character based on a musculoskeletal model.

Character animation is used in various fields such as games, movies, and computer graphics. Technologies such as motion capture methods were used to represent the character's movements. However, motion capture methods have the disadvantage of requiring expensive equipment to obtain motion. Accordingly, methods for creating animations using physical simulation-based characters were also developed. Using physics simulation-based characters, you can obtain motions that interact with various objects in unfamiliar environments.

Korean Patent Publication 10-0856824 B1

Among characters simulated based on physics, there are technologies that create videos based on characters based on musculoskeletal models. A musculoskeletal model is a model created by replicating the human musculoskeletal system. In order to generate the motion of a character based on a conventional musculoskeletal model, the motion was generated by calculating the turning force for each joint. In the prior art, turning force was applied directly to each joint so that the character followed the reference motion. However, in reality, each joint of a person cannot generate turning force. Instead, joints move through muscle contraction and relaxation.

The technology described below provides a method for generating simulated character movements based on the principles of how humans contract and relax their muscles. Furthermore, we provide a method to generate character motion using a reinforcement learning-based model.

A method for generating motion of a character based on a musculoskeletal model includes the steps of receiving, by an image processing device, status information of the character; Inputting, by the image processing device, state information of the character into a reinforcement learning-based model; and controlling, by the image processing device, the state of the character based on the output value of the reinforcement learning-based model.

The character consists of joints, path points formed on the joints, and muscle actuators connected between the path points. The muscle actuators apply forces between path points to move the character. The reinforcement learning-based model is a model that outputs an action value that receives the maximum reward value based on the input state information. The state information of the character includes rigid body state information and muscle state information of the character. The compensation value increases as it reflects the person's walking characteristics. The action value includes a value for whether the muscle actuator will contract or relax.

Using the technology described below, it is possible to create motion of a character based on a musculoskeletal model. In particular, it provides a way to create more human-like motion for characters using a reinforcement learning-based model.

Figure 1 shows the overall process by which the image processing device 100 generates motion of a character based on a musculoskeletal model.
Figure 2 is a flow chart 200 of one embodiment of a musculoskeletal model-based character motion generation method.
Figure 3 is one of the embodiments in which the muscle actuator operates.
Figure 4 shows one example of a muscle actuator.
Figure 5 shows the magnitude of force (Fiber Force) according to the length of muscle fibers (Fiber Length).
Figure 6 shows the relationship between the time derivative of muscle fiber length (velocity) and force.
Figure 7 is one example of how a reinforcement learning model works.
Figure 8 shows the characters used in simulation.
Figure 9 shows the results of generating character motion using the learned reinforcement learning-based model.
Figure 10 shows the result of creating character motion when only metabolic equivalent value (MET) was used as energy compensation among the compensation values.
Figure 11 shows the result of creating a character motion when only the movement cost (Cot) is used as energy compensation among the compensation values.
Figure 12 is the result when no energy compensation is used among the compensation values.
Figure 13 is a graph comparing reward values of the learning results of Figures 10 to 12.
Figure 14 shows the results of an experiment when the initial posture of the character is not specified.
Figure 15 shows the configuration of one embodiment of an image processing device.

The technology described below may be subject to various changes and may have various embodiments. Specific embodiments of the technology described below may be depicted in the drawings of the specification. However, this is for explanation of the technology described below and is not intended to limit the technology described below to specific embodiments. Therefore, it should be understood that all changes, equivalents, and substitutes included in the spirit and scope of the technology described below are included in the technology described below.

In the terms used below, singular expressions should be understood to include plural expressions, unless clearly interpreted differently from the context, and terms such as “including” refer to the described features, numbers, steps, operations, components, parts, or It should be understood that it means the existence of a combination of these, but does not exclude the possibility of the presence or addition of one or more other features, numbers, step operation components, parts, or combinations thereof.

Before providing a detailed description of the drawings, it would be clarified that the division of components in this specification is merely a division according to the main function each component is responsible for. That is, two or more components, which will be described below, may be combined into one component, or one component may be divided into two or more components for more detailed functions. In addition to the main functions it is responsible for, each of the component parts described below may additionally perform some or all of the functions handled by other components, and some of the main functions handled by each component may be performed by other components. Of course, it can also be carried out exclusively by .

In addition, when performing a method or operation method, each process forming the method may occur in a different order from the specified order unless a specific order is clearly stated in the context. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the opposite order.

Hereinafter, the overall process by which an image processing device performs a method for generating motion of a character based on a musculoskeletal system model will be described.

Figure 1 shows the overall process by which the image processing device 100 generates motion of a character based on a musculoskeletal model.

The image processing device 100 may receive character status information. The image processing device 100 may input character status information into a reinforcement learning-based model. The image processing device 100 may control the state of the character based on the output value of the reinforcement learning-based model.

A character may be composed of joints, path points formed at the joints, and muscle actuators connected between the path points. Muscle actuators can move a character by applying forces between path points.

The character's state information may include the character's rigid body state information and muscle state information.

A reinforcement learning-based model is a model that outputs an action value that receives the maximum reward value based on the input state information.

The compensation value can increase as it better reflects the person's walking characteristics.

The action value may include a value for whether the muscle actuator will contract or relax.

Hereinafter, a method for generating character motion based on a musculoskeletal model will be described in detail.

Figure 2 is a flow chart 200 of one embodiment of a musculoskeletal model-based character motion generation method.

The image processing device can receive character status information (210).

A character may be composed of joints, path points formed at the joints, and muscle actuators connected between the path points. There may be multiple joints.

Muscle actuators can move a character by applying forces between path points. In other words, a muscle actuator may simulate the force that causes a muscle to relax or contract. Figure 3 is one of the embodiments in which the muscle actuator operates. The character consists of three joints. A path point is formed at each of the three joints. Muscle actuators are formed between path points. Muscle actuators can apply force between path points to move joints and ultimately move the character. A detailed description of the muscle actuator will be described later.

The character's state information may include the character's rigid body state information and muscle state information.

The character's rigid body state information may include information about the positions and velocities of the character's joints. The position and speed of the character's joints can be expressed by the frame of the character's root (pelvis). The speed of a joint may mean the angular velocity of the joint. Alternatively, the character's rigid body state information may include information about the position and speed of the character's body.

The character's muscle state information may include length information of the muscle actuator. Specifically, the character's muscle status information may refer to the length of muscle fibers in the muscle actuator. Specific details will be described later.

The image processing device may input the character's state information into a reinforcement learning-based model (220).

A reinforcement learning-based model may be a model that outputs an action value that can receive the maximum reward value based on the status information of the input character.

The compensation value can increase as it better reflects the person's walking characteristics. In other words, the compensation value can increase as the character walks forward without falling by using natural walking movements like a human.

In one embodiment, the compensation value is the activation value of the character's muscles when the character's pelvis does not rotate, when the character's center of gravity does not deviate from side to side, when the character is not tilted, and when the character moves at a desired speed. When this is minimized, the value may increase if at least one of the following is satisfied: the character's energy consumption is minimal and the character maintains balance.

The action value may include a value for whether the muscle actuator will contract or relax. Specifically, it may include activation values of muscle actuators.

A detailed description of the reinforcement learning-based model is provided later.

The image processing device can control the character based on the output value of the reinforcement learning-based model (230).

Specifically, the image processing device can control the character by applying the output value of the reinforcement learning-based model to the character's muscle actuator. The force exerted by the muscle actuator can be calculated based on the length information and activation value of the muscle fiber. A detailed explanation of the process of controlling the character will be described later.

Hereinafter, the muscle actuator will be described in detail.

Figure 4 shows one example of a muscle actuator.

The muscle actuator may be a replica of an actual human muscle. Muscle actuators are made up of tendons and muscle fibers.

Muscle fibers can include active and passive parts. The active part may be a part that models the force of muscle contraction. The passive part models the force with which the muscles try to restore their original state. Specifically, the passive part may be a part that models the elastic force that tries to restore the muscle to its original state when it is stretched beyond a certain length.

The length of the muscle actuator can be calculated based on the length of the tendon and the length of the muscle fiber.

Equation 1 is the equation used to calculate the length (l _mtu ) of the muscle actuator.

In Equation 1, l _mtu may mean the length of the muscle actuator. In Equation 1, l _m may mean the length of the muscle fiber. In Equation 1, l _t may mean the length of the tendon. In Equation 1, α may mean the angle between the tendon and the muscle fiber (pennation angle).

The length of the tendon may be a certain length. In other words, the simulation can be performed with the length of the tendon fixed at the resting length. This is to improve simulation speed by reducing the amount of calculation. Each muscle actuator within a character can have a different tendon resting length.

The magnitude of the force generated by the muscle actuator (f _mtu ) can be calculated under the assumption that the force generated by the muscle fiber (f _m ) and the force generated by the tendon (f _t ) are the same. In other words, the magnitude and direction of the force generated by the tendon and the direction of the force generated by the muscle fiber can be calculated under the assumption that they are in equilibrium with each other. Therefore, the relationship shown in Equation 2 can be established.

In Equation 2, f _mtu may mean the force generated by the muscle actuator. In Equation 2, f _t may mean the force generated by the tendon. In Equation 2, f _m may mean the force generated by muscle fibers. In Equation 2, f _ce may mean the force generated by the active part of the muscle fiber. In Equation 2, f _pe may mean the force generated by the passive part of the muscle fiber. In Equation 2, α may mean the angle between the tendon and the muscle fiber (pennation angle).

Equation 4 can be derived by applying Equation 3 to Equation 2. Equation 4 is an equation used to calculate the force produced by the muscle actuator depending on the length of the muscle fiber. For specific details on Equation 4, see the paper 'Locomotion Control for Many muscle Humanoids.' (2014 Yoonsang Lee).

_m은 근섬유의 길이의 시간 미분값을 의미할 수 있다.즉

_m은 근섬유의 길이 변화 속도를 의미할 수 있다.In Equation 3, f _ce may mean the force emitted by the active part. In Equation 3, f _pe may mean the power produced by the passive part. In Equation 3 and Equation 4, l _m may mean the length of the muscle fiber. In Equation 3 and Equation 4

_m may mean the time derivative of the length of the muscle fiber. That is,

_m may refer to the rate of change in length of muscle fibers.

In Equation 4, g _al refers to the relationship between the active part of the muscle and the muscle fiber length. In Equation 4, g _av refers to the relationship between the active part of the muscle and the time derivative of the muscle fiber length. In other words, it can mean the relationship between the active part of the muscle and the rate of change in muscle fiber length. In Equation 4, g _pl refers to the relationship between the passive part of the muscle and the muscle fiber length. In Equation 4, f _mtu refers to the force that the muscle actuator can produce. In Equation 4, cos(α) means the angle (muscle angle) between the tendon and the muscle fiber.

In Equation 4, the activation value (act) can have a value between 0 and 1. That is, when the muscle is relaxed, it can have a value of 0, and when the muscle is contracted, it can have a value of 1.

Figure 5 shows the magnitude of force (Fiber Force) according to the length of muscle fibers (Fiber Length). Specifically, Equation 4 shows the strength of muscle fibers according to changes in g _al (l _m ) and g _pl (l _m ).

G _al (l _m ), the force on the active part, initially increases as the length increases and then tends to decrease. g _pl (l _m ) for the passive part has no effect initially as the length increases, but shows a tendency to increase rapidly beyond a certain length.

'_m)의 변화에 따른 근섬유의 힘을 보여준다. Figure 6 shows the relationship between the time derivative of muscle fiber length (velocity) and force. Specifically, in Equation 4, g _v (

It shows the strength of muscle fibers according to changes in ' _m ).

It can be seen that the faster the length of the muscle fiber changes, the more the strength of the muscle fiber increases.

Below we will look at how to control a character using muscle actuators.

Muscle actuators are located between path points attached to the body. The force generated by the muscle actuator can be applied in a pulling (contraction) or pushing (relaxation) direction between each path point. When the path point changes, the corresponding joints also change. At this time, the forces between path points can balance each other and cancel out.

The force exerted by the muscle actuator can be calculated based on the length of the muscle fiber, the degree of change in muscle fiber length, and the activation value. For this purpose, Equation 4 can be used. At this time, the length of the muscle actuator and the degree of change in the length of the muscle actuator can be calculated based on the character's status information. Additionally, the activation value may be calculated based on the output value of the reinforcement learning model.

To calculate the force exerted by a muscle actuator, the length of the muscle fiber is required. As described above, since the length of the tendon among the muscle actuators is fixed, the length of the muscle fiber can be calculated by subtracting the length of the tendon from the length of the muscle actuator.

Below, the reinforcement learning model will be described in detail.

A reinforcement learning-based model may be a model that receives state values as input and outputs action values. More specifically, a reinforcement learning-based model may be a model that outputs action values to receive maximum reward.

Figure 7 is one example of how a reinforcement learning model works. The reinforcement learning model can receive the character's state information as a state value. The reinforcement learning model can output the activation value of the muscle actuator as an action value and receive a reward value accordingly. Compensation may be related to whether a person's walking characteristics are well reflected. Muscle simulation is performed based on the output value of the reinforcement learning model, and the character's status information can be updated accordingly.

The reward value may increase as the character better reflects the human walking characteristics.

Equation 5 is one of the equations used to determine the reward value.

In Equation 5, r _ori is a term that prevents the character's pelvis from rotating. In other words, r _ori - is a term that gives a penalty the more the pelvis is rotated. r _ori can be calculated through Equation 6.

In Equation 6, w _ori may mean a weight. In Equation 6, θ refers to the three-dimensional rotational degree of freedom (x, y, z) and can mean how much it rotates in each direction.

In Equation 5, r _dev is a term that prevents the character's center of gravity from deviating from side to side. That is, in Equation 5, r _dev is a term that gives a penalty as the center of gravity deviates from left to right. r _dev can be calculated through Equation 7.

In Equation 7, w _dev may mean a weight. In Equation 7, COMz refers to the z-axis position of the center of mass, and can specifically mean how much the character's center of gravity deviates from the z-axis.

In Equation 5, r _up is a term that prevents the character's upper body from tilting. This is the term that allows the upper body (torso) to maintain a straight line upward. r _up can be calculated through Equation 8.

In Equation 8, w _up refers to the weight. In Equation 8, ytorso means the vector in the y-axis direction of the upper body (torso) of the current character. In Equation 8, yglobal <0, 1, 0> means a unit vector that is straight in the y-axis direction.

In Equation 5, r _vel is a term that causes the character to move at the desired speed. In other words, r _vel is a term that gives a penalty the greater the difference between the character's current speed and the target speed. In other words, it is a term for the purpose of allowing the character to move forward at the target speed.

r _vel can be calculated through Equation 9.

In Equation 9, w _vel may mean a weight. In Equation 9, v _desired refers to the target speed, and can refer to the value that starts from 0 m/s at the beginning and accelerates to 1.5 m/s over 1 second. In Equation 9, v _current refers to the character's It means the current speed.

In Equation 6, r _eng is a term that minimizes energy consumption. r _eng can be calculated based on physical quantities such as the degree of muscle activation, muscle skin coverage, and muscle fiber speed.

r _eng can be calculated through two types of energy compensation.

The first is about metabolic equivalent of task (MET). MET may be the rate of change in metabolic energy consumption normalized to mass. MET may be a reward calculated for each episode step (dense reward)

The second concerns cost of transportation (CoT). CoT may be the amount of energy required to move a unit distance. CoT may be a reward that is calculated only once at the end of each episode (sparse reward)

Below, we will look at one example of training a reinforcement learning model.

To achieve static human-like walking movements, the reinforcement learning model was trained in two steps. First, among the compensation values, MET was used as energy compensation, and then CoT was used as energy compensation. Specifically, we learned until convergence with MET, which is a reward given at every episode step, and then switched to CoT reward, which is given only once at the end of the episode, to improve the learned policy and stabilize the operation.

In the initial learning stage, most of the muscle activation values output by the policy consume a lot of energy and produce unstable movements. MET compensation helps you quickly stabilize your movements while learning how to maintain balance across multiple episodic steps. When learning converges in MET and then switches to CoT compensation, it can help effectively increase the distance traveled because it provides a high reward for moving a longer distance even if the same energy is consumed.

The human character's posture at the beginning of the episode is not with both feet on the ground, but with either the left or right leg raised at a random angle. This is because the posture of the human character at the start of the episode can play an important role when learning walking movements similar to humans. In other words, starting with one leg lifted is more likely to explore forward motion while crossing both legs.

After building the aforementioned reinforcement learning-based model, the researcher conducted an experiment to generate character motion based on it. The researcher used PPO (Proximal Policy Optimization) as a reinforcement learning algorithm. 32 CPU cores were used in the experiment, and it took about 10 days to learn a stable walking policy. The size of the network hidden layer is 32 x 32 x 32, and relu was used as the activation function.

Below we look at the results of the experiment conducted by the researcher.

Figure 8 shows the characters used in simulation.

The character used in the simulation weighs 75 kg and consists of 16 bodies. Additionally, the character used in the simulation includes 31 degrees of freedom and 120 muscles. In Figure 8, the blue line indicates a muscle in a relaxed state.

Figure 9 shows the results of generating character motion using the learned reinforcement learning-based model.

Figure 9(a) shows one cycle section of walking. Figure 9(b) shows a total of four cycle sections of walking. The lines expressed in red indicate muscles in a contracted state, and the lines expressed in blue indicate muscles in a relaxed state. Through Figure 9, it can be seen that the character can walk with movements similar to humans.

Figure 10 shows the result of creating character motion when only metabolic equivalent value (MET) was used as energy compensation among the compensation values.

The walking motion itself is similar to Figure 9, but you can see that when landing on the third step, the person cannot maintain balance and falls.

Figure 11 shows the result of creating a character motion when only the movement cost (Cot) is used as energy compensation among the compensation values. Because movement costs are calculated only once, at the end of an episode, they do not provide sufficient information for learning. Therefore, when compared to the metabolic value (Met) given for each step of the episode, it can be seen that it does not behave stably and falls quickly.

Figure 12 is the result when no energy compensation is used among the compensation values. Overall, compared to the previous results, it can be seen that the muscles use a lot of force. Also, you can see that he tries to jump forward but quickly loses his balance and falls.

Figure 13 is a graph comparing reward values of the learning results of Figures 10 to 12.

If energy compensation is not used (no energy), you can see that energy consumption cannot be reduced. As a result, if energy compensation is not used, the amount of energy consumption cannot be reduced, and as a result, the operation is also unstable, so it can be confirmed that high compensation is not obtained. In particular, it can be seen that it is difficult to enter a stable space in the early stages of learning. This is because minimizing energy consumption does not simply limit the size of average force, but can help find states and action spaces that are close to human-like movements.

You can get better rewards by using energy rewards, and in particular, you can see that giving energy rewards at every step shows better results in terms of increased reward value and learning compared to giving energy rewards once at the end of an episode. You can.

Figure 14 shows the results of an experiment when the initial posture of the character is not specified. In other words, it is the result of learning with both feet on the ground.

If you do not specify the character's initial posture, you can see that the jumping motion with both feet has been learned. This appears to be because it is easier to explore the behavior of jumping on two feet rather than lifting one foot in the early stages.

The image processing device will be described below.

Figure 15 shows the configuration of one embodiment of the image processing device 300.

The image processing device 300 may correspond to the image processing device 100 described in FIG. 1 . That is, the image processing device 300 may be a device that performs a method for generating motion of a character based on a musculoskeletal model.

The image processing device 300 may be physically implemented in various forms. For example, the image processing device 300 may take the form of a PC, a laptop, a smart device, a server, or a chipset dedicated to data processing.

The image processing device 300 may include an input device 310, a storage device 320, an arithmetic device 330, an output device 340, an interface device 350, and a communication device 360.

The input device 310 may include an interface device (keyboard, mouse, touch screen, etc.) that receives certain commands or data. The input device 310 may include a component that receives information through a separate storage device (USB, CD, hard disk, etc.). The input device 310 may receive input data through a separate measuring device or through a separate DB. The input device 310 can receive data through wired or wireless communication.

The input device 310 can receive input information and models necessary to execute a method for generating motion of a character based on a musculoskeletal model. The input device 310 can receive character status information. The input device 310 can receive a reinforcement learning-based model.

The storage device 320 can store information input through the input device 310. The storage device 320 can store information generated during calculation by the computing device 330. That is, the storage device 320 may include memory. The storage device 320 can store the results calculated by the computing device 330.

The storage device 320 may store information and models necessary to execute a method for generating motion of a character based on a musculoskeletal model. The storage device 320 can store character status information and a reinforcement learning-based model.

The computing device 330 may be a device such as a processor that processes data and performs certain operations, an AP, or a chip with an embedded program. The computing device 330 may generate a control signal that controls the image processing device 300.

The calculation device 330 may perform calculations necessary to perform a method for generating motion of a character based on a musculoskeletal model.

The computing device 330 can input the character's state information into a reinforcement learning-based model. The computing device 330 can control the state of the character based on the output value of the reinforcement learning-based model.

The output device 340 may be a device that outputs certain information. The output device 340 may output interfaces, input data, analysis results, etc. required for data processing. The output device 340 may be physically implemented in various forms, such as a display, a document output device, etc.

The interface device 350 may be a device that receives certain commands and data from the outside. The interface device 350 may store information and models required to perform a method for generating motion of a character based on a musculoskeletal model from a physically connected input device or an external storage device. The interface device 350 may receive a control signal to control the image processing device 300. The interface device 350 may output the results analyzed by the image processing device 300.

The communication device 360 may refer to a configuration that receives and transmits certain information through a wired or wireless network. The communication device 360 may receive a control signal necessary to control the image processing device 300. The communication device 360 may transmit the results analyzed by the image processing device 300.

The method for generating motion of a character based on the above-described musculoskeletal model may be implemented as a program (or application) including an executable algorithm that can be executed on a computer.

The program may be stored and provided in a temporary or non-transitory computer readable medium.

A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specifically, the various applications or programs described above include CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), and EPROM (Erasable PROM, EPROM). Alternatively, it may be stored and provided in a non-transitory readable medium such as EEPROM (Electrically EPROM) or flash memory.

Temporarily readable media include Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), and Enhanced SDRAM (Enhanced RAM). It refers to various types of RAM such as SDRAM, ESDRAM), synchronous DRAM (Synclink DRAM, SLDRAM), and Direct Rambus RAM (DRRAM).

This embodiment and the drawings attached to this specification only clearly show some of the technical ideas included in the above-described technology, and those skilled in the art can easily understand them within the scope of the technical ideas included in the specification and drawings of the above-described technology. It will be self-evident that all inferable modifications and specific embodiments are included in the scope of rights of the above-described technology.

Claims (17)

A step in which an image processing device receives status information of a character;
Inputting, by the image processing device, state information of the character into a reinforcement learning-based model; and
Including, the image processing device controlling the state of the character based on the output value of the reinforcement learning-based model,
The character consists of a joint, a path point formed on the joint, and a muscle actuator connected between the path points,
The muscle actuator applies force between path points to move the character,
The reinforcement learning-based model is a model that outputs an action value that receives the maximum reward value based on the input state information,
The state information of the character includes rigid body state information and muscle state information of the character,
The compensation value increases as it reflects the person's walking characteristics,
The action value includes a value for whether the muscle actuator will contract or relax. According to paragraph 1,
A method for generating motion of a character based on a musculoskeletal model, wherein the rigid body state information of the character includes information about the positions and speeds of joints of the character. According to paragraph 1,
A method for generating motion of a character based on a musculoskeletal model, wherein the muscle state information of the character includes length information of the muscle actuator. According to paragraph 1,
The compensation value is calculated when the character's pelvis does not rotate, when the character's center of gravity does not deviate from side to side, when the character is not tilted, when the character moves at a desired speed, and when the character's muscles A method for generating motion of a character based on a musculoskeletal model, wherein the value increases when at least one of the following is satisfied: when the activation value is minimized, when the energy consumption of the character is minimized, and when the character maintains balance. According to paragraph 4,
The character's energy consumption includes metabolic equivalent of task (MET) or cost of transport (CoT).
The metabolic equivalent value includes the rate of change in energy consumption normalized by mass,
A method for generating motion of a character based on a musculoskeletal model, wherein the movement cost includes the amount of energy required to move a unit movement distance. According to clause 5,
The reinforcement learning-based model is first learned using the metabolic value as the energy consumption among the compensation values, and then learned again using the movement cost as the energy consumption among the compensation values. A method for generating motion of a character based on a musculoskeletal model. According to paragraph 1,
The muscle actuator consists of tendons and muscle fibers,
A method for generating motion of a character based on a musculoskeletal system model, wherein the muscle fibers include an active part that models the force by which the muscle contracts and a passive part that models the force that the muscle tries to restore to its original state. In clause 7,
A method for generating motion of a character based on a musculoskeletal model, wherein the force (f _mtu ) produced by the muscle actuator is calculated based on the following equation.
[Equation]

In the above equation
In the above equation, l _m may mean the length of the muscle fiber. In the above equation

_m may mean the time derivative of the length of the muscle fiber. in other words

_m may mean the rate of change in length of the muscle fiber. In the above equation, g _al refers to the relationship between the active part of the muscle fiber and the muscle fiber length. In the above equation, g _av refers to the relationship between the active part of the muscle fiber and the time derivative of the muscle fiber length. In other words, it can mean the relationship between the active part of the muscle and the rate of change in muscle fiber length. In the above equation, g _pl refers to the relationship between the passive part of the muscle and the muscle fiber length. In the above equation, f _mtu means the force that the muscle actuator can produce. In the above equation, cos(α) means the angle (muscle angle) between the tendon and the muscle fiber. In the above equation, the activation value (act) may have a value between 0 and 1. That is, when the muscle is relaxed, it can have a value of 0, and when the muscle is contracted, it can have a value of 1. An input device that receives character status information;
a computing device that inputs the character's state information into a reinforcement learning-based model, and allows the image processing device to control the character's state based on an output value of the reinforcement learning-based model; and
A storage device that stores the status information of the character and the reinforcement learning-based model,
The character consists of a joint, a path point formed on the joint, and a muscle actuator connected between the path points,
The muscle actuator applies force between path points to move the character,
The reinforcement learning-based model is a model that outputs an action value that receives the maximum reward value based on the input state information,
The state information of the character includes rigid body state information and muscle state information of the character,
The compensation value increases as it reflects the person's walking characteristics,
An apparatus for generating motion of a character based on a musculoskeletal model, wherein the action value includes a value for whether the muscle actuator will contract or relax. According to clause 9,
An apparatus for generating motion of a character based on a musculoskeletal model, wherein the rigid body state information of the character includes information about the position and speed of joints of the character. According to clause 9,
An apparatus for generating motion of a character based on a musculoskeletal model, wherein the muscle state information of the character includes length information of the muscle actuator. According to clause 9,
The compensation value is calculated when the character's pelvis does not rotate, when the character's center of gravity does not deviate from side to side, when the character is not tilted, when the character moves at a desired speed, and when the character's muscles A motion generating device for a character based on a musculoskeletal model, wherein the value increases when at least one of the following is satisfied: when the activation value is minimized, when energy consumption of the character is minimized, and when the character maintains balance. According to clause 12,
The character's energy consumption includes metabolic equivalent of task (MET) or cost of transport (CoT),
The metabolic equivalent value includes the rate of change in energy consumption normalized by mass,
A motion generating device for a character based on a musculoskeletal model, wherein the movement cost includes the amount of energy required to move a unit moving distance. According to clause 13,
The reinforcement learning-based model is first learned using the metabolic value as the energy consumption among the compensation values, and then learned again using the movement cost as the energy consumption among the compensation values. A device for generating motion of a character based on a musculoskeletal model. According to clause 9,
The muscle actuator consists of tendons and muscle fibers,
The muscle fibers include an active part that models the force by which the muscle contracts and a passive part that models the force that the muscle tries to restore to its original state. A motion generating device for a character based on a musculoskeletal model. According to clause 15,
A motion generating device for a character based on a musculoskeletal model, wherein the force (f _mtu ) produced by the muscle actuator is calculated based on the following equation.
[Equation]

In the above equation
In the above equation, l _m may mean the length of the muscle fiber. In the above equation

_m may mean the time derivative of the length of the muscle fiber. in other words

_m may mean the rate of change in length of the muscle fiber. In the above equation, g _al refers to the relationship between the active part of the muscle fiber and the muscle fiber length. In the above equation, g _av refers to the relationship between the active part of the muscle fiber and the time derivative of the muscle fiber length. In other words, it can mean the relationship between the active part of the muscle and the rate of change in muscle fiber length. In the above equation, g _pl refers to the relationship between the passive part of the muscle and the muscle fiber length. In the above equation, f _mtu means the force that the muscle actuator can produce. In the above equation, cos(α) means the angle (muscle angle) between the tendon and the muscle fiber. In the above equation, the activation value (act) may have a value between 0 and 1. That is, when the muscle is relaxed, it can have a value of 0, and when the muscle is contracted, it can have a value of 1. A computer-readable recording medium recording a program for executing the method for generating motion of a musculoskeletal model-based character according to claim 1.