JP2024055447A - Information processing device, information processing method, and program - Google Patents

Abstract

[Problem] To link a plurality of motion data more naturally. [Solution] An information processing device comprising an acquisition unit that acquires interpolated motion data that interpolates between first motion data and second motion data that are independent in time and space, and a calculation unit that calculates an index related to the motion speed of each motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition unit. [Selected Figure] Figure 3

Description

The present disclosure relates to an information processing device, an information processing method, and a program.

In recent years, animation production and distribution using motion capture to obtain motion information showing a user's movements has become popular. For example, the motion information obtained by motion capture is used to generate motion data that mimics the user's movements, and avatar images based on the motion data are distributed.

As a result of this, the amount of motion data is increasing year by year, and technologies are being developed to reuse previously generated motion data. For example, Patent Document 1 discloses a technology that blends the movements of multiple pieces of motion data and reproduces the movements obtained by blending in real time using an avatar in a virtual space.

International Publication No. 2019/203190

When linking multiple pieces of motion data like this, it may be necessary to perform interpolation editing to join the multiple pieces of motion data more naturally. When performing this type of interpolation editing, parameters such as boundary conditions may be specified in detail, but it may be difficult for the user to set all of the parameters.

This disclosure therefore proposes a new and improved information processing device, information processing method, and program that can link multiple pieces of motion data more naturally.

According to the present disclosure, there is provided an information processing device including an acquisition unit that acquires interpolated motion data that interpolates between first motion data and second motion data that are independent in time and space, and a calculation unit that calculates an index related to the motion speed of each motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition unit.

The present disclosure also provides an information processing method executed by a computer, which includes obtaining interpolated motion data that interpolates between first and second motion data that are independent in time and space, and calculating an index related to the motion speed of each piece of motion data based on the first motion data, the second motion data, and the interpolated motion data.

The present disclosure also provides a program that causes a computer to realize an acquisition function that acquires interpolated motion data that interpolates between first and second motion data that are independent in time and space, and a calculation function that calculates an index related to the motion speed of each motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition function.

FIG. 1 is an explanatory diagram illustrating an information processing system according to an embodiment of the present disclosure. 2 is an explanatory diagram for explaining an example of a functional configuration of a server 10 according to the present disclosure. FIG. 2 is an explanatory diagram for explaining an example of a functional configuration of a PC 20 according to the present disclosure. FIG. FIG. 11 is an explanatory diagram for explaining an example of a linking process of motion data. FIG. 11 is an explanatory diagram for explaining an overview of interpolation editing for interpolating between a plurality of motion data; FIG. 11 is an explanatory diagram illustrating motion blending, which is an example of interpolation editing. FIG. 13 is an explanatory diagram for explaining generation of interpolated motion data C, which is another example of interpolation editing. 11 is an explanatory diagram for explaining a specific example of a process in which the generation unit 241 according to the present disclosure generates interpolated motion data. FIG. FIG. 11 is an explanatory diagram for explaining a specific example of an interpolation parameter. FIG. 11 is an explanatory diagram for explaining an example of a GUI according to the present disclosure. 10 is a flowchart for explaining an example of an operation process of a PC 20 according to the present disclosure. FIG. 2 is a block diagram showing a hardware configuration of a PC 20 according to the present disclosure.

A preferred embodiment of the present disclosure will be described in detail below with reference to the attached drawings. Note that in this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals to avoid redundant description.

The "Mode for Carrying Out the Invention" will be described in the following item order.
1. Overview of Information Processing System 2. Example of Functional Configuration 2.1. Example of Functional Configuration of Server 10 2.2. Example of Functional Configuration of PC 20 3. Details 3.1. Linking of Motion Data 3.2. Interpolation Editing of Motion Data 3.3. Generation of Interpolated Motion Data 3.4. Scoring 3.5. GUI (Graphical User Interface)
4. Operational Processing Example 5. Function and Effect Example 6. Modification Example 7. Hardware Configuration 8. Supplementary Notes

<<1. Overview of the information processing system>>
In order to visualize information on the movement of a moving body such as a human or an animal, for example, skeleton data expressed by a skeleton structure showing the structure of the body is used as the motion data. The skeleton data includes information such as the position and posture of a part. More specifically, the skeleton data includes various information such as the global position of the root joint and the relative posture of each joint. Note that the part in the skeleton structure corresponds to, for example, an end part or a joint part of the body. The skeleton data may also include bones, which are line segments connecting parts. The bones in the skeleton structure may correspond to, for example, human bones, but the positions and number of bones do not necessarily have to match the actual human skeleton. The motion data may further include various information such as skeletal information showing the length of each bone and the connection relationship with other bones, and time-series data of the ground contact information of both feet.

The position and posture of each part in the skeleton data can be obtained using a variety of motion capture technologies. For example, there is a camera-based technology in which markers are attached to each part of the body and the marker positions are obtained using an external camera, and there is a sensor-based technology in which motion sensors are attached to parts of the body and position information from the motion sensors is obtained based on the time-series data obtained by the motion sensors.

Skeleton data has a variety of uses. For example, motion data, which is time-series data of skeleton data, is used to improve form in dance or sports, and is used in applications such as VR (Virtual Reality) or AR (Augmented Reality). Motion data is also used to generate avatar images that mimic the movements of a user, and the avatar images are distributed.

The information processing system according to the present disclosure makes it possible to link multiple pieces of motion data more naturally. First, an overview of the information processing system will be described with reference to FIG. 1.

FIG. 1 is an explanatory diagram for explaining an information processing system according to one embodiment of the present disclosure. As shown in FIG. 1, the information processing system according to the present disclosure includes a network 1, a server 10, and a PC (Personal Computer) 20.

(Network 1)
The network 1 according to the present disclosure is a wired or wireless transmission path for information transmitted from devices connected to the network 1. For example, the network 1 may include public line networks such as the Internet, telephone line networks, and satellite communication networks, as well as various LANs (Local Area Networks) including Ethernet (registered trademark), and WANs (Wide Area Networks). The network 1 may also include dedicated line networks such as IP-VPN (Internet Protocol-Virtual Private Network). The server 10 and the PC 20 are connected via the network 1.

(Server 10)
The server 10 according to the present disclosure is a device that stores multiple pieces of motion data. The server 10 also learns the relationship between two pieces of motion data and parameters related to movement (hereinafter, sometimes referred to as interpolation parameters), and the interpolated motion data, and obtains a generation model that generates the interpolated motion data.

(PC20)
The PC 20 according to the present disclosure is an example of an information processing device that acquires interpolated motion data that interpolates between two motion data, and calculates an index related to the motion speed of each motion data based on the two motion data and the interpolated motion data.

The above describes an overview of the information processing system according to the present disclosure. Next, the functional configurations of the server 10 and the PC 20 will be described in detail with reference to FIG. 2 and FIG. 3.

<<2. Example of functional configuration>>
<2.1. Example of functional configuration of server 10>>
2 is an explanatory diagram for explaining an example of a functional configuration of the server 10 according to the present disclosure. As shown in FIG. 2, the server 10 according to the present disclosure includes a storage unit 110, a learning unit 120, and a communication unit 130.

(Memory unit 110)
The storage unit 110 according to the present disclosure stores software and various data. For example, the storage unit 110 stores a plurality of pieces of motion data. The storage unit 110 may also store a generative model obtained by the learning unit 120.

(Learning Unit 120)
The learning unit 120 according to the present disclosure learns the relationship between two pieces of temporally and spatially independent motion data, the interpolation parameters, and the interpolated motion data, to generate a generative model. Details related to the learning will be described later.

(Communication unit 130)
The communication unit 130 according to the present disclosure transmits and receives various information to and from the PC 20 via the network 1. The communication unit 130 receives, for example, request information requesting motion data from the PC 20. The communication unit 130 also transmits motion data to the PC 20 in response to the request information received from the PC 20.

The communication unit 130 may also transmit the generative model stored in the memory unit 110 to the PC 20.

An example of the functional configuration of the server 10 according to the present disclosure has been described above. Next, an example of the functional configuration of the PC 20 according to the present disclosure will be described with reference to FIG. 3.

<2.2. Example of functional configuration of PC 20>>
3 is an explanatory diagram for explaining an example of a functional configuration of the PC 20 according to the present disclosure. As shown in FIG. 3, the PC 20 according to the present disclosure includes a communication unit 210, an operation display unit 220, a storage unit 230, and a control unit 240.

(Communication unit 210)
The communication unit 210 according to the present disclosure transmits and receives various information to and from the server 10 via the network 1. The communication unit 210 transmits, for example, request information requesting motion data to the server 10. Then, the communication unit 210 receives the motion data transmitted in response to the request information from the server 10.

The communication unit 210 may also receive a generative model for generating the interpolated motion data from the server 10.

(Operation display unit 220)
The operation display unit 220 according to the present disclosure functions as a display unit that displays motion data received from the server 10 and various display information (e.g., linked motion data, etc.) generated by a generating unit 241 described below. The operation display unit 220 also functions as an input unit for the user to select motion data. When certain motion data is selected by the user, the communication unit 210 transmits request information requesting the motion data to the server 10.

The display function is realized, for example, by a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, or an OLED (Organic Light Emitting Diode) device.

The input function can be realized, for example, by a touch panel, keyboard, or mouse.

In FIG. 1, the PC 20 has a configuration in which the functions of the display unit and the operation unit are integrated, but the functions of the display unit and the input unit may be separate.

(Memory unit 230)
The storage unit 230 according to the present disclosure stores software and various data. The storage unit 230 stores a generative model received from the server 10. The storage unit 230 may also store interpolated motion data generated by a generating unit 241 described below. The storage unit 230 may also store linked motion data that combines the interpolated motion data, first motion data, and second motion data generated by the generating unit 241.

(Control unit 240)
The control unit 240 according to the present disclosure controls the overall operation of the PC 20. The PC 20 includes a generating unit 241 and a calculating unit 243, as shown in FIG.

The generation unit 241 according to the present disclosure is an example of an acquisition unit, and generates interpolated motion data that interpolates between first motion data and second motion data that are independent in time and space. The generation of the interpolated motion data will be described in detail below.

For example, the generation unit 241 may input the first motion data, the second motion data, and at least one or more interpolation parameters related to the movement into a generative model to generate the interpolated motion data. The generation unit 241 may also modify the interpolation parameters input into the generative model to generate the interpolated motion data multiple times.

The generation unit 241 may also generate linked motion data by linking the first motion data, the interpolated motion data, and the second motion data.

The calculation unit 243 according to the present disclosure calculates an index related to the motion speed of each piece of motion data based on the first motion data and the second motion data received by the communication unit 210 and the interpolated motion data generated by the generation unit 241. Details of the calculation of the index related to the motion speed will be described later.

The above describes the details of the PC 20 according to the present disclosure. Next, the details of the information processing system according to the present disclosure will be described in order with reference to Figures 4 to 9.

<<3. More details>>
Recently, motion data obtained by motion capture or manually is registered in a database, and applications exist that search for or edit the motion data from the database. For example, in the case of editing motion data, there is a joint editing that connects multiple motion data. First, a specific example of joint editing of a plurality of motion data will be described with reference to FIG.

<3.1. Linking Motion Data>
4 is an explanatory diagram for explaining an example of a motion data linking process. In some cases, a section included in some motion data (hereinafter referred to as first motion data A) is modified by replacing it with other motion data (hereinafter referred to as second motion data B).

For example, a user of an application searches for a certain first motion data A, and selects section A2 from among multiple sections A1 to A3 contained in the first motion data A as the section to be modified.

The user then selects the second motion data B as the target for modifying section A2. As a result, section A2 of the first motion data A may be modified by replacing it with the second motion data B that is the target for modification. Note that the second motion data B here may be any motion data selected by the user on an application, or may be motion data automatically selected by the application.

For example, as shown in FIG. 4, when there are two pieces of second motion data B as candidates for correction of section A2 of first motion data A, the user selects one of the pieces of second motion data B. For example, when the user selects the left image of second motion data B shown in FIG. 4, section A2 of the first motion data A is corrected by replacing it with the left image of motion data B shown in FIG. 4.

However, if section A2 of the first motion data A is simply replaced with the second motion data B, the motion will suddenly switch at the boundary between the first motion data A and the second motion data B, and the movements contained in the motion data may appear unnatural.

Note that the boundary portion here includes the portion switching from the end position of section A1 of the first motion data A to the start position of the second motion data B. The boundary portion also includes the portion switching from the end position of the second motion data B to the start position of section A3 of the first motion data A.

When linking and editing multiple pieces of motion data in this way, it is desirable to perform an interpolation edit that interpolates between the first motion data A and the second motion data B (i.e., the boundary portion).

A specific example of linked editing of multiple motion data has been explained above. Next, we will explain a specific example of interpolation editing that interpolates between multiple motion data.

<3.2. Interpolation editing of motion data>
5 is an explanatory diagram for explaining an outline of the interpolation editing for interpolating between a plurality of motion data. For example, it is assumed that the first motion data A includes a walking motion, and the second motion data B includes a kicking motion.

Here, when there is an abrupt switch from the first motion data A to the second motion data B as shown in FIG. 5, the motion of walking switches instantly to a kicking motion, which can make the motion data unnatural. For example, the joint positions of each part included in the first motion data A transition to the joint positions of each part included in the second motion data B in one frame, which can cause a situation in which the position of a certain part moves (warps) instantaneously. Interpolation editing is a method for suppressing the unnaturalness of movement that can occur when multiple pieces of motion data are linked. A specific example of interpolation editing will now be described with reference to FIG. 6A and FIG. 6B.

Figure 6A is an explanatory diagram for explaining motion blending, which is an example of interpolation editing. For example, motion blending is an interpolation editing that linearly and geometrically combines first motion data A and second motion data B.

In interpolation editing using motion blending, interpolation can be performed in which the movement contained in the first motion data A gradually switches to the movement contained in the second motion data B in the interpolation section BQ. Note that the motion data in which the movement gradually switches in the interpolation section BQ is an example of interpolated motion data C.

Motion blending has few parameters related to interpolation and can be used for interpolation editing that is suitable for situations where real-time performance is required even if some unnaturalness of movement remains. On the other hand, motion blending can result in interpolation editing that is not suitable for situations such as animation editing, where more natural movement is required and there are many parameters related to interpolation.

FIG. 6B is an explanatory diagram for explaining the generation of interpolated motion data C, which is another example of interpolation editing. The generation of the interpolated motion data C includes a process of generating new motion data using machine learning techniques such as Deep Learning. Here, the newly generated motion data is an example of the interpolated motion data C.

When generating the interpolated motion data C, the number of parameters related to the interpolation is greater than in the case of motion blending, but it is possible to generate the interpolated motion data C that can interpolate between the first motion data A and the second motion data B with more natural movements.

However, because there are many parameters related to interpolation, it is difficult for the user to set all of the parameters manually. In addition, the parameters set by the user do not necessarily generate interpolated motion data C that naturally interpolates between motion data. Furthermore, when parameters are set automatically, the type of movement desired by the user also depends on the user's preferences, making it difficult to uniquely determine this when setting the parameters.

Next, the process by which the generation unit 241 according to the present disclosure generates the interpolated motion data C will be described in detail.

<3.3. Generation of Interpolated Motion Data>
7 is an explanatory diagram for explaining a specific example of a process in which the generation unit 241 according to the present disclosure generates interpolated motion data. First, as a preliminary step, the learning unit 120 included in the server 10 generates a generative model for generating the interpolated motion data C. For example, the learning unit 120 generates the generative model by machine learning using a set of two pieces of temporally and spatially independent motion data and parameters related to interpolation (hereinafter referred to as interpolation parameters), and the interpolated motion data as teacher data.

More specifically, as shown in FIG. 7, the learning unit 120 divides the motion data into three parts of arbitrary length, which are the first motion data A, the second motion data B, and the third motion data L. The lengths of the first motion data A, the second motion data B, and the third motion data L may be changed for each use case, or may be variable during learning of the generative model.

Then, the learning unit 120 inputs the first motion data A, the second motion data B, and the interpolation parameters into the generative model. At this time, the third motion data L is considered to be lost (i.e., the third motion data L is not input into the generative model). In this way, the learning unit 120 generates a provisional generative model.

Next, the learning unit 120 uses the generated generative model to calculate the loss between the interpolated motion data C generated from the first motion data A and the second motion data B, and the third motion data (i.e., the correct motion).

The learning unit 120 may then modify the parameters of the interpolated motion data (e.g., joint positions and joint postures) to reduce loss and repeat learning to update the generative model. The communication unit 130 included in the server 10 may then transmit the generative model generated by the learning unit 120 to the PC 20.

The learning unit 120 may generate a generative model from any motion data (multiple motion data including various movements), or may generate a generative model for each specific motion data, such as walking or running.

The generation unit 241 included in the PC 20 according to the present disclosure may generate the interpolated motion data C using the generation model G obtained from the server 10. More specifically, the generation unit 241 may generate the interpolated motion data ^xC using the following formula (1).

In addition, in the formula (1), ^xC is the interpolated motion data, ^xA is the first motion data, ^xB is the second motion data, and _c0 to _cn are the interpolated parameters.

The interpolation parameters are various parameters related to the movement of the motion data, such as boundary conditions or constraint conditions, and may be partially set by the user or may be automatically set based on the first motion data A and the second motion data B.

Here, we will explain a specific example of the interpolation parameters with reference to Figure 8.

(Interpolation parameters)
Fig. 8 is an explanatory diagram for explaining a specific example of the interpolation parameters. Fig. 8 shows a situation in which the first motion data A and the second motion data B are independent in time and space. Here, spatial independence refers to a state in which the first motion data A and the second motion data B are spatially separated, and temporal independence refers to a state in which the time from the start time to the end time of the first motion data A and the time from the start time to the end time of the second motion data B do not overlap.

For example, the position of motion data in three-dimensional space is represented by a position vector P. For example, the position of the first motion data A at time _tA is represented by P _A = (x _A , y _A , z _A ), and the position of the second motion data B at time _tB is represented by P _B = (x _B , y _B , z _B ).

The position of the motion data in three-dimensional space may be, for example, the position of the character's root joint (waist joint), the position of another joint (for example, a toe joint), or a position calculated from the positions of multiple joints.

The posture of the motion data in three-dimensional space is represented by a total joint posture S, which corresponds to the relative postures of all joints. For example, the posture of the first motion data A at time _tA is represented by S _A (t _A ). The posture of the second motion data B at time _tB is represented by S _B (t _B ).

There may be various types of interpolation parameters. In this specification, eight types of parameters (A) to (H) are introduced, but other parameters may be included. Also, some of the parameters (A) to (H) may be manually specified by the user. Also, not all of the parameters (A) to (H) may be used to generate the interpolated motion data. In other words, some of the parameters (A) to (H) may be input to the generative model.

(A) Spatial Position The interpolation parameters may include parameters related to the spatial positions of the motion data. For example, the parameters related to the spatial positions are the distances in three-dimensional space between the first motion data A and the second motion data B. In other words, the parameters related to the spatial positions are the difference between the spatial positions of the first motion data A and the second motion data B, and are represented as ||p _A -p _B ||.

Here, the position _pA of the first motion data or the position _pB of the second motion data may be set manually by the user, or may be set automatically based on the first motion data A and the second motion data B.

For example, the spatial position difference may be automatically set based on the estimation result by estimating the trajectory of a part or joint of the interpolated motion data based on the velocity of each root joint in the first motion data A and the second motion data B.

(B) Interpolation Time The interpolation parameters may include parameters related to an interpolation time to be interpolated by the interpolated motion data. The parameters related to the interpolation time are a time difference on a time line between the first motion data A and the second motion data B, and are represented by t _B -t _A.

In addition, the end time of the first motion data A or the start time of the second motion data B may be set manually by the user or automatically.

For example, the interpolation time may be automatically set based on the difference in the velocity or spatial position of the root joint between the first motion data A and the second motion data B.

(C) Posture of first motion data A (interpolation start posture, boundary conditions)
The interpolation parameters may include parameters related to the posture of the first motion data A. The parameters related to the posture of the first motion data A are parameters for adjusting the posture at the interpolation start time _tA by trimming the motion data A after the interpolation start time _tA , and are represented as S _A (t _A ). Adjusting the parameters related to the posture of the first motion data A can make it possible to make the posture of the interpolated motion data C at the start of the interpolation more natural.

In addition, the parameters related to the posture of the first motion data A may be automatically set based on the most stable posture as the interpolation start posture estimated from the velocity of the root joint of the first motion data A.

(D) Posture of second motion data B (interpolation end posture, boundary conditions)
The interpolation parameters may include parameters related to the posture of the second motion data B. The parameters related to the posture of the second motion data B are parameters for adjusting the posture at the interpolation end time _tB by trimming the motion data B before the interpolation end time _tB , and are represented as S _B (t _B ). Adjusting the parameters related to the posture of the second motion data B can make it possible to make the posture of the interpolated motion data C at the end of the interpolation more natural.

In addition, the parameters related to the posture of the second motion data B may be automatically set based on the most stable posture as the interpolation end posture estimated from the velocity of the root joint of the second motion data B.

(E) Contact Condition The interpolation parameters may include parameters related to the contact condition. The parameters related to the contact condition may be parameters related to contact with the ground, such as the number of steps taken between the interpolation start time _tA and the interpolation end time _tB .

For example, the parameter related to the contact condition may be specified as "n steps" throughout the entire interpolation time from the interpolation start time _tA to the interpolation end time _tB . For example, when a parameter related to the contact condition of "five steps" throughout the entire interpolation time is set, interpolated motion data C is generated in which five steps are taken between the interpolation start time _tA and the interpolation end time _tB .

Furthermore, if the state in which the toes are in contact with the ground is designated as "1" and the state in which the toes are not in contact with the ground (i.e., the toes are off the ground) is designated as "0", the parameters relating to the contact condition may be sequentially designated, such as c(t)={0, 1}, at _each time t between t _A and t _B. For example, if the parameter relating to the contact condition is set as "c(t ₁ )={0, 1}", interpolated motion data C is generated in which the toe of the left foot is not in contact with the ground and the toe of the right foot is in contact with the ground at time t 1.

In addition, the parameters related to the contact condition may be automatically set based on a more appropriate number of steps estimated from the difference in spatial positions or the interpolation time between the first motion data A and the second motion data B.

(F) Positional Constraint Conditions The interpolation parameters may include parameters related to positional constraint conditions. The parameters related to positional constraint conditions are parameters related to fixing intermediate postures of the interpolated motion data C. The parameters related to positional constraint conditions are parameters that fix the postures of all joints in the interpolated motion data C at a certain time t between t _A and t _B , and are represented by S _C (t).

For example, if a user is looking for interpolated motion data in which the knee touches the ground at a certain time t between t _A and t _B , the user may set parameters of all joint postures in which the knee touches the ground at that time t as parameters related to position constraint conditions.

(G) Semantic Conditions The interpolation parameters may include parameters related to semantic conditions. The parameters related to semantic conditions include parameters related to the style of the interpolated motion data C.

The style here is defined by labels expressed as the characteristics of the movements of the motion data and genres, and the learning unit 120 can assign labels to the motion data and train a generative model to set parameters related to semantic conditions.

Specific examples of styles include "dynamically," "dance-like," "jumping," and "masculine (feminine)." Parameters related to semantic conditions may be assigned for each application, or the generating unit 241 may generate the interpolated motion data C without providing parameters related to such semantic conditions.

(H) Random Numbers or Noise The interpolation parameters may include parameters related to random numbers or noise. For example, a generation model that generates the interpolated motion data C may input random numbers as a probabilistic generation model. By using parameters related to random numbers or noise, different interpolated motion data C can be generated even if the other interpolation parameters described above are fixed.

The generation unit 241 may generate various variations of the interpolated motion data C by inputting a random number as a seed, or may generate the interpolated motion data C without providing a random number (i.e., by setting the random number to a fixed value).

Specific examples of the interpolation parameters have been described above. As described above, the generation unit 241 generates the interpolation motion data C based on the first motion data A, the second motion data B, and the interpolation parameters.

The generation unit 241 then generates linked motion data by linking the first motion data A, the interpolated motion data C, and the second motion data. However, depending on the setting value of the interpolation parameter, there is a risk that the motion data included in the generated linked motion data may not be linked naturally (smoothly).

The information processing system according to the present disclosure has a mechanism that enables the first motion data A, the interpolated motion data C, and the second motion data B to be linked more naturally. More specifically, the calculation unit 243 calculates an index related to the motion speed of each piece of motion data based on the first motion data A, the second motion data B, and the interpolated motion data C generated by the generation unit 241.

Next, we will explain specific examples of the indices calculated by the calculation unit 243 and a specific example of the process of presenting candidates for interpolated motion data based on the indices to the user.

3.4. Scoring
First, the generation unit 241 generates the interpolation motion data ^xC based on the first motion data ^xA , the second motion data ^xB , and the interpolation parameters _c0 to _cn using the above-mentioned formula (1). Then, the calculation unit 243 may calculate a total index s including at least one index S using the following formula (2).

For example, the calculation unit 243 may calculate a total index s including five indexes S using the following formula (3):

In formula (3), _{wp, wv,} wj _{, wf} , and _wm are weighting coefficients for each index, and may be set automatically or may be specified for each use case.

Furthermore, the total index s in Equation (3) is defined as a combination of a physical index and a heuristic index. However, the index S included in the total index s is not limited to the physical index and the heuristic index described below. Furthermore, the index S in the total index s does not necessarily include all of the indexes S described below. For example, Equation (3) does not necessarily include the term w _m S _{meta (} meta index). First, the four physical indexes in Equation (3) will be described.

(Physical indicators)
For example, the physical indices include indices related to the motion speeds of the first motion data A, the interpolated motion data C, and the second motion data B (hereinafter, sometimes referred to as each motion data). The motion speeds here include, for example, the speed of the root joint, the speeds of other joints (e.g., the speed of the toes), or a speed obtained by combining the speeds of a plurality of joints. Indices related to the motion speeds may also include indices related to the speed, acceleration, or position of the motion.

For example, the physical index may include a successive position index S _position indicating the degree of succession of each motion data. The successive position index S _position may be an index based on the difference between the maximum value of the velocity of the interpolated motion data C and the average velocity of the first motion data A and the second motion data B in multiple frames before and after the interpolated motion data C.

In particular, in the section where the first motion data A transitions to the interpolated motion data C, and in the section where the interpolated motion data C transitions to the second motion data B, unnatural behavior may occur if the joint positions of the motion data move instantaneously. A continuous position index S _position is defined as an index for reducing such unnatural behavior, and the smaller the value obtained by the following formula (4), the larger the continuous position index S _position is calculated to be.

According to formula (4), the continuous position index S _position is calculated to be larger as the difference between the maximum velocity Max (||v _C ||) of the interpolated motion data C and the average velocity ||v _AB || of the first motion data A and second motion data B in multiple frames before and after the interpolated motion data C becomes smaller. In other words, a larger continuous position index S _position indicates a state in which the joint positions of each motion data are moving continuously.

The physical index may also include a velocity index S _velocity indicating the degree of smoothness of the velocity of each motion data. The velocity index S _velocity may be an index based on the difference between the average velocity of the interpolated motion data C and the average velocities of the first motion data A and the second motion data B.

When the average speed of the first motion data A and second motion data B differs greatly from the average speed of the interpolated motion data C, the interpolated motion data C may suddenly move faster or slower compared to the first motion data A or the second motion data B, resulting in unnatural behavior. A speed index S _velocity is defined as an index for reducing such unnatural behavior, and the smaller the value obtained by the following formula (5), the larger the calculated speed index S _velocity .

According to formula (5), the velocity index S _velocity is calculated to be large as the difference between the average velocity ∥Average(v _C )∥ of the interpolated motion data C and the average velocity ∥v _AB ∥ of the first motion data A and the second motion data B becomes small. In other words, a large velocity index S _velocity indicates a state in which each motion data is moving more smoothly.

The physical index may also include an acceleration index S _jerk that indicates the degree to which the jerk (jerk: rate of change of acceleration) of the interpolated motion data C is small. The acceleration index S _jerk may be an index based on a differential component of the acceleration of the interpolated motion data C.

If the interpolated motion data C generated by the generation unit 241 suddenly accelerates or decelerates, this may result in unnatural behavior. An acceleration index S _jerky is defined as an index for reducing such unnatural behavior, and the smaller the value obtained by the following formula (6), the larger the acceleration index S _jerky is calculated to be.

According to formula (6), the acceleration index S _jerk is calculated to be large when the jerk, which is the differential value of the interpolated motion data C, is small. In other words, the acceleration index S _jerk being calculated to be large indicates a state in which the jerk of each motion data is small (i.e., a state in which there is no sudden acceleration or deceleration).

The physical index may also include a foot index S _foot indicating whether the toe of the interpolated motion data C will slip when touching the ground. The foot index S _foot is an index related to the speed of the tip of the foot of the interpolated motion data C when the tip of the foot is in contact with the ground.

When the toe of the interpolated motion data C is in contact with the ground, if the magnitude of the velocity of the toe is equal to or greater than 0, the toe of the interpolated motion data C is slipping on the ground. In many cases, foot slipping can result in unnatural behavior. A toe index S _foot is defined as an index for reducing such unnatural behavior, and the smaller the value obtained by the following equation (7), the larger the toe index S _foot is calculated to be.

According to equation (7), the toe index S _foot is calculated to be large as the magnitude of the velocity of the toes of the interpolated motion data C decreases. In other words, the toe index S _foot being calculated to be large indicates a state in which the toes of the interpolated motion data C are not slipping on the ground.

A specific example of a physical indicator has been described above. Next, we will explain an example of a heuristic indicator included in formula (3).

(Heuristic Indicators)
The heuristic index may include a meta index S _meta that differs for each use case. The meta index S _meta may be an index according to the use case based on a difference between an attribute of the interpolated motion data and an attribute required for the interpolated motion data.

A user may feel that the attributes of the interpolated motion data C generated by the generation unit 241 are unnatural if they are significantly different from each other depending on the user's sensibility or use case. For example, if the first motion data A and the second motion data B are motions performed by a woman, and the interpolated motion data C generated by the generation unit 241 is motions that look like they were performed by a large man, the connected motion data obtained by connecting the respective motion data may exhibit unnatural behavior. A meta-index S _meta is defined as an index for reducing such unnatural behavior according to a use case. If a function for determining the attributes of the interpolated motion data C is F and the attribute to be sought is m, the smaller the value obtained by the following formula (8), the larger the calculated meta-index S _meta is.

According to formula (8), the smaller the difference between the attribute m determined according to the use case and the attribute F(M _C ) of the interpolated motion data C generated by the generation unit 241, the larger the calculated meta index S _{meta is} . In other words, a state in which the interpolated motion data C generated by the generation unit 241 corresponds to the user's sensibility and the use case is indicated.

After the total index s as described above is calculated by the calculation unit 243, the generation unit 241 changes the values of the interpolation parameters and generates the interpolated motion data ^xC again, and the calculation unit 243 again calculates the total index s from the generated interpolated motion data ^xC .

This series of processes from modifying the interpolation parameters to calculating the total index s may be repeated multiple times.

That is, the generation unit 241 may modify the interpolation parameters and generate the interpolated motion data ^xC again, and the calculation unit 243 may calculate the total index s from the interpolated motion data ^xC generated again by the generation unit 241. This series of processes may be repeated multiple times.

The calculation method for the physical indexes described above is not limited to the above-mentioned formulas, and a more precise calculation method may be used. In addition, the calculation method for the heuristic index may use an index suited to other use cases.

In addition, in the above-mentioned calculation formulas for the physical index and the heuristic index, an example was described in which the smaller the value obtained by each calculation formula, the larger the index S becomes, but the formulas may be modified so that the larger the value obtained by each calculation formula, the larger the index S becomes.

The above describes specific examples of the indices calculated by the calculation unit 243. According to the series of processes described above, multiple combinations of multiple pieces of interpolated motion data C and scores (total indices or indices) calculated from the multiple pieces of motion data C are prepared.

Here, the generation unit 241 may add the interpolated motion data C whose score (total index or index) has been calculated by the calculation unit 243 to the candidate list.

Then, the generation unit 241 may sort the list of candidates for the interpolated motion data C in order of score (more specifically, in order of highest score), and the operation display unit 220 may display the sorted candidates for the interpolated motion data.

This allows the user to view candidates for interpolated motion data C sorted in descending order of score, improving the convenience for the user when viewing the interpolated motion data C.

Next, an example of a GUI (Graphical User Interface) according to the present disclosure will be described with reference to FIG. 9.

<3.5. GUI (Graphical User Interface)>
9 is an explanatory diagram for explaining an example of a GUI according to the present disclosure. First, the user uses the operation display unit 220 to select a first motion data A and insert it into an arbitrary section M1 on the timeline.

Then, the user uses the operation display unit 220 to select the second motion data B and insert it into another arbitrary section M2 on the timeline. Note that the arbitrary sections M1 and M2 may be specified by an operation such as drag and drop, or by inputting the time.

Here, the generation unit 241 sets the section from the end position of the first motion data A (i.e., the end position of the arbitrary section M1) to the start position of the second motion data B (i.e., the start position of another arbitrary section M2) as the interpolation section M3.

The user may also manually set some of the above-mentioned interpolation parameters using the operation display unit 220.

When the user performs an operation related to generating interpolated motion data (for example, pressing an execute button not shown), the generation unit 241 generates the interpolated motion data, and the calculation unit 243 calculates the score of the interpolated motion data generated by the generation unit 241.

Then, the generation unit 241 sorts the interpolated motion data C in order of score, and the operation display unit 220 displays a list of candidates for the interpolated motion data sorted in order of score, as shown in FIG. 9.

The candidate list shown in FIG. 9 is a candidate list in which the first interpolated motion data C1, the second interpolated motion data C2, the third interpolated motion data C3, and the fourth interpolated motion data C4 are calculated to have the highest scores, in that order. Furthermore, if the upper limit of the interpolated motion data to be included in the candidate list is set to four, the four interpolated motion data C1 to C4 as shown in FIG. 9 are displayed on the operation display unit 220. However, if the upper limit of the interpolated motion data to be included in the candidate list is set to five or more, for example, the user can scroll down the candidate list to check the interpolated motion data C5 onwards that have been calculated to have the lowest scores.

Then, when the user selects one piece of interpolated motion data C from the candidate list, the generation unit 241 generates linked motion data that links the first motion data A, the interpolated motion data C selected by the user, and the second motion data B.

Then, the operation display unit 220 displays the linked motion data generated by the generation unit 241.

Although an example of a GUI according to the present disclosure has been described above, the GUI is not limited to the above example. For example, the operation display unit 220 may display linked motion data that simply links the first motion data A and the second motion data B, or linked motion data that is interpolated by motion blending.

The above describes the details of the information processing system according to the present disclosure. Next, an example of the operation process of the PC 20 according to the present disclosure will be described with reference to FIG. 10.

<<4. Operation processing example>>
10 is a flowchart for explaining an example of an operational process of the PC 20 according to the present disclosure. First, the user uses the operation display unit 220 to select the first motion data A (S101).

Next, the user selects the second motion data B using the operation display unit 220 (S105).

Next, the generation unit 241 automatically sets at least one or more interpolation parameters (S109). Here, the user may manually set some of the interpolation parameters using the operation display unit 220.

Then, the generation unit 241 generates the interpolated motion data C based on the first motion data A and the second motion data B selected by the user and the interpolation parameters set by the generation unit 241 (or the user) (S113).

Next, the calculation unit 243 calculates a score (total index or index) based on the interpolated motion data C generated by the generation unit 241 and the first motion data A and second motion data B selected by the user (S117).

Then, the generation unit 241 adds the interpolated motion data C whose score has been calculated by the calculation unit 243 to the candidate list (S121). Note that the generation unit 241 may add to the candidate list the interpolated motion data C whose score calculated by the calculation unit 243 is equal to or greater than a predetermined value, and may not add to the candidate list the interpolated motion data C whose score is less than the predetermined value.

Next, the generation unit 241 sorts the candidate list of the interpolated motion data C in order of score (more specifically, in order of highest score) (S125).

Then, the generation unit 241 determines whether the amount of interpolated motion data C added to the candidate list has reached the upper limit of the candidate list (S129). If the upper limit of the candidate list has been reached (S129: YES), the process proceeds to S133. If the upper limit of the candidate list has not been reached (S129: NO), the process returns to S109 again, where the interpolation parameters are reset (changed), and the processes of S109 to S129 are repeated until the upper limit of the candidate list is reached in S129. Note that the upper limit of the candidate list here corresponds to a predetermined number of times and may be specified by the user or may be set automatically.

If the amount of interpolated motion data C reaches the upper limit of the candidate list (S129: YES), the operation display unit 220 displays a candidate list of interpolated motion data C sorted in order of score (S133).

Then, the operation display unit 220 allows the user to select one of the interpolated motion data C included in the candidate list (S137).

Then, the generation unit 241 generates linked motion data by interpolating the interpolated motion data C selected by the user between the first motion data A and the second motion data B (S141), and the PC 20 according to the present disclosure ends the process.

<<5. Examples of effects>>
According to the present disclosure described above, various operational effects can be obtained. For example, the generation unit 241 according to the present disclosure generates the interpolated motion data C that interpolates between the first motion data A and the second motion data B that are independent in time and space, and the calculation unit 243 calculates an index related to the motion speed of each piece of motion data based on the first motion data A, the second motion data B, and the interpolated motion data C generated by the generation unit 241. This allows the user to determine the degree to which the connected motion data, which is expected when the first motion data A, the second motion data B, and the interpolated motion data C are connected, will behave more naturally. As a result, the generation unit 241 may be able to generate connected motion data that connects a plurality of motion data more naturally.

The generation unit 241 also changes the interpolation parameters to generate the interpolated motion data C multiple times, and the calculation unit 243 calculates a score using each piece of interpolated motion data C generated by the generation unit 241. Furthermore, the operation display unit 220 displays a candidate list of the interpolated motion data C sorted in order of score. This makes it easy for the user to select the interpolated motion data C that the user desires. As a result, the user can confirm linked motion data that meets the user's wishes.

<<6. Modifications>>
(Motion capture interpolation technology)
The interpolation process by generating the interpolated motion data according to the present disclosure can also be applied to the interpolation technology in motion capture. For example, when motion is recorded by a motion capture system, some of the motion may be lost due to occlusion or the like.

In this case, the generation unit 241 may generate interpolated motion data C that smoothly interpolates the first motion data A and the second motion data B, respectively, for the motion before and after the lost section, to interpolate the motion of the lost section.

In this case, when setting the interpolation parameters, among the multiple interpolation parameters, the interpolation parameters related to the spatial position, the interpolation time, the posture of the first motion data A, and the posture of the second motion data B are fixed. The PC 20 modifies the other interpolation parameters (i.e., the contact condition, the position constraint condition, the semantic condition, the random number, etc.) to repeatedly generate the interpolated motion data and calculate the score. Then, after candidates for the interpolated motion data are displayed, the user can specify one of the interpolated motion data to interpolate the lost section. Note that the user does not necessarily have to specify one of the interpolated motion data, and the generation unit 241 may automatically specify the interpolated motion data C with the highest calculated score or the interpolated motion data with a score exceeding a threshold value to interpolate the lost section.

It is also possible that the motion of a part of the body (e.g., a hand) may be lost. In this case, the generation unit 241 may determine a lost section for that part of the body (referred to as a target part).

The generating unit 241 may then generate the first motion data A and the second motion data B for the motion before and after the target part is lost, respectively, and generate interpolated motion data C that smoothly interpolates the target part. This allows the motion of the body excluding the target part to be directly obtained by motion capture, making it possible to interpolate the target part where the motion has been lost.

When the interpolation process for a target part is performed, the target part to be interpolated on the application's timeline may be branched (for example, only the right hand). Also, there may be multiple target parts whose motion is to be interpolated (for example, the right hand + the left hand).

The generating unit 241 may also set position constraint conditions as interpolation parameters for each part other than the target part to be interpolated. This may generate interpolated motion data in which only the target part (for example, only the right hand) is interpolated, while the whole body other than the target part moves according to the motion data.

The calculation unit 243 may also calculate the above-mentioned score from the target parts generated by the generation unit 241. When there are multiple target parts, the calculation unit 243 may change the value of the interpolation parameter for each target part and calculate an index by weighting each target part. Then, the operation display unit 220 may display the interpolated motion data of the target part and the whole-body motion excluding the target part in combination.

In addition, in VR games and applications in the metaverse space, a motion capture system is used to reflect the user's movements on an avatar in real time. In some cases, pre-registered animations such as hand gestures and emote motions are fed into the avatar during real-time motion capture.

Here, the generation unit 421 may generate interpolated motion data C that smoothly interpolates the first motion data A and the second motion data B, with the real-time motion as the first motion data A and the emote motion as the second motion data B. This may enable a natural transition from real-time motion to emote motion. It may also enable new expressions (e.g., VR live) that combine real-time motion and pre-recorded motion.

(Compositing with Motion Blending)
In addition, the interpolation process according to the present disclosure may be a combination of an interpolation process using interpolated motion data generation and an interpolation process using motion blending. For example, the generation unit 241 may perform an interpolation process using motion blending for the upper body of the motion data, and an interpolation process using interpolated motion data generation for the lower body.

In motion blending, the generation unit 421 may, for example, interpolate the connection position of the motion data by linearly interpolating from the interpolation start posture of the first motion data A to the interpolation end posture of the second motion data B.

<<7. Hardware configuration example>>
The embodiment of the present disclosure has been described above. The information processing such as generating the interpolated motion data and calculating the score (total index and index) described above is realized by cooperation between software and the hardware of the PC 20 described below. Note that the hardware configuration described below can also be applied to the server 10.

FIG. 11 is a block diagram showing the hardware configuration of a PC 20 according to the present disclosure. The PC 20 according to the present disclosure includes a CPU (Central Processing Unit) 2001, a ROM (Read Only Memory) 2002, a RAM (Random Access Memory) 2003, and a host bus 2004. The PC 20 also includes a bridge 2005, an external bus 2006, an interface 2007, an input device 2008, an output device 2010, a storage device (HDD) 2011, a drive 2012, and a communication device 2015.

The CPU 2001 functions as an arithmetic processing device and control device, and controls the overall operation of the PC 20 in accordance with various programs. The CPU 2001 may also be a microprocessor. The ROM 2002 stores programs and arithmetic parameters used by the CPU 2001. The RAM 2003 temporarily stores programs used in the execution of the CPU 2001 and parameters that change appropriately during the execution. These are interconnected by a host bus 2004 that is composed of a CPU bus and the like. The functions of the generation unit 241 and the calculation unit 243 described with reference to FIG. 3 can be realized by the cooperation of the CPU 2001, the ROM 2002, the RAM 2003, and software.

The host bus 2004 is connected to an external bus 2006, such as a PCI (Peripheral Component Interconnect/Interface) bus, via a bridge 2005. Note that the host bus 2004, the bridge 2005, and the external bus 2006 do not necessarily need to be configured separately, and these functions may be implemented in a single bus.

The input device 2008 is composed of input means for the user to input information, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 2001. By operating the input device 2008, the user of the PC 20 can input various data to the PC 20 and instruct processing operations.

The output device 2010 includes, for example, a display device such as a liquid crystal display device, an OLED device, and a lamp. Furthermore, the output device 2010 includes an audio output device such as a speaker and headphones. The output device 2010 outputs, for example, played content. Specifically, the display device displays various information such as played video data as text or images. On the other hand, the audio output device converts played audio data, etc. into audio and outputs it.

The storage device 2011 is a device for storing data. The storage device 2011 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage device 2011 is configured, for example, with a HDD (Hard Disk Drive). This storage device 2011 drives the hard disk and stores the programs executed by the CPU 2001 and various data.

The drive 2012 is a reader/writer for storage media, and is built into the PC 20 or attached externally. The drive 2012 reads information recorded on a removable storage medium 30, such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 2003. The drive 2012 can also write information to the removable storage medium 30.

The communication device 2015 is, for example, a communication interface configured with a communication device for connecting to the network 1. The communication device 2015 may be a wireless LAN compatible communication device, an LTE (Long Term Evolution) compatible communication device, or a wired communication device that performs wired communication.

<<8. Supplementary Information>>
Although the preferred embodiment of the present disclosure has been described in detail above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field to which the present disclosure belongs can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

For example, the PC 20 may further include all or part of the functional configuration of the server 10 according to the present disclosure. When the PC 20 includes all of the functional configuration of the server 10 according to the present disclosure, the PC 20 can execute a series of processes from generating the interpolated motion data to calculating the index without communicating via the network 1.

The server 10 may also include a generation unit 241 and a calculation unit 243. In this case, an example of processing relating to the generation of the interpolated motion data and the calculation of the index may be performed on the server, and the results of the processing (e.g., a candidate list sorted in order of score) may be transmitted to the PC 20.

In addition, if the server 10 includes a generation unit, the PC 20 may acquire the interpolated motion data generated by the generation unit of the server 10 from the server 10. In this case, the communication unit 210 included in the PC 20 corresponds to the acquisition unit.

The software may also be used as a standalone application for searching and editing motion data, or the motion interpolation editing function may be used as a plug-in for existing DCC (Digital Content Creation) tools.

Furthermore, although the generation of interpolated motion data and motion blending have been exemplified as specific examples of interpolation editing, the interpolation editing according to the present disclosure is not limited to these examples. For example, the generation unit 241 may interpolate motion data by combining interpolation editing by generating interpolated motion data and interpolation editing by text search of motion data. For example, a user may perform a text search for motion data of an arbitrary body part, and select one motion data from multiple motion data displayed as search results. In this case, the generation unit 241 may interpolate the movement of that body part using the motion data specified by the user, and may interpolate the movement of other body parts by generating interpolated motion data.

Depending on the combination of the first motion data A and the second motion data B, changing the interpolation parameters may not improve the score calculated by the calculation unit 243. In such a case, the operation display unit 220 may present recommendation information such as changing the second motion data B to another second motion data B'.

Furthermore, the steps in the processing of PC 20 in this specification do not necessarily need to be processed in chronological order according to the order described in the flowchart. For example, the steps in the processing of PC 20 may be processed in an order different from the order described in the flowchart.

It is also possible to create a computer program that causes the hardware, such as the CPU, ROM, and RAM, built into the server 10 and the PC 20 to perform functions equivalent to those of the respective components of the server 10 and the PC 20 described above. A storage medium on which the computer program is stored is also provided.

Furthermore, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology disclosed herein may achieve other effects that are apparent to a person skilled in the art from the description in this specification, in addition to or in place of the above effects.

Note that the following configurations also fall within the technical scope of the present disclosure.
(1)
an acquisition unit that acquires interpolated motion data that is interpolated between first motion data and second motion data that are independent in time and space;
a calculation unit that calculates an index related to a motion speed of each piece of motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition unit;
An information processing device comprising:
(2)
The indicator is
an index based on a difference between an average speed of the interpolated motion data and an average speed of the first motion data and the second motion data;
The information processing device according to (1).
(3)
The indicator is
an index based on a difference between a maximum value of the velocity of the interpolated motion data and an average velocity of the first motion data and the second motion data in a plurality of frames before and after the interpolated motion data;
The information processing device according to (1) or (2).
(4)
The indicator is
an index based on a differential component of the acceleration of the interpolated motion data;
The information processing device according to any one of (1) to (3).
(5)
The indicator is
the interpolated motion data including an index related to a velocity of the tip of the foot when the tip of the foot is in contact with the ground;
The information processing device according to any one of (1) to (4).
(6)
The calculation unit is
Calculating an index according to a use case based on a difference between an attribute of the interpolated motion data and an attribute required for the interpolated motion data.
The information processing device according to any one of (1) to (5).
(7)
The calculation unit is
Calculating a plurality of indices related to the speed of each motion data and a plurality of indices related to the use case, and calculating a total index by multiplying each of the plurality of indices by a weighting coefficient and adding them up;
The information processing device according to (6) above.
(8)
The acquisition unit is
obtaining the interpolated motion data based on the first motion data and the second motion data;
The information processing device according to (7) above.
(9)
The acquisition unit is
obtaining the interpolated motion data based on the first motion data, the second motion data, and at least one interpolated parameter associated with a movement;
The information processing device according to (8).
(10)
The acquisition unit is
acquiring the interpolated motion data using a generative model obtained by learning two pieces of temporally and spatially independent motion data and a relationship between the interpolation parameters and the interpolated motion data;
The information processing device according to (9) above.
(11)
The acquisition unit is
modifying the interpolation parameters to obtain the interpolated motion data a plurality of times;
The calculation unit is
calculating the total index from each of the plurality of interpolated motion data acquired by the acquisition unit;
The information processing device according to (10).
(12)
the at least one interpolation parameter is a plurality of interpolation parameters,
one or more interpolation parameters included in the plurality of interpolation parameters are automatically set based on a difference in velocity or spatial position of a root joint between the first motion data and the second motion data;
The information processing device according to any one of (9) to (11).
(13)
The acquisition unit is
obtaining a candidate list of interpolated motion data according to the magnitude of the total index;
The information processing device according to (11) above.
(14)
The acquisition unit is
obtaining a candidate list in which the interpolated motion data is sorted in order according to the magnitude of the sum index;
The information processing device according to (13).
(15)
The acquisition unit is
modifying an interpolation parameter a predetermined number of times to obtain the predetermined number of interpolated motion data, and obtaining a candidate list of the predetermined number of interpolated motion data sorted in descending order of the total index;
The information processing device according to (14) above.
(16)
The acquisition unit is
acquiring linked motion data by linking the first motion data, the second motion data, and the interpolated motion data;
The information processing device according to any one of (13) to (15).
(17)
The acquisition unit is
acquiring linked motion data by linking the first motion data, the second motion data, and one piece of interpolated motion data selected by a user from the candidate list;
The information processing device according to (16).
(18)
Obtaining interpolated motion data that interpolates between first and second motion data that are temporally and spatially independent;
calculating an index related to a motion speed of each motion data based on the first motion data, the second motion data, and the interpolated motion data;
2. An information processing method implemented by a computer, comprising:
(19)
On the computer,
an acquisition function for acquiring interpolated motion data that is interpolated between first motion data and second motion data that are independent in time and space;
a calculation function for calculating an index related to a motion speed of each piece of motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition function;
A program to achieve this.

Reference Signs List 1 Network 10 Server 110 Storage unit 120 Learning unit 130 Communication unit 20 PC
210 Communication unit 220 Operation display unit 230 Storage unit 240 Control unit 241 Generation unit 243 Calculation unit

Claims (19)

an acquisition unit that acquires interpolated motion data that is interpolated between first motion data and second motion data that are independent in time and space;
a calculation unit that calculates an index related to a motion speed of each piece of motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition unit;
An information processing device comprising: The indicator is
an index based on a difference between an average speed of the interpolated motion data and an average speed of the first motion data and the second motion data;
The information processing device according to claim 1 . The indicator is
an index based on a difference between a maximum value of the velocity of the interpolated motion data and an average velocity of the first motion data and the second motion data in a plurality of frames before and after the interpolated motion data;
The information processing device according to claim 1 . The indicator is
an index based on a differential component of the acceleration of the interpolated motion data;
The information processing device according to claim 1 . The indicator is
the interpolated motion data including an index related to a velocity of the tip of the foot when the tip of the foot is in contact with the ground;
The information processing device according to claim 1 . The calculation unit is
Calculating an index according to a use case based on a difference between an attribute of the interpolated motion data and an attribute required for the interpolated motion data.
The information processing device according to claim 1 . The calculation unit is
Calculating a plurality of indices related to the speed of each motion data and a plurality of indices related to the use case, and calculating a total index by multiplying each of the plurality of indices by a weighting coefficient and adding them up;
The information processing device according to claim 6. The acquisition unit is
obtaining the interpolated motion data based on the first motion data and the second motion data;
The information processing device according to claim 7. The acquisition unit is
obtaining the interpolated motion data based on the first motion data, the second motion data, and at least one interpolated parameter associated with a movement;
The information processing device according to claim 8. The acquisition unit is
acquiring the interpolated motion data using a generative model obtained by learning two pieces of temporally and spatially independent motion data and a relationship between the interpolation parameters and the interpolated motion data;
The information processing device according to claim 9. The acquisition unit is
modifying the interpolation parameters to obtain the interpolated motion data a plurality of times;
The calculation unit is
calculating the total index from each of the plurality of interpolated motion data acquired by the acquisition unit;
The information processing device according to claim 10. the at least one interpolation parameter is a plurality of interpolation parameters,
one or more interpolation parameters included in the plurality of interpolation parameters are automatically set based on a difference in velocity or spatial position of a root joint between the first motion data and the second motion data;
The information processing device according to claim 9. The acquisition unit is
obtaining a candidate list of interpolated motion data according to the magnitude of the total index;
The information processing device according to claim 11. The acquisition unit is
obtaining a candidate list in which the interpolated motion data is sorted in order according to the magnitude of the sum index;
The information processing device according to claim 13. The acquisition unit is
modifying an interpolation parameter a predetermined number of times to obtain the predetermined number of the interpolated motion data, and obtaining a candidate list of the predetermined number of the interpolated motion data sorted in descending order of the total index;
The information processing device according to claim 14. The acquisition unit is
acquiring linked motion data by linking the first motion data, the second motion data, and the interpolated motion data;
The information processing device according to claim 13. The acquisition unit is
acquiring linked motion data by linking the first motion data, the second motion data, and one piece of interpolated motion data selected by a user from the candidate list;
The information processing device according to claim 16. Obtaining interpolated motion data that interpolates between first and second motion data that are temporally and spatially independent;
calculating an index related to a motion speed of each motion data based on the first motion data, the second motion data, and the interpolated motion data;
2. An information processing method implemented by a computer, comprising: On the computer,
an acquisition function for acquiring interpolated motion data that is interpolated between first motion data and second motion data that are independent in time and space;
a calculation function for calculating an index related to a motion speed of each piece of motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition function;
A program to achieve this.

Images