JP6868673B1 - Information processing equipment, information processing methods, and information processing programs - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device, an information processing method and an information processing program for supporting an inexperienced person to easily analyze a difference between his / her own movement and an proficient person's movement and obtain a method for correcting his / her movement. To do. SOLUTION: In order to extract a difference between a proficient person's movement and an inexperienced person's movement, an information processing device extracts each part of the proficient person's body extracted from a first moving image of the proficient person's movement. The first position coordinates of the above are acquired, and the second position coordinates of each part of the body of the inexperienced person are acquired, which are extracted from the second moving image of the movement of the inexperienced person. The information processing device estimates the skeleton of the proficient person based on the first position coordinates, and estimates the skeleton of the unskilled person based on the second position coordinates. [Selection diagram] FIG. 12

Description

The present disclosure relates to information processing devices, information processing methods, and information processing programs.

Inexperienced people who want to acquire skills such as sports, traditional performing arts, and work procedures can see reference documents, videos, and actual movements of the proficient person, and within the understanding of the inexperienced person, with themselves and the proficient person. Check the difference in the operation of, and try to improve the proficiency by repeating the difference check and correction.

However, human body shapes (skeleton, muscular strength, etc.) vary, and it is not easy to confirm the difference in movement with a proficient person. In addition, depending on the type and content of the predetermined skill to be acquired, it may be difficult to find a difference from one's own movement or a correction method even by looking at the movement of a proficient person. Further, for example, there is a technique of photographing the movements of a proficient person with a plurality of cameras and analyzing the movements, but it is not easy because it requires large-scale and expensive equipment.

This application has been made in view of the above, and information processing that can assist an inexperienced person to easily analyze the difference between his / her own movement and the movement of an experienced person and obtain a method for correcting his / her own movement. It is an object of the present invention to provide an apparatus, an information processing method, and an information processing program.

One aspect of the information processing apparatus according to the present application is an estimation unit that estimates the skeleton of the proficient person from a moving image of the movement of the proficient person, and the movement and unproficiency of the proficient person from the skeleton estimation result of the proficient person. It is characterized by including an acquisition unit for acquiring the position coordinates of each part of the body of the proficient person for analyzing the difference from the movement of the person.

Further, in another aspect of the information processing apparatus according to the present application, the first motion classification for each of the plurality of proficient persons is determined by the motion patterns of the plurality of proficient persons, and the motion of one unskilled person is determined. The second motion classification for the unskilled person is determined by the pattern, and based on at least the first motion classification and the second motion classification, the unskilled person and the unskilled person are selected from among the plurality of proficient persons. The first part of the body of the first proficient person, which is obtained from the decision unit for determining the first proficient person belonging to the same classification and the first moving image of the movement of the first proficiency person. Position coordinates and the first rotation direction, and the second position coordinates and the second rotation direction of the body part of the inexperienced person obtained from the second moving image of the movement of the inexperienced person. It is characterized by having an extraction unit for extracting the difference between the two.

Further, another aspect of the information processing apparatus according to the present application is the first position coordinates and the first rotation direction of the body part of the proficient person, which is acquired from the first moving image of the movement of the proficient person. Based on the difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction obtained from the second video of the movement of the inexperienced person, the inexperienced person. It is characterized by including a generation unit for generating a correction instruction for the operation of the above and a transmission unit for transmitting the correction instruction to an inexperienced person terminal.

According to one aspect of the embodiment, it is possible to support an inexperienced person to easily analyze the difference between his / her own movement and the movement of the proficient person and obtain a method for correcting his / her movement.

FIG. 1 is a diagram showing a configuration example of an information processing system according to an embodiment. FIG. 2 is a diagram showing a functional configuration example of the information processing apparatus 100 according to the embodiment. FIG. 3 is a diagram showing an example of information stored in the expert information storage unit according to the embodiment. FIG. 4 is a diagram showing an example of information stored in the inexperienced person information storage unit according to the embodiment. FIG. 5 is a diagram showing an example of information stored in the correction instruction storage unit according to the embodiment. FIG. 6 is a diagram showing an example of information stored in the evaluation result storage unit according to the embodiment. FIG. 7 is a diagram showing an example of skeleton estimation according to the embodiment. FIG. 8 is a diagram showing an example of a correction instruction according to the embodiment. FIG. 9 is a diagram showing another example of the correction instruction according to the embodiment. FIG. 10 is a diagram showing an example of the evaluation result according to the embodiment. FIG. 11 is a diagram showing a functional configuration example of the unskilled person terminal 200 according to the embodiment. FIG. 12 is a flowchart showing the flow of the difference analysis process between the operation of the proficient person and the operation of the unskilled person according to the embodiment. FIG. 13 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device.

Hereinafter, the information processing apparatus according to the present application, the information processing method, and a mode for carrying out the information processing program (hereinafter, referred to as “the embodiment”) will be described in detail with reference to the drawings. The information processing apparatus, information processing method, and information processing program according to the present application are not limited by this embodiment. Further, in each of the following embodiments, the same parts are designated by the same reference numerals, and duplicate description is omitted. Hereinafter, the present embodiment will be described in which an inexperienced person can easily analyze the difference between his / her own movement and the movement of the proficient person and receive a correction instruction for his / her own movement.

[1. Information processing system configuration]
The configuration of the information processing system will be described with reference to FIG. FIG. 1 is a diagram showing a configuration example of an information processing system according to an embodiment. As shown in FIG. 1, the information processing system includes an information processing device 100, an inexperienced person terminal 200, and an experienced person terminal 300. As shown in FIG. 1, the information processing device 100, the unskilled person terminal 200 used by the unskilled person P, and the proficient person terminal 300-1 to 300-used by the proficient person M-1 to Mn. n (n is an arbitrary integer. Hereinafter, collectively referred to as "expert terminal 300") is connected to each other via a network N so as to be able to communicate with each other. As the network N, various communication networks such as the Internet can be adopted regardless of whether it is wired or wireless.

The unskilled person terminal 200 is a terminal used by an unskilled person who wants to acquire a skill. The unskilled person terminal 200 may be a mobile terminal such as a smartphone or a tablet PC, or may be a stationary terminal. The unskilled person terminal 200 has a camera function for photographing the movements of the unskilled person, or is connected to a camera device (not shown) to photograph the movements of the unskilled person. The shooting target of the moving image is not limited to the whole body of an inexperienced person, and may be partial such as the upper body and hands only. Further, by photographing the same motion of the unskilled person at different angles by a plurality of unskilled person terminals 200, it is possible to perform a finer difference analysis of the motion and a correction instruction. In addition, a skill acquisition application (skill acquisition application) is installed in the unskilled person terminal 200, and the user receives a correction instruction for his / her own movement and confirms the evaluation result via the skill acquisition application. be able to. The unskilled person terminal 200 does not have to be a terminal owned by an unskilled person, and may be, for example, a terminal owned by a service provider that provides a skill acquisition service.

The proficient terminal 300 is a terminal used by a proficient person of each skill that serves as a model for an unskilled person. The proficient terminal 300 may also be a mobile terminal or a stationary terminal. Further, the proficient person terminal 300 also has a camera function for photographing the movement of the proficient person, or is connected to a camera device to photograph the movement of the proficient person. Further, the moving image of the proficient person may be taken as a part of the body of the proficient person, or may be taken by a plurality of proficient person terminals 300. As shown in FIG. 1, the proficient terminal 300 according to the embodiment does not have to be for each proficient person, and may be, for example, a terminal owned by a service provider that provides a skill acquisition service. It may be the same terminal as the unskilled person terminal 200.

The information processing device 100 is, for example, a server device managed by a service provider that provides a skill acquisition service. The information processing device 100 acquires and stores operation information of various proficient persons from the proficient person terminal 300. Then, when the information processing device 100 acquires the operation information of the inexperienced person from the inexperienced person terminal 200, the information processing device 100 determines the operation information of the inexperienced person as a reference from the accumulated operation information of the inexperienced person, and determines the operation information of the inexperienced person. Perform differential analysis with operation information. Further, the information processing apparatus 100 transmits a correction instruction and an evaluation result for the operation of the unskilled person to the unskilled person terminal 200. The information processing device 100 may be a cloud server device managed by another service provider that provides the cloud service. Further, although the information processing apparatus 100 is shown as one computer in FIG. 1, it may be a distributed computing system composed of a plurality of computers.

[2. Configuration of information processing device 100]
Next, the functional configuration of the information processing apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram showing a functional configuration example of the information processing apparatus 100 according to the embodiment. As shown in FIG. 2, the information processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits / receives information to / from the unskilled person terminal 200 and the proficient person terminal 300 via the network N.

(Memory unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. As shown in FIG. 2, the storage unit 120 includes a proficient person motion information storage unit 121, an unskilled person motion information storage unit 122, an proficient person information storage unit 123, an unskilled person information storage unit 124, and a correction instruction storage unit 125. It has an evaluation result storage unit 126. Hereinafter, each storage unit included in the storage unit 120 will be described in order.

(Experienced person motion information storage unit 121)
The proficient person motion information storage unit 121 stores information about the proficient person's motion. The proficient person motion information stored in the proficient person motion information storage unit 121 is, for example, moving image data obtained by photographing the proficient person's motion. However, since the moving image data is captured by the proficient terminal 300 and transmitted (uploaded) to the information processing device 100, and because the information processing device 100 stores a large number of proficient data. The size of the data can be an issue. Therefore, the proficient person's motion information is the position coordinates (two-dimensional) and rotation direction (three-dimensional) of each part of the proficient person's body acquired by skeleton estimation of the proficient person from the moving image data of the proficient person's motion. It may be time series data. As a result, the size of the data to be handled is smaller than that of the moving image data, and the consumption of various resources can be reduced. The skeleton estimation and the acquisition of the position coordinates and the rotation direction from the moving image data of the proficient person may be performed by the proficient person terminal 300 or the information processing device 100.

(Inexperienced person motion information storage unit 122)
The unskilled person motion information storage unit 122 stores information about the unskilled person's motion. The unskilled person motion information stored in the unskilled person motion information storage unit 122 is, for example, moving image data obtained by photographing the motions of the unskilled person. However, the motion information of the unskilled person is also the time series data of the position coordinates and the rotation direction of each part of the body of the unskilled person, which is obtained from the video data of the motion of the unskilled person by the skeleton estimation of the unskilled person. It may be there. Further, the skeleton estimation from the moving image data of the unskilled person and the extraction of the position coordinates and the rotation direction may be performed by the unskilled person terminal 200 or the information processing device 100.

(Expert Information Memory Unit 123)
The proficient person information storage unit 123 stores attribute information related to the proficient person. FIG. 3 is a diagram showing an example of information stored in the expert information storage unit according to the embodiment. In the example shown in FIG. 3, the proficient person information storage unit 123 stores the proficient person ID, the proficient person name, the acquired skill, the movement classification, the gender, the age, the height, the weight, the dominant arm, the position, the school, and the like in association with each other. To do.

The "professional person ID" is an identifier uniquely indicating the proficient person. "Acquisition skill" is a skill that a proficient person has already acquired. With respect to the same acquired skills as the skills that the unskilled person wishes to acquire, the motion information of the proficient person is positive motion information and can be a standard for the unskilled person.

"Motion classification" indicates a motion classification related to human motion, for example, a classification based on a four-stance theory that classifies humans into four types according to how the body is used. How to use the human body differs depending on such movement classification. For example, even when the same movement is performed, how to move a part of the body differs depending on the movement classification. Therefore, in order to give correction instructions and evaluations of movements that are more suitable for the unskilled person, it is desirable to perform a differential analysis between the movements of the proficient person who belongs to the same movement classification as the unskilled person and the movements of the unskilled person. .. In addition, each data item after "gender" is a classification of proficient persons who can affect the movement, and these classifications are also used to determine the proficient person who performs the movement and differential analysis of the unskilled person as in the "motion classification". Can be used for. Further, even for data items other than those shown in FIG. 3, classifications that may affect the operation can be added to the expert information. In the example of FIG. 3, data such as "learning skill" is shown as a character string, but instead of the character string, a numerical value or the like that uniquely indicates the data may be used. Further, the "age" may be an "age" indicating the actual age of the proficient person.

(Inexperienced person information storage unit 124)
The unskilled person information storage unit 124 stores the attribute information about the unskilled person. FIG. 4 is a diagram showing an example of information stored in the inexperienced person information storage unit according to the embodiment. In the example shown in FIG. 4, the unskilled person information storage unit 124 sets "unskilled person ID, unskilled person name, desired skill to be acquired, motion classification, gender, age, height, weight, dominant arm, position, school" and the like. Store in association with each other.

The "unskilled person ID" is an identifier that uniquely indicates an unskilled person. "Skills desired to be acquired" are skills that an inexperienced person wishes to acquire. In addition, each data item after the "motion classification" is the same as the data item having the same name in the proficient person information (FIG. 3) stored in the proficient person information storage unit 123.

(Correction instruction storage unit 125)
The correction instruction storage unit 125 stores information regarding correction instructions for the movements of an inexperienced person. FIG. 5 is a diagram showing an example of information stored in the correction instruction storage unit according to the embodiment. In the example shown in FIG. 5, the correction instruction storage unit 125 stores "inexperienced person ID, moving image elapsed time, instruction site, instruction content" and the like in association with each other.

The "unskilled person ID" is an identifier indicating an unskilled person who receives a correction instruction, and is associated with the "unskilled person ID" stored in the unskilled person information storage unit 124. The "video elapsed time" indicates the elapsed time for which a correction instruction should be given in the video of the motion of an inexperienced person. For example, the data record in FIG. 5 shows that 1 minute and 45 seconds after the start of video recording, a correction instruction is given to the left foot of an inexperienced person, "Please turn the toes of the left foot outward". There is. The "instruction site" and "instruction content" can be generated by difference analysis between the position coordinates and rotation direction of the body part of the proficient person and the position coordinates and rotation direction of the body part of the inexperienced person. For example, the part where there is a difference by the difference analysis is a part to which a correction instruction should be given, that is, an "instructed part". Further, the "instruction content" can be generated in natural language from the "instruction part" and the difference in the position coordinates and the rotation direction and the degree thereof. By generating correction instructions in natural language, synthesizing the movements of the unskilled person with a video taken, displaying characters, and reading aloud the correction instructions, the unskilled person can understand the correction instructions more easily.

"Instructions" in natural language are, for example, "toes inward", "toes outward", "elbows bent", "elbows extended", "knees bent", "knees extended". , "Put your hips inward", "Put your hips out", "Bend your arms", "Stretch your arms", "Stretch your back", "Bend your back", "Raise your face", "Rise your face" "Lower", "turn the fingertip inward", "turn the fingertip outward", etc., which is a combination of the part to be corrected and the correct movement. In addition, you can combine speed instructions such as "faster, faster, slower, slower, do not move" (eg, turn your right elbow faster and outward). Further, the "instruction content" is not limited to the correct direction and speed for the movement of an inexperienced person, and may include an instruction regarding a distance, a trajectory, and the like.

(Evaluation result storage unit 126)
The evaluation result storage unit 126 stores information regarding the evaluation result for the movement of the inexperienced person. FIG. 6 is a diagram showing an example of information stored in the evaluation result storage unit according to the embodiment. In the example shown in FIG. 6, the evaluation result storage unit 126 is described as "inexperienced person ID, evaluation date, evaluation site 1, site 1 evaluation, site 1 indication, evaluation site 2, site 2 evaluation, site 2 instruction, evaluation site 3". , Site 3 evaluation, site 3 instruction, evaluation site 4, site 4 evaluation, site 4 instruction "and the like are stored in association with each other.

The "unskilled person ID" is an identifier indicating an unskilled person to be evaluated, and is associated with the "unskilled person ID" stored in the unskilled person information storage unit 124. The "evaluation site 1" and the "site 1 instruction" can be generated based on the "instruction site" and the "instruction content" stored in the correction instruction storage unit 125, respectively. The "part 1 evaluation" is an evaluation value relating to the movement of the part shown in the "evaluation part 1", and is, for example, a score on a 5-point scale with a maximum of 5 points. The "part 1 evaluation" can be determined, for example, from the difference between the unskilled person and the proficient person and the degree thereof with respect to the position coordinates and the rotation direction of the part shown in the "evaluation part 1". Note that FIG. 6 is just an example, and the particle size of evaluation, the method of determining each evaluation, the weighting for each evaluation, etc. (collectively referred to as "evaluation method") can be freely changed depending on the skills desired to be acquired by inexperienced persons. can do. As shown in FIG. 6, by accumulating the history of the evaluation results in the evaluation result storage unit 126, it is possible to present the proficiency level (progress) of the skill desired to be acquired to the inexperienced person as described later. it can.

(Control unit 130)
The control unit 130 is a processing unit that controls the entire information processing device 100, and is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like (so-called processor). The control unit 130 expands and executes various programs (for example, the information processing program according to the present application) stored in the storage unit 120 in a RAM serving as a work area. Further, the control unit 130 is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), for example.

As shown in FIG. 2, the control unit 130 includes an acquisition unit 131, an estimation unit 132, a determination unit 133, an extraction unit 134, a generation unit 135, and a transmission unit 136, and realizes each function or operation described below. Execute. The internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 2, and may be any other configuration as long as it executes a differential analysis process between the operation of an experienced person and the operation of an inexperienced person, which will be described later. May be good. Further, the connection relationship of each processing unit included in the control unit 130 is not limited to the connection relationship shown in FIG. 2, and may be another connection relationship.

(Acquisition unit 131)
The acquisition unit 131 acquires moving image data of the unskilled person and the movement of the proficient person from the unskilled person terminal 200 and the proficient person terminal 300. In addition, the acquisition unit 131 acquires the time-series data of the position coordinates and the rotation direction of each part of the body of the unskilled person or the proficient person, which is acquired from the moving image data, from the unskilled person terminal 200 or the proficient person terminal 300. To do. When the time-series data is acquired from the unskilled person terminal 200 or the proficient person terminal 300, it is not necessary to acquire the moving image data of the movements of the unskilled person or the proficient person. In this case, the moving image data is not uploaded to the information processing device 100. Further, the acquisition unit 131 may acquire the time-series data from the moving image data based on the skeleton estimation result of the unskilled person or the proficient person, which will be described later, without acquiring the time series data from the unskilled person terminal 200 or the proficient person terminal 300. Good. The various acquired information is stored in the storage unit 120 by the acquisition unit 131.

(Estimation unit 132)
The estimation unit 132 identifies each part of the body of the unskilled person or the proficient person by image recognition and image processing from the moving image of the unskilled person or the proficient person, and estimates the skeleton. FIG. 7 is a diagram showing an example of skeleton estimation according to the embodiment. As shown in FIG. 7, the skeleton (line) can be estimated by connecting the positions (points) of each part of the body. As a result, the position coordinates of each part of the body and the ratio between each part can be known (the rotation direction of each part can also be known by the transition of the position coordinates), and the relative difference analysis of the motion information of the inexperienced person and the proficient person. Even if there is a physical disparity between the inexperienced person and the proficient person, appropriate correction instructions and evaluations can be given.

(Decision unit 133)
The determination unit 133 determines the movement classification of the unskilled person or the proficient person according to the movement pattern of the unskilled person or the proficient person. The motion classification is, for example, a classification based on the 4-stance theory. Classification based on the four-stance theory can classify humans into four types according to human movement patterns. Such an action classification can be determined, for example, by asking the user some questions in the form of a questionnaire and having the user answer the action pattern. The operation pattern can be answered (user input) using a skill acquisition application or other tools. Further, the determination unit 133 determines a proficient person who belongs to the same classification as the unskilled person from among a plurality of proficient persons based on the motion classification of the unskilled person and the proficient person. If there are a plurality of proficient persons belonging to the same classification as the unskilled person, a list of proficient persons belonging to the same classification as the unskilled person is transmitted to the unskilled person terminal 200, and one person is sent to the unskilled person terminal 200. You can also let the proficient person choose. In addition, this list of proficient persons is close to the inexperienced person by using classifications that can affect movements such as gender, age, and dominant arm, as shown in the proficient person information and the inexperienced person information in FIGS. 3 and 4. The proficient person can be generated to be displayed preferentially at the top of the list, for example.

(Extractor 134)
The extraction unit 134 extracts the difference between the position coordinates and the rotation direction of the body part of the proficient person and the position coordinates and the rotation direction of the body part of the unskilled person. In addition, the relative difference is extracted so as not to be affected by the physical disparity between the unskilled person and the proficient person. Specifically, since the ratio between each part of the body can be known from the skeleton estimation by the estimation unit 132, the difference in the relative positions of the unskilled person and the proficient person is extracted. Further, the body part of the proficient person or the unskilled person to be extracted may be a part designated in advance for each skill. Depending on the skill, there are parts that are not particularly related to the movement, and in order to avoid unnecessary execution of processing, only the parts specified in advance may be extracted.

(Generator 135)
The generation unit 135 generates a correction instruction for the movement of the unskilled person based on the difference between the position coordinate and the rotation direction of the body part of the proficient person and the position coordinate and the rotation direction of the body part of the unskilled person. .. Specifically, the generation unit 135 uses the motion information (position coordinates and rotation direction) of the expert as positive motion information, and makes the motion information of the inexperienced person with a relative difference closer to the positive motion information. Generate correction instructions. In addition, the correction instruction is the correct movement direction for each part, such as "turn the right elbow to the outside faster", depending on the difference in the position coordinates and rotation direction between the proficient person and the unskilled person and the degree thereof. Generate in natural language by combining speeds.

Further, the correction instruction may be a visual instruction. Specifically, it is an arrow that is combined with a video of the movement of an inexperienced person and displayed, and a silhouette of an inexperienced person. The arrow is generated, for example, to visually indicate the correct direction and speed for the movement of the proficient person, starting from a part of the body of the unskilled person. For example, the direction of the arrow is the direction of the correct movement, the length of the arrow is the distance to the correct movement, the color and pattern of the arrow, and the like can indicate the speed of the correct movement. In addition, the trajectory of the arrow itself can be shown as the trajectory of the correct operation. Since the silhouette of the proficient person can know the relative positional relationship of each part of the proficient person and the unskilled person by skeletal estimation, the unskilled person can be corrected by correcting the positional relationship of each part of the proficient person according to the unskilled person. It is possible to generate a silhouette of a proficient person according to the body shape of the person. By synthesizing and displaying such arrows and silhouettes of the proficient person on a frame-by-frame basis with a video of the movement of the unskilled person, the unskilled person can more easily understand the correction instruction.

FIG. 8 is a diagram showing an example of a correction instruction according to the embodiment. As shown in FIG. 8, the unskilled person terminal 200 can output a correction instruction by voice or display according to the operation of the unskilled person during shooting. It should be noted that the correction instruction can be output not only in real time but also for, for example, a video that has already been shot.

Further, FIG. 9 is a diagram showing another example of the correction instruction according to the embodiment. As shown in FIG. 8, the silhouette of the proficient person can be displayed according to the movement of the unskilled person. The silhouette of the proficient person does not have to be the silhouette of the proficient person as shown in FIG. 9, but may be each part and skeleton represented by points and lines as shown in FIG. 7. Further, although FIG. 9 has been described as another example of the correction instruction, it can be output together with the correction instruction as shown in FIG.

In addition, an inexperienced person may practice at a slower speed than the movement of an inexperienced person. Therefore, the differential analysis of the movement and the generation of the correction instruction are performed after adjusting the speeds of the movements of the unskilled person and the proficient person. In addition, machine learning can be performed by feedback on the correction instruction to improve the correction instruction. The feedback referred to here is, for example, user input via a skill acquisition application such as "I understand the correction instruction" or "I do not understand" of an inexperienced person with respect to the correction instruction. Alternatively, it is the degree of improvement in motion obtained by further differential analysis of motion (for example, the evaluation result (FIG. 6) generated by the generation unit 135), indicating whether the motion of the inexperienced person is actually improved in response to the correction instruction. May be).

In addition, the generation unit 135 is based on the difference between the position coordinates and the rotation direction of the body parts of the proficient person and the position coordinates and the rotation direction of the body parts of the unskilled person, for each part of the body of the unskilled person. Generate evaluation results. In addition, as an evaluation result, each part is scored based on the difference between the inexperienced person and the proficient person with respect to the position coordinates and the rotation direction of each part of the body and the degree thereof (for example, a 5-step evaluation with 5 points as the highest point). )can do. In addition, by accumulating the evaluation results each time, it can be presented to an inexperienced person as historical data.

FIG. 10 is a diagram showing an example of the evaluation result according to the embodiment. The generation unit 135 can generate such an evaluation result based on the evaluation result (FIG. 6) stored in the evaluation result storage unit 126. In the example of FIG. 10, a graph is displayed in the upper left, with the total score, which is the total score (evaluation value) for the movement of each part of the inexperienced person (user), as the vertical axis and the time (date) as the horizontal axis. ing. For example, if the data value (line) portion on this graph is selected by tapping or the like, the evaluation result corresponding to that date is displayed (the example in FIG. 10 shows the evaluation result on October 14, 2019). ). At the bottom left of FIG. 10, the correction instructions shown on October 14, 2019 are listed. Further, the "compared teacher" shown in FIG. 10 is a proficient person who has been differentially analyzed with the movement of an unskilled person. Although not shown, it is also possible to change the "compared teacher" to another proficient person and display the evaluation result with the changed proficient person. Further, in the lower right of FIG. 10, the scores for the movements of each part on October 14, 2019 are listed. The "evaluation speed" shown at the bottom of this list is, for example, 5 points, which is the highest point when the movement of an inexperienced person is the same speed as the movement of an experienced person, which is faster than the movement of an experienced person. If you drop, you can give a score of 4 points or less depending on the degree.

(Transmitting unit 136)
The transmission unit 136 transmits the correction instruction and the evaluation result generated by the generation unit 135 to the unskilled person terminal 200. Further, when there are a plurality of proficient persons belonging to the same classification as the unskilled person determined by the determination unit 133, the transmitting unit 136 displays a list of proficient persons belonging to the same classification as the unskilled person as the unskilled person terminal. It can be transmitted to 200.

[3. Configuration of unskilled person terminal 200]
Next, the functional configuration of the unskilled person terminal 200 according to the embodiment will be described with reference to FIG. FIG. 11 is a diagram showing a functional configuration example of the unskilled person terminal 200 according to the embodiment. As shown in FIG. 11, the unskilled person terminal 200 includes a communication unit 210, a storage unit 220, and a control unit 230. The unskilled person terminal 200 is an input device (for example, a keyboard, a mouse, a touch panel) that receives various operations from an unskilled person who uses the unskilled person terminal 200, and an output device (for example, an output device) for displaying various information. , Liquid crystal display or touch panel).

(Communication unit 210)
Like the communication unit 110 of the information processing device 100, the communication unit 210 transmits / receives information to / from the information processing device 100 via the network N.

(Storage 220)
The storage unit 220 is realized by a RAM or the like, like the storage unit 120 of the information processing device 100. As shown in FIG. 11, the storage unit 220 includes a skill acquisition application storage unit 221, an inexperienced person motion information storage unit 222, a correction instruction storage unit 223, and an evaluation result storage unit 224. Hereinafter, each storage unit included in the storage unit 220 will be described in order.

(Skill acquisition application memory section 221)
The skill acquisition application storage unit 221 stores a skill acquisition application for an inexperienced person to receive a correction instruction and an evaluation result for his / her own movement. The skill acquisition application is stored in the skill acquisition application storage unit 221 by being downloaded in advance from the information processing device 100 or the like and installed in the inexperienced person terminal 200.

(Inexperienced person motion information storage unit 222)
The unskilled person motion information storage unit 222 stores information about the motion of the unskilled person. The information stored in the unskilled person motion information storage unit 222 is extracted from the moving image data obtained by capturing the motion of the unskilled person and the moving image data, as in the case of the unskilled person motion information storage unit 122 of the information processing apparatus 100. It is the time series data of the position coordinates and the rotation direction of each part of the body of an inexperienced person. Whether or not the time-series data is stored depends on whether the time-series data is extracted by the inexperienced terminal 200 or the information processing apparatus 100.

(Correction instruction storage unit 223)
The correction instruction storage unit 223 stores information regarding correction instructions for the movement of an inexperienced person. The information stored in the correction instruction storage unit 223 is the same as that of the correction instruction storage unit 125 of the information processing apparatus 100.

(Evaluation result storage unit 224)
The evaluation result storage unit 224 stores information regarding the evaluation result for the movement of the inexperienced person. The information stored in the evaluation result storage unit 224 is the same as that of the evaluation result storage unit 126 of the information processing apparatus 100.

(Control unit 230)
The control unit 230 is a processing unit that controls the entire unskilled person terminal 200, like the control unit 130 of the information processing device 100. As shown in FIG. 11, the control unit 230 includes a photographing unit 231, an estimation unit 232, an acquisition unit 233, a transmission unit 234, an output unit 235, and an input unit 236. The internal configuration of the control unit 230 is not limited to the configuration and connection relationship shown in FIG.

(Photographing unit 231)
The photographing unit 231 photographs the motion of the unskilled person, and stores the photographed moving image data in the unskilled person motion information storage unit 222. Although the photographing unit 231 is a camera function built into the inexperienced person terminal 200, an externally connected camera device (not shown) may be used for photographing. In this case, the control unit 230 does not have to have the photographing unit 231.

(Estimating unit 232)
Similar to the estimation unit 132 of the information processing device 100, the estimation unit 232 identifies each part of the body of the inexperienced person by image recognition and image processing from the moving image of the movement of the inexperienced person, and the skeleton of the inexperienced person. To estimate. When the skeleton estimation of an inexperienced person is performed by the information processing device 100, the control unit 230 does not have to have the estimation unit 232.

(Acquisition unit 233)
Based on the skeleton estimation result of the unskilled person, the acquisition unit 233 acquires the position coordinates of each part of the body of the unskilled person and the time series data in the rotation direction from the moving image data of the motion of the unskilled person. When the information processing apparatus 100 extracts the time-series data, the control unit 230 does not have to have the acquisition unit 233. In addition, the acquisition unit 233 acquires a correction instruction and an evaluation result for the operation of an inexperienced person from the information processing device 100. The various acquired information is stored in the storage unit 220 by the acquisition unit 233.

(Transmission unit 234)
The transmission unit 234 transmits the moving image data captured by the photographing unit 231 to the information processing device 100. When the time-series data of the position coordinates and the rotation direction of each part of the body of an inexperienced person is extracted from the moving image data by the acquisition unit 233, the time-series data is transmitted to the information processing device 100 instead of the moving image data.

(Output unit 235)
The output unit 235 displays the user interface of the skill acquisition application. As a result, the user can upload the operation information to the information processing device 100 via the skill acquisition application, receive a correction instruction for his / her operation, and confirm the evaluation result. The correction instruction output by the output unit 235 is a character display or voice output in natural language, an arrow as described above, a silhouette of a proficient person, or the like. It should be noted that these correction instructions may be combined and output in frame units with a moving image of an inexperienced person's movement. In addition, the output unit 235 can also output correction instructions and evaluation results via an external device (for example, a digital signage or a projector) connected to the inexperienced terminal 200.

(Input unit 236)
The input unit 236 receives, for example, an answer (user input) of an operation pattern to a questionnaire for determining an operation classification of an inexperienced person, such as a classification based on a 4-stance theory, via a skill acquisition application.

[4. Configuration of expert terminal 300]
Since the proficient terminal 300 is also the same mobile terminal as the unskilled terminal 200, the configuration of the proficient terminal 300 is the same as the configuration of the unskilled terminal 200. The proficient terminal 300 does not give a correction instruction to the operation or confirm the evaluation result, but uploads the proficient person's operation information to the information processing device 100 and determines the proficient person's operation classification via the skill acquisition application. It has a configuration for user input of the operation pattern of.

[5. Processing procedure]
Next, with reference to FIG. 12, the procedure of the difference analysis processing between the operation of the proficient person and the operation of the inexperienced person according to the embodiment will be described. FIG. 12 is a flowchart showing the flow of the difference analysis process between the operation of the proficient person and the operation of the unskilled person according to the embodiment. This process analyzes the difference between the motion information of the unskilled person and the motion information of the proficient person who belongs to the same classification as the unskilled person, and gives an instruction to correct the motion to the unskilled person.

As shown in FIG. 12, the proficient person photographs the movement of the proficient person via the proficient person terminal 300 (step S101). The motion is filmed as a video. When transmitting only the position coordinates of each part of the proficient person's body without transmitting the moving image data to the information processing apparatus 100, after step S101, the proficient person terminal 300 sets the skeleton of the proficient person's body from the moving image data. To get the time series data of the position coordinates and the rotation direction.

Next, the expert terminal 300 transmits the operation information to the information processing device 100 (step S102). The motion information referred to here is moving image data of the motion of the proficient person, or time-series data of the position coordinates and the rotation direction of each part of the body of the proficient person.

Next, the acquisition unit 131 of the information processing device 100 determines whether or not the operation information has been received from the expert terminal 300 (step S103). If it is determined that the operation information has not been received (step S103: No), the acquisition unit 131 waits for the reception of the operation information. When it is determined that the operation information has been received (step S103: Yes), the acquisition unit 131 acquires the operation information via the communication unit 110.

Next, the estimation unit 132 estimates the skeleton of the proficient person's body from the motion information of the proficient person, and acquires the time-series data of the position coordinates and the rotation direction of each part of the proficient person's body (step S104). When the position coordinates of each part of the body of the proficient person and the time series data in the rotation direction are acquired from the information processing apparatus 100 as operation information in step S103, it is not necessary to execute step S104. Steps S101 to S104 are processes related to the accumulation of operation information of proficient persons, and this process is performed in advance for the number of proficient persons.

Next, the unskilled person who wants to acquire the skill takes a picture of the movement of the unskilled person via the photographing unit 231 of the unskilled person terminal 200 (step S105). Similar to step S101, when the inexperienced person terminal 200 acquires the time series data of the position coordinates and the rotation direction of each part of the inexperienced person's body, after step S105, the acquisition unit 233 of the inexperienced person terminal 200 receives a moving image. The skeleton of the body of an inexperienced person is estimated from the data, and the time series data of the position coordinates and the rotation direction are acquired.

Next, the transmission unit 234 of the unskilled person terminal 200 transmits the operation information to the information processing device 100 (step S106). The motion information referred to here is moving image data of the motion of an unskilled person, or time-series data of the position coordinates of each part of the body of the unskilled person and the rotation direction.

Next, the acquisition unit 131 of the information processing device 100 determines whether or not the operation information has been received from the unskilled person terminal 200 (step S107). If it is determined that the operation information has not been received (step S107: No), the acquisition unit 131 waits for the reception of the operation information. When it is determined that the operation information has been received (step S107: Yes), the acquisition unit 131 acquires the operation information via the communication unit 110.

Next, the estimation unit 132 estimates the skeleton of the body of the inexperienced person from the motion information of the inexperienced person, and acquires the position coordinates of each part of the body of the inexperienced person and the time series data in the rotation direction (step). S108). If the position coordinates of each part of the body of an inexperienced person and the time series data in the rotation direction are acquired from the information processing apparatus 100 as operation information in step S107, it is not necessary to execute step S108.

Next, the determination unit 133 of the information processing apparatus 100 is a reference proficient person who belongs to the same classification as the unskilled person from among a plurality of proficient persons based on the motion classification of the unskilled person and the proficient person. Is determined (step S109). When there are a plurality of proficient persons belonging to the same classification as the unskilled person, the transmission unit 136 of the information processing apparatus 100 transmits a list of proficient persons belonging to the same classification as the unskilled person to the unskilled person terminal 200. However, an inexperienced person may be allowed to select one proficient person. The determination unit 133 determines the selected proficient person as a reference proficient person. Alternatively, the determination unit 133 determines a proficient person whose classification that can affect the movement, such as gender, age, and dominant arm, as shown in the proficient person information and the unskilled person information in FIGS. 3 and 4, is close to that of the unskilled person. It can also be decided as a standard proficient person.

Next, the extraction unit 134 of the information processing apparatus 100 is relative to the position coordinates and the rotation direction of the body part of the proficient person determined in step S109 and the position coordinates and the rotation direction of the body part of the inexperienced person. The difference is extracted (step S110).

Next, the generation unit 135 of the information processing apparatus 100 generates a correction instruction for the operation of the inexperienced person based on the relative difference extracted in step S110 (step S111).

Next, the transmission unit 136 of the information processing apparatus 100 transmits the correction instruction generated in step S111 to the unskilled person terminal 200 (step S112).

Next, the acquisition unit 233 of the unskilled person terminal 200 determines whether or not the correction instruction has been received from the information processing device 100 (step S113). If it is determined that the correction instruction has not been received (step S113: No), the acquisition unit 233 waits for the correction instruction to be received. When it is determined that the correction instruction has been received (step S113: Yes), the acquisition unit 233 acquires the correction instruction via the communication unit 210.

Next, the output unit 235 of the unskilled person terminal 200 outputs the correction instruction acquired in step S113 (step S114). After step S114, the difference analysis process between the movement of the proficient person and the movement of the unskilled person shown in FIG. 12 is completed.

[6. effect〕
As described above, the information processing apparatus 100 according to the embodiment includes an acquisition unit 131 and an estimation unit 132. The estimation unit 132 estimates the skeleton of the proficient person from the moving image of the movement of the proficient person. The acquisition unit 131 acquires the position coordinates of each part of the body of the proficient person to analyze the difference between the movement of the proficient person and the movement of the unskilled person from the skeleton estimation result of the proficient person.

In this way, the information processing device 100 according to the embodiment estimates the skeleton of the body of the proficient person and acquires the position coordinates of each part of the body of the proficient person so that the unskilled person can perform his / her own movement. It is possible to easily analyze the difference from the movement of an inexperienced person and help to obtain a method for correcting one's own movement.

Further, the information processing apparatus 100 according to the embodiment has a determination unit 133 and an extraction unit 134. The determination unit 133 determines the first motion classification for each of the plurality of proficient persons based on the motion patterns of the plurality of proficient persons. In addition, the determination unit 133 determines the second motion classification for the unskilled person based on the motion pattern of one unskilled person. Further, the determination unit 133 determines a first proficient person who belongs to the same classification as the unskilled person from among a plurality of proficient persons based on at least the first motion classification and the second motion classification. The extraction unit 134 includes the first position coordinates and the first rotation direction of the body part of the first proficient person acquired from the first moving image of the movement of the first proficient person, and the unskilled person. The difference between the second position coordinates and the second rotation direction of the body part of the inexperienced person, which is obtained from the second moving image of the motion of the inexperienced person, is extracted.

As described above, the information processing apparatus 100 according to the embodiment analyzes the difference between the movement of the proficient person who belongs to the same classification as the unskilled person and the movement of the unskilled person, so that the operation is more suitable for the unskilled person. You can easily give correction instructions and evaluations.

Further, the extraction of the difference by the extraction unit 134 of the information processing apparatus 100 according to the embodiment is a relative difference between the first position coordinates and the first rotation direction and the second position coordinates and the second rotation direction. Includes extracting.

As described above, the information processing apparatus 100 according to the embodiment extracts the relative difference between the position coordinates and the rotation direction of the body part of the proficient person and the position coordinates and the rotation direction of the body part of the unskilled person. Therefore, even if there is a difference in body size between the unskilled person and the proficient person, it is possible to analyze the difference between the movement of the proficient person and the movement of the unskilled person.

Further, the body part of the first proficient person and the body part of the unskilled person whose difference is extracted by the extraction unit 134 of the information processing apparatus 100 according to the embodiment are predetermined parts.

As described above, the information processing apparatus 100 according to the embodiment is not particularly related to the movement depending on the skill by performing the differential analysis of the movement on the body parts of the proficient person and the unskilled person designated in advance. It is possible to avoid performing unnecessary processing on the part.

Further, the first motion classification and the second motion classification determined by the determination unit 133 of the information processing apparatus 100 according to the embodiment are classifications based on the 4-stance theory.

As described above, the information processing apparatus 100 according to the embodiment analyzes the difference between the movement of the proficient person who belongs to the same classification as the unskilled person based on the 4-stance theory and the movement of the unskilled person, thereby being unskilled. It is possible to give instructions and evaluations for correction of actions that are more suitable for the person.

Further, the determination of the first proficient person by the determination unit 133 of the information processing apparatus 100 according to the embodiment is an operation of extracting the sex, age or age, dominant arm, body shape, and difference of a plurality of proficient and unskilled persons. It is further determined based on the sport, traditional performing arts, or type of work, as well as the position of the sport or at least one of the sporting or traditional performing arts schools.

As described above, the information processing apparatus 100 according to the embodiment analyzes the difference between the movement of the proficient person whose classification can affect the movement is close to that of the unskilled person and the movement of the unskilled person. It is possible to give a correction instruction and an evaluation of a more suitable operation.

In addition, the information processing device 100 according to the embodiment further includes a transmission unit 136. When there are a plurality of first proficient persons belonging to the same classification as the unskilled person, the transmission unit 136 transmits a list of the first proficient persons belonging to the same classification as the unskilled person to the unskilled person terminal. .. Further, the acquisition unit 131 of the information processing apparatus 100 acquires one first proficient person selected from the list from the unskilled person terminal.

As described above, in the information processing apparatus 100 according to the embodiment, when there are a plurality of proficient persons belonging to the same classification as the unskilled person, the unskilled person is allowed to select one proficient person by the unskilled person. It is possible to give correction instructions and evaluations for suitable operations.

Further, the information processing apparatus 100 according to the embodiment further includes a generation unit 135. The generation unit 135 captures the first position coordinates and the first rotation direction of the body part of the proficient person and the movement of the unskilled person, which are acquired from the first moving image of the movement of the proficient person. Based on the difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction obtained from the moving image of 2, the correction instruction for the movement of the inexperienced person is generated. Further, the transmission unit 136 of the information processing apparatus 100 transmits a correction instruction to an inexperienced person terminal.

As described above, the information processing apparatus 100 according to the embodiment generates a correction instruction based on the difference between the operation of the proficient person and the operation of the unskilled person, and transmits the correction instruction to the unskilled person terminal to the unskilled person. You can easily provide instructions to correct your own movements.

Further, the correction instruction generated by the generation unit 135 of the information processing apparatus 100 according to the embodiment includes an instruction regarding at least one of the correct direction, speed, distance, and locus with respect to the movement of the inexperienced person.

As described above, the information processing apparatus 100 according to the embodiment generates the correction instruction according to the correct direction, speed, distance, and locus with respect to the movement of the inexperienced person, so that the inexperienced person can more easily understand the correction instruction. ..

Further, the correction instruction generated by the generation unit 135 of the information processing apparatus 100 according to the embodiment is generated for each part of the body of an inexperienced person.

As described above, the information processing apparatus 100 according to the embodiment generates the correction instruction for each part of the body of the inexperienced person, so that the inexperienced person can more easily understand the correction instruction.

Further, the generation unit 135 of the information processing apparatus 100 according to the embodiment further synthesizes the correction instruction and the second moving image in frame units of the second moving image to generate the third moving image.

As described above, the information processing apparatus 100 according to the embodiment synthesizes the correction instruction in frame units with the moving image of the motion of the inexperienced person, so that the inexperienced person can more easily understand the correction instruction.

Further, the correction instruction generated by the generation unit 135 of the information processing apparatus 100 according to the embodiment is an instruction in natural language.

As described above, the information processing apparatus 100 according to the embodiment generates the correction instruction in natural language, so that the inexperienced person can more easily understand the correction instruction.

Further, the correction instruction generated by the generation unit 135 of the information processing apparatus 100 according to the embodiment is a visual instruction.

As described above, the information processing apparatus 100 according to the embodiment generates a visual correction instruction, so that an inexperienced person can more easily understand the correction instruction.

Further, the generation unit 135 of the information processing apparatus 100 according to the embodiment is further inexperienced based on the difference between the first position coordinates and the first rotation direction and the second position coordinates and the second rotation direction. Generate evaluation results for each part of the person's body. Further, the transmission unit 136 of the information processing apparatus 100 further transmits the evaluation result to the inexperienced person terminal.

As described above, the information processing apparatus 100 according to the embodiment generates an evaluation result based on the difference between the operation of the proficient person and the operation of the unskilled person, and transmits the evaluation result to the unskilled person terminal to the unskilled person. You can easily provide the evaluation result of your own movement.

[7. Hardware configuration]
The information processing device 100, the unskilled person terminal 200, and the proficient person terminal 300 described above are realized by, for example, a computer 1000 having a configuration as shown in FIG. FIG. 13 is a hardware configuration diagram showing an example of a computer that realizes the functions of each device. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400, and controls each part. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, a program that depends on the hardware of the computer 1000, and the like.

The HDD 1400 stores a program executed by the CPU 1100, data used by the program, and the like. The communication interface 1500 receives data from another device via the network N and sends it to the CPU 1100, and transmits the data collected by the CPU 1100 to the other device via the network N.

The CPU 1100 controls an output device such as a display or a printer, and an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the collected data to the output device via the input / output interface 1600.

The media interface 1700 reads a program or data stored in the recording medium 1800 and provides the program or data to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is an optical recording medium such as a DVD (Digital Versaille Disc) or PD (Phase change rewritable disc), a magneto-optical recording medium such as an MO (Magnet-Optical disc), a tape medium, a magnetic recording medium, or a semiconductor memory. And so on.

For example, when the computer 1000 functions as an information processing device 100 or an inexperienced terminal 200, the CPU 1100 of the computer 1000 realizes the functions of the control units 130 and 230 by executing a program loaded on the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, these programs may be acquired from another device via the network N.

The embodiments of the present application have been described in detail with reference to the drawings, but these are examples, and various modifications and improvements are made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure line of the invention. It is possible to carry out the present invention in other forms described above.

[8. Others]
Further, among the processes described in the above-described embodiment, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or part of it can be done automatically by a known method. In addition, the processing procedure, specific name, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of them may be functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

Further, the above-described embodiments can be appropriately combined as long as the processing contents do not contradict each other.

Further, the above-mentioned "section (module, module, unit)" can be read as "means", "circuit" and the like. For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

100 Information processing device 110 Communication unit 120 Storage unit 121 Expert operation information storage unit 122 Inexperienced person operation information storage unit 123 Expert information storage unit 124 Inexperienced person information storage unit 125 Correction instruction storage unit 126 Evaluation result storage unit 130 Control Part 131 Acquisition part 132 Estimate part 133 Decision part 134 Extraction part 135 Generation part 136 Transmission part 200 Inexperienced person terminal 210 Communication part 220 Storage part 221 Skill acquisition application storage part 222 Skill acquisition application storage part 222 Inexperienced person operation information storage part 223 Correction instruction storage part 224 Evaluation result storage unit 230 Control unit 231 Imaging unit 232 Estimating unit 233 Acquisition unit 234 Transmission unit 235 Output unit 236 Input unit 300 Expert terminal N network

Claims (26)

An estimation unit that estimates the skeleton of the proficient person from a video of the movement of the proficient person,
From the skeleton estimation result of the proficient person, the position coordinates of each part of the body of the proficient person for analyzing the relative difference between the movement of the proficient person and the movement of the unskilled person, and between the parts. An information processing device characterized by having an acquisition unit for acquiring a ratio. The first motion classification for each of the plurality of proficients is determined based on the motion patterns of the plurality of proficients.
The second motion classification for the unskilled person is determined by the motion pattern of one unskilled person.
A determination unit for determining a first proficient person who belongs to the same classification as the unskilled person from among the plurality of proficient persons, based on at least the first motion classification and the second motion classification.
The first position coordinates and the first rotation direction of the body part of the first proficient person, and the movement of the unskilled person, obtained from the first moving image of the movement of the first proficient person. Information characterized by having an extraction unit for extracting the difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction acquired from the second moving image of the photographed person. Processing equipment. The extraction of the difference by the extraction unit includes extracting the relative difference between the first position coordinates and the first rotation direction and the second position coordinates and the second rotation direction. The information processing apparatus according to claim 2, wherein the information processing device is characterized by the above. The third aspect of claim 3, wherein the body part of the first proficient person and the body part of the unskilled person from which the difference is extracted by the extraction unit are predetermined parts. Information processing device. The information processing apparatus according to any one of claims 2 to 4, wherein the first motion classification and the second motion classification determined by the determination unit are classifications based on motion theory. .. Any of claims 2 to 5, wherein the determination of the first proficient person by the determination unit is further determined based on the attribute information of the plurality of proficient persons and the attribute information of the unskilled person. The information processing device according to item 1. When there are a plurality of the first proficient persons belonging to the same classification as the unskilled person, the list of the first proficient persons belonging to the same classification as the unskilled person is transmitted to the unskilled person terminal. Department and
Further including an acquisition unit for acquiring one said first proficient person selected from the list from the inexperienced person terminal, the extraction of the difference by the extraction unit is performed by the selected first proficient person. It is characterized by including extracting the difference between the first position coordinates and the first rotation direction of the body part of the body part and the second position coordinates and the second rotation direction of the body part of the inexperienced person. The information processing apparatus according to any one of claims 2 to 6. From the first position coordinates and the first rotation direction of the body part of the proficient person and the second video of the movement of the unskilled person acquired from the first video of the movement of the proficient person. Based on the acquired difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction , the movement speeds of the inexperienced person and the experienced person are adjusted to match. A generation unit that generates a correction instruction for the operation of the inexperienced person, and
An information processing device including a transmission unit that transmits the correction instruction to an inexperienced terminal. The information processing apparatus according to claim 8, wherein the correction instruction generated by the generation unit includes an instruction regarding at least one of a correct direction, a speed, a distance, and a locus with respect to the movement of the inexperienced person. .. The information processing apparatus according to claim 8 or 9, wherein the correction instruction generated by the generation unit is generated for each part of the body of the inexperienced person. Any of claims 8 to 10, wherein the generation unit further synthesizes the correction instruction and the second moving image in frame units of the second moving image to generate a third moving image. The information processing device according to paragraph 1. The information processing apparatus according to any one of claims 8 to 11, wherein the correction instruction generated by the generation unit is an instruction in natural language. The information processing apparatus according to any one of claims 8 to 12, wherein the correction instruction generated by the generation unit is a visual instruction. The generation unit is further based on the difference between the first position coordinates and the first rotation direction and the second position coordinates and the second rotation direction for each part of the body of the inexperienced person. Generates the evaluation result of
The information processing device according to any one of claims 8 to 13, wherein the transmission unit further transmits the evaluation result to the inexperienced person terminal. The skeleton of the proficient person is estimated from the video of the proficient person's movement, and the skeleton of the proficient person is estimated.
From the skeleton estimation result of the proficient person, the position coordinates of each part of the body of the proficient person for analyzing the relative difference between the movement of the proficient person and the movement of the unskilled person, and between the parts. An information processing method characterized by executing a process of acquiring a ratio. Information processing device
The first motion classification for each of the plurality of proficients is determined based on the motion patterns of the plurality of proficients.
The second motion classification for the unskilled person is determined by the motion pattern of one unskilled person.
Based on at least the first motion classification and the second motion classification, a first proficient person belonging to the same classification as the unskilled person is determined from the plurality of proficient persons.
The first position coordinates and the first rotation direction of the body part of the first proficient person acquired from the first moving image of the movement of the first proficient person, and the movement of the unskilled person. An information processing method characterized by executing a process of extracting the difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction acquired from the second moving image of the photographed person. .. Information processing device
From the first position coordinates and the first rotation direction of the body part of the proficient person and the second video of the movement of the unskilled person acquired from the first video of the movement of the proficient person. Based on the acquired difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction , the movement speeds of the inexperienced person and the experienced person are adjusted to match. Generate a correction instruction for the movement of the inexperienced person,
An information processing method characterized by executing a process of transmitting the correction instruction to an inexperienced person terminal. For information processing equipment
The skeleton of the proficient person is estimated from the video of the proficient person's movement, and the skeleton of the proficient person is estimated.
From the skeleton estimation result of the proficient person, the position coordinates of each part of the body of the proficient person for analyzing the relative difference between the movement of the proficient person and the movement of the unskilled person, and between the parts. An information processing program characterized by executing a process of acquiring a ratio. For information processing equipment
The first motion classification for each of the plurality of proficients is determined based on the motion patterns of the plurality of proficients.
The second motion classification for the unskilled person is determined by the motion pattern of one unskilled person.
Based on at least the first motion classification and the second motion classification, a first proficient person belonging to the same classification as the unskilled person is determined from the plurality of proficient persons.
The first position coordinates and the first rotation direction of the body part of the first proficient person, and the movement of the unskilled person, obtained from the first moving image of the movement of the first proficient person. An information processing program characterized by executing a process of extracting the difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction acquired from the second moving image of the inexperienced person. .. For information processing equipment
From the first position coordinates and the first rotation direction of the body part of the proficient person and the second video of the movement of the unskilled person acquired from the first video of the movement of the proficient person. Based on the acquired difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction , the movement speeds of the inexperienced person and the experienced person are adjusted to match. Generate a correction instruction for the movement of the inexperienced person,
An information processing program characterized by executing a process of transmitting the correction instruction to an inexperienced terminal. The first motion classification for each of the plurality of proficients is determined based on the motion patterns of the plurality of proficients.
The second motion classification for the unskilled person is determined by the motion pattern of one unskilled person.
A determination unit for determining a first proficient person who belongs to the same classification as the unskilled person from among the plurality of proficient persons, based on at least the first motion classification and the second motion classification.
The first position information of the body part of the first proficient person obtained from the first moving image of the movement of the first proficient person and the second movement of the unskilled person are photographed. With an extraction unit that extracts the difference from the second position information of the body part of the inexperienced person acquired from the video.
An information processing device characterized by being equipped with. Information processing device
The first motion classification for each of the plurality of proficients is determined based on the motion patterns of the plurality of proficients.
The second motion classification for the unskilled person is determined by the motion pattern of one unskilled person.
Based on at least the first motion classification and the second motion classification, a first proficient person belonging to the same classification as the unskilled person is determined from the plurality of proficient persons.
The first position information of the body part of the first proficient person acquired from the first moving image of the movement of the first proficient person, and the second position information of the body part of the first proficient person and the second movement of the unskilled person. Extract the difference from the second position information of the body part of the inexperienced person acquired from the video.
An information processing method characterized by executing processing. For information processing equipment
The first motion classification for each of the plurality of proficients is determined based on the motion patterns of the plurality of proficients.
The second motion classification for the unskilled person is determined by the motion pattern of one unskilled person.
Based on at least the first motion classification and the second motion classification, a first proficient person belonging to the same classification as the unskilled person is determined from the plurality of proficient persons.
The first position information of the body part of the first proficient person acquired from the first moving image of the movement of the first proficient person, and the second position information of the body part of the first proficient person and the second movement of the unskilled person. Extract the difference from the second position information of the body part of the inexperienced person acquired from the video.
An information processing program characterized by executing processing. From the first position coordinates and the first rotation direction of the body part of the proficient person and the second video of the movement of the unskilled person acquired from the first video of the movement of the proficient person. A generation unit that generates a correction instruction for the movement of the inexperienced person based on the difference between the acquired second position coordinates of the body part of the inexperienced person and the second rotation direction.
With an output unit that outputs the correction instruction to an inexperienced terminal by voice and display
An information processing device characterized by being equipped with. Information processing device
From the first position coordinates and the first rotation direction of the body part of the proficient person and the second video of the movement of the unskilled person acquired from the first video of the movement of the proficient person. Based on the acquired difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction, a correction instruction for the movement of the inexperienced person is generated.
The correction instruction is output to an inexperienced terminal by voice and display.
An information processing method characterized by executing processing. For information processing equipment
From the first position coordinates and the first rotation direction of the body part of the proficient person and the second video of the movement of the unskilled person acquired from the first video of the movement of the proficient person. Based on the acquired difference between the second position coordinates of the body part of the inexperienced person and the second rotation direction, a correction instruction for the movement of the inexperienced person is generated.
The correction instruction is output to an inexperienced terminal by voice and display.
An information processing program characterized by executing processing .

Images