JP2020175240A - Analysis data collection device, method therefor, program, and data structure - Google Patents

Abstract

To provide an analysis data collection device for making it possible to objectively analyze what perceptual information substantially contributing to exercise control is.SOLUTION: An analysis data collection device includes: a video condition setting unit that as a condition for video, sets video identification information identifying object movement, the video identification information included in video in which an opponent displayed in virtual video performs movement against a physical law; a storage unit that for each of types of object movement, stores movement information to be used in presenting object movement as video; a video synthesis unit that on the basis of the video identification information and using the movement information, synthesizes virtual video presenting an environment incapable of being reproduced in an actual environment; and an analysis data generation unit that takes as analysis data information obtained by associating the video identification information, exercise information obtained from a sensor attached to an analysis target person to which the virtual video is presented or an attachment of the person, and exercise evaluation information showing evaluation of exercise performed by the analysis target person in a state in which the virtual video is presented with one another.SELECTED DRAWING: Figure 2

Description

The present invention is a technique for collecting data for analyzing perceptual information contributing to motor control from a subject playing sports, an analysis technique for analyzing the data, and perceptual information contributing to motor control. Based on techniques for training the subject to improve their athletic performance.

In order to properly acquire the basic movements of sports, it is important for the actor to objectively grasp his / her own physical movements.

Patent Document 1 discloses a technique that makes it possible to objectively grasp the physical movement of an actor when compared with a model by outputting the current exercise state of the actor with respect to the reference exercise state.

Japanese Unexamined Patent Publication No. 2016-150119

For example, when considering the batting movement of baseball, a person reaches the destination of the ball based on various perceptual information such as the movement of the opponent who throws the ball as well as the trajectory and speed of the ball coming toward him. It predicts the arrival time and plans the content and timing of the action (exercise) to be taken by oneself, and controls the exercise. Various perceptual information can be acquired in the real environment, but it is considered important for motor control to process meaningful perceptual information quickly and accurately. However, motion control such as batting is performed in a short period of time, and necessary perceptual information is collected almost unconsciously to make a judgment. Therefore, it is difficult for oneself to grasp what information is used to decide the exercise. In addition, it is difficult to objectively grasp the difference in perceptual information used for motor control between a person who is good at batting and a person who is not good at batting.

In Patent Document 1, by presenting the location and timing of moving muscles with sound or video, it is possible to objectively grasp the difference between a model and one's own exercise regarding which muscle to move, when and how. It is possible to train to match the model. However, in order to carry out such a target exercise in a real sports environment, it is necessary to appropriately recognize and judge the situation and select an appropriate target exercise. In order to do so, it is necessary to understand what kind of information is perceived to control movement (that is, the processing in the brain before it is expressed as body movement), and such perceived information is clarified. If this happens, it will lead to effective training to improve athletic performance.

An object of the present invention is to provide an analysis data collecting device for objectively analyzing what perceptual information has a large contribution to motor control. Furthermore, it is an object of the present invention to provide an analysis device that analyzes perceptual information that greatly contributes to motor control using analysis data, and a training device for training a subject to be able to handle the perceptual information. ..

In order to solve the above problems, according to one aspect of the present invention, the analysis data collecting device captures the movement of an object included in an image in which the other party displayed in the virtual image moves against the laws of physics. The video condition setting unit that sets the video specific information to be specified as the video condition, the storage unit that stores the movement information used when presenting the movement of the object as a video for each type of motion of the object, and the video specific information Based on this, it is attached to a video synthesizer that synthesizes a virtual image that presents an environment that cannot be reproduced in the real environment using motion information, video specific information, and the analysis target person or its accessories to which the virtual image is presented. It includes an analysis data generation unit that uses information in which motion information obtained from the sensor and motion evaluation information indicating the evaluation of the motion performed by the analysis target person while the virtual image is presented as analysis data.

In order to solve the above problem, according to another aspect of the present invention, in the analysis data collecting device, the body movement of the other party and the movement of the object accompanying the body movement of the other party do not match in light of the physical law. The video condition setting unit that sets the video specific information that identifies the movement of the object included in the video as the condition of the video and the motion information used when presenting the movement of the object as a video for each type of movement of the object are stored. A storage unit that synthesizes a virtual image that presents an environment that cannot be reproduced in the real environment using motion information based on the image identification information, an image identification unit, and an analysis that presents the image identification information and the virtual image. The data for analysis is the information obtained by associating the motion information obtained from the sensor attached to the subject or its accessory with the motion evaluation information indicating the evaluation of the motion performed by the analysis subject while the virtual image is presented. Includes a data generation unit for analysis.

In order to solve the above problems, according to another aspect of the present invention, the analysis data collection method is used when the movement of the object is presented as an image in the storage unit for each type of movement of the object. It is assumed that motion information is stored, and video condition setting that sets video specific information that identifies the movement of an object, which is included in the video in which the other party moves against the physical law displayed in the virtual video, as the video condition. A video synthesis step that synthesizes a virtual video that presents an environment that cannot be reproduced in the real environment using motion information based on the steps and video specific information, and the analysis target person who is presented with the video specific information and virtual video. Or, for analysis, the information obtained by associating the motion information obtained from the sensor attached to the accessory with the motion evaluation information indicating the evaluation of the motion performed by the analysis target person while the virtual image is presented is used as the analysis data. Includes data generation steps.

In order to solve the above problems, according to another aspect of the present invention, the analysis data collection method is used when the movement of the object is presented as an image in the storage unit for each type of movement of the object. It is assumed that the motion information is stored, and the image specific information that identifies the movement of the object is included in the image in which the body movement of the other party and the movement of the object accompanying the body movement of the other party do not match in light of the physical law. The video condition setting step to be set as the condition of, the video synthesis step to synthesize the virtual video that presents the environment that cannot be reproduced in the real environment by using the motion information based on the video specific information, the video specific information, and the virtual Information that associates motion information obtained from a sensor attached to the analysis target person or its accessory to which the image is presented with motion evaluation information indicating the evaluation of the motion performed by the analysis target person while the virtual image is presented. Includes an analysis data generation step in which is used as analysis data.

In order to solve the above problems, according to another aspect of the present invention, the data structure is a subject who is presented with a synthesized virtual image in which the opponent displayed in the virtual image moves against the laws of physics. Or, it corresponds to the motion information obtained from the sensor attached to the accessory, the motion evaluation information indicating the evaluation of the motion performed by the analysis target person while the virtual image is presented, and the image identification information that identifies the virtual image. With the attached data structure, the time when there is a large difference between the motion information when the motion evaluation information is high and the motion information when the motion evaluation is low is specified as the attention time, and in the virtual image with movement near this attention time. It is used in the process of identifying a partial area.
In order to solve the above problems, according to another aspect of the present invention, the data structure is synthesized in which the body movement of the other party and the movement of the object accompanying the body movement of the other party do not match in the light of the physical law. Identify the motion information obtained from the sensor attached to the subject or its accessories to which the virtual image is presented, the motion evaluation information indicating the evaluation of the motion performed by the analysis subject while the virtual image is presented, and the virtual image. It is a data structure that associates the video specific information to be performed, and the time when there is a large difference between the motion information when the motion evaluation information is high and the motion information when the motion evaluation is low is specified as the attention time, and the vicinity of this attention time. It is used in the process of identifying a partial area in a moving virtual image.

According to the analysis data collection device of the present invention, it is possible to collect analysis data for objectively analyzing what perceptual information has a large contribution to motor control. Further, according to the analysis device of the present invention, it is possible to estimate perceptual information that greatly contributes to motion control by using the analysis data. Further, according to the training device of the present invention, it is possible to train the subject so that the subject can handle the perceptual information by using the perceptual information having a large contribution to the motor control.

The block diagram of the training system which concerns on 1st Embodiment. The functional block diagram of the data collection apparatus for analysis which concerns on 1st Embodiment. The figure which shows the example of the processing flow of the analysis data collection apparatus which concerns on 1st Embodiment. The functional block diagram of the analysis apparatus which concerns on 1st Embodiment. The figure which shows the example of the processing flow of the analysis apparatus which concerns on 1st Embodiment. The functional block diagram of the training apparatus which concerns on 1st Embodiment. The figure which shows the example of the processing flow of the training apparatus which concerns on 1st Embodiment.

Hereinafter, embodiments of the present invention will be described. In the drawings used in the following description, the same reference numerals are given to the components having the same function and the steps for performing the same processing, and duplicate description is omitted. In the following explanation, the symbol "^" etc. used in the text should be written directly above the character immediately before, but due to the limitation of the text notation, it should be written immediately after the character. In the formula, these symbols are described in their original positions. Further, the processing performed for each element of the vector or matrix shall be applied to all the elements of the vector or the matrix unless otherwise specified.

<First Embodiment>
FIG. 1 shows a configuration diagram of a training system according to the first embodiment. The training system includes an analysis data collection device 100, an analysis device 200, and a training device 300.

Each of the analysis data collection device 100, the analysis device 200, and the training device 300 is a publicly known or dedicated computer having, for example, a central processing unit (CPU), a main storage device (RAM: Random Access Memory), and the like. It is a special device configured by loading a special program into the memory. Each of the analysis data collection device 100, the analysis device 200, and the training device 300 executes each process under the control of, for example, the central processing unit. The data input to each of the analysis data collection device 100, the analysis device 200, and the training device 300 and the data obtained by each process are stored in the main storage device, for example, and the data stored in the main storage device is necessary. It is read out to the central arithmetic processing device according to the above and used for other processing. At least a part of each processing unit of the analysis data collection device 100, the analysis device 200, and the training device 300 may be configured by hardware such as an integrated circuit. Each storage unit included in each of the analysis data collection device 100, the analysis device 200, and the training device 300 includes, for example, a main storage device such as RAM (Random Access Memory), a hard disk, an optical disk, or a flash memory. It can be configured by an auxiliary storage device composed of semiconductor memory elements or middleware such as a relational database or a key value store.

Further, the training system includes a video display unit 91 and a sensor 92 that display an image on the analysis target person 93 when collecting analysis data, and a video display unit 81 that displays an image on the training target person 83 when training. It may be. The video display units 81 and 91 are, for example, head-mounted displays, and are attached to the analysis target person 93 and the training target person 83. The head-mounted display is a device for the training target person 83 and the analysis target person 93 to wear on the head and watch video and audio. As for the head-mounted display, for example, the display is provided in a range corresponding to the field of view when the training target person 83 and the analysis target person 93 are worn, and the headphones are provided at positions corresponding to both ears. Various sensors such as an acceleration sensor and a displacement sensor are mounted on the head-mounted display, and it is possible to recognize the position and posture of the head when the training target person 83 and the analysis target person 93 wear the head-mounted display. As a result, the head-mounted display can present images that match the viewpoints of the training target person 83 and the analysis target person 93. However, the video display units 81 and 91 may be display devices other than the head-mounted display. For example, it may be a 3D display, a multi-faceted box-shaped projection screen (for example, CAVE: Cave Automatic Virtual Environment), or the like. In this case, various sensors such as an acceleration sensor and a displacement sensor mounted on the head-mounted display may be separately attached to the training target person 83 and the analysis target person 93 as necessary.

[Data collection device for analysis 100]
The analysis data collecting device 100 of the present embodiment uses a sports system using virtual reality, and presents an image based on the actual environment and an image of an environment that cannot be reproduced in the actual environment to the analysis target person as a virtual reality image. Exercise information and exercise evaluation information indicating the evaluation of the exercise are acquired from the analysis target person who exercises while watching the virtual image, and the image specific information which is the information for identifying the presented virtual image, the exercise information and the exercise of the subject person. It generates analysis data associated with the evaluation information reaction.

FIG. 2 shows a functional block diagram of the analysis data collection device 100, and FIG. 3 shows an example of its processing flow.

The analysis data collection device 100 includes a storage unit 101, a video condition setting unit 102, a video synthesis unit 103, an evaluation information acquisition unit 106, and an analysis data generation unit 107.

<Memory unit 101>
The storage unit 101 stores in advance motion information (each object in the virtual image) of each object and an image of the surrounding environment, which are the bases when the image synthesis unit 103 synthesizes the image. It is assumed that the movement information of each object is stored in association with the "type of movement" for each "type of movement" of the object.

For example, when trying to acquire analysis data for a baseball batter, an image of a pitcher pitching as a virtual image and a ball flying toward an analysis target person (batter) is displayed on the image display unit 91. It will be. In this case, the motion information of each object stored in the storage unit 101 is
(1) Pitcher movement information An image of the pitcher's pitching form for each pitcher's movement (throwing style and type) (for example, an image of an overhand throw pitcher throwing a straight, an image of a side throw pitcher throwing a curve, and a slider are thrown. Underslow pitcher footage, etc.)
(2) Ball movement information A series of ball position information for each ball movement (ball trajectory), that is, a series of position information indicating where the ball is displayed in each frame (for example, flying straight). Series of position information of the coming ball, series of position information of the ball flying on the curve, series of position information of the ball flying with the slider, etc.).
Is. In addition, the series of position information starts with the position information indicating the position where the ball is displayed at the next time (frame) based on the time when the ball leaves the pitcher's hand, and the position of the ball at a later time. Let the information be arranged in chronological order.

Also, instead of (2), the image of the ball for each movement of the ball (ball trajectory) (the image of a straight ball flying toward the target, and the curved ball toward the target). An image of flying, an image of a slider ball flying toward the target person, etc.) may be stored as ball movement information. In short, the motion information may be any information indicating the motion of the object when the virtual image is generated in the image synthesis unit 103, which will be described later.

<Video condition setting unit 102>
The video condition setting unit 102 sets the conditions for the virtual video to be combined by the video composition unit 103 (S102). As described above, the analysis data collecting device 100 of the present embodiment displays "an image of an environment that cannot be reproduced in an actual environment" on the image display unit 91, but as this "environment that cannot be reproduced in an actual environment". What kind of thing is specifically handled is set as a video condition (hereinafter, also referred to as "video specific information"). In other words, the video specific information is information for specifying what kind of setting the presented virtual video is generated, that is, information for specifying the presented virtual video. As long as a unique ID is assigned to each video condition, that ID may be used, or the video condition itself (for example, in the example of baseball described later, it is a pair of throwing form and ball movement (straight, curve), etc.). Good.

"Environment that cannot be reproduced in the real environment" means an image in which the opponent moves against the laws of physics displayed in the virtual image, or an object such as a ball that is delivered according to the opponent's body movement and the opponent's body movement. It is an image in which the movement of is not consistent with the laws of physics. Taking the case of baseball as an example, the following image is shown.
(1) The image of the pitching form when the pitcher throws a straight ball and the image of the ball approaching the target along the trajectory of the curve are combined (that is, the throwing form and the trajectory of the thrown ball do not match. Video).
(2) An image in which the trajectory of the ball thrown by the pitcher follows a straight trajectory halfway, but then suddenly switches to a movement along a curved trajectory (that is, the physical law that the trajectory of the ball in the first half follows). , Images with different physical laws that the trajectory of the ball in the second half follows).
(3) An image in which a ball coming from a pitcher toward himself disappears in a part of the orbit (that is, an image in which an object disappears where it cannot disappear in the actual environment).

That is, the video condition setting unit 102 identifies the movement of each object (including a person) in the video displayed as a virtual video on the video display unit 91 (a video presenting an environment that cannot be reproduced in the real environment). Is set as a video condition (S102) and output to the video compositing unit 103.

In the above-mentioned example of baseball (1), a set of information for specifying the movement of the pitcher and the ball (straight, curve) is set as a video condition (video specific information).

The video condition may be given by an external input, or the video condition may be generated by randomly selecting one from a plurality of movements prepared in advance for each object. For example, in the above-mentioned baseball example, straights, curves, sliders, etc. are defined in advance as pitcher movements, and straights, curves, sliders, etc. are defined as ball movements in a list. Select movements one by one (curve, slider) and set a set of information that identifies the movement as a condition of the image.

The motion information and the video identification information stored in the storage unit 101 may be appropriately prepared according to the virtual video presented to the analysis target person. For example, in the case of an image in which the trajectory of the ball in (2) moves against the laws of physics, the motion information of the ball thrown by the pitcher along the trajectory of a straight, curve, slider, etc. is prepared halfway. (Stored in the storage unit 101 in advance), prepare motion information along the trajectory of straights, curves, sliders, etc. from the middle (stored in the storage unit 101 in advance), and the trajectory and the middle of the process. The combination with the trajectory from is set as the video specific information that can be set. Also, for example, (3) in the case of an image in which the ball coming from the pitcher toward himself disappears in a part of the orbit, from point A to point B on the orbit, from point B to point C, In ..., the movement information on which the ball disappears is prepared (stored in the storage unit 101 in advance), and the information for specifying the disappearing section is set as the video specific information that can be set.

In addition to the movement of each object, information for specifying the exercise environment (raining / sunny, dark / bright) may be added as a video condition (video specific information).

In addition to the images of the environment that cannot be reproduced in the actual environment, in order to present the images based on the actual environment to the analysis target as virtual reality images, the images that move without violating the laws of physics and the body movements of the other party. Information that identifies an image in which the movement of an object such as a ball that is delivered in accordance with the physical movement of the other party matches with the physical law may also be added as the image identification information. For example, in the above-mentioned example of baseball (1), a set of information specifying the movement such as the movement of the pitcher and the ball (straight, straight) may be set as the video identification information, and in the example of (2), the pitcher. A set of information that specifies the movement, such as the trajectory of the ball thrown by the ball up to the middle of the trajectory and the trajectory from the middle (straight, straight), may be set as the video identification information, or in the example of (3), from the pitcher. The motion information that the ball coming toward itself does not disappear may be set as the image specific information.

<Video synthesis unit 103>
The video compositing unit 103 receives the video specific information as input, and generates a video (virtual video) that synthesizes a video in which each object (including a person) moves according to the video conditions set by the video condition setting unit 102 (S103). , Is output to the video display unit 91. When the video condition includes information on the exercise environment, the video is processed according to the environment and output to the video display unit 91.

Explaining with the above-mentioned baseball example, the image synthesizing unit 103 has a pitching form image corresponding to the pitcher's movement under the condition of the image output from the image condition setting unit 102 and a ball image corresponding to the ball movement. Is read from the storage unit 101, and the combined video is output to the video display unit 91 as a virtual video. Specifically, for example, in the image of the pitching form read from the storage unit 101, the position of the hand throwing the ball (initial position) of the frame image (throwing start frame image) corresponding to the moment when the ball leaves the hand. ) Is superimposed on the image of the ball to generate an image. The image of the frame at the next time is the image of the frame next to the pitching start frame image in the image of the pitching form, and the first position information in the sequence of the ball position information read from the storage unit 101 is the ball of the previous frame. An image in which the image of the ball is superimposed on the position corrected according to the position of is generated. A virtual image is generated by performing the same processing for subsequent frames.

The above-mentioned image display unit 91 displays the virtual image synthesized by the image composition unit 103 on the internal display and presents it to the analysis target person 93 (S104).

The sensor 92 is a predetermined part of the body of the analysis target person 93 who moves while viewing the virtual image presented on the image display unit 91, or / and an object (bat, racket, or the like) held by the analysis target person 93, which is attached. It is attached to an object), acquires sensor data (S105), and outputs it. This acquired sensor data will be referred to as motion information. As the sensor 92, a sensor for detecting various motion information such as an acceleration sensor, a displacement sensor, and a motion sensor, or various biological information such as heartbeat, electrocardiogram, respiration, myoelectricity, and dynamic eye movement are acquired. Sensors and the like can be considered. It is assumed that one or more sensor data acquired from one or more of these sensors is acquired as motion information.

<Evaluation information acquisition unit 106>
The evaluation information acquisition unit 106 evaluates the exercise performed by the subject exercising while viewing the virtual image presented on the image display unit 91 (in other words, the exercise performed by the analysis target person while the virtual image is presented). The information indicating the above is acquired as exercise evaluation information (S106) and output.

"Exercise evaluation information" includes, for example, whether or not batting was successful (whether or not the ball was hit back, whether it was a hit, whether it was a home run, whether it was a missed shot, etc.) and the skill of batting movement (what kind of timing). Information indicating the movement state (for example, beanball (toward the head)) that affects the subject's biological reaction (eye movement, heartbeat, etc.) during the exercise, or how the body was manipulated. Information indicating that the ball) was thrown when it came). The attention state (degree and range of attention) and psychological state of the subject during the exercise may be added as exercise evaluation information.

The motion evaluation information may be input from the analysis target person 93 or a third party by external input, or the evaluation value is calculated based on the motion information acquired by the sensor 92 and the video information displayed by the video display unit 91. However, it may be used as exercise evaluation information.

In the example of baseball, the result of hit determination as to whether or not the ball hits the bat is exercised based on the motion information of the bad swung by the analysis target 93 and the position information of the ball displayed on the video display unit 91. It may be used as evaluation information. For example, the evaluation information acquisition unit 106 receives the video identification information from the video condition setting unit 102, and extracts the ball movement information (series of ball position information) from the storage unit 101 based on the video identification information. Further, the evaluation information acquisition unit 106 receives motion information indicating the position of the bat from the sensor 92 attached to the bat. The evaluation information acquisition unit 106 determines whether or not the ball has hit the bat based on the movement information of the ball and the movement information indicating the position of the bat. If it hits, a high evaluation value may be set, and if it does not hit, a low evaluation value may be set as exercise evaluation information.

<Data generation unit for analysis 107>
The analysis data generation unit 107 inputs the image identification information corresponding to the image condition set by the image condition setting unit 102, the motion information output by the sensor 92, and the motion evaluation information output by the evaluation information acquisition unit 106. Then, the information associated with these information is used as analysis data and output to the analysis device 200. The analysis data collection device 100 stores analysis data in a storage unit (not shown), and a set of analysis data is stored in the storage unit 201 of the analysis device 200, which will be described later.

[Analyzer 200]
The analysis device 200 of the present embodiment improves exercise performance by using a set of analysis data (analysis data set) acquired for different video conditions and different subjects using the above-mentioned analysis data collection device 100. Estimate perceptual information that can contribute (information indicating which movement in the image should be focused on, hereinafter also referred to as "focused visual information").

In FIG. 1, it seems that the analysis data acquired by the analysis data collection device 100 is input to the sequential analysis device 200, but the present invention is not limited to this. It is assumed that the analysis data acquired by the analysis data collection device 100 at various places, various subjects, and different dates and times is stored in the storage unit in advance, and the set of analysis data stored in the storage unit is stored in the analysis device. It is assumed that 200 can be read and used.

FIG. 4 shows a functional block diagram of the analyzer 200, and FIG. 5 shows an example of its processing flow.

The analysis device 200 includes a storage unit 201, a first analysis unit 202, and a second analysis unit 203.

The analysis device 200 receives a set of analysis data collected by the analysis data collection device 100 and stores it in the storage unit 201.

<Memory unit 201>
It is assumed that the storage unit 201 stores a set of analysis data. The set of analysis data includes at least (1) multiple analysis data acquired from the same analysis target person for videos corresponding to a plurality of different video specific information, and (2) video corresponding to the same video specific information. It shall include any of a plurality of analysis data obtained from a plurality of different analysis subjects.

<First analysis unit 202>
The first analysis unit 202 extracts a set of analysis data from the storage unit 201, estimates motion information having a high correlation with the motion evaluation information based on the set of analysis data (S202), and types of the estimated motion information. Outputs information that identifies. The estimated result will be referred to as "analysis target motion information". Therefore, the motion information to be analyzed specifies the types of various biological information (biological responses) such as acceleration sensor, displacement sensor, output value of motion sensor, heartbeat, electrocardiogram, respiration, myoelectricity, and dynamic movement of the eye. Information etc. The motion information to be analyzed can be said to be information that specifies the type of sensor used when obtaining these output values and various biological information (biological responses).

Specifically, when a plurality of analysis data acquired from the same or multiple analysis subjects are analyzed for a video corresponding to the same video specific information, when the motion evaluation is high (when it is successful) and the motion evaluation is performed. The category of motion information (for example, information that identifies the type of sensor) that has a large difference from when is low (when it cannot be done well) is estimated as the analysis target motion information. For example, a category of exercise information (for example, a sensor) in which the time at which the peak value is taken differs greatly depending on whether the evaluation is high or low, or the time-series pattern of the exercise information differs greatly depending on whether the evaluation is high or low. Information that identifies the type, etc.) is used as the exercise information to be analyzed. It should be noted that the motion information to be analyzed is information that can be identified even if the sensors are of the same type, including the part (where the sensor is attached) from which the motion information is acquired.

In addition, (1) when a plurality of analysis data acquired from the same analysis target person are analyzed for videos corresponding to a plurality of different video specific information, there is a difference between when the motion evaluation is high and when the motion evaluation is low. A large category of motion information (for example, information that identifies the type of sensor) is estimated as the motion information to be analyzed.

Alternatively, (2) when a plurality of analysis data acquired from a plurality of different analysis subjects are analyzed for a video corresponding to the same video specific information, a subject with a high exercise evaluation (a subject with a high exercise performance: a professional). Analyzing categories of exercise information (for example, information that identifies the type of sensor) where there is a large difference between subjects with low exercise evaluation (targets with low exercise performance: beginners, substitute athletes, etc.) and those with low exercise evaluation (athletes, regular athletes, etc.) Use as target exercise information.

The exercise information to be analyzed may be one exercise information category or a combination of a plurality of exercise information categories.

When there are a plurality of analysis data, the analysis target motion information may be obtained by performing the above analysis on the analysis data obtained by averaging these data, or the analysis target motion information may be obtained for each analysis data. The majority of the obtained results may be taken, and the most ones may be prioritized as the final analysis target exercise information.

By the above-mentioned processing, a plurality of analysis target motion information corresponding to a plurality of video specific information is acquired from a set of analysis data.

<Second analysis unit 203>
The second analysis unit 203 uses the analysis target motion information and the video identification information included in the set of analysis data stored in the storage unit 201, and the motion information corresponding to the analysis target motion information included in the virtual image. Information that identifies a video partial region (a partial region with movement in a virtual video) that has a high correlation with is estimated as "attention visual information" (S203), and information that associates the analysis target motion information with the attention visual information (information (Also called attention movement type information) is obtained and output. For example, the visual information of interest includes the time at which the behavior indicated by the motion information corresponding to the motion information to be analyzed changes (hereinafter referred to as “change time”) and a partial region in the virtual image that moves near the change time. It includes at least information indicating (a video partial region having a high correlation with the motion information corresponding to the motion information to be analyzed). The information indicating the partial area is information that can identify a moving partial area in the frame, that is, information that specifies the position and size of the partial area in the frame. For example, the information indicating the partial region moves in the vicinity of the time when the movement information corresponding to the analysis target movement information fluctuates greatly (also called the fluctuation time) in the virtual image included in the analysis data when the movement evaluation is high. It may be information that identifies a certain subregion. The visual information of interest is the association of the fluctuation time with the information indicating the partial area. In addition, the visual information of interest may include video identification information that identifies the video information used to specify a partial region in the virtual video.

Here, the analysis data set used by the second analysis unit 203 is the same as the analysis data set used by the first analysis unit 202 to obtain the analysis target motion information.

For example, the second analysis unit 203 changes the behavior of each analysis data in the analysis data set as shown by the motion information acquired from the specific sensor 92 corresponding to the analysis target motion information obtained by the first analysis unit 202. Estimates a partial area in the virtual image (the image displayed on the image display unit 91 of the analysis data acquisition device 100) in which there is movement in the vicinity of the time when This makes it possible to correlate which part of the image moves (partial area in the image) with which movement or movement of the biological response (motion information corresponding to the motion information to be analyzed) has a high correlation. ..

The second analysis unit 203 has, for example, a partial region in the virtual image having a high correlation (presumed) with the change time, the motion information corresponding to the motion information to be analyzed thus obtained, and a virtual including the partial region. The video identification information for specifying the video may be the visual information of interest. In this case, the attention visual information and the analysis target movement information are output as "information relating the analysis target movement information and the attention visual information". In this case, the reason why the visual information of interest includes the video specific information is that it is necessary as the information for specifying the virtual video generated for training by the training device described later.

For example, it can be performed by the following processing.
(1) Regarding the sequence of motion information in the analysis data acquired by the sensor corresponding to the analysis target motion information obtained by the first analysis unit 202, the time when the sensor data fluctuates greatly between adjacent times is set as the change time. Extract.
(2) Regarding the video identification information indicated by the analysis data for which the change time was obtained in (1) above, the vicinity of the change time in the video specified by the video identification information (the video output from the video synthesis unit 103 described above). The images of the two frames at different times are compared, and the partial region in the image in which there is movement between the frames is calculated as the feature of the frame at the change time.

When the above (1) and (2) are performed for each analysis data in the analysis target data set for each type of exercise information indicated by the analysis target exercise information, the partial region and change of the image having a high correlation with each exercise information type. The time can be extracted. If the subregions and change times extracted for each analysis data are different, a majority vote may be taken in the entire analysis data, and a predetermined number of subregions and change times may be selected from the largest number.

The exercise information corresponding to the analysis target exercise information obtained by the first analysis unit 202 differs between when the exercise evaluation is high (or person) and when the exercise evaluation is low (or person).

In the second analysis unit 203, focusing on the movements and the movements of the biological response that differ depending on whether the motion evaluation is high or low, the partial region of the moving image around the time when the movements or the biological response is large is generated. , It is presumed that it is the video (visual) information that contributed to the determination of the movement and the biological response in the brain.

For example, if a person with high batting accuracy in a real environment loses accuracy in batting for an image in which the throwing form and the trajectory of the ball do not match, the perceptual information of at least one of the throwing form or the trajectory of the ball moves. It is considered that it greatly contributes to control and biological response. Therefore, the analysis data when batting in the actual environment by this method is used as the analysis data when the motion evaluation is high, and the analysis data when batting is performed on the image in which the throwing form and the trajectory of the ball do not match is the motion evaluation. If the extracted attention visual information is "a section where the throwing form is moving (before the ball leaves the hand)" by analyzing it as analysis data when the value is low, a player with high athletic performance will have the trajectory of the ball. It is thought that the trajectory of the ball is predicted to some extent from the throwing form before looking at. In particular, it is possible to grasp at what point in time the movement of the pitching form the judgment is made by the attention visual information. Similarly, if the visual information of interest is "the section where the ball flies away from the pitcher's hand", the trajectory of the ball in that section contributes to determining motion control (batting timing and movement) and biological response. You can see that it is doing.

Alternatively, (3) of the image condition setting unit 102 shows an example of erasing the image of the ball in a part section on the trajectory of the ball, but in such an image, the analysis data when the motion evaluation is low. It can be presumed that the information on the section where the ball is erased in the video corresponding to the above may greatly contribute to the movement control and the biological response.

The analysis data acquired by the analysis data collection device 100 is not limited to the method used by the analysis device 200, and it is also possible to manually analyze perceptual information (visual information of interest) that can contribute to the improvement of motor performance. is there.

[Training device 300]
The training device 300 is a training device 300 for training so that the trainee can acquire the visual information corresponding to the visual information of interest.

In FIG. 1, the information output from the analysis device 200 is sequentially input to the training device 300, and it seems that the training device 300 is executed, but the present invention is not limited to this. In general, information that associates the analysis target motion information with the attention visual information is obtained in advance by the analyzer 200 or another manual method, and the information that associates the analysis target motion information with the attention visual information is stored in the storage unit. In the training device 300, the analysis target exercise information and the attention visual information stored in the storage unit are stored in the storage unit at the place where the analysis data is acquired, the acquired target person, the place different from the acquired date and time, the target person, and the date and time. It is assumed that the information associated with is read and executed.

FIG. 6 shows a functional block diagram of the training device 300, and FIG. 7 shows an example of its processing flow.

The training device 300 includes a storage unit 301, a training condition setting unit 302, and a second video composition unit 303.

<Memory unit 301>
The storage unit 301 stores in advance motion information (each object in the virtual image) of each object and an image of the surrounding environment, which are the bases when the image is synthesized by the second image synthesis unit 303. It is assumed that the movement information of each object is stored in association with the "type of movement" for each "type of movement" of the object. The motion information is information used when presenting the motion of an object as an image. Basically, it is the same as that stored in the storage unit 101 of the analysis data collection device.

Further, it is assumed that the storage unit 301 stores information relating the motion information to be analyzed and the visual information of interest. This is the analysis target motion information (type of sensor) obtained by the above-mentioned analysis device 200 or another manual method, and the attention visual information which is the visual information highly correlated with the fluctuation of the analysis target motion information. Information to be associated. The exercise information to be analyzed is information representing the types of exercise information that differ between exercises with high evaluation and exercises with low evaluation, and is also called change exercise type information.

<Training condition setting unit 302>
The training condition setting unit 302 sets the analysis target exercise information to be trained by the training target person as the training target exercise information (S302), and sets the analysis target exercise information stored in the storage unit 301 as information for associating the attention visual information. Based on this, the attention visual information associated with the analysis target exercise information corresponding to the set training target exercise information is output. For example, the analysis data obtained for the trainee by the first analysis unit 202 of the above-mentioned analysis device 200 (or manual analysis, etc.) and the analysis data obtained for the person with higher exercise performance than the trainee. The analysis target exercise with a large difference is obtained, and this is set as the training target exercise information. When a plurality of analysis target exercise information is extracted, one of them to be trained may be selected and set.

Alternatively, when the analysis data acquired by the first analysis unit 202 of the above-mentioned analysis device 200 (or manual analysis or the like) when the exercise evaluation of the trainee is high and when the exercise evaluation of the trainee is low. An analysis target exercise having a large difference from the acquired analysis data may be obtained and set as training target exercise information.

Information that identifies a sensor that measures the movement of the body part or accessory if the instructor or the like grasps the movement of the body part or accessory that the instructor or the like particularly wants to enhance the movement regardless of the analysis device 200. May be set as training target exercise information.

<Second video compositing unit 303>
The second video synthesizing unit 303 receives the attention visual information from the training condition setting unit 302, and synthesizes the virtual image for training by using the attention visual information. The virtual image for training is a virtual image that presents an environment that cannot be reproduced in a real environment, and is an image in which a partial area in the virtual image indicated by the visual information of interest is emphasized.

For example, the second video synthesizing unit 303 synthesizes a virtual image in the same manner as the video synthesizing unit 103 of the analysis data collecting device 100. At this time, in the time frame near the change time included in the attention visual information received from the training condition setting unit 302, a virtual image superimposing a visual effect that emphasizes a partial area in the virtual image indicated by the attention visual information is synthesized. (S303), output to the image display unit 81.

In the video composition unit 103 of the analysis data collection device 100, the video composition conditions are given by the video condition setting unit 102, but the second video composition unit 303 randomly sets the video conditions and analyzes under the set conditions. It is assumed that the virtual video can be synthesized by the same method as the video compositing unit 103 of the data collecting device 100. In the randomly synthesized virtual image, training is performed to pay attention to the partial area indicated by the visual information of interest in the vicinity of the change time. It should be noted that the video specific information may be included in the visual information of interest, and the second video compositing unit 303 may synthesize the virtual video under the video conditions corresponding to the video specific information.

The image in which the visual effect that emphasizes the partial area in the virtual image indicated by the attention visual information set by the training condition setting unit 302 is superimposed is, for example, a visual effect that conspicuously surrounds the partial area in the virtual image. , This is an image in which the image in the area other than the relevant partial area is blurred or the information is reduced in the virtual image. In addition to imparting a certain visual effect, the reference image may be gradually changed according to the progress of the training from the reference image to the above image.

The above-mentioned image display unit 81 displays the virtual image synthesized by the second image compositing unit 303 on the internal display and presents it to the trainee 83 (S304).

The trainee 83 exercises while watching the image displayed on the image display unit 81. As a result, when and what kind of area a person with high exercise performance is visually recognizing and performing exercise control is highlighted in the virtual image, and the trainee 83 acquires visual information by focusing on that part. And can be trained to start exercising.

<Effect>
With the above configuration, the analysis data collection device can collect analysis data to enable objective analysis of what perceived information has a large contribution to motor control, and the analysis device can collect analysis data. Can be used to estimate perceptual information that contributes significantly to motor control, and the training device should train the trainee to handle the perceptual information using perceptual information that contributes significantly to motor control. Can be done.

In this embodiment, by using a virtual image of an environment that cannot be reproduced in the actual environment, it is possible to determine which information the analysis target person uses to exercise, as compared with the case of using only the image based on the actual environment. It has the effect of emphasizing.

<Modification example>
In the present embodiment, the analysis data collection device 100, the analysis device 200, and the training device 300 have been described as separate devices, but these devices may be realized by one known or dedicated computer. Similarly, the video display units 81 and 91 may be realized by one video display unit (for example, a head-mounted display).

In the present embodiment, the output value (set of analysis data) of the analysis data collection device 100 is used by the analysis device 200, and the output value (attention visual information) of the analysis device 200 is used by the training device 300. However, each device does not necessarily require another device and can be used independently. For example, the output value (set of analysis data) of the analysis data collection device 100 does not necessarily have to be used in the analysis device 200, and as described above, perceptual information (attention visual information) that can manually contribute to the improvement of exercise performance. ) Can also be analyzed. Further, in the analysis device 200, if there is data corresponding to a set of analysis data, it does not necessarily have to be the output value of the analysis data collection device 100. For example, the analysis data collecting device presents only the "image based on the actual environment" to the analysis target person, and the analysis target person who exercises while watching the image provides the exercise information and the exercise evaluation information indicating the evaluation of the exercise. You may use a set of analysis data which is acquired and associates the video specific information with the exercise information of the subject and the exercise evaluation information reaction. Further, from the output value (visual information of interest) of the analysis device 200, a part to be noted during training may be presented regardless of the training device. In the training device 300, even if the output value of the analysis device 200 is not used, if the part to be noted is known, the virtual image in which the visual effect emphasizing the part to be noted in the virtual image is superimposed may be synthesized.

In the present embodiment, an example of baseball batting is shown in which the exercise performed by the analysis target person while the virtual image is presented and the exercise to be trained by the training target person are other sports (softball). It goes without saying that it can be applied to batting (and cricket), and even to competitive ball games. Furthermore, even in other exercises, by using a virtual image of an environment that cannot be reproduced in the actual environment, it is possible to determine which information the analysis target person uses to exercise, as compared with the case of using only the image based on the actual environment. Since it becomes easy to analyze, the analysis data collecting device and the analysis device of the present invention can be applied. In other exercises as well, it is possible to efficiently train the trainee by presenting a composite of a virtual image in which a visual effect that emphasizes a partial area in the virtual image including the movement corresponding to the visual information of interest is superimposed. Therefore, the training device of the present invention can be applied.

<Other variants>
The present invention is not limited to the above embodiments and modifications. For example, the various processes described above may not only be executed in chronological order according to the description, but may also be executed in parallel or individually as required by the processing capacity of the device that executes the processes. In addition, changes can be made as appropriate without departing from the spirit of the present invention.

<Programs and recording media>
In addition, various processing functions in each device described in the above-described embodiment and modification may be realized by a computer. In that case, the processing content of the function that each device should have is described by the program. Then, by executing this program on the computer, various processing functions in each of the above devices are realized on the computer.

The program describing the processing content can be recorded on a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disk, a photomagnetic recording medium, a semiconductor memory, or the like.

Further, the distribution of this program is performed, for example, by selling, transferring, renting, or the like a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in the storage device of the server computer and transferring the program from the server computer to another computer via a network.

A computer that executes such a program first, for example, first stores a program recorded on a portable recording medium or a program transferred from a server computer in its own storage unit. Then, when the process is executed, the computer reads the program stored in its own storage unit and executes the process according to the read program. Further, as another embodiment of this program, a computer may read the program directly from a portable recording medium and execute processing according to the program. Further, every time the program is transferred from the server computer to this computer, the processing according to the received program may be executed sequentially. Further, the above-mentioned processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and the result acquisition without transferring the program from the server computer to this computer. May be. In addition, the program shall include information used for processing by a computer and equivalent to the program (data that is not a direct command to the computer but has a property of defining the processing of the computer, etc.).

Further, although each device is configured by executing a predetermined program on a computer, at least a part of these processing contents may be realized by hardware.

Claims (11)

An image condition setting unit that sets image specific information that identifies the movement of an object, which is included in an image in which the other party moves against the laws of physics displayed in the virtual image, as an image condition.
A storage unit that stores motion information used when presenting the motion of an object as an image for each type of motion of the object.
A video synthesizer that synthesizes a virtual video that presents an environment that cannot be reproduced in a real environment using the motion information based on the video specific information.
The image specific information, the motion information obtained from the analysis target person presented with the virtual image or the sensor attached to the accessory thereof, and the evaluation of the motion performed by the analysis target person with the virtual image presented. Includes an analysis data generation unit that uses information associated with the indicated exercise evaluation information as analysis data.
Data collection device for analysis. A video condition setting unit that sets video specific information that identifies the movement of an object, which is included in the video in which the body movement of the other party and the movement of the object accompanying the body movement of the other party do not match in light of the laws of physics. ,
A storage unit that stores motion information used when presenting the motion of an object as an image for each type of motion of the object.
A video synthesizer that synthesizes a virtual video that presents an environment that cannot be reproduced in a real environment using the motion information based on the video specific information.
The image specific information, the motion information obtained from the analysis target person presented with the virtual image or the sensor attached to the accessory thereof, and the evaluation of the motion performed by the analysis target person with the virtual image presented. Includes an analysis data generation unit that uses information associated with the indicated exercise evaluation information as analysis data.
Data collection device for analysis. The analysis data collection device according to claim 2.
An image in which the body movement and the movement of an object accompanying the body movement of the other party do not match in light of the laws of physics is
The opponent's physical movement is an image of a throwing form,
The movement of the object is a trajectory of the movement of the object that is delivered in accordance with the body movement of the other party.
Data collection device for analysis. The analysis data collecting device according to claim 1.
The image displayed in the virtual image in which the other party moves against the laws of physics is
It is an image that the trajectory of the thrown object changes in the middle,
Data collection device for analysis. A data collection device for analysis according to any one of claims 1 to 4.
The exercise performed by the analysis target person is a battle-type ball game, and the virtual image includes an opponent and a ball used in the ball game.
Data collection device for analysis. A data collection device for analysis according to any one of claims 1 to 5.
The exercise performed by the analysis subject is batting.
Data collection device for analysis. It is assumed that the storage unit stores motion information used when presenting the motion of the object as an image for each type of motion of the object.
A video condition setting step that sets video specific information that identifies the movement of an object as a video condition, which is included in the video in which the other party moves against the laws of physics displayed in the virtual video.
A video synthesis step of synthesizing a virtual video that presents an environment that cannot be reproduced in a real environment by using the motion information based on the video specific information.
The image specific information, the motion information obtained from the analysis target person presented with the virtual image or the sensor attached to the accessory thereof, and the evaluation of the motion performed by the analysis target person with the virtual image presented. Includes an analysis data generation step in which information associated with the indicated exercise evaluation information is used as analysis data.
Data collection method for analysis. It is assumed that the storage unit stores motion information used when presenting the motion of the object as an image for each type of motion of the object.
A video condition setting step for setting video specific information that identifies the movement of an object, which is included in a video in which the body movement of the other party and the movement of the object accompanying the body movement of the other party do not match in light of the laws of physics. ,
A video synthesis step of synthesizing a virtual video that presents an environment that cannot be reproduced in a real environment by using the motion information based on the video specific information.
The image specific information, the motion information obtained from the analysis target person presented with the virtual image or the sensor attached to the accessory thereof, and the evaluation of the motion performed by the analysis target person with the virtual image presented. Includes an analysis data generation step in which information associated with the indicated exercise evaluation information is used as analysis data.
Data collection method for analysis. A program for operating a computer as an analysis data collection device according to any one of claims 1 to 6. Motion information obtained from a sensor attached to the subject or its accessories, which is presented with a synthesized virtual image in which the other party displayed in the virtual image moves against the laws of physics,
Exercise evaluation information indicating the evaluation of the exercise performed by the analysis target person while the virtual image is presented, and
Video identification information that identifies the virtual image and
In the data structure associated with, the time when there is a large difference between the motion information when the motion evaluation information is high and the motion information when the motion evaluation is low is specified as the attention time, and the movement is performed in the vicinity of the attention time. A data structure used in a process for identifying a partial area in a virtual image. The movement information obtained from the sensor attached to the subject or its accessory, which is presented with a synthesized virtual image in which the body movement of the other party and the movement of the object accompanying the body movement of the other party do not match in light of the laws of physics,
Exercise evaluation information indicating the evaluation of the exercise performed by the analysis target person while the virtual image is presented, and
Video identification information that identifies the virtual image and
Is a data structure in which A data structure used in a process for identifying a partial area in a virtual image.