JP6691490B2 - Exercise state time information presenting device, exercise state time information presenting method, program - Google Patents

Description

  The present invention relates to a technique for presenting time information of a motion state in a body motion, and particularly to a technique for presenting a relative time relationship at the time when an important motion state in a series of pitching motions occurs.

  When trying to acquire sports skills, there is a desire to train as effectively as possible. It is generally considered that exercises for improving muscle exertion ability and exercises for improving cardiopulmonary function are useful because the expected effect is relatively clear regardless of the type of sport. However, these basic trainings do not immediately lead to the improvement of certain sport skills. In addition to basic training, training specific to each sport is required to acquire techniques specific to each sport, and such training also has various required level differences.

  For example, the pitching movement of a baseball pitcher (pitching movement) is one of the exercise tasks that is very difficult to learn. The purpose of pitching is to throw a ball that is hard for the batter to hit, by adjusting the ability of the ball away from the hand to pass the desired spatial trajectory and the speed of the ball (ball speed). Both abilities are required at the same time. In particular, in order to throw a fast ball, it is important that the motions of the joints of the lower limbs, the trunk, and the upper limbs occur in an appropriate time sequence (hereinafter, referred to as a motion chain) (Non-Patent Document 1).

  However, no training method has been established to surely acquire the kinematic chain that has the effect of increasing the ball speed. For example, what is considered to be effective as a training method to increase the ball speed is that an excellent instructor with experience and a track record provides improvement instruction to learners through words and gestures. Due to individual differences in methods and skill levels, not all learners will receive adequate instruction. Therefore, it is desired to cultivate a means and technique for more learners to receive effective guidance for increasing the ball speed.

  When an instructor teaches a learner how to pitch, it is generally common to record the learner's pitching in an image and check how the exercise should be corrected while watching the reproduced image. This method is useful in that the instructor / learner can share the state of the learner's exercise from an objective viewpoint. However, since the series of pitching actions is very fast, the subjective way of grasping the movement changes depending on how the viewer of the playback image focuses his or her visual or cognitive attention. It is not always possible to get a good guide on what to do.

  Therefore, the visibility of exercise is often increased by stretching the recorded video in the time direction and playing it back (slow playback or frame-by-frame playback). In addition, in order to improve the visibility of the pitching video, there is a technique of extracting and displaying only a human region from the video, and superimposing a motion locus of a specific part such as a hand on the video (Non-Patent Document 2). .

R. F. Escamilla, G. S. Fleisig, S. W. Barrentine, N. Zheng, J. R. Andrews, “Kinematic comparisons of throwing different types of baseball pitches”, Journal of Applied Biomechanics, Vol.14, Issue 1, pp.1-23, 1998. Takatoshi Ichiro, Saito Takafumi, Tanaka Hideyuki, "Video Image Processing for Sports Instruction", IPSJ Research Report Graphics and CAD (CG) 2003 (15 (2002-CG-110)), pp.37-42, 2003.

  However, when you see a motion stretched in the time direction like slow playback or frame advance playback, it is difficult to grasp the influence of external force (gravity, ground reaction force, inertial force) acting on the body, so it is accurate It becomes difficult to obtain the motion correction guideline.

  Further, even if a method for improving visibility such as the technique of Non-Patent Document 2 is used, it is difficult to accurately recognize the temporal structure of the kinematic chain, and it is also difficult to obtain an accurate motion correction guideline.

  Therefore, an object of the present invention is to provide a motion state time information presentation technique capable of presenting, as feedback, a relative time relationship at the time when an important motion state in a series of pitching motions occurs.

  According to an aspect of the present invention, a first camera is a camera for photographing a state in which a forefoot touches the ground during a pitching motion, and a certain area on a plane parallel to a pitch direction and a vertical direction photographed by the first camera is a first camera. The image area of one camera, the second camera is a camera for photographing the state of the trunk rotating horizontally during pitching motion, and the second camera is a certain area on the surface looking down on the ground from above the second camera. A first time point extraction unit that extracts a time point A, which is a time point when the front foot touches the ground, from a first input image obtained by photographing the image area of the first camera as an image area, and a first time point when the image area of the second camera is photographed. A second time point extraction unit that extracts a time point B, which is a time point when the trunk starts horizontal rotation, from two input images, and a relative time relationship between the time points A and B from the time points A and B. Relative time relationship reflecting It generates a readback, and a relative time relationship feedback presentation section for presenting.

  According to the present invention, it is possible to present, as feedback, a relative time relationship at the time when an important motion state in a series of pitching motions occurs.

The schematic diagram of a pitching operation. The figure which shows an example of a structure of the exercise state time information presentation apparatus 100. The figure which shows an example of the image area of the 1st camera 901 and the 2nd camera 902. The figure which shows an example of operation | movement of the exercise state time information presentation apparatus 100. The figure which shows an example of a structure of the 1st time point extraction part 110. The figure which shows an example of operation | movement of the 1st time point extraction part 110. The figure which shows an example of the relationship between the binary image of the image area of the 2nd camera 902, and the trunk vector r ^t (in the case of right hand throwing). The figure which shows an example of the relationship between the binary image of the image area of the 2nd camera 902, and the trunk vector r ^t (in the case of left hand throwing). The figure which shows an example of a structure of the 2nd time point extraction part 120. The figure which shows an example of operation | movement of the 2nd time point extraction part 120. The figure which shows an example of a structure of the exercise state time information presentation apparatus 200. The figure which shows an example of the arrangement | positioning relationship of the 1st camera 901, the 2nd camera 902, and the ball speed measuring device 903. The figure which shows an example of operation | movement of the exercise state time information presentation apparatus 200.

  Hereinafter, embodiments of the present invention will be described in detail. It should be noted that components having the same function are denoted by the same reference numeral, and redundant description will be omitted.

<Technical background>
FIG. 1 is a schematic diagram of a pitching operation. In this continuous motion, the time of occurrence of the movement state (hereinafter, also referred to as an event) of interest is the time A at which the front foot (that is, the foot stepping in the pitching direction) touches the ground, and the horizontal rotation of the trunk ( That is, there are two points at time point B, which is the time point at which the movement of directing the front of the body in the pitching direction) starts. The reason for paying attention to the two points of time A and time B is as follows. The ball speed is nothing but the ground speed of the fingertip at the time when the ball leaves the hand (time C), but according to the kinematic analysis of pitching, it is important to increase the ground speed of the fingertip at time A (the front foot touchdown). ) To time point C (away from the ball), the horizontal rotation speed of the trunk is increased as much as possible (Non-Patent Document 1). Since it is natural that the front of the body faces the pitching direction at time point C, in order to sufficiently increase the horizontal rotation speed of the trunk between time points A and C, the direction of the trunk at time point A and the time point The larger the difference in the orientation of the trunk in C, the more advantageous. It is considered that the relative time relationship between the time points A and B directly affects the difference in the direction.

  Therefore, the relative time relationship between the time points A and B, that is, the time sequence and the time interval between the time points A and B are presented to the instructor or the learner as feedback. Therefore, it is considered that the instructor or the learner can effectively support the correction of the pitching motion.

<First embodiment>
The exercise | movement state time information presentation apparatus 100 is demonstrated with reference to FIGS. As shown in FIG. 2, the exercise state time information presentation device 100 includes a first time point extraction unit 110, a second time point extraction unit 120, a relative time relationship feedback presentation unit 130, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for the processing of the exercise state time information presentation device 100.

  The first camera 901 and the second camera 902 are connected to the exercise state time information presenting apparatus 100, and the images taken by the first camera 901 and the second camera 902 are input to the exercise state time information presenting apparatus 100. There is. FIG. 3 shows a state in which a first pitched camera 901 and a second pitched camera 902 are shooting a pitched ball from a left-handed pitcher. The first camera 901 takes a picture of how the forefoot touches the ground during the pitching operation. The second camera 902 takes a picture of the trunk horizontally rotating during the pitching operation.

  The first input video, which is a video captured by the first camera 901, is a processing target of the first time point extraction unit 110. Because of the processing in the first time point extraction unit 110, there are some conditions required for the first input image. First, the first camera 901 is fixed to the ground, and it is necessary to continue shooting a certain area on the surface parallel to the pitching direction and the vertical direction (hereinafter referred to as the image area of the first camera 901). is there. Further, the image captured by the first camera 901 needs to include the vicinity of the ground contact position when the front foot is stepped in the pitching direction during pitching, but includes the vicinity of the front foot position in the stationary state before the start of pitching. Not preferable. Further, it is preferable that the image captured by the first camera 901 does not include a portion having a color similar to that of the front foot (other than the front foot).

  The second input video, which is a video captured by the second camera 902, is a processing target of the second time point extraction unit 120. Because of the processing in the second time point extraction unit 120, there are some conditions required for the second input video. First, the second camera 902 is fixed to the ground, and it is necessary to continue to photograph a certain area on the surface of the second camera 902 that looks down on the ground from above (hereinafter referred to as the image area of the second camera 902). Further, the image captured by the second camera 902 needs to completely include the trunk being pitched. Furthermore, it is preferable that the image captured by the second camera 902 does not include a portion having a color similar to that of the trunk (other than the trunk).

  When the first time point extraction unit 110 extracts the time point (time point A) at which the front foot touches the ground from the first input image, and when the second time point extraction unit 120 starts horizontal rotation of the trunk from the second input image point (time point B). When extracting, the color information of the forefoot and trunk is used. Therefore, it is preferable that the first input image and the second input image include color information with resolution as high as possible, such as RGB. Further, it is preferable that the time resolution and the spatial resolution are as high as possible.

  The exercise state time information presenting apparatus 100 determines a time point A, which is a time point when the front foot is in contact with the ground, from the first input image obtained by photographing the image region of the first camera and the second input image obtained by photographing the image region of the second camera. Generates and presents relative time relationship feedback that reflects the relative time relationship at time B, which is the time when the horizontal rotation starts.

  The operation of the exercise state time information presentation device 100 will be described with reference to FIG. The first time point extraction unit 110 extracts the time point A when the front foot touches the ground from the first input image obtained by photographing the image area of the first camera (S110). The first time point extraction unit 110 generates a binary image by binarizing the image at each frame time of the first input video imaged in the image area of the first camera 901 by using the color of the forefoot as a reference to generate a binary image and adjoins it. The front foot contact event is detected based on the difference information between the binary images at the two frame times, and the time point A is extracted.

  Hereinafter, the first time point extraction unit 110 will be described with reference to FIGS. As shown in FIG. 5, the first time point extraction unit 110 includes a binary image generation unit 112 and a binary image difference extraction unit 114.

The operation of the first time point extraction unit 110 will be described with reference to FIG. The image of the first input video image at each frame time t (t = 0, 1, ...) Is C ₁ ^t (x, y). Here, (x, y) represents the coordinate position of the pixel in the image area of the first camera 901. That is, C ₁ ^t (x, y) is a p-dimensional vector representing the color of the pixel at (x, y). For example, when using RGB gradation values to represent the color of a pixel, p = 3. Further, in order to binarize the image C ₁ ^t (x, y), a reference value T ₁ of the p-dimensional vector representing the color of the forefoot and a threshold d ₁ for binarization determination are given in advance (however, d ₁ is a positive number). The reference value T ₁ and the threshold value d ₁ may be recorded in the recording unit 190 in advance, or may be input when the process of the binary image generating unit 112 is started.

The binary image generation unit 112 binarizes a binary image B obtained by binarizing the image C ₁ ^t (x, y) of the first input video at the frame time t (t = 0, 1, ...) From the first input video. ₁ ^t (x, y) is generated (S112). The operation of the binary image generation unit 112 will be described in detail. First, C ₁ in the binary image generating unit 112, the coordinate position of the frame time t (t = 0,1, ...) of the first input image in the image ^{_{C 1 t (x, y)}} (x, y) Calculate the distance || C ₁ ^t (x, y) -T ₁ || between ^t (x, y) and the reference value T ₁ that represents the color of the forefoot. To calculate the distance || C ₁ ^t (x, y) -T ₁ ||, for example, the p-dimensional Euclidean distance may be used. Next, the binary image generation unit 112 compares the distance || C ₁ ^t (x, y) -T ₁ || with the threshold value d ₁ to determine the color of the pixel at the coordinate position (x, y). A determination result of whether the color is similar to that of the forefoot is generated, and a binary image B ₁ ^t (x, y) at the frame time t is generated based on the determination result. Specifically, if || C ₁ ^t (x, y) -T ₁ || <d ₁ (similar), B ₁ ^t (x, y) = 1, otherwise (similar) If not), a binary image B ₁ ^t (x, y) is generated with B ₁ ^t (x, y) = 0. ^{_{Incidentally, || C 1 t (x,}} y) instead of _{-T 1 || <d 1, ||} C 1 t (x, y) -T 1 || ≦ d 1 may be used.

The binary image difference extraction unit 114 calculates each coordinate from the binary images B ₁ ^t-1 (x, y) and B ₁ ^t (x, y) at the adjacent frame times t-1 and t generated in S112. Frame the total sum of the absolute values of the differences between B ₁ ^t-1 (x, y) and B ₁ ^t (x, y) at position (x, y) over the entire image area (image area Ω ₁ of the first camera) calculated as an absolute value v ^t forefoot movement speed of the at time t, and extracts the point a by using the absolute value v ^t (S114). The operation of the binary image difference extraction unit 114 will be described in detail. First, binary image difference extraction unit 114, the frame time t-1 in the binary image B ₁ ^t-1 (x, y) two at frame time t value image B ₁ ^t (x, y), the formula ( According to 1), the absolute value v ^t of the moving speed of the forefoot at the frame time t (t = 1, ...) Is calculated.
However, for t = 0, v ⁰ = 0.

Next, the binary image difference extraction unit 114 uses the absolute values v ^t , v ^{t + 1} at the frame times t, ^{t + 1} to determine the frame time t satisfying the expression (2) at the frame time when the front foot touches the ground (hereinafter , T _A ) and is extracted as the time point At _A.
Here, the speed threshold v is a positive number for specifying the frame time when the front foot touches the ground. The speed threshold value v may be recorded in the recording unit 190 in advance, or may be input when the process of the binary image difference extraction unit 114 is started.

It should be noted that the equation (3) may be used in place of the equation (1) to normalize the speed value for determination.

The second time point extraction unit 120 extracts the time point B, which is the time point when the trunk starts horizontal rotation, from the second input image obtained by capturing the image area of the second camera (S120). The second time point extraction unit 120 generates a binary image by binarizing the image at each frame time of the second input video imaged in the image area of the second camera 902 by using the color of the trunk as a reference to generate a binary image. The argument, which is the direction of the trunk, is estimated from the principal component direction of the distribution of the trunk region, the rotation start event in the pitching direction is detected using the argument, and the time point B is extracted. Here, the argument is estimated as the arctangent of the slope of the main component of the distribution of the trunk region. Therefore, the rotation range of the trunk must be limited so that the arctangent function can be defined. Therefore, let r be the vector from the acromion of the right shoulder to the acromion of the left shoulder (trunk vector), and its declination θ is -π / 2 <θ <π / The direction of the second camera 902 is set so that 2 (for right hand throwing) or π / 2 <θ <3π / 2 (for left hand throwing) is satisfied (see FIGS. 7 and 8). Alternatively, after the video is taken by the second camera 902, a fixed coordinate rotation conversion may be performed on the image at each frame time. Hereinafter, the trunk vector at the frame time ^t is represented by r ^t and the argument is represented by θ ^t .

  Hereinafter, the second time point extraction unit 120 will be described with reference to FIGS. 9 to 10. As shown in FIG. 9, the second time point extraction unit 120 includes a binary image generation unit 122 and a binary image principal component extraction unit 124.

The operation of the second time point extraction unit 120 will be described with reference to FIG. The image of the second input video image at each frame time t (t = 0, 1, ...) Is C ₂ ^t (x, y). Here, (x, y) represents the coordinate position of the pixel in the image area of the second camera 902. That is, C ₂ ^t (x, y) is a p-dimensional vector representing the color of the pixel at (x, y). For example, when using RGB gradation values to represent the color of a pixel, p = 3. Further, in order to binarize the image C ₂ ^t (x, y), the reference value T ₂ of the p-dimensional vector representing the color of the trunk and the threshold d ₂ for binarization determination are given in advance (however, d ₂ is a positive number). The reference value T ₂ and the threshold value d ₂ may be recorded in the recording unit 190 in advance, or may be input when the processing of the binary image generation unit 122 is started.

The binary image generation unit 122 binarizes a binary image B obtained by binarizing the image C ₂ ^t (x, y) of the second input video at the frame time t (t = 0, 1, ...) From the second input video. ₂ ^t (x, y) is generated (S122). The operation of the binary image generation unit 122 will be described in detail. First, the binary image generating unit 122, C ₂ at each coordinate position of the frame time t (t = 0,1, ...) of the second input image of the image ^{_{C 2 t (x, y)}} (x, y) Calculate the distance || C ₂ ^t (x, y) -T ₂ || between ^t (x, y) and the reference value T ₂ that represents the color of the trunk. To calculate the distance || C ₂ ^t (x, y) -T ₂ ||, for example, the p-dimensional Euclidean distance may be used. Next, the binary image generation unit 122 compares the distance || C ₂ ^t (x, y) -T ₂ || with the threshold d ₂ to determine the color of the pixel at the coordinate position (x, y). A determination result as to whether the color is similar to the trunk color is generated, and a binary image B ₂ ^t (x, y) at the frame time t is generated based on the determination result. Specifically, if || C ₂ ^t (x, y) -T ₂ || <d ₂ (similar), B ₂ ^t (x, y) = 1, otherwise (similar) If not), a binary image B ₂ ^t (x, y) is generated with B ₂ ^t (x, y) = 0. ^{_{Incidentally, || C 2 t (x,}} y) -T 2 || < instead of ^{_{d 2, || C 2 t (}} x, y) may be used -T ₂ || ≦ d _2.

The binary image principal component extraction unit 124 uses the binary image B ₂ ^t (x, y) at the frame time t generated in S122 to calculate all pixels (that is, B ₂ ^t (x, y) = (X, y)) x coordinate vector X ^t = (x ₁ ^t ,…, x _i ^t ,…, x _Nt ^t ) that satisfies ₁ and y coordinate vector Y ^t = (y ₁ ^t ,…, y _i ^t , ..., y _Nt ^t ) (1 ≦ i ≦ Nt, i is an integer, Nt is the total number of pixels in the trunk region at the frame time t), the eigenvector e ^t = (e _x ^t , e _y ^The arctangent of the slope e _y ^t / e _x ^t of ^t ) is calculated as the argument θ ^t of the trunk vector r ^t at the frame time t, and the time B is extracted using the argument θ ^t (S124). The operation of the binary image principal component extraction unit 124 will be described in detail. First, the binary image principal component extraction unit 124 sets (that is, B ₂ ^t (x, y) = 1 of all pixels in the torso region from the binary image B ₂ ^t (x, y) at the frame time t. (X, y)) x-coordinate vector X ^t = (x ₁ ^t ,…, x _i ^t ,…, x _Nt ^t ) and y-coordinate vector Y ^t = (y ₁ ^t ,…, y _i ^t , ..., y _Nt ^t ) (1 ≦ i ≦ Nt, i is an integer, Nt is the total number of pixels in the trunk region at the frame time t) is extracted. Next, the binary image composed mainly extracting unit 124, X ^t, eigenvectors of the covariance matrix of _{^{Y t e t = (e x}} t, e y t) calculated, eigenvector e ^t calculated by the equation (4) The arc tangent of the slope e _y ^t / e _x ^t of is calculated as the argument θ ^t of the trunk vector r ^t at the frame time t (t = 0, ...).

Finally, the binary image principal component extraction unit 124 uses the argument angles θ ^t and θ ^{t + 1} at the frame times t and ^{t + 1} to calculate the frame time t that satisfies the equation (5R) in the case of right hand throwing. frame time t satisfying the formula (5L) in the case of throwing the left hand, a frame time of horizontal rotation starts of the trunk (hereinafter referred to as t _B) is determined to be, it extracts a time Bt _B.

Here, the declination threshold Θ is a number for specifying the rotation start frame time in the pitching direction of the trunk. In the case of right hand throwing, -π / 2 <Θ <π / 2, and in the case of left hand throwing, It is a number that satisfies π / 2 <Θ <3π / 2. The argument angle threshold Θ may be recorded in the recording unit 190 in advance, or may be input at the time when the process of the binary image principal component extracting unit 124 is started.

Incidentally, for calculating the deflection angle theta ^t by the above-mentioned method is the fact that the inclination of the principal components of the distribution of the gradient e _y ^t / e _x ^t and the trunk area of the eigenvector e ^t match.

  The relative time relation feedback presenting section 130 generates and presents relative time relation feedback reflecting the relative time relation between the time points A and B from the time point A extracted in S110 and the time point B extracted in S120. (S130). For example, sounds can be used to present relative time-related feedback.

  Hereinafter, two examples of the method of presenting the relative time relation feedback as a sound will be described.

(Method 1) Relative time-related feedback is presented by sounding two types of distinguishable sounds s _A and s _B , separated by a time interval | t _B -t _A | between time points At _A and Bt _B. To do. At that time, the sound s _A, presentation order of s _B is, t _A <t _B if s _A, the order of s _B, t _B <t _A if s _B, the order of s _A. When t _A = t _B , the sounds s _A and s _B are presented at the same time. Here, the time length of each of the sounds s _A and s _B may be the same as or different from each other, and is made as short as possible within the audible range. As examples of the sounds s _A and s _B , (1) periodic sounds having fundamental frequencies different from each other so that they can be distinguished from each other, (2) non-periodic sounds having frequency components different from each other so that they can be distinguished from each other, and (3) one of them is a periodic sound. It is conceivable that the other is a sound and the other is an aperiodic sound.

(Method 2) _A sweep sound or a chirp sound with a time length | t _B -t _A | at which the fundamental frequency smoothly increases or decreases monotonically is presented as relative time-related feedback. At that time, the direction of change of the fundamental frequency is two fundamental frequencies f _A and f _B which are distinguishably different from each other. If t _A <t _B , the change from f _A to f _B, and if t _B <t _A, For example, change from f _B to f _A. Also, when t _A = t _B , no sound is presented.

Regarding (Method 1), when the time required for the processing in the first time point extraction unit 110 and the time required for the processing in the second time point extraction unit 120 match, the sound s is immediately output when t _A is obtained. _{It is also possible} to present _{A and} to present sound s _B immediately when t _B is obtained. On the other hand, in (method 2), it is necessary to present a sweep sound or the like after t _A and t _B are extracted.

  Further, the method of presenting the relative time relationship feedback is not limited to sound, and any method that can be perceived by a person such as audiovisual sense may be used. For example, light, skin vibration, or the like can be used.

  Regarding S110 and S120, it is not always necessary to execute S110 and S120 in this order. For example, S120 and S110 may be executed in this order. Further, in order to present the relative time relation feedback in real time, it is preferable to execute S110 and S120 in parallel in time.

  According to the invention of this embodiment, by extracting the front foot contact point and the trunk horizontal rotation start point from two types of video images of the learner's pitching operation during pitching training, a series of pitching operations can be performed. It is possible to present, as feedback, the relative time relationship at the time when an important motor state occurs. Further, by presenting the relative time relationship by sound, the learner and the instructor can hear the sound and grasp the time sequence and time interval in which an important exercise state occurs. As a result, the instructor teaches the correction of the pitching motion, and the learner voluntarily corrects the pitching motion, so that effective training for increasing the ball speed can be performed.

The ideal kinematic chain in pitching motion is that multiple motion states occur one after another in an appropriate sequence within a very short time of less than 1 second. Although it is difficult for a person to accurately recognize the occurrence order and time intervals of a plurality of exercise states, according to the invention of the present embodiment, the learner and the instructor have a plurality of condensed states in such a short time. It is possible to accurately recognize the occurrence sequence and time interval of the motor state of the. In particular, the use of sound for presenting time information relating to the exercise state is extremely effective when considering the cognitive processing ability of human time information. Among the sensory modalities capable of transmitting time information such as hearing, sight, and skin sensation, hearing is particularly excellent in terms of temporal resolution (Reference Non-Patent Document 1), so that presentation by sound is very close in time. It is considered to be the most suitable for the process of accurately grasping the occurrence times of multiple events that occur.
(Reference Non-Patent Document 1) LE Humes, TA Busey, JC Craig, D. Kewley-Port, “The effects of age on sensory thresholds and temporal gap detection in hearing, vision, and touch”, Atten Percept Psychophys, Vol.71. , No.4, pp.860-871, 2009.

<Second embodiment>
In the first embodiment, by extracting from the two types of images two points of time, which are the important motion states during the pitching motion, that is, the forefoot ground contact and the start of the horizontal trunk rotation, a relative guideline for correcting the pitching motion is obtained. Although the time-related feedback is presented so that the pitching training can be performed, the pitching training may be performed while confirming the ball speed measured by the ball speed measuring device. Therefore, in the second embodiment, an exercise state time information presenting apparatus that also inputs the ball speed measured by the ball speed measuring device will be described.

  The exercise state time information presentation device 200 will be described with reference to FIGS. 11 to 13. As shown in FIG. 11, the exercise state time information presentation device 200 includes a first time point extraction unit 110, a second time point extraction unit 120, a relative time relationship feedback presentation unit 130, an optimum time interval estimation unit 240, and a recording unit 190. . The recording unit 190 is a component that appropriately records information necessary for the processing of the exercise state time information presentation device 200.

  The first camera 901 and the second camera 902 are connected to the exercise state time information presenting apparatus 200, and the images taken by the first camera 901 and the second camera 902 are input to the exercise state time information presenting apparatus 200. There is. The requirements and features for the first camera 901, the second camera 902, the first input image, and the second input image are the same as those of the first embodiment.

  The ball speed measuring device 903 is connected to the exercise state time information presenting device 200, and the ball speed measured by the ball speed measuring device 903 is input to the exercise state time information presenting device 200 (see FIG. 12). It is assumed that the ball speed measuring device 903 outputs the speed when detecting an object moving at a speed equal to or higher than a certain threshold, and does not output the speed at other times. Further, the ball speed measuring device 903 appropriately sets the time interval for measuring the speed of the object so that only one output is made at most per pitch.

  The exercise state time information presentation device 200 determines the time point A, which is the time point when the front foot touches the ground, from the first input video imaged of the image area of the first camera, the second input video imaged of the image area of the second camera, and the ball speed. It is a time interval between time A and time B at which the ball speed is expected to be maximized at the same time as generating relative time relationship feedback that reflects the relative time relationship at time B when the core starts horizontal rotation. Estimate and present the optimal time interval.

The operation of the exercise state time information presentation device 200 will be described with reference to FIG. The first time point extraction unit 110 extracts the time point A when the front foot touches the ground from the first input image obtained by photographing the image area of the first camera (S110). The second time point extraction unit 120 extracts the time point B, which is the time point when the trunk starts horizontal rotation, from the second input image obtained by capturing the image area of the second camera (S120). The relative time relation feedback presenting section 130 generates and presents relative time relation feedback reflecting the relative time relation between the time points A and B from the time point A extracted in S110 and the time point B extracted in S120. (S130). The processing from S110 to S130 is the same as that of the exercise state time information presentation device 100. Hereinafter, the time point A is t _A and the time point B is t _B.

The optimum time interval estimation unit 240 determines the time interval between the time A and the time B at which the ball speed is expected to be the maximum from the time At _A extracted in S110, the time Bt _B extracted in S120, and the ball speed measured by the ball speed measuring device 903. Then, the optimum time interval Δt _best is estimated (S240). The specific processing of the optimum time interval estimation unit 240 is as follows. First, the optimum time interval estimation unit 240 inputs the _ball speed v _ball (k) in the k-th pitching motion at the time At _A , which is the time when the front foot touches the ground in the k-th pitching motion, and the horizontal trunk start time. The time interval Δt (k) = t _B -t _A is calculated from a certain time point Bt _B. Next, the optimal time interval estimation unit 240 sets c ₁ , c ₂ , and c ₃ as constants, and the time interval vector T _k = [Δt (1), ..., Δt (k)] having time intervals as elements and its A square time interval vector S _k = [Δt (1) ² , ..., Δt (k) ² ], each of which is the square of each element, and a unit number vector E = [1 that has 1 as the number of units ,…, 1], between the ball velocity vector V _k = [v _ball (1),…, v _ball (k)] with the ball velocity as an element,

Assuming that the relationship of

Estimate c ₁ , c ₂ , and c ₃ that minimize (where 'represents the transpose of the vector). here,

Is. The least squares method or the like can be used to estimate c ₁ , c ₂ , and c ₃ . Also, the time interval vector T _k-1 = [Δt (1),…, Δt (k-1)] and the ball velocity vector V _k-1 = [v _ball (1), , V _ball (k-1)] is recorded in the recording unit 190. Then, if c ₁ <0, the quadratic function f (x) = c ₁ x ² + c ₂ x + c ₃ is convex upward, and its maximum value is given by x = -c ₂ / 2c ₁ . To be On the other hand, when c ₁ ≧ 0, the maximum value of the quadratic function f (x) cannot be defined. Therefore, when c ₁ <0, the optimum time interval estimation unit 240 estimates the optimum time interval Δt _{best at} which the ball speed is expected to be maximum as Δt _best = -c ₂ / 2c ₁ , while c ₁ ≧ 0 At this time, the optimal time interval Δt _best is estimated to be indefinite.

The method of presenting the optimum time interval Δt _best may be the same as that of the relative time relationship feedback presenting unit 130. That is, sound, light, skin vibration, etc. can be used.

When it is not necessary to present the optimum time interval Δt _best for each pitching, the presentation timing may be limited to, for example, when the user gives an instruction for presentation from the outside. When presenting the optimum time interval Δt _best for each pitching, the relative time relation feedback presenting unit 130 presents the relative time relation feedback for each pitch, and then automatically presents it after a few seconds. You may go by the method.

  According to the invention of the present embodiment, a series of pitching motions are extracted by extracting the front foot contact point and the trunk horizontal rotation start point from two types of video images of the learner's pitching motions during pitching training. It is possible to present the relative time relationship at the time when different important motor states occur as feedback. Further, by also acquiring the data regarding the ball speed, it becomes possible to present the optimum time interval which is the time interval between the time A and the time B at which the ball speed is expected to be the maximum. It is expected that this will enable more effective training.

<Modification>
It is needless to say that the present invention is not limited to the above-described embodiment and can be appropriately modified without departing from the spirit of the present invention. The various kinds of processing described in the above embodiments may be executed not only in time series according to the order described, but also in parallel or individually according to the processing capability of the device that executes the processing or the need.

<Additional notes>
The device of the present invention is, for example, as a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating with the outside of the hardware entity. Connectable communication unit, CPU (Central Processing Unit, may include a cache memory or register, etc.), RAM or ROM as memory, external storage device as hard disk, and their input unit, output unit, communication unit , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged among external storage devices. If necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above-described functions and data necessary for processing of this program (not limited to the external storage device, for example, the program is read). It may be stored in a ROM that is a dedicated storage device). In addition, data and the like obtained by the processing of these programs are appropriately stored in the RAM, the external storage device, or the like.

  In the hardware entity, each program stored in an external storage device (or ROM, etc.) and data necessary for the processing of each program are read into the memory as necessary, and interpreted and executed / processed by the CPU as appropriate. . As a result, the CPU realizes a predetermined function (each constituent element represented by the above, ... Unit, ... Means, etc.).

  The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention. Further, the processes described in the above embodiments are not only executed in time series in the order described, but may be executed in parallel or individually according to the processing capability of the device that executes the processes or as necessary. .

  As described above, when the processing functions of the hardware entity (the device of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on the computer, the processing functions of the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded in a computer-readable recording medium. The computer-readable recording medium may be any recording medium such as a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape or the like is used as a magnetic recording device, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), or a CD-ROM (Compact Disc Read Only) is used as an optical disc. Memory), CD-R (Recordable) / RW (ReWritable), MO (Magneto-Optical disc) as a magneto-optical recording medium, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) as a semiconductor memory, etc. Can be used.

  The distribution of this program is performed by selling, transferring, or lending a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, the program may be stored in a storage device of a server computer and transferred from the server computer to another computer via a network to distribute the program.

  A computer that executes such a program first stores, for example, the program recorded on a portable recording medium or the program transferred from the server computer in its own storage device. Then, when executing the processing, this computer reads the program stored in its own recording medium and executes the processing according to the read program. As another execution form of this program, a computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to this computer. Each time, the processing according to the received program may be sequentially executed. Further, the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by executing the execution instruction and acquiring the result without transferring the program from the server computer to the computer. May be It should be noted that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to a computer but has the property of defining computer processing).

  Further, in this embodiment, the hardware entity is configured by executing a predetermined program on the computer, but at least a part of these processing contents may be implemented by hardware.

Claims (8)

The first camera is a camera for photographing the forefoot landing in the pitching motion, and a certain area on a plane parallel to the pitching direction and the vertical direction photographed by the first camera is an image area of the first camera, and pitching. The second camera is a camera for photographing the state in which the trunk is horizontally rotated during the movement, and a certain region on the surface of the second camera which looks down on the ground from above the head is an image region of the second camera,
A first time point extraction unit that extracts a time point A, which is a time point when the front foot touches the ground, from a first input image obtained by photographing the image area of the first camera;
A second time point extraction unit that extracts a time point B, which is a time point at which the trunk starts horizontal rotation, from the second input image obtained by photographing the image area of the second camera;
Motion state time information including a relative time relationship feedback presenting section for generating and presenting relative time relationship feedback reflecting the relative time relationship between the time points A and B from the time points A and B. Presentation device. The exercise state time information presentation device according to claim 1,
The first time point extraction unit is
From the first input image, an image C ₁ ^t (x, y) of the first input image at a frame time t (t = 0,1, ...) (where (x, y) is the image area of the first camera) A binary image generation unit that generates a binary image B ₁ ^t (x, y) by binarizing
Frame time t (t = 1, ...) Calculated by the following formula from binary images B ₁ ^t-1 (x, y) and B ₁ ^t (x, y) at adjacent frame times t-1 and t the absolute value v ^t of the forefoot movement speed of at
(However, for t = 0, v ⁰ = 0) is used, and a motion state time information presentation device including a binary image difference extraction unit that extracts the time point A. The exercise state time information presentation device according to claim 1 or 2,
Δt (k) is the time interval between time A and time B in the kth pitching motion, v _ball (k) is the ball speed in the kth pitching motion measured by the ball speed measuring device,
Using the time intervals Δt (1),…, Δt (k) and the ball speeds v _ball (1),…, v _ball (k), the time interval between the time A and the time B at which the ball speed is expected to be the maximum is determined. A motion state time information presentation device including an optimum time interval estimation unit that estimates a certain optimum time interval. The exercise state time information presentation device according to any one of claims 1 to 3,
The relative time relationship feedback presenting unit,
A motion state time information presentation device, wherein the relative time relation feedback is generated using sound. The first camera is a camera for photographing the forefoot landing in the pitching motion, and a certain area on a plane parallel to the pitching direction and the vertical direction photographed by the first camera is an image area of the first camera, and pitching. The second camera is a camera for photographing the state in which the trunk is horizontally rotated during the movement, and a certain region on the surface of the second camera which looks down on the ground from above the head is an image region of the second camera,
A first time point extraction step in which the exercise state time information presentation device extracts a time point A at which the front foot touches the ground, from a first input image obtained by photographing the image area of the first camera;
A second time point extraction step in which the exercise state time information presentation device extracts a time point B, which is a time point at which the trunk starts horizontal rotation, from a second input image obtained by photographing the image area of the second camera;
Relative time relationship feedback that the exercise state time information presenting device generates and presents relative time relationship feedback reflecting the relative time relationship between the time points A and B from the time points A and B. A method of presenting motion state time information, including a presenting step. The exercise state time information presentation method according to claim 5,
The first time point extraction step includes
From the first input image, an image C ₁ ^t (x, y) of the first input image at a frame time t (t = 0,1, ...) (where (x, y) is the image area of the first camera) (Representing the coordinate position of the pixel in) is a binary image generating step of generating a binary image B ₁ ^t (x, y),
Frame time t (t = 1, ...) Calculated by the following formula from binary images B ₁ ^t-1 (x, y) and B ₁ ^t (x, y) at adjacent frame times t-1 and t the absolute value v ^t of the forefoot movement speed of at
(However, for t = 0, v ⁰ = 0) is used to extract the time point A, and a binary image difference extraction step is included. The exercise state time information presentation method according to claim 5 or 6,
Δt (k) is the time interval between time A and time B in the kth pitching motion, v _ball (k) is the ball speed in the kth pitching motion measured by the ball speed measuring device,
It is expected that the motion state time information presenting device uses the time interval Δt (1), ..., Δt (k) and the ball speed v _ball (1), ..., v _ball (k) to maximize the ball speed. An optimal time interval estimation step of estimating an optimum time interval which is a time interval between time A and time B.   A program for causing a computer to function as the exercise state time information presentation device according to any one of claims 1 to 4.