JP2012094130A - Operation analyzing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation analyzing device which is easily usable for a user.SOLUTION: This operation analyzing device is configured to, when receiving the input of a moving image (S11), display a still image at an operation part (S12). The operation analyzing device is configured to, when an arbitrary part on the still image is designated by a user (S13), retrieve a first feature point in a range having a radius r with the designated point as a center (S14), and to retrieve a second feature point corresponding to the first feature point in adjacent still images temporally posterior to the still image whose first feature point has been retrieved (S15), and to calculate a difference between those feature points (S16), and to calculate the average value of the calculated differences (S17), and to calculate the difference between the feature points for all the adjacent still images of a plurality of continuous still images, and to calculate the average value of the calculated differences (S18: YES), and to successively connect those calculated average values, and to create an orbit in this way (S19).

Description

  The present invention relates to a motion analysis device, and more particularly to a motion analysis device capable of analyzing a motion at a specific location.

  For example, Japanese Unexamined Patent Application Publication No. 2009-20897 (Patent Document 1) discloses a technique related to a golf swing motion analysis. According to Patent Document 1, the position of a specific area is detected for each piece of time-series object image data, and the locus of the specific area is displayed with lines and points by tracking the change in the position of the detected specific area. It is said.

  Specifically, by storing the color of the specific area in advance, areas having the same color as the stored color are detected from the time-series object image data, and the change in the position of the specific area is tracked. Further, the position of the specific area is designated by a manual operation by the user, and the change in the position of the specific area is traced based on the color of the designated area.

JP 2009-20897 A (paragraph numbers 0071 to 0075, etc.)

  However, in the technique disclosed in Patent Document 1, it is necessary to previously store the color of the specific area in order to detect the position of the specific area. In this case, for example, when a color different from the color stored in advance needs to be used quickly, it is necessary to register the color again, which is complicated for the user.

  Further, even when the position of the specific area is designated by the user's manual operation, the change in the position of the specific area is tracked based on the color of the designated area. When it is desired to specify the position of the specific area, it is necessary to use a different color in each of the specific areas.

  An object of the present invention is to provide an operation analysis apparatus that can be easily used by a user.

  The motion analysis apparatus according to the present invention analyzes the motion of a moving image composed of a plurality of continuous still images. The motion analysis device includes a moving image receiving unit that receives an input of a moving image, and an analysis that specifies an analysis target location on the first still image in the first still image constituting the moving image out of the moving images received by the moving image receiving unit. A location specifying means, a first search means for searching for a first feature point within a certain range from the analysis target location, and a second still image that constitutes a moving image and is a still image different from the first still image , A second search means for searching for a second feature point corresponding to the first feature point, and a trajectory creation means for creating a trajectory based on the first and second feature points.

  By doing so, the motion analysis apparatus designates the analysis target location on the first still image, and searches for the first feature point in the first still image and the second feature point in the second still image. To do. Then, a trajectory is created based on the first and second feature points. Therefore, it is possible for the user to analyze the motion of the moving image by specifying the analysis target portion and creating the locus without considering the color. As a result, the user can use it easily.

  Preferably, the analysis location specifying means includes a specification receiving means for receiving a specification from the user so that the user can arbitrarily specify the analysis target location. By doing so, the user can arbitrarily designate a location for analyzing the motion. Thereby, the convenience for the user can be further improved.

  Preferably, the analysis location specifying means includes a recognition means for recognizing the observation target from the first still image, and a form estimation means for estimating the form of the observation target based on the observation target image data recognized by the recognition means; Have Here, the analysis target location is specified based on the image data estimated by the form estimation means. In this way, when starting the motion analysis, the motion analysis device can estimate the initial form of the observation target and automatically determine the initial conditions for the analysis based on the data. Thereby, since the user can omit the operation of specifying the analysis target portion, it is possible to provide an operation analysis device that further improves the convenience for the user.

  Preferably, the recognizing unit includes a target region demarcating unit that demarcates a certain region including the observation target as a target region, and a target region normalizing unit that adjusts the size of the target region to a predetermined image size. . By doing so, it is possible to uniformly perform the morphological estimation and motion analysis of the observation target without being affected by the position and size of the observation target in the first still image.

  Preferably, the recognizing unit detects the contour of the observation target based on the luminance gradient formed in the first still image. By doing so, since the contour of the observation target can be performed with high accuracy, the shape estimation and motion analysis of the observation target can be performed with higher accuracy.

  Preferably, the image processing apparatus further includes a reference image storage unit that stores in advance a plurality of comparison target image data corresponding to the observation target, and the form estimation unit includes the observation target image data recognized by the recognition unit and the reference image storage unit. Reference image selecting means for comparing the stored image data to be compared and selecting image data to be compared whose form approximates the image data to be observed is included. Here, the analysis target portion is specified based on the comparison target image data selected by the reference image selection means. By doing so, it is possible to automatically estimate the form of the observation target and set the analysis target location on the estimated image data. That is, the initial condition of the analysis location at the start point of the motion analysis can be set more reliably and with high accuracy without requiring designation by the user. Thereby, the convenience for the user can be improved and the accuracy of the operation analysis can be improved.

  Preferably, the first search unit searches for a plurality of first feature points within a certain range, and the second search unit stores a plurality of second feature points corresponding to the plurality of first feature points. The trajectory creation means calculates the difference between the plurality of first feature points and the plurality of second feature points, respectively, and obtains the average value of the calculated differences, thereby obtaining the average value. To create a trajectory. In this way, a trajectory can be created based on the average value of differences between a plurality of feature points. Therefore, for example, even if the analysis target location recognized by the motion analysis device is slightly deviated from the location intended by the user, the degree of deviation can be suppressed. As a result, the accuracy of the created trajectory can be improved.

  More preferably, the average value is a point based on the first and second feature points and indicates a change at a predetermined point on the still image constituting the moving image. The trajectory is created by sequentially calculating between adjacent still images of a plurality of continuous still images and sequentially connecting predetermined points indicated by the calculated average value.

  More preferably, the first still image includes a still image located at the beginning of the movie and a still image located at the end of the movie, and the second still image is time-sequentially with the first still image. The second search means includes a first feature point in a still image that is adjacent in time series, and a still image that is adjacent in time series before the first still image. And a second feature point corresponding to the first feature point in a still image adjacent in time series. By doing this, for example, when searching for feature points in order from the front to the back in time series in a plurality of continuous still images, it is no longer possible to search for feature points on the way Even in this case, the feature points can be searched in reverse order from the rear to the front in time series. As a result, the feature point can be reliably searched.

  More preferably, the second search means includes acquisition means for acquiring the speed of motion of the moving image, and searches for the second feature point based on the speed acquired by the acquisition means. In this way, the search for the second feature point can be changed according to the moving speed of the moving image. As a result, the second feature point can be easily searched.

  As one embodiment, a plurality of continuous still images are configured at regular intervals, and the acquisition means acquires the magnitude of the difference between the still images between adjacent still images, thereby speeding up the motion of the moving image. To get.

  Preferably, the trajectory creation unit includes a direction change identification unit that can identify a trajectory before the direction change and a trajectory after the direction change when the direction of the trajectory is changed. By doing this, it becomes possible to clearly grasp the point that becomes the turning point of the operation and the trajectory of the movement before and after that, so that more detailed operation analysis can be easily performed.

  Preferably, the direction change identification unit calculates a velocity vector of a trajectory between still images that are continuous in time series, and a speed vector in a specific still image and a still image that is adjacent to the specific still image after time series. When the inner product with the velocity vector in the image becomes negative, the direction change of the trajectory is detected. By doing so, it becomes possible to detect the point that becomes the turning point of the operation with higher accuracy.

  More preferably, it includes display means for displaying the trajectory created by the trajectory creating means and the moving picture accepted by the moving picture accepting means in an overlapping manner. By doing so, the trajectory and the moving image can be displayed in an overlapping manner, and the analysis result of the moving image operation can be easily understood by the user.

  Such a motion analysis apparatus designates an analysis target location on the first still image, and searches for the first feature point in the first still image and the second feature point in the second still image. Then, a trajectory is created based on the first and second feature points. Therefore, it is possible for the user to analyze the motion of the moving image by specifying the analysis target portion and creating the locus without considering the color. As a result, the user can use it easily.

It is a block diagram which shows the structure of the operation | movement analysis apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows operation | movement of the operation | movement analysis apparatus in the case of analyzing the form of a golf swing. It is a figure which shows a part of several continuous still images. It is a figure which shows a part of several continuous still images, Comprising: It is a still image which shows the next state of FIG. It is a figure which shows a part of several continuous still images, Comprising: It is a still image which shows the next state of FIG. In the still image in FIG. 3 is a diagram illustrating the position of the specified P _0. In the still image of FIG. 3 shows a search feature point _{P 1,} the position of P 2, _{P 3.} FIG. 5 is a diagram illustrating the positions of searched feature points P ₁ ′, P ₂ ′, and P ₃ ′ in the still image of FIG. 4. FIG. 4 is a diagram illustrating a state in which a trajectory is displayed superimposed on the still image of FIG. 3. It is a figure which shows the other example of several continuous still images. It is a figure which shows the other example of several continuous still images, Comprising: It is a still image which shows the next state of FIG. It is a figure which shows the other example of several continuous still images, Comprising: It is a still image which shows the next state of FIG. It is a flowchart which shows operation | movement of the operation | movement analysis apparatus which concerns on other embodiment of this invention. It is a figure which shows the first still picture in time series among several continuous still pictures. FIG. 15 is an enlarged view of the target area shown in FIG. 14. The figure which extracted the outline of the player shown in FIG. 15 is shown. It is a figure which shows an example of the reference image used as comparison object. FIG. 17 is a reference image shown in FIG. 17, wherein (a) represents a state in which first body part regions corresponding to the head, arms, and legs are defined on the image; It is the figure which extracted the outline of the reference player, (c) is a figure showing a mode that only the 1st body part area | region shown to (a) was extracted. It is a figure showing a mode that the 1st body part area | region corresponding to the selected reference image was applied to the image data of observation object. It is a figure which shows the image which adjoins after the time series from the image shown in FIG. 19, Comprising: It is a figure corresponding to FIG. It is a figure which shows the image which adjoins after the time series from the image shown in FIG. 20, Comprising: It is a figure corresponding to FIG. It is the figure which represented the moving image which recorded a series of golf swings with the still image of a total of 6 frames. It is a figure showing the locus | trajectory of the golf swing from which the takeback and the downswing became distinguishable by the direction change identification means. The figure which formed the locus | trajectory of two golf swings and displayed them in parallel is shown. It is a flowchart which shows operation | movement of the operation | movement comparison means which concerns on this invention. The graph which concerns on the frame number N and the difference value x of a 1st and 2nd locus | trajectory is shown.

  Hereinafter, an operation analysis apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a motion analysis apparatus 10 according to an embodiment of the present invention. Referring to FIG. 1, the motion analysis device 10 can analyze the motion of a moving image, and includes a control unit 11 such as a CPU that controls the entire motion analysis device 10, a storage unit 12 that stores images and the like, and a touch panel. An operation unit 13 that receives an operation from a user and displays an image or the like stored in the storage unit 12 and an I / F unit 14 that is a connection interface with an external device. In the present embodiment, the image includes a moving image and a still image.

  Here, a case where the motion analysis device 10 analyzes a golf swing form will be described. FIG. 2 is a flowchart showing the movement of the motion analysis apparatus 10 when analyzing a golf swing form. A description will be given with reference to FIGS.

  First, the user has previously photographed a golf swing form from the address to the finish with a video camera or the like, and the motion analysis apparatus 10 accepts input of the photographed moving image via the I / F unit 14 (FIG. 2, step S11, the following steps are omitted). The moving image is composed of a plurality of continuous still images, and the plurality of still images are at regular intervals. 3 to 5 are diagrams showing a part of a plurality of continuous still images. FIGS. 3 to 5 sequentially show the state from the address immediately after the start of the swing to the takeback among golf swings from the address to the finish. Here, the I / F unit 14 operates as a moving image receiving unit. The motion analysis apparatus 10 displays the still image of FIG. 3 positioned at the head of the continuous still images on the operation unit 13 (S12). Here, the still image of FIG. 3 is a first still image.

Then, the motion analysis apparatus 10 designates an arbitrary location on the still image from the user as an analysis target location to be a motion analysis target (S13). The operation unit 13 is a touch panel type, and an arbitrary portion is designated by touching the screen with a finger or the like. Here, P ₀ is designated as an arbitrary location. 6 shows, in the still image in FIG. 3 is a diagram illustrating the position of the specified P _0. Here, the operation unit 13 operates as a designation receiving unit.

Then, the motion analysis device 10 is centered and P ₀ is the designated point, within the radius r a predetermined range, the search for the first characteristic point performed (S14). A feature point is a point that indicates a location where the luminance changes greatly in a still image, an edge of an object, or the like, and is a point that has some change in a still image. Here, a plurality of first feature points P ₁ , P ₂ , and P ₃ are searched in the still image of FIG.

Note that a feature point search can use a SIFT (Scale Invariant Feature Transform) feature. The SIFT feature value is calculated by smoothing a partial still image within a radius r centered on the designated point P ₀ and generating a DoG (Difference of Gaussian) image from the smoothed image. Then, extreme values are searched for in the DoG image. Then, the feature amount is calculated based on the luminance gradient and angle of the pixels around the extreme value. The SIFT feature quantity is not affected by the rotation of the object, the enlargement / reduction of the object, or the like. Therefore, the golf swing which is a rotating operation as in this embodiment can be appropriately used without any error. FIG. 7 is a diagram illustrating the positions of the searched first feature points P ₁ , P ₂ , and P ₃ in the still image of FIG. Here, the control unit 11 operates as a first search means.

Then, the motion analysis apparatus 10 determines the first feature point P ₁ , P ₂ , P ₃ in the still image adjacent in time series after the still image in FIG. 3, that is, the still image in FIG. The second feature point corresponding to the one feature point P ₁ , P ₂ , P ₃ is searched (S15). Here, “corresponding” means having values similar to the first feature points P ₁ , P ₂ , and P ₃ . In the search, for example, the coordinates of the first feature points P ₁ , P ₂ , P ₃ in the still image of FIG. ₃ are applied to the still image of FIG. 4 and within a predetermined range centered on the position of the coordinates. The second feature points corresponding to the first feature points P ₁ , P ₂ , P ₃ are searched. FIG. 8 is a diagram showing the positions of the searched second feature points P ₁ ′, P ₂ ′, and P ₃ ′ in the still image of FIG. P ₁ 'corresponds to _{P 1, P 2'} corresponds to _{P 2, P 3} 'corresponds to _{P 3.} Here, the still image of FIG. 4 is a second still image, and the control unit 11 operates as a second search unit.

Then, the difference between the feature points P ₁ , P ₂ and P ₃ shown in FIG. 7 and the feature points P ₁ ′, P ₂ ′ and P ₃ ′ shown in FIG. 8 is calculated (S16). ΔP ₁ is calculated from P ₁ ′ −P ₁ = ΔP ₁ . That is, the change amount of the feature point between adjacent still images is calculated. P ₂ ', _{P 3'} is also calculated in the same manner _in, 'calculates the [Delta] P _{2 than-P 2 = ΔP 2, P} 3' P 2 calculates the [Delta] P _{3 from-P 3} = _{ΔP 3.} Then, three average values ΔP _t1 of the calculated differences ΔP ₁ , ΔP ₂ and ΔP ₃ are obtained (S17). The average value ΔP _t1 indicates a change at a predetermined point on the still image that is a point based on the first and second feature points.

Similarly, in the still image adjacent in time series after the still image of FIG. 4 searched for the second feature points P ₁ ′, P ₂ ′, and P ₃ ′, that is, the still image of FIG. the point _P 1 _', P 2', _{P 3} 'to the corresponding feature point _{P 1'', P 2'',} performs a search of the _P 3'' (not shown), the feature point _P 1', _{P 2} ' , P ₃ ′ and the feature points P _{1 ″} , P _{2 ″} , P _{3 ″} are calculated, and an average value ΔP _t2 of the calculated differences is obtained. In this case, the feature point _{P 1 ', P 2',} P 3 ' becomes the first feature points, the feature point _{P 1'', P 2'',} P 3'' is the second aspect.

As described above, the difference between the feature points is calculated for all the adjacent still images of the plurality of continuous still images constituting the moving image received in S11, and the average value ΔP _tn (n = 1) of the calculated differences is calculated. , 2...) In order (NO in S18).

Then, the difference between the feature points is calculated for all the adjacent still images of the plurality of continuous still images, and the average value ΔP _tn (n = 1, 2,...) Of the calculated difference is obtained (S18). And YES), the obtained average value ΔP _tn is connected in order. That is, predetermined points are connected in order. In this way, a trajectory is created based on the average value ΔP _tn (S19). The average value ΔP _tn is a part of the trajectory and becomes the entire trajectory by connecting. FIG. 9 is a diagram illustrating a state in which the trajectory 20 is superimposed on the still image of FIG. In FIG. 9, a part of the trajectory 20 is shown. Here, the control unit 11 operates as a trajectory creation unit. In this way, the motion analysis device 10 analyzes the motion by creating the trajectory 20.

  As described above, when the motion analysis apparatus 10 receives designation of an arbitrary location on the first still image as the analysis target location, the first feature point in the first still image and the second feature in the second still image. Search for feature points. Then, a trajectory is created based on the first and second feature points. Therefore, it is possible for the user to create a trajectory and analyze the motion of the moving image only by designating an arbitrary location without considering the color. As a result, the user can use it easily.

In S13, for example, when the screen is small, the operation unit 13 may accept that the user designates a shaft or the like different from the grip even though the user designates the grip shown in FIG. That is, the designated point P ₀ received by the operation unit 13 may be different from the analysis target portion intended by the user. However, in S14, for searching a plurality of feature points around the P ₀ around a specified point P _0, it is possible to perform the creation of the locus on the basis of a plurality of characteristic points the search. Therefore, even when the analysis target locations intended for the specified point P ₀ and the user are different, it is possible to improve the accuracy of the trajectory to be created.

  When searching for the second feature point in S15, for example, if the motion of the moving image is intense, the search for the second feature point may be difficult. For example, the still images shown in FIGS. 3 to 5 show the state immediately after the start of the swing, so the operation is slow and the change between the still images is small. However, since the still images shown in FIGS. 10 to 12 show the order from the downswing to immediately before the impact, the operation is fast and the change between the still images is large. Thus, when the motion of the moving image is fast and it is difficult to search for the feature points, it can be dealt with by acquiring the speed of the motion of the moving image in advance.

  Specifically, a still image is divided into regions of a predetermined size, and a difference is extracted for each corresponding region between adjacent still images. The magnitude of the difference is acquired, and if the difference is large, it is determined that the operation is fast, and if the difference is small, it is determined that the operation is slow. In this way, the motion speed of the moving image is acquired. Then, when searching for the second feature point in S15, the speed of operation is acquired, and if the operation is fast, the search range is enlarged. Here, the control unit 11 operates as an acquisition unit.

  Further, for example, a still image is divided into small regions of a predetermined size, and a histogram of luminance gradients in which the horizontal axis indicates the luminance gradient and the vertical axis indicates the appearance frequency is calculated for each small region. This luminance gradient is the same as that at the time of SIFT feature value calculation in S14 described above. Then, by comparing the calculated histogram for each corresponding region between adjacent still images, the magnitude of the difference in the histogram is acquired, and if the difference is large, it is determined that the operation is fast, and if the difference is small, the operation is performed. You may judge that it is slow.

  In addition, for example, the entire still image is converted into the frequency domain by performing a two-dimensional DCT (Discrete Cosine Transform) process on the entire still image. And the histogram of a low frequency area | region is calculated except the information of a high frequency area | region among the converted frequency area | regions. Then, by comparing the calculated histograms between adjacent still images, the magnitude of the difference between the histograms is acquired, and if the difference is large, it is determined that the operation is fast, and if the difference is small, the operation is determined to be slow. Also good.

Further, in the above-described embodiment, the case has been described where the feature points can be searched for any still image when searching for the feature points. For example, the feature points P ₁ and P ₁ of the still image shown in FIG. ₂ and P ₃ can be searched, but the feature points P ₁ ′, P ₂ ′, and P ₃ ′ of the still image in FIG. 4 adjacent to the still image in FIG. 3 may not be searched. Such a case can be dealt with by performing an interpolation process.

  Specifically, feature points are searched in order from the still image located at the beginning of the movie to the still image located at the end of the movie, and from the still image located at the end of the movie to the still image located at the beginning of the movie. Repeat in reverse order. That is, when the first feature point is searched for in the still image located at the head, the second feature point is searched for in the still image that is adjacent in time series to the still image located at the head. Further, when the first feature point is searched for in the last still image, the second feature point is searched for in the still image adjacent to the last still image in time series.

Then, a difference between the feature points is calculated, and an average value of the calculated differences is obtained in order as ΔP _t1 and ΔP _t2, and in reverse order as ΔP _t5 and ΔP _t4, and a feature point cannot be searched for ΔP _t3 and the average value If ΔP _t3 cannot be calculated, ΔP _t3 is calculated based on the average values before and after ΔP _t3 , that is, ΔP _t2 and ΔP _t4 . That is, the average of ΔP _t2 and ΔP _t4 is calculated as ΔP _t3 . In this way, the trajectory may be created by performing the interpolation process. In this case, the search for the first feature point in the last still image may be performed by designating the same location as the still image located at the top in the last still image by the user. Good.

In the above-described embodiment, the case has been described where the feature point can be searched for any still image when searching for the feature point. For example, the feature point P ₁ corresponding to the feature point P _{1 is described above.} can find and 'the feature points P ₂ corresponding to the feature point P _2', if it can not find in the feature point P _{3 'corresponding} to the feature point P _3, the feature point P ₃ is discarded, the feature point The average value may be calculated based on only P ₁ , P ₂ , P ₁ ′, and P ₂ ′.

  Further, when creating the trajectory, low-pass filter processing such as a moving average filter may be performed on the average value. By doing so, the created trajectory can be smoothed.

  In the above embodiment, an example of creating a trajectory based on the average value of feature point differences has been described. However, the present invention is not limited to this, and a trajectory is created based on the average value of feature points. May be. That is, a plurality of first feature points are searched and the average value is obtained. In addition, a plurality of second feature points are searched and an average value thereof is obtained. Then, the trajectory may be created by sequentially connecting the obtained average values.

  Further, in the above-described embodiment, an example in which the trajectory 20 and the still image are displayed in an overlapping manner has been described. However, the present invention is not limited thereto, and the trajectory 20 and the moving image may be displayed in an overlapping manner. This makes it easier for the user to understand the analysis result of the motion of the moving image. Here, the operation unit 13 operates as display means.

  Further, for example, by preparing a golf swing video as a model in advance, the model video trajectory and the own video trajectory may be displayed side by side. By doing so, it is possible to easily compare the model form with its own form, and the user can easily improve the form. In this case, when the size of the person is different, the image may be enlarged / reduced so that both sizes are equal. If the time required for the swing is different, the longer moving image may be thinned out so that the time lengths of both are equal.

In the above embodiment, the example in which one point P ₀ is designated by the user in S13 has been described. However, the present invention is not limited to this, and even when two or more points are designated, In each, the process from S14 can be performed to create each trajectory.

In the above embodiment, in S13, P ₀ is designated by the user, and a feature point search is performed within a predetermined radius r centered on the designated point P ₀ . The example in which the feature points of P ₁ , P ₂ , P ₃ different from the designated point P ₀ are retrieved has been described, but the present invention is not limited to this, and the designated point P ₀ may be retrieved as the feature point.

  In the above-described embodiment, the example in which the second still image is a still image adjacent to the first still image in time series has been described, but the first still image is not limited thereto. Any other still image may be used.

In the above-described embodiment, an example of searching for _three of P ₁ , P ₂ , and P ₃ as the first feature points has been described, but the number is not limited to this, and any number is possible.

  In the above embodiment, the example in which the operation unit 13 is a touch panel type has been described. However, the operation unit 13 is not limited to this example. For example, the operation unit 13 includes a display screen and a mouse. The analysis target location may be specified by operating the mouse.

  In the above-described embodiment, an example using a SIFT feature amount as a feature amount has been described. However, the present invention is not limited to this, and other features such as a HOG (Histogram of Oriented Gradients) feature amount, a Harris feature amount, and the like. May be used.

  Next, a motion analysis apparatus 30 according to another embodiment of the present invention will be described with reference to FIG. 1 and FIGS. In addition, the same code | symbol is attached | subjected about the component similar to the said embodiment, and detailed description is abbreviate | omitted. Moreover, in the following description, the case where the present invention is applied to the golf swing motion analysis will be described as in the above-described embodiment.

  Similar to the embodiment shown in FIG. 1, the motion analysis device 30 includes a control unit 11 such as a CPU that controls the entire motion analysis device 30, a storage unit 12 that stores images and the like, and a touch panel type that is received from a user. An operation unit 13 that receives an operation and displays an image or the like stored in the storage unit 12 and an I / F unit 14 that is a connection interface with an external device are provided.

  Here, in the motion analysis apparatus 30 according to this embodiment, the storage unit 12 stores a plurality of images corresponding to observation targets for motion analysis. Specifically, a large number of images of golf players taking the address posture are randomly stored in the storage unit 12 over a wide range of ages, heights, body shapes, genders, and the like. These images are used as reference images for estimating the initial form of the observation target. This process will be described in detail later.

  Next, an operation analysis flow performed by the control unit 11 of the operation analysis apparatus 30 according to this embodiment will be described. First, the motion analysis apparatus 30 accepts an input of a golf swing video imaged in advance via the I / F unit 14 (S21). As described above, the moving image data is configured as an aggregate of a plurality of still images that are continuous in time series. The motion analysis apparatus 30 displays the still image 31 shown in FIG. 14 positioned at the beginning of the time series on the operation unit 13 (S22).

  Next, the motion analysis apparatus 30 scans the still image 31 and sequentially defines an area having a predetermined size as an analysis target area 32 (S23). FIG. 14 shows a case where the player 33 to be observed is included in the target area 32 with an appropriate size. However, as an actual analysis flow, the still image 31 is scanned over the entire area, and the target area is defined one after another while changing the position and size of the target area 32. And in the process mentioned later, it is determined whether the player 33 is included for every object area | region.

  Hereinafter, from the viewpoint of easy understanding, as shown in FIG. 14, a case will be described in which the player 33 to be observed is included in an appropriate size in the target area 32 defined by S23.

  After the target area 32 including the player 33 is defined, the size of the target area 32 is adjusted to a predetermined image size so that the subsequent analysis process can proceed more quickly and easily (S24). . Specifically, the image size of the target area 32 is enlarged / reduced to 64 × 128 dots, for example. As a result, the target area 32 shown in FIG. 14 is cut out as a target area 32A having a predetermined image size as shown in FIG. Similarly, the player 33 included in the target area 32 is also enlarged / reduced correspondingly and becomes the player 33A.

  Next, the contour 34 of the player 33A is detected based on the luminance gradient in the image data of the target area 32 (S25). As a method for detecting the contour, for example, a feature amount based on a luminance gradient such as the SIFT feature amount or the HOG feature amount described above may be applied, or a parameter other than the luminance gradient such as a Harris feature amount may be applied. A feature amount based on the above may be applied. By using these methods, the contour 34 corresponding to the player 33A can be extracted effectively as shown in FIG. In FIG. 16, from the viewpoint of easy understanding, the actual contour 33B of the player 33A and the contour 34 extracted based on the HOG feature amount are shown together. However, in actual processing, the HOG feature is shown. An outline 34 based on the quantity may be detected.

  As described above, the contour 34 of the observation target shown in FIG. 16 can be extracted from the still image 31 shown in FIG. 14 through the steps S23 to S25. That is, in the present invention, the control unit 11 that executes the steps S23 to S25 functions as a person recognition means (A in FIG. 13) for recognizing the player 33 to be observed in the still image 31.

  Next, the motion analysis device 30 compares the contour 34 of the player 33 </ b> A detected by the person recognition means (A) with the reference image stored in the storage unit 12, and selects a reference image that approximates the contour 34 ( S26). This operation will be specifically described below.

  In this embodiment, as described above, the storage unit 12 stores a large number of image data of golf players taking an address posture at random over a wide range of ages, heights, body shapes, genders, and the like. FIG. 17 illustrates a reference image to be compared. Also in the reference image, a target area 32A having the same image size as the target area 32A shown in FIGS. 15 and 16 is defined, and the reference player 35 is accommodated in an appropriate size in the target area 32A. It is remembered. That is, the observation target player 33A and the reference player 35 compared in S26 are adjusted and stored so that the positions and sizes of the persons are substantially the same in the target area 32A of the common size. With this configuration, the comparison between the observation target player 33A and the reference player 35 can be executed with higher accuracy without being affected by the position and size of the player 33 in the still image 31.

  In the storage unit 12, in addition to the still image as shown in FIG. 17, the reference image is a contour image derived from the HOG feature amount, or a partial body region such as the head, arm, or leg. May be stored as a body part region image that defines. 18A to 18C illustrate image data of each format corresponding to FIG. FIG. 18A is a still image corresponding to FIG. 17, and shows a state in which the first body part region 36 corresponding to the head, arms, and legs is defined on the image. . That is, the first head region 36a, the first arm region 36b, and the first leg region 36c are each defined on the image. FIG. 18B is a diagram showing an outline 55 of the reference player 35. FIG. 18C is a diagram illustrating a state where only the first body part region 36 illustrated in FIG.

  In this embodiment, the contour 34 of the player 33A detected by the person recognition means (A) is compared with the contour 55 of the reference player 35 shown in FIG. From a large number of reference images, an image whose form approximates the observation target is selected. Then, the first body part region 36 corresponding to the selected reference image is extracted and applied to the image data to be observed. The result is shown in FIG.

  As shown in FIG. 19, the first body part region 36 applied by the method described above substantially coincides with each part of the body of the player 33A to be observed. Thereby, the arrangement of the head, arms, legs, and the like of the player 33A, that is, the posture of the player 33A in the address state can be roughly estimated.

  As described above, according to the present invention, based on the input data of the still image 31 and the reference image data stored in the storage unit 12, the initial form of the player 33A in the still image 31 (such as the posture and the body shape). Including elements) can be roughly estimated. That is, the control part 11 which performs S26 functions as an initial form estimation means (B) for estimating the initial form of the player 33A.

  In addition, as a result of executing S26, when a reference image that approximates the observation target player 33A is not found, the motion analysis apparatus 30 does not appropriately include a person in the target area 32 determined in S23. And return to S23. Then, S23 to S26 are repeated until a state in which a person is appropriately included in the target area 32 is detected.

  As a result of S26, when the posture of the player 33A can be estimated as shown in FIG. 19, as a next step, an analysis target portion to be subjected to motion analysis is designated based on the image data shown in FIG. (S27). Here, in this embodiment, the analysis object location is determined by the first body part region 36. As an example, the first head region 36a, the lower end portion of the first arm region 36b, and / or the center portion of the first leg region 36c shown in FIG. To do. This is synonymous with automatically specifying the head, grip (near the wrist), and / or knee of the player 33A as the analysis target portion. This is because, as described above, the first body part region 36 is defined so as to substantially coincide with each part of the body of the player 33A to be observed. FIG. 19 shows, as an example, a case where the lower end of the first arm region 36b, that is, the vicinity of the grip of the player 33A is designated as an analysis target location (point 39 in FIG. 19).

  Thus, in this embodiment, the control part 11 which performs said S23-S27 functions as an analysis object location designation | designated means (C) for designating an analysis object location. With this configuration, when the user starts motion analysis, the motion analysis device 30 estimates the initial form of the player 33A to be observed and automatically determines the initial conditions for analysis based on the data. It becomes possible. Thereby, since the user can omit the work of specifying the analysis target portion, the convenience for the user can be further improved.

  After the analysis target location is designated in S27, the motion analysis device 30 searches for the first feature point in the designated first body part region 36 (S28). Here, a plurality of first feature points are searched in the first body part region 36 using the SIFT feature value, the HOG feature value, and the like.

  Next, the motion analysis device 30 corresponds to the first feature point in the adjacent still image after the time series of the still image of FIG. 19 searched for the first feature point, that is, the still image shown in FIG. The second feature point is searched (S29). Based on the searched second feature point, a second body part region 37 corresponding to the first body part region 36 is defined (S30).

  Next, a trajectory is created based on the difference between the first body part region 36 and the second body part region 37 (S31). Specifically, after the second arm region 37b shown in FIG. 20 is newly defined, a position corresponding to the lower end of the second arm region 37b is recognized as an analysis target location (in FIG. 20). Point 40). A trajectory 42 is created based on the difference between the points 39 and 40. Similarly, for example, when the first head region 36a is designated as an analysis target location in S27, the trajectory of the head of the player 33A based on the difference between the first head region 36a and the second head region 37a. Can be created. Further, the knee trajectory of the player 33A can be created based on the difference between the first leg region 36c and the second leg region 37c.

  Subsequently, the third feature point corresponding to the second feature point is searched in the still image that is chronologically adjacent to the still image shown in FIG. 20, that is, the still image shown in FIG. Based on the third feature point, a third body part region 38 corresponding to the second body part region 37 is defined. Then, based on the difference between the second body part region 37 and the third body part region 38, the analysis target part is recognized (point 41 in FIG. 21), and a locus 43 is created.

  In this way, by sequentially creating the locus of the analysis target portion for each continuous still image, the motion of a predetermined part (for example, the head, arms, and / or legs) of the player in a series of golf swing motions. Can be analyzed.

  In the present embodiment, the case where person recognition is performed by detecting the contour 34 of the player 33A has been described. However, the present invention is not limited to this, and any other method may be used as long as the form of the observation target can be extracted. May be.

  Further, in the present embodiment, a case has been described in which the contour that is close to the observation target is selected by comparing the contour 34 of the player 33A and the contour 55 of the reference player 35. 18A may be selected by comparison with still image data, or may be selected by comparison with data relating to the body part region 36 shown in FIG. In addition, any type of data other than that shown in FIG. 18 may be used for comparison as long as the observation target and the reference can be compared.

  Moreover, in this embodiment, the case where the analysis object location 39 which is the initial condition of motion analysis based on the body region 36 was specified was described. However, the present invention is not limited to this. For example, it may be specified based on the still image data shown in FIG. 18A or the data related to the contour 55 shown in FIG. 18B, or may be analyzed based on any other data. The location 39 may be designated.

  When the present invention is applied to the golf swing motion analysis as in the above embodiment, the motion analysis devices 10 and 30 indicate that the swing direction has been changed from the swing takeback to the downswing. There may be further provided a direction change identifying means for detecting and enabling the user to identify it. This will be described with reference to FIG. FIGS. 22A to 22F show moving images in which a series of golf swings are recorded, and the moving images are composed of a total of six frames of still images.

  As shown in FIGS. 22 (a) to 22 (f), the golf swing is a downswing (from FIG. 22 (a) to (c)) in which the golf club is swung down to swing down the golf club. Shifting to FIG. 22 (d) to (f)). That is, in FIG. 22, a swing direction change occurs between FIG. 22 (c) and FIG. 22 (d).

  The direction change identification means detects the direction change point and enables the user to identify the trajectory before the direction change and the trajectory after the direction change. As a result, it becomes possible to clearly grasp the point that becomes the turning point of the motion and the trajectory of the motion before and after that, so that more detailed motion analysis can be performed. A specific algorithm will be described below.

As a premise, FIG. 22 (f) is the current time. The direction change identifying means first calculates a grip speed vector v ₁ in FIG. 22A five frames before the current time series from the present time. This v ₁ is obtained from the frame adjacent in time series to FIG. 22A, that is, the grip position 6 frames before (not shown) from the present time and the grip position 5 frames before. Can do. Similarly, Figure 22 of the previous four frames from the present base rate of grip in (b) vector v _2, 3 frames prior to 22 base speed of the grip in (c) vector v _3, from the moment of two frames before 22 The grip speed vector v ₄ in (d), the grip speed vector v ₅ in FIG. 22 (e) one frame before the current time, and the grip speed vector v ₆ in the current frame are calculated.

Here, the average velocity vector f ₁ of the grip between the front and 3 frames 5 frames before the present time is expressed as follows.

Similarly, the average velocity vector f ₂ of the grip between the frame of two frames before-current from the current time is expressed as follows.

Turning identification means according to the present embodiment, in order to detect the turning points of the grip, to calculate the inner product = f 1 _· f ₂ between f ₁ and f _2. Then, since the direction of the grip speed vector from 5 frames before to 3 frames and the speed vector of the grip from 2 frames before to the present time are reversed, the inner product takes a negative value. . In other words, if a point in time when the inner product becomes negative can be detected, a turning point at which the direction is changed from the takeback of the swing to the downswing can be detected.

  After detecting the turning point in this way, in S31 described above, by forming a different kind of locus line from the locus line before the turning point, the user can take the takeback locus and the downswing locus. Can be clearly distinguished and recognized. As means for enabling identification, for example, changing the color of the locus line (yellow to red, etc.), changing the line type (solid line to dotted line, etc.), blinking the locus line, etc. can be considered.

In this embodiment, from the viewpoint of easy understanding, the average velocity vectors f ₁ and f ₂ are calculated by averaging the velocity vectors of three consecutive frames. The average speed vectors f ₁ and f ₂ may be calculated based on the average of the speed vectors of the frames.

  FIG. 23 shows an example of a golf swing trajectory in which takeback and downswing can be identified by the direction change identifying means. Thus, according to the direction change identification means according to the present invention, the user can clearly grasp the point that becomes the turning point of the swing and the trajectory of the movement before and after the turning point. Analysis can be performed.

  In each of the above-described embodiments, the case where one pattern of golf swing is displayed on the display means (operation unit 13) and the motion analysis is performed has been described. However, the present invention is not limited to this, and the motion analysis apparatuses 10 and 30 may further include an operation comparison means for displaying two or more different operations side by side and making these different operations comparable. Hereinafter, similarly to the above-described embodiment, an operation comparison unit capable of performing a golf swing operation comparison will be described with reference to FIGS.

  The motion comparison unit according to the present embodiment first displays in parallel the videos that store the two golf swings (S41), and forms the trajectory of each golf swing using any of the methods of the above-described embodiments. . FIG. 24 shows a diagram in which two golf swing trajectories are formed and displayed in parallel. Specifically, a first locus 45 is displayed in the first display area 44, and a second locus 47 is displayed in the second display area 46.

  Next, the time axes of the two moving images are adjusted by compressing / expanding, and the time axes of the two moving images are aligned (S42). Here, as described above, the golf swing is constituted by a series of operations such as take back, down swing, and finish from the address posture. The time required for each of these operations varies depending on the person, and even when the same person swings, the time varies depending on the swing. Therefore, when two swings are displayed in parallel and compared, the trajectory of each swing can be more easily visually verified by aligning the start time and end time of each operation.

  The method is to calculate the time from the address state to the end of takeback and the time from the beginning of the downswing to the finish in each of the two videos. It is conceivable to perform a process for aligning the time required for each operation between two moving images by extending the time between them. Note that the turning point from the end of the takeback to the start of the downswing can be detected by the method according to the above-described embodiment.

  More specifically, for example, in the golf swing displayed in the first display area 44, it takes 1.4 seconds from the address to the end of the takeback, and 1.1 seconds from the start of the downswing to the finish. . In the golf swing displayed in the second display area 46, it takes 1.8 seconds from the address to the end of the takeback and 1.5 seconds from the start of the downswing to the finish. In this case, in the moving image displayed in the second display area 46, the still image frame from the address to the end of the takeback is thinned out at a ratio of 1.4 / 1.8, and from the beginning of the downswing. The still image frame until the finish is thinned out at a ratio of 1.1 / 1.5. By this processing, the time axis from the address of two golf swings to the finish can be aligned. With this process, when the two swings are played back in parallel, the trajectories of the two swings can be more easily compared and verified visually.

  Next, processing for enlarging / reducing the image sizes of the two moving images and aligning the positions and sizes of the players in the display areas 44 and 46 is performed (S43). This makes it easier for the user to compare and verify the two swings.

  Next, the centroid point 48 of the first locus 45 and the centroid point 49 of the second locus 47 are respectively calculated (S44). The barycentric points 48 and 49 of each trajectory can be calculated by obtaining an average value of each point constituting the trajectory. In addition, after calculating the gravity center points 48 and 49, the coordinates may be displayed as shown in FIG. 24 or may be stored without being displayed.

  Next, a difference value between the points of the first trajectory 45 and the second trajectory 47 in each frame and the centroid points 48 and 49 of the respective trajectories is calculated, and the frame number and the difference value are recorded ( S45). As reference data, a graph relating to the frame number N and the difference value x is shown in FIG. In FIG. 26, a difference value graph between the first locus 45 and the center of gravity point 48 is shown as a line 50, and a difference value graph between the second locus 47 and the center of gravity point 49 is shown as a line 51.

  Next, based on the data obtained in S45, the difference between the first trajectory 45 and the second trajectory 47 is calculated, and the trajectory portion where the difference is large is displayed in an identifiable manner (S46). . Specifically, when the difference between the line 50 and the line 51 is calculated at each frame number N, as shown in FIG. 26, locus portions 52, 53, and 54 in which the difference is relatively large are detected. In other words, the first trajectory 45 and the second trajectory 47 mean that the trajectories of the swing are relatively greatly different in the trajectory portions 52, 53, and 54. Therefore, for example, the line color, the line type, or the blinking display is performed on the trajectory lines corresponding to the trajectory portions 52, 53, and 54 in the first trajectory 45 or the second trajectory 47. Thus, the user can identify the portion. According to this configuration, for example, when a user records a golf swing every day, it is possible to grasp and verify in detail a portion where the trajectory of the swing has changed. Thereby, it is possible to provide a motion analysis apparatus with higher added value.

  In addition, in this embodiment, although the case where two golf swings were compared was described, the structure which compares not only this but three or more operation | movement may be sufficient.

  Moreover, those skilled in the art will easily understand that the operation algorithm of the operation analysis apparatus according to the present invention can be realized by either software or hardware.

  Further, in each of the above-described embodiments, the case where the golf swing is the target of motion analysis has been described. However, the present invention is not limited to this. That is, the concept of the present invention is to compare the image data at the start point of the motion analysis with the reference data stored in advance, and estimate the form of the observation target at the start point, thereby determining the initial condition (size, In other words, the motion analysis can be performed easily and quickly. Therefore, it is applicable not only to golf swings, but also to sports and performances in all fields where there is a need for grasping and verifying player movements such as baseball swings, dances, performances, martial arts, etc. This will be readily understood by those skilled in the art.

  As mentioned above, although embodiment of this invention was described with reference to drawings, this invention is not limited to the thing of embodiment shown in figure. Various modifications and variations can be made to the illustrated embodiment within the same range or equivalent range as the present invention.

  The present invention is effectively used when it is necessary to check a form in sports.

  10, 30 motion analysis device, 11 control unit, 12 storage unit, 13 operation unit, 14 I / F unit, 20, 42, 43, 45, 47 locus, 31 still image, 32, 32A, 36, 36a, 36b, 36c, 37, 37a, 37b, 37c, 38, 38a, 38b, 38c area, 33, 33A, 35 player, 34, 55 contour, 39, 40, 41, 48, 49 points, 44, 46 display area, 50, 51 line, 52, 53, 54 locus part.

Claims (14)

A motion analysis device that analyzes motion of a moving image composed of a plurality of continuous still images,
Video accepting means for accepting input of the video;
Among the videos received by the video reception means, in the first still image constituting the video, on the first still image, an analysis target location specifying means for specifying an analysis target location that is a target of motion analysis; ,
First search means for searching for a first feature point within a certain range from the analysis target location;
A second search unit configured to search for a second feature point corresponding to the first feature point in a second still image that constitutes the moving image and is a still image different from the first still image;
A motion analysis device comprising: trajectory creation means for creating a trajectory based on the first and second feature points.   The motion analysis apparatus according to claim 1, wherein the analysis target location specifying unit includes a specification receiving unit that receives a specification from a user so that the user can arbitrarily specify the analysis target location. The analysis location specifying means is:
Recognition means for recognizing an observation object from the first still image;
Morphological estimation means for estimating the morphology of the observation target based on the image data of the observation target recognized by the recognition means,
The motion analysis apparatus according to claim 1, wherein the analysis target location is specified based on an estimation result by the form estimation unit. The recognition means is
Target area defining means for defining a certain area including the observation target as a target area;
The motion analysis apparatus according to claim 3, further comprising target region normalizing means that adjusts the size of the target region so as to be a predetermined image size.   5. The motion analysis apparatus according to claim 3, wherein the recognition unit detects an outline of the observation target based on a luminance gradient in the first still image. Reference image storage means for storing a plurality of comparison target images corresponding to the observation target in advance,
The form estimation means includes
The image data of the observation object recognized by the recognition means is compared with the image data of the comparison object stored by the reference image storage means, and the comparison object whose form approximates the image data of the observation object is compared. Reference image selection means for selecting image data is included,
The motion analysis apparatus according to claim 3, wherein the analysis target portion is specified based on the image data to be compared selected by the reference image selection unit. The first search means searches for a plurality of first feature points within the certain range,
The second search means searches for a plurality of second feature points corresponding to the plurality of first feature points,
The trajectory creation means calculates the difference between the plurality of first feature points and the plurality of second feature points, respectively, and obtains an average value of the calculated differences, based on the obtained average value. The motion analysis device according to any one of claims 1 to 6, wherein a trajectory is created. The average value is a point based on the first and second feature points, and indicates a change at a predetermined point on the still image constituting the moving image,
The trajectory creation means creates the trajectory by sequentially calculating the average value between adjacent still images of the plurality of continuous still images and sequentially connecting the predetermined points indicated by the calculated average value. The motion analysis apparatus according to claim 7. The first still image includes a still image located at the beginning of the moving image and a still image located at the end of the moving image,
The second still image includes a still image that is adjacent to the first still image in time series and a still image that is adjacent to the first still image in time series before,
The second search means searches for the second feature point corresponding to the first feature point in the still images that are adjacent in time series, and in the still image that is adjacent in time series. The motion analysis device according to claim 1, wherein the second feature point corresponding to the first feature point is searched.   The second search means includes an acquisition means for acquiring the speed of motion of the moving image, and searches for the second feature point based on the speed acquired by the acquisition means. The motion analysis apparatus according to any one of the above. The plurality of continuous still images are configured at regular intervals,
The motion analysis apparatus according to claim 10, wherein the acquisition unit acquires a motion speed of the moving image by acquiring a magnitude of a difference between still images between adjacent still images.   The said locus | trajectory creation means has a direction change identification means which can identify the locus | trajectory before a direction change, and the locus | trajectory after a direction change, when the direction of the said locus | trajectory is changed. The operation analysis apparatus described. The direction change identifying means includes
The trajectory velocity vector between still images that are continuous in time series is calculated, and the inner product of the velocity vector of a specific still image and the velocity vector of the still image that is adjacent to the specific still image in time series is negative. The motion analysis apparatus according to claim 12, wherein the direction change of the trajectory is detected when it becomes.   The motion analysis apparatus according to claim 1, further comprising a display unit that displays the trajectory created by the trajectory creation unit and the moving image received by the moving image reception unit.

Images