JP2005270500A - Golf swing diagnostic system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide the diagnosis of the swing and advices therefor. <P>SOLUTION: A computer 16 which captures the color moving images taken by image pickup means 14 and 15 has a means which extracts the images of the required attitudes in the swing from a plurality of still images making up the color moving images as the check point images, a means for determining the position coordinates on the check point images of the attention points moving during the swing, a means which sets the diagnosis items according to the trajectory patterns from which the behavior of the balls acquired with a ball motion measuring device 20 is classified and diagnoses the swing of the golfer 11 with respect to the diagnosis items by comparing the data digitized from the position coordinates of the attention points with the judgment data serving as the ideal values and an output means for outputting the advice drills corresponding to the results of the diagnosis. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a golf swing diagnosis system, and more particularly to an apparatus for efficiently performing automatic diagnosis of a golfer's swing form.

  2. Description of the Related Art Conventionally, various devices have been provided that can capture a swing when a golfer is hit, and automatically calculate various information such as a flying distance and a trajectory of the hit ball and display it on a golfer. Although these devices allow golfers to examine the quality of their own flying balls, they have not been able to provide useful information to improve the essential form.

  Therefore, in the swing form diagnostic apparatus disclosed in Japanese Patent Application Laid-Open No. 2003-117045, an image of a golfer swinging is taken and only an image of a specific operating point important for swing form diagnosis is extracted. . Specifically, a frame related to the swing motion is extracted from the photographed subject's video, and a specific motion point during the swing motion is determined according to the analysis result of the moving partial image in the difference image between each frame and the reference image. The frame corresponding to each specific operating point is extracted and displayed as an image. In particular, the image at the time of impact is important for the diagnosis of the swing form, and image extraction focusing on the impact image is performed.

However, even if a frame corresponding to a specific operating point during the swing motion is extracted, the golfer will be specifically provided with the motion of his / her body (joint, etc.) during the swing simply by looking at the extracted image. There is a problem that it is difficult to understand what is in the form and what should be fixed.
Therefore, in the motion diagnosis device disclosed in Japanese Patent No. 2779418, a plurality of motion points are provided on the body of the golf club head and the person to be diagnosed, and the coordinates of the motion points in the swing moving image are acquired to perform the swing diagnosis. Yes. However, it is necessary to perform a calculation for extracting the coordinates of a large number of operation points for all the frames of the swing moving image, and there is a problem that the calculation amount becomes enormous and the misidentification rate of the position coordinates of the operation points increases. In addition, during the operation of the golf swing, the operation point may be hidden by an arm or the like and cannot be visually recognized from the camera, and there is a problem that the position coordinates of the operation point cannot be acquired and the probability of occurrence of a diagnostic error increases. Furthermore, although it is described that numerical data such as arm angle and club angle is calculated from the coordinate data obtained by tracking the operation points P _{1 to} P ₆ , how to actually use it is described. A system that is unknown and that can provide specific swing diagnosis and advice is desired.
JP 2003-1117045 A Japanese Patent No. 2794018

  The present invention has been made in view of the above problems, and it is an object of the present invention to reduce the calculation time and error and to provide swing form diagnosis and advice thereto.

In order to solve the above-described problems, the present invention includes a computer that captures a color moving image obtained by shooting a golfer who holds a golf club and swings.
From a plurality of still images constituting the color moving image, address, take back shaft 8 o'clock, take back shaft 9 o'clock, take back non-dominant arm horizontal, top, down swing non-dominant arm horizontal, down swing shaft 9 o'clock, impact, follow Means for extracting at least one swing posture image selected from the finish at the shaft 3 as a checkpoint image;
Means for acquiring position coordinates on the checkpoint image of a point of interest that operates during swing;
A plurality of diagnostic items including a swing posture and a shaft angle are set for each trajectory pattern, and data obtained by digitizing the position data of the attention point with respect to the diagnostic items is an ideal value input in advance to the computer Means for diagnosing the golfer's swing in comparison with a judgment value;
There is provided a golf swing diagnosis system characterized by comprising means for outputting an advice drill corresponding to the diagnosis result from a database in which a plurality of advice drills, which are practice methods for swing improvement, are registered.

According to the above configuration, diagnostic items are prepared according to the trajectory pattern, so that the diagnostic accuracy of the diagnostic items can be improved and only appropriate diagnostic items can be efficiently diagnosed for each trajectory pattern. That is, for example, hooks and slices are possible only by breaking the back of the wrist at the top, so that appropriate diagnosis can be performed by dividing the ballistic pattern as described above.
In addition to diagnosing the quality of the swing, the advice drill, which is a practice method for improving the swing, is automatically output according to the diagnosis result, so it can be useful for improving the golfer's score. Can do.
Furthermore, since the procedure is to obtain the position coordinates of the golfer's attention point after selectively extracting still images (checkpoint images) useful for golf swing diagnosis, all frames of the captured swing moving images (still images) It is not necessary to acquire the position coordinates of the golfer's attention point for the image), and the calculation cost can be reduced. In addition, by limiting the still image to be calculated to only the checkpoint image, it contributes to a reduction in the misidentification rate of the attention point.

  The color moving image may be captured by connecting a photographing camera to the computer, or may be captured via a DV tape or the like. Further, the trajectory pattern may be automatically acquired by a ball motion measuring device described later, or the golfer himself may manually input his trajectory trajectory.

  The ballistic pattern is preferably classified into pull hook, pull, pull slice, straight hook, straight, straight slice, push hook, push, and push slice. Preferably, the straights are further classified into three types: fade, draw, and straight.

A ball motion measuring device for measuring the behavior of a golf ball hit by the golfer;
A trajectory pattern is acquired from the side spin amount and deflection angle of the ball measured by the ball motion measuring device.

  With the above configuration, the trajectory of the hit ball can be acquired from the side spin amount and the swing angle, and classification based on the trajectory pattern of the swing diagnosis can be easily performed.

The computer is provided with an inquiry means that allows the golfer to input his own ballistic pattern to be diagnosed before hitting,
When the ballistic pattern obtained from the actual ball behavior acquired by the ball motion measuring device matches or resembles the ballistic pattern input by the interrogation means, the diagnosis and advice drill are output.

  With the above configuration, for example, when a golfer wants to correct his ball trajectory due to a defect of a slice, if the slice is input as a trajectory pattern desired to be diagnosed by an inquiry means, the ball of the hit ball If the trajectory pattern is a trajectory pattern that does not wish to be diagnosed, the diagnosis is performed when the trajectory pattern of the slice is struck, so that a diagnosis that meets the needs of the user can be provided.

  A difference silhouette is obtained by performing a difference process between a top image showing the swing posture of the top and an image after a predetermined time has elapsed from the top image, and switching from the top state using the area of the difference silhouette The operation is being diagnosed.

  With the above configuration, since the area of the difference silhouette can be considered as the movement amount of the golfer's body in the turn-back operation from the top, the golfer's turn-back operation has a large amount of movement of the upper body or a large amount of movement of the lower body, etc. Can be diagnosed.

  Further, the turning-back operation may be diagnosed based on an angle difference between the shaft line in the downswing non-dominant arm horizontal image viewed from the rear of the flying ball line and the shaft line in the address image.

  Cock operation when the difference between the wrist angle in a given swing posture and the wrist angle in another swing posture is greater than or equal to a prescribed value, or when the wrist angle in a given swing posture is greater than or equal to a prescribed value Diagnosed that there is.

With this configuration, it is possible to easily diagnose the presence or absence of a cock action in which the wrist angle, which is the angle formed by the shaft line of the golf club and the arm line, changes greatly during swing.
Specifically, if the difference between the golfer's wrist angle in the downswing non-dominant arm horizontal image and the wrist angle in the downswing shaft 9 o'clock image is greater than or equal to a predetermined value, the cock operation is diagnosed. It is good to have. Alternatively, when the downswing non-dominant arm level or the wrist angle at the time of the downswing shaft 9 is a predetermined value or more, it may be diagnosed that there is a cock action.

  Scoring is set in advance for each of the plurality of diagnostic items, and the scoring is added when the determination value that is an ideal value for each diagnostic item is satisfied or not satisfied.

  With the above configuration, the golfer accurately compares the score of the previous diagnosis result with the score of the current diagnosis result, thereby accurately identifying the degree of improvement in the swing form resulting from the practice of swing improvement practice following the advice drill. It can be easily grasped. In addition, it is preferable to add a score when the ideal value is satisfied, and it is preferable to diagnose that the larger the total score is, the better the swing form, but when the ideal value is not satisfied, the score is added, It may be diagnosed that the smaller the total score, the better the swing form.

Further, if the configuration is made such that the points outputted for each diagnosis item are cumulatively added and the total score is outputted as a diagnosis result, the total swing evaluation can be known at a glance.
In addition, it is preferable that the score for each diagnostic item is changed in weight in consideration of the importance to the swing. Furthermore, when the total score is 100 points, for example, if it is 90 points or more, it is a professional grade, if it is 80-90 points, it is an advanced player, if it is 60-80 points, it is an average golfer, if it is less than 60 points, it is called a beginner Thus, it is preferable to output the golfer level by the total score.

As is clear from the above description, according to the present invention, since the diagnostic items are prepared according to the ballistic pattern, it is possible to efficiently perform only the diagnosis of an appropriate diagnostic item for each ballistic pattern, and the diagnostic items The accuracy of diagnosis is improved. In addition to not only diagnosing the swing status of the self, the advice drill, which is a practice method for swing improvement, is automatically output according to the diagnosis result, so the golfer can immediately take appropriate fault countermeasures. Can be done.
Furthermore, since the procedure is to obtain the position coordinates of the golfer's attention point after selectively extracting still images (checkpoint images) useful for golf swing diagnosis, all the frames (still images) of the swing moving image are used. It is not necessary to acquire the position coordinates of the attention point of the golfer, and the calculation cost and the misidentification rate can be reduced.

Embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic diagram of a golf swing measurement system. A computer 16, a monitor 17 as display means connected to the computer 16, a keyboard 18 and a mouse 19 as input means connected to the computer 16, and a computer 16, color CCD cameras 14 and 15 installed at the front position of the golfer 11 and at the rear (side) position of the flying ball line, and a ball motion measuring device 20. A square frame 39 is installed on the ground at a position surrounding the golfer 11 and the ball B.
In this embodiment, the computer 16 to which the color CCD cameras 14 and 15 are connected and the computer 16 to which the ball motion measuring device 20 is connected are shared by the same one, but in order to distribute the processing load, the computer 16 is used. A single computer 16 may be prepared and processed separately.

  The ball motion measuring device 20 is disclosed in Japanese Patent Application Laid-Open No. 2001-264016. A single CCD camera 21 including a multiple shutter 22 that can be continuously opened and closed is provided with four strobes 23 and a striking speed measuring sensor 24. At the same time, it is connected to the computer 16. The striking speed measuring sensor 24 is composed of a pair of light projectors 25 and a light receiver 26. The light projector 25 is provided with two light projecting portions 25a and 25b that emit infrared rays, while the light receiver 26 detects infrared rays 2 Light receiving portions 26a and 26b are provided at different locations.

  A golfer 11 (assumed to be right-handed) to be diagnosed wears measurement clothes 12, which are outerwear with colored marks M <b> 1 to M <b> 7 attached to points of interest, from the top of the private clothes. The measurement clothes 12 are made of a white background, with a yellow colored mark M1 on the left elbow, a red colored mark M2 and a blue colored mark M3 on the left shoulder, and a blue colored mark M4 and a red color on the left shoulder. A colored mark M5, a blue colored mark M6 on the right elbow, and a red colored mark M7 on the waist are provided. The left elbow colored mark M1 has an armband shape, the waist colored mark M7 has a belt shape, and the other colored marks M2 to M6 have a button shape (spherical). Here, the attention points include joints such as the head, neck, shoulders, elbows, hips, knees, ankles, wrists and / or toes of the golfer 11, but the attention points are limited to the body part of the golfer. First, it includes all parts that are useful for swing diagnosis, such as colored club marks and balls attached to the golf club shaft.

  Three colored club marks CM1, CM2, and CM3 are attached to the shaft 13a of the golf club 13 held by the golfer 11 at intervals. The club colored marks CM1 to CM3 are attached at equal intervals from the grip side to the head side, the club colored mark CM1 closest to the grip is yellow, and the club colored mark CM2 arranged in the middle is pink. The color mark M3 for clubs arranged on the head 13b side is yellow so that adjacent marks have different colors. In this embodiment, the distance between the club colored marks CM1 to CM3 is 250 mm, and the distance between the grip end and the club colored mark CM1 is 250 mm.

  The two color CCD cameras 14 and 15 synchronize the photographing timing with the computer 16, and when using a high-speed digital CCD camera, the number of frames per second is 30 frames or more, preferably 60 frames or more. The speed is 1/500 s, preferably 1/1000 s or less.

  The space for shooting the swing (length 3 x width 3 x height 2m) is preferably as bright as possible. However, if an extremely bright spot occurs, halation may occur. It is preferable to set a uniform brightness as a swing environment within a range that does not exist. In addition, it is preferable that the background 20 of the space where the swing is photographed be different in color so that the golfer 11, the colored marks M1 to M7, and the club colored marks CM1 to CM3 can be easily extracted.

  The computer 16 is connected online to the color CCD cameras 14 and 15 using a LAN cable, IEEE 1394, CameraLink standard, and the like, and the swing moving images (a plurality of still images) taken by the color CCD cameras 14 and 15 are stored in the computer 16. Are stored in the hard disk, the memory on the computer 16 or the memory on the board. In addition, as will be described later, the computer 16 binarizes each pixel of the still image with a specific threshold value related to color information, and sets the pixels satisfying the threshold value as the positions of the club colored marks CM1 to CM3. On the basis of the coordinate data of the club colored marks CM1 to CM3, the means for recognizing the movement of the golfer's arm, and the operation data of the shaft 13a. A program including a means for selectively extracting a still image necessary for swing measurement and a means for calculating the ball behavior from the information acquired from the ball motion measuring device 20 is incorporated.

Next, the swing diagnosis of the golfer 11 is performed according to the flowchart of FIG.
First, the golfer 11 inputs a reply using the mouse 19 or keyboard 18 (or in a touch panel format) about the contents of the inquiry displayed on the inquiry screen of the monitor 17 of the computer 16 (S10). At this time, it is preferable that the color CCD cameras 14 and 15 read a background image obtained by photographing only the background 30 in a state where the golfer 11 does not exist.
The contents of the inquiry are prepared for each item of name, sex, age, height, weight, golf history, ball, desired diagnostic content, desired mode, and clothes.

As shown in FIG. 28, the interrogation of the holding ball is a question about a wrinkle of its own ballistic pattern (A to I) such as a slice system, a hook system, a straight system, and particularly none. The default is “None”.
The desired diagnostic contents are the items of the analysis mode such as “I want to fly straight”, “I want to increase the flight distance”, “None in particular” and the diagnostic mode selection items and “I want to learn swing from the basics” Is also provided. The default is “I want to fly straight”.
The diagnosis content of the clothes is a question as to whether the clothes of the golfer 11 who is the subject is a short sleeve, a long sleeve, a swing measurement clothes, or a swing photographing measurement clothes (trousers black).
The above inquiry results are stored in the hard disk of the computer 16 as an initial setting file.

  Next, the golf player 11 makes a trial shot, takes still images of each frame of the swing moving image from the color CCD cameras 14 and 15 into the computer 16 and saves them in the hard disk, the memory in the computer 16 or the memory on the board (S12). ), The movement of the ball B hit by the ball movement measuring device 20 is measured (S15). At this time, for example, when the ballistic pattern obtained from the actual ball behavior acquired by the ball motion measuring device 20 after the five-ball trial hit matches or resembles the ballistic pattern input in the inquiry, the diagnosis described below is executed. . Alternatively, an image to be diagnosed may be selected regardless of match / similarity.

Next, an address image which is a swing posture useful for swing diagnosis from a large number of still images constituting a swing moving image, an image at the takeback shaft 8 o'clock, an image at the takeback shaft 9 o'clock, a horizontal image at the takeback left arm, a top image, Check point images of the cut-back image, the down swing left arm horizontal image, the down swing shaft 9 o'clock image, the pre-impact image, the impact image, the post-impact image, the follow shaft 3 o'clock image, and the finish image are automatically extracted (S13).
By extracting the checkpoint image from a large number of still images of the swing moving image as described above, the attention point of the golfer 11 can be determined by using the colored marks M1 to M7 of the golfer 11 and contour processing performed in a later process. There is an advantage that the calculation for extracting the position coordinates only needs to be performed for the checkpoint image.

Hereinafter, a method for automatically extracting each checkpoint image will be described.
(Address image)
First, an address image extraction method will be described. The address image is a still image in a state where the golfer 11 is in an address posture.
When shooting of the swing moving image is started from the address state, the initial image is set as the address image. However, when a hitting sound at impact or an impact sensor is acquired as a trigger signal and a moving image within a certain time is acquired before and after that, for example, the initial image is waggle (the head is moved back and forth as a preliminary operation before addressing). The initial image is not necessarily an address image. Therefore, in this case, difference processing is performed between the frames (still images), and the frame with the smallest difference is considered as an address image, assuming that the golfer 11 is stationary.

  Next, a method for extracting the takeback shaft 9 o'clock image, the top image, the downswing shaft 9 o'clock image, the pre-impact image, the impact image, the post-impact image, the follow shaft 3 o'clock image, and the finish image will be described.

  Here, the take-back shaft 9 o'clock image is a still image in a state of being at the 9 o'clock position when the shaft is regarded as a clock hand during take-back. The top image is a still image in a top position state in which a transition from takeback to downswing is performed. The down swing shaft 9 o'clock image is a still image in a state of being at the 9 o'clock position when the shaft is regarded as a clock hand during down swing. The pre-impact image is a still image in a state immediately before impact. The impact image is a still image at the moment when the golf club head collides with the ball. The post-impact image is a still image in a state immediately after impact. The 3 o'clock image of the follow shaft is a still image in a state of being at the 3 o'clock position when the shaft is regarded as a clock hand during follow-through. The finish image is a still image in a state where the swing is finished and the movement of the golf club is stopped.

  Since these checkpoint images are basically determined by tracking the coordinates of the club colored marks CM1 to CM3 in each frame, first, an automatic tracking method for the club colored marks CM1 to CM3 will be described. .

A binarization process for automatically recognizing the club colored marks CM1 to CM3 in the address image is performed. In this embodiment, the target of binarization processing is the entire frame. However, when the golfer 11 is limited to be photographed near the center of the image, as shown in FIG. The binarization process may be performed only in the region S where it is considered that exists. Specifically, the region S is set such that the width of the image is W and the height is H, the range of 1/3 W to 2/3 W is the width of the region S, and 1 / 2H to 4 / 5H. Is the height of the region S.
As a binarization method, RGB values or YIQ values may be used, but in the present embodiment, hue, saturation, and lightness that make it easy to recognize the colors of the club colored marks CM1 to CM3 are used. Yes. Specifically, the RGB value for each pixel on the frame is acquired,
The following equation 2 is normalized using the stimulus sum T obtained by the equation 1.
The RGB values are values from 0 to 255 when the color is expressed by 24 bits.

The hue θ is calculated by the following formulas 3 and 4.
However, using 0 ≦ θ ₁ ≦ π,
And

The saturation S is calculated by the following formula 5.

The brightness V is calculated by the following formula 6.

Pixels whose hue, saturation, and lightness values (pixel color information) calculated by Equations 3 to 6 do not satisfy a predetermined condition (reference color information) are set to 0, and pixels satisfying the condition are colored marks. A binarization process is performed by assuming that the color is the same as that of M1 to M3, and 1 pixel is sequentially labeled.
Here, as conditions for hue, saturation, and brightness, for example, in the case of yellow club colored marks M1 and M3, hue θ = 30 to 60 °, saturation S ≧ 0.5, and brightness V ≧ 100. If a threshold is set and the pink colored mark M2 is set, the threshold of hue θ = 320 ° to 360 ° or 0 ° to 10 °, saturation S = 0.3 to 0.6, and lightness V ≧ 80. By setting, a pixel satisfying the condition is regarded as the same color as the mark.

  Here, for example, there is actually only one pink club colored mark CM2, but if there is an unrelated pink color in the image, two or more regions may be extracted. In consideration of such a case, a mark area range is set in advance, a region having an area outside the set range is determined not to be the club colored mark CM2, and a region having an area within the set range Is recognized as a club colored mark CM2. In this embodiment, the area range recognized as the club colored marks CM1 to CM3 is 5 to 60 or 5 to 200 pixels.

When the pixels recognized as club colored marks CM1 to CM3 as described above are set to 1, 2, and 3 by labeling each of the marks CM1 to CM3, the mark color information and the barycentric coordinates are obtained from the pixels of the respective numbers. Here, the mark color information is information including the average color of each pixel in the region, the maximum / minimum RGB values of each pixel, and the fluctuation range thereof.
By performing the above processing, the club colored marks CM1 to CM3 attached to the shaft 13a of the golf club 13 can be automatically extracted with high accuracy.

Next, a process of automatically tracking the second and third images after obtaining the address image for the club colored marks CM1 to CM3 automatically extracted from the address image is performed.
As shown in FIG. 4, rectangular search ranges S1 to S3 are set for the club colored marks CM1 to CM3 with the club colored marks CM1 to CM3 as the center. Here, the search ranges S1 to S3 refer to ranges on the image to be calculated for performing the detection processing of the club colored marks CM1 to CM3. If the concept of the search ranges S1 to S3 is introduced, the club colored marks CM1 to CM3 will be used even if there are locations that are similar in color to the club colored marks CM1 to CM3 outside the search ranges S1 to S3 on the image. Since the CM3 detection process is performed only within the search range S1 to S3, erroneous recognition can be prevented, and the calculation time can be greatly shortened as compared to the case where all pixels in the frame are processed. In the present embodiment, the search ranges S1 to S3 default to a vertical and horizontal (yx) range of 10 × 10 pixels centered on the club colored marks CM1 to CM3. In the image, the horizontal direction is the x axis and the vertical direction is the y axis, the right direction on the image is the positive direction of the x coordinate, and the downward direction is the positive direction of the y coordinate. In addition, the determination of the arrangement of the search ranges S1 to S3 during the automatic tracking is automatically recognized from the image at the previous time because the shaft 13a hardly moves in the second and third images after the acquisition of the address image. Club colored marks CM1 to CM3 are set as center positions.

Next, a color range is set.
The color range is an error that the color information of the target pixel on the image is considered to be the same as the color (reference color information) of the club colored marks CM1 to CM3 when recognizing the club colored marks CM1 to CM3. In this embodiment, each of R (red), G (green), and B (blue) of the mark color information acquired from the above-described address image is a numerical value that is half of the maximum and minimum width centered on the average value. The range is set as the color range.

In the following automatic tracking processing, tracking is performed in order from CM2 and CM3 in order from the club colored mark CM1 arranged near the grip having a slow moving speed during swing.
First, it is determined whether or not each of RGB of the difference pixels in the search range S1 is in the color range, the pixel in the color range is regarded as a pixel for displaying the colored mark CM1, and the color extracted pixel Get the center of gravity position of the group. If tracking cannot be performed by a method using this color range, color extraction may be performed using color information (hue, saturation, brightness). These processes are performed for the search ranges S1 to S3 of the colored marks CM1 to CM3.
If a plurality of candidate mark areas are extracted within the search range, the color mark CM1 is subjected to a difference process from the background image within the search range S1. As a result, the background image is removed within the search range S1, and even when a color similar to the colored mark CM1 exists in the background image or the like when the colored mark CM1 is recognized in the subsequent process, it is not mistaken.

  Next, a method for setting the center position of the search ranges S1 to S3 of the colored marks CM1 to CM3 in the fourth and subsequent frames from the address image will be described. First, in the case of the club colored mark CM1 closest to the grip, for example, with respect to the fourth frame, the movement vector amount V1 between the first (address) and the second frame and the second and third frames The movement vector amount V2 between sheets is obtained, and the movement vector amount {V2 + (V2-V1)} between the third and fourth sheets is predicted in consideration of the increase amount (V2-V1). The position shifted by the movement vector amount {V2 + (V2−V1)} from the center position of the search range S1 at the previous time is set as the center of the search range S2 of the current image (fourth image). (Same procedure for the fifth and subsequent frames)

With respect to the center position of the search ranges S2 and S3 of the club colored marks CM2 and CM3, for example, for the fourth frame, the amount of movement vector obtained using the club colored mark CM1 whose position has been determined above. It is offset from the center position of the search range S2, S3 of the previous time by {V2 + (V2-V1)}, and the shaft angle D1 between the first and second sheets and between the second and third sheets The shaft angle D2 is obtained, and the shaft angle {D2 + (D2-D1)} between the third and fourth sheets is predicted in consideration of the increase (D2-D1), and the color of the fourth frame The mark M1 is rotated by an angle {D2 + (D2-D1)} using the rotation mark as a rotation fulcrum. (Same procedure for the fifth and subsequent frames)
Thus, by determining the center position of the search ranges S2 and S3 by combining the offset movement and the rotational movement, the shaft position can be predicted fairly accurately even when the movement of the shaft 13a such as down swing is fast. This eliminates the need to increase the areas of the search ranges S2 and S3 during tracking. As shown in FIG. 5, the areas of the search ranges S2 and S3 are 20 × 20 pixels.
If a plurality of mark candidate areas are extracted within the search range, the difference between the colored mark M1 and the background image is performed within the search range S1. As a result, the background image is removed within the search range S1, and even when a color similar to the colored mark M1 exists in the background image or the like when the colored mark M1 is recognized in the subsequent process, there is no misidentification.

  If the club colored marks CM1 to CM3 cannot be tracked even by the above method, rebinarization processing is performed in the same manner as the method of automatically extracting the club colored marks CM1 to CM3 from the address image. That is, the main reason why the club colored marks CM1 to CM3 cannot be found in the color range determined by the address image is to track the club colored marks CM1 to CM3 existing in a darker area than the address image. Therefore, it is assumed that the binarization process is performed again by changing the saturation and brightness thresholds of the club colored marks CM1 to CM3.

If tracking is still impossible, if two of the three colored marks CM1 to CM3 for the club are recognized, the position of the remaining one mark is calculated from the positional relationship with respect to the two marks. I will do it. Alternatively, the center of the search range offset by the above method may be considered temporarily as the mark position at the current time.
As described above, the position coordinate data of the club colored marks CM1 to CM3 in the swing operation from the address to the finish is acquired.

Next, each checkpoint image is extracted based on the acquired coordinate data of the club colored marks CM1 to CM3 in the swing.
(Takeback shaft 9 o'clock image)
The take-back shaft 9 o'clock image is extracted by calculating the angle of the shaft 13a using two of the club colored marks CM1 to CM3 and selecting the image with the shaft 13a closest to the horizontal (90 °). . Alternatively, when one of the club colored marks CM1 to CM3 is used, it may be extracted by selecting an image in which the x-direction component of the mark movement vector is minimized. The angle here is positive when the shaft 13a is at 6 o'clock and 0 °.

(Top image)
The top image is extracted by calculating the angle of the shaft 13a using two of the club colored marks CM1 to CM3 and selecting the image with the largest angle of the shaft 13a. Alternatively, when one of the club colored marks CM1 to CM3 is used, it may be extracted by selecting an image in which the x-direction component and the y-direction component of the mark movement vector are minimized.

(Image of downswing shaft 9 o'clock)
The 9 o'clock image of the downswing shaft calculates the angle of the shaft 13a using two of the club colored marks CM1 to CM3, the shaft 13a is closest to the horizontal (90 °), and the time is from the top image. Is extracted by selecting an image later. Alternatively, when one of the club colored marks CM1 to CM3 is used, it is extracted by selecting an image in which the x component of the mark movement vector is minimum and the time is later than the top image. May be.

(Impact image)
The impact image is extracted by calculating the angle of the shaft 13a using two of the club colored marks CM1 to CM3 and selecting an image in which the angle of the shaft 13a is closest to 0 °. Alternatively, when one of the club colored marks CM1 to CM3 is used, it may be extracted by selecting an image in which the y-direction component of the mark movement vector is minimized. The impact image may be extracted using an external trigger signal, or the impact image may be extracted using the head and ball hitting sound at the time of impact.

(Image before impact)
The pre-impact image is extracted by selecting an image obtained by rewinding a predetermined time (or a predetermined number of frames) set in advance from the extracted impact image.

(Image after impact)
The post-impact image is extracted by selecting an image obtained by advancing a predetermined time (or a predetermined number of frames) set in advance from the extracted impact image.

(Follow shaft 3 o'clock image)
The follow shaft 3 o'clock image is extracted by calculating the angle of the shaft 13a using two of the club colored marks CM1 to CM3 and selecting the image in which the angle of the shaft 13a is closest to -90 °. To do. Alternatively, when one of the club colored marks CM1 to CM3 is used, it is extracted by selecting an image in which the x-direction component of the mark movement vector is minimum and the time is after the impact image. Also good.

(Finish image)
The finish image is extracted by calculating the angle of the shaft 13a using two of the club colored marks CM1 to CM3 and selecting the image having the smallest angle of the shaft 13a. Alternatively, when one of the club colored marks CM1 to CM3 is used, the x direction component and the y direction component of the movement vector of the mark are minimized, and an image whose time is later than the top image is selected. It may be extracted.

Next, a method for extracting the takeback left arm horizontal image and the downswing left arm horizontal image will be described.
Here, the takeback left arm horizontal image is a still image in which the forearm portion of the left arm is in a horizontal state during takeback. The downswing left arm horizontal image is a still image in which the forearm portion of the left arm is in a horizontal state during downswing.

In order to recognize an image in which the left arm is horizontal, a template which is an image region including the left arm is created, and an image in which the template angle matched by the template matching process is horizontal is set as the left arm horizontal.
Hereinafter, the contour of the golfer 11 is extracted in order to create a template including the left arm in the still image.

First, an image in which the shaft 13a is in the 6 o'clock state is extracted based on the shaft angle acquired from the coordinates of the club colored marks CM1 to CM3. The grip position is determined by obtaining an inter-mark vector from the club colored mark CM1 closest to the grip and the next club colored mark CM2 closest to the grip. In particular,
(Grip position) = (position of colored mark CM1 for club) −α × (vector between marks)
To calculate the grip position.
Here, α means the ratio of the actual distance between the club colored mark CM1 and the grip to the distance between the club colored mark CM1 and the club colored mark CM2, and in this embodiment α = 0.5.

Next, a background difference process is performed on the shaft 6 o'clock image with a background image (an image in which the golfer 11 is not shown) to extract the silhouette of the golfer 11. Specifically, if the RGB values in the background image are r ′, g ′, b ′, and the RGB values of the pixels on the shaft 6 o'clock image are r, g, b, respectively, If the square root of the sum of the squares of the absolute values of the difference between r, g, b and r ′, g ′, b ′ is less than a preset threshold value, it is regarded as not the silhouette of the golfer 11 and the pixel is set to 0. If it is equal to or greater than the threshold, the binarization process is performed assuming that the silhouette of the golf player 11 is 1, and the labeling process is sequentially performed on one pixel. In the present embodiment, the norm threshold is 40. In addition, background difference processing may be performed using hue, saturation, and lightness. In this case, one or two regions of 5000 or more or 10,000 or more of the labeling regions regarded as silhouettes are used as silhouettes. .

  As shown in FIG. 6A, a scanning process is performed on the binarized image, and contour extraction is performed from the point where it hits one or two pixels. In this contour extraction method, the labeled image is scanned from the top to the bottom in the right direction starting from the upper left pixel of the frame, and the contour is extracted by searching for one or two pixels. Specifically, the pixel of (4, 7) is first found by the scanning process, and as shown in FIG. 5B, the seven pixels around the pixel excluding the pixel immediately before this pixel are clockwise from the upper left pixel. The pixel having the same label as the pixel (1 or 2) first found at (4, 7) is set as the next boundary point. This process is sequentially performed, and the contour extraction is finished when the boundary point returns to (4, 7). Since noise remains in the extracted contour, smoothing is performed by circulating the moving average process over the entire contour.

The moving average process is performed by the following formula 8.

Here, bnd_pt (n) is the coordinate of the nth contour, k is the number of pixels before and after the calculation, and bnd_pt_ido (n) is the coordinate of the contour after moving average.
However, in the case where the contour of the golfer 11 exists from the first to the bnd_num-th (end of the contour number), if the pixel to be subjected to the moving average is n-th, if n <k, A moving average is performed using a certain bnd_num- (kn) -th to bnd_num-th pixel. When bnd_num-n <k, the moving average is performed using the first to k- (bnd_num-n) -th pixels that are the first in the contour number.

  Next, the curvature is calculated from the smoothed contour data, and the left shoulder position of the golfer 11 is obtained. That is, the large curvature portion of the contour 21 that appears first when scanning from the top of the image including the contour data as shown in FIG. 7 is used as the head, the small curvature portion that appears next as the neck, and the large curvature portion that appears next. Recognize it as a shoulder. Here, taking into account wrinkles of clothes and the like, it is preferable to calculate the curvature of each of ± 5 pixels before and after and calculate the average value as the curvature of the center pixel.

Hereinafter, a method for calculating the curvature of the contour will be described.
If the length of the arc of the target contour is S and the angle is θ, the curvature C is expressed by the following formula 9.

Since this equation 9 is calculated only with the pixel for which the curvature is to be obtained and the adjacent points of the pixel, the value fluctuates greatly and a correct value cannot be obtained. The method of calculating by the following formula 10 is used.

Note that Equation 10 is simplified by omitting S in Equation 9. In this embodiment, for the sake of simplicity, as shown in FIG. 9, the curvature C is calculated by the following formula 11 using both ends of the point sequence.

(Takeback left arm horizontal image)
As shown in FIG. 8A, a rectangular template T is set in the region between the left shoulder 22 and the grip 23 extracted as described above, and the length L1 of the long side of the template T is set to the shoulder. The distance between the grips is half of the distance, and the short side length L2 is set to a length that allows the arm to enter the template T (20 pixels in this embodiment).

  The image at the next time is read to acquire the grip position, and the template T of the previous frame is translated in the same manner as the grip position movement vector. Next, as shown in FIG. 8B, the template T is rotated clockwise in increments of 1 ° up to 10 ° with the grip position as a fulcrum, and the angle of the most matched template is calculated, and the template angle is 90 ° (horizontal The image closest to) is extracted as a takeback left arm horizontal image. Further, the matching process may be performed by adding translation to the rotation of the template.

In the template matching process, the RGB value, which is the color information of the pixels in the template T, is converted into the luminance Y by the following formula 12 and evaluated. (Although evaluation may be based on luminance Y, an RGB norm (see Equation 7) may be used.)
For the evaluation, the sum of absolute values (Sum of Absolute Difference: SAD) of pixel value differences expressed by the following Equation 13 is used.

Here, t is the current frame, t-1 is the frame one frame before, (p, q) is the range for translation, (i ₀ , j ₀ ) is the grip position, and m is the long side of the template T. The number of pixels, n is the number of pixels on the short side of the template T, θ is the rotation angle of the template T, α is the template angle obtained one frame before, and g _t (x, y, θ) is the coordinate (x, y) This is a function that represents the luminance Y (or RGB nomels) of a pixel at a template angle θ.

  The position / angle (p, q, θ) of the template T is changed under the above conditions to calculate S (p, q, θ), and it is considered that the best matching is achieved at the position / angle at which this value is minimum. It is said. An image in which the θ value of (p, q, θ) at the time of matching is closest to 90 ° is extracted as a takeback left arm horizontal image.

(Down swing left arm horizontal image)
For the down swing 9 o'clock image, a template including the left arm in the acquired take back left arm horizontal image is used, and the image after the top image that best matches the template is extracted as the down swing left arm horizontal image.
Here, considering the order of the swing images, it is known that the down swing left arm horizontal image is extracted after the top image, so the template matching process may be started from the top image, but it takes time, Or, it is not always possible to see all the arms at the top, so there is a possibility of misrecognition.
Therefore, in this embodiment, the downswing shaft 9 o'clock image is extracted first, and then the downswing left arm horizontal image is extracted by performing template matching while returning the time, thereby reducing calculation time. It prevents misrecognition.

(Takeback shaft 8 o'clock image)
Next, a method for extracting the takeback shaft 8 o'clock image will be described. The takeback shaft 8 o'clock image is a still image in a state of being at the 8 o'clock position when the shaft is regarded as a clock hand during takeback.
By extracting the silhouette of the golfer 11 in the above-mentioned shaft 6 o'clock image, the width of the body (stance width) is obtained, and the time when the perpendicular line passing through the right foot side end and the club colored mark M1 intersect is obtained. The image is selected and extracted as the takeback shaft 8 o'clock image.

As described above, the front checkpoint image as shown in FIG. 10 (address image, takeback shaft 8 o'clock image, takeback shaft 9 o'clock image, takeback left arm horizontal image, top image, down swing left arm horizontal image, down Swing shaft 9 o'clock image, pre-impact image, impact image, post-impact image, follow shaft 3 o'clock image, and finish image) can be extracted, and the color CCD cameras 14 and 15 synchronize the photographing timing with each other. Therefore, by selecting a checkpoint image at the same time as the front checkpoint image, a checkpoint image behind the flying ball line (side surface) as shown in FIG. 11 can be extracted.
Hereinafter, for each of the checkpoint images, the position coordinates of the attention point necessary for the swing diagnosis of the golfer 11 are acquired.

  Silhouette extraction is performed for each checkpoint image, and the contour R of the golfer 11 is extracted as shown in FIG. A curvature is acquired for the contour R, and a straight line portion ST in the contour R is acquired as shown in FIG. The method for acquiring the silhouette / contour / curvature is as described above. Further, in the extraction of the straight line portion ST, the straight line portion ST is a portion where the pixels having a curvature within a range of −10 ° to 10 ° in the acquired contour R are continuous by 5 pixels or more.

Next, the position coordinates of the attention point of the golfer 11 are acquired by recognizing the coordinate positions of the colored marks M1 to M7 of the measurement clothes 12 worn by the golfer 11 using the search range S.
For example, in the method of extracting the right shoulder of the address image (front), the curvature of the contour R is examined counterclockwise from the upper end (head), and the pixel having the extreme curvature is recognized as the right neck, as shown in FIG. As described above, the search range S is determined with the range of −40 pixels in the y direction and +40 pixels in the x direction using the right neck as an end point of the search range S.
Difference processing with the background image is performed within the search range S, and whether or not each of the difference pixels RGB within the search range S is within the color range of the colored mark M4 (blue) or M5 (red) on the right shoulder. The pixel within the color range is regarded as a pixel displaying the colored mark M4 or M5, and color extraction is performed to obtain the barycentric position coordinate.

At this time, the color ranges of the colored marks M1 to M7 are set as shown in Table 1, and pixels satisfying the conditions are regarded as the same color as the colored marks M1 to M7.

  It should be noted that the area range of the mark is set in advance, the area having an area outside the setting range is determined not to be the colored marks M1 to M7, and the area having the area within the setting range is determined as the colored marks M1 to M7. Recognition accuracy is improved. In this embodiment, the area range recognized as the colored marks M1 to M7 is 5 to 60 pixels.

  Next, when the colored marks M1 to M7 are hidden or become shadowed and the recognition of the colored marks M1 to M7 using the search range S fails, or the colored marks M1 to M7 are present. When it is desired to recognize a point of interest that is not to be recognized, the position coordinates of the point of interest are extracted using the contour information (coordinates of the contour R, etc.) of the golfer 11 or the curvature of the contour R.

  For example, as shown in FIG. 15, the extraction method of the head 24 in the address image and the impact image (side surface) is the starting point of contour extraction of the contour R, that is, the pixel having the smallest y coordinate among the pixels constituting the contour R. Is extracted as head 24. Further, in the neck K extraction method, the curvature of the pixel is examined along the contour R from the head 24, and the midpoint of the points B1 and B2 at which the curvature becomes the maximum value is extracted as the neck K.

Further, as shown in FIG. 16, the extraction method of the right foot tip 25 in the address image and the impact image (side surface) is obtained by examining counterclockwise from the lowest point (y coordinate maximum value) of the contour R of the golfer 11 and x The pixel having the maximum coordinate is extracted and set as the right foot 25.
The right ankle 26 is extracted by examining the contour counterclockwise from the right foot tip 25 to extract a pixel P1 having a minimum x coordinate, and another pixel on the contour R having the same y coordinate as the pixel P1. P2 is obtained, and the average of the coordinates of the pixel P1 and the coordinates of the pixel P2 is set as the coordinates of the right ankle 26.
In the method of extracting the left foot tip 27, the contour R is examined counterclockwise from the pixel P1, and the pixel having the maximum x coordinate is defined as the left foot tip 27.

  Next, when the recognition of the attention point by the colored marks M1 to M7, the contour information or the curvature fails, or when it is desired to recognize the attention point where the colored marks M1 to M7 do not exist, or the recognition by the curvature of the contour R When it is difficult, the position coordinates of the target point are extracted using the straight line portion ST extracted on the contour R of the golfer 11.

  For example, as shown in FIG. 17, the extraction method of the right knee 28 in the address image and the impact image (side surface) is based on each of the straight portions ST1 and ST2 above and below the region where the right knee 28 of the contour R is considered to exist. The pixel at the position where the extension lines intersect is specified as the position coordinate of the right knee 28. Specifically, the provisional knee region is assumed to be 30% to 40% from the bottom of the height of the contour R, and it is examined whether the lower end of the straight line portion ST1 and the upper end of the straight line portion ST2 exist in the temporary knee region. At this time, if a plurality of straight portions are found, the straight portion close to the provisional knee region is selected. The value of the maximum value of the x coordinate among the intersections of the extension lines of the two straight portions ST1 and ST2 obtained in this way or the points where the y coordinate and the contour R coincide with each other is defined as the right knee 28. If the straight portions ST1 and ST2 cannot be found because the knee is not bent, the pixel having the maximum x coordinate of the contour R in the provisional knee region is set as the right knee.

As shown in FIG. 18, the method of extracting the wrist 29 in the top image (side surface) is to first scan the initial point I of the silhouette S of the golfer 11 (from left to right and up to down in order from the pixel at the top left of the image). The first pixel found in this case is obtained, and it is determined whether or not the x coordinate of the initial point I is smaller than the x coordinate of the back neck A obtained by the method described later in the takeback left arm horizontal image. If it is small, a straight line portion ST3 having an inclination of 90 ° to 180 ° and a straight line portion ST4 having an inclination of −90 ° to −180 ° with the initial point I as a temporary wrist and the initial point I as a center are extracted. The intersection of the straight line part ST3 and the straight line part ST4 is defined as a wrist 29. If the straight line portion cannot be extracted, the initial point I is set as the wrist.
In the image, the horizontal direction is the x axis and the vertical direction is the y axis, the right direction on the image is the positive direction of the x coordinate, the downward direction is the positive direction of the y coordinate, and the angle is the negative x direction. As a base point, clockwise is a positive angle and counterclockwise is a negative angle.
On the other hand, when the x coordinate of the initial point I is larger than the x coordinate of the back neck A, as shown in FIG. 19, after extracting the skin color described later to obtain the face H1 of the golfer 11, the skin color of the hand is extracted. The center of the skin color region H2 of the hand is the wrist.

Next, as in the case of the right knee in the top image (side surface), if there is no colored mark and it exists inside the contour R, it is difficult to specify using the contour R. The edge is extracted based on the change in the brightness of the pixel on the image, and the body line (including the inside of the contour) of the golfer 11 is acquired to recognize the position of the point of interest.
The procedure for extracting the position coordinates of the right knee in the top image (side surface) will be specifically described below. The basic flow of edge extraction is creation of an edge intensity image → creation of a direction labeling image → creation of a non-maximum value suppression labeling image.

First, an edge strength image as shown in FIG. 20 is created.
Edge strength is extracted from the top image (side surface) using the Sobel operator.
Here, the Sobel operator approximates the change in brightness by weighting the brightness around the center point in order to obtain the place where the brightness changes abruptly (= edge) on the image. In the following, calculation using the RGB values of the color image is shown.

(1) Sobel intensity in the x direction of a color image
Here, as shown in FIG. 23, when the current pixel of interest is E and the eight surrounding pixels are A to D and F to I, Ar is the R (red) value of the A pixel, and Ag is The G (green) value of the A pixel and Ab means the B (blue) value of the A pixel. The same applies to Br, Bg, Bb, to Ir, Ig, and Ib. Moreover, the coefficient of each variable means each weighting.
The direction of the Sobel intensity is sign = 1 when Rx + Gx + Bx> 0, and sign = −1 when Rx + Gx + Bx <0.
Sobel strength Dx in the x direction is
(2) Sobel intensity in the y direction of a color image
(The direction of Sobel intensity is sign = 1 when Ry + Gy + By> 0, and sign = −1 when Ry + Gy + By <0.)
Sobel strength Dy in y direction is
(3) Sobel strength of color image Sobel strength (edge strength) DD of color image is
Is required. Edge strength of pixels whose DD is larger than a threshold value (10 in the present embodiment) with Sobel intensity (edge intensity) and 0 or less below the threshold value and edge-displayed with 256-tone sobel intensity of 0 to 255 as shown in FIG. An image is obtained.

Next, a direction labeling image as shown in FIG. 21 is created using the edge strength in each direction.
The direction is classified into the following four directions.
When tangent = dy / dx (dx is the Sobel intensity in the x direction and dy is the Sobel intensity in the y direction)
When tangent <-tan (3 / 8π), it is labeled “3” and displayed in green.
When -tan (3 / 8π) ≦ tangent <−tan (1 / 8π), label with “4” and display in red,
When -tan (1 / 8π) ≦ tangent <tan (1 / 8π), label with “1” and display in white,
When tangent <tan (3 / 8π), label with “2” and display in blue,
In cases other than the above, “3” is labeled and displayed in green, and the direction labeling image shown in FIG. 21 is obtained.

Next, a non-maximum value suppression labeling image as shown in FIG. 22 is created using the edge intensity image and the direction labeling image.
The non-maximum suppression image is an image obtained by extracting a portion where the intensity is maximum as an edge using the acquired edge intensity and the change direction of brightness.
To classify non-maximal images into four directions:
(1) Regarding the horizontal direction (x direction), when the intensity of the center pixel is the highest among the front and rear of the center pixel and the Sobel intensity of the center pixel, it is regarded as the maximum position (white).
(2) Regarding the diagonal direction (upper left / lower right direction), when the intensity of the central pixel is the highest among the front and rear of the central pixel and the Sobel intensity of the central pixel, it is regarded as the maximum position (red).
(3) In the longitudinal direction (y direction), if the intensity of the central pixel is the highest among the front and rear of the central pixel and the Sobel intensity of the central pixel, it is regarded as the maximum position (green)
(4) Regarding the diagonal direction (lower left / upper right direction), when the intensity of the central pixel is the highest among the front and rear of the central pixel and the Sobel intensity of the central pixel, it is regarded as the maximum position (blue).
As described above, the edge-extracted non-maximum value suppression labeling image shown in FIG. 22 is acquired.
Finally, the position coordinates of the right knee are acquired when the label of the horizontal line passing through the height of the left knee acquired by the straight lines ST1 and ST2 of the contour R and the non-maximum value suppression labeling image is “1” as described above. The intersection with the part (white) is the right knee.

  Next, when it is difficult to recognize a point of interest by coloring marks M1 to M7, contour information, curvature, or edge extraction, or when it is desired to recognize a point of interest where the colored marks M1 to M7 do not exist, a golfer The position coordinates of the target point are extracted using the eleven silhouette information.

  As an example, as shown in FIG. 24, the method for obtaining the grip width W, which is the distance between the body and the grip in the address image (side surface), is the two club colored marks CM1 attached to the shaft 13a as described above. The grip position 30 is obtained from the inter-mark vector passing through the position of CM2, and the pixel P3 on the contour R having the same y coordinate as the grip position 30 is obtained. However, it is a condition that the x coordinate of the pixel P3 is smaller than the x coordinate of the right foot tip. A distance between the grip position 30 and the pixel P3 is defined as a grip width W.

  The position coordinates of the attention point of the golfer 11 in the acquired checkpoint image are stored and held in the memory of the computer 16. As described above, the same calculation is performed until all the attention points necessary for the swing diagnosis are extracted. In other words, instead of extracting the position coordinates of all the points of interest of the golfer in the checkpoint image, it is only necessary to extract the points of interest required for swing diagnosis in each checkpoint image, and further increase the calculation time. Shortening is achieved.

  In addition, since a mechanism for extracting a point of interest by combining a plurality of algorithms without depending on one image processing algorithm, even if the body shape and swing motion of the golfer 11 are significantly different, all points of interest extraction methods are used. Thus, the recognition rate of the position coordinates of the attention point of the golfer 11 can be greatly improved. Further, with respect to the lower body of the golfer 11, the position coordinates of the attention point are extracted by using the above-described contour information, edge extraction or the like instead of recognizing the attention point such as the knee by color extraction using colored marks. Therefore, there is a merit that the golfer 11 does not have to wear the measurement clothes on the lower body.

(Extract attention point in horizontal image (side) of takeback left arm)
Next, as an example of extracting a point of interest in one checkpoint image by combining a plurality of algorithms, a case where the position coordinates of each point of interest in the takeback left arm horizontal image (side surface) are extracted will be described as a representative.

As shown in FIG. 25, the contour extraction process described above is first performed (S100). If the contour R cannot be obtained (S101), the image processing relating to this checkpoint image is terminated (S102), but the contour R A detailed description will be given of the case where the image is removed.
(club)
As described above, the color coordinates of the club colored marks CM1 to CM3 are extracted using the search range S to obtain position coordinates. If it cannot be extracted (S104), it is determined to be a failure, and if it can be extracted, the position coordinates of the club colored marks CM1 to CM3 are stored (S105).

(Grip end and grip center)
If the club colored marks CM1 to CM3 have not been extracted in S104 (S106), it is determined to be unsuccessful. If the club colored marks CM1 to CM3 have not been extracted, the club colored mark CM1 closest to the grip and the club colored mark closest to the next are identified. A mark-to-mark vector is obtained from CM2 to determine the grip end and the grip center. In particular,
(Grip end) = (position of club colored mark CM1) −A × (mark-to-mark vector)
(Grip center) = {(position of colored mark CM1 for club) + (grip end)} / 2
Calculated by
Here, A is the ratio of the actual distance between the club colored mark CM1 and the grip end to the distance between the club colored mark CM1 and the club colored mark CM2.

(Ground)
The position coordinate of the pixel at the lowest point of the contour R described above is regarded as the ground (S107).
(Back neck)
After narrowing the area where the head is present from the silhouette area of the golfer 11 and subtracting the background in that area, the hue θ = 0 to 30, R = 20 to 240, G = 20 to 180, and B = 180 or less are satisfied. The pixel collection region is regarded as skin (S108). In addition, the threshold value of the area when the region is regarded as skin is 30 to 1400 pixels. As shown in FIGS. 27 (A) and 27 (B), the contour point of the extracted face area of the skin color region H where the x coordinate is the minimum is O, and the contour point where the x coordinate is the same as the point O and the minimum is the y coordinate. A is a contour point that has the same y coordinate as that of the point O and a minimum x coordinate, and a straight line passing through the point O is drawn perpendicular to the slope of the straight line AB, and the intersection of the straight line and the contour is the rear neck 31. (S110).

If the skin color cannot be extracted (S109), and the position coordinate of the back neck is obtained in the address image (side surface) (S111), the x coordinate of the back neck position in the address image (side surface) And the intersection of the takeback left arm horizontal image with the contour R, and the curvature of 10 pixels before and after the pixel on the contour R having the smallest y coordinate is extracted from these intersections (S112). To be the rear neck (S113, S110).
When the rear neck cannot be obtained from the address image (side surface) (S111) or when extraction by the curvature cannot be performed (S113), the contour R having a y-coordinate of 90% position from the bottom of the silhouette of the golfer 11 Of the upper pixels, the pixel having the smallest x coordinate is extracted as the back neck (S114, S110).

(Right waist y coordinate)
First, the search range S is defined as a range of -40 to 40 pixels in the x direction and -60 to 20 pixels in the y direction starting from the average height (y) of the ground and golfer silhouette vertices and the silhouette center of gravity (x). The belt-like colored mark M7 is color-extracted to obtain the y coordinate of the right waist (S115, S116).
When the color extraction fails, the height position of 60% from the bottom of the silhouette of the golfer 11 is regarded as the y coordinate of the right waist (S117).

Next, in order to extract both shoulders and both elbows, the subroutines shown in FIGS. 25 to 26 are called.
(Left shoulder)
When the right shoulder of the address image (side surface) has already been acquired (S118), as shown in FIG. 28A, the position within the range of +40 pixels in the x direction and ± 40 pixels in the y direction, starting from the right shoulder The search range S is set to the color of the left shoulder colored mark M3 (S119). If the extraction of the colored mark M2 is successful, the coordinates are stored as the left shoulder (S120, S121).

When the right shoulder cannot be extracted from the address image (side surface) (S118) and when the color cannot be extracted (S120), as shown in FIG. 28 (B), the contour coincides with the y coordinate of the rear neck 31. The pixel on R is extracted as the lower neck 32 (S122). The left shoulder is determined using the undulation information of the contour R from the lower neck 32 (S123).
Specifically, when the contour R is examined clockwise from the neck 32 and the peak (maximum point) is found first, the mountain from the neck 32 to 30 pixels clockwise → from this mountain to 30 pixels clockwise. Valley (minimum point) → A mountain up to 20 pixels clockwise from this valley is the left shoulder 35.
On the other hand, when the contour R is examined clockwise from the lower neck 32 and the valley (minimum point) is found first, the valley from the lower neck 32 to 30 pixels clockwise to → 20 pixels clockwise from this valley. Is the left shoulder 35. (S124, S121)

When the lower neck 32 cannot be extracted (S122), when the contour R cannot be extracted using the undulation (S124), as shown in FIG. 28C, the right waist (S125) is moved in the y direction. On the other hand, the straight line portion ST of the contour R is extracted in the region between [right waist y coordinate] and [right waist y coordinate-30 pixels] (S126), and the intersection of the straight portion ST and the contour R is regarded as the left shoulder 36. (S127, S121). However, the intersection is an area of 80% to 90% from the bottom of the silhouette of the golfer 11, and the y coordinate in the area is the minimum pixel. If a straight line portion cannot be extracted, a straight line is extracted by the least square method using a contour between [right waist y coordinate] and [right waist y coordinate-30 pixels].
When the right waist is not found (S125) or when the left shoulder is not found in 80% to 90%, as shown in FIG. 28D, the position 85% from the bottom of the silhouette (L2 / L1 = The pixel having the largest x in the contour R of 0.85) is set as the left shoulder 37 (S128, S121).

(Right shoulder)
If the rear neck 31 has been extracted (S129), the search range S is set at a position within a range of −50 pixels in the x direction and ± 20 pixels in the y direction starting from the rear neck 31, and the right shoulder is colored. The color of the mark M4 is extracted (S130). If the extraction of the colored mark M4 is successful, the coordinates are stored as the right shoulder (S131, S132).

  If the color extraction fails (S131), if the club colored marks CM1 to CM3 have been extracted (S132), the re-contour extraction is performed to remove the shaft 13a where the shaft 13a intersects the contour R from the contour R. Is determined (S133). If the contour of the shaft 13a intersects the contour of the body (S134), re-contour extraction is performed to remove the contour of the shaft 13a from the contour information (S135). ). If the shaft does not appear in the contour, the process moves to a straight line extraction process (S136) without extracting the contour again.

  It is determined whether or not a straight line portion ST having an inclination of 90 ° to 180 ° exists in the contour R of the takeback left arm horizontal image at a position within ± 10 from the y coordinate of the back neck in the takeback left arm horizontal image (side surface). In addition, it is determined whether or not there is a straight line portion with an inclination of 0 ° to 90 ° in the downward direction between the rear neck and the right waist y-coordinate. If two straight line portions are found, the intersection point is set as the right shoulder. (S137, S132).

  When the back neck cannot be found (S129), when the club colored marks CM1 to CM3 cannot be extracted (S132), when two straight lines are not extracted (S137), the position 80% from the bottom of the silhouette A pixel having the smallest x in the contour R is defined as a right shoulder (S138, S132).

(Left elbow)
If the grip end and left shoulder cannot be extracted (S139), it will be a failure, and if it is extracted, the x direction will be in the range from the grip end to the left shoulder, and the y direction will be in the range from [left shoulder] to [grip end + 40 pixels]. A search range S is set, and color extraction of the left elbow colored mark M1 is performed (S140). If the colored mark M1 is successfully extracted, the coordinates are stored as the left elbow (S141, S142). If the extraction of the colored mark M1 fails, the middle point of the left shoulder and the grip end is regarded as the left elbow (S143, S142).

(Right elbow)
When the right waist of the grip center and the address image (side surface) can be extracted (S144), the search range S is located within the range of ± 50 pixels in the x direction and ± 50 pixels in the y direction starting from the grip center. In addition, the skin color portion extracted above, the vicinity of the shaft, and the area below the right waist on the address image (side surface) are excluded from the search target, and the color extraction of the colored mark M6 on the right elbow is performed. This is performed (S145). If the colored mark M6 is successfully extracted, the coordinates are stored as the right elbow (S146, S147).

If the right waist of the grip center and the address image (side) is not extracted (S144), or if the color cannot be extracted (S146), the left elbow has the same y coordinate and the minimum x coordinate is the starting point. The search range S is set at a position within a range of ± 25 pixels in the x direction and ± 25 pixels in the y direction, and the color of the colored mark M6 of the right elbow is extracted (S149). (S150).
When the left elbow cannot be extracted (S148), the contour point having the minimum x coordinate and 65% in the -y direction from the silhouette ground is set as the right elbow (S151, S147). When color extraction cannot be performed (S150), the starting point of the search range used in the second color extraction is set as the right elbow (S147).

(Spine axis)
Returning to the flowchart of FIG. 25, if the right waist and back neck can be extracted and the right shoulder can be extracted by color extraction (S152), a straight line portion ST exists in the contour R between the right shoulder and the right waist. If this is done (S153), the spine axis (the line connecting the center of both hips and the neck) can be obtained by performing an offset movement so that the straight portion passes through the rear neck 31 while maintaining the angle (S154, S155).
If S152 cannot be extracted, the right waist and the back neck cannot be obtained (S156). On the other hand, if the right waist and the rear neck cannot be obtained, the failure is determined. Using the contour R between the contour point with the smallest coordinate and the back neck, the straight line portion is least squared to be the spine axis (S157, S158, S155).

(Right waist x coordinate)
If the right waist y-coordinate and the spine axis cannot be extracted (S159), the process is unsuccessful. The right waist x coordinate can be acquired (S160, S161).

  As described above, with respect to the takeback left arm horizontal image (side surface) in the flow of FIG. 25 and FIG. 26, the position coordinates of the target point can be acquired using a plurality of image processing algorithms.

On the other hand, as shown in FIG. 2, the ball behavior is calculated by the computer 16 based on the information acquired by the ball motion measuring device 20 (S16).
In other words, it detects that the golf club 13 swung by the golfer 11 has passed between the projector 25 and the light receiver 26, outputs a trigger signal, and the CCD camera 21 that has received the trigger signal continuously opens and closes the multiple shutters 22. At the same time, each flash 23 emits light sequentially in synchronization with the opening and closing, and a plurality of flying balls B are photographed in one video frame. From this frame image, the deflection angle of the ball B (the right and left angles of the ball trajectory with respect to the planned traveling direction (straight) of the ball) is calculated using the same method as in Japanese Patent Laid-Open No. 2001-264016, and the ball The side spin amount (rotation amount) is calculated.

Based on the side spin amount and deflection angle obtained as described above, a ballistic pattern is calculated by the classification method shown in Table 2 (S17). For example, if the side spin amount is 200 rpm or more in the left rotation and the swing angle is 2 degrees in the left direction, the ballistic pattern is determined to be “pull hook”.
It should be noted that the trajectory pattern is specially set to “straight (draw)” when the distance from the right and left of the ball drop point (carry) in the straight direction is within 5 years. Here, the “left / right shift distance” is a distance shifted in the left / right direction of the carry calculated using the trajectory calculation from the side spin amount and the swing angle. For example, when the swing angle is 4 deg to the right and the side spin amount is 500 rpm to the left, the classification shown in FIG. 29 is classified as straight (draw) if the place that becomes a push hook is flying almost in the middle by ballistic calculation. I have to.

  Swing diagnosis is performed based on the trajectory pattern (S17) acquired as described above and the coordinate data (S14) of the point of interest in the checkpoint image (S18). That is, the diagnosis items for subsequent diagnosis are prepared according to the ballistic pattern.

Hereinafter, setting of ideal values of diagnostic items prepared for each checkpoint image will be described with reference to Tables 3 and 4.
In the image, the horizontal direction is the x axis and the vertical direction is the y axis, the right direction on the image is the positive direction of the x coordinate, and the downward direction is the positive direction of the y coordinate.

"Diagnosis with front image"
First, diagnostic items for each checkpoint image in the front image will be described.

(Address status)
As shown in Table 3, in the address state, the diagnostic items viewed from the front are (No. 1) the ball position, (No. 2) the width of the stance, (No. 3) the balance of the upper body (shoulder) , (No.4) Lower body balance (waist) diagnostic items.
Regarding the determination values of (No. 1) to (No. 4), (No. 1) is [x coordinate of ball B position] − [x coordinate of left footpad] = − 5 to 5 cm, (No. 2) is [Width of both feet] / [Width of both shoulders] = 1.5-1.80, (No. 3) is [Midpoint of both shoulders]-[Midpoint of both footpads] =-6-3 cm, (No .4) is [midpoint of both hips (umbilical)]-[midpoint of both footpads] = -3 to 3 cm.

(Takeback shaft at 8 o'clock)
When the takeback shaft is at 8 o'clock, diagnostic items viewed from the front are (No. 5) orientation of the grip end, (No. 6) angle with the ideal shaft line, and (No. 7) right from the address. Each diagnosis item of shoulder movement is provided.
Regarding the judgment values of (No. 5) to (No. 7), (No. 5) is [x coordinate of intersection of shaft line and straight line connecting waist height line]-[mid point of waist at address] (Umbilicus) x coordinate] = − 10 to 10 cm, (No. 6) is [angle between shaft line and y axis] −60 deg = −20 to 20 deg, (No. 7) is [y coordinate of right shoulder] − [Y-coordinate of right shoulder at address] = 10 cm or less.

(Takeback left arm horizontal state)
In the takeback left arm horizontal state, a diagnostic item of (No. 8) list angle is provided as a diagnostic item viewed from the front.
The determination value of (No. 8) is an angle between [shaft line] and [center line of template T from which left arm horizontal is extracted] = 140 deg or less.

(Top state)
In the top state, the diagnostic items viewed from the front are (No.9) Overswing, (No.10) Reverse, (No.11) Left / right movement, (No.12) Right knee position, (No .13) There is a diagnostic item for warping of the upper body.
The determination values of (No. 9) to (No. 11) are as follows: (No. 9) is [angle between [shaft line] and [y axis] ≦ 300 deg, (No. 10) is described later (No. 9). 11) [{x-coordinate of right contour point of umbilical line}-{x-coordinate of right foot}]-[{x-coordinate of right contour point of umbilical line at address}-{x coordinate of right foot side end}] If it is greater than 0, “no weight shift” is set, and if it is 0 or less, “too much sway” is set, and (No. 12) is [x coordinate of contour point of right knee height] − [x coordinate of right foot side end] For> 0, (No. 13), a straight line portion existing between −50 pixels in the y direction along the contour is extracted from the right contour point of the umbilical height, and the average value of the inclination is less than 90 °. The extraction of the straight line portion is the same as the extraction method of the straight line portion ST described above. However, when the straight line portion cannot be extracted, it is between −50 pixels from the right contour point of the umbilical height in the y direction. The contour is linearly interpolated.

(Down swinging shaft at 9 o'clock)
In the down swing shaft 9 o'clock state, as a diagnosis item viewed from the front direction, (No. 14) The diagnosis item of the wrist angle change from the down swing left arm horizontal state to the down swing shaft 9 o'clock state is provided. Yes.
The determination value of (No. 14) is an angle between [shaft line] in the left arm horizontal state and [center line of the template T from which the left arm horizontal is extracted], and [shaft line] in the 9 o'clock state of the shaft. The difference from the angle between [a straight line connecting the wrist and the left shoulder] is <60 deg.
That is, when the difference between the wrist angle in the downswing non-dominant arm horizontal image and the wrist angle in the downswing shaft 9 o'clock image is 60 degrees or more, it is diagnosed that there is a cock action. As another method, it may be diagnosed that the cock operation is present when the downswing left arm level or the downswing shaft at 9 o'clock is a predetermined angle or more.

(Impact state)
In the impact state, the diagnostic items viewed from the front are (No.15) Amount of lateral movement, (No.16) Amount of vertical movement, (No.17) Distance between head and ball, (No.18) Shoulder Each diagnostic item includes the amount of rotation, (No. 19) Spine axis inclination, and (No. 20) Left elbow angle.
Regarding the judgment values of (No. 15) to (No. 20), (No. 15) is [{x coordinate of left contour point at umbilical height in impact state}-{x coordinate of left foot side end in impact state} ] Is less than 0, it is determined to be OK, and if it is smaller than [{x coordinate of the left contour point at the address}-{x coordinate of the navel at the address}], it is assumed that the body weight is not moved. (No. 16) is [umbilical y-coordinate at address]-[umbilical y-coordinate at impact] =-10 to 10 cm, (No. 17) is [ball x-coordinate]-[head x-coordinate] > 0, (No. 18) is [x-direction distance between shoulders in address state] / [x-direction distance between both shoulders in top state] = 1.1 to 1.6, (No. 19) is [ Angle between the line connecting the head and navel and the straight line connecting the right contour point of the navel height to the navel> 70 deg, (No. 20) is the line connecting the left shoulder and the left elbow and the left elbow and the grip end Angle> 170 deg.

Next, diagnosis items for the (No. 10) turn-back determination in the top image will be described.
FIG. 30A shows a difference silhouette obtained by performing inter-frame difference processing between a top image and an image that has passed a predetermined time (80 msec in this embodiment) from the top image. FIG. 30B shows the silhouette of the golfer 11 in the top image.
Here, the area of the difference silhouette near the left shoulder in FIG. 30A is A, the area of the difference silhouette near the left waist is B, the area of the difference silhouette that is the lower arm region is C, and FIG. Let D be the silhouette area of the entire golfer in the top image shown in FIG.

According to the flowchart of FIG. 31, the switching operation from the top state is diagnosed. The definition of the judgment value is as follows (S30).
Judgment value (1) = B / A
Determination value (2) = {(A + B + C) / D} · 100
Judgment value (3) = A + B

First, when the determination value (3) <C (S31), it is diagnosed that the movement of the arm is large and the start from the upper body is too large. Next, when the condition of S31 is not satisfied and 2 <determination value (1) <5 (S32), it is diagnosed that a body turn is made with emphasis on the lower body. Next, when the condition of S32 is not satisfied and the determination value (1)> 5 (S33), it is diagnosed that the start from the lower body is too large. Next, when the condition of S34 is not satisfied and the determination value (2)> 8 (S34), it is diagnosed that the start from the upper body is too large by turning from the arm. Next, when the condition of S34 is not satisfied, it is diagnosed that the upper body and the lower body are started almost simultaneously.
That is, it is possible to diagnose the golfer's turn-back operation by setting the predetermined determination value by regarding the area of the difference silhouette as the amount of movement of the golfer's body in the turn-back operation from the top.

“Diagnosis with side (backward) image”
Next, diagnostic items in the side image will be described for each checkpoint image.

(Address status)
As shown in Table 4, in the address state, as a diagnostic item viewed from the side (behind the flying ball line), (No. 1) whether there is a right shoulder on the thumb ball or (No. 2) thumb Whether there is a right knee on the ball, (No. 3) grip position, (No. 4) spine knee angle diagnostic items are provided.
The determination values of (No.1) to (No.4) are as follows: (No.1) is [x-coordinate of right shoulder]-[x-coordinate of right foot thumb] =-10 to 10 cm, (No.2) Is [x coordinate of right kneecap]-[x coordinate of right foot thumb] = -3 to 3 cm, (No. 3) is [x coordinate of grip end]-[intersection of the horizontal line passing the grip end and the silhouette x coordinate] = 10 to 30 cm, (No. 4) is an angle between [line connecting the back neck and right hip (spine axis)] and [line connecting the right knee and right hip] = 100 to 130 deg.

(Takeback shaft at 8 o'clock)
In the 8 o'clock state of the takeback shaft, the diagnostic items viewed from the side are (No. 5) the angle difference from the original shaft line (shaft line at the address), and (No. 6) the distance from the original shaft line. Each diagnostic item is provided.
The determination values of (No. 5) and (No. 6) are as follows: (No. 5) is [shaft angle with respect to the x axis in the address state] − [shaft angle with respect to the x axis in the shaft 8 o'clock state] = − 10 10 deg, (No. 6) is [distance from grip end to shaft line in address state] = − 10 to 10 mm.

(Takeback left arm horizontal state)
In the horizontal state of the takeback left arm, the diagnostic items viewed from the side include (No. 7) intersection of the shaft and the ball line, (No. 8) difference in height between both elbows, (No. 9) shaft position, (No. 10) There are diagnostic items for shoulder position.
The judgment values of (No.7) to (No.10) are (No.7) is [{x-coordinate of the intersection of the shaft line and the ball line (the line connecting the ball and the left foot)}-(the left foot tip x-coordinate)] / [{x-coordinate of the ball}-{x-coordinate of the left foot}] = 0.66 to 1, (No. 8) is [y-coordinate of the right elbow]-[y-coordinate of the left elbow] = With respect to (No. 9), it is determined that [the x coordinate of the club colored mark CM1 on the hand side]> [the x coordinate of the right foot tip] = “raised forward” It is determined that “x coordinate of club colored mark CM1” <[x coordinate of right footpad] = “drawing behind”, (No. 10) is [straight line connecting right shoulder and left shoulder] and [spine axis] The angle between is set to 80 to 100 deg.

(Top state)
In the top state, the diagnostic items viewed from the side are: (No. 11) Distance of movement of right knee, (No. 12) Distance of movement of left knee, (No. 13) Parallelism between forearm and spine line, ( No.14) Wrist position, (No.15) Parallelism with the target direction of the shaft line, (No.16) Difference in height between both elbows, (No.17) Grip height, (No.18 ) There are diagnostic items for the distance between shoulder and wrist.
The determination values of (No. 11) to (No. 18) are (No. 11): [x coordinate of right knee in address state]-[x coordinate of right knee in top state]>-3 cm, (No. 12) [x coordinate of left knee in address state]-[x coordinate of left knee in top state] <20 cm, (No. 13) [angle between straight line connecting wrist and right elbow and y axis] -[Angle between the straight line connecting the right hip and the back neck and the y-axis] <30 deg, (No. 14) is [x coordinate of the wrist]-[x coordinate of the right shoulder] =-15 to 15 cm, (No .15) is [x coordinate of club colored mark CM1]-[wrist x coordinate] =-20 to 20 cm, and (No. 16) is [left elbow y coordinate]-[right elbow y coordinate] = −10 to 10 cm, (No. 17) is an average value of [angle with respect to x axis of line connecting wrist and left elbow] and [angle with respect to x axis of line connecting wrist and left shoulder] = 45 to 60 deg, (No. 18) is [y coordinate of left shoulder]-[wrist Are the coordinates]> 15cm.

(Down swing left arm horizontal state)
In the horizontal state of the downswing left arm, the diagnostic items viewed from the side are (No. 19) the angle difference between the shaft line and the original shaft line (the shaft line at the time of address), (No. 20) the original shaft line and the grip end And diagnostic distance items are provided.
The determination value of (No. 19) and (No. 20) is that (No. 19) is an angle between [shaft line in address state] and [shaft line in down swing left arm horizontal state] = − 15 For 15 deg, (No. 20), the distance between [grip end] and [shaft line in the addressed state] is 0 to 30 cm.

(Down swinging shaft at 9 o'clock)
In the 9 o'clock state of the downswing shaft, as a diagnostic item viewed from the side, (No. 21) a diagnostic item is provided as to whether a head is present on the shaft line.
The determination value of (No. 21) is that (No. 21) is the shortest distance between [the shaft line in the address state] and [the head in the shaft 9 o'clock state] = − 10 to 10 cm.

(Impact state)
In the impact state, the diagnostic items viewed from the side are (No.22) Angle difference from the original shaft line, (No.23) Change in spine angle, (No.24) Change in knee angle, (No .25) Hip rotation, (No.26) Distance between grip and body, (No.27) Back and forth movement of spine, (No.28) Swing track before and after impact are provided.
The determination values of (No. 22) to (No. 27) are as follows: (No. 22) is [shaft angle with respect to x axis in address state] − [shaft angle with respect to x axis in impact state] = − 5 to 5 degrees , (No.23) is the [angle between the spine axis in the impact state and the straight line connecting the right hip and the right knee]-[between the spine axis in the address state and the straight line connecting the right hip and the right knee Angle] =-10 to 10 deg, (No. 24) is [An angle between a straight line connecting the right knee and the right hip in the impact state and a straight line connecting the right knee and the right ankle]-[the right knee in the address state] The angle between the straight line connecting the right hip and the straight line connecting the right knee and the right ankle] = − 25 to 10 degrees, (No. 25) is [the distance between the right and left contour points at the umbilical height in the impact state] / [address state] Left and right contour point distance at the umbilical height] = 1.1 to 1.5, (No. 26) is [grip X-coordinate]-[x-coordinate of the intersection of the horizontal line passing the grip and the silhouette] = 5 to 15 cm, (No. 27) is [the shortest distance from the spine axis center at the time of impact to the spine axis at the address] The average value of [the shortest distance between the spine axis at the time of address and the spine axis center at the time of impact] = − 5 to 5 cm.

  Describing in detail about (No. 28), the swing trajectory before and after the impact is examined by grasping the position coordinates in the real space of the head portion 13b of the golf club 13 using the front image and the side image.

(1) Estimation of Head Position on Front and Side Images First, the position coordinates of the head portion 13b are estimated from the club colored marks CM1 to CM3 for the front and side images. That is,
(X coordinate of head portion 13b)
= (X of mark CM2) + n · {(x of mark CM3) − (x of mark CM2)}
= (1-n) · (x of mark CM2) + n · (x of mark CM3)
When,
(Y coordinate of the head portion 13b)
= (Y of mark CM2) + n · {(y of mark CM3) − (y of mark CM3)}
= (1-n) · (y of mark CM2) + n · (y of mark CM3)
The xy coordinates of the head unit 13b are calculated for each of the front image and the side image. However, n is a constant and is 2.1 for the front image and 1.8 for the side image.

Using the calculated position coordinates in the front image of the head unit 13, in the front image shown in FIG. 32, the position 40 of the head unit 13b before impact, the position 41 of the head unit 13b at impact, and the post-impact The position 42 of the head portion 13b can be specified.
By displaying the vertical lines 43 to 45 passing through the head positions 40 to 42 on the front image, the projection point on the ground of the head portion 13b before the impact, after the impact, and after the impact exists on the ground. This means that the lines 43 to 45 can be recognized.

(2) Assumptions The following items are assumed when estimating the three-dimensional coordinates of the head portion 13b.
Assumption 1: The camera has the x-axis (horizontal axis) of the coordinate system of the captured image substantially parallel to the ground.
Assumption 2: The optical axis of the camera is almost parallel to the ground.
Assumption 3: The angle between the two cameras in the direction of the optical axis is 45 ° or more, preferably approximately 90 °.
Assumption 4: The position where the head portion 13b in the three-dimensional space is projected on the ground always exists on a vertical line including the position of the head portion 13b in the two-dimensional image.
Assumption 5: In the above-mentioned line, lines in the y-axis direction and the z-axis direction in the three-dimensional space overlap.
Assumption 6: The conversion rate on the plane perpendicular to the depth direction is the same.

(3) Identification of Projection Transformation Matrix 3.1 Acquisition of Control Point The xy coordinate data is acquired so that the four points of the vertexes A to D of the frame 39 correspond to the front and side images as the control points A to D. In addition, although it is preferable that this image is acquired from a background image (an image in which no person is present), four points A to D may be acquired from the image being swung.

3.2 Calculation of Projection Conversion Matrix Using control points A to D, a matrix (projection conversion matrix) for converting a straight line existing on the ground surface of the front image into a side image is calculated. At this time, since four control points exist on the same plane, the number of parameters of the simultaneous equations is reduced from 11 to 8 by performing camera setting as shown in assumptions 1 to 3, and the corresponding points x, If y is four or more sets, the projective transformation matrix can be derived. However, in this embodiment, there are four sets of front and side surfaces for each of points A to D, so that the matrix can be derived. Specifically, if the projective transformation matrix is P, the corresponding point coordinates in the front image are x (x ₁ , x ₂ ), the corresponding point coordinates in the side image are y (y ₁ , y ₂ ), and the scale factor is s. The relationship of the determinant of Formula 19 is established.

Here, since four sets of control points A to D are given for the corresponding points x and y, the equation 19 can be expressed as the following equation 20.
For x _ij and y _ij , x means a side image, y means a front image, i = 1 to 4 correspond to four control points A to D, j = 1 is an x coordinate on the image, J = 2 means the y coordinate on the image. That is, for example, if x _11, will indicate the x coordinate of the control point A on the side image.

Therefore, in the case of four sets of corresponding points, the linear form of the projective transformation matrix P can be rewritten as the following Expression 21 from the Expression 20.

For this relational expression, a projective transformation matrix P is calculated using the least square method.
Next, all the points on the lines 43 to 45 of the front image consisting of the assumption 5 obtained in FIG. 32 are transformed by the formula 22 using the projective transformation matrix P, so that the lines 43 ′ to 43 on the side image shown in FIG. 45 'and lines 43 "to 45".

(4) Estimation of the ground position (swing trajectory) of the head portion 13b in the side image Based on Assumption 4, the perpendiculars 43 "to 45" passing through the head positions 46 to 48 before and after the impact on the side image. Then, the intersections 50 to 52 with the lines 43 ′ to 45 ′ converted from the front image become the head ground positions 50 to 52 where the head portion 13 b is projected on the ground in the side image.
Next, in the side image, projective transformation is performed on the same principle as in (3) above using the position coordinates of the four points A to D of the frame 39 on the screen and the position coordinates of the four points A to D in the real space. Find the matrix. Using this matrix, the head ground positions 50 to 52 on the screen acquired previously are converted into head ground positions in real space.

The swing trajectory of the swing (head) can be obtained by using the converted head ground position. For example, as shown in FIG. 34, the angle θ1 of the first trajectory line KL1 connecting the head ground position 70 before impact after conversion and the head ground position 71 at impact with respect to the flying line HL, and after impact By examining the angle θ2 of the second trajectory line KL2 connecting the head ground position 72 and the head ground position 71 at the time of impact with respect to the flying ball line HL, the swing trajectory is either outside-in, straight, or inside-out. Can be diagnosed.
Specifically, if the value of θ1-θ2 is −5 deg or less, it is diagnosed as outside-in, if it is −5 deg to 10 deg, it is straight, and if it is more than 10 deg, it is diagnosed as inside-out.

Next, when the silhouette cannot be extracted from the front image as shown in FIG. 30 and the turn-back operation cannot be sufficiently diagnosed, as shown in FIGS. Diagnosis of the turning-back operation may be performed using the line of the shaft 13a of the golf club 13 in the image behind the flying ball line (side).
Specifically, the angle difference θ between the shaft line SL1 (original shaft line) in the address image and the shaft line SL2 in the downswing non-dominant arm horizontal image is calculated, and θ is upward as shown in FIG. When it has an angle, it is diagnosed that “starting from the upper body is large”, and when θ has an angle in the downward direction as shown in FIG. 35B, it is diagnosed that “starting from the lower body is too large”. Ideally, if the two shaft lines SL1 and SL2 are parallel, the starting balance between the upper body and the lower body is good.

  As described above, the digitized data in the actual golfer's swing in the diagnostic item and the judgment value (ideal value) in the specific checkpoint image are compared with each other, and whether each item conforms to the judgment value. The diagnosis result and score as shown in Table 3 and Table 4 are output when it is above or below the judgment value.

  Specifically, FIG. 36 shows a schematic flow of diagnosis in a certain ballistic pattern. Swing diagnosis is performed by appropriately selecting the diagnosis items necessary for the swing check among the above-described diagnosis items. If the value (judgment value) is not satisfied, a comment on the diagnosis result is output. On the other hand, if the ideal value (judgment value) is satisfied, a pre-set score is added to each diagnosis item and the total score is obtained. Is output. At this time, it is preferable that the golfer can compare the score of the previous diagnosis result with the score of the current diagnosis result.

The score for each diagnosis item is as shown in Table 3 and Table 4, and the weight is changed in consideration of the importance to swing. Specifically, the address posture has three points because it affects all items of the swing (swing trajectory at impact, face trajectory, hitting point, etc.). Also, the impact trajectory viewed from the side and the original shaft line during downswing affect the swing trajectory during impact, so the number is 3 to 5. In addition, the side elbow and the left elbow at the time of impact have three points because they affect the hitting point at the time of impact. In addition, the distance between the spine angle and the grip and the body at the time of impact is set to 3 points because it affects the hitting point at the time of impact. In addition, the downswing diagnostic items have more impact points on the swing trajectory, the face angle, and the hitting point at the time of impact than the takeback.
As described above, the total score is 100 points (40 points on the front surface and 60 points on the side surface).

The determination method may be one of the choices of whether or not to give the scoring shown in Tables 3 and 4. However, the judgment value may be set in a stepwise manner and the scoring may be given in a stepwise manner.
For example, for a diagnostic item with 5 points, [minimum judgment value−0.5 × SA], [highest judgment value + 0.5 × SA] using the difference value SA between the lowest judgment value and the highest judgment value. ] May be additionally set as an intermediate threshold value, and if the intermediate threshold value is satisfied even if the determination values in Tables 3 and 4 are not satisfied, three points may be given as the scoring points.
As an example of grading setting of scoring, diagnosis items with 5 scoring points are 5 points or 3 points or 0 points, items with 3 scoring points are 3 points or 2 points or 0 points, and items with 2 scoring points are 2 points Or 1 point or 0 point, and the item with 1 point is preferably set as 1 point or 0.5 point or 0 point.
Furthermore, when the total score is 100 points, for example, if it is 90 points or more, it is a professional grade, if it is 80 to 90 points, it is an advanced player, if it is 60 to 80 points, it is an average golfer, if it is less than 60 points, it is a beginner, In this way, the golfer's level may be output and displayed by the total score.

  Next, an advice drill, which is a practice method for overcoming defects, is output according to the diagnosis result. The computer 16 stores a database in which a list of advice drills shown in Table 5, which is a practice method for improving swing, is registered.

For example, when the trajectory pattern is a pull hook, first, the swing trajectory of the head which is the diagnostic item No. 28 is diagnosed in the impact of the side image and the images before and after the impact (S30).
When it is diagnosed that the swing track is outside-in, a swing check on the swing track is performed accordingly (S31). For example, when the spine shaft is standing too much in the impact image of the side surface and the spine axis is standing up, or when the diagnosis is No. 24 and the knee is extended, the drill number 31 in the drill list shown in Table 5 is used. The basic effect and a brief explanation of the "Forehead against the wall" drill are output, and an advice drill as a practice method is presented to the golfer.

If the swing trajectory is diagnosed as being inside-out (S32), for example, a brief description of the “left hand” drill with drill number 4 is output, and an advice drill as a practice method is presented to the golfer.
If the swing trajectory is straight, it is diagnosed that the orientation of the face with respect to the trajectory is closed (S33). For example, a brief explanation of the “left hand” drill with drill number 4 is output and practiced to the golfer. Suggest an advice drill.

  Next, a swing check relating to the orientation of the face is performed (S34). For example, in the case of diagnosing No. 15 in the impact image on the front and diagnosing that the body weight has not moved, a simple explanation or the like is output for the drill with the drill number 25 “shake the buttocks against the wall”.

  Next, a swing check relating to the address position is performed (S35). For example, if the diagnosis is No. 1 in the front address image and the face is open due to impact because the ball is on the outside, a brief explanation of the “posture” drill with drill number 1 etc. Is output.

  Next, a swing check relating to the hitting point is performed (S36). For example, if the diagnosis of No. 27 in the impact image of the side is performed, and it is diagnosed that the upper body is retracted due to the impact with the heel weight, a brief explanation of the drill “create impact shape” with drill number 26, etc. Is output.

  If there is a diagnostic item for outputting an advice drill in the above determination, the process ends as it is. On the other hand, if there is no advice drill output in the above determination, a swing check for a grip is performed (S38). Output a message such as “Please check if it is a square grip”.

In addition, as shown in FIG. 37, diagnosis may be performed from various amounts related to the ball behavior acquired by the ball motion measuring device 20.
For example, the height of the ball trajectory is acquired from the ball motion measuring device 20, and it is determined whether or not the height is within 10 to 30 years (S40). If it is 30 years or more, it is diagnosed as a so-called tempura with a high trajectory, so a swing check on the tempera is performed and an advice drill corresponding to that is presented (S41).
On the other hand, if it is less than 10 years, it is determined that the so-called top has a low trajectory, so a swing check on the top is performed and an advice drill corresponding to the top is presented (S42).

Next, when the ballistic height is within 10 to 30 years, it is determined whether the ratio of the ball speed / head speed is 1.35 or more (S43).
If the above ratio is not 1.35 or more and it is determined that the side spin is a slice, it is diagnosed that the heel is hit, so a swing check on the hitting point is performed and an advice drill is presented accordingly (S44).
If the above ratio is not 1.35 or more and it is determined that the side spin is a hook, it is diagnosed that a toe has been hit, so a swing check on the hitting point is performed and an advice drill is presented accordingly (S45).
If it is determined that the ratio is 1.35 or more, since it is diagnosed that the ball is hit correctly, a swing check is performed to further increase the flight distance, and an advice drill corresponding to that is presented (S46). ).

(Example)
Next, Table 6 shows the results of the above-described golf swing diagnosis system implemented by 10 testers.
For each tester, the ball behavior data related to the swing angle of the hit ball B, the side spin, the lateral displacement distance from the straight direction of the ball drop point (carry), the trajectory height, and the ball speed / head speed are acquired. It can be seen that the ballistic pattern is determined from the spin amount and the deflection angle, and the diagnosis result regarding the diagnostic items prepared for the ballistic pattern and the advice drill for overcoming the defect are presented.

It is a lineblock diagram of a golf swing diagnostic system of an embodiment of the present invention. It is a flowchart of the golf swing measurement system of an embodiment. It is drawing explaining the mark extraction in an address image. It is a figure explaining the mark extraction in the 2nd and 3rd sheet after an address. It is drawing explaining the automatic tracking of a mark. (A) and (B) are drawings explaining contour extraction of a golfer. 2 is a drawing showing an image of a golfer whose contour has been extracted. (A) (B) It is drawing explaining template matching. It is explanatory drawing of curvature calculation. It is a front checkpoint image. It is a checkpoint image behind a flying ball line (side). It is drawing which shows the golfer's outline on a still picture. It is drawing which shows the linear part extracted from the outline on a still image. It is drawing explaining the color extraction of an attention point. It is drawing explaining the extraction using the outline of an attention point. It is drawing explaining the extraction using the curvature of the outline of an attention point. It is drawing explaining the extraction using the straight line part of an attention point. It is drawing explaining the extraction using the straight line part of an attention point. It is drawing explaining the extraction of the attention point using skin extraction. It is drawing which shows an edge strength image. It is drawing which shows a direction labeling image. It is drawing which shows a non-maximum value suppression labeling image. It is drawing explaining the definition of the positional relationship of an adjacent pixel. It is drawing explaining the estimation of the attention point by silhouette information. It is a flowchart which shows the procedure which extracts an attention point in a takeback left arm horizontal image (side). It is a flowchart which shows a part of subroutine of FIG. (A) and (B) are drawings explaining skin extraction. (A) is a drawing based on color extraction, (B) is a drawing based on contour extraction, (C) is a drawing using a straight line portion, and (D) is silhouette information. FIG. It is drawing which shows an inquiry screen. (A) is a drawing showing a differential silhouette, and (B) is a drawing showing a golfer silhouette in a top state. It is a diagnostic flowchart of the return operation | movement from a top state. It is a front image for calculating | requiring a swing track | orbit. It is a side image for calculating | requiring a swing track | orbit. It is explanatory drawing of the diagnosis of a swing track | orbit. (A) (B) is drawing explaining the diagnosis by the side image of the turning operation. It is a diagnostic flowchart after ballistic pattern classification. It is a diagnostic flowchart based on a ball behavior.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Golfer 12 Clothing 13 Golf club 13a Shaft 14, 15 Color CCD camera 16 Computer 20 Ball motion measuring device CM1-CM3 Colored mark M1-M7 Colored mark for club
S, S1-S3 Search range ST, ST1-ST4 Straight line portion R Contour T Template

Claims (9)

A computer that captures a moving color image of a golfer who holds and swings a golf club;
From a plurality of still images constituting the color moving image, address, take back shaft 8 o'clock, take back shaft 9 o'clock, take back non-dominant arm horizontal, top, down swing non-dominant arm horizontal, down swing shaft 9 o'clock, impact, follow Means for extracting at least one swing posture image selected from the finish at the shaft 3 as a checkpoint image;
Means for acquiring position coordinates on the checkpoint image of a point of interest that operates during swing;
A plurality of diagnostic items including a swing posture and a shaft angle are set for each trajectory pattern, and data obtained by digitizing the position data of the attention point with respect to the diagnostic items is an ideal value input in advance to the computer Means for diagnosing the golfer's swing in comparison with a judgment value;
A golf swing diagnosis system comprising: a means for outputting an advice drill corresponding to the diagnosis result from a database in which a plurality of advice drills, which are practice methods for swing improvement, are registered.   The golf swing diagnosis system according to claim 1, wherein the ballistic pattern is classified into pull hook, pull, pull slice, straight hook, straight, straight slice, push hook, push, and push slice. A ball motion measuring device for measuring the behavior of a golf ball hit by the golfer;
3. The golf swing diagnosis system according to claim 1, wherein the ballistic pattern is acquired based on a side spin amount and a swing angle of the ball measured by the ball motion measuring device. The computer is provided with an inquiry means that allows the golfer to input his own ballistic pattern to be diagnosed before hitting,
The diagnosis and advice drill output is performed when the ballistic pattern obtained from the actual ball behavior acquired by the ball motion measuring device matches or resembles the ballistic pattern input by the interrogation means. 4. The golf swing diagnostic system according to any one of items 3.   A difference silhouette is obtained by performing a difference process between a top image showing the swing posture of the top and an image after a predetermined time has elapsed from the top image, and switching from the top state using the area of the difference silhouette The golf swing diagnostic system according to any one of claims 1 to 4, wherein the operation is diagnosed.   5. The switching operation is diagnosed based on an angle difference between the shaft line in the downswing non-dominant arm horizontal image viewed from the rear of the flying ball line and the shaft line in the address image. The golf swing diagnostic system according to claim 1.   Cock operation when the difference between the wrist angle in a given swing posture and the wrist angle in another swing posture is greater than or equal to a prescribed value, or when the wrist angle in a given swing posture is greater than or equal to a prescribed value The golf swing diagnosis system according to any one of claims 1 to 6, wherein the presence is diagnosed.   Claims 1 thru | or the structure which presets the score about each of these several diagnostic items, and adds the said score when the determination value which is an ideal value is satisfy | filled about each diagnostic item, or when not satisfy | filling. The golf swing diagnostic system according to any one of items 7.   The golf swing diagnostic system according to claim 8, wherein the score output for each diagnostic item is cumulatively added and the total score is output as a diagnostic result.