JP2005270484A - Calibration method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily acquire a three-dimensional distance from an image taken by a camera. <P>SOLUTION: This calibration method includes to take a image of a golfer 11 holding a golf club 13 and swinging it, import it in a computer 16, and calibrate position information of highlights P and Q projected on the taken image by the computer 16 to provide actual position information. A frame 20 surrounding the golfer 11 is provided on a ground constituting a swing actual space, at least two straight lines AD and BC constituting the frame 20 are recognized on the image, a conversion rate sf<SB>ave</SB>converting pixel position information in the frame 20 into the actual position information is calculated using the length W2 in the swing actual space of the two straight lines AF and BC and the pixel lengths M1 and M2 on the image and and actual distance L is calculated by multiplying the conversion rate sf<SB>ave</SB>by the pixel distance (a) on the image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a calibration method, and more particularly, to a method for properly calibrating and recognizing the position coordinates of a point of interest on an image during golf swing.

2. Description of the Related Art Conventionally, various devices have been provided that can capture a swing when a golfer is hit, and automatically calculate various information such as a flying distance and a trajectory of the hit ball and display it on a golfer.
For example, in the motion diagnostic device disclosed in Japanese Patent No. 2779418, a plurality of motion points are provided on the body of the golf club head and the person to be diagnosed, and the coordinates of the motion points in the swing moving image are acquired to perform the swing diagnosis. Yes.

  However, in the above publication, when the position coordinates of the operation point are acquired from the image, only the plane coordinates can be acquired because the image is plane information. Therefore, since the position coordinates and distances in the depth direction on the image of the operation point cannot be obtained, it is not possible to provide diagnostic items using three-dimensional position coordinates, or forcibly three-dimensionally from the plane coordinates. If processing is to be performed, there is a problem that the error in diagnosis increases.

  In general, a DLT method (Direct Linear Transformation) used in 3D CG or the like is known as a method for calculating 3D position coordinates from a 2D image. The DLT method uses a plurality of cameras to photograph a plurality of control points installed in real space in advance, and solves 11 simultaneous equations from the position coordinates of the control points photographed by the plurality of cameras. In this method, the relationship between the two-dimensional coordinates and the three-dimensional coordinates is acquired, and the three-dimensional coordinates are separately acquired using a plurality of two-dimensional coordinates of the same point photographed by a plurality of cameras.

However, the DLT method requires 6 or more control points in a three-dimensional space, which makes measurement preparation very laborious and requires a large calibration jig. There is a need to provide a method for easily obtaining three-dimensional position coordinates from a planar image.
Japanese Patent No. 2794018

  The present invention has been made in view of the above problems, and an object thereof is to provide a method for easily obtaining a three-dimensional distance and angle from an image photographed by a camera.

In order to solve the above-mentioned problems, the present invention captures a golfer who holds a golf club and swings it, captures it into a computer, and calibrates position information of a point of interest displayed on the captured image with the computer. A method for acquiring location information of
A frame is provided at a position surrounding the golfer on the ground that constitutes the swing real space,
At least two straight lines constituting the frame are recognized on the image, and the actual length of the two straight lines in the swing real space and the pixel length on the image are used to calculate the actual position from the pixel position information in the frame. There is provided a calibration method characterized in that a conversion rate to be converted into position information is calculated, and an actual distance is calculated by multiplying the conversion rate by a pixel distance on an image.

Since the three-dimensional real space is projected and projected on the image, the pixel position information on the image does not necessarily match the actual correct position information. However, with the above configuration, the depth direction of the frame provided in the swing space is not the same. The conversion rate for converting the pixel position information on the image into the actual position information using the reduced scale can be easily obtained. Therefore, the actual distance can be calculated by simply calibrating the position information in the image.
Note that the conversion rate may be obtained in advance by photographing the frame before the golfer swings, and the frame may be removed from the ground when the golfer actually swings.

  The frame is a virtual frame, and a mark is placed on the ground at a position that is a vertex of the shape of the virtual frame.

  With the above-described configuration, even if a frame-like frame is not provided on the ground on which the golfer performs a swing operation, a virtual frame connecting the vertices can be assumed by only providing marks at positions that become the vertices.

  The conversion rate in the frame is obtained by averaging the conversion rates on two straight lines arranged opposite to the image depth among the straight lines constituting the frame.

  With the above configuration, the average conversion rate in the frame can be obtained very simply by simply obtaining the conversion rates of the two straight lines constituting the frame and averaging them.

  The actual distance between the two points of interest may be obtained by multiplying the pixel distance between the two points of interest on the image by the conversion rate.

Alternatively, the area in the frame is divided into a plurality of blocks,
It is also preferable to obtain the conversion rate in the block by averaging the conversion rates on the two straight lines arranged opposite to the image depth among the straight lines constituting the block.

In other words, instead of using only one conversion rate within a frame, it is possible to provide a plurality of conversion rates corresponding to each block by providing a block obtained by dividing a frame into a plurality of blocks, so that it corresponds to the position where the point of interest exists. Therefore, the optimum conversion rate can be selected and applied, and the accuracy can be improved easily.
It is preferable that the block is formed by dividing an area in the frame into blocks based on the golfer's foot silhouette.
Furthermore, the actual distance between the two points of interest may be obtained by averaging the conversion rates of the blocks to which the points of interest belong and multiplying the pixel distance between the points of interest on the image. .

It is also preferable to obtain the conversion rate in the frame by linear interpolation using at least two straight lines constituting the frame.
In this way, the conversion rate that varies linearly in the image depth direction can be obtained by calculating the conversion rate at each position in the frame while linearly interpolating the conversion rate of the straight lines that make up the frame. The accuracy of conversion from the pixel distance to the actual distance can be improved.

  The actual distance between the two points of interest is calculated by projecting one point of interest on the surface of the other point of interest on the image, and the distance between the projected point and the other point of interest. If the above conversion rate is multiplied, the distance between two points having different positions in the image depth direction can be calculated.

The actual distance between the surface or line existing on the image and the point of interest is
If the point of interest is projected onto the surface constituting the surface or line, and the distance between the projected point and the surface or line is obtained by multiplying the conversion rate on the surface, the position differs in the image depth direction. The distance between a point and a surface (line) can be calculated.

  On the image, the points that intersect the straight lines that are the sides in the depth direction of the frame in the depth direction are defined as vanishing points, a vertical line is dropped from the vanishing point in the vertical direction, and the attention point and the vertical lines The distance between each point of interest is converted according to the distance between them.

  With the above configuration, the distance between the points of interest can be obtained indirectly by using the vertical line from the vanishing point on the image as a reference.

The frame is set in a range including a part of the golf club.
In particular, it is preferable to set the frame within a range that includes the locus of the head portion of the golf club.
In other words, since the head part of the golf club operates with the maximum rotation, if the frame size is set so that the locus of the head part is included, the swing operation is performed using the conversion rate in the frame. The position of the point of interest can be almost covered.

If the length of the side in the direction orthogonal to the flying line direction of the frame is W1, and the length of the side in the flying line direction is W2, it is preferable that W1 / W2 = 1 to 10.
That is, since the swing trajectory tends to increase in the flying ball direction, the frame size can be determined efficiently by using the ratio.

The size of the frame is h / i = 1 to 3 when the sides of the frame are divided by h: i in the flying ball direction from the midpoint of the center of the golfer's feet.
That is, by setting the ratio, the frame size can be determined efficiently so that the swing track is included in the frame with reference to the standing position of the golfer.

The size of the frame is j / k = 1 to 10 when the sides of the frame are divided by j: k in a direction perpendicular to the flying ball direction with the golfer's foot as a base point.
That is, by setting the ratio, the frame size can be determined efficiently so that the swing track is included in the frame with reference to the standing position of the golfer.

According to the present invention, a golfer who holds and swings a golf club is photographed and loaded into a computer, and the position information of a point of interest displayed on the photographed image is calibrated by the computer to obtain actual position information. A method,
For at least four points existing in the same plane on the image taken from two directions, a projection transformation matrix representing the correspondence relationship of the position coordinates between the images in the two directions is obtained in advance.
A vertical line representing the depth direction passing through the attention point on the image on one direction side is converted into a horizontal line on the image on the other direction side by the projective transformation matrix, and passes through the attention point on the image on the other direction side. Obtain the position coordinates on the plane on the image by finding the intersection of the vertical line and the horizontal line,
The position coordinates are converted into the position coordinates on the real space using another projective transformation matrix representing the correspondence relationship between the position coordinates between the image and the real space, and the real space coordinates of the attention point are obtained. A calibration method characterized by the above is provided.

  With this configuration, the position coordinates in the real space can be acquired by calibrating the position coordinates of the target point of the measurement target displayed on the image, using only images in two directions.

It is preferable that the attention point is two points, and the distance or / and angle between the two points is obtained.
That is, by obtaining the distance and angle between two points using real space coordinates, it can be used for diagnosis of a swing trajectory before and after impact.

A frame is provided at a position surrounding the golfer on the ground that constitutes the swing real space,
At least two straight lines constituting the frame are recognized on the image, and the actual length of the two straight lines in the swing real space and the pixel length on the image are used to calculate the actual position from the pixel position information in the frame. Calculate the conversion rate to convert to location information,
A three-dimensional position coordinate in the real space of the target point is obtained by obtaining a pixel distance in the direction of a specific coordinate axis of the target point on the one-direction image and multiplying the pixel distance by the conversion rate. ing.

  With the above configuration, for example, if the pixel distance in the Z-axis direction from the origin to the point of interest is obtained on the image with the ground as the origin in the Z direction (vertical direction), the conversion rate of the position where the point of interest exists is calculated. By simply multiplying the distance, the Z position coordinates in the real space can be acquired, and the three-dimensional position coordinates can be calculated. In addition, it is suitable to use what was calculated | required by the linear interpolation mentioned above as the said conversion rate.

Further, the present invention uses the above calibration method,
The point of interest is the golfer's joint, and the distance between one joint and another joint or golf club shaft, or the distance between one joint and a joint or golf club shaft at another time is obtained. Provides a golf swing measurement system.

  As is clear from the above description, according to the present invention, the conversion rate for converting the pixel position information on the image into the actual position information can be simplified by using the reduced scale in the depth direction of the frame provided in the swing space. The actual distance can be calculated by easily calibrating the position information in the image.

A first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic diagram of a golf swing measurement system. A computer 16, a monitor 17 as display means connected to the computer 16, a keyboard 18 and a mouse 19 as input means connected to the computer 16, and a computer 16, color CCD cameras 14 and 15 installed at the front position of the golfer 11 and at the rear (side) position of the flying ball line.

  A frame-shaped frame 20 is installed around the swing position where the golfer 11 stands on the ground. The frame 20 has a color different from that of the ground and has a quadrangular shape with four points A to D as vertices. As shown in FIG. 2, the lengths of the straight line AB and the line CD are W1, the straight line BC, and the straight line AD. Is W2, and W1 / W2 = 1-10. Further, as shown in FIGS. 3A and 3B, the standing position of the golfer 11 in the frame 20 is such that the ratio h / i at which the straight line CD and the straight line AD are divided at the midpoint between both feet 22 of the golfer 11 is 1. And the ratio k / j at which the straight line AB and the straight line CD are divided at the centers of both feet 22 of the golfer 11 are set to 1 to 10.

  A golfer 11 who is a person to be diagnosed wears a measurement clothes 12 which is an outerwear with colored marks (attention points) P and Q attached to joint positions from above his / her clothes. Three colored club marks CM1, CM2, and CM3 are attached to the shaft 13a of the golf club 13 held by the golfer 11 at intervals. The club colored marks CM1 to CM3 are attached at equal intervals from the grip side to the head 13b side.

  The computer 16 is connected to the color CCD cameras 14 and 15 online using a LAN cable, IEEE1394, CameraLink standard, etc., and a swing moving image (a plurality of still images) and a background image photographed by the color CCD cameras 14 and 15 are used. Are stored in the hard disk of the computer 16, the memory on the computer 16, or the memory on the board. The computer 16 also has means for binarizing each pixel of the still image with a specific threshold relating to color information, and obtaining coordinate data with the pixels satisfying the threshold as the positions of the club colored marks CM1 to CM3. A program including a means for recognizing the movement of the shaft 13a based on the coordinate data of the club colored marks CM1 to CM3 and a means for recognizing the colored marks P and Q of the golfer 11 is incorporated.

  The golfer 11 makes a test shot, takes still images of each frame of the swing moving image from the color CCD cameras 14 and 15 into the computer 16 and stores them in the hard disk, the memory in the computer 16 or the memory on the board, and the color CCD camera 14 In the front image photographed in step S4 and the side image photographed by the color CCD camera 15, the colored marks CM1 to CM3 for clubs and the colored marks P and Q of the golfer 11 are recognized, and each mark is subjected to calibration processing described later. Recognize 3D position coordinates.

Next, a method for calibrating the pixel distance a between the marks P and Q recognized on the still image to the correct distance L will be described.
Although FIG. 4 shows a front image, although the straight line AD and the straight line BC have the same actual length W2, the straight line BC on the depth side appears shorter on the image. The conversion rate for converting the distance to the actual distance is obtained.
In the present embodiment, the average conversion rate of the region in the frame 20 is obtained as the conversion rate and applied to the distance conversion of each mark. That is, this method can be said to be a simple calibration method that utilizes the fact that the golfer 11 performs a swing motion at approximately the center of the frame 20 region.

The average conversion rate sf _ave [cm / pixel] in the frame 20 is such that the measured length of the straight line BC and the straight line AD is W2 [cm], the length of the straight line BC on the image is M1 [pixel], and on the straight line AD image Is M2 [pixel], the following equation 1 is obtained.

Therefore, the distance L [cm] after the calibration between the mark P and the mark Q can be easily obtained by the following formula 2 when the distance between PQs on the image is a [pixel].

  Further, as shown in FIG. 5, the intersection point between the extension line of the straight line AB and the extension line of the straight line CD is acquired as the vanishing point S, and the ball 22 is obtained by using the straight line 21 drawn from the vanishing point S to the ball 22. It is possible to determine whether the position of is appropriate. More specifically, by determining the distance between the heel of the silhouette of the golfer 11 and the straight line 21, it is possible to grasp the installation position of the ball 22 with respect to the foot of the golfer 11.

Note that the method of recognizing the frame 20 on the image may be manually recognized by the operator using a mouse or the like, but it is preferable that the frame 20 is automatically recognized by image processing. Hereinafter, the latter automatic frame recognition method will be described.
The color of the frame 20 is made different from the color of the ground or the vicinity, and binarization processing is performed on each pixel on the image with a specific threshold regarding color information, and the pixel satisfying the threshold is recognized as the frame 20 and the frame is recognized. 20 position information is acquired. As a binarization processing method, RGB values, YIQ values, hue / saturation / lightness, etc. may be used as color information, and a threshold value that can be regarded as the same color of the frame 20 may be set. Further, a straight line constituting the frame 20 may be extracted using Hough transform.

  Alternatively, as shown in FIG. 6, color marks are installed at the four vertices A to D constituting the virtual frame 20 ′, the color marks A to D are different from the surrounding colors, and the color of each pixel on the image is set. Binarization processing is performed with a specific threshold value related to information, pixels satisfying the threshold value are recognized as color marks A to D, the position coordinates of frame vertices are acquired, and the position of the virtual frame 20 ′ is obtained by connecting each color mark Information may be acquired. The color mark placed at the apex of the frame may be anything as long as it is easy to extract colors, such as a spherical body with a diameter of about 3 cm or a cube with a side of about 3 cm.

7 and 8 show a second embodiment.
The difference from the first embodiment is that the conversion rate is calculated for each block by dividing the frame 20 into a plurality of blocks without using the average conversion rate in the frame 20 as the conversion rate.

FIG. 7A shows a front image, FIG. 7B shows a side image, and the straight line BC and the straight line AD are equally divided by two lines passing through the vanishing point S in the frame 20 shown in the image, The straight line AB and the straight line CD are equally divided by two horizontal lines, and the frame 20 is divided into nine blocks into a block 1 to a block 9.
First, the vertical line 23 passing through the mark P recognized on the front image is drawn, and the blocks 2, 5, and 8 through which the vertical line 23 passes are stored. Next, a vertical line 24 passing through the mark P recognized on the side image is drawn, and the blocks 1, 2, 3, and 4 through which the vertical line 24 passes are stored.

The block 2 stored in common between the front image and the side image is determined to be a block where the mark P exists, and the conversion rate sf _B2 [cm / pixel] related to the block 2 is used for the mark P. . Note that the conversion rate calculation method for each block is easily calculated by using the average conversion rate calculation method in the frame 20 of the first embodiment and replacing the frame with the block.

Next, a post-calibration distance L [cm] between the mark P and the mark Q is obtained.
(1) When the mark P and the mark Q exist in the same block, assuming that the conversion rate of the block is sf _B [cm / pixel] and the distance between P and Q on the image is a [pixel], The distance L [cm] after the calibration can be calculated by using Equation 3.

(2) When the mark P and the mark Q exist in adjacent blocks, for example, when the mark P exists in the block 2 and the mark Q exists in the block 1, the conversion rate of the block 2 is set to sf _B2 [ When the conversion rate of the block 1 is sf _B1 [cm / pixel], the distance L [cm] after calibration can be calculated by the following equation 4.

(3) When the mark P and the mark Q are present in non-adjacent blocks, for example, when the mark P is present in the block 2 and the mark Q is present in the block 7, as shown in FIG. Using the point P ′ obtained by projecting the mark P on the vertical plane passing through the block 7 in which Q is present and the blocks 8 and 9 side by side, the distance between P′Q is a ′ [pixel], and the conversion of the blocks 7, 8 and 9 The distance L [cm] after calibration can be obtained by the following formula 5 where the rate is sf _Q [cm / pixel].

Here, the determination method of the point P ′ will be described. First, the point P ₁ is obtained by dropping a vertical line from the mark P in the vertical direction. However, the line on the ground of the mark P is a line horizontally traversing from the center of the block 2. Next, a point that intersects with an extension line of the line SP ₁ connecting the vanishing point S and the point P ₁ and a line that horizontally traverses the centers of the blocks 7, 8, and 9 is defined as a point P ₂ . Then, a point where an extension line of the line SP connecting the vanishing point S and the mark P intersects with a vertical line passing through the point P ₂ is determined as a point P ′. Since other configurations are the same as those of the first embodiment, description thereof is omitted.

9A and 9B show a modification example of block division in the frame 20. In this modification, the inside of the frame 20 is divided into 12 blocks 1 to 12.
In the front image of FIG. 9A, the lines 27 and 28 connecting the points 25 and 26 on both sides of the foot silhouette of the golfer 11 with vanishing points (not shown), respectively, and the midpoint between the both feet of the foot silhouette Is divided into blocks using the line 30 connecting the vanishing point, the line 34 passing through the heels 29 and 30 and parallel to the straight line AD, and the line 33 passing through the lower end 32 of the foot silhouette and parallel to the straight line AD. ing.
In the side image of FIG. 9B, a line 38 connecting the heel 35 of the foot silhouette of the golfer 11 to the vanishing point (not shown), a line 39 connecting the foot tip 36 of the foot silhouette to the vanishing point, Block division is performed using lines 40 and 41 that pass through the tips 36 and 37 and parallel to the straight line AB, and lines 41 that pass through the midpoint between the two toes 36 and 37 and are parallel to the straight line AB.

10 to 13 show a third embodiment.
The difference from the first and second embodiments is that the conversion rate is accurately calculated by linear interpolation without using the average conversion rate in the frame 20 or in the block.

As shown in FIG. 10B, when the vertical line passing through the mark P as seen from the side image divides the straight line CD constituting the frame 20 by α: β, as shown in FIG. Assuming that EF is a line parallel to the straight line BC passing through the intersection with the ground from the mark P as seen from the image, the length in the depth direction of the front image gradually decreases, so that BE: AE = α: β In other words, BE: AE = α ′: β ′. Here, the length of the straight line BC on the image is M1 [pixel], the length of the straight line AD on the image is M2 [pixel], and the length of the line EF on the image is M3 [pixel]. The conversion rate is sf _M1 [cm / pixel], and the conversion rate on the straight line AD is sf _M2 [cm / pixel]. Then, sf _M3 [cm / pixel] of the conversion rate at the line EF is calculated by the following Expression 6.

Here, the unknown value in the right side of Equation 6 is M3 [pixel]. This M3 can be obtained by the following Expression 7.

Next, the calculation procedure regarding α ′ and β ′ on the right side of Equation 7 will be described in detail.
FIG. 11 shows a view looking down from the ceiling side, where the depth direction viewed from the color CCD camera 14 is the Z direction, and the flying ball direction is the Y direction.
If a line passing through the points A, E, and B is bY + cZ + d = 0 (b, c, and d are constants), Y ₀ that is the Y coordinate of the point A is obtained by Equation 8.

Similarly, Y _1, which is the Y coordinate of the point E, is obtained by Equation 9.

Here, the relationship of Y ₀ , y ₀ , f, and Z ₀ is as shown in FIG. Y ₀ is the Y coordinate of the intersection of the line connecting the point A and the camera 14 and the Y axis, f is the distance between the camera 14 and the Y axis, and Z ₀ is the Z coordinate of the point A.

Therefore, Equation 11 is derived from Equation 8 and Equation 10, and Equation 12 is derived from Equation 9 and Equation 10. Y ₁ is the Y coordinate of the intersection of the line connecting the point E and the camera 14 and the Y axis, and Z ₁ is the Z coordinate of the point E.

Next, the following Expression 13 is derived from Expression 11 and Expression 12.

Similarly to Equation 13, Equation 14 is also derived. Y ₂ is the Y coordinate of the intersection of the line connecting the point B and the camera 14 and the Y axis, and Z ₂ is the Z coordinate of the point B.

Since y ₀ , y ₁ , and y ₂ on the Y axis can be considered as points A, E, and B in the front image reflected on the camera 14, α ′ and β ′ in the front image are obtained by the following Expression 15. Can do.

つまり、以上のようにして得られたα’／β’を数式７に代入して算出されるＭ３を更に数式６に代入することで、正面画像からみたマークＰでの変換率ｓｆ_M3を求めることができる。 Here, of the right side of Equation 15, Z ₀ is obtained by actual measurement in the real space, and Z ₁ and Z ₂ are obtained by Equation 16 and Equation 17 below and substituted.

That is, the conversion rate sf _M3 at the mark P as viewed from the front image is obtained by substituting M3 calculated by substituting α ′ / β ′ obtained as described above into Equation 7, further into Equation 6. be able to.

Next, as shown in FIGS. 13A and 13B, a method for calculating the distance between a point and a line (surface) will be described.
As shown in FIG. 13A, the original shaft line SL of the golf club 13 is obtained from the positions of the club colored marks CM1 to CM3 recognized in the front image at the time of addressing. Further, the position of the head portion 13b of the golf club 13 is predicted from the position of the club colored marks CM1 to CM3 and the distance between the club colored mark CM3 and the head portion 13b. Since the head portion 13b can be considered to be located on the ground in the address state, the ratio a: b is determined by which the perpendicular drawn from the head portion 13b divides the straight line AD. In addition, a ratio α: β that a perpendicular drawn from the mark P separates the straight line AD is obtained. Then, from these ratios a: b and α: β, a ′: b ′ and α ′: β ′ are obtained based on the same principle as that of the above-described Expression 15.

Next, in the side image shown in FIG. 13B, an intersection 54 between the perpendicular drawn from the mark P and a line 51 dividing the frame 20 into α ′: β ′ is obtained, and a line connecting the intersection 54 and the vanishing point S. Find the point 55 where 52 intersects the line 53 that divides the frame 20 into a ′: b ′. The intersection of the perpendicular line passing through the point 55 and the line 50 is defined as a point P ′.
Accordingly, the shortest pixel distance between the original shaft line SL and the mark P ′ on the side image is obtained, and the actual distance between the original shaft line SL and the mark P is obtained by multiplying the pixel distance by the conversion rate on the line 53. Can be obtained.
In the present embodiment, the distance between the point (mark P) and the line (original shaft line SL) is obtained, but the distance between the point and the surface can be obtained in the same procedure.

FIG. 14 shows a fourth embodiment.
In the present embodiment, the swing trajectory before and after the impact is examined by grasping the position coordinates in the real space of the head portion 13b of the golf club 13 using the front image and the side image.

(1) Estimation of Head Position on Front and Side Images First, the position coordinates of the head portion 13b are estimated from the club colored marks CM1 to CM3 for the front and side images. (Hereinafter, three-dimensional coordinates in real space are represented by capital letters X, Y, and Z, while planar position coordinates on the image are represented by small letters x and y. X is a flying ball. (Line direction, Y is the front-rear direction viewed from the golfer, Z is the vertical direction, x is the horizontal direction on the image, and y is the vertical direction on the image.)
(X coordinate of head portion 13b)
= (X of mark CM2) + n · {(x of mark CM3) − (x of mark CM2)}
= (1-n) · (x of mark CM2) + n · (x of mark CM3)
When,
(Y coordinate of the head portion 13b)
= (Y of mark CM2) + n · {(y of mark CM3) − (y of mark CM3)}
= (1-n) · (y of mark CM2) + n · (y of mark CM3)
The xy coordinates of the head unit 13b are calculated for each of the front image and the side image. However, n is a constant and is 2.1 for the front image and 1.8 for the side image.

Using the calculated position coordinates of the head unit 13 in the front image, in the front image shown in FIG. 14, the position 60 of the head unit 13b before impact, the position 61 of the head unit 13b during impact, and the post-impact The position 62 of the head portion 13b can be specified.
By displaying the vertical lines 63 to 65 passing through the head positions 60 to 62 on the front image, the projection point on the ground of the head portion 13b before the impact, after the impact, and after the impact exists on the ground. This means that the lines 63 to 65 can be recognized.

(2) Assumptions The following items are assumed when estimating the three-dimensional coordinates of the head portion 13b.
Assumption 1: The camera has the x-axis (horizontal axis) of the coordinate system of the captured image substantially parallel to the ground.
Assumption 2: The optical axis of the camera is almost parallel to the ground. Assumption 3: The angle between the two cameras in the direction of the optical axis is 45 ° or more, preferably about 90 °.
Assumption 4: The position where the head portion 13b in the three-dimensional space is projected on the ground always exists on a vertical line including the position of the head portion 13b in the two-dimensional image.
Assumption 5: In the above-mentioned line, lines in the y-axis direction and the z-axis direction in the three-dimensional space overlap.
Assumption 6: The conversion rate on the plane perpendicular to the depth direction is the same.

(3) Identification of Projection Transformation Matrix 3.1 Acquisition of Control Point The xy coordinate data is acquired so that the four points of the vertices A to D of the frame 20 correspond to the front and side images as the control points A to D. In addition, although it is preferable that this image is acquired from a background image (an image in which no person is present), four points A to D may be acquired from the image being swung.

3.2 Calculation of Projection Conversion Matrix Using control points A to D, a matrix (projection conversion matrix) for converting a straight line existing on the ground surface of the front image into a side image is calculated. At this time, since four control points exist on the same plane, the number of parameters of the simultaneous equations is reduced from 11 to 8 by performing camera setting as shown in assumptions 1 to 3, and the corresponding points x, If y is four or more sets, the projective transformation matrix can be derived. However, in this embodiment, there are four sets of front and side surfaces for each of points A to D, so that the matrix can be derived. Specifically, if the projective transformation matrix is P, the corresponding point coordinates in the side image are x (x ₁ , x ₂ ), the corresponding point coordinates in the front image are y (y ₁ , y ₂ ), and the scale factor is s. The relationship of the determinant of Expression 18 is established.

なお、ｘ_ij、ｙ_ijについては、ｘは側面画像、ｙは正面画像を意味し、ｉ＝１〜４は４つのコントロールポイントＡ〜Ｄに対応し、ｊ＝１は画像上のｘ座標、Ｊ＝２は画像上のｙ座標を意味する。即ち、たとえばｘ₁₁であれば、側面画像上でのコントロールポイントＡのｘ座標を示すことになる。 Here, since four sets of control points A to D are given for the corresponding points x and y, the equation 18 can be expressed as the following equation 19.

For x _ij and y _ij , x means a side image, y means a front image, i = 1 to 4 correspond to four control points A to D, j = 1 is an x coordinate on the image, J = 2 means the y coordinate on the image. That is, for example, if x _11, will indicate the x coordinate of the control point A on the side image.

Therefore, in the case of four sets of corresponding points, the linear form of the projective transformation matrix P can be rewritten as the following Expression 20 from the Expression 19.

For this relational expression, a projective transformation matrix P is calculated using the least square method.
Next, all the points on the lines 63 to 65 of the front image consisting of the assumption 5 obtained in FIG. 14 are transformed by the expression 21 using the projective transformation matrix P, so that the lines 63 ′ to 63 on the side image shown in FIG. 65 ′ and lines 63 ″ to 65 ″.

(4) Estimation of the ground position (swing trajectory) of the head portion 13b in the side image Based on Assumption 4, the perpendiculars 63 "to 65" passing through the head positions 66 to 68 before and after the impact on the side image. Then, the intersections 70 to 72 with the lines 63 ′ to 65 ′ converted from the front image become head ground positions 70 to 72 where the head portion 13 b is projected on the ground in the side image.
Next, in the side image, projective transformation is performed on the same principle as in (3) above using the position coordinates of the four points A to D of the frame 20 on the screen and the position coordinates of the four points A to D in the real space. Find the matrix. Using this matrix, the head ground positions 70 to 72 on the screen acquired previously are converted into head ground positions in real space.

The swing trajectory of the swing (head) can be obtained by using the converted head ground position. For example, as shown in FIG. 16, the angle θ1 of the first trajectory line KL1 connecting the head ground position 70 before impact after conversion and the head ground position 71 at impact with respect to the flying ball line HL, and after impact By examining the angle θ2 of the second trajectory line KL2 connecting the head ground position 72 and the head ground position 71 at the time of impact with the flying ball line HL, the swing trajectory is either outside-in, straight, or inside-out. Can be diagnosed.
Specifically, if the value of θ1-θ2 is −5 deg or less, it is diagnosed as outside-in, if it is −5 deg to 10 deg, it is straight, and if it is more than 10 deg, it is diagnosed as inside-out.

(5) Estimation of the three-dimensional position coordinate of the head portion 13b Up to (4) above, the Z coordinate component on the ground, that is, the Z coordinate component perpendicular to the ground has been treated as 0. Can be obtained in this way.
In the three-dimensional space, when the flying ball direction is the X component, the depth direction viewed from the golfer 11 is the Y component, the vertical direction from the ground is the Z component, and the vertex of the frame 20 is the origin, the X and Y components are A projective transformation matrix obtained by converting the side image into the real space for the ground in 4) is used.

  For example, the Z coordinate of the head position 66 before impact is based on the assumption 6, and the conversion rate sf of the vertical plane including the point 73 where the horizontal line passing through the head ground position 70 intersects the frame 20 and the head ground position 70 is the first. The Z coordinate position of the head position 66 can also be obtained by multiplying the pixel distance between the head position 66 and the head ground position 70 obtained by the linear interpolation of the third embodiment and estimated on the side screen by the conversion rate sf.

It is a lineblock diagram of the golf swing diagnostic system of a 1st embodiment of the present invention. It is a top view of a frame. (A) (B) is drawing which shows the positional relationship of a flame | frame and a golfer. It is drawing which shows a front image. It is drawing explaining utilization of a vanishing point. It is drawing which shows a modification. (A) is a front image of 2nd Embodiment, (B) is drawing which shows a side image. It is drawing which shows a front image. (A) is a front image showing a modified example of block division, and (B) is a drawing showing a side image. (A) is a front image of 3rd Embodiment, (B) is drawing which shows a side image. It is a conceptual diagram explaining the positional relationship of each point. It is drawing which shows a ratio. (A) is a front image, and (B) is a drawing showing a side image. It is drawing which shows the front image of 4th Embodiment. It is drawing which shows a side image. It is explanatory drawing regarding the diagnosis of a swing track | orbit.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Golfer 12 Measurement clothes 13 Golf club 13a Shaft 13b Head part 14, 15 Color CCD camera 16 Computer 20 Frame CM1-CM3 Colored mark P, Q Colored mark for clubs

Claims (19)

A method of capturing a golfer who grips and swings a golf club, capturing it in a computer, calibrating the position information of a point of interest displayed on the captured image with the computer, and obtaining actual position information,
A frame is provided at a position surrounding the golfer on the ground that constitutes the swing real space,
At least two straight lines constituting the frame are recognized on the image, and the actual length of the two straight lines in the swing real space and the pixel length on the image are used to calculate the actual position from the pixel position information in the frame. A calibration method characterized by calculating a conversion rate to be converted into position information and calculating an actual distance by multiplying the conversion rate by a pixel distance on the image.   The calibration method according to claim 1, wherein the frame is a virtual frame, and a mark is placed on the ground at a position that is a vertex of the shape of the virtual frame.   The conversion rate in the said frame is calculated | required by averaging the said conversion rate on two straight lines opposingly arranged by the image depth among the straight lines which comprise the said frame. Calibration method.   The calibration method according to claim 3, wherein an actual distance between the two points of interest is obtained by multiplying a pixel distance between the two points of interest on the image by the conversion rate. Dividing the region in the frame into a plurality of blocks;
The calibration according to claim 1 or 2, wherein a conversion rate in the block is obtained by averaging conversion rates on two straight lines arranged opposite to the image depth among straight lines constituting the block. Method.   The calibration method according to claim 5, wherein the block is formed by dividing an area in the frame into blocks based on a foot silhouette of the golfer.   The actual distance between the two points of interest is obtained by averaging the conversion rates of the blocks to which the points of interest belong and multiplying the pixel distances between the points of interest on the image. The calibration method according to claim 6.   The calibration method according to claim 1 or 2, wherein a conversion rate in the frame is obtained by linear interpolation using at least two straight lines constituting the frame.   The actual distance between the two points of interest is calculated by projecting one point of interest on the surface of the other point of interest on the image, and the distance between the projected point and the other point of interest. The calibration method according to claim 8, which is obtained by multiplying the conversion rate. The actual distance between the surface or line present on the image and the point of interest is
The said attention point is projected on the surface which comprises the said surface or a line, The distance of this projection point and the said surface or a line is calculated | required by multiplying by the conversion rate on the said surface. Calibration method.   On the image, the points that intersect the straight lines that are the sides in the depth direction of the frame in the depth direction are defined as vanishing points. The calibration method according to claim 1, wherein a distance between each point of interest is converted based on a distance between the two points.   The calibration method according to any one of claims 1 to 11, wherein the frame is set in a range including a part of a locus of the golf club. If the length of one side of the frame is W1 and the length of the other side perpendicular to the frame is W2, W1 / W2 =
The calibration method according to claim 1, wherein the calibration method is 1 to 10.   The size of the frame is set to h / i = 1 to 3 when the sides of the frame are divided by h: i in the flying ball direction from the midpoint of the center of the golfer's feet as a base point. 14. The calibration method according to any one of items 13.   The size of the frame is set to j / k = 1 to 10 when the sides of the frame are divided by j: k in a direction orthogonal to the flying ball direction with a golfer's foot as a base point. 14. The calibration method according to any one of items 13. A method of capturing a golfer who grips and swings a golf club, capturing it in a computer, calibrating the position information of a point of interest displayed on the captured image with the computer, and obtaining actual position information,
For at least four points existing in the same plane on the image taken from two directions, a projection transformation matrix representing the correspondence relationship of the position coordinates between the images in the two directions is obtained in advance.
A vertical line that passes through the point of interest on the image in one direction and represents the depth direction is converted into a horizontal line on the image in the other direction by the projection transformation matrix, and passes through the point of interest on the image in the other direction. Obtain the position coordinates on the plane on the image by finding the intersection of the vertical line and the horizontal line,
The position coordinates are converted into the position coordinates on the real space using another projective transformation matrix representing the correspondence relationship between the position coordinates between the image and the real space, and the real space coordinates of the attention point are obtained. A calibration method characterized by that.   The calibration method according to claim 16, wherein the attention point is two points, and a distance or / and an angle between the two points is obtained. A frame is provided at a position surrounding the golfer on the ground that constitutes the swing real space,
At least two straight lines constituting the frame are recognized on the image, and the actual length of the two straight lines in the swing real space and the pixel length on the image are used to calculate the actual position from the pixel position information in the frame. Calculate the conversion rate to convert to location information,
A three-dimensional position coordinate in the real space of the target point is obtained by obtaining a pixel distance in the direction of a specific coordinate axis of the target point on the one-direction image and multiplying the pixel distance by the conversion rate. The calibration method according to claim 16 or claim 17. Using the calibration method according to any one of claims 1 to 18,
The point of interest is the golfer's joint, and the distance between one joint and another joint or golf club shaft, or the distance between one joint and a joint or golf club shaft at another time is obtained. A golf swing measurement system.

Images