JP2017116280A - Camera calibration system, camera calibration program, and camera calibration method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a camera calibration system that can reduce the number of calibration points to be used in calibration of cameras.SOLUTION: A camera calibration system 1 comprises a stationary camera 2 and a computation device 3. The stationary camera 2 is configured to acquire an image having an actual measurement space photographed. The actual measurement space includes at least three calibration points. The computation device 3 is configured to measure a digitized coordinate value on the image of the calibration point, and further the computation device 3 is configured to acquire an optimal solution of a camera parameter of the stationary camera 2 on the basis of a three-dimensional coordinate value of the calibration point in an actual spatial coordinate system, a computation expression for calculating a projection coordinate value having the three-dimensional coordinate value of the calibration point projected onto the digitized coordinate system, a digitized coordinate of the calibration point, in which the computation expression includes the camera parameter of the stationary camera 2.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a camera calibration system, a camera calibration program, and a camera calibration method.

  As a three-dimensional motion analysis method, a DLT method (Direct-Liner-Transformation method) is known (Non-Patent Document 1). The DLT method is generally used for analyzing the three-dimensional motion of a player in a game or competition.

  When the DLT method is used, generally, a calibration point having a known three-dimensional coordinate value is captured by a plurality of cameras, and the camera is calibrated. Specifically, from the geometric relationship between the three-dimensional coordinate values of the calibration points and the digitized coordinate values (coordinate values on the image), 11 calibration coefficients (hereinafter referred to as DLT parameters) for one camera. Is calculated). Then, the camera is calibrated using 11 DLT parameters. When the DLT method is used, generally six or more calibration points that are not included in the same plane are used. Therefore, before the start of a game or competition, it is necessary to install six or more calibration points that are not included in the same plane in an actual measurement space such as a game venue.

Abdel-Aziz, Y.I. and Karara, H.M. (1971): Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. ASP symposium on close-range photogrammetry, American society of photogrammetry, Falls Church, VA: 1-18

  However, in an actual game venue or the like, it may be difficult to install six or more calibration points that are not included in the same plane.

  The present invention has been made in view of the above problems, and an object thereof is to provide a camera calibration system, a camera calibration program, and a camera calibration method capable of reducing the number of calibration points used for camera calibration. .

  The camera calibration system according to the present invention includes a fixed point camera and a calculation device. The fixed point camera acquires an image obtained by imaging the actual measurement space. The actual measurement space includes at least three calibration points. The arithmetic unit measures a digitized coordinate value on the image of the calibration point. Further, the calculation device includes a calculation formula for calculating a three-dimensional coordinate value of the calibration point in a real space coordinate system, and a projection coordinate value obtained by projecting the three-dimensional coordinate value of the calibration point onto a digitized coordinate system, Based on the digitized coordinate value of the calibration point, an optimal solution of the fixed point camera parameter is obtained. The arithmetic expression includes the parameters.

  In one embodiment, the arithmetic unit obtains the optimum solution that minimizes a difference between the digitized coordinate value of the calibration point and the projected coordinate value of the calibration point.

In one embodiment, the digitized coordinate value of the intersection of the optical axis of the fixed point camera and the image is (Uo, Vo), and the three-dimensional coordinate value in the real space coordinate system of the center of the lens included in the fixed point camera is ( Xo, Yo, Zo), the three-dimensional coordinate value of the measurement point in the real space coordinate system is (X, Y, Z), and the projected coordinate value of the projection point obtained by projecting the measurement point onto the digitized coordinate system is ( U, V), F as the focal length of the lens, R as the coordinate transformation matrix from the real space coordinate system to the digitized coordinate system, a as the aspect ratio of the fixed point camera, and camera distortion of the fixed point camera. And s, and the scale factor for setting the value of the third row and the first column included in the determinant indicating the projection point to 1 is h, the arithmetic expressions are the following expressions (1) and ( 2) and formula (3).

  In one embodiment, the arithmetic unit sets the intersection (Uo, Vo) between the optical axis of the fixed point camera and the image at the center of the image and obtains the optimal solution, and then the intersection (Uo, Vo) is added to the variable to obtain the optimal solution.

  In an embodiment, all the calibration points are included in the same plane in the actual measurement space.

  In one embodiment, the arithmetic unit obtains the optimal solution based on a genetic algorithm method.

  The camera calibration program according to the present invention causes a computer to obtain an optimal solution of a parameter of a fixed point camera that images a measured space. The actual measurement space includes at least three calibration points. The camera calibration program causes the computer to measure a digitized coordinate value of the calibration point on an image obtained by imaging the actual measurement space, a three-dimensional coordinate value of the calibration point in a real space coordinate system, and the calibration A calculation formula for calculating a projected coordinate value obtained by projecting a three-dimensional coordinate value of a point onto a digitized coordinate system and a procedure for obtaining the optimum solution based on the digitized coordinate value of the calibration point are executed. The arithmetic expression includes the parameters.

  In one embodiment, the computer is caused to determine the optimal solution that minimizes the difference between the digitized coordinate value of the calibration point and the projected coordinate value of the calibration point.

In one embodiment, the digitized coordinate value of the intersection of the optical axis of the fixed point camera and the image is (Uo, Vo), and the three-dimensional coordinate value in the real space coordinate system of the center of the lens included in the fixed point camera is ( Xo, Yo, Zo), the three-dimensional coordinate value of the measurement point in the real space coordinate system is (X, Y, Z), and the projected coordinate value of the projection point obtained by projecting the measurement point onto the digitized coordinate system is ( U, V), F as the focal length of the lens, R as the coordinate transformation matrix from the real space coordinate system to the digitized coordinate system, a as the aspect ratio of the fixed point camera, and camera distortion of the fixed point camera. And s, and the scale factor for setting the value of the third row and the first column included in the determinant indicating the projection point to 1 is h, the arithmetic expressions are the following expressions (1) and ( 2) and formula (3).

  In one embodiment, when the computer determines the optimum solution, the intersection point (Uo, Vo) between the optical axis of the fixed point camera and the image is set at the center of the image to obtain the optimum solution. And a procedure for adding the intersection (Uo, Vo) to a variable to obtain the optimum solution.

  In an embodiment, all the calibration points are included in the same plane in the actual measurement space.

  In one embodiment, the computer is caused to determine the optimal solution based on a genetic algorithm method.

  In the camera calibration method according to the present invention, a fixed point camera is used to acquire an image obtained by imaging an actual measurement space, and digitized coordinate values on the image of at least three calibration points included in the actual measurement space are measured. A three-dimensional coordinate value of the calibration point in the real space coordinate system, an arithmetic expression for calculating a projected coordinate value obtained by projecting the three-dimensional coordinate value of the calibration point onto the digitized coordinate system, Obtaining an optimal solution of the parameters of the fixed-point camera based on the digitized coordinate value. The arithmetic expression includes the parameters.

  According to the present invention, the number of calibration points used for camera calibration can be reduced.

It is a figure which shows the structure of the camera calibration system which concerns on embodiment of this invention. It is a figure which shows typically the relationship between the three-dimensional coordinate value of the measuring point in a real space coordinate system, and the digitized coordinate value of the measuring point in a digitizing coordinate system. It is a figure which shows the flow of the camera calibration method which concerns on embodiment of this invention. It is a figure which shows the other flow of the camera calibration method which concerns on embodiment of this invention. (A) is a figure which shows the feature point of a throwing runway. (B) is a figure which shows the measurement space which concerns on the Example of this invention. It is a figure which shows 54 calibration points installed in measurement space. It is a figure which shows the change of the reprojection error in the optimization process using the genetic algorithm method which concerns on Example 1 of this invention. It is a figure which shows the measurement result which concerns on Example 3 of this invention. It is a figure which shows the measurement result which concerns on Example 3 of this invention. It is a figure which shows the measurement result which concerns on Example 3 of this invention. It is a figure which shows the measurement result which concerns on Example 3 of this invention. It is a figure which shows the measurement result which concerns on Example 4 of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, in the drawings, the same or corresponding parts are denoted by the same reference numerals, and description thereof will not be repeated. In addition, this invention is not restricted to the following embodiment, It is possible to implement in a various aspect in the range which does not deviate from the summary.

  FIG. 1 is a diagram illustrating a configuration of a camera calibration system according to the present embodiment. As shown in FIG. 1, the camera calibration system 1 includes at least one fixed point camera 2 and an arithmetic device 3. The camera calibration system 1 calibrates the fixed point camera 2 by obtaining an optimal solution of the parameters of the fixed point camera 2 (hereinafter sometimes referred to as camera parameters) included in the predetermined arithmetic expression. Hereinafter, the predetermined arithmetic expression is referred to as an arithmetic expression including a camera parameter. The arithmetic expression including the camera parameter is an expression for calculating a projected coordinate value obtained by projecting the three-dimensional coordinate value of the measurement point in the real space coordinate system onto the digitized coordinate system. In other words, the three-dimensional coordinate value of the measurement point in the real space coordinate system is obtained from the digitized coordinate value on the image of the measurement point (the coordinate value in the digitized coordinate system of the measurement point) based on the arithmetic expression including the camera parameters. be able to.

  Note that a plurality of images are required to obtain the three-dimensional coordinate value in the real space coordinate system from the digitized coordinate value. For a plurality of images, images taken from a plurality of different positions by moving one fixed point camera 2 can be used. Alternatively, images obtained by arranging a plurality of fixed point cameras 2 at a plurality of different positions can be used.

  The fixed point camera 2 is arranged at a position where an actual measurement space set according to the analysis target of the three-dimensional motion can be included in the imaging range. In other words, the fixed point camera 2 is arranged at a position where the analysis target of the three-dimensional motion can be imaged. The type of the fixed point camera 2 is not particularly limited, but a digital video camera is typically used.

  The fixed-point camera 2 images the actual measurement space when the fixed-point camera 2 is calibrated (when an optimal solution for camera parameters is acquired). The actual measurement space includes at least three calibration points whose three-dimensional coordinate values in the real space coordinate system are known. For example, three calibration points are included in any horizontal plane (an example of the same plane) in the measurement space. When using four or more calibration points, only the points included in the same plane may be used as the calibration points, or at least one of the calibration points may be a point not included in the same plane. Also good.

  The arithmetic device 3 acquires an optimal solution of camera parameters from an image (an image including a calibration point) captured by the fixed point camera 2. Specifically, the arithmetic device 3 calculates optimal solutions for internal parameters and external parameters of the fixed point camera 2. The arithmetic device 3 includes a display unit 31, an input unit 32, a storage unit 33, and a control unit 34. Although the apparatus which comprises the arithmetic unit 3 is not specifically limited, For example, a personal computer can be used.

  The display unit 31 displays information processed by the control unit 34 as an image. Although the kind of display part 31 is not specifically limited, For example, a liquid crystal display can be used. When calculating the optimal solution for the camera parameter, the display unit 31 displays an image captured by the fixed point camera 2 (an image including a calibration point). Note that the image data can be taken into the arithmetic device 3 from the fixed point camera 2 using, for example, a cable such as a USB cable or a storage device such as an SD memory card as a medium.

  The input unit 32 is operated by the user and inputs various processing commands and various information to the control unit 34. The input unit 32 typically includes a pointing device such as a mouse and a keyboard. Alternatively, the input unit 32 may include a touch panel. In the present embodiment, the user operates the input unit 32 to specify any point included in the image displayed on the display unit 31 and cause the control unit 34 to recognize the specified point. Can do. Specifically, the user can designate a calibration point included in an image captured by the fixed point camera 2 when calculating the optimal solution of the camera parameter.

  The storage unit 33 stores an OS (Operating System) program, information input by the input unit 32, information processed by the control unit 34, and the like. Although the apparatus which comprises the memory | storage part 33 is not specifically limited, Typically, a non-volatile memory | storage device is used. Specifically, an HDD (Hard Disk Drive) or a flash memory can be used as the nonvolatile storage device.

  The control unit 34 comprehensively controls the operation of the arithmetic device 3. The control unit 34 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory). Specifically, the control unit 34 performs various processes such as numerical calculation, information processing, and device control by causing the CPU to execute various control programs including a camera calibration program.

  The camera calibration program causes the control unit 34 to acquire an optimal solution of camera parameters from an image (an image including a calibration point) captured by the fixed point camera 2. The camera calibration program is stored in advance in the storage unit 33, for example. Alternatively, the camera calibration program is stored in advance in the ROM.

  Specifically, the control unit 34 measures the digitized coordinate value (the coordinate value of the calibration point in the digitized coordinate system) on the calibration point image. Further, the control unit 34, based on the three-dimensional coordinate value of the calibration point included in the actual measurement space, the arithmetic expression including the camera parameters (internal parameters and external parameters of the fixed point camera 2), and the digitized coordinate value of the calibration point, Get the optimal solution for camera parameters. The arithmetic expression including the camera parameters is stored in association with the camera calibration program.

  In the present embodiment, the display unit 31 displays an image (an image including a calibration point) captured by the fixed point camera 2 when calculating the optimal solution for the camera parameter. When the user operates the input unit 32 to specify a calibration point included in the image displayed on the display unit 31, the control unit 34 measures the digitized coordinate value on the calibration point image. That is, the control unit 34 digitizes the calibration point. Next, the control unit 34 obtains an optimal solution of the camera parameter based on the three-dimensional coordinate value of the calibration point included in the actual measurement space, the arithmetic expression including the camera parameter, and the digitized coordinate value on the calibration point image. To do.

  For example, the control unit 34 first calculates a projection coordinate value of the calibration point based on an arithmetic expression including a camera parameter. Thereafter, the control unit 34 minimizes the difference between the digitized coordinate value on the image of the calibration point and the projection coordinate value of the calibration point (hereinafter, may be referred to as a reprojection error) as an optimal solution. Calculate camera parameters. Assuming that there is no digitization error, the projected coordinate value of the calibration point calculated based on the calculation formula including the camera parameter matches the digitized coordinate value on the calibration point image when the camera parameter is correct. . Therefore, the fixed point camera 2 can be calibrated by obtaining a camera parameter that minimizes the reprojection error as an optimal solution. Note that the optimization processing technique for obtaining the optimal solution of the camera parameter is not particularly limited. For example, the control unit 34 can obtain an optimal solution based on a genetic algorithm method.

Next, a specific example of an arithmetic expression including camera parameters, that is, an expression for calculating a projected coordinate value obtained by projecting a three-dimensional coordinate value of a measurement point in a real space coordinate system onto a digitized coordinate system will be described. In the present embodiment, the arithmetic expression including the camera parameter includes the following expressions (1), (2), and (3).

In Expression (1), the first term on the right side indicates an internal parameter of the fixed point camera 2. In other words, the variables included in the following matrix (4) are internal parameters of the fixed point camera 2.

In Expression (1), the second term and the third term on the right side indicate external parameters of the fixed point camera 2. In other words, the variables included in the following matrix (5) are external parameters of the fixed point camera 2.

  The control unit 34 acquires optimal solutions for variables included in the matrix (4) and the matrix (5) when the fixed point camera 2 is calibrated. Furthermore, the control unit 34 replaces the camera parameter included in the equation (1) with the optimal solution. When analyzing the actual three-dimensional motion, the three-dimensional coordinate value in the real space coordinate system of the measurement point is calculated based on the equation (1) after being replaced with the optimal solution.

  Here, with reference to FIG. 2, each variable contained in Formula (1) and Formula (2) is demonstrated. FIG. 2 is a diagram schematically showing the relationship between the three-dimensional coordinate value of the measurement point MX in the real space coordinate system and the digitized coordinate value (coordinate value on the image of the measurement point) of the measurement point in the digitizing coordinate system. is there.

  Expressions (1) and (2) are obtained by combining the three-dimensional geometric relationship between the measurement point MX and the projection point mx shown in FIG. 2 in a homogeneous coordinate system. Specifically, the variables (Uo, Vo) included in the equations (1) and (2) indicate the digitized coordinate values of the intersections between the optical axis L of the fixed point camera 2 and the digitizing plane 21 (image). Variables (Xo, Yo, Zo) indicate three-dimensional coordinate values in the real space coordinate system of the center O of the lens 22 included in the fixed point camera 2. Variables (X, Y, Z) indicate three-dimensional coordinate values of the measurement point MX in the real space coordinate system. Variables (U, V) indicate the projection coordinate value of the projection point mx obtained by projecting the measurement point MX onto the digitized coordinate system. A variable F indicates the focal length of the lens. In the present embodiment, the center of the digitizing plane 21 is the origin of the digitizing coordinate system.

  In Equation (1), R represents a coordinate transformation matrix from the real space coordinate system to the digitized coordinate system, a represents the aspect ratio of the fixed point camera 2, and s represents the camera distortion (skew) of the fixed point camera 2. H represents a scale factor for setting the value of the third row and the first column included in the determinant (see Expression (2)) indicating the projection point mx to 1.

Further, from the third row of the equation (1), the relationship of the following equation (6) is established for the scale factor h.

Further, when the first row and the second row of the formula (1) are combined by the focal length F, the following formula (7) is established.

Therefore, from the expression (7), the following expressions (8), (9), and (10) are established for the focal length F. In Expressions (8) and (9), A ⁺ is a pseudo inverse matrix of A.

  The coordinate transformation matrix R can be represented by Euler angles (θ, ψ, φ). By representing the coordinate transformation matrix R with Euler angles (θ, ψ, φ), the number of camera parameters for which an optimal solution is to be calculated can be reduced.

  When the camera parameter optimization process is performed, the arithmetic device 3 (control unit 34) described with reference to FIG. 1 can perform the optimization process in two stages. Specifically, in the first stage, the control unit 34 sets an intersection (Uo, Vo) between the optical axis L of the fixed point camera 2 and the digitizing plane 21 (image) as the center of the digitizing plane 21 and performs an optimization process. Execute. More specifically, the control unit 34 sets the intersection (Uo, Vo) at the center of the digitizing plane 21. Further, the focal length F is calculated from the equations (8), (9), and (10). Then, an optimal solution of camera parameters (variables) excluding the three camera parameters (Uo, Vo, F) is obtained (calculated). Next, in the second stage, the control unit 34 adds the intersection (Uo, Vo) and the focal length (F) to the variables and executes the optimization process. By performing the camera parameter optimization process in two stages, it is possible to increase the speed of the optimization process and to suppress the possibility that the result of the optimization process falls into a local solution.

When the aspect ratio a is 1 and the skew s is 0, the equation (1) can be converted into the following equation (11). Therefore, the arithmetic device 3 (control unit 34) described with reference to FIG. 1 can obtain an optimal solution of camera parameters based on the following equation (11).

  In Equation (11), the coordinate transformation matrix R is represented by Euler angles (θ, ψ, φ), so that camera parameters (variables) for calculation of the optimal solution are (Uo, Vo, F, Xo, Yo, (Zo, θ, ψ, φ).

  Subsequently, a camera calibration method using the camera calibration system 1 described with reference to FIG. 1 will be described. FIG. 3 is a diagram illustrating a flow of the camera calibration method according to the present embodiment.

  As shown in FIG. 3, when calibrating the fixed point camera 2, first, the measured space is imaged using the fixed point camera 2 (step S31). The actual measurement space includes at least three calibration points whose three-dimensional coordinate values in the real space coordinate system are known. For example, three calibration points are included in any horizontal plane (same plane) in the measurement space.

  Next, the digitizing coordinate value on the image of each calibration point is measured using the arithmetic device 3 (control unit 34) (step S32). Next, camera parameter optimization processing is performed using the arithmetic device 3 (control unit 34) (step S33). Specifically, the optimal solution of the camera parameter is obtained based on the three-dimensional coordinate value of each calibration point included in the actual measurement space, the arithmetic expression including the camera parameter, and the digitized coordinate value on the image of each calibration point. To do.

  Next, another camera calibration method using the camera calibration system 1 described with reference to FIG. 1 will be described. FIG. 4 is a diagram illustrating another flow of the camera calibration method according to the present embodiment.

  The camera calibration method shown in FIG. 4 differs from the camera calibration method shown in FIG. 3 in that the camera parameter optimization process is performed in two stages. That is, in the camera calibration method shown in FIG. 4, the arithmetic device 3 (control unit 34) causes the first optimization process (step S41) and the second optimization process (step S42) to be executed.

  In the first optimization process (step S41), an optimal solution of camera parameters excluding the three camera parameters (Uo, Vo, F) is calculated. Specifically, the intersection (Uo, Vo) between the optical axis L of the fixed point camera 2 and the image (digitized plane 21) is set at the center of the image. Further, the focal length F is calculated from the equations (8), (9), and (10). Then, an optimal solution of the camera parameter is acquired based on the three-dimensional coordinate value of each calibration point included in the actual measurement space, the arithmetic expression including the camera parameter, and the digitized coordinate value on the image of each calibration point. In the second optimization process (step S42), three camera parameters (Uo, Vo, F) are added to the variables, the three-dimensional coordinate values of the respective calibration points included in the actual measurement space, and an arithmetic expression including the camera parameters, Based on the digitized coordinate values on the image of each calibration point, an optimal solution of the camera parameter is obtained.

  The embodiments of the present invention have been described above with reference to the drawings. According to the embodiment of the present invention, the number of calibration points used for camera calibration can be reduced to three. Therefore, it is easy to set a calibration point in an actual game hall or the like.

  Also, in sports game halls and the like, a line whose length is specified by a rule (for example, a line indicating a baseball batter box) is often drawn. Such a line (hereinafter sometimes referred to as a prescribed line) is usually drawn on a horizontal plane. In other words, they are drawn on the same plane. On the other hand, since the DLT method generally used for three-dimensional motion analysis requires six or more calibration points that are not included in the same plane, feature points included in the specified line (for example, each vertex of a baseball batter box) ) Cannot be used to calibrate the camera. On the other hand, according to the embodiment of the present invention, the camera can be calibrated using only calibration points included in the same plane. Therefore, the camera can be calibrated using only the feature points included in the defined line. Furthermore, since the feature points included in the defined line can be used as calibration points, the camera can be calibrated without entering the game hall or the like and installing calibration points.

  Further, according to the embodiment of the present invention, the calibration of the camera can be performed using the calibration point that is stationary. Therefore, when imaging a measurement space (calibration point) from different angles using a plurality of cameras, it is not necessary to synchronize the plurality of cameras. Furthermore, different calibration points can be used to calibrate each camera. Further, the camera can be calibrated by imaging the actual measurement space (calibration point) from different angles using one camera.

  Hereinafter, the present invention will be described more specifically with reference to examples. The present invention is not limited to the scope of the examples. In this example, the camera was calibrated using the feature points included in the throwing runway.

[Example 1 and Comparative Example 1]
[Camera parameters]
In Example 1, two digital video cameras DV1 and DV2 (HC-V750M manufactured by Panasonic Corporation, resolution: 1920 pixels × 1080 pixels) were used as fixed point cameras. The image sensors of the digital video cameras DV1 and DV2 are considered to be square pixels, and the aspect ratio of the digitizing plane 21 is 1. Therefore, the aspect ratio a is 1 and the skew s is 0. That is, as the formula (calculation formula including camera parameters) for calculating the projected coordinate value obtained by projecting the three-dimensional coordinate value of the measurement point in the real space coordinate system onto the digitized coordinate system, the formula (11), formula (2), and Formula (3) was used. Further, the coordinate transformation matrix R is expressed by Euler angles (θ, ψ, φ), and nine values (variables) of (Uo, Vo, F, Xo, Yo, Zo, θ, ψ, φ) are camera parameters. It was. The shooting speed of the digital video cameras DV1 and DV2 was 60 frames per second (progressive), and the shutter speed was 1/500 seconds.

[Calibration point]
FIG. 5A is a diagram showing the characteristic points of the throwing runway 51. In FIG. 5A, the direction indicated by the arrow Y is the throwing direction. The throwing runway 51 is defined by two lines 52 and 53 drawn along the throwing direction. The two lines 52 and 53 are drawn in parallel in the direction orthogonal to the throwing direction (direction indicated by the arrow X) with an interval of 4 m.

  The crossing line 54 (starting line) is drawn in an arc shape. Specifically, the crossing line 54 is an arc-shaped line having a radius of 8 m with both front ends of the two lines 52 and 53 as both ends. Specifically, the arc inside the crossing line 54 is an arc having a radius of 8 m. The center of the crossing line 54 (arc having a radius of 8 m) is located on the middle of the two lines 52 and 53.

In Example 1, the intersection of the line 52 and the crossing line 54 is CP1, the intersection of the line 53 and the crossing line 54 is CP2, and the center of the crossing line 54 is CP3. Specifically, the two intersections between the inner side of each of the two lines 52 and 53 and the inner arc of the crossing line 54 are CP1 and CP2. In addition, CP1 is the origin (0, 0, 0), the direction from CP1 to CP2 is the + X direction, the throwing direction is the + Y direction, and the vertically upward direction is the + Z direction. Therefore, the coordinate value of CP2 is (4, 0, 0), and the coordinate value of CP3 is (2,-(60) ^1/2 , 0). Further, the axis intersecting CP1 and CP2 is the X axis, and the axis along the line 52 is the Y axis. In Example 1, the camera was calibrated using CP1, CP2, and CP3 as calibration points.

[Optimization method]
In Example 1, based on the genetic algorithm method, an optimal camera parameter solution that minimizes reprojection error was obtained. At this time, camera parameter optimization processing was performed in two stages. Specifically, in the first stage, the intersection (Uo, Vo) between the optical axis of the fixed point camera (digital video camera DV1, DV2) and the digitizing plane (image) is set at the center of the digitizing plane. Further, the focal length F was calculated from the equations (8), (9), and (10). Then, based on the genetic algorithm method, an optimal solution of six camera parameters (Xo, Yo, Zo, θ, ψ, φ) excluding the three camera parameters (Uo, Vo, F) was obtained. In the second stage, nine camera parameters (Uo, Vo, F, Xo, Yo, Zo, θ, ψ, The optimal solution of φ) was obtained. Among the variables used in the genetic algorithm method, the number of individuals was set to 1000, the crossover rate was set to 70%, and the mutation rate was set to 5%. With no termination condition, 100 generations of optimization processing were performed at each stage. That is, 200 generations of optimization processing were performed.

[experimental method]
FIG. 5B is a diagram illustrating the actual measurement space 55 according to the first embodiment. In the first embodiment, the actual measurement space 55 has a rectangular parallelepiped shape having the bottom surface of the throwing aid runway 51 described with reference to FIG. Specifically, the actual measurement space 55 is set such that CP1 and CP2 are part of the apex of the actual measurement space 55, and CP3 is included in the lower surface of the actual measurement space 55. Further, the length of the actual measurement space 55 in the Y-axis direction was set to 8.0 m, and the height of the actual measurement space 55 in the Z-axis direction was set to 2.5 m.

  In Example 1, the digital video cameras DV1 and DV2 were calibrated based on images obtained by imaging the actual measurement space 55 with the two digital video cameras DV1 and DV2. That is, the optimal solution of the camera parameters of the two digital video cameras DV1 and DV2 (hereinafter sometimes referred to as the first digital video camera DV1 and the second digital video camera DV2) was obtained. The first digital video camera DV1 was installed as high as possible at a point about −50 m away from the center of the actual measurement space 55 in the X-axis direction. Specifically, it was installed at a position about 9m away from the ground. On the other hand, the second digital video camera DV2 was installed as high as possible at a point about −50 m away from the center of the actual measurement space 55 in the Y-axis direction. Specifically, it was installed at a position about 4 m away from the ground. The reason why the digital video cameras DV1 and DV2 are set as high as possible is to make it easy to identify the characteristic points (calibration points CP1, CP2, and CP3) of the throwing runway 51.

  In Example 1, the measurement accuracy of the digital video cameras DV1 and DV2 after calibration was verified. More specifically, markers are set at eight vertices of the actual measurement space 55, and three-dimensional coordinate values of the eight markers are measured (calculated) from images obtained by imaging the actual measurement space 55 with the calibrated digital video cameras DV1 and DV2. The actual measurement space 55 was reconstructed.

  In Comparative Example 1, the digital video cameras DV1 and DV2 were calibrated using the DLT method. That is, the DLT parameters of the two digital video cameras DV1 and DV2 were obtained. Specifically, a total of 54 calibration points were installed by vertically setting up calibration poles having a length of 2.5 m with calibration points attached at intervals of 0.5 m at nine locations in the actual measurement space 55. FIG. 6 is a diagram showing 54 calibration points installed in the actual measurement space 55. The DLT parameters were calculated using 54 calibration points installed as shown in FIG. Further, similarly to the first embodiment, based on the image obtained by imaging the actual measurement space 55 with the calibrated digital video cameras DV1 and DV2, the three-dimensional coordinate values at the eight vertices of the actual measurement space 55 are measured (calculated). Then, the actual measurement space 55 was reconstructed.

  In FIG. 6, calibration point 1 is calibration point CP1, calibration point 13 is calibration point CP2, and calibration point 43 is calibration point CP3. The coordinates of the calibration point 7 are (2, 0, 0), the coordinates of the calibration point 19 are (0, -4, 0), and the coordinates of the calibration point 25 are (2, -4, 0). Yes, the coordinates of the calibration point 31 are (4, -4, 0).

The DLT parameters can be summarized by the following equation (12), where P is a matrix (first to third terms) relating to the camera parameters on the right side of equation (1).

In general, the variable (P′12) in the third row and the fourth column of Equation (12) is set to 1, and 11 DLT parameters (P1 to P11) shown in Equation (13) below are calculated. Calibrate the camera.

  In Comparative Example 1, the DLT parameter was calculated based on the formula (13). In Expression (13), k is a scale factor for setting the variable (P′12) in the third row and the fourth column on the right side of Expression (12) to 1.

[Data processing]
Using motion analysis software (manufactured by DKH, Frame-DIAS IV), digitized coordinate values (two-dimensional coordinate values in the digitized coordinate system) of calibration points were obtained from images taken by the digital video cameras DV1 and DV2. In Example 1 and Comparative Example 1, in order to reduce the digitization error as much as possible, an average value of data digitized separately by four persons was used. The maximum difference between the digitized data of the four persons was 2 pixels in the horizontal direction (about 0.1% with respect to the resolution). In the vertical direction, it was 3.5 pixels (about 0.3% with respect to the resolution).

[result]
FIG. 7 is a diagram illustrating a change in the reprojection error in the optimization process using the genetic algorithm method according to the first embodiment. In FIG. 7, the vertical axis represents reprojection error, and the horizontal axis represents generation. As shown in FIG. 7, the reprojection error decreased rapidly after the optimization process was started. Thereafter, it gradually decreased and reached almost 0 pixels at the end of the optimization process. Note that, as a result of performing the optimization processing according to Example 1 a plurality of times, all converged to almost the same solution. In other words, it was confirmed that the solution obtained by the optimization process according to the first embodiment does not fall into a local solution.

Table 1 below shows the optimal solution of the camera parameters (Uo, Vo, F, Xo, Yo, Zo, θ, ψ, φ) calculated using the calibration points CP1, CP2, and CP3.

  As shown in Table 1, the center position (Xo, Yo, Zo) of the lens of the first digital video camera DV1 was calculated as (−49.76 m, −4.03 m, 8.92 m). The center position (Xo, Yo, Zo) of the lens of the second digital video camera DV2 was calculated as (1.23 m, −52.75 m, 3.64 m).

Table 2 below shows the measurement results according to Example 1 and Comparative Example 1. Specifically, the difference (error) between the reconstructed actual measurement space 55 and the actual actual measurement space 55 (4.0 m × 8.0 m × 2.5 m) is shown. More specifically, standard errors (SE) and maximum errors (Max) in the X-axis direction (X axis), the Y-axis direction (Y axis), and the Z-axis direction (Z axis) are shown. In Table 2, the standard error of the absolute value (Absolute) is the absolute value of the square root of the value obtained by adding the squares of the standard errors in the X-axis direction, the Y-axis direction, and the Z-axis direction. Is the absolute value of the square root of the value obtained by adding the squares of the maximum errors in the X-axis direction, the Y-axis direction, and the Z-axis direction.

  As shown in Table 2, the standard error in the X-axis direction is 0.002 m (0.05% with respect to the actual measurement range) in Example 1, and 0.003 m (0.003 m with respect to the actual measurement range) in Comparative Example 1. 07%). The standard error in the Y-axis direction was 0.003 m (0.04% with respect to the actual measurement range) in both Example 1 and Comparative Example 1. The standard error in the Z-axis direction was 0.002 m (0.08% with respect to the actual measurement range) in Example 1, and 0.003 m (0.11% with respect to the actual measurement range) in Comparative Example 1. Therefore, there was no significant difference between Example 1 and Comparative Example 1.

  The maximum error in the X-axis direction is 0.012 m (0.30% with respect to the actual measurement range) in Example 1, and 0.013 m (0.33% with respect to the actual measurement range) in Comparative Example 1. It was. The maximum error in the Y-axis direction was 0.016 m (0.20% with respect to the actual measurement range) in Example 1, and 0.013 m (0.16% with respect to the actual measurement range) in Comparative Example 1. The maximum error in the Z-axis direction was 0.008 m (0.32% with respect to the actual measurement range) in Example 1, and 0.012 m (0.48% with respect to the actual measurement range) in Comparative Example 1. Therefore, there was no significant difference between Example 1 and Comparative Example 1.

[Example 2 and Comparative Example 2]
In the second embodiment, a calibrated digital video camera DV1 is provided in the same manner as in the first embodiment, with a 1 m long pole with markers attached to both ends being randomly stopped at 40 locations in the actual measurement space 55. The measured space 55 was imaged by DV2. In Example 2, the three-dimensional coordinate values of the two markers attached to both ends of the pole were calculated based on the image obtained by imaging the actual measurement space 55. Then, the distance between the two markers was measured from the calculated three-dimensional coordinate values of the two markers. Hereinafter, the distance between the two markers measured from the calculated three-dimensional coordinate values of the two markers is referred to as the measured distance between the markers. In Comparative Example 2, as in Comparative Example 1, the digital video cameras DV1 and DV2 were calibrated using the DLT method. Similarly to Example 2, the distance between two markers attached to both ends of the pole was measured using the calibrated digital video cameras DV1 and DV2.

Table 3 below shows the measurement results according to Example 2 and Comparative Example 2. Specifically, an average value (Mean ± SD), a minimum value (Min), and a maximum value (Max) of the distances between the measured markers are shown.

  As shown in Table 3, the minimum value of the measured distance between the markers was 0.994 m in Example 2, and 0.993 m in Comparative Example 2. The maximum distance between the measured markers was 1.008 m in Example 2 and 1.007 m in Comparative Example 2. Therefore, there was no significant difference between Example 2 and Comparative Example 2.

[Example 3]
In Example 3, the influence of the number of calibration points used when obtaining the optimal solution of the camera parameter was verified. Specifically, the digital video cameras DV1 and DV2 are calibrated by changing the number of calibration points, and the difference (error) between the reconstructed actual measurement space 55 and the actual actual measurement space 55 is obtained as in the first embodiment. Verified. Specifically, from the 54 calibration points described with reference to FIG. 6, 3 points, 6 points, 12 points, 18 points, 24 points, 36 points, 45 points, and 54 points are selected. The optimal solution was obtained. The relationship between the number of calibration points and the calibration points used is shown in Table 4 below.

  In addition, when acquiring the optimal solution of a camera parameter using three calibration points, the calibration points 1, 13, and 43 shown in FIG. 6 were used. That is, the three calibration points CP1, CP2, and CP described with reference to FIG. 5 were used. Moreover, when acquiring the optimal solution of a camera parameter using 54 calibration points, all the calibration points 1-54 shown in FIG. 6 were used.

  8 to 11 are diagrams showing measurement results according to Example 3. FIG. Specifically, FIG. 8 shows the relationship between the standard error (Standard error) in the X-axis direction between the reconstructed actual measurement space 55 and the actual actual measurement space 55, and the number of calibration points. FIG. 9 shows the relationship between the standard error in the Y-axis direction between the reconstructed actual measurement space 55 and the actual actual measurement space 55, and the number of calibration points. FIG. 10 shows the relationship between the standard error in the Z-axis direction between the reconstructed actual measurement space 55 and the actual actual measurement space 55, and the number of calibration points. FIG. 11 shows the relationship between the standard error of the absolute value and the number of calibration points. The standard error of the absolute value is the absolute value of the square root of the value obtained by adding the squares of the standard errors in the X axis direction, the Y axis direction, and the Z axis direction. 8 to 11, the vertical axis represents standard error, and the horizontal axis represents the number of calibration points. As shown in FIGS. 8 to 11, the reconstructed measurement space 55 was not affected by the number of calibration points.

[Example 4]
In the fourth embodiment, as in the third embodiment, the influence of the number of calibration points used when obtaining the optimal solution of the camera parameters was verified. Specifically, the digital video cameras DV1 and DV2 were calibrated by changing the number of calibration points in the same manner as in Example 3. Further, as in Example 2, the distance between two markers attached to both ends of the pole was measured.

  FIG. 12 is a diagram illustrating measurement results according to Example 4. Specifically, FIG. 12 shows the relationship between the measured distance between markers and the number of calibration points. In FIG. 12, the vertical axis indicates the distance between the measured markers, and the horizontal axis indicates the number of calibration points. As shown in FIG. 12, the distance between the measured markers was not affected by the number of calibration points.

  The embodiment of the present invention has been described above. According to the results of Examples 1 and 2 and Comparative Examples 1 and 2 (Tables 2 and 3), the camera is calibrated by obtaining an optimal solution of the camera parameters that minimizes the reprojection error. Measurement accuracy comparable to that obtained when the camera was calibrated using a general DLT method was achieved. In other words, it was possible to achieve a measurement accuracy sufficient for analyzing the three-dimensional motion.

  Further, according to the results of Examples 3 and 4 (FIGS. 8 to 12), the measurement accuracy was hardly affected by the number of calibration points. In the embodiment of the present invention, in order to reduce the digitizing error as much as possible, the camera parameter is calculated using the average value of the data digitized by the four persons, and the difference between the digitized data of the four persons is 2 in the horizontal direction at the maximum. Pixel, 3.5 pixels in the vertical direction. From these facts, it was confirmed that when the digitizing error is small, the influence of the number of calibration points on the measurement accuracy is small, and the camera can be calibrated with sufficient accuracy even if the number of calibration points is three. Therefore, calibrating the camera to find the optimal camera parameter solution that minimizes the reprojection error is difficult to accurately measure the coordinate values of a large number of calibration points. This is effective when there is not enough time to measure the coordinate value.

  The present invention is useful for analyzing movements of players who are actually playing games or competitions.

DESCRIPTION OF SYMBOLS 1 Camera calibration system 2 Fixed point camera 3 Arithmetic device 21 Digitize plane 22 Lens 51 Throw runway 54 Crossing line 55 Actual space CP1-CP3 Calibration point L Optical axis MX Measurement point mx Projection point

Claims (13)

A fixed-point camera that acquires an image of the actual measurement space;
An arithmetic unit that obtains an optimal solution of the camera parameters of the fixed-point camera based on the image, and
The measured space includes at least three calibration points;
The arithmetic unit is:
Measuring the digitized coordinate value of the calibration point on the image;
A three-dimensional coordinate value of the calibration point in the real space coordinate system, an arithmetic expression for calculating a projected coordinate value obtained by projecting the three-dimensional coordinate value of the calibration point onto the digitized coordinate system, a digitized coordinate value of the calibration point, and To obtain the optimal solution based on
The camera calibration system, wherein the calculation formula includes the camera parameter.   The camera calibration system according to claim 1, wherein the arithmetic device acquires the optimal solution such that a difference between a digitized coordinate value of the calibration point and a projected coordinate value of the calibration point is minimized. The digitized coordinate value of the intersection of the optical axis of the fixed point camera and the image is (Uo, Vo),
The three-dimensional coordinate value in the real space coordinate system at the center of the lens provided in the fixed point camera is (Xo, Yo, Zo),
The three-dimensional coordinate value of the measurement point in the real space coordinate system is (X, Y, Z),
Projection coordinate values of projection points obtained by projecting the measurement points onto the digitized coordinate system are (U, V),
The focal length of the lens is F,
R is a coordinate transformation matrix from the real space coordinate system to the digitized coordinate system,
The aspect ratio of the fixed point camera is a,
The camera distortion of the fixed point camera is s,
When the scale factor for setting the value of the third row and the first column included in the determinant indicating the projection point to 1 is h,
The camera calibration system according to claim 1 or 2, wherein the calculation formula includes the following formula (1), formula (2), and formula (3).

  The arithmetic unit sets the intersection (Uo, Vo) between the optical axis of the fixed point camera and the image at the center of the image and obtains the optimal solution, and then uses the intersection (Uo, Vo) as a variable. The camera calibration system according to claim 3, which additionally obtains the optimal solution.   The camera calibration system according to claim 1, wherein all the calibration points are included in the same plane in the actual measurement space.   The camera calibration system according to claim 1, wherein the arithmetic device acquires the optimal solution based on a genetic algorithm method. A camera calibration program that causes a computer to find the optimal solution of the camera parameters of a fixed-point camera that captured the actual measurement space,
The measured space includes at least three calibration points;
The camera calibration program is
On the computer,
A procedure for measuring a digitized coordinate value of the calibration point on an image obtained by imaging the actual measurement space;
A three-dimensional coordinate value of the calibration point in the real space coordinate system, an arithmetic expression for calculating a projected coordinate value obtained by projecting the three-dimensional coordinate value of the calibration point onto the digitized coordinate system, a digitized coordinate value of the calibration point, and And a step for obtaining the optimum solution based on
The arithmetic expression includes a camera calibration program including the camera parameters.   The camera calibration program according to claim 7, wherein the computer calculates the optimal solution that minimizes a difference between the digitized coordinate value of the calibration point and the projected coordinate value of the calibration point. The digitized coordinate value of the intersection of the optical axis of the fixed point camera and the image is (Uo, Vo),
The three-dimensional coordinate value in the real space coordinate system at the center of the lens provided in the fixed point camera is (Xo, Yo, Zo),
The three-dimensional coordinate value of the measurement point in the real space coordinate system is (X, Y, Z),
Projection coordinate values of projection points obtained by projecting the measurement points onto the digitized coordinate system are (U, V),
The focal length of the lens is F,
R is a coordinate transformation matrix from the real space coordinate system to the digitized coordinate system,
The aspect ratio of the fixed point camera is a,
The camera distortion of the fixed point camera is s,
When the scale factor for setting the value of the third row and the first column included in the determinant indicating the projection point to 1 is h,
The camera calibration program according to claim 7 or 8, wherein the arithmetic expression includes the following expressions (1), (2), and (3).

When letting the computer find the optimal solution,
A procedure for obtaining the optimal solution by setting the intersection (Uo, Vo) of the optical axis of the fixed point camera and the image at the center of the image;
The camera calibration program according to claim 9, further comprising: a step of adding the intersection (Uo, Vo) to a variable to obtain the optimal solution.   The camera calibration program according to claim 7, wherein all the calibration points are included in the same plane in the actual measurement space.   The camera calibration program according to any one of claims 7 to 11, which causes the computer to obtain the optimal solution based on a genetic algorithm method. Using a fixed point camera to obtain an image of the measured space;
Measuring digitized coordinate values on the image of at least three calibration points contained in the measured space;
A three-dimensional coordinate value of the calibration point in the real space coordinate system, an arithmetic expression for calculating a projected coordinate value obtained by projecting the three-dimensional coordinate value of the calibration point onto the digitized coordinate system, a digitized coordinate value of the calibration point, and Obtaining an optimal solution of the camera parameters of the fixed-point camera based on
The camera calibration method, wherein the calculation formula includes the camera parameter.

Images