JP6317611B2 - Display display pattern generating apparatus and program thereof - Google Patents

Description

  The present invention relates to a display display pattern generation apparatus that generates a display display pattern for camera calibration and a program thereof.

  Camera calibration is to estimate camera parameters (external parameters, internal parameters) of a photographing camera from a photographed image (captured image). As one of the conventional methods, a technique for performing camera calibration by photographing a calibration pattern in which a known pattern figure is arranged has been proposed (for example, Non-Patent Document 1). Here, in the conventional method, a calibration pattern printed on a paper surface is used.

“A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses”, RY Tsai, IEEE J. Robotics & Automation, RA-3, No. 4, August 1987, pp. 323 -344

  However, in the conventional method described above, the calibration pattern cannot be changed because the calibration pattern is printed on the paper. For this reason, according to the conventional method, a calibration pattern suitable for camera calibration may not be photographed depending on the position, orientation, and photographing angle of view of the photographing camera. For example, in the conventional method, when the photographing camera is zoomed out or separated from the calibration pattern, the pattern figure may be crushed on the photographed image and cannot be detected. As a result, the conventional method cannot perform stable camera calibration and cannot improve the accuracy of camera calibration.

  It is an object of the present invention to provide a display display pattern generation apparatus and program for solving the above-described problems and enabling stable and highly accurate camera calibration.

  In view of the above problems, the display display pattern generation device according to the present invention generates a display image for camera calibration including a display display pattern in which pattern figures are arranged according to a predetermined rule, and generates the generated display image. A display display pattern generation device for displaying on a display, comprising a pattern detection unit, a camera calibration unit, a pattern figure position determination unit, and a display image generation unit.

  According to such a configuration, the display display pattern generation device performs pattern detection processing on the captured image obtained by capturing the display image displayed on the display with the capturing camera by the pattern detection unit, so that the pattern graphic is detected from the captured image. The pattern center position is detected. In addition, the display display pattern generation apparatus obtains camera parameters of the photographing camera when the photographed image is photographed by camera calibration by the camera calibration unit.

  Further, the display display pattern generation device calculates a center-to-center distance, which is a distance between adjacent pattern center positions in the captured image, by the pattern figure position determining unit, and a distance reference in which a minimum value of the calculated center-to-center distance is set in advance. The pattern center position of the captured image is corrected so as to be a value. Then, the display display pattern generation device displays, on the display, the pattern center position of the corrected captured image based on the coordinate system correspondence between the display image determined by the camera parameter and the captured image by the pattern figure position determining unit. The pattern center position of the new display image to be determined is determined.

Further, the display display pattern generation device calculates a center-to-center distance that is a distance between adjacent pattern center positions in the new display image by the display image generation unit, and creates a new display image based on the calculated center-to-center distance. Generate.
In this way, the display display pattern generation apparatus can appropriately change the display display pattern according to the position, orientation, and shooting angle of view of the shooting camera.

  In view of the above-described problems, the display display pattern generation device according to the present invention generates a display image for camera calibration that includes a display display pattern in which pattern figures are arranged according to a predetermined rule, and generates the generated display. A display display pattern generation device that displays an image on a display, comprising a pattern detection unit, a distance sum value calculation unit, a pattern figure size determination unit, and a display image generation unit.

According to such a configuration, the display display pattern generation device is configured such that the pattern detection unit
The pattern center position of the pattern figure is detected from the photographed image by performing pattern detection processing on the photographed image obtained by photographing the display image displayed on the display with the photographing camera. In addition, the display display pattern generation device calculates the distance sum value of the pattern center positions adjacent to each other in the captured image and the distance sum value of the pattern center positions adjacent to each other in the display image by the distance sum value calculation unit.

In addition, the display display pattern generation device uses a pattern figure size determination unit to set a size reference value that is set in advance based on the size of the pattern figure located in the center of the photographed image in the ratio of the total distance value between the display image and the photographed image. By multiplying by, the size of the pattern figure in the new display image displayed on the display is calculated. And a display display pattern production | generation apparatus produces | generates a new display image by the display image generation part based on the size of the pattern figure in a new display image.
In this way, the display display pattern generation apparatus can appropriately change the display display pattern according to the position, orientation, and shooting angle of view of the shooting camera.

According to the present invention, the following excellent effects can be obtained.
The display display pattern generation apparatus according to the present invention can appropriately change the display display pattern in accordance with the position, orientation, and shooting angle of view of the shooting camera. As a result, the display display pattern generation device can reduce the situation in which the figure of the pattern is crushed and cannot be detected in the captured image, and can perform highly accurate camera calibration stably.

1 is an overall configuration diagram of a camera calibration system according to a first embodiment of the present invention. In 1st Embodiment of this invention, it is explanatory drawing explaining a display display pattern. (A) And (b) is explanatory drawing showing the relationship between an imaging | photography camera, a display, and a display display pattern. (A) And (b) is explanatory drawing showing the relationship between an imaging | photography camera, a display, and a display display pattern. It is a block diagram which shows the structure of the display display pattern production | generation apparatus of FIG. In 1st Embodiment of this invention, it is explanatory drawing explaining the rotation detection of a display display pattern. In 1st Embodiment of this invention, it is explanatory drawing showing an example of a picked-up image. In 1st Embodiment of this invention, it is explanatory drawing showing the pattern center position of a picked-up image. In 1st Embodiment of this invention, it is explanatory drawing showing the pattern center position of a display image. It is a flowchart which shows operation | movement of the display display pattern production | generation apparatus of FIG. It is a block diagram which shows the structure of the display display pattern production | generation apparatus which concerns on 2nd Embodiment of this invention. In 2nd Embodiment of this invention, it is explanatory drawing showing the pattern center position of a picked-up image. In 2nd Embodiment of this invention, it is explanatory drawing showing the pattern center position of a display image. It is a flowchart which shows operation | movement of the display display pattern production | generation apparatus of FIG. In the modification of this invention, it is explanatory drawing explaining the 1st example of a display display pattern. In the modification of this invention, it is explanatory drawing explaining the 2nd example of a display display pattern.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In each embodiment, means having the same function are denoted by the same reference numerals and description thereof is omitted.

(First embodiment)
[Outline of camera calibration system]
The outline of the camera calibration system 1 according to the first embodiment will be described with reference to FIGS.

As shown in FIG. 1, the camera calibration system 1 performs camera calibration of the photographing camera 3, and includes a display display pattern generation device 2, a photographing camera 3, and a display 4.
In the camera calibration system 1, the photographing camera 3 and the display 4 are installed at a position where camera calibration can be performed, and the photographing camera 3 photographs the entire display 4.

  The display display pattern generation device 2 generates a display image 90 composed of the display display pattern pt and displays the generated display image 90 on the display 4. At this time, the display display pattern generation device 2 obtains camera parameters of the photographing camera 3 using the photographed image 91 and appropriately changes the display display pattern pt.

The display image 90 is a camera calibration image including the display display pattern pt.
The photographed image 91 is an image obtained by photographing the display image 90 displayed on the display 4 by the photographing camera 3. Thus, the captured image 91 is an image corresponding to the display image 90 being displayed on the display 4.

  The display image 90 scheduled to be displayed on the display 4 is referred to as a “new display image 90”. That is, the new display image 90 is an image that is one frame or more after the display image 90, and the display display pattern pt may be different from the display image 90.

<Display pattern>
Here, the display display pattern pt will be described.
As shown in FIG. 2, the display display pattern pt is a calibration pattern in which pattern figures are arranged according to a predetermined rule. In the present embodiment, the display display pattern pt is a square dot pattern in which three circular pattern figures are arranged at equal intervals in the vertical and horizontal directions. For example, in the display display pattern pt, the pattern figure is black and the background portion is white.

In Figure 2, the radius _r x of the pattern shapes, _{r y,} the distance between the centers _d x pattern shapes, _{d y,} the outer edge distance _l x pattern shapes, _{l y,} the frame inner display the display pattern pt length This is indicated by w _x and w _y . Radius _r x, _{r y} of the pattern figure represents the size of the pattern features. The center distances d _x and _dy are distances from the center position of a certain pattern figure to the center position of an adjacent pattern figure. The distances l _x and l _y between the outer edges are distances from the outer edge of a certain pattern graphic to the outer edge of the adjacent pattern graphic.

It marks the number of pattern shapes _n x, as _{n y.} In the example of FIG. 2, the number of pattern figures n _x and n _y is “3”. The subscripts x and y represent the horizontal axis (x axis) and the vertical axis (y axis) of the display image 90, respectively. In FIG. 2, the pattern figure located at the upper right is designated by the symbol α, the pattern figure located at the left of the pattern figure α is designated by the symbol β, and the pattern figure located below the pattern figure α is designated by the symbol γ.

In the display display pattern pt of FIG. 2, the pattern generation rules defined by the following formulas (1) to (6) are established. The pattern generation rule is a rule when generating a display representation pattern pt, center-to-center distance _d x, and _{d y,} the outer edge distance _l x, and _{l y,} the pattern shape radius _r x, and _{r y} Represents a relationship.

Equation (1), the length w _x of the frame inside the display the display pattern pt is a n _x number of pattern shapes of diameter sum 2r x _· n _x, the total length of the background part not drawn pattern feature It indicates that the sum is greater than or equal to the value l _x (n _x +1). That is, the expression (1) relates the number n _x of pattern figures, the radius r _{x of the} pattern figures, the distance l _x between the outer edges, and the length w _x inside the frame of the display display pattern pt ( The same applies to equation (2)).
Here, each of the number n _x, n _y of pattern features, to improve the accuracy of the camera calibration, equation (1), it is preferable that the maximum value among satisfying the equation (2).

Expression (3) indicates that the sum of the distance between the outer edges l _x and the diameter 2r _{x of the} pattern figure is equal to the center distance d _x . In other words, Expression (3) relates the distance l _x between the outer edges, the radius r _{x of the} pattern figure, and the center distance d _x (the same applies to Expression (4)).

Equation (5) indicates that the distance l _x between the outer edges is a _x times the radius r _x of the pattern figure. In other words, the expression (5) relates the distance l _x between the outer edges and the radius r _{x of the} pattern figure (the same applies to the expression (6)).

Since the display display pattern pt in FIG. 2 is a square, w _x = w _y , which are constants. Further, a _x and a _y in the equations (5) and (6) can be arbitrarily set as positive real numbers. According to the pattern generation rules of the above formulas (1) to (6), any of the center-to-center distances d _x and d _y , the outer edge distances l _x and l _y , or the pattern figure radii r _x and r _y If it is determined, the display display pattern pt is also determined.

In the photographed image 91 of FIG. 3A, the pattern figure is crushed because the interval (distance between the centers) of the pattern figures is short, and it is difficult to detect the pattern figure. Also, the captured image 91 in FIG. 3 (b), as compared with the captured image 91 in FIG. 3 (a), reducing the number n _x, n _y of the pattern shapes, that have long intervals between pattern graphic pattern graphic Can be detected. However, in the photographed image 91 of FIG. 3B, since the photographing camera 3 and the display 4 are separated from each other, the display display pattern pt is small in the photographed image 91, and the accuracy of the camera parameter is lowered.

  The captured image 91 in FIG. 4A has a display display pattern pt larger than the captured image 91 in FIG. 3B because the photographing camera 3 and the display 4 are brought close to each other. However, since the captured image 91 in FIG. 4A has a small number of pattern figures, the accuracy of the camera parameters does not improve so much even if camera calibration is performed. Since the number of pattern figures in the captured image 91 in FIG. 4B is increased, camera parameters with higher accuracy can be obtained compared to the captured image 91 in FIG.

  That is, in order to enable camera calibration, the number of pattern figures only needs to be a certain number or more. Furthermore, in order to improve the accuracy of camera calibration, it is only necessary to increase the number of pattern figures within a range where the pattern figures can be detected accurately.

Returning to FIG. 1, the outline of the camera calibration system 1 will be described.
The photographing camera 3 is a general digital video camera including a CCD (Charge Coupled Device) and a CMOS (Complementary MOS). The photographing camera 3 photographs the display 4 on which the display display pattern pt is displayed, and outputs a photographed image 91 to the display display pattern generation device 2.

  The display 4 is a general display device such as a CRT (Cathode Ray Tube) display or a liquid crystal display. The display 4 displays the display image 90 input from the display display pattern generation device 2.

  Hereinafter, an outline of processing in the camera calibration system 1 will be described. First, the display display pattern generation device 2 displays the display image 90 of the initially set display display pattern pt on the display 4. Next, the photographing camera 3 photographs the display 4 on which the display display pattern pt is displayed, and outputs a photographed image 91 to the display display pattern generation device 2. Subsequently, the display display pattern generation device 2 performs camera calibration of the photographing camera 3 using the photographed image 91.

At this time, the display display pattern generation device 2 generates a new display image 90 by appropriately changing the display display pattern pt so that camera parameters with higher accuracy can be obtained. Then, the display display pattern generation device 2 displays a new display image 90 including an appropriate display display pattern pt on the display 4.
Thereafter, the camera calibration system 1 repeats the above-described procedure.

[Configuration of Display Display Pattern Generation Device]
With reference to FIG. 5, the structure of the display display pattern generation device 2 will be described.
As shown in FIG. 5, the display display pattern generation device 2 includes a storage unit 21, a pattern detection unit 22, a camera calibration unit 23, and a pattern generation unit 24.

The storage unit 21 is a storage device such as a memory or a hard disk that stores display information in advance.
The display information is information related to the display 4 such as the screen size of the display 4, the number of pixels, the pixel interval, and the like.

  The pattern detection unit 22 detects the image coordinates of the center position of the pattern figure from the photographed image 91 by inputting the photographed image 91 from the photographing camera 3 and applying pattern detection processing to the inputted photographed image 91. It is. At this time, the pattern detection unit 22 uses the pattern information input from the pattern generation unit 24 (display image generation unit 26) and the display information in the storage unit 21.

The pattern information, the number _n x of pattern features in the display image 90, _{n y,} radius _r x of the pattern shapes, _{r y,} the distance between the centers _d x, _{d y,} the outer edge distance _l x, _{l y,} display the display pattern pt frame inner length _w x, which is information on the display the display pattern pt such _{w y} like. Further, the pattern information includes a center position of a rotation detection reference pattern figure ε described later.

Specifically, the pattern detection unit 22 performs a process for detecting an area (for example, a black area) having the same color as the pattern graphic included in the captured image 91 as the pattern detection process. Since the pattern figure is composed of a plurality of pixels, one pixel at the center is set as an image coordinate. It is preferable that the pattern detection unit 22 detect the center of gravity of the pattern graphic area as the pattern center position because an accurate pattern center position can be quickly obtained.
In addition, the pattern detection unit 22 may obtain a quadrilateral circumscribing the detected pattern figure area, and obtain the diagonal intersection of the obtained quadrilateral as the pattern center coordinates.

  Here, since the display display pattern generation device 2 has pattern information and display information, the world coordinates of the center position of the pattern figure are known. For example, consider a world coordinate system in which the center position of the reference pattern figure located in the center in the display display pattern pt is the origin (0, 0, 0). In the display information, it is assumed that the pixel interval is set on a real-world scale (unit system).

  In this case, the center distance between the pattern figures can be calculated by multiplying the distance (number of pixels) d between the pattern figures and the pixel interval Δ of the display 4. Further, since the pixel interval of the display information is defined on a real world scale, the distance between the centers of the pattern figures can be converted into the world coordinate system (0, dΔ, 0) with the center position of the reference pattern figure as the origin. .

  The pattern detection unit 22 generates, as coordinate information, image coordinates representing the center position of the pattern graphic in the captured image 91 and world coordinates representing the center position of the pattern graphic in the display image 90. Further, the pattern detection unit 22 outputs the generated coordinate information to the camera calibration unit 23 and the pattern generation unit 24.

<Display pattern rotation detection>
Furthermore, it is preferable that the pattern detection unit 22 detects the rotation of the display display pattern pt based on the position change of the rotation detection reference pattern graphic ε included in the captured image 91 as shown in FIG.

The rotation detection reference pattern graphic ε is a pattern graphic (for example, a lower right pattern graphic) at a position that is asymmetric in the display display pattern pt. Moreover, the pattern figure of the position which becomes asymmetric is other than the pattern figure whose position does not change when the display display pattern pt is rotated. Accordingly, the pattern figure located at the center of the display display pattern pt is not set as the rotation detection reference pattern figure ε because the position does not change when it is rotated.
The rotation detection reference pattern graphic ε is set to the display display pattern pt by the display image generation unit 26 described later.

An example of a method for setting the rotation detection reference pattern graphic ε is a method of removing the rotation detection reference pattern graphic ε from the display display pattern pt of the display image 90. In other words, the rotation detection reference pattern graphic ε is not drawn in the display display pattern pt of the display image 90. From the display display pattern pt in which three pattern figures are arranged vertically and horizontally, the pattern figure located at the lower right is extracted as the rotation detection reference pattern figure ε.
In FIG. 6, the rotation detection reference pattern graphic ε is illustrated by a broken line for the sake of explanation.

  As described above, the pattern detection unit 22 knows the center position of the rotation detection reference pattern graphic ε set in the display display pattern pt of the display image 90 from the pattern information. Further, the pattern detection unit 22 can specify the center position of the pattern graphic missing from the display display pattern pt of the captured image 91 as the rotation detection reference pattern graphic ε from the detection result of the pattern center position.

  The pattern detection unit 22 detects a change in the center position of the rotation detection reference pattern graphic ε in the display image 90 and the captured image 91. For example, when the rotation detection reference pattern figure ε located at the lower right in the display image 90 moves to the lower left in the captured image 91, the pattern detection unit 22 displays the display display pattern pt 90 ° clockwise or the photographing camera 3 It is detected that it has rotated 90 ° counterclockwise.

  As described above, the pattern detection unit 22 detects the rotation of the display display pattern pt, thereby associating the vertical axis and the horizontal axis of the display image 90 and the captured image 91 with each other so that the optimal display display pattern pt in the biaxial direction is obtained. Can be requested.

  Note that the rotation detection reference pattern graphic ε may be drawn with a rotation detection color (for example, red) different from other pattern graphics, other than removing from the display display pattern pt. In this case, the pattern detection unit 22 performs processing for detecting a region having the same color as the rotation detection reference pattern graphic ε on the captured image 91. Then, the pattern detection unit 22 detects the center of gravity in the region of the rotation detection reference pattern graphic ε as the center position of the rotation detection reference pattern graphic ε.

Returning to FIG. 5, the configuration of the display display pattern generation device 2 will be described.
The camera calibration unit 23 receives the coordinate information from the pattern detection unit 22 and obtains camera parameters of the photographing camera 3 when the photographed image 91 is photographed by camera calibration using the inputted coordinate information. .

For example, the camera calibration unit 23 can obtain the camera parameters of the photographing camera 3 using the Tsai method described in Reference Document 1 below.
Reference 1: “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses”, RY Tsai, IEEE J. Robotics & Automation, RA-3, No. 4, August 1987 , pp. 323-344

Then, the camera calibration unit 23 outputs the obtained camera parameters to the pattern generation unit 24.
The camera calibration unit 23 may output the camera parameters to the outside in order to use the obtained camera parameters in another device or application.

  The pattern generation unit 24 generates a captured image 91 including the display display pattern pt, and includes a pattern figure position determination unit 25 and a display image generation unit 26. Here, the pattern generation unit 24 receives coordinate information from the pattern detection unit 22 and receives camera parameters from the camera calibration unit 23.

  The pattern figure position determining unit 25 calculates a center-to-center distance that is the distance between adjacent pattern center positions in the captured image 91, and captures the minimum value of the calculated center-to-center distance to a preset distance reference value. The pattern center position of the image 91 is corrected.

<Determination of pattern center position in photographed image>
The determination of the pattern center position will be described with reference to FIGS. 7 and 8 (see FIG. 5 as appropriate).
The pattern figure position determination unit 25 refers to the coordinate information input from the pattern detection unit 22 and calculates the center-to-center distances between adjacent pattern figures for all pattern figures included in the captured image 91.

  Here, it is assumed that the photographing camera 3 has photographed from the lower left side of the display 4. In this case, as shown in FIG. 7, in the photographed image 91, the lower left side is closest to the photographing camera 3 and the upper right side is farthest from the photographing camera 3. For this reason, the interval (center-to-center distance) between adjacent pattern figures becomes shorter from the lower left side to the upper right side of the captured image 91. Hereinafter, the description will be given focusing on the pattern figures α to γ in FIG. 7 as shown in FIG.

For example, the pattern figure position determining unit 25 obtains the center distance between the center position (u _α , v _α ) ^T of the pattern figure α and the center position (u _β , v _β ) ^T of the pattern figure β. The pattern figure position determining unit 25 obtains the center distance between the center position (u _α , v _α ) ^T of the pattern figure α and the center position (u _γ , v _γ ) ^T of the pattern figure γ.

The subscripts u and v represent the horizontal axis (u axis) and the vertical axis (v axis) in the captured image 91, respectively. That is, the u axis of the captured image 91 corresponds to the x axis of the display image 90, and the v axis of the captured image 91 corresponds to the y axis of the display image 90.
A superscript T represents a transposed matrix.
Further, in FIG. 8, for the sake of explanation, the pattern figure is shown in white and the center position of the pattern figure is shown in black (the same applies to FIGS. 9, 12, and 13).

Next, the pattern figure position determining unit 25 obtains the minimum center distance among the center distances in all the pattern figures. In the example of FIG. 8, the pattern shape alpha in u axis direction, the distance between the centers d _u of β is minimized, v-axis direction pattern shapes alpha, center-to-center distance d _v of γ is minimized. Here, the minimum center-to-center distances d _u and d _v can be expressed by the following equations (7) and (8).

Then, the pattern figure position determination unit 25, between the minimum center distance d _u, d _v distance preset reference value D _u, so that the D _v, to correct the pattern center position of the captured image 91. The pattern center position after correction with the pattern figure β is (u ′ _β , v ′ _β ) ^T , and the pattern center position after correction with the pattern figure γ is (u ′ _γ , v ′ _γ ) ^T. In this case, between the distance reference values D _u and D _v and the corrected pattern center position (u ′ _β , v ′ _β ) ^T , (u ′ _γ , v ′ _γ ) ^T , the following formula ( 9) and the relationship of Expression (10) are established.

Here, the corrected pattern center position (u ′ _β , v ′ _β ) ^T is on a straight line connecting the center positions of the pattern figures α and β. The corrected pattern center position (u ′ _γ , v ′ _γ ) ^T is on a straight line connecting the center positions of the pattern figures α and γ. For this reason, the relationship between the u coordinate and the v coordinate is obtained by a function such as similarity. As a result, since the equations (9) and (10) become one variable for one equation, the pattern figure position determining unit 25 determines the corrected pattern center position (u ′ _β , v ′ _β ) ^T. , (U ′ _γ , v ′ _γ ) ^T can be obtained.

<Determination of pattern center position in new display image>
The determination of the pattern center position in the new display image 90 will be described with reference to FIGS. 8 and 9 (see FIG. 5 as appropriate).
The pattern figure position determination unit 25 performs a new display to be displayed on the display 4 from the pattern center position of the corrected captured image 91 based on the coordinate system correspondence between the display image 90 and the captured image 91 determined by the camera parameters. The pattern center position of the image 90 is determined.

  The camera parameters input to the pattern figure position determining unit 25 are camera parameters one frame or more before the new display image 90 scheduled to be displayed on the display 4. Using this camera parameter, the coordinate system (world coordinate system) of the display image 90 and the image coordinate system of the captured image 91 can be related as in the following equation (11).

In Expression (11), the image coordinates of the photographed image 91 are (u, v) ^T , the world coordinates of the display image 90 are (x _w , y _w , z _w ) ^T , and the coordinate system of the photographing camera 3 (camera coordinate system). Depth value at is represented by z. In Expression (11), an internal parameter included in the camera parameters is represented by A, a rotation matrix is represented by R, and a translation vector is represented by T.
In the equation (11), if P = A (R | T), the following equation (12) is established. This equation (12) can be transformed into equation (13).

The pattern figure position determining unit 25 then corrects the pattern center positions (u ′ _β , v ′ _β ) ^T , (u ′ _γ , v ′ _γ ) after correction as in the following formulas (14) and (15). ) Substituting ^T into Equation (13), the pattern center position of the new display image 90 is obtained in the world coordinate system.

As shown in FIG. 9, in the world coordinate system, the center position of the pattern figure α is (x _α , y _α , z _α ) ^T , the center position of the pattern figure β is (x _β , y _β , z _β ) ^T , and the pattern The center position of the figure γ is represented by (x _γ , y _γ , z _γ ) ^T. At this time, the center position (x _β , y _β , z _β ) ^T = (x _α + d ′ _x , y _α , 0) ^{T of} the pattern figure β is obtained from the equation (14). Further, from the expression (15), the center position (x _γ , y _γ , z _γ ) ^T = (x _α , y _α + d ′ _y , 0) ^{T of} the pattern graphic γ is obtained.

Returning to FIG. 5, the configuration of the display display pattern generation device 2 will be described.
The display image generation unit 26 obtains the pattern center position (x _α + d ′ _x , y _α , 0) ^T , (x _α , y _α + d ′ _y ) of the new display image 90 obtained by the pattern figure position determination unit 25. from ^{0) T,} calculate the center distance _d x, _{d y} for a new display image 90, the calculated center-to-center distance _d x, based on the _{d y,} and generates a new display image 90.

Equation (14) and _{d'x, d'y} is the distance between the centers _d x of the new display image 90 of the formula (15), and _{d y.} Further, the pattern generation rule of the equations (1) to (6), center-to-center distance _d x, if _{d y} is determined, determined the display the display pattern pt of the new display image 90. In this case, the number _n x, _{n y} of the pattern feature, an integer.

That is, the display image generation unit 26 calculates the new display image 90 from the center-to-center distances d _x and _dy of the new display image 90 based on the pattern generation rules of Expressions (1) to (6) described above. The distances l _x and l _y between the outer edges and the radii r _x and r _y of the pattern figure are calculated. Then, the display image generating unit 26, calculated center distance _d x, and _{d y,} the outer edge distance _l x, and _{l y,} the radius _r x of the pattern feature, the display displays the pattern figures are arranged according to a _{r y} The pattern pt is obtained. Further, the display image generation unit 26 generates a new display image 90 composed of the obtained display display pattern pt.

Note that the display image generation unit 26 preferably sets the rotation detection reference pattern graphic ε to the display display pattern pt of the new display image 90 (FIG. 6).
Further, the display image generation unit 26 outputs the pattern information of the display display pattern pt included in the new display image 90 to the pattern detection unit 22.

[Operation of display display pattern generator]
The operation of the display display pattern generation device 2 will be described with reference to FIG. 10 (see FIG. 5 as appropriate).

  In the display display pattern generation device 2, the pattern detection unit 22 detects the image coordinates of the center position of the pattern graphic from the captured image 91. Moreover, the display display pattern generation apparatus 2 detects the rotation of the display display pattern pt by the pattern detection unit 22 (step S1).

  The display display pattern generation device 2 uses the camera calibration unit 23 to obtain the camera parameters of the photographing camera 3 by camera calibration (step S2).

  The display display pattern generation device 2 calculates the center-to-center distance of the captured image 91 by the pattern figure position determining unit 25, and sets the pattern center of the captured image 91 so that the minimum value of the calculated center-to-center distance becomes the distance reference value. Correct the position. Moreover, the display display pattern generation apparatus 2 determines the pattern center position of the new display image 90 from the pattern center position of the captured image 91 by the pattern figure position determination unit 25 (step S3).

In the display display pattern generation device 2, the display image generation unit 26 calculates the center-to-center distance from the pattern center position of the new display image 90 (step S4).
In the display display pattern generation device 2, the display image generation unit 26 calculates the distance between the outer edges and the radius of the pattern graphic based on the pattern generation rule, and generates a new display image 90 (step S5).

[Action / Effect]
The display display pattern generation device 2 can appropriately change the display display pattern pt according to the position, orientation, and shooting angle of view of the shooting camera 3. Thereby, the display display pattern generation device 2 can reduce the situation in which the pattern figure is crushed and cannot be detected in the captured image 91, and can perform highly accurate camera calibration stably.

  Furthermore, since the display display pattern generation device 2 generates the display image 90 using the camera parameters, the estimation accuracy of the position, orientation, and shooting angle of view of the shooting camera 3 is improved, and the accuracy of camera calibration is further improved. be able to.

(Second Embodiment)
With reference to FIG. 11, a display display pattern generation device 2B according to the second embodiment of the present invention will be described while referring to differences from the first embodiment.
The display display pattern generation device 2 </ b> B differs from the first embodiment in that when generating a new display image 90, approximate calculation based on the ratio between the captured image 91 and the display image 90 is performed instead of the camera parameter.

As shown in FIG. 11, the display display pattern generation device 2B includes a storage unit 21, a pattern detection unit 22, a camera calibration unit 23, and a pattern generation unit 24B.
The pattern generation unit 24B includes a display image generation unit 26B, a distance total value calculation unit 27, and a pattern figure size determination unit 28.
The means other than the display image generation unit 26B, the distance total value calculation unit 27, and the pattern figure size determination unit 28 are the same as those in the first embodiment, and thus description thereof is omitted.

  The distance sum value calculation unit 27 uses the coordinate information input from the pattern detection unit 22, and the distance sum value of the pattern center positions adjacent in the captured image 91 and the distance sum value of the pattern center positions adjacent in the display image 90. Are calculated.

First, the distance sum calculation unit 27, for all the pattern features contained in the displayed image 90, the distance between the centers d _x between adjacent pattern _features, to calculate a d _y. Then, the distance sum calculation unit 27, as in the following equation (16) and (17), center-to-center distance _d x, the sum of _{d y} distance sum _s x, calculates the _{s y.} At this time, the distance sum value calculation unit 27 calculates the distance sum values s _x and s _y separately for the x axis and the y axis.

The distance sum calculation unit 27, for all the pattern features contained in the captured image 91, and calculates the center-to-center distance d _u, d _v between adjacent pattern features. Then, the distance sum calculation unit 27, as in the following formula (18) and (19), center-to-center distance _d u, _{d v} sum is the distance sum _s u of calculating the _{s v.} At this time, the distance sum value calculation unit 27 calculates the distance sum values s _u and s _v separately for the u axis and the v axis.

The pattern figure size determining unit 28 uses the pattern figure radius r _x in the new display image 90 displayed on the display 4 from the distance sum values s _x , s _y , _su , and s _v calculated by the distance sum value calculating unit 27. , R _y are calculated.

As shown in FIGS. 12 and 13, the ratio of the radii r _u and r _x of the pattern figure η located in the center of the display display pattern pt in the captured image 91 and the display image 90 and the center between the captured image 91 and the display image 90. The ratio of the distance sum values s _u and s _x that is the sum of the distances d _u and d _x becomes equal. Further, the radius _r v pattern shape η located in the center of the display the display pattern pt in the captured image 91 and the display image 90, and the ratio of _{r y,} the distance between the centers _d v of the captured image 91 and the display image 90, the _{d y} The ratio of the distance sum values s _v and s _y that is the sum is equal.
When the pattern figure η does not exist at the center of the display display pattern pt (when the number of pattern figures is an even number), the pattern figure closest to the center is used.

  That is, the following equations (20) and (21) are established. And Formula (20) and Formula (21) can be transformed like Formula (22) and Formula (23).

In the equations (22) and (23), if the radii r _u and r _v of the pattern figure η in the display image 90 are set to the preset radius reference values (size reference values) R _u and R _v , The radii r _x and r _y of the pattern figure in the display image 90 are obtained.

That is, the pattern figure size determination unit 28, as in Equation (24) below, multiplied by the radius reference value R _u of the ratio of the distance sum s _u of the distance sum s _x and the photographed image 91 of the display image 90 Thus, the radius r _x of the pattern figure in the new display image 90 can be calculated. Further, the pattern figure size determining unit 28 multiplies the ratio of the total distance value s _y of the display image 90 and the total distance value s _v of the captured image 91 by the radius reference value R _v as shown in the following equation (25). Thus, the radius r _y of the pattern figure in the new display image 90 can be calculated.

Display image generating unit 26B, the radius r _x of the pattern feature in the new display image _90, based on the r _y, and generates a new display image 90. Specifically, the display image generation unit 26B based on the pattern generation rule of the equations (1) to (6), the radius _r x of the new display image 90, the _{r y,} the new display image 90 center distance _d x, _{d y} and the outer edge distance _l x of calculating the _{l y.} Then, the display image generating unit 26B outputs the calculated center distance _d x, and _{d y,} the outer edge distance _l x, and _{l y,} the radius _r x of the pattern feature, the display displays the pattern figures are arranged according to a _{r y} The pattern pt is obtained. Furthermore, the display image generation unit 26B generates a new display image 90 including the obtained display display pattern pt.

[Operation of display display pattern generator]
The operation of the display display pattern generation device 2B will be described with reference to FIG. 14 (see FIG. 11 as appropriate).
Since the process of step S1, S2 is the same as that of FIG. 10, description is abbreviate | omitted.

  The display display pattern generation device 2B calculates the distance between the centers of adjacent pattern figures in the display image by the distance total value calculation unit 27, and obtains the distance total value of the calculated center distance (step S11).

  The display display pattern generation device 2B uses the pattern figure size determination unit 28 to multiply the ratio of the total distance value between the display image 90 and the captured image 91 by the radius reference value, thereby obtaining the radius of the pattern figure in the new display image 90. Calculate (step S12).

  In the display display pattern generation device 2B, the display image generation unit 26B calculates the distance between the centers and the distance between the outer edges based on the pattern generation rule, and generates a new display image 90 (step S13).

[Action / Effect]
As in the first embodiment, the display display pattern generation device 2B can stably perform highly accurate camera calibration. Furthermore, the display display pattern generation device 2B can improve the execution speed of camera calibration by a simple calculation process.

(Modification)
The display display pattern generation devices 2 and 2B according to the present invention are not limited to the above-described embodiments, and modifications can be made without departing from the spirit of the present invention.

  In the above-described embodiment, the display display pattern generation devices 2 and 2B detect the rotation of the display display pattern pt. However, it is not necessary to detect the rotation of the display display pattern pt. In this case, since the display display pattern generation devices 2 and 2B cannot associate the vertical axis and the horizontal axis of the display image 90 and the captured image 91, the radius of the pattern figure, the center distance, and the outer edge distance in one axis direction. Is calculated.

In the above-described embodiment, the display display pattern pt is a dot pattern, but is not limited thereto.
As shown in FIG. 15, the display display pattern pt may be a square square pattern in which three square pattern figures are arranged at equal intervals in the vertical and horizontal directions. We define the length from the end point in the pattern feature to the midpoint r _x, and r _y. Thereby, in the display display pattern pt of FIG. 15, the pattern generation rule of Formula (1)-Formula (6) can be utilized as it is.
Further, as shown in FIG. 16, the display display pattern pt may be a checkered pattern.

  In the above-described embodiment, the display display pattern generation devices 2 and 2B have been described as independent hardware, but the present invention is not limited to this. For example, the display display pattern generation apparatuses 2 and 2B can also be realized by a display display pattern generation program that causes hardware resources such as a CPU, a memory, and a hard disk included in a computer to operate in cooperation as the above-described units. This program may be distributed through a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

DESCRIPTION OF SYMBOLS 1 Camera calibration system 2, 2B Display display pattern production | generation apparatus 3 Shooting camera 4 Display 21 Memory | storage part 22 Pattern detection part 23 Camera calibration part 24, 24B Pattern generation part 25 Pattern figure position determination part 26, 26B Display image generation part 27 Distance total value calculation unit 28 Pattern figure size determination unit

Claims (7)

A display display pattern generation device that generates a display image for camera calibration including a display display pattern in which pattern figures are arranged according to a predetermined rule, and displays the generated display image on a display.
A pattern detection unit that detects a pattern center position of the pattern figure from the photographed image by performing pattern detection processing on the photographed image obtained by photographing the display image displayed on the display with a photographing camera;
A camera calibration unit that obtains camera parameters of the photographing camera when the photographed image is photographed by camera calibration;
A center-to-center distance that is a distance between adjacent pattern center positions in the captured image is calculated, and the pattern center position of the captured image is corrected so that the calculated minimum value of the center-to-center distance becomes a preset distance reference value. Then, based on the correspondence relationship of the coordinate system between the display image and the captured image determined by the camera parameter, the pattern center position of a new display image to be displayed on the display from the corrected pattern center position of the captured image A pattern figure position determining unit for determining
A display image generation unit that calculates a center-to-center distance that is a distance between adjacent pattern center positions in the new display image, and generates the new display image based on the calculated center-to-center distance;
A display display pattern generation device comprising: The display image generation unit
A pattern generation rule representing a relationship between the distance between the centers, the distance between the outer edges that is the distance between the outer edges of the adjacent pattern figures, and the size of the pattern figure is set in advance,
Based on the pattern generation rule, the distance between the outer edges of the new display image and the size of the pattern figure are calculated from the center-to-center distance of the new display image.
The display according to claim 1, wherein the new display image is generated by arranging the pattern graphic in accordance with the calculated distance between the centers, the distance between the outer edges, and the size of the pattern graphic. Display pattern generation device. A display display pattern generation device that generates a display image for camera calibration including a display display pattern in which pattern figures are arranged according to a predetermined rule, and displays the generated display image on a display.
A pattern detection unit that detects a pattern center position of the pattern figure from the photographed image by performing pattern detection processing on the photographed image obtained by photographing the display image displayed on the display with a photographing camera;
A distance sum value calculating unit for calculating a distance sum value of adjacent pattern center positions in the captured image and a distance sum value of adjacent pattern center positions in the display image;
A new display to be displayed on the display is obtained by multiplying the ratio of the total distance value between the display image and the captured image by a size reference value set in advance based on the size of the pattern figure located in the center of the captured image. A pattern figure size determining unit for calculating the size of the pattern figure in the image;
A display image generation unit that generates the new display image based on the size of the pattern graphic in the new display image;
A display display pattern generation device comprising: The display image generation unit
A pattern generation rule that represents the relationship between the distance between the centers of adjacent pattern center positions in the captured image, the distance between the outer edges that is the distance between the outer edges of the adjacent pattern figures, and the size of the pattern figure is set in advance. And
Based on the pattern generation rules, from the size of the pattern figure in the new display image, calculate the center distance and the outer edge distance of the new display image,
The display according to claim 3, wherein the new display image is generated by arranging the pattern graphic according to the calculated distance between the centers, the distance between the outer edges, and the size of the pattern graphic. Display pattern generation device.   The display pattern generation apparatus according to claim 1, wherein the pattern detection unit detects a center of gravity of the pattern graphic as the pattern center position. The display image generation unit presets a pattern figure at a position that is asymmetric in the display display pattern of the display image as a rotation detection reference pattern figure,
The said pattern detection part detects the rotation of the said display display pattern based on the positional change of the rotation detection reference pattern figure contained in the said picked-up image, The any one of Claims 1-5 characterized by the above-mentioned. The display display pattern production | generation apparatus of description.   A display display pattern generation program for causing a computer to function as the display display pattern generation apparatus according to any one of claims 1 to 6.

Images