JP5707185B2 - All-around field for multi-camera calibration - Google Patents

Description

  The present invention relates to a multi-camera calibration all-around field for accurately measuring an internal parameter and an external parameter, which is a subject when a multi-camera is calibrated.

  Conventionally, in order to obtain an image with little aberration, the internal and external parameters of the camera are analytically determined by photographing and measuring many points that are precisely measured and arranged in three dimensions from multiple directions. Yes.

  As a calibration subject of a conventional camera, there is a calibration three-dimensional field that can be used in a relatively narrow space even in the case of a zoom lens having a variable focal length (see, for example, Patent Document 1). The three-dimensional field for calibration does not change the relative positional relationship between the wide-angle reference mark and the telephoto reference mark on a wall or panel such as a concrete wall in an environment with little temperature change such as a basement. (Paragraph 0022, FIGS. 1 to 4). The calibration three-dimensional field is photographed from a plurality of directions by a single calibration chart photographing camera to be calibrated, and a calibration chart image is obtained.

JP 2004-347544 A

However, when calibrating a multi-camera (see FIG. 7) in which a plurality of cameras are arranged outward and a multi-directional image can be acquired, if a calibration subject such as a calibration three-dimensional field is used, The calibration subject must be moved in each direction to shoot.
Therefore, the present invention is an object used when calibrating a multi-camera, and a calibration object (all-round) that enables highly accurate multi-camera calibration without moving the object in each camera direction. Field).

The all-around field according to the first aspect of the present invention has, for example, a reference camera 11 having an optical axis 11 ′ in the reference direction and an optical axis 12 ′ in a direction different from the reference direction, as shown in FIG. An all-round field 1 for photographing with a multi-camera 10a having another camera 12 and calibrating the multi-camera 10a, which is photographed for measuring an internal parameter or an external parameter of the multi-camera 10a. A plurality of targets with known three-dimensional coordinates arranged in the circumferential field 1; in a predetermined angle range from the reference direction, the targets are arranged at a first density and at a predetermined angle from the reference direction; Outside the range, the targets are arranged at a second density higher than the first density, and the normal of the perimeter field 1 is less Also it has two directions.
The multi-camera refers to a camera system in which a plurality of cameras having different optical axis directions are arranged outwardly and images of fields of view in a plurality of directions can be acquired in synchronization. Therefore, the multi-camera includes not only a camera that can shoot 360 degrees in a horizontal direction with a plurality of cameras, but also a camera that can shoot 360 degrees in any direction with a plurality of cameras. Even if it is not, the thing which can image | photograph the considerable wide range exceeding 180 degree | times with a some camera (camera which image | photographs a very wide range) is also included. The “entire circumference” of the “all circumference field” is not limited to 360 degrees in the horizontal direction , and it is sufficient that a space covering each camera can be formed so that each camera of the multi-camera can photograph the target. That is, any field can be used as long as the target to be imaged can be arranged so as to exist in the imaging region of each camera. What constitutes the all-around field is not limited to the above wall surface, and a panel or the like may be used. The known “three-dimensional coordinates” refers to known three-dimensional coordinates in the world coordinate system as absolute reference points. “World coordinate system” refers to a coordinate system unified for all cameras of a multi-camera.

  With this configuration, high-accuracy calibration is possible even in an image peripheral portion that has a larger lens distortion than the image central portion. In general, a multi-camera uses a wide-angle lens for each camera, and acquires an image with a wide field of view. For this reason, the obtained image is greatly distorted by the lens. In the present invention, the accuracy of calibration in a place with a large distortion can be increased by arranging the targets with a higher density outside the range of the predetermined angle range from the reference direction. Furthermore, in a predetermined angle range from the reference direction, the calculation amount at the time of calibration can be reduced by setting the density lower. Furthermore, it is possible to photograph the target by synchronizing the cameras included in the multi-camera without moving the entire circumference field.

The all-round field according to the second aspect of the present invention is the all-round field 1 according to the first aspect of the present invention, as shown in FIG. The target having the first size and arranged at the second density has a second size larger than the first size, and the first target is set so that the sizes of the captured targets are uniform. The target of the size and the target of the second size are configured.
Note that “the target size is the same” is not limited to the case where the target is completely the same size, and includes cases where the target is slightly different as long as the target can be shot with a size that can guarantee the shooting accuracy. .

  With this configuration, the size of the target is smaller in the central portion than in the peripheral portion of the image, but it can be set to a size that can ensure photographing accuracy.

The all-around field according to the third aspect of the present invention is the same as the reference camera 11 and other cameras in the all-around field 1 according to the first or second aspect of the present invention as shown in FIG. 12 are arranged so as to be photographed with an overlap region 14 provided therebetween, and the targets arranged at the second density are arranged so that the target is photographed in the overlap region 14.
In addition, if the entire circumference field according to the third aspect of the present invention is within a range in which the accuracy of desired calibration can be ensured, the target is more densely arranged than the overlap region, and The case where it is densely arranged in a narrower range than the overlap region is also included. That is, the region where the targets are densely arranged is not limited to the case where the target overlaps completely with the overlap region, and includes a case where there is a slight shift between the region where the targets are arranged and the overlap region.

  When configured in this manner, a target arranged at a higher density is captured in the overlap area, so that the accuracy of calibration in a portion where the image distortion (lens distortion) is large becomes high, and the image is captured by a multi-camera. It is possible to improve the accuracy of connection between images. Therefore, in creating a panoramic image, an image with less sense of discomfort at the connecting portion can be obtained. That is, more accurate calibration between cameras for acquiring a panoramic image is possible. The “panoramic image” includes a panoramic moving image.

  The all-round field according to the fourth aspect of the present invention is a predetermined angle from the optical axis direction of the other camera 12 as shown in FIG. 1, for example, in the all-round field 1 according to the third aspect of the present invention. The range is a region other than the overlap region 14, and the target is arranged at a third density in a predetermined angle range from the optical axis direction of the other camera 12, and the second density is the third density. Higher than the density.

  With this configuration, the target is arranged more roughly in a predetermined angle range from the optical axis direction of the reference camera and the other cameras, so that the amount of calculation at the time of calibration of each camera can be reduced. On the other hand, since the targets are arranged more densely outside the range of a predetermined angle range from the optical axis direction of each camera (overlap region), the calibration accuracy of each camera can be improved.

  The all-around field according to the fifth aspect of the present invention is predetermined from the reference direction of the reference camera 11 in the all-around field 1 according to any one of the first to fourth aspects of the present invention. The angle range is a range that is equal to or less than a predetermined distortion amount of the lens of the camera.

  If comprised in this way, based on the predetermined distortion amount of a lens, a predetermined angle range and the outside of the angle range can be determined. Therefore, the calculation amount at the time of calibration can be reduced more efficiently in a range where the amount of distortion is smaller, and the calibration accuracy can be increased in a range where the amount of distortion is larger.

  The all-around field according to the sixth aspect of the present invention is, for example, as shown in FIG. 3 (a) in the all-around field 1 according to any one of the first to fifth aspects of the present invention. In addition, the entire circumference field is configured to surround the periphery of the multi-camera.

  If comprised in this way, the calibration of the multicamera which can image | photograph the visual field of 360 degrees of horizontal directions will be attained.

The all-round field according to the seventh aspect of the present invention is a reference camera in the all-round field according to any one of the first to fifth aspects of the present invention as shown in FIG. Is an all-around field in which a plurality of other cameras having an optical axis in a direction different from the reference direction are photographed by the multi-cameras 10b, 10c, and 10d arranged around the reference camera. The all-around field includes a side surface surrounding the multi-cameras 10b, 10c, and 10d and an upper surface that covers the multi-cameras 10b, 10c, and 10d.
The “zenith direction” is not limited to the zenith, but also includes the nadir direction, the horizontal direction, and any direction, and includes the case where the reference camera is installed in these directions.

  If comprised in this way, even if it is a multicamera with which each camera was arrange | positioned in the perimeter and the zenith direction, a perimeter field can be image | photographed and the calibration of the multicamera which can produce | generate an omnidirectional panoramic image will be attained. Note that when the reference camera is directed in the horizontal direction, the panoramic image is inverted 90 degrees, and when it is directed toward the nadir, it is inverted 180 degrees.

  The all-around field according to the eighth aspect of the present invention is the all-around field according to any one of the first to seventh aspects of the present invention, as shown in FIG. Is a color coded target.

  If comprised in this way, since the target with a color code is used for a target, each target can be identified easily and the specification of the corresponding point between the images especially in an overlap area | region can be made easy. Furthermore, using a target with a color code, it is possible to correct the brightness reduction phenomenon of each camera and to correct the relative color tone among a plurality of cameras using a known method.

For example, as shown in FIG. 13, the imaging method for the all-around field according to the ninth aspect of the present invention includes a reference camera 11 having an optical axis 11 ′ in the reference direction and an optical axis 12 in a direction different from the reference direction. A method of photographing the entire circumference field 1 for calibration, which is photographed by the multi-camera 10a having the other camera 12 having ', and a step of arranging the multi-camera 10a (S01); and a predetermined angle from the reference direction In the range, a target having a known three-dimensional coordinate is arranged at a first density, and a target having a known three-dimensional coordinate is higher than the first density outside the predetermined angle range from the reference direction. A step (S02) of arranging the circumferential field 1 so as to be arranged at the second density so that the normal direction of the circumferential field 1 is at least two directions; and the optical axis 11 ′ of the reference camera 11 A step of setting the target at the first density (S03); and a step of photographing the plurality of targets with the multi-camera 10a in order to measure an internal parameter or an external parameter of the multi-camera 10a ( S04). In addition, it is preferable to set toward the normal line direction of a perimeter field at the process (S03) of setting toward a target.
The order of placing the multi-camera 10a (S01) and the step of constructing the all-around field 1 (S02) are not limited.

  With this configuration, by placing the target at a higher density outside the range of the predetermined angle range from the reference direction, it is possible to increase the accuracy of calibration in a place where the lens distortion is large, and from the reference direction. In the predetermined angle range, by arranging at a lower density, the whole-field imaging method can reduce the amount of calculation at the time of calibration.

  The all-around field imaging method according to the tenth aspect of the present invention is the same as the all-around field imaging method according to the ninth aspect of the present invention, for example, as shown in FIG. In the step (S02), the reference camera 11 and the other camera 12 are provided with an overlap area to be photographed, and the target arranged at the second density is photographed in the overlap area. Thus, the all-round field 1 is configured.

  With this configuration, since a target arranged at a higher density is photographed in the overlap region, the calibration accuracy is high in a portion where the image distortion (lens distortion) is large. This is an all-around field imaging method that can improve accuracy. Therefore, in creating a panoramic image, an image with less sense of discomfort at the connecting portion can be obtained. That is, more accurate calibration between cameras for acquiring a panoramic image is possible.

  The all-round field of the present invention is a subject when the multi-camera is calibrated, and the multi-camera can be calibrated with high accuracy without moving the subject for each camera.

It is a figure explaining the example which arrange | positions the multi camera 10a which has three cameras, the positional relationship of the perimeter field 1, and a target coarsely and densely. It is a figure at the time of sticking a target on the wall surface (the zenith, front and back, right and left of a multi camera), and comprising the perimeter field. It is a figure which illustrates the shape of a perimeter field. It is a figure (the 1) which shows an overlap area | region, a picked-up image, and a calibration image. FIG. 6 is a second diagram illustrating a captured image and a calibration image. FIG. 6 is a third diagram illustrating a captured image and a calibration image. It is a figure which shows the example of a multi camera. It is a figure which shows the case where a target is image | photographed by making a zenith camera into a reference | standard camera. It is a figure which shows the case where a target is imaged using a side camera as a reference camera. It is FIG. (1) explaining orientation connection. It is FIG. (2) explaining orientation connection. It is a figure explaining conversion to a machine coordinate system. It is a figure which shows the processing flow of the imaging | photography method of a perimeter field. It is a figure explaining single photograph orientation. It is a figure explaining an external parameter (orientation parameter) and an internal parameter (camera calibration parameter). It is a figure which shows target CT1 with a color code. It is a figure which shows the structural example of a color code extraction means. It is a figure which shows the example of a flow of extraction of the target with a color code. It is explanatory drawing of the gravity center position detection using a retro target. It is a figure which shows the example of a processing flow of the target area direction detection process part with a color code. It is a figure which shows the process flow example (continuation) of the target area | region direction detection process part with a color code. It is FIG. (1) for demonstrating the code reading of a retro target. It is FIG. (2) for demonstrating the code reading of a retro target. It is a figure which shows the example of the target with a color code image | photographed with a some image. It is a figure which shows target CT2 with a color code. It is a figure which shows target CT3 with a color code. It is a figure which shows target CT4 with a color code. It is a figure which shows target CT5 with a color code. It is a figure which shows target CT6 with a color code. It is a figure which shows target CT7 with a color code. It is a figure which shows target CT8 with a color code. It is a figure which shows target CT9 with a color code. It is a figure which shows target CT10 with a color code. It is a figure which shows target CT11 with a color code. It is a figure showing target CT12 with a color code which combined a plurality of color retro targets.

  Embodiments of the present invention will be described below with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same or similar reference numerals, and redundant description is omitted. Further, the present invention is not limited to the following embodiments.

[All around field]
An all-round field according to the first embodiment of the present invention will be described. The all-around field to be photographed for calibrating the multi-camera is basically a plurality of targets with known three-dimensional coordinates arranged in a space covering the multi-camera. Note that accurate three-dimensional coordinates in the target world coordinate system are measured by a total station or the like, or are known by using a technique such as digital photogrammetry or free network adjustment.

  For example, FIG. 1 shows an example of target arrangement (roughly or densely) in the case of photographing the entire circumference field 1 using a multi-camera 10a having three cameras. For convenience, any one of the cameras of the multi-camera 10a is set as a reference camera. In FIG. 1, the camera 11 is the reference camera, and the target is arranged at the first density in a predetermined angle range centered on the optical axis 11 ′ of the camera 11. In a predetermined angle range centered on the optical axes 12 ′ and 13 ′ of the other cameras 12 and 13, the target is arranged at the third density. At this time, the first density and the third density may be the same or different. Further, the third densities in a predetermined angle range centered on the optical axes 12 ′ and 13 ′ may be different from each other. Furthermore, in the area where the shooting area of the camera 11 and the shooting area of the camera 12 overlap (overlap area 14) and in the area where the shooting area of the camera 11 and the shooting area of the camera 13 overlap (overlap area 15), the target is set to the second. The first density is smaller than the second density, and the third density is smaller than the second density. That is, as shown in FIG. 1, the targets are arranged more densely in the overlap region than in a specific region centered on the optical axes 11 ', 12', and 13 '.

  The first to third densities at which the targets are arranged may be arranged so that the density is uniform in each region, or may be non-uniform. About 1st density and 3rd density, you may arrange | position so that it may become high density as it leaves | separates from the center of optical axis 11 ', 12', 13 '. In short, it is sufficient if the first density <the second density and the third density <the second density.

As an example, FIG. 2 shows an example in which a room is prepared as a space covering the multi-camera, and a target is pasted on the wall surface (the zenith, front and rear, and left and right with respect to the multi-camera) to configure an all-around field. A target with a color code is used as the target. The target with color code will be described later.
The “target” may be a mark whose three-dimensional coordinates in the world coordinate system are known, and its shape is not particularly limited. For example, the circular retro target shown in FIG. 19 may be used.

  As shown in FIG. 3, the space covering the multi-camera is not limited to one room (cube), but may be other shapes such as an oblique cubic octahedron, a U-shape, and an L-shape. . That is, the all-around field is not limited to being arranged all around (360 degrees in the horizontal direction), and it is only necessary to form a space that covers the multi-camera so that each camera of the multi-camera can capture the target. Further, the target is not limited to the above-described wall surface, and the entire field may be configured using a panel or the like.

  FIG. 4A shows an overlap region 14 (shaded portion) between images taken by the cameras 11 and 12 of the multi-camera 10a. FIG. 4B is an illustration of a photographed image taken by the camera 11, and FIG. 4C is an illustration of a corrected image after calibration. Approximately 10% of the periphery of the captured image (the regions at both ends corresponding to 10% of the width in the figure) are the overlap regions 14 and 15. Therefore, the targets arranged in the overlap regions 14 and 15 are densely arranged, and the accuracy during calibration of the overlap regions 14 and 15 is further increased. Thereby, the connection between images taken by the multi-camera can be made more natural.

In the above example, the density at which the targets are arranged is determined depending on whether or not the region is an overlap region. However, a region where the target is densely arranged and a region where the target is roughly arranged may be determined based on a specific distortion amount of the camera lens. In this way, the camera can be calibrated more effectively. Specifically, as can be seen from the equations (3) and (4) shown in [Calibration] described later, the lens distortion is expressed by coefficients k1 to k4 and p1 to p2, and the lens distortion amount from the equation is, for example, Since it is obtained by the radius r from the principal point position, the density can be determined according to the amount away from the reference optical axis, and the actual target arrangement can be obtained. For example, as shown in Expressions (3) and (4), the fact that the distortion of the lens changes according to the multiplier of the distance r from the center of the image (screen) to the object image on the image, etc. For example, the density of the target to be arranged is coarse in the central part near the optical axis, that is, the target is spaced apart, and the target is fine in the peripheral part, that is, the target spacing, so that the images are captured at substantially the same density. It is desirable to place them together.
That is, using the equations (3) and (4), the distortion amounts dx and dy on the image can be determined according to the distance r from the image center on the image, and the central portion of the image can be determined accordingly. The density of arranging the targets in the peripheral part can be made appropriate. Actually, the lens distortion most depends on the term of k ₁ r ^{2 in} the equations (3) and (4). For example, if the radius r is doubled between the rough region and the dense region, The target may be arranged at a density four times that in the dense region compared to the rough region.

  Furthermore, it is more desirable to change the size of the target so that the image can be captured with a size that can guarantee the capturing accuracy on the image. In FIGS. 5A and 5B, the size of the target to be arranged is relatively small in the central portion near the optical axis and large in the peripheral portion. FIG. 5A is an illustration of a captured image, and FIG. 5B is a calibration correction image. Specifically, the target size (X, Y) can be set by using equations (1) and (2) so that the size (x, y) is 10 pixels or more on the image. Note that it is particularly desirable that the target size is configured so that the target placed in the central portion and the target placed in the peripheral portion are arranged so that the sizes of the captured targets are aligned.

  In the above, an example in which only a target with a color code is shown as the target is shown. However, as shown in FIGS. 6A and 6B, a target with a color code and a color target with a single color (circular retro target in FIG. 6). And may be mixed. Based on the code and the position recognized by the target with the color code, the position and arrangement of the circular color target can be specified. Furthermore, it becomes easy to set the size of the circular color target to be 10 pixels or more on the image, and high measurement accuracy can be guaranteed.

  FIG. 7 shows an example of a multi-camera. The multi-camera is a camera system in which a plurality of cameras are arranged outward, for example, radially, and images of fields of view in a plurality of directions can be acquired in synchronization. (A) is a side view of a multi-camera 10a having three cameras. (B) is a top view of a multi-camera 10b having one camera oriented in the zenith direction and four cameras oriented in the peripheral direction so as to surround it. (C) is a top view of a multi-camera 10c having one camera oriented in the zenith direction and five cameras oriented in the peripheral direction. (D) is a top view of a multi-camera 10d having one camera oriented in the zenith direction and six cameras oriented in the peripheral direction. Note that the multi-camera described above may be arranged such that the camera directed in the zenith direction is directed in the horizontal direction or the nadir direction.

FIG. 8 shows an example in which a multi-camera 10c ′ having six cameras has a zenith camera as a reference camera (Cam6) and the entire field is photographed with cameras (Cam1 to Cam4) arranged around the zenith camera. (Illustration) is shown. Cam5 is not shown because it is behind Cam2 and Cam3.
In addition, in the multi-camera 10c ′ of FIG. 8, when the zenith camera is not used and the entire field is photographed by the five side cameras, each of the five cameras has an overlap area with the adjacent camera, and the image You can connect each other. Therefore, calibration is possible even when one of the side cameras is a reference camera and includes an image that does not overlap with the image of the reference camera.

An all-around field according to the second embodiment of the present invention will be described.
FIG. 9 shows a multi-camera 10c ′ having six cameras, in which one of the side cameras is a reference camera (Cam0), and the cameras (Cam1 and 4) arranged on the left and right of the multi-camera 10c ′ An example of shooting is shown. As described above, the all-around field may be one in which targets are spatially arranged in a space covering a part of the camera included in the multi-camera. For example, for a multi-camera capable of photographing in the horizontal direction (360 degrees), the perimeter field may be, for example, half or one-third instead of one room.

FIG. 10 shows a connection method of orientation connection between corresponding cameras when the multi-camera is rotated when the entire perimeter field is half-surface, and the entire perimeter field is photographed and calibrated by each camera. “Orientation connection” means to unify the orientation (position, inclination) between cameras corresponding to the case where the multi-camera is rotated or moved (see FIGS. 3A to 3C). say.
The multi-camera of FIG. 10 has six cameras, the circle shown in FIG. 10 indicates the camera, and the T-shape indicates the direction of the camera. Moreover, the camera arrange | positioned at the side is set to Cam0-4, and a zenith camera is set to Cam5. (1) First, the entire field is photographed with Cam 4-0-1. (2) Next, the multi-camera is rotated, and the entire field is photographed with Cam0-1-2. (3) to (5) Similarly, the multi-camera is rotated, and the side camera is shifted one by one to photograph the entire circumference field. (6) Finally, in order to obtain the positional relationship between the zenith camera Cam5 and the side camera, the entire circumference field is photographed by the Cam5 and the side camera (Cam1-2-3-4 in FIG. 10).

  A target having three-dimensional coordinates as an absolute reference point is photographed in an image photographed by each camera. Therefore, as shown in FIG. 11, the corresponding camera image plane is fitted in the captured image to obtain the three-axis rotation (ω, φ, κ) and the parallel movement (dx, dy, dz). Next, the orientation of each camera is redefined in the machine coordinate system (camera coordinate system) of the multi-camera. That is, as shown in FIG. 12, in the machine coordinate system, the origin: the average of the side camera positions, the vertical axis: fitting the side camera position to the horizontal plane, the Y axis: the direction of the camera (Cam0) (horizontal 0 degree), the X axis : A direction orthogonal to the Y-axis on a horizontal plane. In this way, the orientation connection between the corresponding cameras enables calibration of multi-cameras (including the zenith direction) that can shoot in the horizontal direction (360 degrees) even if the entire circumference field is half or one third. Is possible.

  An imaging method for the all-around field according to the third embodiment of the present invention will be described. FIG. 13 shows a processing flow of the all-around field imaging method. First, the multi-camera to be calibrated is placed, for example, in the center of the room (S01). Next, the target is arranged on the wall surface of the room so that the target arranged roughly in the central portion of the photographed image is photographed. Further, the target is arranged on the wall surface so that the target arranged more densely than the central part is photographed in the peripheral part of the photographed image, and the all-around field is configured (S02). Next, the optical axis of each camera is set toward a roughly arranged target (S03). Next, the target is photographed by each camera of the multi-camera (S04). In the all-around field configuration (S02), adjacent cameras may have an overlap region, and targets that are arranged more densely in the overlap region may be photographed.

[Calibration]
When a camera is calibrated, it is necessary to obtain an internal parameter (camera calibration parameter) and an external parameter (orientation parameter) of the camera. When obtaining these parameters, a plurality of targets whose three-dimensional coordinates are measured in advance are photographed from different directions, the targets are associated with each other, and the correspondence between the targets and the pre-measured results are obtained. From the above, internal parameters and external parameters of the camera are obtained.

There are various methods for this calibration. As an example, the “bundle adjustment method with self-calibration” used in the photogrammetry field will be described.
The “bundle adjustment method” is based on the collinear condition (see FIG. 14) that the light beam connecting the subject, the lens, and the CCD imaging surface must be on the same line (see FIG. 14). This is a method of adjusting the external parameters (position and tilt) of the camera and the coordinates of the target at the same time using the least square method. “With self-calibration” means that the internal parameters of the camera (image distance, image center, distortion correction) This is a method for obtaining a coefficient (see FIG. 15).

The collinear condition basic expression of the bundle adjustment method with self-calibration is expressed by the following expressions (1) and (2). The correction model formula for the internal parameter is expressed by the following formulas (3) and (4). Solving from these equations and a number of known points and image points (at least 6 points) on the image by self-calibration (bundle adjustment) by single photolocation, the position (projection center: (X ₀ , Y ₀ , Z ₀ )) and slope (ω, φ, κ), and internal parameters are image distance (f), image center (x ₀ , y ₀ ), distortion correction coefficients k _{1 to} k ₄ , p ₁ , p ₂ ). When the camera is calibrated, the calibration accuracy improves as the number of targets or the number of captured images increases (several to several tens). In general, there is a correlation between the number of points whose three-dimensional coordinates are known (reference number) and the number of shots, and considering the accuracy, for example, if the reference number is 100 points or more, the number is 5 or more and the reference number is about 25 points. If so, the number of sheets is preferably 10 or more.


a _{1 to} a ₉ : rotation matrix a ₁ = cosφcosκ
a ₂ = −cosφsinκ
a ₃ = sinφ
a ₄ = cosωsinκ + sinωsinφcosκ
a ₅ = cosωcosκ−sinωsinφsinκ
a ₆ = −sin ω cos φ
a ₇ = sinωsinκ−cosωsinφcosκ
a ₈ = sinωcosκ + cosωsinφsinκ
a ₉ = cosωcosφ
X, Y, Z: Target coordinates _{X 0, Y 0, Z 0} : projection center (principal point)
x, y: image coordinates dx, dy: distortion amount on the image

[Target with color code]
A target with a color code used as a target in the embodiment of the present invention will be described. The color coded target has already been filed as Japanese Patent Application No. 2005-289334 by the present applicant.
FIG. 16 shows an example of a target with a color code. FIG. 16 shows a target CT1 with a color code having three color code unit patterns. The color-coded target CT1 includes a retro target portion (position detection pattern) P1, a reference color portion (reference color pattern) P2, a color code portion (color code pattern) P3, and a white portion (empty pattern) P4. . The retro target portion P1, the reference color portion P2, the color code portion P3, and the white portion P4 are disposed at predetermined positions in the color code-attached target CT1. That is, the reference color portion P2, the color code portion P3, and the white color portion P4 are arranged in a predetermined positional relationship with respect to the retro target portion P1.

  The retro target portion P1 is used for detecting the target itself, detecting its center of gravity, detecting the target direction (inclination direction), and detecting the target area. In FIG. 16, a retro target is used as the retro target portion P1. That is, the color-coded target CT1 has three marks (position detection patterns) whose three-dimensional coordinates are known.

  The reference color portion P2 is used for reference at the time of relative comparison and for color calibration for correcting the color misregistration in order to cope with color misregistration due to photographing conditions such as illumination and a camera. Furthermore, the reference color portion P2 can be used for color correction of the target CT with a color code created by a simple method. For example, when using a target CT with a color code printed by a color printer (inkjet, laser, sublimation type printer, etc.) that is not color-controlled, individual differences will occur in the color of the printer used, etc. By comparing and correcting the colors of P2 and the color code portion P3, the influence of individual differences can be suppressed.

  The color code part P3 expresses a code by a combination of colors for each unit pattern. The number of codes that can be expressed varies depending on the number of code colors used for the code. For example, when the number of code colors is n, the color code target CT1 of FIG. 16 has three unit patterns of the color code portion P3, and therefore can represent n × n × n codes. In order to increase the reliability, even when the condition that the colors used in other unit patterns are not used redundantly, n × (n−1) × (n−2) codes can be expressed. If the number of code colors is increased, the number of codes can be increased. Furthermore, if the condition that the number of unit patterns of the color code part P3 is equal to the number of color codes is imposed, all code colors are used for the color code part P3. Therefore, not only the comparison with the reference color part P2, By relatively comparing the colors between the unit patterns of the color code portion P3, the identification code can be determined by checking the color of each unit pattern, and the reliability can be improved. Furthermore, if a condition for making the area of each unit pattern all the same is added, the target CT with a color code can also be used when detected from an image. This is because the area occupied by each color is the same even between the target CTs with color codes having different identification codes, so that almost the same dispersion value is obtained from the detection light from the entire color code part. In addition, since the boundary between unit patterns is repeated at equal intervals and a clear color difference is detected, it is possible to detect the target CT from the image from such a repeated pattern of detection light.

  The white portion P4 is used for detecting the direction of the color-coded target CT and for correcting color misregistration. Of the four corners of the color-coded target CT, there is a place where the retro target portion P1 (retro target) is not arranged in only one place, and this can be used for detecting the direction of the color-coded target CT. Thus, the white portion P4 may be a pattern different from the retro target portion P1 (retro target). Therefore, a character string such as a number for visually confirming the code may be printed on the white portion, and it may be used as a code area such as a barcode. Furthermore, in order to increase the detection accuracy, it can be used as a template pattern for template matching.

(Color code extraction means)
A procedure for extracting a color code using the color code extraction unit 100 after photographing a target CT with a color code will be described below.
FIG. 17 shows a configuration example of the color code extraction unit 100. The color code extraction unit 100 includes an extraction unit 41 that extracts a target with a color code and an identification code determination unit 46 that determines the color code. The extraction unit 41 includes a search processing unit 110, a retro target grouping processing unit 120, a target detection processing unit with color code 130, and an image / color pattern storage unit 140. Also, the identification code determination unit 46 determines the color code detected by the color code-added target detection processing unit 130 and assigns a code number.

  Reference numeral 20 denotes an imaging unit that photographs a measurement object including a target with a color coat, and a stereo camera or the like is used. Reference numeral 23 denotes an image data storage unit that stores a stereo image taken by the imaging unit 20. Reference numeral 150 denotes a sign information storage unit that stores the position coordinates of the retro target part P1 extracted by the extraction unit 41 and the identification code determined by the identification code determination unit 46 in association with each other. The data stored in is used for orientation by the orientation unit 44, or is used for measurement of the 3D coordinates or 3D shape of the measurement object by the 3D position measurement unit 50.

  The search processing unit 110 detects a position detection pattern (retro target unit P1) such as a retro target pattern from a color image (captured image) captured by the imaging unit 20 or stored in the image data storage unit 23. When a template pattern is used instead of a retro target pattern as a target for position detection, template pattern detection is performed.

  The retro target grouping processing unit 120 determines that the retro targets detected by the search processing unit 110 belong to the same color-coded target CT (for example, those that enter the region of the color-coded target CT having the same position coordinates). Group as candidates that belong to the same group.

  The color code target detection processing unit 130 detects the color code target region direction detection from the retrotarget group (candidate) determined to belong to the same color code target. The color code unit with reference to the processing unit 131, the reference color part P2 of the target CT with color code, the color arrangement in the color code part P3, the color detection processing part 311 for detecting the color in the image, and the reference color pattern P2 P3 and a color correction unit 312 that corrects colors in the image, and a confirmation processing unit 313 that confirms whether the grouping is properly performed.

  The image / color pattern storage unit 140 indicates the type of the color-coded target CT with respect to the read image storage unit 141 that stores the image read into the extraction unit 41 and a plurality of types of color-coded target CTs that are scheduled to be used. A type code number is recorded, and further, a color code target correspondence table 142 is recorded which records the correspondence between pattern arrangement and code number for various color code target CTs.

  The identification code discriminating unit 46 discriminates an identification code from the color arrangement in the color code unit P3 and converts it into an identification code. The color code-added target CT detected by the color code target detection processing unit 130 Based on the direction data, the coordinate conversion processing unit 321 for converting the coordinates of the color-coded target CT and the color code in the color code portion P3 of the coordinate-converted target CT with the color code discriminates the identification code. The code conversion processing unit 322 converts the code into a code.

(Color code extraction flow)
FIG. 18 shows an example of a color code target extraction flow.
First, a color image (photographed image) to be processed is read from the imaging unit 20 or the image data storage unit 23 into the read image storage unit 141 of the extraction unit 41 (S500). Next, a retro target is detected from each read image (S510).

  The search method includes (1) a method of searching for a retro target portion P1 (retro target) which is a position detection pattern in the color code target CT, (2) a method of detecting the color dispersion of the color code portion P3, or ( 4) There are various methods such as a method using a colored position detection pattern. Here, (1), (2) and an example (3) combining these will be described. (4) will be described in (Color Coded Target CT11).

  (1) When a retro-target is included in the color-coded target CT, a pattern with a clear brightness difference is used. The retro target can be easily detected by binarizing this image.

  FIG. 19 is an explanatory diagram of the center-of-gravity position detection using a retro target. (A1) is a retro target in which the brightness of the inner circle portion 204 is bright and the brightness of the outer circle portion 206 is dark, and (A2) is the retro target in (A1). (B1) is a retro target in which the brightness of the inner circle portion 204 is dark and the brightness of the outer circle portion 206 is bright, and (B2) is a brightness distribution diagram in the diameter direction of the retro target of (B1). Show. When the brightness of the inner circle portion 204 is bright as shown in FIG. 19 (A1), the retro target has a bright portion with a large amount of reflected light at the center of gravity in the captured image of the measurement object 1, and thus the light amount distribution of the image. As shown in FIG. 4A2, it is possible to obtain the inner circle portion 204 and the center position of the retro target from the threshold value To of the light amount distribution.

When the target existence range is determined, the barycentric position is calculated by, for example, the moment method. For example, assume that the plane coordinates of the retro target 200 shown in FIG. 19A1 are (x, y). Then, (Expression 1) and (Expression 2) are calculated for points in the x and y directions where the brightness of the retro target 200 is greater than or equal to the threshold value To (* is a multiplication operator).
xg = {Σx * f (x, y)} / Σf (x, y) ---- (Equation 1)
yg = {Σy * f (x, y)} / Σf (x, y) ---- (Formula 2)
(Xg, yg): coordinates of the center of gravity position, f (x, y): density value on (x, y) coordinates In the case of the retro target 200 shown in FIG. (Expression 1) and (Expression 2) are calculated for points in the x and y directions that are less than or equal to the value To.
Thereby, the position of the center of gravity of the retro target 200 is obtained.

(2) Usually, the color code portion of the color-coded target CT uses a large number of code colors and has a feature that the color dispersion value is large. For this reason, the target CT with a color code can be detected by finding a portion having a large dispersion value from the image.
(3) In the case where a retro-target is included in the color-coded target CT, first, a portion with high brightness (possibly having a retro-target portion P1) is scanned and detected over the entire image, and then detected. It can be efficiently detected by finding a portion (a color code portion P3 can be present) having a high color dispersion value around a portion having high brightness.

An example of (1) will be described. Returning to FIG. 18, the retro target detection processing unit 111 stores the coordinates of a plurality of retro targets detected from the color image in the read image storage unit 141.
Next, the retro target grouping processing unit 120 detects a retro target group candidate belonging to the same color-coded target CT from the retro target coordinates stored in the read image storage unit 141 (for example, the coordinates are color). The detected combination CT is detected), and the combination is stored in the read image storage unit 141 as a group (S520). The confirmation is possible, for example, by measuring the distance between the three retro targets in the detected color-coded target CT and the apex angle of the triangle connecting the three retro targets (see S530).
Further, by comparing the detected pattern of the target with color code with the target table 142 with color code, it is confirmed which type of target with the color code.

  Next, the target detection processing unit with color code 130 stores the region of the target CT with color code from the center of gravity position of the retro target in units of retro targets stored in the read image storage unit 141 in the region direction detection processing unit 131. A direction is obtained (S530). Before or after obtaining the region and direction, the color detection processing unit 311 detects the color of the reference color portion P2, the color code portion P3, and the measurement object in the image. If necessary, the color correction unit 312 corrects the color of the color code portion P3 and the measurement object 1 in the image based on the color of the reference color portion P2. In addition, when a target with a color code of a print color that is not a reference is used, the reference color portion is corrected together. Then, the confirmation processing unit 313 confirms whether the grouping is performed properly, that is, whether the center of gravity of the retro targets once grouped belongs to the same color code target CT. If it is determined that they belong to the same group, the process proceeds to the next identification code identification process (S535). If it is determined that they do not belong to the same group, the process returns to the grouping process (S520).

20 and 21 show an example of a processing flow of the target area direction detection processing unit 131 with a color code. The retro target code reading will be described with reference to FIGS. Here, a procedure for reading a code from the color-coded target CT1 in FIG. 16 will be described.
In order to read the code from the target CT1 with color code, it is necessary to know the area and direction of the target CT1 with color code, so the center points of the three position detection retro targets are labeled R1, R2, and R3 (FIG. 22). (See (a)).

The labeling method creates a triangle that passes through the center-of-gravity points R1 to R3 of the three retro targets of interest (S600). Appropriately select one of the centroid points R1 to R3 of the three retro targets and label it temporarily with T1 (S610), and temporarily label the remaining two centroid points with T2 and T3 (S612, (Refer FIG.22 (b)).
Next, the sides passing through the barycentric points are labeled. The side passing through T1 and T2 is set as L12, the side passing through T2 and T3 is set as L23, and the side passing through T3 and T1 is set as L31 (S614, see FIG. 23A).

Next, the inside of the triangle is scanned in an arc shape for pixel values that are separated from each vertex (center of gravity) by a radius R, and the color changes within the scanned range (see FIG. 23B).
At the center of gravity T1, L12 to L31 are scanned clockwise, at the center of gravity T2, L23 to L12 are scanned clockwise, and at the center of gravity T3, L31 to L23 are scanned clockwise (S620 to S625).
The method of determining the radius is determined by multiplying the size of the retro target on the image according to the scanning angle. When the retro target is photographed from an oblique direction, it becomes an ellipse, so the scan range is also an ellipse. The magnification is determined by the size of the retro target and the distance between the retro target center of gravity position and the reference color portion P2.

In order to further reduce the influence of noise and the like, a method of determining a representative value such as an average value between the radius R-Δr and R + Δr by giving a width to the scanning range is also conceivable.
In this example, the scan is performed in an arc shape, but a method of scanning on a straight line perpendicular to the side of the triangle formed by the center of gravity is also conceivable (see FIG. 23C).

  In the example of the color-coded target CT1 in FIG. 16, there is a change in color as a result of scanning around the center of gravity T2, and a change is seen in the values of R (red), G (green), and B (blue). Change peaks appear in the order of R, G, and B. In the scan results around T1 and T3, there is no color change, and the values of R, G, and B are almost constant and no peak appears (see FIG. 23B). Thus, the color change is seen around one centroid point T2, and the direction of the color-coded target CT1 is changed due to the feature that there is no color change around the remaining two centroid points T1 and T3. I can judge.

The confirmation processing for labeling is performed by the confirmation processing unit 313. The barycentric point having the color change in the scan result is set as R1, and the remaining two barycentric points are labeled R2 and R3 by clockwise labeling from the barycentric point having the color change (S630 to S632). In this example, the center of gravity T2 is R1, the center of gravity T3 is R2, and the center of gravity T1 is R3. If one centroid point with a color change and two centroid points with no color change are not detected, a retro target combination error occurs (S633), and three new retro targets are selected. (S634), the process returns to S600. Thus, it can be confirmed from the processing results whether the three selected retro targets belong to the same color-coded target CT1. In this way, the retro target grouping is determined.
The above-described labeling method has been described using the color-coded target CT1 in FIG. 16 as an example, but the same processing can be performed by changing part of the processing for various color-coded targets CT described later.

(Code identification)
Returning to FIG. 18, the identification code determination unit 46 selects the color-coded target CT1 based on the center-of-gravity positions of the retro targets grouped by the coordinate conversion processing unit 321 for the color-coded target CT1 extracted by the extraction unit 41. Then, the code conversion processing unit 322 identifies the color code (S535), performs code conversion to obtain the identification code of the target CT1 with color code (S540), and reads the read image. The data is stored in the storage unit 141 (S545).

  This processing flow will be described with reference to FIG. The photographed image of the target with the color code that is distorted in shape, such as affixed to the curved surface or photographed from an oblique direction, is converted into a front view without distortion using the labels R1, R2, and R3 (S640). ). By converting the coordinates, the retro target portion P1, the reference color portion P2, the color code portion P3, and the white portion P4 can be easily identified with reference to the design value of the target with the color code, and subsequent processing can be facilitated.

  Next, it is checked whether or not there is a white portion P4 as designed on the coordinate-converted target CT1 with color code (S650). If it is not as designed, a detection error occurs (S633). If the white portion P4 is present as designed, it is determined that the color-coded target CT1 has been detected (S655).

Next, the color code of the color-coded target CT1 whose color and color correction have been performed and whose region and direction are known is determined.
The color code part P3 expresses a code by a combination of colors for each unit pattern. For example, when the number of code colors is n and the number of unit patterns is 3, when a condition that an n × n × n code can be expressed and colors used in other unit patterns are not used redundantly is imposed, A code of n × (n−1) × (n−2) can be expressed. When the number of code colors is n, the number of unit patterns is n, and the condition that the colors are not used redundantly is imposed, the code corresponding to the factorial of n can be expressed.
In the code conversion processing unit 322, the identification code determination unit 46 compares and collates the color combination of the unit pattern in the color code unit P3 with the color combination of the target correspondence table 142 with color code to determine the identification code ( S535 in FIG.

  Color discrimination methods include (1) a relative comparison method in which the color of the reference color portion P2 and the color of the color code portion P3 are compared, and (2) the color of the reference color portion P2 and the color of the white portion P4. There are two methods: an absolute comparison method in which color correction of the target CT1 with color code is performed and the code of the color code portion P3 is discriminated with the corrected color. For example, when the number of colors used for the color code portion P3 is small, the reference color is used as a comparative color for relative comparison, and when the number of colors used for the color code portion P3 is large, calibration is performed to correct the colors. Used as a comparative color for absolute comparison. As described above, color detection is performed by the color detection processing unit 311, and color correction is performed by the color correction unit 312. In three-dimensional measurement using images, a plurality of images are taken, and in most cases, color misregistration occurs between the images due to shooting conditions and the like. By using a target with a color code, it is possible to correct a color difference between a plurality of images.

In the code conversion processing unit 322, the identification code determination unit 46 detects the reference color part P2 and the color code part P3 using either the method (1) or (2) (S660, S670), and the color code. Each color of the part P3 is discriminated, the color is converted into a code, and the identification code of the target CT1 with the color code is obtained (S680, S540 in FIG. 18).
For each image, the number of the color-coded target CT1 included in the image is registered in the read image storage unit 141 (S545 in FIG. 18). The data registered in the read image storage unit 141 is used for the orientation by the orientation unit 44 based on the detection position coordinates of a large number of color-coded markers CT, or the tertiary of the measurement object by the three-dimensional position measurement unit 50. Used to measure original coordinates and 3D shapes.

(Association between images)
FIG. 24 shows an example of a target with a color code photographed with a plurality of images. FIG. 24A shows how the stereo images overlap. The basic range to be measured is an overlap range of two (a pair of) stereo shot images. At this time, it is preferable to perform imaging so that the four color-coded target CTs fall within the overlapping range. In this way, three-dimensional measurement is possible using a stereo image. FIG. 24B shows an example of how to overlap adjacent stereo images. In this way, it is preferable to take a series of images so as to overlap, including two color-coded targets CT in the vertical and horizontal directions. In this way, non-contact three-dimensional measurement over a wide area can be automated. The break line is a line indicating the effective area of the image, and the inside of the line connecting the outermost retro targets of the four color-coded targets CT is the effective area.

  FIG. 24C is a diagram for explaining stereo matching area setting. FIG. 24C shows an example in which the color-coded target CT has three retro targets for position detection. The stereo matching area is automatically determined by locating the color-coded target CT near the four corners of the screen and always using the area connecting the outermost retro targets of the color-coded target CT as the matching area. At the same time, the overlap between the model images can be ensured. In this way, by determining the matching region, as shown in FIG. 24B, the overlap between the model images can be reliably obtained. If at least two or more position detection patterns (retro target portions) P1 are arranged on each color code target CT (in the case of two points, they are arranged diagonally), matching area automatic setting processing can be performed.

Other examples of color coded targets are shown below.
(Target CT2 with color code)
FIG. 25 shows an example of the color-coded target CT2. FIG. 25 shows the target CT2 with color code having six color code portions P3. The three unit patterns of the color code pattern P3 of the color-coded target CT1 in FIG. 16 are separated by diagonal lines so that there are six unit patterns. Therefore, the method for detecting the region and direction of the color-coded target CT is shown in FIG. As in the case of, retro target labeling can be performed by changing the color when the inside of a triangle formed by each barycentric point is scanned. That is, two color changes occur around R1, and no color change occurs around R2 and R3. In addition, the reading process of the color code part P3 is increased to six places. If the colors used for the color code are six colors, the number of codes can express 6 × 5 × 4 × 3 × 2 × 1 = 720 codes. The last color code part (the last x1 part) does not affect the number of codes, but is necessary for relative comparison, and can also be used for checking misrecognition when the color code part is recognized.

(Target CT3 with color code)
FIG. 26 shows an example of the color-coded target CT3. The target CT3 with color code in FIG. 26 is obtained by eliminating the reference color portion P2 of the target CT2 with color code in FIG. 25 and enlarging one of the retro target portions P1.
In the color-coded target CT3 in FIG. 26, the retro target corresponding to R1 is large, so that the labeling process corresponding to R1, R2, and R3 of each retro target portion can be performed by detecting the size of the retro target. However, since the reference color portion P2 does not exist because the retro target corresponding to R1 is large, it is difficult to determine the color code.

  As a countermeasure in this case, for example, if the color used for the color code is set to six colors and the same color is not used repeatedly, all code colors appear in the color code portion P3. As a result, the colors of each color code portion P3 need only correspond to the six code colors, and the colors of the color code portions P3 can be compared by relative comparison. The number of codes can represent 720 codes in the same way as the color-coded target CT2 in FIG.

(Target CT4 with color code)
FIG. 27 shows an example of the color-coded target CT4. A color-coded target CT4 in FIG. 27 is a pattern in which a black region P5 is added to the outside of the color-coded target CT1 in FIG. The black region P5 can suppress the influence of the color and pattern around the target.
The color-coded target CT4 in FIG. 27 is the same as the color-coded target CT1 in FIG. 16 because the black region P5 is only formed outside the color-coded target CT1 in FIG. The black region P5 can be formed on a target with a color code other than that shown in FIG.

(Target CT5 with color code)
FIG. 28 shows an example of the color-coded target CT5. The target CT5 with color code in FIG. 28 is obtained by reducing the unit pattern of the color code part P3 and setting the unit pattern of the color code part P3 to nine places.
For this reason, when detecting the region and direction of the color-coded target CT5, two color changes appear around R1, and one color change appears around R2 and R3. Moreover, R1, R2, and R3 can be confirmed by detecting the positional relationship with the white part P4. In addition, the reading process of the color code part P3 is increased to nine places. If nine colors are used for the color code, the number of codes can be expressed as a factorial of 9 = 362880 code. Otherwise, processing can be performed in the same manner as in FIG.

(Target CT6 with color code)
FIG. 29 shows an example of the color-coded target CT6. The color-coded target CT6 in FIG. 29 has the reference color portion P2 arranged in the R2 and R3 portions of the retro target portion P1 and the white portion P4 of the color-coded target CT1 in FIG. It is also used for direction detection. The retro target portion P1 is one portion of R1. A color code pattern P3 can be arranged at the reference color portion P2 of the target CT1 with color code in FIG. 16, and the color code pattern can be increased by one.
When the reference color part P2 is used for detecting the area and direction, the center of gravity of the square pattern is obtained and the brightness difference is detected by the reference color light, but the accuracy is inferior to the retro target. A point can be detected.

(Target CT7 with color code)
FIG. 30 shows an example of the color-coded target CT7. The target CT7 with color code in FIG. 30 has a smaller area of the color code portion P3 of the target CT1 with color code in FIG. 16, and further, the pattern of the retro target portion P1 is one portion of R1, and the color code pattern is four locations. It is a small type. Also in this color code-added target CT7, the reference color portion P2 is used for detecting the area and direction.

(Target CT8 with color code)
FIG. 31 shows an example of the color-coded target CT8. A target CT8 with a color code in FIG. 31 is a small type in which the white portion P4 of the target CT7 with a color code in FIG. 30 is replaced with a color code portion P3 so that there are five color code patterns.

(Target CT9 with color code)
FIG. 32 shows an example of the color-coded target CT9. The color-coded target CT9 in FIG. 32 is obtained by forming a black separation region portion P6 between unit patterns of the color-coded target CT1 in FIG. As a result, errors can be reduced when determining areas and colors.

(Target CT10 with color code)
FIG. 33 shows an example of the color-coded target CT10. The color-coded target CT10 in FIG. 33 is obtained by replacing the retro-target P1 of the color-coded target CT1 in FIG. 16 with a template pattern P7 so as to support template matching detection. Like the retro target, the center of gravity can be detected with high accuracy.

(Target CT11 with color code)
FIG. 34 shows an example of the color-coded target CT11. A color-coded target CT11 in FIG. 34 is obtained by replacing the retro target P1 of the color-coded target CT1 in FIG. 16 with a color retro target P8.
The color-coded target CT11 employs (4) a method using a colored position detection pattern as a search method for target position extraction.

  (4) Different colors are arranged on the three corner retro targets used in the color-coded target CT, and the colors reflected by the respective retro targets are made different. Since different colors are arranged on the retro targets at the three corners, it is easy to distinguish each retro target belonging to one color-coded target CT. In the retro target grouping process, even when a large number of retro targets are used, the process can be simplified by selecting the retro targets of different colors that are closest to each other as candidates for the same group.

When a large number of retro targets are used as the reference point RF, the retro target of the color code target CT and the single retro target are mixed. If the retro target is white, it is easy to distinguish.
For the detection of the position of the center of gravity, refer to the description in FIG.

In addition, it is also possible to use a single or a plurality of color retro targets as the position detection mark instead of the color-coded target CT. For example, in the case of a single unit, it becomes easy to search for a corresponding point and a stereo image pair with respect to a reference point by increasing the type of color arrangement or by randomizing the color arrangement.
It is also possible to use a plurality of color retro targets in combination. For example, as shown in FIG. 35, three color retrotargets P8 are combined into one label (label CT with a color code), and by changing the combination of the color schemes, corresponding points with respect to a reference point, stereo image pairs It becomes easy to search.

  In determining the direction of the target with the color code, the direction was determined by the difference in the number of colors of the inner corners of the triangle that can be created by the retro target. However, the retro target at the three corners of the color code target is colored, and each retro target reflects. It is also conceivable to use different colors. For example, if the upper left retro target used for the color coded target is red, the upper right retro target is blue, and the lower left retro target returns green reflected light, the direction of the color coded target can be determined. It can be easily processed by judging the color of the reflected light.

  The above-described target with a color code has mainly been described as an example of a target with a color code whose unit pattern is a square. However, other shapes such as a bar-shaped pattern and a circular pattern may be used. It may be a color code.

  Further, instead of attaching the target CT with color code, or in combination with the application of the target CT with color code, the target CT with color code may be projected onto the object to be imaged by the projection apparatus. Further, the configuration of the color code extracting means and the flow of extracting the target with the color code can be changed as appropriate.

1 all-round fields 10a, 10b, 10c, 10d Multi-camera 11 Camera, reference camera 12 Camera 13 Camera 11 ′ Optical axis 12 ′ of camera 11 Optical axis 13 ′ of camera 12 Optical axis 14 of camera 13 Overlap region 15 Overlap Area 20 Imaging unit 23 Image data storage unit 41 Extraction unit 44 Orientation unit 46 Identification code determination unit 50 Three-dimensional measurement unit 100 Color code extraction unit 110 Search processing unit 111 Retro target detection processing unit 120 Retro target grouping processing unit 130 Color code Target detection processing unit 131 with color code Target area direction detection processing unit 140 Image / color pattern storage unit 141 Read image storage unit 142 Target table with color code 150 Sign information storage unit 200 Retro target 204 Inner circle 206 Part ¥ 311 color detection processing section 312 color correction portion 313 confirmation processing unit 321 coordinate converting unit 322 code conversion processing section CT, CT1~CT12 color-coded target L12, L23, L31 sides P1 position detection pattern (retro-target section)
P2 standard color pattern (standard color part)
P3 Color code pattern (color code part)
P4 empty pattern (white part)
P5 Black region portion P6 Separation region portion P7 Template pattern P8 Color retro target R1 to R3 Centroid point To Threshold value T1 to T3 Temporary label

Claims (8)

An all-around field for photographing with a multi-camera having a reference camera having an optical axis in a reference direction and another camera having an optical axis in a direction different from the reference direction, and calibrating the multi-camera. ,
Comprising a plurality of targets with known three-dimensional coordinates arranged in the perimeter field that are imaged to measure internal or external parameters of the multi-camera;
In a predetermined angle range from the reference direction, the targets are arranged at a first density,
Outside the range of a predetermined angle range from the reference direction, the target is arranged at a second density higher than the first density,
The normal of the entire circumference field have at least two directions,
The reference camera and the other camera are arranged so as to shoot with an overlap region between each other,
The target arranged at the second density is arranged such that the target is photographed in the overlap area,
All-around field. The targets arranged at the first density are of a first size;
The targets arranged at the second density have a second size larger than the first size,
The target of the first size and the target of the second size are configured so that the sizes of the captured targets are aligned.
The all-around field according to claim 1. The predetermined angle range from the optical axis direction of the other camera is an area other than the overlap area,
In a predetermined angle range from the optical axis direction of the other camera, the target is arranged at a third density,
The second density is higher than the third density;
The all-around field according to claim 1 or claim 2 . The predetermined angle range from the reference direction of the reference camera is a range equal to or less than a predetermined distortion amount of a lens included in the camera.
The all-around field according to any one of claims 1 to 3 . The all-around field is configured to surround the multi-camera.
The all-around field according to any one of claims 1 to 4 . The reference camera is directed to the zenith direction, and a plurality of other cameras having an optical axis in a direction different from the reference direction is an all-around field photographed by a multi-camera arranged around the reference camera,
The all-around field is composed of a side surface that surrounds the multi-camera and an upper surface that covers the multi-camera.
The all-around field according to any one of claims 1 to 4 . The target is a target with a color code.
The all-around field according to any one of claims 1 to 6 . An imaging method for a calibration all-around field, which is taken by a multi-camera having a reference camera having an optical axis in a reference direction and another camera having an optical axis in a direction different from the reference direction,
Arranging the multi-camera;
In a predetermined angle range from the reference direction, targets having known three-dimensional coordinates are arranged at a first density, and outside the predetermined angle range from the reference direction, targets having known three-dimensional coordinates are Arranging at a second density higher than the first density and configuring the circumferential field such that the normal direction of the circumferential field is at least two directions;
Setting the optical axis of the reference camera toward a target arranged at the first density;
Imaging a plurality of the targets with the multi-camera to measure an internal parameter or an external parameter of the multi-camera ;
The step of configuring the all-round field is such that the reference camera and the other camera are photographed by providing an overlap region with each other, and the target disposed at the second density in the overlap region Configure the perimeter field so that is photographed;
How to shoot all around field.