JP5648159B2 - Three-dimensional relative coordinate measuring apparatus and method - Google Patents

Description

  The present invention relates to a three-dimensional relative coordinate measuring apparatus for measuring a relative coordinate between any one of three reference points whose relative coordinates are known and one target point whose coordinates are unknown in a three-dimensional coordinate system.

  A stereo method is known as a method for measuring relative coordinates from an image.

  In the stereo method, two images are obtained by photographing with two cameras placed at different positions or by photographing from two different positions with one camera. In the stereo method, relative coordinate information is obtained from subtle differences in images based on parallax, as in human binocular vision.

  Patent Document 1 discloses a photo measurement system that can perform highly accurate surveying by a stereo method.

JP 10-2221072 A

  However, in Patent Document 1, it is assumed that the position and orientation of the camera at the photographing point are grasped in advance by using a detection device such as an inclination angle sensor.

  Therefore, the present invention has been made in view of such a situation, and provides a three-dimensional relative coordinate measuring apparatus capable of measuring relative coordinates without grasping in advance the position and orientation of a camera at a shooting point. The purpose is to do.

  In order to achieve the above object, a relative coordinate measuring apparatus according to an aspect of the present invention includes any one of three first reference points whose relative coordinates in a three-dimensional coordinate system are known, and the three-dimensional coordinate system. A three-dimensional relative coordinate measuring apparatus that measures relative coordinates with a single target point whose coordinates are unknown, wherein the three first reference points are imaged by a first imaging device from a first viewpoint. An image acquisition unit for acquiring the first acquired image, and information on a first pixel viewpoint projection angle for each pixel, and using the information, the three points projected on the first acquired image A viewpoint projection angle extraction unit that acquires three first reference viewpoint projection angles that are the first pixel viewpoint projection angles corresponding to the first reference point, and the first pixel viewpoint projection angle is Projected to first pixel two-dimensional coordinates, which are the coordinates of each pixel in the first acquired image An angle formed by a line segment connecting each point of the three-dimensional coordinate system and the first viewpoint, and the optical axis of the first imaging device, and the three first reference viewpoint projection angles. And the relative coordinates of the first reference points of the three points, the first reference plane including the first imaging plane that is the imaging plane of the first acquired image and the first reference points of the three points. Relative coordinates between any one of the three first reference points and the target point are measured using an inclination angle calculation unit that calculates a first inclination angle formed with a plane and the first inclination angle. A measuring unit.

  Thereby, by using the viewpoint projection angle, it is possible to measure relative coordinates without grasping in advance the position and orientation of the camera at the photographing point.

  The three first reference points are any three of three or more reference points whose relative coordinates in the three-dimensional coordinate system are known, and the image acquisition unit includes the first reference point. The first reference image of the three first reference points and the target point from the viewpoint are acquired by the first imaging device, and the reference points of the three or more points are acquired from the second viewpoint. A second acquired image obtained by imaging a second reference point, which is any three of the three points, and the target point with a second imaging device that is the same as or different from the first imaging device. The viewpoint projection angle extraction unit further holds information on a second pixel viewpoint projection angle for each pixel, and projects the second acquired image on the second acquired image using the information on the second pixel viewpoint projection angle. Three second reference viewpoints corresponding to the second pixel viewpoint projection angles corresponding to the three second reference points A shadow angle is acquired, and the second pixel viewpoint projection angle is calculated based on each point of the three-dimensional coordinate system projected on a second pixel two-dimensional coordinate that is a coordinate of each pixel in the second acquired image, and An angle formed by a line segment connecting the second viewpoint and the optical axis of the second imaging device, and the tilt angle calculation unit further includes the three second reference viewpoint projection angles and the three points. Using the relative coordinates of the second reference point, the second imaging plane formed by the second imaging plane, which is the imaging plane of the second acquired image, and the second reference plane including the three second reference points. 2, and the measurement unit calculates a relative positional relationship between the first reference point of the three points and the second reference point of the three points, the first inclination angle, and the second inclination point. May be used to measure the relative coordinates between the target point and any one of the three first reference points and the three second reference points.

  As described above, by using the viewpoint projection angle using two images, the relative positional relationship of the camera at each shooting point is determined without grasping in advance the relative posture relationship of the camera at each shooting point. Relative coordinates between any one of the three reference points and the target point can be measured without doing so.

  Further, the tilt angle calculation unit uses the three first and second reference viewpoint projection angles and the three first and second reference points to use the first viewpoint and the first reference point. Determining the first reference plane including the first reference points of the three points on a straight line connecting the first reference points of the three points projected on the acquired image, and the second viewpoint And determining the second reference plane including the three second reference points on a straight line connecting the three second reference points projected onto the second acquired image Also good.

  Further, the viewpoint projection angle extraction unit further acquires a first target viewpoint projection angle that is the first pixel viewpoint projection angle corresponding to the target point projected on the first acquired image, and A second target viewpoint projection angle that is the second pixel viewpoint projection angle corresponding to the target point projected on the two acquired images, and the measurement unit includes the first target viewpoint projection angle, Calculating a first vector passing through the first viewpoint and the target point projected on the first acquired image in the three-dimensional coordinate system using the first inclination angle; A second passing through the second viewpoint and the target point projected on the second acquired image in the three-dimensional coordinate system using the second target viewpoint projection angle and the second inclination angle. And calculating the vector of the three-dimensional coordinate system using the relative positional relationship. The coordinates of the closest point between one vector and the second vector are calculated as relative coordinates between the target point and one of the three first reference points and the three second reference points. May be.

  Also, at least one of the three first reference points may be different from the three second reference points.

  The target point may be the first viewpoint.

  As described above, by using a single image and using the viewpoint projection angle, the relative coordinates between any of the three reference points and the shooting point can be measured without previously knowing the posture of the camera at the shooting point. can do.

  Further, the tilt angle calculation unit projects the first viewpoint and the first acquired image using the three first reference viewpoint projection angles and the three first reference points. The first reference plane including the three first reference points on a straight line connecting each of the three first reference points may be determined.

  As described above, the tilt angle calculation unit can calculate the posture of the camera at the photographing point.

  Further, the measurement unit corresponds to the two angles in the first reference plane by using two of the three first reference viewpoint projection angles and the first inclination angle. Two vectors passing through two of the three first reference points are calculated, and the coordinates of the closest point of the two vectors are set as one of the three first reference points and the You may calculate as a relative coordinate with an object point.

  Note that the present invention can be realized not only as such a three-dimensional relative coordinate measuring apparatus, but also as a step in the operation of the characteristic components included in such a three-dimensional relative coordinate measuring apparatus. It can also be realized as a measurement method or a program for causing a computer to execute these steps.

  The present invention can provide a three-dimensional relative coordinate measuring apparatus capable of measuring relative coordinates without grasping in advance the position and orientation of a camera at a photographing point.

FIG. 1A is a diagram showing a configuration of a system including a three-dimensional relative coordinate measurement device at the time of three-dimensional relative coordinate measurement according to the present invention. FIG. 1B is a diagram showing a configuration of a system including a three-dimensional relative coordinate measuring apparatus at the time of pixel calibration of an image according to the present invention. FIG. 2 is a block diagram showing a characteristic functional configuration of the three-dimensional relative coordinate measuring apparatus according to the present invention. FIG. 3 is a flowchart showing a procedure for pixel calibration of an image according to the present invention. FIG. 4 is a diagram for explaining a viewpoint projection angle according to the present invention. FIG. 5A is a diagram showing an image when a calibration plate is photographed before distortion correction according to the present invention. FIG. 5B is a diagram showing an image when the calibration plate is photographed after distortion correction according to the present invention. FIG. 6 is a diagram for explaining processing for calculating the distance between the calibration board and the viewpoint according to the present invention. FIG. 7 is a flowchart showing a processing procedure of relative coordinate measurement by the three-dimensional relative coordinate measurement apparatus according to Embodiment 1 of the present invention. FIG. 8 is a diagram three-dimensionally showing the relationship among the viewpoint, imaging surface, reference point, and target point according to Embodiment 1 of the present invention. FIG. 9A is a diagram showing a reference point and a target point projected on the first imaging surface according to Embodiment 1 of the present invention. FIG. 9B is a diagram showing a reference point and a target point projected on the second imaging surface according to Embodiment 1 of the present invention. FIG. 10 is a flowchart showing a processing procedure performed by the tilt angle calculation unit according to the present invention. FIG. 11 is a diagram for explaining the processing shown in the flowchart of FIG. 10 according to the present invention. FIG. 12 is a diagram when FIG. 11 is projected onto the Z ₁ -X ₁ plane according to the present invention. FIG. 13 is a flowchart showing a processing procedure of relative coordinate measurement by the three-dimensional relative coordinate measurement apparatus according to Embodiment 2 of the present invention. FIG. 14 is a diagram three-dimensionally illustrating the relationship between the viewpoint, the imaging surface, and the reference point according to Embodiment 2 of the present invention. FIG. 15 is a diagram illustrating a reference point projected on the first imaging surface according to Embodiment 2 of the present invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, the configuration of a system including a three-dimensional relative coordinate measuring apparatus according to the present invention will be described separately for three-dimensional relative coordinate measurement and image pixel calibration according to an embodiment of the present invention.

  FIG. 1A is a diagram showing a configuration of a system including a three-dimensional relative coordinate measurement device at the time of three-dimensional relative coordinate measurement according to the present invention. Note that FIG. 1A shows a system configuration when acquiring two images from two viewpoints. A three-dimensional relative coordinate measuring apparatus 90 according to the present invention is connected to the camera 10 via a cable 30. The cable 30 is, for example, a USB (Universal Serial Bus) cable. As a result, the three-dimensional relative coordinate measuring apparatus 90 can acquire the image data 60 captured by the camera 10 via the cable 30. The cable 30 may be a cable other than USB. The image data 60 may be acquired wirelessly or via a recording medium.

Camera 10, so that the reference structure 40 and the target structure 50 objects appear together, capturing the different first viewpoint M ₁ and second viewpoint M _2, respectively. Here, M ₁ and M ₂ correspond to the positions of the principal points of the photographing lens of the camera 10. In FIG. 1A, O ₁ and O ₂ written by two-dot chain lines indicate the optical axis of the photographing lens (the optical axis of the camera) when the viewpoint is at M ₁ and M ₂ , respectively. Note that when acquiring one image from one viewpoint, the camera 10 may capture an image from the first viewpoint so that the reference structure 40 is captured. The reference structure 40 has three reference points A, B, and C whose relative coordinates are known in a three-dimensional coordinate system. In other words, the triangle defined by the points A, B, and C is referred to as the reference structure 40.

  The target structure 50 has a target point W whose coordinates are unknown in a three-dimensional coordinate system. In addition, when acquiring one image from one viewpoint, the target structure 50 may not be used.

  FIG. 1B is a diagram showing a configuration of a system including a three-dimensional relative coordinate measuring apparatus at the time of pixel calibration of an image according to the present invention. A three-dimensional relative coordinate measuring apparatus 90 according to the present invention is connected to the camera 10 via a cable 30. The cable 30 is, for example, a USB cable. Thus, the three-dimensional relative coordinate measuring apparatus 90 can acquire the image data 61 captured by the camera 10 via the cable 30. The cable 30 may be a cable other than USB. Further, the image data 61 may be acquired wirelessly or via a recording medium.

  The camera 10 captures an image at the viewpoint M so that the calibration plate 70 is captured. Here, M corresponds to the position of the principal point of the taking lens of the camera 10. In FIG. 1B, O written by a two-dot chain line indicates the optical axis of the photographing lens (the optical axis of the camera) when M has a viewpoint.

  The calibration plate 70 is a transparent plate having high rigidity, and has a grid pattern on the surface. A weight 72 is attached to the calibration plate 70 with a thread 71.

  The three-dimensional relative coordinate measuring device 90 is, for example, a computer. In the present embodiment, a common computer is used for three-dimensional relative coordinate measurement and image pixel calibration, but separate computers may be used. Two or more computers may be used.

  FIG. 2 is a block diagram showing a characteristic functional configuration of the three-dimensional relative coordinate measuring apparatus according to the present invention.

  The three-dimensional relative coordinate measurement system 90 includes a three-dimensional relative coordinate measurement unit 20 and an image pixel calibration unit 80 according to the present invention.

  The three-dimensional relative coordinate measurement unit 20 according to the present invention includes an image acquisition unit 100, a two-dimensional coordinate extraction unit 110, a viewpoint projection angle extraction unit 120, a calculation unit 130, and a display unit 140. Further, the image pixel calibration unit 80 includes an adjustment unit 200.

  The adjustment unit 200 performs distortion correction of image pixels from the image data 61 obtained by imaging the calibration plate 70, the thread 71, and the weight 72 from the viewpoint M, and the viewpoint projection angle of the arbitrary image pixel (pixel) (present invention). Of the first and second pixel viewpoint projection angles).

Image acquisition unit 100, from the first viewpoint M _1, the reference point of the 3-point, or a reference point and the target point of the three points (corresponding to the first and second imaging apparatus of the present invention) camera The image data 60 imaged in (1) is acquired as the first acquired image. The image acquisition unit 100, from the second viewpoint M _2, and the reference point and the target point of the three points to obtain the image data 60 captured by the camera as the second acquired image.

  The two-dimensional coordinate extraction unit 110 extracts three reference points projected on the first acquired image acquired by the image acquisition unit 100, or the two reference points and the two-dimensional coordinates of the target point. In addition, the two-dimensional coordinate extraction unit 110 extracts the three reference points and the two-dimensional coordinates of the target points that are projected on the second acquired image acquired by the image acquisition unit 100.

  The viewpoint projection angle extraction unit 120 holds the viewpoint projection angle of an arbitrary image pixel calculated by the adjustment unit 200 in advance, and the two-dimensional coordinate extraction unit 110 extracts the viewpoint projection angle of the arbitrary image pixel to be held. A viewpoint projection angle corresponding to the two-dimensional coordinates is extracted from the viewpoint projection angles of arbitrary image pixels. The definition of the viewpoint projection angle will be described later with reference to FIG.

  Here, the calculation unit 130 further includes an inclination angle calculation unit 130a, a vector calculation unit 130b, a conversion vector calculation unit 130c, and a relative coordinate measurement unit 130d.

  The tilt angle calculation unit 130a calculates the first tilt angle and the second tilt angle using the relative coordinates of the three reference points and the viewpoint projection angle extracted by the viewpoint projection angle extraction unit 120, respectively.

  The vector calculation unit 130b calculates the first vector and the vector 2 using the viewpoint projection angles extracted by the viewpoint projection angle extraction unit 120, respectively. Further, the vector calculation unit 130b calculates two vectors using the viewpoint projection angle extracted by the viewpoint projection angle extraction unit 120 and the first tilt angle calculated by the tilt angle calculation unit 130a.

  The conversion vector calculation unit 130c uses the first inclination angle and the second inclination angle calculated by the inclination angle calculation unit 130a and the vector 2 calculated by the vector calculation unit 130b to calculate the second vector. calculate.

  The relative coordinate measurement unit 130d uses either the first vector calculated by the vector calculation unit 130b and the second vector calculated by the conversion vector calculation unit 130c to obtain either one of the three reference points and the target point. Measure relative coordinates. In addition, the relative coordinate measuring unit 130d measures the relative coordinates between any of the three reference points and the shooting point using the two vectors calculated by the vector calculating unit 130b.

  The display unit 140 displays, for example, the acquired image acquired by the image acquisition unit 100, displays the two-dimensional coordinates extracted by the two-dimensional coordinate extraction unit 110, or is extracted by the viewpoint projection angle extraction unit 120. Display viewpoint projection angles, and display various calculation results performed by the calculation unit 130.

  Next, the operation of a system including the three-dimensional relative coordinate measuring apparatus according to this embodiment configured as described above will be described.

  As a premise, it is necessary to perform pixel calibration of the image before the three-dimensional relative coordinate measurement unit 20 performs relative coordinate measurement.

FIG. 3 is a flowchart showing a procedure for pixel calibration of an image according to the present invention. The pixel calibration image, set the optical center position C _p is the center point of the imaging plane (S101), performs distortion correction (S102), perspective projection angle of an arbitrary image pixels (pixel) theta (present invention (Corresponding to the first and second pixel viewpoint projection angles) (S103). Here, the viewpoint projection angle θ of an arbitrary image pixel is a two-dimensional coordinate (corresponding to the first and second pixel two-dimensional coordinates of the present invention) of an arbitrary image pixel P on the image, as shown in FIG. a line segment connecting the to) to the arbitrary point P ₃ of the three-dimensional space to be projected and viewpoint M, the camera optical axis O (corresponding to the optical axis of the first and second imaging device of the present invention) It is the angle formed by. In FIG. 4, since the optical axis O and the imaging surface of the camera are orthogonal, and the viewpoint M, and the distance x between the optical center position C _p is the intersection of the optical axis O and the imaging surface of the camera, for example, optical the center position C _p as the origin, by using the two-dimensional coordinates of any image pixel P, it is possible to calculate the perspective projection angle θ of the arbitrary image pixels P.

First, the optical center position _Cp is set (S101). First, a weight 72 is attached by a thread 71 to a transparent plate having high rigidity (hereinafter referred to as a calibration plate 70). The calibration plate 70 is assumed to be kept horizontal by a level or the like. Further, it is assumed that grids are marked on the surface of the calibration plate 70. Next, the calibration plate 70 is imaged by the camera 10 from above. Then, the image pickup surface is (1) a grid around the attachment position of the thread 71 so that the attachment position of the calibration plate 70 and the thread 71 and the attachment position of the weight 72 and the thread 71 overlap. when the display of the eye is adjusted to be symmetrical, the mounting position of the thread 71 on the calibration plate 70 is the optical center position C _p.

  Next, the adjustment unit 200 performs image distortion correction (S102). As shown in FIG. 5A, before distortion correction, the image when the calibration plate 70 is photographed is an image in which the grid is distorted as shown in FIG. 5A due to the characteristics of the photographing lens of the camera 10. The adjustment unit 200 adjusts the captured image to an image as illustrated in FIG. 5B by performing normalization processing on the captured image.

  Finally, the adjustment unit 200 calculates the viewpoint projection angle θ of an arbitrary image pixel P as shown in FIG. 4 (S103).

  FIG. 6 is a diagram for explaining processing for calculating the distance between the calibration board and the viewpoint according to the present invention.

First, the adjustment unit 200 acquires an image of a plate before movement, which is an image after distortion correction obtained by photographing the calibration plate 70 as shown in FIG. 5B. Next, the adjustment unit 200 is a distortion-corrected image obtained by capturing the calibration plate 70 when the calibration plate 70 is moved close to the viewpoint M by the distance y while the calibration plate 70 is kept horizontal. Acquire an image of the later board. Then, the adjustment unit 200 uses the image of the plate before movement and the image of the plate after movement, for example, two-dimensional coordinates of the points P ₁ ′ and P ₂ ′ with the optical center position C _p as the origin. To extract.

Here, a line connecting the viewpoint M and the optical center position C _p, the intersection of the surface of the calibration plate 70 and C _1. Further, the arbitrary point of the calibration plate 70 surface and P _1. In addition, an intersection of a straight line connecting the viewpoint M and P ₁ and the imaging surface is defined as P ₁ ′. Further, corresponding to P ₁ of the calibration plate 70 surface before the movement, the point of the calibration plate 70 surface after the movement and P _2. Also, let P ₂ ′ be the intersection of the straight line connecting the viewpoints M and P ₂ and the imaging surface. Further, let Q be the intersection of a straight line passing through the point P ₂ and the point P ₂ ′ and the surface of the calibration plate 70 before the movement.

At this time, the length d ₁ of the line segment P ₁ Q is expressed by Expression 1.

Here, d ₂ represents the length of the line segment C ₁ P ₁ , d ₃ represents the length of the line segment P ₁ ′ P ₂ ′, and d ₄ represents the length of the line segment C _p P ₁ ′.

Further, the viewpoint projection angle θ ₁ of the point Q is expressed by Expression 2.

  Therefore, the distance z between the calibration plate 70 and the viewpoint M is expressed by Equation 3.

Here, d ₅ indicates the length of the line segment C ₁ Q.

Then, using the distance z between the calibration plate 70 and the viewpoint M, for example, the viewpoint projection angle θ ₂ of P ₁ ′ in FIG.

Viewpoint M and the length x of a line connecting the optical center position C _p can be expressed by equation 5.

  Therefore, the viewpoint projection angle θ of an arbitrary pixel point P can be calculated by Expression 6.

Here, d ₆ indicates the length of the line segment C _p P.

That is, since the two-dimensional coordinates of an arbitrary image pixel P with the optical center position C _p as the origin are known, the viewpoint projection angle θ of the arbitrary image pixel P can be calculated.

In this way, the optical center position _Cp is set, distortion correction is performed, and the viewpoint projection angle θ of an arbitrary image pixel is calculated, whereby the pixel calibration of the image is completed.

  The image pixel calibration method has been described based on the embodiment, but the image pixel calibration method is not limited to this embodiment.

  For example, in the present embodiment, the grid of the grid is written on the calibration plate 70, but the calibration plate may be marked with a known interval.

  In this embodiment, an example of using a thread and a weight at the time of calibration has been described. However, instead of the thread and the weight, the light from the lamp light source passes through a cylindrical structure processed perpendicularly to the calibration plate. A light beam or a laser beam may be used.

  Next, the operation in which the three-dimensional relative coordinate measurement unit 20 performs relative coordinate measurement will be described.

(Embodiment 1)
In the first embodiment, the operation in which the three-dimensional relative coordinate measurement unit 20 measures the relative coordinates between any of the three reference points and the target point using two images will be described.

  FIG. 7 is a flowchart showing a processing procedure of relative coordinate measurement by the three-dimensional relative coordinate measurement apparatus according to Embodiment 1 of the present invention.

First, as shown in FIG. 8, the image acquisition unit 100 acquires a first acquired image and a second acquired image (S110). Here, the first acquired image and the second captured image, from the first viewpoint M ₁ and second viewpoint M _2, the reference point A of the 3-point coordinates are known, B, and C, and coordinates This is image data 60 captured by a camera (corresponding to the first and second imaging devices of the present invention) so that the unknown target point W can be seen together.

  In the present embodiment, for the sake of simplicity of explanation, a case where the three reference points included in the first acquired image and the second acquired image are the same will be described as an example. The three reference points included in the second acquired image may be different.

Next, as illustrated in FIG. 9A, the two-dimensional coordinate extraction unit 110, for example, uses the optical center position C _p1 in the first acquired image as an origin, and the reference points A, B, and B projected on the first acquired image. The coordinates of the points A ₀₁ , B ₀₁ , C ₀₁ that are C (first reference two-dimensional coordinates of three points) are extracted from the first acquired image. In addition, the two-dimensional coordinate extraction unit 110 extracts the coordinates of the point W ₀₁ that is the target point W projected on the first acquired image (first target two-dimensional coordinates) from the first acquired image (S120). . Similarly, as illustrated in FIG. 9B, the two-dimensional coordinate extraction unit 110, for example, uses the optical center position _Cp2 in the second acquired image as an origin and the reference points A, B, and B projected on the second acquired image. The coordinates of the points A ₀₂ , B ₀₂ , and C ₀₂ (three second reference two-dimensional coordinates) that are C are extracted from the second acquired image. Further, 2-dimensional coordinate extracting section 110 extracts the second acquired image point which is the target point W which is projected to W ₀₂ coordinates (second target two-dimensional coordinates) from the second acquired image (S120) .

  The viewpoint projection angle extraction unit 120 holds the viewpoint projection angle of an arbitrary image pixel calculated by the adjustment unit 200 in advance, and the two-dimensional coordinate extraction unit 110 out of the viewpoint projection angles of the arbitrary image pixel to be held. The first and second reference two-dimensional coordinates of the three points extracted by the above and the viewpoint projection angles of arbitrary image pixels corresponding to the first and second target two-dimensional coordinates are set as the three first and second references. The viewpoint projection angle and the first and second target viewpoint projection angles are extracted (S130).

  Then, the inclination angle calculation unit 130a includes first and second imaging surfaces that are imaging surfaces of the first and second acquired images, and a reference plane (first and second reference planes) including a reference point. The formed angle (first and second inclination angles) is calculated (S140).

  FIG. 10 is a flowchart showing a processing procedure performed by the tilt angle calculation unit according to the present invention. Moreover, FIG. 11 is a figure explaining the process shown by the flowchart of FIG. 10 based on this invention. Hereinafter, the process of calculating the first inclination angle will be described mainly based on FIGS. 10 and 11.

First, the tilt angle calculation unit 130a has one of the three reference points A, B, and C whose relative coordinates are known, for example, an X ₁ -Y ₁ -Z ₁ coordinate system with the point A as an origin. (First three-dimensional coordinate system) is set (S141). At this time, the inclination angle calculating unit 130a, X ₁ axis and Y ₁ axis is set to be parallel to the first imaging plane taken from the first viewpoint M _1. The inclination angle calculating unit 130a, Z ₁ axis is set to be the first imaging plane and perpendicular captured from the first viewpoint M _1. Note that the X ₁ -Y ₁ plane at this time is also referred to as a first conversion plane.

Then, the inclination angle calculating unit 130a, a relative coordinate reference point A of the three known, B, set C and equivalent first virtual point _{B 1} and _{C 1} of the coordinates (S142). Here, the point B ₁ corresponds to the point B, and the point C ₁ corresponds to the point C. For example, in the present embodiment, the coordinates of the point B ₁ are set as (B _1x , 0, 0), and the coordinates of the point C ₁ are set as (0, C _1y , 0).

Then, the tilt angle calculation unit 130a finely and separately rotates the points B ₁ and C ₁ by an angle α _1n around the X ₁ axis, an angle β _1n around the Y ₁ axis, and an angle γ _1n around the Z ₁ axis. A plurality of first conversion reference points B _1n ′ (B _1nx ′, B _1ny ′, B _1nz ′) and point C _1n ′ (C _1nx ′, C _1ny ′, C _1nz ′) are calculated ( S143). Here, n = 1, 2, 3,...

Next, the inclination angle calculation unit 130a uses the viewpoint projection angle extracted by the viewpoint projection angle extraction unit 120 to project a plurality of points B _1n ′ and C _1n ′ onto the X ₁ -Y ₁ plane. The point B _1n ″ (B _1nx ″, B _1ny ″, 0) and the point C _1n ″ (C _1nx ″, C _1ny ″, 0), which are the first conversion projection reference points, are calculated (S144). Here, n = 1, 2, 3,...

Here, extracted by the viewpoint projection angle extracting unit 120, _Z 1 -X ₁ side perspective projection angle epsilon _1x of point _{B 01,} a perspective projection angle of _Z 1 -Y ₁ side of the point _{B 01} epsilon _1y, viewpoint projection angle of _Z 1 -X ₁ side of the point _{C 01} φ _1x, the perspective projection angle of _Z 1 -Y ₁ side of the point _{C 01} and phi _1y. Reference point A in the first imaging plane, B, that it is C _{A 01,} B _01, a triangle _A 01 _B 01 _{C 01} connecting the _{C 01,} points A, triangle A connecting the points _{B 1n} "point _{C 1n"} , B _1n ″ C _1n ″ is similar, ∠B _1n ′ M ₁ C _p1 and ∠B _1n ″ M ₁ C _p1 are ε _1x as shown in FIG.

Therefore, B _1nx ″ can be calculated using Equation 7.

Similarly, B _1ny ″, C _1nx ″ and C _1ny ″ can be calculated using Equations 8 to 10.

Finally, the inclination angle calculation unit 130a, the triangle A ₀₁ B ₀₁ C ₀₁ connecting the points A ₀₁ , B ₀₁ , C ₀₁ which are the reference points A, B, C on the first imaging surface, and the points A, B By determining the first transformed projection reference points B ₁ ″ and C ₁ ″ that are most similar to the triangle AB _1n ″ C _1n ″ connecting _{1 n} ″ and the point C _1n ″, the first inclination angle (α ₁ , Β ₁ , γ ₁ ) can be calculated (S145).

As a specific similarity calculation method, the line segment A ₀₁ B ₀₁ is made the same length as the line segment AB _1n ″, and the point B ₀₁ and the point C ₀₁ are moved so that the point A ₀₁ coincides with the point A. Then, the first inclination angle (α ₁ , β ₁ , γ ₁ ) is calculated so that the sum of the length of the line segment B ₀₁ B _1n ″ and the length of the line segment C ₀₁ C _1n ″ is minimized. There is a way.

However, the similarity comparison calculation method is not limited to the above method. For example, when a line segment _A 01 _{B 01,} "in the same length as the line segment _A 01 _{B 01} segment _{AB 1n"} segment _{AB 1n} moving the point _{C 01} so as to coincide with point C The first inclination angles (α ₁ , β ₁ , γ ₁ ) may be calculated so that the deviation amount between ₀₁ and the point C _1n ″ is minimized.

Similarly, the inclination angle calculation unit 130a can calculate a second inclination angle (α ₂ , β ₂ , γ ₂ ) that is an angle formed between the second imaging surface and a plane including the reference point. .

  In this way, the first tilt angle and the second tilt angle are calculated.

  Although the calculation of the tilt angle has been described based on the embodiment, the calculation of the tilt angle is not limited to this embodiment.

For example, in the present embodiment, the tilt angle calculation unit 130a has an X ₁ -Y ₁ -Z ₁ coordinate system in which one of the three reference points A, B, and C whose coordinates are known is the origin. However, the origin may be selected from other than the three reference points whose coordinates are known.

  Further, for example, when there are reference points A, B, and C, the point A is placed at a certain position on the viewpoint projection angle line of the point A, and the point B is at a certain pitch on the viewpoint projection angle line of the point B and in a certain range. When the space is sequentially slid, the point C is obtained by being a similar shape based on the line segment AB by the point B at a certain pitch, and the point C is on the circumference with the line segment AB as an axis. I understand that. The condition of the closest approach on the viewpoint projection angle of the point C may be stored, the point B may be sequentially slid to calculate the closest point of the point C at that time, and the inclination angle obtained when the closest approach is obtained.

  Also, for example, when there are reference points A, B, and C, and the length between the respective points is known, the point A is placed at a certain position on the viewpoint A projection angle line of the point A, and the point B at that time is the point B Since the point C is on the circumference around the line segment AB, the condition closest to the point C projection angle line is memorized. The inclination angle obtained when the point A is closest to the viewpoint projection angle line when the above-described calculation is performed for the point A within a certain range may be used.

  With reference to FIG. 7 again, the processing procedure of relative coordinate measurement by the three-dimensional relative coordinate measurement apparatus according to Embodiment 1 of the present invention will be described.

  In FIG. 7, after the calculation of the tilt angle (S140) is completed, the vector calculation unit 130b calculates the first vector and the vector 2 (S150).

First, the first target two-dimensional coordinates of the point W ₀₁ that is the target point W on the first imaging surface are converted into similar shapes on the first conversion plane where the triangles A, B ₁ ″, C ₁ ″ exist. This point is _{defined as} W ₁ (W _1x , W _1y , 0).

Next, when the point on the line segment connecting the point W ₁ and the first viewpoint M ₁ is W ₁ ′ (W _1x ′, W _1z ′, W _1z ′), the first vector is the viewpoint projection. Using the viewpoint projection angle η _1x of the Z ₁ -X ₁ plane of the point W ₀₁ and the viewpoint projection angles η _1y and W _1z ′ of the Z ₁ -Y ₁ plane of the point W ₀₁ extracted by the corner extraction unit 120. , (W _1x −W _1z ′ tan η _1x , W _1y −W _1z ′ tan η _1y , W _1z ′).

  Similarly, the vector calculation unit 130b can calculate the vector 2.

Then, the conversion vector calculation unit 130c generates a line segment connecting any two points of the points A, B ₁ ″, C ₁ ″, for example, the line segment AB ₁ ″ and the points A, B ₂ ″, C ₂ ″. Among them, the line segment AB ₂ ″ corresponding to the line segment AB ₁ ″ is extracted. Then, the conversion vector calculation unit 130 c uses the first inclination angles (α ₁ , β ₁ , γ obtained by the inclination angle calculation unit 130 a. ₁ ) and the second inclination angle (α ₂ , β ₂ , γ ₂ ), the vector 2 is moved so that the line segment AB ₂ ″ and the line segment AB ₁ ″ overlap with each other. Is calculated (S160).

  Finally, the relative coordinate measuring unit 130d calculates the coordinates of the closest point between the first vector and the second vector, so that the origin A and the target point W, which are any one of the reference points, are calculated. Relative coordinates are calculated (S170).

  Note that the vector calculation unit 130b, the conversion vector calculation unit 130c, and the relative coordinate measurement unit 130d described above correspond to the measurement unit of the present invention.

  In this way, the relative coordinates between any of the reference points and the target point are calculated.

(Embodiment 2)
In the second embodiment, using a single image, the three-dimensional relative coordinate measuring unit 20 makes a relative coordinate between any one of the three reference points and the photographing point (corresponding to the first viewpoint of the present invention). The operation of measuring the will be described.

  FIG. 13 is a flowchart showing a processing procedure of relative coordinate measurement by the three-dimensional relative coordinate measurement apparatus according to Embodiment 2 of the present invention.

First, the image acquisition unit 100 acquires a first acquired image as shown in FIG. 14 (S210). Here, the first acquired image is considerably than the first viewpoint M _1, the reference point A of the 3-point coordinates are known, B, the first imaging device to the camera (the present invention as C objects appear together Is the image data 60 imaged in (1).

Next, as illustrated in FIG. 15, the two-dimensional coordinate extraction unit 110, for example, uses the optical center position C _p1 in the first acquired image as an origin, and the reference points A, B, and B projected on the first acquired image. The coordinates of the points A ₀₁ , B ₀₁ and C ₀₁ (three first reference two-dimensional coordinates) that are C are extracted from the first acquired image (S220).

  The viewpoint projection angle extraction unit 120 holds the viewpoint projection angle of an arbitrary image pixel calculated by the adjustment unit 200 in advance, and the two-dimensional coordinate extraction unit 110 out of the viewpoint projection angles of the arbitrary image pixel to be held. The viewpoint projection angles of arbitrary image pixels corresponding to the three first reference two-dimensional coordinates extracted by the above are extracted as three first reference viewpoint projection angles (S230).

  Then, the tilt angle calculation unit 130a calculates an angle (first tilt angle) formed by the first imaging plane, which is the imaging plane of the first acquired image, and the first reference plane including the reference point ( S240).

  Note that the processing procedure performed by the inclination angle calculation unit 130a when calculating the first inclination angle is the same as S141 to S145 in FIG.

  Further, the vector calculation unit 130b calculates two vectors (S250).

Here, the two vectors, for example, _{B 1} determined in (S145) and "the, and the straight line _{B 1 'B 1} connecting the' _{B 1} corresponding to 'point _{B 1n} at that time" (S145) _{C 1} determined in is that of "a, linear _{C 1 'C 1} connecting the' _{C 1} corresponding to the _{'C 1n} terms of their time".

  Finally, the relative coordinate measuring unit 130d calculates the coordinates of the closest point of the two vectors, thereby calculating the relative coordinates between the point A, which is one of the reference points, and the shooting point (S260). ).

Further, for example, when there are reference points A, B, and C, and the length between each point is known, the point A is placed at a certain position on the viewpoint projection angle line of the point A, and the point B at that time is set to Since the point C is on the circumference around the line segment AB, the condition closest to the point C projection angle line is memorized. The inclination angle obtained when the point A is closest to the viewpoint projection angle line when the above-described calculation is performed for the point A within a certain range may be used. This case, the triangle ABM ₁ connecting the point A and point B and the point _{M 1,} because the length and ∠M ₁ AB and ∠M ₁ BA line segment AB is known, using the triangle ABM _1, the reference point The relative coordinates of any one of the points and the shooting point can be calculated. Further, since ∠AM ₁ B is known from the viewpoint projection angle when the line segment AB is viewed from the viewpoint M ₁ , similarly, using the triangle ABM ₁ , any one of the reference points and the shooting point Relative coordinates can be calculated.

  In this way, the relative coordinates between any of the reference points and the shooting point are calculated.

  Although the three-dimensional relative coordinate measuring apparatus according to the embodiment of the present invention has been described above, the present invention is not limited to this embodiment.

In the present embodiment, the viewpoint projection angle extraction unit 120 holds the viewpoint projection angle of an arbitrary image pixel calculated by the adjustment unit 200 in advance, and selects the two-dimensional coordinates from the stored viewpoint projection angles of the arbitrary image. The reference and target viewpoint projection angles corresponding to the two-dimensional coordinates extracted by the extraction unit 110 have been extracted. However, for example, the viewpoint projection angle extraction unit 120 holds Equation 6, and uses the held Equation 6 to use the two-dimensional coordinates extracted by the two-dimensional coordinate extraction unit 110 and the distortion correction equation as a reference and target. The viewpoint projection angle may be calculated. In this case, the adjustment unit 200, by Equation 5, it is necessary to obtain the distance x between the viewpoint M and the optical center position C _p.

  In the present embodiment, the two-dimensional coordinate extraction unit 110 extracts the three reference points and the two-dimensional coordinates of the target points in the first and second acquired images. For example, the image acquisition unit 100 May extract the two-dimensional coordinates. In this case, the two-dimensional coordinate extraction unit 110 is not necessary.

In the present embodiment, the tilt angle calculation unit 130a calculates the first tilt angle and the second tilt angle by setting two different three-dimensional coordinate systems, but one three-dimensional coordinate system is used. The first inclination angle and the second inclination angle may be calculated by setting only. In this case, the vector calculation unit 130b, a first target viewpoint projection angle using the first inclination angle, calculates a first vector which passes through the point M ₁ and the point W ₀₁ in one of the three-dimensional coordinate system . Further, the vector calculation unit 130b, and a second target viewpoint projection angles by using the second inclination angle, calculates a second vector passing through the point M ₂ and the point W ₀₂ in one of the three-dimensional coordinate system. Therefore, in this case, the conversion vector calculation unit 130c is not necessary.

  In the present embodiment, even when the tilt angle calculation unit 130a calculates the first tilt angle and the second tilt angle by setting two different three-dimensional coordinate systems, the conversion vector calculation unit The calculation may be performed by the vector calculation unit 130b instead of 130c.

  In the present embodiment, shooting is performed from two viewpoints, but shooting may be performed from three or more viewpoints, or the number of reference points whose relative coordinates are known may be increased to four or more. .

  In addition, when using four or more reference points whose relative coordinates are known, it is sufficient that at least three of the four or more reference points are captured in each of the two images. Further, the combination of at least three reference points photographed in each image may be different. Further, it is preferable that four or more reference points are arranged in a three-dimensional manner rather than in the same plane. For example, four or more of the apexes of a triangular pyramid or a quadrangular pyramid can be used as the reference point. Thereby, a wide range and stable angle detection can be performed. For example, when only the above-described three reference points are used, an error is likely to occur if there is no angle between the plane including the three reference points and the viewpoint. On the other hand, for example, by photographing different surfaces of the triangular pyramid from two viewpoints, angle detection can be performed stably even in such a case.

  In this case, the inclination angle is an angle with respect to different reference deformation surfaces (first and second reference planes). The relative coordinate measurement unit 130d calculates the relative coordinates of the target point using the relative positional relationship between the three first reference points and the three second reference points. Specifically, the relative coordinate measuring unit 130d calculates the closest point of two vectors using the relative positional relationship. Further, this relative positional relationship may be calculated when calculating the tilt angle. In other words, the calculated tilt angle may be adjusted to one of the reference planes. Alternatively, the calculated tilt angle may be calculated as an angle unified in the three-dimensional coordinate system.

  Alternatively, the first tilt angle and the second tilt angle may be calculated by performing nonlinear approximation calculation using a large number of reference points whose relative coordinates are known.

Further, for example, among the three reference points A, B, and C whose relative coordinates are known and the points A ₀₁ , B ₀₁ , and C ₀₁ that are the reference points A, B, and C on the first imaging surface, When the points A and A ₀₁ , the points B and B ₀₁ , and the points C and C ₀₁ are not associated with each other, the correspondence can be identified by using a special marker or line. .

  In addition, in order to prevent reversal (turn over) due to rotation of the reference point and calculation of an incorrect solution (tilt angle), a reference point for verification whose relative coordinates are known may be used.

  Further, a restriction (for example, a rotation condition) may be added so as not to calculate an incorrect solution.

  Moreover, you may implement | achieve part or all of the function of the three-dimensional relative coordinate measuring device based on embodiment of this invention, when processors, such as CPU, run a program.

  Furthermore, the connection relationship between the components is exemplified for specifically explaining the present invention, and the connection relationship for realizing the function of the present invention is not limited to this.

  In addition, division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, a single functional block can be divided into a plurality of functions, or some functions can be transferred to other functional blocks. May be. Further, the functions of a plurality of functional blocks having similar functions may be processed by the three-dimensional measuring apparatus in parallel or in time division.

  Furthermore, the present invention may be the above program or a non-transitory computer-readable recording medium on which the above program is recorded. Needless to say, the program can be distributed via a transmission medium such as the Internet.

  In addition, the configuration of the three-dimensional measurement apparatus is for illustration in order to specifically describe the present invention, and the three-dimensional measurement apparatus according to the present invention does not necessarily have all of the above-described configurations. In other words, the three-dimensional measuring apparatus according to the present invention only needs to have a minimum configuration that can realize the effects of the present invention.

  Similarly, the three-dimensional measurement method using the three-dimensional measurement apparatus is for illustration in order to specifically describe the present invention, and the three-dimensional measurement method using the three-dimensional measurement apparatus according to the present invention is described above. It is not necessary to include all of the steps. In other words, the three-dimensional measurement method according to the present invention needs to include only the minimum steps that can realize the effects of the present invention.

  In addition, the order in which the above steps are executed is for illustration in order to specifically describe the present invention, and may be in an order other than the above.

  Also, some of the above steps may be executed simultaneously (in parallel) with other steps.

  Further, various modifications in which the present embodiment is modified within the scope conceived by those skilled in the art are also included in the present invention without departing from the gist of the present invention.

  The present invention can be used when three-dimensional relative coordinate measurement, for example, an interval between two points is automatically measured in an environment where relative coordinates cannot be measured mechanically and electrically.

DESCRIPTION OF SYMBOLS 10 Camera 20 Three-dimensional relative coordinate measuring part 30 Cable 40 Reference | standard structure 50 Target structure 60, 61 Image data 70 Calibration board 71 Yarn 72 Weight 80 Image pixel calibration part 90 Three-dimensional relative coordinate measuring apparatus 100 Image acquisition part DESCRIPTION OF SYMBOLS 110 Two-dimensional coordinate extraction part 120 Viewpoint projection angle extraction part 130 Calculation part 130a Inclination angle calculation part 130b Vector calculation part 130c Conversion vector calculation part 130d Relative coordinate measurement part 140 Display part 200 Adjustment part

Claims (4)

At any three of the three first reference points and the three or more reference points, which are any three of the three or more reference points whose relative coordinates in the three-dimensional coordinate system are known A three-dimensional relative coordinate measuring apparatus that measures relative coordinates between any one of the three second reference points and one target point whose coordinates in the three-dimensional coordinate system are unknown,
The first reference image of the three points and the target point from the first viewpoint are acquired by the first imaging device, and the second second of the three points is acquired from the second viewpoint. An image acquisition unit for acquiring a second acquired image captured by a second imaging device having a reference point and the target point that are the same as or different from the first imaging device;
Information on the first pixel viewpoint projection angle for each pixel , which is calculated in advance , is stored, and the information on the first pixel viewpoint projection angle is used to calculate the three points of the three points projected on the first acquired image. Three first reference viewpoint projection angles that are the first pixel viewpoint projection angles corresponding to three first image reference points that are one reference point, and the first image that is projected onto the first acquired image A first target viewpoint projection angle that is the first pixel viewpoint projection angle corresponding to the target point is acquired, information on a second pixel viewpoint projection angle for each pixel calculated in advance is held, and the first The second pixels corresponding to the three second image reference points that are the second reference points of the three points projected on the second acquired image using the information of the two pixel viewpoint projection angles Three second reference viewpoint projection angles, which are viewpoint projection angles, and the target point projected on the second acquired image And a perspective projection angle extracting unit for acquiring second target viewpoint projection angle is the second pixel viewpoint projection angle to respond,
The first pixel viewpoint projection angle is obtained by calculating each point of the three-dimensional coordinate system projected on the first pixel two-dimensional coordinates, which are coordinates of each pixel in the first acquired image, and the first viewpoint. An angle formed by a connecting line segment and the optical axis of the first imaging device;
The second pixel viewpoint projection angle is obtained by calculating each point of the three-dimensional coordinate system projected on the second pixel two-dimensional coordinate which is a coordinate of each pixel in the second acquired image and the second viewpoint. An angle formed by a connecting line segment and the optical axis of the second imaging device;
further,
Using the three first reference viewpoint projection angles and the relative coordinates of the three first reference points, the first imaging surface, which is the imaging surface of the first acquired image, and the three points A first inclination angle formed with a first reference plane including one reference point is calculated, and the three second reference viewpoint projection angles and the relative coordinates of the three second reference points are used. An inclination angle calculation unit for calculating a second inclination angle formed by a second imaging plane which is an imaging plane of the second acquired image and a second reference plane including the three second reference points; ,
Extending in the direction of the first object viewpoints projection angle, and said first calculating a first straight line passing through the target point to be projected in the acquired image, the direction of the second target viewpoints projection angle And the second straight line passing through the target point projected on the second acquired image is calculated, and the first straight line is calculated using the first inclination angle and the second inclination angle. And the second straight line is converted into the same three-dimensional coordinate system, and the three-dimensional coordinates are obtained by using the relative positional relationship between the three first reference points and the three second reference points. In the system, the coordinates of the closest point between the first straight line and the second straight line are set to be either the first reference point of the three points or the second reference point of the three points and the target point. A three-dimensional relative coordinate measurement apparatus comprising a measurement unit that calculates relative coordinates. The three-dimensional relative coordinate measuring apparatus according to claim 1, wherein at least one of the three first reference points is different from the three second reference points. At any three of the three first reference points and the three or more reference points, which are any three of the three or more reference points whose relative coordinates in the three-dimensional coordinate system are known A three-dimensional relative coordinate measurement method for measuring a relative coordinate between any one of the three second reference points and one target point whose coordinates in the three-dimensional coordinate system are unknown,
The first reference image of the three points and the target point from the first viewpoint are acquired by the first imaging device, and the second second of the three points is acquired from the second viewpoint. An image acquisition step of acquiring a second acquired image captured by a second imaging device in which a reference point and the target point are the same as or different from the first imaging device;
Information on the first pixel viewpoint projection angle for each pixel , which is calculated in advance , is stored, and the information on the first pixel viewpoint projection angle is used to calculate the three points of the three points projected on the first acquired image. Three first reference viewpoint projection angles that are the first pixel viewpoint projection angles corresponding to three first image reference points that are one reference point, and the first image that is projected onto the first acquired image A first target viewpoint projection angle that is the first pixel viewpoint projection angle corresponding to the target point is acquired, information on a second pixel viewpoint projection angle for each pixel calculated in advance is held, and the first The second pixels corresponding to the three second image reference points that are the second reference points of the three points projected on the second acquired image using the information of the two pixel viewpoint projection angles Three second reference viewpoint projection angles, which are viewpoint projection angles, and the target point projected on the second acquired image And a perspective projection angle extracting step of obtaining a second target viewpoint projection angle is the second pixel viewpoint projection angle to respond,
The first pixel viewpoint projection angle is obtained by calculating each point of the three-dimensional coordinate system projected on the first pixel two-dimensional coordinates, which are coordinates of each pixel in the first acquired image, and the first viewpoint. An angle formed by a connecting line segment and the optical axis of the first imaging device;
The second pixel viewpoint projection angle is obtained by calculating each point of the three-dimensional coordinate system projected on the second pixel two-dimensional coordinate which is a coordinate of each pixel in the second acquired image and the second viewpoint. An angle formed by a connecting line segment and the optical axis of the second imaging device;
further,
Using the three first reference viewpoint projection angles and the relative coordinates of the three first reference points, the first imaging surface, which is the imaging surface of the first acquired image, and the three points A first inclination angle formed with a first reference plane including one reference point is calculated, and the three second reference viewpoint projection angles and the relative coordinates of the three second reference points are used. An inclination angle calculating step of calculating a second inclination angle formed by a second imaging plane which is an imaging plane of the second acquired image and a second reference plane including the three second reference points; ,
Extending in the direction of the first object viewpoints projection angle, and said first calculating a first straight line passing through the target point to be projected in the acquired image, the direction of the second target viewpoints projection angle And the second straight line passing through the target point projected on the second acquired image is calculated, and the first straight line is calculated using the first inclination angle and the second inclination angle. And the second straight line is converted into the same three-dimensional coordinate system, and the three-dimensional coordinates are obtained by using the relative positional relationship between the three first reference points and the three second reference points. In the system, the coordinates of the closest point between the first straight line and the second straight line are set to be either the first reference point of the three points or the second reference point of the three points and the target point. A three-dimensional relative coordinate measurement method including a measurement step of calculating as relative coordinates.   The program for making a computer perform the step contained in the three-dimensional relative coordinate measuring method of Claim 3.

Images