JP5147055B2 - Distance measuring device and distance measuring method - Google Patents

Description

  The present invention relates to a distance measurement technique using a passive method, and in particular, based on a single image obtained by photographing a double image reflected on a transparent plate or a double-sided half mirror plate with a single imaging device. The present invention relates to a distance measuring device and a distance measuring method for measuring distance.

  In general, the distance measurement method can be divided into a method using an active method and a method using a passive method. Since the active distance measurement method uses a laser light source or spatial light coding, a large-scale apparatus is required and an expensive system is required, and the measurement time is relatively long.

  On the other hand, stereo distance measurement using two or more cameras has long been used as a passive distance measurement method using only passive detection. In the field of computer vision, three-dimensional geometry has been analyzed, and many variations have been proposed based on this result. For example, a multi-camera (see Non-Patent Document 1) or a multi-baseline (see Non-Patent Document 2) is effective as a method for greatly reducing the ambiguity of association when the system cost does not matter much. .

  However, in the stereo distance measuring method using two or more cameras, a plurality of cameras and photographing lenses are required, so that the measuring device becomes expensive. In addition, since it is necessary to perform calibration for the number of cameras, considerable labor is required for measurement preparation. Further, in actual measurement, it is necessary to capture images for the number of cameras into the computer, and it is necessary to secure image memories corresponding to the number of cameras in the computer. In addition, it is necessary to take images simultaneously in time between the cameras in synchronization.

  Recently, distance measurement has come to be used as a traffic measurement in the ITS field (see Non-Patent Document 3), pedestrian detection (see Non-Patent Document 4), and a sensor in the security field. In addition to reliability, robustness, and cost considerations have become necessary.

The cost for distance measurement increases as the number of cameras used increases. The cost at this time includes not only the camera and the taking lens but also the memory and calculation amount required for image capture and image processing. For this reason, a stereo distance measuring method using one camera has been proposed. Stereo distance measurement methods using a single fixed camera (hereinafter also simply referred to as “monocular distance measurement method”) can be classified into three types: image division method, multiple image method, and multiple exposure method.
SBKang, R. Szeliski, J. Chai, "Handling Occlusions in Dense Multi-View Stereo", Proc. On Computer Vision and Pattern Recognition, Vol. I, p.103-110, December 2001 M. Okutomi and T. Kanade, “A Multiple-Baseline Stereo, IEEE Trans. On Pattern Analysis and Machine Intelligence (IEEE Trans. On Pattern Analysis and Machine Intelligence), Vol.15, No.4, p.353-363, April 1993 Co-authored by Z. Sun, G. Bebis, and R. Miller, “On-Road Vehicle Detection using Optical Sensors: A Review ”, Proc. On IEEE Intelligent Transportation Systems Conference, USA, October 2004 "3D Vision Sensing for Improved Pedestrian Safety" by G.Grubb, A.Zelinsky, L.Nilsson, and M.Rilbe for Improved Pedestrian Safety), Proc. On IEEE Intelligent Vehicles Symposium, p.19-24, June 2004 Jointly written by JMGluckman and SKNayar, “Planar Catadioptric Stereo: Geometry and Calibration”, Proc. On Computer Vision and Pattern Recognition ( Proc. On Computer Vision and Pattern Recognition), Volume 1, p.22-28, June 1999 H. Shim, “Light Weight Multi-View Capturing Projects”, http://amp.ece.cmu.edu/projects/MIRRORARRAY/ Co-authored by DH Lee and IS Kweon, “A Novel Stereo Camera System by a Biprism”, IEEE Transformer on Robotic and Auto Mation (IEEE Trans. On Robotics and Automation), Vol.16, No.5, p.528-541, October 2000 Co-authored by Y. Nishimoto and Y, Shirai, "A Feature-Based Stereo Model using Small Disparities", Proc. On IEEE International Work Shop on Industrial Applications Off Machine Vision and Machine Intelligence (Seikei Symposium), p.192-196, Japan, February 1987 Co-authored by C. Gao and N. Ahuja, “Single Camera Stereo using Planar Parallel Plate”, Proc. On International Conference on Pattern Recognition (Proc. On International Conference on Pattern Recognition), Volume IV, p.108-111, United Kingdom, August 2004 Co-authored by EHAdelson and JYA Wang, “Single Lens Stereo with a Plenoptic Camera”, IEEE Trans. On-Pattern Analysis and Machine Intelligence (IEEE Trans. On Pattern Analysis and Machine Intelligence), Vol.14, No.2, p.99-106, February 1992 Co-authored by M. Amtoun and B. Boufama, “Multibaseline Stereo using a Single-Lens Camera”, Proc. On International Conference on Image Processing (Proc. On International Conference on Image Processing), Volume 1, pages 401-404, September 2003 S.Hiura and T.Matsuyama, “Depth Measurement by the Multi-Focus Camera”, Proc. On Computer Vision and Pattern Recognition (Proc On Computer Vision and Pattern Recognition), p.953-959, June 1998 EP Simoncelli and H. Farid, “Direct Differential Range Estimation using Optical Masks”, Proc. On European Conference on Computer Vision (Proc) On European Conference on Computer Vision), Volume 2, pages 82-93, United Kingdom, April 1996. Kenji Yamada, Hideya Takahashi, Eiji Shimizu, “Three-dimensional shape detection method using coded aperture method”, IEICE Transactions, Vol.11, No.11, p.2986-2994 November 1997 Akihiro Kiyohara, Masato Nusui and Kouji Ikeda, “Distance Measurement Using One Camera and Lens Array”, Proceedings of Image Recognition and Understanding Symposium (MIRU2004), Volume I, p.392-397, 2004 July Masao Shimizu and Masatoshi Okutomi, “Distance Measurement Method Using a Single Camera Using Double Images Reflected on a Transparent Plate”, Information Processing Society of Japan CVIM Journal, Vol. 47, SIG10 (CVIM15), p.131- 142, July 2006 Masao Shimizu and Masatoshi Okutomi, “Distance Measurement with a Single Camera Using Double-Side Half-Mirror Transmission Images,” Proceedings of the 9th Symposium on Image Recognition and Understanding (MIRU2006), p.788-793, 2006 July Masao Shimizu and Masatoshi Okutomi, "Transmission-type reflection stereo-Monocular distance measurement using double-sided half-mirror plate transmission images", Proceedings of the 13th Image Sensing Symposium (SSII2007), p.IN1-10-1-8 , June 2007 T.Whitted, "An Improved Illumination Model for Shaded Display", Communications of the ACM (Volume 23, 6) No., p.343-349, June 1980 JYBouguet, “Camera Calibration Toolbox for Matlab”, http://www.vision.caltech.edu/bouguetj/calib_doc/index.html, 2007 February Z. Zhang, "Flexible Camera Calibration by Viewing a Plane from Unknown Orientations", Plock. On International Conference on Computer Vision (Proc. On International Conference on Computer Vision), Volume 1, p.666-673, Greece, September 1999

  The image division method is to divide an image into a plurality of regions using optical means such as a mirror (see Non-Patent Document 5 and Non-Patent Document 6) and a prism (see Non-Patent Document 7). Basically, it is devised so that images from multiple cameras can be taken simultaneously with one camera, and a method of projecting images taken with multiple cameras with different positions and orientations into a single image It is.

  In the image segmentation method, there is no difference in image density, contrast, color, etc. between cameras. In addition, since a plurality of images that are completely synchronized in time are used, it can be applied to dynamic objects and can also be applied to real-time applications. However, since the number of pixels of one camera is divided and used (that is, the number of pixels of a divided image corresponding to an image from one viewpoint is small), there is a problem that measurement accuracy is lowered. In addition, as in the case of stereo distance measurement using a plurality of cameras, it is necessary to calibrate external parameters of the cameras.

  The multiple image method is a method of capturing a plurality of images while optically moving the position of one camera and estimating the distance to the target. For example, the angle with respect to the optical axis of the parallel flat transparent plate arranged on the camera optical axis is changed (see Non-Patent Document 8), or the angle of the transparent plate remains constant and rotates along the camera optical axis (non- (See Patent Document 9), thereby realizing precise translation of the camera position.

  In the multiple image method, since the camera moves in parallel, it is not necessary to estimate external parameters, and distance calculation is easy. However, in principle, it is impossible to measure the distance to a moving object in order to obtain the distance to the target after shooting a plurality of images. In addition, since the optical means is composed of a mechanism including an operating member, it results in an expensive system for its manufacture, and it takes time-consuming measurement for adjustment.

  The multiple exposure method refers to a single lens aperture (see Non-Patent Document 10 and Non-Patent Document 11), a coded aperture (see Non-Patent Document 12, Non-Patent Document 13, and Non-Patent Document 14), a lens array, and the like. (Refer to Non-Patent Document 15) is a method of obtaining distance information by photographing a light beam passing through different positions into one image.

  In the multiple exposure method, since a plurality of openings are provided, the use efficiency with respect to the light amount is reduced, and the baseline length is limited by the lens openings. In addition, problems such as low measurement accuracy due to ambiguity of corresponding point search in captured images using a lens array, and low contrast in captured images using coded apertures, which are in principle taken by pinholes There is.

  In order to solve the above-described conventional problems, the inventors of the present invention, as a monocular distance measurement method belonging to the multiple exposure method, have a reflection image of a transparent plate (consisting of a front-surface reflection image and a back-surface reflection image, and misalignment). Monocular distance measurement method (hereinafter also referred to simply as “reflective reflection stereo”) (see Non-Patent Document 16), and half mirror coating on both sides of the transparent plate A monocular distance measurement method using a transmission image of a double-sided half mirror plate (a double image composed of a direct image and an internal reflection image and having a positional shift) (hereinafter also simply referred to as “transmission reflection stereo”) (non- (See Patent Document 17).

  Hereinafter, the “reflection reflection stereo” and the “transmission reflection stereo” disclosed in Non-Patent Document 16 and Non-Patent Document 17 proposed by the inventors of the present invention are simply referred to as “reflection stereo”. "

  The “reflection stereo” proposed by the inventors of the present invention does not require a movable mechanism or a special optical member, and does not require a movable mechanism (movable part) to display a double image reflected on a transparent plate or a double-sided half mirror plate. There is a feature that distance measurement can be performed with an inexpensive device configuration and a short measurement time based on one image obtained by photographing with one imaging device (camera) that does not have.

  However, in the “reflection stereo” proposed by the inventors of the present invention, a transparent plate or a double-sided half mirror plate (hereinafter, sometimes referred to collectively as a “reflecting plate”) whose both surfaces are parallel planes. The distance is measured using a single image taken through an ideal plane-parallel reflector, and a sufficiently distant object (landscape, etc.) is used when performing calibration of reflection stereo. Images were taken, the non-parallelism of the reflector was estimated from the displacement between the double images, and objects (carpets, walls, etc.) with sufficient texture at a short distance were photographed to estimate the direction of the constraint line. It is assumed that the estimated calibration parameters (external parameters of the distance measuring device including the reflector and the camera) are constant regardless of the position on the image.

  However, when an acrylic transparent plate or glass plate that is easily available as a commercial product and is inexpensive is used as a reflector, the acrylic transparent plate or glass plate is not an ideal parallel flat reflector, that is, a building. As can be seen by observing the reflection of the window glass, the surface of the acrylic transparent plate or glass plate is not completely flat, and both surfaces are not completely parallel, so when performing distance measurement with reflection stereo, Since an external parameter that is invariant to the upper position is used, there is a problem that the measurement result includes an error and the accuracy of distance measurement is limited.

  In order to solve this problem, the inventors of the present invention invented the present invention based on the “reflection stereo” proposed by the inventors.

  In short, the present invention has been made under the circumstances as described above, and the object of the present invention is to create a double image reflected on a wavy (non-parallel and not completely flat) transparent plate or double-sided half mirror plate, Provided is a distance measuring device and a distance measuring method capable of performing high-precision distance measurement with an inexpensive device configuration and a short measurement time based on one image obtained by photographing with one imaging device. There is to do.

  The present invention relates to a distance measuring device, and the object of the present invention is to provide a non-parallel and non-perfect plane of a reflection plate, a single imaging unit, and a double image with a positional deviation reflected on the reflection plate. A double image displacement calculation unit for calculating a displacement between double images from an image obtained by photographing with the imaging means, and a plurality of images obtained by photographing a plurality of calibration planes at a known distance as double images. A calibration processing unit that estimates external parameters representing the shape, orientation, refractive index of the reflector, and the distance along the optical axis from the camera optical center of the imaging means to the reflector, and the estimated external A parameter and a distance calculation unit for calculating a distance to the measurement target from one image obtained by imaging a measurement target at an unknown distance as a double image, and the reflector is a locally non-parallel plane And the above carry External parameters estimated by configuration unit is effectively achieved by having a different value depending on the point of interest on the image obtained by photographing a double image.

Another object of the present invention is to provide a distance measuring device, which is a non-parallel and non-perfect plane of a reflection plate, a single imaging unit, and a double image with a positional deviation reflected on the reflection plate. A double-image displacement calculation unit that calculates a displacement between double images from an image obtained by photographing with the imaging unit, and a plurality of calibration planes at which the double-image displacement calculation unit is at a known distance. A calibration processing unit for estimating an external parameter of the distance measuring device based on the displacement between the double images calculated using a plurality of images taken as double images and the known distance; and between the double images The distance to the measurement object based on the displacement between the double images calculated by using one image obtained by photographing the measurement object at an unknown distance as a double image and the estimated external parameter. And a distance calculation unit for calculating An area (divided plane area) necessary to form a double image corresponding to a point of interest by approximating a reflector corresponding to the vicinity of the point of interest on the captured image with a plane plate having non-parallelism. ) Has an external parameter having the same value as that of the attention point, and the external parameter estimated by the calibration processing unit has a different value depending on the attention point, and the distance calculation unit differs depending on the attention point. By using an external parameter having a value, or the reflecting plate is a transparent plate or a double-sided half mirror plate having a half mirror coating on both sides of the transparent plate, or the imaging means is a solid-state imaging The imaging apparatus using an element, or the external parameter includes the normal vector for both surfaces of the reflector and the reflection from the camera optical center of the imaging means. Up to the distance along the optical axis, the thickness of the reflecting plate, and the relative refractive index of the reflecting plate with respect to the air, or in the calibration processing unit, , External parameter image position
Is approximated by a two-dimensional cubic function
Here, Φ _j (j = 1, 2,..., 10) is a seven-element coefficient vector corresponding to the external parameter, and the coefficient vector Φ _j is a number of image positions.
External parameters estimated for
Use
Or the distance calculation unit, based on the displacement between the double images for a certain point of interest and the external parameter for the point of interest, and the line of sight corresponding to the point of interest The line of sight corresponding to the position of the double image corresponding to the point of interest is obtained, and the distance to the intersection position of the obtained two lines of sight is the distance to the measurement object, or the imaging means In order to obtain the absolute distance from the camera optical center to the calibration plane, the known distance D _zi of the calibration plane is _expressed as D _zi = D _i + D _e [mm] (i = 0, 1,... , 12, also, _{D i.} representing a known distance until i th calibration plane) Distant absolute triplicate distance error _D e = 0, assuming ± 25 [mm], Respectively external parameters
And the residual sum E _d (D _e ) in each estimation is obtained as in the following equation:
An absolute distance error that minimizes E _d (D _e ) by applying the sum of the obtained residuals E _d (−25), E _d (0), E _d (25) to a quadratic function.
The
Error based on the estimated absolute distance
The absolute distance is calculated based on the known distance and the external parameter is estimated again by the calibration processing unit using the calculated absolute distance.

  Furthermore, the present invention relates to a distance measuring method in a distance measuring device composed of a single non-parallel and non-perfect reflecting plate and a single imaging means, and the object of the present invention is to the reflecting plate. A double image displacement calculating step for calculating a displacement between double images from an image obtained by photographing a reflected double image with the imaging means, and a plurality of calibration planes at a known distance. Calibration for estimating external parameters representing the shape, posture, refractive index of the reflecting plate and the distance along the optical axis from the camera optical center of the imaging means to the reflecting plate from a plurality of images taken as double images A processing step; a distance calculating step for calculating a distance to the measurement target from one image obtained by capturing the estimated external parameter and a measurement target at an unknown distance as a double image; The external parameters approximated by a locally non-parallel plane and the external parameters estimated in the calibration processing step have different values depending on the point of interest on the image obtained by capturing the double image. Is achieved.

Another object of the present invention is a distance measuring method in a distance measuring apparatus comprising a single non-parallel and non-perfect flat reflecting plate and a single imaging means, the position reflected on the reflecting plate. A double-image displacement calculation step for calculating a displacement between double images from an image obtained by photographing a double image with a shift by the imaging means, and a known distance by the displacement calculation step between double images. Calibration processing step for estimating an external parameter of the distance measuring device based on the displacement between the double images calculated using a plurality of images obtained by capturing a plurality of calibration planes as a double image and the known distance Based on the displacement between the double images calculated using one image obtained by photographing the measurement object at an unknown distance as a double image in the displacement calculation step between the double images and the estimated external parameter. A distance calculating step for calculating a distance to the measurement object, and approximating a reflecting plate corresponding to the vicinity of the point of interest on the image obtained by photographing the double image with a flat plate having non-parallelism, A region (divided plane region) on the reflector necessary for forming a double image corresponding to a point has an external parameter having the same value as that of the target point, and the external parameter estimated by the calibration processing step is , Having different values depending on the point of interest, and in the distance calculating step, by using an external parameter having a different value depending on the point of interest, or the reflecting plate is formed on the transparent plate or both surfaces of the transparent plate. By being a double-sided half mirror plate with a half mirror coating, or by being an imaging device in which the imaging means uses a solid-state imaging device, or the external parameter The normal vector for both surfaces of the reflector, the distance along the optical axis from the camera optical center of the imaging means to the reflector, the thickness of the reflector, and the relative refraction of the reflector to air In the calibration processing step, the external parameter is set to the image position as shown in the following formula.
Is approximated by a two-dimensional cubic function
Here, Φ _j (j = 1, 2,..., 10) is a seven-element coefficient vector corresponding to the external parameter, and the coefficient vector Φ _j is a number of image positions.
External parameters estimated for
Use
Based on the weighted least square method, or in the distance calculating step, based on the displacement between the double images for a certain point of interest and the external parameter for the point of interest, the line of sight corresponding to the point of interest The line of sight corresponding to the position of the double image corresponding to the point of interest is obtained, and the distance to the intersection position of the obtained two lines of sight is the distance to the measurement object, or the imaging means In order to obtain the absolute distance from the camera optical center to the calibration plane, the known distance D _zi of the calibration plane is _expressed as D _zi = D _i + D _e [mm] (i = 0, 1,... , 12, also, _{D i} represents the known distance to the i th calibration plane.) Distant absolute triplicate distance error _D e = 0, assuming ± 25 [mm] Te, each external parameters
And the residual sum E _d (D _e ) in each estimation is obtained as in the following equation:
An absolute distance error that minimizes E _d (D _e ) by applying the sum of the obtained residuals E _d (−25), E _d (0), E _d (25) to a quadratic function.
The
Error based on the estimated absolute distance
The absolute distance is calculated based on the known distance, and the external parameter is estimated again by the calibration processing step using the calculated absolute distance.

  According to the present invention, a single imaging device and a commercially available inexpensive reflector such as an acrylic transparent plate or glass plate are used, and the “swelled reflector” is not represented by the “divided plane model of reflector”. By approximating with a plane plate with parallelism, and by using external parameters that change depending on the position on the image, not only enables accurate distance measurement in a short measurement time with an inexpensive device configuration, There is an original excellent effect that false detection of the displacement between the double images can be reduced.

Hereinafter, the principle of distance measurement using reflection stereo and the best mode for carrying out the present invention will be described with reference to the drawings.

<1> Principle of Distance Measurement by Reflection Stereo First, here, a transparent plate or a double-sided half mirror plate (hereinafter sometimes referred to collectively as a “reflecting plate”) whose both surfaces are completely parallel and one image pickup. The principle of distance measurement by “reflection stereo” proposed by the inventors of the present invention, which is composed of a device (as an example, a CCD camera, hereinafter simply referred to as “camera”) will be briefly described (Non-Patent Document 16). And non-patent document 17).

  That is, in the following, the reflected image or the transmitted image of the transparent plate or the double-sided half mirror plate becomes a double image, the displacement between the double images varies depending on the distance to the target (measurement target), and between the double images. It will be described that there is a constraint on displacement.

  Note that an imaging device that uses a solid-state imaging device may be used as an imaging device used in reflection stereo. Therefore, the imaging device is not limited to a CCD camera. For example, a CMOS camera using a CMOS image sensor may be used. .

The “distance to the target (measurement target)” in the present invention means a distance to the target along the line of sight. Specifically, when performing distance measurement using reflective reflection stereo, the line of sight is reflected on the surface of the transparent plate, but the “distance to the target (measurement target)” refers to the distance to the target along the reflected line of sight Means. On the other hand, when performing distance measurement using transmission-type reflection stereo, the line of sight is refracted by the double-sided half mirror plate, so the “distance to the target (measurement target)” means the double-sided half from the optical center of the imaging device (camera). This means the distance to the object along the line of sight to the mirror plate.

<1-1> Reflective Reflection Stereo Here, first, the principle of distance measurement using the reflective reflection stereo will be described.

  FIG. 1 is a schematic diagram for explaining the principle of distance measurement by reflection-type reflection stereo using a reflection image of a transparent plate (a double image composed of a front-surface reflection image and a back-surface reflection image and having a positional shift).

As shown in FIG. 1, the angle disparity theta _s between the surface reflection image I _s and rear surface reflection image I _r, the relative refractive index of the transparent plate to air n, the thickness of the transparent plate (hereinafter, simply "thickness also referred to as ".) d, it varies with the distance D _o of the incident angle theta _i of front surface reflection _image, and a transparent plate to the measurement target (object).

Measurement object and the camera optical center (lens optical center), and by considering the light rays on the plane (incidence plane) passing through the normal vector of the transparent plate, between the front surface reflection image I _s and rear surface reflection image I _r A basic relationship between the angle parallax θ _s (angle parallax between double images) and the distance to the measurement target will be described. Since this plane (incident surface) includes the normal vector of the transparent plate, reflection and refraction of light rays coming from the measurement object can be described on the incident surface.

A coordinate system (ξ, υ) is set with the origin at which the light beam from the measurement object is reflected on the transparent plate. The position of the measurement target is (−D _o sinθ _i , D _o cosθ _i ), and the position of the camera optical center is (D _c sinθ _i , D _c cosθ _i ).

In the reflection type reflection stereo, the distance (D _o + D _c ) to the measurement target along the line-of-sight direction is expressed by the following formula 1 because the length of the surface reflection and the back reflection is equal when projected onto the ξ axis. be able to.

Note that the angular parallax θ _s can be obtained by measuring the corresponding position between the double images (hereinafter also simply referred to as “displacement between double images”) on the image obtained by capturing the double image of the transparent plate. . Further, in order to displace the double image on a certain constraint line, a one-dimensional search may be performed as in stereo vision using epipolar constraint. Further, when the distance to the measurement object along the line of sight _(D o ₊ D _c) is infinity, theta _{s =} 0, and the surface reflection image I _s and rear surface reflection image I _r is an exact match.

<1-2> Transmission-type Reflection Stereo Next, the principle of distance measurement using transmission-type reflection stereo will be described.

  FIG. 2 shows a transmission image (consisting of a direct image and an internal reflection image) of a double-sided half-mirror plate (hereinafter simply referred to as “double-sided half-mirror plate”) having a half-mirror coating on both sides of a transparent plate. It is a schematic diagram for demonstrating the distance measurement principle by the transmission-type reflection stereo using a certain double image).

As shown in FIG. 2, the angular parallax θ _s between the direct image I _d and the primary internal reflection image I _i1 is the relative refractive index n of the double-sided half mirror plate with respect to air, as in the reflection type reflection stereo. It varies depending on the thickness d of the double-sided half mirror plate, the incident angle θ _i of the direct image, and the distance D _o from the double-sided half mirror plate to the measurement object.

Similar to the reflection-type reflection stereo, by considering the light ray on the incident surface, the angle parallax θ _s (angle parallax between the double images) between the direct image I _d and the primary internal reflection image I _i1 The basic relationship with the distance to the measurement object will be described.

A coordinate system (ξ, υ) with the origin of the direct image as the origin is set. The position of the camera optical center is defined as (D _c sinθ _i , D _c cosθ _i ).

In transmissive reflection stereo, the distance (D _o + D _i + D _c ) to the measurement object along the line-of-sight direction is equal in length when the direct image and the path of the primary internal reflection image are projected onto the ξ axis. Can be expressed by the following formula 2.

Note that the angular parallax θ _s can be obtained by measuring the corresponding position between the double images (the displacement between the double images) on the image obtained by capturing the double image of the double-sided half mirror plate. Further, in order to displace the double image on a certain constraint line, a one-dimensional search may be performed as in stereo vision using epipolar constraint.

<1-3> How to Find Distance to Measurement Object Measured from the displacement between double images in the reflective reflection stereo described in <1-1> above and the transmissive reflection stereo described in <1-2> above. In order to obtain the distance to the object, in addition to the lens focal length f / δ of the camera, the following five parameters corresponding to external parameters in stereo vision (hereinafter simply referred to as “external parameters of reflection stereo” or “ "Conventional external parameters"). However, it is assumed that the lens distortion and the image center position are corrected from the image obtained by capturing the double image using the camera internal parameter estimated by the existing method.

As "external parameter of reflection stereo"
Normal vector of reflector (transparent plate or double-sided half mirror plate)
(Ie, parameters n ₁ , n ₂ ),
A distance D _co (ie, parameter D _co ) along the optical axis from the camera optical center to the reflector (transparent plate or double-sided half mirror plate),
Thickness d (that is, parameter d) of the reflecting plate (transparent plate or double-sided half mirror plate),
There is a relative refractive index n (that is, parameter n) of the reflecting plate (transparent plate or double-sided half mirror plate) with respect to air.

  If these five conventional external parameters are used, the distance from the displacement between the double images to the measurement object can be calculated as follows.

As shown in FIG. 3, a camera coordinate system with the camera optical center as the origin O is set. Normal vector of reflector (transparent plate or double-sided half mirror plate)
Further, the lens focal length (distance between the origin O and the image plane) with the pixel interval of the CCD as a unit is defined as f / δ.

A point on the image plane (hereinafter, also simply referred to as “attention point”)
The position of the double image corresponding to
The unit direction vector
As a result of searching on the constraint line of
Displacement (displacement between double images)
Suppose it is found at the position of. However,
Represents the intersection position of the constraining straight line. Δ is the point of interest
Represents the distance [pixel] along the constraint straight line. At this time, the angular parallax θ _s between the double images in FIGS. 1 and 2 can be obtained by the following equation (3).

Also noteworthy
The incident angle θ _i (see FIGS. 1 and 2) to the reflection plate (transparent plate or double-sided half mirror plate) can be obtained by the following equation (4).

By using the angular parallax θ _s between the double images obtained by Equation 3 and the incident angle θ _i obtained by Equation 4 in Equations 1 and 2, respectively, the displacement between the double images is calculated.
The distance from the object to the measurement object can be obtained.

  A double image can be considered as an image in which substantially the same image (signal) having a positional shift is overlapped. Such an autocorrelation function of an image (signal) has a maximum at a position corresponding to a positional shift between images (signals) in addition to the maximum at the origin.

Therefore, even if there is no edge or feature point in the image obtained by capturing the double image, this misregistration can be obtained from the secondary maximum position of the autocorrelation function if there is any pattern. In the case of transmissive reflection stereo, the brightness differs between the double images, so that the displacement between the double images is detected using the characteristics near the secondary maximum position (see Non-Patent Document 18). ).

<2> Distance measuring apparatus and distance measuring method according to the present invention Hereinafter, the best mode for carrying out the present invention will be described in detail.

  In the present invention, a transparent plate or a double-sided half mirror plate that is non-parallel and not completely flat (hereinafter, for convenience of explanation, a “transparent plate or double-sided half-mirror plate that is not parallel and not completely flat” is simply referred to as a “reflective plate having a wave” It is possible to perform high-precision distance measurement with an inexpensive device configuration and a short measurement time based on a single image obtained by photographing a double image shown in FIG. The present invention relates to a distance measuring device and a distance measuring method.

  The function of the distance measuring device according to the present invention can be broadly divided into a calibration processing function and a distance measuring function. The calibration processing function is to capture a plurality of planes (calibration planes) at known distances as double images, and use these double images to external parameters (hereinafter simply referred to as “external parameters”) of the distance measuring device. It is also a function that estimates "."

  The distance measurement function (distance calculation function) is an external parameter estimated by the calibration processing function and a single image (image data) obtained by photographing a measurement object at an unknown distance as a double image. ) To calculate (measure) the distance to the measurement target.

  In the distance measurement device according to the present invention, the calibration processing function and the distance measurement function (distance calculation function) are switched by the switching signal.

  In other words, it is necessary to first calibrate the distance measuring device before actually measuring the distance to the measurement object using the distance measuring device according to the present invention, that is, external parameters of the distance measuring device. Need to be estimated. Then, after the distance measurement apparatus is calibrated (after external parameters are obtained), the distance measurement apparatus of the present invention is used to actually measure the distance.

  FIG. 4 is a block configuration diagram showing an embodiment of a distance measuring device according to the present invention. As shown in FIG. 4, the distance measuring device 1 of the present invention includes a reflecting plate 10, an imaging unit 20, a double image displacement calculating unit 30, a calibration processing unit 40, and a distance calculating unit 50. The double image displacement calculated by the double image displacement calculation unit 30 is input to the calibration processing unit 40 or the distance calculation unit 50 by a switching signal. It has become. Although not shown, the distance measuring device 1 of the present invention includes a storage unit such as a memory.

  In the present invention, a single non-parallel transparent plate or double-sided half mirror plate is used as the reflecting plate 10. In this embodiment, a commercially available inexpensive acrylic transparent plate is used as the reflecting plate 10.

  In addition, one imaging device is used as the imaging means 20. In this embodiment, a CCD camera equipped with a CCD image sensor (hereinafter simply referred to as “camera”) is used as the imaging means 20. However, any imaging device using a solid-state imaging device may be used as the imaging means 20 used in the present invention. Therefore, the imaging unit 20 is not limited to a CCD camera. For example, a CMOS camera equipped with a CMOS image sensor may be used. good.

  In the distance measuring device 1 of the present invention, when the distance measuring device 1 itself is calibrated, first, a double image in which a plurality of planes (calibration planes) at known distances are reflected on the reflecting plate 10 is imaged. The image (image data) obtained by photographing in (2) is input to the double-image displacement calculation unit 30, and then the double-image displacement calculated by the double-image displacement calculation unit 30 is changed to a switching signal ( Is input to the calibration processing unit 40 by the calibration processing unit switching signal), and the calibration processing unit 40 estimates the external parameter based on the displacement between the double images and the known distance (true distance). Then, the estimated external parameter is input to the distance calculation unit 50. In this way, the calibration process of the distance measuring device 1 is completed. That is, the calibration method in the distance measuring apparatus of the present invention is performed according to the above procedure.

  And when measuring the distance to the measurement object in an unknown distance using the calibrated distance measurement apparatus 1, in the distance measurement apparatus 1, first, the measurement object in the unknown distance is applied to the reflecting plate 10. A single image (image data) obtained by photographing the reflected double image with the imaging means 20 is input to the inter-double-image displacement calculating unit 30, and then calculated by the inter-double-image displacement calculating unit 30. The distance between the double images is input to the distance calculating unit 50 by a switching signal (switching signal to the distance calculating unit), and the distance calculating unit 50 measures the object based on the displacement between the double images and the external parameters. The distance to is calculated (measured). In this way, the distance measurement of the distance measuring device 1 is completed. That is, the distance measuring method according to the present invention is performed in the above procedure.

In the distance measuring device 1 of the present invention, the external parameters estimated by the calibration processing unit 40 are not input to the distance calculating unit 50 but are stored in the storage means (memory) provided in the distance measuring device 1 itself. You may do it. In such a case, it goes without saying that the distance calculation unit 50 needs to read the external parameter stored in the storage means (memory) before calculating (measuring) the distance to the measurement target.

<3> Reflector plate and distance measurement method in the distance measurement device according to the present invention As described in <2>, in the distance measurement device according to the present invention, as the reflection plate, a transparent plate that is not parallel and not perfectly flat or A double-sided half mirror plate, that is, a wavy transparent plate or a double-sided half mirror plate is used. Hereinafter, the “swelled transparent plate or double-sided half mirror plate” is also simply referred to as “swelled reflector”.

  A commercially available inexpensive acrylic transparent plate or glass plate is used as the reflecting plate of the distance measuring device according to the present invention. “Waviness” exists on the surface of such a highly transparent acrylic transparent plate or glass plate. That is, the surfaces of the acrylic transparent plate and the glass plate are not completely flat, and both surfaces of the acrylic transparent plate and the glass plate are not parallel.

In the present invention, a reflecting plate with waviness is locally approximated by a plane (that is, using a divided plane model of a wavy reflecting plate), and the principle (algorithm) of the ray tracing method (see Non-Patent Document 19) is used. It is characterized by measuring the distance to the measurement object with high accuracy in a short time.

<3-1> Dividing Plane Model of Reflecting Plate Hereinafter, for convenience of explanation, “a reflecting plate with undulation” is also simply referred to as a “reflecting plate”.

  The “divided plane model of the reflector”, which can be said to be the greatest feature of the present invention, is to approximate the reflector corresponding to the vicinity of the point of interest on the image obtained by photographing the double image with a plane plate having non-parallelism. means.

  FIG. 5 is a schematic diagram for explaining a divided plane model of the reflector in the present invention. In FIG. 5, a transparent plate is used as the reflecting plate. However, the present invention is not limited to this, and it goes without saying that a double-sided half mirror plate can be used as the reflecting plate.

As shown in FIG.
A reflecting plate (transparent plate) corresponding to the vicinity is approximated by a flat plate having non-parallelism. That is, in the present invention, the region on the reflector (on the transparent plate) necessary for forming a double image corresponding to a certain point of interest on the image (that is, the divided plane region in FIG. 5) is the point of interest. And can be expressed with the same external parameters.

  As compared with the conventional external parameters described in <1-3>, the external parameters of the present invention have normal vectors for both surfaces of the reflector as follows.

That is, as an external parameter of the distance measuring device according to the present invention,
Normal vectors for both surfaces of the reflector
(Ie, parameters n _R1 , n _R2 , n _S1 , n _S2 ),
A distance D _co (ie, parameter D _co ) along the optical axis from the camera optical center to the reflector,
Reflector thickness d (ie, parameter d),
There is a relative refractive index n (ie, parameter n) of the reflector with respect to air.

These seven external parameters are the points of interest on the image
Different value depending on
have.

  Approximating the reflecting plate corresponding to the vicinity of the point of interest with a flat plate having non-parallelism corresponds to first-order approximation of the surface shape of the reflecting plate having waviness.

In the distance measuring device of the present invention (described in <5-1>) having the transmissive configuration used in the experiment (using a double-sided half mirror plate), when the entire image is projected on a reflecting plate (double-sided half mirror plate) About 45 × 40 [mm]. Corresponds to the attention area (attention point) when the image size is 640 × 480 [pixel], the attention area size (attention point) is 20 × 20 [pixel], and the displacement between the double images is 10 [pixel]. An area (divided plane area) on the reflector necessary for forming a double image is about 1.4 × 2.5 [mm]. Even in a commercially available acrylic transparent plate or glass plate, the surface shape can be approximated to a first order if it is a region having such a small area (divided plane region of about 1.4 × 2.5 [mm]). it can.

<3-2> Ray tracing method In the present invention, the “divided plane model of the reflector” is used to approximate the “reflector with undulation” with a plane plate having non-parallelism.

  Thus, in the present invention, since the reflector having non-parallelism is used, the analytical distance calculation method (distance measurement method) used in the conventional reflection stereo as described in <1> is used. Even if it is used, it is difficult to calculate the distance to the measurement target (distance measurement).

  Therefore, in the present invention, the ray tracing method is used to obtain the line of sight corresponding to the point of interest and the line of sight corresponding to the position of the double image corresponding to the point of interest, respectively. ) Is the distance to the measurement target.

  Here, as a preparation for the distance measurement method according to the present invention described in <3-3>, a calculation method of the ray tracing method will be described.

In the ray tracing method (see Non-Patent Document 19), the path along which the ray coming toward the optical center of the camera follows can be traced by using the following three types of calculation elements.

Calculation element 1: Intersection of ray and plane As shown in FIG.
Direction vector through
A straight line and a point
Unit normal vector containing
Intersection with the plane of
Can be expressed by Equation 5 below.

Calculation element # 2: Regular reflection vector in the plane Unit normal vector as shown in FIG.
Direction vector incident on the plane of
Regular reflection direction vector
Can be expressed by Equation 6 below.

Calculation element # 3: Refraction vector in the plane As shown in FIG. 6, the unit normal vector
Direction vector incident on the plane of
Refraction direction vector into the material with relative refractive index n for
Can be expressed by Equation 7 below.

However,
Further, when k _f <1, it represents a situation where the light is incident on the glass and refracted.

<3-3> Distance Measurement Method in Present Invention Here, the distance measurement method in the present invention will be described in detail. That is, a method for calculating the distance to the measurement object in the distance measuring device of the present invention using a reflector that is locally approximated by a plane (using a divided plane model of the reflector) will be described in detail.

In short, in the present invention, a certain point of interest
External parameters for
As described in detail in <3-3-1> and <3-3-2> by measuring (calculating) the displacement between the double images (the positional deviation between the double images). , The ray tracing method is used to determine the distance to the object to be measured using these functions (external parameters)
And (a function of displacement between double images).

<3-3-1> When using the distance measuring device of the present invention having a reflective configuration Here, the distance measuring device of the present invention having a reflective configuration refers to the present invention using a transparent plate as a reflective plate. It means a distance measuring device.

  When the distance measuring device of the present invention having a reflective configuration is used, the distance to the measurement target is calculated as follows.

As shown in Fig. 7 (A), the point of interest on the image plane
The object to be measured as a surface reflection image is a straight line represented by the following equation 8 with t _S as a parameter.
Exists on.

However,
Is established. Also,
Are the point normal on the transparent plate surface and the unit normal vector of the transparent plate surface, respectively.
Is the origin (camera optical center). And
Is an external parameter
include
With
It can be expressed.

As shown in Fig. 7 (B), the same measurement object
Displacement with respect to
Is detected, the measurement object photographed as a back-surface reflection image is a straight line represented by the following equation 9 using t _R as a parameter.
Exists on.

However,

Is established. Also,
Are the point normal on the back side of the transparent plate and the unit normal vector of the back side of the transparent plate,
Is the origin (camera optical center). And
Is an external parameter
include
With
It can be expressed.
The position of the intersection with, that is, two straight lines
The position where the sum of squares of the distance to the minimum is the position of the measurement target
It is.

However,
Is established.

Position of the measurement target expressed by the above formula 10
Represents the three-dimensional coordinates of the measurement target in the camera coordinate system, and the Y coordinate
Is the distance to the measurement object.

Therefore, in the distance measuring device of the present invention having the reflective configuration, the distance to the measurement object is calculated (measured) as described above.

<3-3-2> When the distance measuring device of the present invention having a transmissive configuration is used Here, the distance measuring device of the present invention having a transmissive configuration is a book using a double-sided half mirror plate as a reflector. It means the distance measuring device of the invention.

  When the distance measuring device of the present invention having a transmissive configuration is used, the distance to the measurement target is calculated as follows.

As shown in FIG. 8 (A), the point of interest on the image plane
The measurement object corresponding to the direct image of a straight line expressed by the following Expression 11 to the t _D and parametric
Exists on.

However,
Is established. Also,
Are the points on the exit light side and the incident light side surface of the double-sided half mirror plate,
Are unit normal vectors on the surfaces of the exit light side and the incident light side of the double-sided half mirror plate, respectively.
Is the origin (camera optical center). And
Is an external parameter
include
With each
It can be expressed.

As shown in FIG. 8B, the internal reflection image corresponding to the direct image is
Displacement with respect to
Can be detected, the measurement object corresponding to the internal reflection image is a straight line represented by the following equation 12 using t _I1 as a parameter.
Exists on.

However,
Is established. Also,
Is the origin (camera optical center).
Intersection point, that is, two straight lines
The position where the sum of squares of the distance at
It is.

However,
Is established.

The position of the measurement target expressed by the above formula 13
Represents the 3D coordinates of the measurement target in the camera coordinate system, and the Z coordinate
Is the distance to the measurement object.

Therefore, in the distance measuring device of the present invention having a transmissive configuration, the distance to the measurement object is calculated (measured) as described above.

<3-4> Displacement constraint between double images in distance measuring device of the present invention The displacement constraint between double images in the distance measuring device of the present invention is different from the displacement constraint between double images in the conventional reflection stereo.

That is, the number 9
And the number 12
Displacement used to determine
That is, the displacement shown in FIG. 7B and FIG. 8B (displacement between double images)
Is the displacement between the double images in the conventional reflection stereo shown in <1-3> and FIG.
Is different.

In the present invention, the split plane model of the reflector is used, that is, the point of interest
Because the corresponding reflector in the vicinity is non-parallel,
There is a restraint.

However,
Is the point of interest
Represents the displacement when the distance to the measurement object is infinite,
Represents a unit direction vector of the constraint line. Δ is the position
Represents the distance [pixel] along the constraint straight line. And
The external parameter is as follows
Ask from.

In the case of using the distance measuring device of the present invention having a transmissive configuration, a certain point of interest
About external parameters
And double image displacement
To the object to be measured

Can be requested. Conversely, the distance to the measurement target
Displacement between double images
Can be obtained as a one-parameter optimization problem.

Distance to measurement target
Gradually increasing the displacement between the double images
Displacement when the change of the image becomes sufficiently small, the displacement between the double images corresponding to the infinity distance
And Further, the unit direction vector of the constrained straight line from the displacement between the double images corresponding to a plurality of distances.
Ask for.

Similarly, even when the distance measuring device of the present invention having a reflective configuration is used, in the same manner as the distance measuring device of the present invention having a transmissive configuration,
Can be requested. However, as the distance to the measurement target, the Y coordinate of the three-dimensional coordinates of the measurement target, that is,
Is used.

Note that
External parameters for
Is obtained using a calibration method (external parameter estimation method) in the distance measuring device of the present invention described in <4>.

<4> Calibration Method in Distance Measurement Device of the Present Invention Here, the calibration method (external parameter estimation method) and the function expression of the external parameter in the distance measurement device of the present invention will be described in detail.

<4-1> External Parameter Estimation Method As already described in <2>, in the distance measurement device of the present invention, the calibration processing unit estimates the external parameter based on the displacement between the double images and the known distance. In other words, the calibration processing unit performs calibration (calibration processing) of the distance measuring device.

  The calibration method in the distance measuring apparatus of the present invention, that is, the external parameter estimation method is performed in the following procedure.

  First, in the present invention, in order to estimate external parameters, a known distance to a plane (hereinafter also simply referred to as a “calibration plane”) and a double image in which the calibration plane is reflected on a reflecting plate are imaged. The displacement between the double images on the image obtained by photographing with the above is used.

The distance D to the calibration plane is the distance D _y along the Y-axis from the camera optical center to the calibration plane in the distance measuring device of the present invention having the reflective configuration, and the distance of the present invention having the transmissive configuration. In the measurement apparatus, the distance _Dz is along the Z axis from the camera optical center to the calibration plane.

  Hereinafter, a calibration method (external parameter estimation method) in the distance measuring apparatus of the present invention having a transmission type configuration will be described.

A point of interest on the image
For, between a double image and the corresponding distance D _z to the calibration plane displacement
If M is found, that is,
Is obtained as in the case of a general camera calibration method (see Non-Patent Document 21) as follows.
Can be estimated.

However,
Is the external parameter estimate. The nonlinear objective function
The second term of is that the external parameter to be estimated is the design value of the external parameter
It is a stabilization term to keep it from moving far away from it. The weighting factor α is experimentally determined as small as possible. Since each element of the external parameter influences each other, in most cases, it does not converge stably unless a stabilization term and an appropriate initial value are given. And this nonlinear objective function
For the minimization, the conjugate gradient method was used with the design values of the external parameters as initial values.

Displacement between double images
Can be measured as the second maximum position of the normalized autocorrelation function without using any displacement constraint between the double images if there is a suitable texture of sufficient density on the calibration plane. Displacement between these double images
Since there are two degrees of freedom, in order to estimate the seven external parameters, observation of M ≧ 4 (imaging and calculation of displacement between double images) is required.

<4-2> Functional expression of external parameters in the present invention Since a commercially available acrylic transparent plate or glass plate having a wave is designed as a parallel flat plate in terms of product performance, in the present invention, such an acrylic transparent plate is used. Even if the attention point (attention point position) on the image of the double image reflected on the plate or glass plate changes, the change in the external parameter is small, and the change in the attention point (attention point position) Suppose that the change of the external parameter is continuous.

  Considering such a continuous change of the external parameter, the external parameter estimated for a specific position on the image (hereinafter also simply referred to as “image position”) is used as it is, and the storage means (memory) of the distance measuring device is used as it is. ) Is less efficient.

Therefore, in the present invention, the external parameter is expressed by a function, that is, the external parameter is represented by the image position as shown in the following Expression 15.
Is approximated by a two-dimensional cubic function.

Here, Φ _j (j = 1, 2,..., 10) is a seven-element coefficient vector corresponding to seven external parameters. These coefficient vectors Φ _j have a number of image positions.
External parameters estimated for
Is obtained by the weighted least square method as shown in the following equation (16).

However, in Equation 16, the residual when the external parameter is estimated using Equation 14
As a weight, the influence of the calibration result of an image position having a large residual is reduced.

Displacement between double images corresponding to infinite distance
And unit direction vector of constrained line
Is calculated in advance by the method described in <3-4> using the estimated external parameters. And like the external parameters, the calculation result
Is also expressed as a function, that is, the calculation result
Is approximated by a two-dimensional cubic function with respect to the image position. In the distance calculation process performed by the distance calculation unit 50, a coefficient vector of an approximate function is used.

In addition, the present invention is not limited to approximating the external parameter by a two-dimensional cubic function with respect to the image position. For example, the external parameter can be expanded by Fourier series. The expression that the thickness is 0th order and the overall non-parallelism is 1st order is intuitive and easy to understand. Actually, the external parameters can be almost expressed by approximation up to the second order, and the third order coefficient is quite small. Further, it has been confirmed that the residual of Equation 16 when such approximation is performed is sufficiently small.

<4-3> Comparison between the calibration method in the present invention and the calibration method in the conventional reflection stereo The calibration method in the conventional reflection stereo (see Non-Patent Document 16) is invariant to the position on the image. Seven external parameters were used. Difference between normal vectors from the center of the image to both surfaces of the transparent plate when using reflective reflection stereo
Or, when transmissive reflection stereo is used, the difference between the normal vectors on both surfaces of the double-sided half mirror plate from the center of the image
The average of the normal vectors on both surfaces, and D _co , d, and n used the design values as they were.

  Further, in the conventional reflection stereo calibration method (hereinafter, also simply referred to as “conventional calibration method”), the displacement between the double images corresponding to the infinite distance is obtained from the center portion of the image, and both of the reflectors are obtained. It was also used to obtain the surface normal vector difference. The constraining straight line approximates that it intersects at one point on the image plane (but outside the image), and the intersection position is obtained from the image.

On the other hand, in the calibration method according to the present invention, the external parameter that changes depending on the position on the image is estimated, and the displacement constraint between the double images is obtained from the estimated external parameter. For this reason, according to the present invention, not only the distance measurement accuracy can be improved, but also more accurate displacement constraint between the double images can be used, so that erroneous detection of the displacement between the double images can be reduced.

As described above, the distance measuring device and the distance measuring method according to the present invention have been described in detail. However, the double image displacement calculating unit, the calibration processing unit (calibration method), and the distance in the distance measuring device of the present invention are described. The calculation unit (distance measurement method) can be implemented by software (computer program) using a computer system, and can be implemented by ASIC (Application Specific Integrated Circuit), GPU (Graphics Processing Unit), or FPGA (Field Programmable Gate). Of course, it can also be implemented by hardware such as Array).

<5> Experimental Results Here, a calibration experiment using real images is performed to show the effectiveness of the present invention. That is, the effectiveness of the calibration method according to the present invention was examined with respect to the conventional calibration method. In <5-1> to <5-3>, the distance measuring device of the present invention having a transmissive configuration is used, and in <5-4>, the distance measuring device of the present invention having a reflective configuration is used. did.

<5-1> Device Configuration of Distance Measuring Device of the Present Invention Having a Transmission Type Configuration As shown in FIG. 9, the double-sided half mirror plate is fixed to 45 degrees with respect to the camera optical axis and attached to the housing. . The reflectance of the half mirror is 75% on both surfaces. For the camera, Flea (Grayscale VGA) manufactured by PointGray was used. The design values of the double-sided half mirror plate are D _co = 66.2 [mm], d = 12.0 [mm], and n = 1.49. The double-sided half mirror plate was crafted so that the upper right thickness on the image was about 0.26 [mm] larger.

In addition, before incorporating the camera into the housing, the internal parameters of the camera were obtained using a publicly available calibration tool (see Non-Patent Document 20). The lens focal length measured by the CCD pixel interval is f / δ = 1255.7. Distortion distortion was removed from the image using the distortion parameter obtained simultaneously.

<5-2> Calibration in the Present Invention FIG. 10 is a diagram showing a state in which calibration is performed on the distance measuring apparatus of the present invention having the transmissive configuration in FIG.

  As shown in FIG. 10, the distance measuring device of the present invention having a transmissive configuration was attached to a parallel movement stage, and a calibration plane perpendicular to the lens optical axis was photographed while moving at an interval of 50 [mm]. A random texture with sufficient density was pasted on the calibration plane so that the secondary maximum position of the normalized autocorrelation function could be easily detected.

  FIG. 11 shows an example of a normalized autocorrelation function for a double image. As can be seen from FIG. 11, two secondary maxima appear symmetrically with respect to the origin, but the displacement direction of the internal reflection image can be determined by a simple preliminary experiment, so one of these displacements is the displacement between the double images. Used as.

  With respect to the entire image obtained by photographing the calibration plane, attention areas of 35 × 35 [pixels] were set at 24 × 18 (= 432) locations, and displacement between the double images was obtained for these attention areas. The number of attention areas (432 locations) is a sufficient number for obtaining the ten parameters of Expression 15 by the weighted least square method. The larger the size of the region of interest is, the more stable the second maximum of the autocorrelation function can be detected.

  However, since the internal reflection image is slightly deformed with respect to the direct image, an extremely large region of interest size is instead reduced to a secondary maximum value. Considering the above, the attention area size was experimentally determined. The displacement between the double images is not always obtained for all the set attention areas due to the chance that the texture is small. Therefore, the direction and magnitude of the obtained double-image displacement are compared with the surroundings, and the double-image displacement determined to be erroneous is not used. Similar to 15, approximation was performed with a two-dimensional cubic function, and the displacement detection error between the double images was smoothed in a sense.

  The distance to the calibration plane was increased from about 400 [mm] to about 1000 [mm] every 50 [mm]. The change in distance (50 [mm]) with respect to the calibration plane can be accurately controlled by the translation stage equipped with the distance measuring device of the present invention.

  However, since it is difficult to accurately know the optical center position (camera optical center) of the photographing lens of the camera incorporated in the housing, it is impossible to accurately know the absolute distance to the calibration plane.

Therefore, in the present invention, the distance D _zi to the calibration plane in Equation 14 is set to D _zi = D _i + D _e = 400 + 50 i + D _e [mm] (i = 0, 1,..., 12 and D _i is represents the known distance to the i-th calibration plane.) Assuming three kinds of absolute distance errors D _e = 0, ± 25 [mm], the external parameters
And the total sum E _d (D _e ) of the residuals in each estimation was obtained as in the following Expression 17.

An error in absolute distance that minimizes E _d (D _e ) by applying the sum of residuals E _d (−25), E _d (0), E _d (25) calculated based on Equation 17 to a quadratic function
Was estimated as shown in Equation 18 below.

In this experiment,
It became.

And the estimated absolute distance error
And the known distance to the calibration plane, the absolute distance from the camera optical center to the calibration plane is calculated. Using the calculated absolute distance, the external parameter is estimated again by the calibration processing unit of the distance measuring apparatus of the present invention.

  FIG. 12 shows the plate thickness distribution and refractive index distribution of the double-sided half mirror plate estimated by the calibration of the present invention. In FIG. 12, black circles in the graph indicate external parameters (thickness and refractive index of the double-sided half mirror plate) estimated with respect to the position on the image, and the mesh in the graph is approximated by the function of Formula 15. External parameters (thickness and refractive index of double-sided half mirror plate) are shown.

As can be seen from FIG. 12, the difference between the estimated external parameter and the external parameter approximated by the function is large in the vicinity of the horizontal position of −160 [pixels]. This is because the influence on the weighted least square method was small. The residual of Formula 14 is considered to be due to an error in estimating the displacement between the double images. When attention is paid to the plate thickness distribution in FIG. 12A, the plate thickness in the upper right part is increased by about 0.25 [mm], and it is clear that the intentionally deformed shape can be accurately estimated.

<5-3> Measurement of distance to calibration plane in the present invention In the present invention, the distance to the calibration plane was re-measured using the estimated external parameter.

  FIGS. 13A and 13B show the average (indicated by a circle) and standard deviation (indicated by a bar) of distance measurement results with respect to the true distance to the calibration plane, and FIG. ) And FIG. 13D show distance maps for a distance of 700 [mm].

In the conventional method (FIGS. 13A and 13C), there is a deviation according to the distance, and the standard deviation is also large. This is because the displacement constraint between the double images is inaccurate, and thus there are many false detections of the displacement between the double images. In contrast, in the present invention (FIGS. 13B and 13D), it can be clearly seen that there is no deviation and the standard deviation is small.

<5-4> Calibration and shape measurement by the distance measuring device of the present invention having a reflective configuration In this experiment, the distance measuring device of the present invention having a reflective configuration was used. In the distance measuring device of the present invention having a reflective configuration, a reflecting plate using a half mirror with a reflectance of 50% on the front surface and a mirror with a reflectance of 100% on the back surface is used. Whereas a transparent plate without a coating is used as a reflection plate, a reflection plate using a half mirror with a reflectance of 50% on the front surface and a mirror with a reflectance of 100% on the back surface (hereinafter simply referred to as “reflection plate”). There is a feature that the reflected image of becomes brighter.

The reflecting plate is fixed at 45 [degrees] with respect to the camera optical axis, and the design values are _Dco = 26.2 [mm], d = 12.0 [mm], and n = 1.49. The cameras and internal parameters used are the same as those used in <5-1> and their internal parameters.

  FIG. 14 shows the plate thickness distribution and refractive index distribution of the reflector estimated by calibration in the distance measuring device of the present invention having a reflective configuration. In FIG. 14, black circles in the graph indicate external parameters (thickness and refractive index of the reflector) estimated with respect to the position on the image, and the mesh in the graph is an external that is approximated by the function of Formula 15. Parameters (thickness and refractive index of the reflector) are shown. Compared with the plate thickness distribution of FIG.

  Similar to FIG. 12, in FIG. 14, there is a part where the difference between the estimated external parameter and the external parameter approximated by the function is large. This is because the residual of Expression 14 is large, and therefore the weighted minimum of Expression 16 is used. This is because the influence on the square method is small.

  FIG. 15A and FIG. 15B show the average of measurement results (shown by ●) and standard deviation (shown by bars) with respect to the true distance to the calibration plane, and FIG. FIG. 15D shows a distance map for the distance 700 [mm].

  In the conventional method (FIGS. 15A and 15C), as in FIG. 13, there is a deviation according to the distance, and the standard deviation is also large. This is because the displacement constraint between the double images is inaccurate, and thus there are many false detections of the displacement between the double images. In contrast, in the present invention (FIGS. 15B and 15D), it can be clearly seen that there is no deviation and the standard deviation is small.

  FIG. 16 shows the shape measurement results measured by the conventional method and the present invention (distance measuring device having a reflective configuration). FIG. 16A shows a measurement object used for distance measurement, and FIG. 16B shows a captured double image. In the distance measuring device of the present invention having a reflective configuration, the image captured by the camera is turned upside down. Incidentally, this double image shown in FIG. 16 (B) was taken by attaching the entire photographing apparatus (entire camera) to a tripod upside down. FIGS. 16C and 16D show distance maps, and FIGS. 16E and 16F show bird's-eye views.

  In the conventional methods (FIG. 16C and FIG. 16E), there is a problem that the background shape cannot be measured. In addition, although the true value of the distance to the measurement target is unknown, the measured distance is different from the present invention (FIG. 16D and FIG. 16F). On the other hand, in the present invention, the shape of the background can also be measured, and the height of the measured shape is close to the true value.

It is a schematic diagram for demonstrating the distance measurement principle by reflection type reflection stereo. It is a schematic diagram for demonstrating the distance measurement principle by transmissive | pervious reflection stereo. It is a schematic diagram for demonstrating the restraint plane and restraint straight line in reflection stereo. It is a block block diagram which shows one Embodiment of the distance measuring device which concerns on this invention. It is a schematic diagram for demonstrating the division | segmentation plane model of the reflecting plate in this invention. It is a schematic diagram for demonstrating the ray tracing method used for this invention. It is a schematic diagram for demonstrating the distance measuring method in the distance measuring device of this invention which has a reflection type structure. It is a schematic diagram for demonstrating the distance measuring method in the distance measuring device of this invention which has a transmissive | pervious structure. It is a figure which shows an example of the external view of the distance measuring device of this invention which has a transmissive | pervious structure. It is a figure which shows the mode at the time of calibrating with respect to the distance measuring device of this invention which has the transmissive | pervious structure of FIG. It is a figure which shows an example of the normalization autocorrelation function with respect to a double image. It is a graph which shows the board thickness distribution and refractive index distribution of the double-sided half mirror board estimated by the calibration of this invention shown in FIG. It is a figure which shows the average of the distance measurement result to the calibration plane, the standard deviation, and distance map which were measured by the conventional method and this invention (distance measuring device which has a transmission type structure of FIG. 9). It is a graph which shows the board thickness distribution and refractive index distribution of the double-sided half mirror board estimated by the calibration in the distance measuring device of this invention which has a reflection type structure. It is a figure which shows the average of the distance measurement result to the calibration plane measured by the conventional method and this invention (distance measuring device which has a reflection type structure), a standard deviation, and a distance map. It is a figure which shows the shape measurement result measured by the conventional method and this invention (distance measuring device which has a reflection type structure).

Explanation of symbols

1 Distance measuring device 10 Reflector (transparent plate or double-sided half mirror plate)
20 Imaging means 30 Double image displacement calculation unit 40 Calibration processing unit 50 Distance calculation unit

Claims (16)

A reflector that is non-parallel and not perfectly flat;
One imaging means;
A double-image displacement calculating unit that calculates a double-image displacement from an image obtained by photographing a double image having a positional deviation reflected on the reflector with the imaging unit;
From a plurality of images taken as a double image of a plurality of calibration planes at a known distance, along the optical axis from the camera optical center of the imaging means to the reflector, and the shape, orientation, refractive index of the reflector A calibration processing unit for estimating an external parameter representing the measured distance;
A distance calculation unit that calculates a distance to the measurement target from the estimated external parameter and a single image obtained by capturing a measurement target at an unknown distance as a double image;
With
Approximating the reflector in a locally non-parallel plane;
The distance measuring apparatus according to claim 1, wherein the external parameter estimated by the calibration processing unit has a different value depending on a point of interest on an image obtained by capturing a double image. A distance measuring device,
A reflector that is non-parallel and not perfectly flat;
One imaging means;
A double-image displacement calculating unit that calculates a double-image displacement from an image obtained by photographing a double image having a positional deviation reflected on the reflector with the imaging unit;
Based on the displacement between the double images calculated by using the plurality of images obtained by photographing the plurality of calibration planes at a known distance as a double image, and the known distance, A calibration processing unit for estimating external parameters of the distance measuring device;
Based on the displacement between the double images calculated by using the single image obtained by photographing the measurement object at an unknown distance as a double image and the estimated external parameter, A distance calculation unit for calculating the distance to the measurement target;
With
An area on the reflector necessary for forming a double image corresponding to the target point by approximating the reflector corresponding to the vicinity of the target point on the image obtained by capturing the double image with a plane plate having non-parallelism. (Split plane area) has an external parameter with the same value as the point of interest,
The external parameter estimated by the calibration processing unit has a different value according to the point of interest, and the distance calculation unit uses an external parameter having a different value according to the point of interest. apparatus.   The distance measuring device according to claim 2, wherein the reflecting plate is a transparent plate or a double-sided half mirror plate in which a half mirror coating is applied to both sides of the transparent plate.   The distance measuring apparatus according to claim 2, wherein the imaging unit is an imaging apparatus using a solid-state imaging device.   The external parameters include normal vectors for both surfaces of the reflector, a distance along the optical axis from the camera optical center of the imaging means to the reflector, the thickness of the reflector, and the thickness of the reflector The distance measuring device according to any one of claims 2 to 4, comprising a relative refractive index with respect to air. The calibration processing unit converts the external parameter to the image position as shown in the following equation.
Is approximated by a two-dimensional cubic function
Here, Φ _j (j = 1, 2,..., 10) is a seven-element coefficient vector corresponding to the external parameter, and the coefficient vector Φ _j is a number of image positions.
External parameters estimated for
Use
6. The distance measuring device according to claim 5, wherein the distance measuring device is obtained by a weighted least square method.   The distance calculation unit corresponds to the line of sight corresponding to the attention point and the position of the double image corresponding to the attention point based on the displacement between the double images with respect to a certain attention point and the external parameter for the attention point. The distance measuring device according to claim 5 or 6, wherein the line of sight is obtained, and the distance to the intersection position of the obtained two lines of sight is the distance to the measurement target. In order to obtain the absolute distance from the camera optical center of the imaging means to the calibration plane,
The known distance D _zi of the calibration plane is defined as D _zi = D _i + D _e [mm] (i = 0, 1,..., 12, and D _i is a known distance to the i-th calibration plane. 3), and assuming that there are three types of absolute distance errors D _e = 0, ± 25 [mm], the external parameters
And the residual sum E _d (D _e ) in each estimation is obtained as in the following equation:
An absolute distance error that minimizes E _d (D _e ) by applying the sum of the obtained residuals E _d (−25), E _d (0), E _d (25) to a quadratic function.
The
Based on
Estimated absolute distance error
And the absolute distance based on the known distance and
The distance measuring device according to claim 5, wherein the external parameter is estimated again by the calibration processing unit using the calculated absolute distance. A distance measuring method in a distance measuring device composed of a non-parallel and non-perfect plane and a single imaging means,
A double-image displacement calculating step for calculating a double-image displacement from an image obtained by photographing a double image having a positional deviation reflected on the reflecting plate with the imaging means;
From a plurality of images taken as a double image of a plurality of calibration planes at a known distance, along the optical axis from the camera optical center of the imaging means to the reflector, and the shape, orientation, refractive index of the reflector Calibration processing step for estimating an external parameter representing the measured distance;
A distance calculating step of calculating a distance to the measurement target from the estimated external parameter and a single image obtained by capturing a measurement target at an unknown distance as a double image;
Have
Approximating the reflector in a locally non-parallel plane;
The distance measurement method according to claim 1, wherein the external parameter estimated in the calibration processing step has a different value depending on a point of interest on an image obtained by capturing a double image. A distance measuring method in a distance measuring device composed of a non-parallel and non-perfect plane and a single imaging means,
A double-image displacement calculating step for calculating a double-image displacement from an image obtained by photographing a double image having a positional deviation reflected on the reflecting plate with the imaging means;
Based on the displacement between double images calculated using a plurality of images obtained by capturing a plurality of calibration planes at a known distance as double images by the displacement calculation step between the double images, and the known distance, A calibration process step for estimating external parameters of the distance measuring device;
Based on the displacement between the double images calculated using one image obtained by photographing the measurement object at an unknown distance as a double image by the displacement between the double images and the estimated external parameter, A distance calculating step for calculating the distance to the measurement target;
Have
An area on the reflector necessary for forming a double image corresponding to the target point by approximating the reflector corresponding to the vicinity of the target point on the image obtained by capturing the double image with a plane plate having non-parallelism. (Split plane area) has an external parameter with the same value as the point of interest,
The external parameter estimated by the calibration processing step has a different value depending on the point of interest, and the distance calculation step uses an external parameter having a different value depending on the point of interest. Method.   The distance measuring method according to claim 10, wherein the reflecting plate is a transparent plate or a double-sided half mirror plate in which a half mirror coating is applied to both sides of the transparent plate.   The distance measuring method according to claim 10 or 11, wherein the imaging means is an imaging device using a solid-state imaging device.   The external parameters include normal vectors for both surfaces of the reflector, a distance along the optical axis from the camera optical center of the imaging means to the reflector, the thickness of the reflector, and the thickness of the reflector The distance measuring method according to any one of claims 10 to 12, comprising a relative refractive index with respect to air. In the calibration processing step, the external parameter is set to the image position as shown in the following equation.
Is approximated by a two-dimensional cubic function
Here, Φ _j (j = 1, 2,..., 10) is a seven-element coefficient vector corresponding to the external parameter, and the coefficient vector Φ _j is a number of image positions.
External parameters estimated for
Use
The distance measurement method according to claim 13, wherein the distance is obtained by a weighted least square method.   In the distance calculation step, the line of sight corresponding to the point of interest and the position of the double image corresponding to the point of interest are determined based on the displacement between the double images for the point of interest and the external parameter for the point of interest. The distance measurement method according to claim 13 or 14, wherein the line of sight is obtained, and the distance to the intersection position of the two lines of sight obtained is set as the distance to the measurement target. In order to obtain the absolute distance from the camera optical center of the imaging means to the calibration plane,
The known distance D _zi of the calibration plane is defined as D _zi = D _i + D _e [mm] (i = 0, 1,..., 12, and D _i is a known distance to the i-th calibration plane. 3), and assuming that there are three types of absolute distance errors D _e = 0, ± 25 [mm], the external parameters
And the residual sum E _d (D _e ) in each estimation is obtained as in the following equation:
An absolute distance error that minimizes E _d (D _e ) by applying the sum of the obtained residuals E _d (−25), E _d (0), E _d (25) to a quadratic function.
The
Based on
Estimated absolute distance error
And the absolute distance based on the known distance and
The distance measurement method according to claim 13, wherein the external parameter is estimated again by the calibration processing step using the calculated absolute distance.