JP4734630B2 - Distance measurement method using double image projected on transparent plate - Google Patents

Description

  The present invention relates to a distance measurement method using a passive technique, and in particular, based on a single image obtained by photographing a double image reflected on a transparent plate with a single imaging device having no movable part. The present invention relates to a distance measurement method that can perform distance measurement with an inexpensive apparatus configuration and a short measurement time.

  In general, the distance measurement method can be divided into a method using an active method and a method using a passive method. Since the active distance measurement method uses a laser light source or spatial light coding, a large-scale apparatus is required and an expensive system is required, and the measurement time is relatively long.

  On the other hand, stereo distance measurement using two or more cameras has long been used as a passive distance measurement method using only passive detection. In the field of computer vision, three-dimensional geometry has been analyzed, and many variations have been proposed based on this result. For example, a multi-camera (see Non-Patent Document 1) or a multi-baseline (see Non-Patent Document 2) is effective as a method for greatly reducing the ambiguity of association when the system cost does not matter much. .

  However, in the stereo distance measuring method using two or more cameras, a plurality of cameras and photographing lenses are required, so that the measuring device becomes expensive. In addition, since it is necessary to perform calibration for the number of cameras, considerable labor is required for measurement preparation. Further, in actual measurement, it is necessary to capture images for the number of cameras into the computer, and it is necessary to secure image memories corresponding to the number of cameras in the computer. In addition, it is necessary to take images simultaneously in time between the cameras in synchronization.

  Recently, distance measurement has come to be used as a traffic measurement in the ITS field (see Non-Patent Document 3), pedestrian detection (see Non-Patent Document 4), and a sensor in the security field. In addition to reliability, robustness, and cost considerations have become necessary.

The cost for distance measurement increases as the number of cameras used increases. The cost at this time includes not only the camera and the taking lens but also the memory and calculation amount required for image capture and image processing. For this reason, a stereo distance measuring method using one camera has been proposed. Stereo distance measurement methods using one fixed camera can be classified into three types: image division method, multiple image method, and multiple exposure method.
Co-authored by SBKang, R. Szeliski, and J. Chai, “Handling Occlusions in Dense Multi-View Stereo”, Proc. On Computer Vision and Pattern Recognition (Proc. On Computer Vision and Pattern Recognition), Volume I, p.103-110, December 2001 M. Okutomi and T. Kanade, “A Multiple-Baseline Stereo, IEEE Trans. On Pattern Analysis and Machine Intelligence (IEEE Trans. On Pattern Analysis and Machine Intelligence), Vol.15, No.4, p.353-363, April 1993 Co-authored by Z.Sun, G. Bebis, and R. Miller, “On-Road Vehicle Detection using Optical Sensors: A Review” , Proc. On IEEE Intelligent Transport Systems Conference, USA, October 2004 Co-authored by G. Grubb, A. Zelinsky, L. Nilsson, and M. Rilbe, “3D Vision Sensing for Improved Pedestrian Safety (3D Vision Sensing for ImprovedPedestrian Safety), Proc. on IEEE IntelligentVehicles Symposium, p.19-24, June 2004 JMGluckman and SKNayar, “Planar Catadioptric Stereo: Geometry and Calibration”, Proc. On Computer Vision and Pattern Recognition ( Proc. On Computer Vision and Pattern Recognition), Volume 1, p.22-28, June 1999 H. Shim, “Light Weight Multi-View Capturing Projects”, http://amp.ece.cmu.edu/projects/MIRRORARRAY/ Co-authored by DHLee and IS Kweon, “A Novel Stereo Camera System by aBiprism”, IEEE Trans. On Robotic and Automation ( IEEE Trans. On Robotics and Automation), Vol.16, No.5, p.528-541, October 2000 Co-authored by Y. Nishimoto and Y, Shirai, “A Feature-Based Stereo Modelusing Small Disparities”, Proc. On IEEE International Workshop On Industrial Applications Off Machine Vision and Machine Intelligence (Proc. On IEEE International Workshop on Industrial Applications of Machine Vision and Machine Intelligence), Seiken Symposium, p.192-196, Japan, February 1987 Co-authored by C. Gao and N. Ahuja, “Single Camera Stereo using Planar Parallel Plate”, Proc. On International Conference on Pattern Recognition (Proc. On International Conferenceon Pattern Recognition), Volume IV, p.108-111, United Kingdom, August 2004 Co-authored by EHAdelson and JYA Wang, “Single Lens Stereo with a Plenoptic Camera”, IEEE Trans. On Pattern Analysis and Machine Intelligence ( IEEE Trans. On Pattern Analysis and Machine Intelligence), Vol.14, No.2, p.99-106, February 1992 Co-authored by M. Amtoun and B. Boufama, “Multibaseline Stereo using a Single-Lens Camera”, Proc. On International Conference on Image Processing ( Proc. On International Conferenceon Image Processing), Volume 1, pages 401-404, September 2003 S.Hiura and T.Matsuyama, "Depth Measurement by the Multi-Focus Camera", Proc. Computer Vision and Pattern Recognition (Proc. on Computer Vision and Pattern Recognition), p. 953-959, June 1998 EP Simoncelli and H. Farid, “Direct Differential Range Estimating using Optical Masks”, Proc. On European Conference on Computer Vision (Proc. on European Conference on Computer Vision), Volume 2, pages 82-93, United Kingdom, April 1996. Kenji Yamada, Hideya Takahashi, Eiji Shimizu, “Three-dimensional shape detection method using coded aperture method”, IEICE Transactions, Vol.11, No.11, p.2986-2994 November 1997 Akihiro Kiyohara, Masato Nusui and Kouji Ikeda, “Distance Measurement Using One Camera and Lens Array”, Proceedings of Image Recognition and Understanding Symposium (MIRU2004), Volume I, p.392-397, 2004 July JYBouguet, “Camera Calibration Toolbox for Matlab”, http://www.vision.caltech.edu/bouguetj/calib_doc/index.html, October 2004

  The image division method is to divide an image into a plurality of regions using optical means such as a mirror (see Non-Patent Document 5 and Non-Patent Document 6) and a prism (see Non-Patent Document 7). Basically, it is devised so that images from multiple cameras can be taken simultaneously with one camera, and a method of projecting images taken with multiple cameras with different positions and orientations into a single image It is.

  In the image segmentation method, there is no difference in image density, contrast, color, etc. between cameras. In addition, since a plurality of images that are completely synchronized in time are used, it can be applied to dynamic objects and can also be applied to real-time applications. However, since the number of pixels of one camera is divided and used (that is, the number of pixels of a divided image corresponding to an image from one viewpoint is small), there is a problem that measurement accuracy is lowered. In addition, as in the case of stereo distance measurement using a plurality of cameras, it is necessary to calibrate external parameters of the cameras.

  The multiple image method is a method of capturing a plurality of images while optically moving the position of one camera and estimating the distance to the target. For example, the angle with respect to the optical axis of the parallel flat transparent plate arranged on the camera optical axis is changed (see Non-Patent Document 8), or the angle of the transparent plate remains constant and rotates along the camera optical axis (non- (See Patent Document 9), thereby realizing precise translation of the camera position.

  In the multiple image method, since the camera moves in parallel, it is not necessary to estimate external parameters, and distance calculation is easy. However, in principle, it is impossible to measure the distance to a moving object in order to obtain the distance to the target after shooting a plurality of images. In addition, since the optical means is composed of a mechanism including an operating member, it results in an expensive system for its manufacture, and it takes time-consuming measurement for adjustment.

  The multiple exposure method refers to a single lens aperture (see Non-Patent Document 10 and Non-Patent Document 11), a coded aperture (see Non-Patent Document 12, Non-Patent Document 13, and Non-Patent Document 14), a lens array, and the like. (Refer to Non-Patent Document 15) is a method of obtaining distance information by photographing a light beam passing through different positions into one image.

  In the multiple exposure method, since a plurality of openings are provided, the use efficiency with respect to the light amount is reduced, and the baseline length is limited by the lens openings. In addition, problems such as low measurement accuracy due to ambiguity of corresponding point search in captured images using a lens array, and low contrast in captured images using coded apertures, which are in principle taken by pinholes There is.

  The present invention has been made under the circumstances as described above, and an object of the present invention is to solve the above-described conventional problems and to capture a double image reflected on a transparent plate with one movable image. Provided is a distance measurement method using a double image reflected on a transparent plate, which can perform distance measurement with an inexpensive apparatus configuration and a short measurement time based on a single image obtained by photographing with an apparatus. There is.

は次の式によって求め、

ただし、θ_ｓは前記表面反射像Ｉ_ｓと前記裏面反射像Ｉ_ｒとの間の角度視差で、ｎは空気に対する前記透明板の相対屈折率で、ｄは前記透明板の厚さで、θ_ｉは前記撮像装置の撮影レンズの光軸と前記透明板との成す角度であることにより、或いは、前記２重像は、反射無し透過像（即ち、透明板内部での反射がない透過像）と２回反射透過像（即ち、透明板内部で２回反射した後の透過像）とから構成され、前記撮像装置の撮影レンズ光学中心から前記計測対象までの距離

は次の式によって求め、

ただし、θ_ｓは前記反射無し透過像と前記２回反射透過像との間の角度視差で、ｎは空気に対する前記透明板の相対屈折率で、ｄは前記透明板の厚さで、θ_ｉは前記撮像装置の撮影レンズの光軸と前記透明板との成す角度であることにより、或いは、前記２重像上の注目点に対する対応位置探索を拘束直線上に限って行うことにより、或いは、前記撮像装置は、固体撮像素子を用いるものであることによって効果的に達成される。 The present invention relates to a distance measuring method using a double image appearing on a transparent plate, the object of the present invention is composed of both sides one and the transparent plate consists of parallel planes, and one of the imaging apparatus Displacement measurement between the double images based on a single image obtained by photographing the double image to be measured with the imaging device having a positional deviation reflected on the transparent plate. Is derived geometrically from the positional relationship between the transparent plate and the optical center of the imaging lens of the imaging device, and the displacement between the double images is measured based on the displacement constraint between the derived double images. Corresponding to the displacement between the double images, the autocorrelation maximum position of the image is searched by obtaining the distance from the optical center of the photographing lens to the measurement object or along the displacement constraint between the double images. By measuring the displacement between the double images , or Image is composed of a front surface reflection image I _s and rear surface reflection image I _r, the distance from the photographing lens optical center of the imaging device to the measurement target

Is obtained by the following formula:

However, theta _s at an angle disparity between the rear surface reflection image I _r and the surface reflection image I _s, n is a relative refractive index of the transparent plate to air, d is a thickness of the transparent plate, theta _i is an angle formed by the optical axis of the photographing lens of the image pickup apparatus and the transparent plate, or the double image is a non-reflection transmission image (that is, a transmission image without reflection inside the transparent plate). And a twice reflected transmission image (that is, a transmission image after being reflected twice inside the transparent plate), and a distance from the optical center of the imaging lens of the imaging device to the measurement target

Is obtained by the following formula:

Where θ _s is the angular parallax between the non-reflective transmitted image and the twice reflected transmitted image, n is the relative refractive index of the transparent plate with respect to air, d is the thickness of the transparent plate, and θ _i Is an angle formed by the optical axis of the photographic lens of the imaging device and the transparent plate , or by performing a corresponding position search with respect to a point of interest on the double image only on a constrained straight line , or The imaging device is effectively achieved by using a solid-state imaging device .

は次の式によって求め、

ただし、ｄは、一定と近似された前記透明板の厚さであり、θ _ｓ は前記表面反射像Ｉ _ｓ と前記裏面反射像Ｉ _ｒ との間の角度視差であり、θ _ｉ は前記撮像装置の撮影レンズの光軸と前記透明板との成す角度であり、θ _ｐ は拘束直線の方向に沿った前記非平行度を表す角度であり、θ _o は前記計測対象から前記透明板の表面に入射する透過光の入射角であり、θ _t は前記透過光が前記透明板の裏面に到達し、前記裏面で前記透過光の一部が反射し、再び前記表面を透過するときの入射角であり、Ｄ _ｃ は前記計測対象からの光線が前記透明板の前記表面で反射する位置を原点とする座標系において、前記原点から前記撮影レンズ光学中心までの距離であり、Ｄ _o は前記原点から前記計測対象までの距離であることにより、或いは、前記距離計測装置のキャリブレーションを行うことにより、前記Ｄ _ｃ を測定することにより、或いは、前記２重像上の注目点に対する対応位置探索を、互いに平行する２本の拘束直線上に限って行うことにより、或いは、前記撮像装置は、固体撮像素子を用いるものであることにより、或いは、前記透明板は透明アクリル板であり、また、前記撮像装置はＣＣＤカメラであることによってより一層効果的に達成される。
In addition, the object of the present invention is to use a distance measuring device composed of one transparent plate having non-parallelism and one imaging device, and a measurement object having a positional deviation reflected on the transparent plate. the double image based on a single image obtained by imaging by the imaging device, the positional relationship between the photographing lens optical center of the transparent plate and the imaging device displacement constraint between double image, and the Geometrically derived from the non-parallelism, and by searching for the autocorrelation maximum position of the image along the displacement constraint between the derived double images, the displacement between the double images in consideration of the non-parallelism is measured. Then, in response to the measured displacement between the double images in consideration of the non-parallelism, the distance from the optical center of the photographing lens to the measurement object is obtained, or the double image is a surface reflection image. is composed of a I _s and the rear surface reflection image I _r, before the pre-Symbol shooting lens optical center The distance to the measurement object

Is obtained by the following formula:

However, d is the thickness of the transparent plate which is approximated as constant, theta _s is the angle disparity between the rear surface reflection image I _r and the surface reflection image I _s, theta _i is the imaging device Is an angle formed by the optical axis of the photographic lens and the transparent plate, θ _p is an angle representing the non-parallelism along the direction of the constraint straight line, and θ _o is the surface of the transparent plate from the measurement target The incident angle of incident transmitted light, θ _t is the incident angle when the transmitted light reaches the back surface of the transparent plate, a part of the transmitted light is reflected on the back surface, and is transmitted through the surface again. There, D _c in the coordinate system a ray as the origin position which is reflected by the surface of the transparent plate from the measurement target, a distance from the origin to the photographing lens optical center, D _o from the origin By being the distance to the measurement object , or of the distance measurement device By performing calibration, the by measuring D _c, or the corresponding position location for the target point on the double image, by performing only two captive on a straight line parallel to each other, or, The image pickup device is achieved more effectively by using a solid-state image pickup device, or the transparent plate is a transparent acrylic plate, and the image pickup device is a CCD camera .

  If the distance measuring method using the double image reflected on the transparent plate according to the present invention is used, the displacement restraint of the double image having a positional deviation reflected on the transparent plate composed of parallel planes on both sides is derived and derived. The displacement between the double images is measured by searching for the autocorrelation maximum position of the image along the displacement constraint of the multiple images, and the distance from the measurement object corresponding to the measured displacement between the double images. Can be easily obtained.

  According to the present invention, a low-cost transparent plate (for example, a transparent acrylic plate) and a single image pickup device (for example, a CCD camera) having no movable part and a very simple apparatus configuration and a short measurement time can be used. An excellent effect of being able to perform distance measurement at high cost and high accuracy is achieved.

  That is, in the distance measuring method of the present invention, expensive parts such as a driving mechanism and a special optical system that are necessary in the conventional stereo distance measuring method using one camera are unnecessary. Moreover, in the present invention, when a transparent acrylic plate is used as the transparent plate, not only a completely parallel plate but also a commercially available mass-produced transparent acrylic plate having non-parallelism can be used.

  Moreover, since the present invention can be applied not only to a static measurement target but also to a dynamic measurement target, the present invention can be applied to a wide range of fields including actual applications including real-time applications. Is possible.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  In the present invention, using a single image taken by a single imaging device (a CCD camera as an example) having a double image reflected on a transparent plate whose both surfaces are parallel planes, Distance measurement is performed.

Meanwhile, light rays incident on the transparent plate, reflected light of the transparent plate surface (hereinafter, this will be referred to as the surface reflection image I _s) and divided into a transmitted light. This transmitted light reaching the backside of the transparent plate, a portion is reflected by the back surface, again transmitted through the surface (hereinafter, will be referred to as a rear surface reflection image I _r). Therefore, an image reflected on such a parallel flat transparent plate can be observed as a double image with a positional shift. This double image can be considered as an image obtained by overlapping stereo images having a small baseline.

  A basic configuration of a distance measuring device used in the distance measuring method according to the present invention is shown in FIG. As shown in FIG. 1A, a distance measuring device used in the distance measuring method of the present invention includes a single imaging device having no movable part, and a transparent plate (hereinafter referred to as a parallel plate). Simply referred to as a parallel flat transparent plate or a transparent plate).

  In the best mode for carrying out the present invention, a CCD camera (hereinafter, also simply referred to as a camera) is used as one image pickup apparatus having no movable part. An example of an image (double image) obtained by the distance measuring device in FIG. 1A is shown in FIG.

  Note that the imaging device of the distance measuring device used in the present invention may be an imaging device that uses a solid-state imaging device, and thus is not limited to a CCD camera. For example, a CMOS camera using a CMOS image sensor is used. Anyway.

As the greatest feature of the distance measuring method of the present invention, the displacement constraint between the double images as shown in FIG. 1B is geometrically derived from the positional relationship between the transparent plate and the camera, and this displacement constraint is By searching for the autocorrelation maximum position along the image, the displacement between the double images is measured, and the distance from the measurement object is obtained in accordance with the measured displacement. Hereinafter, the present invention will be described in detail.

<1> Properties of Double Image Reflected on Parallel Plane Transparent Plate First, the properties of a double image reflected on a parallel plane transparent plate, which is one of the components of the distance measuring device used in the present invention, will be described first. Basics used in the distance measurement method according to the present invention, such as the relationship between the angle parallax between the double images and the distance between the objects, displacement direction constraint, and displacement detection by searching for the maximum position of the autocorrelation of the image The general properties will be described in detail.

<1-1> Angle Parallax Between Double Images and Distance to Object FIG. 2 is a schematic diagram for explaining a basic geometric relationship of the distance measuring device having the configuration of FIG.

As shown in FIG. 2, the angle disparity theta _s between the surface reflection image I _s and rear surface reflection image I _r is the transparent plate relative refractive index n for the air (hereinafter, simply referred to as the refractive index n), a transparent plate Thickness d (hereinafter also simply referred to as thickness d or plate thickness d), an angle (hereinafter also simply referred to as an incident angle) θ _i formed by the optical axis of the camera and the transparent plate, and an object to be measured ( Here, in order to simplify the explanation, the distance varies depending on the distance to one point in the object.

Measurement object by considering the ray on the plane passing through (hereinafter, simply referred to as object) and the camera optical center, and the normal vector of the transparent plate, the angle between the surface reflection image I _s and rear surface reflection image I _r A basic relationship between parallax (hereinafter, also simply referred to as angular parallax between double images) and a distance to a target will be described. Since this plane (that is, a constraining plane described later) includes a normal vector of the transparent plate, reflection and refraction of light rays coming from the object can be described on this plane.

Here, a coordinate system (ξ, υ) with the position where the light beam from the object is reflected on the transparent plate as the origin is set. The distance from the origin of the coordinate system (ξ, υ) to the object is D _o, and the distance from the origin of the coordinate system (ξ, υ) to the camera optical center is D _c .

Accordingly, the object position and the camera optical center position in the coordinate system (ξ, υ) are (−D _o sinθ _i , D _o cosθ _i ) and (D _c sinθ _i , D _c cosθ _i ), respectively, and therefore transparent. By projecting these positions on the ξ axis with respect to the front surface reflection and back surface reflection of the plate, the relationship expressed by the following formula 1 is obtained.

カメラ光学中心から対象までの距離でまとめると、下記数２を得る。

Summing up by the distance from the camera optical center to the object, the following formula 2 is obtained.

数２から、２重像間の角度視差θ_ｓの最大値と最小値を求めることができる。Ｄ_ｏ＝０のとき、すなわち、対象が透明板に接触するまで近距離になったときに、θ_ｓは最大値θ_ｓ ^〈max〉をとる。具体的な例として、例えば、Ｄ_c＝６０[ｍｍ]、ｎ＝１.４９（透明板として透明アクリル板を用いた場合）、ｄ＝１０[ｍｍ]のとき、θ_ｉ≒５５[度]でθ_ｓ ^〈max〉≒７.５[度]となる。この角度に対応する変位まで探索すれば十分である。

From Equation 2, the maximum value and the minimum value of the angular parallax θ _s between the double images can be obtained. When D _o = 0, that is, when the object becomes a short distance until it comes into contact with the transparent plate, θ _s takes the maximum value θ _s ^<max> . As a specific example, for example, when D _c = 60 [mm], n = 1.49 (when a transparent acrylic plate is used as the transparent plate), and d = 10 [mm], θ _i ≈55 [degrees] Therefore, θ _s ^<max> ≒ 7.5 [degrees]. It is sufficient to search to a displacement corresponding to this angle.

Further, when D _o = ∞, that is, when the object is at infinity, θ _s takes the minimum value 0. However, this is when the transparent plate is a completely parallel plane. In the detailed configuration of the distance measuring device used in the present invention, which will be described later, the description will be given even when the transparent plate is not a perfect parallel plane, that is, considering the parallelism of the transparent plate.

Meanwhile, the surface reflection image I _s and rear surface reflection image I _r can be considered as a stereo image having a small baseline. When the object is at infinity, this baseline length can be calculated as

また、このときのステレオ配置は、表面反射像Ｉ_ｓを撮影するカメラの方が対象に近くなるように前後にオフセットしている。このオフセット長は、下記数４のように計算することができる。

Furthermore, stereo placement of this time, towards the camera for photographing the surface reflection image I _s is offset back and forth to be close to the target. This offset length can be calculated as shown in Equation 4 below.

具体的な例として、例えば、ｎ＝１.４９（透明板として透明アクリル板を用いた場合）、ｄ＝１０[ｍｍ]、θ_ｉ＝４５[度]のとき、base＝７.６[ｍｍ]、offset＝１５.１[ｍｍ]である。図３に、ｎ＝１.００（真空や空気）、ｎ＝１.４９（透明板として透明アクリル板、ガラス板を用いた場合）、ｎ＝２.４２（透明板の材質としてダイヤモンドを用いた場合）のときの、入射角度θ_ｉに対するベースライン長baseとオフセットoffsetを示す。それぞれに板厚ｄを乗じることで、実際の長さを求めることができる。

＜１−２＞距離計測装置の別の幾何学関係
本発明で使用される距離計測装置について、図２に示すような基本的な幾何学関係、即ち、表面反射像と裏面反射像を利用することができるだけではなく、透明板内部での反射がない透過像（以下、単に反射なし透過像とも称する）と内部で２回反射した後の透過像（以下、単に２回反射透過像とも称する）を利用することもできる。図４に、本発明で使用される距離計測装置の別の幾何学関係（つまり、透過像を利用する）を示す。

As a specific example, for example, when n = 1.49 (when a transparent acrylic plate is used as the transparent plate), d = 10 [mm], and θ _i = 45 [degree], base = 7.6 [mm] ], Offset = 15.1 [mm]. In FIG. 3, n = 1.00 (vacuum or air), n = 1.49 (when a transparent acrylic plate or glass plate is used as the transparent plate), n = 2.42 (diamond is used as the material of the transparent plate) when when had) shows a baseline length base and offset offset with respect to the incident angle theta _i. The actual length can be obtained by multiplying each by the plate thickness d.

<1-2> Another Geometric Relationship of Distance Measuring Device For the distance measuring device used in the present invention, a basic geometric relationship as shown in FIG. 2, that is, a front surface reflection image and a back surface reflection image are used. In addition, the transmission image without reflection inside the transparent plate (hereinafter also simply referred to as non-reflection transmission image) and the transmission image after reflection twice inside (hereinafter also simply referred to as double reflection transmission image). Can also be used. FIG. 4 shows another geometric relationship (that is, using a transmission image) of the distance measuring apparatus used in the present invention.

  As shown in FIG. 4, when the non-reflection transmission image and the twice-reflection transmission image are projected on the ξ axis in the same manner as Equation 1, the relationship expressed by Equation 5 below is obtained.

角度視差θ_ｓと対象までの距離Ｄ_ｏ＋Ｄ_ｃの関係は、下記数６のようになる。

The relationship between the angle parallax θ _s and the distance D _o + D _c to the object is as shown in the following formula 6.

＜１−３＞透明板の反射率
以下では、透明板の表面反射像と裏面反射像の輝度の関係を示す。透明板の表面や裏面に反射膜や反射防止膜をコーティングすることによって、反射率は大きく異なる。このため、以下のように、コーティングがある場合と、コーティングがない場合に分けて説明する。

＜１−３−１＞コーティングがない場合
図５に示すように、対象からの光線が空気中を通り透明板表面に到達するとき、表面での反射率ｋ_ｒ ^(１)は、下記数７のフレネルの公式で表すことができる。

<1-3> Reflectivity of transparent plate Hereinafter, the relationship between the luminance of the front surface reflection image and the rear surface reflection image of the transparent plate is shown. The reflectance varies greatly by coating the front and back surfaces of the transparent plate with a reflection film or an antireflection film. For this reason, it demonstrates separately with the case where there exists a coating and the case where there is no coating as follows.

<1-3-1> When there is no coating As shown in FIG. 5, when the light beam from the target passes through the air and reaches the transparent plate surface, the reflectance k _r ^{(1) on} the surface is expressed by the following formula 7 Can be represented by the Fresnel formula.

ただし、θ_ｉは入射角で、θ_ｔは屈折角である。また、ｎは屈折率である。透過率ｋ_ｔ ^(１)は、ｋ_ｔ ^(１)＝１−ｋ_ｒ ^(１)である。

However, (theta) _i is an incident angle and (theta) _t is a refraction angle. N is a refractive index. The transmittance k _t ⁽¹⁾ is k _t ⁽¹⁾ = 1−k _r ⁽¹⁾ .

Next, when the light beam reaches the back surface of the transparent plate, the reflectance k _r ⁽²⁾ to the inside of the transparent plate can be similarly expressed by the following equation ⁽⁸⁾ .

さらに、裏面で反射した光線が透明板内部から表面に到達するとき、透明板内部への反射率ｋ_ｒ ^(３)は、下記数９のようになる。

Furthermore, when the light beam reflected on the back surface reaches the surface from the inside of the transparent plate, the reflectance k _r ⁽³⁾ to the inside of the transparent plate is as shown in the following equation ⁽ 9 ⁾ .

透明板から空気中への透過率ｋ_ｔ ^(３)は、ｋ_ｔ ^(３)＝１−ｋ_ｒ ^(３)である。

The transmittance k _t ⁽³⁾ from the transparent plate to the air is k _t ⁽³⁾ = 1−k _r ⁽³⁾ .

Therefore, the composite reflectance k _t ^<1> of the back surface reflection image is as shown in the following formula 10.

この合成反射率ｋ_ｔ ^＜１＞を、１段目の裏面反射像の合成反射率と呼ぶことにする。図６に、表面反射像と裏面反射像に対するそれぞれの反射率ｋ_ｒ ^(１)とｋ_ｔ ^＜１＞を示す。

This combined reflectance k _t ^<1> will be referred to as the combined reflectance of the first-stage back surface reflected image. FIG. 6 shows the respective reflectances k _r ⁽¹⁾ and k _t ^<1> for the front surface reflection image and the back surface reflection image.

In addition, inside the transparent plate, reflection and transmission should be further repeated and observed as a back-surface reflection image. The composite reflectance k _t ^<n> of the ^n-th back surface reflection image is as shown in the following _equation ⁽ 11).

後述するように、透明板への入射角θ_ｉの範囲は、カメラの撮影レンズ画角と透明板取り付け角度によって異なるが、標準レンズ画角の場合には、およそ３０≦θ_ｉ≦６０[度]程度である。この範囲では、ｋ_ｒ ^(１)＜０.０８８なので、ｋ_ｔ ^＜１＞とｋ_ｔ ^＜２＞は、それぞれ下記数１２、数１３に示すような範囲になる。

As described below, the range of incident angle theta _i of the transparent plate may vary depending on photographing lens angle and the transparent plate mounting angle of the camera, in the case of a standard lens angle is approximately 30 ≦ θ _i ≦ 60 [degrees ] About. In this range, since k _r ⁽¹⁾ <0.088, k _t ^<1> and k _t ^<2> are in the ranges shown in the following equations 12 and 13, respectively.

つまり、ｋ_ｔ ^＜１＞／ｋ_ｔ ^＜２＞≒１２９が成立するので、１段目の裏面反射像が２段目の裏面反射像より約１２９倍も明るい反射像で、しかも表面反射像の明るさとほぼ等しい。このため、２段目以後の裏面反射像が無視できる２重像を観測することができる。

That is, since k _t ^<1> / k _t ^<2> ≈129 holds, the first-stage back-surface reflection image is approximately 129 times brighter than the second-stage back-surface reflection image, and the front-surface reflection image It is almost equal to the brightness. For this reason, it is possible to observe a double image in which the back-surface reflection image after the second stage can be ignored.

  However, when there is no reflective film coating on the transparent plate, the reflectivity for forming a double image is generally several percent to 10%, so the double image is quite dark (3 for shooting with a single camera). The aperture is dark.

  On the other hand, in the case of the geometrical relationship as shown in FIG. 4 using the non-reflective transmitted image and the twice-reflected transmitted image, the reflectance corresponding to each image is as shown in the following equations 14 and 15.

２つの像の反射率が１２９倍程度異なるため、この場合は実用的ではない。

＜１−３−２＞コーティングがある場合
ところで、金属（クロム）膜や誘電体多層膜を透明板にコーティングすることで、ハーフミラーを作ることができる。このコーティングによって、反射率を変化させることができる。ここでは、反射膜コーティングによって２重像をより明るく、しかも２重像間の明るさが同じになるような反射率と透過率について検討する。

This is not practical because the reflectance of the two images differs by about 129 times.

<1-3-2> When there is a coating By the way, a half mirror can be made by coating a transparent plate with a metal (chromium) film or a dielectric multilayer film. With this coating, the reflectance can be changed. Here, the reflectance and the transmittance are examined so that the double image becomes brighter and the brightness between the double images becomes the same by the reflection film coating.

As in the case without the coating, the reflectance and transmittance on the transparent plate surface are k _r ^(s) and k _t ^(s) , respectively, and the reflectance and transmittance on the back surface of the transparent plate are k _r ^(r) , respectively. ^Let k _t ^(r) . At this time, k _r + k _t ≦ 1 because there is absorption in the coating film.

The composite reflectance k _t ^<1> of the first-stage back-surface reflection image is _{expressed by the following equation} ⁽ 16).

同様に、ｎ段目の裏面反射像の合成反射率ｋ_ｔ ^＜ｎ＞は、下記数１７のようになる。

Similarly, the composite reflectance k _t ^<n> of the ^n- th stage back-surface reflected image is expressed by the following Expression 17.

従って、２重像間の明るさが同じで、且つ２段目以後の裏面反射像が暗くなる条件は、下記数１８のようになる。

Accordingly, the condition that the brightness between the double images is the same and the back-surface reflection image after the second stage becomes dark is expressed by the following equation (18).

ｋ_ｔ ^(ｓ)を消去して１つの式にまとめると、数１８は下記数１９のようになる。

When k _t ^(s) is eliminated and combined into one equation, Equation 18 becomes Equation 19 below.

コーティングがない場合の反射率の条件は、この条件を満たしている。しかし、２重像は明るくならないので、数１０の２番目の条件をとりやめ、透明板裏面での反射率をｋ_ｒ ^(ｒ)＝１（全反射）とすると、下記数２０で表すような透明板表面での反射率と透過率の関係を得る。

The reflectance condition in the absence of the coating satisfies this condition. However, since the double image does not become bright, if the second condition of Equation 10 is canceled and the reflectance on the back surface of the transparent plate is k _r ^(r) = 1 (total reflection), the transparency expressed by the following Equation 20 is satisfied. Obtain the relationship between the reflectance and transmittance on the plate surface.

例えば、ｋ_ｒ ^(ｓ)＝０.３８、ｋ_ｔ ^(ｓ)＝０.６２とすれば、２重像の明るさは、コーティングがない場合の約４倍（２絞り明るい）になる。ただし、このとき、２段目の裏面反射像の反射率ｋ_ｔ ^＜２＞＝０.２４のために、十分に明るく、３重以上の多重像が観測される。

For example, if k _r ^(s) = 0.38 and k _t ^(s) = 0.62, the brightness of the double image is about four times that of the case where there is no coating (2 stops bright). However, at this time, since the reflectance k _t ^<2> = 0.24 of the back-surface reflected image in the second stage, it is sufficiently bright and a multiple image of three or more layers is observed.

  In the case of the geometrical relationship as shown in FIG. 4 using the non-reflective transmitted image and the twice-reflected transmitted image, the reflectance corresponding to each image is expressed by the following equations (21) and (22).

ｋ_ｒ ^＜ｒ＞ｋ_ｒ ^＜ｓ＞≒１のとき、つまり、表面と裏面での反射率がどちらも１に近いときに、２重像間の明るさは、ほぼ等しくなる。しかし、逆に透過率が小さくなるため、結果として２重像はかなり暗くなる。また、このときにも３重以上の多重像が観測される。

When k _r ^<r> k _r ^<s> ≈1, that is, when the reflectance on the front surface and the back surface is close to 1, the brightness between the double images is substantially equal. However, since the transmittance is small, the double image becomes considerably dark as a result. Also at this time, multiple images of three or more layers are observed.

  In summary, it is convenient to reduce the reflectance when using a reflected image and increase the reflectance when using a transmitted image.

  For this reason, when a distance measuring device based on the use of a reflected image as shown in FIG. 2 is configured, since the transparent plate does not require coating, the transparent plate can be used as it is, and distance measurement can be easily performed. A device can be configured.

  On the other hand, in the case of configuring a distance measuring device based on using a transmission image as shown in FIG. 4, it is necessary to coat so that both surfaces of the transparent plate become half mirrors, and more than three layers. Since it is necessary to obtain the angle parallax from the multiple images, it takes a little time to configure the distance measuring apparatus.

In the following, the distance measuring device configured based on the geometrical relationship of FIG. Further, the distance measurement method of the present invention will be described in detail on the premise that the distance measurement device configured based on the geometrical relationship of FIG.

<1-4> Displacement direction constraint between double images As shown in <1-1>, the displacement between double images with respect to one point in the object is expressed by the normal vector of the transparent plate as shown in FIG. It can be expressed as reflection and refraction on the containing plane. In the present invention, this plane is called a constraining plane. Here, when a reflected image from the transparent plate is taken with a camera, the displacement constraint of the double image corresponding to the position on the image is examined.

は、関係式導出における符号の扱いやすさの都合上、裏面向きに設定している。カメラ座標原点（以下、単に原点とも称する）と画像面との距離は、ＣＣＤの画素間隔を単位として、計測したレンズ焦点距離ｆ／δとする。２重像の変位拘束は、画像面と拘束平面との交線で表すことができる。 As shown in FIG. 7, a coordinate system with the camera optical center O as the camera coordinate origin is set. The transparent plate is disposed at an angle θ _{m with} respect to a plane perpendicular to the Z axis. Transparency normal vector

Is set in the direction of the back side for the convenience of handling the code in the derivation of the relational expression. The distance between the camera coordinate origin (hereinafter also simply referred to as the origin) and the image plane is defined as the measured lens focal length f / δ with the CCD pixel interval as a unit. The displacement constraint of the double image can be expressed by an intersection line between the image plane and the constraint plane.

へのベクトルと、透明板の法線ベクトル

とに直交するので、下記数２３、数２４に示されるように、内積がともに０になる。 The normal vector before normalization of the constraint plane is defined as (c ₁ , c ₂ , 1). The normal vector of the constraint plane is the position on the image plane from the origin.

Vector to and normal vector of transparent board

Therefore, as shown in the following equations 23 and 24, the inner products are both 0.

従って、原点を通る拘束平面ｃ_１Ｘ＋ｃ_２Ｙ＋Ｚ＝０と画像面Ｚ＝ｆ／δとの交線は、画像座標（ｕ,ｖ）を使って、下記数２５のように表すことができる。

Therefore, the intersection line between the constraint plane c ₁ X + c ₂ Y + Z = 0 passing through the origin and the image plane Z = f / δ can be expressed as the following Expression 25 using the image coordinates (u, v).

これは、画像座標上の位置

を通る直線を表している。図８に、例として、画像サイズ１０２４×７６８[画素]、ｆ／δ＝１０７０.０５９１、θ_ｍ＝４５[度]のときの、画像上の各位置に対する拘束直線の方向を示す。図８において、矢印の向きは、裏面反射像に対する表面反射像の向きを表している。

This is the position on the image coordinates

Represents a straight line passing through. FIG. 8 shows, as an example, the direction of the constraint straight line for each position on the image when the image size is 1024 × 768 [pixel], f / δ = 1070.0591, and θ _m = 45 [degrees]. In FIG. 8, the direction of the arrow represents the direction of the front surface reflection image with respect to the back surface reflection image.

に対応する２重像の位置を拘束直線上で探索した結果、位置

から画素単位で表す符号付き距離Δの位置に見つかったとする。このとき、２重像間の角度視差θ_ｓは、下記数２６のように求めることができる。 Position on the image in camera coordinates

As a result of searching for the position of the double image corresponding to

And a signed distance Δ expressed in pixel units. At this time, the angular parallax θ _s between the double images can be obtained as in the following Expression 26.

ただし、

である。

However,

It is.

Further, θ _i can be obtained as in the following Expression 27.

＜１−５＞自己相関による対応位置探索
２重像画像中のある注目位置に対する対応位置は、拘束直線上を探索すればよいことがわかった。ここでは、拘束直線上の画素値を取り出した１次元関数の自己相関を使って、対応位置を探索する方法を説明する。

<1-5> Corresponding position search by autocorrelation It has been found that the corresponding position for a certain target position in the double image may be searched on a constraining straight line. Here, a method for searching for a corresponding position using autocorrelation of a one-dimensional function obtained by extracting pixel values on a constraint line will be described.

は、下記数２８のように定義される。 Autocorrelation function of function f (t) for variable t

Is defined as in Equation 28 below.

数２８で定義される自己相関関数

は、τに関する偶関数で、τ＝０で最大値を取る。関数ｆ(ｔ)に周期性がないときは、その自己相関関数

は、τ＝０で極大且つ最大値を取る。

Autocorrelation function defined by Equation 28

Is an even function related to τ and takes a maximum value when τ = 0. If the function f (t) is not periodic, its autocorrelation function

Takes a maximum and maximum value at τ = 0.

は、下記数２９のようになる。 As a function f (t), consider a function that takes out a pixel value on a constraining line through a certain position of interest in a double image. When the displacement between the double images is Δ, and one of the images constituting the double image is I (t), the double image is expressed as f (t) = I (t) + I (t−Δ). be able to. Autocorrelation function for this f (t)

Is as shown in Equation 29 below.

つまり、２重画像の自己相関関数は、Ｉ(ｔ)の自己相関関数の２倍と、Ｉ(ｔ)の自己相関関数を±Δだけシフトした関数との和になっている。従って、２重像画像中のある注目位置に対する対応位置は、拘束直線上の画素値で構成した関数の自己相関関数を求め、その第２極大位置を探索すればよいことがわかる。

That is, the autocorrelation function of the double image is the sum of twice the autocorrelation function of I (t) and a function obtained by shifting the autocorrelation function of I (t) by ± Δ. Therefore, it is understood that the corresponding position for a certain position of interest in the double image may be obtained by obtaining an autocorrelation function of a function composed of pixel values on the constraint line and searching for the second maximum position.

に対する拘束直線を示す。拘束直線の長さは、探索範囲を示す。対象までの距離が大きくなると、次第に対応位置が注目点に近づき、Ｄ_ｏ＝∞のときには完全に一致する。図９(Ｂ)に、２重像間変位量に対応する自己相関関数の例を示す。対応点は、この自己相関関数の第２極大位置を探索することで求められるが、２重像間変位Δが小さくなると、即ち、対象までの距離が大きくなると、第２極大が分離できなくなることがわかる。しかし、この問題に関しては、後述する簡単な解決策を示している。

＜２＞距離計測装置の構成及びキャリブレーション
＜２−１＞距離計測装置の構成例
カメラと透明板を図１０に示されるように配置して本発明で使用される距離計測装置を構成した。カメラには、市販のディジタルカメラを利用した。撮影レンズの焦点距離は、ｆ＝７.０９[ｍｍ]（公称）である。この焦点距離と公称撮像面サイズ（６.９１×５.１８[ｍｍ]）を使って、下記数３０、数３１を用いて、水平画角ＦＯＶ_ｈと垂直画角ＦＯＶ_ｖを求め、適切な透明板サイズの選択と配置に利用した。 Figure 9 (A) shows the points of interest

A constraining straight line with respect to is shown. The length of the constraint straight line indicates the search range. As the distance to the object increases, the corresponding position gradually approaches the point of interest, and completely matches when D _o = ∞. FIG. 9B shows an example of an autocorrelation function corresponding to the amount of displacement between double images. The corresponding point can be obtained by searching for the second maximum position of the autocorrelation function. However, when the displacement Δ between the double images decreases, that is, when the distance to the object increases, the second maximum cannot be separated. I understand. However, regarding this problem, a simple solution described later is shown.

<2> Configuration and Calibration of Distance Measuring Device <2-1> Configuration Example of Distance Measuring Device A distance measuring device used in the present invention was configured by arranging a camera and a transparent plate as shown in FIG. A commercially available digital camera was used as the camera. The focal length of the photographing lens is f = 7.09 [mm] (nominal). Using this focal length and the nominal imaging surface size (6.91 × 5.18 [mm]), the horizontal field angle FOV _h and the vertical field angle FOV _v are obtained using the following equations 30 and 31, and appropriate values are obtained. Used for selection and placement of transparent plate size.

透明板には、容易に入手可能な透明アクリル板（サイズは１００×１００[ｍｍ]で、板厚は１０[ｍｍ]である）を使用した。透明板の取付角度θ_ｍは４５[度]である。このとき、透明板への入射角度は、約２５〜６５[度]になる。透明板の裏面と側面からの入射光を避けるため、それぞれ遮光のための黒色シートを貼り付けた。

＜２−２＞カメラ内部パラメータのキャリブレーション
ここで、カメラ内部パラメータのキャリブレーション方法について述べる。

As the transparent plate, a readily available transparent acrylic plate (size is 100 × 100 [mm] and plate thickness is 10 [mm]) was used. Mounting angle theta _m of the transparent plate is 45 [degrees]. At this time, the incident angle to the transparent plate is about 25 to 65 [degrees]. In order to avoid incident light from the back and side surfaces of the transparent plate, black sheets for light shielding were respectively attached.

<2-2> Calibration of Camera Internal Parameters Here, a method for calibrating camera internal parameters will be described.

  The camera internal parameters were obtained using a publicly disclosed calibration tool (see Non-Patent Document 16). FIG. 11 shows an example of an image used for calibration. The camera internal parameters were calibrated using 18 such images.

Although the number of pixels of the camera is 2048 × 1536 [pixels], it has been reduced to 1024 × 768 [pixels] by bicubic interpolation in order to shorten the calculation time. The parameters obtained by calibration are the lens focal length f / δ = 1070.0591 measured at the CCD pixel interval, and the image center (c _u , c _v ) = (− 7.8384, −12.1811). Distortion was removed from the image using the distortion parameters obtained at the same time.

<3> Examination of non-parallelism and mounting angle of transparent plate and distance measurement procedure <3-1> Influence of non-parallelism of transparent plate The transparent acrylic plate seems to be a perfect parallel plate, but strictly speaking, Not parallel. A transparent acrylic plate, which is a mass-produced product on the market, is not a perfect parallel plate but has non-parallelism.

  Here, first, the influence of the non-parallelism of the transparent plate is examined on the constraining plane of FIG. 2 (for example, in the case of using a transparent acrylic plate that is commercially available as a transparent plate in the distance measuring device used in the present invention) ).

As shown in FIG. 12, as the non-parallelism affecting the reflection and refraction of light rays on the same constraint plane as Figure 2, consider the angle theta _p. At this time, similarly to Equation 1, the following Equation 32 is obtained.

ただし、透明板の厚さｄは、図１２に示されるように、一定と近似している。カメラ光学中心から対象までの距離でまとめると、下記数３３を得る。

However, the thickness d of the transparent plate is approximated to be constant as shown in FIG. Summing up by the distance from the camera optical center to the object, the following equation 33 is obtained.

このとき、右辺第２項には距離Ｄ_ｃを含むので、距離計測装置を構成したときに、Ｄ_ｃを予め測定する必要がある。

At this time, since the second term on the right side includes the distance D _c , it is necessary to measure D _c in advance when the distance measuring device is configured.

However, when θ _i = 45 [degrees], θ _p = 0.058 [degrees], and θ _s <8 [degrees], when the weight for the first term is actually calculated, the following equation 34 is obtained.

このため、Ｄ_ｃは概略測定で十分である（カメラ光学中心、つまり、撮影レンズの光学中心を正確に求める必要はない）。

For this reason, rough measurement is sufficient for D _c (the camera optical center, that is, it is not necessary to accurately determine the optical center of the photographing lens).

Further, θ _o and θ _t are angles represented by the following formulas 35 and 36, respectively, as shown in FIG.

図１３に、θ_ｐ＝０[度]（透明板が完全な平行板のとき）とθ_ｐ＝０.０５８[度]に対する、２重像間の角度視差θ_ｓと、カメラ光学中心から対象までの距離（Ｄ_ｏ＋Ｄ_ｃ）との関係を示す。図１３から分かるように、θ_ｐ＞０[度]のとき、カメラ光学中心から対象までの距離（Ｄ_ｏ＋Ｄ_ｃ）が大きくなるにつれて角度視差θ_ｓは小さくなるが、一定の角度θ_ｓ ^＊より小さくならず、２重像が重なることはない。本発明では、この性質によって、２重像の自己相関を利用した対応位置探索において、第２極大位置を効果的に分離することができる。

FIG. 13 shows the angle parallax θ _s between double images for θ _p = 0 [degree] (when the transparent plate is a perfect parallel plate) and θ _p = 0.058 [degree], and the object from the camera optical center. The relationship with the distance to (D _o + D _c ) is shown. As can be seen from FIG. 13, when θ _p > 0 [degrees], the angle parallax θ _s decreases as the distance (D _o + D _c ) from the camera optical center to the target increases, but the constant angle θ _s ^* It is not smaller and the double images do not overlap. In the present invention, due to this property, the second maximum position can be effectively separated in the corresponding position search using the autocorrelation of the double image.

  When the object is at infinity, the denominator of Equation 33 becomes 0, and the following Equation 37 is obtained.

すなわち、無限遠方距離に対応するθ_ｓ ^＊とθ_ｐの間には、下記数３８で表す関係が成立する。

That is, the relationship represented by the following formula 38 is established between θ _s ^* and θ _p corresponding to the infinite distance.

数３８にはθ_ｉを含むが、θ_ｉの変化に対する影響は比較的小さい。画像中央部、即ち、θ_ｉ＝４０〜５０[度]付近で、θ_ｓ ^＊を調べれば、θ_ｐを求めることができる。

Equation 38 includes θ _i , but its influence on changes in θ _i is relatively small. If θ _s ^* is examined in the center of the image, that is, in the vicinity of θ _i = 40 to 50 degrees, θ _p can be obtained.

When θ _p <0 [degree], as can be easily imagined from FIG. 13, the angle parallax θ _s becomes 0 [degree] at a certain distance, and the double images overlap. For this reason, in the corresponding position search using autocorrelation, the measurement distance is limited. Further, when the distance increases, the displacement relationship between the surface reflection image I _s and rear surface reflection image I _r is reversed, the angle disparity gradually increases. However, not detect the displacement of the orientation of front surface reflection image I _s and rear surface reflection image I _r from the autocorrelation of the double image. For this reason, a correct distance cannot be obtained in the corresponding position search using autocorrelation. Therefore, when configuring the distance measuring device used in the present invention, it is necessary to arrange the transparent plate so that θ _p > 0 [degrees].

<3-2> Estimation of Non-Parallelity of Transparent Plate Next, a method for obtaining the non-parallelism of the transparent plate using a double image taken with the configuration shown in FIG. 10 will be described.

  For the reason described in <3-1>, the upper, lower, left, and right sides of the transparent plate are selected and arranged so that the object at infinity is in a double image displacement relationship as shown in FIG. This displacement relationship shown in FIG. 14 can be easily determined by observing an image reflected on the transparent plate and blocking a part thereof. Further, the thickness of the transparent plate can be actually measured and judged.

であり、自己相関関数からはＩ_ｓとＩ_ｒの変位関係を求めることはできないが、先に述べて理由で、下記数３９で表す、変位原点に対して右下方向の極大位置

を利用する。 Using the distance measuring device arranged in this way, an image that can be regarded as infinitely far away as shown in FIG. FIGS. 15C and 15D show the results of obtaining the two-dimensional autocorrelation function using the portion surrounded by the red rectangle at the center of the image. From FIG. 15D, the second and third maximum positions are

, And the but from the autocorrelation function can not be obtained displacement relationship I _s and I _r, for reasons stated earlier, expressed by the following Expression 39, the maximum position of the lower right direction with respect to the displacement origin

Is used.

図１６に示すように、透明板の非平行度を、２つの角度θ_ph及びθ_pvで表す。

から、水平方向と垂直方向に対応する角度視差θ_sh ^＊とθ_sv ^＊を求め、数３８を使うことで、θ_phとθ_pvを下記数４０、数４１のように推定することができる。

As shown in FIG. 16, the non-parallel degree of the transparent plate is represented by two angles θ _ph and θ _pv .

Thus, by obtaining the angular parallax θ _sh ^* and θ _sv ^* corresponding to the horizontal direction and the vertical direction and using _Equation 38, θ _ph and θ _pv can be estimated as shown in _Equations 40 and 41 below.

＜３−３＞透明板の非平行度を考慮した対応点探索
以下に、透明板の非平行度によって、どのような直線上で対応点探索を行うべきかを説明する。

<3-3> Corresponding Point Search Considering Non-Parallelity of Transparent Plate Hereinafter, it will be described on which straight line the corresponding point search should be performed based on the non-parallelism of the transparent plate.

  When the transparent plate is a perfect parallel plate, the corresponding point for a certain point of interest on the double image is on a constraining straight line. By interpolating and extracting pixel values on the constrained straight line and treating them as a one-dimensional function, and calculating the autocorrelation function of this one-dimensional function, three local maxima can be detected when the target is a short distance. The corresponding point can be detected by searching for the second maximum position. As the distance of the object increases, these three local maximum positions approach and eventually cannot be separated. In principle, when the object is at infinity, the three maxima coincide at one point.

に対して、２点

を通る拘束直線に平行な２直線上を移動する。

＜３−４＞透明板の取付角度のキャリブレーション
カメラに対する透明板の取付角度には、２自由度の不確かさがある。この不確かさによって、＜１−４＞で示した拘束直線交点位置

は、実際に撮影した画像では異なる位置に現れる。この移動量を

とすると、拘束直線の交点は、

となる。 However, when the transparent plate has non-parallelism, that is, when the transparent plate is not a perfect parallel plate, as shown in FIGS. 15 (C) and (D), even when the object is at infinity, the maximum is Separate into 3 pieces. The positions of the second maximum and the third maximum are as shown in FIG.

2 points

It moves on two straight lines parallel to the constraining straight line passing through.

<3-4> Calibration of mounting angle of transparent plate The mounting angle of the transparent plate with respect to the camera has uncertainty of two degrees of freedom. Due to this uncertainty, the position of the constrained straight line intersection shown in <1-4>

Appear at different positions in the actually captured image. This movement amount

Then, the intersection of the constraint lines is

It becomes.

が正確なほど大きくなると考えることができる。 In the autocorrelation function used when searching for the corresponding position, the size of the second maximum represents the goodness of the corresponding points. The goodness of this correspondence is the direction of the restraint straight line, that is, the intersection of the restraint straight lines.

Can be considered to be larger as the accuracy increases.

を画像上の注目位置

の関数と考え、下記数４２のように、

を求める。 So, the second maximum

The attention position on the image

And the following equation 42,

Ask for.

本発明で使用される距離計測装置において、透明板の取付角度のキャリブレーションに利用される２重像は、自己相関関数の第２極大が明確に現れる画像なら、どのような距離の対象を撮影したものでも利用することができる。

In the distance measuring apparatus used in the present invention, the double image used for calibration of the mounting angle of the transparent plate is an image of any distance object as long as the second maximum of the autocorrelation function appears clearly. You can also use it.

  In this example, an image obtained by photographing a plane (wall surface) at a certain distance (1 [m] from the transparent plate) as shown in FIG. 18 was used for calibration of the mounting angle of the transparent plate. When calculating the autocorrelation function, the maximum value appears more accurately when the sum area is somewhat large.

は離散化極大値を含む３点を使ったパラボラフィッティングによって推定した。

は、画像上に均等に分布する６３個の位置を利用した。

は、水平方向と垂直方向にそれぞれ１０[画素]おきに変化させて総和を計算した。 Therefore, the autocorrelation was calculated using not only the pixel values on the straight line but also the surrounding 21 × 21 [pixel] area. Also, since the autocorrelation function is calculated for the displacement (size of 1 [pixel]) discretized on a straight line,

Was estimated by parabolic fitting using three points including the discretized maximum.

Used 63 positions evenly distributed on the image.

The total was calculated by changing the horizontal direction and the vertical direction every 10 [pixels].

を計算した結果を示す。この結果から、

を得る。 In FIG. 19, for −100 ≦ u _c ≦ + 100, −100 ≦ v _c ≦ + 100,

The result of calculating is shown. from this result,

Get.

を求める。また、Ｄ_ｃを測定する。なお、距離計測装置のキャリブレーションは、予め行うことができる。

ステップＣ：２重像間の角度視差θ_ｓの計算
まず、注目点

に対する対応位置を、次の２直線上の点

で探索する。 As described above, in the present invention, when a transparent acrylic plate that is a mass-produced product that is commercially available is used as a transparent plate, an ideal parallel plate is used by utilizing the non-parallelism of the transparent acrylic plate. The corresponding position search can be performed more effectively than the case. Further, as described above, the error between the non-parallel degree of the transparent plate and the mounting angle of the transparent plate can be estimated by calibration using an image photographed by a distance measuring device constituted by the transparent plate and the camera.

<3-5> Distance Measurement Procedure Based on the above description, the distance measurement procedure in the distance measurement method using the double image reflected on the transparent plate according to the present invention will be described in detail as follows.
Step A: Calibration of camera internal parameters The number of pixels of the image and the lens focal length f / δ are obtained. Remove lens distortion from the image.

Step B: Calibration of the distance measuring device After configuring the distance measuring device,

Ask for. Also, _Dc is measured. The distance measuring device can be calibrated in advance.

Step C: Calculation of angular parallax θ _s between double images

The corresponding position for the point on the next two straight lines

Search with.

ただし、ｔは画素単位の媒介変数である。また、ｔ_maxは、距離計測装置の構成から決定でき、

に応じて変えてもよい。

Here, t is a parameter in units of pixels. T _max can be determined from the configuration of the distance measuring device,

You may change according to.

の大きさを加算して、透明板の非平行度を考慮した２重像間変位Δとする。 Next, as shown in the following formula 44, as a result of the search, it is projected in the direction of the constraining straight line at | t |

Is added to obtain a displacement Δ between the double images in consideration of the non-parallelism of the transparent plate.

そして、数４４で算出された透明板の非平行度を考慮した２重像間変位Δに基づき、数２６によって、２重像間の角度視差θ_ｓを計算する。

ステップＤ：カメラ光学中心から対象までの距離の計算
まず、下記数４５で表すように、２つの角度θ_phとθ_pvを、拘束直線の方向に沿った非平行度を表す角度θ_ｐに合成する。

Then, based on the displacement Δ between the double images in consideration of the non-parallel degree of the transparent plate calculated in Expression 44, the angular parallax θ _s between the double images is calculated according to Expression 26.

Step D: Calculation of the distance from the optical center of the camera to the object First, as expressed by the following formula 45, the two angles θ _ph and θ _pv are combined into an angle θ _p representing the non-parallel degree along the direction of the constraint straight line. To do.

そして、数３３によって、カメラ光学中心から対象までの距離（Ｄ_ｏ＋Ｄ_ｃ）を計算する。

ステップＥ：距離計測結果の補正
本発明の距離計測方法では、カメラ光学中心（レンズ光学中心）から対象までの距離を測定している。カメラ光軸に投影した距離に補正するために、画像位置（ｕ_ｐ,ｖ_ｐ）に対して計算した距離に、

を乗じる。

＜４＞実験結果
ここでは、実画像を使って、本発明の距離計測方法を用いて実際に距離計測が可能なことを確認する。
＜４−１＞正対する平面（壁面）
本発明の距離計測方法によって距離計測が可能なことを確認するために、まず、図２０に示すように、距離計測装置に正対する壁面を撮影して距離測定実験を行った。実験に利用した距離計測装置の構成では、Ｄ_ｃ＝６０[ｍｍ]である。

Then, the distance (D _o + D _c ) from the camera optical center to the target is calculated by _Equation 33.

Step E: Correction of Distance Measurement Result In the distance measurement method of the present invention, the distance from the camera optical center (lens optical center) to the object is measured. In order to correct the distance projected on the camera optical axis, the distance calculated for the image position (u _p, v _p),

Multiply

<4> Experimental Results Here, it is confirmed that actual distance measurement is possible using the distance measurement method of the present invention using an actual image.
<4-1> Plane (wall surface) facing directly
In order to confirm that distance measurement is possible by the distance measurement method of the present invention, first, as shown in FIG. 20, a wall surface directly facing the distance measurement device was photographed to perform a distance measurement experiment. In the configuration of the distance measuring apparatus used for the experiment, D _c = 60 [mm].

FIG. 21 shows an autocorrelation function value for an input image (double image) when the distance D _o to the wall surface is 0.5 [m]. Figure 21 (A) is an input image center _{(u p, v p) =} (0,0) shows an autocorrelation function values for each point of the vertical line passing through the value of the red part of the value is large blue area is small Represents that. The horizontal axis is the searched parameter t, and the vertical axis is the vertical position on the input image. Although the measurement target is a plane (wall surface), it can be seen that the amount of displacement varies depending on the vertical position in the input image. FIG. 21B shows an autocorrelation function with respect to the center position of the input image, that is, a cross-sectional view with respect to the red horizontal line in FIG. The autocorrelation function was calculated for an area of 41 × 41 [pixels].

Figure 22 shows the distance (D _o) maps the measurement results obtained by changing the D _o from 0.5 [m] to 3.0 [m]. FIG. 23 shows the linearity of the distance measurement result of FIG.

In FIG. 23, a region having an area of 1/4 of the image is set at the center of the image, and the distances of 17 × 17 = 289 locations in the region are measured. The average value of the measured distances is a black circle, and the ± standard deviation is Shown with error bars. This result, at a distance D _o to the object is 2 to 2.5 [m], it is possible to read the difference in distance to the object and the variation in the measured value can not be determined. Therefore, in the apparatus configuration shown in FIG. 20, distance measurement up to about 2 [m] is possible.

<4-2> Person Next, a distance measurement experiment is performed by using an input image (double image of a person) in a more natural environment as shown in FIG. It was confirmed that distance measurement was possible by the distance measurement method.

When the input image (double image) is used as it is, the change of the value becomes gentle around the maximum position of the autocorrelation function, and the second maximum position cannot be effectively separated. For this reason, the low frequency component of the input image was reduced before processing. FIG. 24B shows the distance map measurement result. In the distance map measurement result, the distance is obtained for 224 × 224 positions excluding the peripheral portion of the input image. The distance to the object is distributed from about 0.5 [m] (blue) to 1.5 [m] (green). The reason why the top and bottom of the image is reversed is that the mirror is reversed upside down. The white area is an area where the corresponding position cannot be obtained because there is no texture. This result uses only the method of detecting the corresponding position from the autocorrelation function described so far, and does not perform any other error recovery processing.

Although specific embodiments of the present invention have been described above, the present invention is not limited thereto, and for example, multiple images (multiple reflection images or multiple transmission images) are used instead of double images. It is also possible. At that time, multiple maxima appear in the autocorrelation function, and there is a possibility that the corresponding position can be searched with higher accuracy by using the period.

FIG. 1A is a basic configuration diagram of a distance measuring device used in the distance measuring method of the present invention. FIG. 1B shows an example of a double image obtained by the distance measuring device shown in FIG. It is a schematic diagram for demonstrating the basic geometric relationship of the distance measuring device which has the structure of FIG. It is a figure which shows the relationship between an incident angle and baseline length, and the relationship between an incident angle and offset with respect to the transparent plate which has a different refractive index. It is a schematic diagram for demonstrating another geometric relationship (namely, using a transmitted image) of the distance measuring device used by this invention. It is a schematic diagram for demonstrating the surface reflection and back surface reflection in a transparent plate. It is a figure which shows the relationship between each reflectance with respect to a surface reflected image and a back surface reflected image, and an incident angle. In the distance measuring device used by this invention, it is a schematic diagram for demonstrating the relationship between a transparent plate and a camera. It is a figure which shows the direction of the restraint straight line in each position on an image. FIG. 9A shows an example of a constraint straight line with respect to the point of interest. FIG. 9B is a diagram illustrating an example of an autocorrelation function for pixel values on a constraint line. It is a figure for demonstrating the specific arrangement | positioning relationship of a transparent plate and a camera in the distance measuring device used by this invention. It is a figure which shows an example of the image utilized for calibration. In the distance measuring device used by this invention, it is a schematic diagram for demonstrating the influence of the non-parallelism of a transparent plate. It is a figure which shows the relationship between the angle parallax between double images, and the distance from a camera optical center to object. It is a figure which shows the displacement relationship of a double image. It is a figure which shows an infinitely far image and a two-dimensional autocorrelation function. In the distance measuring device used by this invention, it is a schematic diagram for demonstrating two angles showing the non-parallel degree of a transparent plate. In this invention, it is a figure for demonstrating the corresponding point search which considered the non-parallel degree of the transparent plate. FIG. 17A shows an example of a constraint line (in the case of two constraint lines) for the target point. FIG. 17B is a diagram illustrating an example of the maximum position of the autocorrelation function. It is a figure which shows the image utilized for the calibration of the attachment angle of a transparent plate in the distance measuring device used by this invention. In this invention, it is a figure which shows the evaluation value when changing the correction amount of a restraint straight line intersection position. It is a schematic diagram for demonstrating the distance measurement experiment to a wall surface by applying this invention. FIG. 21A shows an autocorrelation function map. FIG. 21B shows an example of the autocorrelation function. It is a figure which shows the measurement result of a distance map. It is a figure for demonstrating the linearity of a distance map measurement result. FIG. 24A shows an input image of a person. FIG. 24B shows the distance map measurement result.

Claims (12)

Both sides one and the transparent plate consists of parallel planes, with the configured distance measuring device and a single imaging apparatus, a positional deviation reflected on the transparent plate, a double image of the measurement object the Based on a single image obtained by photographing with an imaging device, displacement constraints between double images are geometrically derived and derived from the positional relationship between the transparent plate and the optical center of the imaging lens of the imaging device. The displacement between the double images is measured based on the displacement constraint between the double images, and the distance from the photographic lens optical center to the measurement object is determined in accordance with the measured displacement between the double images. The distance measurement method using the double image reflected on the transparent board characterized by this.   The distance using the double image reflected on the transparent plate according to claim 1, wherein the displacement between the double images is measured by searching for an autocorrelation maximum position of the images along the displacement constraint between the double images. Measurement method. The double image is composed of a front surface reflection image I _s and rear surface reflection image I _r,
Distance from the optical center of the taking lens to the measurement target

Is obtained by the following formula:

However, theta _s at an angle disparity between the rear surface reflection image I _r and the surface reflection image I _s, n is a relative refractive index of the transparent plate to air, d is a thickness of the transparent plate, theta _The distance measurement method using a double image reflected on a transparent plate according to claim 1, wherein _i is an angle formed by an optical axis of a photographing lens of the image pickup device and the transparent plate. The double image is composed of a non-reflection transmission image (that is, a transmission image without reflection inside the transparent plate) and a double reflection transmission image (that is, a transmission image after being reflected twice inside the transparent plate). ,
Distance from the optical center of the taking lens to the measurement target

Is obtained by the following formula:

Where θ _s is the angular parallax between the non-reflective transmitted image and the twice reflected transmitted image, n is the relative refractive index of the transparent plate with respect to air, d is the thickness of the transparent plate, and θ _i The distance measuring method using a double image reflected on the transparent plate according to claim 1, wherein is an angle formed by an optical axis of a photographing lens of the imaging device and the transparent plate.   The distance measurement method using the double image reflected on the transparent plate according to claim 3 or 4, wherein the corresponding position search for the point of interest on the double image is performed only on the constraint line.   The distance measuring method using a double image reflected on a transparent plate according to claim 1, wherein the imaging device uses a solid-state imaging device. Using a distance measuring device composed of one transparent plate having non-parallelism and one image pickup device, a double image of a measurement object having a positional deviation reflected on the transparent plate is obtained by the image pickup device. Based on a single image obtained by photographing, the displacement constraint between the double images is geometrically derived from the positional relationship between the transparent plate and the photographing lens optical center of the imaging device, and the non-parallelism. The displacement between the double images in consideration of the non-parallel degree is measured by searching for the autocorrelation maximum position of the image along the displacement constraint between the derived double images, and the measured non-parallel degree A distance measurement method using a double image reflected on a transparent plate, wherein a distance from the optical center of the photographing lens to the measurement object is obtained in correspondence with a displacement between the double images in consideration of the above . The double image is composed of a front surface reflection image I _s and rear surface reflection image I _r,
Distance from front Symbol shooting lens optical center to the measurement target

Is obtained by the following formula:

Where d is the thickness of the transparent plate approximated to be constant ,
theta _s is the angle disparity between the rear surface reflection image I _r and the surface reflection image I _s,
θ _i is an angle formed by the optical axis of the imaging lens of the imaging apparatus and the transparent plate,
θ _p is an angle representing the non-parallel degree along the direction of the constraint straight line,
θ _o is the incident angle of transmitted light incident on the surface of the transparent plate from the measurement object,
θ _t is an incident angle when the transmitted light reaches the back surface of the transparent plate, a part of the transmitted light is reflected on the back surface, and is transmitted through the surface again.
D _c is the distance from the origin to the photographic lens optical center in a coordinate system having the origin at which the light beam from the measurement object is reflected by the surface of the transparent plate,
The distance measurement method using a double image reflected on a transparent plate according to claim 7, wherein _Do is a distance from the origin to the measurement target . 9. The distance measuring method using a double image reflected on a transparent plate according to claim 8, wherein the _Dc is measured by performing calibration of the distance measuring device.   The distance measurement method using the double image reflected on the transparent plate according to claim 8 or 9, wherein the corresponding position search for the target point on the double image is performed only on two constraining straight lines parallel to each other. .   The distance measuring method using a double image reflected on a transparent plate according to claim 7, wherein the imaging device uses a solid-state imaging device.   The distance measuring method using a double image reflected on a transparent plate according to any one of claims 7 to 10, wherein the transparent plate is a transparent acrylic plate, and the imaging device is a CCD camera.

Images