JP7410806B2

JP7410806B2 - stereo camera

Info

Publication number: JP7410806B2
Application number: JP2020104206A
Authority: JP
Inventors: 良太川俣; 和良山崎; 聰陳
Original assignee: Hitachi Ltd
Current assignee: Hitachi Ltd
Priority date: 2020-06-17
Filing date: 2020-06-17
Publication date: 2024-01-10
Anticipated expiration: 2040-06-17
Also published as: JP2021196544A

Description

本発明は、ステレオカメラに関し、例えば、双曲面ミラーを備えたステレオカメラに関する。 The present invention relates to a stereo camera, and for example, to a stereo camera equipped with a hyperboloid mirror.

自動運転の実現において、自動車の全周囲（３６０°）の物体を画像によって認識する方法、および、距離測定に基づいた３次元センシングが必要とされる。 In order to realize autonomous driving, a method for recognizing objects all around the vehicle (360°) using images and three-dimensional sensing based on distance measurement are required.

例えば、特許文献１には、双曲面ミラーを用いて全周囲カメラを構成し、画像から全周囲の物体の画像を取得する技術が開示されている。特許文献２には、魚眼ステレオカメラを用いて全周囲を３次元センシングする技術が開示されている。特許文献３には、双曲面ミラーとその反射光とを撮影可能な２組のカメラを上下に設けることで、ステレオカメラを構成し、全周囲の３次元空間認識を行う技術が開示されている。特許文献４には、２枚の双曲面ミラーを同じ方向に設け、１台のカメラによってそれらの反射光を撮影することで、ステレオカメラを構成し、全周囲の３次元空間認識を行う技術が開示されている。 For example, Patent Document 1 discloses a technique for configuring an all-around camera using a hyperboloid mirror and acquiring images of all-around objects from the images. Patent Document 2 discloses a technology for three-dimensional sensing of the entire surrounding area using a fisheye stereo camera. Patent Document 3 discloses a technology that configures a stereo camera by providing two sets of cameras above and below that can photograph a hyperboloid mirror and its reflected light, and performs three-dimensional spatial recognition of the entire surroundings. . Patent Document 4 discloses a technology that configures a stereo camera by providing two hyperboloid mirrors in the same direction and photographing their reflected light with a single camera, and performs three-dimensional spatial recognition of the entire surroundings. Disclosed.

特許文献５には、互いの頂点が対向するように、上下に設けられた２つの双曲面ミラーが開示されている。そして、一方の双曲面ミラーは、互いの円錐定数が異なる外周側双曲面ミラーおよび内周側双曲面ミラーを備えている。非特許文献１には、互いに対向するように２枚の双曲面ミラーを設け、一方の双曲面ミラーの内部にカメラを設けることで、全周囲の３次元検知を行う技術が開示されている。 Patent Document 5 discloses two hyperboloid mirrors provided one above the other so that their apexes face each other. One of the hyperboloid mirrors includes an outer hyperboloid mirror and an inner hyperboloid mirror having different conic constants. Non-Patent Document 1 discloses a technique in which two hyperboloid mirrors are provided to face each other and a camera is provided inside one of the hyperboloid mirrors to perform three-dimensional detection of the entire surroundings.

特開２００３－１３２３４８号公報Japanese Patent Application Publication No. 2003-132348 国際公開第２０１３／１６５００６号International Publication No. 2013/165006 特開２０１０－２５６２９６号公報Japanese Patent Application Publication No. 2010-256296 特開平１１－９５３４４号公報Japanese Patent Application Publication No. 11-95344 特開２０２０－４６２６２号公報JP2020-46262A

インターネット掲載ビデオ動画「Visual Odometry with a Single-Camera Stereo Omnidirectional System」 https://ubuntuslave.github.io/project/vo_sos/Video posted on the Internet "Visual Odometry with a Single-Camera Stereo Omnidirectional System" https://ubuntuslave.github.io/project/vo_sos/

夜間における物体の検知および距離測定は、可視光線だけではなく遠赤外線にも対応していることが必要とされる。また、建築現場などの作業現場においても、画像によって全周囲の物体を把握すること、および、距離測定に基づいた３次元センシングが必要とされる。この場合も、夜間における物体の把握および物体の温度情報の取得のために、３次元センシングが遠赤外線に対応していることが必要とされる。 Detecting objects and measuring distances at night requires not only visible light but also far-infrared light. Furthermore, at work sites such as construction sites, it is necessary to grasp all surrounding objects using images and three-dimensional sensing based on distance measurement. In this case as well, three-dimensional sensing is required to be compatible with far-infrared rays in order to grasp the object and obtain temperature information of the object at night.

しかしながら、上述の特許文献１～５および非特許文献１には、波長に関する記述はなく、多波長を用いた情報の取得が想定されていない。本願において開示される発明は、このような状況を鑑みて成されたものであり、ステレオカメラ（全周囲カメラ）におけるある波長での３次元センシング情報に、他の波長での情報を付加することを主な目的とする。 However, the above-mentioned Patent Documents 1 to 5 and Non-Patent Document 1 do not include any description regarding wavelengths, and do not assume the acquisition of information using multiple wavelengths. The invention disclosed in this application was made in view of this situation, and is a method of adding information at other wavelengths to three-dimensional sensing information at a certain wavelength in a stereo camera (all-around camera). The main purpose is

その他の課題および新規な特徴は、本明細書の記述および添付図面から明らかになる。 Other objects and novel features will become apparent from the description herein and the accompanying drawings.

本願において開示される実施の形態のうち、代表的なものの概要を簡単に説明すれば、次のとおりである。 A brief overview of typical embodiments disclosed in this application will be as follows.

一実施の形態におけるステレオカメラは、凸型の第１双曲面ミラーと、前記第１双曲面ミラーと中心軸を共通とし、前記第１双曲面ミラーと離れて設けられ、且つ、その頂点が前記第１双曲面ミラーの頂点と対向する凸型の第２双曲面ミラーと、前記第１双曲面ミラーの頂点付近に設けられ、且つ、第１波長の光を収束させるための第１結像光学系と、前記第１結像光学系を介して入射した光に基づいて、前記第１波長の光の出力画像を生成するための第１イメージセンサと、前記第２双曲面ミラーの頂点付近に設けられ、且つ、第２波長の光を収束させるための第２結像光学系と、前記第２結像光学系を介して入射した光に基づいて、前記第２波長の光の出力画像を生成するための第２イメージセンサと、を備える。 In one embodiment, the stereo camera includes a convex first hyperboloid mirror that shares a central axis with the first hyperboloid mirror, is provided apart from the first hyperboloid mirror, and has an apex that is located at a distance from the first hyperboloid mirror. a convex second hyperboloid mirror facing the apex of the first hyperboloid mirror; and a first imaging optical system provided near the apex of the first hyperboloid mirror for converging light of a first wavelength. a first image sensor for generating an output image of light of the first wavelength based on the light incident through the first imaging optical system; and a first image sensor near the apex of the second hyperboloid mirror. and a second imaging optical system for converging the light of the second wavelength, and an output image of the light of the second wavelength based on the light incident through the second imaging optical system. and a second image sensor for generating the image.

一実施の形態によれば、ステレオカメラにおいて、ある波長での３次元センシング情報に、他の波長での情報を付加することができる。 According to one embodiment, in a stereo camera, three-dimensional sensing information at one wavelength can be supplemented with information at other wavelengths.

実施の形態１におけるステレオカメラを示す模式図である。1 is a schematic diagram showing a stereo camera in Embodiment 1. FIG. 実施の形態１におけるステレオカメラを示す断面図である。1 is a cross-sectional view showing a stereo camera in Embodiment 1. FIG. 実施の形態１において取得される第１波長の光の出力画像を示す模式図である。FIG. 3 is a schematic diagram showing an output image of light of a first wavelength obtained in the first embodiment. 実施の形態１において取得される第２波長の光の出力画像を示す模式図である。FIG. 3 is a schematic diagram showing an output image of light of a second wavelength acquired in Embodiment 1. FIG. 実施の形態１における各双曲面ミラーのパラメータを示す表である。3 is a table showing parameters of each hyperboloid mirror in Embodiment 1. FIG. 実施の形態２におけるステレオカメラを示す断面図である。FIG. 3 is a cross-sectional view showing a stereo camera in a second embodiment. 実施の形態３におけるステレオカメラを示す模式図である。FIG. 7 is a schematic diagram showing a stereo camera in Embodiment 3. 実施の形態３において取得される第２波長の光の出力画像を示す模式図である。7 is a schematic diagram showing an output image of light of a second wavelength acquired in Embodiment 3. FIG. 実施の形態４におけるステレオカメラを示す模式図である。FIG. 7 is a schematic diagram showing a stereo camera in Embodiment 4. 実施の形態４におけるステレオカメラを示す断面図である。FIG. 7 is a cross-sectional view showing a stereo camera in Embodiment 4. 実施の形態４において取得される第２波長の光の出力画像を示す模式図である。7 is a schematic diagram showing an output image of light of a second wavelength acquired in Embodiment 4. FIG.

以下、実施の形態を図面に基づいて詳細に説明する。なお、実施の形態を説明するための全図において、同一の機能を有する部材には同一の符号を付し、その繰り返しの説明は省略する。また、以下の実施の形態では、特に必要なとき以外は同一または同様な部分の説明を原則として繰り返さない。 Hereinafter, embodiments will be described in detail based on the drawings. In addition, in all the drawings for explaining the embodiment, members having the same function are given the same reference numerals, and repeated explanation thereof will be omitted. Furthermore, in the following embodiments, descriptions of the same or similar parts will not be repeated in principle unless particularly necessary.

（実施の形態１）
＜ステレオカメラ１０の構成＞
以下に図１～図５を用いて、実施の形態１におけるステレオカメラ１０について説明する。ステレオカメラ１０は、少なくとも２種類以上の波長（多波長）を扱うことができ、全周囲（３６０°）における物体の検知を行うことができる多波長全周囲カメラである。 (Embodiment 1)
<Configuration of stereo camera 10>
The stereo camera 10 according to the first embodiment will be described below using FIGS. 1 to 5. The stereo camera 10 is a multi-wavelength omnidirectional camera that can handle at least two types of wavelengths (multiple wavelengths) and can detect objects all around (360°).

図１に示されるように、ステレオカメラ１０は、上側双曲面ミラー２０、下側双曲面ミラー３０、第１波長の光を収束させるための結像光学系４１、第２波長の光を収束させるための結像光学系４２、第１波長の光の出力画像を生成するためのイメージセンサ５１、第２波長の光の出力画像を生成するためのイメージセンサ５２、駆動制御部８０および画像処理部９０を備える。 As shown in FIG. 1, the stereo camera 10 includes an upper hyperboloid mirror 20, a lower hyperboloid mirror 30, an imaging optical system 41 for converging light of a first wavelength, and a focusing optical system 41 for converging light of a second wavelength. an image forming optical system 42, an image sensor 51 for generating an output image of light of a first wavelength, an image sensor 52 for generating an output image of light of a second wavelength, a drive control section 80, and an image processing section. 90.

上側双曲面ミラー２０および下側双曲面ミラー３０は、それぞれ凸型の双曲面を有するミラーである。下側双曲面ミラー３０は、上側双曲面ミラー２０と中心軸ＣＡを共通とし、上側双曲面ミラー２０と離れて設けられている。下側双曲面ミラー３０の頂点は、上側双曲面ミラー２０の頂点と対向している。 The upper hyperboloid mirror 20 and the lower hyperboloid mirror 30 are mirrors each having a convex hyperboloid surface. The lower hyperboloid mirror 30 shares a central axis CA with the upper hyperboloid mirror 20 and is provided apart from the upper hyperboloid mirror 20. The apex of the lower hyperboloid mirror 30 faces the apex of the upper hyperboloid mirror 20.

また、下側双曲面ミラー３０は、外周側双曲面ミラー３１および内周側双曲面ミラー３２を含む。外周側双曲面ミラー３１は、内周側双曲面ミラー３２と中心軸ＣＡを共通とし、内周側双曲面ミラー３２の外周に設けられている。なお、外周側双曲面ミラー３１と内周側双曲面ミラー３２との境界には、多少の段差があってもよい。また、後で詳細に説明するが、内周側双曲面ミラー３２の円錐定数κの絶対値は、外周側双曲面ミラー３１の円錐定数κの絶対値と異なり、外周側双曲面ミラー３１の円錐定数κの絶対値よりも大きくなるように設定されている。 Further, the lower hyperboloid mirror 30 includes an outer hyperboloid mirror 31 and an inner hyperboloid mirror 32. The outer hyperboloid mirror 31 shares a common central axis CA with the inner hyperboloid mirror 32 and is provided on the outer periphery of the inner hyperboloid mirror 32 . Note that there may be some level difference at the boundary between the outer hyperboloid mirror 31 and the inner hyperboloid mirror 32. Further, as will be described in detail later, the absolute value of the conic constant κ of the inner hyperboloid mirror 32 is different from the absolute value of the conic constant κ of the outer hyperboloid mirror 31, and the absolute value of the conic constant κ of the outer hyperboloid mirror 31 is It is set to be larger than the absolute value of the constant κ.

また、結像光学系４１、結像光学系４２、イメージセンサ５１およびイメージセンサ５２も、中心軸ＣＡと重なる位置に設けられている。 Further, the imaging optical system 41, the imaging optical system 42, the image sensor 51, and the image sensor 52 are also provided at positions overlapping with the central axis CA.

実施の形態１では、第１波長の光は、例えば可視光線であり、第２波長の光は、例えば遠赤外線である。上側双曲面ミラー２０および下側双曲面ミラー３０（外周側双曲面ミラー３１、内周側双曲面ミラー３２）は、それぞれ、第１波長の光および第２波長の光を反射可能な材料からなり、例えば金属材料からなる鏡である。 In the first embodiment, the first wavelength light is, for example, visible light, and the second wavelength light is, for example, far infrared rays. The upper hyperboloid mirror 20 and the lower hyperboloid mirror 30 (the outer hyperboloid mirror 31 and the inner hyperboloid mirror 32) are each made of a material that can reflect light of the first wavelength and light of the second wavelength. , for example, a mirror made of metal material.

上側双曲面ミラー２０の頂点付近には、第１開口部６１が設けられ、下側双曲面ミラー３０の頂点付近には、第２開口部６２が設けられている。第１波長用の結像光学系４１は、上側双曲面ミラー２０の頂点付近に設けられ、第１開口部６１内に設けられている。第２波長用の結像光学系４２は、下側双曲面ミラー３０の頂点付近に設けられ、第２開口部６２内に設けられている。 A first opening 61 is provided near the apex of the upper hyperboloid mirror 20 , and a second opening 62 is provided near the apex of the lower hyperboloid mirror 30 . The imaging optical system 41 for the first wavelength is provided near the apex of the upper hyperboloid mirror 20 and is provided within the first opening 61 . The imaging optical system 42 for the second wavelength is provided near the apex of the lower hyperboloid mirror 30 and is provided within the second opening 62 .

結像光学系４１および結像光学系４２は、それぞれ１枚以上のレンズから成る。また、被写体１００から出射される光ＬＩは、第１波長の光ＬＩ１および光ＬＩ２と、第２波長の光ＬＩ３とを含む。 The imaging optical system 41 and the imaging optical system 42 each include one or more lenses. Furthermore, the light LI emitted from the subject 100 includes light LI1 and light LI2 having a first wavelength, and light LI3 having a second wavelength.

被写体１００から出射される光ＬＩのうち第１波長の光ＬＩ１および光ＬＩ２は、第１波長用の結像光学系４１を介して第１波長用のイメージセンサ５１に入射する。すなわち、結像光学系４１は、上側双曲面ミラー２０で反射され、更に下側双曲面ミラー３０の内周側双曲面ミラー３２で反射された第１波長の光ＬＩ１と、下側双曲面ミラー３０で反射された第１波長の光ＬＩ２とを収束する。その後、収束された光ＬＩ１および光ＬＩ２は、イメージセンサ５１に入射する。 Of the light LI emitted from the subject 100, light LI1 and light LI2 of the first wavelength enter the image sensor 51 for the first wavelength via the imaging optical system 41 for the first wavelength. That is, the imaging optical system 41 uses the first wavelength light LI1 reflected by the upper hyperboloid mirror 20 and further reflected by the inner hyperboloid mirror 32 of the lower hyperboloid mirror 30, and the lower hyperboloid mirror. 30 and the first wavelength light LI2 is converged. Thereafter, the converged light LI1 and light LI2 enter the image sensor 51.

同様に、被写体１００から出射される光ＬＩのうち第２波長の光ＬＩ３は、第２波長用の結像光学系４２を介して第２波長用のイメージセンサ５２に入射する。すなわち、結像光学系４２は、上側双曲面ミラー２０で反射された第２波長の光ＬＩ３を収束する。その後、収束された光ＬＩ３は、イメージセンサ５２に入射する。 Similarly, of the light LI emitted from the subject 100, the light LI3 of the second wavelength enters the image sensor 52 for the second wavelength via the imaging optical system 42 for the second wavelength. That is, the imaging optical system 42 converges the second wavelength light LI3 reflected by the upper hyperboloid mirror 20. Thereafter, the converged light LI3 enters the image sensor 52.

第１波長用のイメージセンサ５１および第２波長用のイメージセンサ５２は、それぞれ、半導体デバイスであり、例えばＣＭＯＳイメージセンサである。イメージセンサ５１およびイメージセンサ５２は、駆動制御部８０に電気的に接続され、駆動制御部８０は、画像処理部９０に電気的に接続されている。 The image sensor 51 for the first wavelength and the image sensor 52 for the second wavelength are each semiconductor devices, such as CMOS image sensors. The image sensor 51 and the image sensor 52 are electrically connected to a drive control section 80, and the drive control section 80 is electrically connected to an image processing section 90.

駆動制御部８０は、イメージセンサ５１およびイメージセンサ５２の各々の駆動を制御する。イメージセンサ５１は、駆動制御部８０からの制御信号に従って、結像光学系４１を介して入射した第１波長の光ＬＩ１および光ＬＩ２に基づいて、第１波長の光の出力画像ＩＭ１を生成する。イメージセンサ５２は、駆動制御部８０からの制御信号に従って、結像光学系４２を介して入射した第２波長の光ＬＩ３に基づいて、第２波長の光の出力画像ＩＭ２を生成する。出力画像ＩＭ１および出力画像ＩＭ２は、図３および図４に示されている。 The drive control unit 80 controls the drive of each of the image sensor 51 and the image sensor 52. The image sensor 51 generates a first wavelength light output image IM1 based on the first wavelength light LI1 and light LI2 that are incident through the imaging optical system 41 in accordance with a control signal from the drive control unit 80. . The image sensor 52 generates an output image IM2 of light of the second wavelength based on the light LI3 of the second wavelength incident through the imaging optical system 42 in accordance with a control signal from the drive control unit 80. Output image IM1 and output image IM2 are shown in FIGS. 3 and 4.

駆動制御部８０は、イメージセンサ５１から出力された出力画像ＩＭ１と、イメージセンサ５２から出力された出力画像ＩＭ２とを画像処理部９０へ供給する。画像処理部９０は、供給された出力画像ＩＭ１に基づいて、被写体１００の第１波長の光の３次元位置情報を生成する。また、画像処理部９０は、供給された出力画像ＩＭ２に基づいて、被写体１００の第２波長の光の画像情報を生成し、第２波長の光の画像情報を第１波長の光の３次元位置情報に足し合わせる。 The drive control unit 80 supplies the output image IM1 output from the image sensor 51 and the output image IM2 output from the image sensor 52 to the image processing unit 90. The image processing unit 90 generates three-dimensional position information of the first wavelength light of the subject 100 based on the supplied output image IM1. Further, the image processing unit 90 generates image information of the light of the second wavelength of the subject 100 based on the supplied output image IM2, and converts the image information of the light of the second wavelength into a three-dimensional image of the light of the first wavelength. Add to location information.

このように、ステレオカメラ１０では、ある波長での被写体の全周囲の３次元位置情報と、別の波長で撮影した全周囲の画像情報とが同時に得られる。なお、第２波長の光として遠赤外線を利用した場合、第２波長の光の画像情報は、暗視画像または熱画像である。 In this manner, the stereo camera 10 can simultaneously obtain three-dimensional position information around the object at a certain wavelength and image information around the object photographed at a different wavelength. Note that when far infrared rays are used as the second wavelength light, the image information of the second wavelength light is a night vision image or a thermal image.

ここで、上側双曲面ミラー２０、外周側双曲面ミラー３１および内周側双曲面ミラー３２などの双曲面ミラーの一般的な性質について説明する。 Here, general properties of hyperboloid mirrors such as the upper hyperboloid mirror 20, the outer hyperboloid mirror 31, and the inner hyperboloid mirror 32 will be described.

上側双曲面ミラー２０、外周側双曲面ミラー３１および内周側双曲面ミラー３２の各々の双曲面は、以下の数式１によって表される２次曲面において、円錐定数κが－１よりも小さい場合に相当する。ここで、数式１におけるｚ（ｒ）は、光軸上の頂点を原点とする光軸方向の面のサグ量である。ｃは、光軸上の曲率（軸上曲率）である。ｒは、光軸からの半径座標である。 Each of the hyperboloids of the upper hyperboloid mirror 20, the outer hyperboloid mirror 31, and the inner hyperboloid mirror 32 is a quadratic surface expressed by the following formula 1, and when the conic constant κ is smaller than -1. corresponds to Here, z(r) in Equation 1 is the amount of sag of a surface in the optical axis direction whose origin is the vertex on the optical axis. c is the curvature on the optical axis (on-axis curvature). r is the radial coordinate from the optical axis.

一般に、双曲面は２つの焦点を有する。２つの焦点の座標は、面頂点を基準として以下の数式２によって表される。 Generally, a hyperboloid has two foci. The coordinates of the two focal points are expressed by Equation 2 below with respect to the surface apex.

なお、数式２における±が＋である場合、それは、双曲面の内側にある焦点の座標を表す。数式２における±が－である場合、それは、双曲面の外側にある焦点の座標を表す。以下では、双曲面の内側にある焦点を第１焦点と称し、双曲面の外側にある焦点を第２焦点と称する。 Note that when ± in Equation 2 is +, it represents the coordinates of the focal point inside the hyperboloid. When ± in Equation 2 is -, it represents the coordinates of the focal point outside the hyperboloid. Hereinafter, the focus located inside the hyperboloid will be referred to as the first focus, and the focus located outside the hyperboloid will be referred to as the second focus.

双曲面ミラーは、第１焦点に向かう光線を双曲面で反射した場合、その反射光を第２焦点に集光させる性質を有する。逆に、双曲面ミラーは、第２焦点から出射された光を、第１焦点から出射された光のように反射する性質を有する。 A hyperboloid mirror has a property of condensing the reflected light at a second focal point when a ray of light directed toward a first focal point is reflected by a hyperbolic surface. Conversely, a hyperboloid mirror has the property of reflecting the light emitted from the second focal point like the light emitted from the first focal point.

以下に図２を用いて、上側双曲面ミラー２０および下側双曲面ミラー３０（外周側双曲面ミラー３１、内周側双曲面ミラー３２）において反射される光ＬＩ１～ＬＩ３について説明する。 The lights LI1 to LI3 reflected by the upper hyperboloid mirror 20 and the lower hyperboloid mirror 30 (outer hyperboloid mirror 31, inner hyperboloid mirror 32) will be described below with reference to FIG.

図２に示されるように、上側双曲面ミラー２０の第２焦点２０ｂは、内周側双曲面ミラー３２の第１焦点３２ａと略一致する。また、この一致点は、外周側双曲面ミラー３１の第１焦点３１ａと一致する必要はないが、第１焦点３１ａの近傍に位置する。 As shown in FIG. 2, the second focal point 20b of the upper hyperbolic mirror 20 substantially coincides with the first focal point 32a of the inner hyperbolic mirror 32. Further, this matching point does not need to match the first focal point 31a of the outer circumferential hyperboloid mirror 31, but is located near the first focal point 31a.

また、外周側双曲面ミラー３１の第２焦点３１ｂは、内周側双曲面ミラー３２の第２焦点３２ｂと略一致する。この一致点には、結像光学系４１が設けられている。この一致点は、上側双曲面ミラー２０の第１焦点２０ａと一致する必要はないが、第１焦点２０ａの近傍に位置する。 Further, the second focal point 31b of the outer hyperboloid mirror 31 substantially coincides with the second focal point 32b of the inner hyperboloid mirror 32. An imaging optical system 41 is provided at this coincident point. This coincident point does not need to coincide with the first focal point 20a of the upper hyperboloid mirror 20, but is located near the first focal point 20a.

このような関係によって、被写体１００からの光ＬＩのうち、上側双曲面ミラー２０の第１焦点２０ａ（上側視点）に向かう光ＬＩ１は、上側双曲面ミラー２０によって反射され、上側双曲面ミラー２０の第２焦点２０ｂに集光される。 Due to this relationship, among the light LI from the subject 100, the light LI1 directed toward the first focal point 20a (upper viewpoint) of the upper hyperboloid mirror 20 is reflected by the upper hyperboloid mirror 20, and The light is focused on the second focal point 20b.

上側双曲面ミラー２０の第２焦点２０ｂは、内周側双曲面ミラー３２の第１焦点３２ａと略一致する。従って、第２焦点２０ｂに集光される光ＬＩ１は、内周側双曲面ミラー３２の第１焦点３２ａに向かう光と見做せるので、光ＬＩ１は、内周側双曲面ミラー３２で再び反射され、内周側双曲面ミラー３２の第２焦点３２ｂに集光される。 The second focal point 20b of the upper hyperboloid mirror 20 substantially coincides with the first focal point 32a of the inner hyperboloid mirror 32. Therefore, the light LI1 condensed at the second focal point 20b can be regarded as light heading toward the first focal point 32a of the inner hyperboloid mirror 32, so the light LI1 is reflected again by the inner hyperboloid mirror 32. The light is focused on the second focal point 32b of the inner hyperboloid mirror 32.

また、被写体１００からの光ＬＩのうち、外周側双曲面ミラー３１の第１焦点３１ａ（下側視点）に向かう光ＬＩ２は、外周側双曲面ミラー３１で反射され、外周側双曲面ミラー３１の第２焦点３１ｂに集光される。 Furthermore, among the light LI from the subject 100, the light LI2 directed toward the first focal point 31a (lower viewpoint) of the outer hyperboloid mirror 31 is reflected by the outer hyperboloid mirror 31, and the light LI2 is reflected by the outer hyperboloid mirror 31. The light is focused on the second focal point 31b.

このように集光された光ＬＩ１および光ＬＩ２は、第１波長用の結像光学系４１によって収束され、第１波長用のイメージセンサ５１に同時に入射する。 The light LI1 and the light LI2 thus condensed are converged by the imaging optical system 41 for the first wavelength, and simultaneously enter the image sensor 51 for the first wavelength.

図３に示されるように、第１波長用のイメージセンサ５１は、内周側に第１焦点２０ａ（上側視点）から見た被写体１００の像ＩＭ１ａを生成し、同時に、外周側に第１焦点３１ａ（下側視点）から見た被写体１００の像ＩＭ１ｂを生成する。このようにして、イメージセンサ５１は、像ＩＭ１ａおよび像ＩＭ１ｂからなる第１波長の光の出力画像ＩＭ１を生成できる。すなわち、イメージセンサ５１は、異なる２つの視点から被写体１００を撮像した場合に対応し、且つ、２つの第１波長の光の像ＩＭ１ａ、ＩＭ１ｂからなる出力画像ＩＭ１を生成可能である。 As shown in FIG. 3, the image sensor 51 for the first wavelength generates an image IM1a of the subject 100 seen from the first focal point 20a (upper viewpoint) on the inner circumferential side, and at the same time generates an image IM1a of the subject 100 on the outer circumferential side. An image IM1b of the subject 100 viewed from 31a (lower viewpoint) is generated. In this way, the image sensor 51 can generate the output image IM1 of the light of the first wavelength, which is composed of the image IM1a and the image IM1b. That is, the image sensor 51 corresponds to the case where the subject 100 is imaged from two different viewpoints, and can generate an output image IM1 consisting of two images IM1a and IM1b of light of the first wavelength.

一方、被写体１００からの光ＬＩのうち第２波長の光ＬＩ３は、上側双曲面ミラー２０によって反射され、第２波長用の結像光学系４２に集光される。集光された光ＬＩ３は、結像光学系４２によって収束され、第２波長用のイメージセンサ５２に入射する。図４に示されるように、第２波長用のイメージセンサ５２は、被写体１００の第２波長の光の像ＩＭ２ａからなる出力画像ＩＭ２を生成できる。 On the other hand, of the light LI from the subject 100, the light LI3 having the second wavelength is reflected by the upper hyperboloid mirror 20 and is focused on the imaging optical system 42 for the second wavelength. The collected light LI3 is converged by the imaging optical system 42 and enters the image sensor 52 for the second wavelength. As shown in FIG. 4, the image sensor 52 for the second wavelength can generate an output image IM2 consisting of an image IM2a of the subject 100 of light at the second wavelength.

このようにして得られた第１波長の光の出力画像ＩＭ１および第２波長の光の出力画像ＩＭ２は、駆動制御部８０を介して画像処理部９０に供給される。画像処理部９０は、出力画像ＩＭ１を、像ＩＭ１ａが写されている内周領域と、像ＩＭ１ｂが写されている外周領域とに分離し、それぞれに射影変換処理を行う。そして、画像処理部９０は、射影変換処理で得られた２枚の画像を用いてステレオマッチング処理を行い、被写体１００の第１波長の光の３次元位置情報を生成する。 The output image IM1 of the light of the first wavelength and the output image IM2 of the light of the second wavelength thus obtained are supplied to the image processing section 90 via the drive control section 80. The image processing unit 90 separates the output image IM1 into an inner peripheral area where the image IM1a is shown and an outer peripheral area where the image IM1b is shown, and performs projective transformation processing on each. Then, the image processing unit 90 performs stereo matching processing using the two images obtained by the projective transformation processing, and generates three-dimensional position information of the light of the first wavelength of the subject 100.

また、画像処理部９０は、第２波長の光の出力画像ＩＭ２から暗視画像または熱画像などの第２波長の光の画像情報を生成し、第２波長の光の画像情報を第１波長の光の３次元位置情報と統合する。 The image processing unit 90 also generates image information of the light of the second wavelength, such as a night vision image or a thermal image, from the output image IM2 of the light of the second wavelength, and converts the image information of the light of the second wavelength into the image information of the light of the first wavelength. This is integrated with the 3D position information of the light.

以上のように、実施の形態１におけるステレオカメラ１０では、ある波長での被写体の全周囲の３次元位置情報と、別の波長で撮影した全周囲の画像情報とが同時に得られる。このため、全周囲の物体の３次元位置情報に、別の波長から得られる情報を付加させることができる。 As described above, the stereo camera 10 according to the first embodiment can simultaneously obtain three-dimensional positional information around the subject at a certain wavelength and image information around the entire circumference photographed at a different wavelength. Therefore, information obtained from another wavelength can be added to the three-dimensional position information of objects all around.

また、第１波長の光とは別の第２波長の光を利用しているので、第１波長の光だけでは物体の正確な検知が困難な状況においても、物体の検知を安定して行うことができる。例えば第２波長の光が遠赤外線である場合、物体の温度情報から画像情報を取得できる。これにより、物体の周囲の環境が、夜間または悪天候のような場合であっても、物体の検知を安定して行うことができる。すなわち、実施の形態１によれば、ステレオカメラにおける物体検知の冗長性を向上させることができる。 In addition, since it uses light with a second wavelength that is different from the light with the first wavelength, it can stably detect objects even in situations where it is difficult to accurately detect objects with only light with the first wavelength. be able to. For example, if the second wavelength light is far infrared rays, image information can be obtained from the temperature information of the object. As a result, the object can be stably detected even when the environment around the object is night or bad weather. That is, according to the first embodiment, the redundancy of object detection in the stereo camera can be improved.

＜上側双曲面ミラー２０、外周側双曲面ミラー３１および内周側双曲面ミラー３２と、その他の関係とについて＞
ステレオカメラ１０においては、内周側双曲面ミラー３２の円錐定数κの絶対値が、外周側双曲面ミラー３１の円錐定数κの絶対値と異なり、外周側双曲面ミラー３１の円錐定数κの絶対値よりも大きくなるように設定されている。これにより、第１波長の光の出力画像ＩＭ１においては、上側視点２０ａから見た被写体１００の像の大きさと、下側視点３１ａから見た被写体１００の像の大きさとが揃えられている。 <About the upper hyperboloid mirror 20, the outer hyperboloid mirror 31, the inner hyperboloid mirror 32, and other relationships>
In the stereo camera 10, the absolute value of the conic constant κ of the inner hyperboloid mirror 32 is different from the absolute value of the conic constant κ of the outer hyperboloid mirror 31; is set to be greater than the value. As a result, in the output image IM1 of the light of the first wavelength, the size of the image of the subject 100 seen from the upper viewpoint 20a is the same as the size of the image of the subject 100 seen from the lower viewpoint 31a.

第１波長の光の出力画像ＩＭ１における内周側の像および外周側の像のサイズを揃えることで、２つの像の解像度を合わせることができ、ステレオマッチング処理の精度を向上させることができる。 By making the sizes of the inner image and the outer image of the output image IM1 of the first wavelength light the same, the resolutions of the two images can be matched, and the accuracy of the stereo matching process can be improved.

また、実施の形態１におけるステレオカメラ１０では、上側双曲面ミラー２０には、下側双曲面ミラー３０よりも口径が大きい双曲面ミラーが採用されている。上側双曲面ミラー２０の口径を下側双曲面ミラー３０の口径よりも大きくすることで、下側視点３１ａの上向き画角の光線を上側視点２０ａで受光でき、ステレオ視を行うことができる上下の画角範囲を最大限まで広く確保することが可能となる。 Further, in the stereo camera 10 according to the first embodiment, a hyperboloid mirror having a larger aperture than the lower hyperboloid mirror 30 is used as the upper hyperboloid mirror 20. By making the aperture of the upper hyperboloid mirror 20 larger than the aperture of the lower hyperboloid mirror 30, the upper and lower viewpoints 20a can receive the light rays at the upward viewing angle of the lower viewpoint 31a, allowing stereo viewing. It is possible to secure the widest possible viewing angle range.

なお、仮に、上側双曲面ミラー２０の口径が下側双曲面ミラー３０の口径と略等しい場合、下側双曲面ミラー３０が外周側双曲面ミラー３１および内周側双曲面ミラー３２に分離されているので、下側視点３１ａの視野範囲は、必然的に狭くなってしまう。 Note that if the aperture of the upper hyperboloid mirror 20 is approximately equal to the aperture of the lower hyperboloid mirror 30, the lower hyperboloid mirror 30 is separated into an outer hyperboloid mirror 31 and an inner hyperboloid mirror 32. Therefore, the viewing range of the lower viewpoint 31a inevitably becomes narrower.

ところで、ステレオカメラ１０では、口径が大きい上側双曲面ミラー２０が上側に設けられ、口径が小さい下側双曲面ミラー３０が下側に設けられている。しかし、上側双曲面ミラー２０および下側双曲面ミラー３０の上下の位置関係が反対であってもよく、互いの頂点が対向していればよい。 By the way, in the stereo camera 10, the upper hyperboloid mirror 20 with a large diameter is provided on the upper side, and the lower hyperboloid mirror 30 with a small diameter is provided on the lower side. However, the vertical positional relationship of the upper hyperboloid mirror 20 and the lower hyperboloid mirror 30 may be reversed, as long as their apexes are opposite to each other.

図５は、上側双曲面ミラー２０、外周側双曲面ミラー３１および内周側双曲面ミラー３２に関する曲率半径ｒ、円錐定数κ、頂点位置および口径の一例を示している。 FIG. 5 shows an example of the radius of curvature r, conic constant κ, vertex position, and aperture regarding the upper hyperboloid mirror 20, the outer hyperboloid mirror 31, and the inner hyperboloid mirror 32.

ここでは、上側双曲面ミラー２０では、曲率半径ｒは１５ｍｍ、円錐定数κは－３．１、頂点位置は０ｍｍ、口径は３４ｍｍに設定されている。外周側双曲面ミラー３１では、曲率半径ｒは－１７ｍｍ、円錐定数κは－２．９４、頂点位置は－１８．５０５６ｍｍ、口径は２５ｍｍに設定されている。内周側双曲面ミラー３２では、曲率半径ｒは－２２ｍｍ、円錐定数κは－１３０、頂点位置は－１８．５０８６ｍｍ、口径は９ｍｍに設定されている。 Here, in the upper hyperboloid mirror 20, the radius of curvature r is set to 15 mm, the conic constant κ is set to -3.1, the apex position is set to 0 mm, and the aperture is set to 34 mm. In the outer circumference side hyperboloid mirror 31, the radius of curvature r is set to -17 mm, the conic constant κ is set to -2.94, the apex position is set to -18.5056 mm, and the aperture is set to 25 mm. The inner hyperboloid mirror 32 has a radius of curvature r of −22 mm, a conic constant κ of −130, an apex position of −18.5086 mm, and an aperture of 9 mm.

ただし、これらのパラメータは一例に過ぎず、ステレオカメラ１０に要求される仕様に応じて、適切なパラメータを有する上側双曲面ミラー２０、外周側双曲面ミラー３１および内周側双曲面ミラー３２を採用することができる。 However, these parameters are only examples, and depending on the specifications required for the stereo camera 10, the upper hyperboloid mirror 20, the outer hyperboloid mirror 31, and the inner hyperboloid mirror 32 having appropriate parameters are adopted. can do.

（実施の形態２）
以下に図６を用いて、実施の形態２におけるステレオカメラ１０を説明する。なお、以下では、主に実施の形態１との相違点について説明する。 (Embodiment 2)
The stereo camera 10 according to the second embodiment will be described below using FIG. 6. Note that, below, differences from Embodiment 1 will be mainly explained.

実施の形態２では、外周側双曲面ミラー３１および内周側双曲面ミラー３２を含む下側双曲面ミラー３０は、第１波長の光を反射可能であり、且つ、第２波長の光を透過可能な材料からなる。なお、内周側双曲面ミラー３２だけに、上記材料を適用してもよい。すなわち、下側双曲面ミラー３０の全部または一部が、上記材料で構成される。第１波長の光が可視光線であり、第２波長の光が遠赤外線である場合、上記材料の例として、例えばゲルマニウムが挙げられる。 In the second embodiment, the lower hyperboloid mirror 30 including the outer hyperboloid mirror 31 and the inner hyperboloid mirror 32 is capable of reflecting light of a first wavelength and transmitting light of a second wavelength. Made of possible materials. Note that the above material may be applied only to the inner hyperboloid mirror 32. That is, all or part of the lower hyperboloid mirror 30 is made of the above material. When the light of the first wavelength is visible light and the light of the second wavelength is far infrared rays, examples of the above-mentioned material include germanium.

図６に示されるように、第１波長の光ＬＩ１および光ＬＩ２の反射に関して、実施の形態２は、実施の形態１と同様であり、第１波長用のイメージセンサ５１で取得される出力画像ＩＭ１および第１波長の光の３次元位置情報に変更はない。 As shown in FIG. 6, the second embodiment is similar to the first embodiment with respect to the reflection of the first wavelength light LI1 and the light LI2, and the output image obtained by the image sensor 51 for the first wavelength. There is no change in the three-dimensional position information of IM1 and the first wavelength light.

一方、実施の形態２では、第２波長の光が下側双曲面ミラー３０を透過する。それ故、実施の形態１において下側双曲面ミラー３０の頂点付近に設けられていた第２開口部６２は、必須ではない。すなわち、下側双曲面ミラー３０に第２開口部６２を設けずに、第２波長用の結像光学系４２が下側双曲面ミラー３０に覆われていてもよい。 On the other hand, in the second embodiment, light of the second wavelength is transmitted through the lower hyperboloid mirror 30. Therefore, the second opening 62 provided near the apex of the lower hyperboloid mirror 30 in the first embodiment is not essential. That is, the imaging optical system 42 for the second wavelength may be covered by the lower hyperboloid mirror 30 without providing the second opening 62 in the lower hyperboloid mirror 30.

この場合、結像光学系４２およびイメージセンサ５２の配置位置の自由度を高めることができる。実施の形態１では、第２開口部６２は、結像光学系４２およびイメージセンサ５２の配置に依存して設けられている必要があるので、第２開口部６２を広くすることは、下側双曲面ミラー３０の領域を狭めることになる。そうすると、第１波長の光の出力画像ＩＭ１を構成する像ＩＭ１ａを写すための内周領域が、狭まってしまう。 In this case, the degree of freedom in the arrangement positions of the imaging optical system 42 and the image sensor 52 can be increased. In the first embodiment, the second opening 62 needs to be provided depending on the arrangement of the imaging optical system 42 and the image sensor 52, so widening the second opening 62 means that the lower side This will narrow the area of the hyperboloid mirror 30. In this case, the inner peripheral region for capturing the image IM1a that constitutes the output image IM1 of the light of the first wavelength becomes narrower.

実施の形態２では、図６に示されるように、例えば、結像光学系４２を下側に移動させ、上側双曲面ミラー２０の第２焦点２０ｂと、結像光学系４２とを略一致させている。このため、イメージセンサ５２で取得される第２波長の光の出力画像ＩＭ２を構成する像ＩＭ２ａの視点が、下側視点３１ａと一致し、第１波長の光の３次元位置情報と第２波長の光の画像情報との統合が、容易になる。 In the second embodiment, as shown in FIG. 6, for example, the imaging optical system 42 is moved downward so that the second focal point 20b of the upper hyperboloid mirror 20 and the imaging optical system 42 are substantially aligned. ing. Therefore, the viewpoint of the image IM2a constituting the output image IM2 of the light of the second wavelength acquired by the image sensor 52 coincides with the lower viewpoint 31a, and the three-dimensional position information of the light of the first wavelength and the second wavelength The integration of light with image information becomes easier.

（実施の形態３）
以下に図７および図８を用いて、実施の形態３におけるステレオカメラ１０を説明する。なお、以下では、主に実施の形態１との相違点について説明する。 (Embodiment 3)
The stereo camera 10 according to the third embodiment will be described below using FIGS. 7 and 8. Note that, below, differences from Embodiment 1 will be mainly explained.

実施の形態３では、第２波長用の結像光学系４２は、広角レンズである。 In the third embodiment, the imaging optical system 42 for the second wavelength is a wide-angle lens.

図７に示されるように、第１波長の光ＬＩ１および光ＬＩ２の反射に関して、実施の形態２は、実施の形態１と同様であり、第１波長用のイメージセンサ５１で取得される出力画像ＩＭ１および第１波長の光の３次元位置情報に変更はない。 As shown in FIG. 7, the second embodiment is similar to the first embodiment with respect to the reflection of the first wavelength light LI1 and the light LI2, and the output image obtained by the image sensor 51 for the first wavelength. There is no change in the three-dimensional position information of IM1 and the first wavelength light.

一方で、結像光学系４２には、上側双曲面ミラー２０で反射した第２波長の光ＬＩ３に加えて、第２波長の光ＬＩ４が直接入射する。 On the other hand, in addition to the second wavelength light LI3 reflected by the upper hyperboloid mirror 20, the second wavelength light LI4 directly enters the imaging optical system 42.

従って、図８に示されるように、第２波長用のイメージセンサ５２は、光ＬＩ３に基づいて、内周側に上側視点２０ａから見た被写体１００の像ＩＭ２ａを生成し、同時に、光ＬＩ４に基づいて、外周側に視点位置が結像光学系４２である被写体１００の像ＩＭ２ｂが得られる。このようにして、イメージセンサ５２は、像ＩＭ２ａおよび像ＩＭ２ｂからなる第２波長の光の出力画像ＩＭ３を生成できる。すなわち、イメージセンサ５２は、異なる２つの視点から被写体１００を撮像した場合に対応し、且つ、２つの第２波長の光の像ＩＭ２ａ、ＩＭ２ｂからなる出力画像ＩＭ２を生成可能である。 Therefore, as shown in FIG. 8, the image sensor 52 for the second wavelength generates an image IM2a of the subject 100 seen from the upper viewpoint 20a on the inner circumferential side based on the light LI3, and at the same time generates an image IM2a of the subject 100 seen from the upper viewpoint 20a based on the light LI3. Based on this, an image IM2b of the subject 100 whose viewpoint position is the imaging optical system 42 on the outer circumferential side is obtained. In this way, the image sensor 52 can generate an output image IM3 of light of the second wavelength consisting of the image IM2a and the image IM2b. That is, the image sensor 52 corresponds to the case where the subject 100 is imaged from two different viewpoints, and can generate an output image IM2 consisting of two images IM2a and IM2b of light of the second wavelength.

このようにして得られた第１波長の光の出力画像ＩＭ１および第２波長の光の出力画像ＩＭ３は、駆動制御部８０を介して画像処理部９０に供給される。画像処理部９０は、出力画像ＩＭ３を、像ＩＭ２ａが写されている内周領域と、像ＩＭ２ｂが写されている外周領域とに分離し、それぞれに射影変換処理を行う。そして、画像処理部９０は、射影変換処理で得られた２枚の画像においてステレオマッチング処理を行い、被写体１００の第２波長の光の３次元位置情報を生成し、生成された第２波長の光の３次元位置情報を第１波長の光の３次元位置情報と組みわせる。 The output image IM1 of the light of the first wavelength and the output image IM3 of the light of the second wavelength thus obtained are supplied to the image processing section 90 via the drive control section 80. The image processing unit 90 separates the output image IM3 into an inner circumferential area where the image IM2a is shown and an outer circumferential area where the image IM2b is shown, and performs projective transformation processing on each of them. The image processing unit 90 then performs stereo matching processing on the two images obtained by the projective transformation processing, generates three-dimensional position information of the second wavelength light of the subject 100, and generates three-dimensional position information of the second wavelength light of the subject 100. The three-dimensional position information of the light is combined with the three-dimensional position information of the light of the first wavelength.

実施の形態３では、第２波長の光の画像情報よりも情報量の多い第２波長の光の３次元位置情報が生成できるので、実施の形態１と比較して、第１波長とは異なる物体の性質を反映した３次元位置情報の取得と、３次元位置情報取得の冗長性の向上が可能となる。 In the third embodiment, it is possible to generate three-dimensional position information of the light of the second wavelength, which has a larger amount of information than the image information of the light of the second wavelength. It is possible to acquire three-dimensional position information that reflects the properties of an object and to improve the redundancy of acquiring three-dimensional position information.

なお、実施の形態３で開示した技術と、実施の形態２で開示した技術とを組み合わせて適用することも可能である。 Note that it is also possible to apply a combination of the technology disclosed in Embodiment 3 and the technology disclosed in Embodiment 2.

（実施の形態４）
以下に図９～図１１を用いて、実施の形態４におけるステレオカメラ１０を説明する。なお、以下では、主に実施の形態１との相違点について説明する。 (Embodiment 4)
The stereo camera 10 according to the fourth embodiment will be described below with reference to FIGS. 9 to 11. Note that, below, differences from Embodiment 1 will be mainly explained.

図９に示されるように、実施の形態４では、上側双曲面ミラー２０と下側双曲面ミラー３０との間に、中間双曲面ミラー７０が設けられている。中間双曲面ミラー７０は、凸型の双曲面を有するミラーであり、上側双曲面ミラー２０および下側双曲面ミラー３０と中心軸ＣＡを共通とする。中間双曲面ミラー７０の頂点は、下側双曲面ミラー３０の頂点と対向している。 As shown in FIG. 9, in the fourth embodiment, an intermediate hyperboloid mirror 70 is provided between the upper hyperboloid mirror 20 and the lower hyperboloid mirror 30. The intermediate hyperboloid mirror 70 is a mirror having a convex hyperboloid surface, and shares a central axis CA with the upper hyperboloid mirror 20 and the lower hyperboloid mirror 30. The apex of the intermediate hyperboloid mirror 70 faces the apex of the lower hyperboloid mirror 30.

図１０に示されるように、中間双曲面ミラー７０は、中間外周側双曲面ミラー７１および中間内周側双曲面ミラー７２を含む。中間外周側双曲面ミラー７１は、中間内周側双曲面ミラー７２と中心軸ＣＡを共通とし、中間内周側双曲面ミラー７２の外周に設けられている。なお、中間外周側双曲面ミラー７１と中間内周側双曲面ミラー７２との境界には、多少の段差があってもよい。また、中間内周側双曲面ミラー７２の円錐定数κの絶対値は、中間外周側双曲面ミラー７１の円錐定数κの絶対値と異なり、中間外周側双曲面ミラー７１の円錐定数κの絶対値よりも大きくなるように設定されている。 As shown in FIG. 10, the intermediate hyperboloid mirror 70 includes an intermediate outer hyperboloid mirror 71 and an intermediate inner hyperboloid mirror 72. The intermediate outer hyperboloid mirror 71 shares a central axis CA with the intermediate inner hyperboloid mirror 72, and is provided on the outer periphery of the intermediate inner hyperboloid mirror 72. Note that there may be some level difference at the boundary between the intermediate outer circumference side hyperboloid mirror 71 and the intermediate inner circumference side hyperboloid mirror 72. Further, the absolute value of the conic constant κ of the intermediate inner hyperboloid mirror 72 is different from the absolute value of the conic constant κ of the intermediate outer hyperboloid mirror 71; is set to be larger than .

中間外周側双曲面ミラー７１および中間内周側双曲面ミラー７２を含む中間双曲面ミラー７０は、第１波長の光を透過可能であり、且つ、第２波長の光を反射可能な材料からなる。なお、中間内周側双曲面ミラー７２だけに、上記材料を適用してもよい。すなわち、中間双曲面ミラー７０の全部または一部が、上記材料で構成される。第１波長の光が可視光線であり、第２波長の光が遠赤外線である場合、上記材料の例として、例えばガラスが挙げられる。 The intermediate hyperboloid mirror 70 including the intermediate outer hyperboloid mirror 71 and the intermediate inner hyperboloid mirror 72 is made of a material that can transmit light of a first wavelength and reflect light of a second wavelength. . Note that the above material may be applied only to the intermediate inner circumference side hyperboloid mirror 72. That is, all or part of the intermediate hyperboloid mirror 70 is made of the above material. When the light of the first wavelength is visible light and the light of the second wavelength is far infrared rays, examples of the material include glass.

なお、第１波長の光ＬＩ１および光ＬＩ２の反射に関して、実施の形態４は、実施の形態１と同様であり、第１波長用のイメージセンサ５１で取得される出力画像ＩＭ１および第１波長の光の３次元位置情報に変更はない。 Regarding the reflection of the first wavelength light LI1 and the light LI2, the fourth embodiment is similar to the first embodiment, and the output image IM1 obtained by the image sensor 51 for the first wavelength and the reflection of the first wavelength light LI1 and the first wavelength light LI2 are the same as the first embodiment. There is no change in the three-dimensional position information of the light.

図１０に示されるように、中間外周側双曲面ミラー７１の第２焦点７１ｂは、中間内周側双曲面ミラー７２の第２焦点７２ｂと略一致する。この一致点には、第２波長用の結像光学系４２が設けられている。この一致点は、外周側双曲面ミラー３１の第１焦点３１ａと一致する必要はないが、第１焦点３１ａの近傍に位置する。 As shown in FIG. 10, the second focal point 71b of the intermediate outer hyperboloid mirror 71 substantially coincides with the second focal point 72b of the intermediate inner hyperboloid mirror 72. As shown in FIG. An imaging optical system 42 for the second wavelength is provided at this coincident point. This coincident point does not need to coincide with the first focal point 31a of the outer hyperboloid mirror 31, but is located near the first focal point 31a.

また、外周側双曲面ミラー３１の第２焦点３１ｂは、内周側双曲面ミラー３２の第２焦点３２ｂと、中間内周側双曲面ミラー７２の第１焦点７２ａとに略一致する。また、この一致点は、中間外周側双曲面ミラー７１の第１焦点７１ａと一致する必要はないが、第１焦点７１ａの近傍に位置する。 Further, the second focus 31b of the outer hyperboloid mirror 31 substantially coincides with the second focus 32b of the inner hyperboloid mirror 32 and the first focus 72a of the intermediate inner hyperboloid mirror 72. Further, this matching point does not need to match the first focal point 71a of the intermediate outer circumferential side hyperboloid mirror 71, but is located near the first focal point 71a.

このような関係によって、実施の形態４では、被写体１００からの光ＬＩのうち、中間外周側双曲面ミラー７１の第１焦点７１ａ（第２波長の上側視点）に向かう第２波長の光ＬＩ５は、中間外周側双曲面ミラー７１によって反射されて、中間外周側双曲面ミラー７１の第２焦点７１ｂに集光される。 Due to this relationship, in the fourth embodiment, out of the light LI from the subject 100, the light LI5 of the second wavelength directed toward the first focal point 71a (the upper viewpoint of the second wavelength) of the intermediate outer circumferential side hyperboloid mirror 71 is , is reflected by the intermediate outer circumferential side hyperboloid mirror 71, and is focused on the second focal point 71b of the intermediate outer circumferential side hyperboloid mirror 71.

外周側双曲面ミラー３１の第２焦点３１ｂは、中間内周側双曲面ミラー７２の第１焦点７２ａと略一致する。従って、外周側双曲面ミラー３１で反射し、外周側双曲面ミラー３１の第２焦点３１ｂに集光される光ＬＩ３は、中間内周側双曲面ミラー７２の第１焦点７２ａに向かう光と見做せるので、この光ＬＩ３は、中間内周側双曲面ミラー７２で再び反射され、中間内周側双曲面ミラー７２の第２焦点７２ｂに集光される。 The second focal point 31b of the outer hyperboloid mirror 31 substantially coincides with the first focal point 72a of the intermediate inner hyperboloid mirror 72. Therefore, the light LI3 reflected by the outer hyperboloid mirror 31 and focused on the second focal point 31b of the outer hyperboloid mirror 31 can be regarded as light heading toward the first focus 72a of the intermediate inner hyperboloid mirror 72. Therefore, this light LI3 is reflected again by the intermediate inner hyperboloid mirror 72 and focused on the second focal point 72b of the intermediate inner hyperboloid mirror 72.

このように集光された光ＬＩ３および光ＬＩ５は、第２波長用の結像光学系４２によって収束され、第２波長用のイメージセンサ５２に同時に入射する。 The light LI3 and the light LI5 thus collected are converged by the imaging optical system 42 for the second wavelength, and simultaneously enter the image sensor 52 for the second wavelength.

図１１に示されるように、第２波長用のイメージセンサ５２は、外周側に第１焦点７１ａ（上側視点）から見た被写体１００の像ＩＭ２ａを生成し、同時に、内周側に第２焦点７１ｂ（下側視点）から見た被写体１００の像ＩＭ２ｂを生成する。このようにして、イメージセンサ５１は、像ＩＭ２ａおよび像ＩＭ２ｂからなる第２波長の光の出力画像ＩＭ４を生成できる。すなわち、イメージセンサ５２は、異なる２つの視点から被写体１００を撮像した場合に対応し、且つ、２つの第２波長の光の像ＩＭ２ａ、ＩＭ２ｂからなる出力画像ＩＭ４を生成可能である。 As shown in FIG. 11, the image sensor 52 for the second wavelength generates an image IM2a of the subject 100 seen from a first focal point 71a (upper viewpoint) on the outer circumferential side, and at the same time generates an image IM2a of the subject 100 on the inner circumferential side. An image IM2b of the subject 100 viewed from 71b (lower viewpoint) is generated. In this way, the image sensor 51 can generate an output image IM4 of light of the second wavelength, which is composed of the image IM2a and the image IM2b. That is, the image sensor 52 corresponds to the case where the subject 100 is imaged from two different viewpoints, and can generate an output image IM4 consisting of two images IM2a and IM2b of light of the second wavelength.

このようにして得られた第１波長の光の出力画像ＩＭ１および第２波長の光の出力画像ＩＭ４は、駆動制御部８０を介して画像処理部９０に供給される。画像処理部９０は、出力画像ＩＭ４を、像ＩＭ２ａが写されている外周領域と、像ＩＭ２ｂが写されている内周領域とに分離し、それぞれに射影変換処理を行う。そして、画像処理部９０は、射影変換処理で得られた２枚の画像を用いてステレオマッチング処理を行い、被写体１００の第２波長の光の３次元位置情報を生成し、第１波長の光の３次元位置情報と第２波長の光の３次元位置情報ＰＩ２とを組みわせる。 The output image IM1 of the light of the first wavelength and the output image IM4 of the light of the second wavelength thus obtained are supplied to the image processing section 90 via the drive control section 80. The image processing unit 90 separates the output image IM4 into an outer peripheral area where the image IM2a is shown and an inner peripheral area where the image IM2b is shown, and performs projective transformation processing on each of them. Then, the image processing unit 90 performs stereo matching processing using the two images obtained by the projective transformation processing, generates three-dimensional position information of the second wavelength light of the subject 100, and generates three-dimensional position information of the second wavelength light of the subject 100. The three-dimensional position information of the light of the second wavelength is combined with the three-dimensional position information PI2 of the light of the second wavelength.

ここで、中間外周側双曲面ミラー７１の第１焦点７１ａと、上側双曲面ミラー２０の第１焦点２０ａとを一致させることが可能である。この場合、第１波長の光の出力画像ＩＭ１の内周側の像の視点位置（第１焦点２０ａ）と、第２波長の光の出力画像ＩＭ４の外周側の像の視点位置（第１焦点７１ａ）とを一致させることが可能となる。これにより、出力画像ＩＭ１から得られる第１波長の光の３次元位置情報と、出力画像ＩＭ４から得られる第２波長の光の３次元位置情報との統合が、容易になる。 Here, it is possible to make the first focal point 71a of the intermediate outer circumference side hyperboloid mirror 71 and the first focal point 20a of the upper hyperboloid mirror 20 coincide. In this case, the viewpoint position (first focal point 20a) of the image on the inner peripheral side of the output image IM1 of light of the first wavelength, and the viewpoint position (first focal point) of the image on the outer peripheral side of the output image IM4 of light of the second wavelength. 71a). This facilitates the integration of the three-dimensional positional information of the first wavelength light obtained from the output image IM1 and the three-dimensional positional information of the second wavelength light obtained from the output image IM4.

実施の形態４でも、第２波長の光の画像情報よりも情報量の多い第２波長の光の３次元位置情報が生成できる。従って、第１波長とは異なる物体の性質を反映した３次元位置情報の取得と、３次元位置情報取得の冗長性の向上とが可能となる。 In the fourth embodiment as well, it is possible to generate three-dimensional position information of the light of the second wavelength, which has a larger amount of information than image information of the light of the second wavelength. Therefore, it is possible to acquire three-dimensional position information that reflects the properties of the object that are different from the first wavelength, and to improve the redundancy of acquiring three-dimensional position information.

以上、本発明をその実施の形態に基づき具体的に説明したが、本発明は、上記実施の形態に限定されるものではなく、その要旨を逸脱しない範囲で種々の変更が可能である。 Although the present invention has been specifically described above based on the embodiments thereof, the present invention is not limited to the above embodiments, and various changes can be made without departing from the gist thereof.

１０ステレオカメラ（多波長全周囲カメラ）
２０上側双曲面ミラー
２０ａ上側双曲面ミラー２０の第１焦点
２０ｂ上側双曲面ミラー２０の第２焦点
３０下側双曲面ミラー
３１外周側双曲面ミラー
３１ａ外周側双曲面ミラー３１の第１焦点
３１ｂ外周側双曲面ミラー３１の第２焦点
３２内周側双曲面ミラー
３２ａ内周側双曲面ミラー３２の第１焦点
３２ｂ内周側双曲面ミラー３２の第２焦点
４１第１波長用の結像光学系
４２第２波長用の結像光学系
５１第１波長用のイメージセンサ
５２第２波長用のイメージセンサ
６１第１開口部
６２第２開口部
７０中間双曲面ミラー
７１中間外周側双曲面ミラー
７１ａ中間外周側双曲面ミラー７１の第１焦点
７１ｂ中間外周側双曲面ミラー７１の第２焦点
７２中間内周側双曲面ミラー
７２ａ中間内周側双曲面ミラー７２の第１焦点
７２ｂ中間内周側双曲面ミラー７２の第２焦点
８０駆動制御部
９０画像処理部
１００被写体
ＣＡ中心軸
ＩＭ１第１波長の光の出力画像
ＩＭ１ａ第１波長の光の出力画像ＩＭ１の上側視点からの像
ＩＭ１ｂ第１波長の光の出力画像ＩＭ１の下側視点からの像
ＩＭ２、ＩＭ３、ＩＭ４第２波長の光の出力画像
ＩＭ２ａ第２波長の光の出力画像ＩＭ２の上側視点からの像
ＩＭ２ｂ第２波長の光の出力画像ＩＭ２の下側視点からの像
ＬＩ被写体からの光
ＬＩ１上側双曲面ミラー２０の第１焦点２０ａに向かう光
ＬＩ２外周側双曲面ミラー３１の第１焦点３１ａに向かう光
ＬＩ３上側双曲面ミラー２０で反射され第２波長用の結像光学系４２に向かう光
ＬＩ４第２波長用の結像光学系に直接入射する光
ＬＩ５中間外周側双曲面ミラー７１によって反射され第２波長用の結像光学系４２に向かう光 10 Stereo camera (multi-wavelength all-around camera)
20 Upper hyperboloid mirror 20a First focus 20b of upper hyperboloid mirror 20 Second focus 30 of upper hyperboloid mirror 20 Lower hyperboloid mirror 31 Outer hyperboloid mirror 31a First focus 31b of outer hyperboloid mirror 31 Outer periphery Second focal point 32 of the side hyperboloid mirror 31 Inner circumference side hyperboloid mirror 32a First focus 32b of the inner circumference side hyperboloid mirror 32 Second focus 41 of the inner circumference side hyperboloid mirror 32 Imaging optical system for the first wavelength 42 Imaging optical system for second wavelength 51 Image sensor for first wavelength 52 Image sensor for second wavelength 61 First aperture 62 Second aperture 70 Intermediate hyperboloid mirror 71 Intermediate outer hyperboloid mirror 71a Intermediate First focus 71b of outer hyperboloid mirror 71 Second focus 72 of intermediate outer hyperboloid mirror 71 Intermediate inner hyperboloid mirror 72a First focus 72b of intermediate inner hyperboloid mirror 72 Intermediate inner hyperboloid Second focal point 80 of mirror 72 Drive control section 90 Image processing section 100 Subject CA Central axis IM1 Output image of light of first wavelength IM1a Output image of light of first wavelength Image IM1b from the upper viewpoint of light of first wavelength IM1b Light of first wavelength Output image IM1 from the lower viewpoint Image IM2, IM3, IM4 Output image IM2 of the light of the second wavelength IM2a Output image of the light of the second wavelength IM2 Image IM2b from the upper viewpoint of the light of the second wavelength IM2 Image LI from the lower viewpoint Light from the subject LI1 Light LI2 directed toward the first focal point 20a of the upper hyperboloid mirror 20 Light LI3 directed toward the first focal point 31a of the outer hyperboloid mirror 31 Reflected by the upper hyperboloid mirror 20 Light LI4 directed toward the imaging optical system 42 for the second wavelength Light LI5 directly incident on the imaging optical system 42 for the second wavelength Reflected by the intermediate outer circumferential side hyperboloid mirror 71 and directed to the imaging optical system 42 for the second wavelength the light heading towards

Claims

a convex first hyperboloid mirror;
a convex second hyperboloid mirror that shares a central axis with the first hyperboloid mirror, is provided apart from the first hyperboloid mirror, and has an apex facing the apex of the first hyperboloid mirror; ,
a first imaging optical system provided near the apex of the first hyperboloid mirror and configured to converge light of a first wavelength;
a first image sensor for generating an output image of light of the first wavelength based on the light incident through the first imaging optical system;
a second imaging optical system provided near the apex of the second hyperboloid mirror and configured to converge light of a second wavelength;
a second image sensor for generating an output image of light of the second wavelength based on the light incident through the second imaging optical system;
Equipped with
Of the light emitted from the object, the first light having the first wavelength is reflected by the first hyperboloid mirror, reflected by the second hyperboloid mirror, and transmitted through the first imaging optical system to the first light. 1 incident on the image sensor,
Of the light emitted from the subject, the second light having the first wavelength is reflected by the second hyperboloid mirror and enters the first image sensor via the first imaging optical system,
The first image sensor corresponds to the case where the subject is imaged from two different viewpoints based on the incident first light and the second light, and the first image sensor captures images of the subject from two different viewpoints based on the incident first light and the second light. It is possible to generate a first output image that is
Among the lights emitted from the subject, third light having the second wavelength is reflected by the first hyperboloid mirror and enters the second image sensor via the second imaging optical system,
The second image sensor is a stereo camera that can generate a second output image consisting of an image of light of the second wavelength based on the third light that has entered .

The stereo camera according to claim 1,
The second hyperboloid mirror includes a first inner hyperboloid mirror and a first outer hyperboloid mirror,
The first outer hyperboloid mirror has the same central axis as the first inner hyperboloid mirror, and is provided on the outer periphery of the first inner hyperboloid mirror,
The stereo camera, wherein the absolute value of the conic constant of the first inner hyperboloid mirror is larger than the absolute value of the conic constant of the first outer hyperboloid mirror.

The stereo camera according to claim 1,
The first hyperboloid mirror and the second hyperboloid mirror are each made of a material capable of reflecting the first wavelength light and the second wavelength light, respectively.

The stereo camera according to claim 3 ,
A first opening is provided near the apex of the first hyperboloid mirror,
The first imaging optical system is provided within the first opening,
A second opening is provided near the apex of the second hyperboloid mirror,
The second imaging optical system is a stereo camera provided within the second opening.

The stereo camera according to claim 1,
The first hyperboloid mirror is made of a material capable of reflecting light of the first wavelength and light of the second wavelength,
A stereo camera in which all or part of the second hyperboloid mirror is made of a material that can reflect the light of the first wavelength and transmit the light of the second wavelength.

The stereo camera according to claim 5 ,
A first opening is provided near the apex of the first hyperboloid mirror,
The first imaging optical system is provided within the first opening,
The second imaging optical system is a stereo camera covered by the second hyperboloid mirror.

a convex first hyperboloid mirror;
a convex second hyperboloid mirror that shares a central axis with the first hyperboloid mirror, is provided apart from the first hyperboloid mirror, and has an apex facing the apex of the first hyperboloid mirror; ,
a first imaging optical system provided near the apex of the first hyperboloid mirror and configured to converge light of a first wavelength;
a first image sensor for generating an output image of light of the first wavelength based on the light incident through the first imaging optical system;
a second imaging optical system provided near the apex of the second hyperboloid mirror and configured to converge light of a second wavelength;
a second image sensor for generating an output image of light of the second wavelength based on the light incident through the second imaging optical system;
Equipped with
The second imaging optical system is a wide-angle lens,
Of the light emitted from the object, the first light having the first wavelength is reflected by the first hyperboloid mirror, reflected by the second hyperboloid mirror, and transmitted through the first imaging optical system to the first light. 1 incident on the image sensor,
Of the light emitted from the subject, the second light having the first wavelength is reflected by the second hyperboloid mirror and enters the first image sensor via the first imaging optical system,
The first image sensor corresponds to the case where the subject is imaged from two different viewpoints based on the incident first light and the second light, and the first image sensor captures images of the subject from two different viewpoints based on the incident first light and the second light. It is possible to generate a first output image that is
Among the lights emitted from the subject, third light having the second wavelength is reflected by the first hyperboloid mirror and enters the second image sensor via the second imaging optical system,
Among the lights emitted from the subject, the fourth light having the second wavelength is directly incident on the second image sensor via the second imaging optical system,
The second image sensor corresponds to the case where the subject is imaged from two different viewpoints based on the incident third light and the fourth light, and the second image sensor captures images of the subject from two different viewpoints based on the incident third light and the fourth light. A stereo camera capable of generating a third output image.

a convex first hyperboloid mirror;
a convex second hyperboloid mirror that shares a central axis with the first hyperboloid mirror, is provided apart from the first hyperboloid mirror, and has an apex facing the apex of the first hyperboloid mirror; ,
a first imaging optical system provided near the apex of the first hyperboloid mirror and configured to converge light of a first wavelength;
a first image sensor for generating an output image of light of the first wavelength based on the light incident through the first imaging optical system;
a second imaging optical system provided near the apex of the second hyperboloid mirror and configured to converge light of a second wavelength;
a second image sensor for generating an output image of light of the second wavelength based on the light incident through the second imaging optical system;
The first hyperboloid mirror and the second hyperboloid mirror share the same central axis, are provided between the first hyperboloid mirror and the second hyperboloid mirror, and have their apexes aligned with the second hyperboloid mirror. a convex third hyperboloid mirror facing the apex of the curved mirror ;
Equipped with
The first hyperboloid mirror and the second hyperboloid mirror are each made of a material capable of reflecting light of the first wavelength and light of the second wavelength,
A stereo camera in which all or part of the third hyperboloid mirror is made of a material that can transmit the light of the first wavelength and reflect the light of the second wavelength.

The stereo camera according to claim 8 ,
Of the light emitted from the object, the first light having the first wavelength is reflected by the first hyperboloid mirror, reflected by the second hyperboloid mirror, and transmitted through the first imaging optical system to the first light. 1 incident on the image sensor,
Of the light emitted from the subject, the second light having the first wavelength is reflected by the second hyperboloid mirror and enters the first image sensor via the first imaging optical system,
The first image sensor corresponds to the case where the subject is imaged from two different viewpoints based on the incident first light and the second light, and the first image sensor captures images of the subject from two different viewpoints based on the incident first light and the second light. It is possible to generate a first output image that is
Of the light emitted from the object, the third light having the second wavelength is reflected by the second hyperboloid mirror, reflected by the third hyperboloid mirror, and transmitted through the second imaging optical system to the third light having the second wavelength. incident on the second image sensor,
Among the lights emitted from the subject, a fifth light having the second wavelength is reflected by the third hyperboloid mirror and enters the second image sensor via the second imaging optical system,
The second image sensor corresponds to the case where the subject is imaged from two different viewpoints based on the incident third light and the fifth light, and the second image sensor captures images of the subject from two different viewpoints based on the incident third light and the fifth light. A stereo camera capable of generating a fourth output image.

The stereo camera according to claim 8 ,
The third hyperboloid mirror includes a second inner hyperboloid mirror and a second outer hyperboloid mirror,
The second outer hyperboloid mirror has the same central axis as the second inner hyperboloid mirror, and is provided on the outer periphery of the second inner hyperboloid mirror,
In the stereo camera, the absolute value of the conic constant of the second inner hyperboloid mirror is larger than the absolute value of the conic constant of the second outer hyperboloid mirror.