JP2010281733A - Three-dimensional optical image-forming system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a three-dimensional camera which can perform three-dimensional capturing with high resolution in a simple hardware configuration. <P>SOLUTION: A three-dimensional imager 30 includes: light source equipment 42 for emitting a series of IR light pulses; a camera 60 including an array of photosensitive cells where each of them is sensitive to IR light; an imaging lens for sending light from a subject to the photosensitive cells so that each ray of light passes a common focus; two memories 64, 66 for alternately storing images captured by the camera 60 in synchronization with the emission of light pulses; and a signal processing unit for computing a depth map of the subject by utilizing two continuous images stored in the two memories 64, 66, and positions of the light source equipment 42, the camera 60, and an object point 102. Light pulses emitted from the light source equipment 42 have the first intensity and the second intensity alternately. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a 3D imaging system for capturing a 2D color image and a 3D surface profile of a subject. In particular, the present invention relates to a 3D optical system suitable for capturing high-definition 3D images in real time.

  Over the years, various types of 3D display technologies such as the well known red / green stereoscopic glasses display, volume display, autostereoscopic display, etc. have been developed.

  Transmission and (audio-video) multimedia technologies are advancing rapidly, trying to achieve a real sense in media broadcast content. In order to achieve a sense of reality on the viewer side, it is necessary to deliver surreal audio and video content to the viewer. Here we focus on the acquisition of three-dimensional (3D) visual content.

  In contrast to conventional 2D television, most 3D televisions not only display the 2D visual appearance of the subject, but also in various forms such as stereo images, multi-view images or 2D plus parallax images. Depth information is also displayed. If the subject color information and the 3D surface profile are available, all of these forms of visual information can be regenerated. Therefore, the target here is the following two points. That is, (1) capturing a 2D appearance (color information) and (2) calculating depth information for each pixel synchronously in real time.

  So far, various techniques for 3D image capture have been proposed. In one extreme, the depth map is estimated using an image-based approach using a conventional calibrated stereoscopic camera. In another extreme example, depth is captured by an optical-based camera, such as one using the infrared time-of-flight (TOF) principle.

  In the invention disclosed in Non-Patent Document 1, the TOF 3D camera emits a sinusoidally modulated infrared light signal. The time required for light to reach the object from the measurement system and back to the system is measured and used in calculating the depth of each pixel in the image.

B. Batgen, T. Ogier and M.C. Lehmann, “Ccd / cmos Lock-in Pixel for Range Images: Challenges, Limits and State-of-the-Art Technology”, Daily Proceedings of the First Range Image Study, 21-32, Ingensand / Carlmann (ed.), Zurich 2005.

  (B. Buttgen, T. Oggier, and M. Lehmann, "Ccd / cmos lock-in pixel for range imaging: challenges, limitations and state-of-the-art," in Proceedings of 1st range imaging research day, pp. 21-32, Ingensand / Kahlmann (eds.), Zurich, 2005.)

  The best distance accuracy achievable with a TOF 3D camera is on the order of a few millimeters to a few centimeters, depending on the distance from the object to the camera. Furthermore, this technique has a distance limit imposed by the modulation frequency of the infrared light, and measurements are inaccurate outside this distance limit range. Furthermore, although the TOF range camera can capture synchronized distance data and a color image, it is difficult to capture high resolution with this method. Hardware complexity and associated costs are also high. Therefore, the approach of capturing with a TOF camera seems not suitable as a consumer 3D imaging camera for demanding applications such as personal use instead of 2D-only cameras or 3D television broadcasting. .

  Accordingly, one of the objects of the present invention is to provide a 3D camera capable of high-resolution 3D capture synchronized with color image capture.

  Another object of the present invention is to provide a 3D camera capable of high-resolution 3D capture with a simple hardware configuration.

  An optical imaging system according to one aspect of the present invention includes a light source device that emits two separate pulses of light of invisible wavelength, the pulses of light alternating with a first intensity (with the same behavior). And second light intensity limited to a longer path relative to the first light intensity, and each sensitive to invisible wavelengths, each receiving light from the subject A camera having a photosensor surface having a first array of photosensitive cells that outputs an electrical signal representative of the intensity of the received light, thereby capturing an image of the subject, said camera comprising: An imaging lens system that operates in synchronization with the emission of a pulse, and further directs light from the subject to a photosensor surface so that each ray of the light passes through a predetermined common point; and Ki by camera Two memories for alternately storing captured images in synchronization with the emission of the light pulses by the light source device, images stored in the two memories, a predetermined 3D coordinate system And a signal processing device for calculating a depth map of the subject using the position of the light source device with respect to and the position of the camera with respect to the 3D coordinate system.

  According to one embodiment, the light source device has a light source for emitting a pulse of light with an invisible wavelength, and the light source comes to a predetermined first position every other period. A mechanism for moving the light source so that the light source is at a predetermined second position in another cycle, and another predetermined position so as to spread light from the light source uniformly. Placed optical device.

  According to another embodiment, the light source device is placed in another predetermined position to uniformly spread light coming from the light source and a light source positioned in a predetermined place. An optical device, and a means for defining two optical paths between the light source and the optical device, the two optical paths having different lengths. And means for switching the two optical paths in synchronization with the emission of light pulses by the light source device.

  In this configuration, there is a second photosensitive cell, each of which is sensitive to a predetermined visible wavelength, and outputs an electrical signal indicating the intensity of the visible wavelength light received from the subject. It is also possible to output a visible image of the subject.

  The second photosensitive cells may be divided into a plurality of groups each including a plurality of cells sensitive to different visible wavelengths.

  Preferably, the light source device emits the light pulse so that the light pulse spreads uniformly from a predetermined position fixed with respect to the 3D coordinate system. Each of the photosensitive cells is placed at a known position with respect to the 3D coordinate system, and the predetermined common point is placed at a known position within the 3D coordinate system. The signal processing device includes an intensity output from the first photosensitive cell in two successive cycles, a position of the light source device with respect to the 3D coordinate system, a position of the common point with respect to the 3D coordinate system, and the 3D coordinate system. The depth of the target point corresponding to the first photosensitive cell with respect to the 3D coordinate system is calculated using the position of the photosensitive cell with respect to.

  More preferably, the 3D coordinate system is fixed to a plane defined by the array of photosensitive cells.

  More preferably, the light source device and the camera are fixed to each other.

1 is a diagram illustrating an overall configuration of a 3D image forming system 30 according to a first embodiment of this invention. It is a figure which shows the motion of the light source 92 in 1st Embodiment. 3 is a diagram schematically showing an optical path length of the 3D imaging system 30. FIG. FIG. 6 is a diagram schematically showing a geometric relationship between the optical device 94, the center of the sensor surface 110, and the object point 102; FIG. 6 is a diagram schematically illustrating a geometric relationship between a focal point 112, a pixel 114, and a camera origin 116 on a sensor surface 110 of an optical device 94 and a four-channel camera. It is a figure which shows roughly the relationship between the focus 112, the camera origin 116, the pixel 114, and the object point 102. It is a timing chart of the illumination of IR light source 92, the color image image | photographed in each period, and the distance image (depth map) each calculated from two continuous IR images. It is a figure which shows the whole structure of the optical path in 2nd Embodiment of this invention. It is a figure which shows the whole structure of the optical path in 3rd Embodiment of this invention. It is a figure which shows the whole structure of the 3D image creation system 330 in the 4th Embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. In the figures, similar elements bear the same reference numerals. Therefore, detailed description thereof will not be repeated.

First Embodiment Introduction A method and apparatus for accomplishing this function consists of two isotropic light sources that are placed in slightly different geometric positions and that are invisible, eg operating in the infrared wavelength range, and light rays projected onto the subject. And an optical device for aligning the image and a high-precision camera having sensitivity in the invisible wavelength region. To determine distance information for each pixel, the camera system captures two sequential images during the strobe operation from each successive light source. Then, using the light intensity at each pixel position in these two images, the distance information of the target point corresponding to the pixel position in the image is calculated, and the 3D surface profile of the subject is generated in real time.

  Depending on the design choice, the color image is either a dichroic prism mechanism or a four channel camera with four bandpass filter systems, three of which are visible red, green and blue channels It is possible to capture synchronously by a camera that is for use and the fourth is an IR channel. In the first embodiment, a single four-channel camera having four band pass filters is used.

2. Principle of High-Definition 3D Sensing Camera System This section describes the principle of a 3D sensing camera system that can capture both high-definition color images and distance data in real time in synchronization. The basic principle is as follows.

  Two energy sources with known propagation characteristics, placed in slightly different geometries, alternately emit a series of energy pulses (in the form of pulses of light in the invisible wavelength range). The energy pulse is reflected from the surface of the object. The receiver takes out the amplitude of the reflected energy pulse. Thus, the amplitude of the energy pulses received from these two energy sources defines the distance between the reference point and the target point along the energy path. For example, the propagation characteristics, gravity, electric field, sound and radiation of a point light source can be explained by the inverse square law. Therefore, they can be used for 3D sensing with appropriate design.

  In this embodiment, distance data and color image data are captured using two light sources as an ejector and a high-precision camera as a receiver. This system utilizes two physical properties of light. That is, (1) propagation characteristics described by the inverse-square law, and (2) linearity of the visual effect of light of different wavelengths, also known as Abney's Law. Specifically, the system that achieves the above-described function is composed of two isotropic light sources that are arranged at slightly different positions along the optical axis and operate in the invisible wavelength range, a light guiding optical device, and a high accuracy. Imaging camera (preferably 16 bits or more).

3. System Overview Referring to FIG. 1, a 3D imaging system 30 according to a first embodiment of the present invention acquires two consecutive image frames to calculate depth information for each pixel of a subject 44. There is a need to. Depending on the design choice, the color image is either a 4 channel camera with a dichroic prism mechanism that requires two cameras to separate IR and visible light, or with four optical bandpass filter systems. Thus, three of the channels are optimized for red, green and blue (visible light), and the fourth channel is optimized for the wavelength range of the IR light source. In this embodiment, a single 4-channel camera is used.

  Referring to FIG. 1, an imaging system 30 includes a light source device 42 for emitting IR light pulses at a predetermined interval, and a light source device 42 that emits light pulses in synchronism with the light pulse emission of each frame. And a four-channel camera 40 for capturing RGB color images and IR images. In this way, a series of invisible wavelength light pulses are emitted from the light source device 42. The light pulses alternately have a first luminous intensity and a second luminous intensity at a timing different from the first luminous intensity.

The light source device 42 includes a cylinder 90 having an inner wall for absorbing light and an opening at one end, and two isotropic IR point light sources S _A and S _B housed in the cylinder 90, which are respectively Intensities S _A = S and S _B = kS affect that equally in all directions. However, k = S _{B_ray (j)} / S _{A_ray (j)} is an arbitrary scaler and is constant for all rays (j = 1, 2,... N) projected onto the subject. The value of k may be known in advance or may be measured dynamically on-the-fly. There is also an optical device 94, which is typically a lens system, located at the open end of the tube 90, and similarly spreading the light from the light sources S _A and S _B , so that j = 1,2, ... k = S _{B_ray (j)} / S _{A_ray (j)} is maintained for all pixels corresponding to N.

Referring to FIG. 2, a single light source 92 is actually provided in the cylinder 90. The light source 92 reciprocates between positions indicated as light sources S _A and S _{B in} FIG. 1 as shown in FIGS. 2 (A) and 2 (B). When in the position where uniform light source 92 is indicated by the light source S _A, which operates as a light source S _A. This is when in the position indicated by the light source S _B, which operates as a light source S _B. Therefore, in this embodiment, the constant k of S _B = kS described above is equal to 1.

The distances from the center of the optical device 94 to the light sources S _A and S _B are v and w, respectively, as shown in FIG.

  Referring again to FIG. 1, the four-channel camera 40 is positioned on a predefined image plane with a lens system (not shown) for focusing the image of the subject 44 on the predefined image plane. A 4-channel CCD (Charge Coupled Device) 60 for capturing RGB images and IR images of the subject 44 and outputting RGB signals and IR signals of each frame; A frame memory 70 connected to receive RGB signals and storing RGB images at each frame time slot; and frame memories 64 and 66 alternately storing IR images captured by the CCD 60 (computer program or hardware) Electronic switch 62, which can be implemented with an input Are connected to receive the IR signal from the CCD 60, and the two outputs are connected to the inputs of the frame memories 64 and 66, respectively, to the frame memory 64 or 66 at each frame time slot in synchronization with the emission of the light pulse. A switch for alternately providing IR signals from the CCD 60 is connected to receive IR images stored in the frame memories 64 and 66. The frame memories 64 and 66 are synchronized with the emission of light pulses in each frame time slot. And a signal processing device 68 for calculating the depth profile of the subject 44 based on the IR image stored in.

The light sources S _A and S _B are controlled to emit IR light alternately in each frame. That is, in the first frame, the light source S _A emits IR light, and the light source S _B does not emit. In the next frame, the light source S _B emits IR light and the light source S _A does not emit.

E _A and E _B represent the intensity on the surface of any visible object point _x ₀ (where _ is attached below the letter) during the respective illumination periods of S _A and S _B To do. Similarly, the I _A and I _B, respectively denote the captured intensity values by imager by S _A and S _B. Here, it can be said that I _B for I _A is the same as E _B for E _A. Therefore, this relationship can be described as follows.
I _A : I _B = E _A : E _B (1)
This is because, except for the surface having a light constancy of the light-absorbing surface or the like, _I A _.alpha.E A independent and reflectance of the target surface _{(I A} is proportional to _{E A)} and a so _I B .alpha.E _B. Using light propagation characteristics, E _A and E _B can be defined mathematically as follows:

ここでγは光学装置によるスケーリングファクタである。式１に、式２及び式３によるＥ_Ａ及びＥ_Ｂの定義を挿入すれば、以下が得られる。

Here, γ is a scaling factor by the optical device. Inserting the definitions of E _A and E _B according to Equation 2 and Equation 3 into Equation 1 yields:

ここで図３を参照して、ｒは光学装置９４の焦点と被写体４４の対象点１０２との間の距離を示し、ｖ及びｗは、図３に例示するように、光学装置９４とＳ_Ａ及び９４とＳ_Ｂとの距離をそれぞれ表わす。式４の両辺の平方根をとり、変形すると、単一の変数ｒについて以下の１次多項式が得られる。

Referring now to FIG. 3, r represents the distance between the target point 102 of the focus and the object 44 of the optical device 94, v and w, as illustrated in FIG. 3, the optical device 94 and the S _A and respectively represent the distances between the 94 and _{S B.} Taking the square root of both sides of Equation 4 and modifying it, the following first order polynomial is obtained for a single variable r.

（図７（Ｃ）の画像Ａ及び画像Ｂの画素位置に属する強度比の平方根）。従って、ｒについて解くと以下のようになる。

(The square root of the intensity ratio belonging to the pixel positions of the image A and the image B in FIG. 7C). Therefore, the solution for r is as follows.

ここでｒは光学装置９４の焦点と、被写体４４の対象点１０２との間の距離である。

Here, r is the distance between the focal point of the optical device 94 and the target point 102 of the subject 44.

4). 3D Measurement Using Camera Origin in Cartesian Coordinates This section describes how to represent distance data in Cartesian coordinates using the camera origin as a reference.

  That is, if the geometric relationship between the focal point of the optical device 94 and the optical center of the CCD 60 (see FIG. 3) of the four-channel camera 40 is known, the camera specifications and Cartesian coordinates (x, y, z) Using the estimated distance r between the optical device 94 and the target point 102 (or converting it to spherical coordinates), the camera coordinates fixed on the CCD 60 are used as a reference. The 3D position of the target point 102 can be calculated. Note that the optical device 94 emits light onto the subject and is captured on the sensor surface of the CCD 60. Each ray from the object point passes through a common focal point of the lens 100 when it strikes the photosensitive cell.

  Using the notations in FIGS. 3 and 4, the cosine theorem is as follows.

ここでｒ、ｕ及びρは図３及び図４に例示された距離であり、ｒとｕとは既知である。しかしながら、ρとθとは未知である。図４において、対象点１０２の画像は３Ｄ座標系のセンサ面１１０上の位置（ｘ_ｉ，ｙ_ｉ）のセル１１４上に形成され、これはセンサ面１１０の中心（ｃ_ｘ，ｃ_ｙ）でカメラ座標系の原点１１６からｘ方向及びｙ方向にそれぞれｄ_ｘ及びｄ_ｙだけずれている。式７を用いて、ここでの目標は、値ρを決定することである。従って、まずはじめに角度θを決定する必要がある。ｆが４チャンネルカメラ４０の撮像レンズの焦点１１２とセンサ面１１０に固定された３Ｄ座標系の座標（０，０，０）で示される原点１１６との距離を示すものとして、図５の表記を用いれば、２つのベクトル→ｕ及び→ρ（式中→は各文字の上に付される）の内積は以下のようになる。

Here, r, u, and ρ are distances illustrated in FIGS. 3 and 4, and r and u are known. However, ρ and θ are unknown. In FIG. 4, the image of the target point 102 is formed on the cell 114 at the position (x _i , y _i ) on the sensor surface 110 in the 3D coordinate system, which is the center (c _x , c _y ) of the sensor surface 110. is shifted by _{d x} and _{d y} are the x and y directions from the origin 116 of the camera coordinate system. Using Equation 7, the goal here is to determine the value ρ. Therefore, it is necessary to first determine the angle θ. 5 represents the distance between the focal point 112 of the imaging lens of the four-channel camera 40 and the origin 116 indicated by coordinates (0, 0, 0) of the 3D coordinate system fixed to the sensor surface 110. If used, the inner product of the two vectors → u and → ρ (where → is attached to each character) is as follows.

ここで内積は２つのベクトル間のドットによって表わされ、｜→ｘ｜は→ｘのノルムを表す。画素位置（ｘ_ｉ，ｙ_ｉ）のずれ情報ｄ_ｘ及びｄ_ｙと、光学装置９４とカメラ原点１１６との間の長さとを直接用いて、→ｕの値を決定できる。しかしながら、カメラ投影マトリックスを用いて決定できるのは→ρに平行な単位ベクトルのみである。コンピュータビジョンでは、投影マトリックスＰは３Ｄ対象点からカメラの２Ｄ画像点へのマッピングを記述する３×４のマトリックスである。マトリックスＰは、ピンホールカメラモデルによって定義するか、又は、カメラ較正処理を用いて決定できる。

Here, the inner product is represented by a dot between two vectors, and | → x | represents the norm of → x. Pixel position _(x i, _{y i)} using the shift information _{d x} and _{d y} in, and a length between the optical device 94 and the camera origin 116 can directly determine the value of the → u. However, only the unit vector parallel to → ρ can be determined using the camera projection matrix. In computer vision, the projection matrix P is a 3 × 4 matrix that describes the mapping from 3D object points to 2D image points of the camera. The matrix P can be defined by a pinhole camera model or determined using a camera calibration process.

を、それぞれ同次座標系における以下の画像点及び対象点の対を表すものとする。

その場合、３Ｄ対象点＿ｘ_０から２Ｄ画像点＿ｘ_ｉ（式中下線は各文字の下に付される）へのマッピングは以下で与えられる。 here

Denote the following pairs of image points and object points in the homogeneous coordinate system, respectively.

In that case, the mapping from 3D object point_x ₀ to 2D image point_x _i (where the underline is attached below each character) is given below.

ここでｗ_ｉは画像メモリの（ｘ_ｉ，ｙ_ｉ）画素位置に対し一意である任意の非ゼロのスケーラである。従って、スケールファクタにより＿ｘ_０を次のように決定できる。

Where w _i is an arbitrary non-zero scaler that is unique to the (x _i , y _i ) pixel location in the image memory. Therefore, we determine the _x ₀ as follows by the scale factor.

さて、ｗ_ｉが未知であるが、ｗ_ｉ＿ｘ_０が得られたので、式１０と図５の表記を用いて、ベクトル→ρに沿った新たなベクトル→ｐを定義できる。

Now, w _i is unknown, but w _{i — x 0} is obtained, so a new vector → p along the vector → ρ can be defined using Equation 10 and the notation of FIG.

ここでδ＞１．０であり、δは任意に選択可能な定数スケーラである。→ｖが、→ｐをそのノルムで正規化して得られる以下の単位ベクトルであるものとする。

Here, δ> 1.0, and δ is an arbitrarily selectable constant scaler. Let v be the following unit vector obtained by normalizing p with its norm.

→ｖ／／→ρであるから、式８において→ρに代えて→ｖを用いて角度θを以下のように決定できる。

Since → v // → ρ, the angle θ can be determined as follows by using → v instead of → ρ in Equation 8.

ここでａｃｏｓは逆コサイン演算を表す。式７を書換えれば、単一の変数ρについて以下の２次の二次方程式が得られる。

Here, acos represents an inverse cosine operation. If Equation 7 is rewritten, the following quadratic quadratic equation is obtained for a single variable ρ.

ρについて解くと、２つの解ρ＝ρ_１とρ＝ρ_２とが得られる。正の実数値を有するものが→ρのノルムである。→ρを以下によって一意に決定できる。

Solving for ρ gives two solutions ρ = ρ ₁ and ρ = ρ ₂ . Those having positive real values are norms of → ρ. → ρ can be uniquely determined by:

→ρは作像装置上で（ｄ_ｘ，ｄ_ｙ，０）で示される座標位置から始まり、カメラの焦点を通過する対象点（ｘ_０，ｙ_０，ｚ_０）で終わる（図５を参照）。

→ ρ starts from the coordinate position indicated by (d _x , d _y , 0) on the image forming device and ends at the target point (x ₀ , y ₀ , z ₀ ) passing through the camera focus (see FIG. 5). ).

Therefore, in order to determine the target point _x ₀ in the 3D space with respect to the camera origin 116, as shown in FIG. 6, from the origin 116 of (0, 0, 0), (d _x , _dy , 0) The vector → s defined as up to the position of → must be added to → ρ as shown below.

５．動作
図７を参照して、作像システム３０は以下のように動作する。

5). Operation Referring to FIG. 7, the image forming system 30 operates as follows.

Source _{S A,} as shown in FIG. 7 (A), the frame 130, 132, ... emit lights at. On the other hand, as shown in FIG. 7B, the light source S _B has frames 140, 142, and 144 located between the frames 130 and 132, between 132 and 134, between 134 and 136, respectively. ... and emit light. In this way, a series of light pulses having invisible wavelengths are emitted from the light source device 42. The light pulse has a first light intensity and a second light intensity alternately.

In each frame time slot, the CCD 60 generates an RGB image (color images 200, 202, 204, 206, 208, 210,...) And an IR image (IR images 150, 152, 154, 156, 158, 160,...). Images are taken and signals of these images are output. Since the light sources S _A and S _B alternately emit IR light, the IR image output from the CCD 60 is an alternating image of IR light emitted from the light sources S _A and S _B and reflected from the same target point 102 (image 150). , 154, 158,... And images 152, 156, 160,.

  By applying the calculations shown in Equations 7 through 16 to each of the pixels in the IR image, the IR image pairs 150 and 152, 152 and 154, 154 and 156, 156 and 158, 158 and 160,. Distance images 180, 182, 184, 186, 188, 190,... Can be obtained from each.

Second Embodiment In the first embodiment, the light source device 42 includes a single light source 92 that reciprocates the positions of the light sources S _A and S _B. However, the present invention is not limited to such an embodiment.

  Referring to FIG. 8, a light source device 230 according to a second embodiment of the present invention includes two projecting portions, a rectangular tube 240 having a light-absorbing inner wall, and one end portion of the projecting portion. IR light source 92 provided in There is an opening at the end of the other protrusion of the tube 240. The light source device 230 is further provided at four corners of the tube 240, and four mirrors 246, 248, 250, and 252 facing the center of the tube 240, and can be reversed at right angles to reflect the light from the light source 92 to reflect the mirror 248 Alternatively, the mirror 244 that can be directed in the direction of the mirror 250 and the mirror 240 can be reversed at right angles, and either the light from the mirror 250 or the light from the mirror 246 is reflected in synchronization with the inversion of the mirror 244 to A mirror 242 directed toward the opening of the tube 240 and an optical device 94 provided in the opening of the tube 240. Mirrors 244, 252, 250 and 242 define a longer optical path for light from light source 92. Mirrors 244, 248, 246 and 242 define a shorter optical path.

  As is apparent from FIG. 8, when the mirrors 244 and 242 are “down” as in FIG. 8A, the light from the light source 92 travels a shorter path. When mirrors 244 and 242 are “up” as in FIG. 8B, the light travels a longer path. Therefore, the position of the light source 92 does not change, but the intensity of light emitted from the optical device 94 changes. Therefore, the light source device 230 can be used instead of the light source device 42 of the first embodiment.

Third Embodiment In the third embodiment of the present invention, a light source device 280 shown in FIG. 9 is used. Referring to FIG. 9, light source device 280 has tube 290 having an opening 322 at one end and opening 302 formed on the side wall, and is placed in the opening of tube 290 and enters through tube 290. It includes an optical device 314 that similarly disperses light from two different light sources, and an IR light source 296 placed at the other end of the tube 290.

  The light source device 280 further has an opening on the side wall, the tube 292 attached to the tube 290 so that the opening is aligned with the opening 302 of the tube 290, and the tube 290 at the end distal from the opening 302. The IR light source 298 placed inside, the reflection prism 300 attached to the other end of the tube 292, and the opening 302 so that the IR light from the light source 298 is reflected by the reflection prism 300 toward the opening 302. A reflective shutter 294 operably attached to the tube 290 with the opening 302 so that the posture can be changed between the first posture and the second posture. In the first state, the shutter 294 includes the opening 302. In the second state, the shutter 294 opens the opening 302 in the closed state. In the second state, one end of the shutter 294 comes into contact with the inner wall of the cylinder 290 and reflects light from the reflecting prism 300 to the opening 322. In order to move the state of the shutter 294, the light source device 280 includes a shutter mechanism (not shown).

  In the first state shown in FIG. 9A, the light 310 from the IR light source 296 travels directly to the optical device 314 and is dispersed outside the cylinder 290. The light source 298 is turned off.

  In the second state shown in FIG. 9B, the IR light source 296 is turned off and the light source 298 is turned on. Light from the light source 298 travels to the prism 300 where it is reflected and travels toward the shutter 294. The light is reflected again by the shutter 294 and travels toward the optical device 314 and is dispersed outside the tube 290 as in the above case.

  Obviously, the light source device 280 of this embodiment can be used in place of the light source device 42 of the first embodiment shown in FIGS. Therefore, this third embodiment achieves the same result as the first embodiment.

Fourth Embodiment In the first to third embodiments, a four-channel camera having the ability to capture RGB color images and IR images is used. However, the present invention is not limited to such an embodiment. As described elsewhere, the embodiment of the present invention can also be realized by separating IR and visible light using a dichroic prism mechanism.

  Referring to FIG. 10, a 3D imaging system 330 according to the fourth embodiment includes a dichroic prism 340 for dividing incoming light into visible light and IR light, and an optical path of IR light from the dichroic prism 340. A normal color camera 344 that receives visible light from the dichroic prism 340 and captures an RGB image, and a high-accuracy IR camera 346 that receives IR light reflected from the mirror 342 and captures an IR image. ,including.

  The IR camera 346 is similar to the 4-channel camera 40, and includes a CCD 60, two frame memories 64 and 66 for storing IR images captured by the CCD 60, and IR images stored in the frame memories 64 and 66. And a signal processor 68 for calculating the depth profile of the subject 44 based on the IR images stored in the frame memories 64 and 66 in synchronism with the emission of light pulses.

  In this embodiment, light from the subject object is separated by the dichroic prism 340. The light in the visible wavelength range travels through the dichroic prism 340 and is directly received by the color camera 344. The color camera 344 captures a color image of the object and outputs an RGB color signal for each frame.

  The IR light is reflected by the dichroic prism 340 and travels to a mirror 342 that directs the IR light to two frame memories 64 and 66 for storing the IR image captured by the CCD 60. The IR image capture by the CCD 60, the alternating storage of the IR images by the frame memories 64 and 66, and the calculation of the 3D distance by the signal processing device 68 based on the images stored in the frame memories 64 and 66 are the first embodiment. This is performed in exactly the same manner as the four-channel camera 40.

  As is clear from the above description, the 3D image forming system 330 of the fourth embodiment operates in the same manner as the first embodiment. Accordingly, a 3D color image and a highly accurate 3D depth image are generated by the 3D imaging system 330.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are included. Including.

30, 330 3D imaging system 40 4-channel camera 42, 230, 280 Light source device 60 CCD
62 Switch 64, 66, 70 Frame memory 68 Signal processor 90, 240, 290, 292 Tube 92, 296, 298 Light source 94, 314 Optical device 110 Sensor surface 242, 244, 246, 248, 250, 252, 342 Mirror 340 Dichroic prism 344 Color camera 346 IR camera

Claims (8)

An optical imaging system,
A light source device that periodically emits a series of invisible wavelength light pulses, the light pulses alternately having a first light intensity and a second light intensity different from the first light intensity, each of which is invisible A photosensor surface having a first array of photosensitive cells that is sensitive in wavelength and receives light from the subject and outputs an electrical signal representing the intensity of the light received by each, thereby capturing an image of the subject The camera operates in synchronization with the emission of light pulses by the light source device, and further directs light from the subject toward the photosensor surface so that each light ray of the light is defined in advance. An imaging lens system for sending through, and
Two memories for alternately storing images captured by the camera in synchronism with the emission of the light pulses by the light source device;
For calculating a depth map of a subject using the images stored in the two memories, the position of the light source device with respect to a predetermined 3D coordinate system, and the position of the camera with respect to the 3D coordinate system An optical imaging system comprising: a signal processing device; The light source device is a light source for emitting a pulse of light of the invisible wavelength;
For moving the light source so that the light source comes to a predetermined first position in every other period, and so that the light source comes to a predetermined second position in other periods. Mechanism,
The optical imaging system according to claim 1, further comprising: an optical device placed at another predetermined position so as to spread light from the light source uniformly. The light source device is
A light source positioned at a predetermined location;
An optical device placed at another predetermined position to uniformly spread light coming from the light source;
And a means for defining two light paths between the light source and the optical device, wherein the two light paths have different lengths, and The optical imaging system according to claim 1, comprising means for switching the two optical paths in synchronism with the emission of light pulses by the light source device. Each of the arrays has a second photosensitive cell that is sensitive to a predetermined visible wavelength and outputs an electric signal representing the intensity of the visible wavelength light received from the subject. The optical imaging system according to claim 1, wherein a visible image of the subject is output thereby. The optical imaging system according to claim 4, wherein the second photosensitive cells are divided into a plurality of groups each including a plurality of cells sensitive to different visible wavelengths. The light source device emits the light pulse so that the light pulse spreads uniformly from a predetermined position of the 3D coordinate system,
Each of the photosensitive cells is placed at a known position relative to the 3D coordinate system;
The predetermined common point is placed at a known position in the 3D coordinate system;
The signal processing device includes an intensity output from the first photosensitive cell in two successive cycles, a position of the light source device with respect to the 3D coordinate system, a position of the common point with respect to the 3D coordinate system, and the 3D coordinate system. The optical imaging system according to claim 1, wherein the position of the photosensitive cell with respect to is used to calculate the depth of the first photosensitive cell with respect to the 3D coordinate system. The optical imaging system according to claim 6, wherein the 3D coordinate system is fixed to a photosensor surface defined by the array of photosensitive cells. The optical imaging system according to claim 7, wherein the light source device and the camera are fixed to each other.

Images