JP3517628B2 - Range finder device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a range finder device for measuring three-dimensional shape or distance information of an object.

[0002]

2. Description of the Related Art FIG. 24 is a diagram showing an example of the configuration of a conventional range finder apparatus. The range finder device shown in FIG. 24 projects light on a subject, and from the observed image,
The three-dimensional shape measurement is performed based on the principle of triangulation, and it is proposed as a device that can operate in real time.

In FIG. 24, 101A and 101B are laser light sources having slightly different wavelengths, and 102 is a laser light source 1.
Half mirrors for synthesizing laser light from 01A and 101B, 103 for a light source control unit for controlling the light intensity of the laser light sources 101A, 101B, 104 for a rotary mirror for scanning the laser light, and 105 for a rotary control unit for controlling the rotary mirror. 1
06 is a subject, 107 is a lens for forming an image on the CCD, 108A and 108B are laser light sources 101A and 101
An optical wavelength separation filter for separating light of wavelength B, 109
A and 109B are CCDs for capturing monochrome images, and 109
C is a CCD for capturing a color image, 110A, 110B
Is a signal processing unit of a monochrome camera, 111 is a signal processing unit of a color camera, 112 is a distance calculation unit that calculates the distance or shape of the subject from the intensity of the laser light captured by the CCDs 109A and 109B, and 113 is the synchronization of the entire device. It is a control unit that does.

The operation of the range finder device shown in FIG. 24 will be described.

The laser light sources 101A and 101B emit laser lights having slightly different wavelengths. This laser light is slit light having an optical cross section that is perpendicular to the scanning direction of the rotating mirror described later, and when the rotating mirror scans in the horizontal direction,
It becomes the slit light in the vertical direction.

FIG. 25 is a diagram showing the wavelength characteristics of the two light sources 101A and 101B. Here, the reason why the two light sources 101A and 101B having wavelengths close to each other are used is to make it less susceptible to the wavelength dependency of the reflectance of the subject 106.
The laser light emitted from the laser light sources 101A and 101B is combined by the half mirror 102, and the rotating mirror 1
The subject 106 is scanned by 04.

The rotation control unit 105 drives the rotating mirror 104 at a field cycle, whereby the light sources 101A,
The laser beam projected from 101B is scanned. At that time, the light source control unit 103 determines that both the light sources 101A and 101
The light intensity of B is changed within one field cycle as shown in FIG. That is, the change of the light intensity of the laser light and the driving of the rotating mirror are performed in synchronization.

The lens 107 is a CCD 109A, 109
An image of the subject is formed on B and 109C. The light wavelength separation filter 108A transmits only the light of the wavelength of the light source 101A and reflects the light of other wavelengths. The light wavelength separation filter 108B transmits only the light of the wavelength of the light source 101B and reflects the light of other wavelengths. As a result, the reflected light from the subject 106 of the projected light from the light sources 101A and 101B is respectively reflected by the CCD.
Photographed by 109A and 109B, and light of other wavelengths is photographed by CCD 109C as a color image.

The light source A signal processing section 110A and the light source B signal processing section 110B are respectively connected to CCDs 109A and 10C.
The signal processing similar to that of a normal monochrome camera is performed on the output of 9B. The color camera signal processing unit 111 is a CCD
The signal processing of a normal color camera is performed on the output of 109C.

The distance calculation unit 112 includes CCDs 109A and 10A.
Distance calculation is performed for each pixel from the output of 9B, the baseline length, and the coordinate value of the pixel.

The distance calculation unit 112 is composed of the CCDs 109A, 10
From the output of 9B, the light intensity ratio of the reflected light of the laser light is calculated. The time in one scanning cycle can be measured from this light intensity ratio, and the rotation angle φ of the rotary mirror 104 can be obtained from this time. For example, as shown in FIG. 26B, when the light intensity ratio is Iao / Ibo, the scanning time is measured as t0, and the rotation angle φ of the rotating mirror 104 can be known from the measured value. This angle φ corresponds to the angle of the subject viewed from the light source side.

Scanning time t and rotation angle φ of the rotating mirror 104
Since the correspondence relationship between and is known in advance, another characteristic table showing the correspondence relationship between the light intensity ratio Ia / Ib and the rotation angle φ in which the horizontal axis of FIG. You may keep it. In this case, the rotation angle φ can be directly specified from the calculated light intensity ratio without passing through the scanning time t.

FIG. 27 is a diagram for graphically explaining the distance calculation in the distance calculation unit 112. In FIG. 27, O is the center of the lens 107, P is the point on the subject, and Q is the position of the rotation axis of the rotating mirror. Further, the position of the CCD 109 is shown folded back to the subject side for the sake of simplicity. Note that f is the CCD 109 and the lens 107.
Is the distance from the center O. The length of OQ, that is, the base line length is L, the angle of the point P viewed from the rotation axis position Q in the XZ plane is φ, the angle of the point P viewed from the lens center O in the XZ plane is θ, and the lens in the YZ plane is Ω is the angle of the point P viewed from the center O
Then, the three-dimensional coordinates (X,
Y, Z) is calculated by the following formula. X = L · tan φ / (tan θ + tan φ) Y = L · tan θ · tan φ · tan ω / (tan θ +
tanφ) Z = L · tanθ · tanφ / (tanθ + tanφ) Regarding the angle φ in the above equation, as described above, the CCD 10
9A and 109B are calculated by the light intensity ratio of the reflected light. For angles θ and ω, the CCD 109
It is calculated from the coordinate values of the pixels on A and 109B.

By calculating the three-dimensional coordinates (X, Y, Z) for all the pixels in the camera image according to the above equation, the three-dimensional information of the subject can be calculated. Further, the distance image can be obtained by calculating only the Z coordinate value.

Also, a low-cost range finder apparatus has been proposed in which the mechanism on the light source side does not perform a mechanical operation. This range finder apparatus has a light source section capable of projecting a plurality of lights having different two-dimensional radiation patterns, instead of having a laser light source and a light sweeping means such as a rotating mirror.

That is, in this range finder device, a plurality of lights having mutually different two-dimensional radiation patterns are radiated to a subject in a time division manner, reflected light from the subject is imaged by a camera, and the light intensity of the imaged image is used. Measure the distance. Then, at the time of distance measurement, the three-dimensional information is obtained according to the coordinate value in the image by using the light intensity ratio of the reflected light or the correspondence relationship between the light intensity and the projection light angle.

In such a configuration, all three-dimensional measurement is realized by electronic operation, so that the reliability of the apparatus becomes higher and the measurement performance becomes stable. Further, there is an advantage that the cost is relatively low because no mechanical mechanism is required.

[0018]

However, the conventional range finder apparatus has the following problems.

[0019] In rangefinder using the record over. The light source is usually a slit light spread in a direction perpendicular to the light emitted from the laser light source by a lens system, which is swept by the rotating mirror. In such a device, the distance measurement of the subject is performed on the assumption that the light intensity of the projected light changes only in the horizontal direction and is constant in the vertical direction. However, due to the effect of lens shading, that is, peripheral dimming of the lens system, the light intensity of the slit light becomes weaker at both end portions than at the slit central portion. Therefore, the light intensity changes even in the vertical direction, and therefore the observed light intensity ratio of the reflected light and the projection direction of the projected light do not correspond one-to-one, which makes it difficult to measure the distance with high accuracy. Become. Further, even when a plurality of lights having a two-dimensional radiation pattern are projected, the same problem may occur.

[0020] viewed Kan said problems, the present invention is, as rangefinder, even light characteristic of the reflected light and the projection direction of the projection light in a case that does not correspond one-to-one in the camera image, An object is to enable highly accurate measurement of three-dimensional information.

[0021]

According to a first aspect of the present invention, there is provided a light source unit for projecting projection light, the characteristics of which are changed according to the projection direction, as a range finder device, and the projection light. A camera unit for capturing reflected light from the subject, and a three-dimensional information generating unit for generating three-dimensional information of the subject from the reflected light image captured by the camera unit. Multiple in camera image
The regions are respectively provided in correspondence, it the correspondence between the projection direction of the light characteristics of the reflected light the projection light
Is have a plurality of look-up table describing, in accordance with the pixel position in the camera image, the one of the plurality of look-up table to select each of the three-dimensional information by referring to the lookup table selected To generate.

According to the first aspect of the present invention, the three-dimensional information generating section has a plurality of look-up tables in which the correspondence between the light characteristics of the reflected light and the projection direction of the projected light is described. Either look-up table is selected depending on the pixel position in the image. Therefore, even if
Even if the correspondence between the light characteristics of the reflected light and the projection direction of the projected light does not correspond one-to-one in the camera image, the lookup table selected according to the pixel position is referred to, so the accuracy is high. Three-dimensional information can be generated.

According to the invention of claim 2 , the light source section in the range finder device of claim 1 is provided with a flash lamp.

According to the invention of claim 3 , the plurality of look-up tables in the range finder device of claim 1 are respectively provided corresponding to epipolar lines in a camera image.

According to the invention of claim 3 , on the epipolar line, the light characteristics of the reflected light and the projection direction of the projected light always have a one-to-one correspondence. Therefore, by providing a lookup table corresponding to the epipolar line. , It is possible to efficiently and economically provide a plurality of lookup tables.
Therefore, it is possible to accurately generate three-dimensional information within a practical calculation amount and storage capacity range.

According to the invention of claim 4 , the three-dimensional information generation unit in the range finder device of claim 3 corresponds to the epipolar line in the vicinity of the epipolar line for the pixel position on the epipolar line where the lookup table is not provided. The lookup table thus provided is interpolated to obtain the lookup table.

According to a fifth aspect of the present invention, in the light source unit and the camera unit in the range finder apparatus according to the third aspect , the straight line connecting the light source unit and the camera unit coincides with the X axis in the world coordinate system, and the camera. It is assumed that they are installed so as to be parallel to the x axis in the coordinate system.

According to the invention of claim 5 , the epipolar line is parallel to the x-axis in the camera image. Therefore, the construction of the look-up table and the distance calculation become much easier.

Further, in the means for solving the problems of the sixth aspect of the invention, a set of the light patterns for projecting a set of light patterns in which the light intensity ratio changes according to the projection direction and the set of the projected light patterns are used. The three-dimensional image processing apparatus includes: a camera unit that captures each reflected light image of the subject; and a three-dimensional information generation unit that generates three-dimensional information of the subject from a set of reflected light images captured by the camera unit. The information generator is a camera
It is provided for each of the multiple areas in the image.
Ri, a plurality of look-up table describing each correspondence relationship between the projection direction and the pair of the light intensity ratio of the light pattern possess <br/>, in accordance with the pixel position in the camera image, the plurality of look-up The three-dimensional information is generated by selecting any one of the tables and referring to the selected look-up table.

[0030]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

( Reference Example ) FIG. 1 is a diagram showing a configuration of a range finder apparatus according to a reference example of the present invention. In FIG. 1, the light source unit 2 is composed of two light sources 2a and 2b of flash lamps,
The projection light is projected so that the intensity of the light changes in position on the subject 106. The camera unit 1 images the reflected light from the subject 106 of the projection light projected from the light source unit 2. The light source control unit 5 synchronizes with the vertical synchronization signal of the camera unit 1,
a and 2b are made to emit light alternately every frame period or field period. Distance calculation unit 6 as three-dimensional information generation unit
Generates a distance image as three-dimensional information of the subject 106 from the reflected light image captured by the camera unit 1.

The light sources 2a and 2b of the light source unit 2 are
Light receiving elements 21a and 21b for detecting the intensity of light emitted from each of the light sources 2a and 2b are provided. The light intensity detected by the light receiving elements 21a and 21b is sent to the light source controller 5 and further sent to the distance calculator 6 as predetermined correction information. In the distance calculation unit 6, the reflected light image is corrected based on this correction information, and a distance image is generated using the corrected image.

FIG. 2A is a perspective view showing a configuration example of the light sources 2a and 2b. As shown in FIG. 2A, the light sources 2a,
As 2b, for example, a flash light source 7 or 8 such as a xenon flash lamp is vertically arranged on the same axis, and the range of the light reflecting direction of the rear reflectors 9 and 10 is shifted by a predetermined angle with respect to the axis. To use. In addition, the light receiving element 21
Reference numerals a and 21b are photodiodes provided on the reflection plates 9 and 10, respectively, and detect the light intensity emitted from each of the light sources 2a and 2b. 2 (b) is shown in FIG.
It is a top view of a structure of (a). As shown in FIG. 2B, the light sources 2a and 2b radiate (project) light in the ranges A and B, respectively. It is desirable that the xenon lamp used here has a small light emitting portion and can be regarded as a point light source when viewed two-dimensionally.

FIG. 3 is a diagram showing optical patterns radiated from the light sources 2a and 2b shown in FIG. In FIG. 3, the solid line L
Reference characters a and Lb represent the brightness of the screen surface when light is projected from the light sources 2a and 2b onto the imaginary screen Y. The degree of brightness is represented by the height of solid lines La and Lb in the → direction. As can be seen from FIG. 3, the projected light from each of the light sources 2a and 2b has a characteristic that the light intensity is strongest on the central axis in the projection direction (that is, the brightest) and weaker toward the periphery (that is, darker). . This characteristic results from the fact that the semi-cylindrical reflectors 9 and 10 are arranged behind the flash light sources 7 and 8. The projection lights of the light sources 2a and 2b partially overlap with each other depending on the directions of the reflection plates 9 and 10.

FIG. 4 is a graph showing the relationship between the position and the light intensity in the horizontal direction (H direction) of FIG. In the part α of the light pattern shown in FIG. 4, the light emitted from the light sources 2a and 2b to the subject space is one where the right side is bright and the left side is dark and the other side is the left side and the right side is dark as viewed from the light source. Has become. Here, the light intensity characteristics of the light sources 2a and 2b in the portion α are represented as Ia and Ib, respectively. However, in FIG. 4, the lens center of the camera unit 1 is passed and y
The coordinate indicates a light intensity characteristic on a plane having a predetermined value (for example, 0), and the distribution of the light intensity characteristic differs depending on the y coordinate value.

FIG. 5 is a graph showing the relationship between the projection light angle φ from the light source unit 2 and the light intensity ratio Ib / Ia in the portion α of FIG. As shown in FIG. 5, in the portion α, the light intensity ratio Ib / Ia and the projection light angle φ have a one-to-one correspondence.

In order to measure the distance, two types of light patterns are alternately projected in advance on a vertically standing plane, and the reflected light is imaged by the camera unit 1 as shown in FIG. The relationship between the light intensity ratio and the projection light angle must be obtained in advance for each y coordinate. Then, if the light source unit 2 is arranged so that the line segment connecting the lens center of the camera unit 1 and the light source unit 2 is parallel to the x-axis of the imaging surface, the light intensity ratio and the projection light for each y coordinate obtained in advance. An accurate distance calculation can be performed by using the data of the relationship with the angle.

FIG. 6 is a diagram showing the internal structure of the distance calculation unit 6. As shown in FIG. 6, the distance calculation unit 6 includes a camera unit 1
First and second image memories 61a, 61 for temporarily storing the reflected light image signal picked up by each frame
b, first and second light intensity correction units 62a and 62b for respectively correcting the light intensity of the images stored in the first and second image memories 61a and 61b using the correction information,
A light intensity ratio calculator 63 that calculates a light intensity ratio from the light intensities of the images corrected by the light intensity correctors 62a and 62b.
And a distance image generation unit 64 that generates a distance image of the subject 106 from the calculated light intensity ratio.

The reflected light image by the projected light of the light source 2a is the first
Image memory 61a, and the reflected light image of the projection light of the light source 2b is stored in the second image memory 61b.
In the correction information, the light intensity detected by the light receiving element 21a is given to the first light intensity correcting section 62a, and the light intensity detected by the light receiving element 21b is given to the second light intensity correcting section 62b.

Further, the first and second light intensity correction units 6
Reference values Ira and Irb used for light intensity correction are stored in advance in 2a and 62b, respectively. For example, the actual light intensity projected from each light source 2a, 2b is measured by the light receiving elements 21a, 21b a plurality of times,
The average values of the measured values are stored in the first and second light intensity correction units 62a and 62b as reference values Ira and Irb, respectively.

The operation of distance calculation of the range finder device according to this reference example will be described below. Here, a case where the point P on the subject 106 in FIG. 1 is set as a point of interest and the depth value Z (Z-axis coordinate value) of the point P is calculated will be described.
In particular, the operation of correcting the light intensity in the distance calculator 6 will be described in detail.

FIG. 7 is a timing chart showing the operation of the range finder apparatus according to this reference example .

As shown in FIG. 7, first, the light source controller 5
In synchronization with the vertical synchronization signal (FIG. 7A) of the camera unit 1,
The light emission control signals a and b (FIGS. 7B and 7C) are alternately output for each frame cycle or field cycle, whereby the light sources 2a and 2b are alternately emitted. Light receiving element 21
Reference numerals a and 21b sequentially detect the light intensities of the light sources 2a and 2b. The light source control unit 5 takes in the light intensities detected by the light receiving elements 21a and 21b at the timings shown in FIGS. 7D and 7E, respectively.

The camera unit 1 time-divisionally captures the reflected light image. That is, in the first imaging period, the light source 2a
The reflected light image by the projection light of is captured, and the reflected light image by the projection light of the light source 2b is captured in the second imaging period. The reflected light image of the projected light of the light source 2a is written in the first image memory 61a, and the reflected light image of the projected light of the light source 2b is written in the second image memory 61b.

The light source controller 5 uses the light intensity data of the light source 2a detected by the light receiving element 21a as the correction information Isa.
To the light intensity correction unit 62a of the distance calculation unit 6. Similarly, the light intensity data of the light source 2b detected by the light receiving element 21b is used as the correction information Isb as the light intensity correction unit 62b.
Send to.

Next, the distance calculator 6 generates a distance image.

First, the light intensity correction units 62a and 62b convert the reflected light images written in the image memories 61a and 61b into correction information Isa and Isb sent from the light source control unit 5.
And the reference values Ira and Irb stored in advance are used for the correction.

That is, the light intensity correction section 62a and the correction information Isa sent with the reference value Ira stored in advance.
Is used to calculate the conversion ratio Ira / Isa, and the conversion ratio Ira / Isa is multiplied by each light intensity data of the reflected light image stored in the image memory 61a. The result of this multiplication is output to the light intensity ratio calculator 63 as the corrected light intensity Ia. Similarly, the light intensity correction unit 62b also calculates a conversion ratio Irb / Isb using the reference value Irb stored in advance and the correction information Isb sent in advance, and the conversion ratio Irb / Ib is calculated.
sb is multiplied by each light intensity data of the reflected light image stored in the image memory 61b. The result of this multiplication is output to the light intensity ratio calculator 63 as the corrected light intensity Ib.

The light intensity ratio calculator 63 calculates the light intensity ratio Ib / Ia using the corrected light intensities Ia and Ib and outputs it to the distance image generator 64.

The distance image generating unit 64 uses the correspondence relationship between the light intensity ratio Ib / Ia and the projection light angle φ and the correspondence relationship between the pixel coordinate value x and the angle θ as shown in FIG. With respect to the pixel of, the angles φ and θ are specified, the distance Z is obtained, and the distance image is output.

The distance image generating unit 64 is prepared in advance for each y coordinate with a corresponding relationship between the light intensity ratio and the projection light angle φ as shown in FIG. The distance image generation unit 64 determines the light intensity ratio and the angle φ corresponding to the y coordinate value of the point P.
And the projection light angle φ of the point P is specified from the selected correspondence. Further, the angle θ with respect to the point P viewed from the camera unit 1 is determined from the pixel coordinate value of the point P and the camera parameter (focal length f, optical center position of the lens system). Then, the calculated two angles φ and θ and the distance between the light source position and the optical center position of the camera unit, that is, the baseline length D
From the above, the depth value Z of the point P is calculated by the principle of triangulation.

That is, with the optical center of the camera section 1 as the origin, the optical axis direction of the camera section 1 is set as the Z axis, the X axis is set in the horizontal direction, and the Y axis is set in the vertical direction. Assuming that the angle between the direction and the X axis is φ, the angle between the direction of the point of interest viewed from the camera section and the X axis is θ, and the light source position is (−D, 0, 0), that is, the baseline length is D, the point of interest P The depth value Z of is calculated by the equation (1).

As described above, by correcting the light intensity in the reflected light image taken by the camera section 1 by the correction information, it is possible to obtain a more accurate range image.

Further, here, the distance calculation unit 6 calculates only the distance Z as the three-dimensional information and outputs the calculation result as a distance image. However, the present invention is not limited to this, and for example, FIG.
Using the angle ω shown in, the three-dimensional coordinate value X,
All Y and Z may be calculated and output as three-dimensional information of the subject. X = Z / tan θ Y = Z · tan ω (2)

In this reference example , the light intensity correction unit 62
The correction in a and 62b is performed by the conversion ratio Ira / Isa, I
Although the simple process of multiplying the light intensity data by rb / Isb is performed, the present invention is not limited to this, and for example, it is desirable to perform the offset process before the multiplication.

For example, the reflected light images stored in the image memories 61a and 61b are subjected to offset processing for subtracting the black level component (for example, "10") from the image data, and then the conversion ratios Ira / Isa and Irb / Isb are set. Correction is performed by multiplying. The value of the black level component may be measured only once in advance. By performing such processing, when the emission intensity of the light source fluctuates by about 5%, the measurement error is 10
% Can be reduced. That is, it is possible to perform correction with higher accuracy.

Also, instead of subtracting the black level component,
An offset process may be executed to subtract the light intensity of the reflected light image when there is no projected light, that is, when there is only background light. In this case, not only the influence of the background light but also the influence of the black level component is canceled, so that it is possible to perform the correction with higher accuracy.

FIG. 9 is a diagram showing another example of the internal configuration of the distance calculation section. The distance calculation unit 6A shown in FIG. 9 is for light intensity ratio calculation having a lookup table (LUT) for each conversion ratio, instead of the light intensity correction units 62a and 62b and the light intensity ratio calculation unit 63 in the configuration of FIG. The LUT unit 71 is provided as a light intensity correction unit.

In the configuration of FIG. 9, it is necessary to prepare the LUT for the conversion ratio in the range required for the light intensity ratio calculation so that it can correspond to all the correction information sent from the light source control section 5. For example, when the emission intensity of the light sources 2a and 2b fluctuates about ± 5% with respect to the reference emission intensity, 0.95 to
It is necessary to provide the LUT for the conversion ratio in the range of 1.05. Then, in each LUT, the light intensity ratio data corrected by the conversion ratio is stored according to the light intensity data.

The light intensity ratio calculation LUT unit 71 obtains a conversion ratio for correction from the correction information sent from the light source control unit 5, and selects the LUT corresponding to the calculated conversion ratio. Then, the light intensity data is extracted from each of the image memories 61a and 61b for each pixel, the corrected light intensity ratio corresponding to each light intensity data is directly obtained by referring to the selected LUT, and the distance image generation unit is obtained. To 64.

In this reference example , the light sources 2a and 2b are made to emit light in a time-division manner, but not limited to this, for example,
It may be configured to emit light at the same time. As a result, the measurement time can be shortened. However, in this case, the light source 2
It is necessary to make the light wavelengths a and 2b different from each other and to provide the camera unit 1 with a filter element capable of selectively receiving each wavelength.

Further, in this reference example , the light receiving elements 21a, 2a
Although 1b is provided inside the light sources 2a and 2b, the present invention is not limited to this and may be provided outside the light sources 2a and 2b as long as the light intensity of the light sources 2a and 2b can be detected.

<Modification of Reference Example > In the above-mentioned reference example , in order to obtain predetermined correction information, the light receiving elements 21a, 21 for detecting the light intensity of the light sources 2a, 2b.
Although b is provided, it is not always necessary to provide a light receiving element in order to obtain the correction information.

FIG. 10 is a diagram showing an example of the internal configuration of the distance calculation section according to the modification of this reference example . Here, the brightness of a predetermined area in the captured image of the camera unit 1 is used as the predetermined correction information. That is, the first light intensity correction unit 62a uses the brightness of a predetermined area of the captured image data stored in the first image memory 61a as the correction information, and the second light intensity correction unit 62b uses the second light intensity correction unit 62b. Image memory 6
The brightness of a predetermined area of the captured image data stored in 1b is
It is used as correction information. In this case, for example, an object serving as a reference for brightness, such as a white paper, may be arranged in the area to be imaged, and the brightness of the captured image of this object may be used as the correction information.

( First Embodiment ) FIG. 11 is a view showing the arrangement of a range finder apparatus according to the first embodiment of the present invention. In FIG.
The same components as in the conventional configuration shown in FIG. 24 are assigned the same reference numerals as those in FIG. 24, and detailed description thereof will be omitted here. In FIG. 11, 210A and 210B are laser light sources. The light source unit 20 includes the laser light sources 210A and 210B, the half mirror 102, and the rotating mirror 104.
0 is configured, and the camera unit 3 includes the lens 107, the light wavelength separation filters 108A and 108B, the CCDs 109A and 109B, and the signal processing units 110A and 110B.
00 is configured.

In the present embodiment, in order to represent a three-dimensional position, an xy coordinate system (this is called a "camera coordinate system") is used as a coordinate system in the field of view of the camera, and in a real space. An XYZ coordinate system (this is referred to as a "world coordinate system") is used as the coordinate system.

FIG. 12 shows the light source 210 in the configuration of FIG.
It is a figure which shows the structure of the optical system which concerns on A. In FIG. 12, reference numeral 211 denotes a lens which is disposed between the laser light source 210A and the rotating mirror 104 and which shapes the light flux from the light source 210A into slit light 212 and converges it on the center 104a of the rotating mirror. The optical system related to the light source 210B is similar to that shown in FIG.

In the optical system shown in FIG. 12A, the converging point 213 of the slit light 212 coincides with the center 104a of the rotating mirror, which can be said to be the most desirable configuration. On the other hand, FIG.
As in the optical system shown in FIG. 2B, the convergence point 213 of the slit light 212 may not necessarily coincide with the center 104a of the rotating mirror. In this case, in principle, the distance between the convergence point 213 and the center 104a causes an error in distance measurement. However, this interval can be made relatively small, and as will be described later, since the lookup table (LUT) is calculated so that the measured distance value to the calibration surface matches the known Z value, it is practical. It doesn't matter.

Further, the LUT holding section 220 has the light intensity ratio of the reflected light of the projected light from each of the light sources 210A and 210B and the rotation angle of the rotating mirror (that is, the light from the light sources 210A and 210B as described in FIG. 26). A plurality of types of LUTs that show a one-to-one correspondence with projection angles) are held.
The distance calculation unit 230 uses a plurality of LUTs held in the LUT holding unit 220 according to the pixel position in the camera image.
Any one of the above is selected, and a distance image as three-dimensional information is generated using the selected LUT. The LUT holding unit 220 and the distance calculation unit 230 form a three-dimensional information generation unit 400.

FIG. 13 is a diagram showing the internal structure of the LUT holding unit 220 and the distance calculating unit 230. CCD109A
The image data captured by the CCD 109B is converted into a light intensity ratio for each pixel in the light intensity ratio calculator 231. The coordinate value generation unit 232 controls the control unit 113.
From the sync signal of the image data given by
Generate coordinate values. The LUT selection unit 233 selects the LUT 221 to be referred to according to the y coordinate value generated by the coordinate value generation unit 232.

The selected LUT 221 converts the value of the light intensity ratio obtained by the light intensity ratio calculator 231 into the projected light angle information 1 / tan φ. On the other hand, the coordinate value generation unit 23
The x-coordinate value generated by 2 is converted into angle information 1 / tan θ from the camera in the conversion unit 234.
The calculation unit 235 calculates the baseline length D and the angle information 1 / tan.
The Z value is calculated from θ and 1 / tan φ according to the following equation (3).

[0072]

[Equation 1]

The above formula (3) corresponds to the denominator numerator on the right side of the formula (1) described in the reference example divided by tan θ · tan φ, and is essentially the same as the formula (1). However, since values near 90 ° are used for both θ and φ during measurement, the values are undefined when θ and φ are 90 ° during numerical calculation.
1 / tanθ, 1 / tanφ instead of nθ, tanφ
The calculation is more stable when is used as the angle information.

Here, the light intensity data is set to 8 bits, and the projection light angle information 1 output from each LUT 221 is set.
If / tanφ is represented by 2 bytes, LUT22
The size per 1 is about 130 KB (≈2 ⁸
× 2 ⁸ × 2).

The operation of the range finder apparatus according to this embodiment will be described below. The main basic operation for obtaining the three-dimensional coordinate values (X, Y, Z) is the same as that described using FIG. A feature of the present embodiment is that the LUT is used properly according to the pixel position on the camera image when obtaining the projection light angle φ.

FIG. 14 is a schematic diagram showing the correspondence between camera images and LUTs in this embodiment. 14
Then, to simplify the explanation, 5 CCD lines 1
Only 001 to 1005 are shown. Then, for the CCD line 1001, the LUT 1 is the CCD line 100.
It is assumed that LUT2 is prepared in advance for 3 and LUT3 is prepared in advance for CCD line 1005. Actually, the CCD line where the LUT is provided is 1
It may be about 0. These LUTs are respectively
It is created in advance at the design and manufacturing stage of this range finder device.

In this embodiment, each LUT is provided corresponding to the epipolar line. The "epipolar line" refers to a straight line reflected on a camera image when a straight line in space is photographed, and here, a straight line reflected by a straight laser beam projected from a light source.

In this embodiment, the CCD lines 1001 to 1005 are aligned with the epipolar line,
In other words, the camera installation conditions are set so that the epipolar lines are arranged horizontally on the camera image. Specifically, the lens center 107a of the camera and the center 1 of the rotating mirror
It is necessary to determine the arrangement of the cameras so that the straight line connecting to 04a is on the X axis of the world coordinate system, and this straight line is parallel to the x axis of the camera coordinate system.

3 corresponding to one epipolar line
On the surface in the dimensional space, the correspondence between the light intensity ratio and the projection light angle is always 1: 1. Therefore, by providing the LUT for each epipolar line, the distribution of the light intensity varies, and even if the optical characteristics of the reflected light and the projection direction of the projected light are in a one-to-one correspondence in the camera image. Even without it, you can perform distance calculation with high accuracy,
Moreover, a plurality of LUTs can be provided efficiently and economically.

That is, the distance calculation unit 230 calculates the light intensity ratio for each pixel of the CCD and obtains the coordinate value P (x, y) of the pixel of interest P in the camera coordinate system,
Decide which LUT to use. For example, when the target pixel P is on the CCD line 1001 as shown in FIG. 14, the LUT1 is used and the CCD lines 1003, 1
When it is on 005, LUT2 and LUT3 are used respectively. When the pixel of interest P is on the CCD line 1002 that does not have a corresponding LUT, C in the vicinity thereof
L provided corresponding to the CD lines 1001 and 1003
UT1 and LUT2 are used after being linearly interpolated.

When the projection light angle is specified using the LUT,
The subsequent processing is the same as that described with reference to FIG. In this way, since an appropriate LUT is selected according to the CCD line in which the pixel of interest exists, that is, the epipolar line, accurate distance calculation can be executed even if there is peripheral dimming in the lens system, for example.

Next, the principle of operation according to the present embodiment will be described together with a theoretical description of the epipolar line.

FIG. 15 is a diagram for explaining the operation principle of the range finder apparatus according to this embodiment, and is a schematic perspective view expressed based on the world coordinate system. Also,
16 is a schematic plan view of FIG.

In the arrangements of FIGS. 15 and 16, the above-mentioned camera installation conditions are satisfied, and the lens center 107a of the camera 1104 and the center 104a of the rotating mirror are arranged so as to be on the X axis of the world coordinate system. ing. Further, the light source 1105 is a virtual light source device for explaining the operation principle, and is a light source that radiates laser linear light of two types of light patterns as shown in FIG. 25 in a time division manner.

CP1 and CP2 are calibration surfaces placed at Z coordinate values Z1 and Z2 (Z1 <Z2), respectively. In FIG. 15, the calibration planes CP1 and CP2 are drawn only in the upper half of the range reflected in the visual field of the camera 1104. The lower half of the field of view of the camera is symmetrical to the upper half, and therefore is not shown in FIG. 15 for the sake of simplicity.

Here, the angle between the vertical plane including the point of interest and the rotation axis of the rotating mirror 104 and the X axis is φ. For example, the angle between the vertical plane VPA including the point of interest PA1 and the rotation axis of the rotating mirror 104 and the X axis is φA, and the vertical plane VP including the point of interest PB1 and the rotating axis of the rotating mirror 104.
The angle between B and the X axis is φB.

Further, the point of interest and the viewpoint (lens center 107a
Or a straight line YZ connecting the center 104a of the rotating mirror
The angle formed by the projection on the plane with the Z axis is defined as the tilt angle ω. Here, due to the installation conditions of the camera, the lens center 10
7a and the center 104a of the rotating mirror are both on the X-axis, the tilt angle when the same point of interest is seen when the lens center 107a is viewed and when the rotating mirror center 104a is viewed. Will always be the same. The angle θ is as shown in FIG.

The linear laser beam emitted from the light source 1105 is reflected at one point of the center 104a of the rotating mirror and is swept by the rotation of the rotating mirror 104. At the same time, the light source 1105 changes its posture,
The emission angle ωL of the linear laser light is sequentially changed. This emission angle ωL corresponds to the above-described tilt angle ω.

The reason for assuming such a light source 1105 is as follows.
This is for expressing the peripheral dimming in the Y-axis direction of the actual projected light from the light source unit. That is, the actual light source unit is a device that sweeps and emits slit light parallel to the Y-axis, and the light intensity of the projection light is adjusted so as to change according to the sweep direction (X-axis direction). However, actually, also in the Y-axis direction, a phenomenon called so-called peripheral dimming occurs in which the light intensity decreases near the periphery as compared with the central portion of the slit light. Therefore, assuming a light source 1105 that emits a linear laser beam,
The peripheral dimming in the actual light source is reproduced by using the emission angle ωL as a parameter.

When one linear light is irradiated in the space and the Z coordinate value of the calibration surface is changed from 0 to infinity, the reflection point on the calibration surface is y on the camera image. An image is taken as a straight line trajectory having constant coordinate values. This straight line corresponds to the epipolar line described above.

Two kinds of light intensity I in the same direction
Light intensity ratio ρ = Ia when lights a and Ib are alternately emitted
Focusing on / Ib, if the angle φ is in the same direction, the light intensity ratio ρ can be regarded as constant regardless of the distance traveled by the light. Therefore, in this description, for easy understanding, linear light with a constant light intensity ratio is assumed as the linear light emitted in the same direction.

FIG. 17 shows each point of interest P on the camera image.
It is a schematic diagram which shows the relationship between the position of A1, PA2, PB1, PB2, and a CCD line. PA1 and PA2 are linear light LA having a light intensity ratio ρA (constant value) and calibration surfaces CP1 and CP2.
Is the intersection with. PB1 and PB2 are the light intensity ratio ρ
It is the intersection of the linear light LB of B (constant value) and the calibration surfaces CP1 and CP2.

The following can be said from FIGS. 15, 16 and 17.

(1) Since the linear light LA passes through the point PA1 on the calibration plane CP1, the plane L including the point PA1 and the lens center 107a and determined by the tilt angle ω from the Z axis
It is included in the same plane as P1 (a cross-hatched surface).

(2) Further, since the linear light LA passes through the point PA2 on the calibration plane CP2, the point PA2 and the lens center 107a.
And is included in the same plane as the plane LP2 (hatched surface) determined by the tilt angle ω from the Z axis.

(3) The plane LP1 and the plane LP2 are the same plane, and the point P on the xy plane of the camera coordinate system.
A1 and PA2 are reflected on the same CCD line (CCD line 1001 in FIG. 17) left and right separately. That is, regarding the linear light LA having a constant light intensity ratio, when the Z coordinate value of the point of interest is different, the pixel position in the camera coordinate system due to the reflected light is always different position on the same CCD line.

(4) The same applies to the above (1) to (3) with respect to the linear light LB in which the rotation angle φ is changed from φA to φB without changing the tilt angle ω. That is, the linear light LB is generated at the point PB1 on the calibration plane CP1 and the lens center 1
07a and a plane determined by a tilt angle ω from the Z axis, a point PB2 on the calibration plane CP2, and a lens center 1
07a and are included in the plane determined by the tilt angle ω from the Z axis. Therefore, the straight line LZ1 connecting the points PA1 and PB1 on the calibration plane CP1 and the straight line LZ2 connecting the points PA2 and PB2 on the calibration plane CP2 always exist on the CCD line 1001 in the camera coordinate system shown in FIG. .

From the above, the following can be understood.

First, the plane LP1, with a predetermined tilt angle ω,
All the points of interest included in LP2 have the same y-coordinate value on one CC on the camera image, regardless of their Z-coordinate values.
An image is captured on the D line 1001.

Secondly, regarding one straight light beam LA having a constant light intensity ratio ρ, the reflected light beams from the respective points having different Z coordinate values have the same y coordinate value on the camera image. CCD
Although they are located on the line 1001, their x coordinate values are always different from each other. Therefore, it is possible to distinguish each point from the x coordinate value.

Third, at the same pixel position on the camera image,
Even if the reflected lights from a plurality of points having different spatial positions are received, the light intensity ratios of the reflected lights from the respective points are always different from each other. Therefore, also in this case, it is possible to distinguish each point from the observed light intensity ratio.

Therefore, as in this embodiment, (a)
An appropriate LUT is provided in advance for each CCD line having a constant y coordinate value, and (b) both the pixel position (x, y) on the camera image and the light intensity ratio ρ observed at that pixel position are obtained.
(C) The LUT corresponding to the CCD line corresponding to the pixel position can be selected, and (d) the selected LUT can be used to specify the Z coordinate value of the target point P on the subject corresponding to the pixel position. it can.

The LUT is created, for example, as follows.
A calibration plane (vertical plane) is arranged at a position where the Z value is known, and the calibration plane is irradiated with light from the light sources 210A and 210B. Then, an LUT is created for each y coordinate so that the distance measurement value to the calibration surface matches the known Z value. At this time,
In principle, there may be only one calibration surface, but from multiple calibration surfaces L
By making the UT, the measurement accuracy can be practically improved.

For all CCD lines, LU
You may prepare T. Alternatively, LUTs are provided only for specific CCD lines at predetermined intervals, and for pixels on CCD lines that are not provided with LUTs, LUTs provided in the vicinity of the pixels, for example, the CCD lines above and below it
May be linearly interpolated to create the LUT.

Instead of the LUT, function fitting may be performed on the entire camera image using the light intensity ratio ρ and the pixel position (x, y) as parameters.

( Second Embodiment ) In the first embodiment described above, it is necessary to set the camera installation conditions so that the epipolar line is parallel to the x axis of the camera coordinate system and coincides with the CCD line. It was That is, the lens center 107a of the camera and the center 1 of the rotating mirror.
It is necessary to determine the arrangement of the cameras so that the straight line connecting to 04a is on the X axis of the world coordinate system, and this straight line is parallel to the x axis of the camera coordinate system. Therefore, the degree of freedom in arranging the camera and the light source is small.

In the second embodiment of the present invention, the first embodiment
The degree of freedom in arranging the camera and the light source is further expanded rather than the form . That is, the basic configuration and operation are the same as those in the first embodiment , but the camera installation conditions, epipolar line setting method, LUT creation method,
And the operation of the distance calculation unit is different.

FIG. 18 is a view for explaining the operating principle of the range finder apparatus according to this embodiment, and is a schematic perspective view expressed based on the world coordinate system.

The camera installation conditions in this embodiment are as follows.

The lens center 107a of the camera 1104 is set as the origin of the world coordinate system, the optical axis of the camera 1104 is aligned with the Z axis, and the x axis direction (direction of the CCD line) of the camera coordinate system is aligned with the X axis. However, it is not necessary that the center 104a of the rotating mirror be on the X axis. In this respect, the degree of freedom of arrangement is expanded as compared with the first embodiment .

Under the camera installation conditions of this embodiment,
Unlike the first embodiment , the epipolar lines are not parallel straight lines on the camera image. Therefore, in order to create the LUT, it is first necessary to find the shape of the epipolar line on the camera image.

<Determination of Epipolar Line Shape> FIG. 19 is a diagram showing the shape of the epipolar line in this embodiment. As shown in FIG. 19B, in this embodiment, each epipolar line is a straight line that radially extends from the vanishing point S (xm, ym). Therefore, if the coordinate value of the vanishing point S can be determined, the shape of each epipolar line can be specified.

A method of determining the coordinate values of the vanishing point S will be described with reference to FIGS. 18 and 19.

First, the unit direction vector u from the light source 1105 to the point P is defined by the following equation (4). Further, in order to separate the term of ω and the term of φ, equation (4)
Each component of is divided by cosω and defined as a direction vector U. U = (cos φ, tan ω, sin φ) (5) The reason why the direction vector U is used is that in the formula (5), the X component and the Z component are represented by only φ, and the Y component is represented by only ω. This is because the calculation is easier than using the unit vector u of the equation (4).

The definition of each angle is as shown in FIG.
The angle ω in the present embodiment is not the tilt angle defined in the first embodiment , but the unit direction vector u and XZ
It is the angle formed by the plane.

Next, as shown in FIG. 19A, the epipolar lines are different from the rotation center 104a on the camera image with respect to the plurality of calibration planes CP1 and CP2 whose Z coordinate values Z1 and Z2 are known. , And the light intensity ratio (ρ1, ρ
2, a plurality of (three in the figure) linear light L1, L2, L3 having different ρ3) are emitted. P11, P21, P31
Is a reflection point on the calibration surface CP1 of each linear light L1, L2, L3, and P12, P22, P32 are each linear light L1, L.
It is a reflection point on the calibration plane CP2 of 2, L3.

Further, in FIG. 19B, six reflection points P
The imaging positions of the camera images of 11 to P32 and the epipolar lines by the three linear lights L1, L2, and L3 between the two calibration surfaces CP1 and CP2 are shown.

Then, with respect to these reflection points P11 to P32, the position in the world coordinate system and the position in the camera coordinate system are measured. From the world coordinate values of P11 to P32 of the six points, the coordinate value A (Xm, Ym, Zm) of the rotation center 104a of the rotating mirror is obtained. In addition, FIG.
As shown in (b), the coordinate values (xm, ym) of the vanishing point S are obtained as the intersections of the three epipolar lines from the camera coordinate values of the six points P11 to P32.

<Generation of LUT> From the vanishing point S thus obtained, an epipolar line is drawn in a desired direction, and an LUT corresponding thereto is created. This method will be described with reference to FIGS. 20, 21 and 22.

First, as shown in FIG. 20, the calibration plane CP1
The point B whose coordinate value (X1, Y1, Z1) is known is imaged. Then, the known coordinate values (X1, Y1, Z1) are substituted into the general relational expression (6) between the coordinate values (x, y) on the camera image and the (X, Y, Z) of the world coordinate system. Then, the formula (7) is obtained. fx and fy are system parameters. x = fx · X / Z y = fy · Y / Z (6) x = fx · X1 / Z1 y = fy · Y1 / Z1 (7) where (x, y) is a measurement on the camera image Since the values are values, fx and fy can be obtained from Expression (7).

Next, as shown in FIG. 21 (a), two types of pattern light with light intensity modulation are applied in the direction of vector V, and the reflected light on the calibration plane CP1 of Z = Z1 is reflected by the camera 1.
The image is taken by 104. The light intensities of the respective pattern lights are Ia2 and Ib2, respectively, and the light intensity ratio thereof is ρ2 (=
Ia2 / Ib2). 21 (b), one epipolar line 1701 is formed on the camera image by the linear light having the light intensity ratio ρ2.

Pixel position B (x1, y1) on the camera image
And the light intensity ratio ρ2 is measured, the known Z coordinate value Z
1 and the system parameter fx obtained by the above equation (7),
From fy, X1 and Y1 are obtained by the following equation (8) using the equation (6). X1 = x1 · Z1 / fx Y1 = y1 · Z1 / fy (8) Further, the direction vector V from the point A to the point B is expressed by the following equation.

When the magnitude of the direction vector V is transformed so as to be the same as the magnitude of the direction vector U in the equation (5), the following equation (10) is obtained. Comparing the first and third components of equation (10) and equation (5), equation (11) is obtained. Φ is determined from the equation (11). In addition, the value of tan ω is determined from the second component of equation (10).

As a result of such processing, at the point B, the correspondence relationship between the light intensity ratio ρ and the rotation angle φ and the correspondence relationship between the light intensity ratio ρ and the tilt angle ω in the vertical plane including the straight line AB are shown. I want it. Similarly for the other plurality of points forming the epipolar line 1701, the correspondence between the light intensity ratio ρ and the rotation angle φ and the correspondence between the light intensity ratio and the tilt angle are obtained.

FIG. 22 shows the LU for one epipolar line.
It is a figure which shows the information stored in T, In the figure, (a) is a graph showing the correspondence of a light intensity ratio and a rotation angle, (b) is a correspondence of a light intensity ratio and a tilt angle (tan ω). It is a graph showing a relationship. In the figure, 1801a and 1801b are shown in FIG.
This is a point corresponding to point B of. For each point forming the same epipolar line, points 1801a, 1801
It is possible to create an LUT corresponding to one epipolar line by plotting on a graph as in the case of b and fitting the plotted distribution by a cubic expression.

For other desired epipolar lines, LUTs are created in the same manner as above.

<Method of Measuring Z Coordinate Value> Next, a method of measuring the Z coordinate value of the target point P will be described.

The LUT to be used is determined from the coordinate value (x, y) of the pixel of interest P in the camera coordinate system. And
From the measured light intensity ratio ρ of the target pixel P, the L determined
By referring to the UT, the direction vector (cosφ, tanω, sinφ) from the light source to the point of interest P is determined.

Therefore, the straight line connecting the light source and the point of interest P is
It is expressed as the following equation (12) using the parameter t.

[0130]

[Equation 2]

Further, when the equation (6) is modified, the following equation (13) is obtained.
Then, the following expression (14) is obtained from the expression (13). fx · X−x · z = 0 fy · Y−y · z = 0 (13) fx · X + fy · Y- (x + y) Z = 0 (14) The straight line represented by the formula (12) and the formula (12). Since the point of intersection with the plane represented by 14) is the point of interest P, the following equation (15) is solved to obtain the coordinate value (X, X,
Y, Z) is obtained.

[0132]

[Equation 3]

When it is not necessary to measure all the coordinate values X, Y and Z and only the Z coordinate value is measured, the equation (12) is used.
From equations (16) and (13), which are part of
Solve 7) to get the Z coordinate value. Xm + cos φ · t = X Zm + sin φ · t = Z (16)

[0134]

[Equation 4]

In this case, as the LUT, only the one relating to the correspondence between the light intensity ratio ρ and the rotation angle φ may be used.

As described above, according to this embodiment, the degree of freedom in arranging the camera and the light source can be expanded more than that in the first embodiment .

<Modification of Second Embodiment > In the second embodiment described above, the lens center 107 is used.
The case where a and the origin O of the world coordinate system coincide has been described. However, the lens center 107a and the origin O
Does not necessarily have to match. However, in this case, it is necessary to correct the coordinate system.

FIG. 23 is a diagram showing the positional relationship between the origin O of the world coordinate system and the lens center 107a in this modification. In the case as shown in FIG. 23, a known camera calibration method (see Shokoido “3D image measurement” by Seiji Iguchi and Kosuke Sato, p. 92-95) is used to determine the lens center 107a and the origin O. It suffices to measure the positional relationship between the camera parameters and correct the coordinate system. First, the camera parameter determinant (18) is calculated by a known camera calibration method.
Ask for.

[0139]

[Equation 5]

From the camera parameter matrix of equation (18), the camera coordinates (x, y) and the world coordinates (X, Y,
Z) is related with the following equation (19).

[0141]

[Equation 6]

Position A of the center of rotation 104a of the rotating mirror
The determination of (Xm, Ym, Zm) and the vanishing point S (xm, ym), and the generation of the LUT are performed in the same manner as in the second embodiment .

The method of measuring the Z coordinate value of the point of interest P is basically the same as that of the second embodiment , but instead of obtaining the intersection of equations (12) and (14), equation (12) ) And the formula (1
9), the following equation (20) is solved to obtain coordinate values (X, Y,
Z) is obtained.

[0144]

[Equation 7]

When obtaining the angles φ and ω from the light intensity ratio ρ, both x and y coordinate values on the camera image are required.

According to this modification, the degree of freedom in installing the camera and the light source can be further expanded.

Although the LUT is provided for each predetermined epipolar line in the first and second embodiments , the present invention is not limited to this, and for example, LUTs are provided for all pixels based on the same principle. I don't care. However, in this case, since information is duplicated between the LUTs, the configuration becomes redundant.

Even when the epipolar line is not parallel to the x axis as in the second embodiment , L is set for each y coordinate.
By providing the UT, the accuracy of the three-dimensional information can be improved as compared with the related art. Further, for example, even if an LUT is provided for each x coordinate, one LU is used for the entire camera image.
The accuracy of the generated three-dimensional information is improved as compared with the case where T is provided.

Further, in the first and second embodiments , the device configuration is provided with two light sources, but the present invention is not limited to this, and for example, one light source is provided and two types of modulated light are alternately arranged at a predetermined cycle. It may be configured to irradiate. Further, the device configuration using the laser light source and the rotating mirror has been described, but the present invention is not limited to this, and the same effect can be obtained even with a configuration using a flash lamp as the light source as in the reference example . In this case, in order to consider that the epipolar lines of the two flash lamps are practically coincident with each other within the measurement range,
It is desirable that the vertical distance between both lamps is as small as possible.

[0150]

According to the present invention, even if the correspondence between the light characteristics of the reflected light and the projection direction of the projected light is 1 in the camera image.
Even if it does not correspond to the pair 1, the lookup table selected according to the pixel position is referred to, so
It is possible to generate three-dimensional information.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a range finder device according to a reference example of the present invention.

2A and 2B are diagrams showing a configuration of a light source in the range finder apparatus of FIG. 1, wherein FIG. 2A is a perspective view and FIG. 2B is a plan view.

FIG. 3 is a diagram showing a light pattern emitted from the light source shown in FIG.

4 is a graph showing a relationship between a horizontal position and light intensity in the optical pattern of FIG.

5 is a graph showing a relationship between a projection light angle and a light intensity ratio in a portion α of FIG.

6 is a diagram showing an internal configuration of a distance calculation unit in the range finder apparatus of FIG.

7A to 7F are timing charts showing the operation of the range finder device shown in FIG.

FIG. 8 is a conceptual diagram of an angle ω used for calculation of three-dimensional positions X, Y, and Z.

9 is a diagram showing another example of the internal configuration of a distance calculation unit in the range finder device of FIG.

FIG. 10 is a diagram illustrating an example of an internal configuration of a distance calculation unit according to a modified example of the reference example .

FIG. 11 is a diagram showing a configuration of a range finder apparatus according to the first embodiment of the present invention.

12A and 12B are diagrams showing the configuration of an optical system related to a light source in the apparatus of FIG.

13 is a diagram showing an internal configuration of an LUT holding unit and a distance calculation unit in the device of FIG.

FIG. 14 is a schematic diagram showing a correspondence relationship between camera images and LUTs.

FIG. 15 is a schematic perspective view showing the operating principle of the range finder apparatus according to the first embodiment of the present invention based on the world coordinate system.

16 is a schematic plan view of FIG.

17 is a schematic diagram showing the relationship between the position of each point of interest in FIG. 15 on the camera image and the CCD line.

FIG. 18 is a schematic perspective view showing the operation principle of the range finder apparatus according to the second embodiment of the present invention based on the world coordinate system.

FIG. 19 is a diagram showing the shape of an epipolar line according to the second embodiment of the present invention, (a) showing three linear lights with different epipolar lines on a camera image, (b).
FIG. 6 is a diagram showing an epipolar line formed by each of the linear lights shown in (a).

FIG. 20 is a schematic plan view in the case of irradiating linear light with a constant light intensity ratio.

21A and 21B are diagrams for explaining generation of an LUT in the second embodiment of the present invention, FIG. 21A is a diagram showing irradiation of pattern light, and FIG. 21B is an epipolar by the pattern light shown in FIG. 21A. It is a figure which shows a line.

22A and 22B are graphs showing information stored in the LUT for one epipolar line, where FIG. 22A is a graph showing a correspondence relationship between a light intensity ratio and a rotation angle, and FIG. 22B is a light intensity ratio and a tilt angle. It is a graph showing a correspondence relationship with.

FIG. 23 is a diagram showing a positional relationship between an origin O of the world coordinate system and a lens center in a modified example of the second embodiment of the present invention.

FIG. 24 is a configuration diagram of a conventional range finder device.

FIG. 25 is a characteristic diagram showing wavelength characteristics of a light source of a conventional range finder device.

26A and 26B are characteristic diagrams of intensity modulation of a light source of a conventional range finder device.

27A and 27B are measurement principle diagrams in the range finder.

[Explanation of symbols]

1 camera section 2 light source 6,6A, 6B Distance calculation unit (three-dimensional information generation unit) 21a, 21b Light receiving element 62a, 62b Light intensity correction unit 106 subject 200 light source 221 lookup table 300 camera section 400 3D information generator

Front page continued (58) Fields surveyed (Int.Cl. ⁷ , DB name) G01B 11/00-11/30 G01C 3/06 G06T 1/00 G06T 7/00

Claims (6)

(57) [Claims] 1. A light source section for projecting projection light whose characteristics of light change according to its projection direction, a camera section for imaging reflected light of the projection light from a subject, and a reflection section imaged by the camera section. A three-dimensional information generation unit that generates three-dimensional information of the subject from an optical image, the three-dimensional information generation unit being provided corresponding to each of a plurality of regions in a camera image.
And which, have a plurality of lookup tables corresponding relationship described each of the optical characteristics of the reflected light and the projection direction of the projection light, in accordance with the pixel position in the camera image, the plurality of look-up tables Select any one of them and refer to the selected look-up table.
A range finder device characterized by generating dimensional information. 2. The range finder device according to claim 1 , wherein the light source unit includes a flash lamp. 3. The range finder apparatus according to claim 1 , wherein each of the plurality of lookup tables is provided corresponding to an epipolar line in a camera image. 4. The range finder device according to claim 3 , wherein the three-dimensional information generation unit is provided for a pixel position on an epipolar line where a look-up table is not provided, corresponding to an epipolar line near the epipolar line. A range finder device, characterized in that the lookup table obtained is interpolated to obtain the lookup table. 5. The range finder apparatus according to claim 3 , wherein in the light source unit and the camera unit, a straight line connecting the light source unit and the camera unit coincides with the X axis in the world coordinate system, and the camera coordinate system is used. A range finder device, which is installed so as to be parallel to the x-axis in the system. 6. A camera for respectively capturing a light source unit for projecting a set of light patterns whose light intensity ratio changes according to the projection direction, and a reflected light image of the object by the set of projected light patterns. Section and a set of reflected light images captured by the camera section,
A three-dimensional information generation unit that generates three-dimensional information of the subject, the three-dimensional information generation unit being provided corresponding to each of a plurality of regions in a camera image.
And a plurality of look-up tables each describing the correspondence between the light intensity ratio of the set of light patterns and the projection direction.
Have a, in accordance with the pixel position in the camera image, the one of the plurality of look-up table to select respectively, that by referring to the lookup table selected, and generates the three-dimensional information Characteristic range finder device.