JP2020148475A - Ranging sensor - Google Patents

Abstract

To provide a ranging sensor capable of measuring a long distance with high accuracy at a high speed.SOLUTION: A beam deflecting device 2 includes a light emitting structure 20 which emits linear light Li having a linear far-field pattern from a surface of the device and is configured so that a deflection angle θs of the linear light Li is variable. A camera 4 captures an image of an object irradiated with the linear light Li. A processing unit 8 varies the deflection angle θs of the linear light Li to vary an irradiation position of the linear light Li and measures a distance to the object or a three-dimensional shape of the object on the basis of a pixel position of the linear light Li in image data of the camera 4.SELECTED DRAWING: Figure 1

Description

The present invention relates to a distance measuring sensor.

In recent years, the applications of three-dimensional measurement are expanding. Laser radar (LIDAR) mounted on automobiles, drones, robots, etc., 3D sensors mounted on personal computers and smartphones for face recognition, safety monitoring systems, automatic inspection devices at manufacturing sites, etc. are representative of 3D measurement. is there.

The distance measurement method can be broadly divided into a triangulation method and a TOF (Time Of Flight) method. The triangulation method is used for measuring short distances with high accuracy, and the TOF method is used for measuring long distances.

Japanese Unexamined Patent Publication No. 2004-333369 U.S. Pat. No. 8,320,621B2 U.S. Patent Application Publication No. 2014/0211215A1 U.S. Patent Application Publication No. 2016/0025993A1

T. Matsuda, F. Abe, and H. Takahashi, "Laser printer scanning system with a parabolic mirror" Appl. Opt., Vol. 17, no. 6, pp. 878-884, Mar. 1978. P. F. V. Dessel, L. J. Hornbeck, R. E. Meier, and M. R. Douglass, "A MEMS-based projection display," Proc. IEEE, vol. 86, no. 8, pp. 1687-1704, Aug. 1988. K. Nakamura, J. Miyazu, M. Sasaura, and K. Fujiura, "Wide-angle, low-voltage electro-optic beam deflection based onspace-charge-controlled mode of electrical conduction in KTa1xNbxO3," Appl. Phys. Lett. , vol. 89, no. 3, pp. 131115-1-131115-3, Sep. 2006. Y. Kurosaka, S. Iwahashi, Y. Liang, K. Sakai, E. Miyai, W. Kunishi, D. Ohnishi, and S. Noda, "On-chip beam-steering photonic-crystal lasers," Nat. Photon. , vol. 4, no. 7, pp. 447-450, May 2010. J. K. Doylend, et.al., "Two-dimensional free-space beam steering with an optical phased array on silicon-on-insulator," Optics Express, vol. 19, no.22, pp.21595-21604, 2011. X. Gu, T. Shimada, and F. Koyama, "Giant and high-resolution beam steering using slow-light waveguide amplifier," Opt. Exp., vol. 19, no. 23, pp. 22 675-22 683, Nov. 2011. M. Nakahama, X. Gu, T. Shimada, and F. Koyama, "On-Chip high-resolution beam scanner based on Bragg reflector slow-light waveguide amplifier and tunable micro-electro-mechanical system vertical cavity surface emitting laser," Jpn. J. Appl. Phys., Vol. 51, no. 4, pp. 040208-1-040208-3, Mar. 2012.

There are cases where a triangulation method using a monocular camera or a stereo camera is adopted for in-vehicle applications targeting medium and long distances, but both methods involve enormous image processing and are due to the high frame rate. Requires a high-speed processor, which has the problem of high cost.

As a triangulation method, an optical cutting method is also known (Patent Document 1). The light cutting method scans and irradiates an object with line-shaped light to perform triangulation, and can reduce the amount of calculation as compared with the method using a monocular or stereo camera. However, the use of the conventional optical cutting method is limited to a short distance, and it is difficult to measure the distance at a high frame rate. Further, in order to scan the line light, it is necessary to use a movable stage or a galvanometer mirror, which causes a problem that the apparatus becomes complicated.

The present invention has been made in such a situation, and one of the exemplary objects of the embodiment is to provide a triangulation distance measuring sensor for medium and long distances.

One aspect of the present invention relates to a ranging sensor. The ranging sensor can emit line light having a line shape in a far-field image from the device surface, and is irradiated with a beam deflection device including a light emission structure in which the deflection angle θs of the line light is variably configured. By changing the deflection angle θs of the line light and the image sensor that captures the image of the object, the line light is swept, and the distance to the object or the three dimensions of the object is based on the image data of the image sensor. It is provided with a processing unit for measuring the shape.

According to an aspect of the present invention, long-distance distance measurement can be increased in speed or accuracy.

It is a schematic block diagram of the ranging sensor which concerns on this embodiment. It is a figure explaining the schematic operation of the distance measurement sensor shown in FIG. It is a block diagram of the distance measuring sensor which concerns on 1st Embodiment. It is sectional drawing of the light radiation structure which has the VCSEL structure which concerns on embodiment. It is a conceptual diagram of the line light pattern emitted from the light emission structure shown in FIG. It is a figure explaining the line length of a line light. It is sectional drawing of the beam deflection device of a thermal drive system. It is sectional drawing of the beam deflection device of the micromachine drive system. It is a conceptual diagram when the distance measuring sensor of an embodiment is applied to a vehicle. It is explanatory drawing of the distance measuring method of a single camera system. It is a figure which shows the distance measurement distance and the pixel position in an image sensor when the line light is swept from a beam deflection device in a single camera system. It is explanatory drawing for calculating the line peak in one pixel. It is explanatory drawing of the pixel range (deflection angle direction) of an image sensor. It is explanatory drawing of the pixel range (diffraction angle direction) of an image sensor. It is explanatory drawing of the pixel range on an image sensor in a single camera system. It is a flowchart of 3D distance moving image generation. It is explanatory drawing of the distance measurement method of a dual camera system. It is explanatory drawing which calculates the pixel range (deflection angle direction) of the image sensor which synchronizes with a time profile in a dual camera system. It is explanatory drawing of the pixel range on an image sensor in a dual camera system. It is a block diagram of the distance measuring sensor which concerns on 2nd Embodiment. It is explanatory drawing which shows the relationship between noise and line peak position accuracy. It is explanatory drawing of the noise measurement. It is explanatory drawing of the process in a noise subtraction part. It is a figure explaining various mounting examples when the distance measuring sensor of an embodiment is applied to a vehicle.

Hereinafter, embodiments will be described in detail with reference to some drawings. In the following description, components having substantially the same function and configuration are designated by the same reference numerals, and duplicate explanations will be given only when necessary.

FIG. 1 is a schematic block diagram of the distance measuring sensor 1 according to the present embodiment. The distance measuring sensor 1 mainly includes a beam deflection device 2, a camera 4, and a processing unit 8.

The beam deflection device 2 irradiates the measurement object Ob with a line light sweep pattern 200 based on a time profile described later. In FIG. 1, a virtual plane or a reference plane RS is shown as the object to be measured Ob.

The beam deflection device 2 includes a light emission structure 20 that emits line light Li (i = 1, 2, ... N) whose far-field image has a line shape from the device surface, and deflects the line light Li to be emitted. The angle θs is variably configured. Here, θs = 0 in the direction perpendicular to the device surface.

The processing unit 8 sweeps the irradiation position of the line light Li by changing the deflection angle θs of the line light Li. This function is executed by the light sweep control unit 3 of the processing unit 8. FIG. 1 shows an example of a sweep pattern 200 obtained as a result of sweeping the line light. The sweep pattern 200 includes a plurality of line lights L1 to Ln separated in the first direction (X direction in the drawing), and each line light Li (represented by the subscript i) has a second direction (Y direction). It is long. This is defined as the line length. It should be noted that a plurality of line lights L1 to Ln are not simultaneously irradiated to the measurement target, and only one line light Li is irradiated to the measurement target at a certain time.

The light sweep control unit 3 determines the deflection angles θs of the line lights L1 to Ln emitted from the light emission structure 20 based on the time profile for generating the desired line light sweep pattern 200 on the predetermined reference surface RS. Control.

The camera 4 sequentially captures the line lights L1 to Ln projected on the measurement object Ob on the image sensor.

The processing unit 8 measures the distance to the object or the three-dimensional shape of the object based on the image data obtained by the image sensor 4. This function is executed by the distance image generation unit 5 of the processing unit 8.

The distance image generation unit 5 converts the information on the pixel position of the line light Li in the X direction included in the image data captured by the camera 4 into the distance information from the beam deflection device 2 to the irradiation spot of the line light Li. The light sweep control unit 3 and the distance image generation unit 5 operate in synchronization with the frequency of the frame rate corresponding to the moving image, and the distance image generating unit 5 can output a three-dimensional distance moving image corresponding to the moving image. The processing unit 8 may be one piece of hardware such as a CPU (Central Processing Unit) or an FPGA (Field Programmable Gate Array). Alternatively, the light sweep control unit 3 and the distance image generation unit 5 may be implemented as separate hardware.

FIG. 2 is a diagram illustrating a schematic operation of the distance measuring sensor shown in FIG. In the ranging sensor of this embodiment, a desired line light sweep pattern 200 is formed based on the time profile.

The light sweep control unit 3 first sets the ranges θmin and θmax of the deflection angles θs to be measured. The number of line lights n to be swept is set within the range of this deflection angle. The number of line lights n is the resolution of the finally obtained three-dimensional distance image. FIG. 2 shows an example in which the light sweep control unit 3 continuously sweeps the deflection angles θs (θ1 to θn) of the line light emitted from the beam deflection device 2 with time t.

For example, when the deflection angle θs becomes θ1 at time t1, the intensity Ip of the line light L1 is changed with time in a pulsed manner. By repeating this n times, the sweep pattern 200 of the line light of FIG. 1 can be obtained. Here, the times t1 to tn are defined as the emission timing ts of the line light. Further, although FIG. 2 shows an example in which the deflection angle θs is continuously swept at a constant inclination, the sweep of the deflection angle θs may be temporarily stopped during the light emission period of each line light Ln. The accuracy of the deflection angle θs can be improved by stopping the sweep of the deflection angle θs.

The above is the operation of the distance measuring sensor 1. Next, the advantages will be described. According to the distance measuring sensor 1, three-dimensional measurement at medium and long distances is possible with a significantly smaller amount of calculation than a triangulation method using a monocular camera or a stereo camera. In addition, since a movable stage or a galvanometer mirror is not required for scanning line light, the device can be simplified and the cost can be reduced.

The present invention extends to various devices and circuits grasped as the block diagram of FIG. 1 or derived from the above description, and is not limited to a specific configuration. Hereinafter, more specific examples and modifications will be described not for narrowing the scope of the present invention but for helping to understand the essence and operation of the invention and clarifying them.

(First Embodiment)
FIG. 3 is a block diagram of the distance measuring sensor according to the first embodiment. Similar to FIG. 1, the ranging sensor 1 emits a beam emitted from a beam deflection device 2, a light sweep control unit 3 for controlling a beam emitted from the beam deflection device 2, and a beam deflection device 2, and projects a beam projected on an object to be measured. Various types of cameras including a camera 4 that detects imaging on a two-dimensional pixel, a distance image generation unit 5 that determines a three-dimensional distance of a measurement object from an imaging signal of the camera 4 and generates a three-dimensional distance image, and a time profile. It has a memory 6 for storing control data. Further, it includes a synchronization unit 7 that synchronizes and controls various parameters related to the time profile.

The beam deflection device 2 has a light emission structure 20 in which a far-field image emits line-shaped line light directly from the device surface, and a light sweep structure that sweeps line light in directions having different deflection angles (θ1 to θn in the figure). Has 21. Further, the light sweep control unit 3 sequentially sweeps the line light emitted directly from the surface of the device at a predetermined deflection angle in time, and a predetermined number of lines are provided at a predetermined line interval on the reference surface RS separated by a distance Zs. A line light sweep pattern 200 comprising the above is generated.

The light sweep control unit 3 includes a time profile setting unit 31 for setting a sweep pattern 200 of line light projected on the reference surface RS, and a wavelength setting unit 32 for setting a control wavelength according to the line light sweep pattern 200. It also has a drive unit 33 that drives the beam deflection device 2.

The camera 4 has a lens 41 having a focal length f and an image sensor 42 that captures a sweep pattern of line light projected onto a measurement object. The image sensor 42 is preferably a global shutter type image sensor, but a rolling shutter type image sensor may be used as long as the operating speed is high.

The distance image generation unit 5 selects the pixel range of the image sensor 42 in synchronization with the time profile. The line light line based on the pixel position and pixel value of the line light imaged by the pixel range selection unit 501 and the image sensor 42. Set the peak position calculation unit 502 that calculates the peak with the accuracy of the sub-pixel, the distance measurement unit 503 that measures the distance to the measurement target using the line peak with respect to the deflection angle θs of the line light, and the frame rate of image generation. It has a frame rate setting unit 504.

The distance image generation unit 5 triangulates the distance (three-dimensional position) to the measurement object based on the line distortion of the line light captured by the camera 4, the emission timing ts of the line light, and the deflection angle θs. Is measured using. A three-dimensional distance image is generated by obtaining the three-dimensional distance of the object to be measured for all the line lights L1 to Ln forming the line light sweep pattern 200.

The memory 6 has a time profile for controlling the beam emitted from the beam deflection device 2, control data for synchronization, calibration data for three-dimensional distance measurement, and a three-dimensional distance image of the measured object to be measured. Save various data such as data.

The synchronization unit 7 synchronizes the light sweep control unit 3 and the distance image generation unit 5 with various data of the memory 6 so that the three-dimensional distance image data can be acquired. For example, one frame of an image is formed by a time profile that sweeps within a predetermined deflection angle range with a line light having a predetermined resolution, and this time profile is repeated in synchronization with the frame rate of a predetermined moving image to move. A three-dimensional distance image of the object to be measured is generated. In the figure, the parameter fr is the frame rate, fs is the sweep frequency of the line light, ts is the emission timing of the line light, and θs is the deflection angle of the line light. These parameters are parameters for illustration purposes, and the parameters used in the synchronization unit 7 are not limited to these.

The distance image generation unit 5 outputs the acquired three-dimensional distance image in a predetermined format such as a still image or a moving image. Then, it is output to an AI (Artificial Intelligence) processing unit or a monitor (not shown) that analyzes the feature points of the three-dimensional distance image data. Further, a three-dimensional distance image may be superimposed on the visible image and displayed.

FIG. 4 is a cross-sectional view of the light radiation structure 20 according to the embodiment. The light radiation structure 20 is a semiconductor device having a VCSEL (Vertical Cavity Surface Emitting Laser) structure 22 integrated on the semiconductor substrate 23, and the VCSEL structure 22 is a lower DBR (lower DBR) formed on the semiconductor substrate 23. A Distributed Bragg Reflector) 24, an active layer 25, and an upper DBR26 are provided, and slow light light multiple reflected by the upper DBR26 and the lower DBR24 is propagated. The VCSEL structure 22 has a unique oscillation wavelength λ2 determined by the resonance length in the vertical direction (Z direction).

Since the light radiation structure 20 propagates the slow light light while amplifying it in the active layer 25, the waveguide length is lengthened to about 2 mm to 10 mm. The drive unit 33 injects a current larger than the oscillation threshold current into the optical radiation structure 20 and oscillates at a wavelength λ2 determined by the resonator length of the VCSEL structure 22. In this state, when coherent incident light Li is input from the incident port 27 provided on one end side of the light radiation structure 20, the incident light Li propagates while being amplified as slow light light that is multiple reflected in a substantially vertical direction. The emission light Lo having a radiation angle θs is emitted from the emission port 28 formed on the upper surface of the light emission structure 20, and the far-field image of the emission light Lo has a line shape.

Here, the light sweep structure 21 can use a seed light source 21s that generates coherent incident light Li, and has a tunable laser diode (TLD) structure. The seed light source 21s may be an end-face emission type TLD, but a structure in which the seed light source 21s is integrated with the light radiation structure 20 is more effective for miniaturization.

The light radiation structure 20 emits emitted light Lo at a radiation angle θs corresponding to the wavelength λ1 of the seed light Li. Equation (1) holds when the multiple reflection angle of the slow light light in the light radiation structure 20 is θi and the emission angle of the emitted light Lo is θs.

sinθs = nsinθi = n√ (1- (λ1 / λc) ² )… (1)
n is the refractive index of the waveguide of the light radiation structure 20, and λc is the cutoff wavelength of the waveguide, which is equal to λ2.

Since the light emission structure 20 is injected with a current larger than the threshold current, the oscillating state at the single oscillation wavelength λ2 is maintained when the incident light Li is not incident, but the incident light Li at the wavelength λ1 is present. When input, the incident light Li propagates while being amplified as slow light light that is multiple reflected, so that the component of the single oscillation wavelength λ2 becomes very small.

At this time, the emitted light Lo has a beam spreading angle Δθdiv that is extremely narrow in the radiation angle θs direction because it becomes coherent light with a uniform wave surface. The beam spread angle Δθdiv is given by the equation (2) using the aperture width WL in the longitudinal direction of the exit port 28 of the light radiation structure 20.

Δθdiv≈λ1 / (WL · cosθs)… (2)
That is, the longer the opening width WL of the exit port 28, the narrower the beam spread angle Δθdiv, and at the same time, higher output is achieved.

FIG. 5 is a conceptual diagram of a sweep pattern of line light emitted from the light emission structure 20 of FIG. The sweep pattern of the line light generated in the present embodiment will be described with reference to FIG. When seed light of wavelength λ1 is input from the left side of the light emission structure 20, slow light light that is multiple-reflected in the light emission structure 20 propagates, and the beam spread angle is very narrow in the deflection angle θs direction according to equation (1). The light beam is emitted from the exit port 28. Now, consider a reference surface RS parallel to the plane on which the light radiation structure 20 is arranged and the distance from the light radiation structure 20 is Zs, and on this reference surface RS, for example, n lines at equal intervals (n is a natural number). Consider forming a line light sweep pattern consisting of the line light of. The light beam emitted from the exit port 28 has a line shape on the reference surface RS, and the deflection angle θs changes by sweeping the wavelength λ1 of the seed light in time (t1 to tun), and the reference surface RS A multi-line line light sweep pattern (L1 to Ln) is formed on the top. As shown in the equation (2), the width of the line light emitted from the light emitting structure 20 can be narrowed by increasing the length of the light emitting structure 20, and the light output can be increased. The light intensity can be increased. This makes it possible to increase the S / N ratio in a bright environment.

The maximum deflection angle θs_max is determined by the sweep range of the wavelength λ1 of the seed light. Specifically, when the wavelength λ1 of the seed light is swept from 990 nm to 950 nm near the cutoff wavelength by about 40 nm, the deflection angle becomes about 10 ° to 75 °, and the maximum deflection angle θs_max exceeds 60 °. Has been done. The number of resolution points at this time is a high resolution exceeding 1000.

Next, the line length of each line on the reference surface RS will be described. FIG. 6 is a diagram for explaining the line length of the line light. For example, the line length with respect to the line light L1 radiating in the substantially vertical direction (that is, θs = 0) in FIG. 5 is largely determined by the lateral opening width WT of the exit port 28. Assuming that the electric field of the emitted light is uniform with the aperture width, the far-field image at a distance becomes Fraunhofer diffraction and the intensity distribution of the square of the SINC function. If the line length at which the electric field strength 40 is 0 is defined at the line end, the line length LT is determined by the diffraction angle θd, and the equations (3a) and (3b) hold.
LT = 2Zs · tan (θd / 2)… (3a)
θd≈λ / WT ... (3b)
The line lengths LT1 and LT2 can be obtained corresponding to the distance Zs (Zs1, Zs2 in the figure) from the light radiation structure 20. That is, it is necessary to set an appropriate opening width WT according to the size of the object to be measured and the distance Zs to the object to be measured to obtain a desired line length LT.

FIG. 7 is a cross-sectional view of a heat-driven beam deflection device according to an embodiment. In this beam deflection device 2, the seed light source 21s and the light radiation structure 20 are monolithically integrated. Here, the light sweep structure 21 includes a seed light source 21s, a light radiation structure 20, heaters 51 and 52 for heating them, and a heat drive unit 53 for temperature control.

In this integrated beam deflection device, a seed light source 21s having a VCSEL structure 22s and a light emitting structure 20 having a VCSEL structure 22 are formed on the same semiconductor substrate.

The VCSEL structure 22s has the same structure as the VCSEL structure 22 of the light radiation structure 20, and includes a lower DBR 24s, an active layer 25, and an upper DBR 26s formed on the semiconductor substrate 23, and the seed light source 21s and the light radiation structure 20 include. It is in a photobonded state in which the active layer 25 is common via the bonding surface 54. Further, it is preferable that the seed light source 21s and the optical radiation structure 20 have a configuration in which the oscillation wavelengths are slightly different from each other and the resonator length is changed so that λ1 <λ2. Further, in consideration of the reflectance, the number of DBR layers of the upper DBR 26s is designed to have high reflection unlike that of the light radiation structure 20.

The seed light source 21s is oscillated in the vertical direction at the wavelength λ1 by injecting a current equal to or larger than the threshold current from the drive unit 33 from the drive electrode 55. A part of the light intensity distribution 56 in this oscillation mode exudes as seed light 56s of the light radiation structure 20.

On the other hand, a current larger than the threshold current is also injected from the drive unit 33 into the light radiation structure 20 to bring it into an oscillating state at the wavelength λ2. At this time, a part of the light intensity distribution 56 of the seed light source 21s becomes the seed light input Li of the light radiation structure 20 and photocouples, and the slow light light having a wavelength λ1 propagates in the longitudinal direction of the light radiation structure while being amplified. .. Then, the emitted light Lo is emitted from the exit port 28 formed on the surface of the light emitting structure 20.

The control of the thermal drive unit 53 will be described. In FIG. 7, for the sake of explanation, the heater 51 for heating the seed light source 21s and the heater 52 for heating the light radiation structure 20 are shown under the substrate 23, but are arranged on the upper surface of the beam deflection device 2 from the viewpoint of heating efficiency. Further, due to thermal isolation, a structure capable of heat separation such as a void 57 is formed in the vicinity of the bonding surface 54. By adjusting the temperature of each of the heaters 51 and 52, the refractive index changes, and the oscillation wavelengths λ1 and λ2 change accordingly, so that the deflection angle θs of the emitted light Lo can be swept.

As can be seen from the equation (1), it is possible to change the deflection angle of the emitted light Lo by changing the cutoff wavelength λc of the light radiation structure 20. In the light radiation structure 20 having the VCSEL structure 22, the cutoff wavelength λc is equal to the single oscillation wavelength λ2 of the light radiation structure 20, so the equation (1) is rewritten as the equation (4).

sinθr = n · sinθi = n√ (1- (λ1 / λ2) ² )… (4)
That is, efficient deflection control can be performed by controlling the heaters 51 and 52 so that the changes of the single oscillation wavelengths λ1 and λ2 of the seed light source 21s and the light radiation structure 20 are opposite to each other. Of course, the temperature of either the seed light source 21s or the light radiation structure 20 may be changed.

FIG. 8 is a cross-sectional view of a beam deflection device with a micromachine structure. The difference from FIG. 7 is that the seed light source 21s and the upper DBRs 26s and 26 of the light radiation structure 20 are provided with air gap layers 58s and 58, respectively, and the thickness of the air gap layers 58s can be changed by the MEMS (Micro Electro Mechanical Systems) structure. It is a point that is composed. By changing the thickness of the air gap layer 58s, the position of the high-reflection mirror composed of the upper DBR26s can be controlled, whereby the cavity length of the seed light source 21s can be changed and the oscillation wavelength λ1 can be changed. Further, a MEMS driving unit 60 for controlling the MEMS structure is provided. As a result, the light sweep structure 21 has a seed light source 21s having a MEMS structure and a MEMS driving unit 60. In FIG. 8, the drive unit 33 described in FIG. 7 is omitted.

The MEMS driving unit 60 changes the layer thickness of the air gap layer 58 of the seed light source 21s by changing the voltage applied to the MEMS control electrode 59, and controls to sweep the oscillation wavelength λ1.

A MEMS structure that changes the air gap layer 58 may also be formed on the light radiation structure 20 side. In that case, it is efficient to control the single oscillation wavelengths λ1 and λ2 so that the changes are opposite to each other, as in the case of thermal control.

Using the beam deflection device 2 configured as described above, the light sweep control unit 3 sweeps the line light in time, and sweeps the line light based on the time when the line light is transmitted and the deflection angle of the line light. Generate a pattern. As a result, it is possible to generate various line light sweep patterns composed of a predetermined number of lines at predetermined line intervals.

(Single camera method)
FIG. 9 is a conceptual diagram when the distance measuring sensor of the present embodiment is applied to a vehicle. Further, FIG. 10 is an explanatory diagram of a single camera type distance measuring method. A single camera type distance measuring method will be described with reference to FIGS. 9 and 10 by taking as an example a case where the distance measuring sensor 1 of the present embodiment is mounted on the upper center of the windshield of the vehicle 90.

In FIG. 9, it is assumed that the beam deflection device 2 projects line light Ln at a deflection angle θs with respect to a cube-shaped measurement object Ob located in front of the vehicle 90. At this time, the camera 4 having the focal length f and the viewing angle θr captures the line light projected on the measurement object Ob on the image sensor 42. Assuming that the distance of the object to be measured Ob to the front surface of the cube is Zs1 and the distance to the rear surface of the cube is Zs2, images are taken on the image sensor 42 at pixel positions x1 and x2 having different thicknesses (Zs2-Zs1) of the object to be measured Ob. Will be done. That is, the difference in distance to the object to be measured is observed as line distortion on the image sensor 42.

In FIG. 10, the distance from the beam deflection device 2 to the object to be measured Ob is Zs, the deflection angle of the line light Ln is θs, the distance between the beam deflection device 2 and the center of the image sensor 42 is D, and the focal length of the lens 41 is. f. Assuming that the distance in the X direction from the center of the image sensor 42 that captures the line light of the object to be measured Ob is p · x (p: pixel size, x: number of pixels), the beam deflection device is based on the principle of triangular measurement. The distance Zs from 2 to the object to be measured Ob can be obtained.
Zs = D ・ f / (f ・ tanθs + p ・ x)… (5)

FIG. 11 is a diagram showing the distance measurement distance and the pixel position on the image sensor when the deflection angle of the line light is swept from −30 ° to 30 ° from the beam deflection device. Here, the pixel position x represents the number of pixels x obtained by dividing the distance p · x from the center of the image sensor 42 by the pixel size p. The parameters used in the calculation are D = 100 mm, f = 10 mm, and pixel size 10 μm. At this time, the deflection angle control on the transmitting side controls 0.05 ° per line of light, and the resolution is 1200.

Since the pixel position x on the image sensor 42 that captures the line light is an integer value, the maximum ranging distance Zs_max is given by Eq. (6), and the ranging error ΔZs is given by Eq. (7).

Zs_max = D ・ f / p… (6)
ΔZs = Zs ² · p / (D · f + Zs · p)… (7)
For example, using the above parameters, the maximum distance measurement distance Zs_max is 100 m, and the distance measurement error ΔZs is 50 m. As described above, since the stereo camera system utilizes parallax, it has a large ranging error ΔZs at a ranging distance Zs of about 100 m.

In order to solve this, it is effective to reduce the pixel size, but in the present embodiment, the peak position calculation unit 502 calculates and estimates the line peak imaging position within one pixel of the line light. As a result, a line peak with subpixel accuracy can be obtained.

FIG. 12 is an explanatory diagram for calculating a line peak within one pixel. The intensity of the line light Ln imaged on the image sensor 42 is output as a pixel value in which the amount of charge received by the photodetector in each pixel is digitized by a predetermined number of bits. Further, the resolution of the line light is determined by the pixel size in the X direction. First, the pixel x having the maximum pixel value Sx obtained by imaging the line light Ln is obtained, and the pixels before and after the pixel x are considered. In the example of FIG. 12, two front and rear pixels are considered with respect to the center pixel x.

As a method of calculating the line peak in one pixel, for example, the following method can be considered.
(1) Template matching method.
The intensity profile of the line light Ln in the X direction is assumed to be a Gaussian distribution, and this is used as a template. Then, an intensity profile that matches each pixel value (Qx-1 to Qx + 1) is obtained. In the example of FIG. 12, the intensity profile 120a (solid line) having a deviation σw similar to the pixel size p is matched and the line peak Xp is obtained.

(2) Equal-angle linear interpolation method Three pixels of the center pixel x and the pixels x-1 and x + 1 on both sides are used. First, of the slope of the straight line connecting the center pixel x and the pixel x-1 and the slope of the straight line connecting the center pixel x and the pixel x + 1, the straight line 120b having the larger slope is adopted. Next, the slope of the straight line 120b is reversed, and the straight line 120c passing through the pixel x-1 on the opposite side is obtained. Then, the line peak Xp is obtained from the position where the two straight lines 120b and 120c intersect. Alternatively, the line peak Xp may be obtained by fitting these three pixels with a quadratic curve.

(3) Center of gravity method The line peak Xp is obtained by taking the weighted average shown in the equation (8) for the central pixel x and its peripheral pixels.
Xp = Σx ・ Qx / ΣQx… (8)

(4) Gaussian filter method Natural smoothing is performed by weighting the Gaussian distribution (convolution processing) on the pixel values of the central pixel x and its peripheral pixels, and the line peak Xp is obtained.

(5) Laplacian filter method Both edges of the line light are obtained by weighting (convolution processing) the Laplacian with respect to the pixel values of the center pixel x and its peripheral pixels, and the center is obtained as the line peak Xp. In this method, the width σw of the line light can also be estimated from the widths of both edges. From the width σw of the line light, it is possible to estimate the minimum number of peripheral pixels to be selected in the convolution process.

These methods are appropriately selected in consideration of the processing speed and position accuracy required by the system. Of course, a method other than the above method may be used. When the width of the intensity profile of the line light Ln is smaller than the pixel size p as in the intensity profile 120d indicated by the one-point chain line, the pixel value of the adjacent pixel of the central pixel x may be close to 0 and may be buried in noise. In such a case, since the above-mentioned line peak cannot be estimated accurately, it is necessary to set the width σw and the pixel size p of the line light intensity profile so that an accurate pixel value can be detected with at least 3 pixels. There is.

Next, the operation of the pixel range selection unit 501 will be described with reference to FIG. The parameters used in the calculation are the same as in FIG. When the line light is swept from −30 ° to 30 ° in deflection angle θs in synchronization with the time profile, the ranging curve 130 moves in the direction of the arrow marked Scan. From this, it can be seen that it is not necessary to expose all the pixels on the image sensor 42 or read out the pixel values of all the pixels. For example, assuming that the ranging range is 1 m to 100 m, the required horizontal pixel range ROI_h is determined for each ranging curve 130. In the case of FIG. 13, ROI_h is about 100 pixels. Since the pixel range on the image sensor 42 is set based on the deflection angle θs, the pixel range is set appropriately even if the object to be measured does not exist in the distance measurement range (even if there is no reflected light of the line light). can do.

Further, the pixel range ROI_v in the vertical direction will be described with reference to FIG. As described with reference to FIG. 6, the line length LT is determined by the diffraction angle θd. The pixel range ROI_v on the image sensor 42 is determined by the diffraction angle θd and the focal length f of the lens, and is given by the equation (9).
ROI_v = 2f · tan (θd / 2)… (9)
If the camera 4 has a zoom mechanism capable of changing the focal length f, ROI_v is changed by changing the focal length f.

FIG. 15 is an explanatory diagram of a pixel range on the image sensor 42. The pixel range 150 is set by the ROI_h described with reference to FIG. 13 and the ROI_v described with reference to FIG. 14, and this pixel range is moved in synchronization with the time profile (sweep of line light). The straight line 151 indicates the pixel position corresponding to the maximum distance measurement value, and the pixel corresponding to the distance measurement range (for example, 1 m to 100 m) is set to be included in the pixel range 150.

This pixel range can be the exposure range of the pixels of the image sensor 42 and the reading range of the pixel values. Further, the exposure range of the pixel and the read range of the pixel value may be set differently. Limiting the exposure range of the pixels of the image sensor 42 brings about stabilization of the shutter speed and sensitivity setting, and at the same time, it is possible to reduce the influence of ambient light, reflected light, ambient light, and the like. Further, by limiting the reading range of the pixel value, the distance measuring process can be speeded up. Variations of the pixel range setting method will be described later. In the embodiment where the influence of sunlight is small such as at night, the exposure range of the pixels may not be set, but only the reading range of the pixel values may be set.

For the pixel value of the line light captured in the pixel range selected by the pixel range selection unit 501, the line peak of the line light is calculated by the peak position calculation unit 502, and the distance measurement unit 503 measures based on this line peak. Calculate the distance to the object.

FIG. 16 is a flowchart for generating a three-dimensional distance moving image. First, in step ST1, the frame rate fr of the three-dimensional distance moving image to be acquired is set by the frame rate setting unit 504. For example, 30 fps (frames per second).

In step ST2, the time profile setting unit 31 reads the time profile corresponding to the reciprocal (1 / fr) of the set frame rate fr from the memory 6 and sweeps the line light resolution number n for generating one frame. The range of the deflection angle of the line light is set.

Subsequently, in step ST3, the sweep frequency fs for repeatedly sweeping the line light, the emission timing ts (t1 to tun) for emitting the line light within the repeating cycle of the sweep frequency fs, and the deflection angle θs (θ1 to θ1 θn) is set. Then, in the wavelength setting unit 32, the control wavelengths λ1 and λ2 corresponding to the deflection angles θs are set, and the projection of the line light is started.

In step ST4, the pixel range selection unit 501 sets the pixel range on the image sensor 42 based on the deflection angle θs. The line light that projects the object to be measured within the distance measurement range within this pixel range is imaged. The shutter is opened for exposure for this pixel range, and the pixel value of this pixel range is read out.

In step ST5, the peak position calculation unit 502 obtains the maximum value in the horizontal direction (row direction) among the read pixel values, and uses the adjacent peripheral pixel values to obtain the line peak in one pixel.

In step ST6, the distance measuring unit 503 calculates the distance Zs to the measurement target using the deflection angle θs and the line peak. This completes the distance measurement for one line of light.

Further, in step ST7, all the line lights having the resolution n are swept, and it is determined whether the distance measurement process is completed. If all the line lights of resolution n are swept (ST7: Yes), a one-frame image is generated, and if all the line lights of resolution n are not swept (ST7: No), the process returns to step ST3 and the next step is performed. Line light is projected by setting ts and θs.

By the above steps, a one-frame three-dimensional distance image is generated, and by repeating this one-frame generation operation, a three-dimensional distance moving image is generated.

(Dual camera method)
FIG. 17 is an explanatory diagram of a distance measuring method of the dual camera system. Cameras 4R and 4L were prepared on the left and right of the beam deflection device 2. The imaging distances of the image sensors 42R and 42L are expressed as pr and pl. For the sake of explanation, the distance D and the focal length f between the beam deflection device 2 and the left and right cameras 4R and 4L are the same, but different distances D and focal length f may be set.

In the dual camera system, the distance Zs from the beam deflection device 2 to the object to be measured Ob is calculated by the equation (10).
Zs = D · f / (pr-pl)… (10)
Since the deflection angle θs is not used in the equation (10), accurate distance measurement can be performed even when the control accuracy of the deflection angle θs cannot be obtained.

FIG. 18 is an explanatory diagram for calculating the pixel range (deflection angle direction) of the image sensor selected in the dual camera system. The parameters used in the calculation are the same as those in FIGS. 11 and 13. The ranging curves 130R and 130L for the image sensors 42R and 42L are displayed at the same time. In the example of FIG. 18, the same pixel range ROI_h is set for the image sensors 42R and 42L. Control can be standardized by setting the same pixel range. ROI_v is the same as the single camera system.

FIG. 19 shows a variation of the pixel range setting on the image sensors 42R and 42L. FIG. 19A shows a case where the pixel range is selected centering on the straight line 151 indicating the pixel having the maximum distance measurement value, as shown in FIG. FIG. 19B shows a case where different pixel ranges 150R and 150L are set for the image sensors 42R and 42L. FIG. 19C shows a case where a relatively wide pixel range of 150 W is specified, the pixel range of 150 W is fixed for a certain period of time, and the line light is swept.

As described above, according to the first embodiment, since the method of sweeping the line light is adopted, it is possible to increase the intensity of the laser light that projects the light to be measured. Therefore, it is possible to project light on a long-distance measurement object. Moreover, since the S / N ratio can be increased even in a bright environment, stable distance measurement can be performed.

The stereo camera method has a drawback that the distance measurement accuracy over a long distance is not good because it uses parallax, but in the present embodiment, the line peak within one pixel can be obtained in synchronization with the sweep of the line light. Therefore, it is possible to obtain sufficient accuracy even in long-distance distance measurement. Further, by setting the exposure and readout pixel ranges of the image sensor in synchronization with the sweeping of the line light, the speeding up of the distance measuring process is achieved.

The beam deflection device having a VCSEL structure is small and capable of high resolution beam sweeping. Further, since the beam sweep does not require an optical component such as a lens, a high-speed and compact distance measuring sensor can be realized. Since it can be made much smaller than the conventional one, when the distance measuring sensor of this embodiment is applied to an in-vehicle vehicle, it can be easily applied to the front and rear of the vehicle (for example, near the license plate) and the side surface (for example, near the door and door mirror). It is possible to install it in the vehicle, and it is possible to improve the reliability of automatic operation.

(Second Embodiment)
FIG. 20 is a block diagram of the distance measuring sensor according to the second embodiment. In addition to the first embodiment, the distance image generation unit 5 includes a noise measurement unit 505, a noise removal unit 506, and a distance accuracy determination unit 507.

The noise measuring unit 505 measures the noise on the image sensor 42. In particular, noise caused by ambient light such as sunlight and headlights of oncoming vehicles greatly affects the distance measurement accuracy. Of course, the camera 4 may be equipped with a filter that cuts light of these wavelengths.

FIG. 21 is an explanatory diagram showing the relationship between noise and the position accuracy of the line peak. Consider matching a Gaussian intensity profile for 5 pixels of the pixel x having the largest pixel value and its adjacent peripheral pixels. At this time, if random noise is added in which each pixel value has a deviation σn and the pixel value receives only the line light and deviates from the pixel value, the position of the line peak also deviates accordingly. The deviation at this time is σx.

Assuming that the deviation σx of the line peak is caused by noise, the deviation σZs of the distance measurement due to this is obtained by differentiating the equation (5) into the equation (11) in the case of the single camera method.
σZs = Zs ²・ p ・ σx / (D ・ f)… (11)
This equation (11) means that if σx is close to one pixel, the distance measurement accuracy cannot be obtained over a long distance of about 100 m, as in the discussion of the equation (7). Therefore, the noise measuring unit 505 of the present embodiment employs a method as shown in FIG. 22 for measuring noise.

FIG. 22 is an explanatory diagram of noise measurement. Since the noise measurement of the image sensor 42 is performed in synchronization with the time profile, it is displayed superimposed on the emission timing of the line light of FIG. The image sensor 42 exposes and reads the pixel value within the time RTs synchronized with the emission timing ts (t1 to tun) of the line light. As a result, the pixel value for the line light including the background light noise can be read out. Further, following this, the exposure and the pixel value are read at the time RTn synchronized with the non-emission timing of the line light. As a result, the pixel value of only the background light noise can be read out.

FIG. 23 is an explanatory diagram of processing in the noise removing unit. The noise removing unit 506 removes the background light noise by subtracting the pixel value read out at the time RTn synchronized with the non-emission timing of the line light from the pixel value read out at the time RTs synchronized with the light emission timing ts of the line light. Find the pixel value. This noise removal processing is performed on the pixel range selected by the deflection angle θs. Furthermore, by performing this noise removal processing only on the pixels necessary for obtaining the line peak, the distance measurement processing can be further speeded up.

The pixel value from which noise has been removed is sent to the peak position calculation unit 502 to calculate a line peak. As a result, background light noise such as sunlight, ambient light, and reflected light can be removed, and a highly accurate line peak can be calculated.

However, when the intensity of the background light is strong, the distance measurement accuracy may not be improved if the signal component of the line light is small even if the noise is removed. The distance accuracy determination unit 507 acquires an SNR (signal-to-noise ratio) from the noise removal unit 506. Since the distance accuracy is inversely proportional to the SNR, the distance accuracy is determined and output to the outside. If the SNR is not greater than or equal to a predetermined value, an alarm may be issued.

As described above, according to the second embodiment, in addition to the effect of the first embodiment, background light noise caused by sunlight, ambient light such as headlights of an oncoming vehicle, and reflected light is removed. It is possible to measure the distance with high accuracy.

FIG. 24 shows various mounting examples when the distance measuring sensor of the embodiment is applied to a vehicle. FIG. 24A is a mounting example shown in the first embodiment, and is suitable for recognizing an obstacle in front of the vehicle. FIG. 24B is an example in which the distance measuring sensors 1 are mounted on the front and rear (for example, near the license plate) and left and right (for example, near the door or the door mirror) of the vehicle. In such a mounting example, it is suitable for applications such as automatic parking support. Since the three-dimensional device of this embodiment does not include a rotating mechanical component or a large optical component, it can be realized in a size that can be mounted on a smartphone. Therefore, it can be freely mounted anywhere in the vehicle.

FIG. 24C shows an example in which the beam deflection device 2 and the camera 4 are mounted independently. As the distance D between the beam deflection device 2 and the camera 4 is increased, the distance measurement accuracy at a long distance can be improved. Further, since the light sweep control unit 3 and the distance image generation unit 5 can be separated and arranged in the cockpit or the like, they may be linked with the car navigation function or synchronized with the camera image of a drive recorder or the like.

Further, although the present embodiment has been described for in-vehicle use as an example, it goes without saying that the present embodiment is not limited to this. It can be used for various purposes such as 3D shape measurement and face recognition.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. The present embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and the equivalent scope thereof.

1 Distance measurement sensor 2 Beam deflection device 3 Light sweep control unit 4 Camera 5 Distance image generation unit 6 Memory 7 Synchronization unit 8 Processing unit 20 Light radiation structure 21 Light sweep structure 21s Seed light source 22 VCSEL structure 31 Time profile setting unit 32 Wavelength setting Unit 33 Drive unit 501 Pixel range selection unit 502 Peak position calculation unit 503 Distance measurement unit 504 Frame rate setting unit 505 Noise measurement unit 506 Noise removal unit 507 Distance accuracy determination unit

Claims (10)

A beam deflection device including a light emission structure capable of emitting line light having a line shape in a far-field image from the device surface and having a variable deflection angle θs of the line light.
An image sensor that captures an object irradiated with the line light and
By changing the deflection angle θs of the line light, the irradiation position of the line light is changed, and the distance to the object or the object is based on the pixel position of the line light in the image data of the image sensor. A processing unit that measures the three-dimensional shape of
A distance measuring sensor characterized by being provided with. The ranging sensor according to claim 1, wherein the light emitting structure has a VCSEL (Vertical Cavity Surface Emitting Laser) structure in which slow light light propagates. The image sensor is one,
The distance measuring sensor according to claim 1 or 2, wherein the distance to the object is calculated based on the deflection angle θs and the pixel position of the line light in the image data. There are two image sensors.
The distance measurement according to claim 1 or 2, wherein the distance to the object is calculated based on the pixel position of the line light in each of the two image data generated by the two image sensors. Sensor. One frame of an image is formed by a time profile that sweeps within a predetermined deflection angle range with the line light, and this time profile is repeated in synchronization with the frame rate of a predetermined moving image image to obtain a three-dimensional distance moving image of an object to be measured. The distance measuring sensor according to any one of claims 1 to 4, wherein the distance measuring sensor is generated. The processing unit assumes that the beam profile of the line light has a Gaussian distribution, and by matching the pixel position and the pixel value of the line light imaged by the image sensor, the peak position of the line light is determined by the accuracy of the subpixel. The distance measuring sensor according to any one of claims 1 to 5, wherein the distance measuring sensor is calculated according to the above. The processing unit obtains a pixel having the maximum pixel value of the line light imaged by the image sensor, and uses a method of equiangular linear interpolation method or a center of gravity method for the pixel values of peripheral pixels adjacent to the pixel. The distance measuring sensor according to any one of claims 1 to 5, wherein the peak position of the line light is calculated with a subpixel accuracy. The distance measuring sensor according to any one of claims 1 to 7, further comprising a pixel range selection unit that selects a pixel range of the image sensor based on the distance measuring range of the line light. Claims 1 to 8 further include a noise removing unit that subtracts the pixel value read out in synchronization with the non-emission timing of the line light from the pixel value read out in synchronization with the emission timing of the line light. The ranging sensor according to any one of. Claims 1 to 9 further include a noise measuring unit that measures the SN ratio of the line light imaged by the image sensor, and a distance accuracy determining unit that determines the distance accuracy based on the SN ratio. The ranging sensor described in either.

Images