KR20230099402A - 3d image acquisition device - Google Patents

Abstract

The present invention relates to a three-dimensional image acquisition device, comprising: a body housing on one side of which a light-transmitting window is disposed for emitting pulsed laser light to the outside and transmitting light entering from the outside; a control board mounted on an inner upper end of the main body housing to control timing of light transmission and reception, and to transmit detected signals to an external device; and a light transmitting/receiving unit coupled to the control board to transmit laser light and receive incident external light, wherein the light transmitting/receiving unit is irradiated from a laser irradiation module for irradiating laser light and then reflected back to a target. a beam splitter that transmits and reflects light; a first TOF receiving sensor generating a first structural image by receiving light of a first wavelength transmitted through the beam splitter; and a second TOF receiving sensor configured to generate a second structural image by receiving light of a second wavelength reflected by the beam splitter.

Description

3D image acquisition device {3D IMAGE ACQUISITION DEVICE}

The present invention relates to a 3D image acquisition device, and more particularly, to a 3D image acquisition device capable of acquiring images of surrounding terrain and a target such as a subject and measuring a spatial structure thereby.

As a conventional 3D image acquisition device, the passive stereo vision using two cameras to obtain distance information with the corresponding pixel of the target object and the active stereo vision replacing one camera of the stereo vision with a projector are applied. there is. However, passive stereo vision has the advantage of high speed, but has limitations in obtaining accurate 3D images because it is severely affected by monotonous environments or ambient lighting. In addition, active stereo vision using visible light-based structural light Vision provides a somewhat improved 3D image, but due to the pattern of visible light, when used in a home robot or the like, it may be annoying to the human eye.

In recent years, as a TOF (Time Of Flight) method, the distance or depth between the camera and the recognition object is determined by using the temporal difference between the light emission time by irradiating light to the recognition object and the light receiving time reflected from the recognition object. A method of measuring and obtaining a 3D image from this is provided.

In addition, as a conventional technique for 3D image acquisition, an infrared flash type active 3D distance image measuring device is known from Korean Patent Publication No. 10-2005-0026949 (Mar. 16, 2005). Accordingly, the above technology relates to a 3D distance image measurement device using a DMD (Digital Micromirror Device) element to perform real-time distance measurement using a flash-type infrared light source and various light irradiation patterns. It is possible to measure the spatial structure by combining the color image image with the TOF receiving method that receives the laser light of the received pulse pattern using the laser irradiation module and the beam splitter that irradiate the laser light and measures the phase difference according to the time timing. There is a difference with the image acquisition device that exists.

The present invention was derived to solve the above-mentioned problems of the prior art, and the 3D image acquisition device according to the present invention receives light having different wavelengths reflected from a target through a plurality of TOF receiving sensors, 2 Acquire depth information on the structural image, perform an algorithm process to fuse the two images, generate a high-resolution stereoscopic image, and precisely measure the 3D spatial structure of the target through the combined image through the algorithm want to do it

In addition, the 3D image acquisition device of the present invention measures distance and depth information for a subject using a TOF method, and further includes an image sensor capable of detecting a color image to increase the resolution of information, and to detect surrounding terrain and data with a simple operation. It is intended to be able to measure a precise 3D spatial structure of a target such as a subject or provide a 2D image.

In addition, the 3D image acquisition device of the present invention is provided with a front lens on the optical path, but even as one single lens, it can perform the function of an optical lens for receiving light, and has a certain form for controlling the incident light. It is intended to provide an optical lens system as a lens assembly in which lens elements are combined in a complex manner so as to enable improvement in optical performance and miniaturization of products on an optical path.

A three-dimensional image acquisition device according to an embodiment of the present invention for solving the above technical problem is a main body housing on one side of which a light-transmitting window is disposed for emitting pulsed laser light to the outside and transmitting light entering from the outside; a control board mounted on an inner upper end of the main housing to control timing of light transmission and reception, and to transmit detected signals to an external device; and a light transmission/reception unit coupled to the control board to emit laser and receive incident external light, wherein the light transmission/reception unit is irradiated by a laser irradiation module that emits laser light and then reflected back to a target. a beam splitter that transmits and reflects; a first TOF receiving sensor generating a first structural image by receiving light of a first wavelength transmitted through the beam splitter; and a second TOF receiving sensor configured to generate a second structural image by receiving light of a second wavelength reflected by the beam splitter.

According to another embodiment of the present invention, the laser irradiation module may irradiate light of a plurality of wavelengths, and the first TOF receiving sensor and the second TOF receiving sensor may each receive light of a plurality of wavelengths.

According to another embodiment of the present invention, the first TOF receiving sensor may receive wavelengths in a first range, and the second TOF receiving sensor may receive wavelengths in a second range.

According to another embodiment of the present invention, the first TOF receiving sensor and the second TOF receiving sensor may be composed of an avalanche photodiode (APD), a single-photon avalanche detector (SPAD), or a silicon photomultiplier (SiPM). .

According to another embodiment of the present invention, the control board combines the first structure image of the target received from the first TOF receiving sensor and the second structural image of the target received from the second TOF receiving sensor to obtain a stereoscopic image. It may be configured to include a 3D image processing module that generates.

According to another embodiment of the present invention, the laser irradiation module is modularized in the form of a plurality of dot arrays and transmits a laser pulse pattern toward a target by plane emission, or a laser irradiating laser light toward a target. It can be configured as a diode module.

According to another embodiment of the present invention, the optical lens system may be formed of a certain type of lens assembly to adjust incident light for each lens element.

A 3D image acquisition apparatus according to the present invention obtains depth information on first and second structural images by receiving light having different wavelengths reflected from a target through a plurality of TOF receiving sensors, and an algorithm for fusing the two images. A stereoscopic image with high resolution can be generated by performing the process, and a 3D spatial structure of the target can be precisely measured through the image combined through the algorithm.

In addition, the 3D image acquisition device of the present invention measures distance and depth information for a subject using a TOF method, and further includes an image sensor capable of detecting a color image to increase the resolution of information, and to detect surrounding terrain and data with a simple operation. There is an effect of measuring a precise 3D spatial structure of a target such as a subject or providing a 2D image.

In addition, the 3D image acquisition device of the present invention includes a front lens on the optical path, but even a single lens can perform the function of an optical lens for receiving light, and a certain type of lens for controlling incident light. By providing an optical lens system as a lens assembly in which elements are combined in a complex manner, there is an effect of enabling improvement in optical performance and miniaturization of a product on an optical path.

In addition, the 3D image acquisition device of the present invention can be applied not only to improve the performance of LIDAR devices of vehicles in general, but also to mobile devices that can move such as robots, ships, helicopters, drones, and buildings. It has the advantage that it can be applied without limitation to fixed devices with limited movement such as columns, towers, etc.

In addition, information data such as the spatial structure of a target and surrounding topography detected by the 3D image acquisition device of the present invention can be transmitted to an external device, so that the data can be reprocessed over a network.

1 is an external perspective view showing a body housing of a 3D image acquisition device according to an embodiment of the present invention.
FIG. 2 is an exemplary diagram showing the internal structure of the 3D image acquisition device of FIG. 1 .
3 is an exemplary view showing the structure of an optical transmission/reception unit according to an embodiment of the present invention.
FIG. 4 is an exemplary view showing a plate-type beam splitter in the structure of the optical transmission/reception unit of FIG. 3 according to another embodiment of the present invention.
5 is an exemplary diagram for explaining the entire structure and path of an optical transmission/reception unit in an optical transmission/reception unit according to an embodiment of the present invention.
6 is a block diagram showing the configuration of a 3D image acquisition device according to an embodiment of the present invention.
7 is a diagram illustrating surface light emission of a 3D image acquisition device having a quadrangular box shape according to an embodiment of the present invention.
8 is a diagram illustrating surface light emission of a cylindrical 3D image acquisition device according to another embodiment of the present invention.
9 is a diagram for explaining an optical transmission/reception unit of a 3D image acquisition device according to another embodiment of the present invention.

The terms or words used in this specification and claims should not be construed as being limited to ordinary or dictionary meanings, and the inventors may appropriately define the concept of terms in order to explain their invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention based on the principle that there is.

Therefore, since the embodiments described in this specification and the configurations shown in the drawings are only one of the most preferred embodiments of the present invention and do not represent all of the technical ideas of the present invention, various equivalents that can replace them at the time of the present application It should be understood that there may be waters and variations.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is an external perspective view showing a body housing of a 3D image acquisition device according to an embodiment of the present invention.

As shown in FIG. 1, the 3D image acquisition device of the present invention emits pulsed laser light to the outside from a body housing 100 having a sealed inner space and one side of the body housing 100, and transmits light coming from the outside. Facilitating windows 110 are arranged. The window 110 may be formed of a light-transmitting member for facilitating light transmission into and out of the 3D image capture device and protecting the body housing 100 .

Accordingly, the body housing 100 of the present invention may be formed in the form of a rectangular box of a certain height as shown in FIG. 1, but is not limited thereto, and may also be applied to a housing formed in a cylindrical shape as shown in FIG. 8.

In addition, FIG. 2 is an exemplary view showing the internal structure of the 3D image acquisition device of FIG. 1, FIG. 3 is an exemplary view showing the structure of an optical transmission and reception unit according to an embodiment of the present invention, and FIG. 4 is an exemplary view of the present invention. It is an exemplary view showing a plate-type beam splitter in the structure of the optical transmission and reception unit of FIG. 3 according to another embodiment, and FIG. 5 is a description of the overall structure and path of the optical transmission and reception unit in the optical transmission and reception unit according to an embodiment of the present invention. , and FIG. 6 is a block diagram showing the configuration of a 3D image acquisition device according to an embodiment of the present invention.

Referring to FIGS. 2 to 6, the three-dimensional image acquisition device of the present invention includes a control board 300 coupled and mounted at the bottom of the inside of the main housing 100, transmission of light from the top of the control board 300, and An optical transmission/reception unit 200 for reception is disposed.

At this time, the light transmitting/receiving unit 200 includes a front lens 210, a laser irradiation module 220, a beam splitter 240, a first TOF receiving sensor 230, and a second TOF receiving sensor 250. .

In addition, the optical transmission and reception unit 200 is made to be protected in a form wrapped by the unit case 201, whereby the front end of the unit case 201 is located in the direction of the window 110 of the main housing 100 A front lens 210 is positioned, and at the rear end of the front lens 210, a beam splitter 240 is provided to separate and reflect light incident from the outside by wavelength, and the beam splitter 240 At the rear and lower ends in the horizontal direction, first and second TOF receiving sensors 230 and 250 respectively detecting light of a wavelength range are provided.

The beam splitter 240 of the present invention is formed in the form of a triangular prism as shown in FIG. 3 to form an incident surface and an exit surface, or as shown in FIG. The surface and the exit surface can be implemented.

At this time, the unit case 201 of the present invention forms a space for assembling and mounting certain components of the optical transmission and reception unit 200, and is a protective case in which certain components are assembled and coupled to the inside, and the light emitted from the inside and It has a structure that is sealed so that light incident from the outside does not leak out, and the front lens 210, the laser irradiation module 220, the beam splitter 240, the first TOF receiving sensor 230 and the second TOF receiving sensor (250) is made to be protected in a form wrapped by the unit case (201).

Meanwhile, according to an embodiment of the present invention, the laser irradiation module 220 may be configured as a VCSEL module or a Laser Diode (LD) module.

The big cell module 220 is a module in which a Vertical-Cavity Surface Emitting Laser (VCSEL) is modularized in the form of a plurality of dot arrays to transmit a fast pulse pattern laser as surface light emission. The laser diode module is a module that irradiates laser light toward a target.

The laser irradiation module 220 configured as described above is positioned by being mounted on the top of the inside of the unit case 201 at a certain distance from the rear end of the front lens 210 . At this time, the laser irradiation module 220 emits near-infrared wavelengths such as 905 nm, 940 nm, and 1550 nm.

In addition, as shown in FIGS. 2 to 5, the laser irradiation module 220 emits surface-emitted laser light in a pulse pattern to the outside through a portion of the front lens 210 at the front end. Accordingly, the laser irradiation module ( 220) may be provided as a filter lens 211 for dissipating the light emitted from the laser irradiation module 220 to the outside. At this time, the filter lens 211 may be a lens functioning as a band pass filter for passing only wavelengths used in the laser light source of the laser irradiation module 220 itself.

That is, the front lens 210 provided in the unit case 201 performs the function of an optical lens for receiving light as a whole, but at a certain portion located at the front end of the laser irradiation module 220 for emitting light. The filter lens 211 is inserted at a predetermined position of the front lens 210, or is made to have a configuration formed by being additionally mounted on the outside.

Hereinafter, with reference to FIG. 5 , the overall structure and path of the optical transmission/reception unit in the optical transmission/reception unit according to an embodiment of the present invention will be described in detail.

As the light emitted from the laser irradiation module 220 passes through the filter lens 211 and is irradiated to the target, light reflected from the target and incident is separately separated for each wavelength by the beam splitter 240 .

Therefore, in the first TOF receiving sensor 230, a first structure image including depth information is formed by receiving light of a first wavelength by a large amount of projected pulsed laser, and similarly, the second TOF receiving sensor 250 In , a second structure image including depth information is formed by receiving light of a second wavelength by a large amount of projected pulsed laser.

That is, since the beam splitter 240 is configured to transmit and reflect the incident light reflected from the target into two wavelengths, the first TOF receiving sensor 230 is transmitted through the beam splitter 240. A first structure image is generated by receiving light of a first wavelength, and the second TOF receiving sensor 250 receives light of a second wavelength reflected from the beam splitter 240 to generate a second structure image. can In this case, the first wavelength may be 940 nm and the second wavelength may be 905 nm, or one of the first wavelength and the second wavelength may be 940 nm or 905 nm, and the other may be 1550 nm may consist of

Through this, according to the present invention, it is possible to construct a three-dimensional image with higher resolution by combining the first and second structural images.

Meanwhile, the first TOF receiving sensor 230 and the second TOF receiving sensor 250 are configured to receive a plurality of lights of each wavelength, or the first TOF receiving sensor 230 transmits a wavelength within a first range. and the first TOF receiving sensor 250 may be configured to receive wavelengths in a second range.

In addition, the first TOF receiving sensor 230 and the second TOF receiving sensor 250 configured as described above may be composed of an avalanche photodiode (APD), a single-photon avalanche detector (SPAD), or a silicon photomultiplier (SiPM). there is.

On the other hand, when the laser irradiation module 220 is composed of a big cell module, a modulation 221 is provided to flicker at very fast intervals when shooting light, where the laser of the laser irradiation module 220 While it is turned on is called in phase, and while the laser is off is called out phase, each TOF receiving sensor (230, 250) consists of two receptor pairs, each cell is reflected light is executed to be exactly synchronized with the emitted light pulse in synchronization with the flickering interval of the modulation 221, and can be configured to be measured according to time timing.

In other words, general infrared (IR) sensors measure only signal strength and are easily affected by the reflectance of objects, but TOF receivers receive lasers from the laser irradiation module and calculate the distance based on the time-of-flight method, resulting in a more precise value. can be obtained.

The TOF receiving sensors 230 and 250 in the present invention transmit resolution data of a certain resolution (for example, VGA 640x480) at a certain pixel pitch (for example, 15 μm) and a certain frame rate (for example, up to 150 frames per second). can provide

In addition, if the signals from the TOF receiving sensors 230 and 250 are analog signals, they are converted into digital signals by the A/D converter (ADC, Analog to Digital Converter) 223 and transmitted to the 3D image processing module 320 It can be.

As shown in FIG. 6, according to an embodiment of the present invention, the control board 300 includes a first structure image of a target received by the first TOF receiving sensor 230 and the second TOF receiving sensor 250. A high-resolution stereoscopic image may be generated by combining the second structural images of the target received from the target.

More specifically, the control board 300 includes a control module 310 and a 3D image processing module 320 .

The control module 310 controls light emission for applying a driving signal for fast blinking of the pulsed laser of the laser irradiation module 220, and the first TOF receiving sensor 230 and the second TOF receiving sensor ( 250) can be controlled.

In addition, the 3D image processing module 320 interlocks the first TOF receiving sensor 230 and the second TOF receiving sensor 250 to obtain first and second structural image data for a target such as a surrounding terrain and a subject. Received and combined, it is possible to construct a stereoscopic image of a 3D space for the target.

That is, the 3D image processing module 320 receives image data of a target and performs a pre-processing process of removing noise, and first and second structure images received from the first and second TOF receiving sensors 230 and 250. It is possible to generate a stereoscopic image with high resolution by calculating depth information for and performing an algorithm process of fusing the two images. In this way, the 3D spatial structure of the target can be precisely measured through the images combined through the algorithm.

At this time, the 3D image processing module 320 may generate 2D image data using the result of measuring the 3D image or the 3D spatial structure configured as described above, thereby providing more diverse image processing and image information. this is possible

To this end, the control board 300 may be implemented with at least one device selected from a logic circuit, a programming logic controller, a microcomputer, a microprocessor, and the like, and may be implemented as a Field-Programmable Logic Array (FPGA) board encompassing them.

In addition, the control board 300 may include a communication module 330 for a communication connection means for wireless WiFi or wired LAN (Local Area Network) connection, or may be configured integrally. The communication module 330 communicates with an external user PC or a user mobile device such as a smartphone through an intranet, the Internet, a vehicle network, etc., and transmits information data about a target such as a detected surrounding terrain and a subject to the user device. can be sent to This is to provide convenience in which data can be converted into digital images and reprocessed in a user computing device interlocked with an external network such as the Internet.

On the other hand, the control board 300 may include a wire, adapter, or power supply means for supplying power to the 3D image acquisition device of the present invention, and the power supply means may be provided as an internal power source or a rechargeable power supply device. there is.

7 is a view illustrating surface light emission of a rectangular box-shaped 3D image acquiring device according to an embodiment of the present invention, and FIG. 8 is a surface of a cylindrical 3D image acquiring device according to another embodiment of the present invention. It is a drawing illustrating light emission.

As shown in FIGS. 7 and 8, according to the present invention, the light emitted from the laser irradiation module 220 inside the body housings 100 and 100a is a filter lens that is held at one end of the front lens 210 ( 211) and emitted from a specific portion of the body window 110, 110b to the outside, and can be irradiated with surface light emission.

9 is a diagram for explaining an optical transmission/reception unit of a 3D image acquisition device according to another embodiment of the present invention.

The light transmission/reception unit of the 3D image acquisition device according to the embodiment of FIG. 9 may include a beam splitter 240 , a TOF reception sensor 230 and an image sensor 253 .

The beam splitter 240 horizontally transmits infrared rays emitted from a laser irradiation module that irradiates laser light and then reflected and returned to a target, and vertically reflects incident visible light.

The TOF receiving sensor 230 receives the infrared laser reflected from the beam splitter 240 to generate a structural image in the infrared region.

In addition, the image sensor 253 controls the focus by moving the camera lens 251 composed of a plurality of lens elements to receive visible light passing through the beam splitter 240 and moving the camera lens forward and backward in the direction of the optical axis. A color image of the target may be generated by the configuration of the lens driving unit 252 .

That is, in the embodiment of FIG. 9, the camera lens 251 composed of a plurality of lens elements on the side of the beam splitter 240 and the camera lens 251 composed of a plurality of lens elements in order to receive light in the visible light wavelength range transmitted by the beam splitter 240 to the horizontal side. The lens driver 252 enabling focus adjustment by moving the camera lens 251 forward and backward in the direction of the optical axis and detecting the RGB light signal of the visible light wavelength band incident to the camera lens 251 from the side of the lens driver 252 A camera module made of an image sensor 253 for the image sensor 253 may be disposed.

At this time, the image sensor 253 may be a CMOS (Complementary Metal-Oxide-Semiconductor) type RGB Image sensor supporting a certain frame (eg 30 fps) and a certain resolution (eg 1280 x 720).

In addition, in another embodiment, without the configuration of the camera lens 251 and the lens driving unit 252, only the image sensor 253 is positioned on the side of the beam splitter 240 to obtain a color image of the target. You may.

As described above, the 3D image acquisition device of the present invention may be formed by combining components of various embodiments into one, or by selectively using each component according to preferred embodiments. That is, those skilled in the art will understand that the present invention can be variously modified and changed without departing from the spirit and scope described in the claims below.

Claims (7)

a body housing on one side of which a light-transmitting window is disposed for emitting pulsed laser light to the outside and transmitting light coming from the outside;
a control board mounted on an inner upper end of the main body housing to control timing of light transmission and reception, and to transmit detected signals to an external device; and
A light transmission/reception unit coupled to the control board for emitting a laser and receiving incident external light;
The optical transmission and reception unit,
a beam splitter that transmits and reflects the light emitted from the laser irradiation module that irradiates the laser light and then reflected back to the target;
a first TOF receiving sensor generating a first structural image by receiving light of a first wavelength transmitted through the beam splitter; and
and a second TOF receiving sensor configured to generate a second structural image by receiving light of a second wavelength reflected by the beam splitter. The method of claim 1,
The laser irradiation module irradiates light of a plurality of wavelengths,
The first TOF receiving sensor and the second TOF receiving sensor each receive a plurality of wavelengths of light. The method of claim 1,
The first TOF receiving sensor,
Receiving wavelengths in a first range;
The second TOF receiving sensor,
A 3D image acquisition device receiving wavelengths in the second range. The method of claim 1,
The first TOF receiving sensor and the second TOF receiving sensor,
A 3D image acquisition device consisting of an avalanche photodiode (APD), Single-Photon Avalanche Detector (SPAD) or Silicon Photomultiplier (SiPM). The method of claim 1,
The control board,
3D image acquisition including a 3D image processing module generating a stereoscopic image by combining the first structural image of the target received from the first TOF receiving sensor and the second structural image of the target received from the second TOF receiving sensor Device. The method of claim 1,
The laser irradiation module,
A big cell module that is modularized in the form of a plurality of dot arrays and transmits a laser of a pulse pattern to the target side by surface emission;
A 3D image acquisition device composed of a laser diode module that irradiates laser light toward a target. The method of claim 1,
The optical lens system,
A three-dimensional image acquisition device comprising a lens assembly of a certain shape so as to adjust incident light for each lens element.

Images