WO2020130440A1

WO2020130440A1 - Three-dimensional distance measuring camera

Info

Publication number: WO2020130440A1
Application number: PCT/KR2019/017173
Authority: WO
Inventors: 손현성
Original assignee: ㈜미래컴퍼니
Priority date: 2018-12-20
Filing date: 2019-12-06
Publication date: 2020-06-25
Also published as: KR102165091B1; KR20200077208A

Abstract

The present invention relates to a three-dimensional distance measuring camera comprising: a light source for emitting light at an object; an image sensor for recognizing light reflected from the object; and a control IC for controlling operations of the light source and the image sensor, wherein the control IC performs control so that the image sensor acquires at least one piece of image information in a first image acquisition section without emission from the light source, and performs control so that the exposure time of the image sensor in a second image acquisition section after the first image acquisition section is different from the exposure time in the first image acquisition section. According to the present invention, a distance error caused by ambient light such as sunlight can be minimized in a photographing process of a camera using a light source, such as a three-dimensional distance measuring camera.

Description

3D distance measuring camera

The present invention relates to a three-dimensional distance measuring camera using a light source. More specifically, the present invention relates to an active ambient light noise reduction method of a 3D distance measuring camera using a light source capable of minimizing the distance error due to ambient light such as sunlight.

A 3D distance measuring camera is a camera that irradiates light onto an object and senses the reflected light signal to calculate the distance. Due to its characteristics, signals caused by unsynchronized ambient light (noise), such as sunlight, in addition to the light source irradiated on the subject, There is a problem that an error occurs in the distance calculation by reaching the sensor surface.

As an example, in the case of a robot cleaner equipped with a 3D distance measuring camera, the distance from an object in front is sensed using a light source in the camera module to perform cleaning without collision with an object, but various external Since the cleaning is performed under the condition, an incorrect distance calculation may occur due to the ambient light noise, which may cause a problem that cannot accurately guide the route.

As one of the prior arts for eliminating such optical noise, a technique for reducing optical noise using a light source having a wavelength range of 940 nm, which is difficult to find in a real life environment, is known.

However, as is generally known, since sunlight is distributed over a wide range of wavelengths including infrared rays and ultraviolet rays, as well as visible rays, errors due to ambient light such as sunlight may occur even when a light source of a specific wavelength band is used. There is a problem that can not be prevented.

In particular, these problems are greatly strengthened in an outdoor environment in which sunlight is strong, and thus serve as a major factor deteriorating the quality of a 3D distance measuring camera using a light source.

[Advanced technical literature]

Republic of Korea Patent Publication No. 10-2016-0054156 (published date: May 16, 2016, name: distance measuring device)

Republic of Korea Patent Publication No. 10-2017-0051752 (published date: May 12, 2017, name: TOF camera control method)

An object of the present invention is to provide a 3D distance measuring camera using a light source capable of minimizing the distance error due to ambient light such as sunlight.

The present invention for solving this technical problem is a three-dimensional distance measuring camera including a light source for irradiating light on a subject, an image sensor for recognizing light reflected from the subject, and a control IC for controlling the operation of the light source and the image sensor. In the above, the control IC controls the image sensor to acquire at least one image information without irradiation of the light source in the first image acquisition section, and in the second image acquisition section after the first image acquisition section, the image sensor The exposure time is controlled to be different from the exposure time in the first image acquisition section.

In the three-dimensional distance measuring camera according to the present invention, the control IC senses noise information from image information acquired by the image sensor without irradiating the light source.

In the three-dimensional distance measuring camera according to the present invention, when the control IC has noise in the image information obtained in the first image acquisition section, the exposure time of the image sensor in the second image acquisition section is the first. 1 Control so that it is shorter than the exposure time in the image acquisition section.

In the three-dimensional distance measuring camera according to the present invention, when the control IC has no noise in the image information obtained in the first image acquisition section, the exposure time of the image sensor in the second image acquisition section is the first. 1 Control so that it is longer than the exposure time in the image acquisition section.

In the three-dimensional distance measuring camera according to the present invention, the control IC generates at least one frame signal for light source irradiation in the image acquisition section and allows the image sensor to synchronize with the frame signal.

In the 3D distance measuring camera according to the present invention, between the first image acquisition section and the second image acquisition section, at least one of noise information, image information, or distance measurement information sensed in the first image acquisition section is external. It further includes a section output to the device.

In the three-dimensional distance measuring camera according to the present invention, when the control IC detects noise even in the second image acquisition section, the exposure time of the image sensor in the subsequent image acquisition section gradually increases until no noise is detected. Control it to be short.

In the three-dimensional distance measuring camera according to the present invention, when the control IC detects noise even in the second image acquisition section, the exposure time of the image sensor in the subsequent image acquisition section is gradually increased until noise is detected. Control it to be long.

According to the present invention, there is an effect of providing a 3D distance measuring camera using a light source capable of minimizing the distance error due to ambient light such as sunlight.

In addition, the image obtained to calculate the distance information, that is, an image capable of inferring the intensity of ambient light, including sunlight, etc., in addition to the image for distance measurement, that is, an image for noise determination is additionally acquired and analyzed to perform sunlight According to the intensity of the ambient light including the light, it is possible to actively respond to various environments by varying the exposure time.

In addition, in environments where there is no ambient light including sunlight, the distance can be increased by increasing the exposure time, and in environments where there is a lot of ambient light including sunlight, the exposure time is shortened to reduce the ambient light including sunlight. There is an effect that can minimize the distance error caused by.

1A is an exploded perspective view of a 3D distance measuring camera according to an embodiment of the present invention,

1B is a view showing a cross section of a 3D distance measuring camera according to an embodiment of the present invention,

2 is a view showing an active ambient light noise reduction method of a 3D distance measuring camera according to an embodiment of the present invention,

3 is, in one embodiment of the present invention, ambient light noise is present through an image for distance measurement obtained when the light source is turned on and a noise determination image obtained when the light source is turned off. It is a drawing showing the case,

4 is, in one embodiment of the present invention, the ambient light noise is present through the image for distance measurement and the noise determination image obtained when the light source is turned off (OFF). It is a drawing showing a case where it is not

5 is a view showing the relationship between the size and exposure time of a noise component included in an image for noise determination obtained in a state in which the light source is turned off in an embodiment of the present invention.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in the present specification are exemplified for the purpose of explaining the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention are in various forms It can be implemented with and is not limited to the embodiments described herein.

Embodiments according to the concept of the present invention can be applied to various changes and can have various forms, so the embodiments will be illustrated in the drawings and described in detail herein. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosure forms, and includes all changes, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various components, but the components should not be limited by the terms. The above terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of rights according to the concept of the present invention, the first component may be referred to as the second component and similarly the second component The component may also be referred to as the first component.

When a component is said to be "connected" to or "connected" to another component, it should be understood that other components may be directly connected or connected to the other component, but may exist in the middle. will be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle. Other expressions describing the relationship between the components, such as "between" and "immediately between" or "adjacent to" and "directly neighboring to," should be interpreted similarly.

The terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “include” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described herein exists, one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, actions, components, parts or combinations thereof are not excluded in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Terms, such as those defined in the commonly used dictionary, should be interpreted as having meanings consistent with meanings in the context of related technologies, and are not to be interpreted as ideal or excessively formal meanings unless explicitly defined herein. .

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1A is an exploded perspective view of a 3D distance measuring camera according to an embodiment of the present invention, and FIG. 1B is a view showing a cross section of a 3D distance measuring camera according to an embodiment of the present invention

Referring to FIG. 1A, the camera module includes a light source board 1 on which components such as a light source 8 are mounted, a heat sink 2 emitting heat generated from the light source board 1, and a lens module 5 Lens base 6, a control board 7 having a control IC 4 for overall control of the operation of the camera module and other coupling members 3-1, 3-2, 3-3, 3-4 ).

Referring to FIG. 1B, an image sensor 100 for condensing external light reflected from a subject may be provided under the lens module 5, and the image sensor 100 includes a CCD (Charge Coupled Device), which is an electronic coupling element. ) Or Complementary Metal Oxide Semiconductor (CMOS), which is a complementary metal oxide semiconductor, may be applied, but is not limited thereto.

2 shows a signal applied to each device to implement an active ambient light noise reduction method of a camera using a light source according to an embodiment of the present invention.

Since the light source 8 for generating light in the camera module and the image sensor 100 for condensing light reflected from the subject are synchronized with each other, when the light source is generated (On), the image sensor 100 is also exposed (On) mode Works. By synchronizing the two pulse signals, light generated from the light source 8 is reflected to the subject and is incident on the image sensor 100, thereby securing an image of the subject. The number of subject image information required for distance measurement with a subject may vary depending on the type of various products on which the camera module is mounted. In the present invention, as an example, a total of 5 frame signals are used to subject images. The case of securing information and noise information will be described as an example. After the image acquisition section that secures the subject image information and noise information, the section that transmits the calculated distance information and the measured noise information from the acquired image information and image information to the final product equipped with the camera module (noise determination and distance calculation/ Video output section), it is possible to implement active noise reduction technology.

Hereinafter, an active noise reduction technique according to the present invention will be described in more detail with reference to FIGS. 1A, 1B, and 2.

When the external power signal is applied (On), the control IC 4 applies a driving pulse as shown in FIG. 2 to the light source 8 and the image sensor 100, respectively. As described above, each driving pulse is synchronized, and whenever one driving pulse is applied, light by the light source 8 is irradiated to the subject, and the irradiated light is reflected from the subject and is reflected in the image sensor 100 in the camera module again. ). In addition, one image frame information for the incident subject is input to the control IC 4 through the image sensor 100. The driving pulse is repeatedly applied on/off, and sequentially, from the first to the fourth image frames, the light source is applied (On) and the image information reflected by the applied light source is reflected through the image sensor (100). (4). After securing the subject image information using the light source, the control IC 4 does not apply a driving pulse to the light source (Off) and exposes only the image sensor 100 (On) to dummy frame information of external noise light. ). When the first image acquisition section, which obtains the image information of the subject and the external noise light through the driving pulse, ends, the control IC 4 outputs the secured image information to the final product along with the calculated distance information, and at the same time, the presence or absence of external noise light. The section for determining (first noise determination and distance calculation/image output section) is executed. At this time, if the control IC 4 senses external noise light, then the control IC 4 drives the pulse so that the width of the driving pulse in the second image acquisition section is narrower than the width of the driving pulse in the first image acquisition section. Is applied. If the external noise light is also detected in the second image acquisition section, then the width of the driving pulse applied in the third image acquisition section is applied to be narrower. By repeating this process, the control IC 4 can finally obtain the object image information without external noise light, thereby calculating more accurate distance information to the object and transmitting it to the final product.

After the control IC 4 controls the width of the driving pulse to be narrow to obtain the subject image information without external noise light, it is gradually controlled so that the width of the driving pulse is wide again to make the intensity of the subject exposed by the light source stronger. Through this, it is possible to reduce the distance measurement error through more accurate image information reflected from the subject.

Controlling the width of the above-described driving pulse means that digitally controlling the time when the power is applied to the light source 8 and the time when the image sensor 100 is exposed (On).

Through the above-mentioned method, it is possible to provide the final product with image information that has actively reduced noise depending on the presence or intensity of external noise light. It is possible to improve the performance of the.

3 and 4 show an image image actually measured when the active driving pulse according to FIG. 2 is applied to each device in one embodiment of the present invention.

3 and 4, (a) the video image corresponds to the video image according to the driving pulse of the first frame of FIG. 2, (b) the video image is the second frame, (c) the video image is the third frame , (d) The image image corresponds to the fourth frame, and (e) the image image corresponds to the image image according to the driving pulse of the dummy frame.

In FIG. 3, the image images from (a) to (d) show image image information received by the image sensor 100 when the light source 8 is sequentially turned on. And (e) the image image is obtained by exposing (On) only the image sensor 100 in a state in which the light source 8 is turned off, and in this case, it is determined that there is noise due to external light in the image (e).

Referring to the case of FIG. 3, a method of solving with the active noise removal method according to FIG. 2, since noise due to external light or the like was detected in the first image acquisition section, the control IC 4 is then driven in the second image acquisition section. The width of the pulse is reduced and the exposure time of the image sensor 100 is reduced. If noise is also detected in the dummy frame of the second image acquisition section, the width of the driving pulse in the third image acquisition section is further reduced, and as shown in FIG. 4, the image sensor 100 until no more noise is detected. ) Will continuously reduce the exposure (On) time. In addition, if noise is no longer detected in the dummy frame as shown in FIG. 4, the control IC 4 applies the driving pulse width in the subsequent image acquisition section wider than the pulse width in the previous section to expose the image sensor 100. (On) can be longer. By doing this, a stronger light source can be irradiated to the subject, and the distance measurement precision can be improved by sensing a more accurate image of the subject.

As described above, the 3D distance measuring camera according to the present invention adopts a noise reduction method capable of actively responding to ambient noise situations, and an image sensor according to whether the control IC 4 is noise sensed through a dummy frame. By controlling the exposure time of (100) variably and providing noise information, image information, and distance information to a final product such as a robot cleaner, product performance can be improved.

FIG. 5 shows the relationship between the size of the noise component and the exposure time included in the image information obtained only by exposure of the image sensor 100 in a state where the light source 8 is turned off in an embodiment of the present invention. It is a drawing. For the first time, when the exposure time is 2 ms, it is measured that there is noise 525, and then, when the exposure time is reduced to 0.2 ms, which is reduced to 1.8 ms, the size of the measured noise component is reduced to 440 to about 85. Afterwards, it can be seen that the noise level continuously decreases as the exposure time is continuously reduced.

As described in detail above, according to the present invention, there is an effect of providing a 3D distance measuring camera using a light source capable of minimizing a distance error due to ambient light such as sunlight.

[Description of drawing symbols]

1: Light board

2: Heat sink

3-1, 3-2, 3-3, 3-4: Coupling members

4: Control IC

5: Lens module

6: Lens base

7: Control board

8: Light source

100: image sensor

Claims

A light source irradiating light to the subject;

An image sensor that recognizes light reflected from the subject; And

In the three-dimensional distance measuring camera including a control IC for controlling the operation of the light source and the image sensor,

The control IC controls the image sensor to acquire at least one image information without irradiation of the light source in the first image acquisition section, and the exposure time of the image sensor in the second image acquisition section after the first image acquisition section The 3D distance measuring camera is controlled to be different from the exposure time in the first image acquisition section.
According to claim 1,

The control IC senses noise information from image information acquired by the image sensor without irradiating the light source.
According to claim 2,

The control IC controls the exposure time of the image sensor in the second image acquisition section to be shorter than the exposure time in the first image acquisition section when there is noise in the image information acquired in the first image acquisition section. 3D distance measuring camera.
According to claim 2,

When there is no noise in the image information acquired in the first image acquisition section, the control IC controls the exposure time of the image sensor in the second image acquisition section to be longer than the exposure time in the first image acquisition section. 3D distance measuring camera.
According to claim 1,

The control IC generates at least one frame signal for irradiating a light source in the image acquisition section and allows the image sensor to be synchronized with the frame signal, a three-dimensional distance measuring camera.
According to claim 1,

Between the first image acquisition section and the second image acquisition section further comprises a section for outputting at least one of the noise information, image information or distance measurement information sensed in the first image acquisition section to an external device, 3D Distance measuring camera.
According to claim 3,

The control IC continuously controls the exposure time of the image sensor in the image acquisition section to be gradually shortened until noise is not detected when noise is detected even in the second image acquisition section.
According to claim 4,

If the control IC is no noise is detected even in the second image acquisition section, the 3D distance measurement camera continuously controls the exposure time of the image sensor in the subsequent image acquisition section to gradually increase until noise is detected.