KR20230160598A - Camera apparatus - Google Patents

Abstract

A camera device according to an embodiment of the present invention includes an optical output unit that radiates an optical signal of a predetermined pattern consisting of a plurality of dots to an object, an optical input unit that receives an optical signal reflected from the object, and the optical signal that the optical input unit receives. a depth information generator that generates depth information of the object using disparity of an optical signal, wherein the optical output unit includes a first optical system that radiates the optical signal at a first angle of view to the object, and A zoom optical system and an image comprising a second optical system that radiates the optical signal of a second angle of view smaller than the first angle of view to the object, wherein the optical input unit is driven at one of a first magnification and a second magnification higher than the first magnification. It includes a sensor, and when the first optical system is driven, the zoom optical system is driven at the first magnification, and when the second optical system is driven, the zoom optical system is driven at the second magnification.

Description

Camera device{CAMERA APPARATUS}

The present invention relates to a camera device.

3D content is applied in many fields such as education, manufacturing, and autonomous driving as well as games and culture, and depth information (Depth Map) is required to obtain 3D content. Depth information is information representing distance in space, and represents perspective information of another point with respect to one point in a two-dimensional image. Methods for acquiring depth information include structured light methods, stereo camera methods, and TOF (Time of Flight) methods.

Among these, according to the structured light method, a predetermined pattern of IR (Infrared) structured light, which is distinct from surrounding lighting, is irradiated to the object, and the distance is calculated by receiving the light signal reflected from the object and analyzing the distortion.

The method of projecting IR structured light onto an object has relatively high accuracy at close range compared to other methods, but the accuracy decreases significantly as the distance increases, so the drivable distance is limited to a short range.

The technical problem to be achieved by the present invention is to provide a camera device for acquiring depth information.

A camera device according to an embodiment of the present invention includes an optical output unit that radiates an optical signal of a predetermined pattern consisting of a plurality of dots to an object, an optical input unit that receives an optical signal reflected from the object, and the optical signal that the optical input unit receives. a depth information generator that generates depth information of the object using disparity of an optical signal, wherein the optical output unit includes a first optical system that radiates the optical signal at a first angle of view to the object, and A zoom optical system and an image comprising a second optical system that radiates the optical signal of a second angle of view smaller than the first angle of view to the object, wherein the optical input unit is driven at one of a first magnification and a second magnification higher than the first magnification. It includes a sensor, and when the first optical system is driven, the zoom optical system is driven at the first magnification, and when the second optical system is driven, the zoom optical system is driven at the second magnification.

It further includes a control unit that controls the optical output unit, the optical input unit, and the depth information generator, wherein the first optical system includes a first light source, and a second light source that diffuses the light output by the first light source to the first angle of view. 1 includes a diffractive optical element (DOE), and the second optical system may include a second light source and a second diffractive optical element that diffuses the light output by the second light source to the second angle of view. You can.

When the first light source is turned on, the control unit may control the zoom optical system to be driven at the first magnification, and when the second light source is turned on, the control unit may control the zoom optical system to be driven at the second magnification. there is.

The first light source and the second light source output light of the same pattern, and the light output by the first light source is radiated n*n times by the first diffractive optical element and irradiated to the object. 2 The light output by the light source is radiated n*n times by the second diffractive optical element and irradiated to the object, where n may be an integer of 2 or more.

The zoom optical system may include a zoom lens and an actuator that drives the zoom lens to one of the first magnification and the second magnification.

It further includes a control unit that controls the optical output unit, the optical input unit, and the depth information generator, and the control unit can control one of the first optical system and the second optical system to be selectively driven.

The optical output unit irradiates the light signal of the first angle of view to the object if the generated depth information is less than the preset distance, and if the generated depth information exceeds the preset distance, the light signal of the second angle of view is irradiated to the object. An optical signal may be irradiated to the object.

A method of generating depth information in a camera device according to an embodiment of the present invention includes the steps of an optical output unit irradiating an optical signal of a predetermined pattern consisting of a plurality of points to an object, linking the optical output unit and the optical input unit, and An input unit receiving an optical signal reflected from the object, and generating depth information of the object using disparity of the optical signal received by the optical input unit, and irradiating the optical signal. The step includes the optical output unit irradiating one of the optical signal at a first angle of view and the optical signal at a second angle of view smaller than the first angle of view to the object, and linking the optical output unit and the optical input unit. The step of irradiating the optical signal of the first angle of view to the object, the zoom lens of the optical input unit is driven at a first magnification, and the optical output unit transmits the optical signal of the second angle of view to the object. When irradiating an object, the zoom lens of the optical input unit is driven at a second magnification higher than the first magnification.

The step of irradiating the optical signal to the object may include the light output unit selectively irradiating the optical signal of the first angle of view to the object or the optical signal of the second angle of view to the object according to the generated depth information. It may include investigating.

The step of irradiating the optical signal to the object includes, if the generated depth information is less than a preset distance, the optical output unit irradiates the optical signal of the first angle of view to the object, and the generated depth information is the preset distance. When the distance is exceeded, the light output unit may irradiate the light signal of the second angle of view to the object.

According to an embodiment of the present invention, a camera device for acquiring depth information may be provided. In particular, according to an embodiment of the present invention, depth information can be obtained with high accuracy not only at short distances but also at medium or longer distances using the structured light method. Additionally, according to an embodiment of the present invention, the overall power consumption and computation amount of the camera device can be reduced by not using a high-pixel image sensor.

1 is a block diagram of a camera device according to an embodiment of the present invention.
Figure 2 is a schematic cross-sectional view of a camera device according to an embodiment of the present invention.
Figure 3 is an example of an optical signal with a predetermined pattern.
Figure 4 is a diagram explaining the principle of generating depth information using structured light.
Figure 5 is a graph explaining the change in disparity according to the difference between the reference distance and the distance between the object.
Figure 6 shows disparity measurement performance according to the number of pixels of the image sensor.
Figure 7 is a block diagram of a camera device according to an embodiment of the present invention.
Figure 8 is a conceptual diagram of a camera device according to an embodiment of the present invention.
Figure 9 is a block diagram of a zoom optical system according to an embodiment of the present invention.
Figure 10 is a flowchart showing a method for generating depth information in a camera device according to an embodiment of the present invention.
FIG. 11 is a flowchart showing steps in which the control unit links the optical output unit and the optical input unit in the depth information generation method of FIG. 10.
Figure 12 is a flowchart showing a method of controlling a camera device according to an embodiment of the present invention.
Figure 13 shows an example in which disparity measurement performance is improved when the optical input unit includes a zoom lens according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

However, the technical idea of the present invention is not limited to some of the described embodiments, but may be implemented in various different forms, and as long as it is within the scope of the technical idea of the present invention, one or more of the components may be optionally used between the embodiments. It can be used by combining and replacing.

In addition, terms (including technical and scientific terms) used in the embodiments of the present invention, unless explicitly specifically defined and described, are generally understood by those skilled in the art to which the present invention pertains. It can be interpreted as meaning, and the meaning of commonly used terms, such as terms defined in a dictionary, can be interpreted by considering the contextual meaning of the related technology.

Additionally, the terms used in the embodiments of the present invention are for describing the embodiments and are not intended to limit the present invention.

In this specification, the singular may also include the plural unless specifically stated in the phrase, and when described as "at least one (or more than one) of A and B and C", it is combined with A, B, and C. It can contain one or more of all possible combinations.

Additionally, when describing the components of an embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used.

These terms are only used to distinguish the component from other components, and are not limited to the essence, sequence, or order of the component.

And, when a component is described as being 'connected', 'coupled' or 'connected' to another component, the component is not only directly connected, coupled or connected to that other component, but also is connected to that component. It can also include cases where other components are 'connected', 'combined', or 'connected' due to another component between them.

Additionally, when described as being formed or disposed "above" or "below" each component, "above" or "below" refers not only to cases where two components are in direct contact with each other, but also to one This also includes cases where another component described above is formed or placed between two components. In addition, when expressed as "top (above) or bottom (bottom)", it may include not only the upward direction but also the downward direction based on one component.

A camera device according to an embodiment of the present invention may refer to a camera that extracts depth information using a structured light method. Therefore, the camera device can be used interchangeably with a depth information extraction device, a 3D information extraction device, etc.

FIG. 1 is a block diagram of a camera device according to an embodiment of the present invention, and FIG. 2 is a schematic cross-sectional view of a camera device according to an embodiment of the present invention.

1 to 3, the camera device 1 according to an embodiment of the present invention includes an optical output unit 10, an optical input unit 20, a depth information generator 30, and a control unit 40. .

The optical output unit 10 generates an output optical signal and then radiates it to the object. At this time, the optical output unit 10 may output an optical signal of a predetermined pattern. Figure 3 is an example of an optical signal with a predetermined pattern. Referring to FIG. 3, an optical signal with a predetermined pattern may be composed of a plurality of dots and may be referred to as structured light. Here, the predetermined pattern may be a unique pattern and may be generated using a pre-designed algorithm. The optical signal with a predetermined pattern may be an IR (Infrared) optical signal. In this specification, output light refers to light output from the light output unit 10 and incident on an object, and input light is output from the light output unit 10, reaches the object, and then is reflected from the object to the light input unit 20. ) may refer to light input. From the object's perspective, output light can be incident light, and input light can be reflected light.

The light output unit 10 may include a light source 100 and a lens assembly 110.

First, the light source 100 generates light. The light generated by the light source 100 may be infrared light with a wavelength of 770 to 3000 nm, or visible light with a wavelength of 380 to 770 nm. The light source 100 may use a light emitting diode (LED), and may have a plurality of light emitting diodes arranged according to a certain pattern. In addition, the light source 100 may include an organic light emitting diode (OLED) or a laser diode (LD). Alternatively, the light source 100 may be a Vertical Cavity Surface Emitting Laser (VCSEL). VCSEL is one of the laser diodes that converts electrical signals into optical signals, and can output a wavelength of about 800 to 1000 nm, for example, about 850 nm or about 940 nm. One VCSEL may have a plurality of emitters, for example, hundreds of emitters, and a pattern consisting of dots may be output by each emitter. That is, structured light with a predetermined pattern can be output by hundreds of emitters. The light source 100 may repeat blinking (on/off) at regular time intervals, and the regular time interval may be the frequency of the output light signal.

The lens assembly 110 may converge light output from the light source 100 and output the condensed light to the outside. The lens assembly 110 may be disposed on top of the light source 100 and spaced apart from the light source 100 . Here, the upper part of the light source 100 may refer to the side where light is output from the light source 100. The lens assembly 110 may include at least one lens. When the lens assembly 110 includes a plurality of lenses, each lens may be aligned with respect to the central axis to form an optical system. Here, the central axis may be the same as the optical axis of the optical system.

The lens assembly 110 may be accommodated or supported in the housing 120 . According to one embodiment, the housing 120 may be combined with a driving module (not shown), and the lens assembly 110 may be moved in the direction of the optical axis or in a direction perpendicular to the optical axis by the driving module (not shown).

Meanwhile, the optical input unit 20 receives light reflected from an object. To this end, the optical input unit 20 includes a lens assembly 130 that focuses input light reflected from an object, a filter (not shown), and an image sensor 140 that converts the input light passing through the lens assembly 130 into an electrical signal. ) may be included, and the lens assembly 130, filter (not shown), and image sensor 140 may be accommodated or supported in the housing 150. The housing 120 on the optical output unit 10 side and the housing 150 on the optical input unit 20 side are shown as spaced apart from each other, but are not limited thereto, and the housing 120 on the optical output unit 10 side ) and the housing 150 on the optical input unit 20 side may be a single housing.

The optical axis of the lens assembly 130 may be aligned with the optical axis of the image sensor 140. A filter (not shown) is disposed between the lens assembly 130 and the image sensor 140 and can filter light having a predetermined wavelength range. For example, a filter (not shown) may pass light in the wavelength band of the output light output from the optical output unit 10.

The image sensor 140 may receive an input light signal according to the blinking cycle of the light source 100. The image sensor 140 may be configured with a structure in which a plurality of pixels are arranged in a grid shape. The image sensor 140 may be a Complementary Metal Oxide Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

The optical input unit 20 may be arranged side by side with the optical output unit 10. The optical input unit 20 may be placed next to the optical output unit 10. The optical input unit 20 may be arranged in the same direction as the optical output unit 10.

The depth information generator 30 may generate depth information of an object using an input light signal input to the light input unit 20. According to an embodiment of the present invention, the depth information generator 30 may generate depth information of an object using disparity of an optical signal.

The control unit 40 controls the operation of the optical output unit 10, the optical input unit 20, and the depth information generator 30. The depth information generator 30 and the control unit 40 may be implemented in the form of a printed circuit board (PCB) 50 on which the optical output unit 10 and the optical input unit 20 are mounted. That is, the depth information generator 30 or the control unit 40 may be implemented by a circuit pattern or IC chip disposed on the PCB 50. Alternatively, the PCB 50 may be connected to the connector through an FPCB (not shown). Alternatively, the PCB and FPCB may be implemented as RFPCB (Rigid Flexible PCB). Additionally, the depth information generator 30 and the control unit 40 may be implemented in other configurations.

The light source 100 of the optical output unit 10 is disposed on the PCB 50 and may be electrically connected to a circuit pattern on the PCB 50. Alternatively, the control unit 40 may be included in an electronic device in which the camera device 1 according to an embodiment of the present invention is disposed. For example, the control unit 40 may be implemented in the form of an application processor (AP) of an electronic device on which the camera device 1 according to an embodiment of the present invention is mounted.

Figure 4 is a diagram explaining the principle of generating depth information using structured light. As described above, in this specification, structured light refers to an optical signal with a predetermined pattern consisting of a plurality of dots. Referring to FIG. 4, the distance (object distance, h') between the camera device 1 and the object may vary depending on the disparity (Δx) of the points forming the structured light. In other words, according to the structured light method, depth information can be extracted according to the degree of disparity caused by distance change. Accordingly, the accuracy of disparity may affect the accuracy of depth information.

More specifically, extraction of depth information using structured light can follow the following equation.

Here, h is the reference distance, h' is the object distance, b is the length of the baseline, and Δx is the disparity.

Referring to Equations 1 to 3, the length of the baseline (b) affects the disparity, and the disparity per unit length of the object distance (h') increases as the FOV (field of view) becomes smaller, and the baseline You can see that the bigger it gets, the bigger it gets. If the size of the object is smaller than half the baseline, points within a certain pattern may overtake adjacent points due to disparity, and as the object distance increases, disparity may decrease. Accordingly, in order to calculate accurate depth information, it is necessary to extract disparity based on the center of the point.

The performance of depth information extraction according to the structured light method may vary depending on the measurement accuracy level of the center of the point and the measurement resolution of the disparity.

Figure 5 is a graph explaining the change in disparity according to the difference between the reference distance and the distance between the object. Here, the reference distance was set to 300 nm as an example.

Referring to FIG. 5, it can be seen that as the object distance increases, the rate of change in disparity decreases. For example, when the object distance is 900 mm or less, the slope of the graph, that is, the amount of change in disparity relative to the change in distance, is large, but when the object distance is greater than 900 mm, the slope of the graph is significantly small. In particular, it can be seen that as the object distance increases, the slope of the graph becomes smaller.

From this, it can be seen that as the difference between the reference distance and the object distance increases, the disparity measurement accuracy decreases.

To solve this problem, the number of pixels in the image sensor can be increased.

Figure 6 shows disparity measurement performance according to the number of pixels of the image sensor.

Referring to FIG. 6, FIG. 6(a) and FIG. 6(b) show pixel values in an image sensor having a pixel number of 9*9, and FIG. 6(c) and FIG. 6(d) show pixel values in FIG. 6, respectively. This is the result of showing pixel values using an image sensor with a number of pixels of 3*3 for the same object as in (a) and FIG. 6(b).

Referring to FIGS. 6(a) and 6(b), when using a high-pixel image sensor with a number of pixels of 9*9, the values of FIGS. 6(a) and 6(b) are differentiated, but Referring to c) and FIG. 6(d), it can be seen that when using a low-pixel image sensor with the number of pixels of 3*3, the values of FIG. 6(c) and FIG. 6(d) are not distinguished.

However, when using a high-pixel image sensor, the overall power consumption and computation amount of the camera device increases, and the cost of the camera device significantly increases.

In an embodiment of the present invention, an attempt is made to improve the measurement accuracy of disparity at long distances without using a high-pixel image sensor.

FIG. 7 is a block diagram of a camera device according to an embodiment of the present invention, and FIG. 8 is a conceptual diagram of a camera device according to an embodiment of the present invention.

Referring to Figures 7 and 8, the camera device 700 according to an embodiment of the present invention includes an optical output unit 710, an optical input unit 720, a depth information generator 730, and a control unit 740. do. The optical output unit 710, the optical input unit 720, the depth information generator 730, and the control unit 740 are the optical output unit 10, the optical input unit 20, and the depth information described with reference to FIGS. 1 to 4, respectively. It is a configuration corresponding to the generator 30 and the control unit 40, and for convenience of explanation, duplicate descriptions of content that is the same as the description of the camera device 1 described with reference to FIGS. 1 to 4 will be omitted.

The optical output unit 710 generates an output optical signal and then radiates it to the object. At this time, the optical output unit 710 may output an optical signal of a predetermined pattern.

According to an embodiment of the present invention, the optical output unit 710 includes a first optical system 712 that irradiates an optical signal of a first field of illumination (FOI) to an object and light of a second angle of view smaller than the first angle of view. It includes a second optical system 714 that radiates signals to the object.

To be more specific, when the first optical system 712 of the optical output unit 710 is driven as shown in FIG. 8, the first optical system 712 transmits light of a predetermined pattern consisting of a plurality of dots at a first angle of view. Examine the signal to the object. Then, when the second optical system 714 of the optical output unit 710 is driven, the second optical system 714 radiates an optical signal of a predetermined pattern consisting of a plurality of dots to the object at a second angle of view that is smaller than the first angle of view. .

According to this, when the first optical system 712 and the second optical system 714 output optical signals of the same pattern, the dot density of the optical signal output through the first optical system 712 for the same distance is the same as that of the second optical system 714. It may be lower than the dot density of the optical signal output through 714. That is, when the first optical system 712 and the second optical system 714 output optical signals of the same pattern, the distance that the optical signal output through the second optical system 714 reaches for the same dot density is the distance reached by the first optical system 714. It may be longer than the distance that the optical signal output through 712 reaches. Here, point density may mean the number of points per unit area.

Accordingly, the first optical system 712, which radiates an optical signal of a first angle of view to the object, is used to generate depth information at a short distance, and the second optical system 712, which irradiates an optical signal of a second angle of view smaller than the first angle of view to the object 714 can be used to generate depth information at a longer distance than the distance measured by the first optical system 712. For example, the first optical system 712 may be used to generate depth information when the object distance is less than 900 mm, and the second optical system 714 may be used to generate depth information when the object distance is approximately greater than 900 mm.

To this end, the first optical system 712 includes a first light source 800 and a first diffractive optical element (DOE, 802) that diffuses the light output by the first light source 800 to a first angle of view. The second optical system 714 may include a second light source 810 and a second diffractive optical element 812 that diffuses the light output by the second light source 810 to a second angle of view. At this time, the first light source 800 and the second light source 810 output light of the same pattern, and the light output by the first light source 800 is divided by n*n times by the first diffractive optical element 802. The light output by the second light source 810 may be copied n*n times by the second diffractive optical element 812 and irradiated to the object. At this time, n may be an integer of 2 or more. To this end, the first diffractive optical element 802 and the second diffractive optical element 812 have the same shape, but the angle at which the light passing through the first diffractive optical element 802 spreads is the second diffractive optical element ( 812) can be designed to be larger than the angle at which the light passing through spreads out.

According to this, the pattern shape of the light signal irradiated to the object through the first optical system 712 and the pattern shape of the light signal irradiated to the object through the second optical system 714 may be the same. If the pattern shape of the light signal irradiated to the object through the first optical system 712 and the pattern shape of the light signal irradiated to the object through the second optical system 714 are the same, the depth information generator 730 Since the same operation can be applied to the optical signal input after being irradiated to the object through 712 and the optical signal input after being irradiated to the object through the second optical system 714, the depth information generator 730 The computational complexity can be reduced.

Meanwhile, according to an embodiment of the present invention, the first optical system 712 may further include a first collimator lens 804, and the second optical system 714 may further include a second collimator lens 814. A collimator lens refers to a lens that condenses light output from a light source.

The first collimator lens 804 is disposed between the first light source 800 and the first diffractive optical element 802, and focuses the light output by the first light source 800 into the first diffractive optical element 802. You can do it. The second collimator lens 814 is disposed between the second light source 810 and the second diffractive optical element 812, and focuses the light output by the second light source 810 into the second diffractive optical element 812. You can do it.

Here, the first optical system 712 is described as including the first light source 800, and the second optical system 714 is described as including the second light source 810, but the embodiment of the present invention is not limited thereto. . Although not shown, the first optical system 712 and the second optical system 714 include a common light source, and the light output from the light source is generated by aligning or moving the first collimator lens 804 and the second collimator lens 814. The light may be aligned to be directed toward the first diffractive optical element 802 or may be aligned to be directed toward the second diffractive optical element 812.

The first diffractive optical element 802, the first collimator lens 804, the second diffractive optical element 812, and the second collimator lens 814 described herein are the optical output unit 10 described with reference to FIGS. 1 and 2. ) may be part of the lens assembly 110.

According to an embodiment of the present invention, the optical input unit 720 includes a zoom optical system 722 and an image sensor 724 driven at a first magnification or a second magnification higher than the first magnification. Referring to FIG. 8, an IR bandpass filter 726 that passes IR light may be further disposed between the zoom optical system 722 and the image sensor 724.

Figure 9 is a block diagram of a zoom optical system according to an embodiment of the present invention. Referring to FIG. 9 , the zoom optical system 722 may include a zoom lens 900 and an actuator 902 that drives the zoom lens 900 at a first magnification or a second magnification higher than the first magnification. The zoom lens 900 includes one or more lenses and can be moved by an actuator 902. The zoom lens 900 includes a plurality of lenses, and some of the plurality of lenses may be a fixed group and others may be a movable group. As the zoom lens 900 moves, the distance between the zoom lens 900 and the image sensor 724 changes, and the magnification can be adjusted accordingly.

For example, the first magnification may be 1x, and the second magnification may be 3x, but are not limited thereto, and the second magnification may be 2x, 3x, 4x, 5x, 6x, or 7x. There could be at least one. Alternatively, the zoom optical system 722 may be a continuous zoom optical system that can be adjusted to any magnification of 1 to 7 times. Alternatively, the zoom optical system 722 may include a folded lens. According to this, the volume occupied by the zoom optical system 722 within the optical input unit 720 can be minimized.

In this way, if the optical input unit 720 includes the zoom optical system 722, the optical signal received through the optical input unit 720 can be enlarged. According to this, since the plurality of points forming the optical signal and the distance between them can be enlarged, the position and disparity of the center of the point can be measured more precisely.

According to an embodiment of the present invention, the control unit 740 interlocks the optical output unit 710 and the optical input unit 720.

As described above, the second optical system 714, which has a narrower viewing angle than the first optical system 712, can be applied to generate depth information over a longer distance than the measurement distance of the first optical system 712. However, as explained with reference to FIG. 5, as the object distance increases, the rate of change in disparity decreases. As explained with reference to FIG. 6, the number of pixels in the image sensor can be increased to solve this problem, but this causes The power consumption, computational complexity, and cost of the camera device may increase.

However, according to an embodiment of the present invention, when the control unit 740 links the optical output unit 710 and the optical input unit 720, precise depth information is provided not only at short distances but also at mid-to-long distances without increasing the number of pixels of the image sensor. You can obtain a camera device that can create .

That is, according to an embodiment of the present invention, when the first optical system 712 of the optical output unit 710 is driven, the control unit 740 controls the zoom optical system 722 to be driven at the first magnification, and the optical output unit 710 When the second optical system 714 of 710 is driven, the control unit 740 controls the zoom optical system 722 to be driven at a second magnification higher than the first magnification. For example, when the first light source 800 of the first optical system 712 is turned on, the control unit 740 controls the zoom optical system 722 to be driven at the first magnification and the second optical system 714 is turned on. When the light source 810 is turned on, the control unit 740 can control the zoom optical system 722 to be driven at the second magnification. In this way, when the zoom optical system 722 is driven at the second magnification when the second optical system 714 is driven, the points of the optical signal input to the optical input unit 720 and the distance between them are adjusted to the first magnification when the optical system 722 is driven. Because it is enlarged compared to when driven with , the center and disparity of the point can be measured precisely.

According to an embodiment of the present invention, the control unit 740 can control one of the first optical system 712 and the second optical system 714 to be selectively driven. For example, the control unit 740 may control the first optical system 712 to be driven and, in conjunction with this, control the zoom optical system 722 to be driven at the first magnification. Alternatively, the control unit 740 may control the second optical system 714 to be driven and, in conjunction with this, control the zoom optical system 722 to be driven at the second magnification.

According to an embodiment of the present invention, the first optical system 712 may be set to be driven preferentially by initial setting, and the control unit 740 may be changed to drive from the first optical system 712 to the second optical system 714. You can control it. Alternatively, the second optical system 714 may be set to be driven preferentially by initial setting, and the control unit 740 may control the drive to change from the second optical system 714 to the first optical system 712. Here, the control unit 740 can control the change operation based on information input through the user interface unit or information input from an external device. Alternatively, the control unit 740 may control change driving based on depth information generated by the depth information generator 730.

Hereinafter, a method for generating depth information of a camera device according to an embodiment of the present invention will be described in more detail.

FIG. 10 is a flowchart showing a method for generating depth information of a camera device according to an embodiment of the present invention, and FIG. 11 is a flowchart showing steps in which the control unit links the optical output unit and the optical input unit in the depth information generating method of FIG. 10. .

Referring to FIG. 10, the optical output unit 710 radiates an optical signal of a predetermined pattern consisting of a plurality of dots to the object (S1000). Here, the optical output unit 710 radiates an optical signal of a first angle of view to the object, or irradiates an optical signal of a second angle of view smaller than the first angle of view to the object.

Next, the control unit 740 interlocks the optical output unit 710 and the optical input unit 720 (S1010). For example, referring to FIG. 11, when the optical output unit 710 radiates an optical signal of the first angle of view to the object (S1100), the control unit 740 adjusts the zoom lens of the optical input unit 720 to the first magnification. Control it to drive (S1110). When the optical output unit 710 does not irradiate the optical signal of the first angle of view to the object but radiates the optical signal of the second angle of view to the object (S1120), the control unit 740 sets the zoom lens of the optical input unit 720 to the object. It is controlled to be driven at a second magnification higher than 1 magnification (S1130).

Next, referring again to FIG. 10, the optical input unit 720 receives the optical signal input after reflection from the object (S1020), and the depth information generator 730 receives the optical signal input to the optical input unit 720. Depth information of the object is generated using the disparity (S1030).

Meanwhile, according to an embodiment of the present invention, the control unit 740 causes the optical output unit 710 to irradiate an optical signal of a first angle of view to the object or to irradiate an optical signal of a second angle of view smaller than the first angle of view to the object. Can be controlled selectively. For example, the control unit 740 causes the optical output unit 710 to irradiate an optical signal of a first angle of view to an object or an optical signal of a second angle of view based on information input through a user interface or information input from an external device. You can selectively control to interrogate the object. Alternatively, the control unit 740 causes the light output unit 710 to irradiate the light signal of the first angle of view to the object or emit light of the second angle of view according to the depth information generated in the step of generating depth information of the object (S1030). The signal can be selectively controlled to irradiate to the object.

Figure 12 is a flowchart showing a method of controlling a camera device according to an embodiment of the present invention.

Referring to FIG. 12, the optical output unit 710 irradiates an optical signal at a first angle of view to an object (S1200), the optical input unit 720 receives an optical signal at a first magnification (S1210), and generates depth information. Unit 730 generates depth information using the received optical signal (S1220).

At this time, if the generated depth information is less than the preset distance (S1230), the control unit 740 causes the optical output unit 710 to irradiate the optical signal of the first angle of view to the object (S1200), and the optical input unit 720 radiates the optical signal of the first angle of view to the object. An optical signal is received at a magnification of 1 (S1210), and the depth information generator 730 can be controlled to generate depth information using the received optical signal (S1220). Here, the preset distance refers to the object distance and may be, for example, 900mm.

On the other hand, if the generated depth information exceeds the preset distance, the control unit 740 causes the optical output unit 710 to irradiate an optical signal of the second angle of view to the object (S1240), and the optical input unit 720 irradiates the optical signal of the second angle of view to the object. An optical signal is received at a magnification (S1250), and the depth information generator 730 can be controlled to generate depth information using the received optical signal (S1220).

According to this, in a state where the first optical system 712 of the optical output unit 710 is initially set to operate, when an object is at a mid-distance or longer, for example, a distance exceeding 900 mm, the angle of view and light of the optical output unit 710 The magnification of the input unit 720 can be adjusted adaptively.

In addition, when the second optical system 714 of the optical output unit 710 is driven and an object is at a short distance, for example, a distance of 900 mm or less, the angle of view of the optical output unit 710 and the optical input unit 720 The magnification can be adjusted adaptively for close range use.

According to this, the user can automatically convert from short-distance mode to long-distance mode or from long-distance mode to short-distance mode without the user having to set anything separately, so depth information can be extracted precisely and quickly for a moving object.

Figure 13 shows an example in which disparity measurement performance is improved when the optical input unit includes a zoom lens according to an embodiment of the present invention.

FIG. 13(a) is an image of the image sensor when the optical input unit does not include a zoom lens, and FIG. 13(b) is an image of the image sensor when the optical input unit includes a zoom lens according to an embodiment of the present invention. am.

Referring to FIG. 13(a), it can be seen that since some of the points overlap each other, it is difficult to precisely calculate the center and disparity of the points.

On the other hand, referring to FIG. 13(b), when the image is enlarged 5 times using a zoom lens, the center and disparity of the points can be calculated more precisely.

Accordingly, according to an embodiment of the present invention, the problem that the sensitivity of disparity decreases over a long distance can be solved, and a structured light camera device can be used to generate depth information not only at a short distance but also at a mid-to-long distance.

In this specification, the description is centered on an example in which the zoom optical system is driven at a first magnification or a second magnification higher than the first magnification, but is not limited thereto. The zoom optical system can be driven at a magnification of 2 or more. For example, the zoom optical system may be driven at a first magnification, a second magnification higher than the first magnification, or a third magnification higher than the second magnification. Alternatively, the zoom optical system may be a continuous zoom optical system in which magnification continuously changes.

The camera device according to an embodiment of the present invention may refer to a camera device mounted on a car and measures the distance between the car and an object, but is not limited thereto. For example, a camera device according to an embodiment of the present invention may be a LIDAR (Light Detection and Ranging) camera.

Although the above description focuses on the examples, this is only an example and does not limit the present invention, and those skilled in the art will be able to You will see that various variations and applications are possible. For example, each component specifically shown in the examples can be modified and implemented. And these variations and differences in application should be construed as being included in the scope of the present invention as defined in the appended claims.

1, 700: camera device
10, 710: Optical output unit
20, 720: Optical input unit
30, 730: Depth information generation unit
40, 740: control unit
712: First optical system
714: Second optical system
722: Zoom optics
724: Image sensor
726: IR bandpass filter
800: first light source
802: First diffractive optical element
804: first collimator lens
810: second light source
812: second diffractive optical element
814: second collimator lens

Claims (10)

An optical output unit that radiates an optical signal of a predetermined pattern consisting of a plurality of dots to an object,
An optical input unit that receives an optical signal reflected from the object, and
A depth information generator that generates depth information of the object using disparity of the optical signal received by the optical input unit,
The optical output unit,
A first optical system for irradiating the light signal of a first angle of view to the object, and a second optical system for irradiating the light signal of a second angle of view smaller than the first angle of view to the object,
The optical input unit includes a zoom optical system and an image sensor driven at one of a first magnification and a second magnification higher than the first magnification,
When the first optical system is driven, the zoom optical system is driven at the first magnification,
When the second optical system is driven, the zoom optical system is driven at the second magnification. According to paragraph 1,
Further comprising a control unit that controls the optical output unit, the optical input unit, and the depth information generator,
The first optical system includes a first light source and a first diffractive optical element (DOE) that diffuses the light output by the first light source to the first angle of view,
The second optical system includes a second light source and a second diffractive optical element that diffuses the light output by the second light source to the second angle of view. According to paragraph 2,
When the first light source is turned on, the control unit controls the zoom optical system to be driven at the first magnification,
When the second light source is turned on, the control unit controls the zoom optical system to be driven at the second magnification. According to paragraph 3,
The first light source and the second light source output light of the same pattern,
The light output by the first light source is radiated n*n times by the first diffractive optical element and irradiated to the object,
The light output by the second light source is radiated n*n times by the second diffractive optical element and irradiated to the object,
A camera device where n is an integer of 2 or more. According to paragraph 1,
The zoom optical system is a camera device including a zoom lens and an actuator that drives the zoom lens to one of the first magnification and the second magnification. According to paragraph 1,
Further comprising a control unit that controls the optical output unit, the optical input unit, and the depth information generator,
The control unit controls one of the first optical system and the second optical system to be selectively driven. According to paragraph 1,
The optical output unit,
If the generated depth information is less than a preset distance, radiating the light signal of the first angle of view to the object,
A camera device that irradiates the light signal of the second angle of view to the object when the generated depth information exceeds the preset distance. In a method of generating depth information of a camera device,
A step where the optical output unit radiates an optical signal of a predetermined pattern consisting of a plurality of dots to the object,
Linking the optical output unit and the optical input unit,
the optical input unit receiving an optical signal reflected from the object, and
Generating depth information of the object using disparity of the optical signal received by the optical input unit,
The step of irradiating the optical signal includes the optical output unit irradiating one of the optical signal at a first angle of view and the optical signal at a second angle of view smaller than the first angle of view to the object,
The step of linking the optical output unit and the optical input unit includes: when the optical output unit irradiates the optical signal of the first angle of view to the object, the zoom lens of the optical input unit is driven at a first magnification, and the optical output unit When the optical signal of the second angle of view is irradiated to the object, the zoom lens of the optical input unit is driven at a second magnification higher than the first magnification,
How to generate depth information. According to clause 8,
The step of irradiating the optical signal to the object may include the light output unit selectively irradiating the optical signal of the first angle of view to the object or the optical signal of the second angle of view to the object according to the generated depth information. A method of generating depth information that includes examining . According to clause 9,
The step of irradiating the optical signal to the object includes, if the generated depth information is less than a preset distance, the optical output unit irradiates the optical signal of the first angle of view to the object, and the generated depth information is the preset distance. When the distance is exceeded, the light output unit irradiates the light signal of the second angle of view to the object.

Images