KR20220047155A - Erectronic apparatus and contrl method thereof - Google Patents

Abstract

An electronic device and a control method thereof are disclosed. The electronic device comprises a sensor and a processor for obtaining first information related to a distance from the sensor to an object and second information related to the reliability of the first information, identifying a plurality of cells that do not include distance information among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on the first information, determining the reliability of the cells identified based on the second information, changing the parameter settings of the sensor based on the determined reliability, obtaining third information related to a distance to the object from the sensor whose parameter setting is changed, obtaining fourth information including distance information of the cells based on the first and third information, and obtaining distance information with the object based on the distance information included in the fourth information. According to various embodiments of the present disclosure, an electronic device capable of detecting a distance to an object having a dark color such as black, etc. and a control method thereof may be provided.

Description

Electronic device and its control method {ERECTRONIC APPARATUS AND CONTRL METHOD THEREOF}

The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device capable of detecting a distance to an object and a control method thereof.

With the development of electronic technology, various electronic devices are being developed. In particular, recently, autonomous vehicles that drive on behalf of people, automated guided vehicles that classify goods by themselves and transport goods to their destinations, and robot cleaners that clean while driving in a space in the home have recently been developed. Electronic devices are being developed.

On the other hand, such a mobile electronic device needs to detect an object around the electronic device or a distance to the object in order to prevent collision with an object while driving. To this end, in particular, recently, an electronic device equipped with a light source-based sensor (eg, an image sensor or a lidar sensor) has been developed.

However, when the amount of light reflected by the object is insufficient, the conventional electronic device equipped with a light source-based sensor has a problem in that it cannot detect the distance to the object. This imposes restrictions on the driving of the electronic device, and in particular causes a concern about the occurrence of a collision between the electronic device and an object.

The present disclosure has been made to solve the above problems, and an object of the present disclosure is to provide an electronic device capable of receiving light having an amount of light greater than or equal to a minimum threshold value required for distance measurement through a change in a parameter setting of a sensor, and a method for controlling the same is in providing.

An electronic device according to an embodiment of the present disclosure for achieving the above object obtains first information related to a distance from a sensor and an object from the sensor and second information related to reliability of the first information, , identify a plurality of cells that do not include distance information from among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on the first information, and identify a plurality of cells that do not include distance information based on the second information determining reliability, changing the parameter setting of the sensor based on the determined reliability, obtaining third information related to the distance to the object from the sensor whose parameter setting is changed, and adding to the first and third information based on the fourth information including the distance information of the plurality of cells, and based on the distance information included in the fourth information may include a processor for obtaining the distance information to the object.

The processor may set a region of interest (ROI) including the plurality of identified cells, and may change parameter settings of the sensor based on reliability of the plurality of cells included in the ROI.

The processor determines the number of cells belonging to each reliability range in the ROI for each reliability range, and receives parameter setting information corresponding to the reliability distribution of a plurality of pixel cells included in the ROI based on the reliability-parameter setting information. and change the parameter setting of the sensor based on the parameter setting information.

The processor may change at least one of a parameter value for adjusting the light intensity of the sensor and a parameter value for adjusting an exposure time to light based on the determined reliability.

The reliability of the third information is relatively higher than that of the first information, and in the third information, a plurality of cells included in the ROI set based on the first information may include distance information.

The processor determines, from the third information, a plurality of cells included in a set ROI based on the first information, and obtains distance information of a plurality of cells included in the ROI based on the third information, , the fourth information may be acquired based on the first information and the acquired distance information.

The sensor determines a cell having a reliability equal to or less than a threshold value among the plurality of cells based on the second information, and sets distance information of a cell having a reliability equal to or less than the threshold value to a preset value. information can be created.

The processor identifies a plurality of cells included in a preset ROI from among a plurality of cells corresponding to a plurality of pixels constituting the sensor, and determines reliability of the plurality of cells included in the ROI based on the second information and may change the parameter setting of the sensor based on the determined reliability.

The processor determines the number of cells having a reliability value equal to or less than a preset threshold value among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on the second information, If the number of cells having a reliability value is equal to or greater than a preset number, the parameter setting of the sensor may be changed, and the third information including distance information may be acquired from the sensor whose parameter setting has been changed.

The processor may apply a weight to the determined reliability of the plurality of cells based on the identified positions of the plurality of cells, and may change the parameter setting of the sensor based on the reliability of the plurality of cells to which the weight is applied. .

The processor is configured to identify, based on the first information, at least one cell having a reliability equal to or less than a threshold value from among a plurality of cells including distance information, and based on the first and third information, the identified at least one The fourth information in which the distance information of the cell exceeds the threshold value may be obtained.

Meanwhile, a method of controlling an electronic device according to an embodiment of the present disclosure includes acquiring first information related to a distance to an object from a sensor and second information related to reliability of the first information, the first information identifying a plurality of cells that do not include distance information among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on , changing the parameter setting of the sensor based on the determined reliability, obtaining third information related to distance information from the object from the sensor whose parameter setting is changed, and based on the first and third information , obtaining fourth information including distance information of the plurality of cells, and obtaining distance information with the object based on distance information included in the fourth information.

The changing of the parameter setting may include setting a region of interest (ROI) including the identified plurality of cells, and changing the parameter setting of the sensor based on reliability of the plurality of cells included in the ROI. .

The changing of the parameter setting includes determining the number of cells belonging to each reliability range in the ROI for each reliability range, and corresponding to the reliability distribution of a plurality of cells included in the ROI based on reliability-parameter setting information It is possible to obtain parameter setting information to be used, and change the parameter setting of the sensor based on the parameter setting information.

The changing of the parameter setting may include changing at least one of a parameter value for adjusting the light intensity of the sensor and a parameter value for adjusting an exposure time to light based on the determined reliability.

The reliability of the third information is relatively higher than that of the first information, and in the third information, a plurality of cells included in the ROI set based on the first information may include distance information.

The obtaining of the fourth information may include, in the third information, determining a plurality of cells included in the ROI set based on the first information, and determining a plurality of cells included in the ROI based on the third information. The distance information may be obtained, and the fourth information may be obtained based on the first information and the obtained distance information.

The sensor determines a cell having a reliability equal to or less than a threshold value among the plurality of cells based on the second information, and sets distance information of a cell having a reliability equal to or less than the threshold value to a preset value. information can be created.

The changing of the parameter setting includes identifying a plurality of cells included in a preset ROI among a plurality of cells corresponding to a plurality of pixels constituting the sensor, and a plurality of cells included in the ROI based on the second information. may determine the reliability of the cell of , and change the parameter setting of the sensor based on the determined reliability.

The obtaining of the third information may include determining the number of cells having a reliability value equal to or less than a preset threshold value among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on the second information, If the number of cells having a reliability value less than or equal to a preset threshold is greater than or equal to the preset number, the parameter setting of the sensor may be changed, and the third information including distance information may be acquired from the sensor whose parameter setting has been changed.

The changing of the parameter setting includes applying a weight to the reliability of the plurality of cells determined based on the identified positions of the plurality of cells, and a parameter of the sensor based on the reliability of the plurality of cells to which the weight is applied. You can change the settings.

The acquiring of the fourth information may include identifying, based on the first information, at least one cell having a reliability equal to or less than a threshold value among a plurality of cells including distance information, and based on the first and third information The fourth information may be obtained in which the distance information of the identified at least one cell exceeds the threshold value.

According to various embodiments of the present disclosure as described above, an electronic device capable of detecting a distance to an object having a dark color such as a black color and a control method thereof may be provided.

1 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.
2A is a diagram for describing first information according to an embodiment of the present disclosure.
2B is a diagram for describing second information according to an embodiment of the present disclosure.
3 is a diagram for explaining an embodiment of setting a region of interest (ROI) based on at least one cell in which distance information is not detected, according to an embodiment of the present disclosure.
4A is a diagram illustrating a first reliability distribution according to an embodiment of the present disclosure.
4B is a diagram illustrating a second reliability distribution according to an embodiment of the present disclosure.
4C is a diagram illustrating reliability-parameter setting information according to an embodiment of the present disclosure.
5 is a diagram for explaining third information according to an embodiment of the present disclosure.
6 is a diagram for describing fourth information according to an embodiment of the present disclosure.
7 is a view for explaining an embodiment of changing a parameter setting of a sensor based on a weight according to an embodiment of the present disclosure.
8A is a diagram for explaining a preset ROI according to an embodiment of the present disclosure.
8B is a diagram for explaining an embodiment of changing a parameter setting of a sensor based on the reliability of all pixels constituting the sensor according to an embodiment of the present disclosure.
9A is a diagram for describing first information including distance information according to an embodiment of the present disclosure.
9B is a diagram for describing second information including reliability information according to an embodiment of the present disclosure.
9C is a diagram for explaining third information including distance information according to an embodiment of the present disclosure.
9D is a diagram for explaining fourth information including reliability information according to an embodiment of the present disclosure.
9E is a diagram for explaining fifth information according to an embodiment of the present disclosure.
10A is a detailed block diagram illustrating an electronic device according to an embodiment of the present disclosure.
10B is a detailed block diagram illustrating an electronic device according to an embodiment of the present disclosure.
11 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.
12 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

First, the terms used in the specification and claims were selected in consideration of the function of the present disclosure. However, these terms may vary depending on the intention of a person skilled in the art, legal or technical interpretation, and emergence of new technology. Also, some terms are arbitrarily selected by the applicant. These terms may be interpreted in the meanings defined herein, and in the absence of specific definitions, they may be interpreted based on the general content of the present specification and common technical common sense in the art.

In addition, in describing the present disclosure, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present disclosure, the detailed description thereof will be abbreviated or omitted.

Further, an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings and the contents described in the accompanying drawings, but the present disclosure is not limited or limited by the embodiments.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

The electronic device 100 according to an embodiment of the present disclosure may include a sensor 110 , a processor 120 , and a memory 130 .

The sensor 110 is a configuration that outputs information about the distance. Referring to FIG. 1 , the sensor 110 according to an embodiment of the present disclosure includes a light emitting unit 10 , a light receiving unit 20 and an MCU 30 . It may be a ToF sensor that includes The ToF sensor 110 is a sensor that measures the distance to the object by measuring the time until the light irradiated through the light emitting unit 10 is reflected by the object and received by the light receiving unit 20, where the light emitting unit ( 10) may be, for example, an IR light source irradiating infrared modulated with a signal of a specific frequency, and the light receiving unit 20 may be, for example, an image sensor receiving light reflected by an object. However, this is an example, and the light emitting unit 10 of the present disclosure may be implemented with various light sources capable of irradiating light such as ultrasonic waves or laser, and the light receiving unit 20 may receive light such as an RGB sensor. It can be implemented with various light receiving sensors.

The MCU 30 of the sensor 110 may control the light emitting unit 10 to emit light based on a control signal received from the processor 120 . Then, the MCU 30 receives information on the distance between the sensor 110 and the object based on the time from when light is irradiated by the light emitting unit 10 to being reflected by the object and received by the light receiving unit 20 . can be printed out. To this end, the MCU 30 performs an operation based on the speed c of light and the time t from when the light is irradiated by the light emitting unit 10 to being reflected by the object and received by the light receiving unit 20 to the sensor 110 and The distance between objects can be determined.

On the other hand, this is an embodiment, and the MCU 30 detects a phase change with respect to the time until the light irradiated by the light emitting unit 10 is reflected by the object and received by the light receiving unit 20, and the sensor ( 110) and information about the distance between the objects may be output. To this end, the MCU 30 operates the sensor ( 110) and the distance between the objects can be determined. As described above, the distance between the sensor 110 and the object can be determined by using the phase change. Hereinafter, for convenience of explanation, the MCU 30 transmits the light irradiated by the light emitting unit 10 to the object. A case in which the distance between the sensor 110 and the object is determined based on the time t until the reflection by the light-receiving unit 20 is received by the light-receiving unit 20 will be described.

Also, hereinafter, information on the distance between the sensor 110 and the object output by the sensor 110 will be collectively referred to as first information. According to an embodiment, the first information may be called distance information, depth information, or a distance map or a depth map. An example of the first information will be described later with reference to FIG. 2A.

Meanwhile, in FIG. 1 , the sensor 110 includes the MCU 30 , but according to an embodiment, the sensor 110 includes the light emitting unit 10 and the light receiving unit 20 , and the above-described MCU 30 . The function of may be performed by the processor 120 . In this case, the light emitting unit 10 irradiates light based on a control signal received from the processor 120 , and the processor 120 reflects the light irradiated by the light emitting unit 10 by an object to the light receiving unit ( 20), the distance between the sensor 110 and the object may be determined based on the time until it is received.

The electronic device 100 according to an embodiment of the present disclosure may include a memory 130 .

As will be described later, according to the control of the MCU 30 and the processor 130 , the memory 130 stores the first information and the second information output by the sensor 110 in order to determine the distance between the sensor 110 and the object. and reliability-parameter setting information may be pre-stored. A detailed description thereof will be provided later.

2A is a diagram for describing first information according to an embodiment of the present disclosure.

As described above, the MCU 30 of the sensor 110 is based on the time until the light irradiated by the light emitting unit 10 is reflected by the object and received by the light receiving unit 20, the sensor 110 and a distance between the objects may be determined.

Specifically, the MCU 30 may determine the distance to the object that reflects the light for each of a plurality of pixels constituting the light receiving unit 20 . Here, each of the plurality of pixels constituting the light receiving unit 20 according to an example may include an in-phase receptor and an out-phase receptor.

For example, while the light emitting unit 10 is irradiating light, the in-phase receptor is activated to receive light, and while the light emitting unit 10 is not irradiating light (eg, the light emitting unit 10 is off) During this period), the out-phase receptor is activated and can receive light.

For example, when the distance between the sensor 110 and the object is 0, the light emitting unit 10 may receive only the in-phase receptor and not the out-phase receptor of the light reflected by the object. As another example, if the distance between the sensor 110 and the object is not 0, the light emitting unit 10 takes a certain time to reach the light receiving unit 20 after the light emitting unit 10 is irradiated and reflected on the object. Part of the light reflected by the irradiated object can be received by the in-phase receptor, and the rest can be received by the out-phase receptor.

Here, the MCU 30 may identify the distance to the object based on a difference between the time the in-phase receptor receives the light and the time the out-phase receptor receives the light.

For example, as shown in FIG. 2A , the MCU 30 includes a light irradiated by the light emitting unit 10 to the object (refer to FIG. 2A , the object is, for example, a mouse 1, a human arm 2) , the wall surface 3, etc.) and received by each of a plurality of pixels, it is possible to determine the time at which the light is received for each of the plurality of pixels.

To this end, the plurality of pixels constituting the light receiving unit 20 includes a timer for measuring a reception time of light and a Time to Digital (TDC) converter for converting the measured time into a digital signal, and the MCU 30 includes a TDC Based on the signal output from the converter, it is possible to determine the time the light is received for each of the plurality of pixels. Then, the MCU 30 performs a time delay, that is, the time from when the light is irradiated by the light emitting unit 10 until it is received by each of the plurality of pixels and the operation based on the movement speed of the light, to the object for each of the plurality of pixels. The distance may be determined and the first information 210 including information on the determined distance may be output. For example, the MCU 30 may identify the time when each of the in-phase receptor and the out-phase receptor constituting the pixel receives light, and may identify the distance to the object based on the phase difference between the two times. However, this is an example and is not limited thereto.

As another example, each of the in-phase receptor and the out-phase receptor may receive a plurality of lights, and a distance to the object may be identified based on a phase difference at a time when each of the plurality of lights is received. For example, the light emitting unit 10 may sequentially irradiate a plurality of lights at regular time intervals (eg, 0.1 s intervals). Accordingly, each of the in-phase receptor and the out-phase receptor may receive a plurality of lights irradiated by the light emitting unit 10 at regular time intervals, and the MCU 30 transmits a plurality of lights to each of the in-phase receptor and the out-phase receptor. Of course, the received time may be identified, and a distance to the object may be identified based on the identified phase difference of a plurality of times. Of course, the specific time interval in the above-described example is an example for convenience of description and is not limited thereto.

That is, the MCU 30 may identify a distance between each of the plurality of pixels constituting the light receiving unit 20 and the object, and store the first information 210 including information on the identified distance in the memory 130 . there is.

As shown in FIG. 2A , the first information 210 may include distance information between each of the plurality of pixels and the object identified in each of the plurality of pixels constituting the light receiving unit 20 . First, each of a plurality of cells (or cells) constituting the first information 210 shown in FIG. 2A corresponds to a pixel, and the number written in the plurality of cells is an example of distance information, and the above-described operation It is a value expressing the distance between the pixel and the object identified by the MCU 30 through a numerical value, and the unit may be cm, but is not necessarily limited thereto.

Meanwhile, referring to FIG. 2A , the object may include a mouse 1 , a human arm 2 , a wall surface 3 , and the like. Objects corresponding to each of the plurality of pixels constituting the light receiving unit 30 may be the same or different.

For example, among the plurality of pixels constituting the light receiving unit 20 , an object corresponding to (eg, located in front) of the first pixel is the mouse 1 , and the first pixel is reflected by the mouse 1 . light can be received. In this case, the distance information of the first pixel identified by the MCU 30 may mean a distance between the sensor 110 and the mouse 1 .

As another example, an object corresponding to the second pixel among the plurality of pixels constituting the light receiving unit 20 may be a human arm 2 , and the second pixel may receive light reflected by the human arm 2 . In this case, the distance information of the second pixel identified by the MCU 30 may mean a distance between the sensor 110 and the human arm 2 .

Accordingly, referring to the first information 210 of FIG. 2A , each of the plurality of pixels is reflected from a different object (eg, any one of the mouse 1 , the human arm 2 , and the wall surface 3 ). Even when each of the plurality of pixels receives light reflected from the same object, each of the plurality of pixels receives the reflected light from a different area of the same object (for example, the top or bottom of the wall surface 3, etc.) Since light is received, it can be confirmed that distance information corresponding to a plurality of cells is different.

Meanwhile, as described above, each of the plurality of cells corresponds to a plurality of pixels constituting the light receiving unit 20 , and a number written in the plurality of cells may mean a distance from an object identified in the corresponding pixel.

If described in more detail with reference to the first information 210 of FIG. 2A , it can be seen that distance information corresponding to a plurality of pixels is different. This is because, when the distance between the sensor 110 and the object is different, the time at which the light irradiated by the light emitting unit 10 is received by each of the plurality of pixels (ie, the flight time of the light) is different. For example, referring to FIG. 2A , distance information of a cell corresponding to a pixel that has received light reflected by an object (eg, a human arm, etc.) located relatively close from the sensor 110 is, 110), it can be confirmed that information on a distance relatively shorter than distance information of a cell corresponding to a pixel that has received the light reflected by an object (eg, a wall surface) located relatively far away is included.

Meanwhile, referring to the first information 210 of FIG. 2A , it can be confirmed that distance information corresponding to some cells among a plurality of cells is 0.

As described above, in that the sensor 110 measures the distance to the object based on the light reflected by the object, when the amount of light reflected from the object and received by the light receiving unit 20 is less than or equal to a threshold value, the sensor ( 110), the distance to the object cannot be measured due to the insufficient amount of light. For example, in the case of an object having a dark color such as a black color (for example, referring to FIG. 2A , a partial region of the mouse 1), the light irradiated to the object is mostly absorbed by the surface of the object. , the amount of light reflected from the object and received by the light receiving unit 20 may be less than or equal to a threshold value.

In this case, the sensor 110 outputs distance information of a cell corresponding to a pixel that has received light having an amount of light equal to or less than a threshold value as 0. For example, as shown in FIG. 2A , distance information of a cell corresponding to a pixel that has received light reflected from a black object 1-1 (eg, a partial area of the mouse 1) among a plurality of cells is 0. can On the other hand, such an output value of 0 is an embodiment, and according to an embodiment, the sensor 110 stores distance information of a cell corresponding to a pixel that has received light having an amount of light equal to or less than a threshold value to another specific value such as an infinity value. You can also output it as a value.

Here, when the distance information is 0, it may mean that the distance between the sensor 110 and the object is 0, or it may mean that the distance to the object is not properly measured due to an insufficient amount of light. Accordingly, in the present disclosure, a cell having an output value of 0 is distance information (ie, a value of 0) of a cell having an output value of 0 based on reliability information (hereinafter, referred to as second information) for each of a plurality of cells. ) can be determined. Hereinafter, it will be described with reference to FIG. 2B.

2B is a diagram for describing second information according to an embodiment of the present disclosure.

The sensor 110 may output second information including trust information for the first information 210 .

Specifically, the MCU 30 of the sensor 110 may output the second information 220 indicating the reliability of the distance information corresponding to each cell constituting the first information 210 . Here, the reliability may be a value for the accuracy of distance information corresponding to each cell constituting the first information 210 . To this end, the MCU 30 performs distance information measured for each pixel constituting the light receiving unit 20 based on a plurality of charges measured as a plurality of lights having different phase shifts are received by the light receiving unit 20 . reliability can be judged. For example, when a plurality of lights having four phase shifts (eg, 0 degrees, 90 degrees, 180 degrees, 270 degrees) are sequentially irradiated through the light emitting unit 10 under the control of the MCU 30 , the light receiving unit (20) may sequentially receive a plurality of lights having four phase shifts.

Also, the MCU 30 may determine the reliability of the distance information corresponding to each of the plurality of pixels by calculating confidence = root[(Q1-Q2)^2+(Q3-Q4)^2]. For example, in the MCU 30 , the amount of charge Q1 measured when the first pixel receives the light of the first phase shift among the plurality of pixels constituting the light receiving unit 20, and the first pixel receives the light of the second phase shift The root[ (Q1-Q2)^2+(Q3-Q4)^2] operation is performed, and confidence with respect to distance information included in the cell corresponding to the first pixel may be obtained. The MCU 30 may acquire reliability of distance information corresponding to each of the plurality of pixels by using the above-described method (operation).

However, this is an exemplary embodiment, and the reliability of the first information 210 outputs the reliability corresponding to each pixel using the difference in distance information between a specific pixel and adjacent pixels or distance information measured for each pixel as input data. It can be determined through various methods, such as using a neural network model that has been trained to As another example, under the control of the MCU 30 , the light emitting unit 10 may have less than 4 phase shifts (eg, 0 degrees, 180 degrees) or more than 4 phase shifts (eg, 0 degrees, 45 degrees). degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, 315 degrees, 360 degrees, etc.) are sequentially irradiated, and the light receiving unit 20 sequentially receives a plurality of lights having different phases can do. Next, the MCU 30 measures the reliability in the pixel based on the amount of charge measured in the pixel according to the plurality of sequentially received lights, and accordingly, the MCU 30 provides distance information corresponding to each of the plurality of pixels. reliability can be obtained.

Since the second information 220 includes trust information on the first information 210 as described above, it may be called confidence information or a confidence map according to an embodiment.

For example, as shown in FIG. 2B , when light is irradiated toward an object, a plurality of pixels constituting the light receiving unit 20 receives the light reflected by the object, and the MCU 30 performs the above-described calculation method. Through this, the second information 220 including the reliability value for each cell corresponding to each pixel may be output. Subsequently, the MCU 30 may store the second information 220 in the memory 130 .

As shown in FIG. 2B , the second information 220 may include a cell corresponding to each of a plurality of pixels constituting the light receiving unit 20 and reliability information corresponding to the cell. Here, the reliability information is information including the reliability value calculated through the above-described calculation method, and the reliability value may have, for example, a value of 0 to 255, but is not limited thereto. In addition, it can be seen that a cell having a relatively high reliability value includes distance information with higher accuracy than a cell having a relatively low reliability value. Meanwhile, the shape of the second information 220 including cells corresponding to each of the plurality of pixels illustrated in FIG. 2B is an example and is not limited thereto.

Meanwhile, referring to the second information 220 according to an embodiment of the present disclosure illustrated in FIG. 2B , the reliability value of a cell corresponding to a pixel that has received light having an amount of light equal to or less than a threshold value is different from that of other pixels. It can be confirmed that the cell has a relatively small value (eg, a reliability value of 50 or less) when compared with the reliability value of the corresponding cell. Here, the pixel that has received the light having the amount of light equal to or less than the threshold value may be the pixel that has received the light reflected from the object having a dark color such as a black color, as described with reference to FIG. 2A .

In this case, the MCU 30 determines a cell having a reliability value equal to or less than a threshold value among a plurality of cells based on the second information 220 , and among a plurality of cells included in the first information 210 , a threshold value The first information 210 as shown in FIG. 2A may be output by setting the distance information of cells having the following reliability values to 0.

Meanwhile, as described above, in that the sensor 110 measures the distance to the object based on the light reflected by the object, if the amount of light reflected from the object and received by the light receiving unit 20 is insufficient, even the object It may be difficult to measure the distance of For example, the amount of light received by the light receiving unit 20 may be insufficient as most of the light irradiated to the black object is absorbed by the surface of the object.

In this case, the sensor 110 may not be able to measure the distance to the black-colored object due to an insufficient amount of light received by the light receiving unit 20 .

In particular, this imposes restrictions on the driving of an electronic device (eg, a mobile robot or an autonomous vehicle, etc.) that operates based on the distance to the object, and may cause a risk of collision between the electronic device and the object.

To solve this problem, the processor 120 of the present disclosure changes the parameter setting of the sensor 110 and sets the parameter when the first information 210 includes at least one cell for which distance information is not detected. Third information including distance information may be additionally obtained from the changed sensor 110 . Here, the change of the parameter setting refers to changing a parameter value set in at least one parameter among a plurality of parameters, and the parameter is, for example, a parameter for adjusting the luminous intensity of the light emitting unit 10 or an exposure time to light can be a parameter for adjusting However, this is an embodiment, and the parameter of the sensor 110 may be various parameters such as a parameter for changing the modulation frequency of light, and the technical spirit of the present disclosure is not limited to the embodiments described in this specification. . This will be described in detail with reference to FIG. 3 below.

3 is a diagram for explaining an embodiment of setting a region of interest (ROI) based on at least one cell in which distance information is not detected, according to an embodiment of the present disclosure.

The processor 120 may control the sensor 110 to emit light through the light emitting unit 10 . In addition, the processor 120 may receive the first information 210 generated by the MCU 30 from the sensor 110 as the light reflected by the object is received by the light receiving unit 20 . Here, the first information 210 may include distance information as described above.

Meanwhile, as described above, the first information 210 may be generated by the processor 120 . Specifically, the processor 120 controls the light emitting unit 10 of the sensor 110 to irradiate light, and when the light irradiated by the light emitting unit 10 is reflected by the object and received by the light receiving unit 20 , , based on the flight time of the light, it is possible to determine (or calculate) the distance between the sensor 110 and the object.

Also, the processor 120 may receive the second information 220 including trust information on the distance information of the first information 210 from the sensor 110 .

As described above, the sensor 110 may output the second information 220 indicating the reliability of the distance information corresponding to each pixel constituting the light receiving unit 20 . Here, the reliability may be a value for the accuracy of distance information corresponding to each pixel constituting the light receiving unit 20 .

Meanwhile, the second information 220 may be generated by the processor 120 . Specifically, the processor 120 controls the light emitting unit 10 of the sensor 110 to irradiate light, and when the light irradiated by the light emitting unit 10 is reflected by the object and received by the light receiving unit 20 , , it is possible to determine the reliability of distance information corresponding to each pixel constituting the light receiving unit 20 by using the charge Q and the phase shift measured based on the amount of light received by the light receiving unit 20 . For example, when the light receiving unit 20 receives light with four different phase shifts, the processor 120 calculates each Reliability of distance information corresponding to a pixel may be determined.

As described above, when the amount of light reflected by the object and received by the light receiving unit 20 is insufficient, that is, the amount of light received by the light receiving unit 20 is less than or equal to the minimum threshold required for distance measurement. , a value of 0 is output as distance information of the cell corresponding to the pixel that has received the light reflected from the object.

When the first information 210 including distance information is received from the sensor 110 , the processor 120 has no distance information among a plurality of cells corresponding to a plurality of pixels constituting the light receiving unit 20 based on the distance information. It is possible to identify at least one cell having an output value of 0 representing . Here, the identified at least one cell may be a cell corresponding to a pixel that has received an amount of light equal to or less than a minimum threshold required for distance measurement. In addition, the processor 120 may set a region of interest (ROI) including at least the identified cells to the first information 210 and the second information 220 . Here, the ROI may be, for example, a rectangular region having a minimum size including at least one cell having an output value of 0. However, the present invention is not necessarily limited thereto, and the ROI may have a region having various shapes, such as a circle including at least one cell, a closed curve, and a polygon. For example, as shown in FIG. 3 , the processor 120 sets a value of 0 indicating no distance information based on the first information 210 or the second information 220 among a plurality of pixels constituting the light receiving unit 20 . At least one pixel having an output value may be identified, and the ROI 1000 including a cell corresponding to the identified pixel may be set.

In addition, the processor 120 may change the parameter setting of the sensor 110 based on the trust information corresponding to the ROI 1000 .

To this end, when the ROI 1000 is set, the processor 120 may determine the reliability of at least one cell constituting the ROI 1000 based on the trust information included in the second information 220 .

In addition, the processor 120 may change the parameter setting of the sensor 110 based on the reliability of at least one cell constituting the ROI 1000 .

First, in order for the processor 120 to change the parameter setting of the sensor 110 , the processor 120 may identify a reliability distribution (or histogram) of a plurality of cells constituting the ROI 1000 . Subsequently, the processor 120 may acquire a parameter for changing the operation and setting of the sensor 110 based on the identified reliability distribution and the reliability-parameter setting information previously stored in the memory 130 .

For example, the parameter of the sensor 110 may include a parameter for adjusting the luminous intensity of the light emitting unit 10 or a parameter for adjusting the exposure time to light. However, this is an embodiment, and the parameter of the sensor 110 may be various parameters such as a parameter for changing the modulation frequency of light, and the technical spirit of the present disclosure is not limited to the embodiments described in this specification. .

Specifically, the processor 120 may determine the reliability of each of the plurality of cells included in the ROI 1000 . In addition, the processor 120 may determine the number of cells belonging to the corresponding reliability range for each reliability range.

As an example, referring to FIGS. 4A and 4B , the processor 120 determines the number of cells having a reliability between 0 and 25 in the ROI identified based on the first information 210 , and a reliability between 26 and 50. The number of cells, .., and the number of cells having a reliability between 226 and 255 may be determined based on the second information 220 . Meanwhile, the reliability range of 25 units is an example, and the reliability range may be set in various ways according to embodiments. In addition, the examples of FIGS. 4A and 4B are diagrams for explaining an embodiment of determining the number of cells belonging to a corresponding reliability range for each reliability range. It will be seen that it can be applied to the case of determining the number of cells per range of .

Subsequently, the processor 120 determines a parameter of the sensor 110 corresponding to the ROI 1000 based on the reliability distribution of the plurality of cells constituting the ROI 1000 and the reliability pre-stored in the memory 130-parameter setting information. can be obtained. Subsequently, the processor 130 may change the parameter setting of the sensor 110 to correspond to the acquired parameter.

An example of the reliability-parameter setting information pre-stored in the memory 130 is shown in FIG. 4C . For example, the reliability-parameter setting information may include parameter setting information for changing the parameter setting of the sensor 110 based on the number of cells having a reliability value lower than a minimum threshold value for distance measurement. For example, referring to FIG. 4C , in the reliability-parameter setting information, a first plurality (eg, 51 or more) of cells having a reliability value lower than the minimum threshold value for distance measurement are included in the ROI. Parameter setting information for increasing the luminous intensity of the unit 10 by a1 times, parameter setting information for increasing the exposure time to light by b1 times, or increasing the luminous intensity of the light emitting unit 10 by a11 times and setting the exposure time to light by b11 Parameter setting information for multiplying may be stored.

And, the reliability-parameter setting information includes a second plurality (eg, 31) or more and less than a first plurality (eg, 51) of cells having a reliability value lower than the minimum threshold value for distance measurement in the ROI, Parameter setting information for increasing the luminous intensity of the light emitting unit 10 by a2 times, parameter setting information for increasing the exposure time to light by b2 times, or increasing the luminous intensity of the light emitting unit 10 by a22 times and the exposure time to light Parameter setting information for increasing b22 times may be stored.

In addition, the reliability-parameter setting information includes a third plurality (eg, 11) or more and less than a second plurality (eg, 31) of cells having a reliability value lower than the minimum threshold value for distance measurement in the ROI, Parameter setting information for increasing the light intensity of the light emitting unit 10 by a3 times, parameter setting information for increasing the exposure time to light by b3 times, or increasing the light intensity of the light emitting unit 10 by a33 times and the exposure time to light Parameter setting information for increasing b33 times may be stored. And, in the reliability-parameter setting information, when the number of cells having a reliability value lower than the minimum threshold value for distance measurement is less than a third plurality (eg, 11) in the ROI, for maintaining the parameter value of the light emitting unit 10 Parameter setting information may be stored.

Accordingly, the processor 120 may change the parameter setting of the sensor 110 based on the reliability distribution and the reliability-parameter setting information of the plurality of cells constituting the ROI 1000 .

On the other hand, in the reliability-parameter setting information, the number of cells having a reliability value lower than the minimum threshold value included in the ROI, a parameter for increasing the light intensity, and a parameter for increasing the exposure time to light may have a proportional relationship. there is. For example, when the number of cells having a reliability value lower than the minimum threshold value included in the ROI is less than the third plurality (eg, 11), when the number of cells is greater than or equal to the first plurality (eg, 51), increase the luminance A parameter for increasing the exposure time to light and a parameter for increasing the exposure time to light may increase in proportion to the number of cells.

As an example, as shown in FIG. 4B , if a first plurality (eg, 51 or more) of cells having a reliability value less than or equal to the minimum threshold value for distance measurement (eg, reliability 50) are included in the ROI, The processor 120 increases the luminous intensity of the light emitting unit by a1 times (eg, 3.5 times), increases the exposure time to light by b1 times (eg, 3.5 times), or the light emitting unit 10 based on the reliability-parameter setting information. The parameter settings of the sensor 110 may be changed to increase the luminous intensity of α11 times (eg, 2.3 times) and increase the exposure time to light b11 times (eg, 1.5 times).

Alternatively, as shown in FIG. 4B , a second plurality of cells (eg, 31 or more) and less than the first plurality of cells having a reliability value less than or equal to a minimum threshold value for distance measurement (eg, reliability 50) are in the ROI (eg, less than 50), the processor 120 increases the luminous intensity of the light emitting unit 10 by a2 times (eg, 2 times) based on the reliability-parameter setting information, and increases the exposure time to light b2 sensor 110 to increase by a factor of (eg, a factor of 2), or increase the luminous intensity of the light emitter 10 by a22 times (eg, 1.3 times) and increase the exposure time to light by b22 times (eg, by a factor of 1.5) You can change the parameter settings of

Thereafter, the processor 120 may obtain third information including information on the distance between the sensor 110 and the object from the sensor 110 whose parameter setting is changed. Specifically, the processor 120 changes the parameter setting of the sensor 110 according to the reliability-parameter setting information, and sends a control signal for generating third information including distance information to the sensor 110 whose parameter setting is changed. can be transmitted In this case, the MCU 30 of the sensor 110 controls the light emitting unit 10 according to the control signal to irradiate light, and based on the time until the light is reflected by the object and received by the light receiving unit 20 . Third information including information on the distance to the object may be output.

Here, the third information 510 may include information on a distance to an object sensed by the sensor 110 that increases the luminous intensity of the light emitting unit 10 or increases the exposure time to light. In particular, the distance information included in the ROI 1000 of the third information 510 may have a relatively high reliability when compared with the distance information included in the ROI 1000 of the first information 210 . This is because as the luminous intensity of the light emitting unit 10 is increased, the light receiving unit 20 can receive light greater than or equal to the minimum threshold required for distance measurement, and as the exposure time to light is increased, the light receiving unit 20 is relatively This is because by receiving light for a long time, light above the minimum threshold required for distance measurement can be received.

For example, if there is an ROI including a large number of cells having a reliability value lower than the minimum threshold value for distance measurement, the processor 120 may improve accuracy of driving path identification or , it is possible to increase the exposure time to light in order to reduce the probability of collision with an object.

For example, when the time for the light emitting unit 10 to irradiate light is increased, the activation time of the in-phase receptor provided in the light receiving unit 20 that is activated during the time the light emitting unit 10 is irradiated with light, the light emitting unit ( The activation time of the out-phase receptor, which is activated during the time when 10) is not irradiated with light, can also be increased. In this case, as the activation time of the out-phase receptor provided in the pixel increases, light having an amount of light greater than or equal to the minimum threshold required for distance measurement may be received, and the MCU 30 has a reliability greater than or equal to the threshold corresponding to the pixel. Distance information having a value may be obtained.

For example, if there is an ROI including a plurality of cells having a reliability value lower than the minimum threshold for distance measurement, the processor 120 moves the electronic device 100 even though the time required to measure the distance increases. The exposure time to light may be increased in order to improve the accuracy of path identification or to reduce the probability of collision with an object.

As another example, if there is an ROI including a large number of cells having a reliability value lower than the minimum threshold for distance measurement, it may be somewhat unreasonable on the hardware specifications of the IR light source provided in the light emitting unit 10 (for example, , although there may be a risk of malfunction), to obtain reliable distance information, to improve the accuracy of identification of the driving path of the electronic device 100 , or to reduce the probability of collision with an object, the intensity of the light emitted by the light emitting unit 10 is adjusted. can increase

For example, referring to the third information 510 output by the sensor 110 in which the parameter of FIG. 5 is changed, unlike the first information output by the sensor 110 before the parameter change, it is included in the ROI 1000 It can be confirmed that the plurality of cells included distance information.

When the third information 510 is received from the sensor 110 whose parameter setting is changed, the processor 120 determines the number of cells included in the ROI 1000 of the third information 510 as shown in FIG. 6B . Based on the distance information and distance information of a plurality of cells included in the ROI 1000 of the first information 210 , fourth information 610 including distance information of a plurality of cells may be acquired.

Specifically, when the third information 510 is received from the sensor 110 , the processor 120 determines, from the third information 510 , a plurality of cells included in the ROI set based on the first information 210 . can do. Then, the processor 120 determines the distances of the plurality of cells included in the ROI 1000 of the first information 210 based on the distance information of the plurality of cells included in the ROI 1000 of the third information 510 . You can change the information. Accordingly, in the electronic device 100 of the present disclosure, as shown in FIG. 6 , distance information of a plurality of cells included in the ROI of the first information 210 is a plurality of cells included in the ROI of the third information 510 . The changed fourth information 610 may be acquired based on the cell distance information. Alternatively, the processor 120 fuses a plurality of distance information corresponding to a plurality of cells obtained from the first information 210 and a plurality of distance information corresponding to a plurality of cells obtained from the third information 510, The fourth information 610 including distance information of a plurality of cells may be generated (or acquired). In this case, the processor 120 fuses the ROI 1000 of the third information 510 with the ROI 1000 of the first information 210 to generate the fourth information 610 including distance information of a plurality of cells. Of course, it is also possible to generate the fourth information 610 including distance information of a plurality of cells by fusing the entire area of the third information 510 with the entire area of the first information 210 . .

Thereafter, the processor 120 may perform a process such as driving, stopping, and controlling the driving direction of the electronic device 100 based on the distance information included in the fourth information 610 . As described above, in that the electronic device 100 of the present disclosure performs a process such as driving based on the fourth information 610, the conventional electronic device does not detect the distance to an object of a dark color such as black color. It is possible to prevent problems (eg, collision with a black-colored object, etc.)

Meanwhile, according to an embodiment, the present disclosure may change parameter settings of the sensor 110 based on the reliability values of the plurality of cells and the weights of the plurality of cells.

Specifically, the processor 120 applies the first weight to the reliability value of the cell located at the lower end among the plurality of cells included in the ROI 1000, and applies the first weight to the reliability value of the cell located at the upper end. A second weight may be applied. Here, the first weight may be higher than the second weight. In this case, a cell located at the lower end of the plurality of cells is a cell for detecting an object closer to the electronic device 100 than a cell located at the upper end, and a higher weight is given to accurately measure the distance to the object. because there is a need.

For example, referring to FIG. 7A , the processor 120 applies a first weight to a reliability value of a cell located in the fourth row 4 among a plurality of cells included in the ROI 1000 , and applies a first weight to the fifth row. The second weight may be applied to the reliability value of the cell located in (5), and the third weight may be applied to the reliability value of the cell located in the sixth row (6). Similarly, the processor 120 applies a fourth weight to the reliability value of the cell positioned in the seventh row 7 among the plurality of cells included in the ROI 1000 and is positioned in the eighth row 8 . A fifth weight may be applied to the reliability value of the cell, and a sixth weight may be applied to the reliability value of the cell located in the ninth row 9 . Here, the application of the weight may be an operation of multiplying the reliability value of the cell by the weight. That is, if the weight is w and the reliability value is k, the weighted reliability value may be a value of w * k.

In addition, the processor 120 sums up values to which a weight is applied to each cell included in the ROI 1000 , and divides the summed value by the number n of a plurality of cells included in the ROI 1000 to obtain a calculated value. there is. As an example, in the ROI, four pixels with weight w1 and reliability value k1, pixels with weight w1 and reliability value k2, pixels with weight w2 and reliability value k3, and pixels with weight w2 and reliability value k4 If included, the processor 120 may calculate (w1*k1 + w1*k2 + w2*k3 + w2*k4)/4.

In addition, the processor 120 may change the parameter setting of the sensor 110 based on the calculated value obtained through the above-described operation. For example, when the first calculation value is obtained, the processor 120 changes the parameter setting of the sensor 110 to increase the luminous intensity of the light emitting unit 10 x1 times or increase the exposure time to light by y1 times, , when the second calculated value is obtained, the parameter setting of the sensor 110 may be changed to increase the luminous intensity of the light emitting unit 10 x2 times or increase the exposure time to light by y2 times. To this end, a plurality of parameter setting information corresponding to a plurality of calculation values may be stored in a memory (not shown) of the electronic device 100 . For example, as shown in FIG. 7B , in the memory (not shown), if the calculated value obtained through the above-described operation is greater than or equal to 0 and less than the first value, the parameter setting value of the sensor 110 is maintained, and the calculated value is If it is equal to or greater than the first value and less than the second value, the luminous intensity of the light emitting unit 10 is increased by x1, the light exposure time is increased by y1, or the luminous intensity of the light emitting unit 10 is increased x11 times and the light exposure time is increased by y11 times. For parameter setting information may be stored. In addition, in the memory (not shown), if the calculated value is greater than or equal to the second value and less than the third value, the luminous intensity of the light emitting unit 10 is increased by x2, the light exposure time is increased by y2, or the luminous intensity of the light emitting unit 10 is increased by x2. If the parameter setting information and calculation value for increasing x22 times and increasing the light exposure time by y22 times is equal to or greater than the third value, the luminous intensity of the light emitting unit 10 is increased by x3 times, the light exposure time is increased by y3 times, or the light emitting unit Parameter setting information for increasing the light intensity of (10) x33 times and increasing the light exposure time by y33 times may be stored.

Meanwhile, although it has been described above that the ROI is set based on a plurality of cells that do not include distance information, the ROI may be preset according to an embodiment.

For example, referring to FIG. 8A , the ROI 2000 may be preset as an area including a plurality of cells in the lower center area among a plurality of cells.

In this case, the processor 120 may identify a plurality of cells included in the preset ROI 2000 among the plurality of cells, and determine the reliability of the plurality of cells included in the ROI 2000 based on the second information. there is. In addition, the processor 120 may determine the reliability distribution (or histogram) of a plurality of cells constituting the ROI 2000 . Specifically, the processor 120 may determine the reliability of each of the plurality of cells included in the ROI 2000 , and may determine the number of cells belonging to the corresponding reliability range for each reliability range. In addition, the processor 120 may change the parameter setting of the sensor 110 based on the reliability distribution and reliability of the plurality of cells constituting the ROI 2000 -parameter setting information.

Meanwhile, the processor 120 may change the parameter setting of the sensor 110 based on the reliability of all cells corresponding to all pixels constituting the light receiving unit 20 . In this case, the ROI may be a region 3 including all cells of the second information 220 , as shown in FIG. 8B .

Specifically, the processor 120 may determine the number of cells having a reliability value equal to or less than a preset threshold value among a plurality of cells included in the ROI 3 based on the second information. Here, the preset threshold value may be, for example, 50 as a minimum reliability value required for distance measurement, but is not necessarily limited thereto. Then, the processor 120 sets the parameters of the sensor 110 when the number of cells having a reliability value less than or equal to a preset threshold among a plurality of cells included in the ROI 3 is greater than or equal to a preset number (eg, 50). can be changed

Meanwhile, in the above, an embodiment in which distance information of an ROI including at least one cell having an output value of 0 is changed based on distance information obtained by the sensor 110 whose parameter setting has been changed has been described. However, this is an embodiment, and although the electronic device 100 of the present disclosure outputs a non-zero value, the sensor 110 whose parameter setting is changed also for distance information of an ROI including at least one cell having a reliability value equal to or less than a threshold value. ) can be changed based on the distance information obtained by

As an example, by the sensor 110, the first information 910-1 including distance information corresponding to the plurality of cells as shown in FIG. 9A and the second information including reliability information corresponding to the plurality of cells as shown in FIG. 9B A case in which 2 information 910-2 is obtained will be described as an example.

9A and 9B , the processor 120 corresponds to distance information corresponding to a plurality of cells included in the first information 910-1 and a plurality of cells included in the second information 910-2. Based on the reliability information, at least one cell having distance information (ie, outputting a non-zero value) but having a reliability value of less than or equal to a preset threshold value (eg, 40) can be identified from among the plurality of cells there is. In addition, the processor 120 may set the ROI 2000 including the identified at least one cell to the first information 910 - 1 and the second information 910 - 2 . Here, the ROI 2000 may be, for example, a rectangular region with a minimum size including at least one identified cell as shown in FIGS. 9A and 9B , but is not necessarily limited thereto. It may be various shapes, such as a polygonal shape.

Then, as described above, the processor 120 changes the parameter setting of the sensor 110 as at least one cell having an output value of 0 is included in the first information 910-1, and the parameter setting is changed. Third information 920 - 1 including distance information and fourth information 920 - 2 including reliability information may be additionally obtained from the sensor 110 .

Then, the processor 120 calculates the output values of the plurality of cells included in the ROI 2000 of the first information 910 - 1 of the plurality of cells included in the ROI 2000 of the third information 920 - 1 . It can be changed (or corrected, updated) based on the output value.

Specifically, the processor 120 applies the first weight to the output value of the cell included in the ROI 2000 of the first information 910 - 1 , and applies the first weight to the ROI 2000 of the third information 920 - 1 . A second weight may be applied to the output value of the included cell. Here, the first weight is the reliability value of the cell included in the ROI 2000 of the second information 910-2, the reliability value of the cell included in the second information 910-2, and the fourth information 920 -2) may be a value divided by the sum of the reliability values of the cells included in -2), and the second weight is the reliability value of the cell included in the ROI 2000 of the fourth information 920-2, the second information It may be a value divided by the sum of the reliability value of the cell included in 910 - 2 and the reliability value of the cell included in the fourth information 920 - 2 . That is, when the reliability value of the same cell is C1 in the second information 910-2 and C2 in the fourth information 920-2, the first weight may be C1/(C1+C2), 2 weight may be C2/(C1+C2).

Then, the processor 120 sets the output value of the cell included in the ROI 2000 of the first information 910-1 to the output value of the cell included in the ROI 2000 of the first information 910-1. The value to which the first weight is applied and the value to which the second weight is applied to the output value of the cell included in the ROI 2000 of the third information 920 - 1 may be changed to a sum value.

For example, the reliability value of the first cell included in the ROI 2000 of the second information 910-2 is 25, and the reliability of the first cell included in the ROI 2000 of the fourth information 920-2 is 25. If the value is 55, the first weight may be 25/80, and the second weight may be 55/80. And, the output value of the first cell included in the ROI 2000 of the first information 910-1 is 110, and the output value of the first cell included in the ROI 2000 of the third information 920-1 is If is 150, the processor 120 sets the output value 110 of the first cell to 127 (for convenience of explanation, the decimal point is round off). Similarly, the processor 120 includes output values of the second to n-th cells included in the ROI 2000 of the first information 910-1 in the ROI 2000 of the second information 910-2. It may be changed based on the plurality of reliability values of the plurality of cells and the plurality of reliability values of the plurality of cells included in the ROI 2000 of the fourth information 920 - 2 .

Accordingly, in the electronic device 100 of the present disclosure, as shown in FIG. 9E , the distance information of the cell included in the ROI 2000 of the first information 910 - 1 is also obtained by the sensor 110 whose parameter setting is changed. The changed fifth information 930 may be acquired based on the acquired distance information. Through this, the present disclosure can provide distance information having high quality across all cells.

Meanwhile, according to an embodiment, the first weight is determined based on the standard deviation of a plurality of output values included in the ROI 2000 of the first information 910 - 1 , and the second weight is the third information 920 - 1 ) may be determined based on the standard deviation of a plurality of output values included in the ROI 2000 . Specifically, the output average value of N cells included in the ROI 2000 of the first information 910-1 is m1, and the output average value of the plurality of cells included in the ROI 2000 of the first information 910-1 is m1. If the output value is an x1 value to an xn value, the standard deviation a1 of the plurality of output values included in the ROI 2000 of the first information 910 - 1 is root((x1-m1)^2+(x2-m1) )^2+...+(xn-m1)^2)/N). In addition, the output average value of the N cells included in the ROI 2000 of the third information 920-1 is m2, and the output of a plurality of cells included in the ROI 2000 of the third information 920-1 is m2. If the value is the y1 value to the yn value, the standard deviation a2 of the plurality of output values included in the ROI 2000 of the third information 920-1 is root((y1-m2)^2+(y2-m2) It can be obtained by the operation ^2+...+(yn-m2)^2)/N). In addition, the processor 120 may apply a higher weight as the standard deviation is smaller. Specifically, the electronic device 100 may store information on a plurality of weights corresponding to a plurality of standard deviations to which a higher weight is matched as the standard deviation is smaller (hereinafter referred to as standard deviation-weight information). , the processor 120 may determine a weight corresponding to the standard deviation obtained through the above-described operation from among a plurality of pre-stored weights. For example, the processor 120 may determine a first weight corresponding to the standard deviation a1 and a second weight corresponding to the standard deviation a2 based on the pre-stored standard deviation-weight information, wherein the standard deviation a1 is If it is greater than the standard deviation a2, the first weight may be less than the second weight.

Then, the processor 120 determines the output value of the first cell included in the ROI 2000 of the first information 910-1, and the first cell included in the ROI 2000 of the first information 910-1. may be changed to a sum value of a value to which a first weight is applied and a value to which a second weight is applied to an output value of the first cell included in the ROI 2000 of the third information 920 - 1 . For example, the output value of the first cell included in the ROI 2000 of the first information 910-1 is 110, and the output value of the first cell included in the ROI 2000 of the third information 920-1 When this 150, the first weight is 0.4, and the second weight is 0.6, the processor 120 sets the output value 110 of the first cell included in the ROI 2000 of the first information 910-1, 110*0.4+ It can be changed to 134 obtained through 150*0.6 operation. Also, similarly, the processor 120 may change the output values of the second to n-th cells included in the ROI 2000 of the first information 910 - 1 based on the first and second weights.

In this way, the distance information of the ROI including at least one cell that outputs a non-zero value but whose reliability value is less than or equal to the threshold value is also changed based on the distance information obtained by the sensor 110 whose parameter setting is changed. Initiation can acquire distance information having high quality across all cells.

10A is a detailed block diagram illustrating an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 10A , the electronic device 100 according to an embodiment of the present disclosure includes a sensor 110 , a memory 130 , a driving unit 140 , a manipulation unit 150 , a display 160 , and a communication unit 170 . and a processor 120 .

Meanwhile, this is an embodiment, and the electronic device 100 may be implemented except for some of the above-described plurality of components, or may be implemented with additional components other than the above-described plurality of components. Hereinafter, parts overlapping with the above-described parts will be omitted or abbreviated.

The memory 130 may store an operating system (OS) for controlling overall operations of the components of the electronic device 100 and commands or data related to the components of the electronic device 100 .

Accordingly, the processor 120 may control a plurality of hardware or software components of the electronic device 100 using various commands or data stored in the memory 130 , and receive from at least one of the other components. The command or data may be loaded and processed into the volatile memory, and various data may be stored in the non-volatile memory.

In particular, the memory 130 may store reliability-parameter setting information. Here, the reliability-parameter setting information may include information on a plurality of parameter values corresponding to a plurality of reliability distributions. For example, in the reliability-parameter setting information, when a first plurality of pixels having a reliability value lower than the minimum threshold value for distance measurement are included in the ROI, a parameter for increasing the luminance of the light emitting unit 10 by a1 times Information on values and/or parameter values for increasing the exposure time to light by b1 may be stored. Alternatively, when a second plurality or more pixels having a reliability value lower than the minimum threshold value for distance measurement are included in the ROI, the second plurality of pixels less than the first plurality are included in the reliability-parameter setting information by increasing the luminance of the light emitting unit 20 by a2 times Information about a parameter value for increasing the exposure time to light and/or a parameter value for increasing the exposure time to light b2 times may be stored.

Also, information about a plurality of parameter values corresponding to a plurality of operation values may be stored in the memory 130 . Here, the calculated value may be a value calculated based on the reliability values of the plurality of pixels and the weights of the plurality of pixels.

The driving unit 140 may move the electronic device 100 . Here, the driving unit 140 includes a driving unit (not shown) and a motor (not shown) connected to the driving unit (not shown), and the driving unit (not shown) of the driving unit 140 is a wheel or a leg of a robot. etc., the motor (not shown) of the driving unit 140 may control the driving unit (not shown) according to the control of the processor 120 to move the electronic device 1000 .

For example, when the driving unit (not shown) is implemented as a left wheel and a right wheel, the processor 120 includes a motor for rotating the left wheel in order to move the electronic device 100 in a direction that does not collide with an object in front. By transmitting a control signal for generating a first rotational force and transmitting a control signal for generating a second rotational force different from the first rotational force to a motor that rotates the right wheel, the driving direction of the electronic device 100 can be changed. there is.

The manipulation unit 150 includes a first motor (not shown), a robot arm (not shown) connected to the first motor (not shown), a second motor (not shown), and a robot hand connected to the second motor (not shown). (not shown) may be included. Here, the robot arm (not shown) and the robot hand (not shown) are connected through a connector, and the robot arm (not shown) is 3 according to the driving of a first motor (not shown) connected to the robot arm (not shown). Dimensional movement or rotation can be performed. In addition, the robot hand (not shown) may perform three-dimensional movement, rotation, or product grip according to the driving of a second motor (not shown) connected to the robot hand (not shown).

The display 160 may display various screens. For example, the display 160 may display information about an object around the electronic device 100 or a distance to the object. Also, the display 160 may display a depth map based on the first information or a confidence map based on the second information.

The display 160 may be implemented as a liquid crystal display panel (LCD) type display. However, depending on the embodiment, the display 160 may be implemented with various types of displays, such as a light emitting diode (LED), organic light emitting diodes (OLED), liquid crystal on silicon (LCoS), digital light processing (DLP), etc. there is. In addition, the display 160 may include a driving circuit, a backlight unit, and the like, which may be implemented in the form of an a-si TFT, a low temperature poly silicon (LTPS) TFT, or an organic TFT (OTFT).

In addition, the display 160 may be implemented as a touch screen in combination with a touch sensing unit.

The communication unit 170 is configured to communicate with an external device. For example, the communication unit 170 communicates with various external devices through a wireless communication method such as BT (Bluetooth), BLE (Bluetooth Low Energy), WI-FI (Wireless Fidelity), Zigbee, or IR (Infrared) communication method. can be done Meanwhile, the communication unit 170 may be mounted on the processor 120 , and may be included in the electronic device 100 as a configuration separate from the processor 120 .

Meanwhile, the electronic device 100 according to an embodiment of the present disclosure may further include various configurations in addition to the aforementioned configuration.

As an example, the electronic device 100 may further include an input unit (not shown) capable of receiving a user input. Here, the input unit (not shown) may be implemented as a button or a touch screen, and may receive various user commands such as a user command for detecting a distance from an object or a user command for moving the electronic device 100 . there is.

Also, the electronic device 100 may further include a speaker (not shown) capable of outputting various types of audio data.

Also, the electronic device 100 may further include a microphone (not shown) capable of receiving a user's voice. Here, the user's voice may be a user's voice for executing a task of the electronic device 100 .

Meanwhile, in FIG. 10A , the sensor 110 includes the MCU 30 , but according to an embodiment, the sensor 110 includes a light emitting unit 10 and a light receiving unit 20 as shown in FIG. 10B . In addition, the above-described functions of the MCU 30 may be performed by the processor 120 . In this case, the light emitting unit 10 irradiates light based on a control signal received from the processor 120 , and the processor 120 reflects the light irradiated by the light emitting unit 10 by an object to the light receiving unit ( 20), the distance between the sensor 110 and the object may be determined based on the time until it is received.

11 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

The electronic device 100 may obtain first information related to the distance from the sensor and second information related to the reliability of the first information ( S1110 ).

Here, the first information may include information about the distance to the object. Specifically, the first information may include information on a plurality of distances corresponding to a plurality of pixels included in the sensor. In addition, the second information may include information on reliability of each pixel constituting the first information.

The electronic device 100 may identify a plurality of pixels that do not include distance information among a plurality of pixels constituting the sensor based on the first information ( S1120 ).

Specifically, when the first information including distance information is received from the sensor, the electronic device 100 includes at least one output value of 0 indicating no distance information among a plurality of pixels constituting the sensor based on the distance information. Pixels can be identified. Here, the identified at least one pixel may be a pixel that has received light having an amount of light equal to or less than a minimum threshold required for distance measurement.

Then, the electronic device 100 may determine the reliability of the identified plurality of pixels based on the second information ( S11130 ).

Specifically, the electronic device 100 may set an ROI including at least the identified pixels. Here, the ROI may be, for example, a rectangular region having a minimum size including at least one pixel having an output value of 0.

Then, the electronic device 100 may determine the reliability of the plurality of pixels included in the ROI based on the second information.

Thereafter, the electronic device 100 may change the parameter setting of the sensor based on the determined reliability ( S1140 ).

Here, the parameter of the sensor may be, for example, a parameter for adjusting the luminous intensity of the light emitting unit or a parameter for adjusting the exposure time to light.

Specifically, the electronic device 100 may change the parameter setting of the sensor based on the reliability distribution (or histogram) of the plurality of pixels constituting the ROI. To this end, the electronic device 100 may determine the reliability of each of a plurality of pixels included in the ROI. In addition, the electronic device 100 may determine the number of pixels belonging to the reliability range for each reliability range. In addition, the electronic device 100 may change the parameter setting of the sensor based on the reliability distribution of the plurality of pixels constituting the ROI and the reliability-parameter setting information.

The electronic device 100 may obtain third information including distance information from the sensor whose parameter setting has been changed ( S1150 ). Here, the third information may include information on a distance to an object sensed by a sensor that increases the luminous intensity of the light emitting unit or increases the exposure time to light.

The electronic device 100 may obtain fourth information in which distance information of a plurality of pixels identified in the first information is changed based on the third information ( S1160 ). Specifically, the electronic device 100 may acquire the fourth information in which the output value included in the ROI of the first information is changed based on the output value included in the ROI of the third information.

The electronic device 100 may determine the distance to the object based on the distance information included in the fourth information (S1170). In addition, the electronic device 100 may perform a process such as driving, stopping, and controlling the driving direction of the electronic device 100 based on the distance information included in the fourth information.

As described above, in that the electronic device 100 of the present disclosure performs a process such as driving based on the fourth information, the conventional electronic device fails to detect a distance to an object of a dark color such as a black color. Problems (eg, collision with a black-colored object, etc.) can be prevented.

12 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

The processor 120 may transmit a signal requesting generation (or transmission) of the first information related to the distance to the object and the second information related to the reliability of the first information to the sensor 110 ( S1210 ). . For example, when a user command for moving the electronic device 100 is input or when a user command for detecting a distance to an object is input, the processor 120 requests generation of the first information and the second information. A signal may be transmitted to the sensor 110 .

The sensor 110 may generate the first information and the second information under the control of the processor 120 (S1220).

Specifically, the MCU 30 of the sensor 110 may control the light emitting unit 10 to emit light based on a signal received from the processor 120 . Then, the MCU 30 determines the distance between the sensor 110 and the object based on the time until the light is irradiated by the light emitting unit 10, reflected by the object and received by the light receiving unit 20, and , may generate first information including information on a distance to an object for each of a plurality of cells corresponding to a plurality of pixels. In addition, the MCU 30 of the sensor 110 receives distance information measured for each pixel constituting the light receiving unit 20 based on a plurality of charges measured as a plurality of lights of different phase shifts are received by the light receiving unit 20 . Reliability may be determined, and second information including a reliability value may be generated for each cell corresponding to each pixel.

Then, the sensor 110 transmits the first information and the second information to the processor 120 (S1230), and the processor 120 based on the first information a plurality of cells that do not include distance information among the plurality of cells. can be identified.

Specifically, when the first information is received from the sensor 110 , the processor 120 receives at least an output value of 0 indicating no distance information among a plurality of cells based on the distance information of each cell included in the first information. One cell can be identified. Here, the identified at least one cell may be a cell corresponding to a pixel in which light having an amount of light equal to or less than a minimum threshold required for distance measurement is received.

Then, the processor 120 may determine the reliability of the identified plurality of cells based on the second information.

Specifically, the processor 120 may set an ROI including at least a cell identified based on the first information to the second information. Here, the ROI may be, for example, a rectangular region having a minimum size including at least one cell having an output value of 0.

Then, the processor 120 may determine the reliability of the plurality of cells included in the ROI based on the second information.

Thereafter, the processor 120 may transmit a signal requesting to change the parameter setting of the sensor to the sensor 110 based on the determined reliability ( S1240 ).

Here, the parameter of the sensor may be, for example, a parameter for adjusting the luminous intensity of the light emitting unit or a parameter for adjusting the exposure time to light.

Specifically, the processor 120 may transmit a signal requesting to change the parameter setting of the sensor to the sensor 110 based on the reliability distribution (or histogram) of the plurality of cells constituting the ROI. To this end, the processor 120 may determine the reliability of each of the plurality of cells included in the ROI based on the second information. Then, the processor 120 determines the number of cells belonging to the corresponding reliability range for each reliability range, and based on the reliability distribution and reliability of a plurality of cells constituting the ROI-parameter setting information, changes the parameter setting of the sensor. A request signal may be transmitted to the sensor 110 .

The sensor 110 may change the parameter setting based on the signal received from the processor 120 ( S1250 ), and transmit a signal for completing the parameter setting change to the processor 120 .

When it is determined that the parameter setting of the sensor 110 has been changed, the processor 120 may transmit a signal requesting generation of the third information to the sensor 110 ( S1260 ).

The sensor 110 generates third information including information on the distance to the object based on the signal received from the processor 120 (S1270), and transmits the third information to the processor 120 (S1280). can Here, the third information may include information on a distance to an object sensed by the sensor 100 that increases the luminous intensity of the light emitting unit or increases the exposure time to light.

The processor 120 may generate fourth information in which distance information of a plurality of cells identified in the first information is changed based on the third information ( S1290 ). Specifically, the processor 120 may acquire the fourth information in which the output value included in the ROI of the first information is changed based on the output value included in the ROI of the third information.

The processor 120 may obtain distance information with an object based on the distance information included in the fourth information, and perform processes such as driving, stopping, and controlling the driving direction of the electronic device 100 based on this. .

As described above, in that the electronic device 100 of the present disclosure performs a process such as driving based on the fourth information, the conventional electronic device fails to detect a distance to an object of a dark color such as a black color. Problems (eg, collision with a black-colored object, etc.) can be prevented.

Meanwhile, the above-described methods according to various embodiments of the present disclosure may be implemented in the form of software or applications that can be installed in an existing electronic device.

In addition, the above-described methods according to various embodiments of the present disclosure may be implemented only by software upgrade or hardware upgrade of an existing electronic device.

In addition, various embodiments of the present invention described above may be performed through an embedded server provided in the electronic device or a server external to the electronic device.

Meanwhile, a non-transitory computer readable medium in which a program for sequentially executing the method of controlling an electronic device according to the present invention is stored may be provided.

On the other hand, the non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specifically, the various applications or programs described above may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: electronic device
110: sensor
120: processor

Claims (20)

In an electronic device,
sensor; and
Obtaining first information related to a distance to an object from the sensor and second information related to reliability of the first information,
A plurality of cells that do not include distance information are identified among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on the first information, and reliability of the plurality of identified cells based on the second information to judge,
change the parameter setting of the sensor based on the determined reliability, and obtain third information related to the distance to the object from the sensor whose parameter setting is changed,
A processor configured to obtain fourth information including distance information of the plurality of cells based on the first and third information, and obtain distance information to the object based on distance information included in the fourth information An electronic device comprising; According to claim 1,
The processor is
An electronic device configured to set a region of interest (ROI) including the identified plurality of cells, and change parameter settings of the sensor based on reliability of the plurality of cells included in the ROI. 3. The method of claim 2,
The processor is
For each reliability range, the number of cells belonging to each reliability range is determined in the ROI, and parameter setting information corresponding to the reliability distribution of a plurality of cells included in the ROI is obtained based on the reliability-parameter setting information, and the An electronic device configured to change a parameter setting of the sensor based on parameter setting information. 3. The method of claim 2,
The processor is
and changing at least one of a parameter value for adjusting the light intensity of the sensor and a parameter value for adjusting an exposure time to light based on the determined reliability. According to claim 1,
The reliability of the third information is relatively higher than that of the first information, and in the third information, a plurality of cells included in the ROI set based on the first information includes distance information. According to claim 1,
The processor is
In the third information, determining a plurality of cells included in a set ROI based on the first information, obtaining distance information of a plurality of cells included in the ROI based on the third information, and the first The electronic device obtains the fourth information based on the information and the obtained distance information. According to claim 1,
The processor is
Identifies a plurality of cells included in a preset ROI from among a plurality of cells corresponding to a plurality of pixels constituting the sensor, determines reliability of a plurality of cells included in the ROI based on the second information, and An electronic device that changes a parameter setting of the sensor based on the determined reliability. According to claim 1,
The processor is
Determine the number of cells having a reliability value less than or equal to a preset threshold value among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on the second information, and having a reliability value less than or equal to the preset threshold value If the number of cells is greater than or equal to a preset number, the electronic device is configured to change a parameter setting of the sensor and acquire the third information including distance information from the sensor whose parameter setting is changed. According to claim 1,
The processor is
An electronic device that applies a weight to the determined reliability of the plurality of cells based on the identified positions of the plurality of cells, and changes parameter settings of the sensor based on the reliability of the plurality of cells to which the weight is applied. According to claim 1,
The processor is
Based on the first information, at least one cell having a reliability equal to or less than a threshold value is identified among a plurality of cells including distance information, and distance information of the identified at least one cell based on the first and third information to obtain the fourth information exceeding the threshold value. A method for controlling an electronic device, comprising:
acquiring first information related to a distance to an object from a sensor and second information related to reliability of the first information;
A plurality of cells that do not include distance information are identified among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on the first information, and reliability of the plurality of identified cells based on the second information judging;
changing the parameter setting of the sensor based on the determined reliability and acquiring third information related to distance information from the object from the sensor whose parameter setting is changed; and
obtaining fourth information including distance information of the plurality of cells based on the first and third information, and obtaining distance information with the object based on distance information included in the fourth information; ; Control method of an electronic device comprising a. 12. The method of claim 11,
Changing the parameter setting includes:
A method for controlling an electronic device, comprising setting a region of interest (ROI) including the identified plurality of cells, and changing parameter settings of the sensor based on reliability of the plurality of cells included in the ROI. 13. The method of claim 12,
Changing the parameter setting includes:
For each reliability range, the number of cells belonging to each reliability range is determined in the ROI, and parameter setting information corresponding to the reliability distribution of a plurality of cells included in the ROI is obtained based on the reliability-parameter setting information, and the A method of controlling an electronic device, wherein the parameter setting of the sensor is changed based on parameter setting information. 13. The method of claim 12,
Changing the parameter setting includes:
Changing at least one of a parameter value for adjusting the light intensity of the sensor and a parameter value for adjusting an exposure time to light based on the determined reliability. 12. The method of claim 11,
Reliability of the third information is relatively higher than that of the first information, and in the third information, a plurality of cells included in the ROI set based on the first information includes distance information. Way. 12. The method of claim 11,
The step of obtaining the fourth information includes:
In the third information, determining a plurality of cells included in a set ROI based on the first information, obtaining distance information of a plurality of cells included in the ROI based on the third information, and the first A method of controlling an electronic device, obtaining the fourth information based on the information and the obtained distance information. 12. The method of claim 11,
Changing the parameter setting includes:
Identifies a plurality of cells included in a preset ROI from among a plurality of cells corresponding to a plurality of pixels constituting the sensor, determines reliability of a plurality of cells included in the ROI based on the second information, and A method of controlling an electronic device, wherein the parameter setting of the sensor is changed based on the determined reliability. 12. The method of claim 11,
The step of obtaining the third information comprises:
Determine the number of cells having a reliability value less than or equal to a preset threshold value among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on the second information, and having a reliability value less than or equal to the preset threshold value When the number of cells is equal to or greater than a preset number, the parameter setting of the sensor is changed, and the third information including distance information is obtained from the sensor whose parameter setting is changed. 12. The method of claim 11,
Changing the parameter setting includes:
Control of an electronic device, applying a weight to the reliability of the plurality of cells determined based on the identified positions of the plurality of cells, and changing parameter settings of the sensor based on the reliability of the plurality of cells to which the weight is applied Way. 12. The method of claim 11,
The step of obtaining the fourth information includes:
Based on the first information, at least one cell having a reliability equal to or less than a threshold value is identified among a plurality of cells including distance information, and distance information of the identified at least one cell based on the first and third information Acquires the fourth information exceeding the threshold value, the control method of the electronic device.

Images