KR20180136294A - Method of sensing depth of object applying external light and and device implementing thereof - Google Patents

Abstract

The present invention relates to a method for sensing a depth of an object by applying an external light which corrects distortion of information in which a depth is sensed by a strong external light, and a device realizing the same. According to an embodiment of the present invention, the method for sensing a depth of an object by applying an external light comprises the steps of: storing, in a storage unit, first depth information of an object sensed at a first point by a depth camera unit of a depth sensing module; storing, in the storage unit, second depth information of the object sensed at a second point by the depth camera unit; comparing the generated first depth information with the generated second depth information by a sensing data filtering unit of the depth sensing module so as to identify a filtering area in the second depth information; and adjusting a depth value of the filtering area from the second depth information by a control unit of the depth sensing module.

Description

FIELD OF THE INVENTION [0001] The present invention relates to a method of sensing the depth of an object by reflecting external light and a device implementing the same.

The present invention relates to a method of sensing the depth of an object by reflecting external light and an apparatus for implementing the method.

In order for robots to operate in spaces where human and material exchanges are active, such as airports, schools, government offices, hotels, offices, factories, gymnasiums, and cultural facilities such as concert halls, It is necessary to move. In this process, the robot can set the traveling route based on various information sensed during the traveling process. The robot influences the sensing of the object according to the external light (illumination, sunlight, etc.) Can receive.

Particularly, a depth sensing module for acquiring depth information or a depth sensing device on which such a module is mounted provides a distance to an external object as three-dimensional image information. In this process, strong light or an external light such as sunlight have. In particular, when an object reflects external light, a problem arises that the distance from the object to the outside is farther or closer to the actual distance.

This is a problem that can often occur in spaces where human and material exchanges are active, such as airports, schools, government offices, hotels, offices, factories, gymnasiums, and cultural facilities such as concert halls. Or the brightness of the internal illumination is high. Accordingly, in order to solve such a problem, the present invention proposes a method of reflecting the state of external light in the process of sensing the depth of objects.

SUMMARY OF THE INVENTION In order to solve the above problems, the present invention provides a method and apparatus for correcting distortion of depth-sensitive information due to strong external light.

Also, in this specification, a method and an apparatus for enhancing the accuracy of depth sensing information using spatial and temporal information capable of generating strong external light are provided.

In addition, the present invention aims at providing a method and an apparatus for safely moving a robot by raising the accuracy of the above-described depth sensing information.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention which are not mentioned can be understood by the following description and more clearly understood by the embodiments of the present invention. It will also be readily apparent that the objects and advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

A method of sensing depth of an object reflecting external light according to an exemplary embodiment of the present invention includes storing a first depth information of an object sensed at a first viewpoint of a depth camera module of a depth sensing module in a storage unit, Storing the second depth information of the object sensed at the second time point in the storage unit, comparing the first depth information and the second depth information generated by the sensing data filtering unit of the depth sensing module, and filtering the second depth information from the second depth information And adjusting the depth value of the filtering region in the second depth information by the control unit of the depth sensing module.

The depth sensing module for sensing the depth of an object in accordance with another embodiment of the present invention includes a storage unit for storing depth information, a first depth information and a first depth information of the object at a first point in time, A depth camera unit for generating second depth information of the object at a following second viewpoint, a sensing data filtering unit for comparing the generated first depth information and the generated second depth information to identify an area to be filtered in the second depth information, And a control unit for changing the depth value of the filtering area in the second depth information or removing the second depth information from the storage unit.

A robot for sensing depth of an object reflecting external light according to another embodiment of the present invention includes a storage unit for storing depth information, a storage unit for storing depth information of the object at a first time point, A depth camera unit for generating second depth information of an object at a second time point, which is a preceding or succeeding point, with the first depth information and the second depth information generated by the depth camera unit to identify a region to be filtered from the second depth information, A depth sensing module for changing the depth value of the filtering area or removing the second depth information from the storage part in the second depth information, an object sensing module for sensing an object disposed outside, And a robot control unit for controlling the movement path of the robot based on the sensed result of the depth sensing module and the object sensing module, The.

When the embodiments of the present invention are applied, in the case where the depth camera unit 110 is used outdoors or in a space where a lot of sunlight or light is introduced, or when the depth value is distorted and sensed in an environment where sunlight is reflected, So that normal depth information can be calculated.

In addition, when the embodiments of the present invention are applied, it is possible to increase the filtering speed by checking whether the depth value is to be filtered based on information about whether external light such as illumination or sunlight is generated according to the current position and the situation.

The effects of the present invention are not limited to the effects described above, and those skilled in the art of the present invention can easily derive the various effects of the present invention in the constitution of the present invention.

FIG. 1 is a block diagram of a depth sensing module according to an embodiment of the present invention. Referring to FIG.
FIG. 2 and FIG. 3 are diagrams showing a configuration of depth information according to an embodiment of the present invention.
4 is a diagram illustrating a process of processing depth information by a depth sensing module according to an exemplary embodiment of the present invention.
5 is a diagram illustrating a configuration of an optical information providing unit of a depth sensing module according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating a configuration of a map storage unit according to an embodiment of the present invention. Referring to FIG.
7 is a diagram illustrating a configuration of a light source period information unit and a context unit according to an embodiment of the present invention.
FIG. 8 is a view showing a configuration of a robot according to an embodiment of the present invention.
9 is a diagram illustrating a depth sensing process according to an embodiment of the present invention.
10 is a diagram illustrating a process of generating depth information according to an exemplary embodiment of the present invention and filtering the generated depth information.
11 and 12 are views illustrating a process of sensing a depth value of an object by reflecting the influence of external light in a depth sensing module according to an exemplary embodiment of the present invention.
13 is a flowchart illustrating a process of changing a depth value according to an exemplary embodiment of the present invention.
FIG. 14 is a diagram illustrating an example of newly generating and storing depth information according to an embodiment of the present invention.
FIG. 15 is a detailed diagram illustrating a process of adjusting a depth value according to an embodiment of the present invention. Referring to FIG.
16 is a diagram illustrating a process of identifying a filtering area by the filtering unit for sensing data according to an exemplary embodiment of the present invention.
FIG. 17 is a diagram illustrating a process of updating sensed external light information in a moving process of a robot according to an exemplary embodiment of the present invention. Referring to FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings so that those skilled in the art can easily carry out the present invention. The present invention may be embodied in many different forms and is not limited to the embodiments described herein.

In order to clearly illustrate the present invention, parts not related to the description are omitted, and the same or similar components are denoted by the same reference numerals throughout the specification. Further, some embodiments of the present invention will be described in detail with reference to exemplary drawings. In the drawings, like reference numerals are used to denote like elements throughout the drawings, even if they are shown on different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In describing the components of the present invention, the terms first, second, A, B, (a), (b), and the like can be used. These terms are intended to distinguish the components from other components, and the terms do not limit the nature, order, order, or number of the components. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; intervening "or that each component may be" connected, "" coupled, "or " connected" through other components.

The present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. As shown in FIG.

In the following description, the depth sensing module refers to a device on which a module is mounted, for example, a device for acquiring depth image information of an object in a space where the robot moves and calculating the distance of the object. The depth sensing module may be referred to as a depth sensing device when it exists as an independent device instead of being mounted on another device.

Meanwhile, in the present specification, the robot to which the depth sensing module is coupled includes a moving device having a specific purpose (cleaning, security, monitoring, guidance, etc.) or providing functions according to the characteristics of a space in which the robot moves. Therefore, the robot in this specification collectively refers to a device that has a moving means capable of moving using predetermined information and a sensor, and provides a predetermined function. In addition to the above-described depth sensing module, the sensor means various types of sensors that sense the presence, distance, and characteristics of an external object.

In this specification, a robot can move while holding a map. A map is information about a fixed object such as a fixed wall, a stair, or the like that has been confirmed not to move in space. In addition, information about dynamic objects that are placed periodically can also be stored on the map. In one embodiment, information about obstacles disposed within a certain range based on the traveling direction of the robot can also be stored on the map.

On the other hand, the robot can store information about external light such as sunlight, illumination, etc. on the map. For example, when external light such as sunlight is strong at a specific time or in a specific situation, or when external light is generated due to a specific lighting being turned on at a specific time, information on the external light can be stored on the map. The information about the external light includes information on the direction in which the light comes in, intensity of light, time, or cycle. Storing external light information on the above-described map can be applied to both the robot or the depth sensing module.

FIG. 1 is a block diagram of a depth sensing module according to an embodiment of the present invention. Referring to FIG. The depth sensing module includes a depth camera 110 for primarily generating depth information of an external object and a sensing data filtering unit 120 for identifying an area to be filtered from the depth information that is generated primarily by calculating the influence of external light A storage unit 130 for storing depth information, and a controller 150 for generating new depth information by applying previous or subsequent depth information to a region to be filtered.

The apparatus may further include an optical information providing unit 140 for providing information on characteristics of external light at a location where the depth sensing module is located. The optical information providing unit 140 may provide the optical information of the current position or the entering position by reflecting the position or the moving speed of the depth sensing module 100. In addition, an optical sensing unit 111 for sensing external light may be disposed. The optical sensing unit 111 may update or store the intensity or other characteristics of the light sensed during the movement in the optical information providing unit 140.

In more detail, the depth camera unit 110 generates the first depth information of the object at a first point in time and the second depth information of the object at a second point in time preceding or following the point of time with respect to the first point, ).

The sensing data filtering unit 120 compares the first depth information and the second depth information and identifies an area to be filtered from the second depth information. The identification of the filtering area will be described later. The controller 150 changes the depth value of the filtering area in the second depth information or removes the second depth information from the storage unit 130 so that the depth information can be accurately calculated.

In the components of FIG. 1, some components may be implemented in a robot when the depth sensing module is a device other than a single device, for example, a robot or the like. For example, the optical information providing unit 140 may be included in the depth sensing module 100 or may be included in a robot that is another apparatus to which the depth sensing module 100 is connected. In the latter case, the controller 150 may receive optical information and position information from the robot, and may provide the sensing information to the sensing data filtering unit 120 or may use the depth information to generate new depth information from which the influence of light is removed.

In the embodiment of the present invention, the depth sensing module sequentially stores depth information in a storage unit 130 in the same manner as a queue for a predetermined period of time, and, among the stored depth information, depth information strongly influenced by external light It is judged as a garbage value and not used. In this process, the speed at which the depth sensing module moves, for example, the moving speed of the depth sensing module, or the moving speed of the robot equipped with the depth sensing module, can be reflected.

FIG. 2 and FIG. 3 are diagrams showing a configuration of depth information according to an embodiment of the present invention. The depth information 10 indicates depth values of each depth cell of a depth image photographed in one frame. In addition, the depth information 10 is stored in the storage unit 130, and a plurality of pieces of depth information, which are photographed at different points in time, that is, corresponding to frames of a plurality of points of view, may be stored in the storage unit 130.

In one embodiment, the depth value of each x-y coordinate of an image photographed at a specific point in time may be configured. For example, the depth camera 110 may calculate a depth value for a depth image at a specific time point, that is, a depth cell divided into max_x horizontally and max_y horizontally corresponding to a frame at a specific time point. In one embodiment, each cell in the depth image may indicate one pixel. A depth cell indicates a unit in which one depth value is calculated including a pixel or a larger range. The size of the depth cell can be increased or decreased according to the resolution of the depth image calculated by the depth camera unit 110. The number on the upper side of the depth information 10 indicates the x coordinate, and the number on the left indicates the y coordinate.

One depth cell has one depth value. Therefore, the value of one depth cell defined on the x, y coordinates can be expressed as follows.

Depth Value: 61.3 (X: 15, Y: 25): Frame Number: 52

The frame number may be identification information for identifying one depth image. The depth value indicates a depth value of a depth cell indicated by X and Y of a specific frame.

As shown in FIG. 3, the depth information of the depth image for each frame generated by the depth camera unit 110 may continuously change. In one embodiment, the depth value of the specific depth cell constituting the depth information can be continuously changed. In one embodiment, the depth values of the depth cells 15a, 15b, and 15c having (x, y) 1 and 2 can be changed every frame. Of course, the depth value of another depth cell may also change. In summary, the depth information means a set of depth values constituting one frame acquired by the depth sensing module 100, that is, a depth image captured at one point in time.

As shown in FIG. 3, the depth values of all depth cells are increased by 1 at a time between the depth information 11 of the first frame and the depth information 12 of the second frame. On the other hand, it can be seen that there is a large difference between the depth values of the depth cells of (x, y) 1 and 2 between the depth information 12 of the second frame and the depth information 13 of the third frame.

That is, there is a large difference from 15a to 557, 15b to 558, and 15c to 795. Since the depth value of the other depth cell is uniformly increased by one in three frames, the controller 150 of the depth sensing module 100 can determine that the depth value of 15c is a garbage value due to external light.

The sensing data filtering unit 120 analyzes the accumulated depth information when the depth value is larger than a preset reference, and if the garbage value is a garbage value due to the influence of external light, the sensing value data is temporally previous or later, It is possible to extract the depth value of the corresponding cell from the depth information of the previous or subsequent depth image and convert the extracted depth value to remove the garbage value due to the external light and calculate the depth information reflecting the depth value of the actual objects.

Also, since a sudden change in the depth value may cause a sudden appearance of a real object, a change in the depth value may occur. Therefore, the optical information providing unit 140 provides information necessary for determining whether a large change in the depth value is due to the garbage value The sensing data filtering unit 120 may filter the variation of the depth value of the depth cell.

4 is a diagram illustrating a process of filtering a depth value in depth information in a process of accumulating depth information in a depth sensing module according to an embodiment of the present invention. 4, the depth sensing module senses N depth information for a predetermined time, identifies an error in the depth information due to external light, and corrects or removes the depth information.

The depth camera unit 110, which is a depth sensor, stores N depth information (depth data) in a storage unit (S21). For example, N may be a natural number of 1 or more. For example, five pieces of depth information may be stored in the storage unit 130 in the form of a queue.

Meanwhile, the controller 150 acquires the external environment information such as the characteristics of the external light corresponding to the current position of the sensing module and the moving speed of the depth sensing module (S22). The control unit 150 may acquire the external environment information through the optical information providing unit 140 or may acquire the external environment information from another device, for example, a robot, to which the depth sensing module 100 is coupled.

The sensed data filtering unit 120 reflects the external environment information and identifies a region having a variation of the depth value in the N depth information as a filtering region (S23). For example, the sensing data filtering unit 120 checks five queues and compares the depth information of each depth cell of the depth information. If the depth value is greater than a predetermined distance (for example, 100 mm), the sensed data filtering unit 120 determines the garbage value Do not use.

In addition, regions with large differences are identified as filtering regions. The reference distance may be calculated in consideration of the moving speed of the depth sensing module and the time at which the depth information is generated. For example, assuming that the moving speed of the robot equipped with the depth sensing module is 500 mm / s and the moving speed of the person is 1000 mm / s, a distance difference of 1500 mm per second may occur. However, if the depth camera unit 110 generates the depth information at a period of 33 mm / s, this means that about 30 depth information is generated per second. This is because 1500mm / 30 = about 50mm / frame.

Therefore, when the depth information of one frame is compared with the depth information of the previous frame or the subsequent frame, it is difficult for the difference of 50 mm or more to be generated. Therefore, when the depth value changes significantly, the sensing data filtering unit 120 It can be determined that the depth value is changed by the intensity of the external light instead of the depth value.

In order to increase the accuracy, the controller 150 can determine whether the space where the depth sensing module 100 is located is a space that is highly affected by sunlight or illumination, and provide the sensed data filtering unit 120 with the sensing result. In this case, when the robot equipped with the depth sensing module 100 or the depth sensing module 100 is located at a specific place and / or at a specific time, it is possible to filter depth values affected by external light, It is possible to eliminate the delay of the movement of the robot due to the ineffective obstacle detection by controlling and operating to identify the external obstacle in the moving process of the robot by removing it from the depth value.

Then, the controller 150 sets the depth values of the depth cells included in the filtering area using the other depth information stored in the storage unit 130.

5 is a diagram illustrating a configuration of an optical information providing unit constituting a depth sensing module according to an embodiment of the present invention or a robot equipped with a depth sensing module. The optical information providing unit 140 is for confirming the external environment information necessary for removing the garbage value from the depth information sensed by the depth sensing module 100. In the space where the depth sensing module moves, And so on. Also, the depth sensing module 100 can update the information about the external light to the optical information providing unit 140 when the external light sensed in the moving process has an intensity of a certain magnitude or more or the external light is periodically sensed have.

A map storage unit 141 in which reflection information or transmission information of external light is stored, a light source period information unit 145 in which period information in which external light is turned on and off is stored, And a context unit 146 in which a context in which external light is turned on and off is stored.

The map storage unit 141 stores information on a wall, a window, and a position of a light source in a space where the depth sensing module 100 moves. Particularly, the space wall stores information about the degree of reflection of light, so that the depth sensing module 100 can process the depth information by reflecting the intensity of the reflected light. The position of the light source may additionally include intensity of the light source, for example, illumination, color of the light source, and the like. If there is a space between the walls and the walls, this can be judged to be free. Values for the light transmittance of these glasses can also be stored in the map storage unit 141. [

The light source period information unit 145 is information on a period during which a specific light source is turned on and off. For example, a light source may be turned on every day, all day. Also, in the case of other light sources, it may be turned on only in the afternoon or evening. The light source period information unit 145 stores information about a cycle of turning on and off the light source so that the depth sensing module 100 can determine that there is no influence of external light when the light source is turned off.

The context unit 147 stores context information that is not included in the period information of the light source such as context information on the on / off state of the light source or context information on the window. In one embodiment, information associated with the season or weather can be stored in a context in which sunlight can enter the window. Also, when the light sources of the space are turned on or off according to a specific condition, context information on the light sources may be stored in the context unit 147. For example, in the case of an airport, on / off conditions of a light source to be turned on or off corresponding to a specific flight schedule may be stored in the context unit 147. Or the on / off condition of the light source that is turned on or off corresponding to the specific weather can be stored in the context unit 147. [

Accordingly, the depth sensing module 100 can confirm the information about the light source of the currently moving space and the environment in which the light sources are turned on or off, and remove the garbage value from the depth information by reflecting the information.

FIG. 6 is a diagram illustrating a configuration of a map storage unit according to an embodiment of the present invention. Referring to FIG.

The map storage unit 141a can be configured as a kind of bit map. It is possible to configure each bit in the bitmap of the image file to represent one unit area. Each unit area can point to the left bottom (0, 0) and the right top (19, 19). In addition, 141a in FIG. 6 may have a 20x20 data structure. For example, it may contain information about whether an object is placed for each location. When a fixed object is placed, the value of the corresponding matrix can be set to a predetermined value.

In addition, the map storage unit 141a may store characteristic information of light reflection or transmission properties of the fixed objects. The characteristic information about the objects may be classified into categories and stored in the map storage unit 141a have. In the map storage unit 141a of FIG. 6, a portion indicated by black indicates that the reflective value of the light is 10, which is an attribute object of low light reflectivity.

In addition, the fixed object indicated by 212 shows that the reflectance of light is 30, and the external light is reflected by 30%. The fixed object indicated by 213a and 213b shows that the reflectance of light is 70 and the external light is reflected by 70%. The category of the light reflectivity of the fixed object can be variously set, and the present invention is not limited thereto.

On the other hand, the portion of the fixed objects that are joined by T and the number shows the light transmission. For example, "T9 " indicated by 211a indicates that a transparent object, such as a transparent window, of a transparent material that transmits light at 90% or more is disposed. As a result, the depth sensing module 100 can confirm that the sunlight affects the calculation of the depth information when the depth sensing module 100 moves from the time when the sun rises and the vicinity of the direction 211a in the corresponding direction.

Quot; T5 "indicated by 211b means that a fixed object of semitransparent material, for example, translucent glass window, through which light is transmitted at 50% is disposed.

In the map storage unit 141a, L1 or L3 is displayed in a space where light can enter from the ceiling or a point where the light is arranged. Such as L1 or L3, may be labels that group windows or lights in the ceiling, and may, in one embodiment, be illuminance of light or light intensity due to the transmission of a window or group name of a particular category of lights.

 L1, L3, and the like represent the intensity of the sunlight or illumination of the light entering through the windows of the ceiling, they can be categorized and categorized, which can be categorized from 1 to 5 in one embodiment, (L1) of the intensity corresponding to the category "1" and the external light (L3) of the intensity corresponding to the category "3" are displayed.

In an embodiment, when the ceiling is transparent glass, it can be grouped into a group having a high light transmittance, and when the ceiling is opaque glass, grouping can be performed with a low light transmittance.

In yet another embodiment, the lights may be grouped and categorized in addition to the intensity of external light or illumination illumination. For example, groups of lights that are turned on or off at the same time can be categorized and labeled as L1, L2.

L1, L3, etc. provide specific information of external light, such as representing the intensity of sunlight entering through the windows of a ceiling or a group of lights or indicating the illuminance of illumination, while more detailed external light information, e.g., The information on the on-off period or the on-off context (such as a lighting condition in a specific space) in which the contexts (weather, latitude, longitude, etc.) And the context part 147. [

6, information about the degree of reflection of light by the material of the floor may also be stored in the map storage unit 141. For example, as shown in the following table, Information about the degree of reflection of the light.

Position Reflective (0, 0) to (19, 19) Reflective 50

Table 1 shows that (0, 0) corresponds to the bottom left region, and (19, 19) to the top right region corresponds to the category "50". This may reflect information that light may be reflected from the floor in the process of filtering the depth value sensed by the depth sensing module 100. Alternatively, the spatial information may be segmented to provide floor reflection information as shown in Table 2.

Position Reflective (0, 0) to (19, 10) Reflective 50 (0, 11) to (19, 19) Reflective 20

When the robot or the depth sensing module moves from the first region ((0, 0) to (19, 10)) in Table 2, the reflection degree of the ceiling is reflected in the category of 50 The depth information can be filtered in space. When moving to the second area ((0, 11) to (19, 19)), the depth information is filtered in the space belonging to the category corresponding to the degree of reflection of the ceiling light by reflecting the information that the reflectivity is 20 .

7 is a diagram illustrating a configuration of a light source period information unit and a context unit according to an embodiment of the present invention. The light source period information section 145a presents the external light grouped in Fig. 6, i.e., a period in which sunlight is illuminated or lights are turned on. L1 is set to be turned on for 6:00 to 22:00 hours (Time field) from Monday to Friday (Day field) regardless of the position (POS field is ALL). On the other hand, the external light (sunlight or illumination) grouped with L3 is set to be turned on for a predetermined time (10: 00-17: 00) on Saturday and Sunday.

If the time is set by day or by day, it may correspond to lighting, and in the case of sunlight, sunset / sunrise time may apply. The context part 147a presents, in addition to the light source cycle, a condition under which the sunlight is turned on, the light is turned on or off, that is, context information. It is mainly applied to lighting, but it can combine weather information such as sunlight to construct context information.

The lights arranged at the positions (positions indicated by 15 in Fig. 6) of the light sources of Ll (7, 17) can be adjusted to be turned off or on according to the external weather. This is because the L1 light source of FIG. 6 is provided with a transparent object 211a of a transparent material for transmitting external light. Thus, context information can be stored that turns off the illumination in "Sunny_Weather" and turns on the illumination in the "Night / Rainy_weather" situation as shown in 147a. The L3 light sources may all be set to ON when the space is crowded. And OFF (OFF) on Saturdays and Sundays.

6 and 7, when a robot mounted with the depth sensing module 100 or the depth sensing module 100 moves through a section affected by light, the current position and time of the robot The robot or depth sensing module 100 accumulates and learns the sections affected by sunlight or illumination and removes the influence of sunlight or light when moving at the same place at the same time And the influence of the external light is removed to calculate the accurate depth information.

Activation of the filter means that the sensing data filtering unit 120 of FIG. 1 filters the garbage value in the depth information, compares the depth values of the respective depth cells in the accumulated depth information according to an embodiment, It can be determined whether the depth value is an error. If the accumulated depth information increases in this process, it is possible to increase the accuracy of determining whether the depth value is a garbage value. This may cause a delay in confirming the depth information.

However, in order to filter the garbage value in the depth information within a delay time that the robot does not cause a problem in judging the obstacle by reflecting the moving speed of the depth sensing module 100 or the robot, The depth sensing module 100 can be configured. In this process, the information constituting the optical information providing unit 140 may be updated through learning to enhance the accuracy of the filtering for a faster filtering effect.

FIG. 8 is a view showing a configuration of a robot according to an embodiment of the present invention. The robot 1000 includes the above-described depth sensing module 100. The optical information providing unit 140 constitutes the robot 1000 and may be a component of the robot 1000 independently of the depth sensing module 100. Meanwhile, an object sensing module 190 for sensing external objects in the movement of the robot 1000, a moving unit 300 for moving the robot 1000, a function unit 400 for performing a specific function, The communication unit 500 is presented.

 8 shows the configuration of the robot hierarchically, but this is due to the logical representation of the components of the robot, and the physical configuration may be different. A plurality of logical components may be included in one physical component or a plurality of physical components may be implemented in a single logical component. Also, the hierarchy of FIG. 8 does not necessarily have to be maintained.

The object sensing module 190 senses external objects and provides the sensed information to the robot control unit 900. In one embodiment, the object sensing module 190 calculates the material and distance of external objects such as walls, glass, and metal doors at the current position of the robot in the intensity and reflected time (velocity) of the signal (Lidar ) Sensing unit 191. [0033] FIG. The object sensing module 190 may further include a vision sensing unit 192 for acquiring an image of an external object of the robot.

The vision sensing unit 192 may include a camera. The vision sensing unit 192 can take an image of the objects around the robot. In particular, the robot can identify whether an external object is a moving object by distinguishing an image in which the variation is not so much like a fixed object and an image in which the moving object is disposed.

In addition, the object sensing module 190 may include an ultrasonic sensor 193 for sensing the presence or absence of objects disposed within a predetermined distance from the robot 1000. The ultrasound sensing unit 193 provides information for determining whether objects exist within a certain distance from the robot 1000. [

In addition, a plurality of auxiliary sensing units 194 including a thermal sensing unit, an ultrasonic sensing unit, and the like may be disposed. These auxiliary sensing units provide auxiliary sensing information necessary to generate a map or to sense an external object. In addition, these auxiliary sensing units also sense objects disposed outside and provide information when the robot travels.

The sensing data analysis unit 195 analyzes the information sensed by the plurality of sensing units and transmits the analyzed information to the robot control unit 900. For example, when an object disposed in the periphery of the robot is detected by a plurality of sensing units, each sensing unit can provide information about a characteristic and a distance of the object. The sensing data analyzer 195 may combine these values and transmit them to the robot controller 900.

The robot controller 900 provides the sensed distance information to the depth sensing module 100 so that it can be referred to in the process of filtering the depth information. For example, the sensing data analysis unit 195 provides information to the robot control unit 900 that the object does not exist within 5 meters, and the robot control unit 900 provides such information to the depth sensing module 100 The depth sensing module 100 can determine whether the depth cell is distorted by external light and filter the depth cell if the depth value within 5 meters is sensed in the depth information generated by the depth sensing module 100. [

The moving unit 300 is a means for moving the robot 1000 such as a wheel, and moves the robot 1000 under the control of the robot control unit 900. At this time, the robot control unit 900 can confirm the current position of the robot 1000 in the area stored in the map storage unit 141 and provide a movement signal to the movement unit 300. The robot controller 900 may generate a route in real time using various information stored in the map storage unit 141 or may generate a route in the movement process.

The function unit 400 means providing a specialized function of the robot. For example, in the case of a cleaning robot, the function unit 400 includes components necessary for cleaning. In the case of a guide robot, the function unit 400 includes components necessary for guidance. In case of a secure robot, the function unit 400 includes components necessary for security.

The functional unit 400 may include various components according to functions provided by the robot, but the present invention is not limited thereto. The communication unit 500 provides a function of transmitting the information acquired by the robot to an external server or other robots, or receiving information from the server or another robot described above.

In particular, the communication unit 500 receives information about the current weather from external devices and confirms whether the condition is such that strong external light such as sunlight is generated, and provides the information to the robot control unit 900. The robot control unit 900 The light generation condition is provided to the depth sensing module 100 so that it can be applied to the filtering of the depth information.

The robot control unit 900 of the robot 1000 may generate or update a map of the map storage unit 200. [ In addition, the robot controller 900 can collect the sensing information provided by the depth sensing module 100 and the sensing information provided by the object sensing module 190 in the driving process, and control the movement path of the robot. That is, the robot controller 900 checks the object around the robot based on the sensed result of the depth sensing module 100 and the object sensing module 190, and controls the movement path of the robot 1000.

In addition, the optical information providing unit 140 may be disposed independently of the depth sensing module 100. In this process, the robot control unit 900 transmits the position information of the external light obtained in the moving process to the optical information providing unit 140 Can be stored. If it is determined that the attribute of reflection or transmission of light on the wall or floor of the moving space is changed in the map storage unit 141 in the process of generating the map of the moving space of the robot, Can also be changed.

The robot controller 900 may provide distance information of the object sensed by the object sensing module 190 to the depth sensing module 100 in an area corresponding to the depth information generated by the depth sensing module 100. [ When the distance information of the object provided through the robot control unit 900 is inconsistent with a part or all of the depth information, the control unit 150 of the depth sensing module 100 transmits information about an inconsistent region in the depth information to the sensing data filtering unit (120) to allow the sensing data filtering unit (120) to more accurately identify the area to be filtered.

9 is a diagram illustrating a depth sensing process according to an embodiment of the present invention. 9 is a view for explaining a case where the current position of the robot or the depth sensing module is determined to be a position affected by sunlight using the information stored in the map storage unit of the optical information providing unit described above, (Input data) of each depth cell is analyzed and filtered.

The depth camera unit 110 generates depth information composed of the sensed depth values and stores the depth information in the storage unit 130 (S31). Then, it is checked whether the current position and time of the robot are outside light, in particular, a period influenced by sunlight or strong illumination (S32). This can be confirmed by using information stored in the depth sensing module 100 or the optical information providing unit 140 constituting the robot 1000. As a result of checking, in the case of the section affected by external light, depth information composed of the most recent five frames stored in the storage unit 130, i.e., depth values, is stored in the queue (S33).

The above-mentioned five frames means the depth information photographed at the previous five times based on the current time point. Then, it compares the depth values of each depth cell of the previous frame with the depth values of each depth cell in the currently confirmed depth information, and determines whether to check whether the data is actual data or garbage value and filter it. If it is a garbage value, the process proceeds to step S31 so that the depth information is excluded and new depth information is sensed and stored. On the other hand, if the value is not the garbage value, the corresponding depth data is used (S35).

If it is determined in step S34 of FIG. 9 that the value is the garbage value, the depth value of the depth cell corresponding to the garbage value can be newly calculated. For example, if the depth value of the depth cell which is a specific position in the five depth information corresponding to the previous five frames continues to decrease by one but suddenly decreases by 1000, it can be judged as the garbage value. However, step S31 may be performed to discard the garbage value and newly calculate the depth information, or the depth information may be calculated by correcting the garbage value. For example, if the garbage value is identified as 15c in FIG. 3, the depth value of the corresponding cell can be converted into 559 based on the depth information 11, 12 of the previous frame.

10 is a diagram illustrating a process of generating depth information according to an exemplary embodiment of the present invention and filtering the generated depth information. 41 to 43 are shapes of objects disposed on the front surface of the depth sensing module 100 or the robot 1000, and show the shapes of objects on the front surface of each frame. 51 to 53 show a part of the depth information generated corresponding to these 41 to 43.

The depth information includes a depth value for each depth cell by dividing 41 to 43 into depth cells. 51 to 53 show depth values for specific depth cells 41p, 41q, 42p, 42q, 43p, and 43q. 41p, 42p and 43p have the coordinates of (30, 75) at 41, 42 and 43, respectively, and 41q, 42q and 43q are 41, 42, and 43 are (75, 25).

As shown in FIG. 2 and FIG. 3, the depth information may be stored for each depth cell. For convenience of explanation, only depth values of specific cells are displayed in FIG.

The depth values of 41p and 41q are sensed at 1530 and 3500, respectively. The depth values of 52 to 42p and 42q are sensed at 1480 and 3450, respectively. A portion of the sunlight 50 is introduced at 42, but the depth sensing is not affected.

As shown in 53, when the sunlight 50 appears on the front face at 43, 43p is 1430 which is reduced by 50 by a constant value when compared with 1480 which is the depth value of the same depth cell (the cell whose position is (30, 75) 43q are irregularly sensed as 0 when compared with 3450 which is the depth value of the same depth cell (the cell whose position is (75, 25)). This means that the value sensed by the depth sensing module is input to the garbage value due to the sunlight (50).

Accordingly, the sensing data filtering unit 120 may filter a value for which a significant change occurs at a specific value such as 43q of 53 among the depth values, and the controller 150 removes the filtered depth information 53 or the previous depth information The depth information 54 having the new depth value 3400 can be generated and stored by referring to the first depth value 51 and the second depth value 52.

In this case, the control unit 150 determines whether to filter the depth value, which is generated based on the sensed value of the object sensing module 190 of the robot 1000, as a garbage value or a new object, Lt; / RTI > For example, when an object is suddenly approaching the object sensing module 190, the sensed data filtering unit 120 may use the depth value without filtering. Alternatively, the depth camera unit 110 may again generate the depth information for accuracy.

As shown in FIG. 10, the process of analyzing the depth information by the depth sensing module 100 is summarized as follows.

Even if there is no obstacle in front of the depth camera unit 110, the depth value within a specific range under the influence of the sunlight changes in such a way that the depth value in each frame is excessively changed between frames, If it is confirmed to disappear, such depth information can be filtered.

For example, a situation where a depth value of a specific depth cell between 400 and 1000 mm in the front part is successively detected in depth information of 2 to 3 frames and a value self zero is detected can be repeated. In particular, if the same value does not arrive continuously as a depth information of 2 or 3 frames, but if the depth value is not linearly increased or decreased such as 200, 700, 400, 0, 1000 or the like, The depth sensing module 100 can confirm that the depth value changes due to the influence of external light rather than the depth sensing, and can filter the corresponding depth value.

In this process, the depth sensing module 100 can refer to the sensing result of the external objects sensed by the object sensing module 190 to check whether the object is actually disposed on the front surface, The depth value can be filtered on the basis of the possibility that the external light may exist by extracting the arrangement of the external light at the current position, the intensity of the light, and the context information. In addition, when the optical sensing unit 111 senses the intensity of external light and the sensed light has a degree that affects the depth sensing, the depth value can be adjusted.

In FIG. 10, information of the map storage unit 141 may be used. In FIG. 6, reflection of the material of the wall of the specific space or the degree of transmission of external light due to the arrangement of the window is reflected, Or whether there is reflected external light. Also, as shown in Table 1, it is possible to confirm the reflectivity of the light at the bottom and check the position of the illumination at the map storage unit 141a, thereby removing the influence of light from the depth information, thereby accurately calculating the depth value.

Although it is possible to refer to the specific position (x, y) of the depth information in order to check the change of the value of the specific depth cell in FIG. 10 through 41 to 43, according to another embodiment of the present invention, The relative coordinates corresponding to this point can be obtained. For example, when the vision sensing unit 192 confirms that the cylindrical column 41 is gradually approaching at 42 and 43, the position of the depth cell can be confirmed in proportion to the distance between the area where the value rapidly changes and the cylindrical column.

In one embodiment, the position of 42p is (30,75), but the position of 42p may be (28,73) at the corresponding point. In order to analyze a more accurate transition of the depth cell, the vision sensing unit 192 senses Information of objects can be used.

11 and 12 are views illustrating a process of sensing a depth value of an object by reflecting the influence of external light in a depth sensing module according to an exemplary embodiment of the present invention. It can be implemented in the depth sensing module 100 that operates independently or in the robot 1000 in which the depth sensing module 100 is mounted.

In FIG. 11, the depth camera unit 110 of the depth sensing module 100 stores the first depth information of the object sensed at the first viewpoint in the storage unit 130 (S61). Then, the depth camera unit 110 stores the second depth information of the object sensed at the second view point in the storage unit 130 (S62). A plurality of depth information can be stored even before the first point in time.

Referring to FIG. 12, the depth information generated by each view point, i.e., each frame, which is composed of a plurality of depth cells, may be stored in n storage units 130. Since the depth values of the depth cell at the same position of the depth information stored in the storage unit 130 are sensed by the depth camera unit 110 in a state where the depth sensing module 100 moves within a short time, .

This may vary depending on the moving speed of the depth sensing module 100 or the robot 1000, the moving speed of an external object, and the interval (time interval for each frame) at which the depth camera unit 110 senses. Therefore, the controller 150 or the sensing data filtering unit 120 of the depth sensing module 100 can set the maximum variation reference value of the depth value based on the moving speed and the sensing interval. In one embodiment, when the moving speed of the depth sensing module 100 is 500 mm / s and the maximum movable speed of the external object is 1000 mm / s, a distance difference of about 1500 mm per second may occur.

On the other hand, assuming that the interval at which the depth camera unit 110 is sensed is 30 frames per second, a depth value of 50 mm (1500 mm / 30 seconds) maximum may be generated in the depth information generated for each frame. Therefore, after setting the value of the maximum variation value of the depth value to 50 mm or a value slightly larger than this (100 mm), the depth value of the specific depth cell is compared with the previous frame in the depth information accumulated in FIG. The sensing data filtering unit 120 can check whether the depth value is a garbage value.

11, the sensing data filtering unit 120 of the depth sensing module 100 compares the generated first depth information and the generated second depth information to identify an area to be filtered from the second depth information (S63 ). When the sensing data filtering unit 120 previously identifies the depth cells, the depth cells corresponding to the depth values are identified as areas to be filtered if the variation of the depth values is greater than a predetermined reference value (maximum variation reference value).

Alternatively, the external sensing unit may be configured to detect the external light characteristic through the optical information providing unit 140 or to sense the intensity of the external light by the optical sensing unit 111 or the sensing result of the external object sensed by the object sensing module 190 of the robot 1000 If the depth value of the corresponding depth cell is determined as a garbage value based on information such as existence or distance, the sensing data filtering unit 120 identifies a region to be filtered composed of one or more depth cells.

The area to be filtered may be one or more depth cells in the second depth information, and the controller 150 adjusts the depth value of the filtering area in the second depth information (S64).

One embodiment of the adjustment is one in which the second depth information is removed from the storage unit 130 and depth information is generated at a new new point.

As another embodiment of the next adjustment, the depth value of the filtering area may be changed in the second depth information. This can change the depth value of the depth cells constituting the filtering region by referring to the first depth information or other temporally adjacent depth information. The depth information sensed as 53 in FIG. 10 is changed to 54 as an embodiment.

13 is a flowchart illustrating a process of changing a depth value according to an exemplary embodiment of the present invention.

As shown in FIG. 12, the depth cells at the same positions (X1, Y1) of the 1-1th-depth information to the 1-4th depth information have values of D11 to D14, respectively. D11 to D4 changes in a decreasing manner, and the difference between D11-D12, D12-D13 and D13-D14 is within a predetermined error range.

However, if the depth value D2 sensed in the second depth information is close to 0 or 0 or set to a value deviating from the previous D11 to D14 change direction, the sensing data filtering unit 120 determines that the sensing value is a garbage value. The depth value of the depth cell at the same position (X1, Y1) of the second depth information can be set to D2 'based on the previously sensed depth information or the subsequently sensed depth information.

In order to calculate D2, the controller calculates D2 using the previous depth information as it is or calculates D2 'generated by applying a weight to the previous depth information, and the value of D2' is the depth cell of the second depth information DCell (X1, Y1 Can be set to the depth value. That is, a depth value is newly calculated for each depth cell in the filtering area to generate new depth information, and the control unit 150 stores the depth information in the storage unit.

The following is summarized. The controller 150 may generate the new depth information by applying the first depth information (or the 1-1 to 1-4 depth information) to the area to be filtered of the second depth information.

Alternatively, the controller 150 applies the depth information generated by applying the weight to the first depth information (or the 1-1 to 1-4 depth information) to the area to be filtered of the second depth information at the second time point, The depth information can be generated. The controller 150 may store the newly generated depth information in the storage unit 130 as the depth information at the second time point. The moving speed of the robot can be reflected in the process of applying the weight.

FIG. 14 is a diagram illustrating an example of newly generating and storing depth information according to an embodiment of the present invention. The depth sensing module 100 generates depth information such as 10p, 10q, and 10s in chronological order. In this process, among the most recent depth information of 10s, an area (DCell1, DCell2 area of 10s) in which the difference of the value of the previous depth information occurs is identified as a filtering area. In one embodiment, the sensing data filtering unit 120 confirms that the first depth value of the first depth cell (Dcell1 or DCell2) of the first depth information 10p and 10q is decreased by 10 or 9, respectively.

Since the second depth values of the second depth cell (Dcell1 or DCell2) of the second depth information (10s) corresponding to the first depth cell (Dcell1 or DCell2) of 10p and 10q are respectively "0" The filtering unit 120 can compare the occurrence of the difference with the previous values.

More specifically, the sensing data filtering unit 120 can determine that the depth value sensed at the last "0 " is a garbage value since DCell1 has a depth value changed to 90, 80, Similarly, the sensing data filtering unit 120 can determine that the depth value sensed at the last "0 " is the garbage value since the depth value of DCell2 is changed to 80, 71, As a result, the sensing data filtering unit 120 can generate the new depth information 10s'. It can be confirmed that the values of the depth cells constituting the filtering area are changed to 70 and 62, respectively. The change of the depth value reflects the change process of 10p and 10q.

Of course, the sensing data filtering unit 120 may use only the immediately preceding depth information. For example, if 10q is referenced and there is no influence of external light, the maximum depth value that can be generated between 10q and 10s is 20, then the depth value that caused 20 or more difference is set as the garbage value, Can be calculated.

13 and 14, the depth information is based on the variation of the depth value of the depth cell of the same X / Y point in the depth information, but the present invention is not limited thereto. For example, when the subject is photographed by the vision sensing unit 192 of the object sensing module 190 as moving toward the right, the depth value of the associated depth cell may be reflected by reflecting the change.

In one embodiment, if the first depth information indicates that (1, 1) is the first depth cell, but the second depth information indicates that the same object has moved and moved to (1, 2) , 2). For this purpose, the vision sensing unit 192 can track the movement of the sensed object and analyze the associated depth cells.

In summary, the position of the first depth cell of the first depth information and the position of the second depth cell of the second depth information may be the same or different, and when they are different, the movement of the object is tracked, Different positions can be selected on the basis of the results.

FIG. 15 is a detailed diagram illustrating a process of adjusting a depth value according to an embodiment of the present invention. Referring to FIG. The steps S63 and S64 of FIG. 11 will be described in detail and the embodiment of FIG. 14 will be described.

The sensing data filtering unit 120 calculates the first depth value D1 (D11 to D14 in Fig. 14 as one embodiment) of the first depth cell of the first depth information of the object sensed at the first time point (S631 ). The sensing data filtering unit 120 calculates the second depth value D2 of the second depth cell of the second depth information of the object sensed at the second point of time (S632). If the difference between D1 and D2 is equal to or greater than the preset reference (S633), the sensing data filtering unit 120 performs filtering as in S635. If the difference is equal to or less than the set reference, the second depth information including D2 is stored in the storage unit (S649).

In step S633, it is determined whether the difference between D1 and D2 is equal to or greater than a preset reference. The difference between D1 and D2 is determined based on the position of the depth sensing module 100 at the first time point and the position of the depth sensing module at the second time point. It is confirmed that D2 is likely to have a garbage value in a case where the difference is greater than a predetermined threshold such as a maximum variation reference value in consideration of the moving speed of the object.

In order to calculate a new D2 ', the controller 150 sets D1 to D2' (new depth value) based on the difference in position of the depth sensing module at the first and second points of time . The fact that the depth difference is based on the difference means that the previously accumulated variation of the depth values is applied or the depth sensing module reflects the distance information moved between the two points. Then, the controller 150 sets the newly generated D2 'to the depth value of the second depth cell.

FIG. 16 is a diagram illustrating a process of using external light characteristic information in a process of identifying a filtering region by the sensing data filtering unit according to an exemplary embodiment of the present invention. As described above, the optical information providing unit 140 includes a map storage unit 141, a light source period information unit 145, and a context unit 147. The sensing data filtering unit 120 includes a sensing data filtering unit 120, The effect can be analyzed to identify the filtering area.

The control unit 150 extracts characteristic information of the external light stored in the optical information providing unit 140 (S71). The map storage unit 141 confirms whether there is illumination around the current position of the depth sensing module 100 or the robot 1000 as an embodiment. Also, the map storage unit 141 can confirm whether or not there is a fixed object of a material that transmits an external light such as a window in front of the depth sensing module 100.

More specifically, the control unit 150 can confirm whether external light such as illumination or sunlight is currently generated through the light source period information unit 145 and the context unit 147. [ In addition, the optical sensing unit 111 can be used to confirm whether there is strong external light in the periphery. As a result, the controller 150 can calculate information on the presence or absence of the external light, and the intensity of the external light. The intensity of the external light reflects the reflectivity of the bottom or the wall material, .

In other words, even if the illuminance of high illuminance is placed on the ceiling, the influence of the depth information due to the external light is low in the bottom direction if the light reflectance of the floor is very low. Therefore, the controller 150 lowers the possibility of external light, And may be provided to the filtering unit 120.

On the other hand, if the low-illuminance illumination is placed on the ceiling and the light reflectivity of the floor is very high, the influence of the depth information due to the external light will be high in the bottom direction. Therefore, the controller 150 increases the possibility of external light, (120). Accordingly, the sensing data filtering unit 120 can identify the filtering region by the external light by reflecting the characteristic information of the external light in the process as shown in FIG.

FIG. 17 is a diagram illustrating a process of updating sensed external light information in a moving process of a robot according to an exemplary embodiment of the present invention. Referring to FIG. The update includes both operations of changing the property information of the external light stored in advance, adding new external property information, or deleting the previously stored external property information.

The depth sensing module 100 or the robot 1000 includes an optical sensing unit 111. The optical sensing unit 111 may be a component of the robot 1000 independently of the depth sensing module 100.

The robot controller 900 compares the intensity of the external light sensed by the optical sensing unit 111 or the time point of turning on and off the external light during the movement of the robot 1000 with the characteristic information of the external light stored in the optical information providing unit 140 S81). If the comparison result is different (S82), the information about the external light intensity or on-off time is changed and stored in the optical information providing unit 140 (S83).

Instead of storing the characteristic information of the external light sensed by the unconditional optical information providing unit 140 in a different case in S82, the robot control unit 900 temporarily stores the characteristic information of the external light sensed in real time in another embodiment of the present invention The characteristic information of the external light temporarily stored in the optical information providing unit 140 may be stored in the optical information providing unit 140 when the data is accumulated in the optical information providing unit 140 so as to have a regularity enough to store the changed information.

When the embodiments of the present invention are applied, the sensors constituting the depth camera unit 110 of the depth sensing module 100, for example, a space in which an infrared (IR) sensor is used outdoors, Or when the depth value is distorted and sensed in an environment in which sunlight is reflected, it is filtered to calculate the normal depth information.

In particular, since a large number of windows and many lights are disposed in an open large area such as an airport, a school, a hospital, a high-speed bus terminal, a train station, etc., light is reflected or transmitted to affect the depth camera unit 110 many. When the present invention is applied in such an environment, the depth camera 110 can analyze the depth values that are affected and filter depth values distorted by the light to identify the garbage value.

Particularly, since the optical information providing unit 140 according to the embodiment of the present invention provides information on whether external light such as illumination or sunlight is generated for each time, place, and context, whether the depth value is to be filtered, It can be selected appropriately according to the state of light.

In the case of applying the present invention, the robot or the depth sensing module can distinguish the object to be avoided by the robot or the depth sensing module, such as a real obstacle or a human, from distorted information due to external light, from the calculated depth information. Particularly, in a wide area such as an airport, a school, a hospital, a terminal, a train station, a complex mall, etc., since robots are not affected by sunlight at all places, By analyzing the position and time, it is possible to control the robot to operate efficiently by applying the filtering algorithm only in the section where the robot is affected by the external light such as sunlight or strong illumination.

In the filtering process, a delay may occur in the process of comparing with the depth information accumulated in the previous frame. In this process, using the information of the influence of the external light at the current position and the current position of the robot, It is possible to increase the data processing speed. Further, the optical sensing unit 111 can be used to apply filtering based on context information for cases where external light is strong or not.

As shown in FIG. 17, when the robot 1000 travels repeatedly in the same space, it can accumulate information on the generation time or occurrence of external light.

In particular, as shown in FIG. 6, in order to filter what is recognized as data when light is reflected, information about the position of the light, the position of the window, and the degree of light reflection on the wall and the floor is maintained, It is possible to calculate the likelihood to be filtered based on this, or to perform no filtering operation.

In addition, it is to be understood that the present invention is not necessarily limited to these embodiments, and that all the elements within the scope of the present invention Or may be selectively coupled to one or more of them. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. The codes and code segments constituting the computer program may be easily deduced by those skilled in the art. Such a computer program can be stored in a computer-readable storage medium, readable and executed by a computer, thereby realizing an embodiment of the present invention. The storage medium of the computer program includes a magnetic recording medium, an optical recording medium, and a storage medium including a semiconductor recording element. Also, a computer program embodying the present invention includes a program module that is transmitted in real time through an external device.

While the present invention has been particularly shown and described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It is therefore to be understood that such changes and modifications are intended to be included within the scope of the present invention unless they depart from the scope of the present invention.

100: Depth sensing module 110: Depth camera section
120: Sensing data filtering unit 140: Optical information providing unit
141: map storage unit 145: light source period information unit
147: context unit 190: object sensing module
191: Lada sensing part 192: Vision sensing part
195: Sensing data analysis unit 300:
400: function unit 500: communication unit
900: robot control unit 1000: robot

Claims (15)

Storing the first depth information of the object sensed at the first viewpoint by the depth camera unit of the depth sensing module in a storage unit;
Storing the second depth information of the object sensed at the second viewpoint by the depth camera unit in a storage unit;
The sensing data filtering unit of the depth sensing module compares the generated first depth information and the second depth information to identify an area to be filtered in the second depth information; And
Wherein the controller of the depth sensing module adjusts the depth value of the filtering area in the second depth information.
The method according to claim 1,
The step of changing the depth value
Generating the fourth depth information by applying any one of the third depth information or the first depth information generated by applying the weight to the first depth information to the area to be filtered of the second depth information, ; And
Wherein the control unit stores the fourth depth information in the storage unit as depth information at the second time point, wherein the depth information is reflected by the external light.
The method according to claim 1,
The sensing data filtering unit may be configured to detect the first depth value of the first depth cell of the first depth information and the second depth value of the second depth cell of the second depth information corresponding to the first depth cell, Identifying a region to be compared and filtering, the external light reflecting the depth of the object.
The method of claim 3,
The sensing data filtering unit may be configured such that the sensing data filtering unit determines that the difference between the first depth value and the second depth value is between the position of the depth sensing module at the first time point and the position of the depth sensing module at the second time point. And setting the second depth value of the second depth cell to a garbage value when the difference value is greater than or equal to a preset reference,
Wherein the adjusting step comprises: the controller generating the first depth value and the third depth value based on the position difference of the depth sensing module at the first viewpoint and the second viewpoint; And
Wherein the controller is further configured to set the third depth value to a depth value of the second depth cell, wherein the depth of the object is reflected by the external light.
The method according to claim 1,
The depth sensing module may further include an optical information providing unit for storing external light characteristic information at a position of the depth sensing module,
The identifying step may include extracting characteristic information of the external light stored in the optical information providing unit;
Wherein the sensing data filtering unit identifies the filtering region by reflecting characteristic information of the extracted external light.
A storage unit for storing depth information;
A depth camera unit for generating first depth information of the object at a first time point and second depth information of the object at a second time point temporally preceding or following the first time point;
A sensing data filtering unit for comparing the generated first depth information and the second depth information to identify an area to be filtered in the second depth information; And
And a control unit for changing a depth value of the filtering area in the second depth information or removing the second depth information from the storage unit, the depth sensing module reflecting the external light.
The method according to claim 6,
Wherein the controller generates the fourth depth information by applying any one of the third depth information or the first depth information generated by applying the weight to the first depth information to the area to be filtered of the second depth information, And stores the fourth depth information in the storage unit as depth information at the second viewpoint, the depth sensing module reflecting the external light and sensing the depth of the object.
The method according to claim 6,
The sensing data filtering unit
A first depth value of the first depth information of the first depth information is compared with a second depth value of a second depth cell of the second depth information corresponding to the first depth cell, A depth sensing module that senses the depth of an object by reflecting light.
9. The method of claim 8,
Wherein the sensing data filtering unit is configured to determine whether the difference between the first depth value and the second depth value is greater than a difference between the position of the depth sensing module at the first time point and the position of the depth sensing module at the second time point Or sets the second depth value of the second depth cell to a garbage value when the second depth value is greater than a preset reference,
The controller generates the third depth value based on the position difference of the depth sensing module at the first viewpoint and the second viewpoint, and outputs the third depth value to the depth value of the second depth cell And a depth sensing module for sensing the depth of the object by reflecting external light.
The method according to claim 6,
And an optical information providing unit for storing the characteristic information of the external light at the position of the depth sensing module,
The control unit extracts characteristic information of the external light stored in the optical information providing unit,
The sensing data filtering unit reflects the extracted characteristic information of the external light and identifies the filtering area. The depth sensing module senses the depth of the object by reflecting external light.
11. The method of claim 10,
The optical information providing unit
A map storage unit in which the position of the fixed object in the space in which the depth sensing module generates the depth information and the reflection or transmission information of the external light are stored;
A light source period information unit storing period information on the external light on and off; And
And a context unit storing a context in which the external light is turned on and off, the depth sensing unit reflecting the external light and sensing the depth of the object.
A depth camera unit for generating first depth information of an object at a first viewpoint and second depth information of an object at a second viewpoint temporally preceding or following the first viewpoint at a first viewpoint, A sensing data filtering unit for comparing the generated first depth information and the second depth information to identify an area to be filtered by the second depth information, and a controller for changing a depth value of the filtering area in the second depth information, A depth sensing module including a control unit for removing two-depth information from the storage unit;
An object sensing module for sensing an object disposed outside;
A moving unit for moving the robot; And
And a robot controller for controlling an object around the robot based on the sensed result of the depth sensing module and the object sensing module to control a movement path of the robot, robot.
The method according to claim 1,
The depth sensing module or the robot further includes an optical information providing unit for storing external light characteristic information at a position of the depth sensing module,
Wherein the control unit of the depth sensing module extracts the characteristic information of the external light stored in the optical information providing unit and the sensing data filtering unit of the depth sensing module identifies the filtering area by reflecting characteristic information of the extracted external light, A robot that senses the depth of an object by reflecting external light and moves it.
14. The method of claim 13,
The optical information providing unit
A map storage unit in which the position of the fixed object in the space in which the depth sensing module generates the depth information and the reflection or transmission information of the external light are stored;
A light source period information unit storing period information on the external light on and off; And
And a context in which the context in which the external light is turned on and off is stored, the robot sensing the depth of the object by reflecting external light.
14. The method of claim 13,
The depth sensing module or the robot further includes an optical sensing unit,
When the intensity of the external light sensed by the optical sensing unit or the time point when the external light is turned on and off is different from the characteristic information of the external light stored in the optical information providing unit during the movement of the robot, the robot controller controls the intensity of the external light Or on and off points of the object, and stores the changed information.

Images