KR20230173277A - Robot and controlling method therof - Google Patents

Abstract

The robot starts. The robot includes a memory storing cleaning scenarios and neural network models according to the type of area to be cleaned, a camera, a distance sensor, a driving unit, and a processor. The processor inputs the image acquired through the camera and the distance data acquired through the distance sensor into a neural network model to obtain type information about at least one cleaning target area included in the image, and creates a cleaning scenario corresponding to the obtained type information. can be obtained from memory, and the driving unit can be controlled based on the obtained cleaning scenario. The driving part consists of a motor, a plate designed to rotate, a rail placed on the plate and having a slope inclined at a certain angle to the ground, a slider arranged to be able to travel back and forth on the rail, and one side is coupled to the slider, and the slider is connected to the rail. It may include an arm that moves while maintaining a constant angle with the ground as it reciprocates on the surface, and a suction unit coupled to the other side of the arm.

Description

Robot and its control method {ROBOT AND CONTROLLING METHOD THEROF}

The present invention relates to a robot and a method of controlling the same, and more specifically, to a robot that travels in space and performs a cleaning task and a method of controlling the same.

Recently, technology development for robots that are deployed in indoor spaces and provide services to users has become active. Robots can travel indoor spaces and provide a variety of services such as cleaning, guiding, serving, patrolling, or responding to emergency situations. In particular, the demand for robots that clean spaces on their own without user intervention is increasing day by day.

When a robot provides a cleaning service, there is a problem that blind spots where the robot cannot run may be formed in the space due to the structural characteristics of the space or obstacles located within the space, which makes it difficult to provide a smooth service. Accordingly, there has been a continuous demand for a method that can provide satisfactory cleaning services for various types of cleaning areas.

The present disclosure is in response to the above-described need, and the purpose of the present disclosure is to provide a robot that acquires type information about a cleaning target area based on image and distance data and performs cleaning based on a cleaning scenario corresponding to the obtained type information. and providing a control method thereof.

A robot according to an embodiment of the present invention to achieve the above object includes a memory storing a cleaning scenario and a neural network model according to the type of area to be cleaned, a camera, a distance sensor, a driving unit, an image acquired through the camera, and Inputting the distance data obtained through the distance sensor into the neural network model to obtain type information about at least one cleaning target area included in the image, and obtaining a cleaning scenario corresponding to the obtained type information from the memory. and a processor that controls the driving unit based on the obtained cleaning scenario, wherein the driving unit includes a motor, a plate designed to rotate, and a slope disposed on the plate and inclined at a certain angle with the ground. A rail provided, a slider arranged to be able to reciprocate on the rail, an arm whose one side is coupled to the slider and moves while maintaining a certain angle with the ground as the slider reciprocates on the rail, and the other arm of the arm It may include a suction part coupled to the side.

Here, when image and distance data are input, the neural network model is trained to output type information about at least one cleaning target area included in the image, or is trained to output information about the object included in the image. , when information about the object included in the image is obtained from the neural network model, the processor may obtain type information about the area to be cleaned included in the image based on the obtained information about the object.

In addition, the driving unit further includes a first wire coupled to one side of the suction unit and a second wire coupled to the first wire at a predetermined distance from the one side of the suction unit, and the processor is configured to operate the cleaning target area. Depending on the type, the suction part winds the first wire in a first direction so that the suction part maintains a first angle with the ground, and depending on the type of the cleaning target area, the suction part maintains a second angle with the ground. The second wire may be wound in a second direction so as to do so.

Here, when the object adjacent to the cleaning target area is identified as a movable object, the processor winds the first wire in the first direction so that the suction part contacts the object, and when the suction part contacts the object, the processor winds the first wire in the first direction. , the motor can be controlled so that the plate rotates in one direction.

Additionally, if the object is identified as an immovable object, the processor may provide a guide UI (User Interface) requesting movement of the object.

In addition, if the area to be cleaned is a specific type, the processor winds the first wire in the first direction, and if the area to be cleaned is a type other than the specific type, the processor winds the second wire in the second direction. Can be wound.

Additionally, when the level difference between the cleaning target area and the ground is greater than or equal to a threshold value, the processor may control the position of the suction unit toward the cleaning target area by moving the slider rearward.

Additionally, the distance sensor includes a LIDAR sensor, and the processor can control the slider based on distance data acquired through the LIDAR sensor while performing the cleaning.

Additionally, when an area that cannot be cleaned is identified in the image, the processor may provide a UI that guides the area that cannot be cleaned.

Meanwhile, the robot control method according to an embodiment of the present invention inputs the image acquired through the camera and the distance data obtained through the distance sensor into a neural network model to determine at least one cleaning target area included in the image. Obtaining type information, obtaining a cleaning scenario corresponding to the obtained type information, and controlling the driving unit to perform cleaning based on the obtained cleaning scenario, wherein the driving unit includes a motor, A plate designed to be rotatable, a rail disposed on the plate and having a slope inclined at a certain angle with the ground, a slider disposed to be reciprocatable on the rail, one side is coupled to the slider, and the slider is It may include an arm that moves while maintaining a constant angle with the ground as it reciprocates on the rail, and a suction unit coupled to the other side of the arm.

Here, when image and distance data are input, the neural network model is trained to output type information about at least one cleaning target area included in the image, or is trained to output information about the object included in the image. , The step of acquiring type information about the at least one cleaning target area includes, when information about the object included in the image is obtained from the neural network model, based on the obtained information about the object included in the image. Type information about the area to be cleaned can be obtained.

In addition, the driving unit further includes a first wire coupled to one side of the suction unit and a second wire coupled to the first wire at a predetermined distance from the one side of the suction unit, depending on the type of the area to be cleaned. Winding the first wire in a first direction so that the suction part maintains a first angle with the ground, and according to the type of the cleaning target area, the suction part maintains a second angle with the ground. 2 The step of winding the wire in the second direction may be further included.

Here, the step of winding the first wire in the first direction includes, when an object adjacent to the cleaning target area is identified as a movable object, winding the first wire in the first direction so that the suction unit contacts the object. The control method may further include rotating the plate in one direction when the suction unit contacts the object.

Additionally, if the object is identified as an immovable object, the method may further include providing a guide UI (User Interface) requesting movement of the object.

In addition, the step of winding the first wire in the first direction includes, if the area to be cleaned is a specific type, winding the first wire in the first direction and winding the second wire in the second direction. If the area to be cleaned is of a type other than the specific type, the second wire may be wound in the second direction.

Additionally, in the step of performing the cleaning, when the level difference between the cleaning target area and the ground is greater than or equal to a threshold value, the slider may be moved rearward to position the suction unit in the cleaning target area.

Additionally, the step of performing the cleaning may involve moving the slider based on distance data acquired through a LiDAR sensor while performing the cleaning.

Additionally, when an area that cannot be cleaned is identified in the image, a step of providing a UI to guide the area that cannot be cleaned may be further included.

According to various embodiments of the present disclosure, since a robot can provide satisfactory cleaning services for various types of cleaning areas, user convenience can be improved.

Figure 1 is a diagram schematically explaining the provision of cleaning services by a robot.
Figure 2 is a block diagram for explaining the configuration of a robot according to an embodiment of the present disclosure.
3A and 3B are diagrams for explaining the operation of an arm according to an embodiment of the present disclosure.
Figure 4 is a diagram for explaining the operation of the suction unit according to an embodiment of the present disclosure.
Figure 5 is a diagram for explaining a neural network model according to an embodiment of the present disclosure.
FIGS. 6A to 6D are diagrams for explaining various cleaning scenarios according to an embodiment of the present disclosure.
FIGS. 7A and 7B are diagrams for explaining the operation of a robot depending on whether an object can be moved according to an embodiment of the present disclosure.
Figure 8 is a diagram for explaining a cleaning operation of a robot in a step area according to an embodiment of the present disclosure.
Figure 9 is a block diagram for specifically explaining the configuration of a robot according to an embodiment of the present disclosure.
Figure 10 is a flowchart for explaining a control method according to an embodiment of the present disclosure.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

The terms used in the embodiments of the present disclosure have selected general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but this may vary depending on the intention or precedent of a technician working in the art, the emergence of new technology, etc. . In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description part of the relevant disclosure. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

In the present disclosure, expressions such as “have,” “may have,” “includes,” or “may include” refer to the presence of the corresponding feature (e.g., component such as numerical value, function, operation, or part). , and does not rule out the existence of additional features.

The expression at least one of A or/and B should be understood as referring to either “A” or “B” or “A and B”.

Expressions such as “first,” “second,” “first,” or “second,” used in the present disclosure can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components.

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as “connected to,” it should be understood that a certain component can be connected directly to another component or connected through another component (e.g., a third component).

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “consist of” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are intended to indicate the presence of one or more other It should be understood that this does not exclude in advance the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof.

In the present disclosure, a “module” or “unit” performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Additionally, a plurality of “modules” or a plurality of “units” are integrated into at least one module and implemented by at least one processor (not shown), except for “modules” or “units” that need to be implemented with specific hardware. It can be.

In this disclosure, 'user' may refer to a person who receives services from a robot, but is not limited thereto.

Figure 1 is a diagram schematically explaining the provision of cleaning services by a robot.

The robot 100 according to an embodiment of the present disclosure is placed in a specific space and can provide various services to users who live in or temporarily visit the space. The robot 100 can provide services such as cleaning, guiding, serving, patrolling, or responding to emergency situations to the user. However, in this specification, the operation of the robot 100 is based on the premise that the robot 100 provides cleaning services. Let me explain.

When the robot 100, which provides cleaning services while traveling in a space, is identified as being spaced within a critical distance from an object located in the space, it can perform cleaning based on a cleaning scenario for an area adjacent to the object. For example, the robot 100 may clean the corner 10 area based on a cleaning scenario corresponding to the corner 10 formed by the object.

For example, the robot 100 may acquire an image and distance data including the corner 10, and identify a cleaning area of the type of corner 10 included in the image based on the acquired image and distance data. . Additionally, the robot 100 may obtain a cleaning scenario corresponding to the identified area and perform cleaning based on the obtained cleaning scenario.

Additionally, the robot 100 may include a special type of driving unit that can variably adjust the cleaning range based on the main body of the robot 100 in order to effectively clean a specific type of cleaning target area. The robot 100 can effectively clean the identified cleaning target area by controlling the driving unit based on the acquired cleaning scenario.

Hereinafter, various embodiments of obtaining type information about a cleaning target area based on image and distance data and performing cleaning based on a cleaning scenario corresponding to the obtained type information will be described in more detail.

Figure 2 is a block diagram for explaining the configuration of a robot according to an embodiment of the present disclosure.

According to FIG. 2, the robot 100 according to an embodiment of the present disclosure may include a memory 110, a camera 120, a distance sensor 130, a driver 140, and a processor 150.

The memory 110 may store data necessary for various embodiments of the present disclosure. The memory 110 may be implemented as a memory embedded in the robot 100 or as a memory detachable from the robot 100 depending on the data storage purpose. For example, data for driving the robot 100 is stored in a memory embedded in the robot 100, and data for expansion functions of the robot 100 is stored in a memory that is detachable from the robot 100. It can be. Meanwhile, in the case of memory embedded in the robot 100, volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (e.g. : OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g. NAND flash or NOR flash, etc.) , a hard drive, or a solid state drive (SSD). In addition, in the case of memory that is removable from the robot 100, a memory card (e.g., compact flash (CF), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (e.g. For example, it may be implemented in a form such as USB memory).

The memory 110 according to one example may store a cleaning scenario according to the type of the cleaning target area and a neural network model used to obtain information on the type of the cleaning area.

According to one example, the neural network model may be a model learned to output type information about at least one cleaning target area included in the image when image and distance data are input. In this case, when image and distance data are input, the neural network model outputs type information on the area that can be cleaned (hereinafter referred to as the area to be cleaned) among multiple segment areas included in the image or areas where different segments are in contact. It may be a learned model.

According to another example, a neural network model may be a model learned to output information about objects included in the image when image and distance data are input. In this case, when image and distance data are input, the neural network model can output information about each of a plurality of segments included in the image.

The camera 120 is configured to acquire images including objects around the robot 100. The camera 120 may include a lens that focuses visible light and other optical signals reflected by an object and received into an image sensor, and an image sensor that can detect visible light and other optical signals. Here, the image sensor may include a 2D pixel array divided into a plurality of pixels.

The processor 150 may receive optical signals in various wavelength ranges, such as visible light or infrared light, through the lens of the camera 120, and process the received optical signals through the image sensor of the camera 120 to obtain an image. Additionally, the camera 120 according to one example may be implemented as a depth camera, and in this case, the processor 150 may acquire a depth image through the camera 120.

Meanwhile, the camera 120 may be implemented in the form of a camera module including a plurality of lenses and a plurality of image sensors. According to one example, the camera 120 may be a stereo camera including two lenses and two image sensors, but is not limited thereto.

The distance sensor 130 may acquire distance data. Specifically, the distance sensor 130 may measure the distance between the position of the robot 100 and the position of the object, and obtain distance data based on the measurement result. The distance sensor 130 according to an example may include at least one of LIDAR (Light Detection And Ranging), RADAR (RAdio Detection And Ranging), or SONAR (RAdio Detection And Ranging), but is limited thereto. no.

The driving unit 140 is configured to drive the robot 100 and provide cleaning services. Regarding the driving of the robot 100, the driving unit 140 may adjust the driving direction and driving speed according to the control of the processor 150. The driving unit 140 according to one example is a power generator that generates power for the robot 100 to travel (e.g., a gasoline engine, a diesel engine, or a liquefied petroleum gas (LPG) depending on the fuel (or energy source) used. ) engines, electric motors, etc.), steering devices to control the driving direction (e.g., manual steering, hydraulics steering, electronic control power steering, EPS, etc.), robots depending on power. It may include a traveling device (e.g., wheels, propeller, etc.) that drives (100). Additionally, the driving unit 140 may be modified depending on the driving type of the robot 100 (e.g., wheel type, walking type, flying type, etc.).

In relation to the cleaning function of the robot 100, the driving unit 140 can suction and remove foreign substances distributed in the cleaning target area under the control of the processor 150. The driving unit 140 according to an example includes a motor, a plate designed to be rotatable, a rail disposed on the plate and having a slope inclined at a certain angle with the ground, a slider arranged to be reciprocatable on the rail, and one side It may include an arm that is coupled to the slider and moves while maintaining a certain angle with the ground as the slider reciprocates on the rail, and a suction portion coupled to the other side of the arm.

Additionally, the driving unit 140 may include a first wire coupled to one side of the suction unit and a second wire coupled to one side of the suction unit at a predetermined distance from the first wire.

The processor 150 generally controls the operation of the robot 100. Specifically, the processor 150 is connected to each component of the robot 100 and can generally control the operation of the robot 100. For example, the processor 150 may be connected to the memory 110, the camera 120, the distance sensor 130, and the driver 140 to control the operation of the robot 100.

According to one embodiment, the processor 150 includes a digital signal processor (DSP), a microprocessor, a central processing unit (CPU), a micro controller unit (MCU), and a micro processing unit (MPU). It may be named by various names such as unit), NPU (Neural Processing Unit), controller, and application processor (AP), but in this specification, it is referred to as processor 150.

The processor 150 may be implemented as a System on Chip (SoC), large scale integration (LSI), or may be implemented as a Field Programmable Gate Array (FPGA). Additionally, the processor 150 may include volatile memory such as SRAM.

The processor 150 according to an embodiment of the present disclosure inputs the image acquired through the camera 120 and the distance data obtained through the distance sensor 130 into the neural network model stored in the memory 110 to obtain the image included in the image. Type information about at least one cleaning target area can be obtained.

According to one example, when image and distance data are input and a neural network model learned to output type information about at least one cleaning target area included in the image is used, the processor 150 outputs the result value through the neural network model. Based on this, type information about at least one cleaning target area included in the image can be obtained without separate post-processing. Here, the type of the area to be cleaned may be at least one of normal, shallow gap, corner, deep gap, narrow gap, or step area, but is not limited thereto.

According to another example, a neural network model may be a model learned to output information about objects included in the image when image and distance data are input. When using a neural network model learned in this way, the processor 150 generates type information for at least one cleaning target area included in the image through separate post-processing based on information about the object output through the neural network model. can be obtained. Specifically, the processor 150 may obtain type information of the area to be cleaned based on the relationship between objects corresponding to segments included in the image through post-processing.

Subsequently, the processor 150 may identify a cleaning scenario corresponding to the obtained type information among the plurality of cleaning scenarios stored in the memory 110. The processor 150 may perform cleaning of the area to be cleaned by controlling the driver 140 based on the identified cleaning scenario.

In addition, the processor 150 may wind the first wire in a first direction so that the suction unit maintains a first angle with the ground, or wind the second wire in a second direction so that the suction unit maintains a second angle with the ground, depending on the type of area to be cleaned. It can be wound in any direction. Here, when the object adjacent to the area to be cleaned is identified as a movable object, the processor 150 winds the first wire in the first direction so that the suction part contacts the object, and rotates the plate in one direction when the suction part contacts the object. The motor can be controlled.

Additionally, if an object is identified as an immovable object, the processor 150 may provide a guide UI (User Interface) to request movement of the object.

Additionally, the processor 150 may wind the first wire in a first direction if the area to be cleaned is identified as being a specific type, and may wind the second wire in a second direction if the area to be cleaned is of a type other than the specific type.

Additionally, if the level difference between the area to be cleaned and the ground is greater than or equal to a threshold value, the processor 150 may control the position of the suction unit to the area to be cleaned by moving the slider backward.

In addition, when the distance sensor 130 includes a LIDAR sensor, the processor 150 obtains type information about the area to be cleaned based on distance data acquired through the LIDAR sensor while cleaning. And, the slider can be controlled based on the cleaning scenario corresponding to the acquired type information.

Additionally, when an area that cannot be cleaned is identified in the image, the processor 150 may provide a UI that guides the area that cannot be cleaned. Here, the uncleanable area refers to an area where cleaning cannot be performed at all, considering the form factor of the object itself and the robot 100 included in the image, as well as an area where problems such as damage to the object may occur when cleaning is performed. It may include, but is not limited to this.

3A and 3B are diagrams for explaining the operation of an arm according to an embodiment of the present disclosure.

The driving unit 140 may include a motor. The driving unit 140 may include a single motor related to the driving and cleaning functions of the robot 100, or may include a first motor related to driving and a second motor related to the cleaning function.

The driving unit 140 may include a plate 142 designed to be rotatable. The plate 142 may be designed to be rotatable based on a direction perpendicular to the ground, and a rail 143 having a slope inclined at a certain angle to the ground may be disposed on the plate 142. According to one example, the rail 143 may be implemented as a pair of straight rails, but is not limited thereto.

The rail 143 may be coupled to the slider 144, and parts such as bearings may be included in the joint portion of the rail 143 and the slider 144. The slider 144 is arranged to be reciprocatable on the rail 143, and the slider 144 according to one example may include a pair of coupling parts corresponding to a pair of rails.

The slider 144 is combined with the arm 145. The arm 145 can move while maintaining a constant angle with the ground as the slider 144 reciprocates on the rail 143. In this case, the arm 145 can move along the slope included in the rail 143 while maintaining a constant angle with the ground. One side of the arm 145 coupled to the slider 144 may include a hinge, etc. to help the arm 145 operate smoothly.

A suction unit 146 may be coupled to the other side of the arm 145. The suction unit 146 is a suction unit for sucking and removing foreign substances distributed in the area to be cleaned, and increases friction between the suction unit 146 and the object when the suction unit 146 contacts an object adjacent to the area to be cleaned. It may include non-slip members, etc.

The arm 145 may include a wire 147 for controlling the operation of the suction unit 146. For example, the wire 147 may include a first wire coupled to one side of the suction portion 146 and a second wire coupled to one side of the suction portion 146 at a certain distance from the first wire. It is not limited to this.

According to FIG. 3A, the processor 150 may control the slider 144 to move in the opposite direction of the arm 145. In this case, the arm 145 moves together in the direction of movement of the slider 144, and as a result, the length of the arm 145 protruding out of the rail 143 can be shortened. That is, as the slider 144 retreats, the range that the robot 100 can clean based on the position of the robot 100 may decrease.

Additionally, as the slider 144 retracts, the distance between the suction unit 146 and the ground may increase. In this case, since the suction of foreign substances by the suction unit 146 may not be smoothly achieved, the processor 150 controls the wire 147 through the motor 141 to bring the suction unit 146 closer to the ground. The position of unit 146 can be adjusted.

According to FIG. 3B, the processor 150 can control the slider 144 to move in the direction of the arm 145. In this case, the arm 145 moves together in the direction of movement of the slider 144, and as a result, the length of the arm 145 protruding out of the rail 143 may be extended. That is, as the slider 144 moves forward, the range that the robot 100 can clean may increase based on the position of the robot 100.

Additionally, as the slider 144 moves forward, the distance between the suction unit 146 and the ground may decrease. In this case, because the suction unit 146 may be temporarily bonded to the ground or friction between the suction unit 146 and the ground may occur during driving, the processor 150 controls the wire 147 through the motor 141 to The position of the suction unit 146 can be adjusted so that the suction unit 146 is away from the ground.

Figure 4 is a diagram for explaining the operation of the suction unit according to an embodiment of the present disclosure.

According to FIG. 4, the angle between the ground and the suction unit 146 can be adjusted by winding the wire 147 by the motor 141. For example, the suction unit 146 may be driven to have a range of a first angle or more and a second angle or less with respect to the ground. The suction unit 146 may maintain an angle (hereinafter, the first angle) smaller than the angle at which the slope is inclined relative to the ground according to the winding of the wire 147 by the motor 141, and the angle at which the slope is inclined A smaller angle (hereinafter referred to as the second angle) may be maintained. When the suction unit 146 maintains a first angle with the ground, the robot 100 can perform an operation to move an object adjacent to the cleaning target area, and the suction unit 146 maintains a second angle with the ground. In this case, the robot 100 can provide normal cleaning services.

In the process of the robot 100 providing a normal cleaning service, the suction unit 146 may maintain a state 401 bent downward from the incline direction of the slope.

According to one example, the processor 150 may identify whether an object adjacent to the cleaning target area needs to be moved based on type information of the cleaning target area. When it is identified that the object needs to be moved, the processor 150 winds the first wire coupled to one side of the suction unit 146 in a first direction so that the suction unit 146 maintains a first angle with the ground. The motor 141 can be controlled. In this case, the suction unit 146 may maintain a state 402 bent upward from the slope direction of the slope.

In addition, when the object adjacent to the cleaning target area is identified as a movable object, the processor 150 may control the motor 141 to wind the first wire in the first direction so that the suction unit 146 contacts the object. . The processor 150 may move the object by controlling the motor 141 so that the plate 142 rotates in one direction when the suction unit 146 contacts the object.

Additionally, the processor 150 may wind the first wire in the first direction if the area to be cleaned is of a specific type. For example, if the area to be cleaned is identified as a ‘narrow gap,’ the processor 150 may wind the first wire in the first direction and then move an object adjacent to the area to be cleaned. If the area to be cleaned is of a type other than a specific type, the processor 150 may provide a normal cleaning service by winding the second wire in the first direction.

Figure 5 is a diagram for explaining a neural network model according to an embodiment of the present disclosure.

The function related to obtaining type information about the area to be cleaned according to the present disclosure is operated through the processor 150 and memory 110. The processor 150 may be comprised of one or multiple processors. At this time, one or more processors may be a general-purpose processor such as a CPU, AP, or DSP (Digital Signal Processor), a graphics-specific processor such as a GPU or VPU (Vision Processing Unit), or an artificial intelligence-specific processor such as an NPU. One or more processors may process input data according to the neural network model stored in the memory 110. If one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor is specialized for processing a specific neural network model. It can be designed as a hardware structure.

The neural network model 500 stored in the memory 110 is characterized by being created through learning. Here, being created through learning means that the basic neural network model is learned using a large number of learning data by a learning algorithm, thereby creating a predefined operation rule or neural network model set to perform the desired characteristics (or purpose). do. Such learning may be performed on the device itself on which various embodiments according to the present disclosure are performed, or may be performed through a separate server and/or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

The neural network model 500 may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and neural network calculation is performed through calculation between the calculation result of the previous layer and the plurality of weights. The plurality of weights possessed by the plurality of neural network layers can be optimized based on the learning results of the neural network model. For example, a plurality of weights may be updated so that loss or cost values obtained from the neural network model are reduced or minimized during the learning process. Artificial neural networks may include deep neural networks (DNN), for example, Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), or Deep Q-Networks, etc., but are not limited to the examples described above.

According to an embodiment of the present disclosure, the neural network model 500 generates a segment included in the image when the image 511 acquired through the camera 120 and the distance data 512 acquired through the distance sensor 130 are input. It may be a model learned to output (521) and type of segment (522). Here, the type of segment 522 may be information about an object included in the image 511, such as a floor, wall, slope, or obstacle, and may be normal, shallow gap, corner, deep gap, narrow gap, or step. It may be type information about at least one cleaning target area included in the image 511, such as an area.

FIGS. 6A to 6D are diagrams for explaining various cleaning scenarios according to an embodiment of the present disclosure.

According to FIG. 6A, the robot 100 may identify the type of area to be cleaned as a ‘shallow gap’ and perform cleaning based on a cleaning scenario corresponding to the shallow gap 610. For example, the robot 100 may perform cleaning by traveling along the shallow gap 610 with the suction unit 146 positioned on the shallow gap 610.

According to FIG. 6B, the robot 100 identifies the type of area to be cleaned as 'corner' and performs cleaning based on the cleaning scenario corresponding to the corner 620. For example, cleaning can be performed by rotating the plate 142 while maintaining the suction unit 146 of the robot 100 close to the object constituting the corner 620. In this process, the processor 150 may reciprocate the slider 144 through the motor 141 to variably control the range that the robot 100 can clean based on the position of the robot 100.

According to FIG. 6C, the robot 100 identifies the type of area to be cleaned as ‘deep crevice’ and performs cleaning based on the cleaning scenario corresponding to the deep crevice 630. For example, the robot 100 may control the motor 141 so that the plate 142 alternately rotates in the first direction or the second direction. When the robot 100 travels along the deep gap 630 at the same time as the alternating rotation motion of the plate 142, the degree to which the suction part 146 enters the deep gap 630 continuously changes and cleans. Because cleaning is performed, cleaning of a wider area can be performed in a short period of time.

According to FIG. 6D, the robot 100 identifies the type of area to be cleaned as ‘narrow gap’ and performs cleaning based on the cleaning scenario corresponding to the narrow gap 640. For example, the robot 100 may reciprocate the slider 144 to clean the area between the start and end points of the narrow gap. Through this, the arm 145 can continuously protrude or recover from the main body of the robot 100, thereby enabling efficient cleaning of the narrow gap 640.

FIGS. 7A and 7B are diagrams for explaining the operation of a robot depending on whether an object can be moved according to an embodiment of the present disclosure.

According to FIG. 7A, the robot 100 can clean a cleaning target area where the movable object 710 is adjacent based on the specifications of the robot 100. During the cleaning process, the robot 100 may identify whether the object 710 is a movable object. For example, the robot 100 inputs image and distance data into a neural network model and identifies the object 710 as a movable object based on the output results, and the suction unit 146 contacts the object 710. If possible, the wire 147 can be controlled through the motor 141.

When the suction unit 146 contacts an object, the processor 150 may control the motor 141 to rotate the plate 142 in one direction to move the object 710. Thereafter, the processor 150 may control the driving unit 140 to clean the area adjacent to the existing location of the object 710.

According to FIG. 7B, the robot 100 may further include a user interface 160. The robot 100 inputs image and distance data into a neural network model in the process of cleaning a cleaning target area adjacent to an immovable object 720 based on the specifications of the robot 100, and cleans the object 720 based on the output results. is identified as an object that cannot be moved, and a guide UI 700 requesting movement of the object 720 may be provided through the user interface 160 .

The guide UI 700 according to an example may include information such as text that alerts the user to move the object 720, a confirmation button 701, and an ignore button 702. Here, when the user selects the confirm button 701, the robot 100 may stop driving and wait until the object 720 is moved. Conversely, when the user selects the ignore button 702, the robot 100 may leave the current location to clean the next cleaning target area regardless of the movement of the object 720.

Figure 8 is a diagram for explaining a cleaning operation of a robot in a step area according to an embodiment of the present disclosure.

According to FIG. 8, in the space where the robot 100 is located, an object 800 forming an area having a level difference greater than a threshold value from the ground (hereinafter referred to as a level difference area) may be disposed. The processor 150 according to an example may control the position of the suction unit 146 to the step area by moving the slider 144 rearward when the step area, which is the cleaning target area, is identified as having a step difference greater than a threshold value from the ground. there is.

In this process, the arm 145 is recovered in the direction of the main body of the robot 100 along the slope included in the rail 143, so the distance between the suction part 146 and the ground increases, and accordingly, the suction part 146 It can be maintained at an appropriate distance from the step area.

Meanwhile, of course, in addition to the above-described method, the robot 100 can control the motor 141 by winding the wire 147 so that the suction part 146 is maintained at an appropriate distance from the step area.

Figure 9 is a block diagram for specifically explaining the configuration of a robot according to an embodiment of the present disclosure.

According to FIG. 9, the robot 100 may include a memory 110, a camera 120, a distance sensor 130, a driver 140, a processor 150, a user interface 160, and a communication interface 170. You can. Among the configurations shown in FIG. 9, detailed descriptions of configurations that overlap with those shown in FIG. 2 will be omitted.

The camera 120 may be implemented in the form of a camera module including an RGB camera 121 and an infrared camera 122. According to one example, the camera 120 may include an RGB camera 121 including a visible light sensor and an infrared camera 122 including an infrared sensor. In this case, the processor 150 uses the RGB camera 121. The visible light image obtained through the infrared camera 122, the infrared image obtained through the infrared camera 122, and the distance data can be input into the neural network model to obtain type information about the area to be cleaned.

The distance sensor 130 may include LIDAR 131, RADAR 132, and SONAR 133.

LIDAR (131) can fire a high-output laser pulse and measure the time it takes for the laser to reach the object and then reflect back (Tof, Time of Flight method). Through this, LIDAR 131 can acquire distance data to the object. Additionally, LIDAR can also obtain data about the shape of an object based on a point cloud corresponding to the object surface area. The processor 150 controls the motor 141 so that the cleaning range is adjusted within the range where the suction unit 146 does not collide with the object based on the distance data acquired through the LIDAR 131 while the robot 100 performs cleaning. ) can be used to control the slide 144.

RADAR 132 can acquire distance data to an object in the same way as LIDAR 131 (ToF). However, unlike the LIDAR 131, the RADAR 132 uses radio waves and can obtain distance data to the object by measuring the time the radio waves take to reflect and return to the object.

SONAR (133) is equipment that can acquire distance data to an object using sound waves. SONAR 133 according to one example may have a cone-shaped angle of view, and may obtain distance data to the object by measuring the time it takes for a sound wave to reflect and return to an object.

The processor 150 provides type information about the area to be cleaned based on the image acquired through the camera 120 and a plurality of distance data respectively acquired through LIDAR 131, RADAR 132, and SONAR 133. It can be obtained.

The driving unit 140 may include a motor 141, a plate 142, a rail 143, a slider 144, an arm 145, a suction unit 146, and a wire 147. Since each component included in the driving unit 140 has been described above, detailed description will be omitted here.

The user interface 160 is a component that helps the robot 100 interact with a user. For example, the user interface 160 may include, but is not limited to, at least one of a display, a touch screen, a button, a jog dial, a switch, a proximity sensor, various types of biometric sensors, a microphone, or a speaker. If it is impossible to move an object adjacent to the cleaning target area, the processor 150 may provide a guide UI requesting movement of the object through the user interface 160, and may provide a guide UI requesting movement of the object in the image acquired through the camera 120. Once the area is identified, a UI that guides the area that cannot be cleaned can be provided through the user interface 160.

The communication interface 170 can input and output various types of data. For example, the communication interface 170 is AP-based Wi-Fi (Wireless LAN network), Bluetooth, Zigbee, wired/wireless LAN (Local Area Network), and WAN (Wide Area Network). , Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES/EBU (Audio Engineering Society/ European Broadcasting Union), Optical ), coaxial, etc. to external devices (e.g. source devices), external storage media (e.g. USB memory), external servers (e.g. web hard drives) and various types of data. can be sent and received.

The processor 150 may provide various UIs through the user interface 160, but may also provide various UIs by transmitting UI information to the user terminal through the communication interface 160. In addition, when the operation of acquiring type information of the area to be cleaned using the neural network model according to the present disclosure is performed through an external device (e.g., a server, etc.), the processor 150 may use the image acquired through the camera 120 and the distance sensor. The communication interface 170 may be controlled to transmit the distance data obtained through 130 to an external device, and information on the type of area to be cleaned may be received from the external device through the communication interface 170.

Figure 10 is a flowchart for explaining a control method according to an embodiment of the present disclosure.

The control method according to an embodiment of the present disclosure inputs an image acquired through a camera and distance data obtained through a distance sensor into a neural network model to obtain type information about at least one cleaning target area included in the image ( S1010).

Next, a cleaning scenario corresponding to the obtained type information is acquired (S1020).

Finally, cleaning can be performed by controlling the driving unit based on the obtained cleaning scenario (S1030).

Here, the driving part includes a motor, a plate designed to rotate, a rail disposed on the plate and having a slope inclined at a certain angle with the ground, a slider arranged to be able to reciprocate on the rail, and one side is coupled to the slider and the slider It may include an arm that moves while maintaining a constant angle with the ground as it moves back and forth on the rail, and a suction portion coupled to the other side of the arm.

Here, when image and distance data are input, the neural network model is trained to output type information about at least one cleaning target area included in the image, or is trained to output information about objects included in the image, and is trained to output at least one cleaning target area. In the step of acquiring type information about the target area (S1010), when information about the object included in the image is obtained from the neural network model, type information about the cleaning target area included in the image is acquired based on the information about the acquired object. can do.

In addition, the driving unit further includes a first wire coupled to one side of the suction unit and a second wire coupled to one side of the suction unit at a predetermined distance from the first wire. In this case, the control method may be used to control the suction unit according to the type of the area to be cleaned. Winding the first wire in a first direction to maintain a first angle with the ground and winding the second wire in a second direction so that the suction portion maintains a second angle with the ground depending on the type of area to be cleaned. Additional steps may be included.

Here, the step of winding the first wire in the first direction includes winding the first wire in the first direction so that the suction unit contacts the object when the object adjacent to the cleaning target area is identified as a movable object. In this case, the control method is suction. The method may further include rotating the plate in one direction when it comes into contact with an additional object.

Additionally, if the object is identified as an immovable object, the step of providing a guide UI (User Interface) to request movement of the object may be further included.

In addition, the step of winding the first wire in the first direction is winding the first wire in the first direction when the area to be cleaned is a specific type, and the step of winding the second wire in the second direction is to do so when the area to be cleaned is a specific type. For other types, the second wire can be wound in the second direction.

Additionally, in the cleaning step (S1030), if the level difference between the cleaning target area and the ground is greater than a critical value, the slider can be moved backward to position the suction unit in the cleaning target area.

Additionally, in the cleaning step (S1030), the slider can be moved based on distance data acquired through the LiDAR sensor while cleaning.

Additionally, when an area that cannot be cleaned is identified in the image, a step of providing a UI to guide the area that cannot be cleaned may be further included.

Additionally, the methods according to various embodiments of the present disclosure described above can be implemented only by upgrading software or hardware for an existing robot.

Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in a robot or at least one external server.

Meanwhile, the various embodiments described above may be implemented in a recording medium that can be read by a computer or similar device using software, hardware, or a combination thereof. In some cases, embodiments described herein may be implemented by the processor 150 itself. According to software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

Meanwhile, computer instructions for performing processing operations of the robot 100 according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. . Computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations in the robot 100 according to the various embodiments described above.

A non-transitory computer-readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specific examples of non-transitory computer-readable media may include CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

In the above, preferred embodiments of the present disclosure have been shown and described, but the present disclosure is not limited to the specific embodiments described above, and may be used in the technical field pertaining to the disclosure without departing from the gist of the disclosure as claimed in the claims. Of course, various modifications can be made by those with the knowledge, and these modifications should not be understood individually from the technical ideas or perspectives of the present disclosure.

100: Robot 110: Memory
120: Camera 130: Distance sensor
140: driving unit 141: motor
142: plate 143: rail
144: Slider 145: Arm
146: Suction part 147: Wire
150: processor

Claims (18)

In robots,
A memory storing cleaning scenarios and neural network models according to the type of cleaning target area;
camera;
distance sensor;
driving part; and
Input the image acquired through the camera and the distance data obtained through the distance sensor into the neural network model to obtain type information about at least one cleaning target area included in the image,
Obtain a cleaning scenario corresponding to the acquired type information from the memory,
It includes a processor that controls the driving unit based on the obtained cleaning scenario,
The driving unit,
motor;
Plate designed to be rotatable;
a rail disposed on the plate and having a slope inclined at a certain angle with the ground;
a slider arranged to move back and forth on the rail;
An arm whose one side is coupled to the slider and moves while maintaining a constant angle with the ground as the slider reciprocates on the rail; and
A robot comprising a suction unit coupled to the other side of the arm. According to paragraph 1,
The neural network model is,
When image and distance data are input, it is learned to output type information about at least one cleaning target area included in the image,
Learned to output information about objects included in the image,
The processor,
When information about the object included in the image is obtained from the neural network model, the robot acquires type information about the cleaning target area included in the image based on the obtained information about the object. According to paragraph 1,
The driving unit,
a first wire coupled to one side of the suction unit; and
It further includes a second wire coupled to the first wire at a predetermined distance apart from the first wire on one side of the suction unit,
The processor,
Winding the first wire in a first direction so that the suction unit maintains a first angle with the ground according to the type of the area to be cleaned,
A robot that winds the second wire in a second direction so that the suction unit maintains a second angle with the ground according to the type of the cleaning target area. According to paragraph 3,
The processor,
When an object adjacent to the cleaning target area is identified as a movable object, winding the first wire in the first direction so that the suction unit contacts the object,
A robot that controls the motor so that the plate rotates in one direction when the suction unit contacts the object. According to paragraph 3,
The processor,
A robot that provides a guide UI (User Interface) to request movement of the object when the object is identified as an immovable object. According to paragraph 3,
The processor,
If the area to be cleaned is a specific type, winding the first wire in the first direction,
If the area to be cleaned is of a type other than the specific type, the robot winds the second wire in the second direction. According to paragraph 1,
The processor,
A robot that moves the slider rearward to control the position of the suction unit to the cleaning target area when the level difference between the cleaning target area and the ground is greater than a threshold value. According to paragraph 1,
The distance sensor is,
Includes a LIDAR sensor,
The processor,
A robot that controls the slider based on distance data acquired through the lidar sensor while performing the cleaning. According to paragraph 1,
The processor,
When an area that cannot be cleaned is identified in the image, a robot provides a UI to guide the area that cannot be cleaned. In the control method of a robot including a driving unit,
Inputting an image acquired through a camera and distance data obtained through a distance sensor into a neural network model to obtain type information about at least one cleaning target area included in the image;
Obtaining a cleaning scenario corresponding to the obtained type information; and
A step of controlling the driving unit to perform cleaning based on the obtained cleaning scenario,
The driving unit,
A motor, a plate designed to be rotatable, a rail disposed on the plate and having a slope inclined at a certain angle with the ground, a slider disposed to be reciprocatable on the rail, one side of which is coupled to the slider and the slider A control method comprising an arm that moves while maintaining a constant angle with the ground as it reciprocates on the rail, and a suction portion coupled to the other side of the arm. According to clause 10,
The neural network model is,
When image and distance data are input, it is learned to output type information about at least one cleaning target area included in the image,
Learned to output information about objects included in the image,
The step of obtaining type information about the at least one cleaning target area includes:
When information about the object included in the image is obtained from the neural network model, type information about the area to be cleaned included in the image is obtained based on the obtained information about the object. According to clause 10,
The driving unit,
It further includes a first wire coupled to one side of the suction portion and a second wire coupled to the first wire at a predetermined distance from the one side of the suction portion,
Winding the first wire in a first direction so that the suction unit maintains a first angle with the ground according to the type of the area to be cleaned; and
The control method further comprising: winding the second wire in a second direction so that the suction unit maintains a second angle with the ground according to the type of the area to be cleaned. According to clause 12,
The step of winding the first wire in the first direction,
When an object adjacent to the cleaning target area is identified as a movable object, winding the first wire in the first direction so that the suction unit contacts the object,
The control method is,
A control method further comprising: rotating the plate in one direction when the suction unit contacts the object. According to clause 12,
If the object is identified as an immovable object, providing a guide UI (User Interface) requesting movement of the object. According to clause 12,
The step of winding the first wire in the first direction,
If the area to be cleaned is a specific type, winding the first wire in the first direction,
The step of winding the second wire in the second direction,
If the area to be cleaned is of a type other than the specific type, the control method includes winding the second wire in the second direction. According to clause 10,
The steps for performing the cleaning are:
When the level difference between the cleaning target area and the ground is greater than or equal to a threshold value, the control method moves the slider rearward to position the suction unit in the cleaning target area. According to clause 10,
The steps for performing the cleaning are:
A control method for moving the slider based on distance data acquired through a lidar sensor while performing the cleaning. According to clause 10,
When an area that cannot be cleaned is identified in the image, providing a UI that guides the area that cannot be cleaned.

Images