KR20180134155A - Robot cleaner and robot cleaning system - Google Patents

Abstract

The present invention relates to an artificial intelligence (AI) robot cleaner capable of correctly recognizing the presence of an obstacle and a robot cleaning system including the same. To achieve this, according to one embodiment of the present invention, an autonomous driving cleaner comprises: a main body; a driving unit to move the main body; a sensor to detect information related to environments including the main body; a communication unit to transmit or receive information to or from either or both of a mobile terminal and a server; a memory to store training data for analyzing the information detected in the sensor; and a control unit determining whether the main body enters a dangerous situation based on the information detected in the sensor, using the training data and an analysis algorithm stored in the memory to analyze the information related to the dangerous situation when it is determined that the main body enters the dangerous situation, and setting an operation mode corresponding to the dangerous situation.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a robot cleaner,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a cleaner and a control method thereof, and more particularly, to a cleaner capable of recognizing an obstacle and performing autonomous traveling, and a robot cleaning system having such a cleaner.

In general, robots have been developed for industrial use and have been part of factory automation. In recent years, medical robots, aerospace robots, and the like have been developed, and household robots that can be used in ordinary homes are being developed.

A representative example of the domestic robot is a robot cleaner, which is a type of household appliance that sucks and cleanes dust or foreign matter around the robot while traveling in a certain area by itself. Such a robot cleaner is generally equipped with a rechargeable battery and has an obstacle sensor capable of avoiding obstacles during traveling, so that it can run and clean by itself.

In recent years, research has been actively carried out to utilize the robot cleaner in various fields such as health care, smart home, and remote control by moving away from the cleaning operation by merely autonomously moving the cleaning area by the robot cleaner.

In addition, the robot cleaner may collect various information, and may communicate with the external network or the mobile terminal, thereby processing the collected information in various ways.

For example, the camera of the robot cleaner photographs the vicinity of the body of the robot cleaner, and the controller of the robot cleaner can detect information related to the obstacles existing in the photographed image.

In this regard, Korean Patent Laid-open Publication No. 10-2013-0027355 (published on March 15, 2013) discloses a remote control system for a robot cleaner capable of variously controlling a robot cleaner by connecting a terminal device and a robot cleaner through a network System.

However, in the robot cleaner system according to Korean Patent Laid-Open No. 10-2013-0027355, since the robot cleaner does not utilize the machine learning technique in detecting the obstacles in the vicinity, it is difficult to accurately detect obstacles .

That is, the robot cleaner included in the general robot cleaning system detects information related to the obstacle around the robot based only on the algorithm or the comparison group data stored in advance in the memory of the robot cleaner by the user or the designer, so that a new type of obstacle The obstacle information can not be detected accurately.

In addition, if the robot cleaner can not accurately detect the obstacle information, the obstacle avoidance operation can not be performed smoothly, and the cleaning operation can not be efficiently performed for the predetermined cleaning area.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a robot having a machine learning function capable of accurately recognizing the presence or absence of an obstacle around a main body of the robot cleaner without being limited to the initial performance of a robot cleaner designed by a designer or a user And a robot cleaning system having the robot cleaner.

The present invention also provides a robot cleaner equipped with a machine learning function for improving the performance of recognizing an obstacle of a robot cleaner as the use cases are accumulated by sharing data related to use cases of the robot cleaner, And a robot cleaning system.

According to another aspect of the present invention, there is provided a robot cleaner system for performing a machine learning, the robot cleaner comprising: a robot cleaner for providing a robot cleaner with analysis results related to a dangerous situation occurring in the vicinity of the robot cleaner, A robot cleaner equipped with a machine learning function capable of performing obstacle recognition requiring a large amount of data processing, and a robot cleaning system equipped with the robot cleaner.

The present invention also provides a robot cleaner equipped with a machine learning function for recognizing an object existing in a part of the peripheral region of the robot cleaner while reflecting the user's intention while the robot cleaner transmits the monitoring result to the mobile terminal. And a robot cleaning system having the same.

In order to solve the technical problems of the present invention, a vacuum cleaner according to the present invention includes a main body, a driving unit for moving the main body, a sensor for detecting information related to an environment where the main body is located, A memory for storing training data for analyzing the information sensed by the sensor, and a sensor for receiving or transmitting information with at least one of the sensors, and determining whether the main body enters a dangerous state based on the information sensed by the sensor And analyzing information related to the dangerous situation using the training data and the analysis algorithm stored in the memory and determining an operation mode corresponding to the dangerous situation based on the analysis result And a control unit.

In one embodiment, the control unit controls the communication unit to transmit a training data update request to the server to update the training data, when it is determined that the main body enters a dangerous situation.

In one embodiment, the control unit updates the training data using the training data update information received from the server, in response to the training data update request.

In one embodiment, the control unit analyzes the information related to the dangerous situation using the updated training data.

In one embodiment, the control unit uses the updated training data to identify information related to a subject of a photographed image when a dangerous situation is detected.

In one embodiment, the controller identifies information related to the type of the obstacle detected when a dangerous situation is detected by using the updated training data.

In one embodiment, the control unit sets an operation mode corresponding to the dangerous situation based on an analysis result of information related to the dangerous situation.

In one embodiment, the controller sets a first operation mode for avoiding the obstacle when it is determined that an obstacle exists on the traveling path of the cleaner, based on the analysis result.

In one embodiment of the present invention, when it is determined based on the analysis result that there is an object or organism obstructing the driving of the vacuum cleaner in the cleaning area of the vacuum cleaner, Is set.

In one embodiment, if it is determined that there is a dangerous object or organism in the cleaning area of the cleaner based on the analysis result, And a third operation mode for providing information is set.

In one embodiment, the sensor includes an encoder for sensing information related to the operation of the driving unit, and the controller determines that a dangerous situation has occurred in the cleaner based on the detection result of the encoder.

In one embodiment, the controller determines that a dangerous situation has occurred in the cleaner based on the detection result of the encoder, when it is determined that the main body has deviated from the traveling path.

In one embodiment, the control unit determines that a dangerous situation has occurred in the cleaner when wheels of the driving unit are restrained or idle based on the detection result of the encoder.

In one embodiment, the sensor includes a camera for photographing an image related to the periphery of the main body, and the control unit determines that a dangerous situation has occurred in the cleaner based on the image photographed by the camera .

In one embodiment, the control unit determines that a dangerous situation occurs in the cleaner when a difference is detected between a plurality of images photographed by the camera at a predetermined time interval while the main body is stopped. do.

In one embodiment, the control unit controls the communication unit to transmit an analysis algorithm update request to the server, in order to update the analysis algorithm, when it is determined that the main body enters a dangerous situation.

In one embodiment, the control unit updates the analysis algorithm using analysis algorithm update information received from the server, in response to the analysis algorithm update request.

In addition, the robot cleaner system according to the present invention may include a vacuum cleaner that performs autonomous traveling and a server that communicates with the vacuum cleaner.

In one embodiment, the vacuum cleaner includes a body, a driver for moving the body, a sensor for sensing information related to the environment in which the body is located, a communication unit for communicating with the server, And a control unit for controlling the communication unit to transmit information related to the dangerous situation to the server when it is determined that the main body has entered a dangerous situation and the main body has entered a dangerous situation, Analyzing information related to a dangerous situation transmitted from the cleaner using the training data stored in the server, transmitting the analysis result to the cleaner, and using the information related to the dangerous condition transmitted from the cleaner, And the control unit of the cleaner updates the analysis result received from the server And a control unit for setting an operation mode corresponding to the dangerous situation on the basis of the operation mode.

In one embodiment, the control unit of the cleaner controls the communication unit such that information related to an object existing around the main body obtained by the sensor after the execution of the set operation mode is completed is transmitted to the server, And the accuracy of the analysis result is determined using information related to an object existing around the main body received from the cleaner after execution of the set operation mode is completed.

In one embodiment, the mobile terminal further includes a mobile terminal that communicates with the cleaner and the server, and the mobile terminal outputs monitoring information received from the cleaner, and when the monitoring information is output, And transmits information related to the dangerous situation to the server based on the user input.

According to the present invention, the robot cleaner is not limited in performance in manufacturing or designing, and an effect of recognizing an obstacle present around the main body is derived.

In addition, according to the present invention, since the robot cleaner can perform an appropriate counter operation against a dangerous situation generated around the main body by using machine learning, the operation performance of the robot cleaner is improved.

According to the present invention, even when the control unit of the robot cleaner has a performance that is not sufficient to perform machine learning, the obstacle recognition result using the machine learning can be obtained by the server performing the communication with the robot cleaner . That is, it is possible to improve the obstacle recognition performance of the robot cleaner without increasing the manufacturing cost of the robot cleaner.

In addition, according to the present invention, as the data related to the use case of the robot cleaner is accumulated, the robot cleaner can accurately determine the substance of the dangerous situation generated around the main body. That is, according to the present invention, the obstacle recognition performance of the robot cleaner can be improved.

In addition, according to the present invention, while the user of the robot cleaner performs remote monitoring of the place where the robot cleaner is present through the mobile terminal, whether or not a risk element exists in the place can be informed to the user, Can be improved.

1 is a perspective view illustrating an example of a vacuum cleaner that performs an autonomous running according to the present invention.
FIG. 2 is a plan view of the vacuum cleaner shown in FIG. 1; FIG.
FIG. 3 is a side view of the vacuum cleaner shown in FIG. 1; FIG.
4 is a block diagram illustrating components of a vacuum cleaner that performs autonomous traveling according to an embodiment of the present invention.
5 is a conceptual diagram showing a system of a vacuum cleaner that performs autonomous traveling according to an embodiment of the present invention.
6 is a flowchart showing a control method of a system of a vacuum cleaner that performs autonomous traveling according to an embodiment of the present invention.
7 is a flowchart showing a control method of a system of a vacuum cleaner that performs autonomous traveling according to an embodiment of the present invention.
8 is a flowchart showing a control method of a system of a vacuum cleaner that performs autonomous traveling according to an embodiment of the present invention.
9 is a flowchart showing a control method of a system of a vacuum cleaner that performs autonomous traveling according to an embodiment of the present invention.
10 is a flowchart showing a control method of a system of a vacuum cleaner that performs autonomous traveling according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. .

FIG. 1 is a perspective view showing an example of a robot cleaner 100 according to the present invention, FIG. 2 is a plan view of the robot cleaner 100 shown in FIG. 1, and FIG. 3 is a schematic view of the robot cleaner 100 shown in FIG. Fig.

For reference, in this specification, a mobile robot, a robot cleaner, and a vacuum cleaner that performs autonomous travel may be used in the same sense.

Referring to FIGS. 1 to 3, the robot cleaner 100 performs a function of cleaning a floor while traveling on a certain area by itself. Cleaning of the floor includes suction of dust on the floor (including foreign matter) or mopping the floor.

The robot cleaner 100 includes a cleaner main body 110, a suction unit 120, a sensing unit 130, and a dust box 140.

The cleaner main body 110 is provided with a control unit (not shown) for controlling the robot cleaner 100 and a wheel unit 111 for traveling the robot cleaner 100. The robot cleaner 100 can be moved forward or backward or to the left or to the right by the wheel unit 111.

The wheel unit 111 includes a main wheel 111a and a sub wheel 111b.

The main wheel 111a is provided on both sides of the cleaner main body 110 and is configured to be rotatable in one direction or another direction according to a control signal of the control unit. Each of the main wheels 111a can be configured to be drivable independently of each other. For example, each main wheel 111a may be driven by a different motor.

The sub wheel 111b supports the cleaner main body 110 together with the main wheel 111a and assists the running of the robot cleaner 100 by the main wheel 111a. The sub wheel 111b may also be provided in the suction unit 120 described later.

As described above, the control unit controls the driving of the wheel unit 111, so that the robot cleaner 100 is made to run on the floor autonomously.

Meanwhile, a battery (not shown) for supplying power to the robot cleaner 100 is mounted on the cleaner main body 110. The battery may be configured to be rechargeable and may be detachably attached to the bottom of the cleaner body 110.

The suction unit 120 is arranged to protrude from one side of the cleaner main body 110 to suck air containing dust. The one side may be the side where the cleaner main body 110 travels in the forward direction F, that is, the front side of the cleaner main body 110.

In this figure, it is shown that the suction unit 120 is protruded from one side of the cleaner main body 110 to both front and left and right sides. Specifically, the front end of the suction unit 120 is disposed at a position spaced forward from the one side of the cleaner main body 110, and the left and right ends of the suction unit 120 are spaced apart from one side of the cleaner main body 110 .

The cleaner main body 110 is formed in a circular shape and both sides of the rear end of the suction unit 120 are protruded from the cleaner main body 110 to the left and right sides, Space, that is, a gap can be formed. The empty space is a space between both left and right ends of the cleaner main body 110 and both left and right ends of the suction unit 120 and has a shape recessed inward of the robot cleaner 100.

If an obstacle is caught in the empty space, the robot cleaner 100 may be blocked by an obstacle. In order to prevent this, a cover member 129 may be arranged to cover at least a part of the empty space. The cover member 129 may be provided in the cleaner main body 110 or the suction unit 120. A cover member 129 is formed on both sides of the rear end of the suction unit 120 so as to cover the outer peripheral surface of the cleaner main body 110. [

The cover member 129 is disposed to fill at least part of the empty space, that is, the empty space between the cleaner main body 110 and the suction unit 120. Accordingly, it is possible to prevent the obstacle from being caught in the empty space, or to easily remove the obstacle from the obstacle even if the obstacle is caught in the empty space.

The cover member 129 protruding from the suction unit 120 can be supported on the outer circumferential surface of the cleaner main body 110. The cover member 129 may be supported on the rear portion of the suction unit 120. In this case, According to the structure, when the suction unit 120 is impacted by an impact with an obstacle, a part of the impact is transmitted to the cleaner main body 110 so that the impact can be dispersed.

The suction unit 120 may be detachably coupled to the cleaner main body 110. When the suction unit 120 is separated into the cleaner main body 110, a mop module (not shown) may be detachably coupled to the cleaner main body 110 in place of the separate suction unit 120. Accordingly, when the user desires to remove dust on the floor, the suction unit 120 may be mounted on the cleaner main body 110, and the mop module may be mounted on the cleaner main body 110 when the floor is to be cleaned.

When the suction unit 120 is mounted on the cleaner main body 110, the mounting can be guided by the cover member 129 described above. That is, the cover member 129 is disposed so as to cover the outer peripheral surface of the cleaner main body 110, whereby the relative position of the suction unit 120 with respect to the cleaner main body 110 can be determined.

A sensing unit 130 is disposed in the cleaner main body 110. The sensing unit 130 may be disposed on one side of the cleaner body 110 where the suction unit 120 is located, that is, on the front side of the cleaner body 110.

The sensing unit 130 may be arranged to overlap the suction unit 120 in the vertical direction of the cleaner main body 110. The sensing unit 130 is disposed at an upper portion of the suction unit 120 so as to detect an obstacle or feature in front of the suction unit 120 so that the suction unit 120 positioned at the forefront of the robot cleaner 100 does not hit the obstacle.

The sensing unit 130 is further configured to perform a sensing function other than the sensing function. This will be described in detail later.

The cleaner main body 110 is provided with a dust receptacle 113. The dust receptacle 140 separates and collects the dust in the sucked air is detachably coupled to the dust receptacle 113. [ As shown in the figure, the dust receptacle 113 may be formed on the other side of the cleaner main body 110, that is, behind the cleaner main body 110.

A part of the dust container 140 is received in the dust container receiving portion 113 and another portion of the dust container 140 protrudes toward the rear of the cleaner body 110 (i.e., the reverse direction R opposite to the forward direction F) .

The dust box 140 is formed with an inlet 140a through which air containing dust is introduced and an outlet 140b through which dust is separated from the dust box 140. When the dust box 140 is installed in the dust box receiving portion 113, 140a and the outlet 140b are configured to communicate with the first opening 110a and the second opening 110b formed in the inner wall of the dust container receiving portion 113, respectively.

The suction flow path in the cleaner main body 110 corresponds to the flow path from the inflow port (not shown) communicating with the communicating portion 120b "to the first opening 110a, and the air flow path from the second opening 110b to the air outlet 112).

The air containing the dust introduced through the suction unit 120 flows into the dust container 140 through the intake air passage in the cleaner main body 110 and the filter or cyclone of the dust container 140 Air and dust are separated from each other as they pass through. The dust is collected in the dust container 140 and the air is discharged from the dust container 140 and then discharged to the outside through the discharge port 112 of the cleaner main body 110 and finally through the discharge port 112.

4, an embodiment related to the components of the robot cleaner 100 will be described below.

The robot cleaner 100 or the mobile robot according to an embodiment of the present invention includes a communication unit 1100, an input unit 1200, a driving unit 1300, a sensor 1400, an output unit 1500, a power unit 1600, A controller 1700 and a controller 1800, or a combination thereof.

At this time, it is needless to say that the components shown in FIG. 4 are not essential, so that a robot cleaner having components having more or fewer components can be implemented. Hereinafter, each component will be described.

First, the power supply unit 1600 includes a battery that can be charged by an external commercial power supply, and supplies power to the mobile robot. The power supply unit 1600 supplies driving power to each of the components included in the mobile robot to supply the operating power required for the mobile robot to travel or perform a specific function.

At this time, the controller 1800 senses the remaining power of the battery and controls the battery 1800 to move to the charge connected to the external commercial power supply if the remaining power is insufficient, so that the charging current can be supplied from the charging stand to charge the battery. The battery is connected to the battery sensing unit, and the battery remaining amount and the charging state can be transmitted to the control unit 1800. The output unit 1500 can display the battery remaining amount on the screen by the control unit.

The battery may be located at the bottom of the center of the robot cleaner, or may be located at either the left or right side. In the latter case, the mobile robot may further include a balance weight to eliminate weight biases of the battery.

The driving unit 1300 includes a motor. By driving the motor, the left and right main wheels can be rotated in both directions to rotate or move the main body. The driving unit 1300 can advance the main body of the mobile robot forward, backward, leftward, rightward, curved, or rotated in place.

Meanwhile, the input unit 1200 receives various control commands from the user for the robot cleaner. The input unit 1200 may include one or more buttons, for example, the input unit 1200 may include an OK button, a setting button, and the like. The OK button is a button for receiving a command for confirming the detection information, the obstacle information, the position information, and the map information from the user, and the setting button is a button for receiving a command for setting the information from the user.

In addition, the input unit 1200 may include an input reset button for canceling the previous user input and receiving the user input again, a delete button for deleting the preset user input, a button for setting or changing the operation mode, A button for receiving an input, and the like.

In addition, the input unit 1200 may be installed on the upper portion of the mobile robot using a hard key, a soft key, a touch pad, or the like. In addition, the input unit 1200 may have the form of a touch screen together with the output unit 1500.

On the other hand, the output unit 1500 can be installed above the mobile robot. Of course, the installation location and installation type may vary. For example, the output unit 1500 may display a battery state, a traveling mode, and the like on the screen.

Also, the output unit 1500 can output the state information inside the mobile robot detected by the sensor 1400, for example, the current state of each configuration included in the mobile robot. Also, the output unit 1500 can display the external status information, the obstacle information, the position information, the map information, and the like detected by the sensor 1400 on the screen. The output unit 1500 may include any one of a light emitting diode (LED), a liquid crystal display (LCD), a plasma display panel, and an organic light emitting diode (OLED) As shown in FIG.

The output unit 1500 may further include sound output means for audibly outputting an operation process or an operation result of the mobile robot performed by the control unit 1800. [ For example, the output unit 1500 may output a warning sound to the outside according to the warning signal generated by the control unit 1800.

In this case, the sound output means may be a means for outputting sound such as a beeper, a speaker, and the like, and the output unit 1500 may output sound using audio data or message data having a predetermined pattern stored in the memory 1700, And output to the outside through the output means.

Accordingly, the mobile robot according to an embodiment of the present invention can output environmental information about the traveling region through the output unit 1500 or output it as sound. According to another embodiment, the mobile robot may transmit map information or environment information to the terminal device through the communication unit 1100 so that the terminal device outputs a screen or sound to be output through the output unit 1500. [

On the other hand, the communication unit 1100 is connected to a terminal device and / or another device (referred to as "home appliance" in this specification) located in a specific area and a communication method of one of wired, wireless, And transmits and receives signals and data.

The communication unit 1100 can transmit and receive data with other devices located in a specific area. In this case, the other device may be any device capable of connecting to a network and transmitting / receiving data. For example, the device may be an air conditioner, a heating device, an air purifier, a lamp, a TV, The other device may be a device for controlling a door, a window, a water supply valve, a gas valve, or the like. The other device may be a sensor for detecting temperature, humidity, air pressure, gas, or the like.

On the other hand, the memory 1700 stores a control program for controlling or driving the robot cleaner and data corresponding thereto. The memory 1700 may store audio information, image information, obstacle information, position information, map information, and the like. Also, the memory 1700 can store information related to the traveling pattern.

The memory 1700 mainly uses a nonvolatile memory. Here, the non-volatile memory (NVM) is a storage device capable of continuously storing information even when power is not supplied. Examples of the storage device include a ROM, a flash memory, A storage device (e.g., a hard disk, a diskette drive, a magnetic tape), an optical disk drive, a magnetic RAM, a PRAM, and the like.

Meanwhile, the sensor 1400 may include at least one of an external signal sensor, a front sensor, a cliff sensor, a bottom camera sensor, an upper camera sensor, a three-dimensional camera sensor, an encoder, and a microphone.

The external signal detection sensor can detect the external signal of the mobile robot. The external signal detection sensor may be, for example, an infrared ray sensor, an ultrasonic sensor (Ultra Sonic Sensor), a RF sensor (Radio Frequency Sensor), or the like.

The mobile robot can detect the position and direction of the charging base by receiving the guidance signal generated by using the external signal detection sensor. At this time, the charging base can transmit a guidance signal indicating a direction and a distance so that the mobile robot can return. That is, the mobile robot can receive the signal transmitted from the charging station, determine the current position, set the moving direction, and return to the charging state.

On the other hand, the front sensor may be installed at a predetermined distance in front of the mobile robot, specifically along the side surface of the side of the mobile robot. The front sensing sensor is located at least on one side of the mobile robot and detects an obstacle in front of the mobile robot. The front sensing sensor senses an object, especially an obstacle, existing in the moving direction of the mobile robot and transmits detection information to the controller 1800 . That is, the front sensor can detect protrusions on the moving path of the mobile robot, household appliances, furniture, walls, wall corners, and the like, and transmit the information to the controller 1800.

For example, the frontal detection sensor may be an infrared sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, or the like, and the mobile robot may use one type of sensor as the front detection sensor or two or more kinds of sensors have.

Ultrasonic sensors, for example, can typically be used to detect distant obstacles in general. The ultrasonic sensor includes a transmitter and a receiver. The controller 1800 determines whether the ultrasonic wave radiated through the transmitter is reflected by an obstacle or the like and is received by the receiver, The distance to the obstacle can be calculated.

Also, the controller 1800 can detect the information related to the size of the obstacle by comparing the ultrasonic waves radiated from the transmitter and the ultrasonic waves received by the receiver. For example, the control unit 1800 can determine that the larger the obstacle is, the more the ultrasonic waves are received in the receiving unit.

In one embodiment, a plurality (e. G., Five) of ultrasonic sensors may be installed along the outer circumferential surface on the front side of the mobile robot. At this time, preferably, the ultrasonic sensor can be installed on the front side of the mobile robot alternately with the transmitting part and the receiving part.

That is, the transmitting unit may be disposed to be spaced left and right from the front center of the main body, and one or two transmitting units may be disposed between the receiving units to form a receiving area of the ultrasonic signal reflected from the obstacle or the like. With this arrangement, the receiving area can be expanded while reducing the number of sensors. The angle of origin of the ultrasonic waves can maintain an angle range that does not affect different signals to prevent crosstalk. Also, the receiving sensitivity of the receiving units may be set differently.

In addition, the ultrasonic sensor may be installed upward by a predetermined angle so that the ultrasonic wave emitted from the ultrasonic sensor is outputted upward, and the ultrasonic sensor may further include a predetermined blocking member to prevent the ultrasonic wave from being radiated downward.

On the other hand, as described above, the front sensor can use two or more kinds of sensors together, so that the front sensor can use any one of an infrared sensor, an ultrasonic sensor, and an RF sensor .

For example, the front sensing sensor may include an infrared sensor in addition to the ultrasonic sensor.

The infrared sensor may be installed on the outer surface of the mobile robot together with the ultrasonic sensor. The infrared sensor can also detect the obstacles existing on the front or side and transmit the obstacle information to the controller 1800. That is, the infrared sensor senses protrusions existing on the moving path of the mobile robot, household appliances, furniture, walls, wall corners, and the like, and transmits the information to the controller 1800. Therefore, the mobile robot can move within a specific area without collision with the obstacle.

On the other hand, a cliff sensor (or Cliff Sensor) can detect obstacles on the floor supporting the main body of the mobile robot by mainly using various types of optical sensors.

That is, it is a matter of course that the cliff detection sensor is installed on the rear surface of the bottom mobile robot, but may be installed at a different position depending on the type of the mobile robot. The obstacle detection sensor is located on the back surface of the mobile robot and detects an obstacle on the floor. The obstacle detection sensor includes an infrared sensor, an ultrasonic sensor, an RF sensor, a PSD (Position Sensitive Detector) sensor.

For example, one of the cliff detection sensors may be installed in front of the mobile robot, and the other two cliff detection sensors may be installed relatively behind.

For example, the cliff detection sensor may be a PSD sensor, but it may be composed of a plurality of different kinds of sensors.

The PSD sensor uses a semiconductor surface resistance to detect the shortest path distance of incident light at one p-n junction. The PSD sensor includes a one-dimensional PSD sensor for detecting light in only one direction and a two-dimensional PSD sensor for detecting a light position on a plane, all of which can have a pin photodiode structure. The PSD sensor is a type of infrared sensor that measures the distance by measuring the angle of the infrared ray reflected from the obstacle after transmitting the infrared ray by using the infrared ray. That is, the PSD sensor uses the triangulation method to calculate the distance to the obstacle.

The PSD sensor includes a light emitting unit that emits infrared rays to an obstacle, and a light receiving unit that receives infrared rays that are reflected from the obstacle and is returned to the obstacle. When an obstacle is detected by using the PSD sensor, a stable measurement value can be obtained irrespective of the reflectance and the color difference of the obstacle.

The control unit 1800 can detect the depth of the cliff by measuring the infrared angle between the light emission signal of the infrared ray emitted from the cliff detection sensor toward the ground and the reflection signal received by being reflected by the obstacle.

The control unit 1800 can determine whether to pass or not according to the ground state of the detected cliff by using the cliff detection sensor, and can determine whether the cliff passes or not according to the determination result. For example, the control unit 1800 determines whether or not a cliff is present and the depth of the cliff through the cliff detection sensor, and then passes through the cliff only when a reflection signal is detected through the cliff detection sensor.

As another example, the control unit 1800 may determine the lifting phenomenon of the mobile robot using a cliff detection sensor.

On the other hand, the lower camera sensor is provided on the back surface of the mobile robot, and acquires image information on the lower side, that is, the bottom surface (or the surface to be cleaned) during the movement. The lower camera sensor is also referred to as an optical flow sensor in other words. The lower camera sensor converts the downward image inputted from the image sensor provided in the sensor to generate image data of a predetermined format. The generated image data can be stored in the memory 1700.

Also, one or more light sources may be installed adjacent to the image sensor. The at least one light source irradiates light onto a predetermined area of the bottom surface, which is photographed by the image sensor. That is, when the mobile robot moves in a specific region along the bottom surface, a certain distance is maintained between the image sensor and the bottom surface when the bottom surface is flat. On the other hand, when the mobile robot moves on the bottom surface of a nonuniform surface, it is distant by a certain distance due to the unevenness and obstacles on the bottom surface. At this time, one or more light sources can be controlled by the control unit 1800 to adjust the amount of light to be irradiated. The light source may be a light emitting device capable of adjusting the amount of light, for example, an LED (Light Emitting Diode).

Using the lower camera sensor, the control unit 1800 can detect the position of the mobile robot irrespective of the slip of the mobile robot. The control unit 1800 may compute and analyze the image data photographed by the lower camera sensor over time to calculate the moving distance and the moving direction, and calculate the position of the mobile robot on the basis of the movement distance and the moving direction. By using the image information of the lower portion of the mobile robot using the lower camera sensor, the controller 1800 can correct the slip-resistant correction of the position of the mobile robot calculated by other means.

On the other hand, the upper camera sensor can be installed to the upper side or the front side of the mobile robot, and can photograph the vicinity of the mobile robot. When the mobile robot includes a plurality of upper camera sensors, the camera sensors may be formed on the upper or side surface of the mobile robot at a certain distance or at an angle.

The three-dimensional camera sensor may be attached to one side or a part of the main body of the mobile robot to generate three-dimensional coordinate information related to the surroundings of the main body.

That is, the three-dimensional camera sensor may be a 3D depth camera that calculates the near distance of the mobile robot and the object to be photographed.

Specifically, the three-dimensional camera sensor can capture a two-dimensional image related to the surroundings of the main body, and can generate a plurality of three-dimensional coordinate information corresponding to the captured two-dimensional image.

In one embodiment, the three-dimensional camera sensor includes two or more cameras that acquire an existing two-dimensional image, and a stereoscopic vision sensor that combines two or more images obtained from the two or more cameras to generate three- . ≪ / RTI >

Specifically, the three-dimensional camera sensor according to the embodiment includes a first pattern irradiating unit for irradiating light of a first pattern downward toward the front of the main body, and a second pattern irradiating unit for irradiating the light of the second pattern upward toward the front of the main body A second pattern irradiating unit, and an image acquiring unit acquiring an image in front of the main body. Thus, the image acquiring unit can acquire an image of a region where light of the first pattern and light of the second pattern are incident.

In another embodiment, the three-dimensional camera sensor includes an infrared ray pattern emitting unit for irradiating an infrared ray pattern with a single camera, and captures a shape of the infrared ray pattern irradiated by the infrared ray pattern emitting unit onto the object to be photographed, The distance between the sensor and the object to be photographed can be measured. The three-dimensional camera sensor may be an IR (Infra Red) type three-dimensional camera sensor.

In another embodiment, the three-dimensional camera sensor includes a light emitting unit that emits light together with a single camera, receives a part of the laser emitted from the light emitting unit reflected from the object, analyzes the received laser, The distance between the camera sensor and the object to be photographed can be measured. The three-dimensional camera sensor may be a time-of-flight (TOF) type three-dimensional camera sensor.

Specifically, the laser of the above-described three-dimensional camera sensor is configured to irradiate a laser beam extending in at least one direction. In one example, the three-dimensional camera sensor may include first and second lasers, wherein the first laser irradiates a linear laser beam intersecting each other, and the second laser irradiates a single linear laser can do. According to this, the lowermost laser is used to detect obstacles in the bottom part, the uppermost laser is used to detect obstacles in the upper part, and the intermediate laser between the lowermost laser and the uppermost laser is used to detect obstacles in the middle part .

The encoder can sense information related to the operation of the motor that drives the wheels of the driver 1300.

5, a system including a vacuum cleaner that performs autonomous driving according to an embodiment of the present invention will be described.

5, a robot cleaning system 50 according to an embodiment of the present invention includes a vacuum cleaner 100, an AP apparatus 400, a server 500, a network 550, (200a, 200b).

The cleaner 100, the AP device 400, the mobile terminals 600a, and the like may be disposed in a building 10 such as a house.

The vacuum cleaner 100 is a vacuum cleaner that can perform automatic running and automatic cleaning. In addition to the traveling function and the cleaning function, the vacuum cleaner 100 includes a communication unit 1100 in the interior thereof, and electronic apparatuses in the internal network 10 or electronic apparatuses connectable through the external network 550 Exchangeable. To this end, the communication unit 1100 can perform data exchange with the AP apparatus 400, either wirelessly or wirelessly.

An access point (AP) device 400 may provide an internal network 10 to an adjacent electric device. In particular, a wireless network can be provided.

On the other hand, the AP apparatus 400 can allocate a wireless channel according to a predetermined communication method to the electronic devices in the internal network 10, and perform wireless data communication through the corresponding channel. Here, the predetermined communication method may be a WiFi communication method.

At this time, the mobile terminal 600b located in the internal network 10 can perform monitoring, remote control, and the like with respect to the cleaner 100 by connecting to the cleaner 100 through the AP apparatus 400 .

Meanwhile, the AP apparatus 400 can perform data communication with an external electronic device via the external network 550 in addition to the internal network 10. [

For example, the AP apparatus 400 can perform wireless data communication with an external mobile terminal 200a through an external network 550. [

At this time, the mobile terminal 200a located in the external network 550 accesses the cleaner 100 through the external network 550 and the AP apparatus 400 to monitor the cleaner 100, And so on.

As another example, the AP apparatus 400 can perform wireless data communication with an externally located server 500 through the external network 550. [

The server 500 may include a speech recognition algorithm. In receiving the voice data, the received voice data can be converted into text format data and output.

The server 500 may store firmware information, operation information (course information, etc.) for the cleaner 100, and register product information on the cleaner 100. For example, the server 500 may be a server operated by the manufacturer of the cleaner 100. As another example, the server 500 may be a server operated by an open application store operator.

Unlike the drawing, the server 500 may be a home server that is provided in the home and stores state information about home appliances in the home, or stores contents shared in home appliances. If the server 500 is a home server, it may store information related to the foreign object, for example, a foreign object image and the like.

Meanwhile, the cleaner 100 according to an embodiment of the present invention captures an image including a foreign substance through a camera provided therein, and transmits an image including foreign matter and image related information to the mobile terminal 200 Or cleaning the periphery of the foreign substance on the basis of the cleaning execution information on the foreign substance from the mobile terminal 200 or the server 500, You may not clean your surroundings. As a result, cleaning can be selectively performed on the foreign matter.

Meanwhile, according to another embodiment of the present invention, the cleaner 100 captures an image including a foreign object through a stereo camera provided therein, and performs signal processing on an image including a foreign substance obtained from the stereo camera Thereby generating cleaning execution information or cleaning circumvention information for the foreign object based on the object related to the identified foreign object, and cleaning the periphery of the foreign object based on the cleaning execution information, Based on the cleaning avoidance information, the periphery of the foreign matter can be cleaned. As a result, cleaning can be selectively performed on the foreign matter.

6, a control method of the robot cleaning system 50 according to the present invention will be described.

Referring to FIG. 6, a robot cleaning system 50 according to the present invention may include a cleaner 100 and a server 500.

In addition, the embodiment shown in FIG. 6 may be configured such that the controller 1800 of the cleaner 100 can analyze information related to a dangerous situation, and receive training data used for analyzing information related to the dangerous situation from the server 500 To update, training data update information may be transmitted.

For reference, training  The data may be defined as data used by the control unit 1800 when the control unit 1800 drives a learning engine for analyzing a dangerous situation.

In other words, training  The data show that in the past several other cleaners, Was  The data collected by the cleaner 100 may be included. Specifically training  The data may include coordinate information for the cleaning area, Sensing  Value, and an image taken around the main body.

In other words, the embodiment shown in FIG. 6 can install the learning engine for analyzing the dangerous situation by the cleaner 100 itself.

6, the controller 1800 of the cleaner 100 may detect a dangerous situation (S601).

Specifically, when the vacuum cleaner 100 detects various conditions Examples include  It is described as follows.

The control unit 1800 determines that a dangerous situation has occurred in the cleaner 100 when it is determined that an obstacle exists on the traveling path of the cleaner 100 based on the detection result of the sensor 1400 .

The control unit 1800 may determine that a dangerous situation has occurred in the cleaner 100 if it is determined that the wheels of the driving unit 1300 are constrained based on the detection result of the sensor 1400. [

The control unit 1800 may determine that a dangerous situation has occurred in the cleaner 100 if it is determined that the wheels of the drive unit 1300 are idling based on the detection result of the sensor 1400. [

The control unit 1800 determines that one point of the main body of the cleaner 100 is spaced from the bottom of the cleaning area by a predetermined distance or more based on the detection result of the sensor 1400, ) May be considered to have caused a hazardous situation.

In another embodiment, the control unit 1800 may determine that a dangerous situation has occurred in the cleaner 100 when it is determined that the main body has deviated from the traveling path, based on the detection result of the sensor 1400. [

In another embodiment, when it is determined that the main body can not escape from the specific area for a predetermined time interval or more based on the detection result of the sensor 1400, the control unit 1800 determines that a dangerous situation has occurred in the cleaner 100 It can be judged. That is, if the controller 1800 judges that the cleaner 100 is placed under the bed or in a narrow area, the controller 1800 can determine that a dangerous situation has occurred in the cleaner 100.

In another embodiment, when a significant difference is detected between a plurality of images photographed at predetermined time intervals while the main body is not moving, the control unit 1800 determines that a dangerous situation has occurred in the cleaner 100 . Here, the control unit 1800 may determine whether there is a significant difference between the plurality of images by comparing the pixel values included in the first image and the second image, respectively, of the plurality of images.

In another embodiment, the control unit 1800 can determine that a dangerous situation has occurred in the cleaner 100 when an abnormal sound source occurs in the vicinity of the main body, based on the detection result of the sensor 1400.

Here, the memory 1700 of the vacuum cleaner 100 may store a sound source database including an abnormal sound source and a non-abnormal sound source.

For example, there are situations in which there are intruders from outside such as a sound of falling or breaking of various objects, a sound of cracking of glass or the like, a sound of drill rotation, a dog barking, The noise generated from the inside of the robot cleaner 100, the noise generated from appliances such as a refrigerator, a washing machine, and a water purifier may be generated irrespective of intrusion or not, Various kinds of noise can be stored as a non-abnormal sound source.

In this case, the control unit 1800 determines whether the sound source obtained through the sound acquisition means included in the sensor 1400 is similar to at least one sound source stored in the sound source database, It is possible to judge whether or not it is an abnormal sound source. That is, the control unit 1800 can determine the coincidence and / or similarity of the two sound sources by calculating the similarity between the sound source obtained through the sound acquisition means and the sound sources stored in the sound source database.

The control unit 1800 can control the sensor 1400 so that information related to the above described dangerous situation can be periodically collected.

Referring to FIG. 6, the server 500 may collect training data (S602).

In particular, the server 500 may collect information related to the hazardous situation of the cleaner from a plurality of other cleaners, and may convert the collected information into training data.

In addition, the server 500 can update the training data stored in the server 500 using the training data input by the server manager.

6, when a dangerous situation is detected around the cleaner 100, the controller 1800 transmits a training data update request to the server 500 to update the training data stored in the memory 1700 in advance, (Step S603).

In one embodiment, the controller 1800 may send a training data update request to the server 500 whenever a critical condition is detected.

In another embodiment, the control unit 1800 may analyze the information related to the risk situation when a dangerous situation is detected, and may transmit a training data update request to the server 500 based on the analysis result.

The control unit 1800 may transmit a training data update request to the server 500 when it is difficult to detect an object corresponding to the image photographed by the camera.

Similarly, when it is difficult to detect the voice data corresponding to the voice input by the microphone among the voice data included in the training data stored in advance in the memory 1700, the control unit 1800 transmits a training data update request to the server 500 .

If it is determined that the training data update request is required as a result of analyzing the information related to the dangerous situation, the control unit 1800 may include the identification information indicating the type of information related to the dangerous situation in the training data update request.

For example, the control unit 1800 may control the communication unit 1100 to transmit a training data update request related to an image to the server 500 when the analysis target is an image.

In this manner, the control unit 1800 detects the type of information related to the dangerous situation, and transmits a training data update request related to the detected type to the server 500, thereby detecting the amount of training data update information received from the server 500 Can be minimized.

In another embodiment, the controller 1800 may send a training data update request to the server 500 so that the training data is updated at a predetermined period.

Specifically, the control unit 1800 may control the communication unit 1100 to transmit the training data update request to the server 500 after a predetermined time interval elapses from the latest update of the training data. The control unit 1800 may check the latest update time of the training data when a dangerous situation is detected or may periodically check the latest update time of the training data irrespective of the dangerous situation.

Meanwhile, the training data update request includes identification information of the cleaner 100, information related to the version of the training data stored in the memory 1700 of the cleaner 100, information related to the version of the learning engine mounted on the cleaner 100, Identification information indicating a type of information related to the dangerous situation, and information related to a cleaning area in which the cleaner 100 is disposed.

Referring to FIG. 6, the controller 1800 receives the training data update information from the server 500 (S604).

In addition, the controller 1800 may update the training data stored in the memory 1700 using the training data update information received from the server 500 (S605).

In addition, the control unit 1800 may analyze the information related to the risk situation obtained when the dangerous situation is detected, using the updated training data (S606).

In one embodiment, the control unit 1800 may use updated training data to identify information associated with the subject of the photographed image when a dangerous situation is detected.

In another embodiment, the control unit 1800 may use updated training data to identify information related to the type of obstacle detected when a dangerous situation is detected.

In another embodiment, the control unit 1800 may use updated training data to identify information associated with a difference portion generated between a plurality of images photographed when a dangerous situation is detected.

In another embodiment, the controller 1800 may use updated training data to identify information associated with a sound source that was generated when a dangerous situation was detected.

Referring to FIG. 6, the controller 1800 may set an operation mode corresponding to the dangerous situation based on the analysis result of information related to the dangerous situation (S607).

In addition, the control unit 1800 can control the driving unit 1300 according to the set driving mode (S608).

In one embodiment, the control unit 1800 may set a first driving mode for avoiding an obstacle when it is determined that an obstacle exists on the traveling path of the cleaner, based on the analysis result.

That is, the control unit 1800 may set a first operation mode for avoiding an obstacle detected when a dangerous situation is detected. The control unit 1800 may control the driving unit 1300 to avoid the detected obstacle according to the set first operation mode.

In another embodiment, the control unit 1800 determines whether an object or an organism that interrupts the driving of the vacuum cleaner exists in the cleaning area of the vacuum cleaner 100, based on the analysis result, It is possible to set the second operation mode for stopping the operation.

That is, the control unit 1800 can set a second operation mode for temporarily stopping the operation of the cleaner 100 based on information related to the photographed image when a dangerous situation is detected.

The controller 1800 can set the second operation mode based on the training data stored in the memory 1700 when it is determined that the subject of the photographed image interferes with the operation of the cleaner 100. [

The control unit 1800 may control the driving unit 1300 to temporarily stop the cleaning or running according to the set second operating mode. In addition, when the second operation mode is being executed, the control unit 1800 may transmit an image of the cleaning area in which the cleaner 100 is present to the mobile terminal registered in advance in the cleaner 100.

In another embodiment, the control unit 1800 determines whether or not the user of the cleaner 100 has been cleaned (i.e., the cleaner 100 is present) based on the information related to the difference portion generated between the plurality of images photographed when the dangerous situation is detected It is possible to set a third operation mode for providing monitoring information on the area.

That is, if it is determined that the object or the creature corresponding to the difference portion generated between the plurality of images threatens the security of the cleaning area based on the training data, the control unit 1800 sets the third operation mode .

The control unit 1800 may control the driving unit 1300 to temporarily stop the cleaning or the traveling according to the set third operating mode. When the controller 1800 is in the third operation mode, the control unit 1800 can transmit an image of the cleaning area in which the cleaner 100 exists to the mobile terminal registered in advance in the cleaner 100.

In another embodiment, the control unit 1800 can set the third operation mode based on information related to the sound source generated when a dangerous situation is detected.

That is, the control unit 1800 may set the third operation mode based on the training data if the sound source is determined as a threat to the security of the cleaning area in which the cleaner 100 exists.

7, a control method of the robot cleaning system 50 according to the present invention will be described.

Referring to FIG. 7, a robot cleaning system 50 according to the present invention may include a cleaner 100 and a server 500.

7 may analyze the information related to the dangerous situation that the control module (not shown) of the server 500 transmits to the server 500 by the cleaner 100, The communication module (not shown) may transmit the analysis result to the cleaner 100.

That is, the embodiment shown in FIG. 7 can receive the analysis result related to the dangerous situation from the server 500 without directly analyzing the dangerous situation by the cleaner 100. At this time, the server 500 may include a learning engine for analyzing a dangerous situation, and may continuously update the learning engine by collecting information related to various dangerous situations from a plurality of cleaners connected through a network .

7, the controller 1800 of the cleaner 100 may sense a dangerous situation (S701). The risk situation is as defined in Fig.

The control unit 1800 of the cleaner 100 may transmit information related to the dangerous situation to the server 500 when a dangerous situation is detected (S702).

Here, the information related to the dangerous situation may include information obtained by the sensor 1400 when the control unit 1800 detects a dangerous situation. That is, the information related to the risk situation may include information collected by the sensor 1400, which causes the control unit 1800 to detect a dangerous situation.

Next, the control module of the server 500 may update the training data stored in the memory (not shown) of the server 500 using the information related to the dangerous situation received from the cleaner 100 (S703) .

In addition, the control module of the server 500 may analyze the information related to the received risk using the updated training data (S704).

In one embodiment, the information associated with the hazard situation may include an image taken by the camera of the cleaner 100. At this time, the control module of the server 500 can use the updated training data to identify information related to the subject of the image photographed by the cleaner 100 when a dangerous situation is detected.

In another embodiment, the information related to the hazard situation may include information related to the obstacles present around the cleaner 100. For example, At this time, the control module of the server 500 can use the updated training data to identify information related to the type of obstacle detected when a dangerous situation is detected.

In another embodiment, the information associated with the hazardous situation may include a plurality of images taken by the camera of the cleaner 100 when a hazardous condition is detected. At this time, the control module of the server 500 may use the updated training data to identify information related to a difference portion generated between a plurality of images photographed when a dangerous situation is detected.

In another embodiment, the information associated with the hazard situation may include a sound source obtained by the microphone of the cleaner 100. [ At this time, the control module of the server 500 can identify the information related to the sound source generated when the dangerous situation is detected by using the updated training data.

The communication module of the server 500 may transmit the analysis result performed by the control module to the cleaner 100 (S705).

The control unit 1800 of the cleaner 100 can set an operation mode corresponding to the detected dangerous situation based on the received analysis result (S706).

In addition, the control unit 1800 can control the driving unit 1300 based on the set operation mode (S707).

In one embodiment, the control unit 1800 may set a first operating mode to avoid the detected obstacle when a dangerous situation is detected. The control unit 1800 may control the driving unit 1300 to avoid the detected obstacle according to the set first operation mode.

In another embodiment, the control unit 1800 can set a second operation mode for temporarily stopping the operation of the cleaner 100, based on information related to the subject of the photographed image when a dangerous situation is detected.

That is, the control unit 1800 can set the second operation mode using the analysis result provided from the server 500, if it is determined that the object corresponding to the analyzed image interferes with the operation of the cleaner 100.

The control unit 1800 may control the driving unit 1300 to temporarily stop the cleaning or running according to the set second operating mode. In addition, when the second operation mode is being executed, the control unit 1800 may transmit an image of the cleaning area in which the cleaner 100 is present to the mobile terminal registered in advance in the cleaner 100.

In another embodiment, the control unit 1800 determines whether or not the user of the cleaner 100 has been cleaned (i.e., the cleaner 100 is present) based on the information related to the difference portion generated between the plurality of images photographed when the dangerous situation is detected It is possible to set a third operation mode for providing monitoring information on the area.

That is, the control unit 1800 can use the analysis result provided by the server 500 to set the third operation mode when it is determined that the object or the creature corresponding to the difference portion threatens the security of the cleaning area have.

The control unit 1800 may control the driving unit 1300 to temporarily stop the cleaning or the traveling according to the set third operating mode. When the controller 1800 is in the third operation mode, the control unit 1800 can transmit an image of the cleaning area in which the cleaner 100 exists to the mobile terminal registered in advance in the cleaner 100.

In another embodiment, the control unit 1800 can set the third operation mode based on information related to the sound source generated when a dangerous situation is detected.

That is, the control unit 1800 can set the third operation mode using the analysis result provided from the server 500 if the sound source is determined to be a threat to the security of the cleaning area in which the cleaner 100 exists .

The control unit 1800 may control the sensor 1400 to acquire information related to an object existing around the main body of the cleaner 100 (S708).

Specifically, when the execution of the first to third operation modes is completed, the control unit 1800 can control the sensor 1400 to acquire information related to objects existing within a predetermined distance from the main body of the cleaner 100 .

For example, information related to an object existing within a predetermined distance from the main body can be obtained by at least one of the camera of the vacuum cleaner 100, the infrared sensor, the ultrasonic sensor, the microphone, and the RF sensor.

That is, the information related to an object existing within a predetermined distance from the main body may include at least one of image information photographed by a camera, audio information obtained by a microphone, and ultrasound information acquired by an ultrasonic sensor of the vacuum cleaner 100 .

In addition, the controller 1800 may transmit information related to an object existing around the main body obtained after the execution of the set operation mode is completed to the server 500 (S709).

The control module of the server 500 may determine the accuracy of the analysis result using the information related to the object existing around the main body received from the cleaner 100 after the analysis result transmission step S706 is performed ).

Specifically, the control module of the server 500 determines whether or not the information related to the dangerous situation transmitted after the cleaner 100 senses the dangerous situation and the information related to the dangerous condition transmitted to the cleaner 100 after the execution of the operation mode in which the cleaner 100 is set up is completed. It is possible to determine whether or not the risk situation is eliminated by comparing information related to an object existing in the vicinity of the main body.

The control module of the server 500 can determine the accuracy of the analysis result based on the result of the determination as to whether or not the risk situation is resolved.

In one example, if the accuracy is determined to be below a predetermined level, the control module may delete the updated portion of the training data of the server 500 in the update step S703.

In another example, the control module can set the priority of the training data using the determined accuracy.

8, a control method of the robot cleaning system 50 according to the present invention will be described.

Referring to FIG. 8, the robot cleaning system 50 according to the present invention may include a cleaner 100 and a server 500.

In addition, the embodiment shown in FIG. 8 can analyze the information related to the dangerous situation that the control module of the server 500 transmits to the server 500 by the cleaner 100, as in the embodiment shown in FIG. 7 , The communication module of the server 500 may transmit the analysis result to the cleaner 100.

8, the controller 1800 of the cleaner 100 may sense a dangerous situation (S801). The risk situation is as defined in Fig.

The control unit 1800 of the cleaner 100 may transmit information related to the dangerous situation to the server 500 when a dangerous situation is detected (S802).

The step S802 of transmitting the information related to the dangerous situation to the server 500 corresponds to the step S702 of transmitting the information related to the dangerous situation shown in FIG. 7 to the server 500.

Next, the control module of the server 500 may analyze information related to the dangerous situation received from the cleaner 100 using the training data stored in advance in the memory of the server 500 (S803).

In one embodiment, the information associated with the hazard situation may include an image taken by the camera of the cleaner 100. At this time, the control module of the server 500 can use the training data stored in advance in the server 500 to identify information related to the subject of the image photographed by the cleaner 100 when a dangerous situation is detected.

In another embodiment, the information related to the hazard situation may include information related to the obstacles present around the cleaner 100. For example, At this time, the control module of the server 500 can identify information related to the type of the obstacle detected when the dangerous situation is detected by using the training data stored in advance.

In another embodiment, the information associated with the hazardous situation may include a plurality of images taken by the camera of the cleaner 100 when a hazardous condition is detected. At this time, the control module of the server 500 can identify the information related to the difference generated between the plurality of images photographed when the dangerous situation is detected by using the training data stored in advance.

In another embodiment, the information associated with the hazard situation may include a sound source obtained by the microphone of the cleaner 100. [ At this time, the control module of the server 500 can identify information related to the sound source generated when the dangerous situation is detected by using the training data stored in advance.

The communication module of the server 500 may transmit the analysis result performed by the control module to the cleaner 100 (S804).

The control unit 1800 of the cleaner 100 can set an operation mode corresponding to the detected dangerous situation based on the received analysis result (S805).

In addition, the control unit 1800 can control the driving unit 1300 based on the set operation mode (S806).

The steps S805 and S806 performed by the cleaner 100 correspond to steps S706 and S707 shown in Fig. 7, respectively.

The control unit 1800 may control the sensor 1400 to acquire information related to an object existing around the main body of the cleaner 100 (S807).

Specifically, when the execution of the first to third operation modes is completed, the control unit 1800 can control the sensor 1400 to acquire information related to objects existing within a predetermined distance from the main body of the cleaner 100 .

For example, information related to an object existing within a predetermined distance from the main body can be obtained by at least one of the camera of the vacuum cleaner 100, the infrared sensor, the ultrasonic sensor, the microphone, and the RF sensor.

That is, the information related to an object existing within a predetermined distance from the main body may include at least one of image information photographed by a camera, audio information obtained by a microphone, and ultrasound information acquired by an ultrasonic sensor of the vacuum cleaner 100 .

In step S808, the control module of the server 500 may classify information related to the dangerous situation received from the cleaner 100 based on the analysis result after transmitting the analysis result in step S804 (S808) .

Specifically, the control module of the server 500 may set label information of information related to the dangerous situation received from the cleaner 100.

In addition, the control module of the server 500 may update the training data stored in advance based on the classification result (S809).

More specifically, if the set label already exists in the pre-stored training data, the control module of the server 500 can add information related to the dangerous situation received from the cleaner 100 to the data group of the corresponding label. The data group corresponding to the label set in the training data can be newly generated if the set label is not present in the training data stored in advance.

Referring to FIG. 8, the controller 1800 may transmit information related to an object existing around the main body obtained after the execution of the set operation mode is completed to the server 500 (S810).

The control module of the server 500 may determine the accuracy of the analysis result using the information related to the object existing around the main body received from the cleaner 100 after the analysis result transmission step S810 is performed ).

The step of determining the accuracy of FIG. 8 (S811) corresponds to the step of determining the accuracy shown in FIG. 7 (S710).

9, a control method of the robot cleaning system 50 according to the present invention will be described.

Referring to FIG. 9, a robot cleaning system 50 according to the present invention may include a cleaner 100 and a server 500.

In addition, the embodiment shown in FIG. 9 may be configured such that the controller 1800 of the cleaner 100 can analyze information related to a dangerous situation, and receive training data used for analyzing information related to the dangerous situation from the server 500 To update, training data update information may be transmitted.

In other words, the embodiment shown in FIG. 9 can mount the learning engine for analyzing a dangerous situation by itself in the cleaner 100, as in the embodiment shown in FIG.

Specifically, referring to FIG. 9, the controller 1800 of the cleaner 100 may detect a dangerous situation (S901). Here, the risk situation is as defined in Fig.

9, when a dangerous situation is detected in the vicinity of the cleaner 100, the controller 1800 transmits an analysis algorithm update request to the cleaner 100 in order to update the analysis algorithm pre- The communication unit 1100 may be controlled so as to transmit the data to the communication unit 500 (S902).

In one embodiment, the controller 1800 may send an analysis algorithm update request to the server 500 whenever a critical condition is detected.

In another embodiment, the control unit 1800 may analyze the information related to the dangerous situation when a dangerous situation is detected, and send an analysis algorithm update request to the server 500 based on the analysis result.

More specifically, the control unit 1800 can transmit an analysis algorithm update request to the server 500 when it is difficult to detect a subject corresponding to the image photographed by the camera.

Similarly, when it is difficult to detect the speech data corresponding to the speech inputted by the microphone among the speech data included in the training data stored in advance in the memory 1700, the controller 1800 transmits an analysis algorithm update request to the server 500 .

If it is determined that the analysis algorithm update request is required as a result of analyzing the information related to the dangerous situation, the control unit 1800 may include identification information indicating the type of information related to the dangerous situation in the analysis algorithm update request.

For example, when the object of analysis is an image, the control unit 1800 may control the communication unit 1100 to transmit an analysis algorithm update request related to the image to the server 500.

In this way, the control unit 1800 detects the type of information related to the dangerous situation, and transmits an analysis algorithm update request related to the detected type to the server 500, thereby detecting the amount of analysis algorithm update information received from the server 500 Can be minimized.

In another embodiment, control 1800 may send an analysis algorithm update request to server 500 such that the analysis algorithm is updated at predetermined intervals.

Specifically, the control unit 1800 may control the communication unit 1100 to transmit the analysis algorithm update request to the server 500 after a predetermined time interval elapses from the latest update time of the analysis algorithm. The control unit 1800 may check the latest update timing of the analysis algorithm when a dangerous situation is detected or may periodically check the latest update timing of the analysis algorithm regardless of a dangerous situation.

On the other hand, the analysis algorithm update request includes the identification information of the cleaner 100, the information related to the version of the analysis algorithm stored in the memory 1700 of the cleaner 100, the information related to the dangerous situation, And information related to the cleaning area in which the cleaner 100 is disposed.

Referring to FIG. 9, the control module of the server 500 can check the version of the analysis algorithm mounted on the cleaner 100, based on the analysis algorithm update request received from the cleaner 100 (S903).

The server 500 may send analysis algorithm update information to the cleaner 100 based on the verified version (S904).

In addition, the controller 1800 may update the analysis algorithm stored in the memory 1700 using the analysis algorithm update information transmitted from the server 500 (S905).

In addition, the control unit 1800 may analyze the information related to the obtained dangerous situation when the dangerous situation is detected by using the updated analysis algorithm (S906).

In one embodiment, the control unit 1800 can use the updated analysis algorithm to identify information related to the subject of the photographed image when a dangerous situation is detected.

In another embodiment, controller 1800 may use updated analysis algorithms to identify information associated with the type of obstacle detected when a hazard condition is detected.

In another embodiment, controller 1800 may use updated analytical algorithms to identify information associated with a difference portion generated between a plurality of images photographed when a critical condition is detected.

In another embodiment, the controller 1800 may use the updated analysis algorithm to identify information associated with a sound source that was generated when a dangerous situation was detected.

Referring to FIG. 9, the controller 1800 may set an operation mode corresponding to the dangerous situation based on the analysis result of the information related to the dangerous situation (S907).

In addition, the control unit 1800 may control the driving unit 1300 according to the set operation mode (S908).

10, a control method of the robot cleaning system 50 according to the present invention will be described.

Referring to FIG. 10, a robot cleaning system 50 according to the present invention may include a cleaner 100, a mobile terminal 200, and a server 500.

In addition, the embodiment shown in FIG. 10 can analyze information related to the dangerous situation that the control module of the server 500 transmits to the server 500 by the cleaner 100, as in the embodiment shown in FIG. 7 , The communication module of the server 500 may transmit the analysis result to the cleaner 100.

The mobile terminal 200 may include a wireless communication unit, an input unit, a sensor, a display having a touch screen, a memory, an interface unit, a control unit, and a power supply unit.

Since the components of the mobile terminal 200 are well known in the art, a detailed description thereof will be omitted.

10, the mobile terminal 200 may transmit a monitoring request to the cleaner 100 (S1001).

In addition, the controller 1800 of the cleaner 100 may control the communication unit 1100 to transmit monitoring information to the mobile terminal 200 in response to the monitoring request (S1002).

For example, the monitoring information may include an image captured by the camera of the vacuum cleaner 100 or a voice obtained by the microphone of the vacuum cleaner 100.

The display (not shown) of the mobile terminal 200 may output the monitoring information received from the cleaner 100 (S1003).

An input unit (not shown) of the mobile terminal 200 can receive a user input related to a dangerous situation while monitoring information is output to the display (S1004).

Specifically, the display of the mobile terminal 200, along with the monitoring information, may output a predetermined icon to receive user input related to the risk situation.

Also, the control unit of the mobile terminal 200 may determine some of the monitoring information as information related to the dangerous situation based on the user input applied to the display.

That is, when the user of the mobile terminal 200 outputs the monitoring information to the display, it is determined that a dangerous situation has occurred in the vicinity of the cleaner 100, a touch input is applied to the predetermined icon, By selecting, the cleaner 100 can request the server 500 to cope with a dangerous situation.

Referring to FIG. 10, when a user input related to a dangerous situation is applied to the mobile terminal 200, the control unit of the mobile terminal 200 may transmit information related to the dangerous situation to the server 500 based on the user input (S1005).

Next, the control module of the server 500 may update the training data previously stored in the memory of the server 500 using information related to the dangerous situation received from the mobile terminal 200 (S1006).

In addition, the control module of the server 500 may analyze the information related to the received risk using the updated training data (S1007).

In one embodiment, the information related to the risk situation may include monitoring information output to the display of the mobile terminal 200 at the time when user input is authorized. That is, when the display of the mobile terminal 200 outputs the monitoring information in real time, the information related to the dangerous situation may include the image of the screen of the display at the time when the user input is applied.

In another embodiment, the information associated with the risk situation may include a portion selected based on user input of the monitoring information. That is, the user can specify a portion of the monitoring information, which is recognized as a dangerous situation, as information related to the dangerous situation by applying user input to the mobile terminal 200. [

The communication module of the server 500 may transmit the analysis result performed by the control module to the cleaner 100 (S1008).

The control unit 1800 of the cleaner 100 may set an operation mode corresponding to the detected dangerous situation based on the received analysis result (S1009).

In addition, the control unit 1800 can control the driving unit 1300 based on the set operation mode (S1010).

In one embodiment, the control unit 1800 may set a first operation mode for avoiding an obstacle when it is determined that an obstacle exists on the traveling path of the cleaner 100, based on the analysis result received. The control unit 1800 may control the driving unit 1300 to avoid the detected obstacle according to the set first operation mode.

In yet another embodiment, the control unit 1800 may use the analysis result provided from the server 500 to determine whether the subject corresponding to the monitoring information specified by the user is information related to the dangerous situation, It is possible to set the second operation mode.

The control unit 1800 may control the driving unit 1300 to temporarily stop the cleaning or running according to the set second operating mode.

In another embodiment, when the control unit 1800 determines that an object or life threatening the security of the cleaning area exists in the cleaning area in which the cleaner 100 exists using the analysis result provided from the server 500 The amount of monitoring information provided to the mobile terminal 200 can be increased.

Specifically, the control unit 1800 may decrease the transmission period to the mobile terminal 200, decrease the shooting period of the camera, or increase the shooting angle of the camera.

10, the control unit 1800 may control the sensor 1400 to acquire information related to an object existing around the main body of the cleaner 100 (S1011).

Specifically, when the execution of the first to third operation modes is completed, the control unit 1800 can control the sensor 1400 to acquire information related to objects existing within a predetermined distance from the main body of the cleaner 100 .

For example, information related to an object existing within a predetermined distance from the main body can be obtained by at least one of the camera of the vacuum cleaner 100, the infrared sensor, the ultrasonic sensor, the microphone, and the RF sensor.

That is, the information related to an object existing within a predetermined distance from the main body may include at least one of image information photographed by a camera, audio information obtained by a microphone, and ultrasound information acquired by an ultrasonic sensor of the vacuum cleaner 100 .

In addition, the control unit 1800 may retransmit the monitoring information after the execution of the set operation mode is completed (S1012).

At this time, the monitoring information to be retransmitted may include information related to an object existing around the main body obtained after execution of the set operation mode is completed.

In addition, the input unit of the mobile terminal 200 may receive a user input related to whether or not the risk situation is eliminated (S1013).

Specifically, the display of the mobile terminal 200 may output a button for confirming whether or not the risk situation is eliminated together with the monitoring information retransmitted from the cleaner 100. For example, the button for confirming whether or not the risk situation is solved may include a first icon indicating that the risk situation has been solved and a second icon indicating that the risk situation has not been solved.

The control unit of the mobile terminal 200 controls the communication unit of the mobile terminal 200 to transmit information related to the received user input to the server 500 when the user input is received in the button for confirming whether or not the risk situation is eliminated .

In addition, the control module of the server 500 may determine the accuracy of the analysis result based on the received user input (S1015).

According to the present invention, the robot cleaner is not limited in performance in manufacturing or designing, and an effect of recognizing an obstacle present around the main body is derived.

In addition, according to the present invention, since the robot cleaner can perform an appropriate counter operation against a dangerous situation generated around the main body by using machine learning, the operation performance of the robot cleaner is improved.

According to the present invention, even when the control unit of the robot cleaner has a performance that is not sufficient to perform machine learning, the obstacle recognition result using the machine learning can be obtained by the server performing the communication with the robot cleaner . That is, it is possible to improve the obstacle recognition performance of the robot cleaner without increasing the manufacturing cost of the robot cleaner.

In addition, according to the present invention, as the data related to the use case of the robot cleaner is accumulated, the robot cleaner can accurately determine the substance of the dangerous situation generated around the main body. That is, according to the present invention, the obstacle recognition performance of the robot cleaner can be improved.

Claims (20)

main body;
A driving unit for moving the main body;
A sensor for sensing information related to the environment in which the body is located;
A communication unit for receiving or transmitting information with at least one of a mobile terminal and a server;
A memory for storing training data for analyzing information sensed by the sensor; And
Based on information detected by the sensor, whether or not the main body enters a dangerous situation,
Analyzing information related to a risk situation using training data and an analysis algorithm stored in the memory when the main body is determined to have entered a dangerous situation,
And a controller for setting an operation mode corresponding to the dangerous situation based on the analysis result. The method according to claim 1,
Wherein,
Wherein the control unit controls the communication unit to transmit a training data update request to the server to update the training data when it is determined that the main body enters a dangerous situation. 3. The method of claim 2,
Wherein,
Wherein the training data updating unit updates the training data using the training data update information received from the server in response to the training data update request. The method of claim 3,
Wherein,
And analyzing information related to the dangerous situation using the updated training data. 5. The method of claim 4,
Wherein,
Wherein the information related to the subject of the photographed image is identified using the updated training data when a dangerous situation is detected. 5. The method of claim 4,
Wherein,
Wherein the information related to the type of the obstacle detected when the dangerous situation is detected is identified by using the updated training data. 5. The method of claim 4,
Wherein,
And sets an operation mode corresponding to the dangerous situation based on an analysis result of information related to the dangerous situation. 8. The method of claim 7,
Wherein,
Wherein the first operation mode is set to avoid the obstacle when it is determined that an obstacle exists on the traveling path of the cleaner based on the analysis result. 8. The method of claim 7,
Wherein,
And setting a second operation mode for stopping the operation of the vacuum cleaner when it is determined based on the analysis result that an object or living organism obstructing the operation of the vacuum cleaner exists in the cleaning area of the vacuum cleaner Vacuum cleaner to carry. 8. The method of claim 7,
Wherein,
A third operation for providing monitoring information on the cleaning area to the user of the cleaner when it is determined that there is an object or organism dangerous to the security of the cleaning area in the cleaning area of the cleaner, And the mode is set. The method according to claim 1,
Wherein the sensor includes an encoder for sensing information related to operation of the drive,
Wherein,
Wherein the controller determines that a dangerous situation has occurred in the cleaner based on the detection result of the encoder. 12. The method of claim 11,
Wherein,
Wherein the controller determines that a dangerous situation has occurred in the cleaner when it is determined that the main body has deviated from the driving route based on the detection result of the encoder. 12. The method of claim 11,
Wherein,
Wherein the control unit determines that a dangerous situation has occurred in the cleaner when wheels of the driving unit are restrained or idle based on the detection result of the encoder. The method according to claim 1,
Wherein the sensor includes a camera for capturing an image associated with the periphery of the body,
Wherein,
Wherein the controller determines that a dangerous situation has occurred in the cleaner based on the image photographed by the camera. 15. The method of claim 14,
Wherein,
Wherein the controller determines that a dangerous situation has occurred in the cleaner when a difference is detected between a plurality of images photographed by the camera at a predetermined time interval while the main body is in a stopped state. The method according to claim 1,
Wherein,
Wherein the control unit controls the communication unit to transmit an analysis algorithm update request to the server to update the analysis algorithm if it is determined that the main body enters a dangerous situation. 17. The method of claim 16,
Wherein,
And in response to the analysis algorithm update request, updates the analysis algorithm using analysis algorithm update information received from the server. 1. A robot cleaner system comprising a vacuum cleaner for performing autonomous traveling and a server for communicating with the vacuum cleaner,
The cleaner includes:
main body;
A driving unit for moving the main body;
A sensor for sensing information related to the environment in which the body is located;
A communication unit for communicating with the server; And
Wherein the control unit judges whether or not the main body enters a dangerous situation based on the information detected by the sensor, and transmits the information related to the dangerous situation to the server, And a control unit for controlling the control unit,
The server comprises:
Analyzing information related to a dangerous situation transmitted from the cleaner using the training data stored in the server, transmitting analysis results to the cleaner,
Updating the training data using information related to a dangerous situation transmitted from the cleaner,
The control unit of the vacuum cleaner,
And a controller for setting an operation mode corresponding to the dangerous situation based on the analysis result transmitted from the server. 19. The method of claim 18,
The control unit of the vacuum cleaner,
Controls the communication unit so that information related to an object existing around the main body obtained by the sensor after the execution of the set operation mode is completed is transmitted to the server,
The server comprises:
Wherein the accuracy of the analysis result is determined using information related to an object existing around the main body received from the cleaner after execution of the set operation mode is completed. 19. The method of claim 18,
Further comprising a mobile terminal that communicates with the cleaner and the server,
The mobile terminal includes:
Outputting the monitoring information received from the cleaner,
In response to the monitoring information being output, receiving user input related to a risk situation,
And transmits information related to the dangerous situation to a server based on the user input.

Images