WO2020045732A1

WO2020045732A1 - Method for controlling mobile robot

Info

Publication number: WO2020045732A1
Application number: PCT/KR2018/011837
Authority: WO
Inventors: 이성훈; 조원철
Original assignee: 엘지전자 주식회사
Priority date: 2018-08-27
Filing date: 2018-10-08
Publication date: 2020-03-05
Also published as: KR20200027072A; KR102290983B1

Abstract

A mobile robot according to one aspect of the present invention is capable of uttering speech to a user, thus being able to converse and interact with the user through speech. The mobile robot can proactively provide information and recommend services, functions, etc., prior to receiving a request, and can thus derive increased levels of trust, favor, and product utilization from the user.

Description

Mobile Robot Control Method

The present invention relates to a mobile robot and a control method thereof, and more particularly, to a mobile robot and a control method capable of actively providing information and services to a user.

Robots have been developed for industrial use and have been a part of factory automation. Recently, the application of robots has been further expanded, medical robots, aerospace robots, and the like have been developed, and home robots that can be used in general homes have also been made. Among these robots, a moving robot capable of traveling by magnetic force is called a mobile robot.

A representative example of a mobile robot used at home is a robot cleaner, which is a device that cleans a corresponding area by inhaling dust or foreign matter while driving around a certain area by itself.

The mobile robot is capable of moving by itself and is free to move, and is provided with a plurality of sensors for avoiding obstacles and the like while driving, so that the robot can travel to avoid obstacles.

Meanwhile, in order to improve user convenience, voice recognition technology is applied to various devices, and researches on a method of controlling a mobile robot using voice recognition technology are increasing.

For example, prior art document 1 (Korean Patent Publication No. 10-2012-0114670, published on October 17, 2012) discloses that the robot cleaner has a speech recognition unit and recognizes a user's speech signal to provide a corresponding control command. Is running.

In Prior Art 1, the voice input is unidirectional from the user to the robot cleaner, so that it stays in the additional means of the control operation of pressing a button or operating with a remote controller. Accordingly, there is a problem that the voice recognition function is hard to give the user more than simple control, and cannot provide other functions and services other than the addition of control input means.

The object of the present invention is to provide a mobile robot capable of interacting with a user through a voice and a control method thereof by using voice recognition as a single control input means.

An object of the present invention is to provide a mobile robot with various information and services to a user.

Disclosure of Invention An object of the present invention is to provide a mobile robot and a control method thereof capable of user-friendly operation to improve user reliability and preference.

In order to achieve the above or another object, the mobile robot according to an aspect of the present invention may utter a voice for guiding predetermined information or a service to a user, and may communicate with and interact with the user through the voice. .

In order to achieve the above or another object, the mobile robot according to an aspect of the present invention first increases the reliability, preference, and product utilization of the user by actively providing predetermined information and recommending predetermined services and functions before requesting them. Can be.

In order to achieve the above or another object, a method of operating a mobile robot according to an aspect of the present invention includes: performing a cleaning operation while moving, determining a special region based on existing data obtained by performing a previous cleaning; And outputting a voice guidance message for the special region when the determined special region arrives, thereby providing voice guidance for the special region that needs to be actively provided before the user's request. have.

Here, the special area may be a dangerous area in which an inoperable state has occurred, or a low efficiency area in which the running speed or cleaning rate is lower than the average value or the number of rotations is greater than the reference number.

In addition, the existing data obtained by the previous cleaning operation may include driving history data and cleaning history data.

In addition, the existing data obtained by performing the previous cleaning may further include obstacle information registered in the special area, and in this case, the voice guidance message may include a guide message for the obstacle present in the special area. have.

Meanwhile, the voice guidance message may include a message for requesting confirmation of cleaning of the special area. In this case, the voice guidance message may further include at least one of a description of the reason for determining as the special region or a guide for an example of a command that a user can input voice.

According to an aspect of the present invention, there is provided a method of operating a mobile robot, the method comprising: receiving voice feedback of a user with respect to the voice guidance message, identifying a voice command included in the received voice feedback, and identifying the voice command. The method may further include performing a corresponding operation.

In this case, the identification of the voice command may be performed by itself in the mobile robot, in the server, or stepwise in the mobile robot and the server.

According to an aspect of the present invention, there is provided a method of operating a mobile robot, the method comprising: receiving at least one voice command of a user and recommending at least one of functions used less than a reference number based on a usage record; Outputting a voice announcement message, identifying a feedback voice command included in the received voice feedback when the user's voice feedback is received, and a predetermined function based on the identified feedback voice command. By including the setting step, it is possible to actively recommend useful functions among the functions not frequently used by the user.

Here, the usage record includes the number of times the general cleaning is completed, the number of times of returning to the charging station, the number of cleaning functions for each area, the operating time, the average cleaning intensity for each area, the amount of dust for each area, the frequency of use of the cleaning mode for each area, the ratio of the cleaning mode for each area, and the cleaning. It may include at least one of the number of uses per function and additional functions.

In addition, the mobile robot may select and recommend at least one of functions used less than the reference number of times according to a predetermined priority. In this case, the preset priority may be an order of high importance for each preset function or an order of small number of times of use.

The method of operating a mobile robot according to an aspect of the present invention may further include identifying a voice command of a user. In this case, an operation corresponding to the voice command of the identified user may be performed, and a function associated with the voice command of the identified user may be selected and recommended from among functions used less than the reference number.

Meanwhile, the identification of the voice command and the feedback voice command may be performed by the mobile robot by itself, by the server, or step by step by the mobile robot and the server.

According to at least one of the embodiments of the present disclosure, the mobile robot may speak a voice to the user, and may communicate with and interact with the user through the voice.

In addition, according to at least one of the embodiments of the present invention, the mobile robot may actively provide information and recommend services, functions, and the like before requesting, thereby increasing the reliability, preference, and product utilization of the user.

In addition, according to at least one of the embodiments of the present invention, by actively guiding a dangerous area, a low-efficiency area that is not easy to clean, to improve the stability of the mobile robot and user convenience, driving efficiency, cleaning efficiency Etc. can be improved.

In addition, according to at least one of the embodiments of the present invention, by actively recommending useful functions among functions that are not frequently used by the user, it is possible to increase the reliability, preference, and product utilization of the user.

In addition, according to at least one of the embodiments of the present invention, by recommending another function associated with the user's voice command, the user can easily set up and use the associated function without additional effort.

In addition, according to at least one of the embodiments of the present invention, the speech recognition is performed by the mobile robot by itself, by the server, or by the mobile robot and the server step by step, thereby enabling effective speech recognition.

In addition, according to at least one of the embodiments of the present invention, by using artificial intelligence and machine learning for voice recognition, obstacle recognition, product data analysis, etc., an evolving user experience may be provided.

On the other hand, various other effects will be disclosed directly or implicitly in the detailed description according to the embodiment of the present invention to be described later.

1 is a block diagram of a home appliance network system according to an embodiment of the present invention.

2 is a perspective view showing a mobile robot according to an embodiment of the present invention.

3 is a plan view of the mobile robot of FIG. 2.

4 is a side view of the mobile robot of FIG. 2.

5 is a block diagram showing a control relationship between major components of a mobile robot according to an embodiment of the present invention.

FIG. 6 is a view referred to for describing learning using product data according to an embodiment of the present invention.

7 is an example of a simplified internal block diagram of a server according to an embodiment of the present invention.

8 is a flowchart illustrating a control method of a mobile robot according to an embodiment of the present invention.

9 is a view referred to for describing the control method of the mobile robot according to an embodiment of the present invention.

10 is a flowchart illustrating a control method of a mobile robot according to an embodiment of the present invention.

11 is a flowchart illustrating a control method of a mobile robot according to an embodiment of the present invention.

12 is a view referred to for describing the method for controlling the mobile robot according to the embodiment of the present invention.

13 is a flowchart illustrating a control method of a mobile robot according to an embodiment of the present invention.

14 is a view referred to for describing the control method of the mobile robot according to the embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings will be described an embodiment of the present invention; However, the present invention is not limited to these embodiments and may be modified in various forms.

In the drawings, parts not related to the description are omitted in order to clearly and briefly describe the present invention, and the same reference numerals are used for the same or extremely similar parts throughout the specification.

On the other hand, the suffixes "module" and "unit" for the components used in the following description are merely given in consideration of ease of preparation of the present specification, and do not give particular meaning or role in themselves. Therefore, the "module" and "unit" may be used interchangeably.

The mobile robot 100 according to an embodiment of the present invention refers to a robot that can move itself by using a wheel or the like, and may be a home helper robot or a robot cleaner. Hereinafter, a robot cleaner having a cleaning function among mobile robots will be described with reference to the drawings, but the present invention is not limited thereto.

Referring to FIG. 1, a home appliance network system may include a home appliance including a communication module, which may communicate with another device, the server 70, or connect to a network.

For example, the home appliance may correspond to an air conditioner 10 having a communication module, a cleaner 20, a refrigerator 31, a washing machine 32, and the like.

The air conditioner 10 may include at least one of an air conditioner 11, an

air cleaner

12 and 13, a humidifier 14, and a hood 15.

In addition, the cleaner 20 may be a vacuum cleaner 21, a robot cleaner 22, or the like.

Meanwhile, the communication module included in the

home appliances

10, 20, 31, and 32 may be a Wi-Fi communication module, and the present invention is not limited to the communication method.

Alternatively, the

home appliances

10, 20, 31, and 32 may include other types of communication modules or may include a plurality of communication modules. For example, the

home appliances

10, 20, 31, and 32 may include an NFC module, a Zigbee communication module, a Bluetooth ™ communication module, and the like.

The

home appliances

10, 20, 31, and 32 may be connected to a predetermined server 70 through a Wi-Fi communication module, and may support smart functions such as remote monitoring and remote control.

The home appliance network system according to an embodiment of the present invention may include a mobile terminal 50 such as a smart phone and a tablet PC.

The user may check information on the

home appliances

10, 20, 31, and 32 in the home appliance network system or control the

home appliances

10, 20, 31, and 32 through the portable terminal 50.

Meanwhile, the home appliance network system according to an embodiment of the present invention may include a plurality of Internet of Things (IoT) devices (not shown). Thus, the home appliance network system may include

home appliances

10, 20, 31, and 32, portable terminal 50, and Internet of Things (IoT) devices.

The home appliance network system according to an embodiment of the present invention is not limited to a communication scheme constituting a network.

For example, the

home appliances

10, 20, 31, and 32, the portable terminal 50, and the Internet of Things (IoT) devices may be connected through a wired / wireless router (not shown).

In addition, devices in the home appliance network system may form a mesh topology that is individually communicated with each other.

The

home appliances

10, 20, 31, and 32 in the home appliance network system may communicate with the server 70 or the mobile terminal 50 via a wired / wireless router (not shown). In addition, the

home appliances

10, 20, 31, and 32 in the home appliance network system may communicate with the server 70 or the portable terminal 50 by Ethernet.

2 is a perspective view illustrating a mobile robot according to an embodiment of the present invention, FIG. 3 is a plan view of the mobile robot of FIG. 2, and FIG. 4 is a side view of the mobile robot of FIG. 2.

2 to 4, the mobile robot 100 may drive a certain area by itself. The mobile robot 100 may perform a function of cleaning the floor. Cleaning of the floor here includes suctioning dust (including foreign matter) from the floor or mopping the floor.

The mobile robot 100 includes a main body 110. The main body 110 includes a cabinet forming an appearance. The mobile robot 100 may include a suction unit 130 and a dust container 140 provided in the main body 110. The mobile robot 100 includes an image acquisition unit 120 that detects information related to an environment around the mobile robot. The mobile robot 100 includes a driving unit 160 for moving the main body. The mobile robot 100 includes a control unit 181 for controlling the mobile robot 100. The controller 181 is provided in the main body 110.

The driving unit 160 includes a wheel unit 111 for traveling of the mobile robot 100. The wheel unit 111 is provided in the main body 110. The mobile robot 100 may be moved back, front, left, and right by the wheel unit 111 or rotated. As the controller controls the driving of the wheel unit 111, the mobile robot 100 may autonomously travel the floor. The wheel unit 111 includes a main wheel 111a and a sub wheel 111b.

The main wheels 111a are provided at both sides of the main body 110, and are configured to be rotatable in one direction or the other direction according to the control signal of the controller. Each main wheel 111a may be configured to be driven independently of each other. For example, each main wheel 111a may be driven by different motors.

The sub wheel 111b supports the main body 110 together with the main wheel 111a, and is configured to assist driving of the mobile robot 100 by the main wheel 111a. The sub wheel 111b may also be provided in the suction unit 130 described later.

The suction unit 130 may be disposed to protrude from the front side F of the main body 110. The suction unit 130 is provided to suck air containing dust.

The suction unit 130 may have a form protruding from the front of the main body 110 to both left and right sides. The front end of the suction unit 130 may be disposed in a position spaced forward from one side of the main body 110. The left and right both ends of the suction unit 130 may be disposed at positions spaced apart from the main body 110 to the left and right sides.

The main body 110 is formed in a circular shape, and as both rear ends of the suction unit 130 protrude from the main body 110 to the left and right sides, respectively, an empty space, that is, between the main body 110 and the suction unit 130. Gaps may be formed. The empty space is a space between the left and right both ends of the main body 110 and the left and right both ends of the suction unit 130, and has a shape recessed inside the mobile robot 100.

The suction unit 130 may be detachably coupled to the main body 110. When the suction unit 130 is separated from the main body 110, the mop module (not shown) may be detachably coupled to the main body 110 in place of the separated suction unit 130.

The image acquisition unit 120 may be disposed in the main body 110. The image acquisition unit 120 may be disposed in front of the main body 110. The image acquisition unit 120 may be disposed to overlap the suction unit 130 in the vertical direction of the main body 110. The image acquisition unit 120 may be disposed above the suction unit 130.

The image acquisition unit 120 may detect an obstacle around the mobile robot 100. The image acquisition unit 120 may detect an obstacle or a feature in front of the suction unit 130 located in the front of the mobile robot 100 so as not to collide with the obstacle. The image acquisition unit 120 may further perform other sensing functions to be described later in addition to the sensing function.

The main body 110 may be provided with a dust container accommodating part (not shown). The dust container 140 is detachably coupled to the dust container 140 which separates and collects dust in the sucked air. The dust container accommodation part may be formed at the rear side R of the main body 110. Part of the dust container 140 is accommodated in the dust container receiving portion, the other part of the dust container 140 may be formed to protrude toward the rear (R) of the main body 110.

The dust container 140 has an inlet (not shown) through which air containing dust is introduced and an outlet (not shown) through which air from which dust is separated is formed. When the dust container 140 is mounted on the dust container housing, the inlet and the outlet of the dust container 140 are configured to communicate with the first opening (not shown) and the second opening (not shown) respectively formed on the inner wall of the dust container accommodation. .

A suction flow path (not shown) for guiding air from the suction port of the suction unit 130 to the first opening is provided. An exhaust passage (not shown) for guiding air to an exhaust port (not shown) opened toward the outside of the second opening is provided.

The air containing the dust introduced through the suction unit 130 is introduced into the dust container 140 through the intake passage inside the main body 110, and the air and the dust are passed through the filter or the cyclone of the dust container 140. Are separated from each other. Dust is collected in the dust container 140, the air is discharged from the dust container 140, and then through the exhaust flow path inside the main body 110 is finally discharged to the outside through the exhaust port.

2 to 5, the mobile robot 100 includes a main body 110 and an image acquisition unit 120 that acquires an image around the main body 110.

The mobile robot 100 includes a driving unit 160 for moving the main body 110. The driving unit 160 includes at least one wheel unit 111 for moving the main body 110. The driving unit 160 includes a driving motor (not shown) connected to the wheel unit 111 to rotate the wheel unit 111.

The image acquisition unit 120 photographs the driving zone and may include a camera module. The camera module may include a digital camera. The digital camera includes at least one optical lens and an image sensor (eg, a CMOS image sensor) including a plurality of photodiodes (eg, pixels) formed by the light passing through the optical lens. The apparatus may include a digital signal processor (DSP) that forms an image based on signals output from the photodiodes. The digital signal processor may generate not only a still image but also a moving image composed of frames composed of the still image.

Multiple cameras may be installed for each part for photographing efficiency. The image photographed by the camera may be used to recognize a kind of material such as dust, hair, floor, etc. present in the corresponding space, whether to clean or check the cleaning time.

The camera may photograph a situation of an obstacle or a cleaning area existing on the front of the moving direction of the mobile robot 100.

According to an embodiment of the present invention, the image acquisition unit 120 may acquire a plurality of images by continuously photographing the periphery of the main body 110, and the obtained plurality of images may be stored in the storage unit 105. Can be.

The mobile robot 100 improves the accuracy of spatial recognition, location recognition, and obstacle recognition using a plurality of images, or selects one or more images from the plurality of images and uses effective data to provide spatial recognition, location recognition, and obstacle recognition. You can increase the accuracy.

In addition, the mobile robot 100 may include a sensor unit 170 including sensors for sensing various data related to the operation and state of the mobile robot.

For example, the sensor unit 170 may include an obstacle detecting sensor detecting a front obstacle. In addition, the sensor unit 170 may further include a cliff detection sensor for detecting the presence of a cliff on the floor in the driving zone, and a lower camera sensor for obtaining an image of the floor.

The obstacle detecting sensor may include an infrared sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, a position sensitive device (PSD) sensor, and the like.

On the other hand, the position and type of the sensor included in the obstacle detection sensor may vary depending on the type of the mobile robot, the obstacle detection sensor may include more various sensors.

On the other hand, the sensor unit 170 may further include a motion detection sensor for detecting the operation of the mobile robot 100 according to the driving of the main body 110 and outputs the motion information. For example, a gyro sensor, a wheel sensor, an acceleration sensor, or the like may be used as the motion detection sensor.

The gyro sensor detects the rotation direction and detects the rotation angle when the mobile robot 100 moves according to the driving mode. The gyro sensor detects the angular velocity of the mobile robot 100 and outputs a voltage value proportional to the angular velocity. The controller 150 calculates the rotation direction and the rotation angle by using the voltage value output from the gyro sensor.

The wheel sensor is connected to the wheel unit 111 to sense the number of revolutions of the wheel. Here, the wheel sensor may be a rotary encoder.

The acceleration sensor detects a change in the speed of the mobile robot 100, for example, a change in the mobile robot 100 due to start, stop, direction change, collision with an object, and the like.

In addition, the acceleration sensor may be built in the controller 150 to detect a speed change of the mobile robot 100.

The controller 150 may calculate a position change of the mobile robot 100 based on the motion information output from the motion detection sensor. This position becomes a relative position corresponding to the absolute position using the image information. The mobile robot can improve the performance of position recognition using image information and obstacle information through the relative position recognition.

On the other hand, the mobile robot 100 may include a power supply unit (not shown) for supplying power to the mobile robot by having a rechargeable battery.

The power supply unit supplies driving power and operation power to each component of the mobile robot 100, and when the remaining power is insufficient, power may be supplied and charged from a charging stand (not shown).

The mobile robot 100 may further include a battery detector (not shown) that detects a charging state of the battery and transmits a detection result to the controller 150. The battery is connected to the battery detector so that the battery remaining amount and the charging state are transmitted to the controller 150. The battery remaining amount may be displayed on the display 182 of the output unit 180.

In addition, the mobile robot 100 includes an input unit 125 for inputting on / off or various commands. The input unit 125 may include a button, a dial, a touch screen, and the like. The input unit 125 may include a microphone for receiving a user's voice command. Through the input unit 125, various control commands necessary for the overall operation of the mobile robot 100 may be input.

In addition, the mobile robot 100 may include an output unit 180 to display reservation information, a battery state, an operation mode, an operation state, an error state, etc. as an image or output a sound.

The output unit 180 may include a sound output unit 181 for outputting an audio signal. The sound output unit 181 may output a warning message, such as a warning sound, an operation mode, an operation state, an error state, etc., under the control of the controller 150. The sound output unit 181 may convert an electrical signal from the controller 150 into an audio signal and output the audio signal. To this end, a speaker or the like may be provided.

In addition, the output unit 180 may further include a display 182 that displays reservation information, a battery state, an operation mode, an operation state, an error state, and the like as an image.

Referring to FIG. 5, the mobile robot 100 includes a controller 150 for processing and determining various information such as recognizing a current location, and a storage 105 for storing various data. In addition, the mobile robot 100 may further include a communication unit 190 for transmitting and receiving data with an external terminal.

The external terminal includes an application for controlling the mobile robot 100, and displays an map of the driving area to be cleaned by the mobile robot 100 through execution of the application, and designates an area to clean a specific area on the map. have. Examples of the external terminal may include a remote controller, a PDA, a laptop, a smartphone, a tablet, and the like, having an application for setting a map.

The external terminal may communicate with the mobile robot 100 to display a current location of the mobile robot together with a map, and information about a plurality of areas may be displayed. In addition, the external terminal updates and displays its position as the mobile robot travels.

The controller 150 controls the image acquisition unit 120, the input unit 125, the driving unit 160, the suction unit 130, etc. constituting the mobile robot 100 to control the overall operation of the mobile robot 100. To control.

The controller 150 may process a voice input signal of the user received through the microphone of the input unit 125 and perform a voice recognition process. In some embodiments, the mobile robot 100 may include a voice recognition module that performs voice recognition inside or outside the controller 150.

According to an exemplary embodiment, simple voice recognition may be performed by the mobile robot 100 itself, and high-level voice recognition such as natural language processing may be performed by the server 70.

The storage unit 105 records various types of information necessary for the control of the mobile robot 100 and may include a volatile or nonvolatile recording medium. The recording medium stores data that can be read by a microprocessor, and includes a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic Tapes, floppy disks, optical data storage devices, and the like.

In addition, the storage unit 105 may store a map for the driving zone. The map may be input by an external terminal, a server, or the like, which may exchange information with the mobile robot 100 through wired or wireless communication, or may be generated by the mobile robot 100 by learning itself.

The map may display the locations of the rooms in the driving zone. In addition, the current position of the mobile robot 100 may be displayed on the map, and the current position of the mobile robot 100 on the map may be updated during the driving process. The external terminal stores the same map as the map stored in the storage unit 105.

The storage unit 105 may store cleaning history information. Such cleaning history information may be generated every time cleaning is performed.

The map of the driving zone stored in the storage unit 105 stores a navigation map used for driving during cleaning, a Simultaneous Localization and Mapping (SLAM) map used for location recognition, an obstacle, and the like. It may be a learning map used for learning cleaning, a global location map used for global location recognition, an obstacle recognition map in which information about the recognized obstacle is recorded.

Meanwhile, as described above, maps may be stored and managed in the storage unit 105 for each use, but the map may not be clearly classified for each use. For example, a plurality of pieces of information may be stored in one map to be used for at least two purposes.

The controller 150 may include a driving control module 151, a map generation module 152, a position recognition module 153, and an obstacle recognition module 154.

1 to 5, the driving control module 151 controls the driving of the mobile robot 100, and controls the driving of the driving unit 160 according to the driving setting. In addition, the driving control module 151 may determine the driving path of the mobile robot 100 based on the operation of the driving unit 160. For example, the driving control module 151 may determine the current or past moving speed of the mobile robot 100, the distance traveled, and the like based on the rotational speed of the wheel unit 111, and the mobile robot thus identified ( Based on the driving information of the 100, the position of the mobile robot 100 on the map may be updated.

The map generation module 152 may generate a map of the driving zone. The map generation module 152 may generate a map by processing the image acquired through the image acquisition unit 120. That is, a cleaning map corresponding to the cleaning area can be created.

In addition, the map generation module 152 may recognize the global position by processing the image acquired through the image acquisition unit 120 at each position in association with the map.

The position recognition module 153 estimates and recognizes a current position. The position recognition module 153 uses the image information of the image acquisition unit 120 to determine the position in connection with the map generation module 152 to estimate the current position even when the position of the mobile robot 100 suddenly changes. Can be recognized.

In addition, the location recognition module 153 may recognize the property of the current location, that is, the location recognition module 153 may recognize the space.

The mobile robot 100 may recognize a position during continuous driving through the position recognition module 153, and also, through the map generation module 152 and the obstacle recognition module 154, without the position recognition module 153. Learn, estimate your current location, and more.

While the mobile robot 100 is traveling, the image acquisition unit 120 acquires images around the mobile robot 100. Hereinafter, an image acquired by the image acquisition unit 120 is defined as an 'acquisition image'.

The acquired image includes various features such as lightings on the ceiling, edges, corners, blobs, and ridges.

The map generation module 152 detects a feature from each of the acquired images, and calculates a descriptor based on each feature point.

The map generation module 152 classifies at least one descriptor into a plurality of groups according to a predetermined sub-classification rule for each acquired image based on descriptor information obtained through the acquired image of each position, and according to the predetermined sub-representation rule, the same group. Descriptors included in each can be converted into lower representative descriptors.

As another example, all descriptors collected from acquired images in a predetermined area, such as a room, are classified into a plurality of groups according to a predetermined sub-classification rule, and descriptors included in the same group according to the predetermined sub-representation rule are respectively represented by lower representative descriptors. You can also convert to.

The map generation module 152 can obtain the feature distribution of each location through this process. Each positional feature distribution can be represented by a histogram or an n-dimensional vector. As another example, the map generation module 152 may estimate an unknown current position based on a descriptor calculated from each feature point without passing through a predetermined sub classification rule and a predetermined sub representative rule.

In addition, when the current position of the mobile robot 100 is unknown due to a position leap or the like, the current position may be estimated based on data such as a previously stored descriptor or a lower representative descriptor.

The mobile robot 100 obtains an acquired image through the image acquisition unit 120 at an unknown current position. Through the image, various features such as lightings on the ceiling, edges, corners, blobs, and ridges are identified.

The position recognition module 153 detects features from the acquired image and calculates a descriptor.

The position recognition module 153 is based on at least one descriptor information obtained through an acquired image of an unknown current position, and position information (for example, feature distribution of each position) to be compared according to a predetermined lower conversion rule. Convert to comparable information (sub-recognition feature distribution).

According to a predetermined sub-comparison rule, each position feature distribution may be compared with each recognition feature distribution to calculate each similarity. Similarity (probability) may be calculated for each location corresponding to each location, and the location where the greatest probability is calculated may be determined as the current location.

As described above, the controller 150 may distinguish the driving zone and generate a map composed of a plurality of regions, or recognize the current position of the main body 110 based on the pre-stored map.

When the map is generated, the controller 150 may transmit the generated map to an external terminal, a server, etc. through the communication unit 190. As described above, the controller 150 may store the map in the storage 105 when a map is received from an external terminal, a server, or the like.

In this case, the map may be divided into a plurality of cleaning areas, and include a connection path connecting the plurality of areas, and may include information about obstacles in the area.

When the cleaning command is input, the controller 150 determines whether the position on the map matches the current position of the mobile robot. The cleaning command may be input from a remote controller, an input unit, or an external terminal.

If the current position does not match the position on the map or the current position cannot be confirmed, the controller 150 recognizes the current position and recovers the current position of the mobile robot 100 based on the current position. The driving unit 160 may be controlled to move to the designated area.

If the current position does not match the position on the map or if the current position cannot be confirmed, the position recognition module 153 analyzes the acquired image input from the image acquisition unit 120 to estimate the current position based on the map. can do. In addition, the obstacle recognition module 154 or the map generation module 152 may also recognize the current position in the same manner.

After recognizing the position and restoring the current position of the mobile robot 100, the driving control module 151 calculates a driving route from the current position to the designated region and controls the driving unit 160 to move to the designated region.

When the cleaning pattern information is received from the server, the driving control module 151 may divide the entire driving zone into a plurality of areas according to the received cleaning pattern information, and set at least one area as a designated area.

In addition, the driving control module 151 may calculate the driving route according to the received cleaning pattern information, travel along the driving route, and perform cleaning.

The controller 150 may store the cleaning record in the storage unit 105 when cleaning of the set designated area is completed.

In addition, the controller 150 may transmit the operation state or cleaning state of the mobile robot 100 to the external terminal and the server at a predetermined cycle through the communication unit 190.

Accordingly, the external terminal displays the location of the mobile robot along with the map on the screen of the running application based on the received data, and outputs information on the cleaning state.

The mobile robot 100 according to an embodiment of the present invention moves in one direction until an obstacle or a wall surface is detected, and when the obstacle recognition module 154 recognizes the obstacle, the robot moves straight, rotates, etc. according to the recognized characteristics of the obstacle. The pattern can be determined.

On the other hand, the controller 150 may control to perform the avoidance driving in a different pattern based on the recognized property of the obstacle. The controller 150 may control to avoid driving in different patterns according to the properties of obstacles such as non-hazardous obstacles (general obstacles), dangerous obstacles, and movable obstacles.

For example, the controller 150 may control the dangerous obstacle to be bypassed in a state where a safe distance of a longer distance is secured.

In addition, in the case of the movable obstacle, if the obstacle does not move after a predetermined waiting time, the controller 150 may control to perform the avoiding driving corresponding to the general obstacle or the avoiding driving corresponding to the dangerous obstacle. Alternatively, when the avoidance driving pattern corresponding to the movable obstacle is set separately, the controller 150 may control to travel accordingly.

The mobile robot 100 according to the embodiment of the present invention may perform obstacle recognition and avoidance based on machine learning.

The controller 150 may drive the driving unit 160 based on an obstacle recognition module 154 that recognizes an obstacle previously learned by machine learning in an input image and an attribute of the recognized obstacle. It may include a driving control module 151 for controlling.

Meanwhile, although FIG. 5 illustrates an example in which the plurality of

modules

151, 152, 153, and 154 are separately provided in the controller 160, the present invention is not limited thereto.

For example, the position recognition module 153 and the obstacle recognition module 154 may be integrated into one recognizer and constitute one recognition module 155. In this case, the recognizer may be trained using a learning technique such as machine learning, and the learned recognizer may recognize attributes of an area, an object, and the like by classifying data input thereafter.

According to an embodiment, the map generation module 152, the position recognition module 153, and the obstacle recognition module 154 may be configured as one integrated module.

In the following description, the position recognition module 153 and the obstacle recognition module 154 are integrated as one recognizer and described with reference to an embodiment configured as one recognition module 155, but the position recognition module 153 and the obstacle recognition are described. The module 154 may operate in the same manner even when each is provided.

The mobile robot 100 according to an exemplary embodiment of the present invention may include a recognition module 155 in which attributes of objects and spaces are learned by machine learning.

Machine learning means that a computer can learn from data and let the computer take care of a problem without having to instruct the computer directly to the logic.

Deep Learning Based on Artificial Neural Networks (ANN) for constructing artificial intelligence, this is an artificial intelligence technology that teaches a computer's way of thinking to a person without the need for a person. .

The artificial neural network (ANN) may be implemented in software or in the form of hardware such as a chip.

The recognition module 155 may include an artificial neural network (ANN) in the form of software or hardware in which properties of an object, such as an object of a space or an obstacle, are learned.

For example, the recognition module 155 may include a deep neural network (DNN) such as a convolutional neural network (CNN), a recurrent neural network (RNN), a deep belief network (DBN), and the like that have been learned by deep learning. It may include.

The recognition module 155 may determine an attribute of a space and an object included in the input image data based on weights among nodes included in the deep neural network DNN.

On the other hand, the driving control module 151 may control the driving of the driving unit 160 based on the recognized space and the properties of the obstacle.

The recognition module 155 may recognize attributes of spaces and obstacles included in the selected specific viewpoint image based on data previously learned by machine learning.

The storage unit 105 may store space, input data for determining object properties, and data for learning the deep neural network DNN.

The storage unit 105 may store the original image obtained by the image acquisition unit 120 and the extracted images from which the predetermined region is extracted.

In some embodiments, the storage 105 may store weights and biases forming the deep neural network (DNN).

Alternatively, weights and biases constituting the deep neural network structure may be stored in an embedded memory of the recognition module 155.

On the other hand, the recognition module 155 performs a learning process by using a predetermined image as training data whenever the image acquisition unit 120 acquires an image or extracts a partial region of the image, or a predetermined number or more. After the image is acquired, the learning process may be performed.

Alternatively, the mobile robot 100 may receive data related to machine learning from the predetermined server through the communication unit 190.

In this case, the mobile robot 100 may update the recognition module 155 based on data related to machine learning received from the predetermined server.

Referring to FIG. 6, product data obtained by the operation of a predetermined device such as a mobile robot 100 may be transmitted to the server 70.

For example, the mobile robot 100 may transmit a space, an object, and usage related data to the server 70 to the server 70.

Here, the space and object related data may be a space recognized by the mobile robot 100 and data related to recognition of an object, or a space obtained by the image acquisition unit 120. And image data about an object.

In addition, the usage-related data (Data) is data obtained according to the use of a predetermined product, for example, the mobile robot 100, the use history data, the sensing data obtained from the sensor unit 170, etc. Can be.

On the other hand, the control unit 150, more specifically, the recognition module 155 of the mobile robot 100 may be equipped with a deep neural network structure (DNN) such as a convolutional neural network (CNN).

The learned deep neural network structure DNN may receive input data for recognition, recognize a property of an object and a space included in the input data, and output the result.

In addition, the learned deep neural network structure (DNN) may receive input data for recognition, analyze and learn the usage-related data (Data) of the mobile robot 100 to recognize the use pattern, the use environment, and the like. have.

Meanwhile, the space, object, and usage related data may be transmitted to the server 70 through the communication unit 190.

The server 70 may generate a configuration of learned weights, and the server 70 may learn a deep neural network (DNN) structure using training data.

After the server 70 learns the deep neural network (DNN) based on the received data, the server 70 may transmit the updated deep neural network (DNN) structure data to the mobile robot 100 to be updated.

Accordingly, home appliance products such as mobile robot 100 may become smarter and provide an evolving user experience (UX) as they are used.

Referring to FIG. 7, the server 70 may include a communication unit 720, a storage unit 730, a learning module 740, and a processor 710.

The processor 710 may control the overall operation of the server 70.

Meanwhile, the server 70 may be a server operated by a home appliance manufacturer such as the mobile robot 100 or a server operated by a service provider, or may be a kind of cloud server.

The communication unit 720 may receive various data such as status information, operation information, operation information, and the like from a portable terminal, a home appliance such as the mobile robot 100, a gateway, or the like.

The communication unit 720 may transmit data corresponding to the received various information to a portable terminal, a home appliance such as the mobile robot 100, a gateway, or the like.

To this end, the communication unit 720 may include one or more communication modules, such as an internet module and a mobile communication module.

The storage unit 730 may store the received information and may include data for generating result information corresponding thereto.

In addition, the storage unit 730 may store data used for machine learning, result data, and the like.

The learning module 740 may serve as a learner of a home appliance such as the mobile robot 100.

The learning module 740 may include an artificial neural network, for example, a deep neural network (DNN) such as a convolutional neural network (CNN), a recurrent neural network (RNN), and a deep belief network (DBN). You can learn neural networks.

Meanwhile, the controller 710 may control to update the artificial neural network structure of the home appliance such as the mobile robot 100 to the learned artificial neural network structure after learning according to a setting.

In addition, the learning module 740 may receive input data for recognition, recognize a property of an object and a space included in the input data, and output the result. In this case, the communication unit 720 may transmit the recognition result to the mobile robot 100.

In addition, the learning module 740 may analyze and learn usage-related data of the mobile robot 100 to recognize a usage pattern, a usage environment, and the like, and output the result. In this case, the communication unit 720 may transmit the recognition result to the mobile robot 100.

Accordingly, home appliance products such as the mobile robot 100 may receive a recognition result from the server 70 and operate by using the received recognition result.

In addition, as the server 70 becomes smarter by learning using the product data, it is possible to provide an evolving user experience (UX) as using the home appliance product.

Meanwhile, the mobile robot 100 and the server 70 may also use external information.

For example, the mobile robot 100 and the server 70 may be obtained from spatial information of a specific home appliance product such as the mobile robot 100, object information, internal information such as a usage pattern, and other products, or the server 70. Can provide excellent user experience by comprehensively using external information obtained from other linked service servers.

According to an embodiment of the present invention, it is possible to optimize a product operating sequence in a home appliance network system connected by Wi-Fi through artificial intelligence and to present a user with an optimal operating sequence.

For example, cleaning the house with the mobile robot 100, dust management during cleaning with the

air cleaners

12 and 13, and controlling the humidity and temperature of the house with the air conditioner 11, washing machine 32 Washing may be performed such that washing is finished in accordance with the time when the user arrives at home.

The server 70 may perform voice recognition by receiving a voice input signal spoken by a user. To this end, the server 70 may include a speech recognition module, and the speech recognition module may include an artificial neural network trained to perform speech recognition on input data and output a speech recognition result.

Meanwhile, the server 70 may include a voice recognition server for voice recognition. In addition, the voice recognition server may also include a plurality of servers that share a predetermined process of the voice recognition process. For example, a speech recognition server may include an automatic speech recognition (ASR) server that receives speech data and converts the received speech data into text data, and the text from the automatic speech recognition server. It may include a natural language processing (NLP) server that receives the data, and analyzes the received text data to determine the voice command. In some cases, the speech recognition server may further include a text to speech (TTS) server that converts the text speech recognition result output from the natural language processing server into speech data and transmits the speech data to another server or a home appliance. .

According to the present invention, the mobile robot 100 and / or the server 70 may perform voice recognition, so that a user voice may be used as an input for controlling the mobile robot 100.

In a conventional robot cleaner supporting voice recognition, voice recognition is merely a means for control. As a result, the robot cleaner only executes the recognized control commands. Therefore, the analysis of which function is used a lot and which parts are less cleaned has a disadvantage of finding out by means other than voice.

However, in the present invention, the mobile robot 100 may ignite to provide a simple interaction function than simple terminal control.

In addition, according to the present invention, the mobile robot 100 may provide a variety of active control functions to the user by actively providing information or outputting a voice recommending a function or service.

In addition, according to the present invention, the mobile robot 100 can learn and understand the usage pattern of the user. Accordingly, the mobile robot 100 may interact while first suggesting a predetermined function to the user, thereby performing a more efficient and user friendly operation.

8 is a flowchart illustrating a method for controlling a mobile robot according to an embodiment of the present invention, and FIG. 9 is a view referred to for describing a method for controlling a mobile robot according to an embodiment of the present invention.

Referring to FIG. 8, the mobile robot 100 may move according to a command or a setting and start cleaning (S810). For example, the mobile robot 100 may move based on a navigation map or a simulaneous localization and mapping (SLAM) map stored in the storage unit 105 according to a cleaning start command or cleaning reservation setting.

The mobile robot 100 may move and perform cleaning, and may store sensing data acquired by the sensor unit 170, image data obtained by the image acquisition unit 120, and the like in the storage unit 105.

In addition, the mobile robot 100 may store data obtained by performing cleaning, such as driving history data and cleaning history data, in the storage unit 105.

The data obtained by performing the cleaning may be a record of use of the mobile robot 100. For example, the number of normal cleaning completions, charging station return count, number of cleaning functions per area, operating time, average cleaning intensity per area, amount of dust for each area, cleaning mode for each area (cleaning, quick cleaning, general cleaning, etc.) ) Frequency and ratio, the number of times the function is used, such as monitoring, designated cleaning, intensive cleaning, cleaning with virtual wall (virtual wall), and at least one of the time-phase concentration of the above-mentioned record.

In addition, not only a cleaning function performed by the mobile robot 100 but also a usage record regarding other functions may be stored.

For example, cleaning usage history such as designated cleaning, intensive cleaning, and cleaning with virtual walls, monitoring of a specified area, recognizing moving objects, and photographing and transmitting the recognized objects to a specified device. Usage records can be used for environmental sensing and driving strategies such as home guard functions, mapping functions that mobile robots can provide, object search, temperature, humidity, and air quality dust measurement.

By using the data obtained by such driving, the mobile robot 100 and / or the server 70 may use the user's mobile robot 100 usage pattern, spatial information for each area in the usage environment, such as a house, or an object existing in the usage environment. Analyze and learn information.

In addition, the mobile robot 100 may determine the special area based on the existing data obtained by the previous cleaning operation (S820). The mobile robot 100 may identify the current location and space on the basis of the image obtained from the image acquisition unit 120 and may determine whether a special area exists. In addition, the mobile robot 100 may additionally check region-specific information of the corresponding space on the basis of the currently identified position and space information to determine whether the special area is present.

In this case, the special area may be a dangerous area in which an inoperable state has occurred. For example, the dangerous area may be an area in which an inoperable situation occurs such as a stuck, a trap, a fall, a power off.

Alternatively, the special area may be a low efficiency area in which the running speed or the cleaning rate is lower than the average value or the number of rotations is greater than the reference number. For example, the low-efficiency area may correspond to a driving area below a predetermined average speed in a certain area, an area frequently turning, and an area where a predetermined number of times corresponding to a corresponding motion occurs.

When the mobile robot 100 arrives at the determined special region while moving (S830), the mobile robot 100 may output notification information about the special region (S840).

Preferably, the sound output unit 181 of the mobile robot 100 may output a voice guidance message for the special area (S840). Accordingly, the user's attention can be attracted, and the user can naturally check the situation of the special area by turning or moving his / her head in the direction of sound.

Meanwhile, the existing data obtained by performing the previous cleaning may further include obstacle information registered in the special area. In this case, the voice guide message spoken by the mobile robot 100 may include a guide message for an obstacle existing in the special area. Accordingly, the user may immediately determine the type of the obstacle and the degree of danger to instruct the next operation of the mobile robot 100.

The voice guide message may include a message for requesting confirmation of cleaning of the special area.

The voice guidance message may further include at least one of a description of a reason for the determination as the special region or a guide for an example of a command that a user can input voice.

By providing a description of the reason for the determination as the special area, it can help the user to understand the special area more quickly.

Providing guidance on an example of a command that a user can input voice can help the user input the feedback voice accurately. In addition, when the user speaks the illustrated command, the mobile robot 100 and / or the server 70 may recognize the user's voice command more quickly and accurately.

In addition, the voice can be informed simultaneously such as why the danger zone, what commands can be used. For example, the mobile robot 100 may say, "It is an area where past operation was stopped. If you want to clean, please answer" Cleaning. ""

The mobile robot 100 may determine a special area such as a dangerous area and a low efficiency area (S820). When arriving in a special area (S830), it may be inquired first whether or not to perform cleaning for the special area (S840).

Referring to FIG. 9, the mobile robot 100 arriving at a danger zone 910 having many wires may utter a voice guide message 920 for guiding the danger zone.

The voice guide message 920 may include a content 921 that the wire is complicated, a content 922 that is difficult to clean, and the like.

The mobile robot 100 according to an embodiment of the present invention, when entering a special area, such as a cleaning danger zone in which a dangerous obstacle exists, a cleaning failure zone having a cleaning failure history, and is recognized through a plurality of cleaning, Can inform you of areas for cleaning.

For example, there is a possibility that a situation in which movement of the mobile robot 100 is stuck in a driving environment of a house where a dangerous obstacle such as an electric wire exists is present, and there is a high probability that a continuous restraint situation occurs at a place where a restraint situation occurs once.

Ongoing arrests can lead to increased user complaints and lower product reliability.

Therefore, the mobile robot 100 according to an embodiment of the present invention may notify the user that the dangerous situation is recognized through ignition when entering the dangerous section. Accordingly, the reliability and preference of the user can be increased.

In addition, for an area that is too complicated to drive, it is possible to induce the user to clean up and provide additional help to complete the cleaning.

For example, when entering a special area, the mobile robot 100 utters a voice guidance message such as “here I am difficult to clean,” and the user can check and remove the complicated wires or toys in the area. Thereafter, the mobile robot 100 may safely perform the cleaning of the corresponding area.

On the other hand, according to the feedback response of the user, it is possible to skip the cleaning of the special area, so that the cleaning of other areas can be prevented or the overall cleaning time can be reduced. For example, if the user says "don't clean it", the mobile robot 100 does not clean the area.

On the other hand, when the user does not know the voice guidance message due to the cleaning noise during ignition and say "What?", The mobile robot 100 may repeat the voice guidance message just output.

On the other hand, the mobile robot 100 may record the place where the inability to run situation such as restraint, power off when cleaning is performed, and may classify the area as a dangerous area when the inability to travel for a certain number of times occurs.

In this case, the guidance intensity of the voice guidance message and the like may be increased according to the frequency of occurrence of the incapability of driving.

For example, at the start of the cleaning, the mobile robot 100 is in danger of encountering a habitually incapacitated situation, “Master, come here. It's really hard for me to clean here. If you put away a little, I'll try harder. "

In addition, the mobile robot 100 may interact with the user by first asking whether the mobile robot 100 does not need to perform an inefficient operation in a section in which an inefficient operation (running speed or cleaning rate is lower than the average value or the number of rotations is greater than the reference number). Can be.

The special area is a dangerous area that may adversely affect the risk of safety and the cleaning of other areas such as the movement of the mobile robot 100 is restricted, or a low efficiency area that is difficult to perform cleaning.

Therefore, the user is asked whether or not to clean the special area (S840), if the user feedback is received (S850), it may operate according to the received feedback (S860).

The mobile robot 100 may receive a user's voice feedback on the voice guidance message during a predetermined voice waiting time (S850).

When the voice feedback of the user is received, the voice command included in the received voice feedback may be identified and an operation corresponding to the identified voice command may be performed (S860).

The user's feedback response may be determined to be clean, if a positive vocabulary such as clean, proceed, continue, okay, uh, yes, is used.

If the user's feedback response includes a negative vocabulary such as do not clean or do not clean, it may be determined that the user wants to clean.

According to an embodiment, the mobile robot 100 may identify the voice command with respect to the voice feedback of the user.

For example, the mobile robot 100 may include an artificial neural network trained to recognize a voice included in the input data, and the artificial neural network may recognize a voice command included in the received voice feedback.

Alternatively, the voice command regarding the voice feedback of the user may be performed by the server 70.

In this case, identifying the voice command included in the received voice feedback may include transmitting the received voice feedback data to a voice recognition server including an artificial neural network trained to recognize the voice included in the input data. The method may include receiving a voice recognition result of the received voice feedback data from the voice recognition server.

Here, the speech recognition server may be configured as part of the server 70, but may be configured as a dedicated server for speech recognition separately from the server 70.

Alternatively, the operation may be performed step by step in the mobile robot 100 and the server 70. In the case of a simple voice command, the mobile robot 100 may primarily recognize the server, and if the mobile robot 100 does not recognize the voice command, the server 70 may be used.

In this case, the step of identifying the voice command, if the received voice feedback includes a predetermined keyword, identifying the voice command corresponding to the keyword, the received voice feedback is Transmitting the received voice feedback data to a voice recognition server including an artificial neural network trained to recognize a voice included in the input data when the set keyword is not included, and the voice recognition server Receiving a voice recognition result of the received voice feedback data from the.

That is, the mobile robot 100 may recognize a simple keyword such as a positive vocabulary, a negative vocabulary, or a specific vocabulary guided by an example of a command, and operate according to the recognized keyword.

In addition, if the keyword is not recognized, the mobile robot 100 may transmit voice feedback data of the user to the server 70 to receive a voice recognition result from the server 70.

On the other hand, if the user's feedback is not received for a predetermined time (S850), the mobile robot 100 may perform a predetermined avoidance operation (S870).

According to an embodiment, if the user's feedback is not received for a predetermined time (S850), the mobile robot 100 may be set to force the cleaning.

According to an embodiment of the present invention, the mobile robot 100 may start to speak to the user after a predetermined time has elapsed while cleaning only initially.

That is, the mobile robot 100 may perform deep learning with data obtained by performing a plurality of cleaning operations, and recognize the situation of a dangerous obstacle or its location through deep learning, and may explain the corresponding situation in a complex manner.

In addition, it is possible to communicate with users by continuously updating hazardous areas and low-efficiency areas in the changing home environment.

According to the present invention, not only the simple control of the mobile robot 100 through voice but also interaction with the user can improve the voice recognition function and satisfaction of the entire product.

It also controls areas that are difficult to clean and receive user input by voice. The cleaning efficiency can also be improved by cleaning or not cleaning according to the user's voice input.

Referring to FIG. 10, the mobile robot 100 according to an embodiment of the present invention may receive a voice command of a user (S1010).

The conventional voice recognition mobile robot merely recognizes a received voice command and immediately performs a corresponding operation.

However, the mobile robot 100 according to an embodiment of the present invention may check and confirm existing data stored such as a usage record according to the user's voice command reception (S1010) (S1020).

Here, the usage record may include the number of general cleaning completions, charging station return counts, area cleaning function completions, operating time, area cleaning intensity average, area dust amount, area cleaning mode use frequency, area cleaning mode use ratio, It may include at least one of the cleaning function and the number of times of use of each additional function. In addition, the concentration of the items included in the above-described usage record may be further included.

In addition, a usage record regarding other functions as well as a cleaning function performed by the mobile robot 100 may be stored and later checked.

The mobile robot 100 may output a voice guidance message recommending at least one of the functions used less than the reference number of times based on the usage record (S1030).

According to the exemplary embodiment of the present invention, the mobile robot 100 may induce a user to use other functions by proposing a voice to the user for functions that were not normally used.

In addition, a user who inputs a predetermined voice is located at a short distance suitable for communicating with the mobile robot 100. Therefore, the mobile robot 100 may recommend a predetermined function or provide information to a user located at a short distance before performing a command corresponding to the user's voice, and may allow natural interaction.

In addition, the mobile robot 100 may provide an improved cleaning service by first asking about a function and a case considered to be necessary by the user.

The controller 150 may control the sound output unit 181 to output at least one recommended voice guidance message by selecting at least one of functions used less than a reference number according to a preset priority.

Here, the predetermined priority may be in the order of high importance for each predetermined function. For example, when the SLAM success rate decreases, the mapping function is ranked first, the uncleaned area cleaned second, the missed cleaning, home guard, etc. according to schedule, and the third uncleaned area virtual wall. In addition, the danger zone setting function can be set to four priority levels. In this case, the user may recommend a function having the highest priority among functions that the user does not frequently use.

Alternatively, the preset priority may be an order of decreasing number of times of use. In other words, you can recommend a feature that is not used or has a low frequency. In this case, in the case of a function having the same number of times of use, it may be randomly recommended, or a predetermined function pool may be set and the functions included in the pool may be preferentially recommended.

In addition, the controller 150 may determine the recommended function by checking and confirming necessary data according to the preset priority.

Meanwhile, according to the reception of the user's voice command (S1010), the mobile robot 100 and / or the server 70 may identify the user's voice command.

In this case, among functions used less than the reference number of times, a function associated with the voice command of the identified user may be selected and recommended.

To this end, the mobile robot 100 and / or the server 70 may recognize the received voice command before or after checking the usage record (S1020).

According to an embodiment of the present invention, a similar cleaning function or a helpful function may be recommended based on a control command determined by voice recognition. For example, if the user's voice command is determined as a meticulous cleaning command, the local concentrated cleaning performance function with a large amount of dust may be recommended, and if the user's voice command is determined as a quick cleaning command, a local skip may be recommended.

On the other hand, when the voice feedback of the user is received for the voice guidance message within a predetermined time (S1040), the feedback voice command included in the received voice feedback is identified, and a predetermined function is based on the identified feedback voice command. It may be set (S1050).

Thereafter, the mobile robot 100 may perform an operation corresponding to the voice command of the user identified in response to the reception of the user voice command (S1010) (S0160).

Meanwhile, as described with reference to FIG. 8, the mobile robot 100 may identify the voice command with respect to the voice feedback of the user.

According to the present invention, the mobile robot 100 can recognize and analyze a user's usage pattern and talk with the user while first suggesting functions that are not normally used.

According to the present invention, not only the control of the mobile robot 100 through voice but also interaction with the user can improve the voice recognition function and product satisfaction.

In addition, when the mobile robot 100 determines that a better cleaning strategy (cleaning mode, schedule) is determined, the determined cleaning strategy or the like may be recommended by voice.

The function can be set and used according to the user's feedback voice input.

Hereinafter, various embodiments in which the mobile robot 100 actively interacts with a user will be described with reference to the drawings.

11 is a flowchart illustrating a control method of a mobile robot according to an embodiment of the present invention, and FIG. 12 is a view referred to for describing a control method of a mobile robot according to an embodiment of the present invention.

Referring to FIG. 11, the mobile robot 100 may recognize that a front door is opened (S1110). The recognition of the front door opening may be recognized based on a signal received from another device such as a sensor or a server attached to the front door.

When the front door opening is recognized, the mobile robot 100 may move to the front door (S1120) and recognize the user (S1130).

For example, user recognition may be performed by recognizing an image acquired through the image acquisition unit 120. Or other related home appliance or server.

When the user is recognized (S1130), the mobile robot 100 may briefly perform an operation brief operation of the mobile robot 100 by voice (S1140).

The mobile robot 100 may briefly perform an operation brief operation of the mobile robot 100 by voice in operation S1140.

Referring to FIG. 12, a voice guidance message 1210 for approaching a user entering from the front door of the mobile robot 100 and briefing an operation of the mobile robot 100 in the absence of the user is provided through the sound output unit 181. You can print

When not in the house, when a person comes in, the mobile robot 100 may pick up and provide voice feedback at the front door. After the briefing, when there is a lot of fine dust or a user's request, the mobile robot 100 may proceed to clean the surroundings.

In addition, the mobile robot 100 may brief the operation status and results in the absence of the user of the home appliance included in the home appliance network system connected by Wi-Fi communication.

By detecting the return of the user, the mobile robot 100 may first guide the operation record of the home appliances on the day.

The mobile robot 100 may follow a user's movement and provide a voice briefing.

For example, as a pet greets a person who has returned home, the mobile robot 100 may recognize a user who has entered the home and then follow the user and talk about the cleaning record of the day.

On the other hand, the mobile robot 100 may operate in the user following driving mode in response to the user's speech such as "Roboking, Come!".

In addition, the mobile robot 100 may provide the user with predetermined data such as a picture of an uncleaned area, which is not cleaned when cleaning the house while the user is absent.

According to an embodiment of the present invention, the mobile robot 100 may behave similarly to the 'pet animal' to make the user feel good and fun.

In addition, without the user having to check the status of the home appliances in the home, it is possible to know the operation status of the home appliances in the absence by the voice of the mobile robot 100 approaching the user after returning home.

In addition, it is possible to increase the satisfaction of the product by giving feedback that the user is actively recognized.

If the user is not recognized (S1130), the user recognition may be performed until the failure criterion set by the time or the number of times is satisfied (S1150).

If the user is not recognized until the failure criterion is satisfied (S1150), a predetermined operation such as waiting after returning to the charging stand and a home guard may be performed (S1160).

13 is a flowchart illustrating a control method of a mobile robot according to an embodiment of the present invention, and FIG. 14 is a view referred to for describing a control method of a mobile robot according to an embodiment of the present invention.

13 and 14, the mobile robot 100 may receive predetermined data from another device included in the home appliance network system in operation S1810.

For example, the air cleaner 200 may convert dust or air quality index data for each space / area through an indoor air sensor. In addition, the air cleaner 200 may transfer dust or air quality index data to the mobile robot 100. ) Can be shared with.

The air cleaner 200 may detect the presence of dust or detect a space in which a lot of people gather.

When the air cleaner 200 detects dust generation, the air cleaner 200 may start driving and transmit dust generation position data to the mobile robot 100.

On the other hand, the mobile robot 100 may establish and recommend a cleaning plan based on the data received from the air cleaner 200 and previously learned data (S1320).

For example, the mobile robot 100 may induce by recommending intensive cleaning of a dusty space and recommending a direction change of the air cleaner 200.

In addition, the mobile robot 100 may propose a cleaning plan after a predetermined time to the user in consideration of the degree of dust generation, the number of people detected, and the like. At this time, the mobile robot 100 may utter a voice guidance message 1410 including a current plan, a cleaning plan after one hour, and the like.

The mobile robot 100 may proceed with cleaning depending on whether the user responds.

If there is a voice response 1420 of the user including cleaning start two hours later, the mobile robot 100 may start cleaning two hours later.

According to an embodiment of the present invention, it is possible to provide a positive synergy effect through the interworking use scenario between home appliances.

In addition, the home appliance first detects a situation that needs to be cleaned and suggests it to the user, thereby increasing product reliability.

The mobile robot according to the present invention is not limited to the configuration and method of the embodiments described as described above, but the embodiments may be selectively combined with all or part of the embodiments so that various modifications can be made. It may be configured.

On the other hand, the control method of the mobile robot according to an embodiment of the present invention, it is possible to implement as a processor readable code on a processor-readable recording medium. The processor-readable recording medium includes all kinds of recording devices that store data that can be read by the processor. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet. . The processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

Claims

Moving and performing cleaning;

Determining a special area based on existing data obtained by a previous cleaning operation; And,

Outputting a voice guidance message for the special area when the determined special area arrives.
The method of claim 1,

The special area,

Hazardous areas where incapacitation has occurred, or

A control method for a mobile robot, characterized in that the running speed or cleaning rate is lower than the average value, or the low efficiency area where the number of rotations is greater than the reference number.
The method of claim 1,

The existing data obtained by the previous cleaning operation includes driving history data and cleaning history data.
The method of claim 3,

Existing data obtained by the previous cleaning operation,

Further includes obstacle information registered in the special area,

The voice guidance message,

And a guide message for obstacles existing in the special area.
The method of claim 1,

The voice guidance message,

And a message requesting confirmation of cleaning of the special area.
The method of claim 5,

The voice guidance message,

The control method of the mobile robot, characterized in that it further comprises at least one of a description of the reason for determining as the special area, or a guide for an example of a command that the user can voice input.
The method of claim 1,

Receiving a user's voice feedback for the voice announcement message;

Identifying a voice command included in the received voice feedback; And,

And performing an operation corresponding to the identified voice command.
The method of claim 7, wherein

Identifying the voice command,

Transmitting the received speech feedback data to a speech recognition server including an artificial neural network trained to recognize speech included in the input data;

Receiving a voice recognition result of the received voice feedback data from the voice recognition server.
The method of claim 7, wherein

The mobile robot includes an artificial neural network learned to recognize the voice included in the input data,

Identifying the voice command,

And the artificial neural network recognizes a voice command included in the received voice feedback.
The method of claim 7, wherein

Identifying the voice command,

When the received voice feedback includes a preset keyword, identifying a voice command corresponding to the keyword,

If the received voice feedback does not include a predetermined keyword, transmitting the received voice feedback data to a voice recognition server including an artificial neural network trained to recognize a voice included in the input data. , And,

Receiving a voice recognition result of the received voice feedback data from the voice recognition server.
Receiving a voice command of a user;

Outputting a voice guidance message recommending at least one of the functions used less than a reference number based on the usage record;

Identifying a feedback voice command included in the received voice feedback when the voice feedback of the user is received for the voice guide message; And,

Setting a predetermined function based on the identified feedback voice command.
The method of claim 11,

The usage record,

Normal cleaning completion count, charging station return count, cleaning function completion count by area, operating time, cleaning intensity average by region, dust level by area, cleaning mode usage frequency by region, cleaning mode usage ratio by region, cleaning function and usage function by additional function Control method of a mobile robot, characterized in that it comprises at least one of.
The method of claim 11,

The voice guide message output step,

The control method of the mobile robot, characterized in that for selecting at least one of the functions used less than the reference number in accordance with a predetermined priority.
The method of claim 13,

The predetermined priority is a control method of a mobile robot, characterized in that the predetermined order of importance of each function or the order of use is small.
The method of claim 11,

Identifying a voice command of the user.
The method of claim 15,

And performing an operation corresponding to the identified voice command of the user.
The method of claim 15,

The voice guide message output step,

And selecting and recommending a function associated with the identified voice command from among the functions used less than the reference number of times.
The method of claim 11,

Identifying the feedback voice command,

Transmitting the received speech feedback data to a speech recognition server including an artificial neural network trained to recognize speech included in the input data;

Receiving a voice recognition result of the received voice feedback data from the voice recognition server.
The method of claim 11,

The mobile robot includes an artificial neural network learned to recognize the voice included in the input data,

Identifying the voice command,

And the artificial neural network recognizes the feedback voice command included in the received voice feedback.