KR20190003118A - Robot system including Mobile robot and Moving terminal - Google Patents

Abstract

According to an aspect of the present invention, a mobile terminal comprises: a wireless communication part for receiving information on a plurality of mobile robots included in a robot system; a display part for displaying a user interface screen for a first mobile robot among the plurality of mobile robots; and a control part for controlling the display part to display a designated area cleaning setting screen capable of designating one or more areas in an area allocated to the first mobile robot when an input to the designated area cleaning setting item included in the user interface screen is received. The user can conveniently designate the cleaning area of the robot included in the robot system.

Description

[0001] The present invention relates to a robot system including a mobile robot and a mobile terminal,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot system including at least one mobile robot and a mobile terminal, and more particularly to a mobile robot for cleaning a cleaning area allocated in a public place and a mobile terminal will be.

Robots have been developed for industrial use and have been part of factory automation. In recent years, medical robots, aerospace robots, and the like have been developed, and household robots that can be used in ordinary homes are being developed. Among these robots, mobile robots capable of traveling by magnetic force are called mobile robots.

A representative example of the mobile robot is a cleaning robot (a robot cleaner), which is a device for cleaning a corresponding area by suctioning dust or foreign matter around the robot while running a certain area by itself.

Using a cleaning robot has the advantage that the working ability is higher than when it is operated by personnel, and it is possible to work without rest, and there is no restriction on the cleaning work time.

On the other hand, cleaning robots are also effective in keeping clean and swiftly in public areas throughout the year, including vulnerable areas that need frequent cleaning, even in public places with large spaces such as airports, train stations, department stores and harbors.

A plurality of cleaning robots need to be disposed and operated for cleaning and managing such a large space. In this case, a plurality of cleaning robots perform a cleaning operation for the assigned cleaning area.

Therefore, when a plurality of cleaning robots are operated, it is required to effectively control a plurality of cleaning robots in order to improve cleaning efficiency.

Also, when a cleaning robot is operated in a public place, prevention of safety accidents caused by a cleaning robot is a top priority.

In particular, multiple safety devices are needed to prevent safety accidents that may occur when a cleaning robot is controlled by a mobile terminal.

An object of the present invention is to provide a robot system and a control method thereof that can be safely operated in a public place.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a robot system and its control method capable of effectively cleaning a wide space.

SUMMARY OF THE INVENTION An object of the present invention is to provide a robot system and a control method thereof that enable a user to conveniently check and control information.

According to one aspect of the present invention, there is provided a robot system including a display unit displaying at least one mobile robot and a user interface screen including a manual control item for manually manipulating the mobile robot, A control unit for setting a manual control item to a locked state and controlling the display unit to display a guidance message indicating that the locked state can be released by a display operation of the mobile robot when an input to the manual control item is received By including the terminal, it is possible to prevent the mobile robot from manually operating the mobile terminal in a dangerous situation.

According to at least one of the embodiments of the present invention, it is possible to provide a robot system and a control method thereof that can be safely operated in a public place.

Further, according to at least one of the embodiments of the present invention, a large space can be effectively cleaned.

In addition, according to at least one embodiment of the present invention, it is possible to improve the usability of the user by providing a robot system and a control method thereof that enable a user to easily check and control information.

Meanwhile, various other effects will be directly or implicitly disclosed in the detailed description according to the embodiment of the present invention to be described later.

1 is a front view of a mobile robot included in a robot system according to an embodiment of the present invention.
2 is a rear view of a mobile robot included in a robot system according to an embodiment of the present invention.
3 is a block diagram showing a control relationship between main components of a mobile robot included in the robot system according to an embodiment of the present invention.
4 is a conceptual diagram of a robot system including a plurality of mobile robots and a mobile terminal according to an embodiment of the present invention.
5 is a simplified block diagram of a mobile terminal according to an embodiment of the present invention.
6 and 7 are views showing a state in which a head of a mobile robot is opened according to an embodiment of the present invention.
8 is a view illustrating a user interface screen provided through a display of a mobile robot according to an embodiment of the present invention.
FIGS. 9 to 17 are diagrams illustrating a user interface screen provided through a display unit of a mobile terminal according to an exemplary embodiment of the present invention, and a method of controlling a robot system using the same.
18 is a conceptual diagram of a robot system including a plurality of mobile robots and a mobile terminal according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it is needless to say that the present invention is not limited to these embodiments and can be modified into various forms.

In the drawings, the same reference numerals are used for the same or similar parts throughout the specification.

The suffix " module " and " part " for components used in the following description are given merely for convenience of description and do not give special significance or role in themselves. Accordingly, the terms " module " and " part " may be used interchangeably.

The mobile robot 100 according to an embodiment of the present invention refers to a robot that can move by itself using wheels or the like, and may be a robot cleaner or the like. Hereinafter, a cleaning robot (robot cleaner) having a cleaning function will be described as an example of a mobile robot with reference to the drawings, but the present invention is not limited thereto.

FIG. 1 is a front view of a mobile robot included in a robot system according to an embodiment of the present invention. FIG. 2 is a view illustrating a rear portion of a mobile robot included in the robot system according to an embodiment of the present invention. to be.

3 is a block diagram showing a control relationship between main components of a mobile robot included in the robot system according to an embodiment of the present invention.

4 is a conceptual diagram of a robot system including a plurality of mobile robots and a mobile terminal according to an embodiment of the present invention.

The robot system according to an embodiment of the present invention may include a plurality of mobile robots exemplified with reference to Figs. In addition, the robot system according to an embodiment of the present invention may further include a server, a mobile terminal, and the like.

4, a robot system according to an embodiment of the present invention includes a plurality of mobile robots 100a, 100b, 100c, 100d, 100e and a plurality of mobile robots 100a, 100b, 100c, 100d, And a mobile terminal 200 that can control the mobile terminal 200.

The robot system according to an embodiment of the present invention includes a plurality of mobile robots 100a, 100b, 100c, 100d and 100e, a mobile terminal 200 and a server (not shown) of a robot manufacturer or a service provider .

In addition, the robot system according to an embodiment of the present invention may further include a robot other than a mobile robot having a cleaning function.

1 to 3, the mobile robot 100 may include a main body 101 and a traveling unit 160 for moving the main body 101.

The portion facing the ceiling in the running zone is defined as the upper surface portion, the portion facing the floor in the running zone is defined as the bottom portion, and the main body 101 is defined between the upper surface portion and the bottom surface portion. , A portion facing the running direction is defined as a front portion, and a portion facing the opposite direction of the front portion is defined as a rear portion.

The mobile robot (100) includes a traveling unit (160) for moving the main body (101). The driving unit 160 includes at least one driving wheel 136 for moving the main body 101. The driving unit 160 includes a driving motor (not shown) connected to the driving wheels 136 to rotate the driving wheels. The driving wheels 136 may be provided on the left and right sides of the main body 101, respectively, and are hereinafter referred to as left and right wheels, respectively.

The left wheel and the right wheel may be driven by a single drive motor, but may be provided with a left wheel drive motor for driving the left wheel and a right wheel drive motor for driving the right wheel, respectively, if necessary. The running direction of the main body 101 can be switched to the left or right side by making a difference in the rotational speeds of the left and right wheels.

A suction port (not shown) for sucking air can be formed on the bottom surface of the main body 101. A suction device (not shown) for providing a suction force to suck air through the suction port is provided in the main body 101 , And a dust container (not shown) for collecting the dust sucked together with the air through the suction port. In addition, a dust container cover (not shown) may be provided in the dust container to discard the dust therein.

The main body 101 includes a body part 102 forming a space for accommodating various components constituting the mobile robot 100 and a head part 110 disposed on the upper side of the body part 102 so as to be openable and closable .

The head unit 110 may include a head 111 that can be opened and closed and a coupling unit 112 coupled to the head 111 so as to be openable and closable.

In addition, according to the embodiment, a switch or a sensor for detecting whether the head 111 is open or closed may be disposed on the head 111 and / or the coupling part 112.

In addition, the head 111 may further include an emergency operation button 114. The emergency operation button 114 may perform a function of immediately stopping the operation of the mobile robot 100 while the mobile robot 100 is stopped or running. For example, the emergency operation button 114 may be located behind the mobile robot 100 so that the emergency operation button 114 can be easily operated even if the mobile robot 100 is traveling forward have.

In addition, the mobile robot 100 may include a display 182 housed in the main body 101.

The user can open the head 111 and insert and remove the dust container in the main body 101. [

In some embodiments, the head portion 110 may be openable and closable in a plurality of ways or may include one or more opening and closing mechanism structures.

For example, when the head 111 is opened by lifting the head 111, a button is provided on the head 110, and a predetermined cover is opened toward the user when the user presses a button, Can be implemented.

In this case, the display 182 may be arranged in a predetermined area partitioned in the inner storage space, and the opening / closing mechanism structure opened by the user's button operation may be configured with a dedicated opening / closing structure of the area where the display 182 is disposed . Accordingly, when the user presses a button, the display 182 can be configured to protrude toward the user.

The mobile robot 100 includes a brush-type brush (not shown) having a brush which is exposed through a suction port, a brush (not shown) located on the front side of the bottom surface of the main body 101, The auxiliary brush 135 may be provided. By the rotation of the brushes 135, the dusts are separated from the floor in the traveling zone, and the dusts separated from the floor are sucked through the suction port and collected in the dustbin.

Meanwhile, the mobile robot 100 may include a power supply unit (not shown) provided with a rechargeable battery (not shown) to supply power to the mobile robot 100.

The power supply unit supplies driving power and operating power to the respective components of the mobile robot 100 and can be charged by receiving a charging current from a charging stand (not shown) when the remaining power is insufficient.

The mobile robot 100 may further include a battery sensing unit (not shown) that senses a charged state of a battery (not shown) and transmits the sensed result to the control unit 140. The battery is connected to the battery sensing unit, and the battery remaining amount and the charging state are transmitted to the control unit 140. The battery remaining amount may be displayed on the display 182 of the output unit 180. [

The battery (not shown) supplies not only the drive motor but also the power required for the overall operation of the mobile robot 100. The body 102 may include an openable cover 103 for battery checking and / or replacement. The user can open the cover 103 to check or replace the battery condition.

When the battery is discharged, the mobile robot 100 can return to the charging stand (not shown) for charging. During the return travel, the mobile robot 100 can detect the position of the charging stand by itself .

The charging stand may include a signal transmitting unit (not shown) for transmitting a predetermined return signal. The return signal may be an ultrasonic signal or an infrared signal, but is not limited thereto.

The mobile robot 100 may include a signal sensing unit (not shown) for receiving a return signal. The charging unit may transmit an infrared signal through a signal transmission unit, and the signal sensing unit may include an infrared sensor that senses an infrared signal. The mobile robot 100 moves to the position of the charging base according to the infrared signal transmitted from the charging base and docks with the charging base. (Not shown) of the mobile robot 100 and the charging terminal (not shown) of the charging base by such docking.

The image acquiring unit 120 photographs the periphery of the main body 101, a traveling zone, an external environment, and the like, and may include a camera module. The camera module may include a digital camera. A digital camera includes an image sensor (e.g., a CMOS image sensor) configured with at least one optical lens, and a plurality of photodiodes (e.g., pixels) that are formed by light passing through the optical lens, And a digital signal processor (DSP) that forms an image based on signals output from the photodiodes. The digital signal processor is capable of generating moving images composed of still frames as well as still images.

The image captured by the camera can be used to identify the kind of material such as dust, hair, floor, etc. existing in the space, whether it is cleaned, or to confirm the cleaning time.

The image acquiring unit 120 may acquire an image by photographing the periphery of the main body 101, and the acquired image may be stored in the storage unit 130. [

These cameras can be installed in several parts for shooting efficiency. For example, the image acquiring unit 120 may include an upper camera 120a for capturing an image of the upper side of the front of the main body 101 to acquire an image of the ceiling within the traveling region, A front camera 120b, and a depth camera 120c. However, the number, arrangement, type, and photographing range of the cameras provided in the image obtaining unit 120 are not necessarily limited to these.

The upper camera 120a can acquire an image of the ceiling in the travel zone and the mobile robot 100 can use the image acquired by the upper camera 120a for SLAM (Simultaneous Localization and Mapping).

The front camera 120b can photograph an image of a user, a situation of an obstacle, a cleaning area, or the like existing in the front of the mobile robot 100 in the traveling direction.

In this case, the status notification light 151 may be embodied as a circular ring around the upper camera 120a.

In addition, the mobile robot 100 may include a sensor unit 170 including sensors for sensing various data related to the operation and state of the mobile robot.

For example, it may include a sensor (not shown) for detecting whether the head 111 is open or closed. The sensors for detecting whether the head 111 is opened or closed can be various known sensors.

Meanwhile, the sensor unit 170 may include an obstacle detection sensor 131 for sensing an obstacle ahead. In addition, the sensor unit 170 may further include a cliff detection sensor (not shown) for detecting the presence or absence of a cliff on the floor in the driving area, and a lower camera sensor (not shown) for acquiring a bottom image.

According to an embodiment, the obstacle detection sensor 131 may include a plurality of sensors installed on the outer circumferential surface of the mobile robot 100 at regular intervals.

For example, the sensor unit 170 may include a first sensor and a second sensor disposed on the front surface of the main body 101 so as to be spaced left and right.

The obstacle detection sensor 131 may include an infrared sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, a position sensitive device (PSD) sensor, and the like.

Meanwhile, the position and type of the sensor included in the obstacle detection sensor 131 may vary depending on the type of the mobile robot, and the obstacle detection sensor 131 may include more various sensors.

Also, the sensor unit 170 may include light detection and ranging (Lidar) 132a and 132b.

The lidarities 132a and 132b transmit the object such as an obstacle based on the TOF (Time of Flight) of the transmission signal and the reception signal or the phase difference between the transmission signal and the reception signal through the laser light Can be detected.

The plurality of Ladas 132a and 132b may be provided. For example, the lidarities 132a and 132b are a first ray 132a for detecting an object located in front of the mobile robot 100 and a first ray 132a for detecting an object located behind the mobile robot 100 And a second RL 132b.

In addition, the RLAs 132a and 132b can detect the distance to the object, the relative speed with the object, and the position of the object.

The lines 132a and 132b may be provided as part of the configuration of the obstacle detection sensor 131. [

Also, the RLAs 132a and 132b may be provided as a sensor for creating a map.

For example, the map generation module 143 may generate a map of the running area. The map generation module 143 can generate a map by processing the image acquired through the image acquisition unit 120 and supplementarily or independently generate the map on the basis of the sensing data of the lines 132a and 132b Can be created.

The obstacle detection sensor 131 senses an object, particularly an obstacle, existing in a traveling direction (movement direction) of the mobile robot, and transmits the obstacle information to the control unit 140. That is, the obstacle detection sensor 131 can sense the moving path of the mobile robot 100, protrusions existing on the front or side of the mobile robot 100, household appliances, furniture, walls, wall corners, etc. and transmit the information to the control unit have.

At this time, the controller 140 detects the position of the obstacle based on at least two signals received through the ultrasonic sensor, and controls the movement of the mobile robot 100 according to the position of the detected obstacle.

According to an embodiment, the obstacle detection sensor 131 provided on the outer surface of the main body 101 may include a transmitting unit and a receiving unit.

For example, the ultrasonic sensor may be provided such that at least one transmitting portion and at least two receiving portions are staggered from each other. Also, it is possible to radiate signals at various angles and to receive signals reflected from obstacles at various angles.

According to an embodiment, the signal received by the obstacle detection sensor 131 may undergo signal processing such as amplification, filtering, and the like, and then the distance and direction to the obstacle may be calculated.

The sensor unit 170 may further include a motion detection sensor for detecting motion of the mobile robot 100 according to driving of the main body 101 and outputting motion information. For example, a gyro sensor, a wheel sensor, an acceleration sensor, or the like can be used as the motion detection sensor.

The gyro sensor senses the direction of rotation and detects the rotation angle when the mobile robot 100 moves according to the operation mode. The gyro sensor detects the angular velocity of the mobile robot 100 and outputs a voltage value proportional to the angular velocity. The control unit 140 calculates the rotation direction and the rotation angle using the voltage value output from the gyro sensor.

The wheel sensor is connected to the left and right wheels to detect the number of revolutions of the wheel. Here, the wheel sensor may be a rotary encoder. The rotary encoder detects and outputs the number of rotations of the left and right wheels.

The control unit 140 can calculate the rotational speeds of the left and right wheels using the number of rotations. Also, the control unit 140 can calculate the rotation angle using the difference in the number of rotations of the left and right wheels.

The acceleration sensor senses a change in the speed of the mobile robot 100, for example, a change in the mobile robot 100 due to a start, a stop, a change of direction, a collision with an object or the like. The acceleration sensor is attached to the main wheel or the adjoining positions of the auxiliary wheels, so that the slip or idling of the wheel can be detected.

In addition, the acceleration sensor is built in the control unit 140 and can detect a speed change of the mobile robot 100. [ That is, the acceleration sensor detects the amount of impact according to the speed change and outputs a corresponding voltage value. Thus, the acceleration sensor can perform the function of an electronic bumper.

The control unit 140 can calculate the positional change of the mobile robot 100 based on the operation information output from the motion detection sensor. Such a position is a relative position corresponding to the absolute position using the image information. The mobile robot can improve the performance of the position recognition using the image information and the obstacle information through the relative position recognition.

In addition, the mobile robot 100 may display the reservation information, the battery status, the operation mode, the operation status, the error status, and the like, including the output unit 180, or may output it as an audio.

The output unit 180 may include a display 182 for displaying the user interface screen such as the reservation information, the battery status, the operation mode, the operation status, and the error status as an image.

The display 182 may be slid or pushed to the front to allow the user to more easily recognize the user interface screen provided through the display 182 can do.

Also, the display 182 may be configured as a touch screen by forming a mutual layer structure with the touch pad. In this case, the display 182 may be used as an input device capable of inputting information by a user's touch in addition to the output device.

The output unit 180 may further include an audio output unit 181 for outputting an audio signal. The sound output unit 181 can output a sound message such as a warning sound, an operation mode, an operation state, an error state, etc., under the control of the control unit 140. [ The sound output unit 181 can convert an electric signal from the control unit 140 into an audio signal and output it. To this end, the sound output section 181 may include a speaker or the like.

Referring to FIG. 3, the mobile robot 100 includes a controller 140 for processing and determining various information such as recognizing a current position, and a storage unit 130 for storing various data. The mobile robot 100 may further include a communication unit 190 for transmitting and receiving data to and from a mobile terminal, a server, another mobile robot, a guide robot, and the like.

The mobile terminal 200 is provided with an application for controlling the mobile robot 100 and displays a map of the traveling area to be cleaned by the mobile robot 100 through the execution of the application, Can be specified. The mobile terminal 200 may be, for example, a remote controller, a PDA, a laptop, a smart phone, or a tablet on which an application for setting a map is mounted.

The mobile terminal 200 can communicate with the mobile robot 100 to display the current position of the mobile robot together with the map, and information about a plurality of regions can be displayed. In addition, the mobile terminal 200 updates its position and displays it according to the traveling of the mobile robot.

Further, the mobile robot 100 can share data or transmit / receive data with the server or other robots through the communication unit 190.

For example, the mobile robot 100 can transmit current position information, current state information, and the like to the server through the communication unit 190.

The communication unit 190 may include an access point (AP). Accordingly, the mobile robot 100 can establish a connection with the Internet network 500 through a wireless LAN (AP) and be provided with a service.

In the robot system according to the present invention, a plurality of mobile robots can share status information such as cleaning progress status information with each other. Also, the shared state information can be transmitted to the server and managed, and the server can transmit various information to the user's mobile terminal.

The control unit 140 controls the overall operation of the mobile robot 100 by controlling the image acquisition unit 120, the driving unit 160, the display 182, and the like, which constitute the mobile robot 100.

The storage unit 130 records various kinds of information required for controlling the mobile robot 100, and may include a volatile or nonvolatile recording medium. The storage medium stores data that can be read by a microprocessor, and includes a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD- Tape, floppy disk, optical data storage, and the like.

In addition, the storage unit 130 may store a map of a driving area. The map may be input by a mobile terminal, a server, or the like capable of exchanging information with the mobile robot 100 through wired or wireless communication, or may be one generated by the mobile robot 100 itself.

The map may indicate the location of the compartmentalized zones within the driving zone. In addition, the current position of the mobile robot 100 can be displayed on the map, and the current position of the mobile robot 100 on the map can be updated in the course of travel. The mobile terminal may store the same map as the map stored in the storage unit 130. [

The storage unit 130 may store cleaning history information. Such cleaning history information can be generated each time cleaning is performed.

The map of the traveling area stored in the storage unit 130 may include a navigation map used for traveling during cleaning, a simultaneous localization and mapping (SLAM) map used for location recognition, A learning map used for learning cleaning, a global position map used for global position recognition, an obstacle recognition map for recording information on the recognized obstacle, and the like.

Meanwhile, as described above, the maps can be stored and managed in the storage unit 130 separately for each use. However, the maps may not be clearly classified for each use. For example, a plurality of pieces of information may be stored in one map for use in at least two applications.

The control unit 140 may include a travel control module 141, a location recognition module 142, a map generation module 143, and an obstacle recognition module 144.

1 to 3, the travel control module 141 controls the travel of the mobile robot 100, and controls the travel of the travel unit 160 according to the travel setting. In addition, the travel control module 141 can grasp the travel path of the mobile robot 100 based on the operation of the travel unit 160. [ For example, the travel control module 141 can grasp the current or past traveling speed and the traveling distance of the mobile robot 100 based on the rotational speed of the driving wheel 136, The current or past redirection process can also be identified. Based on the travel information of the mobile robot 100, the position of the mobile robot 100 on the map can be updated.

The map generating module 143 may generate a map of the running area. The map generation module 143 can process the image acquired through the image acquisition unit 120 to generate a map. That is, a cleaning map corresponding to the cleaning area can be created.

Also, the map generation module 143 can process the image acquired through the image acquisition unit 120 at each position and recognize the global position in association with the map.

The position recognition module 142 estimates and recognizes the current position. The position recognition module 142 grasps the position in cooperation with the map generation module 143 by using the image information of the image acquisition unit 120 so that even if the position of the mobile robot 100 suddenly changes, .

The mobile robot 100 is capable of recognizing the position during the continuous running through the position recognition module 142 and is capable of recognizing the map through the map generation module 143 and the obstacle recognition module 144 without the position recognition module 142 You can learn and estimate your current location.

During traveling of the mobile robot 100, the image acquisition unit 120 acquires images around the mobile robot 100. Hereinafter, the image acquired by the image acquisition unit 120 is defined as an 'acquired image'.

The acquired image includes various features such as lights, edges, corners, blobs, ridges, etc., located on the ceiling.

The map generation module 143 detects the feature from each of the acquired images. Various methods for detecting features from an image in the field of Computer Vision (Feature Detection) are well known. Several feature detectors suitable for detecting these features are known. For example, Canny, Sobel, Harris & Stephens / Plessey, SUSAN, Shi & Tomasi, Level curve curvature, FAST, Laplacian of Gaussian, Difference of Gaussians, Determinant of Hessian, MSER, PCBR and Gray-level blobs detector.

The map generation module 143 calculates a descriptor based on each minutiae point. The map generation module 143 may convert a feature point into a descriptor using a Scale Invariant Feature Transform (SIFT) technique for feature detection. The descriptor may be denoted by an n-dimensional vector.

The SIFT can detect unchanging features with respect to the scale, rotation, and brightness change of the object to be photographed. Even if the same region is photographed with a different attitude of the mobile robot 100 (i.e., -invariant) can be detected. Of course, various other techniques (e.g., HOG: Histogram of Oriented Gradient, Haar feature, Fems, Local Binary Pattern (LBP), Modified Census Transform (MCT)) may be applied.

The map generation module 143 classifies at least one descriptor for each acquired image into a plurality of groups according to a predetermined lower classification rule on the basis of the descriptor information obtained through the acquired image of each position, Can be converted into lower representative descriptors, respectively.

As another example, it is also possible to classify all descriptors gathered from acquired images in a predetermined area, such as a room, into a plurality of groups according to a predetermined sub-classification rule, and write descriptors included in the same group according to the predetermined lower representative rule, . ≪ / RTI >

The map generation module 143 can obtain the feature distribution of each position through such a process. Each position feature distribution can be represented as a histogram or an n-dimensional vector. As another example, the map generation module 143 can estimate an unknown current position based on descriptors calculated from each feature point, without going through a predetermined lower classification rule and a predetermined lower representative rule.

In addition, when the current position of the mobile robot 100 becomes an unknown state due to a positional jump or the like, the current position can be estimated based on data such as a previously stored descriptor or a lower representative descriptor.

The mobile robot 100 acquires an acquired image through the image acquisition unit 120 at an unknown current position. Through the image, various features such as lights, edges, corners, blobs, ridges, etc., are found on the ceiling.

The position recognition module 142 detects the features from the acquired image. Various methods of detecting features from an image in the field of computer vision technology and descriptions of various feature detectors suitable for detecting these features are as described above.

The position recognition module 142 calculates the recognition descriptor through the recognition descriptor calculation step based on each recognition feature point. Herein, the recognition minutiae and the recognition descriptor are used to describe the process performed by the position recognition module 142, and are intended to distinguish the terms used to describe the process performed by the map generation module 143. [ However, the characteristics of the external world of the mobile robot 100 are merely defined by different terms.

The location recognition module 142 may convert the recognized minutiae point into the recognition descriptor using the Scale Invariant Feature Transform (SIFT) technique for detecting the feature. The recognition descriptor may be denoted by an n-dimensional vector.

As described above, the SIFT selects characteristic points that are easily distinguishable, such as corner points, from the acquired image, and then determines the distribution characteristics of the brightness gradient of pixels belonging to a certain region around each characteristic point ) Is an image recognition technique that obtains an n-dimensional vector (vector) in which the degree of change in each direction is a numerical value for each dimension.

Based on at least one recognition descriptor information obtained through an acquired image of an unknown current position, the position recognition module 142 generates position information (for example, feature distribution of each position) (Lower recognition feature distribution).

According to a predetermined lower comparison rule, each position feature distribution can be compared with each recognition feature distribution to calculate each similarity. The similarity degree (probability) is calculated for each position corresponding to each position, and the position where the greatest probability is calculated can be determined as the current position.

In this way, the control unit 140 can recognize the current location of the main body 101 based on the pre-stored map, or can generate a map composed of a plurality of regions by distinguishing the travel regions.

When the map is generated, the control unit 140 can transmit the generated map to the mobile terminal, the server, and the like through the communication unit 190. Also, as described above, the control unit 140 can store the map in the storage unit when the map is received from the mobile terminal, the server, and the like.

In addition, when the map is updated during travel, the controller 140 transmits the updated information to the mobile terminal 200 so that the map stored in the mobile terminal 200 and the mobile robot 100 are the same. As the map stored in the mobile terminal 200 and the mobile robot 100 are kept the same, the mobile robot 100 can clean the designated area with respect to the cleaning command from the mobile terminal, So that the current position of the mobile robot can be displayed.

At this time, the map is divided into a plurality of areas for the cleaning area, and includes a connection path for connecting the plurality of areas, and may include information about the obstacles in the area.

When the cleaning command is inputted, the control unit 140 determines whether the position on the map matches the current position of the mobile robot 100 or not. The cleaning command may be input from the remote controller, the display, or the mobile terminal 200.

If the current position does not coincide with the current position on the map or the current position can not be confirmed, the controller 140 recognizes the current position and restores the current position of the mobile robot 100, The traveling section 160 can be controlled to move to the designated area.

If the current position does not coincide with the position on the map or the current position can not be confirmed, the position recognition module 142 analyzes the acquired image input from the image obtaining unit 120 and estimates the current position based on the map can do. In addition, the obstacle recognition module 144 or the map generation module 143 may also recognize the current position in the same manner.

After recognizing the position and restoring the current position of the mobile robot 100, the travel control module 141 calculates the travel route from the current position to the designated area and controls the travel unit 160 to move to the designated area.

In the case of receiving the cleaning pattern information from the server, the travel control module 141 can divide the entire traveling area into a plurality of areas and set one or more areas as designated areas according to the received cleaning pattern information.

Also, the travel control module 141 may calculate the travel route in accordance with the received cleaning pattern information, travel along the travel route, and perform cleaning.

The control unit 140 may store the cleaning history in the storage unit 130 when the cleaning of the designated area is completed.

The control unit 140 may transmit the operation state or the cleaning state of the mobile robot 100 to the mobile terminal 200 and the server through the communication unit 190 at predetermined intervals.

Accordingly, the mobile terminal 200 displays the position of the mobile robot together with the map on the screen of the application being executed based on the received data, and also outputs information about the cleaning state.

The mobile robot 100 according to an embodiment of the present invention moves until an obstacle or a wall surface is sensed in one direction and when the obstacle is recognized through the sensor unit 170 and the obstacle recognition module 144, It is possible to determine a running pattern such as a straight run or a rotation.

For example, if the recognized obstacle is an obstacle of a kind that can be passed over, the mobile robot 100 can continue to go straight. Alternatively, if the recognized obstacle is an obstacle of a type that can not be surpassed, the mobile robot 100 rotates and moves a certain distance, moves to a distance in which the obstacle is sensed in the direction opposite to the initial moving direction, and is moved in a zigzag . ≪ / RTI >

The mobile robot 100 according to the embodiment of the present invention can perform obstacle recognition and avoidance based on machine learning.

The control unit 140 includes an obstacle recognition module 144 for recognizing an obstacle learned through machine learning in the input image and a driving unit for driving the driving unit 160 based on the recognized obstacle characteristics And a travel control module 141 for controlling the vehicle.

The mobile robot 100 according to an exemplary embodiment of the present invention may include an obstacle recognition module 144 in which an attribute of an obstacle is learned by machine learning.

Machine learning means that the computer learns from the data through the computer, and the computer takes care of the problem, even though the computer does not instruct the person directly with the logic.

Deep Learning (Deep Learning). It is based on Artificial Neural Networks (ANN) for constructing artificial intelligence. It is an artificial intelligence technology that allows a computer to learn like a human being without learning it as a way of teaching people how to think.

The ANN may be implemented in a software form or a hardware form such as a chip.

The obstacle recognition module 144 may include an artificial neural network (ANN) in the form of software or hardware in which the property of the obstacle is learned.

For example, the obstacle recognition module 144 may include a Deep Neural Network (DNN) such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), DBN (Deep Belief Network) . ≪ / RTI >

The obstacle recognition module 144 may determine an attribute of the obstacle included in the input image data based on the weights among the nodes included in the DNN.

If the sensor unit 170 senses an obstacle while the mobile robot 100 is moving, the control unit 140 determines whether the image acquisition unit 120 acquires the obstacle according to the direction of the obstacle detected by the sensor unit 170 It is possible to control to extract a partial area of the image.

The image obtaining unit 120 may obtain an image within a predetermined angle range in the moving direction of the mobile robot 100. [

The obstacle recognition module 144 can recognize the obstacle based on the data learned by the machine learning in the image acquired by the image acquisition unit 120. [

In addition, the obstacle recognition module 144 may identify the type of object to be cleaned.

1 to 3, a mobile robot 100 according to an embodiment of the present invention includes a lamp unit 150 having various lamps having a signal function for informing people of the traveling state of the mobile robot 100, ).

The lamp unit 150 includes a status notification lamp 151 for outputting light representing the current state of the mobile robot 100 and a status notification lamp 151 disposed on the front surface of the main body 101, And a backward direction indicator 153 disposed on the rear surface of the main body 101 and turned on according to the traveling direction of the mobile robot 100. [

The lamps 151, 152, and 153 of the lamp unit 150 may include one or more light sources. For example, the status notification light 151, the forward direction indicator 152, and the backward direction indicator 153 may include one or more light emitting diodes (LEDs).

Conventional analog lighting has a limitation in precisely controlling the illuminance, but the light emitting diode (LED) can precisely control the illuminance by adjusting the amount of current applied and the width of the driving pulse. Also, when the light emitting diodes (LEDs) of R, G, and B colors are provided in combination, the light of a specific color can be provided and the adjustment of the color temperature can be facilitated.

The light emitting diode (LED) may be a single color light emitting diode (LED) such as Red, Blue, Green, and White. According to an embodiment, the light emitting diode (LED) may be a multicolor light emitting diode (LED) capable of reproducing a plurality of colors.

The status indicator 151 may include a plurality of light emitting diodes (LEDs), and the plurality of light emitting diodes (LEDs) may emit white light to provide white light. Red, Blue, and Green light emitting diodes (LEDs) may be combined to provide a particular color of illumination or white light.

For example, the status notification light 151 may include a first color (Yellowish White) indicating a standby / stop state, a second color (Blueish Green) indicating a cleaning in progress state, a third And a corresponding fourth color (sky blue) light is output when the robot search signal is received.

Meanwhile, the status notification light 151 may be turned off after a predetermined time period after outputting light of a first color indicating a standby / stop state.

In a case where the status notification light 151 includes a plurality of light emitting diodes, a plurality of light emitting diodes may be turned on simultaneously, and then a plurality of light emitting diodes may be sequentially turned off at regular intervals.

In addition, at the start of the initial cleaning, all the light emitting diodes of the status notification light 151 blink at the same time and can output light of the second color indicating the cleaning in progress status.

The status notification light 151 may display a current progress state of the output light through color, and may serve as a kind of signal light for notifying stop, avoidance, and state change for avoiding the human body.

For example, upon a pause / error, the status notification light 151 may provide red light.

In addition, when avoiding a human body, the state notifying light 151 may blink light of a specific color to notify people of the avoidance driving.

The mobile robot 100 according to an embodiment of the present invention can be set to avoid an obstacle immediately when a human body is sensed or to avoid obstacles depending on whether a person is moving after waiting for a predetermined time. The state notifying lamp 151 can output light of a predetermined color constantly for a predetermined time before blinking and during avoidance running, or blink.

In addition, when a human body is detected, the sound output unit 181 can output and provide voice reminders to people such as " Please take a moment. &Quot;

The forward direction indicator 152 and the backward direction indicator 153 may indicate direction indication and emergency stop.

The forward direction indicator 152 and the backward direction indicator 153 may be turned on and blink according to the traveling direction of the mobile robot 100. [ In addition, the forward direction indicator 152 and the backward direction indicator 153 may be synchronized with each other and turned on or blink together.

The front direction indicator light 152 and the rear direction indicator light 153 may include indicator lamps corresponding to the left and right directions, respectively.

1, the front direction indicator light 152 may include a left direction indicator light 152L and a right direction indicator light 152R. Referring to FIG. 2, the rear direction indicator light 153 includes a left direction indicator 153L , And a right turn indicator 153R.

When the mobile robot 100 turns left or right during traveling, the turn signal lamps 152L, 152R, 153L, and 153R corresponding to the direction in which the direction is to be turned on may turn on or blink.

1, the forward direction indicator light 152 includes a leftward direction indicator 152L and a rightward direction indicator 152R arranged adjacent to each other, but is not limited to the position of the direction indicator in the present invention. For example, the forward direction indicator light 152 may also be arranged such that the indicator light 152L corresponding to the left direction and the indicator light 152R corresponding to the right direction are spaced apart from each other by a predetermined distance.

Further, at the avoidance traveling, the forward direction indicator light 152 and the rearward direction indicator light 153 can also notify the area departure as an emergency flashing.

Meanwhile, the control unit 140 may control the lamp unit 150.

For example, the control unit 140 may control the status notification light 151 to output lights of different colors according to the current state of the mobile robot 100. Also, the controller 140 may control the status notification lamp 151 to blink at predetermined intervals for a predetermined period of time.

The control unit 140 may control the forward direction indicator 152 and the backward direction indicator 153 to be turned on according to the traveling direction of the mobile robot 100. In addition, the controller 140 may control the forward direction indicator 152 and the backward direction indicator 153 to flicker at predetermined intervals for a predetermined time.

Meanwhile, the forward direction indicator 152 and the backward direction indicator 153 may be synchronously driven.

That is, the controller 140 can control the forward direction indicator 152 and the backward direction indicator 153 to turn on or blink in the same manner as the left / right rotation.

The control unit 140 controls the forward direction indicator 152 and the backward direction indicator 153 to output light having the same color as the light output from the status indicator 151 in a specific situation Can be controlled.

Meanwhile, referring to FIG. 1, the status indicator light 151 may have a circular ring shape. In addition, the status notification light 151 may be in the shape of a circular ring disposed along the outer edge of the front surface of one of the cameras included in the image acquisition unit 120. [

Meanwhile, as described above, the controller 140 can recognize the attribute of the obstacle based on the image acquired by the image acquiring unit 120. [0034] FIG.

In addition, the controller 140 may control driving of the driving unit 160 based on the recognized property of the obstacle.

For example, the control unit 140 may control the movement of the main body 101 to stop when the recognized obstacle is at least a part of the human body.

In this case, the controller 140 may control to resume the existing movement when the recognized movement of the body is detected within a predetermined waiting time. In addition, the control unit 140 may control to avoid the recognized body if the recognized movement of the body is not detected within a predetermined waiting time.

Also, the controller 140 may control the sound output unit 181 to output a predetermined sound when the recognized obstacle property is at least a part of the human body.

Since public areas such as airports and terminals have a very wide area to be cleaned, the entire area can be set as a plurality of cleaning areas. Further, one or more mobile robots can be allocated to each cleaning area.

The allocation of the cleaning area may be automatically divided so that a plurality of cleaning areas have similar areas based on the maximum area of the entire area.

Alternatively, a plurality of cleaning zones may be manually designated and stored by an administrator.

4, a robot system according to an embodiment of the present invention includes a plurality of mobile robots 100a, 100b, 100c, 100d, and 100e and a mobile terminal 200, and a plurality of mobile robots 100a, 100b, 100c, 100d, and 100e are each assigned a cleaning zone to perform a cleaning operation for the assigned cleaning zone. One or more mobile robots can be assigned to each cleaning zone. It is also possible to allocate a plurality of mobile robots to one or more cleaning zones depending on the setting of the user.

5 is a simplified block diagram of a mobile terminal according to an embodiment of the present invention.

5, the mobile terminal 200 includes a wireless communication unit 210, an A / V input unit 220, a user input unit 230, a sensing unit 240, an output unit 250, A memory 260, an interface unit 270, a control unit 280, and a power supply unit 290.

The mobile terminal 200 according to one aspect of the present invention includes a display unit 251, a wireless communication unit 210 for exchanging data with other electronic devices such as a server, And a control unit 280 for controlling the application screen to be displayed on the display unit 251.

The mobile robot related application screen may include a main screen capable of controlling a plurality of mobile robots.

The main screen may be based on information on the mobile robots received through the wireless communication unit 210. [

The main screen may include a map area including one or more maps divided into a plurality of areas and a robot information area including robot information items for a mobile robot allocated to the plurality of areas.

The user interface screen and the control of the mobile robot provided through the mobile terminal 200, such as the main screen, will be described later in detail with reference to Figs. 9 to 17.

On the other hand, the wireless communication unit 210 can directly receive position information, status information, and the like from the mobile robot, or receive position information, status information, and the like of the mobile robot through the server.

The wireless communication unit 210 may include a broadcast receiving module 211, a mobile communication module 213, a wireless Internet module 215, a short distance communication module 217, and a GPS module 219.

The broadcast receiving module 211 may receive at least one of a broadcast signal and broadcast related information from an external broadcast management server through a broadcast channel. At this time, the broadcast channel may include a satellite channel, a terrestrial channel, and the like.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 211 may be stored in the memory 260.

The mobile communication module 213 transmits and receives a radio signal to at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to a voice call signal, a video call signal, or a text / multimedia message transmission / reception.

The wireless Internet module 215 refers to a module for wireless Internet access, and the wireless Internet module 215 can be embedded in the mobile terminal 200 or externally. For example, the wireless Internet module 215 can perform wireless communication based on WiFi or wireless based on WiFi Direct.

The short-range communication module 217 is for short-range communication, and includes Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB) (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology.

The short-range communication module 217 may communicate with the mobile terminal 200 and the wireless communication system between the mobile terminal 200 and another mobile terminal 200 or between the mobile terminal 200 and the mobile terminal 200 through wireless local area networks ) And another mobile terminal, or a network in which an external server is located. The short-range wireless communication network may be a short-range wireless personal area network.

The GPS (Global Position System) module 219 may receive position information from a plurality of GPS satellites.

Meanwhile, the wireless communication unit 210 can exchange data with the server using one or more communication modules.

The wireless communication unit 210 may include an antenna 205 for wireless communication, and may include an antenna for receiving broadcast signals in addition to an antenna for communication.

The A / V (Audio / Video) input unit 220 is for inputting an audio signal or a video signal, and may include a camera 221 and a microphone 223.

The user input unit 230 generates key input data that the user inputs for controlling the operation of the terminal. The user input unit 230 may include a key pad, a dome switch, a touch pad (static pressure / static electricity), and the like. In particular, when the touch pad has a mutual layer structure with the display unit 251, it can be called a touch screen.

The sensing unit 240 senses the current state of the mobile terminal 200 such as the open / close state of the mobile terminal 200, the position of the mobile terminal 200, A sensing signal can be generated.

The sensing unit 240 may include a sensing sensor 241, a pressure sensor 243, a motion sensor 245, and the like. The motion sensor 245 can detect the movement or the position of the mobile terminal 200 using an acceleration sensor, a gyro sensor, a gravity sensor, or the like. In particular, the gyro sensor is a sensor for measuring the angular velocity, and it can sense the direction (angle) of rotation about the reference direction.

The output unit 250 may include a display unit 251, an acoustic output module 253, an alarm unit 255, and a haptic module 257.

The display unit 251 displays and outputs information processed by the mobile terminal 200.

Meanwhile, when the display unit 251 and the touch pad have a mutual layer structure to constitute a touch screen, the display unit 251 may be used as an input device capable of inputting information by a user's touch in addition to the output device.

The audio output module 253 outputs audio data received from the wireless communication unit 210 or stored in the memory 260. [ The sound output module 253 may include a speaker, a buzzer, and the like.

The alarm unit 255 outputs a signal for notifying the occurrence of an event of the mobile terminal 200. For example, it is possible to output a signal in a vibration mode. .

The haptic module 257 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 257 is a vibration effect.

The memory 260 may store a program for processing and controlling the control unit 280 and may store a function for temporarily storing input or output data (e.g., a phone book, a message, a still image, .

The interface unit 270 serves as an interface with all the external devices connected to the mobile terminal 200. The interface unit 270 may receive data from the external device or supply power to the respective components in the mobile terminal 200 and may transmit data in the mobile terminal 200 to the external device .

The control unit 280 typically controls the operation of the respective units to control the overall operation of the mobile terminal 200. For example, perform related controls and processing for voice calls, data communications, video calls, and the like. In addition, the control unit 280 may include a multimedia playback module 281 for multimedia playback. The multimedia playback module 281 may be configured in hardware in the controller 280 or separately from software in the controller 280. [

The power supply unit 290 receives external power and internal power under the control of the controller 280 and supplies power necessary for operation of the respective components.

Meanwhile, the block diagram of the mobile terminal 200 shown in FIG. 5 is a block diagram for an embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted depending on the specifications of the mobile terminal 200 actually implemented.

That is, two or more constituent elements may be combined into one constituent element, or one constituent element may be constituted by two or more constituent elements, if necessary. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and apparatuses do not limit the scope of the present invention.

6 and 7 are views showing a state in which a head of a mobile robot is opened according to an embodiment of the present invention.

Referring to FIGS. 6 and 7, the mobile robot 100 may include a display 182 housed inside the main body 101.

Referring to FIG. 6, the display 182 may be disposed in the dust container cover 136.

Alternatively, referring to FIG. 7, the display 182 may be provided separately from the dust container 135.

The display 182 may be slid or pushed to the front to allow the user to more easily recognize the user interface screen provided through the display 182 can do.

8 is a view illustrating a user interface screen provided through a display 182 of a mobile robot according to an embodiment of the present invention.

According to an embodiment of the present invention, when the user opens the head 111 of the mobile robot 100, a user interface screen may be provided through the display 182. [

Or a user interface screen may be provided through the mobile terminal 200 irrespective of whether the head 111 is opened.

A predetermined screen may be displayed on the display 182 or the display unit 251 prior to providing the main screen including the top menu items.

For example, a screen including a message for welcoming the user (Welcome Scree), a screen for user authentication, or the like may be displayed on the display 182 or the display unit 251.

8, the main screen 800 provided through the display 182 includes a status bar area 810 in which current date / time information and the like are displayed, various information of the mobile robot 100 are displayed And a main menu area 830 including a plurality of main menu items.

In addition to date / time information, an icon corresponding to predetermined status information such as a network status may be displayed in the status bar area 810.

The information displayed in the monitoring area 820 may include battery information, status information, and status guidance for each error condition.

The battery information may include an icon corresponding to the current battery charge value and the battery state. The icon corresponding to the battery status may be displayed as a graphic image that indicates the level corresponding to the current battery charge level by dividing the battery charge status into a plurality of levels.

The state information may include a current state of the mobile robot 100 in the standby state, during a pause, an emergency error occurrence, a battery discharge, an accessory state, and the like.

In addition, the status guidance message may be a guidance message corresponding to the current state of the mobile robot 100.

For example, if an error occurs in the side brush, the side brush error may be displayed in the monitoring area 820 as graphic and / or text, and a guideline such as 'please remove foreign substances in the side brush' may be displayed have.

Alternatively, when the user selects the command to go to the charging station, the monitoring area 820 displays 'Go to charging station' or 'Pause to go to charging station' as status information, and the head of the cleaning robot If you close it, the robot can go back to the charging station! '.

In addition, the monitoring area 820 may include an emergency call button for calling an upper manager, and an operator can quickly call an upper manager by pressing an emergency call button if necessary.

On the other hand, the main menu area 830 is an area for displaying the main menu. The main menu area 830 includes a return item 831 for inputting a move command to a specified place (for example, a charging station) A start item 832, and a robot management item 833.

The control unit 140 controls the display unit 820 to display a guidance message to guide the user to close the head 111 when a selection input to the return entry 831 or the cleaning start entry 832 is received .

Alternatively, the control unit 140 may cause the display 182 to display a pop-up message for guiding the head 111 to close when receiving a selection input for the return item 831 or the cleaning start item 832, Can be controlled to be displayed.

Thereafter, when the user closes the head 111, the control unit 140 may control to perform an operation corresponding to the input command.

The control unit 140 can control the robot management detailed screen to be displayed on the display 182 when a selection input to the robot management item 833 is received.

For example, the robot management detailed screen may include an accessory status item, an error history item, a password setting item, a robot version information item, and the like.

The control unit 140 may control the display unit 182 to display various notification messages based on signals received through the communication unit 190 or data sensed by the sensor unit 170. [ The notification message can be displayed in the form of a pop-up window.

FIGS. 9 to 17 are diagrams illustrating a user interface screen provided through a display unit of a mobile terminal according to an exemplary embodiment of the present invention, and a method of controlling a robot system using the same.

The mobile terminal 200 according to one aspect of the present invention includes a wireless communication unit 210 for exchanging data with other electronic devices such as a display unit 251, a mobile robot, a server, and the like, And a control unit 280 for controlling the mobile robot related application screen to be displayed on the display unit 251. [

The mobile robot related application screen may include a main screen capable of controlling a plurality of mobile robots.

According to an embodiment, a predetermined screen may be displayed on the display unit 251 prior to providing the main screen including the top-level menu items.

For example, a screen including a message for welcoming the user (Welcome Scree), a screen for user authentication, and the like may be displayed on the display unit 251.

9, the main screen 900 includes a map area 930 including one or more maps 931 and 932 divided into a plurality of areas A, B, C, D and E, And a robot information area 940 including robot information items 941, 942, 943, 944, and 945 for the mobile robot to be allocated to the plurality of zones A, B, C, D, .

The main screen 900 may be based on information on the mobile robots received through the wireless communication unit 210. [

The wireless communication unit 210 can receive information on the mobile robots from the mobile robots directly or via the server.

For example, the information on the received mobile robots may include at least one of position information, state information, disposed zone information, and battery state information of the mobile robots.

9, the main screen 900 provided through the display unit 251 includes a status bar for displaying current date / time information, connected communication network information, battery information of the mobile terminal 200, A title area 920 in which an application related button such as an area 910, a name information, a notification center, and a basic setting is placed, a map area 930 in which one or more maps are displayed, And a robot information area 940 in which information is displayed.

In the map area 930, a service area full map can be displayed. Alternatively, the map area 930 may display one or more maps set by the administrator among the entire maps.

Each of the maps 931 and 932 may be a SLAM map used by the mobile robot 100 when traveling, or may be a map generated in a different manner.

Further, each of the maps 931 and 932 may be a simplified map based on SLAM maps or other maps.

On the other hand, FIG. 9 illustrates a case where the mobile robot provides a cleaning service at an airport, and therefore, the maps 931 and 932 may be maps for some areas in the airport.

For example, when an area where a duty-free shop or the like is located after passing through an emigration control center of an airport is classified into an air area and the remaining area is classified into a land area, the first map 931 is divided into three areas of an airport land area And the second map 932 may be a first floor map of the airport land area.

If a map other than the first and second maps 931 and 932 is further displayed in the map area 930, a scroll button 930 for vertically scrolling the screen Or a switch button for changing the displayed map or the like may be further displayed.

On the other hand, the first map 931 can be divided into a plurality of zones A, B and C. Also, the second map 932 may be divided into a plurality of zones D and E, respectively.

Icons of the mobile robots disposed in the respective zones may be displayed in the plurality of zones A, B, C, D, and E.

More preferably, the current position of the allocated mobile robot can be displayed in a plurality of zones A, B, C, D, and E included in the map area 930, B, C, D, and E), the icon of the mobile robot may be displayed on a point corresponding to the current position of the mobile robot.

On the other hand, when a touch input to any one of the plurality of zones A, B, C, D, and E is received, the control unit 280 determines whether the touch input of the mobile robot So that the individual robot detailed screen can be displayed on the display unit 251.

The control unit 280 may control the display unit 251 to display the individual robot detailed screen of the mobile robot having received the touch input when the touch input to the icon of the mobile robot is received.

On the other hand, state information of the mobile robots allocated to each of the plurality of zones A, B, C, D, and E included in the map area 930 can be displayed as text. For example, depending on the current state of each mobile robot, status information may be displayed in the A, C, D,

The colors of the plurality of zones A, B, C, D, and E included in the map area 930 may vary depending on the status information of the mobile robot.

For example, the area where the mobile robot having a failure is disposed is displayed in red, the area where the mobile robot is waiting is displayed in green, and the area where the mobile robot is being cleaned can be displayed in blue.

On the other hand, when an input to touch the first one of the plurality of zones A, B, C, D, and E to the second zone is received, So that the mobile robot and the mobile robot assigned to the second zone are mutually exchanged.

That is, according to an embodiment of the present invention, the allocation area of the mobile robots can be easily changed by drag & drop.

In the robot information area 940, robot information items 941, 942, 943, 944, and 945 corresponding to a plurality of mobile robots can be displayed.

The robot information items 941, 942, 943, 944, and 945 may include name information, status information, disposed zone information, battery status information, and one or more operation buttons of the corresponding mobile robot.

Here, the state information of the mobile robot may include a current state such as during cleaning, waiting, going to a charging station, during an emergency stop, or during a pause. Further, when the state information of the mobile robot is being cleaned, information on the cleaning progress time can be further displayed.

The robot information items 941, 942, 943, 944, and 945 may include a cleaning start button that can remotely start cleaning and / or a recharge button to return to the charging station.

On the other hand, when the mobile robot is being cleaned, a pause button may be displayed on the robot information items 941, 942, 943, 944, and 945.

On the other hand, when the number of robot information items 941, 942, 943, 944, and 945 displayed in the robot information area 940 is large and it is difficult to display on one screen, the robot information area 940 is moved up and down A scroll button for scrolling the screen, a switching button for changing the displayed item, and the like.

On the other hand, when a touch input to any one of the robot information items 941, 942, 943, 944, and 945 is received, the control unit 280 determines whether the touch input corresponds to the received robot information item So that the individual robot detailed screen of the mobile robot to be displayed on the display unit 251 can be controlled.

That is, the mobile terminal 200 according to an embodiment of the present invention receives information on a plurality of mobile robots included in the robot system through the wireless communication unit 210, and displays a main screen based on the information on the display unit 251).

When the user selects a zone allocated to the first mobile robot or the first mobile robot among the plurality of mobile robots through the main screen, the control unit 280 displays a user interface screen for the selected first mobile robot on the display unit (251). Here, the user interface screen for the first mobile robot may be the individual robot detailed screen of the first mobile robot.

10 is a diagram showing an example of a user interface screen displayed when an 'A zone' or an 'airport cleaning robot 1' robot information item 941 is selected, that is, when the first mobile robot is an 'airport cleaning robot 1' (1000).

10, the individual robot detailed screen 1000 of the first mobile robot provided through the display unit 251 displays current date / time information, connected communication network information, battery information of the mobile terminal 200, A status bar area 1010 to be displayed, title information, a title area 1020 in which individual robot basic setting buttons are arranged, a map item 1031 corresponding to an assigned zone, A map and remote control area 1030 including a remote control item 1032 including one or more operation buttons for controlling the robot and a designated area cleaning setting item 1035 for specifying a cleaning area of the first mobile robot, And a detailed menu including a detailed menu item 1041 including a plurality of upper menus, a manual control item 1042 for manually operating the first mobile robot, and a manual control area 1040 .

The map item 1031 of the map and remote control area 1030 may include a map of the 'A zone' allocated to the 'airport cleaning robot 1' and current location information of the 'airport cleaning robot 1' displayed on the map have.

The remote control item 1032 of the map and remote control area 1030 may include a cleaning start button that can remotely start cleaning and / or a recharge station go back button that can command a return to the charging station.

When the user selects the cleaning start button of the remote control item 1032, the map and remote control area 1030 can be switched to the screen as shown in FIG.

11, the current position of the mobile robot, the area where the cleaning is completed, and the area where the cleaning is not performed are displayed on the map, and the cleaning progress information 1132 may be displayed in the map item 1131. [ Herein, the cleaning progress information 1132 may include information on the cleaning progress time and the cleaning progress rate.

On the other hand, when the mobile robot is being cleaned, the remote control item 1132 may include a pause button.

Referring to FIGS. 10 and 11, the pause button may be displayed at a position where the cleaning start button is displayed.

Referring to FIGS. 10 and 12, when the user selects the button for the charging station of the remote control item 1032, the map and remote control area 1030 can be switched to the screen as shown in FIG.

Referring to FIG. 12, the current position of the mobile robot may be displayed on the map in the map item 1231, and the remote control item 1232 may include a pause button.

Referring to FIGS. 10 and 12, the pause button may be displayed at a position where the go to charging station button is displayed.

Referring to FIG. 10, the detailed menu and the manual control area 1040 may include a manual control item 1042 for manually operating the mobile robot.

The manual control item 1042 may include an operation button for moving the mobile robot 100 forward, backward, turning or stopping, and an operation button for operating the suction strength.

Meanwhile, in order to prevent a safety accident, the manual control item 1042 may be set to a locked state by default. In this case, the locked state of the manual control item 1042 can be released by a predetermined pre-set unlock command.

The robot system according to an embodiment of the present invention may include at least one mobile robot 100 and a mobile terminal 200 capable of controlling the mobile robot 200.

10, the display unit 251 of the mobile terminal 200 can display a user interface screen including a manual control item 1042 capable of manually operating the mobile robot 100 have.

13, the control unit 280 of the mobile terminal 200 may set the manual control item 1042 to a locked state. The control unit 280 may also set a default value of the manual control item 1042 ) State is set to the locked state, it is possible to prevent the situation in which the mobile robot 100 is manually operated in a state in which the safety is not ensured by the user, or in which the mobile robot 100 is operated by unintentional unintentional touch have.

Also, the control unit 280 can control the display state of the manual control item 1042 in the locked state to be different from the displayed state in which the locked setting is released.

 For example, when the locked state is set and the setting is canceled, the control unit 280 controls the operation buttons that allow the mobile robot 100 to move forward, backward, change direction, stop, Lt; / RTI >

Also, the control unit 280 can control the operation buttons included in the manual control item 1042 to be displayed dimmed or dimmed in the locked state.

Referring to FIG. 14, when an input to any one of various buttons included in the manual control item 1042 is received, the control unit 280 can release the locked state by the display operation of the mobile robot The guidance message 1410 may be displayed on the display unit 251. Here, the guidance message 1410 may be provided in a pop-up form.

The guide message 1410 may be displayed by selecting a close button displayed therein, receiving a predetermined signal from the mobile robot 100, or automatically terminating display after a predetermined time elapses.

The mobile terminal 200 and the mobile robot 100 according to the present invention can wirelessly communicate according to a communication standard such as LTE and Wi-Fi, and a user can operate the mobile terminal 200 to remotely move the mobile robot 200 away from the mobile robot 100, (100).

On the other hand, the mobile robot 100 can be controlled through the mobile terminal 200 even if there is a wall or an obstacle between the user and the mobile robot 100 due to the characteristics of wireless communication.

However, when the user and the mobile robot 100 are too far away, or when there is a wall or an obstacle between the user and the mobile robot 100, the user can not check the current state and the surrounding environment of the mobile robot 100 A safety accident may occur when the mobile robot 100 is operated manually.

For example, when the user selects the forward or backward button included in the manual control item 1042 and the forward or backward movement of the mobile robot 100 occurs, an accident may occur in which the user is moving or colliding with a nearby person .

Accordingly, in the present invention, the manual control item 1042 is set such that the locked state can not be released only by the operation of the mobile terminal 200, so that the user can check the current state and the surrounding environment of the mobile robot 100 And to release the locked state. Accordingly, it is possible to prevent an accident that may occur during the manual operation of the mobile robot 100.

The wireless communication unit 210 of the mobile terminal 200 may transmit the first signal corresponding to the input to the manual control item to the mobile robot 100 under the control of the control unit 280. [

On the other hand, the display 182 of the mobile robot 100 can be housed in the main body 101. The display 182 may display a predetermined user interface screen such as the user interface screen described with reference to FIG. 8 when the head 111 is opened or another opening / closing structure is opened under the control of the controller 140.

Further, the mobile robot 100 can receive the first signal through the communication unit 190. [ According to an embodiment, the communication between the mobile robot 100 and the mobile terminal 200 may be performed via a predetermined server.

The control unit 140 of the mobile robot 100 displays the display 182 to display a lock release confirmation button capable of releasing the locked state setting of the manual control item 1042 upon receipt of the first signal Can be controlled.

FIG. 15 illustrates an example of the unlock pop-up 1510 including a unlocking confirmation button 1512 that can be unlocked from the locked state of the manual control item 1042 displayed on the display 182. FIG.

The unlock pop-up 1510 may include a cancel button 1511 and a lock release confirmation button 1512. [

If the user selects the cancel button 1511, the control unit 140 can control the display of the unlocking pop-up 1510 to be terminated. In this case, the mobile terminal 200 does not release the locked state of the manual control item 1042 because it does not receive the feedback signal corresponding to the unlocking.

Alternatively, the control unit 140 may control the communication unit 190 to transmit a signal informing the selection of the cancel button 1511 of the user to the mobile terminal 200.

On the other hand, the display 182 may display an authentication screen for user authentication under the control of the controller 140 when the selection input for the unlocking confirmation button 1512 is received.

Referring to FIG. 16, the authentication screen may be a password input screen 1600 for inputting a preset password.

16 illustrates a user interface screen for inputting a password when the user authentication using the password input method is used. However, the present invention is not limited to this, and another user authentication method can be used.

16, the password input screen 1600 is arranged at the top of the screen and displays a status bar (status bar) for displaying status information of the mobile robot 100 such as information on parts replacement, an error, ).

Also, the password input screen 1600 may include an input button area for inputting a password. The input button area may include a text guide such as 'Please enter 4 digits of password' or 'Try again'.

In addition, the input button area may include a password input button such as a numeric panel and a confirmation key.

In addition, the input button area may include a 4-digit password input window and a number delete key, which are displayed when the input number is displayed for a while.

When a legitimate user authentication input is received through the password input screen 1600, the communication unit 190 transmits a second signal corresponding to the legitimate user authentication input to the mobile terminal 200, under the control of the control unit 140, Lt; / RTI >

The mobile terminal 200 receives the second signal through the wireless communication unit 210 and the control unit 280 releases the locked state of the manual control item 1042 in response to the reception of the second signal can do.

That is, according to the present embodiment, the selection of the manual control item 1042 through the mobile terminal 200, the selection of the unlocking confirmation button 1512 through the mobile robot 100, the user authentication through the password input screen 1600 , It is possible to prevent the manual control item 1042 from unintentionally releasing or manually moving the mobile robot 100 inadvertently.

Particularly, by requesting the operation through the mobile robot 100, when the user is not in the adjacent position where the state of the mobile robot 100 and the surrounding environment can be checked, the locked state of the manual control item 1042 can not be released Can be set.

Accordingly, the risk of a safety accident caused by manual operation of the mobile robot 100 can be remarkably reduced.

The communication unit 190 of the mobile robot 100 may select the unlocking confirmation button 1512 under the control of the control unit 140. In this case, And may transmit a third signal corresponding to the input to the mobile terminal 200.

The mobile terminal 200 receives the third signal through the wireless communication unit 210 and the control unit 280 releases the lock state of the manual control item 1042 in response to the reception of the third signal can do.

That is, according to the present embodiment, by requiring a two-step procedure such as selection of the manual control item 1042 through the mobile terminal 200 and selection of the unlocking confirmation button 1512 via the mobile robot 100, It is possible to prevent the control item 1042 from unintentionally releasing or manually moving the mobile robot 100 inadvertently.

Even in this case, since the operation through the mobile robot 100 is essentially required, the risk of a safety accident caused by the manual operation of the mobile robot 100 can be remarkably reduced.

The communication unit 190 of the mobile robot 100 may include a wireless router AP and the wireless communication unit 210 of the mobile terminal 200 may transmit the identification information of the wireless router AP Or can access the wireless router (AP).

Here, the identification information of the wireless router (AP) may be a service set ID (SSID) which is a service group name, and a MAC address that can identify each wireless router (AP).

The SSID can be referred to as a network identifier because it can identify the network to which the access point belongs, and the MAC address is an identifier of the access point because it can identify the access point. Other information may also be used to identify the network or access point in addition to the SSID or MAC address.

Meanwhile, the user who confirmed the guidance message 1410 can access the mobile robot 100 by holding the mobile terminal 200 in order to operate the mobile robot 100.

When the distance between the mobile terminal 200 and the mobile robot 100 is within a predetermined distance according to a user's approach, the wireless communication unit 210 of the mobile terminal 200 transmits a signal to the wireless router AP) of the mobile terminal.

The control unit 240 of the mobile terminal 200 can control to release the locked state of the manual control item 1042 when the identification information of the wireless router AP is recognized.

According to the present embodiment, since the locked state of the manual control item 1042 is released until the mobile robot 100 is physically within a predetermined distance, the user 100 can control the mobile robot 100 while watching the mobile robot 100 ,

 It is also possible to reduce the risk of remote control in a situation where the user 100 can not confirm the state of the mobile robot 100 and the surrounding environment and also to open the head of the mobile robot 100 and press the OK button displayed on the display 182 There is convenience that there is no need to press.

On the other hand, the control unit 280 of the mobile terminal 200 may set the manual control item 1042 to the locked state again if there is no input within a predetermined time after the lock setting is released.

The control unit 280 of the mobile terminal 200 displays a guidance message indicating that the locked state can be released by the display operation of the mobile robot only when the current state of the mobile robot 100 is in the standby state .

17A illustrates an individual robot detail screen 1700 displayed when the mobile robot 100 is in the cleaning state.

17A, in the map and remote control area 1710, there are displayed map items in which the current position of the mobile robot, the area where the cleaning is completed and the area where the cleaning is not performed are displayed on the map, the cleaning progress time, , A pause button, a button to go to the charging station, etc., can be displayed.

17A, the manual control item 1720 can be set to the locked state.

Even if an input to the manual control item 1720 is received when the mobile robot 100 is being cleaned, it can be set to prohibit the lock state release in the cleaning state.

In addition, the control unit 280 can control the guidance message to be displayed only when the current state of the mobile robot 100 is in the standby state.

Therefore, when the mobile robot 100 is being cleaned, a guidance message indicating that the locked state can be released by the display operation of the mobile robot is not displayed even if the input to the manual control item 1720 is received.

17B, when the current state of the mobile robot 100 is not in the standby state, the control unit 280 transmits a guidance message to inform that the manual control is possible only when the mobile robot 100 is in the standby state The control unit 1730 controls the display unit 251 to display the image.

18 is a conceptual diagram of a robot system including a plurality of mobile robots and a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 18, a robot system according to an embodiment of the present invention includes a plurality of mobile robots 100a, 100b,... And a plurality of mobile robots 100a, And may include a terminal 200.

The robot system according to an embodiment of the present invention may include a plurality of mobile robots 100a and 100b, a mobile terminal 200, and a server 50 of a robot manufacturer or a service provider.

The plurality of mobile robots 100a and 100b transmit map data in which the position information of the mobile robots 100a and 100b is stored in accordance with communication standards such as LTE and WiFi through the communication units 190a and 190b 50).

The map data may be an entire map of the service area stored in the storage units 130a and 130b of the mobile robots 100a and 100b or a map of the area allocated to each of the mobile robots 100a and 100b.

On the other hand, the control units 140a and 140b of the mobile robots 100a and 100b can determine the current position based on data acquired from the sensor units 170a and 170b or the image acquisition unit (not shown) It is possible to control map data stored in the robot 100a or 100b to be transmitted to the server 50.

Alternatively, the control units 140a and 140b may control the location information of the mobile robots 100a and 100b to be transmitted to the server 50. The server 50 may store the received location information in a pre- Can be combined into the data.

The server 50 is a user interface capable of displaying map data including the start (current) position of the mobile robots 100a and 100b or a map screen including the start (current) positions of the mobile robots 100a and 100b (UI) data to the mobile terminal 200.

The control unit 280 of the mobile terminal 200 controls the display unit 251 to display the user interface screen such as the main screen or the individual robot detailed screen described above on the basis of the data received through the wireless communication unit 210 .

On the other hand, the control unit 280 can control the various information to be stored in the memory 260. [

The control unit 280 may control the display unit 251 to display an individual robot detailed screen when one of the robots is selected through the main screen displayed on the display unit 251. [

When the manual control item included in the individual robot detailed screen is selected, the control unit 280 controls the display unit 251 to display a guidance message indicating that the locked state can be released by the display operation of the mobile robot, can do.

The control unit 280 can also control the wireless communication unit 210 so that a signal indicating the selection of the manual control item is transmitted to the server 50 or the mobile robots 100a and 100b.

For example, when the predetermined mobile robot 100a is selected on the main screen, the controller 280 displays the guide message on the display unit 251, and the wireless communication unit 210 transmits a predetermined signal to the server 50 To the mobile robot 100a.

The control unit 140a of the mobile robot 100a transmits a guidance message including a lock release confirmation button that can cancel the locked state setting of the manual control item in response to the signal received directly or via the server 50 So that it can be controlled to be displayed on a display (not shown).

The control unit 140a may control the communication unit 190a to transmit a signal to the mobile terminal 200 if there is a legitimate user input.

Meanwhile, the control unit 280 of the mobile terminal 200 may cancel the locked state setting of the manual control item in response to the legitimate user's input.

According to at least one of the embodiments of the present invention, it is possible to provide a robot system and a control method thereof that can be safely operated in a public place.

Further, according to at least one of the embodiments of the present invention, a large space can be effectively cleaned.

In addition, according to at least one embodiment of the present invention, it is possible to improve the usability of the user by providing a robot system and a control method thereof that enable a user to easily check and control information.

The mobile robot, the mobile terminal, and the robot system according to the present invention can be applied to all or some of the embodiments of the present invention so that various modifications can be made to the embodiments and methods of the embodiments described above. Some of which may be selectively combined.

Meanwhile, the control method of the mobile robot, the mobile terminal, and the robot system according to the embodiment of the present invention can be implemented as a code that can be read by a processor in a recording medium readable by the processor. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

Mobile robots 100, 100a, 100b, 100c, 100d, 100e
Body: 101
Control section: 140
Status notifications: 151
Front turn indicator: 152
Rear turn indicator: 153
Driving section: 160
Mobile terminal: 200
Wireless communication part: 210
Display unit: 251
Control section: 280

Claims (14)

One or more mobile robots; And
A display unit for displaying a user interface screen including a manual control item capable of manually operating the mobile robot; and a display unit for setting the manual control item to a locked state, and when an input for the manual control item is received, And a control unit for controlling the display unit to display a guidance message indicating that the robot can be released by a display operation of the mobile robot. The method according to claim 1,
The mobile terminal includes:
And a wireless communication unit for transmitting a first signal corresponding to an input to the manual control item to the mobile robot. 3. The method of claim 2,
The mobile robot includes:
A display unit for displaying a user interface screen for the mobile robot, a communication unit for receiving the first signal, and a unlocking confirmation button for releasing the locked state setting of the manual control item on receipt of the first signal And a control unit for controlling the display to control the display. The method of claim 3,
Wherein the display of the mobile robot displays an authentication screen for user authentication when a selection input for the unlocking confirmation button is received. 5. The method of claim 4,
Wherein the authentication screen is a password input screen for inputting a preset password. 5. The method of claim 4,
Wherein the communication unit of the mobile robot transmits a second signal corresponding to the legitimate user authentication input to the mobile terminal when a legitimate user authentication input is received. The method according to claim 6,
Wherein the control unit of the mobile terminal releases the locked state of the manual control item in response to the reception of the second signal. 8. The method of claim 7,
Wherein the control unit of the mobile terminal sets the manual control item to the locked state again if there is no input to the manual control item within a predetermined time. The method of claim 3,
Wherein the communication unit of the mobile robot transmits a third signal corresponding to a selection input to the unlocking confirmation button to the mobile terminal when a selection input to the unlocking confirmation button is received. 10. The method of claim 9,
Wherein the control unit of the mobile terminal releases the locked state of the manual control item in response to the reception of the third signal. 11. The method of claim 10,
Wherein the control unit of the mobile terminal sets the manual control item to the locked state again if there is no input to the manual control item within a predetermined time. The method according to claim 1,
Wherein the control unit of the mobile terminal controls the guidance message to be displayed only when the current state of the mobile robot is in the standby state. 13. The method of claim 12,
When the current state of the mobile robot is not in the standby state, the control unit of the mobile terminal controls to display on the display unit a guidance message informing that manual control is possible only when the mobile robot is in standby A robot system characterized by. The method according to claim 1,
The mobile robot includes a communication unit including a wireless router (AP)
Wherein the control unit of the mobile terminal releases the locked state of the manual control item when identification information of the wireless router (AP) is recognized.

Images