KR20130027339A - Mobile robot, and system and method for remotely controlling the same - Google Patents

Abstract

PURPOSE: A mobile robot, a remote control system thereof, and a method are provided to watch surrounding of the mobile robot in remote and to rapidly confirming a state of the mobile robot by checking the state in real time. CONSTITUTION: A mobile robot comprises an image detecting unit, a driving unit, and a control unit. The image detecting unit makes a film of surroundings and generates image information. The driving unit includes one or more wheels and drives the wheels according to a driving signal. The control unit generates the driving signal based on a movement command and generates the image information by controlling the image detecting unit. [Reference numerals] (AA) Mobile robot; (BB) Terminal device; (S110) Inputting a moving command; (S120) Generating a control signal; (S130) Control signal; (S140) Extracting the moving command; (S150) Executing the moving command; (S160) Generating image information; (S170) Image information; (S180) Displaying the image information

Description

MOBILE ROBOT, AND SYSTEM AND METHOD FOR REMOTELY CONTROLLING THE SAME}

The present invention relates to a remotely controllable mobile robot, a terminal device for remotely controlling the mobile robot, and a remote control system and method for a mobile robot including the same.

In general, robots have been developed for industrial use and have been a part of factory automation. In recent years, medical robots, aerospace robots, and the like have been developed, and household robots that can be used in ordinary homes are being developed.

A representative example of the domestic robot is a robot cleaner, which is a kind of electronic equipment that sucks dust and foreign matter around while driving a certain area by itself. Such a robot cleaner is generally equipped with a rechargeable battery and has an obstacle sensor capable of avoiding obstacles during traveling, so that it can run and clean by itself.

On the other hand, a method for controlling the mobile robot includes a method using a remote control which is a user interface, a method using a button provided in the mobile robot body. In addition, mobile robots having magnetic location recognition and mapping functions have been developed using cameras or various sensors.

Embodiments of the present invention have an object to provide a mobile robot that can be controlled remotely using a terminal device.

Embodiments of the present invention have another object to provide a terminal device capable of remotely controlling a mobile robot and checking the state of the mobile robot in real time.

Embodiments of the present invention can remotely control a mobile robot, can remotely monitor the surroundings of the mobile robot, and remote control consisting of a mobile robot and a terminal device to respond quickly by checking the status of the mobile robot in real time Another object is to provide a system and method.

According to an embodiment of the present invention, a mobile robot includes: an image detecting unit configured to photograph surroundings to generate image information, a driving unit including one or more wheels, and to drive the wheels according to a driving signal; And a control unit which generates a signal to move along the movement path and controls the image detection unit to generate the image information.

The remote control system of a mobile robot according to an embodiment of the present invention extracts a movement command from a control signal to perform the movement command, and moves based on the movement command while photographing a surrounding while moving along a movement path to generate image information. And a terminal device for controlling the mobile robot using the control signal and receiving and providing the image information from the mobile robot.

In a remote control method of a mobile robot according to an embodiment, the remote control system of a mobile robot including a mobile robot and a terminal device for controlling the mobile robot, the mobile robot receives a control signal from the terminal device And extracting a movement command from the control signal by the mobile robot, moving the robot along a moving path by performing the movement command, and during the execution of the movement command, Generating image information by capturing the image, transmitting the image information to the terminal apparatus by the mobile robot, and displaying the image information on the screen by the terminal apparatus.

The mobile robot according to the embodiments of the present invention can be easily controlled by using a terminal device such as a mobile terminal from the outside by embedding a communication means for remote control.

The mobile robot according to the embodiments of the present invention may patrol a certain area or move to a point desired by the user, and provide an image of a point desired by the movement path or the user through the terminal device. In addition, the mobile robot may be provided with illumination to provide a high quality image robustly to the surrounding environment.

The mobile robot according to the embodiments of the present invention performs a charging operation by frequently checking the state of the battery during patrol or movement. In addition, the mobile robot transmits state information to the terminal device in real time so that a user or the like can easily check the state of the mobile robot using the terminal device.

The terminal device according to the embodiments of the present invention executes a remote control program to easily control a mobile robot from a remote location, and allows a user or the like to check the status or surrounding conditions of the mobile robot in real time.

The remote control system and method according to the embodiments of the present invention enable the mobile robot to be controlled in more various ways by connecting the terminal device and the mobile robot through a network. In addition, the remote control system and method, it is possible to check the situation of the house in real time from the outside, and to be able to respond quickly according to the state of the mobile robot.

Embodiments of the present invention may improve the convenience of the user and increase the operational efficiency and stability of the mobile robot. In addition, embodiments of the present invention, when connecting to the mobile robot through the terminal device, it is possible to prevent the abuse of the mobile robot by performing the connection authentication operation.

1 is a perspective view showing the appearance of a mobile robot according to embodiments of the present invention;
2 and 3 are block diagrams showing the detailed configuration of a mobile robot according to embodiments of the present invention;
4 is an enlarged view illustrating an output unit of a mobile robot according to one embodiment;
5 is a schematic view of a remote control system of a mobile robot according to one embodiment;
6 is a block diagram illustrating a configuration of a terminal device according to an exemplary embodiment;
7A to 7C are diagrams illustrating an example of screen switching of a terminal device displaying data of a mobile robot;
8A to 8C show screens of a terminal device for explaining manual operation of a mobile robot using the terminal device;
9A and 9B illustrate screens of a terminal device for explaining an operation of manipulating an internal map displayed on a screen;
10A to 10C illustrate screens of a terminal device for explaining an operation of operating a control screen displayed on a screen;
11A and 11B show screens of the terminal apparatus displaying the cleaning screen and the patrol screen switching being performed by the mobile robot;
12A to 12C illustrate screens of a terminal device for explaining an operation of transmitting a cleaning command during patrol of a mobile robot;
13A to 13D illustrate screens of a terminal device for explaining an operation of performing a patrol command during charging of a mobile robot;
14A to 14F illustrate screens of a terminal device for explaining an operation of setting a movement path of a mobile robot and transmitting a movement command;
15A-15C illustrate another example of FIG. 14;
16A to 16C show screens of a terminal device for explaining switching between a patrol command and a move command;
16D to 16I illustrate screens of a terminal device for explaining an operation of performing a patrol command or a movement command during generation of an internal map of a mobile robot;
17A to 17E illustrate screens of a terminal device for explaining an operation of controlling lighting of a mobile robot;
18A to 18D show screens of a terminal device for explaining automatically performing a charging operation during patrol of a mobile robot;
19 is a view showing a screen of a terminal device displaying a wheel catch error during patrol of a mobile robot;
20 and 21 are flowcharts schematically illustrating a remote control method of a mobile robot according to embodiments of the present disclosure.

Referring to FIG. 5, a remote control system of a mobile robot according to an embodiment extracts a movement command from a control signal to perform a movement command, and takes a picture of surroundings while moving along a movement path based on the movement command. And a terminal device 200 for controlling the mobile robot using a control signal and receiving and providing image information from the mobile robot. Here, the movement path is a path according to a movement command for moving the mobile robot from the current position or a certain point to another point. In addition, the movement route may be a patrol route connected to return to the current position. At this time, the move command becomes a patrol command for moving the patrol path, and while performing the patrol command, the mobile robot transmits the image information to the terminal device.

The terminal device is divided into a mobile terminal and a stationary terminal according to whether the mobile terminal is mobile or not, and includes both a mobile terminal and a fixed terminal. The terminal device is divided into a handheld terminal and a vehicle mount terminal according to whether the user can move directly or not, and includes both a mobile terminal and a stationary terminal. For example, the terminal device may be a cellular phone, a PCS phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player) Navigation). The remote control system can be changed according to the type of the terminal device. For example, the remote control system may use a mobile communication network such as 3G, CDMA, WCDMA, etc. in the case of a mobile phone or a smart phone, and the mobile robot and the terminal device transmit a radio signal to a base station, an external terminal, a server, and the like on the mobile communication network. Send and receive

1 and 2, the mobile robot 100 according to an embodiment includes an image detecting unit 110, a driving unit 120, and a control unit 130. The image detecting unit 110 generates image information by photographing a surrounding, and the driving unit 120 includes one or more wheels, and drives the wheels according to a driving signal. The control unit 130 generates the driving signal based on the movement command to move along the movement path, and controls the image detection unit to generate the image information.

The mobile robot 100 according to an embodiment may further include a communication unit 140 that receives a control signal from an external terminal device 200 and transmits the image information to the terminal device. The control unit 130 controls the mobile robot according to the control signal, and generates a response signal according to the control result. In addition, the control unit 130 extracts the movement command from the control signal, performs the movement command to drive the driving unit to move the mobile robot.

The communication unit 140 is connected to the terminal device in one of wired, wireless, and satellite communication schemes, that is, one of currently available communication schemes, to transmit and receive signals and data. The communication unit 140 receives a control signal from the terminal device. The communication unit 140 transmits state information, obstacle information, location information, image information, an internal map (cleaning map), and the like of the mobile robot. In addition, the communication unit 140 may communicate with the terminal device in one of short-range wireless communication methods such as radio frequency (RF) communication, Bluetooth, infrared communication (IrDA), wireless LAN (LAN), and Zigbee. Communication can be performed. For example, when the terminal device is a smartphone, the mobile robot may include a communication unit according to the communication scheme available for the smartphone.

The image detection unit 110 is provided to face upward or forward, and is provided with an upper camera sensor to detect the image information by capturing the surroundings of the mobile robot. When the image detection unit 110 includes a plurality of upper camera sensors, the camera sensors may be formed on the top or side surfaces of the mobile robot at a predetermined distance or at an angle. The image detection unit 110 can be used as another type of position recognition unit. Referring to FIG. 3, the image detection unit 110 may include a camera 111 for photographing a subject, a lens 112 connected to the camera 111 to focus the subject, and an adjusting unit 111a for adjusting the camera. And, it may further include a lens adjusting unit (112a) for adjusting the lens. The lens uses a lens having a wide angle of view so that all the surrounding areas, for example, all areas of the ceiling can be photographed even at a predetermined position. The image information is continuously transmitted to the terminal device at the request of the terminal device for a predetermined time. The control unit 130 may extract a feature point from the image information captured by the image detection unit, recognize the position of the mobile robot using the feature point, and generate a cleaning map for the cleaning area.

The image detection unit 110 further includes an illumination 113 for adjusting the brightness of the surroundings. The lighting 113 may be automatically adjusted according to the ambient illumination. In addition, the lighting 113 may be controlled to open or close according to a control command of the terminal device, that is, a command for opening and closing the power of the lighting.

The mobile robot has left and right main wheels which are movable on both lower sides thereof. On either side of the main wheel, a handle may be provided to facilitate user's grip. Referring to FIG. 3, the driving unit 120 is connected to the left and right main wheels 121 and includes a predetermined wheel motor for rotating the wheels to drive the mobile robot by driving the wheel motor. Move it. The wheel motors are respectively connected to the main wheels so that the main wheels rotate, and the wheel motors operate independently of each other and can rotate in both directions. In addition, the mobile robot is provided with one or more auxiliary wheels on the rear side to support the main body, to minimize friction between the lower surface and the bottom surface (cleaned surface) of the main body and to facilitate the movement of the mobile robot.

Referring to FIG. 2 or FIG. 3, the mobile robot according to an embodiment includes an obstacle detecting unit 151 that includes one or more sensors and detects obstacles around the vehicle using the detection signal of the sensor and outputs obstacle information. It is configured to further include. Here, the control unit 130 generates a cleaning map using the obstacle information.

The obstacle detecting unit 151 includes a first sensor installed at a predetermined interval on the outer circumferential surface of the mobile robot, that is, as shown in FIG. 1. In addition, the obstacle detecting unit 151 may include a second sensor installed to have a surface protruding outside the body. The position and type of the first sensor and the second sensor may vary according to the type of the mobile robot, and the obstacle detecting unit may include more various sensors. The first sensor detects an object in the moving direction of the mobile robot, in particular an obstacle, and transmits detection information to the control unit 130. That is, the first sensor detects protrusions, household appliances, furniture, walls, wall edges, etc. existing on the movement path of the mobile robot and transmits the information to the control unit. The first sensor may be an infrared sensor, an ultrasonic sensor, a RF sensor, a geomagnetic sensor, or the like. The second sensor senses an obstacle present on the front or side and transmits the obstacle information to the control unit. That is, the second sensor detects protrusions, household appliances, furniture, walls, wall edges, etc. existing on the movement path of the mobile robot and transmits the information to the control unit. The second sensor may be an infrared sensor, an ultrasonic sensor, an RF sensor, a position sensitive device (PSD) sensor, or the like.

The obstacle detection unit 151 may further include a cliff sensor installed on a lower surface (bottom surface) of the main body and sensing an obstacle on the floor surface, for example, a cliff. The cliff sensor is configured to obtain a stable measurement value regardless of the reflectance and color difference of the bottom surface, and may be in the form of an infrared module such as a PSD sensor.

In addition, the obstacle detecting unit may further include a charging signal sensor for receiving a guide signal from the charging station. The mobile robot checks the position and direction of the charging station by receiving a guide signal generated by the charging station using the charging signal sensor. The charging station sends a guide signal indicating the direction and distance so that the mobile robot can return. The mobile robot receives the signal from the charging station to determine the current position, sets the direction of movement, and returns to the charging station. The charging signal sensor may be an infrared ray sensor, an ultrasonic sensor (Ultra Sonic Sensor), an RF sensor (Radio Frequency Sensor), or the like. In general, an infrared sensor is used. The charging signal sensor is provided at one side of the inside or outside of the mobile robot, and for example, as shown in FIG. 4, the charging signal sensor may be installed below the output unit 170 or around the image detection unit 110. .

Referring to FIG. 3, the mobile robot according to an embodiment includes a location recognition unit 152 having one or more sensors and recognizing the location of the mobile robot itself using the detection signal of the sensor and outputting location information. It is configured to include more. Here, the control unit 130 corrects the cleaning map using the position information recognized by the position recognition unit 152. [

The position recognizing unit 152 is provided on the rear surface of the mobile robot and includes a lower camera sensor which photographs the lower side, that is, the bottom surface and the surface to be cleaned during the movement. The lower camera sensor is an optical flow sensor, and converts a downward image input from an image sensor provided in the sensor to generate image data of a predetermined format. The lower camera sensor may detect the position of the mobile robot regardless of the sliding of the mobile robot. The control unit 130 compares the image data photographed by the lower camera sensor with time to calculate a moving distance and a moving direction, and thus calculates a position of the mobile robot. By observing the lower side of the mobile robot by using the lower camera sensor, the control unit can correct the sliding against the position calculated by other means.

The position recognizing unit 152 further includes an acceleration sensor that detects a change in the moving speed due to a speed change of the mobile robot, for example, starting, stopping, turning, collision with an object, and the like. The acceleration sensor is attached to the adjacent position of the main wheel or the auxiliary wheel, and can detect slippage or idle of the wheel. In this case, the speed may be calculated using the acceleration detected by the acceleration sensor, and the position of the mobile robot may be checked or corrected by comparing with the command speed. However, in general, the acceleration sensor is embedded in the control unit 130 to detect a change in speed of the mobile robot itself generated when cleaning and moving. That is, the acceleration sensor detects the impact amount according to the speed change and outputs a voltage value corresponding thereto. Thus, the acceleration sensor can perform the function of an electronic bumper.

The position recognizing unit 152 may further include a gyro sensor that detects a rotation direction and detects a rotation angle when the mobile robot moves, patrols or cleans. The gyro sensor detects the angular velocity of the mobile robot and outputs a voltage value proportional to the angular velocity. The control unit 130 calculates the rotation direction and the rotation angle using the voltage value output from the gyro sensor.

As shown in FIG. 2, the position recognizing unit 152 may further include a wheel sensor 122 connected to the left and right main wheels 121 and detecting a rotation speed of the main wheel. The wheel sensor 122 mainly uses a rotary encoder and detects and outputs rotational speeds of the main wheels on the left and right sides when the moving robot is cleaned or moved. The control unit 130 can calculate the rotational speeds of the left and right wheels using the number of revolutions.

The control unit 130 can accurately recognize the position using the detection information of the acceleration sensor, the gyro sensor, the wheel sensor, the lower camera sensor, and the image information of the image detection unit. Further, the control unit 130 can precisely generate the cleaning map using the obstacle information detected by the obstacle detecting unit and the position recognized by the image detecting unit. The communication unit 140 transmits data including image information, obstacle information, location information, a cleaning map, a cleaning area, and the like to the terminal device 200.

2 or 3, the mobile robot further includes an input unit 160 that directly receives a control command. Further, the user or the like may input a command through the input unit to output one or more pieces of information stored in the storage unit. The input unit 160 may be formed of one or more buttons. For example, the input unit 160 may include a confirmation button and a setting button. The confirmation button inputs a command for confirming obstacle information, location information, image information, a cleaning area or a cleaning map, a cleaning path, a moving path, a patrol path, and the like. The setting button inputs a command for setting the above information. The input unit may include a reset button, a delete button, a cleaning start button, a stop button, and the like for inputting a command for resetting the information. As another example, the input unit 160 may include a button for setting or deleting reservation information. In addition, the input unit 160 may further include a button for setting or changing a cleaning mode. In addition, the input unit 160 may further include a button for receiving a command to return to the charging station. As illustrated in FIG. 1, the input unit 160 may be installed on the upper part of the mobile robot using a hard key, a soft key, a touch pad, or the like. In addition, the input unit 160 may have a form of a touch screen together with the output unit.

The output unit 170 is provided in the upper part of the mobile robot, as shown in FIG. Of course, the installation location and installation type may vary. For example, the output unit 170 displays, on the screen, a cleaning method or a traveling method such as reservation information, battery status, intensive cleaning, space expansion, zigzag driving, and the like. The output unit 170 may output the current state and the current cleaning state of each unit constituting the mobile robot. In addition, the output unit 170 may display obstacle information, location information, image information, an internal map, a cleaning area, a cleaning map, a cleaning path, a moving path, a patrol path, and the like on the screen. The output unit 170 may be any one of a light emitting diode (LED), a liquid crystal display (LCD), a plasma display panel, and an organic light emitting diode (OLED). It can be formed as an element of. The output unit 170 may further display the remaining battery level on the screen. In addition, the terminal device 200 may receive the state of charge of the battery and the remaining amount of battery from the mobile robot 100 and display the state of charge of the battery and the remaining amount of battery on one side of the screen of the display unit.

Referring to FIG. 3, the mobile robot further includes a storage unit 180 that stores at least one of image information, obstacle information, location information, a cleaning map, a cleaning area, a cleaning path, a moving path, and a patrol path. . The storage unit 180 stores a control program that controls (drives) the mobile robot and data accordingly. In addition, the storage unit 180 may further store a cleaning method, a traveling method, and a location of the charging station. The storage unit 180 mainly uses a nonvolatile memory. Here, a non-volatile memory (NVM, NVRAM) is a storage device that keeps stored information even when power is not supplied. Non-volatile memory includes ROM, flash memory, magnetic computer storage devices (e.g., hard disk, diskette drive, magnetic tape), optical disk drive, magnetic RAM, PRAM, and the like.

Referring to FIG. 3, the mobile robot further includes a power supply unit 190. The power supply unit 190 includes a rechargeable battery to supply power to the mobile robot. The power supply unit supplies driving power to each unit, and operating power for moving, patrolling, or cleaning the mobile robot. If the remaining power is insufficient, the power supply unit is charged to receive a charging current. The mobile robot further includes a battery detection unit (not shown) that detects a charging state of the battery and transmits a detection result to the control unit 130. The battery is connected to the battery detection unit, and the remaining battery level and charge state are transmitted to the control unit 130. [ The battery remaining amount can be displayed on the screen of the output unit. As shown in FIG. 5, the battery may be located at the lower center of the mobile robot, or may be located at either the left or the right side so that the dust container is located at the bottom of the main body. In the latter case, the mobile robot may further comprise a counterweight to eliminate the weight bias of the battery.

The control unit 130 previously sets a reference value (remaining battery level), and compares the remaining battery level with a reference value. As a result of the comparison, if the detection result is equal to or less than the reference value, the control unit 130 moves the mobile robot to the charging station to perform charging. In one example, the control unit 130 may stop the other operation according to the charging command from the terminal device and move the charging operation to the charging station to perform charging. As another example, the control unit 130 may extract a charge command, perform a charge command according to a result of comparison between the remaining battery level and the reference value, or may perform the previous operation as it is.

When the mobile robot is a robot cleaner, the mobile robot may further include a cleaning unit (not shown). The cleaning unit includes a dust container in which dust collected is stored, a suction fan providing power to suck dust in the cleaning area, and a suction motor rotating the suction fan to suck air, thereby removing surrounding dust or foreign matter. Inhale.

Referring to FIG. 6, a terminal device according to an embodiment includes a wireless communication unit 210, a controller 220, and a display 230.

The wireless communication unit 210 transmits a control signal generated by the control unit 220 to the mobile robot 100, and receives one or more data from the mobile robot 100. Here, the one or more data includes image information, obstacle information, location information, an internal map (or cleaning map), a cleaning area, state information, and the like. The wireless communication unit 210 includes one or more modules that enable wireless communication between the terminal device 200 and the wireless communication system, between a plurality of terminal devices, or between a network located between the terminal device and the mobile robot 100. can do. For example, the wireless communication unit 210 may include a broadcast receiving module, a mobile communication module, a wireless Internet module, a short distance communication module, and a location information module.

The broadcast receiving module receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. Mobile communication module Transmits and receives radio signals to and from a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception. The wireless Internet module refers to a module for wireless Internet access, and may be built in or enclosed in a terminal device. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like. The short-range communication module literally means a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

In the display 230, a touch recognition area 231 for receiving a control command is formed and displays a control screen generated by the controller. In addition, the display 230 may display an icon according to a communication method (eg, Wi-Fi, 3G), display a communication sensitivity, and display a battery remaining amount of the mobile robot. In addition, as shown in FIG. 7A or less, the display unit 230 includes a first area S232 having a predetermined size and a size smaller than or equal to the first area in which the control screen is displayed and receives the control command. The touch recognition area may further include a second area S233 of FIG. Of course, in the case of a mobile phone or a notebook computer having no touch screen (touch pad), the touch recognition area is not formed, and an input part for receiving a control command instead of touch recognition and an output part for displaying a control screen can be distinguished.

The display unit 230 alternately displays a moving start icon for receiving a moving start command or a patrol start command, a patrol starting icon, a moving stop icon for receiving a moving stop command or a patrol stop command, and a patrol stopping icon in the touch recognition area. can do. The display unit 230 displays information processed by the terminal device. That is, the display unit 230 displays a control screen. For example, when the terminal device is in the call mode, a UI (User Interface) or GUI (Graphic User Interface) associated with the call is displayed. The display unit 230 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display).

In the case where the display unit 230 and the touch sensor that detects the touch operation form a mutual layer structure, the display unit 230 may be a touch screen that can be input in addition to the output. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like. The touch sensor may be configured to convert a change in a pressure applied to a specific part of the display part 230 or a capacitance generated in a specific part of the display part 230 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch. The touch sensor may be a proximity sensor capable of performing proximity-touch in which a pointer is not actually touched on the screen but approaches a predetermined distance from the screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. In the present invention, the touch recognition is a proximity touch that allows the user to recognize that the pointer is positioned on the touch screen without being touched on the touch screen, and the pointer is actually touched on the touch screen. It includes both contact touch.

Referring to FIG. 6, the terminal device may further include a memory 240. The memory 240 may store a program for the operation of the control unit 220. The memory 240 may also store input / output data (e.g., phonebook, message, still image, moving picture, etc.). The memory 240 may pre-pattern and store a control signal for controlling the mobile robot and a control command corresponding thereto.

The terminal apparatus may further include an A / V (Audio / Video) input unit, a user input unit, a sensing unit, an interface unit, and a power supply unit.

The A / V (Audio / Video) input unit is for inputting an audio signal or a video signal, and may include a camera and a microphone. The user input unit generates input data for controlling the operation of the terminal device by the user. The user input unit may include a key pad, a dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like. In particular, when the touch pad has a mutual layer structure with the display unit 230, it is referred to as a touch screen. The sensing unit senses the state of the terminal device such as the open / close state of the terminal device, the position of the terminal device, the presence of the user, the orientation of the terminal device, and the acceleration / deceleration of the terminal device, and generates a sensing signal for controlling the operation.

The interface unit serves as a pathway to all the external devices connected to the terminal device 200. The interface unit receives data from an external device, receives power from the external device, transfers the data to each component in the terminal device, or allows data in the terminal device to be transmitted to the external device. The power supply unit receives external power and internal power under the control of the controller 220 and supplies power necessary for operation of the respective components.

The control unit 220 generally controls the overall operation of the terminal device. For example, in the case of a mobile phone or smart phone, it performs related control and processing for voice call, data communication, video call, and the like. In addition, the control unit 220 can perform pattern recognition processing that can recognize the handwriting input or drawing input performed on the display unit 230 as characters and images, respectively.

The controller 220 generates a control signal corresponding to a control command for the mobile robot 100, and generates a control screen using data and a response signal. Here, the control command includes a cleaning command, a moving command, a patrol command, a charging command, a setting change, and the like. In addition, the control command may be a command for opening and closing the power of the illumination provided in the mobile robot. The display 230 may further display a lighting icon for opening and closing the power of the lighting 113 provided in the mobile robot 100. As another example, the terminal device may transmit a control signal to the mobile robot and allow the mobile robot to open and close the external light by transmitting and receiving a signal to and from a light controller of an external light present at the same location.

When the controller 220 receives a touch input for one or more points on the internal map displayed on the screen of the display unit, the controller 220 may set a movement path using one or more points. The movement path is a path according to a movement command for moving the mobile robot from the current position or a certain point to another point. When the user touches the internal map so that the user or the like is connected to the current location, the controller 220 may set a patrol route connected to return to the current location. At this time, the move command becomes a patrol command for moving the patrol path, and while performing the patrol command, the mobile robot transmits the image information to the terminal device.

Referring to FIG. 20, in a remote control method of a mobile robot according to an embodiment, the mobile robot receives a control signal from a terminal device (S130), and the mobile robot extracts a movement command from the control signal. (S140), the mobile robot performs the movement command to move along the movement path (S150), and during the execution of the movement command, the mobile robot photographs the surroundings to generate image information (S160). ), And transmitting, by the mobile robot, the image information to the terminal device (S170), and displaying, by the terminal device, the image information on a screen (S180). Hereinafter, a remote control system of a mobile robot including a mobile robot and a terminal device for controlling the mobile robot will be described with reference to FIGS. 1 to 6.

The terminal device 200 receives a control command and generates a control signal according to it (S110, S120). Here, the control command includes a move start command, a patrol start command, a move stop command, a patrol stop command, a cleaning command, a charge command, a setting change, and the like. When a user inputs a control command through an input method such as touching a patrol start icon or a patrol stop icon (S110), the terminal device generates a control signal in accordance with a communication protocol (S120), and transmits the control signal through a network. Transfer to the mobile robot 100 (S130). The mobile robot extracts a control command contained in the control signal (S140), and performs a movement command or patrol command (S150). At this time, the mobile robot can continuously transmit data such as image information, internal maps, obstacle information, etc. to the terminal device (S170), and the terminal device generates a control screen using the data and displays the generated control screen. (S180).

Referring to FIG. 21, while moving or patrolling, the mobile robot may continuously detect a charging state of the battery (S310) and transmit a detection result to the terminal device (S320). In this case, even when there is no charging command of the terminal device, the mobile robot may move to the charging station based on the detection result (S330) and perform charging (S340).

Referring to FIG. 21, in the method of remotely controlling a mobile robot, the terminal apparatus requests a connection to the mobile robot (S10), the mobile robot authenticates the connection (S20), and the mobile robot Approving and confirming the connection of the terminal device (S30) and the terminal device may further include displaying an initial control screen (S40).

First, when a terminal device executes a remote control program for remotely controlling a mobile robot, a login window for inputting login information for the mobile robot is generated. The remote control program may be stored in the memory of the terminal device or downloaded from a server storing the remote control program. For example, in the case of a smartphone, a remote control program is downloaded in the form of an app (App.), Displayed on the screen of the display unit in the form of an icon, and then executed when the user touches the icon. When the user inputs the login information, the terminal device requests a connection to the corresponding mobile robot (S10). The mobile robot performs authentication on the login information, and the terminal device may display a message on the screen during the connection.

The terminal device may be connected to one of the plurality of mobile robots. The display unit displays a plurality of connectable mobile robots in the form of a list or an icon. When the user or the like selects one of the plurality of mobile robots, the mobile robot performs authentication on the terminal device.

The mobile robot transmits an access confirmation signal to the terminal device after authentication (S30), and the terminal device displays an initial control screen (S40). The initial control screen may be displayed including the option menu, or only data such as image information and cleaning map may be displayed. In the latter case, when the touch input to the first area is received, the display unit can display the option menu. The control unit may display the data after the display unit temporarily displays the option menu on the initial control screen.

If the login information does not match the stored authentication information, the mobile robot transmits a connection rejection signal to the terminal device. Then, the terminal device 'does not match the ID or password. Please check and try again. " If the mobile robot to be connected is connected to another terminal device or the mobile robot to be connected is being used by another user, the display unit may display an error message. The terminal device can display the guidance message according to the communication method. If the mobile robot selected by the user is not registered or the network is not connected, an error message may be displayed.

Hereinafter, an operation and a method of a remote control system of a mobile robot according to embodiments will be described with reference to FIGS. 7A to 19.

7A and 7B show an initial control screen as an example. When the user inputs the login information, the terminal device requests a connection to the corresponding mobile robot (S10). After the authentication operation of the mobile robot (S20), when the mobile robot transmits a connection confirmation signal to the terminal device (S30), the display unit displays an initial control screen (S40). The terminal device can display an option menu together for a predetermined time as shown in FIG. 7A and then display a previously stored initial control screen such as image information and an internal map as shown in FIG. 7B. As shown in FIGS. 7B and 7C, the areas displaying the internal map and the image information may be switched. For example, when the first area is touched and dragged to the second area, or after the second area is touched and dragged to the first area, data displayed in each area may be switched.

Referring to FIG. 8A, when a touch is input to a first area while a control command of a mobile robot is being executed or in standby, the display unit displays an option menu. The configuration of the option menu may vary depending on the design. For example, as illustrated in FIG. 8B, the option menu may include menus such as recording, interphone, lighting, cleaning schedule, setting, and the like. In addition, as illustrated in FIG. 8B or 8C, the display unit may further display an icon that enables manual operation of the mobile robot. When the cleaning command is input through the manual operation, the terminal device generates a control signal and transmits it to the mobile robot.

9A to 10C, the terminal device may edit and display an internal map displayed on the screen of the display unit. 9A and 9B, the display unit displays icons such as zoom in, zoom out, and rotate at the bottom of the control screen, and rotates and displays an internal map when a user or the like touches the icon. As shown in FIG. 10A, when a user or the like multi-touches an internal map using two fingers and drags outwards, the display enlarges and displays the internal map. Referring to FIG. 10B, when a user or the like touches and drags a predetermined point, the display moves and displays an internal map in a direction dragged by the user or the like. When the user touches a certain point, the display unit moves and displays the internal map around the touched point. Referring to FIG. 10C, when a user or the like multi-touches an internal map using two fingers and drags inward, the display reduces and displays the internal map. In addition, after the display unit iconizes and displays the mobile robot on the internal map, the display unit may move and display the internal map around the current position of the mobile robot when the user or the like touches the icon.

11A and 11B, the display unit may display a cleaning tab showing a screen for receiving a cleaning start command or a cleaning stop command and a patrol tab for displaying a screen for receiving a patrol command. As shown in FIG. 11A, the terminal device receives a touch input for a patrol tap after displaying a control screen for receiving a cleaning command. Then, as shown in FIG. 11B, the terminal device can display the patrol screen on the screen of the display unit.

12A, during patrol (movement) of a mobile robot, a user or the like may touch the cleaning tab. In this case, as shown in Fig. 22B, the display unit switches to the cleaning screen and displays it. Then, when the user or the like touches the cleaning start icon, as shown in Fig. 12C, the display unit is currently patrolling. Do you want to stop patrol and clean? '

Referring to FIG. 13A, a user or the like may input a patrol start command or the like while the mobile robot is being charged. As shown in FIG. 13B, the terminal device generates a control signal corresponding to the patrol start command and transmits it to the mobile robot, and the mobile robot performs patrolling according to the patrol start command. The terminal device displays a control screen generated using the data received from the mobile robot on the display unit. As shown in FIG. 13C or FIG. 13D, the mobile robot checks the state of charge of the battery and transmits the status information to the terminal device even while performing the patrol command, and the terminal device is "battery low" or "battery low." Please execute it afterwards'.

Referring to FIG. 21, the method for remotely controlling a mobile robot may include: detecting, by the mobile robot, an obstacle around the robot (S231); generating, by the mobile robot, an internal map using obstacle information (S232); The mobile robot may further include transmitting the internal map to the terminal device (S233). The remote control method may further include displaying, by the terminal device, the internal map, and receiving, by the terminal device, the movement route on the internal map (S251).

The mobile robot may generate an internal map while patrolling or moving. The terminal device can display an internal map generation operation of the mobile robot on the screen. When the mobile robot detects an image and transmits image information, and generates and transmits an internal map, the display unit may display image information in a first area and an internal map in a second area. If the display unit drags the first area to the second area or drags the second area to the first area, the display may display the image information and the internal map generation operation interchangeably.

As shown in Figs. 14A and 14B, when a user or the like touches the patrol start icon to start the patrol start command (or inputs a moving command) (S252), the terminal apparatus, as shown in Fig. 14C, The path setting screen can be displayed on the display unit. As shown in FIGS. 14D to 14F, when a user or the like sequentially touches certain points on the internal map, the controller connects the points to set a moving path (patrol path) (S251). As illustrated in FIGS. 15A to 15C, the user or the like may set an internal map to be displayed on the first area, and then set a movement route or a patrol route.

16A to 16C, the display unit may automatically display the patrol start button and the move start button alternately according to the route setting on the internal map. When the user or the like connects the route to the current position as shown in Fig. 16B, the control unit sets the patrol route, and the display unit displays the patrol start button on the screen. On the other hand, when the user or the like sets the path so as not to be repeated, as shown in Fig. 16C, the control unit sets the moving path, and the display unit displays the moving start button on the screen.

As shown in Figs. 16D to 16I, the mobile robot can generate an internal map while performing a movement or patrol. The terminal device can display an internal map generation operation of the mobile robot on the screen. As shown in FIG. 16D, when a user or the like issues a patrol start command to the terminal device, the display unit displays a message screen to set a moving point (patrol point) on an internal map. In this case, when the internal map is not generated, the internal map is generated and transmitted to the terminal device while moving. As shown in Figs. 16F to 16I, the display unit displays an internal map generated by the mobile robot. When a user or the like sequentially touches certain points on the internal map, the controller connects the points to set a movement route or a patrol route.

Referring to FIG. 17A, the terminal device receives a control command for opening and closing the power of the lighting 113 provided in the mobile robot, and the controller 220 generates a control signal. As shown in Fig. 17A, an icon for a light control command may be set to be provided in an option menu. As shown in FIG. 17B or 17D, when a user or the like touches a light icon, the light icon is toggled, and the display 230 displays a message such as 'light is turned on' or 'light is turned off'. Can be marked on. In addition, as shown in FIG. 17C or FIG. 17E, the display unit may display video information after opening and closing the illumination.

Referring to FIGS. 18A to 18D, the mobile robot may detect the state of charge of the battery during patrol (S310) and move to the charging station according to the detection result (S330) to perform charging (S340). At this time, as shown in Figure 18A, the display unit 230 'needs charging. Will stop the current operation and move to the charging station. ' In addition, the wireless communication unit 210 receives status information from the mobile robot at any time (S320), and as shown in Figs. 18B to 18D, the display unit is moving to charge, 'battery power is insufficient', ' It is being charged, such as a message may be displayed (S350).

As illustrated in FIG. 19, when an error such as a wheel jam occurs during movement or patrol of a mobile robot, the terminal device receives an error message, generates a control screen, and displays the same. If the user or the like confirms this and then touches (clicks) the confirmation button, the display unit may redisplay the previously displayed screen.

As described above, embodiments of the present invention enable the mobile robot to be controlled in various ways by connecting the terminal device and the mobile robot through a network. In addition, embodiments of the present invention, it is possible to check the status of the house in real time from the outside, and to be able to quickly respond to the state of the mobile robot. The mobile robot may patrol a certain area or move to a point desired by the user, and may provide an image of a point desired by the movement path or the user through the terminal device. In addition, the mobile robot may be provided with illumination to provide a high quality image robustly to the surrounding environment.

100: mobile robot 200: terminal device
110: image detection unit 120: drive unit
130: control unit 140: communication unit
151: Obstacle detection unit 152: Position recognition unit
160: input unit 170: output unit
180: storage unit 210: wireless communication unit
220: control unit 230: display unit
240: Memory

Claims (17)

An image detecting unit for photographing the periphery to generate image information;
A driving unit having at least one wheel and driving the wheel in accordance with a driving signal; And
And a control unit generating the driving signal based on a movement command to move along the movement path, and controlling the image detection unit to generate the image information. The method according to claim 1,
A communication unit for receiving a control signal from an external terminal device and transmitting the video information to the terminal device;
Wherein the control unit comprises:
And extracting the movement command from the control signal and performing the movement command. The method of claim 2,
And an obstacle detection unit having at least one sensor and detecting an obstacle around the obstacle using the sensing signal of the sensor and outputting obstacle information,
Wherein the control unit comprises:
A mobile robot, characterized in that for generating an internal map using the obstacle information. The method of claim 3,
And a location recognizing unit having one or more sensors, the location recognizing unit outputting location information by recognizing the location of the mobile robot itself using the detection signal of the sensor.
Wherein the control unit comprises:
And the internal map is modified using the recognized location information. 5. The method of claim 4,
The image detection unit,
A camera for photographing a subject; And a lens for focusing the subject.
And the control command is a camera control command for adjusting the camera or a lens control command for adjusting the lens. 6. The method of claim 5,
The image detection unit,
It further comprises a lighting for adjusting the brightness of the surroundings,
And the control command is a command for opening and closing the power of the lighting. 7. The method according to any one of claims 1 to 6,
The movement path is a patrol path connected to the current position of the mobile robot,
And the movement command comprises a patrol command for moving the patrol path. A mobile robot extracting a moving command from a control signal to perform the moving command, and generating image information by photographing a surrounding while moving along a moving path based on the moving command; And
And a terminal device which controls the mobile robot using the control signal and receives and provides the image information from the mobile robot. The method of claim 8,
The terminal apparatus comprises:
A wireless communication unit which transmits the control signal to the mobile robot and receives one or more data including the image information and an internal map from the mobile robot;
A controller configured to generate the control signal corresponding to the movement command and to generate a control screen using the data; And
And a display unit configured to receive the control command, and to display the control screen. 10. The method of claim 9,
The touch recognition area may further include a first area having a predetermined size on which the control screen is displayed and receiving the movement command, and a second area having a size smaller than or equal to the first area. Remote control system. 10. The method of claim 9,
The control unit,
When the touch input for one or more points on the internal map is received, the remote control system of the mobile robot, characterized in that for setting the movement path using the one or more points. 10. The method of claim 9,
The display unit includes:
The remote control system of the mobile robot further comprises a lighting icon for opening and closing the power of the light provided in the mobile robot. 13. The method according to any one of claims 8 to 12,
The movement path is a patrol path connected to the current position of the mobile robot,
And the movement command comprises a patrol command for moving the patrol path. In the remote control system of a mobile robot including a mobile robot and a terminal device for controlling the mobile robot,
Receiving, by the mobile robot, a control signal from the terminal device;
The mobile robot extracting a movement command from the control signal;
The mobile robot performing the movement command to move along a movement path;
Generating image information by photographing the surroundings by the mobile robot during the movement command;
Transmitting, by the mobile robot, the image information to the terminal device; And
And displaying, by the terminal device, the image information on a screen. 15. The method of claim 14,
Detecting, by the mobile robot, an obstacle around it;
Generating, by the mobile robot, an internal map using obstacle information; And
And transmitting, by the mobile robot, the internal map to the terminal device. The method of claim 15,
Displaying, by the terminal device, the internal map; And
And receiving, by the terminal device, the movement path on the internal map. 17. The method of claim 16,
The movement path is a patrol path connected to the current position of the mobile robot,
The movement command is a remote control method for a mobile robot, characterized in that the patrol command to move the patrol path.

Images