KR101356161B1 - Robot cleaner, and system and method for remotely controlling the same - Google Patents

Abstract

Embodiments of the present invention, by connecting the terminal device and the mobile robot through a network to enable the mobile robot to be controlled in more various ways. In addition, embodiments of the present invention, it is possible to check the status of the house in real time from the outside, and to be able to quickly respond to the state of the mobile robot. In addition, embodiments of the present invention can implement a voice call function, an interphone function, or a video call function using a terminal device and a mobile robot.

Description

ROBOT CLEANER, AND SYSTEM AND METHOD FOR REMOTELY CONTROLLING THE SAME}

The present invention relates to a remotely controllable robot cleaner, a terminal device for remotely controlling the robot cleaner, and a remote control system and method for a robot cleaner including the same.

In general, robots have been developed for industrial use and have been part of factory automation. In recent years, medical robots, aerospace robots, and the like have been developed, and household robots that can be used in ordinary homes are being developed.

A representative example of the domestic robot is a robot cleaner, which is a kind of electronic equipment that sucks dust and foreign matter around while driving a certain area by itself. Such a robot cleaner is generally equipped with a rechargeable battery and has an obstacle sensor capable of avoiding obstacles during traveling, so that it can run and clean by itself.

On the other hand, a method for controlling the mobile robot includes a method using a remote control which is a user interface, a method using a button provided in the mobile robot body. In addition, mobile robots having magnetic location recognition and mapping functions have been developed using cameras or various sensors.

SUMMARY Embodiments of the present invention provide a mobile robot that can be remotely controlled and recognizes voice using a terminal device.

Embodiments of the present invention have another object to provide a terminal device capable of remotely controlling a mobile robot and checking the state of the mobile robot in real time.

Embodiments of the present invention have another object to provide a remote control system and method that can perform a voice call or a video call using a mobile robot and a terminal device.

Embodiments of the present invention can remotely control a mobile robot, can remotely monitor the surroundings of the mobile robot, and remote control consisting of a mobile robot and a terminal device to respond quickly by checking the status of the mobile robot in real time Another object is to provide a system and method.

According to an embodiment of the present invention, a robot cleaner includes a video detecting unit configured to generate image information by photographing a surrounding in order to recognize a location or generate a map, wherein the robot cleaner extracts and recognizes a voice signal from surrounding sounds. A control unit receives a control signal from a recognition unit, a remote terminal apparatus, transmits a response signal to the control signal to the terminal apparatus, and extracts a control command from the control signal, and performs a voice signal from the control signal. And a control unit for extracting and recognizing an image, wherein the robot cleaner transmits the image information through the image detection unit and the voice signal through the voice recognition unit together to the terminal device through the communication unit. Outputting video information and an audio signal received from the terminal device; And a gong.

delete

delete

The remote control system of the robot cleaner according to an embodiment of the present disclosure may extract and recognize a voice signal from a control signal, and control the robot cleaner using a robot cleaner that remotely outputs the voice signal and a control command from the remote controller, And a terminal device for receiving an audio signal and generating the control signal and transmitting the control signal to the robot cleaner, wherein each of the robot cleaner and the terminal device simultaneously transmits image information of the surroundings and the received audio signal together. And outputting the received image information and an audio signal, wherein the robot cleaner acquires the image information using an image detection unit for recognizing a location or generating a map.

The terminal device according to an embodiment may include a wireless communication unit transmitting the control signal to the robot cleaner and receiving one or more data including a response signal from the robot cleaner, and using the voice signal or the control command. A control unit for generating a control signal and generating a control screen using the data, a touch recognition area for receiving the control command, a display unit for displaying the control screen, and a microphone for receiving the voice signal. Include.

The terminal device according to an exemplary embodiment may further include a camera for capturing the periphery of the terminal device to detect image information.

In a remote control system of a mobile robot according to an embodiment, the mobile robot and the terminal device may transmit and receive image information and an audio signal to each other.

In a remote control method of a robot cleaner according to an embodiment, in a remote control system of a robot cleaner including a robot cleaner and a terminal device for controlling the robot cleaner, the terminal device detects image information by photographing the surroundings and Receiving a voice signal, converting the voice signal into a control signal by the terminal apparatus, and transmitting the voice signal and the image information to the remote robot cleaner together; And extracting the voice signal from the robot cleaner and outputting the image information and the voice signal to the surroundings, wherein the robot cleaner receives sound from the surroundings, and the robot cleaner recognizes a position. Image information using an image detection unit that captures the surroundings The method may further include detecting the received sound, and transmitting the sound received by the robot cleaner and the captured image information to the terminal device.

The mobile robot according to the embodiments of the present invention can be easily controlled by using a terminal device such as a mobile terminal from the outside by embedding a communication means for remote control.

The mobile robot according to the embodiments of the present invention may execute a control command by performing voice recognition and perform a voice call through the terminal device.

The mobile robot according to the embodiments of the present invention performs a charging operation by frequently checking the state of the battery while cleaning or moving. In addition, the mobile robot transmits state information to the terminal device in real time so that a user or the like can easily check the state of the mobile robot using the terminal device.

The terminal device according to the embodiments of the present invention executes a remote control program to easily control a mobile robot from a remote location, and allows a user or the like to check the status or surrounding conditions of the mobile robot in real time.

The remote control system and method according to the embodiments of the present invention may perform a voice call function or an interphone function by connecting a terminal device and a mobile robot through a network.

The remote control system and method according to the embodiments of the present invention enable the mobile robot to be controlled in more various ways by connecting the terminal device and the mobile robot through a network. In addition, the remote control system and method, it is possible to check the situation of the house in real time from the outside, and to be able to respond quickly according to the state of the mobile robot.

Embodiments of the present invention may improve the convenience of the user and increase the operational efficiency and stability of the mobile robot. In addition, embodiments of the present invention, when connecting to the mobile robot through the terminal device, it is possible to prevent the abuse of the mobile robot by performing the connection authentication operation.

1 is a perspective view showing the appearance of a mobile robot according to embodiments of the present invention;
2 and 3 are block diagrams showing the detailed configuration of a mobile robot according to embodiments of the present invention;
4 is a schematic view of a remote control system of a mobile robot according to one embodiment;
5 is a block diagram illustrating a configuration of a terminal device according to an embodiment;
6A to 6F illustrate screens of a terminal device for explaining an operation of performing a voice call using the terminal device and a mobile robot;
7A-7E illustrate another example of FIG. 6;
8 is a flowchart schematically illustrating a method of remotely controlling a mobile robot according to an exemplary embodiment.

Referring to FIG. 4, a remote control system of a mobile robot according to an exemplary embodiment may include a mobile robot 100 that extracts and recognizes a voice signal from a control signal, and outputs the voice signal to the surroundings, using a control command. And a terminal device 200 that controls the mobile robot, receives the voice signal, generates the control signal, and transmits the control signal to the mobile robot.

The terminal device is divided into a mobile terminal and a stationary terminal according to whether the mobile terminal is mobile or not, and includes both a mobile terminal and a fixed terminal. The terminal device is divided into a handheld terminal and a vehicle mount terminal according to whether the user can move directly or not, and includes both a mobile terminal and a stationary terminal. For example, the terminal device may be a cellular phone, a PCS phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player) Navigation). The remote control system can be changed according to the type of the terminal device. For example, the remote control system may use a mobile communication network such as 3G, CDMA, WCDMA, etc. in the case of a mobile phone or a smart phone, and the mobile robot and the terminal device transmit a radio signal to a base station, an external terminal, a server, and the like on the mobile communication network. Send and receive

1 and 2, the mobile robot 100 according to an embodiment includes a communication unit 110 and a control unit 130. The communication unit 110 receives a control signal from an external terminal device 200 and transmits a response signal to the control signal to the terminal device. The control unit 130 extracts a control command from the control signal and performs the extracted control command. In addition, the control unit 130 extracts and recognizes a voice signal from the control signal.

The communication unit 110 is connected to the terminal device via one of the wired, wireless, and satellite communication methods, that is, one of the currently available communication methods, and transmits and receives signals and data. The communication unit 110 receives a control signal from the terminal device. The communication unit 110 transmits state information, obstacle information, location information, image information, an internal map (cleaning map), and the like of the mobile robot. The communication unit 110 may be connected to the terminal device in one of the short range wireless communication methods such as radio frequency (RF) communication, Bluetooth, infrared communication (IrDA), wireless LAN (LAN), ZigBee, Communication can be performed. For example, when the terminal device is a smartphone, the mobile robot may include a communication unit according to the communication scheme available for the smartphone.

Referring to FIG. 2, the mobile robot according to an embodiment may further include a voice output unit 151 provided at one side of the mobile robot and outputting a voice signal recognized by the control unit to the periphery. The voice output unit 151 is a means for outputting sound and voice, and includes a beeper and a speaker. When a user of the terminal device or the like inputs a voice through a microphone provided in the terminal device, the terminal device converts the received voice into a control signal. The terminal device transmits a control signal to the mobile robot, and the control unit 130 extracts and recognizes a voice signal from the control signal. The voice output unit 151 provides the voice signal extracted by the control unit 130 to a user or the like around the mobile robot. At this time, the mobile robot performs an interphone function. The interphone function may include a video call function described later.

According to another exemplary embodiment, the mobile robot includes a communication unit 110 and a control unit 130, and the control unit 130 extracts and recognizes a voice signal from the control signal. The control command is extracted from the voice signal. In this case, the mobile robot may further include a storage unit 160 that stores the voice signal and the control command corresponding to the voice signal. The storage unit 160 stores the voice signal in advance, and stores in advance a patterned, databased control command for the voice signal. Here, the control command includes a cleaning command such as cleaning start, cleaning start, patrol command, moving command, charging command, setting change, and the like.

The mobile robot according to the embodiments further includes an input unit 140 that directly receives a control command. Further, the user or the like may input a command through the input unit to output one or more pieces of information stored in the storage unit. The user or the like may store a voice signal and a control command thereof in advance in the storage unit through the input unit 140. The input unit 140 may be formed of one or more buttons. For example, the input unit 140 may include a confirmation button and a setting button. The confirmation button inputs a command for confirming obstacle information, location information, image information, a cleaning area or a cleaning map, a cleaning path, a moving path, a patrol path, and the like. The setting button inputs a command for setting the above information. The input unit may include a reset button, a delete button, a cleaning start button, a stop button, and the like for inputting a command for resetting the information. As another example, the input unit 140 may include a button for setting or deleting reservation information. In addition, the input unit 140 may further include a button for setting or changing a cleaning mode. In addition, the input unit 140 may further include a button for receiving a command to return to the charging station. As shown in FIG. 1, the input unit 140 may be installed on the upper portion of the mobile robot by a hard key, a soft key, a touch pad, or the like. In addition, the input unit 140 may have a form of a touch screen together with the output unit.

The output unit 150 is provided in the upper part of the mobile robot, as shown in FIG. Of course, the installation location and installation type may vary. For example, the output unit 150 displays reservation information, battery status, intensive cleaning, space expansion, cleaning method such as zigzag driving or driving method on the screen. The output unit 150 may output a current state and a current cleaning state of each unit constituting the mobile robot. In addition, the output unit 150 may display obstacle information, location information, image information, an internal map, a cleaning area, a cleaning map, a cleaning path, a moving path, a patrol path, and the like on the screen. The output unit 150 may be any one of a light emitting diode (LED), a liquid crystal display (LCD), a plasma display panel, and an organic light emitting diode (OLED). It can be formed as an element of. The output unit 150 may further display the remaining battery level on the screen. In addition, the terminal device 200 may receive the state of charge of the battery and the remaining amount of battery from the mobile robot 100 and display the state of charge of the battery and the remaining amount of battery on one side of the screen of the display unit.

The storage unit 160 further stores one or more information among the image information, the obstacle information, the location information, the cleaning map, the cleaning area, the cleaning path, the moving path, and the patrol path, in addition to the control signal for the voice signal and the voice signal. The storage unit 160 stores a control program for controlling (driving) the mobile robot and data accordingly. In addition, the storage unit 160 may further store the cleaning method, the traveling method, and the positions of the charging stations. The storage unit 160 mainly uses nonvolatile memory. Here, a non-volatile memory (NVM, NVRAM) is a storage device that keeps stored information even when power is not supplied. Non-volatile memory includes ROM, flash memory, magnetic computer storage devices (e.g., hard disk, diskette drive, magnetic tape), optical disk drive, magnetic RAM, PRAM, and the like.

Referring to FIG. 3, the mobile robot is configured to face upward or forward, and further includes an image detection unit 171 that includes an upper camera sensor and detects image information by capturing the periphery of the mobile robot. . When the image detection unit 171 includes a plurality of upper camera sensors, the camera sensors may be formed on the top or side surfaces of the mobile robot at a predetermined distance or at an angle. The image detection unit 171 may be used as another type of location recognition unit. Referring to FIG. 3, the image detection unit 171 may further include a camera photographing a subject, a lens connected to the camera to focus the subject, an adjusting unit adjusting the camera, and a lens adjusting unit adjusting the lens. can do. The lens uses a lens having a wide angle of view so that all the surrounding areas, for example, all areas of the ceiling can be photographed even at a predetermined position. The control unit 130 may extract a feature point from the image information captured by the image detection unit, recognize the position of the mobile robot using the feature point, and generate a cleaning map for the cleaning area.

Referring to FIG. 3, the mobile robot further includes an obstacle detection unit 172 that includes one or more sensors and detects obstacles around the vehicle using the detection signal of the sensor and outputs obstacle information. Here, the control unit 130 generates a cleaning map using the obstacle information.

The obstacle detecting unit 172 includes a first sensor installed in front of the mobile robot, that is, at a predetermined interval on the outer circumferential surface as shown in FIG. 1. In addition, the obstacle detecting unit 172 may include a second sensor installed to have a surface protruding outward of the main body. The position and type of the first sensor and the second sensor may vary according to the type of the mobile robot, and the obstacle detecting unit may include more various sensors. The first sensor detects an object in the moving direction of the mobile robot, in particular an obstacle, and transmits detection information to the control unit 130. That is, the first sensor detects protrusions, household appliances, furniture, walls, wall edges, etc. existing on the movement path of the mobile robot and transmits the information to the control unit. The first sensor may be an infrared sensor, an ultrasonic sensor, a RF sensor, a geomagnetic sensor, or the like. The second sensor senses an obstacle present on the front or side and transmits the obstacle information to the control unit. That is, the second sensor detects protrusions, household appliances, furniture, walls, wall edges, etc. existing on the movement path of the mobile robot and transmits the information to the control unit. The second sensor may be an infrared sensor, an ultrasonic sensor, an RF sensor, a position sensitive device (PSD) sensor, or the like.

The obstacle detection unit 172 may be installed on the bottom surface (bottom surface) of the main body, and may further include a cliff sensor for detecting an obstacle on the bottom surface, for example, a cliff. The cliff sensor is configured to obtain a stable measurement value regardless of the reflectance and color difference of the bottom surface, and may be in the form of an infrared module such as a PSD sensor.

In addition, the obstacle detecting unit may further include a charging signal sensor for receiving a guide signal from the charging station. The mobile robot checks the position and direction of the charging station by receiving a guide signal generated by the charging station using the charging signal sensor. The charging station sends a guide signal indicating the direction and distance so that the mobile robot can return. The mobile robot receives the signal from the charging station to determine the current position, sets the direction of movement, and returns to the charging station. The charging signal sensor may be an infrared ray sensor, an ultrasonic sensor (Ultra Sonic Sensor), an RF sensor (Radio Frequency Sensor), or the like. In general, an infrared sensor is used. The charging signal sensor is provided at one side of the inside or outside of the mobile robot. For example, as shown in FIG. 4, the charging signal sensor may be installed below the output unit 150 or around the image detection unit 171. .

Referring to FIG. 3, the mobile robot further includes a location recognizing unit 173 including one or more sensors and recognizing the location of the mobile robot itself using the detection signal of the sensor and outputting location information. . Here, the control unit 130 corrects the cleaning map using the positional information recognized by the position recognition unit 173.

The position recognizing unit 173 is provided on the rear surface of the mobile robot and includes a lower camera sensor which photographs the lower side, that is, the bottom surface and the surface to be cleaned, during the movement. The lower camera sensor is an optical flow sensor, and converts a downward image input from an image sensor provided in the sensor to generate image data of a predetermined format. The lower camera sensor may detect the position of the mobile robot regardless of the sliding of the mobile robot. The control unit 130 compares the image data photographed by the lower camera sensor with time to calculate a moving distance and a moving direction, and thus calculates a position of the mobile robot. By observing the lower side of the mobile robot by using the lower camera sensor, the control unit can correct the sliding against the position calculated by other means.

The position recognizing unit 173 further includes an acceleration sensor that detects a change in a moving speed due to a speed change of the mobile robot, for example, starting, stopping, turning, collision with an object, and the like. The acceleration sensor is attached to the main wheel or the adjoining positions of the auxiliary wheels, so that the slip or idling of the wheel can be detected. In this case, the speed may be calculated using the acceleration detected by the acceleration sensor, and the position of the mobile robot may be checked or corrected by comparing with the command speed. However, in general, the acceleration sensor is embedded in the control unit 130 to detect a change in speed of the mobile robot itself generated when cleaning and moving. That is, the acceleration sensor detects the amount of impact according to the speed change and outputs a corresponding voltage value. Thus, the acceleration sensor can perform the function of an electronic bumper.

The position recognition unit 173 may further include a gyro sensor that detects a rotation direction and detects a rotation angle when the mobile robot moves, patrols or cleans. The gyro sensor detects the angular velocity of the mobile robot and outputs a voltage value proportional to the angular velocity. The control unit 130 calculates the rotation direction and the rotation angle using the voltage value output from the gyro sensor.

The position recognizing unit 173 may further include a wheel sensor connected to the left and right main wheels and detecting a rotation speed of the main wheel. The wheel sensor mainly uses a rotary encoder, and detects and outputs the number of revolutions of the main wheels on the left and right sides when the moving robot is cleaned or moved. The control unit 130 can calculate the rotational speeds of the left and right wheels using the number of revolutions.

The control unit 130 can accurately recognize the position using the detection information of the acceleration sensor, the gyro sensor, the wheel sensor, the lower camera sensor, and the image information of the image detection unit. Further, the control unit 130 can precisely generate the cleaning map using the obstacle information detected by the obstacle detecting unit and the position recognized by the image detecting unit. The communication unit 110 transmits data including video information, obstacle information, location information, cleaning map, and cleaning area to the terminal device 200.

The mobile robot has left and right main wheels which are movable on both lower sides thereof. On either side of the main wheel, a handle may be provided to facilitate user's grip. Referring to FIG. 3, the driving unit 180 is connected to the left and right main wheels and includes a predetermined wheel motor for rotating the wheels to move the mobile robot by driving the wheel motor. The wheel motors are respectively connected to the main wheels so that the main wheels rotate, and the wheel motors operate independently of each other and can rotate in both directions. In addition, the mobile robot is provided with one or more auxiliary wheels on the rear side to support the main body, to minimize friction between the lower surface and the bottom surface (cleaned surface) of the main body and to facilitate the movement of the mobile robot.

Referring to FIG. 3, the mobile robot further includes a power supply unit 190. The power supply unit 190 includes a rechargeable battery to supply power to the mobile robot. The power supply unit supplies driving power to each unit, and operating power for moving, patrolling, or cleaning the mobile robot. If the remaining power is insufficient, the power supply unit is charged to receive a charging current. The mobile robot further includes a battery detection unit that detects a charging state of the battery and transmits a detection result to the control unit 130. The battery is connected to the battery detection unit, and the remaining battery level and charge state are transmitted to the control unit 130. [ The battery remaining amount can be displayed on the screen of the output unit. The battery may be located at the lower center of the mobile robot, or may be located at either the left or the right side so that the dust container is located at the bottom of the main body. In the latter case, the mobile robot may further comprise a counterweight to eliminate the weight bias of the battery.

The control unit 130 previously sets a reference value (remaining battery level), and compares the remaining battery level with a reference value. As a result of the comparison, if the detection result is equal to or less than the reference value, the control unit 130 moves the mobile robot to the charging station to perform charging. In one example, the control unit 130 may stop the other operation according to the charging command from the terminal device and move the charging operation to the charging station to perform charging. As another example, the control unit 130 may extract a charge command, perform a charge command according to a result of comparison between the remaining battery level and the reference value, or may perform the previous operation as it is.

When the mobile robot is a robot cleaner, the mobile robot may further include a cleaning unit (not shown). The cleaning unit includes a dust container in which dust collected is stored, a suction fan providing power to suck dust in the cleaning area, and a suction motor rotating the suction fan to suck air, thereby removing surrounding dust or foreign matter. Inhale.

2 or 3, the mobile robot according to the embodiments further includes a voice recognition unit 120 that extracts and recognizes a voice signal from surrounding sounds. Here, the control unit 130 converts the voice signal recognized by the voice recognition unit 120 into a response signal. The communication unit 110 transmits a response signal to the terminal device, and the terminal device outputs a voice signal through the built-in speaker. At this time, the mobile robot performs a telephone function.

Referring to FIG. 3, the voice recognition unit 120 extracts a voice signal from a signal input unit 121 for receiving sound, a noise filter 122 for removing noise from the sound, and a sound from which the noise is removed. It is configured to include a voice signal extraction unit 123. The signal input unit 121 is a device that receives external sound such as a microphone. The noise filter 122 extracts a signal of a voice signal band from external sound and removes noise. For example, the speech signal extractor 123 may extract the speech signal by sampling the speech signal from which the noise is removed by the noise filter at a predetermined cycle.

The communication unit 110 transmits the video information detected by the video detection unit 171 and the audio signal recognized by the audio recognition unit 120 to the terminal device together with the video information and the control signal ( Voice signal). In this way, a video call function can be implemented between the mobile robot 100 and the terminal device 200. That is, the mobile robot is equipped with an image detecting unit and a voice recognition unit to compress the input image information and the audio signal and transmit the compressed image information and the audio signal to a user of a terminal device remotely connected. At the same time, the audio signal and the image information transmitted from the remote terminal device can be decoded and output to the user around the mobile robot. For example, embodiments of the present invention, by directly transmitting and receiving data between the mobile robot and the terminal device through the hole punching (Hole Punching) or server, etc. to secure a data transmission environment and to perform voice, video compression, and two-way streaming Real-time video call can be enabled.

Referring to FIG. 5, a terminal device according to an embodiment includes a wireless communication unit 210, a control unit 220, a display unit 230, and a microphone 240. The microphone 240 may be included in the A / V input unit.

The wireless communication unit 210 transmits a control signal generated by the control unit 220 to the mobile robot 100, and receives one or more data including a response signal from the mobile robot 100. Here, the one or more data further includes image information, obstacle information, location information, an internal map (or a cleaning map), a cleaning area, state information, and the like. The wireless communication unit 210 includes one or more modules that enable wireless communication between the terminal device 200 and the wireless communication system, between a plurality of terminal devices, or between a network located between the terminal device and the mobile robot 100. can do. For example, the wireless communication unit 210 may include a broadcast receiving module, a mobile communication module, a wireless Internet module, a short distance communication module, and a location information module.

The broadcast receiving module receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. Mobile communication module Transmits and receives radio signals to and from a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception. The wireless Internet module refers to a module for wireless Internet access, and may be built in or enclosed in a terminal device. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies. The short-range communication module literally means a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

In the display 230, a touch recognition area 231 for receiving a control command is formed and displays a control screen generated by the controller. In addition, the display 230 may display an icon according to a communication method (eg, Wi-Fi, 3G), display a communication sensitivity, and display a battery remaining amount of the mobile robot. In addition, as shown in FIG. 7A or less, the display unit 230 includes a first area S232 having a predetermined size and a size smaller than or equal to the first area in which the control screen is displayed and receives the control command. The touch recognition area may further include a second area S233 of FIG. Of course, in the case of a mobile phone or a notebook computer having no touch screen (touch pad), the touch recognition area is not formed, and an input part for receiving a control command instead of touch recognition and an output part for displaying a control screen can be distinguished.

The display unit 230 alternately displays a moving start icon for receiving a moving start command or a patrol start command, a patrol starting icon, a moving stop icon for receiving a moving stop command or a patrol stop command, and a patrol stopping icon in the touch recognition area. can do. The display unit 230 displays information processed by the terminal device. That is, the display unit 230 displays a control screen. For example, when the terminal device is in the call mode, a UI (User Interface) or GUI (Graphic User Interface) associated with the call is displayed. The display unit 230 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display).

In the case where the display unit 230 and the touch sensor that detects the touch operation form a mutual layer structure, the display unit 230 may be a touch screen that can be input in addition to the output. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like. The touch sensor may be configured to convert a change in a pressure applied to a specific part of the display part 230 or a capacitance generated in a specific part of the display part 230 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch. The touch sensor may be a proximity sensor capable of performing proximity-touch in which a pointer is not actually touched on the screen but approaches a predetermined distance from the screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. In the present invention, the touch recognition is a proximity touch that allows the user to recognize that the pointer is positioned on the touch screen without being touched on the touch screen, and the pointer is actually touched on the touch screen. It includes both contact touch.

The terminal device may further include a memory (not shown). The memory may store a program for the operation of the controller 220. In addition, the memory may store input / output data (for example, a phone book, a message, a still image, a video, etc.). The memory may store in advance a control signal for controlling the mobile robot and a control command corresponding thereto in advance.

As another example, the memory may further store a voice signal and a control command corresponding thereto in advance. At this time, the controller 220 generates a control signal using a voice signal corresponding to the control command. The mobile robot extracts and executes a control command from the voice signal received from the terminal device.

The terminal apparatus may further include an A / V (Audio / Video) input unit, a user input unit, a sensing unit, an interface unit, and a power supply unit.

The A / V (Audio / Video) input unit is for inputting an audio signal or a video signal, and may include a camera and a microphone. The user input unit generates input data for controlling the operation of the terminal device by the user. The user input unit may include a key pad, a dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like. In particular, when the touch pad has a mutual layer structure with the display unit 230, it is referred to as a touch screen. The sensing unit senses the state of the terminal device such as the open / close state of the terminal device, the position of the terminal device, the presence of the user, the orientation of the terminal device, and the acceleration / deceleration of the terminal device, and generates a sensing signal for controlling the operation.

The interface unit serves as a pathway to all the external devices connected to the terminal device 200. The interface unit receives data from an external device, receives power from the external device, transfers the data to each component in the terminal device, or allows data in the terminal device to be transmitted to the external device. The power supply unit receives external power and internal power under the control of the controller 220 and supplies power necessary for operation of the respective components.

The control unit 220 generally controls the overall operation of the terminal device. For example, in the case of a mobile phone or smart phone, it performs related control and processing for voice call, data communication, video call, and the like. In addition, the control unit 220 can perform pattern recognition processing that can recognize the handwriting input or drawing input performed on the display unit 230 as characters and images, respectively.

The controller 220 generates a control signal corresponding to a control command for the mobile robot 100, and generates a control screen using data and a response signal. Here, the control command includes a cleaning command, a moving command, a patrol command, a charging command, a setting change, and the like. In addition, the controller 220 may generate a control signal by converting the voice signal into a control signal. The mobile robot performs the interphone function by outputting the received voice signal to a nearby user or the like.

The terminal device may further include a camera which photographs the surroundings of the terminal device to detect image information. The mobile robot and the terminal device may implement a video call function by transmitting and receiving image information and audio signals. That is, the mobile robot is equipped with an image detecting unit and a voice recognition unit to compress the input image information and the audio signal and transmit the compressed image information and the audio signal to a user of a terminal device remotely connected. At the same time, the audio signal and the image information transmitted from the remote terminal device can be decoded and output to the user around the mobile robot. For example, embodiments of the present invention, by directly transmitting and receiving data between the mobile robot and the terminal device through the hole punching (Hole Punching) or server, etc. to secure a data transmission environment and to perform voice, video compression, and two-way streaming Real-time video call can be enabled.

As another example, the controller 220 may extract a control command from the voice signal. When the voice signal corresponds to a control command for controlling the mobile robot, the controller 220 generates a control signal corresponding to the control command. The wireless communication unit 210 may transmit a control signal generated by the controller to the mobile robot, and the mobile robot may extract the control command from the control signal and perform the control command.

Referring to FIG. 8, in a method of remotely controlling a mobile robot according to an embodiment, the terminal device receives a voice signal (S110), and the terminal device converts the voice signal into a control signal to the mobile robot. Transmitting (S120, S200), the mobile robot extracting the voice signal from the control signal (S300), and the mobile robot outputting the voice signal to the surroundings (S400). do. Hereinafter, a remote control system of a mobile robot including a mobile robot and a terminal device for controlling the mobile robot will be described with reference to FIGS. 1 to 5.

The terminal device 200 receives a voice (S110), and generates a voice signal according to it (S120). The terminal device converts the voice signal into a control signal and transmits it to the mobile robot (S200). The mobile robot extracts and recognizes a voice signal from the received control signal (S300), and outputs the recognized voice signal to the surroundings using a speaker or the like (S400). At this time, the mobile robot can continuously transmit data such as image information, internal maps, obstacle information, etc. to the terminal device (S500), and the terminal device generates a control screen using the data and displays the generated control screen. (S710, S720).

As another example, the terminal device 200 may receive a control command for controlling the mobile robot through voice. The terminal device receives a control command and generates a control signal accordingly. Here, the control command includes a move start command, a patrol start command, a move stop command, a patrol stop command, a cleaning command, a charge command, a setting change, and the like. When a user or the like inputs a control command by voice using a microphone, the terminal device generates a control signal in accordance with a communication protocol, and transmits the control signal to the mobile robot through a network. The mobile robot extracts a control command carried in the control signal and performs the control command. At this time, the mobile robot can continuously transmit data such as image information, internal maps, obstacle information, etc. to the terminal device (S500), and the terminal device generates a control screen using the data and displays the generated control screen. (S710, S720).

Referring to FIG. 8, the mobile robot may continuously detect the charging state of the battery during cleaning, moving, or patrolling (S810), and may transmit a detection result to the terminal device (S900). In this case, even when there is no charging command of the terminal device, the mobile robot may move to the charging station and perform charging based on the detection result (S820).

Referring to FIG. 8 again, the method for remotely controlling a mobile robot may include: requesting, by the terminal apparatus, a connection to the mobile robot (S10), authenticating, by the mobile robot (S20), and transmitting the mobile robot. The method may further include acknowledging and confirming the access of the terminal device (S30), and displaying, by the terminal device, an initial control screen (S40).

First, when a terminal device executes a remote control program for remotely controlling a mobile robot, a login window for inputting login information for the mobile robot is generated. The remote control program may be stored in the memory of the terminal device or downloaded from a server storing the remote control program. For example, in the case of a smartphone, a remote control program is downloaded in the form of an app (App.), Displayed on the screen of the display unit in the form of an icon, and then executed when the user touches the icon. When the user inputs the login information, the terminal device requests a connection to the corresponding mobile robot (S10). The mobile robot performs authentication on the login information, and the terminal device may display a message on the screen during the connection.

The terminal device may be connected to one of the plurality of mobile robots. The display unit displays a plurality of connectable mobile robots in the form of a list or an icon. When the user or the like selects one of the plurality of mobile robots, the mobile robot performs authentication on the terminal device.

The mobile robot transmits an access confirmation signal to the terminal device after authentication (S30), and the terminal device displays an initial control screen (S40). The initial control screen may be displayed including the option menu, or only data such as image information and cleaning map may be displayed. In the latter case, when the touch input to the first area is received, the display unit can display the option menu. The control unit may display the data after the display unit temporarily displays the option menu on the initial control screen.

If the login information does not match the stored authentication information, the mobile robot transmits a connection rejection signal to the terminal device. Then, the terminal device 'does not match the ID or password. Please check and try again. " If the mobile robot to be connected is connected to another terminal device or the mobile robot to be connected is being used by another user, the display unit may display an error message. The terminal device can display the guidance message according to the communication method. If the mobile robot selected by the user is not registered or the network is not connected, an error message may be displayed.

Hereinafter, an operation and a method of a remote control system of a mobile robot according to an exemplary embodiment will be described with reference to FIGS. 6A to 7E.

Referring to FIG. 6A, when the touch is input to the first area while the mobile robot is executing a control command or is waiting, the display unit displays an option menu. The configuration of the option menu may vary depending on the design, but may include menus such as recording, interphone, illumination, cleaning reservation, and setting, as shown in FIG. 6A, for example. The display unit may further display an icon that enables manual operation of the mobile robot. When the cleaning command is input through the manual operation, the terminal device generates a control signal and transmits it to the mobile robot. When the terminal device detects a touch input for the interphone icon from a user or the like as illustrated in FIG. 6A, the terminal device starts the interphone function. Referring to FIG. 6B, the display unit may display a message “Connecting the interphone” on the screen. As shown in FIG. 6C, when the user or the like touches the screen of the display unit during the interphone connection, an option menu is displayed again. When the user touches the interphone icon once again as shown in FIG. 6D, the terminal device releases the interphone connection. At this time. As illustrated in FIG. 6E, the display unit may display a message of 'released interphone connection' on the screen. If an error occurs in the network during the interphone connection process, as illustrated in FIG. 6F, the display may display a notification message such as “Please check the network connection status”.

7A to 7E illustrate examples of another screen configuration of FIG. 6. When the terminal device detects a touch input for the interphone icon from the user or the like as illustrated in FIG. 7A, the terminal device starts the interphone function. At this time, the terminal device may receive a command to open and close the speaker provided in the mobile robot through the interphone icon. As shown in FIG. 7B or 7C, the terminal device displays a microphone icon on the screen of the display unit, and when the user or the like touches the microphone icon (by pressing the microphone button), the terminal device performs the interphone function. In addition, as shown in FIG. 7D, the display unit may further display a previous button for switching to the previous control screen. When the user or the like presses the previous button, the display unit displays the previous control screen (Fig. 7E).

In another embodiment, the mobile robot receives a sound from the outside through a built-in voice recognition unit, extracts and recognizes a voice signal from the sound, and transmits the sound signal to the terminal device. In addition, the terminal device receives a voice of the terminal device user through a built-in microphone, generates a control signal from the terminal device and transmits the control signal to the mobile robot. In this way, the telephone function can be implemented. In this case, the mobile robot may include a voice output unit such as a speaker, and output the voice signal received from the terminal device to the periphery.

In another embodiment, the mobile robot may store a voice signal and a control command corresponding to the voice signal in advance, extract a control command corresponding to the voice signal received from the terminal device, and perform the control command. At this time, the voice signal transmitted by the terminal device to the mobile robot is a signal received by the terminal device is converted to a control signal and transmitted to the mobile robot. On the contrary, the terminal apparatus stores the voice signal and a control command corresponding to the voice signal in advance, and if the input voice corresponds to a control command for controlling the mobile robot, the terminal device converts the control signal into a control signal corresponding to the control command and transmits the same to the mobile robot. Can be. The mobile robot which has received the control signal may perform a control command by extracting the control command from the control signal.

As described above, embodiments of the present invention enable the mobile robot to be controlled in various ways by connecting the terminal device and the mobile robot through a network. In addition, embodiments of the present invention, it is possible to check the status of the house in real time from the outside, and to be able to quickly respond to the state of the mobile robot. In addition, embodiments of the present invention can implement a voice call function, an interphone function, or a video call function using a terminal device and a mobile robot.

100: mobile robot 200: terminal device
110: communication unit 120: speech recognition unit
130: control unit 140: input unit
150: output unit 160: storage unit
171: Image detection unit 172: Obstacle detection unit
173: position recognition unit 180: drive unit
190: power supply unit 210: wireless communication unit
220: control unit 230: display unit
240: microphone

Claims (20)

An image detection unit generating image information by photographing a surrounding to recognize a location or generate a map;
In the robot cleaner comprising:
A speech recognition unit for extracting and recognizing a speech signal from ambient sounds;
A communication unit which receives a control signal from a remote terminal device and transmits a response signal to the control signal to the terminal device; And
And a control unit extracting and performing a control command from the control signal, and extracting and recognizing a voice signal from the control signal.
The robot cleaner transfers the image information through the image detection unit and the voice signal through the voice recognition unit together to the terminal device through the communication unit, and simultaneously receives the image information and the audio signal received from the terminal device. Robot cleaner, characterized in that output. delete The method according to claim 1,
And the control unit extracts the control command from the voice signal. The method of claim 3,
And a storage unit which stores the voice signal and the control command corresponding to the voice signal. delete delete The method of claim 1, wherein the speech recognition unit,
A signal input unit receiving the sound;
A noise filter for removing noise from the sound; And
And a voice signal extractor configured to extract the voice signal from the sound from which the noise is removed. The method according to claim 1,
And the communication unit transmits the video information detected by the video detection unit and the audio signal recognized by the audio recognition unit to the terminal device. A robot cleaner which extracts and recognizes a voice signal from a control signal and outputs the voice signal to the surroundings; And
And a terminal device that remotely controls the robot cleaner using a control command, receives the voice signal, generates the control signal, and transmits the control signal to the robot cleaner.
Each of the robot cleaner and the terminal device simultaneously transmits the captured image information and the received audio signal to each other and outputs the received image information and the audio signal. The remote control system of the robot cleaner, characterized in that to obtain the image information using an image detection unit for recognizing or generating a map. 10. The method of claim 9,
The terminal apparatus comprises:
A wireless communication unit which transmits the control signal to the robot cleaner and receives one or more data including a response signal from the robot cleaner;
A controller configured to generate the control signal using the voice signal or the control command and to generate a control screen using the data;
A display unit in which a touch recognition area for receiving the control command is formed and displays the control screen; And
And a microphone for receiving the voice signal. The method of claim 10,
The control unit generates the control signal using the voice signal corresponding to the control command,
The robot cleaner is a remote control system of the robot cleaner, characterized in that for extracting and performing the control command from the voice signal. The method according to claim 10 or 11,
The touch recognition area may further include a first area having a predetermined size on which the control screen is displayed and receiving the control command, and a second area having a size smaller than or equal to the first area. Remote control system. 13. The method of claim 12,
The display unit includes:
And a microphone icon for opening and closing a voice output unit provided in the robot cleaner. delete delete In the remote control system of the robot cleaner comprising a robot cleaner and a terminal device for controlling the robot cleaner,
Photographing the surroundings by the terminal device to detect image information and receiving an audio signal;
Converting, by the terminal device, the voice signal into a control signal and transmitting the voice signal and the image information together to the remote robot cleaner;
Extracting the voice signal from the control signal by the robot cleaner; And
And outputting, by the robot cleaner, the image information and the audio signal to the surroundings.
The robot cleaner receiving sound from a surrounding;
Detecting, by the robot cleaner, image information using an image detection unit photographing a surrounding to recognize a location or generate a map;
Transmitting the sound received by the robot cleaner and the captured image information to the terminal device together; Further comprising the step of controlling the robot cleaner. 17. The method of claim 16,
Receiving, by the terminal device, the voice signal corresponding to a control command; And
And extracting, by the robot cleaner, the control command from the voice signal. delete delete delete

Images