KR102118055B1 - remote controller for a robot cleaner and a control method of the same - Google Patents

Abstract

The present invention relates to a remote control device for a robot cleaner and a control method therefor.
According to an embodiment of the present invention, in a control method of a remote control device having a camera and a display unit and remotely controlling a robot vacuum cleaner, generating image information about the robot cleaner and surrounding areas of the robot vacuum cleaner through the camera ( A) step; (B) detecting the robot cleaner based on the image information and generating location information of the robot cleaner; And it can provide a control method of a remote control device characterized in that it comprises the step (C) for displaying the augmented reality image on the preview image of the robot cleaner displayed on the display unit.

Description

Remote controller for a robot cleaner and a control method thereof

The present invention relates to a robot cleaner system and a control method thereof. The present invention relates to a remote control device for a robot cleaner and a control method therefor.

A vacuum cleaner is a device for cleaning floors or carpets in a room, and is provided inside the cleaner body to suck air containing foreign substances from the outside by driving an air suction device composed of a motor and a fan that generate air suction power. After this, it is a device that separates and collects foreign substances and discharges the purified air from which the foreign substances are separated to the outside of the cleaner.

The vacuum cleaner may include a manual vacuum cleaner that is directly operated by the user and a robot cleaner that performs cleaning on its own without user interaction.

Here, the robot vacuum cleaner sucks foreign substances such as dust from the floor while driving itself in the area to be cleaned. In addition, the robot cleaner may be controlled to automatically clean while driving the cleaning area using an obstacle sensor or other sensor provided therein, or manually to clean the robot cleaner while driving by using a remote controller that is wirelessly connected to the robot cleaner.

However, since the configuration for accurately measuring the relative position between the user and the robot cleaner is not provided in the robot cleaner, there is a problem in that various location based services based on the relative position between the user and the robot cleaner cannot be provided.

In addition, in the case of manual adjustment of the robot cleaner, the movement path of the robot cleaner must be changed in real time using the direction keys provided on the remote control, etc. Whenever the direction keys are input, a delay occurs in the operation of the robot cleaner, and the operability of the direction keys is inconvenient. Therefore, there is a problem that the robot cleaner cannot be precisely adjusted.

In addition, the wireless adjustment of the robot cleaner has a problem that the user is forced to perform while directly watching the robot cleaner. In other words, when cleaning a cleaning area in which a plurality of areas are partitioned from each other, the robot cleaner and the user are always located in the same area in order to wirelessly control the robot cleaner. That is, if the user is located in another area and cannot observe the robot cleaner, wireless control of the robot cleaner is effectively impossible.

On the other hand, when wirelessly controlling a conventional robot cleaner, there is a problem in that it cannot provide pleasure to a user. Therefore, it is common for the user to use only the basic automatic cleaning mode, and there is very little need to perform radio control.

The present invention basically aims to solve the problems of the above-described robot cleaner and the robot cleaner system including the same.

Through one embodiment of the present invention, it is intended to provide a robot cleaner system and a control method thereof that can improve product reliability and satisfaction by providing a user with a unique function of a robot cleaner called cleaning.

Through one embodiment of the present invention, to provide a robot cleaner system and a control method for accurately measuring the relative position between the robot cleaner and the user.

Through one embodiment of the present invention, once the wireless control is started, it is intended to provide a robot cleaner system and a control method of the robot cleaner that can effectively control the robot cleaner wirelessly without directly looking at the robot cleaner.

Through one embodiment of the present invention, to provide a robot cleaner system and a control method for effectively mapping a cleaning area recognized by a display of a remote control device and an actual cleaning area.

Through one embodiment of the present invention, to prevent the damage of the external design of the robot cleaner, and furthermore, to provide a robot cleaner system and a control method thereof, which allows a user to easily grasp whether or not wireless adjustment is performed through the robot cleaner do.

Through an embodiment of the present invention, to provide a location-based service based on the location information between the robot cleaner and the user, to provide a robot cleaner system and a control method for enhancing convenience.

Through an embodiment of the present invention, a robot cleaner system and a control method for controlling the robot cleaner precisely and intuitively through a touch input on an image of the robot cleaner and its surrounding area displayed on the display unit of the remote control device Want to provide

Through one embodiment of the present invention, to provide a robot cleaner system and a control method thereof that can more intuitively and easily grasp the state information of the robot cleaner through a remote control device.

Through one embodiment of the present invention, to increase the utilization of applications related to the robot cleaner and to provide a robot cleaner system and a control method that can provide more content.

Through one embodiment of the present invention, to provide a robot cleaner system and a control method for providing a user with a basic robot cleaner function as well as additional game elements through a remote control device and a robot cleaner.

In order to achieve the above object, according to an embodiment of the present invention, a robot cleaner that performs cleaning while automatically driving; Remotely controlling the robot cleaner, including a remote control device having a camera and a display, wherein the remote control device includes position information and posture information of the robot cleaner through image information of the robot cleaner generated through the camera. It is possible to provide a robot cleaner system for extracting and displaying the augmented reality image on the preview image of the robot cleaner displayed on the display so that the augmented reality image is superimposed.

The remote control device may generate mapping information between a real area and a virtual area displayed on the display through image information on the robot cleaner and surrounding areas of the robot cleaner. In addition, the augmented reality image may be superimposed on the preview image of the robot cleaner displayed on the display through the mapping information.

In order to extract location information and posture information of the robot cleaner or to generate the mapping information, it is preferable that the robot cleaner is provided with a recognition means.

Preferably, the remote control device compares actual shape information of the recognition means with relative shape information identified by the image information, and extracts location information and posture information of the robot cleaner or generates the mapping information.

The remote control device includes: a display unit that displays the image information; And it is preferable to include an input unit for inputting a control signal of the robot cleaner. together,

Preferably, the display unit functions as the input unit including a touch sensor that senses a touch input.

The recognition means may include a recognition mark provided on the outer peripheral surface of the robot cleaner body.

The recognition means may include an LED light that supplies light from the inside of the body of the robot cleaner to the outside of the body of the charging base.

The outer wall of the main body comprises a color layer and a reflective layer from the outside, and the LED light is provided below the reflective layer, so that the LED light is invisible to the outside of the main wall of the body while the LED light is inactive. It is preferred to be excluded.

The LED lighting is preferably activated through a control signal of the remote control device when the application for controlling the robot cleaner is executed in the remote control device.

It is preferable that the LED light is activated by an application for controlling the robot cleaner in the remote control device and input through a specific icon.

The augmented reality image may be displayed to overlap part or all of the preview image.

The augmented reality image may be provided to display status information of the robot cleaner. In addition, a plurality of the status information may be provided.

The status information includes a synchronization state of the robot cleaner and the remote control device, a waiting state for cleaning of the robot cleaner, a request state of charging of the robot cleaner, a cleaning progress state of the robot cleaner, and a state in which the robot cleaner approaches the user. It may include at least one of.

An augmented reality image distinguished from each other may be displayed for each state information. In addition, it is desirable to have a sound distinct from each other for each state information and motion of the augmented reality image.

The augmented reality image may be implemented differently through a plurality of characters. A menu for selecting any one of the plurality of characters may be displayed on the display.

It is preferable that an additional augmented reality image in the form of a character is displayed separately from the augmented reality image superimposed on the preview image of the robot cleaner.

Through an embodiment of the present invention, it is possible to provide a robot cleaner system and a control method thereof that can improve product reliability and satisfaction by providing a user with a unique function of a robot cleaner called cleaning.

Through one embodiment of the present invention, it is possible to provide a robot cleaner system capable of accurately measuring a relative position between a robot cleaner and a user and a control method thereof.

Through one embodiment of the present invention, it is possible to provide a robot vacuum cleaner system and a control method for controlling the robot vacuum cleaner wirelessly without a user directly looking at the robot vacuum cleaner.

Through one embodiment of the present invention, it is possible to provide a robot cleaner system and a control method for effectively mapping a cleaning area recognized by a display of a remote control device and an actual cleaning area.

Through an embodiment of the present invention, damage to the external design of the robot cleaner can be prevented, and furthermore, a robot cleaner system and a control method thereof are provided for the user to easily grasp whether or not to perform wireless adjustment through the robot cleaner. Can be.

Through one embodiment of the present invention, it is possible to provide a robot cleaner system and a control method for improving convenience by providing a location-based service based on location information between a robot cleaner and a user.

Through an embodiment of the present invention, a robot cleaner system and a control method for controlling the robot cleaner precisely and intuitively through a touch input on an image of the robot cleaner and its surrounding area displayed on the display unit of the remote control device Can provide

1 is a perspective view showing a robot cleaner according to an embodiment of the present invention.
2 is a perspective view showing the internal configuration of a robot cleaner according to an embodiment of the present invention.
3 is a bottom perspective view of a robot cleaner according to an embodiment of the present invention.
4 is a block diagram of a robot cleaner constituting a robot cleaner system according to an embodiment of the present invention.
5 is a block diagram of a remote apparatus device constituting a robot cleaner system according to an embodiment of the present invention.
6 is a perspective view of a remote control device constituting a system when cleaning a robot according to an embodiment of the present invention, as viewed from the front.
7 is a rear perspective view of a remote control device constituting a robot cleaner system according to an embodiment of the present invention.
8 is a view for explaining a method of generating image information of the robot cleaner 100 in the robot cleaner system according to an embodiment of the present invention,
9 is a view for explaining a method of generating location information of a robot cleaner in a robot cleaner system according to an embodiment of the present invention.
9A is a view showing the actual shape of the recognition mark provided on the robot cleaner.
9B is a view showing the relative shape of the recognition mark grasped by the imager of the image information.
9C is a diagram for explaining generation of position information of a robot cleaner using a remote control device.
9D is a diagram for detecting the posture of the robot cleaner.
10 to 13 are views for explaining a location-based service of a robot cleaner provided by the robot cleaner system according to an embodiment of the present invention.
10A is a view for explaining a method of setting a cleaning area of a robot cleaner through a touch input to a display unit.
10B is a view for explaining a state in which the robot cleaner performs cleaning in a preset cleaning area.
11 is a view for explaining a method of setting a movement path of a robot cleaner through a touch input to a display unit.
12 is a view for explaining a method of setting a cleaning prohibited area of a robot cleaner through a touch input to a display unit.
13 is a view for explaining a method of calling the robot cleaner to the user's location.
14 is a flowchart illustrating a basic configuration of a method for controlling a robot cleaner system according to an embodiment of the present invention.
15 is a flowchart illustrating a step of detecting a robot cleaner in a method of controlling a robot cleaner system according to an embodiment of the present invention.
16 is a flowchart illustrating a step of providing a location-based service in a method of controlling a robot cleaner system according to an embodiment of the present invention.
17 is a flowchart illustrating a step of detecting a robot cleaner in a control method of a robot cleaner system according to another embodiment of the present invention.
18 is a perspective view showing an example of a charging station that may be included in the robot cleaner system according to an embodiment of the present invention.
19 is a conceptual diagram showing a lighting image as an example of a recognition means of a robot cleaner according to an embodiment of the present invention.
20 is a view comparing before and after activation of the illumination image shown in FIG. 19.
21 is a view showing one form of the recognition means.
22 is an initial screen of executing an application on a remote control device.
23 is a screen showing an example of a screen of a remote control device according to an embodiment of the present invention.
24 to 28 are screens showing shapes that vary depending on the state of the robot cleaner on the screen shown in FIG. 23.

Hereinafter, preferred embodiments of the present invention capable of specifically realizing the above object will be described with reference to the accompanying drawings.

In this process, the size or shape of components shown in the drawings may be exaggerated for clarity and convenience of description. In addition, terms specifically defined in consideration of the configuration and operation of the present invention may vary depending on the intention or custom of the user or operator. Definitions of these terms should be made based on the contents throughout this specification.

The robot cleaner system according to an embodiment of the present invention may include a robot cleaner 100 that automatically performs cleaning while driving, and a remote control device 200 that remotely controls the robot cleaner 100. .

It is preferable that the remote control device 200 has a robot cleaner 100 and a camera 221' that generates image information about a peripheral area of the robot cleaner 100. The remote control device 200 may generate location information of the robot cleaner 100 based on the image information. That is, on the basis of the image information, an area appearing on the remote control device and an actual cleaning area can be mapped. Hereinafter, a system for cleaning a robot according to an embodiment of the present invention capable of specifically realizing the above object will be described with reference to the accompanying drawings.

First, with reference to FIGS. 1 to 3, the configuration of the robot cleaner constituting the robot cleaner system will be described as an example.

1 is a perspective view showing a robot cleaner according to an embodiment of the present invention, Figure 2 is a perspective view showing the internal configuration of a robot cleaner according to an embodiment of the present invention, Figure 3 is a robot cleaner according to an embodiment of the present invention 4 is a block diagram of a robot cleaner constituting a robot cleaner system according to an embodiment of the present invention.

The robot cleaner 100 includes a vacuum cleaner body 110 that forms an exterior, a suction device 120 provided inside the cleaner body 110, and a suction nozzle that sucks dust on the floor by driving the suction device 120 ( 130), and a dust collecting device 140 for collecting foreign substances in the air sucked by the suction nozzle 130.

Here, the cleaner body 110 of the robot cleaner 100 may be a cylindrical shape having a lower height relative to the diameter, that is, a flat cylindrical shape, and may have a square shape with rounded corners. In addition, a suction device 120, a suction nozzle 130, and a dust collecting device 140 communicating with the suction nozzle 130 may be provided inside the cleaner body 110.

In addition, on the outer circumferential surface of the cleaner body 110, a sensor (not shown) that detects a distance from an indoor wall or an obstacle, a bumper (not shown) that cushions an impact in the event of a collision, and wheels for moving the robot cleaner 100 150 may be provided.

The wheel 150 may be provided with left and right driving wheels 152 and 154 respectively installed on both sides of the cleaner body 110. The left and right driving wheels 152 and 154 are configured to rotate by the left wheel motor 152a and the right wheel motor 154a controlled by the cleaner control unit 160, respectively, so that the left wheel motor 152a and the right wheel motor 154a are rotated. According to the driving, the robot cleaner 100 may perform indoor cleaning while changing directions by itself.

In addition, at least one auxiliary wheel 156 is provided on the bottom of the cleaner body 110 to minimize the friction between the robot cleaner 100 and the floor while guiding the movement of the robot cleaner 100.

On the other hand, the cleaner body 110 may be provided with a recognition means. As described later, the recognition means may be provided to map the real area and the virtual area where the cleaner is located. In addition, the recognition means may have various shapes and various shapes, and may be provided inside the cleaner body or may be provided outside the cleaner body. That is, in any case, it is preferable that the recognition means is provided outside the cleaner to be recognized through a camera or the like.

In Fig. 1, a recognition mark 112 is shown as an example of the recognition means. The recognition mark 112 may be displayed on the upper surface of the cleaner body 110 as shown, and may have various patterns or patterns. In addition, the position of the recognition mark 112 may be variously modified and the number may also be variously modified. Through this recognition means, a real area and a virtual area in which the robot cleaner 100 is located in the external device can be mapped. For example, a real area in which the robot cleaner 100 is located and a virtual area displayed on an external device may be mapped. More details on this will be described later.

4 is a block diagram based on the control unit 160 of the robot cleaner 100. Inside the cleaner body 110, for example, the front part may be provided with a cleaner control unit 160 that is connected to various parts of the robot cleaner 100 and controls the driving of the robot cleaner 100. In addition, a battery 170 for supplying power to the suction device 120 may be provided inside the cleaner body 110, for example, behind the cleaner control unit 160.

A suction device 120 for generating air suction force may be provided at the rear of the battery 170, and the dust collecting device 140 is detachable from the rear in the dust collecting device mounting part 140a provided at the rear of the suction device 120. It can be mounted as much as possible.

In addition, a suction nozzle 130 is provided on the lower side of the dust collecting device 140 to inhale foreign substances on the floor together with air. Here, the suction device 120 is installed obliquely between the battery 170 and the dust collecting device 140, and is connected to a motor (not shown) that is electrically connected to the battery 170 and a rotating shaft of the motor to flow air. It is preferably configured to include a forced fan (not shown).

On the other hand, the suction nozzle 130 is exposed to the lower side of the cleaner body 110 through an opening (not shown) formed in the bottom of the cleaner body 110, thereby making contact with the floor of the room.

In order to control the robot cleaner 100 from the outside, the robot cleaner 100 according to the present embodiment preferably includes a first wireless communication unit 180 capable of wireless communication with an external device.

The first wireless communication unit 180 may include one or more modules that enable the robot cleaner 100 to wirelessly communicate with an external device or an external network. For example, the first wireless communication unit 180 may include a wireless Internet module, a short-range communication module, and the like.

The wireless Internet module refers to a module for wireless Internet access, and may be built in or external to the robot cleaner 100. Here, as a wireless Internet technology, a wireless LAN (WLAN) (Wi-Fi), a wireless broadband (Wibro), a world interoperability for microwave access (Wimax), and a high speed downlink packet access (HSDPA) may be used.

The short-range communication module refers to a module for short-range communication. Here, Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, etc. may be used as short range communication technology.

Next, an embodiment of the remote control device 200 constituting the robot cleaner system will be described with reference to FIGS. 5 to 7.

5 is a block diagram of a remote apparatus device constituting a robot cleaner system according to an embodiment of the present invention.

In the present invention, the remote control device 200 is a device that is remotely connected to the robot cleaner 100 and controls driving of the robot cleaner 100, for example, a mobile phone, a smart phone, and PDAs (Personal Digital Assistants). ), PMP (Portable Multimedia Player) and the like can be used as the remote control device 200.

Hereinafter, for convenience of description, it will be described based on the use of a smart phone as the remote control device 200 of the robot cleaner 100.

The remote control device 200 includes a wireless communication unit 210, an audio/video (A/V) input unit 220, a user input unit 230, an output unit 240, a memory 250, an interface unit 260, and a terminal. A control unit 270 and a power supply unit 280 may be included. The components shown in FIG. 5 are not essential, so a remote control device 200 with more or fewer components may be implemented.

Hereinafter, the components will be described in turn.

The second wireless communication unit 210 may include one or more modules that enable wireless communication between the remote control device 200 and the wireless communication system or between the remote control device 200 and the network where the remote control device 200 is located. Can be. For example, the second wireless communication unit 210 may include a mobile communication module 211, a wireless Internet module 212, and a short-range communication module 213.

The mobile communication module 211 transmits and receives a radio signal to and from at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text/multimedia message.

The wireless Internet module 212 refers to a module for wireless Internet access, and may be built in or external to the remote control device 200. Wireless Internet technology (Wireless LAN) (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) may be used.

The short-range communication module 213 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, etc. may be used as short range communication technology.

Referring to FIG. 5, the A/V (Audio/Video) input unit 220 is for inputting an audio signal or a video signal, which may include a camera 221 and a microphone 222. The camera 221 processes image frames such as still images or moving images obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display unit 241.

The image frames processed by the camera 221 may be stored in the memory 250 or transmitted to the outside through the second wireless communication unit 210. Two or more cameras 221 may be provided according to the use environment.

The microphone 222 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes it as electrical voice data. The processed voice data may be converted into a form transmittable to a mobile communication base station through the mobile communication module 211 and output. The microphone 222 may be implemented with various noise removal algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 230 generates input data for the user to control the operation of the remote control device 200. The user input unit 230 may be configured with a key pad dome switch, a touch pad (static pressure/blackout), a jog wheel, a jog switch, or the like.

The output unit 240 is for generating output related to vision, hearing, or tactile sense, and includes a display unit 241, an audio output module 242, an alarm unit 243, and a haptic module 244. Can be.

The display unit 241 displays (outputs) information processed by the remote control device 200. For example, when the remote control device 200 is in a call mode, it displays a user interface (UI) or a graphical user interface (GUI) related to the call. When the remote control device 200 is in a video call mode or a photographing mode, the photographed and/or received video or UI, GUI is displayed.

The display unit 241 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (flexible) display) and a 3D display.

Two or more display units 241 may be present depending on the implementation form of the remote control device 200. For example, the remote control device 200 may have a plurality of display units 241 spaced apart or integrally disposed on one surface, or may be disposed on different surfaces.

When the display unit 241 and a sensor that senses a touch operation (hereinafter, referred to as a “touch sensor”) form a mutual layer structure (hereinafter, referred to as a “touch screen”), the display unit 241 may be used in addition to an output device. It can also be used as an input device. The touch sensor may have a form of, for example, a touch film, a touch sheet, or a touch pad.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display portion 241 or capacitance generated in a specific portion of the display portion 241 into an electrical input signal. The touch sensor may be configured to detect not only the touched position and area, but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal(s) is sent to the touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the terminal control unit 270. As a result, the terminal control unit 270 can know which area of the display unit 241 has been touched.

The audio output module 242 may output audio data received from the second wireless communication unit 210 or stored in the memory 250 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, or the like. The sound output module 242 may also output sound signals related to functions (for example, call signal reception sound, message reception sound, etc.) performed by the remote control device 200. The sound output module 242 may include a receiver, a speaker, and a buzzer.

The alarm unit 243 outputs a signal for notifying the occurrence of an event of the remote control device 200. Examples of events generated by the remote control device 200 include call signal reception, message reception, key signal input, and touch input. The alarm unit 243 may output a signal for notifying the occurrence of an event by other forms, for example, vibration, in addition to a video signal or an audio signal. The video signal or the audio signal may also be output through the display unit 241 or the audio output module 242, so that they 241 and 242 may be classified as part of the alarm unit 243.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic module 244 is vibration. The intensity and pattern of vibration generated by the haptic module 244 is controllable. For example, different vibrations may be synthesized and output or sequentially output.

The memory 250 may store a program for the operation of the terminal control unit 270, and may temporarily store input/output data (eg, a phone book, message, still image, video, etc.). The memory 250 may store data related to various patterns of vibration and sound output when a touch is input on the touch screen.

The memory 250 is a flash memory type, a hard disk type, a multimedia card micro type, a memory of a card type (for example, SD or XD memory, etc.), RAM (Random Access Memory, RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), magnetic memory, magnetic It may include a storage medium of at least one type of disk, optical disk. The remote control device 200 may operate in connection with a web storage performing a storage function of the memory 250 on the Internet.

The interface unit 260 serves as a passage with all external devices connected to the remote control device 200. The interface unit 260 receives data from an external device, receives power, and transmits data to each component inside the remote control device 200, or allows data inside the remote control device 200 to be transmitted to the external device. For example, wired/wireless headset port, external charger port, wired/wireless data port, memory card port, port for connecting devices equipped with an identification module, audio input/output (I/O) port, A video input/output (I/O) port, an earphone port, and the like may be included in the interface unit 260.

The identification module is a chip that stores various information for authenticating the usage rights of the remote control device 200, a user identification module (UIM), a subscriber identification module (SIM), and a universal user authentication module (Universal Subscriber Identity Module, USIM). The device provided with the identification module (hereinafter referred to as'identification device') may be manufactured in a smart card format. Therefore, the identification device may be connected to the remote control device 200 through a port.

The terminal controller (controller, 270) usually controls the overall operation of the remote control device 200. For example, it performs related control and processing for voice calls, data communication, video calls, and the like. The terminal control unit 270 may include a multimedia module 271 for playing multimedia. The multimedia module 271 may be implemented in the terminal control unit 270 or may be implemented separately from the terminal control unit 270.

The terminal control unit 270 may perform a pattern recognition process capable of recognizing handwriting input or picture drawing input performed on a touch screen as text and images, respectively.

The power supply unit 280 receives external power and internal power under the control of the control unit 270 and supplies power required for the operation of each component.

One embodiment of the present invention may be implemented in a computer- or similar device readable recording medium using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions. These may be implemented by the terminal control unit 270.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code can be implemented by software applications written in the appropriate programming language. The software code is stored in the memory 250 and can be executed by the terminal control unit 270.

6 is a perspective view of a remote control device constituting a system when cleaning a robot according to an embodiment of the present invention, as viewed from the front.

The disclosed remote control device 200 has a bar-shaped body. However, the present invention is not limited to this, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are movably coupled.

The body includes a case (casing, housing, cover, etc.) forming an exterior. In this embodiment, the case may be divided into a front case 201 and a rear case 202. Various electronic components are embedded in a space formed between the front case 201 and the rear case 202. At least one intermediate case may be additionally disposed between the front case 201 and the rear case 202.

The cases may be formed by injecting synthetic resin or may be formed to have a metal material, for example, a metal material such as stainless steel (STS) or titanium (Ti).

The body of the remote control device 200, mainly the front case 201, has a display unit 241, an audio output unit 242, a camera 221, an input unit 230/231,232, a microphone 222, an interface 160 Etc. can be arranged.

The display portion 241 occupies most of the main surface of the front case 201. An acoustic output unit 151 and a camera 221 are disposed in an area adjacent to one end of both ends of the display unit 241, and an input unit 231 and a microphone 222 are arranged in an area adjacent to the other end. The input unit 232 and the interface 160 may be disposed on side surfaces of the front case 201 and the rear case 202.

The user input unit 230 is operated to receive a command for controlling the operation of the remote control device 200, and may include a plurality of operation units 231 and 232. The operation units 231 and 232 may also be collectively referred to as a manipulation portion, and any method may be employed as long as the user operates with a tactile feeling.

Contents input by the first or second operation units 231 and 232 may be variously set. For example, the first operation unit 231 receives commands such as start, end, scroll, and the like, and the second operation unit 232 adjusts the volume of sound output from the sound output unit 242 or the display unit 241 ) To a touch recognition mode.

7 is a rear perspective view of a remote control device constituting a robot cleaner system according to an embodiment of the present invention.

Referring to FIG. 7, a camera 221 ′ may be additionally mounted on the rear of the body of the remote control device 200, that is, the rear case 202. The camera 221 ′ has a shooting direction substantially opposite to the camera 221 (see FIG. 6 ), and may be a camera having different pixels from the camera 221.

For example, in the case of a video call, the camera 221 has a low pixel so as not to overwhelm the user's face and transmit it to the other party. It is desirable to have high pixels in many cases. The cameras 221 and 221' may be installed on the body of the remote control device 200 to be rotatable or pop-up.

A flash 123 and a mirror 124 are additionally disposed adjacent to the camera 221'. When the flash 123 photographs the subject with the camera 221', light is directed toward the subject. The mirror 124 allows the user to see his/her face or the like when the user wants to take a picture (self shot) using the camera 221'.

A sound output unit 242' may be additionally disposed on the rear of the body of the remote control device 200. The sound output unit 242' may implement a stereo function together with the sound output unit 242 (see FIG. 6), and may be used to implement a speakerphone mode during a call.

The body of the remote control device 200 is equipped with a power supply unit 280 for supplying power to the remote control device 200. The power supply unit 280 may be built in the body of the remote control device 200, or may be configured to be detachable directly from the outside of the body of the remote control device 200.

8 is a diagram for explaining a method of generating image information of a robot cleaner 100 in a robot cleaner system according to an embodiment of the present invention, and FIG. 9 is a robot cleaner system according to an embodiment of the present invention In, it is a diagram for explaining a method of generating position information of a robot cleaner. Hereinafter, a method of generating image information and location information of the robot cleaner 100 using a remote control device will be described with reference to FIGS. 8 and 9.

As described above, the robot cleaner system according to an embodiment of the present invention includes a remote control device 200 for remotely controlling the driving of the robot cleaner 100. However, in order for the remote control device 200 to control the driving of the robot cleaner 100, accurate location information of the robot cleaner 100 is required. Specifically, mapping between a real area and a virtual area is required.

To solve this, in the robot cleaner system according to an embodiment of the present invention, the robot cleaner 100 is detected by detecting the robot cleaner 100 based on image information generated by using the camera 221 ′ of the remote control device 200. 100) location information.

First, as shown in FIG. 8 with the camera 221 ′ provided on the rear of the body of the remote control device 200, the robot cleaner 100 and the surrounding areas of the robot cleaner 100 are photographed to generate image information. The generated image information of the robot cleaner 100 is transmitted to the terminal control unit 270.

Next, the remote control device 200 maps the virtual area indicated by the image information and the actual area where the robot cleaner 100 is located based on the image information. It can be said that the remote control device 200 detects the robot cleaner 100.

As described above, a recognition mark 112 provided to detect the robot cleaner 100 using an external device may be provided on the upper surface of the robot cleaner 100 (see FIG. 1 ). Here, the shape of the recognition mark 112 is not particularly limited. For example, the recognition mark 112 may be formed in a circular shape as illustrated in FIG. 9A, in which case the width W and height H of the recognition mark 112 have the same value.

The actual shape of the recognition mark 112 is stored in the memory 250 of the remote control device 200. That is, information including the actual size of the recognition mark 112 is stored. The terminal control unit 270 can detect the robot cleaner 100 by extracting the recognition mark 112 from the image of the image information and determining whether the extracted recognition mark 112 is a valid recognition mark 112. have.

Here, the terminal controller 270 may check, for example, the shape or pattern of the recognition mark 112 or a specific color of the recognition mark 112 in order to extract the recognition mark 112 from the image information image.

However, when the image quality of the image of the video information is poor, it is difficult for the terminal control unit 270 to detect the recognition mark 112. Accordingly, the terminal control unit 270 may improve the image quality of the image information before extracting the recognition mark 112 from the image of the image information.

Here, in order to improve the image quality of the image of the image information, the terminal control unit 270 may perform image brightness adjustment, noise reduction, color correction, and the like.

Next, the remote control device 200 may generate the location information of the robot cleaner 100 by comparing the actual shape information of the recognition mark 112 with the relative shape identified by the image information. That is, the mapping of the real area and the virtual area can be performed.

When the user photographs the robot cleaner 100 and the surrounding area using the camera 221' of the remote control device 200, the user makes a photograph while the remote control device 200 is raised to a predetermined height. do. That is, the user is photographed while looking down the robot cleaner 100 at an angle.

Therefore, as illustrated in FIG. 9B, the recognition mark 112 of the robot cleaner 100 on the image of the image information according to the angle at which the user looks down the robot cleaner 100 is reduced in height compared to the width.

In addition, the relative shape and size of the recognition mark 112 recognized from the image information is changed according to the posture of the robot cleaner 100 and the distance between the robot cleaner 100 and the remote control device 200.

As described above, the actual shape of the recognition mark 112 and the relative shape grasped by the image information are different from each other in correspondence with the position and posture of the robot cleaner 100. Therefore, the position and posture of the robot cleaner 100 can be grasped by comparing the actual shape of the recognition mark 112 with the relative shape grasped by the image information. In addition, the distance and the shooting angle between the user and the robot cleaner 100 can be grasped, and the scale between the virtual area and the real area appearing in the image information can be grasped.

Hereinafter, a method for recognizing the position and posture of the robot cleaner 100 by comparing the actual shape of the recognition mark 112 with the relative shape identified by the image information, or for a specific method for mapping between the virtual region and the real region I will explain.

 The following equations can be used to recognize the position and posture of the robot cleaner 100 from the relative shape of the recognition mark 112.

9C is an example of an image of image information on the robot cleaner 100 and its surrounding area generated by the camera 221'. Here, the horizontal photo resolution of the image is a [pixel], the vertical photo resolution b [pixel], and the horizontal angle of view of the camera 221' is c°. This means that the image of the video information has a*b resolution and a horizontal viewing angle c°.

First, the distance between the robot cleaner 100 and the remote control device 200 may be obtained using Equation 1 and Equation 2.

The horizontal angle of view v° of the recognition mark 112 is the horizontal angle of view c° of the camera 221', the vertical length Δh [pixel] of the recognition mark 112 on the image information image, and the image information image as described in equation (1). It can be obtained through the horizontal length Δw [pixel] of the recognition mark 112 on the image.

By substituting the horizontal angle of view v° of the recognition mark 112 thus obtained into Equation 2, the horizontal distance L between the robot cleaner 100 and the remote control device 200 can be obtained as shown in FIG. 9C. have.

Next, as shown in FIG. 9C using Equation 3, the upward angle u°, which is the angle at which the remote control device 200 looks down the robot cleaner 100, may be obtained.

Next, the azimuth angle of the recognition mark 112 indicating the posture of the robot cleaner 100 may be obtained using Equations 4 and 5.

The recognition mark 112 on the image information image has a circular shape according to the upward angle u° or an ellipse shape whose width Δw is longer than the height Δh. Therefore, as illustrated in FIG. 9D, the azimuth angle θ of the recognition mark 112 on the image information image is the azimuth angle θ ₀ of the actual recognition mark 112 according to the upward angle u°. It has the same or less value. That is, when the upward angle of the remote control device 200 is 90°, θ=θ ₀ , and when the upward angle is less than 90°, θ<θ ₀ .

Accordingly, by correcting Equation 4 for obtaining the relative azimuth angle θ from the image of the image information by applying r(H/W ratio), Equation 5 for the actual azimuth angle θ ₀ of the recognition mark 112 is derived. Can be. As a result, the actual azimuth angle θ ₀ of the recognition mark 112 can be obtained to see what attitude the robot cleaner 100 is currently taking.

As described above, in the robot cleaner 100 system according to an embodiment of the present invention, the robot cleaner 100 having the recognition mark 112 is detected using the remote control device 200 having the camera 221'. In addition, it is possible to generate location information of the robot cleaner 100 indicating the position and posture of the robot cleaner 100.

Meanwhile, the robot cleaner 100 may be assumed to travel on a flat surface. That is, the actual cleaning area is displayed on a plane, and the actual cleaning area may be displayed on the remote control device through image information. That is, it may be displayed as a virtual area having a different scale from the actual cleaning area. Therefore, the actual cleaning area and the virtual area can be mapped through the above-described equations.

The location information of the robot cleaner 100 may be displayed to the user through the display unit 241 provided on the remote control device 200, and based on the location information of the robot cleaner 100, the location information of the robot cleaner 100 may be displayed to the user. Provide location-based services. That is, it is possible to provide various types of location-based services through the virtual area through mapping between the virtual area and the real area. Details of the location-based service of the robot cleaner 100 will be described later.

Meanwhile, in the robot cleaner system according to another embodiment of the present invention, the shape information of the robot cleaner 100 instead of the recognition mark 112 is stored in the memory of the remote control device 200, and the robot cleaner ( 100) location information.

That is, the position and posture of the robot cleaner 100 may be recognized by comparing the actual shape of the robot cleaner 100 with a relative shape identified by image information. Another embodiment of the present invention is the same as in the case of one embodiment of the present invention, except that the position and posture are grasped by using the shape information of the robot cleaner 100. do.

10 to 13 are views for explaining a location-based service of a robot cleaner provided by the robot cleaner system according to an embodiment of the present invention. Hereinafter, the location-based service of the robot cleaner 100 will be described with reference to FIGS. 10 to 13.

As described above, the remote control device 200 may generate image information for the robot cleaner 100 and its surrounding areas, and detect the robot cleaner 100 based on the image information to generate location information. .

In addition, the display unit 241 outputs image information about the robot cleaner 100 and its surrounding areas, and in particular, the terminal control unit 270 is a screen of the display unit 241 based on the location information of the robot cleaner 100. The robot cleaner 100 is selectively selectable on the screen. Accordingly, the user can control the driving of the robot cleaner 100 while observing the current state of the robot cleaner 100 through the display unit 241.

On the other hand, in order to control the driving of the robot cleaner 100, the remote control device 200 may further include an input unit for inputting a control signal of the robot cleaner 100, the microphone 222, the user input unit 230 And a display unit 241 provided with a touch sensor that senses a touch input, and may function as an input unit.

When the control signal of the robot cleaner 100 is input from the remote control device 200, the second wireless communication unit of the remote control device 200 transmits a control signal to the first wireless communication unit 180 of the robot cleaner 100. , The robot cleaner 100 is driven according to the control signal by the cleaner controller 160.

Hereinafter, the location-based services of the robot cleaner 100 that can be provided through the input unit of the remote control device 200 will be described with reference to the drawings.

first. The location-based service of the possible robot cleaner 100 provided through the touch input to the display unit 241 will be described as an example.

For example, a cleaning area of the robot cleaner 100 may be set through a touch input. As illustrated in FIG. 10A, when a virtual area A is designated on the screen of the display unit 241, the terminal control unit 270 cleans the actual area A′ of the indoor space corresponding to the area A. Is specified as Accordingly, as illustrated in FIG. 10B, the robot cleaner 100 may move to a cleaning area to perform cleaning.

That is, a virtual area appearing on the display unit 241 may be mapped to a real area to transmit actual movement coordinates to the cleaner.

For example, assuming that the virtual area A designated by the display unit 241 is an area 5 centimeters to the right from the center of the cleaner of the display unit 241, the actual area A'is right from the center of the actual cleaner It can be mapped to an area 1 meter away. Of course, such mapping may be performed through mapping information comparing the shape and size of the actual recognition means with the shape and size of the relative recognition means through image information, as described above. Accordingly, mapping information mapped to a virtual area is transferred to the cleaner 100, and the cleaner can perform cleaning based on the mapping information.

That is, in one embodiment of the present invention, a cleaning area may be simply set through a touch input through an image (ie, a virtual area) of an indoor space output to the display unit 241 as a medium. As a result, it is possible to perform cleaning only on the area desired by the user, thereby shortening the cleaning time and reducing electricity consumption.

Meanwhile, a movement path of the robot cleaner 100 may be set through a touch input. As illustrated in FIG. 11, when the path B is specified on the screen of the display unit 241, the terminal control unit 270 displays the path B'in the indoor space corresponding to the path B, the robot cleaner 100. It is designated as a moving path. Accordingly, the robot cleaner 100 may move to a target point along a movement path.

That is, in one embodiment of the present invention, a moving path of the robot cleaner 100 may be set through a touch input through an image of an indoor space output on the display unit 241. As a result, the robot cleaner 100 automatically moves along a route determined by the user, so that a delay time that may occur when the user changes the direction of the robot cleaner in real time using a direction key or the like can be eliminated.

In addition, it is possible to freely compose the movement path of the robot cleaner by mixing curves and straight lines, enabling sophisticated manipulation.

On the other hand, the cleaning prohibited area of the robot cleaner 100 may be set through a touch input. As shown in FIG. 12, when a forbidden line C is designated on the screen of the display unit 241, the terminal control unit 270 displays an area outside the partition line C'in the indoor space corresponding to the forbidden line C. Set as a cleaning prohibited area. Accordingly, the robot cleaner 100 may clean only the area inside the partition line C'.

That is, in one embodiment of the present invention, the cleaning prohibition area may be set by simply dividing the indoor space through the image of the indoor space output on the display unit 241. Therefore, it is very economical to set the cleaning prohibition area without using separate members such as magnet stripe and virtual wall.

In this way, the control of the robot cleaner 100 through the touch input is performed through the images of the robot cleaner 100 and its surrounding areas output from the display unit 241, thereby intuitively and precisely controlling the robot cleaner 100. Drive can be controlled.

On the other hand, various location-based services provided through touch input are performed by selecting an icon (not shown) of the service displayed on the screen of the display unit 241 after applying a touch input of a preset pattern as described above. Can be. For example, as illustrated in FIG. 11, when a driving route is set on the display unit 241 and an icon is touched, driving route information and a driving command are transmitted from the remote control device 200 to the robot cleaner 100. Can be. Of course, the service may be selected by voice through the microphone 222 or through the user input unit 230.

Next, a location-based service of the robot cleaner 100 that can be provided through voice input through the microphone 222 will be described as an example.

For example, the robot cleaner 100 may be called to a user's location through voice input. As illustrated in FIG. 13, when a user voice inputs a preset mant with respect to the microphone 222 of the remote control device 200, the robot cleaner 100 positions the remote control device 200 based on the location information, that is, You can go to your location.

As described above, it can be seen that wireless control of the robot cleaner is possible through the display unit through the mapping of the real area and the virtual area. Therefore, once the mapping is performed, the user does not need to directly look at the real area or the robot cleaner. That is, when mapping is performed, wireless control of the robot cleaner is possible through the display unit even if the user moves to another area. Of course, in this case, the wireless communication between the remote control device 200 and the robot cleaner 100 should be maintained.

* Generally, the area cleaned by the robot cleaner can be fixed. In this case, detection of the robot cleaner through the remote control device 200 or mapping of the real area and the virtual area may be continuously maintained. That is, it is possible to use a single detection or mapping as it is. Therefore, it may not be necessary to photograph the cleaner every time the wireless control is performed. That is, it is possible to store the conventionally captured image information and display the image information on the display unit whenever wireless control is performed. Thereafter, wireless control may be repeatedly performed through a virtual area displayed on the display unit. In this case, if the environment in which wireless communication between the robot cleaner 100 and the remote control device 100 is possible, the user may have the remote control device 100 and perform wireless control of the robot cleaner 100 from outside.

For example, the robot cleaner may be wirelessly connected through a Wi-Fi AP in the home, and the user's remote control device 100 may be wirelessly connected to the robot cleaner through a server and the Wi-Fi AP. Therefore, the user can wirelessly control the robot cleaner in the home from the outside.

Of course, in order to enable such wireless control, the robot cleaner 100 must first be moved to an initial position. That is, it is preferable to move to the position of the robot cleaner 100 when shooting is performed. Therefore, it is preferable that initial position information (including coordinate information and posture information) is stored in the robot cleaner, and when the wireless control is started, it is desirable that the robot cleaner is moved to the initial position.

The wireless control of the robot cleaner 100 through the remote control device 200 may not only perform the original function of the cleaner called cleaning, but additionally provide amusement to the user.

So far, the robot cleaner 100 system according to an aspect of the present invention has been described. Hereinafter, a method of controlling a system in cleaning a robot according to another aspect of the present invention will be described with reference to the drawings.

14 is a flowchart illustrating a basic configuration of a method for controlling a robot cleaner system according to an embodiment of the present invention, and FIG. 15 is a step of detecting a robot cleaner in a method for controlling a robot cleaner system according to an embodiment of the present invention 16 is a flowchart for explaining a step of providing a location-based service in a method for controlling a robot cleaner system according to an embodiment of the present invention.

Here, the contents overlapping with the robot cleaner system according to the embodiment of the present invention described above will be omitted or briefly described, and the same components will use the same reference numerals.

The control method of the robot cleaner system according to an embodiment of the present invention is a remote control device 200 having a robot cleaner 100 and a camera 221' and a display unit 241 schematically as shown in FIG. In the robot cleaner system comprising a), generating image information about the robot cleaner 100 and its surrounding area through the camera 221', and detecting the robot cleaner 100 based on the image information It may include. In addition, it may include the step of generating the location information of the robot cleaner 100. That is, the step of mapping the real region and the virtual region through the generated image information may be performed.

Meanwhile, a step of outputting image information and location information to the display unit 241 may be included, and this step may be performed by generating image information. Thereafter, a step of providing a location-based service based on the location information may be performed.

Hereinafter, a control method of a robot cleaner system according to an embodiment of the present invention will be described in detail with reference to FIGS. 14 to 16.

First, a step of generating image information about the robot cleaner 100 and its surrounding areas may be performed through the camera 221 ′ of the remote control device 200 (S100 ).

Next, a step of sensing the robot cleaner 100 based on the image information may be performed (S200 ). Here, the step of sensing the robot cleaner 100 may be performed by sensing the recognition mark 112 of the robot cleaner 100.

* In accordance with this, the step of detecting the robot cleaner 100 (S200) is a step of improving an image of image information as shown in FIG. 15 (S210) and detecting a recognition mark 112 of the robot cleaner 100. (S220 ), determining whether the detected recognition mark 112 is a valid recognition mark 112 (S230 ), and recognizing the robot cleaner 100 (S240 ).

In addition, the step (S120) of improving the image of the image information is performed by the terminal control unit 270 of the remote control device 200, and more specifically, includes at least one of image brightness adjustment, noise removal, and color correction. Can be performed.

In addition, the step (S240) of recognizing the robot cleaner 100 may be performed by the terminal controller 270 recognizing an object having a valid recognition mark 112 as the robot cleaner 100.

On the other hand, in the step (S 230) of determining whether the detected recognition mark 112 is a valid recognition mark 112, if the recognition mark 112 is not a valid recognition mark 112, the robot is again through the camera 221'. Steps for generating image information about the cleaner 100 and its surrounding areas may be performed (S260).

Next, a step of generating location information of the robot cleaner 100 may be performed (S300).

Specifically, the step S300 of generating position information of the robot cleaner 100 may be performed by comparing the actual shape of the recognition mark 112 with the relative shape identified by the image information. Here, the terminal control unit 270 generates location information of the robot cleaner 100 based on factors such as a horizontal angle of view, a H/W ratio, an upward angle, and an azimuth angle of the recognition mark 112 identified by the image information. can do. Therefore, through this, it is possible to map the real area and the virtual area.

Next, a step of outputting the image information and the location information of the robot cleaner 100 to the display unit 241 output to the display unit 241 may be performed (S400). That is, after the photographing, the screen of the display unit 241 is switched and a mapping step is performed, and when the mapping step ends, the captured image information and location information may be output from the display unit 241. Of course, after the shooting is completed, the photographed screen is continuously output from the display unit 241, and it is also possible to perform the mapping step. Accordingly, in this case, the wireless control through the display unit 241 will be possible only after the mapping is completed.

Here, based on the image information and the location information, the terminal control unit 270 may selectively display the robot cleaner 100 on the screen of the display unit 241.

Next, a step of providing a location-based service based on the location information may be performed (S500).

Here, the step in which the robot cleaner 100 provides the location-based service includes: generating a control signal of the robot cleaner 100 in the remote control device 200, transmitting a control signal to the robot cleaner 100, and The robot cleaner 100 may include a step of being driven according to a control signal.

In addition, the step of generating a control signal of the robot cleaner 100 may be performed by applying a touch input to the display unit 241. In particular, the touch input is applied through the image of the robot cleaner 100 and its surrounding area output from the display unit 241 as a medium. As a result, as a touch input of a predetermined pattern is applied to drive the robot cleaner 100 to provide a location-based service, a control signal corresponding thereto may be generated.

Here, the predetermined pattern is, for example, by drawing a circle on the screen of the display unit 241 to designate a predetermined area, to draw a path made of curves and straight lines, or to draw a partition line that divides the screen of the display unit 241. have.

As described above, as a control signal is input through a touch input, the location-based service may include at least one of a movement path setting function, a cleaning area designation function, and a cleaning prohibition area designation function.

Meanwhile, the location-based service may include a calling function in which the robot cleaner 100 moves to a location of the remote control device 200 when a voice call signal is input to the remote control device 200.

Since the remote control device 200 is provided with a microphone 222, the remote control device 200 transmits a voice call signal to the robot cleaner 100 through the microphone 222 to remotely control the robot cleaner 100. The location of the 200 may be called, that is, the location of the user.

17 is a flowchart illustrating a step of detecting a robot cleaner in a control method of a robot cleaner system according to another embodiment of the present invention.

The control method of the robot cleaner system according to another embodiment of the present invention is the same as the robot cleaner system according to an embodiment of the present invention in a basic configuration, but the robot cleaner 100 is formed through the shape of the robot cleaner 100. ), and has a difference in that it generates location information.

First, the step of detecting the robot cleaner 100 (S 200) is a step of improving the image of the image information as shown in FIG. 17 (S 210'), and detecting the shape of the robot cleaner 100 (S) 220'), determining whether the detected shape of the robot cleaner 100 is a valid shape (S 230'), and recognizing the robot cleaner 100 (S 240).

Here, the shape of the robot cleaner 100 is pre-stored in the terminal control unit 270 of the remote control device 200, and the shape of the robot cleaner 100 can be detected using the image information.

On the other hand, in the step of determining whether the detected shape of the robot cleaner 100 is a valid shape, if it is not a valid shape, generating image information about the robot cleaner 100 and the surrounding area again through the camera 221' Can be performed. (S 260')

Next, a step of generating location information of the robot cleaner 100 may be performed. (S 300)

In addition, the step of generating location information of the robot cleaner 100 may be performed by comparing the actual shape of the robot cleaner 100 with the relative shape identified by the image information. Here, the terminal control unit 270 generates location information of the robot cleaner 100 based on factors such as a horizontal angle of view, an H/W ratio, an upward angle, and an azimuth angle of the robot cleaner 100 identified by the image information. can do.

Hereinafter, another embodiment of the recognition means will be described with reference to FIG. 18.

First, unlike in FIG. 1, it is preferable that a plurality of recognition means is provided instead of one. This is because the recognition means may be distorted by photography. Also, a small error may actually appear as a large error in the mapping process. In addition, the recognition means may be distorted and photographed through an external light source, for example, sunlight or lighting.

Therefore, it is preferable that at least three recognition means are provided to compensate for the distortion.

Meanwhile, as illustrated in FIG. 1, recognition means such as a recognition mark 112 may be provided on the outer surface of the cleaner body 110. Therefore, the external design of the cleaner may be damaged due to the recognition mark 112. In recent years, the design of robot cleaners is in progress to exclude artificial design patterns or patterns and to have a natural texture. It may not be desirable to have a means of recognition against this trend.

Therefore, a recognition means such as the recognition mark 112 can be selectively provided outside the main body. For example, the recognition mark 112 may be manufactured in the form of a detachable sheet to perform wireless control, or the recognition mark 112 may be attached to the outside of the body when photographing a cleaner for this purpose. In addition, the recognition mark 112 can be removed from the outside of the main body in the usual time.

For example, a plurality of magnets may be provided on the recognition mark 112, and a plurality of magnets corresponding to the magnets may be provided inside the body. Therefore, when the recognition mark 112 is removed, damage to the original appearance design of the cleaner can be prevented. In addition, it is possible to attach and use the recognition mark 112 to the cleaner body 110 only when necessary. The removed recognition mark 112 may be attached to the door of a household appliance such as a refrigerator. Therefore, it can be manufactured to be easy to store.

Meanwhile, the robot cleaner system according to an embodiment of the present invention may include a charging stand 300 in which the robot cleaner is charged. Such a charging stand can be said to be a very common configuration in a robot cleaner. However, the above-described recognition means may be provided in the robot cleaner 100 and/or the charging stand, and the same or similar location-based service to the robot cleaner may be provided through the charging stand recognition means.

18 shows an example of the charging base 300.

The charging unit 300 may be provided with a sensor unit 350, and the sensor unit may transmit and receive signals to and from the short-range communication module 182 of the robot cleaner 100. The robot cleaner 100 may be induced to return to the charging base 200 through communication through the sensor unit 350 or the like.

The charging base 300 may include a charging base body 310, and a slot 350 may be provided in the main body 310 so that the robot cleaner 100 may be mounted, and in the slot 350 The charging terminal 330 may be provided.

In general, the robot cleaner 100 recognizes its own position and posture within a home (cleaning area) through a SLAM (simultaneous location and mapping) technology. In addition, the position of the charging base 300 can be grasped, and relative position and posture of the person can also be recognized based on the charging base 300.

Accordingly, it may be possible to map the real area and the virtual area based on the charging base 300, similar to mapping the real area and the virtual area based on the robot cleaner 100 described above.

Specifically, the charging stand 300 may be provided with the same or similar recognition means as described above. 18 shows that the recognition mark 312 is provided as an example of the recognition means.

The user can generate the same image information through the remote control device 200. That is, through the recognition mark 312 provided on the charging base 300, a real region around the charging base 300 and a virtual region displayed on the display unit of the remote control device 200 may be mapped. In addition, the same location-based service can be provided through the mapping as described above.

For example, the user may designate an area near the charging base 300 as a cleaning area while looking at the display unit. The virtual cleaning area may be transmitted to the charging base 300 or the robot cleaner 100 as a real area based on mapping information.

Since the robot cleaner 100 grasps the current position based on the charging base 300, it is possible to grasp the designated real area.

For example, when a user designates an area 5 centimeters to the right from the charging base 300 in the virtual area, it is possible to actually designate an area 3 meters to the right through the mapping information. At this time, the robot cleaner may be located 5 meters away from the charging base 300 and may not appear in the virtual area. Nevertheless, since the robot cleaner 100 grasps its own position with respect to the charging base, it is possible to perform cleaning by traveling to an actual area.

Therefore, the same effect can be obtained by providing the charging stand 300 with a recognition means. However, in this case, since the actual robot cleaner 100 is not photographed, and the charging stand 300 is generally fixed, the controllable area will be limited to a region near the charging stand 300.

As described above, the robot cleaner system and the control method capable of providing a location-based service by mapping the virtual area and the real area through the recognition means provided in the robot cleaner 100 or the charging table 300 will be described in detail. Did. Specifically, the recognition mark has been described in detail as an example as a recognition means.

However, recognition means such as the recognition marks 112 and 312 may affect the appearance design of the charging table 300 of the robot cleaner 100. In recent years, a minimalist design has been preferred that allows a simple yet unique texture or color of the material to be expressed as an exterior design. For such a minimal design, recognition means such as the recognition marks 112 and 312 may cause damage to the exterior design.

Therefore, there is a need to provide a recognition means that can be easily detected without compromising the unique design of the robot cleaner 100 or the charging base 300.

19, the main body 110 of the robot cleaner 100 may include an outer wall 111. The exterior is formed through the outer wall 111 manufactured in the form of a panel, and various components may be provided inside the outer wall 111.

In this embodiment, the illumination image 412 may be provided as a recognition means. The illumination image 412 may be selectively activated. That is, through selective light emission or activation of the light source 400, the illumination image 412 may be generated or removed.

Specifically, the outer wall 111 may include a color layer 111b and a reflective layer 111c. That is, the color layer 111b is formed from the outside, and the reflective layer 111c may be formed inside thereof. The color layer 111b expresses the external design color of the main body 110.

Basically, a light source from the outside is not transmitted inside the outer wall 111 through the reflective layer 111c. Therefore, the user cannot see inside the reflective layer 111c. In addition, a transparent layer 111a is provided on the outside of the color layer 111b, so that a smoother and more glossy appearance design can be realized.

On the other hand, for the selective generation or activation of the illumination image 412, an illumination device 400 may be provided inside the reflective layer 111c. For example, the lighting device 400 may include an LED lighting device, and may include an LED element. Therefore, light may be radiated from the inside of the main body 111 to the outside of the main body by activation of the LED lighting device or the LED element. The light may project the reflective layer 111c to form a constant illumination image 111a on the outer wall 111.

That is, as illustrated in FIG. 19, when the lighting device 400 is inactivated, the lighting image 412 is not generated on the outer wall 111 and the lighting device 400 is not visible to the outside. Therefore, in the case of inactivation of the lighting device 400, the product-specific design is not damaged.

Conversely, when the lighting device 400 is activated, the lighting image 412 is generated on the outer wall 111, and the remote control device 200 maps the virtual area and the real area through the lighting image 412. .

On the other hand, when the lighting image 412 is used as the recognition means, the recognition means can be recognized more clearly in a dark environment as well as maintaining the design.

20 shows the appearance of the robot cleaner 100 before and after the illumination image 412 is activated. That is, the appearance design of the general robot cleaner 100 may be maintained as it is before the illumination image 412 is activated. However, when the illumination image 412 is activated, the illumination image 412 is generated outside the main body 110 so that it can be easily recognized from the outside.

The illumination image may be arranged such that single illumination images per LED element have various shapes. Further, it is also possible to form an illumination image in a certain area through an LED lighting device having a plurality of LED elements. For example, it is also possible to form a lighting image having a circular or polygonal shape through an LED lighting device.

Since the illumination image is formed by light, it can be more easily recognized even in a dark environment. Therefore, even in a slightly dark room, it can be easily used even when the lighting is not turned on.

Here, it is preferable that the illumination image 412 is not always activated. That is, it is preferably activated only when necessary.

As illustrated in FIG. 14, it may be activated when the user remotely controls the robot cleaner 100 through the remote control device 200 or wants to use a location-based service. That is, the illumination image 412 may be activated in the initialization (S10) step for such control. In addition, subsequent steps may be performed through the initialization (S10).

For example, a service such as the remote control may be performed through an application installed in the remote control device 200. Accordingly, when the user executes the application on the remote control device 200, the remote control device 200 may transmit an execution command to the robot cleaner 100. Based on the execution command, the robot cleaner 100 may apply power to the lighting device 400 to activate the lighting image 412.

That is, the illumination image 412 is activated through the remote control device 100 in the initializing step (S10), and can be maintained continuously during the location-based service through the application. And, through the termination of the application, the activation of the lighting image 412 may be released.

As described above, the illumination image 412 is preferably activated only when using the remote control device 100. Therefore, not only the user but also other users can intuitively recognize that the remote control device 100 is being used by looking at the state of the robot cleaner 100. In addition, a new design as well as a unique design may be implemented through the lighting image 412, and it is possible to provide a play function as well as a product-specific function to the user.

On the other hand, as described above, the recognition means may be formed in various forms, and the location may also be variously formed. In addition, the shape, number, and location of the recognition means may be variously modified.

Preferably, the recognition means are used to map the real and virtual areas very accurately. Therefore, it is desirable that the error is minimized through the recognition means. This is because when the error in the small area is mapped to the large area, the error becomes much larger.

Therefore, for accurate and precise mapping, it may be desirable for the recognition means to be provided in a plurality of locations, for example at least three locations. And, in this case, it is preferable that recognition means having various shapes are provided.

In general, circular images have the least risk of distortion at various angles. This is because the center of the circle can always be grasped easily. Therefore, it is preferable to include a recognition means such as a circular image or mark among a plurality of recognition means.

In general, a window or an illumination light may be reflected outside the main body 110 by a light source such as external illumination or sunlight. However, the upper portion of the main body 110 may be manufactured in a very gentle curved form. Therefore, this reflection occurs only in a part of the upper portion of the body 110. Of course, this reflection can distort the means of recognition.

In order to respond flexibly to this problem, it is preferable that a plurality of recognition means is provided as described above. 1, the recognition means may be provided on the upper left and lower sides of the upper surface of the main body, and on the upper right and lower sides of the upper surface of the main body, respectively. Therefore, it is possible to prevent the recognition means from being distorted by an external light source or the like, which causes a large error.

21 shows various types of recognition means. Of course, the illustrated recognition means may be in the form of a mark and may be a lighting image. It can have a variety of colors, and it can also have a particularly vivid primary color series. For example, it may be in the form of a visual mark as shown in FIGS. 21A to 21C, and may be in the form of an IR mark as shown in FIG. 21D. In such a recognition means, the straight portion needs to be clearly identified when projected in a plane. Therefore, in each recognition means, it is necessary that at least two points are connected to each other so that the straight portion is clearly displayed.

22 is an example of a screen displayed on the remote control device 200, and may be referred to as an initial screen in which an application is executed. Various contents can be selected through the initial screen of the application.

For example, an icon called remote control may be displayed on the initial screen. Through the remote control icon, it is possible to execute the above-described location-based control service.

On the other hand, the initial screen may be provided with icons that can use various contents in addition to the remote control icon, for example, an icon called "virtual eye" may be provided. As described below, the content through which the augmented reality image is implemented may be used through the icon, and through this, the user's satisfaction may be improved and intuitive information may be obtained.

Hereinafter, an augmented reality service that can be distinguished from the above-described location-based control service or remote control will be described in detail.

First, the configuration and features of the above-described robot cleaner and remote control device may be applied in the same way. In addition, generating image information of the robot cleaner through the remote control device, and extracting location information and posture information of the robot cleaner through the same may be the same or similar. In addition, extraction of such information may be possible through the above-described recognition means.

However, the service according to the present embodiment may also be referred to as a location-based control service. However, in terms of remote control, it may be somewhat different from the above-described embodiments.

When the user selects a specific icon shown in FIG. 22, for example, an icon called virtual die, the screen shown in FIG. 23 may appear. That is, the screen illustrated in FIG. 23 may be said to be a state in which the user is pointing the cleaner through the camera through the display. Therefore, basically, a cleaner image and an image around the cleaner may be displayed on the display. Of course, as described above, such an image may be referred to as an image displayed in a display area, that is, a virtual area, not an actual area. Such an image can be referred to as a preview image.

Here, the remote control device 200 may extract position information and posture information of the robot cleaner through the above-described recognition means. Through this location information and posture information, the remote control device 200 may grasp the location and posture information of the preview image of the robot cleaner in the display area.

Through this location and posture information, the remote control device 200 may display an augmented reality image superimposed on the preview image of the robot cleaner.

As an example, the image of the robot cleaner shown in FIG. 23 may be a preview image, or an augmented reality image superimposed on the preview image. In addition, a part of the preview image of the robot cleaner may be an image in which an augmented reality image is superimposed.

The remote control device 200 may generate mapping information between a real area and a virtual area displayed on the display through image information of the robot cleaner 100 or image information about the robot cleaner 100 and its surrounding areas. have. Through this mapping information, the augmented reality image can be superimposed on the preview image of the robot cleaner displayed on the display. Since the mapping information can be more precisely performed through the above-described recognition means, the augmented reality image can be more precisely superimposed on the preview image.

Meanwhile, the recognition means may be the above-described LED lighting, and by selecting the icon named virtual eye shown in FIG. 22, the LED lighting may be activated.

As described above, FIG. 23 can be said to be an example of an initial screen in this embodiment. Menus for a plurality of inputs may be displayed on the initial screen. For example, at least one of a cleaning start menu, a character change menu, and a follow and menu may be displayed.

As described above, through the initial screen, position information and posture information or mapping information is generated. Of course, before the generation of such information, the initialization step S10 of the remote control device 200 and the robot cleaner 100 should be performed. In this initialization step, both wireless connection or synchronization may be performed.

When synchronization is performed, the screen illustrated in FIG. 23 may be gradually changed to the screen illustrated in FIG. 24. Such a screen may be implemented as a video, and may be displayed to change its appearance like a living character. In other words, the augmented reality image can be implemented as a dynamic image rather than a static image.

Of course, all of the images of the cleaner illustrated in FIG. 24 may be augmented reality images, and only a part of them may be augmented reality images. For example, it may be an augmented reality image in which a light projected image, an eyelid image, and an eye image are superimposed on the preview image. Alternatively, the entire robot cleaner image including such an image may be an augmented reality image.

The augmented reality mode can be easily understood through the screen shown in FIG. 24, which can be more clearly grasped through the motion or sound of the augmented reality image. Therefore, it can be clearly determined whether or not the synchronization state through FIG. This is because not only an intuitive image, but also text and the like can be displayed in augmented reality.

25 is a screen for an example of a command waiting state after synchronization is completed. That is, it is possible to more intuitively indicate that the current state of waiting is through an eye shape, an eyelid shape, and a character. Of course, all of these shapes and characters can be said to be augmented reality images.

26 is a screen for an example of a state in which charging is required. As an example, it is possible to intuitively indicate that the current state of charging is required through an eye shape or a letter, and an icon indicating a figure, a battery shape, and a charge amount. In addition, it is possible to provide an effective image to draw attention to the user through an augmented reality image with different colors.

27 is a screen for an example of a cleaning state after pressing the start menu displayed on the display. If cleaning is being performed, the actual cleaner will move. In this case, the movement path and posture of the actual cleaner should be mapped and displayed on the display. Therefore, the preview video image is also moved. Accordingly, the augmented reality moves according to the preview image of the moving robot cleaner, that is, synchronized. Through the movement of the augmented reality image, the user can grasp intuitively that cleaning is being performed.

28 is a view of selecting a menu to follow. As described above, the cleaner can know the location of the user through the recognition means. Therefore, when a follow and menu is selected, the robot cleaner is in a state of approaching the user. In this state, the augmented reality image may represent a robot cleaner receiving a master like a pet. That is, a more friendly image may be displayed through an eye shape or an image such as a heart.

As described above, the state information of the robot cleaner may vary, and according to the plurality of state information, the augmented reality image may be changed. In addition, the augmented reality image may be expressed in motion, and such motion may be changed according to state information. In addition, different sounds may be generated according to the state information.

24 to 28 are examples of augmented reality images in various states. However, such an augmented reality image may be implemented in a three-dimensional manner, rather than a two-dimensional image, and may be displayed as a dynamic image rather than a static image.

For example, as illustrated in FIG. 27, after the light has no shape and gradually spreads to the upper portion, text or other images may be displayed. That is, the augmented reality image is dynamically implemented to enhance user satisfaction. In other words, a dynamic augmented reality image such as a video is displayed in each state, so that the user can be satisfied.

Meanwhile, a character change menu may be provided on the display.

The character may display a preview image of the robot cleaner so that all specific characters overlap, and may be an additional augmented reality image in a character form separate from the augmented reality image for the robot cleaner.

For example, a puppy character may be displayed on the robot cleaner as an augmented reality image, and the current state may be interestingly displayed through various motions, characters, and expressions of the puppy.

In addition, the dog character following the robot cleaner may be displayed as a separate augmented reality image. Therefore, the user may feel like watching a cartoon movie through the display.

Therefore, it is possible to provide amusement to the user by adding game elements such as pets or cartoon characters as well as augmented reality images of a simple robot cleaner. Through this, the robot cleaner can approach the user more intimately and deliver satisfaction to the user as if raising a pet.

This not only provides a basic cleaning function of the robot vacuum cleaner, but also can provide a user with pleasure, thereby increasing user satisfaction. In addition, it is possible to make it easy for the user to feel the feeling of enjoying the game.

The robot cleaner system and control method described above are not limited to the configuration and method of the above-described embodiments, and the embodiments are all or part of each embodiment so that various modifications can be made. It can be configured in combination.

100: robot cleaner 112: recognition mark
180: first wireless communication unit 200: remote control unit 210: second wireless communication unit 222': camera 241: display unit 300: charging stand

Claims (21)

In the control method of the remote control device to remotely control the robot cleaner with a camera and a display,
(A) generating image information about the robot cleaner and the surrounding area of the robot cleaner through the camera;
(B) detecting the robot cleaner based on the image information and generating location information of the robot cleaner; And
And (c) displaying an augmented reality image overlapping on a preview image of the robot cleaner displayed on the display unit,
When the robot cleaner moves, a preview image of the robot cleaner is moved on the display, and the augmented reality image is synchronized with the preview image and moved. According to claim 1,
The control method of the remote control device, characterized in that for generating the posture information of the robot cleaner in step (B). According to claim 1,
The step (B) includes a mapping step of mapping the actual area where the robot cleaner is located and the virtual area where the image information appears to generate mapping information. The method of claim 3,
The mapping step, the control method of the remote control device comprising the step of detecting the robot cleaner based on the image information. The method of claim 4,
The mapping is performed by comparing the actual shape of the robot cleaner with a relative shape identified through the image information. The method of claim 4,
The mapping is performed by comparing actual shape information of the recognition means provided in the robot cleaner with relative shape information identified by the image information. The method according to any one of claims 1 to 6,
The augmented reality image is a control method of a remote control device, characterized in that it is displayed overlapping a part or all of the preview image. The method according to any one of claims 1 to 6,
The remote control device is a control method of a remote control device, characterized in that an application for using the content to which the augmented reality image is implemented is installed. The method of claim 8,
Control method of a remote control device, characterized in that an icon for using the content on which the augmented reality image is implemented is displayed on the initial screen of the application. The method of claim 9,
When the icon is selected, the preview image is displayed on the display unit, so that the step (a) is performed. According to claim 1,
A control method of a remote control device, characterized in that an initialization step is performed in which wireless connection and synchronization between the remote control device and the robot cleaner are performed before generating the location information. The method of claim 11,
When the synchronization is performed, the control method of the remote control device, characterized in that the screen displaying the preview image is changed to a screen displaying the augmented reality image. The method of claim 12,
The augmented reality image control method of a remote control device, characterized in that the dynamic image. The method according to any one of claims 1 to 6,
In the step (c), the control method of the remote control device is characterized by displaying a command waiting state of the robot cleaner through an augmented reality image that is a shape or a character. The method according to any one of claims 1 to 6,
In the step (c), a control method of a remote control device, characterized in that a state requiring charging is displayed through the augmented reality image. The method according to any one of claims 1 to 6,
In the step (c), at least one of a cleaning start menu, a character change menu, and a follow and menu is displayed. The method of claim 16,
When the cleaning start menu is pressed, a control method of a remote control device characterized in that a movement path and posture of an actual robot cleaner is mapped and displayed on the display unit. The method of claim 17,
A method of controlling a remote control device, characterized in that the augmented reality image is also synchronized with the moving preview image of the robot cleaner. The method according to any one of claims 1 to 6,
In the step (c), a method of controlling a remote control device, characterized in that it displays status information of the robot cleaner. The method of claim 19,
The status information,
Control method of a remote control device, characterized in that it comprises at least one of a waiting state for cleaning of the robot vacuum cleaner, a request state for charging the robot cleaner, a cleaning progress state of the robot vacuum cleaner, and a state in which the robot cleaner approaches the user. . The method of claim 19,
The state information is plural, and the control method of the remote control device is characterized in that augmented reality images that are distinguished from each other are displayed for each state information.

Images