KR102613624B1 - Cleaning robot for airport and method thereof - Google Patents

Abstract

An airport cleaning robot according to an embodiment of the present invention includes a communication unit that transmits and receives data, a location recognition unit that reads location information, a driving unit that performs cleaning and movement, a camera that photographs the appearance of an object, and the airport cleaning robot. It includes a control unit that controls the operation, and when receiving a cleaning request message, the control unit reads the location information included in the cleaning request message, moves to the cleaning location, and collects the first cleaning object information included in the cleaning request message. read, scan a cleaning object at the cleaning location, read second cleaning object information with an image of the scanned cleaning object, and if the first cleaning object information and the second cleaning object information match, the cleaning object It is characterized in that it is controlled to clean.

Description

Airport cleaning robot and its operation method {CLEANING ROBOT FOR AIRPORT AND METHOD THEREOF}

The present invention relates to a robot deployed at an airport and its operating method. More specifically, it is about an airport cleaning robot that is deployed at the airport and cleans trash within the airport. Airport cleaning robots can receive data from various devices deployed within the airport. Additionally, it relates to a cleaning robot that performs efficient cleaning based on various types of data received and its operating method.

Recently, the functions of robots are expanding due to developments in deep learning technology, autonomous driving technology, automatic control technology, and the Internet of Things.

To explain each technology in detail, deep learning corresponds to a field of machine learning. Deep learning is a technology that allows the program to make similar decisions in various situations, rather than checking conditions and setting commands in advance in the program. Therefore, according to deep learning, computers can think similar to the human brain and enable the analysis of vast amounts of data.

Autonomous driving is a technology that allows machines to make their own decisions to move and avoid obstacles. According to autonomous driving technology, robots can autonomously recognize their location through sensors and move and avoid obstacles.

Automatic control technology refers to a technology that automatically controls the operation of a machine by feeding back the measured values of the machine's status to the control device. Therefore, control without human operation is possible, and the desired control object can be automatically adjusted to reach the target value, that is, within the desired range.

The Internet of Things refers to intelligent technologies and services that connect all objects based on the Internet to communicate information between people and objects, and between objects. Through the Internet of Things, devices connected to the Internet exchange information and communicate autonomously without human assistance.

The application fields of robots are generally classified into industrial, medical, space, and undersea applications. For example, in machining industries such as automobile production, robots can perform repetitive tasks. In other words, many industrial robots are already in operation that can repeat the same movements for hours after being taught just once what a human arm does.

The purpose of the present invention is to improve cleaning completeness using airport robots and various devices deployed within the airport.

Another object of the present invention is to improve cleaning efficiency by placing a cleaning robot in the right place.

Another purpose of the present invention is to allow users to easily perform cleaning by controlling a cleaning robot within the airport.

The airport cleaning robot according to the present invention can receive data from CCTV, servers, user terminals, etc. in the airport. Airport cleaning robots can detect cleaning objects from received data.

The airport cleaning robot according to the present invention may be composed of a plurality of types depending on the cleaning method. Additionally, the server or management robot can clean by calling a cleaning robot of a type that can perform cleaning according to the cleaning object.

The airport cleaning robot according to the present invention can perform data communication with a remote control device through a communication unit. Additionally, the user can control the cleaning robot by touching the display unit of the mobile terminal.

The airport cleaning robot according to the present invention can detect cleaning objects by receiving data from CCTV, servers, user terminals, etc. in the airport. As a result, it is possible to find cleaning objects that the cleaning robot could not find, which has the effect of increasing cleaning completeness.

The server or management robot that manages the airport cleaning robot according to the present invention can select a matching cleaning robot based on cleaning object information. Therefore, cleaning efficiency can be improved by immediately calling an appropriate cleaning robot that matches the cleaning object.

The airport cleaning robot according to the present invention can be controlled by a remote control device such as a user's terminal. As a result, users within the airport can easily control the cleaning robot using their smartphones to directly clean the cleaning objects.

Figure 1 is a block diagram showing the hardware configuration of an airport robot according to an embodiment of the present invention.
Figure 2 is a diagram illustrating in detail the configuration of a microcomputer and AP of an airport robot according to another embodiment of the present invention.
Figure 3 is a diagram for explaining the structure of an airport robot system according to an embodiment of the present invention.
Figure 4 is a block diagram for explaining a mobile terminal related to the present invention.
Figure 5 is a diagram for explaining various types of cleaning robots according to an embodiment of the present invention.
Figures 6 and 7 are diagrams to explain an example in which a server manages cleaning robots and CCTV images installed in an airport according to an embodiment of the present invention.
Figures 8 to 10 are diagrams to explain an example in which a user in an airport photographs a cleaning object and transmits it to a server.
11 and 12 are diagrams for explaining an example in which a management robot calls a cleaning robot according to an embodiment of the present invention.
Figures 13 to 15 are diagrams to explain an example in which a cleaning robot organizes a trash can using Internet of Things technology according to an embodiment of the present invention.
Figures 16 to 19 are diagrams to explain an example in which a user controls a cleaning robot using a remote control device according to an embodiment of the present invention.
Figure 20 is a block diagram showing the configuration of an airport cleaning robot according to an embodiment of the present invention.

Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings.

The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves.

Figure 1 is a block diagram showing the hardware configuration of an airport robot according to an embodiment of the present invention.

As shown in FIG. 1, the hardware of the airport robot 100 according to an embodiment of the present invention may be composed of a Micom group and an AP group. The microcomputer 110 group may include a microcomputer 110, a power supply unit 120, an obstacle recognition unit 130, and a travel drive unit 140. The AP group may include an AP (150), a user interface unit (160), an object recognition unit (170), a location recognition unit (180), and a LAN (190). The user interface unit 160 may be called a communication unit.

Among the hardware of the airport robot, the microcomputer 110 can manage the power supply unit 120 including a battery, the obstacle recognition unit 130 including various sensors, and the travel driving unit 140 including a plurality of motors and wheels. .

The power supply unit 120 may include a battery driver 121 and a lithium-ion battery 122. The battery driver 121 can manage charging and discharging of the lithium-ion battery 122. The lithium-ion battery 122 can supply power to drive the airport robot. The lithium-ion battery 122 can be configured by connecting two 24V/102A lithium-ion batteries in parallel.

The obstacle recognition unit 130 may include an IR remote control receiver 131, USS 132, Cliff PSD 133, ARS 134, Bumper 135, and OFS 136. The IR remote control receiver 131 may include a sensor that receives a signal from an IR (Infrared) remote control for remotely controlling the airport robot. USS (Ultrasonic sensor, 132) may include a sensor for determining the distance between an obstacle and an airport robot using ultrasonic signals. Cliff PSD 133 may include a sensor to detect cliffs or cliffs in the 360-degree omnidirectional airport robot driving range. ARS (Attitude Reference System, 134) may include a sensor that can detect the attitude of the airport robot. The ARS 134 may include a sensor consisting of three acceleration axes and three gyro axes for detecting the rotation amount of the airport robot. Bumper 135 may include sensors that detect collisions between the airport robot and obstacles. The sensor included in Bumper (135) can detect collisions between airport robots and obstacles in a 360-degree range. OFS (Optical Flow Sensor, 136) may include a sensor that can measure the spinning phenomenon of the airport robot while driving and the driving distance of the airport robot on various floor surfaces.

The traveling drive unit 140 includes motor drivers (141), wheel motors (142), rotation motors (143), main brush motors (144), side brush motors (145), and suction motors (Suction Motors, 146). It can be included. The motor driver 141 may serve to drive wheel motors, brush motors, and suction motors for driving and cleaning the airport robot. The wheel motor 142 can drive a plurality of wheels for driving the airport robot. The rotation motor 143 may be driven to rotate the main body of the airport robot or the head of the airport robot left and right, up and down, or to change the direction or rotate the wheels of the airport robot. The main brush motor 144 can drive a brush that sweeps up dirt from the airport floor. The side brush motor 145 can drive a brush that sweeps away dirt from the area around the outer surface of the airport robot. The suction motor 146 may be driven to suction dirt from the airport floor.

AP (Application Processor, 150) can function as a central processing unit that manages the entire hardware module system of the airport robot. The AP 150 can use location information received through various sensors to drive an application program for driving and transmit user input/output information to the microcomputer 110 to drive a motor, etc.

The user interface unit 160 includes a user interface processor (UI Processor) 161, an LTE router (162), a WIFI SSID (163), a microphone board (164), a barcode reader (165), a touch monitor (166), and It may include a speaker 167. The user interface processor 161 can control the operation of the user interface unit responsible for user input and output. The LTE router 162 can receive necessary information from the outside and perform LTE communication to transmit information to the user. The WIFI SSID 163 can perform location recognition of a specific object or airport robot by analyzing the signal strength of WiFi. The microphone board 164 can receive a plurality of microphone signals, process the voice signal into voice data, which is a digital signal, and analyze the direction of the voice signal and the corresponding voice signal. The barcode reader 165 can read barcode information written on a plurality of tickets used at the airport. The touch monitor 166 may include a touch panel configured to receive user input and a monitor to display output information. The speaker 167 may perform the role of informing the user of specific information by voice.

The object recognition unit 170 may include a 2D camera 171, an RGBD camera 172, and a recognition data processing module 173. The 2D camera 171 may be a sensor for recognizing people or objects based on two-dimensional images. RGBD camera (Red, Green, Blue, Distance, 172) for detecting people or objects using captured images with depth data obtained from cameras with RGBD sensors or other similar 3D imaging devices. It could be a sensor. The recognition data processing module 173 can recognize people or objects by processing signals such as 2D images/videos or 3D images/videos obtained from the 2D camera 171 and the RGBD camera 172.

The location recognition unit 180 may include a stereo board (Stereo B/D) 181, Lidar (182), and a SLAM camera (183). SLAM cameras (Simultaneous Localization And Mapping cameras, 183) can implement simultaneous location tracking and mapping technology. The airport robot can detect information about the surrounding environment using the SLAM camera 183, process the obtained information, create a map corresponding to the mission performance space, and estimate its absolute position at the same time. Lidar (Light Detection and Ranging: Lidar, 182) is a laser radar and may be a sensor that performs location recognition by irradiating a laser beam and collecting and analyzing backscattered light among the light absorbed or scattered by aerosol. The stereo board 181 can process and process sensing data collected from the lidar 182 and the SLAM camera 183 and manage data for location recognition and obstacle recognition of the airport robot.

The LAN 190 may communicate with the user interface processor 161, the recognition data processing module 173, the stereo board 181, and the AP 150 related to user input and output.

Figure 2 is a diagram illustrating in detail the configuration of a microcomputer and AP of an airport robot according to another embodiment of the present invention.

As shown in FIG. 2, the microcomputer 210 and AP 220 can be implemented in various embodiments to control the recognition and behavior of the airport robot.

As an example, the microcomputer 210 may include a data access service module (Data Access Service Module, 215). The data access service module 215 includes a data acquisition module (211), an emergency module (212), a motor driver module (213), and a battery manager module (214). may include. The data acquisition module 211 may acquire sensed data from a plurality of sensors included in the airport robot and transmit it to the data access service module 215. The emergency module 212 is a module that can detect an abnormal state of the airport robot. When the airport robot performs a predetermined type of action, the emergency module 212 detects that the airport robot has entered an abnormal state. You can. The motor driver module 213 can manage the driving control of the wheels, brushes, and suction motors for driving and cleaning the airport robot. The battery manager module 214 is responsible for charging and discharging the lithium-ion battery 122 of FIG. 1 and can transmit the battery status of the airport robot to the data access service module 215.

The AP 220 can receive various cameras, sensors, user input, etc., recognize and process them, and control the operation of the airport robot. The interaction module 221 integrates the recognition data received from the recognition data processing module 173 and the user input received from the user interface module 222, and manages software that allows users and airport robots to interact with each other. It may be a module that does this. The user interface module 222 receives the user's short-range commands such as the display unit 223, which is a monitor for the current status and operation/information provision of the airport robot, and keys, touch screens, readers, etc., or operates the airport robot. It is possible to manage user input received from the user input unit 224, which receives a long-distance signal such as a signal from an IR remote control for remote control, or receives a user input signal from a microphone or barcode reader. When at least one user input is received, the user interface module 222 may transmit the user input information to the state management module (State Machine module, 225). The state management module 225 that receives user input information can manage the overall state of the airport robot and issue appropriate commands in response to user input. The planning module 226 can determine the start and end time/actions for a specific operation of the airport robot according to the command received from the state management module 225, and calculate which path the airport robot should move. The navigation module 227 is responsible for the overall driving of the airport robot and can cause the airport robot to drive according to the driving route calculated in the planning module 226. The motion module 228 can perform basic airport robot operations in addition to driving.

Additionally, the airport robot according to another embodiment of the present invention may include a location recognition unit 230. The location recognition unit 230 may include a relative location recognition unit 231 and an absolute location recognition unit 234. The relative position recognition unit 231 can correct the movement amount of the airport robot through the RGM mono (232) sensor, calculate the movement amount of the airport robot for a certain period of time, and recognize the current surrounding environment of the airport robot through LiDAR (233). can do. The absolute location recognition unit 234 may include a Wifi SSID 235 and UWB 236. Wifi SSID (235) is a UWB sensor module for absolute location recognition of airport robots, and is a WIFI module for estimating the current location through Wifi SSID detection. The Wifi SSID (235) can recognize the location of the airport robot by analyzing the Wifi signal strength. The UWB 236 can sense the absolute position of the airport robot by calculating the distance between the transmitter and the receiver.

Additionally, the airport robot according to another embodiment of the present invention may include a map management module 240. The map management module 240 may include a grid module 241, a path planning module 242, and a map division module 243. The grid module 241 can manage a grid-shaped map generated by the airport robot through a SLAM camera or map data of the surrounding environment for location recognition input to the airport robot in advance. The path planning module 242 may be responsible for calculating the driving path of the airport robots in dividing the map for collaboration between a plurality of airport robots. Additionally, the path planning module 242 can also calculate the driving path that the airport robot must travel in an environment in which one airport robot operates. The map division module 243 can calculate in real time the areas each of the plurality of airport robots will be responsible for.

Data sensed and calculated from the location recognition unit 230 and the map management module 240 may be transmitted back to the state management module 225. The state management module 225 may issue a command to the planning module 226 to control the operation of the airport robot based on data sensed and calculated from the location recognition unit 230 and the map management module 240.

Next, Figure 3 is a diagram for explaining the structure of an airport robot system according to an embodiment of the present invention.

The airport robot system according to an embodiment of the present invention may include a mobile terminal 310, a server 320, an airport robot 300, and a camera 330.

The mobile terminal 310 can transmit and receive data with the server 320 within the airport. For example, the mobile terminal 310 may receive airport-related data such as flight time schedule, airport map, etc. from the server 320. The user can obtain necessary information by receiving it from the server 320 at the airport through the mobile terminal 310. Additionally, the mobile terminal 310 can transmit data such as photos, videos, and messages to the server 320. For example, a user can send a photo of a lost child to the server 320 to register a lost child, or take a photo of an area that needs cleaning in the airport with a camera and send it to the server 320 to request cleaning of the area.

Additionally, the mobile terminal 310 can transmit and receive data with the airport robot 300.

For example, the mobile terminal 310 may transmit a signal calling the airport robot 300, a signal commanding the airport robot 300 to perform a specific operation, or an information request signal to the airport robot 300. The airport robot 300 may move to the location of the mobile terminal 310 in response to a call signal received from the mobile terminal 310 or perform an operation corresponding to a command signal. Alternatively, the airport robot 300 may transmit data corresponding to the information request signal to each user's mobile terminal 310.

Next, the airport robot 300 can perform roles such as patrolling, guiding, cleaning, quarantine, and transport within the airport.

The airport robot 300 can transmit and receive signals with the mobile terminal 310 or the server 320. For example, the airport robot 300 can transmit and receive signals including situation information within the airport with the server 320. Additionally, the airport robot 300 can receive image information captured from each area of the airport from the camera 330 within the airport. Therefore, the airport robot 300 can monitor the airport situation by combining the image information captured by the airport robot 300 and the image information received from the camera 330.

The airport robot 300 can receive commands directly from the user. For example, a command can be received directly from the user through a touch input or voice input on the display unit provided in the airport robot 300. The airport robot 300 may perform operations such as patrolling, guiding, and cleaning according to commands received from a user, mobile terminal 310, or server 320.

Next, the server 320 may receive information from the mobile terminal 310, the airport robot 300, and the camera 330. The server 320 can integrate, store, and manage information received from each device. The server 320 may transmit the stored information to the mobile terminal 310 or the airport robot 300. Additionally, the server 320 may transmit a command signal for each of the plurality of airport robots 300 deployed at the airport.

The camera 330 may include a camera installed within the airport. For example, the camera 330 may include a plurality of closed circuit television (CCTV) cameras, infrared heat detection cameras, etc. installed within the airport. The camera 330 may transmit the captured image to the server 320 or the airport robot 300.

Figure 4 is a block diagram for explaining a mobile terminal related to the present invention.

The mobile terminal 310 includes a wireless communication unit 410, an input unit 420, a sensing unit 440, an output unit 450, an interface unit 460, a memory 470, a control unit 480, and a power supply unit 490. ), etc. may be included. The components shown in FIG. 4 are not essential for implementing the mobile terminal, so the mobile terminal described herein may have more or fewer components than the components listed above.

More specifically, among the above components, the wireless communication unit 410 is used between the mobile terminal 310 and the wireless communication system, between the mobile terminal 310 and another mobile terminal 310, or between the mobile terminal 310 and an external server. It may include one or more modules that enable wireless communication between the devices. Additionally, the wireless communication unit 410 may include one or more modules that connect the mobile terminal 310 to one or more networks.

This wireless communication unit 410 may include at least one of a broadcast reception module 411, a mobile communication module 412, a wireless Internet module 413, a short-range communication module 414, and a location information module 415. .

The input unit 420 includes a camera 421 or video input unit for inputting video signals, a microphone 422 or an audio input unit for inputting audio signals, and a user input unit 423 for receiving information from the user, for example. , touch keys, push keys (mechanical keys, etc.). Voice data or image data collected by the input unit 420 may be analyzed and processed as a user's control command.

The sensing unit 440 may include one or more sensors for sensing at least one of information within the mobile terminal, information on the surrounding environment surrounding the mobile terminal, and user information. For example, the sensing unit 440 includes a proximity sensor (441), an illumination sensor (442), a touch sensor, an acceleration sensor, a magnetic sensor, and a gravity sensor. Sensor (G-sensor), gyroscope sensor, motion sensor, RGB sensor, IR sensor (infrared sensor), fingerprint scan sensor, ultrasonic sensor , optical sensors (e.g., cameras (see 421)), microphones (see 422), battery gauges, environmental sensors (e.g., barometers, hygrometers, thermometers, radiation detection sensors, It may include at least one of a heat detection sensor, a gas detection sensor, etc.) and a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the mobile terminal disclosed in this specification can utilize information sensed by at least two of these sensors by combining them.

The output unit 450 is for generating output related to vision, hearing, or tactile sense, and includes at least one of a display unit 451, an audio output unit 452, a haptip module 453, and an optical output unit 454. can do. The display unit 451 can implement a touch screen by forming a layered structure or being integrated with the touch sensor. This touch screen functions as a user input unit 423 that provides an input interface between the mobile terminal 310 and the user, and can simultaneously provide an output interface between the mobile terminal 310 and the user.

The interface unit 460 serves as a passageway for various types of external devices connected to the mobile terminal 310. This interface unit 460 connects devices equipped with a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, and an identification module. It may include at least one of a port, an audio input/output (I/O) port, a video input/output (I/O) port, and an earphone port. In response to the external device being connected to the interface unit 460, the mobile terminal 310 may perform appropriate control related to the connected external device.

Additionally, the memory 470 stores data supporting various functions of the mobile terminal 310. The memory 470 may store a number of application programs (application programs or applications) running on the mobile terminal 310, data for operating the mobile terminal 310, and commands. At least some of these applications may be downloaded from an external server via wireless communication. Additionally, at least some of these applications may be present on the mobile terminal 310 from the time of shipment for the basic functions of the mobile terminal 310 (e.g., incoming and outgoing calls, receiving and sending functions). Meanwhile, the application program may be stored in the memory 470, installed on the mobile terminal 310, and driven by the control unit 480 to perform the operation (or function) of the mobile terminal.

In addition to operations related to the application program, the control unit 480 typically controls the overall operation of the mobile terminal 310. The control unit 480 can provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the components discussed above, or by running an application program stored in the memory 470.

Additionally, the control unit 480 may control at least some of the components examined with FIG. 4 in order to run an application program stored in the memory 470. Furthermore, the control unit 480 may operate at least two of the components included in the mobile terminal 310 in combination with each other in order to run the application program.

The power supply unit 490 receives external power and internal power under the control of the control unit 480 and supplies power to each component included in the mobile terminal 310. This power supply unit 490 includes a battery, and the battery may be a built-in battery or a replaceable battery.

At least some of the above components may operate in cooperation with each other to implement operation, control, or a control method of a mobile terminal according to various embodiments described below. Additionally, the operation, control, or control method of the mobile terminal may be implemented on the mobile terminal by running at least one application program stored in the memory 470.

Below, before looking at various embodiments implemented through the mobile terminal 310 discussed above, the components listed above will be looked at in more detail with reference to FIG. 4.

First, looking at the wireless communication unit 410, the broadcast reception module 411 of the wireless communication unit 410 receives broadcast signals and/or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channels may include satellite channels and terrestrial channels. Two or more broadcast reception modules may be provided in the mobile terminal 310 for simultaneous broadcast reception of at least two broadcast channels or broadcast channel switching.

The mobile communication module 412 uses technical standards or communication methods for mobile communication (e.g., Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV -DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced, etc.), wireless signals are transmitted and received with at least one of a base station, an external terminal, and a server on a mobile communication network constructed according to (Long Term Evolution-Advanced), etc.).

The wireless signal may include various types of data based on voice call signals, video call signals, or text/multimedia message transmission and reception.

The wireless Internet module 413 refers to a module for wireless Internet access and may be built into or external to the mobile terminal 310. The wireless Internet module 413 is configured to transmit and receive wireless signals in a communication network based on wireless Internet technologies.

Wireless Internet technologies include, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), and WiMAX (Worldwide). Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc., and the wireless Internet module ( 413) transmits and receives data according to at least one wireless Internet technology, including Internet technologies not listed above.

From the perspective that wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is achieved through a mobile communication network, the wireless Internet module 413 performs wireless Internet access through the mobile communication network. ) may be understood as a type of the mobile communication module 412.

The short-range communication module 414 is for short-range communication, including Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and NFC. Short-distance communication can be supported using at least one of (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. This short-range communication module 114 is used between the mobile terminal 100 and a wireless communication system, between the mobile terminal 310 and another mobile terminal 310, or between the mobile terminal 310 through wireless area networks. ) can support wireless communication between a network where another mobile terminal (310, or an external server) is located. The short-range wireless communication networks may be wireless personal area networks.

Here, the other mobile terminal 310 is a wearable device (e.g., smartwatch, smart glass) capable of exchanging data with (or interoperating with) the mobile terminal 310 according to the present invention. (smart glass), HMD (head mounted display)). The short-range communication module 414 may detect (or recognize) a wearable device capable of communicating with the mobile terminal 310 around the mobile terminal 310 . Furthermore, if the detected wearable device is a device authenticated to communicate with the mobile terminal 310 according to the present invention, the control unit 480 sends at least a portion of the data processed by the mobile terminal 310 to the short-range communication module ( 414) can be transmitted to a wearable device. Accordingly, the user of the wearable device can use data processed in the mobile terminal 310 through the wearable device. For example, according to this, when a call is received on the mobile terminal 310, the user makes a phone call through a wearable device, or when a message is received on the mobile terminal 310, the user makes a phone call through the wearable device. It is possible to check the message.

The location information module 415 is a module for acquiring the location (or current location) of the mobile terminal, and representative examples thereof include a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, if a mobile terminal uses a GPS module, it can acquire the location of the mobile terminal using signals sent from GPS satellites. As another example, when a mobile terminal utilizes a Wi-Fi module, the location of the mobile terminal can be obtained based on information from the Wi-Fi module and a wireless AP (Wireless Access Point) that transmits or receives wireless signals. If necessary, the location information module 415 may replace or additionally perform any of the functions of other modules of the wireless communication unit 410 to obtain data regarding the location of the mobile terminal. The location information module 415 is a module used to obtain the location (or current location) of the mobile terminal 310, and is not limited to a module that directly calculates or obtains the location of the mobile terminal.

Next, the input unit 420 is for inputting video information (or signal), audio information (or signal), data, or information input from the user. For input of video information, the mobile terminal 310 is one Alternatively, a plurality of cameras 421 may be provided. The camera 421 processes image frames, such as still images or moving images, obtained by an image sensor in video call mode or shooting mode. The processed image frame may be displayed on the display unit 451 or stored in the memory 470. Meanwhile, the plurality of cameras 421 provided in the mobile terminal 310 may be arranged to form a matrix structure, and through the cameras 421 forming the matrix structure, the mobile terminal 310 can be viewed at various angles or focuses. A plurality of image information may be input. Additionally, the plurality of cameras 421 may be arranged in a stereo structure to acquire left and right images for implementing a three-dimensional image.

The microphone 422 processes external acoustic signals into electrical voice data. The processed voice data can be utilized in various ways depending on the function (or application program being executed) being performed in the mobile terminal 310. Meanwhile, various noise removal algorithms may be implemented in the microphone 422 to remove noise generated in the process of receiving an external acoustic signal.

The user input unit 423 is for receiving information from the user. When information is input through the user input unit 423, the control unit 480 can control the operation of the mobile terminal 310 to correspond to the input information. . This user input unit 423 is a mechanical input means (or mechanical key, for example, a button located on the front, rear, or side of the mobile terminal 310, a dome switch, a jog wheel, a jog switches, etc.) and touch input means. As an example, the touch input means consists of a virtual key, soft key, or visual key displayed on the touch screen through software processing, or a part other than the touch screen. It can be done with a touch key placed in . Meanwhile, the virtual key or visual key can be displayed on the touch screen in various forms, for example, graphic, text, icon, video or these. It can be made up of a combination of .

Meanwhile, the sensing unit 440 senses at least one of information within the mobile terminal, information about the surrounding environment surrounding the mobile terminal, and user information, and generates a sensing signal corresponding thereto. Based on these sensing signals, the control unit 480 may control the driving or operation of the mobile terminal 310 or perform data processing, functions, or operations related to an application program installed on the mobile terminal 310. Representative sensors among the various sensors that may be included in the sensing unit 440 will be looked at in more detail.

First, the proximity sensor 441 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing nearby without mechanical contact using the power of an electromagnetic field or infrared rays. This proximity sensor 441 may be placed in the inner area of the mobile terminal surrounded by the touch screen as seen above or near the touch screen.

Examples of the proximity sensor 441 include a transmissive photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high-frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. When the touch screen is capacitive, the proximity sensor 441 may be configured to detect the proximity of a conductive object by changing the electric field according to the proximity of the object. In this case, the touch screen (or touch sensor) itself can be classified as a proximity sensor.

Meanwhile, for convenience of explanation, the act of approaching an object on the touch screen without touching it so that the object is recognized as being located on the touch screen is called "proximity touch," and the touch The act of actually touching an object on the screen is called “contact touch.” The position at which an object is close-touched on the touch screen means the position at which the object corresponds perpendicularly to the touch screen when the object is close-touched. The proximity sensor 441 can detect proximity touch and proximity touch patterns (e.g., proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.). there is. Meanwhile, the control unit 480 processes data (or information) corresponding to the proximity touch operation and proximity touch pattern detected through the proximity sensor 441, as described above, and further provides visual information corresponding to the processed data. It can be output on the touch screen. Furthermore, the control unit 480 may control the mobile terminal 310 to process different operations or data (or information) depending on whether a touch to the same point on the touch screen is a proximity touch or a contact touch. .

The touch sensor detects a touch (or touch input) applied to the touch screen (or display unit 451) using at least one of several touch methods such as resistive method, capacitive method, infrared method, ultrasonic method, and magnetic field method. sense

As an example, a touch sensor may be configured to convert changes in pressure applied to a specific part of the touch screen or capacitance occurring in a specific part into an electrical input signal. The touch sensor may be configured to detect the position, area, pressure at the time of touch, capacitance at the time of the touch, etc., at which the touch object that touches the touch screen is touched on the touch sensor. Here, the touch object is an object that applies a touch to the touch sensor, and may be, for example, a finger, a touch pen or stylus pen, or a pointer.

In this way, when there is a touch input to the touch sensor, the corresponding signal(s) are sent to the touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the control unit 480. As a result, the control unit 480 can determine which area of the display unit 451 has been touched. Here, the touch controller may be a separate component from the control unit 480 or may be the control unit 480 itself.

Meanwhile, the control unit 480 may perform different controls or the same control depending on the type of touch object that touches the touch screen (or touch keys provided other than the touch screen). Whether to perform different controls or the same controls depending on the type of touch object may be determined depending on the current operating state of the mobile terminal 310 or the application program being executed.

Meanwhile, the touch sensor and proximity sensor discussed above are used independently or in combination to provide short (or tap) touch, long touch, multi touch, and drag touch on the touch screen. ), flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, etc. Touch can be sensed.

An ultrasonic sensor can recognize the location information of a sensing object using ultrasonic waves. Meanwhile, the control unit 480 is capable of calculating the location of the wave generator through information sensed from an optical sensor and a plurality of ultrasonic sensors. The location of the wave generator can be calculated using the property that light is much faster than ultrasonic waves, that is, the time for light to reach the optical sensor is much faster than the time for ultrasonic waves to reach the ultrasonic sensor. More specifically, the location of the wave source can be calculated using the time difference between the arrival time of the ultrasonic waves using light as a reference signal.

Meanwhile, looking at the configuration of the input unit 420, the camera 421 includes at least one of a camera sensor (eg, CCD, CMOS, etc.), a photo sensor (or image sensor), and a laser sensor.

The camera 421 and the laser sensor can be combined with each other to detect the touch of the sensing object for a 3D stereoscopic image. A photo sensor can be laminated on the display element, and this photo sensor is made to scan the movement of a detection object close to the touch screen. More specifically, the photo sensor mounts photo diodes and TR (transistors) in rows/columns and scans the contents placed on the photo sensor using electrical signals that change depending on the amount of light applied to the photo diode. In other words, the photo sensor calculates the coordinates of the sensing object according to the amount of change in light, and through this, location information of the sensing object can be obtained.

The display unit 451 displays (outputs) information processed in the mobile terminal 310. For example, the display unit 451 may display execution screen information of an application running on the mobile terminal 310, or UI (User Input) and GUI (Graphic User Input) information according to this execution screen information. .

Additionally, the display unit 451 may be configured as a three-dimensional display unit that displays a three-dimensional image.

A three-dimensional display method such as a stereoscopic method (glasses method), an autostereoscopic method (glasses-free method), or a projection method (holographic method) may be applied to the three-dimensional display unit.

The audio output unit 452 may output audio data received from the wireless communication unit 410 or stored in the memory 470 in call signal reception, call mode or recording mode, voice recognition mode, broadcast reception mode, etc. The sound output unit 452 also outputs sound signals related to functions performed in the mobile terminal 310 (eg, call signal reception sound, message reception sound, etc.). This sound output unit 452 may include a receiver, speaker, buzzer, etc.

The haptic module 453 generates various tactile effects that the user can feel. A representative example of a tactile effect generated by the haptic module 453 may be vibration. The intensity and pattern of vibration generated by the haptic module 453 can be controlled by the user's selection or settings of the control unit. For example, the haptic module 453 may synthesize and output different vibrations or output them sequentially.

In addition to vibration, the haptic module 453 responds to stimulation such as pin arrays moving perpendicular to the contact skin surface, blowing force or suction force of air through a nozzle or inlet, grazing the skin surface, contact with electrodes, and electrostatic force. A variety of tactile effects can be generated, including effects by reproducing hot and cold sensations using elements that can absorb heat or heat.

The haptic module 453 can not only deliver a tactile effect through direct contact, but can also be implemented so that the user can feel the tactile effect through muscle senses such as fingers or arms. Two or more haptic modules 453 may be provided depending on the configuration of the mobile terminal 310.

The optical output unit 454 uses light from the light source of the mobile terminal 310 to output a signal to notify that an event has occurred. Examples of events occurring in the mobile terminal 310 may include receiving a message, receiving a call signal, a missed call, an alarm, a schedule notification, receiving an email, receiving information through an application, etc.

The signal output by the optical output unit 454 is realized as the mobile terminal emits light of a single color or multiple colors to the front or back. The signal output may be terminated when the mobile terminal detects the user's confirmation of the event.

The interface unit 460 serves as a passageway for all external devices connected to the mobile terminal 310. The interface unit 460 receives data from an external device, receives power and transmits it to each component inside the mobile terminal 310, or transmits data inside the mobile terminal 310 to an external device. For example, wired/wireless headset port, external charger port, wired/wireless data port, memory card port, port for connecting a device equipped with an identification module. A port, an audio input/output (I/O) port, a video input/output (I/O) port, an earphone port, etc. may be included in the interface unit 460.

Meanwhile, the identification module is a chip that stores various information for authenticating the use authority of the mobile terminal 310, and includes a user identification module (UIM), subscriber identity module (SIM), and general user authentication. It may include a module (universal subscriber identity module; USIM), etc. A device equipped with an identification module (hereinafter referred to as an 'identification device') may be manufactured in a smart card format. Therefore, the identification device can be connected to the terminal 310 through the interface unit 460.

In addition, the interface unit 460 serves as a path through which power from the cradle is supplied to the mobile terminal 310 when the mobile terminal 310 is connected to an external cradle, or provides power input from the cradle by the user. It can be a passage through which various command signals are transmitted to the mobile terminal 310. Various command signals or the power input from the cradle may be operated as signals for recognizing that the mobile terminal 310 is correctly mounted on the cradle.

The memory 470 can store programs for operating the control unit 480, and can also temporarily store input/output data (eg, phone book, messages, still images, videos, etc.). The memory 470 can store data on various patterns of vibration and sound output when a touch is input on the touch screen.

The memory 470 is a flash memory type, hard disk type, solid state disk type, SDD type (Silicon Disk Drive type), and multimedia card micro type. ), card-type memory (e.g. SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), EEPROM (electrically erasable programmable read) -only memory), PROM (programmable read-only memory), magnetic memory, magnetic disk, and optical disk may include at least one type of storage medium. The mobile terminal 100 may be operated in connection with web storage that performs the storage function of the memory 470 on the Internet.

Meanwhile, as discussed above, the control unit 480 controls operations related to application programs and generally controls the overall operation of the mobile terminal 310. For example, if the state of the mobile terminal satisfies a set condition, the control unit 480 can execute or release a lock state that restricts the user's input of control commands to applications.

In addition, the control unit 480 performs control and processing related to voice calls, data communications, video calls, etc., or performs pattern recognition processing to recognize handwriting input or drawing input made on the touch screen as text and images, respectively. You can. Furthermore, the control unit 480 may control any one or a combination of the components described above in order to implement various embodiments described below on the mobile terminal 310 according to the present invention.

The power supply unit 490 receives external power and internal power under the control of the control unit 480 and supplies power necessary for the operation of each component. The power supply unit 490 includes a battery, and the battery can be a built-in battery that can be recharged, and can be detachably coupled to the terminal body for charging, etc.

Additionally, the power supply unit 490 may be provided with a connection port, and the connection port may be configured as an example of the interface 460 to which an external charger that supplies power for charging the battery is electrically connected.

As another example, the power supply unit 490 can charge the battery wirelessly without using the connection port. In this case, the power supply unit 490 uses one or more of the inductive coupling method based on magnetic induction phenomenon or the magnetic resonance coupling method based on electromagnetic resonance phenomenon from an external wireless power transmitter. Power can be transmitted.

Meanwhile, various embodiments below may be implemented in a recording medium readable by a computer or similar device, for example, using software, hardware, or a combination thereof.

Next, we will look at a communication system that can be implemented through the mobile terminal 310 according to the present invention.

First, communication systems may utilize different wireless interfaces and/or physical layers. For example, wireless interfaces available by the communication system include Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), and Code Division Multiple Access (CDMA). ), Universal Mobile Telecommunications Systems (UMTS) (especially LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced)), Global System for Mobile Communications (GSM), etc. This may be included.

Hereinafter, for convenience of explanation, the description will be limited to CDMA. However, it is obvious that the present invention can be applied to all communication systems, including not only CDMA wireless communication systems but also OFDM (Orthogonal Frequency Division Multiplexing) wireless communication systems.

The CDMA wireless communication system includes at least one terminal 100, at least one base station (BS (may be named Node B or Evolved Node B)), and at least one base station controller (Base Station Controllers, BSCs). ), and may include a mobile switching center (MSC). The MSC is configured to connect to the Public Switched Telephone Network (PSTN) and BSCs. BSCs can be paired and connected to a BS through a backhaul line. The backhaul line may be provided according to at least one of E1/T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL, or xDSL. Accordingly, multiple BSCs may be included in a CDMA wireless communication system.

Each of the plurality of BSs may include at least one sector, and each sector may include an omni-directional antenna or an antenna pointing in a specific radial direction from the BS. Additionally, each sector may include two or more antennas of various types. Each BS may be configured to support multiple frequency allocations, and each of the multiple frequency allocations may have a specific spectrum (eg, 1.25 MHz, 5 MHz, etc.).

The intersection of sector and frequency allocation may be called a CDMA channel. BS may be called Base Station Transceiver Subsystem (BTSs). In this case, one BSC and at least one BS may be collectively referred to as a “base station.” A base station may also refer to a “cell site.” Alternatively, each of a plurality of sectors for a specific BS may be referred to as a plurality of cell sites.

A broadcasting transmitter (BT) transmits broadcast signals to terminals 310 operating within the system. The broadcast reception module 411 shown in FIG. 4 is provided in the terminal 310 to receive a broadcast signal transmitted by BT.

In addition, the CDMA wireless communication system may be linked to a global positioning system (GPS) to confirm the location of the mobile terminal 310. The satellite helps determine the location of the mobile terminal 310. Useful location information may be obtained by two or more satellites. Here, the location of the mobile terminal 310 can be tracked using not only GPS tracking technology but also any technology that can track location. Additionally, at least one of the GPS satellites may be selectively or additionally responsible for satellite DMB transmission.

The location information module 415 provided in the mobile terminal is used to detect, calculate, or identify the location of the mobile terminal, and representative examples may include a Global Position System (GPS) module and a Wireless Fidelity (WiFi) module. If necessary, the location information module 415 may replace or additionally perform any of the functions of other modules of the wireless communication unit 410 to obtain data regarding the location of the mobile terminal.

The GPS module 415 calculates distance information and accurate time information from three or more satellites and then applies trigonometry to the calculated information to accurately calculate three-dimensional current location information according to latitude, longitude, and altitude. can do. Currently, a method of calculating location and time information using three satellites and correcting errors in the calculated location and time information using another satellite is widely used. Additionally, the GPS module 415 can calculate speed information by continuously calculating the current location in real time. However, it is difficult to accurately measure the location of the mobile terminal using a GPS module in areas where satellite signals are shadowed, such as indoors. Accordingly, WPS (WiFi Positioning System) can be used to compensate for GPS-based positioning.

The WiFi Positioning System (WPS) uses a WiFi module provided in the mobile terminal 310 and a wireless AP (Wireless Access Point) that transmits or receives wireless signals with the WiFi module. As a technology for tracking the location of a device, it refers to a location determination technology based on WLAN (Wireless Local Area Network) using WiFi.

The Wi-Fi location tracking system may include a Wi-Fi location determination server, a mobile terminal 310, a wireless AP connected to the mobile terminal 310, and a database storing arbitrary wireless AP information.

The mobile terminal 310 connected to the wireless AP may transmit a location information request message to the Wi-Fi location determination server.

The Wi-Fi location determination server extracts information on the wireless AP connected to the mobile terminal 310 based on the location information request message (or signal) of the mobile terminal 310. Information on the wireless AP connected to the mobile terminal 310 may be transmitted to the Wi-Fi location determination server through the mobile terminal 310, or may be transmitted from the wireless AP to the Wi-Fi location determination server.

Based on the location information request message of the mobile terminal 310, the extracted wireless AP information includes MAC Address, Service Set IDentification (SSID), Received Signal Strength Indicator (RSSI), Reference Signal Received Power (RSRP), and RSRQ ( Reference Signal Received Quality), channel information, privacy, network type, signal strength, and noise strength.

As described above, the Wi-Fi location determination server can receive information on the wireless AP connected to the mobile terminal 310 and extract wireless AP information corresponding to the wireless AP to which the mobile terminal is connected from a pre-built database. At this time, information on any wireless APs stored in the database includes MAC Address, SSID, channel information, Privacy, Network Type, latitude and longitude coordinates of the wireless AP, name of the building where the wireless AP is located, number of floors, and detailed indoor location information (GPS coordinates). available), and may be information such as the AP owner’s address and phone number. At this time, in order to remove mobile APs or wireless APs provided using illegal MAC addresses from the positioning process, the Wi-Fi location determination server may extract only a certain number of wireless AP information in order of high RSSI.

Thereafter, the Wi-Fi location determination server may extract (or analyze) the location information of the mobile terminal 310 using at least one wireless AP information extracted from the database. By comparing the included information with the received wireless AP information, the location information of the mobile terminal 310 is extracted (or analyzed).

As a method for extracting (or analyzing) the location information of the mobile terminal 310, the Cell-ID method, the fingerprint method, the triangulation method, and the landmark method can be used.

The Cell-ID method is a method of determining the location of the wireless AP with the strongest signal strength among the surrounding wireless AP information collected by the mobile terminal as the location of the mobile terminal. It has the advantage of being simple to implement, requiring no additional costs, and being able to obtain location information quickly, but has the disadvantage of lowering the positioning accuracy if the installation density of wireless APs is low.

The fingerprint method is a method of collecting signal strength information by selecting a reference location in the service area and estimating the location through signal strength information transmitted from the mobile terminal based on the collected information. In order to use the fingerprint method, it is necessary to create a database of radio wave characteristics in advance.

The triangulation method is a method of calculating the location of a mobile terminal based on the distance between the coordinates of at least three wireless APs and the mobile terminal. To measure the distance between a mobile terminal and a wireless AP, the signal strength is converted into distance information, or the time the wireless signal is transmitted (Time of Arrival, ToA), and the time difference between the signal is transmitted (Time Difference of Arrival, TDoA). , the angle at which the signal is transmitted (Angle of Arrival, AoA), etc. can be used.

The landmark method is a method of measuring the location of a mobile terminal using a landmark transmitter whose location is known.

In addition to the methods listed, various algorithms can be used as a method to extract (or analyze) location information of a mobile terminal.

The location information of the mobile terminal 310 extracted in this way is transmitted to the mobile terminal 310 through the Wi-Fi location determination server, so that the mobile terminal 310 can obtain the location information.

The mobile terminal 310 can obtain location information by connecting to at least one wireless AP. At this time, the number of wireless APs required to obtain location information of the mobile terminal 310 may vary depending on the wireless communication environment in which the mobile terminal 310 is located.

Figure 5 is a diagram for explaining various types of cleaning robots according to an embodiment of the present invention.

As shown in Figure 5, cleaning robots according to an embodiment of the present invention can be classified according to the object of cleaning. For example, there may be a first cleaning robot 510 whose main task is cleaning the floor, such as wiping dry floors or cleaning water. Additionally, there may be a second cleaning robot 520 whose main task is to suck up dust or small foreign substances. Furthermore, there may be a third cleaning robot 530 whose main task is to pick up large trash or large foreign substances. Various cleaning tasks may be required within the airport, and the first cleaning robot 510, the second cleaning robot 520, and the third cleaning robot 530 may be placed in the right place.

Figures 6 and 7 are diagrams to explain an example in which a server manages cleaning robots and CCTV images installed in an airport according to an embodiment of the present invention.

As shown in FIG. 6, a plurality of CCTVs 611, 612, and 613 may be randomly placed within the airport. A plurality of CCTVs (611, 612, 613) can capture the situation within the airport in real time. Additionally, a plurality of CCTVs 611, 612, and 613 may transmit real-time captured image data to at least one airport robot 601, 602, 603, 604, 605, and 606. At least one airport robot (601, 602, 603, 604, 605, 606) can search for cleaning objects in various places within the airport using video data received from CCTV. Additionally, a plurality of CCTVs 611, 612, and 613 may transmit real-time captured image data to the server 620. In addition, at least one airport robot (601, 602, 603, 604, 605, 606) can receive real-time video data captured by a specific CCTV from the server.

Also, as shown in FIG. 7, the manager uses various data received by the server 620 to operate a plurality of CCTVs (611, 612, 613) and at least one airport robot (601, 602, 603, 604, 605, 606) can be managed. For example, a manager can discover a cleaning object in video data captured by a specific CCTV. Additionally, the manager may select a robot that can process the cleaning object among the first cleaning robot 510, the second cleaning robot 520, and the third cleaning robot 530. Additionally, the manager may transmit a message to the cleaning robot located closest to the cleaning object to clean the cleaning object. Additionally, the manager can control video data captured by the specific CCTV to be transmitted to the cleaning robot. The cleaning robot that receives the video data captured by the specific CCTV from the server can discover the cleaning object in the video data captured by the specific CCTV. Additionally, the cleaning robot can detect information about the location where a specific CCTV is filming. Then, the cleaning robot can move to the relevant location and clean the cleaning object.

Figures 8 to 10 are diagrams to explain an example in which a user in an airport photographs a cleaning object and transmits it to a server.

The cleaning robots 510, 520, and 530 according to an embodiment of the present invention can independently detect various cleaning objects within the airport. Additionally, the cleaning robots 510, 520, and 530 may receive cleaning object information from the server 830 or the manager. As described above, the server 830 can sense cleaning object information through video data captured by CCTV. In addition, the server 830 can sense cleaning object information through images transmitted by the airport user 800.

For example, as shown in FIG. 8 , an airport user 800 may discover a cleaning object 820 while moving within the airport. In this case, the airport user 800 can photograph the cleaning object 820 through his or her mobile terminal 810. Additionally, the airport user 800 can directly transmit the captured image to the server 830. Alternatively, as shown in FIG. 9 , the airport user 800 may run the airport management application 815 on the mobile terminal 810. Additionally, the airport user 800 may upload a captured image of the cleaning object 820 to the airport management application 815 to request cleaning. As shown in FIG. 10 , the airport management application 815 may transmit a captured image of the cleaning object 820 to the server 830 . Additionally, the server 830 or the manager of the server 830 may discover the cleaning object 820 from the transmitted captured image. In addition, the server 830 or the manager of the server 830 may select a robot that can process the cleaning object among the first cleaning robot 510, the second cleaning robot 520, and the third cleaning robot 530. . Additionally, the server 830 or the manager of the server 830 may transmit a message to the cleaning robot located closest to the cleaning object 820 to clean the cleaning object. Additionally, the server 830 or the manager of the server 830 may control the image captured by the airport user 800 to be transmitted to the cleaning robot. The cleaning robot that receives the image taken by the airport user 800 from the server 830 can discover the cleaning object 820. And, the cleaning robot can detect information about the shooting location. Then, the cleaning robot can move to the relevant location and clean the cleaning object.

11 and 12 are diagrams for explaining an example in which a management robot calls a cleaning robot according to an embodiment of the present invention.

Cleaning robots 510, 520, and 530 may be deployed in certain areas within the airport. Cleaning robots 510, 520, and 530 may move a predetermined route and clean cleaning objects. In this case, an event may occur in which the cleaning robots 510, 520, and 530 miss the cleaning object or the cleaning robots 510, 520, and 530 do not pass a specific area. Therefore, in order to prevent such events, management robots that manage the cleaning robots 510, 520, and 530 may be placed in certain areas within the airport. Additionally, the management robot can wander around a certain area within the airport and discover objects to be cleaned. Additionally, the management robot may select a cleaning robot that can process the cleaning object among the first cleaning robot 510, the second cleaning robot 520, and the third cleaning robot 530. Additionally, the management robot may call the cleaning robot located closest to the cleaning object. Additionally, the management robot may transmit a message to the called cleaning robot to clean the corresponding cleaning object.

For example, as shown in FIG. 11, the management robot 1100 may wander around a certain area within the airport and discover a cleaning object 1120. In addition, the management robot 1100 may select the cleaning robot 1110 that can process the cleaning object 1120 among the first cleaning robot 510, the second cleaning robot 520, and the third cleaning robot 530. there is. Additionally, the management robot 1100 may call the cleaning robot 1110 located closest to the cleaning object 1120. And as shown in FIG. 12, the cleaning robot 1110 can come within a predetermined range from the cleaning object 1120. Additionally, the management robot 1100 may transmit a message to the cleaning robot 1110 to clean the corresponding cleaning object 1120.

Figures 13 to 15 are diagrams to explain an example in which a cleaning robot organizes a trash can using Internet of Things technology according to an embodiment of the present invention.

Internet of Things technology can be applied to airport facilities where cleaning robots according to an embodiment of the present invention are deployed. The term Internet of Things (IoT) first appeared in 1998 at the Auto-ID Lab at the Massachusetts Institute of Technology (MIT). The Internet of Things can refer to an environment where everyday objects are connected through wired and wireless networks to share information. The Internet of Things is a more advanced stage than the Internet or mobile Internet based on existing wired communications, and devices connected to the Internet can exchange and process information with each other without human intervention. It is similar to existing ubiquitous or M2M (Machine to Machine: Machine-to-Machine) communication in that objects communicate without relying on humans, but the concept of M2M, which primarily aims at communicating between communication equipment and people, is extended to the Internet. Therefore, it can be said to be an evolved concept that interacts not only with objects but also with all information in the real and virtual world. Technical elements for implementing IoT services include 'sensing technology' that obtains information from tangible objects and the surrounding environment, 'wired and wireless communication and network infrastructure technology' that supports objects to connect to the Internet, and suitable for various service fields and types. The key is 'service interface technology', which processes and processes information or integrates various technologies, and there is 'security technology' to prevent hacking or information leakage of Internet of Things components such as large amounts of data.

For example, as shown in FIG. 13, the cleaning robot 1300 according to an embodiment of the present invention can transmit/receive data to/from the trash can 1310. At this time, the trash can 1310 may transmit a request message to the cleaning robot 1300 to clean the trash can. Or as shown in Figure 14. The cleaning robot 1300 and the trash can 1310 can send/receive data messages through the server 1330. That is, the trash can 1310 may transmit a request message to the server 1330 to clean out the trash can. Additionally, the server 1330 can detect the cleaning robot 1300 that can clean the trash can among cleaning robots around the trash can 1310. Additionally, the server 1330 may transmit a message requesting to clean the trash can 1310 to the cleaning robot 1300. And as shown in FIG. 15 , the cleaning robot 1300 that receives a message from the trash can 1310 or the server 1330 can move the trash can 1310 to the loading dock 1320. And, the cleaning robot 1300 can organize the trash can 1310.

Figures 16 to 19 are diagrams to explain an example in which a user controls a cleaning robot using a remote control device according to an embodiment of the present invention.

It may include a remote control device 1610 for remotely controlling the operation of the cleaning robot 1600 according to an embodiment of the present invention. The remote control device 1610 may be the mobile terminal 310 described above. Additionally, in order for the remote control device 200 to control the operation of the cleaning robot 1600, accurate location information of the cleaning robot 1600 may be required. Specifically, mapping between real areas and virtual areas may be necessary. To solve this problem, the cleaning robot 1600 is detected based on image information generated using the camera of the remote control device 1610 for controlling the cleaning robot 1600 according to an embodiment of the present invention. (1600) location information can be generated.

First, image information can be generated by photographing the cleaning robot 1600 and the surrounding area of the cleaning robot 1600, as shown in FIG. 16, using a camera provided on the rear of the body of the remote control device 1610. Image information of the cleaning robot 1600 generated in this way can be transmitted to the server.

Next, the remote control device 1610 maps the actual area where the cleaning robot 1600 is located and the virtual area represented by the image information based on the image information. This can be said to be the remote control device 1610 detecting the cleaning robot 1600. A recognition mark may be provided on the upper surface of the cleaning robot 1600 to enable the cleaning robot 1600 to be detected using an external device. Here, the shape of the recognition mark is not particularly limited. For example, the recognition mark may be formed in a circular shape, and in this case, the width (W) and height (H) of the recognition mark may have the same value.

The actual shape of the recognition mark may be stored in the memory of the remote control device 1610. That is, information including the actual size of the recognition mark may be stored. The server can detect the cleaning robot 1600 by extracting a recognition mark from the image of the video information and determining whether the extracted recognition mark is a valid recognition mark. Here, in order to extract the recognition mark from the video information image, the server may check, for example, the shape or pattern of the recognition mark or the specific color of the recognition mark.

However, when the image quality of the video information is poor, it is difficult for the server to detect the recognition mark. Accordingly, the server can improve the quality of the image of the video information before extracting the recognition mark from the image of the video information.

Here, the server may perform, for example, image brightness adjustment, noise removal, color correction, etc. to improve the image quality of the image information.

Next, the remote control device 1610 may generate location information of the cleaning robot 1600 by comparing the actual shape information of the recognition mark and the relative shape identified by the image information. That is, mapping between the real area and the virtual area can be performed.

As described above, the remote control device 1610 generates image information about the cleaning robot 1600 and its surrounding area, and detects the cleaning robot 1600 based on this image information to generate location information. . In addition, the display unit of the remote control device 1610 outputs image information about the cleaning robot 1600 and its surrounding area. In particular, the server displays the cleaning robot 1600 on the screen of the display unit based on the location information of the cleaning robot 1600. ) can be specified as selectable. Accordingly, the user can control the driving of the cleaning robot 1600 while observing the current state of the cleaning robot 1600 through the display unit. Meanwhile, in order to control the operation of the cleaning robot 1600, the remote control device 1610 may further include an input unit for inputting a control signal of the cleaning robot 1600. A microphone, a user input unit, and a display unit equipped with a touch sensor that detects touch input may function as an input unit.

When a control signal for the cleaning robot 1600 is input from the remote control device 1610, the wireless communication unit of the remote control device 1610 may transmit the control signal to the wireless communication unit of the cleaning robot 1600. And, the cleaning robot 1600 can be driven according to control signals.

For example, the cleaning area of the cleaning robot 1600 can be set through a touch input. As shown in FIG. 17, if a virtual area (A) is designated on the screen of the display unit, the server can designate the actual area (A') of the indoor space corresponding to the area (A) as the cleaning area. Accordingly, the cleaning robot 1600 can move to the cleaning area and perform cleaning.

That is, the virtual area appearing on the display unit can be mapped to the real area, and the actual movement coordinates can be transmitted to the cleaning robot 1600. For example, assuming that the virtual area (A) specified in the display unit is an area 5 centimeters to the right from the center of the vacuum cleaner in the display unit, the actual area (A') is mapped to an area 1 meter to the right from the center of the actual vacuum cleaner. It can be. Of course, as described above, this mapping can be performed through mapping information that compares the shape and size of the actual recognition means with the shape and size of the relative recognition means through image information. Accordingly, mapping information that maps the virtual area to the real area is transmitted to the cleaning robot 1600, and the cleaning robot 1600 can perform cleaning based on this.

That is, in one embodiment of the present invention, the cleaning area can be simply set through a touch input using an image of an indoor space (i.e., a virtual area) displayed on the display unit. As a result, cleaning can be performed only on the area desired by the user, shortening cleaning time and reducing electricity usage.

Additionally, the user 1810 can control the movement of the cleaning robot 1800 through voice input through a microphone. For example, the user 1810 may call the cleaning robot 1800 to the user's 1810 location through voice input. As shown in FIG. 18, when the user 1810 inputs a preset mant into the microphone of the remote control device 1610, the cleaning robot 1800 determines the location of the remote control device 1610 based on the location information, That is, it is possible to move to the location of the user 1810.

As described above, it can be seen that wireless control of the robot vacuum cleaner is possible through the display unit through mapping between the real area and the virtual area. Therefore, once the mapping is performed, there is no need for the user to look directly at the actual area or the cleaning robot. In other words, once mapping is performed, wireless control of the robot vacuum cleaner is possible through the display unit even if the user moves to another area. Of course, in this case, wireless communication between the remote control device 1610 and the cleaning robot 1800 must be maintained.

Generally, the area cleaned by the cleaning robot can be fixed. In this case, detection of the cleaning robot 1600 or mapping of the real area and the virtual area through the remote control device 1610 may be continuously maintained. In other words, it is possible to use a single detection or mapping as is. Therefore, there may be no need to film the cleaning robot every time it is wirelessly controlled. That is, conventionally captured image information can be stored and displayed on the display unit each time wireless control is performed.

Afterwards, wireless control can be repeatedly performed through the virtual area displayed on the display unit. In this case, if there is an environment where wireless communication between the cleaning robot 1600 and the remote control device 1610 is possible, the user can wirelessly control the cleaning robot 1600 from outside with the remote control device 1610.

For example, the cleaning robot 1600 may be connected to wireless communication through a Wi-Fi AP within the airport, and the user's remote control device 1610 may be connected to the cleaning robot 1600 through wireless communication through a server and the Wi-Fi AP. Accordingly, users can wirelessly control cleaning robots placed in airport lobbies, etc., from an external office, etc., rather than the airport lobby.

Using FIG. 19, we will explain how to remotely control a cleaning robot with a remote control device such as a mobile terminal.

In the airport cleaning robot remote control system including a cleaning robot 1600 and a remote control device 1610 having a camera and a display unit, image information about the cleaning robot 1600 and its surrounding area is generated through the camera. It may include a step (S1920) and a step (S1930) of detecting the cleaning robot 1600 based on image information. In addition, a step of generating location information of the cleaning robot 1600 (S1940) may be included. In other words, a step of mapping the real area and the virtual area can be performed through the generated image information.

Meanwhile, a step of outputting image information and location information to the display unit (S1950) may be included, and this step may be performed by generating image information. Afterwards, a step of providing location-based services based on location information (S1960) may be performed.

Figure 20 is a block diagram showing the configuration of an airport cleaning robot according to an embodiment of the present invention. To summarize the airport assistance robot according to an embodiment of the present invention described in FIGS. 5 to 19, the block diagram shown in FIGS. 1 and 2 can be simplified to FIG. 20.

The airport cleaning robot 2000 according to an embodiment of the present invention may include a communication unit 2040 that transmits and receives data. And the airport cleaning robot 2000 may include a location recognition unit 2010 that reads location information. And the airport cleaning robot (2000),

It may include a driving unit 2020 that performs cleaning and movement. Additionally, the airport cleaning robot 2000 may include a camera 2030 that photographs the exterior of an object. And the airport cleaning robot 2000 may include a control unit 2050. And upon receiving the cleaning request message, the control unit 2050 can move to the cleaning location by reading the location information included in the cleaning request message. And the control unit 2050 can read the first cleaning object information included in the cleaning request message. And the control unit 2050 can scan the cleaning object at the cleaning location. And the control unit 2050 can read the second cleaning object information using the scanned image of the cleaning object. And the control unit 2050 may determine whether the first cleaning object information and the second cleaning object information match. And if the control unit 2050 determines that there is a match, it can control the cleaning object to be cleaned. The first cleaning object information and the second cleaning object information may include a cleaning method, object size, external image of the object, and surrounding image information of the object. A cleaning request message may be received from a management robot deployed at the airport. A cleaning request message may be received from an airport cleaning management server. A cleaning request message may be received from an airport cleaning application.

A computer-readable storage medium containing computer-executable instructions for execution by a processing system according to an embodiment of the present invention, wherein the computer-executable instructions for controlling an airport cleaning robot include: Commands for scanning, commands for determining a type of the cleaning object, commands for determining a type of cleaning robot corresponding to the determined type of cleaning object, commands for determining a position of the cleaning object, and the determined cleaning robot. Among the cleaning robots that match the type, commands for transmitting a cleaning request message to the cleaning robot closest to the location of the cleaning object may be included. The commands for scanning the cleaning object may include commands for receiving CCTV image data and commands for scanning the cleaning object in the received CCTV image data. The commands for scanning the cleaning object may include commands for receiving image data from an airport cleaning application and commands for scanning the cleaning object in the received image data. The types of cleaning robots include a first type of cleaning robot whose main task is to wipe the floor, a second type of cleaning robot whose main task is to suck foreign substances of less than a predetermined size, and removal of foreign substances of a predetermined size or larger. It may include a third type of cleaning robot that performs its main mission. The computer-executable instructions may further include instructions for transmitting a cleaning request message to a plurality of robots according to distance priority when the cleaning object is larger than a predetermined size.

An airport cleaning robot according to another embodiment of the present invention may include a main body 2000 that forms the exterior, a driving unit 2020 that moves and cleans, and a communication unit 2040 that transmits and receives data with a remote control device. there is. In order to provide location-based services for the airport cleaning robot through a remote control device, the main body 2000 maps the actual area where the airport cleaning robot is located and the virtual area displayed on the display unit of the remote control device. A means of recognition may be provided. The recognition means may include a recognition mark provided on the outer peripheral surface of the main body 2000. The recognition mark may be provided on the outer peripheral surface of the main body 2000 to be selectively removable. Recognition means may be provided at multiple locations in the main body. Location-based services can be performed through touch input on the airport cleaning robot and surrounding image information of the airport cleaning robot displayed on the display unit of the remote control device. The location-based service may include at least one of a function for setting a movement path of the robot cleaner, a function for specifying a cleaning area, a function for specifying a no-cleaning area, and a function for setting access to a user's location.

A computer-readable storage medium containing computer-executable instructions for execution by a processing system according to an embodiment of the present invention, wherein the computer-executable instructions for controlling an airport cleaning robot include: Among commands to scan, commands to determine the type of the cleaning object, commands to determine the type of cleaning robot corresponding to the type of the cleaning object, and cleaning robots matching the determined type of the cleaning robot, the cleaning It may include commands that call the cleaning robot closest to the location of the object. The computer-executable instructions may include instructions for transmitting a message instructing the called cleaning robot to clean the cleaning object.

A computer-readable storage medium containing computer-executable instructions for execution by a processing system according to another embodiment of the present invention, wherein the computer-executable instructions for controlling an airport cleaning robot are transmitted from a server. It may include commands for receiving a cleaning request message, commands for determining the location of the cleaning object included in the cleaning request message, and commands for transmitting the cleaning request message to the cleaning robot closest to the location of the cleaning object. . The cleaning request message may be transmitted to the server through an airport cleaning application or MMS (Multimedia messaging service) message.

The above-described present invention can be implemented as computer-readable code on a program-recorded medium. Computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. It also includes those implemented in the form of carrier waves (e.g., transmission via the Internet). Additionally, the computer may include the AP 150 of the airport robot. Accordingly, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (20)

In the airport cleaning robot,
A communication unit that transmits and receives data;
A location recognition unit that reads location information;
A driving unit that performs cleaning and movement;
A camera that captures the exterior of an object; and
It includes a control unit that controls the operation of the airport cleaning robot,
Upon receiving a cleaning request message received through the communication unit,
The control unit,
Move to the cleaning location by reading the location information included in the cleaning request message,
Read the first cleaning object information included in the cleaning request message,
Scanning a cleaning object through the camera at the cleaning location,
Reading second cleaning object information from the image of the scanned cleaning object,
Determine whether the first cleaning object information and the second cleaning object information read from the cleaning request message match, and if the first cleaning object information and the second cleaning object information match, control to clean the cleaning object. do,
Each of the first cleaning object information and the second cleaning object information includes object size, an external image of the object, and surrounding image information of the object,
The control unit
When the object size, external image of the object, and surrounding image information of the object included in each of the first cleaning object information and the second cleaning object information match each other, controlling to clean the cleaning object.
Airport cleaning robot. delete According to claim 1,
The cleaning request message is received from a management robot deployed at the airport,
Airport cleaning robot.
According to claim 1,
The cleaning request message is received from the airport cleaning management server,
Airport cleaning robot.
According to claim 1,
The cleaning request message is received from the airport cleaning application,
Airport cleaning robot.
delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images