KR20180074404A - Robot for airport and method thereof - Google Patents

Abstract

The present invention relates to an airport robot and an operation method thereof. The airport robot according to an embodiment of the present invention comprises: a display unit; a user interface unit for receiving an input from a user; a position recognition unit for sensing a position of a human body and an object; and a control unit for controlling an operation of the airport robot. The control unit senses an operation of the user through a motion sensor, interprets a sensed signal, and recognizes the sensed signal as an individual operation corresponding to a situation. In addition, the control unit controls the recognized individual operation to perform the operation corresponding to the situation and application.

Description

[0001] ROBOT FOR AIRPORT AND METHOD THEREOF [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot disposed at an airport and a method of operating the same, and more particularly, to a guide robot for an airport and an operation method thereof, which are disposed at an airport and provide guidance to users.

What distinguishes a robot from an existing information device is that it has an operating device such as a moving device capable of moving, a robotic arm capable of lifting the object, and a mechanism capable of adjusting the angle of the built-in camera. These actuators evolved from industrial and military robotics.

Generally, industrial and military robots operate according to pre-written programs, so after the start of operation, it is difficult to control the operation to interact with the user. However, service robots for human convenience are required to perform intelligent tasks and to interact with users in order to adapt to various needs of users. In addition, since recent service robots have various functions similar to those of a computer, it is necessary to have an interactive method similar to GUI provided by computer software. Therefore, various user interface devices capable of performing the interaction between the user and the service robot are devised and installed in the service robot.

An interface device between a robot and a user can be largely divided into an output device, an input device, and a bidirectional device. The output device mainly output information by using video and audio. The video output device requires a lot of space inside the robot to be mounted on the robot, and there is a problem that the power consumption is large and the distance from the user is close to use the output image.

The voice output apparatus has advantages in terms of space and power consumption as compared with the video output apparatus, and has recently been adopted as a function of transmitting voice information to the user by utilizing voice synthesis technology. Recently, when a user inputs an image through a camera, the user can recognize the user's face and motion using the image recognition software. However, due to the nature of the camera, there are spatial and optical constraints in the input range, which makes it inconvenient for the user to be located within the range of the camera and there is a technical limitation in application of the image recognition technology to the dynamic environment.

Speech recognition software and a voice synthesizer (TTS) can be utilized when a voice is input using a microphone. However, there is a problem that it is difficult to expect a natural conversation due to the problem of misrecognition although the interaction between the robot and the user can be performed by communicating commands or communicating with the user.

The robot can database the behaviors of the robot corresponding to various emotional expressions and then process the moving image input by the image camera in the motion detection method to determine the behavior of the user. Then, the emotion can be expressed by moving the head, body, arm, and expression of the robot. Although the robot can express various behavior patterns, there is a problem that the robot does not express the behavior pattern corresponding to the user's behavior. Therefore, in order to grasp the user's behavior, the user's behavior style should be grasped by various vision technologies and a corresponding standardized and general emotional expression method should be provided.

An object of the present invention is to provide a robot that analyzes a user's operation and automatically performs an appropriate response in accordance with a user's request.

Another object of the present invention is to enable emotional interaction between an airport user and an airport robot, not merely information transmission.

The robot for an airport according to the present invention can detect an operation of a user with an operation sensor and match an operation of the detected user with a plurality of user messages stored in a database, thereby accurately grasping the intention of the user.

The airport robot according to the present invention can store various emotional emoticons outputted in a specific situation in a database and suitably output the emotional emoticon corresponding to the user and the situation to the display unit.

The airborne robot according to the present invention analyzes an operation of a user and provides an appropriate GUI (Graphic User Interface) in accordance with a request of a user, thereby providing a more efficient interface with a user.

The airport robot according to the present invention displays various emotion emotions suitable for the current state and the user, thereby improving the user's interaction satisfaction.

1 is a block diagram showing a hardware configuration of an airport robot according to an embodiment of the present invention.
FIG. 2 is a detailed view showing the configuration of a micom and AP of an airport robot according to another embodiment of the present invention.
3 is a view for explaining the structure of an airport robot system according to an embodiment of the present invention.
4 is a view for explaining an example in which the airport robot detects a human body or an object at predetermined distances according to an embodiment of the present invention.
5 is a block diagram schematically showing an operation-based interface system of an airport robot according to an embodiment of the present invention.
6 is a block diagram showing the configuration of an operation signal input unit according to an embodiment of the present invention.
7 is a block diagram showing a configuration of an operation recognition unit, a command analysis unit, and a situation management unit according to an embodiment of the present invention.
8 is a diagram showing a configuration of an airport robot application software according to an embodiment of the present invention.
9 is a flowchart schematically illustrating an operation-based interface method according to an embodiment of the present invention.
10 is a hardware block diagram of an airport robot according to another embodiment of the present invention.
11 is a block diagram showing a method of expressing emotions of an airport robot according to another embodiment of the present invention.
FIG. 12 is a chart showing an example of behaviors corresponding to intimacy between airport robots according to another embodiment of the present invention.
FIG. 13 is a flowchart of a method in which an airport robot determines the behavior of a user according to the present invention and expresses emotions by moving the body.
14 is a configuration diagram of an airport robot for expressing emotions in the body by judging the behavior of the user.
15 is a view for explaining an example in which an airport robot according to an embodiment of the present invention outputs emotional emoticons to a display unit.

Hereinafter, embodiments related to the present invention will be described in detail with reference to the drawings.

The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

1 is a block diagram showing a hardware configuration of an airport robot according to an embodiment of the present invention.

As shown in FIG. 1, the hardware of the airport robot 100 according to an embodiment of the present invention may be composed of a Micom group and an AP group. The microcomputer 110 may include a microcomputer 110, a power source 120, an obstacle recognition unit 130, and a driving unit 140. The AP group may include an AP 150, a user interface unit 160, an object recognition unit 170, a location recognition unit 180, and a LAN 190. The user interface unit 160 may be referred to as a communication unit.

The microcomputer 110 may manage a driving unit 120 including a battery or the like among the hardware of the airport robot, an obstacle recognizing unit 130 including various sensors, and a driving driving unit 140 including a plurality of motors and wheels .

The power supply unit 120 may include a battery driver 121 and a lithium-ion battery 122. The battery driver 121 can manage the charging and discharging of the lithium-ion battery 122. The lithium-ion battery 122 can supply power for driving the airport robot. The lithium-ion battery 122 can be constructed by connecting two 24V / 102A lithium-ion batteries in parallel.

The obstacle recognizing unit 130 may include an IR remote control receiver 131, a USS 132, a Cliff PSD 133, an ARS 134, a Bumper 135, and an OFS 136. The IR remote control receiving unit 131 may include a sensor for receiving a signal of an IR (infrared) remote control for remotely controlling the airport robot. USS (Ultrasonic sensor) 132 may include a sensor for determining the distance between the obstacle and the airport robot using ultrasonic signals. The Cliff PSD 133 may include sensors for detecting cliffs or cliffs in an airport robot traveling range 360 degrees in all directions. The ARS (Attitude Reference System) 134 may include a sensor capable of detecting the attitude of the airport robot. The ARS 134 may include a sensor consisting of three axes of acceleration and three axes of acceleration for detecting the amount of rotation of the airport robot. The bumper 135 may include a sensor that detects a collision between the airport robot and the obstacle. The sensor included in the bumper 135 can detect a collision between the airport robot and the obstacle in the range of 360 degrees. The OFS (Optical Flow Sensor) 136 may include a sensor for measuring the traveling state of an airport robot and a sensor for measuring the distance traveled by the airport robot on various floor surfaces.

The travel driving unit 140 includes a motor driver 141, a wheel motor 142, a rotation motor 143, a main brush motor 144, a side brush motor 145 and a suction motor 146 . The motor driver 141 may serve to drive a wheel motor, a brush motor, and a suction motor for traveling and cleaning the airport robot. The wheel motor 142 may drive a plurality of wheels for driving the airport robot. The rotation motor 143 may be driven for left-right rotation, up-down rotation of the main body of the airport robot or the head portion of the airport robot, or for turning or rotating the wheels of the airport robot. The main brush motor 144 can drive a brush that scavenges dirt from the airport floor. The side brush motor 145 can drive a brush for sweeping dirt in the area around the outer surface of the airport robot. The suction motor 146 can be driven to suck the dirt on the airport floor.

The AP (Application Processor) 150 can function as a central processing unit for managing the entire system of hardware modules of the airport robot. The AP 150 can drive an application program for traveling and user input / output information to the microcomputer 110 by using position information received through various sensors, thereby driving a motor or the like.

The user interface unit 160 includes a user interface processor 161, an LTE router 162, a WIFI SSID 163, a microphone board 164, a barcode reader 165, a touch monitor 166, And a speaker 167. The user interface processor 161 can control the operation of the user interface unit responsible for user input and output. The LTE router 162 can receive the necessary information from the outside and can perform LTE communication for transmitting information to the user. The WIFI SSID 163 can perform position recognition of a specific object or an airport robot by analyzing the signal strength of WiFi. The microphone board 164 receives a plurality of microphone signals, processes the voice signal into voice data, which is a digital signal, and analyzes the direction of the voice signal and the voice signal. The barcode reader 165 can read the barcode information written in the plurality of tickets used in the airport. The touch monitor 166 may include a touch panel configured to receive user input and a monitor for displaying output information. The speaker 167 can play a role of notifying the user of specific information by voice.

The object recognition unit 170 may include a 2D camera 171, an RGBD camera 172, and a recognition data processing module 173. The 2D camera 171 may be a sensor for recognizing a person or an object based on a two-dimensional image. For detecting a person or object using captured images having depth data obtained from a camera having RGBD sensors or other similar 3D imaging devices as RGBD cameras (Red, Green, Blue, Distance, 172) Sensor. The recognition data processing module 173 can process a signal such as a 2D image / image or 3D image / image obtained from the 2D camera 171 and the RGBD camera 172 to recognize a person or an object.

The position recognition unit 180 may include a stereo board 181, a lidar 182, and a SLAM camera 183. A SLAM camera (Simultaneous Localization And Mapping camera, 183) can implement simultaneous location tracking and mapping techniques. The airport robot can detect the surrounding information by using the SLAM camera 183 and process the obtained information to create a map corresponding to the task execution space and estimate its own absolute position. Light Detection and Ranging (Lidar) 182 is a laser radar, which can be a sensor that irradiates a laser beam and collects and analyzes backscattered light among light absorbed or scattered by an aerosol to perform position recognition. The stereo board 181 processes and processes sensing data collected from the rider 182 and the SLAM camera 183 to manage the data for recognition of the position of an airport robot and recognition of obstacles.

The LAN 190 may communicate with the user input / output related user interface processor 161, the recognition data processing module 173, the stereo board 181 and the AP 150.

FIG. 2 is a detailed view showing the configuration of a micom and AP of an airport robot according to another embodiment of the present invention.

As shown in FIG. 2, the microcomputer 210 and the AP 220 may be implemented in various embodiments to control the recognition and behavior of the airport robot.

As one example, the microcomputer 210 may include a data access service module 215. The data access service module 215 includes a data acquisition module 211, an emergency module 212, a motor driver module 213 and a battery manager module 214, . ≪ / RTI > The data acquisition module 211 can acquire sensed data from a plurality of sensors included in the airport robot and transmit the sensed data to the data access service module 215. The emergency module 212 is a module capable of detecting an abnormal state of the airport robot. When the airport robot performs a predetermined type of action, the emergency module 212 detects that the airport robot has entered an abnormal state . The motor driver module 213 can manage driving control of a wheel, a brush, and a suction motor for driving and cleaning the airport robot. The battery manager module 214 may charge and discharge the lithium-ion battery 122 of FIG. 1 and may transmit the battery state of the airport robot to the data access service module 215.

The AP 220 receives various types of cameras and sensors, user input, etc., and performs recognition and processing to control the operation of the airport robot. The interaction module 221 synthesizes the recognition data received from the recognition data processing module 173 and the user input received from the user interface module 222 to collectively manage software that allows the user and the airport robot to interact with each other Lt; / RTI > The user interface module 222 receives a user's close command such as a display unit 223, a key, a touch screen, a reader, or the like, which is a monitor for present state of the airport robot and operation / Such as a remote control signal for remote control, or a user input received from a user input 224 that receives a user's input signal from a microphone or bar code reader. When at least one user input is received, the user interface module 222 may pass user input information to a state machine module 225. The state management module 225 receiving the user input information can manage the entire state of the airport robot and issue an appropriate command corresponding to the user input. The planning module 226 can determine the start and end points / actions for the specific operation of the airport robot in accordance with the command received from the state management module 225, and calculate the route to which the airport robot should move. The navigation module 227 is responsible for the overall running of the airport robot, and may cause the airport robot to travel according to the travel route calculated by the planning module 226. [ The motion module 228 can perform the operation of the basic airport robot in addition to the traveling.

In addition, the airport robot according to another embodiment of the present invention may include a position recognition unit 230. [ The position recognition unit 230 may include a relative position recognition unit 231 and an absolute position recognition unit 234. [ The relative position recognition unit 231 can calculate the amount of movement of the airport robot for a predetermined period of time by correcting the amount of movement of the airport robot through the RGM mono sensor 232 and recognize the current environment of the airport robot through the LiDAR 233 can do. The absolute position recognition unit 234 may include a Wifi SSID 235 and a UWB 236. [ The Wifi SSID 235 is a UWB sensor module for recognizing the absolute position of the airport robot, and is a WIFI module for estimating the current position by detecting the Wifi SSID. The Wifi SSID 235 can recognize the position of the airport robot by analyzing the signal strength of the Wifi. The UWB 236 can calculate the distance between the transmitter and the receiver and sense the absolute position of the airport robot.

In addition, the airport robot according to another embodiment of the present invention may include a map management module 240. The map management module 240 may include a grid module 241, a path planning module 242, and a map partitioning module 243. The grid module 241 can manage a grid-shaped map generated by the airport robot through the SLAM camera, or map data of the surrounding environment for the position recognition input to the airport robot in advance. The path planning module 242 can take charge of the travel path calculation of the airport robots in the map division for cooperation among the plurality of airport robots. In addition, the path planning module 242 can calculate a traveling route through which the airport robot should move in an environment where one airport robot operates. The map division module 243 can calculate the area to be managed by the plurality of airport robots in real time.

The data sensed and calculated from the position recognition unit 230 and the map management module 240 may be transmitted to the state management module 225 again. The state management module 225 can command the planning module 226 to control the operation of the airport robot based on the data sensed and calculated from the position recognition module 230 and the map management module 240. [

3 is a view for explaining a structure of an airport robot system according to an embodiment of the present invention.

The airport robot system according to an embodiment of the present invention may include a mobile terminal 310, a server 320, an airport robot 300, and a camera 330.

The mobile terminal 310 can transmit and receive data to and from the server 320 in the airport. For example, the mobile terminal 310 may receive airport-related data such as flight time schedules, airport maps, and the like from the server 320. The user can obtain the necessary information from the server 320 through the mobile terminal 310 and obtain it. In addition, the mobile terminal 310 can transmit data such as a photograph, a moving picture, a message, and the like to the server 320. For example, the user may request the cleaning of a corresponding area by sending a photograph of a missing person to the server 320, receiving a photograph of the missing person, photographing a photograph of the area requiring cleaning in the airport, and transmitting the photograph to the server 320.

In addition, the mobile terminal 310 can exchange data with the airport robot 300.

For example, the mobile terminal 310 may transmit a signal for calling the airport robot 300, a signal for requesting to perform a specific operation or an information request signal to the airport robot 300. The airport robot 300 may move to the position of the mobile terminal 310 or perform an operation corresponding to the command signal in response to the paging signal received from the mobile terminal 310. [ Or the airport robot 300 may transmit the data corresponding to the information request signal to the mobile terminal 310 of each user.

Next, the airport robot 300 can perform patrol, guidance, cleaning, disinfection, and transportation within the airport.

The airport robot 300 can transmit / receive signals to / from the mobile terminal 310 or the server 320. For example, the airport robot 300 can transmit and receive signals including the server 320 and the information on the situation in the airport. In addition, the airport robot 300 can receive image information of each area of the airport from the camera 330 in the airport. Therefore, the airport robot 300 can monitor the status of the airport by synthesizing the image information captured by the airport robot 300 and the image information received from the camera 330.

The airport robot 300 can receive commands directly from the user. For example, it is possible to directly receive a command from a user through an input or a voice input touching a display unit provided in the airport robot 300. The airport robot 300 may perform operations such as patrol, guidance, and cleaning according to a command received from the user, the mobile terminal 310, or the server 320 and the like.

Next, the server 320 can receive information from the mobile terminal 310, the airport robot 300, and the camera 330. The server 320 may collectively store and manage the information received from the respective devices. The server 320 may transmit the stored information to the mobile terminal 310 or the airport robot 300. In addition, the server 320 may transmit a command signal to each of a plurality of airport robots 300 disposed at an airport.

The camera 330 may include a camera installed in the airport. For example, the camera 330 may include a plurality of closed circuit television (CCTV) cameras installed in an airport, an infrared heat sensing camera, and the like. The camera 330 may transmit the photographed image to the server 320 or the airport robot 300.

4 is a view for explaining an example in which the airport robot detects a human body or an object at predetermined distances according to an embodiment of the present invention.

As shown in FIG. 4, the airport robot 400 according to an embodiment of the present invention may operate different object recognition modes for different distances or ranges. For example, when a human body or an object is detected in the first range 410, the user interface unit 160 of the airport robot 400 may automatically wake up. The user can interface with the airport robot without manually waking up the airport robot 400 when the user approaches the vicinity of the first range 410 of the sensing range of the airport robot 400. Further, when an inte or an object is detected in the second range 420, the airport robot 400 may execute a mode for interfacing with the user. If the user is in the second range 420, the airport robot 400 may deliver a message to the speaker or text informing that the interface is available before the user makes a request. In addition, when the airport robot 400 detects that the user is within the third range 430, the airport robot 400 may activate the dangerous mode. At this time, the airport robot 400 outputs a warning sound to a speaker, or outputs a warning message to a monitor, thereby notifying the user that the warning sound is too close to the user. Then, when the user moves away from the third range 430 to the second range 420, the user can stop the dangerous mode and notify the user that the safe distance has been reached. Alternatively, when the user detects that the user is within the third range 430, the airport robot 400 maintains a certain distance from the user himself or the airport robot 400 itself moves away from the user in the second range 420, It can move in the falling direction.

Further, the airport robot 400 can provide the user with the guidance service while traveling to a specific destination. In this case, the airport robot 400 may move while continuing to sense the distance from the user so that the user is located at least within the first range 410. Therefore, the airport robot 400 can provide the guidance service while keeping the distance from the user always within a certain range.

5 is a block diagram schematically showing an operation-based interface system of an airport robot according to an embodiment of the present invention. 6 is a block diagram showing the configuration of an operation signal input unit according to an embodiment of the present invention. 7 is a block diagram showing a configuration of an operation recognition unit, a command analysis unit, and a situation management unit according to an embodiment of the present invention. 8 is a diagram showing a configuration of an airport robot application software according to an embodiment of the present invention. 9 is a flowchart schematically illustrating an operation-based interface method according to an embodiment of the present invention.

The operation-based interface system between an airport robot and a human being according to the present invention includes an operation signal input unit 5100 for sensing a user's operation and transmitting a sensed signal, an operation recognition unit 5200 for interpreting the sensed signal and recognizing the sensed signal as an individual operation, A command interpreting unit 5300 for interpreting the recognized individual operation as a proper command according to the situation, a situation in which the operation recognition unit 5200 and the command interpreting unit 5300 are controlled by managing and communicating a situation according to the application of the robot context management unit 5400 and a robot application software program 5800 for operating the robot 5500, the video output apparatus 5600 and the audio output apparatus 5700 according to a command generated from the user's operation.

The operation signal input unit 5100 senses the operation of the user and transmits the sensed signal to the motion recognition unit 5200. The operation signal input unit 5100 includes an operation sensor device 5101 for detecting the operation of the user, a memory device 5103 for temporarily storing the sensed signal or storing the ID of the device and the user, And a wireless transmitting / receiving unit 5104 for transmitting the provided sensing signal to the operation recognition unit 5200. The wireless transmitting /

The operation signal input unit 5100 includes a power source unit 5106 for supplying power and a package and a wearing unit 5105 formed to be worn by a user.

The motion sensor device 5101 is a part for sensing the operation of the user and outputting a signal. The motion sensor device 5101 may use various sensors capable of sensing the motion of the user, and preferably a three-axis inertial sensor (acceleration sensor) is used.

The control device 5102 collects the signals output from the motion sensor device 5101, appropriately processes the signals, and transmits the processed signals to the motion recognizer 5200 through the wireless transceiver 5104.

The memory device 5103 is embedded to temporarily store the processed signal in the control device 5102 or store the device and the user's ID so that the necessary information can be used by the control device 5102. [

The device and the user identifier (ID) stored in the memory device 5103 are used to determine the position of the user wearing the operation signal input unit 5100 or to grasp the basic information about the current device or the user, Gt; 5102 < / RTI >

The wireless transmission / reception unit 5104 transmits the signal processed by the control unit 5102 to the motion recognition unit 5200. [ The wireless transceiver 5104 also functions to receive a signal, including a request for information about the device and the user, if necessary, and to transmit information related thereto.

The package and wearing portion 5105 is formed in a shape that can be worn by the user, and the shape and size of the wearing portion 5105 are determined by the selection of the detailed parts. The wearable form can be worn in various places of the body, and as a typical example, a wearing form such as a ring can be used.

The operation recognition unit 5200 includes a wireless transceiver 5210 that receives a signal transmitted from the operation signal input unit 5100, a recognition unit 5220 that analyzes the operation data of the received signal and recognizes the operation data as an individual operation, The learned neural network 5330 which is used in the process of recognizing the signal as an individual operation and variously exist according to the user and the robot application and the neural network 5330 which is used according to the control command of the situation management section 5400 among the learned various neural networks 5330, And a neural network selector 5240 for selecting the neural network selector 5230.

The neural network 5230 may exist in various ways depending on the user and the robot application, and each neural network 5230 may be configured by a learning algorithm.

For example, if there is an action from top to bottom, the direction recognition neural network recognizes the motion signal in the corresponding direction, but the numerical recognition neural network recognizes the number 1.

The neural network selector 5240 selects a required neural network 5230 according to a control command of the situation management unit 5400. [ The neural network selector 5240 selects and loads the neural network that has been learned and prepared in advance according to various situations such as the robot and the robot application registered in the situation management unit 5400.

As a method of recognizing a signal received by the motion recognition unit 5200 as an individual operation, a recognition apparatus 5220 may be configured using a soft computing technique (artificial intelligence technique) in addition to a method using a neural network. However, it is preferable to use the learned neural network when recognizing various actions by inputting simple operations once. Since the motion recognition unit 5200 using the neural network is composed of a simple arithmetic operation of arithmetic operation level for the operation signal, it is simpler and faster than the image and speech recognition.

The command interpreting section 5300 operates the robot application software 5800 through the airport robot 5500, the video output device 5600 and the audio output device 5700 using the recognition result of the motion recognition section 5200, Quot; information " The command interpreting section 5300 includes an operation including data information about a context-sensitive command corresponding to an individual operation, an operation by the control of the command interpreting table 5320 and the situation managing section 5400, the command interpreting table 5320 A command for selecting an analysis table 5330 for selecting an interpretation table 5320 and a command analysis table 5320 to generate commands corresponding to the current situation and application based on the selected operation and command interpretation table 5320, And a command interpreter 5310 for transferring the command to the command interpreter 5800.

The operation of the command interpreter 5300 and the command interpreter table 5320 and the interpreter table 5330 are controlled by the operation registered in the situation manager 5400 and the control of the command interpreter 5320 and the controller 5410 An interpretation table selection 5330 may be made.

The situation management unit 5400 controls the operation recognition unit 5200 and the command analysis unit 5300 according to the currently available or selected application.

The context management unit 5400 mainly includes a management unit 5410, a user status unit 5420, an application status unit 430, an environment status unit 5440, (5450) to collect and manage the situation.

The user status section 5420 records the user's ID, application preference, and neural network characteristics.

The application status section 5430 and the robot status section 5450 record externally released control commands and statuses.

In order to interpret commands transmitted to the airport robot 5500 and the robot application software 5800, the related operation and command interpretation table may be registered in the application status section 5430. [

When the application status is registered, the situation management unit 5400 selects an appropriate interpretation table from the registered interpretation table, and the command interpretation apparatus 5310 generates an instruction accordingly.

The format of the command depends on how the system is implemented. The message format is preferred. For example, the "forward push operation" is interpreted as an operation of pressing a button output on the screen output device, and a "push" message is generated and transmitted to the robot application software 5800. Or "repeated up and down shaking" operation generates a "stop" message by stopping the motion of the robot and transmits the message to the robot application software 5800. [

The environment status section 5440 records detailed information of the video and audio output apparatuses available to the robot and the application.

Specific situation information is managed so that it can be added as needed. In order to add the situation information by necessity, it is preferable to use an XML method in which format and contents are separated.

The robot application software 5800 controls the operation of the robot 5500 or outputs related information to the video output device 5600 and the audio output device 5700 by using the command generated by the command analysis unit 5300. [

The airport robot 5500 and the robot application software 5800 execute commands based on an operation generated from a user's operation. The airport robot 5500 may have conventional interface devices such as video and speech recognition based interfaces. The robot application software 5800 can also use the existing interface and the operation-based interface of the present invention together.

It is preferable that the airport robot 5500 and the robot application software 5800 use the video output device 5700 and the audio output device 5700 to give information necessary for interaction to the user. Since it is impossible to memorize all the currently available operations, it is preferable to inform the user through the video output device 5600 and the audio output device 5700. [ It is also desirable that the contents of the input operation be confirmed and the result of the misrecognition is confirmed by the user through the video and audio output apparatuses 5600 and 5700.

The robot application software 5800 may receive the command generated from the command interpreter 5300 and analyze the information contained in the command by the application logic unit 5810. [ Control information 5820 for controlling the robot 5820, image information 5830 output through the video output device 5600 and audio information 5840 output through the audio output device 5700, The airport robot 5500, the video output device 5600, and the audio output device 5700, respectively.

The airport robot 5500 performs an operation by the control information 5820. [ The airport robot 5500 can receive the control information through the robot application software 5800 but can directly receive the command from the command interpreter 5300 and perform the operation directly using the control information included therein .

The video output device 5600 and the audio output device 5700 output the video information 5830 and the audio information 5840 generated from the command and the user confirms the output contents to check the result of the input operation, .

An operation-based interface method between an airport robot and a person can sense a user's operation through a motion sensor as a signal. An operation signal input step (S5100) for transmitting the sensed signal and an individual operation recognition step (S5200) for receiving the transmitted signal and interpreting the received signal and recognizing the received signal as an individual operation corresponding to the situation may be included. And an instruction interpretation step (S5300) for interpreting the recognized individual operation as a command suited to the situation and application. And an airport robot and a robot application operation step (S5400) for operating the airport robot and the robot application according to the information included in the interpreted command.

The individual motion recognizing step (S5200) and the command interpreting step (S5300) are controlled by a situation management unit for managing and delivering the situation according to the robot application.

The individual motion recognizing step (S5200) recognizes the signal received from the operation signal inputting step (S5100) as a separate operation through the neural network selected by the situation manager based on a plurality of neural networks suitably learned for the application and the user.

The command interpretation step S5300 interprets and transmits the operation according to the recognition result obtained in the individual operation recognition step S5200, the operation provided by the situation manager, and the command interpretation table as commands suitable for the application of the robot apparatus.

The airport robot and robot application operation step (S5400) provides information necessary for interaction by the video output device and the audio output device to the user.

The operation signal input step (S5100) is performed by a device mounted by the user.

A simple application scenario of how an airport robot and human beings interface based on an operation by an operation-based interface system and method between an airport robot and a person of the present invention will be described. The scenario is that a user sitting at the airport wears an operation signal input unit 5100 and then calls a robot somewhere in the airport.

The user performs a predefined operation with the hand wearing the operation signal input unit 5100. For example, three quick shake actions are performed. The operation signal generated at this time is sensed by the motion sensor device 5101 and transmitted to the motion recognizer 5200 through the wireless transceiver 5104.

The situation management unit 5400 loads the standby state neural network 5230 that has been learned and prepared in advance in the operation recognition unit 5200 because the airport robot 5500 is currently in the standby state. The motion recognition unit 5200 recognizes that the wiggling is recognized three times faster through the atmospheric state neural network and transmits the result to the command analysis unit 5300.

The situation management unit 5400 loads the operation analysis table 5320 corresponding to the waiting situation into the instruction analysis unit 5300 and the instruction analysis unit 5300 uses the operation and the instruction analysis table 320 to determine whether the " Three quick shakes "operation is interpreted as a" robot call "command and a" robot call "command is generated.

The generated command is delivered to the robot with the argument "who is who" provided by the robot system or other application, and the robot is approached to the calling user. At this time, the user's location is ascertained by using the device and the user identification (ID) stored in the memory device 5103 of the operation signal input unit 5100 installed by the user, and approaches the user.

If the video output apparatus 5600 is being used, the user can show the currently input operation and the result through the video output apparatus 5600 to the user. For example, the display device in the airport can output video output information in cooperation with the robot application software 5800. [

10 is a hardware block diagram of an airport robot according to another embodiment of the present invention. 11 is a block diagram showing a method of expressing emotions of an airport robot according to another embodiment of the present invention. FIG. 12 is a chart showing an example of behaviors corresponding to intimacy between airport robots according to another embodiment of the present invention.

Referring to FIG. 10, an airport robot according to another embodiment of the present invention includes a sensing unit for sensing a state of the robot outside, a controller for controlling movement of the robot using information sensed by the sensing unit, And a driving unit for driving the operation of the robot by an instruction from the control unit.

The sensing unit is for inputting external information to the robot and may be provided in various forms. For example, it includes a touch sensor provided in the robot and capable of sensing human contact with the robot, a microphone capable of sensing sound or sound input from the outside, and a camera capable of receiving an external image .

The controller may include a determination unit for determining preferences of a robot user based on various external information input through the sensing unit and determining a tendency of the robot, and a control unit for controlling the movement of the robot based on the robot orientation determined by the determination unit. And an operation control unit. The control unit includes a microcontroller, a memory, and various registers in hardware.

In addition, the driving unit is composed of various parts for moving the robot. For example, if the robot includes a wheel and a motor for moving the robot, and a head and an arm of the robot, various motors and sensors for moving the components are included.

The robot according to the present invention as described above can be configured in various forms. In the case of a wheel-based robot, a wheel may be provided. In the case of a legged robot, a leg and a plurality of motors may be included.

First, the behavioral tendency of the robot is determined (S1010) through the interaction with the robot according to the present invention.

Here, the behavior tendency of the robot means that the airport robot analyzes and judges the type or behavior pattern preferred by the user, that is, the user, so that when the robot performs arbitrary actions without the action of the user, . In other words, it reflects the preference of the user to the robot, which means that the robot has personality similar to human personality.

In order for the robot to determine the behavioral tendency, the robot is initially manufactured by the robot manufacturer and various predefined operations are stored at the time when the robot is released to the market.

If the robot commands the user to perform a specific operation, the robot will perform the operation commanded by the user. However, if the user does not issue any command, the robot arbitrarily performs various operations stored in the robot. In this case, when the robot performs various actions arbitrarily, the user may input a specific input to the robot, so that the robot can grasp what the user prefers. When the user does not issue any command to the robot, that is, when the robot arbitrarily acts without receiving a command from the user, the robot acts more on the user's preferred action than other actions.

More specifically, when the robot is operated arbitrarily, various operations stored in the robot at the beginning are randomly operated by the same weight. However, when the robot performs a specific action desired by the user in the course of arbitrarily performing various actions, the user touches the touch sensor of a specific part mounted on the robot.

Through these inputs, the robot will be weighted up for the particular action the user prefers. Therefore, when the robot arbitrarily operates, the probability that a user will act on a specific operation is increased. That is, an operation that the user touches the touch sensor of a specific part means that the user positively recognizes the specific operation in which the robot operates.

For example, when the robot outputs a specific song, the user touches the touch sensor mounted on the robot if the song is a favorite song. In this case, since the robot recognizes that the user prefers a specific song, the weight of the specific song is increased, and the robot has a tendency to output a specific song frequently.

Or when the robot performs a specific operation (for example, when the tail is shaken), the user touches the touch sensor mounted on the robot when the action is his or her preferred action. By the input of the user The weight for a specific operation is increased. Accordingly, the robot recognizes that the user prefers this particular operation, and when the robot is operated at random, the robot tends to frequently perform such a specific operation.

In the foregoing, the user has touched a touch sensor installed at a specific position of the robot for a specific song or action, and inputting a user's preferred operation to the robot has been described, but the present invention is not necessarily limited to such a method. For example, when the user touches the touch sensor with a predetermined intensity or less, it is possible to realize the action as a user's favorite behavior. On the contrary, if the touch sensor is touched by a predetermined intensity or more, the robot may be recognized as a behavior that the user does not like.

In addition, when a user performs a specific operation (for example, a user shakes his or her hand), the input is inputted to the robot through the camera, and the inputted image is judged by the image processing module to determine which operation the user prefers .

In this way, the interaction between the robot and the user enables the robot to determine the user's preferred action or song, and the behavior of the robot is determined by reflecting the user's preferred behavior.

Next, the robot determines whether another robot is present within the identification range in which another robot can be identified (S1020). At this time, various methods can be used as a method for determining whether or not another robot exists.

For example, an image input to the camera using a camera mounted on the robot may be analyzed using an image processing module, and based on the analyzed result, it may be determined whether another robot is within the identification range.

Alternatively, a unique RFID chip is built in each robot, and a wireless reader capable of recognizing the RFID is provided. When the robot comes within a distance allowing recognition of RFID of another robot, it is judged that another robot exists You can also use the

Next, a description will be given of a method of expressing emotions between the robots according to the behavior of the robot (S1030). Expressing the feelings of robots does not mean emotions of the same level as humans but it means that the intimacy between the robot and the robot is quantified and the robot performs specific actions according to the numerical intimacy. Through such actions, people express similar emotions.

If it is detected in step S1020 that another robot exists within a distance at which the robot can recognize another robot, the robot determines the degree of intimacy with the other robot. The intimacy level between the robot and the robot can be set to be divided into several levels. According to the embodiment of the present invention, the degree of familiarity between the robot and the robot is set to be higher as the number of intimates is divided into levels of 1 to 5 and the number increases.

The intimacy between the robot and the robot is set to 1 at first, but the intimacy between the robots is gradually increased by various methods. For example, if the number of times or the number of times the robot and other robots exist within the identification range, the intimacy between the robot and the robot becomes high.

Or when a robot having the same behavioral tendency as the behavior tendency of the robot determined by the user is encountered, the intimacy between the robots is increased. For example, if another robot judged to exist within the identification range of the robot is perceived as performing a specific song or operation preferred by the user, the familiarity of the robot and the other robot becomes high. It is possible for another robot to express a specific song through a microphone and a voice processing module provided in the robot and to perform a specific operation through a camera and an image processing module provided in the robot.

Then, depending on the intimacy between the robot and another robot, the robot expresses intimacy to the other robot. That is, when a robot having a high affinity is encountered, the corresponding operation is performed. However, if you encounter a robot with a low intimacy, you will not take any action.

When a robot meets another robot with an affinity of 1, the robot does not react. However, when the robot meets another robot having an affinity of 2, the robot moves the arm provided on the robot. When the robot meets another robot with an affinity of 3, the robot shakes the arm and displays a welcome message on the LCD screen of the robot or causes the LED to blink. Then, when the robot meets another robot having an intimacy of 5, the robot turns around and expresses the feelings of the robot while outputting a welcome message through the speaker.

In this way, the emotion between the robot and the robot can be expressed by allowing the robot to perform actions corresponding to the intimacy between the robot and the other robot. And it is possible to realize more intelligent behavior of the robot through emotional expression between these robots.

FIG. 13 is a flowchart of a method in which an airport robot determines the behavior of a user according to the present invention and expresses emotions by moving the body. 14 is a configuration diagram of an airport robot for expressing emotions in the body by judging the behavior of the user.

The user's moving image is input to the image camera mounted on the airport robot (S1310). And generates difference image information in which the behavior of the user is changed in the moving image of the user photographed by the image camera (S1311). In order to obtain the image information, the moving image of each frame is blocked with a small rectangle of 8 * 8 (S1312). The blocked motion picture frame is compared with the previous motion picture frame to define difference image information based on the changed color (S1313). The differential image information is analyzed based on a database defined for an action performed by the user to define a user action (S1314). It is determined whether the user action is expected in the program executed in the robot (S1315). If the user action is an action expected by the robot, a positive emotional expression is displayed using the SDK of the robot, and a negative emotional expression is displayed if the user action is not an expected action (S1316).

The actions and the motion control that can be provided for each robot may be different, and the robot's emotions can not be expressed only through the same action. The motion control which can be generally used is as follows. In other words, wheel control moves back and forth, left and right, head control is controlled up, down, left and right, and arm control is rotated left, right and front and back. LED control is controlled in mouth, ear, arm, wheel part. Through the above control, it is defined in terms of happiness / surprise / normal / disappointment (sadness) / shame, and is operated in contents. In definition, it is provided by a combination of wheel, head, arm, LED, And expressions are different from each other.

14, the airport robot 1400 according to another embodiment of the present invention includes an image camera 1410, an image generating unit 1420, a DB 1430, a determining unit 1440, an output unit 1450, And a control unit 1460. The image camera 1410 photographs the behavior of the user and outputs it to the screen. The image generating unit 1420 generates the difference image information changed in the action of the photographed user. The DB 1430 has a list defining the behavior of the changed difference image information. The determination unit 1440 determines whether the difference image information is an expected behavior in the robot. The output unit 1450 expresses the emotion of the robot as at least one of a wheel, a head, an arm, and an LED according to the determination of the determination unit. The controller 1460 controls the overall operation of the airport robot.

15 is a view for explaining an example in which an airport robot according to an embodiment of the present invention outputs emotional emoticons to a display unit.

As shown in FIG. 15, the airport robot according to an embodiment of the present invention can arrange the display unit on the front surface or the rear surface. The control unit of the airport robot can control the display unit to output emotion emoticons for displaying emotional changes by applying facial expression elements such as eyebrows, eyes, and mouth of the human body.

The control unit of the airport robot outputs the emotion emoticon to the display unit for the purpose of transmitting a message to the user in predetermined predetermined situations such as when the battery of the airport robot is insufficient or when the voice recognition inputted by the user is not correctly recognized Can be controlled.

In addition, the control unit of the airport robot may recognize the user's facial expression, predict the current emotional state of the user, and then select and output the emotional emoticons so that they have the same emotional state or opposite emotional state. For example, when it is determined that the user's facial expression is currently depressed, the control unit of the airport robot can control to output emotional emoticons of bright facial expression to the display unit.

The control unit of the airport robot may control the emotional emoticons matching the negative emotions when the response to the input requested by the user is negative and the emotional emoticons matching the positive emotions when the reply is positive have.

Also, the control unit of the airport robot can control the display unit to output an emoticon associated with surprise when the user collides with the user or when the user performs voice input more than a predetermined speed.

The present invention described above can be embodied as computer-readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, , And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet). The computer may also include an AP 150 of an airport robot. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

Claims (9)

In an airport robot,
A display unit;
A user interface unit for receiving input from a user; And
And a control unit for controlling the operation of the airport robot,
Wherein the user interface unit includes an operation sensor for detecting an operation of a user,
Wherein,
Wherein the controller controls the operation of the user detected by the operation sensor to match one of a plurality of user messages stored in the database and to output a GUI (Graphic User Interface) mapped to the matched user message to the display unit,
Airport robots.
The method according to claim 1,
Wherein,
When receiving at least one information output request signal from the user,
And outputting a positive expression emoticon to the display unit when the request signal is correctly recognized and outputting a negative expression emoticon to the display unit when the information output request signal is not correctly recognized,
Airport guidance robot.
The method according to claim 1,
Wherein,
The state of the battery of the airport robot is checked,
Outputting a positive expression emoticon to the display unit when the battery condition is equal to or greater than a predetermined value and outputting a negative expression emoticon to the display unit when the battery condition is less than a predetermined value,
Airport robots.
3. The method of claim 2,
Wherein the airport robot includes a camera,
The camera captures an image of a behavior change of the user,
Wherein,
A method of analyzing a frame of a captured image based on a database,
Airport robots.
5. The method of claim 4,
The image captured by the camera is received at a rate of 30 frames per second,
Airport robots.
5. The method of claim 4,
Wherein,
Determining the accuracy of the information output according to the defined user action,
Airport robots.
The method according to claim 6,
Wherein,
Outputting a positive expression emoticon to the display unit when the accuracy of the information is not less than a predetermined value and outputting a negative expression emoticon to the display unit when the accuracy of the information is less than the predetermined value,
Airport robots,
5. The method of claim 4,
Wherein,
Wherein the controller is configured to recognize the face of the user, analyze the sex and age of the user according to the recognition result, and output the facial expression emoticons matching the current state of the user on the display unit,
Airport robots.
The method according to claim 1,
Wherein,
Controlling the emotional emoticon corresponding to the surprise with the warning sound to be output to the display unit when the proximity of the user is detected within the predetermined distance,
Airport robots.

Images