KR20180054528A - Concierge robot system, concierge service method and concierge robot - Google Patents

Abstract

Provided are a concierge robot system, a concierge service method, and a concierge robot. The system is the concierge robot system for providing an artificial intelligence concierge service, and comprises: a user interface device which receives external images and external voices, and outputs data on a screen or outputs to voices; a storage device which stores a program providing data through the user interface device based on learning data generated using a neural network model; and a processor which carries out the program. In this regard, the program includes instructions which recognizes emotions of a user identified from the external images based on the learning data, outputs data expressing emotions caused by the emotion recognition on the screen, outputs to voices by generating conversation sentences corresponding to a natural language of web data collected from the outside through web scrapping based on the learning data, outputs to voices by generating conversation sentences corresponding to an answer by a recognition result of the external voices, generates user recommendation data for the identified user based on the learning data, and outputs the user recommendation data on the screen or outputs conversation sentences corresponding to the process of the natural language to voices.

Description

CONCIERGE ROBOT SYSTEM, CONCIERGE SERVICE METHOD AND CONCIERGE ROBOT BACKGROUND OF THE INVENTION 1. Field of the Invention [0001]

The present invention relates to a concierge robot system, a concierge service method, and a concierge robot.

Intelligent robots that provide services to human beings seem to be highly utilized in the same places as restaurants, and studies of robots suitable for them are underway.

In the era of industrialization of robots, differentiated services are required to enhance customer satisfaction and to use the customers in a time when expectation of utilization of robots in service providing places such as restaurants is increasing.

Conventionally, a kiosk, which is an unmanned general information guidance system, is installed at a service providing place such as a restaurant, and automation service is provided. However, such a kiosk has attempted to computerize only by installing network devices, but it is far from being interactive with customers or offering differentiated services to customers.

In addition, the simplicity is only the function of a ticket vending machine that automates the menu selection and settlement according to the order, so it does not satisfy various needs of the customer.

In addition, there is a case in which robot technology is applied to the unmanned general information guidance system. However, since the output of the robot is defined in advance in a specific input of the user, the robot's behavior is simply repetitively repeated, It is merely a simple operation of outputting the instruction that is instructed. Therefore, service robots for human convenience are required to perform intelligent tasks and to interact with users in order to meet various needs of users.

The present invention provides a concierge robot system, a concierge service method, and a concierge robot that provide customized information by accurately determining user's feelings and requirements when providing a concierge service such as face-to-face service using a robot .

According to one aspect of the present invention, a concierge robot system is a concierge robot system for providing an artificial intelligent concierge service, comprising: a user interface device for inputting an external image and an external voice, A storage device for storing a program for providing data through the user interface device based on learning data generated using a model, and a processor for executing the program, the program comprising: The method includes the steps of recognizing a user's emotion identified from an external image, outputting data expressing an emotion according to the emotion recognition to the screen, and transmitting a conversation sentence corresponding to a natural language of web data collected from the outside through web scraping to the learning Based on the data Generating a dialogue sentence corresponding to an answer according to a result of recognition of the external voice and outputting it as a voice, generating user recommendation data for the identified user based on the learning data, And instructions for outputting the data to the screen or natural language processing and outputting a corresponding conversation sentence by voice.

Wherein the concierge robot system further comprises a communication device for transmitting and receiving wireless data in a communication area of a limited space, the program broadcasting a uniform resource locator (URL) through the communication device, And generating concierge service data corresponding to the user in response to a user request obtained through the concierge service providing URL, wherein the concierge service providing URL includes a member registration guide information, a personal information use consent request, A concierge service including at least one of service shop information, service menu information, waiting number information, product or service order, and online payment service.

The program may include at least one of service recommendation information, promotion information, and advertisement information derived by matching CRM (Customer Relationship Management) data with a plurality of user personal data including at least one of a user's visit history, service use history, And broadcasting the concierge service providing URL to provide the marching information including one.

The program generates a relationship between a plurality of user personal data, emotion learning data, and CRM (Customer Relationship Management) data including at least one of a visit history of a user, a service use history and a payment history by a neural network, And backward propagation to generate the user recommendation data.

Wherein the concierge robot system further comprises a server interface device for transmitting and receiving data to / from an external cloud server by wire or wireless, wherein the program transmits a facial image recognized from the external image to the cloud server, Acquiring emotion learning data by performing corresponding emotion learning, transmitting a face image input from the user interface device to the cloud server, requesting emotion recognition corresponding to the face image, and receiving the emotion recognition data.

Wherein the program is configured to perform natural language processing of data expressing the emotion and the user recommendation data to generate a corresponding natural language, to learn a conversation sentence corresponding to a natural language using the neural network model, Lt; / RTI >

The concierge robot system may further include a server interface device for transmitting and receiving data to / from an external cloud server by wire or wireless, and the program may further comprise a data processing unit for processing the data expressing the emotion and the user recommendation data, And transmitting the natural language to the cloud server, learning the conversation sentence corresponding to the natural language using the neural network model, and receiving the generated conversation sentence using the learning result.

The program is configured to apply a concierge service defined by a model, a view and a controller (MVC, Model, View and Controller), process the user data, the emotion learning data and the CRM data, And may include instructions for generating data.

The concierge robot system further includes a point of sale (POS) device for processing a product or service order according to the speech recognition result and performing payment processing in cooperation with a payment server connected to the communication network, And collecting order details and payment details from the force device and storing the collected personal information as user personal data.

The interface device includes a video input device for capturing the external video, a display device for visually displaying data on the screen, a voice input device for inputting the external voice, a voice output device for outputting the conversation text by voice, And a light emitting unit including at least one LED (Light Emitting Diode) driven according to a signal, wherein the program outputs the conversation sentence to the sound output apparatus, and transmits the user recommendation data and the data expressing the emotion And outputting a signal for controlling at least one of a light emission time, a light emission color, and a light emission amount to the light emitting unit according to data expressing the emotion.

According to another aspect of the present invention, a concierge service method is a concierge service method performed in a concierge robot system operated by at least one processor. When an external image is input, the concierge service method is based on user characteristic data extracted through learning of a neural network model Identifying the user from the facial image recognized from the external image, calculating emotion corresponding to the plurality of facial images from the facial image based on emotion learning data learned using the neural network model, Generating a sentence corresponding to the natural language of the web data collected from the outside through the web scraping to correspond to a natural language using the neural network model; The study of conversation sentence Generating on the basis of the result, and a step of outputting to the outside through the sound output apparatus a voice corresponding to the conversation sentence.

Generating an answer to the recognized natural language from the external voice, and outputting a voice corresponding to the answer.

Broadcasting a uniform resource locator (URL) by wireless in a communication area of a limited space, generating concierge service data responsive to a user request obtained through the concierge service providing URL, The method of claim 1, further comprising the step of processing the concierge service data in a natural language and outputting a conversation sentence corresponding to a natural language as a voice, wherein the concierge service providing URL includes a member registration guide information, A concierge service including at least one of store information, service menu information, waiting number information, product or service order, and online payment service.

The concierge service providing URL includes at least one of service recommendation information derived by matching a plurality of user personal data including at least one of a visit history of a user, service usage history and payment history, and CRM (Customer Relationship Management) data, And information about the at least one of the at least one of the at least one of the at least two of the at least one of the at least two of the at least two.

A method of generating a relationship between a plurality of user personal data including at least one of a visit history of a user, a service use history and a payment history, the emotion learning data, and CRM (Customer Relationship Management) data by a neural network, and generating user recommendation data by processing the user recommendation data in a natural language, generating a dialogue sentence corresponding to the natural language based on a learning result of a dialogue sentence corresponding to a natural language using the neural network model, The method comprising the steps of:

According to another aspect of the present invention, there is provided a concierge robot comprising: a body having a predetermined shape; a display device mounted on the body for outputting data to a screen; a speaker mounted on the body for outputting sound; A printer mounted on the main body and mounted on the main body so as to be in contact or non-contact with the main body, An input / output port mounted on the main body and connected to a communication network and an external device, a memory mounted inside the main body and storing a program, and a control unit including at least one processor for executing the program Based on learning data generated using a neural network model, A display unit for displaying a sentence corresponding to a natural language of the web data collected from the outside through web scraping; And generates a dialogue sentence corresponding to an answer according to a result of recognition of the external voice and outputs the generated dialogue sentence to the speaker, and processes the product or service order according to the recognition result of the external voice And outputs the settlement processing result to the printer, processes the result of the order processing in a natural language, and outputs the corresponding conversation sentence to the speaker.

And an actuating member mounted to the main body and driven by a driving signal of the actuator driving unit to rotate in a predetermined angle range or to move in a predetermined direction can do.

And a light emitting device mounted on the main body and including at least one LED (Light Emitting Diode) driven according to a control signal, wherein the control unit controls the light emitting device to emit light in accordance with data expressing the emotion, And can output a signal for controlling at least one to the light emitting unit.

According to the embodiment of the present invention, based on learned neural network technology, AI-based differentiation that accurately determines requirements by recognizing visiting customers (speech recognition, face recognition, emotion recognition, etc.) It is possible to improve competitiveness and profit structure through intelligent services.

In addition, it is possible to improve the quality of service in the field of repetitive business and to maintain more close interaction relation with the user, so that the frequency of use of the robot is increased, thereby providing high satisfaction to both consumers and producers.

1 is a configuration diagram of a concierge robot system according to an embodiment of the present invention.
2 is a configuration diagram of a concierge robot system according to another embodiment of the present invention.
3 is a front perspective view of a concierge robot according to an embodiment of the present invention.
4 is a front view of a concierge robot according to an embodiment of the present invention.
5 is a rear perspective view of a concierge robot according to an embodiment of the present invention.
6 is a flowchart illustrating a concierge service method using a robot according to an embodiment of the present invention.
7 is a flowchart illustrating a concierge service method using a robot according to another embodiment of the present invention.
8 is a flowchart illustrating a concierge service method using a robot according to another embodiment of the present invention.
9 is a flowchart illustrating a concierge service method using a robot according to another embodiment of the present invention.
10 is a flowchart illustrating a concierge service method using a robot according to another embodiment of the present invention.
11 is a block diagram illustrating a hardware configuration of a concierge robot system according to another embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise. Also, the terms " part, "" ... "," module ", and the like described in the specification mean a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software .

In the present specification, the Concierge Service is defined as a kind of gateway service that faces the customer as a secretarial service for judging and fulfilling the customer's needs for the convenience of the customer. Such a concierge service may be provided, for example, in a restaurant, a coffee shop, a fast food restaurant, a hotel, a shop, or the like.

A concierge robot system, a concierge service method, and a concierge robot according to an embodiment of the present invention will now be described with reference to the drawings.

1 is a configuration diagram of a concierge robot system according to an embodiment of the present invention.

1, the concierge robot system 100 includes an image capturing unit 101, a face recognizing unit 103, an emotion recognizing unit 105, a user recognizing unit 107, a voice input unit 109, The conversation sentence generation unit 113, the concierge service processing unit 115, the display device 117, the light emitting device 119, the audio output device 121, the behavior expression unit 123, the short distance communication device 125 A storage device 127, a POS device 129, an input / output port 131, and a communication device 133.

At this time, the image capturing unit 101, the voice input unit 109, the display device 117, the light emitting device 119, and the voice output device 121 constitute a user interface device. That is, it acquires information from a user and performs an interface function of outputting concierge service data to the user based on the acquired information.

The image capturing unit 101 captures an external image and outputs a video signal corresponding to the image. A CCD (Charge Coupled Device) camera, a depth camera, a pan-tilt-zoom camera, or the like, which is a digital camera, may be used as the image photographing unit 101.

The face recognizing unit 103 recognizes the face of the person in the image using the image corresponding to the image signal outputted from the image photographing unit 101. [

The emotion recognition unit 105 learns a plurality of face images using a neural network model of deep learning technology, in particular, a Convolutional Neural Network (CNN) model, and learns the emotions of the user represented by the face images do. Thereafter, the emotion recognition unit 105 can recognize the user emotion corresponding to the face by using the CNN model as the face image recognized by the face recognizing unit 103 as an input. For example, one of the seven emotions learned in the embodiment of the present invention, namely anger, happiness, surprise, disgust, sadness, fear, and neuter, is recognized through the CNN model learned for the facial image, And output the result.

The user recognition unit 107 can learn the image output from the face recognition unit 103 using the CNN model and generate learning information for a previously registered user.

Based on the learning information about the pre-registered user, the user recognition section 107 can recognize the presence or absence of the pre-registered user by using the CNN model with the face image recognized by the face recognition section 103 as an input have. If it is recognized by the user recognition unit 107 that the user is not a previously registered user, the registration process as a user may proceed to the user.

The user recognition unit 107 extracts feature points that distinguish 128 individuals from each other through deep CNN learning through images of the same person and the same person and an image of a completely different control group, And stores it as information.

Also, the user recognition unit 107 can recognize whether the user is a previously registered user through a SVM (Support Vector Machine) algorithm. Here, the SVM is a map learning method that provides an optimal separation boundary for classifying data, and the optimal separation boundary is learned to be located in the middle between data of each class.

The voice input unit 109 is configured to receive voice from outside, for example, a microphone array can be used.

The voice recognition unit 111 recognizes an external voice input through the voice input unit 109. [ Since the speech recognition technology is well known, a detailed description thereof will be omitted here.

The conversation sentence generation unit 115 collects web data from various external servers 200 connectable via the Internet, inputs a natural language generated by the natural language processing of the collected web data, and generates a conversation sentence corresponding thereto . Here, the learning about the generation of the conversation sentence corresponding to the natural language of the web data can be performed through a Recurrent Neural Network (RNN) model, for example. One of the deep learning algorithms, which is a representative technology of artificial intelligence representation learning, which is a field of researching how to extract useful information from a machine, which is a big issue recently, is an artificial neural network The connection between the constituent units refers to a neural network constituting a directed cycle. Unlike the Feedforward Neural Network, the cyclic neural network can utilize the memory inside the neural network to process arbitrary input, especially the model used to extract the time series correlation of input data. Due to these characteristics, the circular neural network is utilized in fields such as handwriting recognition and exhibits a high recognition rate. Since such a circular neural network is already well known, a detailed description thereof will be omitted here.

The conversation sentence generation unit 115 processes the web data collected in real time from the external server 200 at the time of service, generates a conversation sentence corresponding to the input of the natural language using the already-learned circulating neural network, Can be output.

The conversation sentence generation unit 115 generates a conversation sentence corresponding to the natural language of the web data collected from the outside through Web scraping based on the learning data of the CNN model and outputs it as a voice. Then, a conversation sentence corresponding to an answer based on the recognition result of the external speech is generated and outputted as a voice.

The concierge service processing unit 115 can generate and output concierge service data using the data output from the emotion recognition unit 105, the user recognition unit 107, and the conversation sentence generation unit 113. [ The concierge service processing unit 115 includes a video capture unit 101, a face recognition unit 103, an emotion recognition unit 105, a user recognition unit 107, a voice input unit 109, a voice recognition unit 111, A display device 117, a light emitting device 119, a sound output device 121, an action expression unit 123, a short range communication device 125, a storage device 127, a POS device Output port 131, and communication device 133 to control these configurations and process signals therebetween.

The concierge service processing unit 115 provides the concierge service data through the user interface device based on the learning data generated using the neural network model.

The concierge service processing unit 115 outputs data expressing emotion based on the emotion recognition on the screen of the display device 117 based on the learning data. That is, a display for interacting with a user based on the current emotional state of the user according to the emotional recognition, may be variously expressed by emoticons, graphics, and the like.

The concierge service processing unit 115 generates the user recommendation data for the user based on the learning data of the CNN model. According to an embodiment of the present invention, a relationship between a plurality of user personal data, emotion learning data, and CRM (Customer Relationship Management) data including at least one of a visit history of a user, service utilization history, User recommendation data can be generated by backward propagation of the neural network.

The concierge service processing unit 115 applies a concierge service defined by a model, a view and a controller (MVC, Model, View and Controller), performs a big data process on a large number of user personal data, emotion learning data, It can be used to generate.

Here, the user recommendation data may be favorite menu recommendation data for frequent visitors or menu recommendation data that can be freely enjoyed through automatic point accumulation.

Alternatively, if the user referral data is classified as a loyal customer group who visit three or more times a week, the guide information can be directly applied to the corresponding promotions and discounts on the basis of monthly / quarterly / have.

Alternatively, the user recommendation data may be loyalty customers, departing customers, and general customers through linkage with CRM, and promotional information such as differentiated coupons and discount events according to customer segments.

The concierge service processing unit 115 may output the user recommendation data on the screen in cooperation with the conversation sentence generation unit 113 or output the corresponding conversation sentence by voice in a natural language process.

The concierge service processing unit 115 outputs the user recommendation data and the user emotion state through the display device 117. [

The concierge service processing unit 115 outputs a signal for controlling at least one of the light emission time, the color of light emission, and the light emission amount to the light emitting device 119 according to the user emotion state.

The concierge service processing unit 115 can control the operation of the behavior expression unit 123 according to the user's emotional state.

The concierge service processing unit 115 broadcasts a uniform resource locator (URL) through the short distance communication device 125. And generates concierge service data corresponding to the user according to the user request obtained through the concierge service providing URL.

Herein, the concierge service providing URL is a service providing URL of the concierge service to the user terminal (not shown) to which the connected user guide information (not shown), personal information use agreement request, service shop information, service menu information, waiting information guide, And provides at least one concierge service. At this time, the user terminal (not shown) may be a smart phone or the like.

Also, the concierge service providing URL may include at least one of service recommendation information, promotion information, and advertisement information derived by matching CRM data with a plurality of user personal data including at least one of a user's visit history, service use history, May provide the < RTI ID = 0.0 >

Such a concierge service providing URL may include a connection address of a server providing information for providing a concierge service. Or the concierge service providing URL may include an address of a web space or a storage device storing information for providing a concierge service.

The concierge service processing unit 115 performs natural language processing on the emotion recognition result and the user recommendation data in cooperation with the emotion recognition unit 105 and the conversation sentence generation unit 115 to generate a corresponding natural language and transmits a conversation sentence corresponding to the natural language to the neural network You can use the model to learn, and you can create conversation sentences using the learning results.

The display device 117 displays the concierge service data output by the concierge service processing unit 115 and displays the user recognition information recognized by the user recognition unit 107, Quot; is displayed. The display device 117 includes a display, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, a thin film transistor-liquid crystal display (TFT LCD) And may include at least one of an organic light emitting diode (OLED), a flexible display, a field emission display (FED), and a three-dimensional display (3D display). In addition, a plurality of display devices 117 may be provided.

The light emitting device 119 may include at least one light emitting diode (LED) that is driven according to a control signal output from the concierge service processing unit 115. [

The voice output apparatus 121 outputs voice corresponding to the conversation sentence generated by the concierge service processing unit 115 to the outside. The audio output device 121 may be a speaker having a TTS (Text to Speech) function.

The behavior expressing unit 123 includes an actuator driving unit (not shown) for generating a driving signal under the control of the concierge service processing unit 115, and an actuator driving unit (not shown) Or an operation member moving in a predetermined direction. That is, the concierge service processing unit 115 may generate a control signal to rotate or move and output it to the actuator driving unit. For example, when the order is completed, you can turn around and point to the designated place, coffee pick-up stand.

The short range communication device 125 transmits and receives wireless data in a limited space communication area. At this time, it is possible to use a short-range communication technology such as Bluetooth or Wi-Fi.

The storage device 127 stores a program for providing the concierge service data through the user interface device based on the learning data generated using the neural network model. The storage device 127 may store information related to the pre-registered user. The storage device 127 may store instructions for performing operations of the face recognition unit 103, the user recognition unit 107, the voice recognition unit 111, the emotion recognition unit 105 and the concierge service processing unit 115 have.

Also, the storage device 127 may store the learning information in the user recognition section 107 and the emotion recognition section 105, respectively.

In addition, the storage device 127 may store a plurality of user personal data including at least one of a visit history, service usage history, and payment history of the user.

Also, the storage device 127 generates a relation between a plurality of user personal data, emotional learning data, and CRM (Customer Relationship Management) data in a neural network, and generates learning data and analysis data that are analyzed by backward propagation of the neural network Can be stored.

The storage device 127 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.) A random access memory (SRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an electrically erasable memory (EEPROM) Programmable Read-Only Memory).

The POS device 129 processes a product or service order according to a voice recognition result and performs payment processing in cooperation with a payment server 200 connected to a communication network. Such order details and payment history can be stored in the storage device 127. [

The input / output port 131 is connected to an external device and provides a path for inputting / outputting data. At this time, the input / output port 131 may be connected to, for example, a kitchen or a store server provided in the service space, and used for delivering order details. Or a LAN port connected to the network device.

The communication device 133 is connected to the external server 200 through a communication network and transmits and receives data. Here, the external server 200 is a concept that includes the singular or plural. That is, a plurality of servers having different functions can be included. The plurality of servers may include a marketing server, a CRM server, a payment server, and the like, which provide the marketing information.

Recently, a cloud service, which is a service for downloading various kinds of data after storing it in an external cloud server instead of an internal storage space such as a user's computer or a smart phone, has attracted attention. Such a cloud service includes a cloud server As shown in FIG. For example, IBM's artificial intelligence technology, Watson and Google's TesnorFlow, are well known as open software. They use artificial intelligence to understand natural language questions, analyze and synthesize relevant data from vast amounts of data, and provide appropriate answers. They are providing cloud services that can use artificial intelligence technology through API (Application Programming Interface).

FIG. 2 is a configuration diagram of a concierge robot system according to another embodiment of the present invention, which corresponds to an embodiment using a cloud service.

Referring to FIG. 2, the description of the same configuration as that of FIG. 1 will be omitted. Unlike FIG. 1, it may further include a server interface device 135.

The server interface device 135 is connected to the cloud server 300. Audio information, and big data to an external cloud server 300 connected via an API through the server interface device 135 and receives corresponding user recognition results, emotion recognition results, and big data learning results use. In other words, the user recognizing learning through the image, the user emotion recognizing learning, the user recognition based on the learning result and the user emotion recognition, and the corresponding chat sentence using the web data collected from the outside are learned, The conversation robot corresponding to the newly collected web data generates a large number of user personal data, emotion learning data, and CRM data, and performs neural network learning and backpropagation analysis on the conciousness robot system 100 But may receive the results to be performed in the external cloud server 300.

Such a concierge robot system can be implemented in the form of a robot, for example, as shown in Figs. 3 to 5.

5 is a rear perspective view of a concierge robot according to an embodiment of the present invention. FIG. 5 is a front view of a concierge robot according to an embodiment of the present invention .

3 and 4, the image pickup section 101 and the audio input section 109 are mounted on the head of the main body 137 having a robot shape similar to a human being. A first display device 117-1 having a screen for displaying an emoticon type or the like on the face portion of the main body 137 is mounted. A second display device 117-3 having a monitor screen for outputting concierge service data, voice recognition result, emotion recognition result, user recognition result, and the like is mounted on the body of the main body 137. On the side of the second display device 117-3, a settlement means recognizer 141 such as an NFC reader or a card reader is mounted. Therefore, the user can check the order details displayed on the screen and then make a payment directly.

A printer 139 is mounted on the lower end of the main body 137 to output order details or payment details.

Referring to FIG. 5, the LED 119 is mounted on the face portion and the body portion of the body 137, respectively. Then, the voice output device 121 is mounted around the entire body.

An I / O box 147 including an input / output port 131 and a reset button 143 and a power button 145 is mounted on the rear surface of the main body 137.

Now, a method of providing various concierge services using a robot according to an embodiment of the present invention will be described.

6 is a flowchart illustrating a concierge service method using a robot according to an embodiment of the present invention.

Referring to FIG. 6, a user image is captured through the image capturing unit 101 (S101). Then, the face is recognized from the photographed image through the face recognizing unit 103 (S103).

Next, the emotion corresponding to the face image is recognized using the CNN model for the face image recognized by the face recognition unit 103 through the emotion recognition unit 105 (S105). For example, one of the seven emotions learned in the embodiment of the present invention, namely anger, happiness, surprise, disgust, sadness, fear, and neuter, is recognized through the CNN model learned for the facial image, And outputs the result (S107). At this time, in step S107, the emotion recognition result is processed as a natural language, and a conversation sentence corresponding to the natural language is generated using the CNN model and outputted as a voice.

Alternatively, the emotional expression of the robot corresponding to the emotion recognition result may be outputted through the LED light emission, or may be displayed on the display screen with the predefined avatar display corresponding to the emotion recognition result.

In step S109, the user recognizes whether the face recognized in step S103 is a face of a previously registered user by using the CNN model through the user recognition unit 107. Then, At this time, if the face of the user is recognized as the face of the previously registered user, the user recognition is completed and the information of the user can be known by extracting the recognized user information from the storage device 125. However, if it is not the face of the pre-registered user, since it is the face of the new user, a procedure of performing the user registration for the new use can be started.

If the user is identified, user recommendation data for the identified user is generated based on the previously generated learning data (S111). The user recommendation data is processed in a natural language (S113) to generate a conversation sentence corresponding to the natural language (S115). Then, the conversation sentence is converted into speech and output (S117).

FIG. 7 is a flowchart illustrating a concierge service method using a robot according to another embodiment of the present invention, and shows the detailed procedure of step S111 of FIG.

7, when the concierge service processing unit 115 receives user personal data collected from the local communication device 125 or the POS device 129, the emotion learning data provided from the emotion recognition unit 105, (S201) the CRM data provided through the external server 200 or provided through the storage device 127. [

Next, the relationship between the user personal data, the emotion learning data, and the CRM data is generated in the neural network (S203). Then, the neural network backward propagates to search for a service related to the user's emotional state or a service expected to be preferred by the user, and generates user recommendation data of the search result (S205). At this time, the user pattern data generated in the neural network algorithm can be trained through back propagation learning to derive user recommendation data.

FIG. 8 is a flowchart illustrating a concierge service method using a robot according to another embodiment of the present invention, and may be added to another embodiment after step S109 of FIG.

Referring to FIG. 8, after recognizing the user from the face image through the user recognition unit 107 (S301), it is determined whether the user is a registered user (S303).

If the user is a registered user, the user recommendation data is processed in a natural language through the conversation sentence generation unit 113, and a conversation sentence corresponding to the natural language is generated and output as speech (S305).

If the user is not a registered user, the Web data collected from the outside through Web scraping is processed in a natural language process (S307) through a conversation sentence generation unit 113, a conversation sentence corresponding to a natural language is generated (S309) (Step S311).

9 is a flowchart illustrating a concierge service method using a robot according to another embodiment of the present invention.

Referring to FIG. 9, a uniform resource locator (URL) is wirelessly broadcast in a communication area of a limited space through a concierge service processing unit (S401).

When the user accesses the concierge service providing URL (S403), it is determined whether the member is a member (S405), and if the member is not a member, the membership registration is induced (S407).

If the user is a member, the concierge service screen is provided (S409), and the information requested by the user or selected by the user is obtained on the concierge service screen (S411). Then, the payment process is performed (S413).

Next, concierge service data corresponding to the user request is generated (S415). The concierge service data is output to the display device (S417). Then, the concierge service data is processed in a natural language and a conversation sentence corresponding to a natural language is output as a voice (S419). Here, the concierge service data may be, for example, a message indicating that the ordered goods have been prepared.

FIG. 10 is a flowchart illustrating a method of providing a concierge service using a robot according to another embodiment of the present invention, and may be performed sequentially or simultaneously after step S101 in FIG.

Referring to FIG. 10, in step S503, a user's voice input through the speech recognition unit is recognized in step S503, and a response sentence based on the speech recognition result is generated in the conversation sentence generation unit 113 in step S505. Then, the answer sentence is output by voice (S507).

11 is a block diagram illustrating a hardware configuration of a concierge robot system according to another embodiment of the present invention.

11, the concierge robot system 400 includes a speaker 401 for outputting a voice, a microphone 403 for receiving an external voice, a communication device 405 for performing short-range communication or Internet connection, a contact- An input / output port 409 connected to the communication network and the external device, a display device 411 for visually displaying data on the screen, a printer 413 for outputting the order and payment details ), A memory 415, and a processor 417, and stores a program to be executed in combination with hardware at a specified location. The hardware has a configuration and performance capable of executing the configuration and / or the method according to the embodiments described in Figs. 1-10. The program includes instructions that may execute the configuration and / or methods in accordance with the embodiments described in Figures 1 through 10 and may be implemented in hardware, such as memory 415 and processor 417, Implements the invention.

As described above, according to the embodiment of the present invention, the concierge robot system stores the characteristics and information of the user through speech recognition, face recognition, etc., and uses the artificial intelligence (AI) Or analyzing and storing the user's repeated input information and providing a customized service through a robot optimized for the user. Therefore, the staff at the coffee shop, the fast food store, etc., It can replace and simplify simple repetitive tasks.

The embodiments of the present invention described above are not implemented only by the apparatus and method, but may be implemented through a program for realizing the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (18)

A concierge robot system providing an artificial intelligent concierge service,
A user interface device for receiving an external video and an external audio and outputting the data to the screen or outputting it as a voice,
A storage device for storing a program for providing data via the user interface device based on learning data generated using a neural network model; and
And a processor for executing the program,
The program includes:
Recognizing the emotion of the user identified from the external image based on the learning data, outputting data expressing emotion according to the emotion recognition to the screen, and responding to the natural language of the web data collected from the outside through web scraping And generating a dialogue sentence corresponding to an answer based on the result of recognition of the external voice and outputting the sentence as a voice, and outputting, as a voice, a user for the identified user based on the learning data, Instructions for generating recommendation data, outputting the user recommendation data to the screen or performing natural language processing and outputting a corresponding conversation sentence by voice. The method of claim 1,
Further comprising a communication device for transmitting and receiving wireless data in a communication area of a limited space,
The program includes:
Instructions for broadcasting a uniform resource locator (URL) for providing a concierge service through the communication device and generating concierge service data for responding to a user request obtained through the concierge service providing URL,
The concierge service providing URL includes:
Providing a concierge service including at least one of a membership registration guide information, a personal information use consent request, a service store information, a service menu information, a waiting number guide, a product or service order, Robot system. 3. The method of claim 2,
The program includes:
And at least one of service recommendation information, promotion information, and advertisement information derived by matching CRM (Customer Relationship Management) data with a plurality of user personal data including at least one of a user's visit history, service use history, And broadcasting the concierge service providing URL for providing the marching information. The method of claim 1,
The program includes:
(CRM) data, including at least one of a user's visit history, a service use history and a payment history, and transmits the backward propagation ) To generate the user recommendation data. 5. The method of claim 4,
Further comprising a server interface device for transmitting and receiving data to / from an external cloud server by wire or wireless,
The program includes:
Transmitting facial images recognized from the external image to the cloud server to perform emotion learning corresponding to the facial images to obtain emotion learning data,
And transmitting the face image input from the user interface device to the cloud server to request and receive emotion recognition corresponding to the face image. 5. The method of claim 4,
The program includes:
Generating a dialogue sentence corresponding to a natural language by using the neural network model and generating the dialogue sentence by using the result of learning; Gt; a < / RTI > concierge robot system. 5. The method of claim 4,
Further comprising a server interface device for transmitting and receiving data to / from an external cloud server by wire or wireless,
The program includes:
The data representing the emotion and the user recommendation data are processed in a natural language to generate a corresponding natural language, the natural language is transmitted to the cloud server, the conversation sentence corresponding to the natural language is learned using the neural network model, And receiving the generated conversation sentence using the result. 5. The method of claim 4,
The program includes:
A concierge service defined by a model, a view, and a controller (MVC, Model, View and Controller) is applied, and the user recommendation data is generated by subjecting the user personal data, the emotion learning data, ≪ / RTI > 5. The method of claim 4,
Further comprising a point of sale (POS) device for processing a product or service order according to the speech recognition result and performing a payment process in cooperation with a payment server connected to the communication network,
The program includes:
And collecting order details and payment details from the force device and storing the personal details as user personal data. The method of claim 1,
Wherein the interface device comprises:
A video input device for capturing the external video,
A display device for visually displaying data on a screen,
A voice input device for receiving the external voice,
A voice output device for outputting the conversation sentence by voice, and
And a light emitting unit including at least one LED (Light Emitting Diode) driven according to a control signal,
The program includes:
And outputting the user recommendation data and the data expressing the emotion through the display device, wherein at least one of the emissive time, the emissive hue, and the emissive amount is changed according to the data expressing the emotion, And outputting a signal for controlling the light emitting unit to the light emitting unit. A concierge service method performed in a concierge robot system operated by at least one processor,
Identifying a user from a face image recognized from the external image based on extracted user characteristic data through learning of a neural network model,
Recognizing emotions corresponding to a plurality of facial images from the facial images based on emotion learning data learned using the neural network model;
Generating emotion expression data according to the emotion recognition and outputting the emotion expression data through a display device,
Generating a dialogue sentence corresponding to the natural language of the web data collected from the outside through web scraping based on the learning result of the dialogue sentence corresponding to the natural language using the neural network model; and
Outputting a voice corresponding to the conversation sentence to the outside via a voice output device
The method comprising the steps of: 12. The method of claim 11,
Generating an answer to the recognized natural language from the external voice, and outputting a voice corresponding to the answer
Further comprising the steps of: 12. The method of claim 11,
Broadcasting a uniform resource locator (URL) wirelessly in a communication area of a limited space,
Generating concierge service data responsive to a user request obtained through the concierge service providing URL; and
Further comprising processing the concierge service data in a natural language and outputting a conversation sentence corresponding to a natural language as a voice,
The concierge service providing URL includes:
Providing a concierge service including at least one of a membership registration guide information, a personal information use consent request, a service store information, a service menu information, a waiting number guide, a product or service order, Service method. The method of claim 13,
The concierge service providing URL includes:
And at least one of service recommendation information, promotion information, and advertisement information derived by matching CRM (Customer Relationship Management) data with a plurality of user personal data including at least one of a user's visit history, service use history, A method of concierge service that provides the information of the marching. 12. The method of claim 11,
A method of generating a relationship between a plurality of user personal data including at least one of a visit history of a user, a service use history and a payment history, the emotion learning data, and CRM (Customer Relationship Management) data by a neural network, propagation to generate user recommendation data, and
Generating a dialogue sentence corresponding to a natural language based on a result of a dialogue sentence corresponding to a natural language using the neural network model,
Further comprising the steps of: A main body having a predetermined shape,
A display device mounted on the main body and outputting data to a screen,
A speaker mounted on the main body for outputting sound,
A camera mounted on the main body for photographing an external image,
A microphone mounted on the main body for receiving an external sound,
A printer mounted on the main body for outputting order and payment details,
A payment recognizing unit mounted on the main body and recognizing the payment means in a contact or non-contact manner,
An input / output port mounted on the main body and connected to a communication network and an external device,
A memory mounted in the main body to store a program, and
And a control unit configured of at least one processor for executing the program,
Wherein,
The method includes recognizing emotions of a user identified from the external image based on learning data generated using a neural network model, outputting data expressing emotions according to the emotional recognition to the display device, And generates a dialogue sentence corresponding to an answer based on the result of recognition of the external speech and outputs the dialogue sentence to the speaker, Processing a product or service order according to a recognition result of an external voice, performing payment processing in cooperation with a payment server connected to the communication network, outputting a payment processing result to the printer, And outputs the corresponding conversation sentence to the speaker. 17. The method of claim 16,
An actuator driver mounted on the main body and generating a drive signal,
An actuator member mounted on the main body and driven in accordance with a driving signal of the actuator driving unit to rotate in a predetermined angle range,
Further comprising: a concierge robot. The method of claim 17,
And a light emitting device mounted on the main body, the light emitting device including at least one LED (Light Emitting Diode) driven according to a control signal,
Wherein,
And outputs a signal for controlling at least one of a light emission time, a light emission color, and a light emission amount to the light emitting unit according to data expressing the emotion.

Images