JP6952453B2 - Customer service system and customer service method - Google Patents

Description

The present invention relates to a customer service system and a customer service method for serving customers at a restaurant or the like.

In recent years, the development of information processing technology such as robots and artificial intelligence has been remarkable, and robots and information processing technology have been adopted in fields such as long-term care and customer service, which have been difficult to mechanize in the past. For example, it has been proposed to adopt a robot equipped with artificial intelligence for customer service such as reception and customer service (see, for example, Patent Document 1).

The customer service provided by robots is also required to provide service, manners (courtesy), and hospitality, similar to the customer service provided by humans in the past. Here, the service is the main purpose of customer service, and for example, in the case of dealing with a visitor at a company reception, it corresponds to guiding the customer and bringing the customer and the person in charge of the company together. Manners are not just to provide services, but to express the same politeness as people do when serving customers. For example, in the case of dealing with customers at the company reception, the robot greets the customers. This applies to bowing, etc. (which is not always necessary to provide services). Hospitality is to provide comfort to customers by responding to the situation through attention, attention, and concern. It is not just a standard greeting, but it is not easy to get in the rain, for example. This corresponds to the attention given to the place where the umbrella is placed, expressing gratitude for having come to the middle. In particular, the hospitality of customer service in Japan is delicate and delicate, as it is introduced overseas by the word "hospitality" (Omotenashi).

Japanese Unexamined Patent Publication No. 2009-248193

As described above, it is desired that even in the case of customer service by robots, it is possible to realize hospitality or "hospitality" (hereinafter referred to as "customer service hospitality") as well as simply providing services with manners. However, customer service hospitality is difficult to create into a manual, and it is not easy for robots to realize it. Recently, with the development of artificial intelligence, the ability of robots to grasp individual situations has increased, but how to respond to the grasped situation to realize or improve customer service hospitality is made into a manual. Is difficult.

The present invention has been made in view of the above problems, and an object of the present invention is to realize or improve customer service hospitality in customer service using a robot.

As described above, the inventor of the present application proposes the following solutions in view of the difficulty of the robot exhibiting customer service hospitality. That is, in the first solution, the collaboration between humans and robots realizes customer service hospitality that is difficult to achieve by humans alone and difficult to achieve by robots alone. Further, in the second solution, a mechanism is realized in which a person causes a robot to learn customer service hospitality by an OJT method.

The customer service system according to the first aspect of the present invention is a customer service system including a robot, a controller operated by a service provider, and an instruction request means for generating an instruction request based on request information indicating a request of a service user. Is. The robot includes an action means for executing a customer service action for the service beneficiary and a robot communication means for transmitting and receiving information. The controller includes a controller input means for receiving an input from the service provider, a controller output means for outputting information to the service provider, and a controller communication means for transmitting and receiving information. The controller output means outputs the instruction request, the controller input means receives an input of an instruction from a service provider in response to the instruction request, and the controller communication means transmits the instruction to the robot. The robot communication means receives the instruction, and the action means executes the customer service action according to the instruction received by the robot communication means.

With this configuration, the service provider can instruct the customer service action in response to the request of the service beneficiary, and the robot can execute the customer service action according to the instruction of the service provider. Therefore, it is possible to respond to the requests of service recipients that the service provider cannot grasp or respond to, and even if the robot cannot determine the appropriate customer service action for the request, the service provider makes a decision. Appropriate customer service actions can be performed at.

The customer service system of the second aspect of the present invention further includes, in the customer service system of the first aspect, a request grasping means for grasping the request of the service beneficiary based on the request information, and the action means is the request. The customer service action corresponding to the request grasped by the grasping means is executed, and the instruction requesting means is based on the request information when the request grasping means cannot grasp the request of the service beneficiary based on the request information. To generate an instruction request.

With this configuration, if the request can be grasped by the request grasping means, the customer service action can be executed without requesting the instruction from the service provider, and if the request cannot be grasped by the request grasping means, the service provider is notified. You can ask for instructions.

The customer service system of the third aspect of the present invention further includes a request grasping means for determining a customer service action corresponding to the request in the customer service system of the first aspect, and the action means is the request grasping means. The determined customer service action is executed, and the instruction request means generates an instruction request based on the request information when the request grasping means cannot determine the customer service action corresponding to the request.

With this configuration, if the customer service action corresponding to the request can be determined by the request grasping means, the customer service action can be executed without requesting the instruction from the service provider, and the customer service action corresponding to the request can be performed by the request grasping means. If the determination cannot be made, the service provider can be instructed.

The customer service system of the fourth aspect of the present invention further includes an agent operated by the service beneficiary in the customer service system of the second aspect, and the agent is an agent input means that accepts input by the service beneficiary. The request grasping means grasps the customer's request by using the input transmitted from the agent communication means as the request information.

With this configuration, the service beneficiary can input the request using the agent. The agent may be a device installed in the robot and provided by the service provider, or a mobile terminal of the service beneficiary who has installed a predetermined application and is paired with the robot. You may.

In the customer service system of the fifth aspect of the present invention, in the customer service system of the second aspect, the robot further includes a microphone that accepts voice input, and the customer service system further includes a voice input to the microphone. A voice analysis means for analyzing the contents is provided, and the request grasping means grasps the request of the service beneficiary by using the analysis result of the voice analysis means as the request information.

With this configuration, the service beneficiary can input a request by talking to the robot.

In the customer service system of the sixth aspect of the present invention, in the customer service system of the second aspect, the robot further includes a microphone that accepts voice input, and the customer service system further includes a voice input to the microphone. The request grasping means includes a voice analysis means for analyzing the contents and an interpreting means for translating the contents of the voice analyzed by the voice analysis means into another language, and the request grasping means uses the translation result of the interpreting means as the request information. Understand the needs of service recipients.

With this configuration, even if the service beneficiary is a foreigner, the request can be communicated to the robot or the service provider by talking to the robot in a foreign language.

In the customer service system of the seventh aspect of the present invention, in the customer service system of the second aspect, the robot further includes a camera for taking an image, and the customer service system analyzes an image taken by the camera. An image analysis means is provided, and the request grasping means grasps the request of the service beneficiary by using the analysis result of the image analysis means as the request information.

With this configuration, even if the service beneficiary does not input the request, the robot can positively grasp the service beneficiary's request and execute an appropriate customer service action.

In the customer service system of the eighth aspect of the present invention, in the customer service system of the second aspect, the controller input means receives the request of the service beneficiary grasped by the service provider as the request information, and the controller communication. The means transmits the request information input to the controller input means, and the request grasping means grasps the request of the service beneficiary based on the request information transmitted from the controller communication means.

With this configuration, even if the service beneficiary does not make a request and the robot does not notice the service beneficiary's request, the service provider can grasp the service beneficiary's request and execute an appropriate customer service action through the robot. ..

The customer service system of the ninth aspect of the present invention further includes a learning means for learning an instruction by the service provider for the request or the request information in the customer service system of any one of the first to eighth aspects. ..

The customer service system according to the tenth aspect of the present invention includes a robot provided with an action means for executing a customer service action for the service beneficiary, a controller operated by the service provider, and a request of the service beneficiary based on the request information. The action means includes a request grasping means for instructing the action means of a customer service action corresponding to the grasped request, and the action means executes the customer service action instructed by the request grasping means and grasps the request. When the means cannot grasp the request by the request information, the service provider executes the customer service action according to the instruction input to the controller for the request information.

With this configuration, if the request can be grasped by the request grasping means, the customer service action can be executed without requesting the instruction from the service provider, and if the request cannot be grasped by the request grasping means, the service provider is notified. You can perform customer service actions according to the instructions.

The customer service system according to the eleventh aspect of the present invention includes a robot provided with an action means for executing a customer service action for a service beneficiary, a controller operated by the service provider, and a customer service action corresponding to the request of the service beneficiary. The action means includes a request grasping means for instructing the action means of the determined customer service action, the action means executes the customer service action instructed by the request grasping means, and the request grasping means performs the request. When the customer service action corresponding to the above cannot be determined, the customer service action is executed according to the instruction input by the service provider to the controller with respect to the request.

With this configuration, if the customer service action corresponding to the request can be determined by the request grasping means, the customer service action can be executed without requesting the instruction from the service provider, and the customer service action corresponding to the request can be performed by the request grasping means. If the determination cannot be made, the customer service action can be executed according to the instruction to the service provider.

The customer service method according to the twelfth aspect of the present invention includes a robot, a controller operated by the service provider of the restaurant, and an instruction request means for generating an instruction request based on request information indicating a request of the service beneficiary. This is a customer service method using the provided customer service system. The robot includes an action means for executing a customer service action for the service beneficiary and a robot communication means for transmitting and receiving information. The controller includes a controller input means for receiving an input from the service provider, a controller output means for outputting information to the service provider, and a controller communication means for transmitting and receiving information. In the customer service method, the controller output means outputs the instruction request, the controller input means receives an input of an instruction from a service provider for the instruction request, and the controller communication means transmits the instruction to the robot. Then, the robot communication means receives the instruction, and the action means is used to execute the customer service action according to the instruction received by the robot communication means.

With this configuration as well, the service provider can instruct the customer service action in response to the request of the service beneficiary, and the robot can execute the customer service action according to the instruction of the service provider. Therefore, it is possible to respond to the requests of service recipients that the service provider cannot grasp or respond to, and even if the robot cannot determine the appropriate customer service action for the request, the service provider makes a decision. Appropriate customer service actions can be performed at.

According to the present invention, customer service hospitality can be realized or improved by collaboration between a robot and a human.

The figure which shows the whole structure and arrangement of the customer service system of embodiment of this invention. A block diagram showing a configuration of a customer service system according to an embodiment of the present invention. The figure explaining the 1st use mode of the customer service system of embodiment of this invention. The figure explaining the 2nd use mode of the customer service system of embodiment of this invention. The figure explaining the 3rd use mode of the customer service system of embodiment of this invention. A flow chart showing an operation example from the visit of the customer to the self-introduction of the robot according to the embodiment of the present invention. A flow chart showing an operation example from the ordering of the embodiment of the present invention to the start of cooking. A flow diagram showing an operation example in which a customer of the embodiment of the present invention talks to a robot and the robot responds to the robot to perform a customer service action. Flow chart of the reception process according to the embodiment of the present invention A flow chart showing an operation example in which the robot according to the embodiment of the present invention photographs a table and performs a customer service action according to the situation of the table. Flow chart of the reception process according to the embodiment of the present invention Flow chart of an operation example in which a clerk of an embodiment of the present invention grasps a customer's request and causes a robot to perform a customer service action. Flow chart of the reception process according to the embodiment of the present invention

Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the embodiments described below show an example of the case where the present invention is carried out, and the present invention is not limited to the specific configuration described below. In carrying out the present invention, a specific configuration according to the embodiment may be appropriately adopted.

(Configuration of customer service system 1)
FIG. 1 is a diagram showing an overall configuration and arrangement of a customer service system according to an embodiment of the present invention. FIG. 1 shows an example in which the customer service system 1 according to the embodiment of the present invention is installed in a restaurant. As shown in FIG. 1, the restaurant includes a reception / accounting area E, a floor area F, and a kitchen area K. The reception / accounting area E is provided with a reception terminal 400 and an accounting device 500. A table T used by a customer C who is a service recipient is installed in the floor area F, and each table T is provided with a robot 100 and an agent 200. The clerk S, who is a service provider who serves customers in the floor area F, carries the controller 300. An order receiving device 600 is installed in the kitchen area K, and a management control device 700 is installed between the kitchen area K and the floor area F.

The reception terminal 400, the accounting device 500, the robot 100, the agent 200, the controller 300, and the order receiving device 600 can each communicate with the management control device 700, and in the present embodiment, wireless communication is possible via a wireless router. Is. It should be noted that terminals such as the reception terminal 400, the accounting device 500, and the order receiving device 600, in which the troublesome wiring is not a major problem, may be connected to the management control device 700 by wire so as to be communicable.

The reception terminal 400 is provided with a touch panel, and is operated to issue a numbered ticket when the customer C who has entered the store needs to wait in line. The reception terminal 400 can input the number of customers C and requests for seats (non-smoking / smoking, etc.), and the clerk S guides the customer C to the table T in order according to the contents input to the reception terminal 400.

The accounting device 500 performs accounting processing for each table T for the customer C who leaves the store after eating and drinking. To this end, the accounting device 500 presents the payment amount according to the order received via the agent 200 or the controller 300. The clerk S in charge of accounting collects the price according to this payment amount.

The order receiving device 600 includes a monitor and displays an order from customer C on the monitor. For this purpose, the order receiving device 600 receives the order received via the agent 200 or the controller 300 via the management control device 700 and displays it on the monitor. The clerk S in the kitchen area K cooks according to the order displayed by the order receiving device 600. The order receiving device 600 may be provided with a printer instead of the monitor or in addition to the monitor, and may print out the received order in the form of an order receiving slip.

The robot 100 is a humanoid robot whose hands, feet, hips, and neck can be driven by a drive mechanism 110 (see FIG. 2). Further, the robot 100 has a size that does not interfere with the eating and drinking of the customer C even when placed on the table T, and for example, the height is about 20 cm. The robot 100 has a display panel 120 (see FIG. 2) on the chest, a lens of the camera 150 (see FIG. 2) on the head, and a speaker 130 (see FIG. 2) on the mouth. The ear part is equipped with a microphone 160 (see FIG. 2).

The agent 200 is provided in each table T together with the robot 100. The agent 200 is a device such as a smartphone or a tablet, which is provided with a touch panel and can be operated by a customer C attached to a table T. Like a smartphone, the controller 300 is also equipped with a touch panel and is a device that can be carried by a clerk S who provides customer service. The controller 300 may be in a wearable form.

In the present embodiment, the robot 100, the agent 200, and the controller 300 have a wireless communication function, and a wireless router (not shown) is installed between them and the management control device 700, and the robot 100, the agent 200, and the controller 300 are wirelessly managed and controlled. Communicate with device 700.

FIG. 2 is a block diagram showing the configuration of the customer service system 1, and in particular, shows the internal configuration of the robot 100 and the controller 300. As shown in FIG. 2, the robot 100 has a drive mechanism 110, a display panel 120, a speaker 130, a wireless communication module 140, a camera 150, a microphone 160, and a learning storage unit 170 as hardware configurations. It includes a storage device used and a processor used as a control unit 180. Further, although not shown, the robot 100 is provided with a storage device that stores a customer service program of the present embodiment for causing the processor to function as a control unit 180, and the processor reads and executes the customer service program. As a result, various functions shown in FIG. 2 are realized.

The drive machine 110 drives the hands, feet, hips, and neck of the robot under the control of the control unit 180. The display panel 120 displays the screen under the control of the control unit 180. The speaker 130 outputs sound according to the control of the control unit 180. The wireless communication module 140 performs wireless communication with the management control device 700, transmits information from the robot 100 to the management control device 700, and receives information from the management control device 700.

The camera 150 acquires an optical image through a lens installed on the head portion of the robot 100 and generates video data and image data. The camera 150 is constantly driven, shoots an image at a predetermined frame rate, and shoots an image (still image) in the direction in which the face of the robot 100 is facing at the timing instructed to shoot. The microphone 160 is always driven, and acquires the voice of the customer C attached to the table T to generate voice data.

The learning storage unit 170 stores the result of learning the correspondence relationship between the request information and the request, and the result of learning the correspondence relationship between the request and the customer service action. The control unit 180 includes an action control unit 181, a transmission / reception control unit 182, an image analysis unit 183, a voice analysis unit 184, a learning unit 188, an instruction request unit 187, a request grasping unit 186, and an interpreter unit 185. It has.

The action control unit 181 outputs a command for driving the drive mechanism 110 to the drive mechanism 110, outputs video data or image data to be displayed on the display panel 120 to the display panel 120, and outputs audio data output by the speaker 130 to the speaker 130. Output to.

The transmission / reception control unit 182 acquires information to be transmitted to the management control device 700 and outputs it to the wireless communication module 140, and also distributes the information received by the wireless communication module 140 to each unit. If the received information is foreign language text data or voice data from the age circle and 200, the transmission / reception control unit 182 outputs the received information to the interpretation unit 185.

The image analysis unit 183 analyzes the video frame or image (hereinafter, simply referred to as “image”) input from the camera 150, and recognizes the information necessary for acquiring the request of the customer C from the content of the image. For this recognition, an image recognition technique by machine learning using features can be used. The image analysis unit 183, for example, photographs the table T and recognizes glasses, plates, etc., and further recognizes empty glasses, empty plates, and dishes served on the plates. be able to. Further, the image analysis unit 183 recognizes the gesture by the customer C's hand. As a result of the image analysis, the image analysis unit 183 outputs information indicating, for example, the number of empty glasses, the number of empty plates, the type of dishes served on the plates, and the type of gestures made by the customer. ..

The voice analysis unit 184 analyzes the voice frame input from the microphone 160 and recognizes the information necessary for acquiring the request of the customer C from the content of the voice. For this recognition, a voice recognition technique by machine learning using features can be used. The voice analysis unit 184 recognizes the words spoken by the customer C into the microphone. For this purpose, the voice analysis unit 184 first determines the type of voice language of customer C (Japanese, English, Chinese, French, Portuguese, Korean, etc.). Further, the voice analysis unit 184 converts the voice into text and outputs the text data obtained by the return.

When the voice analysis unit 184 determines that the language is a foreign language, the interpretation unit 185 inputs the text data from the voice analysis unit 184, converts it into Japanese (interpretation), and sends it to the request grasping unit 186. Output. Machine learning technology can also be used for this interpretation (translation).

The request grasping unit 186 grasps the request of the customer C distributed from the transmission / reception control unit 182, and grasps the request of the customer C by analyzing the information output from the image analysis unit 183, and the voice analysis unit 184 or the interpreter. The request of the customer C is grasped by analyzing the text data output from the unit 185. The machine learning technique can also be used for grasping the request in the request grasping unit 186, and the learning result is stored in the learning storage unit 170. Hereinafter, the information input from the transmission / reception control unit 182, the image analysis unit 183, the voice analysis unit 184, and the interpreter unit 185 to the request grasping unit 186 is referred to as "request information". The request grasping unit 186 grasps the request indicated by the request information. The request grasping unit 186 is provided with a timer, and if necessary, time counting by the timer is also used to grasp the request.

For example, when the image analysis unit 183 recognizes the empty glass from the image, the request grasping unit 186 grasps the request for replenishment of water, and when the customer C inputs an additional order to the agent 200, the request grasping unit 186 grasps the request. The content of the additional order is grasped as a request, and when the customer C voice-inputs an inquiry of the recommended product through the microphone 160, the request for guidance of the recommended product is grasped.

The request is prepared in advance, and the request grasping unit 186 maps (corresponds) the request information input from the image analysis unit 183, the voice analysis unit 184, or the interpreter unit 185 to any of the prepared requests. To grasp the request by doing (attach). For example, when a request for "replenishment of water" is prepared in advance, text data such as "please give me water", "replace cold", and "please add water" input from the voice analysis unit 184. Is grasped by mapping to the request of "replenishment of water". As described above, machine learning technology is used to grasp the demand by mapping text data.

Further, the request grasping unit 186 refers to the learning storage unit 170 with respect to the grasped request, and determines the customer service action corresponding to the request. When the customer service action corresponding to the request is obtained from the learning storage unit 170, the request grasping unit 186 instructs the action control unit 181 to perform the customer service action. The action control unit 181 executes a customer service action including driving by the drive mechanism 110, display by the display panel 120, voice output from the speaker 130, or a combination thereof, in accordance with the instruction from the request grasping unit 186.

For example, regarding the request for "replenishment of water", the learning memory unit 170 outputs a voice asking whether or not replenishment of water is necessary from the speaker 130 and a customer service action of moving the limbs. Are associated with each other. Further, in response to the request for an additional order, the learning storage unit 170 is associated with a customer service action of reciting the order contents on the speaker 130 and displaying the ordered product on the display panel 120 in accordance with the recitation of the order. ing.

The request grasping unit 186 outputs the request information to the instruction requesting unit 187 when the request cannot be grasped from the input request information or when the customer service action corresponding to the grasped request cannot be found in the learning storage unit 170. .. The instruction request unit 187 generates an instruction request for inquiring the controller 300 of the clerk S about the customer service action for the request information output from the request grasping unit 186, and outputs the instruction request to the transmission / reception control unit 182.

For example, if the voice analysis unit 184 or the interpreter unit 185 inputs text data as request information to the request grasping unit 186, but the request grasping unit 186 cannot grasp the request of the customer C from the text data, the text The data (request information) is output to the instruction request unit 187. When the request information is input from the request grasping unit 186, the instruction request unit 187 transmits the request information to the clerk S, generates an instruction request for asking for an instruction, and outputs the instruction request to the transmission / reception control unit 182. If the request grasping unit 186 cannot grasp the request of the customer C from the analysis result of the image analysis unit 183, the instruction requesting unit 187 generates an instruction request including the image and outputs it to the transmission / reception control unit 182. ..

The learning unit 188 learns the combination of the request information or request related to the instruction request generated by the instruction request unit 187 and the customer service action for the request information or request. The customer service action is shown in the instruction by the clerk S, and the instruction is received by the wireless communication module 140 from the controller 300 via the management control device 700, and is distributed to the learning unit 188 by the transmission / reception control unit 182. The result of learning by the learning unit 188, that is, the request information or the combination of the request and the customer service action is stored in the learning storage unit 170.

As shown in FIG. 1, in the restaurant, by installing the robot 100 on each table T, a plurality of robots 100 are installed in the restaurant as a whole. The learning result is not only stored in the learning storage unit 170 of the robot, but may be shared by a plurality of robots 100. That is, the content stored in the learning storage unit 170 of the plurality of robots 100 may be common to the plurality of robots 100.

The controller 300 includes a wireless communication module 310, a control unit 320, and a touch panel 330. The wireless communication module 310 wirelessly communicates with the management control device 700 via a wireless router. The control unit 320 performs various types of information processing by executing a predetermined program. The touch panel 330 displays a screen and accepts touch input by the clerk S. The control unit 320 performs information processing according to a program based on the information received by the wireless communication module 310 and the input to the touch panel 330, transmits the information to the management control device 700 through the wireless communication module 310, and also causes the touch panel 330. Display the screen on.

The agent 200, the accounting device 500, the receiving terminal 400, and the order receiving device 600 also have the same configuration as the controller 300, although not shown, and by executing each program, each information processing described later is performed. .. In particular, the agent 200 is also provided with a microphone, and can acquire the voice of the customer C and generate voice data. Customer C can select and input the agent 200 by using a button or the like displayed on the touch panel 330, can input text using the touch panel 330, and input voice to the microphone. Can be done. The selection data, text data, and voice data generated by the input of customer C are transmitted by the wireless communication module.

Similarly, the management control device 700 includes a wireless communication module and a control unit, and receives and receives information from the robot 100, the agent 200, the controller 300, the reception terminal 400, the accounting device 500, and the order receiving device 600. Determine the content of the information and transfer the information to one of the devices.

(Usage mode of customer service system 1)
Next, a usage mode of the customer service system 1 of the present embodiment will be described. 3 to 5 are views for explaining the usage mode of the customer service system 1. In FIGS. 3 to 5, the management control device 700 and the like are not shown.

FIG. 3 shows a first usage mode. In the first usage mode, a request is issued from the customer C (step s11) and input to the request grasping unit 186 of the robot 100. Specifically, the customer C can input the request information to the request grasping unit 186 via the microphone 140 by voice by talking to the robot 100. Further, the customer C can input the request information into the request grasping unit 186 by inputting the text using the agent 200. The contents of the request of the customer C in this usage mode are, for example, "Please replace the water" and "Please bring the menu".

The robot 100 that receives the request grasps the request of the customer C in the request grasping unit 186, and if it can grasp the request, determines the corresponding customer service action by referring to the learning storage unit 170. When the corresponding customer service action is stored in the learning storage unit 170, the action control unit 181 controls the drive mechanism 110, the display panel 120, and / or the speaker 130 so as to execute the customer service action. The drive mechanism 110, the display panel 120, and / or the speaker 130 execute a customer service action under the control of the action control unit 181 (step s12).

When the request grasping unit 186 cannot grasp the request from the text or voice request information input from the customer C, or when the customer service action corresponding to the grasped request is not stored in the learning storage unit 170, the instruction request is made. The instruction request of the customer C is generated by the unit 187, and the instruction request is transmitted from the wireless communication module 140 to the controller 300 under the control of the transmission / reception control unit 182 (step s13).

When the controller 300 receives the instruction request, it displays it on the touch panel 330. The clerk S inputs an instruction for the instruction request displayed on the touch panel 330 to the touch panel 330. This instruction is given by entering text and / or selecting a prepared customer service action. The instruction input to the controller 300 is transmitted to the robot 100 (step s14), and the robot 100 receives this and executes a customer service action in response to the instruction (step s12).

For example, in the controller 300, when a text is input by the clerk S and a customer service action of voice output is instructed, the action control unit 181 of the robot 100 that receives this instruction reads out the text from the speaker 130. Output.

As described above, in the first usage mode, it is possible to realize customer service in which the customer C issues a request to the robot 100 and the robot 100 responds to the request. If the robot 100 cannot determine the customer service action in response to the request of the customer C, the clerk S can make a determination and cause the robot 100 to execute the action, but it seems as if the robot 100 corresponds to the customer C. ..

As another example of the first usage mode, the customer C makes a predetermined gesture toward the robot 100 (camera 130), photographs the gesture with the camera 130, and then the image analysis unit 183 and the request grasping unit. The 186 may analyze the content of the gesture to grasp the request. In this case as well, it is possible to realize customer service in which the customer C issues a request message (gesture) to the robot and the robot 100 responds to the request.

FIG. 4 shows a second usage mode. In the second usage mode, the robot 100 captures the situation of the table T with the camera 130, the image analysis unit 183 analyzes the image, and the request grasping unit 186 grasps the request of the customer C (step s21). Specifically, the camera 130 always shoots on the table T and outputs a video frame to the image analysis unit 183. The image analysis unit 183 performs image analysis on continuously input video frames, and outputs the result as request information to the request grasping unit 186. The request grasping unit 186 acquires the request of the customer C by grasping the request based on the result of the image analysis and the timing by the timer.

After the request grasping unit 186 grasps the request, as in the first usage mode, the learning storage unit 170 is referred to, and if the corresponding customer service action is obtained, the customer service action is executed (step s22). If not obtained, an instruction request is transmitted to the controller 300 (step s23). Then, when an instruction is returned in response to the instruction request (step s24), the customer service action is executed according to the instruction (step s22).

For example, when it is recognized that the plate is empty as a result of image analysis, when the empty plate is placed on the table T for a predetermined time, a voice such as "Should I lower the plate?" Output and execute a customer service action asking if it is necessary to put away the dishes. Further, for example, when the dessert is not ordered, a voice prompting the addition of the dessert is output after the food is served and a predetermined time has elapsed.

On the other hand, for example, when a predetermined time (long time) has passed without the plate being emptied, customer C is still eating slowly (the plate should not be lowered) or has stopped eating (plate). Since the request grasping unit 186 cannot determine whether or not the table T may be lowered), the instruction requesting unit 187 generates an instruction request including an image of the situation of the table T and the elapsed time, and the transmission / reception control unit 182 performs this. The instruction request is transmitted to the controller 300. The clerk S considers the image and the elapsed time in the instruction request sent, and if necessary, visually inspects the situation of the table T and the customer C and asks whether it is necessary to clean up the dishes. It is possible to instruct pre-prepared customer service actions such as asking whether or not reheating is necessary.

As described above, in the second usage mode, even if the customer C does not make a request by himself / herself, the robot 100 can grasp the request and execute the customer service action in consideration of the image analysis result and the passage of time. .. If the robot 100 cannot grasp the request or determine the customer service action to be executed, the clerk S can make a determination and let the robot 100 execute it, but the robot 100 responds to the customer C as if the robot 100 were all in charge. Looks like it is. In this usage mode, the robot 100 can grasp the request that the clerk S does not notice and it is difficult for the customer C to request the clerk S, and can execute the customer service action by the robot 100, and the customer service hospitality can be improved.

FIG. 5 shows a third usage mode. In the third usage mode, the clerk S visually confirms the situation of the table T or the customer C and grasps the request of the customer C (step s31). The clerk S inputs a customer service action for the grasped request to the controller 300, and transmits this as an instruction from the controller 300 to the robot 100 (step s32). The robot 100 receives the instruction and executes the instructed customer service action (step s33).

This third usage mode is effective when the clerk S is away from the table T and the situation of the table T and the customer C can be grasped but the customer service cannot be provided. By remotely controlling the robot 100, the clerk S can have the robot 100 provide the customer service to the customer C on behalf of the clerk S. For example, when a distant clerk S notices that the customer C has dropped the fork from the table, the clerk S prepares a replacement fork and gives an instruction to the robot 100 to "replace" the robot 100. I'd like you to bring my fork. Is your clothes dirty? "

As described above, in the third usage mode, when the clerk S notices the request of the customer C, the robot 100 can execute the customer service action. In this usage mode, the man-hours of the clerk S can be reduced, and the customer service hospitality can be improved even when the number of the clerk S is insufficient. Further, since the clerk S grasps the request, even if the situation is difficult for the robot 100 to deal with, it seems to the customer C that the robot 100 makes a judgment and executes the customer service action.

(Operation example of customer service system 1)
Hereinafter, a specific operation example of the customer service system 1 will be described.

FIG. 6 is a flow chart showing an operation example from the visit of the customer C to the self-introduction of the robot 100. When the customer C comes to the store (step S210), the clerk S greets the customer C and greets him (step S110), and guides the input to the reception terminal 400 (step S111). Customer C inputs necessary items such as the number of people and requests for a table into the reception terminal 400 (step S211), and waits with a number tag (step S212).

The clerk S sets the table (step S113), calls in the waiting customer C (step S114), and guides the customer to the table T (step S116). Meanwhile, the clerk S presses the "guidance" button on the controller 300 (step S115), and notifies the management control device 700 that the waiting state of the customer C has ended. The management control device 700 releases the waiting state of the customer C input to the reception terminal 400. The customer C guided to the table T is seated at the table T (step S213).

When the clerk S confirms that the customer C is seated at the table T, the clerk S presses the "greeting" button on the controller 300 (step S117). This serves as a greeting instruction and is transmitted from the controller 300 to the robot 100 via the management control device 700. In the robot 100, the wireless communication module 140 receives this instruction, and the transmission / reception control unit 182 outputs the instruction to the action control unit 181. The action control unit 181 executes a greeting customer service action according to this instruction (step S310). Specifically, the robot 100 drives the hip joint and bows while outputting a voice of a greeting (“Welcome”, etc.).

The clerk S confirms that the attention of the customer C is directed to the robot 100 by the greeting of the robot 100, and guides the customer C to select a language by using the agent 200 (step S118). Customer C uses the agent 200 to select a language. This selection is transmitted from the agent 200 to the robot 100 via the management control device 700, and the robot 100 receives this as a self-introduction instruction and performs a self-introduction customer service action in the selected language. Specifically, the robot 100 outputs a voice of self-introduction prepared in advance in a selected language while driving a hand or a neck.

The operation example of FIG. 6 is an operation example in which the clerk S gives an instruction to the robot 100 while visually observing the situation, and corresponds to the above-mentioned third usage mode.

FIG. 7 is a flow chart showing an operation example from ordering to the start of cooking. When the self-introduction (step S311 in FIG. 6) is completed, the robot 100 subsequently provides guidance on the ordering method and recommended products (step S320). Customer C operates the agent 200 according to this guidance to start processing an order (step S220). In the processing of the order, when the customer C operates the screen of the agent 200 to request the product explanation (step S221), the robot 100 gives a detailed explanation of the product in response to this (step S321), and the customer C is out of stock. When a product or a product having a long provision time is selected (step S222), the robot 100 notifies by voice that the product is out of stock or the product is provided for a long time (step S322).

When the customer C instructs the agent 200 to confirm the order (step S223), the robot 100 repeats the selected product (step S323). At this time, the robot 100 displays an image of the product selected according to the repeat on the display panel 120. When the customer C confirms the order by the agent 200 (step S224), the robot 100 announces the order confirmation (step S324).

At this time, the confirmed order is transmitted from the agent 200 to the order receiving device 600 via the management control device 700, and the order slip is created in the order receiving device 600 (step S120). As described above, this order slip may be a screen or a printout. The clerk in the kitchen area K starts cooking according to this order slip.

FIG. 8 is a flow chart showing an operation example in which the customer C talks to the robot 100 and the robot 100 responds to the robot 100 to perform a customer service action. When the customer C operates the agent 200 and presses the "speak" button (step S230), the robot 100 starts the "speak" guidance in response to this (step S330). When the customer C speaks to the agent 200 according to the guidance (step S231), the robot 100 performs a response process (step S331).

FIG. 9 is a flow chart of the response process. In the response process, the wireless communication module 140 of the robot 100 receives voice from the agent 200 via the management control device 700 (step S3310), and the transmission / reception control unit 182 outputs the received voice to the voice analysis unit 184. The voice analysis unit 184 analyzes the received voice to generate a text (step S3311), and determines whether or not the language is a foreign language (step S3312). If it is a foreign language (YES in step S3312), the interpreting department 185 interprets it into Japanese (step S3313). If it is Japanese (NO in step S3312), the text data obtained by the voice analysis unit 184 is translated into Japanese by the interpreter unit 185 if it is in a foreign language (YES in step S3312). , Each is output to the request grasping unit 186 as request information.

The request grasping unit 186 grasps (acquires) the request from the request information (step S3314). If the request can be obtained (YES in step S3314), the customer service action for the request is searched from the learning storage unit 170 (step S3315). When the customer service action for the request is stored in the learning storage unit 170 and can be handled by the robot 100 (YES in step S3315), the customer service action is executed (see also step S333 and FIG. 8).

If the request grasping unit 186 cannot grasp the request (NO in step S3314) or the customer service action corresponding to the learning storage unit 170 is not stored (NO in step S3315), the request grasping unit 186 The request information (text data) is output to the instruction request unit 187. The instruction request unit 187 generates an instruction request and transmits the instruction request to the controller 300 via the transmission / reception control unit 182 and the wireless communication module 140 (see also step S332 and FIG. 8).

The operation example of FIG. 8 is an operation example in which the customer C positively conveys a request to the robot 100, and corresponds to the above-mentioned first usage mode.

FIG. 10 is a flow chart showing an operation example in which the robot 100 photographs the table T and performs a customer service action according to the situation of the table T. The robot 100 constantly photographs the table T using the camera 130, and outputs video frames to the image analysis unit 183 at predetermined frame intervals (step S340). The robot 100 that has obtained the video frame performs a response process (step S341).

FIG. 11 is a flow chart of the response process. In the response process, the image analysis unit 183 of the robot 100 analyzes the received video frame (step S3411) and outputs the analysis result as request information to the request grasping unit 186. The request grasping unit 186 grasps (acquires) a request from the analysis result (step S3412). If the request can be obtained (YES in step S3412), the customer service action for the request is searched from the learning storage unit 170 (step S3413). When the customer service action for the request is stored in the learning storage unit 170 and can be handled by the robot 100 (YES in step S3413), the customer service action is executed (see also step S343 and FIG. 10).

If the request grasping unit 186 cannot acquire the request (NO in step S3412) or the customer service action corresponding to the learning storage unit 170 is not stored (NO in step S3413), the request grasping unit 186 The video frame is output to the instruction request unit 187. The instruction request unit 187 generates an instruction request and transmits the instruction request to the controller 300 via the transmission / reception control unit 182 and the wireless communication module 140 (see also step S342 and FIG. 10).

When the controller 300 receives the instruction request, it outputs the instruction request to the touch panel 330 (step S140). When the clerk S inputs an instruction to the touch panel 330 in response to the instruction request (step S150), the controller 300 transmits the instruction from the wireless communication module 310 to the robot 100 (step S142).

When an instruction is transmitted from the controller 300, the robot 100 receives the instruction in the wireless communication module 310, the transmission / reception control unit 182 inputs the instruction to the action control unit 181 and the action control unit 181 follows the instruction to drive the mechanism. The customer service action is executed by controlling the 110, the display panel 120, and / or the speaker 130 (step S343). Further, in the response process (step S341), when the request grasping unit 186 can grasp the request (YES in step S3412) and can respond by referring to the learning storage unit 170 (YES in step S3413). , The request grasping unit 186 instructs the action control unit 181 to perform the corresponding customer service action, and executes the customer service action (step S343).

When the instruction is transmitted from the controller 300 (step S142), the customer service action is executed as described above (step S343), and the transmission / reception control unit 182 also outputs the instruction to the learning unit 188. The learning unit 188 learns the relationship between the request related to the instruction request and the instruction by the instruction request generated by the instruction request unit 187 and the instruction input from the transmission / reception control unit 182 (step S344). The learned result is stored in the learning storage unit 170.

The operation example of FIG. 10 is an operation example in which the robot 100 grasps the request of the customer C by itself based on the captured image and performs the customer service action, and corresponds to the above-mentioned second usage mode.

FIG. 12 is a flow chart of an operation example in which the clerk S grasps the request of the customer C and causes the robot 100 to perform a customer service action. When the clerk S recognizes the request by looking at the situation of the table T or the customer C while patrolling the floor area F, the clerk S inputs the recognized request to the controller 300 as request information (step S150). The controller 300 transmits the input request information to the robot 100 (step S151). The robot 100 receives the request information from the controller 300 and performs a response process (step S350).

FIG. 13 is a flow chart of the response process. In the response process, the wireless communication module 140 receives the request information from the controller 300 (step S3501), and the transmission / reception control unit 182 inputs this to the request grasping unit 186. The request grasping unit 186 grasps the request based on the request information (step S3502), and if the request can be grasped (YES in step S3502), searches the customer service action for the request from the learning storage unit 170 (step S3502). S3413). When the customer service action for the request is stored in the learning storage unit 170 and can be handled by the robot 100 (YES in step S3503), the customer service action is executed (see also step S352 and FIG. 12).

If the request grasping unit 186 cannot grasp the request (NO in step S3502) or the customer service action corresponding to the learning storage unit 170 is not stored (NO in step S3503), the request grasping unit 186 The request information is output to the instruction request unit 187. The instruction request unit 187 generates an instruction request and transmits the instruction request to the controller 300 via the transmission / reception control unit 182 and the wireless communication module 140 (see also step S351 and FIG. 12). The instruction request in this case may be a notification that the received request information cannot be handled.

When the controller 300 receives the instruction request, it outputs the instruction request to the touch panel 330 (step S152). When the clerk S inputs an instruction to the touch panel 330 in response to the instruction request (step S153), the controller 300 transmits an instruction from the wireless communication module 310 to the robot 100 (step S154).

When an instruction is transmitted from the controller 300, the robot 100 receives the instruction in the wireless communication module 310, the transmission / reception control unit 182 inputs the instruction to the action control unit 181 and the action control unit 181 follows the instruction to drive the mechanism. The customer service action is executed by controlling the 110, the display panel 120, and / or the speaker 130 (step S352). Further, in the response process (step S350), when the request grasping unit 186 can grasp the request (YES in step S3502) and can respond by referring to the learning storage unit 170 (YES in step S3503). , The request grasping unit 186 instructs the action control unit 181 to perform the corresponding customer service action, and executes the customer service action (step S352).

When the instruction is transmitted from the controller 300 (step S154), the customer service action is executed as described above (step S352), and the transmission / reception control unit 182 also outputs the instruction to the learning unit 188. The learning unit 188 learns the relationship between the request related to the instruction request and the instruction based on the instruction request generated by the instruction request unit 187 and the instruction input from the transmission / reception control unit 182 (step S353). The learned result is stored in the learning storage unit 170.

The operation example of FIG. 12 is an operation example in which the clerk S grasps the request of the customer C and causes the robot to perform the customer service action, and corresponds to the above-mentioned third usage mode.

As described above, according to the customer service system 1 of the present embodiment, the robot 100 can determine the request of the customer C by himself / herself and execute the customer service action, and when the robot 100 cannot determine, the clerk S carries it. Since the controller 300 can give an instruction to the robot 100, not only the necessary customer service can be provided, but also the customer service hospitality can be realized. In addition, the robot 100 can learn the customer service hospitality provided by the clerk S by the OJT method.

(Modification example)
Hereinafter, various modification examples will be described. In the above embodiment, the restaurant is provided with the reception / accounting area E, the floor area F, and the kitchen area K, but an office or a clerk break room may be further provided. Further, a plurality of cameras are set in the floor area F, a monitor for displaying the captured images is set in the office, and the clerk S acts as a monitoring specialist on the monitor in the office, and the table T and the customer C are used. You may monitor the situation of. The clerk S specializing in monitoring may use the controller 300 to receive an instruction request and give an instruction to the robot 100 of each table T. In this case, the controller 300 may be a non-portable personal computer or the like.

Further, in the above embodiment, the agent 200 is provided at each table T of the restaurant, but instead of the agent 200, a smartphone owned by the customer C may be used. In this case, by downloading the application for this customer service system to the smartphone of customer C and then pairing with the robot 100 installed on the seated table T, this smartphone can be used instead of the agent 200. Can be used. Further, the customer service system 1 of the present invention does not include the agent 200, and the customer C may directly input the selection input, the text input, or the voice input to the robot 100.

Further, in the above embodiment, a plurality of clerk S carry the controller 300, and any clerk S can give an instruction to the robot 100, but one controller 300 can give an instruction to the robot 100. It may be limited to the controller 300 of the unit. That is, only an experienced clerk S may be able to give an instruction to the robot 100 using the controller 300.

Further, in the above-described embodiment, all communication between the robot 100, the agent 200, the controller 300, and the like is performed via the management control device 700, but these may perform direct communication. In particular, the agent 200 may perform short-range wireless communication with the robot 100 such as Bluetooth (registered trademark), and may be able to communicate with the controller 300 or the like via the robot 100.

Further, in the above embodiment, the control unit 180 of the robot 100 includes an image analysis unit 183, a voice analysis unit 184, an interpreter unit 185, a request grasping unit 186, an instruction request unit 187, and a learning unit 188. Although the storage unit 170 is also provided in the robot 100, a part or all of them may be provided in an external device of the robot 100. In this case, the external device may communicate with the robot 100, the agent 200, and the controller 300 by wire or wirelessly. The management control device 700 of the above embodiment may serve as this external device.

Further, in the above embodiment, when the robot 100 asks the clerk S for an instruction, the robot 100 sends an instruction request to the controller 200, but whether or not the robot 100 needs to ask the clerk S for an instruction. Not limited to this, the robot 100 may instruct the controller 300 of the clerk S to instruct the customer service to be performed by the clerk S in response to the request of the customer C grasped by the robot 100. For example, when the robot 100 grasps that the water in the cup of the table T is empty and the customer C requests the replenishment of water, the robot 100 may instruct the clerk S to replenish the water. When the robot 100 grasps the request of the customer C to urge the food when the food is not served, the robot 100 issues a food reminder alarm to the controller 300 and the order receiving device 600 installed in the kitchen area K. You may instruct.

In this case, the contents that the robot 100 can instruct the clerk S may be limited in consideration of the division of roles between the robot 100 and the clerk S (person) and teamwork.

Further, in the above embodiment, the robot 100 is installed on the table T in the restaurant, but the robot 100 may be installed at the reception desk of the restaurant. In this case, the robot 100 may perform waiting time guidance, recommended product guidance, and the like as customer service actions.

Further, in the above embodiment, the example in which the customer service system 1 is applied to a restaurant has been described, but the customer service system of the present invention can also be applied to customer service in, for example, a movie theater, a hotel, a department store, etc. other than the restaurant. Is. When the customer service system 1 of the present invention is introduced into a hotel, the robot 100 is provided in each guest room used by the guest who is the service recipient, and the employee of the hotel who is the service provider carries the controller 300. Often, the controller 300 may be a personal computer used by the above-mentioned monitoring specialist staff. When the customer service system 1 of the present invention is introduced into a department store, the robot 100 can be installed in a resting place or an information center used by shoppers who are service recipients, and an employee of the department store who is a service provider can install the robot 100. The controller 300 may be carried, and the controller 300 may be a personal computer used by the above-mentioned staff specializing in monitoring.

As described above, the customer service system of the present invention has an effect that customer service hospitality can be realized or improved by collaboration between a robot and a person, and is useful as a customer service system or the like that provides customer service at a restaurant or the like.

1 Customer service system 100 Robot 110 Drive mechanism 120 Display panel 130 Speaker 140 Wireless communication module 150 Camera 160 Microphone 170 Learning storage unit 180 Control unit 181 Action control unit 182 Transmission / reception control unit 183 Image analysis unit 184 Voice analysis unit 185 Interpretation unit 186 Request grasping Department 187 Instruction request unit 188 Learning unit 200 Agent 300 Controller 310 Wireless communication module 320 Control unit 330 Touch panel 400 Reception terminal 500 Accounting device 600 Order receiving device 700 Management control device E Reception / accounting area F Floor area K Kitchen area T Table C Customer S clerk

Claims (8)

A robot provided in a table used by a service beneficiary and equipped with an action means for executing a customer service action for the service beneficiary, a robot communication means for transmitting and receiving information, and a camera for capturing an image of the table. ,
A controller operated by a service provider, which includes a controller input means for receiving input from a service provider, a controller output means for outputting information to the service provider, and a controller communication means for transmitting and receiving information.
An image analysis means for analyzing an image taken by the camera and
A request grasping means that grasps the request of the service beneficiary using the analysis result of the image analysis means as request information and instructs the action means of a customer service action corresponding to the grasped request.
When the request grasping means cannot grasp the request of the service beneficiary based on the request information, the instruction request means for generating the instruction request based on the request information and the instruction request means.
It is a customer service system equipped with
The controller output means outputs the instruction request and outputs the instruction request.
The controller input means receives an input of an instruction from a service provider in response to the instruction request, and receives the input of an instruction.
The controller communication means transmits the instruction to the robot, and the controller communication means transmits the instruction to the robot.
The robot communication means receives the instruction and receives the instruction.
The action means executes the customer service action instructed by the request grasping means, and also executes the customer service action in accordance with the instruction received by the robot communication means.
Customer service system. The customer service system further comprises an agent operated by the service beneficiary.
The agent
An agent input means that accepts input by the service beneficiary, and
The agent communication means for transmitting the input received by the agent input means is provided.
The request grasping means further grasps the request of the service beneficiary by using the input transmitted from the agent communication means as the request information.
The customer service system according to claim 1. The robot is further equipped with a microphone that accepts voice input.
The customer service system further includes a voice analysis means for analyzing the content of the voice input to the microphone.
The request grasping means further grasps the request of the service beneficiary by using the analysis result of the voice analysis means as the request information.
The customer service system according to claim 1. The robot is further equipped with a microphone that accepts voice input.
The customer service system further
A voice analysis means for analyzing the content of the voice input to the microphone, and
An interpreter that translates the content of the voice analyzed by the voice analysis means into another language, and
With
The request grasping means further grasps the request of the service beneficiary by using the translation result of the interpreting means as the request information.
The customer service system according to claim 1. The controller input means receives the request of the service beneficiary grasped by the service provider as the request information, and receives the request information.
The controller communication means transmits the request information input to the controller input means, and the controller communication means transmits the request information.
The request grasping means further grasps the request of the service beneficiary based on the request information transmitted from the controller communication means.
The customer service system according to claim 1. The customer service system according to any one of claims 1 to 5, further comprising a learning means for learning an instruction by the service provider for the request or the request information. A robot equipped with an action means for executing a customer service action for the service beneficiary at a table used by the service beneficiary, and a camera for taking an image of the table.
The controller operated by the service provider and
An image analysis means for analyzing an image taken by the camera and
A request grasping means that grasps the request of the service beneficiary using the analysis result of the image analysis means as request information and instructs the action means to take a customer service action corresponding to the grasped request.
With
The action means executes the customer service action instructed by the request grasping means, and when the request grasping means cannot grasp the request by the request information, the service provider obtains the request information. A customer service system that executes a customer service action according to an instruction input to the controller. A customer service method using the customer service system according to claim 1.
When the request grasping means cannot grasp the request of the service beneficiary based on the request information by using the instruction request means, an instruction request is generated based on the request information.
The instruction request is output by the controller output means, and the instruction request is output.
The controller input means accepts the input of an instruction from the service provider in response to the instruction request, and receives the input.
The instruction is transmitted from the controller communication means to the robot,
Upon receiving the instruction by the robot communication means,
The desire grasping means, when grasping the needs of the service receiver analysis results as demand information of the image analysis device, instructs the customer action corresponding to desire grasped on the action means,
Using the action means, the customer service action instructed by the request grasping means is executed, and the customer service action is executed according to the instruction received by the robot communication means.
How to serve customers.

Images