JP4798577B2 - Dialogue support system - Google Patents

Description

  The present invention relates to a dialog support system, for example, a dialog support system that supports a dialog when a person interacts with a person via a robot.

An example of a conventional system of this type is disclosed in Patent Document 1. According to this prior art, when a user interacts with a partner, the control unit acquires the partner's profile from the database, and the acquired profile is displayed on the display unit. As a result, the dialogue is activated.
JP 2003-157224 A

    However, in the prior art, a personal computer, a portable terminal, and an Internet television are assumed as information devices operated by the user. In personal computers and mobile terminals, since the object to be faced is inorganic, for example, a child may feel resistance when proceeding with a dialogue, and activation of the dialogue may be hindered. In addition, since the other party's face is displayed on the Internet TV, activation of the dialogue may be hindered for users who react sensitively to changes in the other party's face impression and facial expression.

    Therefore, a main object of the present invention is to provide a novel dialogue support system capable of eliminating such a sense of hindrance and activating dialogue.

The dialogue support system (10) according to the invention of claim 1 includes a robot (12) facing a user (CS), a database server (16) for managing dialogue support information, and a terminal device operated by an operator (OP). (18), in which the operator interacts with the user via the robot and the terminal device. The robot includes an identification unit (12b) for identifying a user and a first interaction processing unit (S9) for executing a dialogue process between the user and the operator, and the database server supports dialogue support that matches the identification result of the identification unit Extraction means (S23, S25) for extracting information, and the terminal device provides presentation means (S47) for presenting the dialogue support information extracted by the extraction means to the operator, and dialogue processing between the user and the operator. Second dialog processing means (S55) to be executed is provided.

  The robot faces the user, the database server manages the dialogue support information, and the terminal device is operated by the operator. With respect to the robot, the identification unit identifies the user, and the first dialogue processing unit executes dialogue processing between the user and the operator. As for the database server, the extraction means extracts dialogue support information that matches the identification result of the identification means. As for the terminal device, the presenting means presents the dialog support information extracted by the extracting means to the operator, and the second dialog processing means executes the dialog processing between the user and the operator.

  That is, the operator operates the terminal device while the user faces the robot. Further, when a dialogue is performed between the operator and the user, dialogue support information is presented to the operator through the terminal device. A message issued by the operator referring to the dialogue support information is transmitted to the user through the robot. A message issued by the user to the robot is transmitted to the operator through the terminal device. Thus, since the operator refers to the dialogue support information and the user faces the robot, an active dialogue led by the operator is realized.

The dialogue support system according to the invention of claim 2 is dependent on claim 1, and the first dialogue processing means transmits an instruction from the operator to the user by at least one of voice and gesture, and information from the user is voice and gesture. by at least one of Ru communicated to the operator.

  The dialogue support system according to the invention of claim 3 is dependent on claim 1 or 2, and the second dialogue processing means sends the dialogue information from the user and the dialogue support information from the database server transmitted to the operator to the operator. Information is transmitted to the user via the robot based on the dialogue information and the dialogue support information.

  According to the present invention, since the operator refers to the dialogue support information and the user faces the robot, an active dialogue led by the operator is realized.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a dialogue support system 10 of this embodiment includes a communication robot (hereinafter simply referred to as “robot”) 12 facing a user CS holding a member card CD in which a tag TG is embedded, and a user CS. A database server 16 for managing personal information and additional information of a plurality of users, and a terminal device 18 operated by an operator OP. All of the robot 12, the database server 16, and the terminal device 18 are connected to the public Internet network 14. The dialogue between the user CS and the operator OP is realized by using such a dialogue support system 10.

Robot 12 is humanoid, head to have eye camera 12a and a speaker 12c, each having a microphone 12d and tag reader 12b to the chest and abdomen. The appearance of the user CS is captured by the eye camera 12a, and the voice of the user CS is captured by the microphone 12d. Further, the voice of the operator OP is output from the speaker 12c, and the information on the tag TG is read by the tag reader 12b. The terminal device 18 includes a microphone 18a that captures the speech of the operator OP, a speaker 18b that outputs the speech of the user CS, and a monitor 18c that reflects the appearance of the user CS.

  Referring to FIG. 2, the robot 12 includes a CPU 12j for overall control. The CPU 12j is connected to a sensor input / output board 12e, an audio input / output board 12f, a memory 12g, a motor control board 12h, and a wireless LAN board 12k via a bus 12n.

Sensor output board 12e is, for example, a DSP (Digital Signal Processor), and a tag information read by the image data and the tag reader 12b outputted from the eye camera 12a, and inputs the CPU12j.

  The speaker 12c is supplied with audio data from the CPU 12j via the audio input / output board 12f, and outputs audio corresponding to the audio data. The voice data corresponding to the voice of the user CS input from the microphone 12c is taken into the CPU 12j via the voice input / output board 12f.

  Although not shown, the memory 12g includes a ROM, a HDD, and a RAM, and various data such as a program for controlling the robot 12 as a whole are written in the ROM and the HDD in advance. The RAM is used as a temporary storage memory and a working memory.

  The motor control board 12h is also constituted by a DSP, and operates the robot 12 in accordance with control data from the CPU 12j. Various motors (right arm motor system, left arm motor system, head motor system, waist motor system, wheel motor system, right eye motor system, and left eye motor system) are driven.

  The wireless LAN board 12k is connected to the public Internet network 14 (see FIG. 1) via the I / F 12m. The voice data corresponding to each of the voice message to be output from the speaker 12c and the voice message taken in from the microphone 12d, the video signal of the user CS from the camera 12a, the tag information from the tag reader 12b, etc. are transmitted via this I / F 12m. Sent and received.

  As shown in FIG. 3, the database server 16 includes a database 16a, a CPU 16b, a memory 16c, and an I / F 16d. The database 16a stores the personal information and additional information described above. The CPU 16b executes a program stored in the memory 16c. The I / F 16 d is connected to the public Internet network 14.

  As shown in FIG. 4, the communication terminal 18 includes a microphone 18a, a speaker 18b, a monitor 18c, a keyboard 18d, a memory 18e, a CPU 18f, and an I / F 18g. The microphone 18a generates voice data corresponding to the voice of the operator OP, and the speaker 18b outputs voice corresponding to the voice data given from the I / F 18g. The keyboard 18d is operated by the operator OP, and the CPU 18f executes the program stored in the memory 18e. The I / F 16 d is connected to the public Internet network 14.

  The exchange of information among the robot 12, the database server 16, and the terminal device 18 is executed as shown in FIG. When the user CS comes in front of the robot 12, information on the tag TG is read by the tag reader 12b of the robot 12. The read tag information, that is, the person identification information is transmitted to the terminal device 18 via the I / F 12m and the public Internet network 14.

  The terminal device 18 requests the database server 16 to inquire about the person identified by the received tag information. This inquiry request includes the received tag information.

  The database server 16 takes in the transmitted inquiry request through the I / F 16d, and extracts personal information and additional information that match the tag information added to the incorporated inquiry request from the database 16a. The dialogue support information including the extracted personal information and additional information is transmitted to the terminal device 18 via the I / F 16d and the public Internet network 14.

  The terminal device 18 fetches the transmitted dialogue support information through the I / F 18g, and displays the fetched dialogue support information on the monitor 18c. The operator OP refers to the dialogue support information presented in this way, and determines a topic for proceeding with the dialogue with the user CS. When the operator OP performs a dialog start operation on the keyboard 18d, the microphone 18a, the speaker 18b, and the monitor 18c are turned on, and a dialog start command is sent to the robot 12 and the database server 16 via the I / F 18g and the public Internet network 14. Sent.

  When the robot 12 fetches the transmitted dialogue start command from the I / F 12m, the robot 12 turns on the speaker 12c, the microphone 12d, and the eye camera 12a. The dialogue process between the operator OP and the user CS is started after the microphone 18a, speaker 18b and monitor 18c on the terminal device 18 side, and the speaker 12c, microphone 12d and eye camera 12a on the robot 12 side are turned on.

  The voice data indicating the content of the dialogue performed between the operator OP and the user CS and the image data indicating the appearance (particularly the face) of the user CS are also given to the database server 16. In response to the dialog start command received from the terminal device 18, the CPU 16b of the database server 16 starts a recording process in which such audio data and image data are stored in the database 16a in association with the user CS.

  When the operator OP performs a dialog end operation on the keyboard 18d, the microphone 18a, the speaker 18b, and the monitor 18c are turned off, and a dialog end command is sent to the robot 12 and the database server 16 via the I / F 18g and the public Internet network 14. Sent. The robot 12 turns off the speaker 12c, the microphone 12d, and the eye camera 12a when the transmitted dialogue end command is received from the I / F 12m. As a result, the interactive process is terminated. The database server 16 ends the above-described recording process in response to the dialog end command from the terminal device 18.

  As described above, the operator OP operates the terminal device 18, while the user CS faces the robot 12. Further, when a dialogue is performed between the operator OP and the user CS, dialogue support information related to the user CS is presented to the operator OP through the terminal device 18. A message issued by the operator OP referring to the dialogue support information is transmitted to the user CS through the robot 12. Further, a message issued by the user CS to the robot 12 is transmitted to the operator OP through the terminal device 18. Since the operator OP refers to the dialogue support information and the user CS faces the robot 12, an active dialogue led by the operator OP is realized. Further, the dialogue contents between the operator OP and the user CS and the face image of the user CS are stored in the database 16 a of the database server 16. As a result, each time the user CS visits the robot 12, personal information of the user CS is enriched.

  The CPU 12j of the robot 12 executes the process shown in FIG. 6 according to the control program stored in the memory 12g.

  First, it is determined in step S1 whether or not the tag TG information possessed by the user CS has been read by the tag reader 12b. If “YES” here, the read tag information is transmitted to the database server 16 in a step S3. In step S5, it is determined whether or not a dialogue start command is given. If YES, the microphone 12d, speaker 12c and eye camera 12a are turned on in step S7, and dialogue processing is executed in step S9. In step S11, it is determined whether or not a dialogue end command has been given. If NO, the dialogue processing in step S9 is repeated. If YES, microphone 12d, speaker 12c and eye camera 12a are turned off in step S13. Return to S1.

  The CPU 16b of the database server 16 executes the process shown in FIG. 7 according to the control program stored in the memory 16d.

  First, in step S21, it is determined whether or not an inquiry request has been received from the terminal device 18. If YES here, the personal information of the user CS is extracted from the database 16a in step S23, and the additional information is extracted from the database 16a based on the personal information extracted in step S25. In step S27, the dialogue support information including the extracted personal information and additional information is transmitted to the terminal device 18.

  In step S29, it is determined whether or not a dialogue start command has been received from the terminal device 18. If YES, the recording process is started in step S31. When a dialog end command is received from the terminal device 18, YES is determined in step S35, the recording process is ended in step S35, and the process returns to step S21.

  The CPU 18f of the terminal device 18 executes the processes shown in FIGS. 8 and 9 according to the control program stored in the memory 18e.

  First, it is determined in step S41 whether or not tag information has been received from the robot 12. If YES, an inquiry request including the received tag information is transmitted to the database server 16. When the dialog support information is received from the database server 16, it is determined YES in step S45, and the received dialog support information is displayed on the monitor 18c in step S47. When the dialog start operation is performed by the operator, YES is determined in step S49, and a dialog start command is transmitted to the robot 12 and the database server 16 in step S51. Further, the microphone 18a, the speaker 18b, and the monitor 18c are turned on in step S53, and a dialogue process is executed in step S55. In step S57, it is determined whether or not a dialog end operation has been performed by the operator OP. If NO, the process returns to step S55. If YES, a dialog end command is transmitted to the robot 12 and the database server 16. Thereafter, in step S61, the microphone 18a, the speaker 18b, and the monitor 18c are turned off, and the process returns to step S41.

  Next, an actual usage example of the dialogue support system of the present invention will be described.

  When the dialogue support system 10 is arranged in a department store (the robot 12 is arranged in a men's clothing department), the personal information of the user CS includes the address, name, age, gender, married / unmarried, last visit date and time, this department Purchase history at the store, videos of the last five visits, and so on. Additional information includes greetings (“Thank you from the distance” for people who are far away, “Thank you often for those who came within a week”), history, age, and gender. The recommended product (tie or suit) judged, the color of the product judged from the purchase history, etc. can be considered.

  When the user CS is a man in his 30's who has visited from other prefectures for the first time, the operator OP determines from the explanation of the products handled at the men's clothing department following the greeting “Thank you from the distance” and the figure of this man. You can make proposals for products with the same design.

  When the dialogue support system 10 is arranged in an amusement park, the personal information may include an address, name, age, sex, married / unmarried, last visit date and time, and the like. Also, as additional information, recommended attractions (attractions that were newly added since the last time, attractions that were not fully boarded at the last time, vacant attractions that have not been experienced yet, crowded but estimated from age, gender (Recommended attractions) and examples of questions that may be asked (time of the last train on the way home, list of new attractions, scheduled circus performance).

  If the visitor visits the user CS for the third time and there is a new attraction that was not present at the second visit, the operator OP will introduce the new attraction following “Thank you for your frequent visits”. Can do.

When the dialogue support system 10 is placed in a nursery school or an elementary school, personal information includes: name, age, birthday, gender, call name, favorite color, name list of children playing together, lateness and sickness Frequency, frequency of going to the health room, precautions from parents (cold, soba allergy), favorite things (drawing pictures), poor things (eating vegetables) are considered. For additional information, examples of questions and guidelines for developing what you like ("Did you have a good picture?
"," Now I Do? "And whether to draw any picture, sample questions and guidance guidelines in order to overcome the poor (" take you to the vegetable garden "," eat a tomato ") can be considered.

  When the user CS visits the robot 12 for the first time in January and has a toy car in his hand, the operator OP begins with a greeting saying “How are you doing? "Do you like it?"

  In this embodiment, the robot 12, the database server 16, and the terminal device 18 are prepared one by one. However, one database server 16 is prepared, while the robot 12 and the terminal device 18 are N (N: 2 or more). Integer)) may be prepared. In this embodiment, the user CS is identified based on the tag information. However, the user CS may be identified by the face or voice of the user CS.

  Furthermore, in this embodiment, the dialogue by voice is described, but in addition to this, the robot 12 may be caused to perform gestures and hand gestures by the operation of the operator OP. For example, when the user CS is a kindergarten child, a behavior of hugging the user CS by hand can be considered.

It is a block diagram which shows the structure of one Example of this invention. It is a block diagram which shows an example of the internal structure of the robot applied to the FIG. 1 Example. It is a block diagram which shows an example of the internal structure of the database server applied to the FIG. 1 Example. It is a block diagram which shows an example of an internal structure of the terminal device applied to the FIG. 1 Example. It is an illustration figure which shows a part of operation | movement of FIG. 1 Example. It is a flowchart which shows a part of operation | movement of CPU of the robot applied to the FIG. 2 Example. FIG. 5 is a flowchart showing one portion of behavior of a CPU of a database server applied to the embodiment in FIG. 3; FIG. 5 is a flowchart showing a part of the operation of the CPU of the terminal device applied to the embodiment in FIG. 4; It is a flowchart which shows a part of other operation | movement of CPU of the terminal device applied to the FIG. 4 Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Dialog support system 12 ... Robot 16 ... Database server 18 ... Terminal device

Claims (3)

A dialogue support system comprising a robot that faces a user, a database server that manages dialogue support information, and a terminal device operated by an operator, wherein the operator interacts with the user via the robot and the terminal device. And
The robot includes identification means for identifying the user, and first dialogue processing means for executing dialogue processing between the user and the operator,
Presentation means said database server includes an extraction means for extracting a compatible interaction support information to the identification result of the identifying means, and the terminal device, for presenting the interactive assistance information extracted by the extracting unit to the operator, And a dialogue support system comprising second dialogue processing means for executing dialogue processing between the user and the operator.   The said 1st dialog processing means conveys the instruction | indication from the said operator to the said user by at least one of an audio | voice and a gesture, and conveys the information from the said user to the said operator by an at least one of an audio | voice and a gesture. Dialogue support system.   The second dialog processing means transmits the dialog information from the user transmitted through the robot and the dialog support information from the database server to the operator, and based on the dialog information and the dialog support information, The dialogue support system according to claim 1, wherein an operator inputs information to be transmitted to the user via the robot.

Images