JP2022086817A - Question answering system and question answering method - Google Patents

Abstract

To provide a question answering system and a question answering method improving usability.SOLUTION: A question answering system 1 includes: a user terminal device 10 configured to output a response to a question inputted by a user; and a respondent terminal device 20 configured to transmit a response by a respondent to the user terminal device 10. The user terminal device 10 includes an answering processing part 191 configured to execute processing of: outputting, as a response, an automatic response being a response automatically generated according to a learning outcome through a machine learning to a question; and connecting to the respondent terminal device 20 when it is determined that the automatic response has not attained an appropriate response according to an evaluation index regarding a response to a question.SELECTED DRAWING: Figure 1

Description

Application for application of Article 30, Paragraph 2 of the Patent Law Website publication date: November 18, 2nd year of Reiwa Website address: https: // www. jrest. co. jp / press / 2020/20201118_ho01. pdf

The present invention relates to a question answering system and a question answering method.

In recent years, a question answering system that outputs an answer to a questioner's question has been known (see, for example, Patent Document 1). In such a question answering system, for example, the answer to the question is automatically determined and output by artificial intelligence using machine learning.

Japanese Patent No. 6726800

However, in the conventional question answering system, there is a limit to automatically answering a question, and an appropriate answer may not be obtained. Therefore, there is a demand for a question answering system with a wider range of answers and higher usability for the questioner.

The present invention has been made to solve the above problems, and an object of the present invention is to provide a question answering system and a question answering method capable of improving usability.

In order to solve the above problem, one aspect of the present invention is a user terminal device that outputs an answer to a question input by a user, and a respondent terminal that transmits the answer by the respondent to the user terminal device. The user terminal device includes an apparatus, and the user terminal device outputs an automatic answer, which is an answer automatically generated based on a learning result by machine learning, as the answer to the question, and evaluates the answer to the question. A question-and-answer system comprising a response processing unit that executes a process of connecting to the respondent terminal device when it is determined that the automatic answer has not reached an appropriate answer based on an index. be.

Further, one aspect of the present invention includes a user terminal device that outputs an answer to a question input by the user, and a respondent terminal device that transmits the answer by the respondent to the user terminal device. The user terminal device outputs the user information acquisition unit that acquires the user's information and the automatic answer, which is an answer automatically generated based on the learning result by machine learning, as the answer to the question. At the same time, a response that executes a process of connecting to the respondent terminal device when it is determined that the automatic answer cannot reach an appropriate answer based on the user information acquired by the user information acquisition unit. It is a question-and-answer system characterized by having a processing unit.

Further, one aspect of the present invention includes a question response including a user terminal device that outputs an answer to a question input by the user and a respondent terminal device that transmits the answer by the respondent to the user terminal device. In the question answering method of the system, the user terminal device outputs an automatic answer, which is an answer automatically generated based on a learning result by machine learning, as the answer to the question, and also outputs the answer to the question. It is characterized by including a response processing step for executing a process of connecting to the respondent terminal device when it is determined that the automatic answer has not reached an appropriate answer based on the evaluation index related to the answer. This is a question answering method.

Further, one aspect of the present invention is a question response including a user terminal device that outputs an answer to a question input by the user and a respondent terminal device that transmits the answer by the respondent to the user terminal device. In the question answering method of the system, the user information acquisition step in which the user terminal device acquires the user information, and the user terminal device responds to the question by learning results by machine learning. An automatic answer, which is an answer automatically generated based on the above, is output as the answer, and it is determined that the automatic answer cannot reach an appropriate answer based on the user information acquired by the user information acquisition step. The question-and-answer method is characterized by including a response processing step of executing a process of connecting to the respondent terminal device when the question is answered.

According to the present invention, usability can be improved.

It is a block diagram which shows an example of the question answering system by this embodiment. It is a figure which shows an example of the conversation history by the automatic answer in this embodiment. It is a figure explaining the display example of the user terminal apparatus in this embodiment. It is a figure explaining the display example of the respondent terminal apparatus in this embodiment. It is a figure which shows an example of the operation of the question answering process by the automatic answering in this embodiment. It is a figure which shows an example of the operation of the switching process from the automatic answer to the answer by a respondent in this embodiment. It is a figure which shows an example of the operation of the answer processing by a respondent in this embodiment. It is a figure which shows the modification of the operation of the switching process from the automatic answer to the answer by a respondent in this embodiment.

Hereinafter, a question answering system and a question answering method according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a schematic block diagram showing an example of a question answering system 1 according to the present embodiment.
As shown in FIG. 1, the question answering system 1 includes a user terminal device 10, a respondent terminal device 20, a server device 30, and an external cooperation server 40. The user terminal device 10, the respondent terminal device 20, the server device 30, and the external cooperation server 40 can be connected to each other via the network NW1.

In the present embodiment, the question answering system 1 is installed in a facility such as a service infrastructure industry such as a local government, a railway, a distribution, a retail store, and a hotel, and outputs an answer to a question from a user U1. Provide services. The question answering system 1 includes an automatic answer automatically generated using AI (artificial intelligence) based on the learning result by machine learning, and an interpersonal answer in which the respondent A1 such as a staff member receives and answers the question from the user U1. Is used together to output the answer.

The server device 30 is a cloud server device (for example, an AI (artificial intelligence) server device) connected to the network NW1 and can generate an automatic answer to a question using the AI function or use a native language (for example, Japanese). Performs processing such as mutual translation with a foreign language (for example, English). The server device 30 includes a NW (network) communication unit 31, a server storage unit 32, and a server control unit 33.

The NW communication unit 31 connects to the network NW1 by a wired LAN (Local Area Network) or the like, and performs various communications via the network NW1. The NW communication unit 31 connects to, for example, the user terminal device 10 and the respondent terminal device 20 via the network NW 1, and communicates with the user terminal device 10 and the respondent terminal device 20.

The server storage unit 32 stores various information used by the server device 30. The server storage unit 32 includes a learning result storage unit 321 and a history information storage unit 322.

The learning result storage unit 321 stores the learning result by machine learning. The learning result is, for example, a generative model (automatic answer generation model) that generates (estimates) an answer to a question using AI.
The history information storage unit 322 stores input / output history information such as the operation history and conversation history of the user U1 in the user terminal device 10.

The server control unit 33 is, for example, a processor including a CPU (Central Processing Unit) and the like, and controls the server device 30 in an integrated manner. The server control unit 33 includes a sound signal conversion unit 331, a translation processing unit 332, an answer generation unit 333, and a machine learning unit 334.

The sound signal conversion unit 331 executes a voice recognition process for converting sound data (voice signal) into text data and a voice synthesis process for converting text data into sound data. The sound signal conversion unit 331 converts input sound data such as a question of user U1 into text data, and converts answer text data (for example, automatic answer text data) into output sound data. Further, when the sound signal conversion unit 331 translates between a native language (first language) and a foreign language (second language), which will be described later, the speech signal conversion unit 331 performs speech recognition processing and speech synthesis processing of the native language, and a foreign country. Performs both speech recognition processing and speech synthesis processing of words.

The translation processing unit 332 mutually translates the native language (first language) used by the respondent A1 and the foreign language (second language) used by the user U1. For example, when the native language of the respondent A1 is Japanese and the foreign language used by the user U1 is English, the translation processing unit 332 translates Japanese and English into each other.

The answer generation unit 333 automatically generates an automatic answer, which is an answer, based on the learning result by machine learning for the question. The answer generation unit 333 generates an automatic answer based on the learning result (automatic answer generation model) stored in the learning result storage unit 321 for the question received from the user terminal device 10, for example. The answer generation unit 333 automatically generates an answer and outputs the AI determination similarity (certainty) for the answer. The AI determination similarity (confidence) indicates the certainty of the automatic answer in the process of automatic generation based on the learning result, and is the similarity indicating how close the distance to the pre-registered answer is. Further, the AI determination similarity (confidence) is an example of an evaluation index regarding the answer to the question.

The machine learning unit 334 relearns by machine learning and reconstructs an automatic answer generation model. For example, the machine learning unit 334 executes machine learning using the history information stored in the history information storage unit 322 as learning data, and constructs an automatic answer generation model. The machine learning unit 334 stores the constructed answer automatic generation model as a learning result in the learning result storage unit 321.

The external cooperation server 40 is, for example, a Web server that acquires railway transfer guidance information, peripheral information (shop and facility information, etc.), map information, videos, etc., and is related to an answer via network NW1. Send (deliver) various information.

The user terminal device 10 is a terminal device installed in a facility such as a station and used by the user U1 in the guidance service provided by the question answering system 1. The user terminal device 10 outputs an answer to the question input by the user U1. The user terminal device 10 includes a NW communication unit 11, an input unit 12, a display unit 13, a microphone 14, a speaker 15, a camera 16, a motion sensor 17, a terminal storage unit 18, and a terminal control unit. It is equipped with 19.

The NW communication unit 11 connects to the network NW1 by, for example, a wireless LAN or a wired LAN, and performs various communications via the network NW1. The NW communication unit 11 communicates between the respondent terminal device 20, the server device 30, and the external cooperation server 40, for example, and the user terminal device 10.

The input unit 12 is, for example, an input device such as an operation button, a touch panel, and a touch sensor, and receives various input information such as operation information input by the user U1. The input unit 12 outputs the received input information to the terminal control unit 19.

The display unit 13 is, for example, a display device such as a liquid crystal display, and displays various information. The display unit 13 is an example of an output unit, and outputs (displays) various information output from the terminal control unit 19.

The microphone 14 (an example of the user sound collecting unit) collects the sound including the voice of the user U1 and outputs the sound data. For example, the microphone 14 collects the voice of the user U1 as input sound data and outputs the input sound data to the terminal control unit 19. The microphone 14 outputs the question of the user U1 by voice as input sound data.

The speaker 15 (an example of the user sound emitting unit) outputs sounds based on various sound data to the outside of the user terminal device 10. The speaker 15 outputs, for example, an answer to a question by voice. That is, the speaker 15 outputs the output sound data based on the answer output by the terminal control unit 19 as voice.

The camera 16 (an example of an image pickup unit) takes an image of the periphery of the user terminal device 10 and outputs image data including the user U1.
The motion sensor 17 detects the user U1 who uses the user terminal device 10. The motion sensor 17 detects, for example, the approach of the user U1 to the user terminal device 10, and uses the detected signal as a tricker signal for starting the use of the question answering system 1.

In the present embodiment, the input unit 12, the microphone 14, and the camera 16 are examples of an input unit that accepts a question from the user U1, and the display unit 13 and the speaker 15 output an answer to the question. This is an example of the output unit.

The terminal storage unit 18 stores various information used by the user terminal device 10. The terminal storage unit 18 includes an operation history storage unit 181, a conversation history storage unit 182, and an image information storage unit 183.

The operation history storage unit 181 stores the operation history in which the user U1 operates the user terminal device 10. The operation history storage unit 181 stores, for example, the input information received by the input unit 12 and the date and time information as operation history information in association with each other.

The conversation history storage unit 182 stores the history of conversations between the user U1 and the user terminal device 10. The conversation history storage unit 182 stores, for example, the question content input by the user U1 or the answer content output by the user terminal device 10 as conversation history information in association with the date and time information. The answer output by the user terminal device 10 includes an automatic answer generated by using AI and an answer input by the respondent A1.

The image information storage unit 183 stores various image information used when explaining the answer to the question. The image information storage unit 183 is, for example, a QR code (registered trademark) for linking image data such as a map, transfer information, shop information, and the image data with a mobile terminal (not shown) such as a smartphone of the user U1. Remember.

The terminal control unit 19 is, for example, a processor including a CPU and the like, and controls the user terminal device 10 in an integrated manner. The terminal control unit 19 includes a response processing unit 191 and a history processing unit 192.

The response processing unit 191 executes a process of outputting an answer to the question of the user U1. When the response processing unit 191 detects the user U1 by the motion sensor 17 (or the camera 16), the response processing unit 191 starts providing a guidance service to the user U1 and processes according to various operations received by the input unit 12. To execute. Further, the response processing unit 191 acquires the question input by the input unit 12 or the microphone 14, and gives an automatic answer, which is an answer automatically generated based on the learning result by machine learning, to the question of the user U1. As an answer, output to an output unit (for example, display unit 13 or speaker 15).

When a question is input by voice from the microphone 14, the response processing unit 191 transmits the input sound data of the question acquired from the microphone 14 to the server device 30 via the NW communication unit 11 and causes the server device 30 to transmit the input sound data. The sound signal conversion unit 331 is converted into the text data of the question. The response processing unit 191 acquires the text data of the question from the server device 30 via the NW communication unit 11.

Further, when the user U1 inputs a question in a foreign language (for example, English), the response processing unit 191 transmits the text data of the question to the server device 30 via the NW communication unit 11. Then, the translation processing unit 332 of the server device 30 is made to translate the input text data of the question in the foreign language (English) into the text data of the question in the native language (for example, Japanese) of the respondent A1. The response processing unit 191 acquires the translated text data of the question from the server device 30 via the NW communication unit 11.

Further, the response processing unit 191 causes the answer generation unit 333 of the server device 30 to generate an automatic answer from the text data of the question using AI. The response processing unit 191 transmits the text data of the question to the server device 30 via the NW communication unit 11 and causes the answer generation unit 333 of the server device 30 to generate the text data of the automatic answer. The response processing unit 191 acquires the text data of the automatic response from the server device 30 via the NW communication unit 11.

The response processing unit 191 transmits the acquired text data of the automatic response to the server device 30 via the NW communication unit 11, and causes the sound signal conversion unit 331 of the server device 30 to convert the acquired text data into the output sound data of the automatic response. The response processing unit 191 acquires the output sound data of the automatic answer from the server device 30 via the NW communication unit 11, and outputs the output sound data of the automatic answer to the speaker 15 as the voice of the answer.

When the question is input in a foreign language (for example, English) by the user U1, the response processing unit 191 uses the text data of the automatic answer in the native language (for example, Japanese) in a foreign language (for example, Japanese). , English) automatic answer text data is translated, and then the sound signal conversion unit 331 converts it into output sound data.

Further, the response processing unit 191 executes a process of connecting to the respondent terminal device 20 when it is determined that the automatic answer has not reached an appropriate answer based on the evaluation index regarding the answer to the question. The response processing unit 191 can reach an appropriate answer when the user U1 does not obtain a satisfactory answer by the automatic answer, as in (1) to (4) in the following cases. It is determined that this is not the case, and a process of connecting to the respondent terminal device 20 is executed.

(1) When the user U1 does not obtain a satisfactory evaluation as a result of multiple exchanges by automatic answering (2) An automatic answer is obtained for the user U1's question, but the AI judgment When the degree of similarity (confidence) is low (3) When the automatic answer to the question of user U1 is not obtained (when the automatic answer by AI is not supported)
(4) When the user U1 cannot understand the content of the automatic answer (for example, he / she does not understand the word used).

The case shown in (1) above corresponds to, for example, a case where the evaluation of the response by the user U1 is selected multiple times from the input unit 12 such as "not satisfied". That is, the above-mentioned evaluation index includes the evaluation result of the automatic answer input by the user U1, and the response processing unit 191 indicates that the automatic answer has not reached an appropriate answer based on the evaluation result. judge.

Further, the case shown in (2) above corresponds to, for example, a case where an answer having a low AI determination similarity (certainty) to the automatic answer generated by the answer generation unit 333 continues a plurality of times. That is, the above-mentioned evaluation index includes a certainty degree (for example, AI judgment similarity) indicating the certainty of the automatic answer in the automatic generation process based on the learning result, and the response processing unit 191 has the certainty degree equal to or less than the threshold value. When the automatic answer is continued a predetermined number of times, it is determined that the automatic answer has not reached an appropriate answer.

Further, in the case of the above-mentioned (3), for example, it corresponds to the case where the answer to the question is not learned by machine learning.
Further, in the case of the above-mentioned (4), since the user U1 does not understand the content of the automatic answer, for example, a similar question (a question with a different expression) may be asked a plurality of times. That is, the above-mentioned evaluation index includes the number of questions input by the user U1, and the response processing unit 191 reaches an appropriate answer when the number of questions reaches the threshold number. Judge that it is not done.

Further, the response processing unit 191 can reach an appropriate answer when the user U1 is uncertain about how to use the guidance system, as in (5) to (7) in the following cases. It is determined that it is not, and the process of connecting to the respondent terminal device 20 is executed.

(5) When the input by the user U1 (voice input or operation to the input unit 12) has not been performed for a certain period after the user terminal device 10 detects the user U1. (6) The user U1 When the input by the user U1 (voice input or the operation to the input unit 12) has not been performed for a certain period from the start of use (7) When the user U1 repeats the same operation.

The cases shown in (5) and (6) above correspond to, for example, the case where the period of no input reaches the threshold period. That is, the above-mentioned evaluation index includes a non-input period in which the user U1 does not input the user terminal device 10, and the response processing unit 191 automatically performs the non-input period when the threshold period is reached. Determine that the answer has not reached the appropriate answer.

Further, the case shown in (7) above corresponds to the case where the user U1 performs the same operation or a similar operation a plurality of times. That is, the above-mentioned evaluation index includes the number of operations of the same operation or similar operation by the user U1, and the response processing unit 191 gives an appropriate answer when the number of operations reaches the threshold number. Is not reached.

Here, with reference to FIG. 2, a specific example in which the response processing unit 191 determines that the automatic response has not reached an appropriate answer will be described.
FIG. 2 is a diagram showing an example of a conversation history by automatic answering in the present embodiment.

In FIG. 2, the “AI” of the “caller” indicates the output of the automatic response, and the “system” indicates the output by the response processing unit 191. Further, "evaluation: not satisfied" in the "outgoing content" indicates that the content not to be evaluated is selected from the input unit 12.

In the example shown in FIG. 2, as the evaluation result of the automatic answer by the user U1, it is determined that the automatic answer has not reached an appropriate answer because the content not evaluated has reached twice.
Further, as an example of using another evaluation index, the automatic response in which the AI judgment similarity is equal to or less than the threshold value (for example, 80% or less) continues a predetermined number of times (for example, twice), so that the automatic answer is given. Judge that the appropriate answer has not been reached.

Further, as a process of connecting to the respondent terminal device 20, the response processing unit 191 may perform a process of blinking a connection button connected to the respondent terminal device 20 used by a respondent such as a staff member, or a process of blinking a connection button on the display unit 13. Respondent A process for displaying a message (display information) prompting connection to the terminal device 20 is executed. Further, the response processing unit 191 may automatically connect to the respondent terminal device 20 as a process of connecting to the respondent terminal device 20.
When the user terminal device 10 and the respondent terminal device 20 are connected, the response processing unit 191 directly talks between the user U1 and the respondent A1 such as a staff member, and the respondent A1 sends a direct conversation. The answer is output to the output unit (for example, the speaker 15).

Here, with reference to FIG. 3, a display example of the user terminal device 10 in the processing of the series of response processing units 191 described above will be described.
FIG. 3 is a diagram illustrating a display example of the user terminal device 10 in the present embodiment.

FIG. 3A shows a display example when the response processing unit 191 determines that the automatic response has not reached an appropriate response. As shown in FIG. 3A, the response processing unit 191 blinks the "call staff" button BT1 and displays a message prompting the respondent to connect to the terminal device 20. In this way, the response processing unit 191 starts the connection with the respondent terminal device 20 as a process of connecting to the respondent terminal device 20 when it is determined that the automatic answer has not reached an appropriate answer. Display information prompting the user U1 is displayed on the display unit 13.

Further, FIG. 3B is a display example of a confirmation screen displayed when the button BT1 is pressed by the user U1. As shown in FIG. 3B, when the button BT1 is pressed, the response processing unit 191 displays the button BT2 indicating "call" on the display unit 13 to connect to the respondent terminal device 20. Prompt for confirmation.

Further, FIG. 3C shows an example of display after the user terminal device 10 is connected to the respondent terminal device 20. As shown in FIG. 3C, after connecting to the respondent terminal device 20, the response processing unit 191 has, for example, an image SG1 of the respondent A1 such as a staff member, and an image SG2 of the user U1 captured by the camera 16. An image SG3 for explaining an answer such as a map, an image SG4 of a two-dimensional code, and the like are displayed. Here, the image SG1, the image SG3, and the image SG4 are images received from the respondent terminal device 20 via the network NW1, and the image SG2 is image data captured by the camera 16. It is assumed that the image data of the images SG1 to SG4 are stored in the image information storage unit 183, for example.

Further, the two-dimensional code of the image SG4 is, for example, a QR code (registered trademark), and information that enables image data such as the image SG3 to be acquired and displayed by a mobile terminal (not shown) owned by the user U1. Includes. The two-dimensional code of the image SG4 includes, for example, a URL (Uniform Resource Locator) for displaying image data such as the image SG3.

Returning to the description of FIG. 1, the history processing unit 192 stores the operation history information by the input unit 12 in the operation history storage unit 181 and stores the conversation history information by the response processing unit 191 in the conversation history storage unit 182. .. Further, the history processing unit 192 has the operation history information stored in the operation history storage unit 181 and the conversation history information stored in the conversation history storage unit 182 when connected to the respondent terminal device 20 by the response processing unit 191. Is transmitted to the respondent terminal device 20 as the takeover information.

Further, the history processing unit 192 transmits the operation history information and the conversation history information to the server device 30 periodically or when the user U1 ends the use, and stores the operation history information and the conversation history information in the history information storage unit.

The respondent terminal device 20 is a terminal device for the respondent A1 such as a staff member to input an answer to the question of the user U1, and transmits the answer by the respondent A1 to the user terminal device 10. The respondent terminal device 20 is installed in a station staff room or the like when used in, for example, a railway company, and in this case, the station staff is the respondent A1.
The respondent terminal device 20 includes a NW communication unit 21, an input unit 22, a display unit 23, a microphone 24, a speaker 25, a camera 26, a warning light 27, a terminal storage unit 28, and a terminal control unit 29. And prepare.

The NW communication unit 21 connects to the network NW1 by, for example, a wireless LAN or a wired LAN, and performs various communications via the network NW1. The NW communication unit 21 communicates, for example, between the user terminal device 10, the server device 30, and the external cooperation server 40, and the respondent terminal device 20.

The input unit 22 is, for example, an input device such as an operation button, a touch panel, a touch sensor, a keyboard, and a mouse, and receives various input information input by the respondent A1. The input unit 22 outputs the received input information to the terminal control unit 29.
The display unit 23 is, for example, a display device such as a liquid crystal display, and displays various information.

The microphone 24 (an example of the respondent sound collecting unit) collects the sound including the voice of the respondent A1 and outputs the sound data. For example, the microphone 24 collects the voice of the respondent A1 as sound data and outputs the sound data to the terminal control unit 29. The microphone 24 outputs the answer of the respondent A1 by voice as output sound data.

The speaker 25 (an example of the respondent sound emitting unit) outputs sounds based on various sound data to the outside of the respondent terminal device 20. The speaker 25 outputs, for example, a question from the user U1 by voice. That is, the speaker 25 outputs the input sound data of the question output by the terminal control unit 29 as voice.

The camera 26 (an example of the image pickup unit) captures the periphery of the respondent terminal device 20 and outputs image data including the respondent A1.
The warning light 27 is, for example, a patrol light (registered trademark), and causes the respondent A1 to recognize that there is a connection request from the user terminal device 10 by lighting.

The terminal storage unit 28 stores various information used by the respondent terminal device 20. The terminal storage unit 28 includes a takeover information storage unit 281, a response candidate storage unit 282, and an image information storage unit 283.

The hand-over information storage unit 281 connects the user terminal device 10 and the respondent terminal device 20, and stores the hand-over information when answering by a direct conversation between the user U1 and the respondent A1. The takeover information storage unit 281 stores operation history information and conversation history information for the user terminal device 10 of the user U1 until the user terminal device 10 and the respondent terminal device 20 are connected.

The answer candidate storage unit 282 stores the answer generated by the answer generation unit 333 of the server device 30, the AI determination similarity, and the like.

The image information storage unit 283 stores various image information used when explaining the answer to the question. The image information storage unit 283 is, for example, a QR code (registered trademark) for linking image data such as a map, transfer information, shop information, and the image data with a mobile terminal (not shown) such as a smartphone of the user U1. Remember. It should be noted that these image data are acquired from, for example, the external cooperation server 40.

The terminal control unit 29 is, for example, a processor including a CPU and the like, and controls the respondent terminal device 20 in an integrated manner. The terminal control unit 29 includes a response processing unit 291.

When the response processing unit 291 is connected to the user terminal device 10, the response processing unit 291 displays the transfer information based on the input / output history between the user terminal device 10 and the user U1 on the display unit 23, and the respondent. The answer from A1 is acquired, and the acquired answer is transmitted to the user terminal device 10. The input / output history includes the operation history of the user terminal device 10 by the user U1 and the history information of the above-mentioned questions and automatic answers (for example, the conversation history madoka in the user terminal device 10). The response processing unit 291 acquires history information (for example, operation history information and conversation history information) from the user terminal device 10 when connected to the user terminal device 10.

Further, in addition to the takeover information, the response processing unit 291 displays, for example, image data of the user U1, image data of the respondent A1, image data for explaining the answer acquired from the external cooperation server 40, and the like. To display.
Here, with reference to FIG. 4, a display example when the response processing unit 291 is connected to the user terminal device 10 will be described.

FIG. 4 is a diagram illustrating a display example of the respondent terminal device 20 in the present embodiment.
FIG. 4A shows a display example when a connection request is received from the user terminal device 10. When the response processing unit 291 receives a connection request from the user terminal device 10 via the NW communication unit 21, the warning light 27 is turned on and a display screen as shown in FIG. 4A is displayed. Display on 23.

Further, FIG. 4B is an operation for the respondent A1 such as a staff member to display when the “OK” button BT3 is pressed by the respondent A1 on the display screen of FIG. 4A described above. A display example of the screen is shown. As shown in FIG. 4B, the response processing unit 291 includes an image SG4 of connection operation information, an image SG5 of user U1, an image SG6 of respondent A1 captured by the camera 26, an image SG7 of takeover information, and a map. The image SG8 for explaining the answer such as is displayed on the display unit 23.

Further, the answer processing unit 291 receives, for example, the input sound data of the question by the user U1 via the NW communication unit 21, and the input sound data of the received question is transmitted to the sound signal conversion unit 331 of the server device 30. , The answer generation unit 333 is converted into the text data of the question, and the answer candidate for the question (text data of the answer candidate) and the AI judgment similarity of each candidate are generated. The response processing unit 291 causes the display unit 23 to display the response candidate and the AI determination similarity generated by the response generation unit 333 as reference information for the response.

Further, the answer processing unit 291 outputs the input sound data of the question by the user U1 received from the user terminal device 10 from the speaker 25 via the NW communication unit 21, and the respondent who picks up the sound by the microphone 24. The answer sound data, which is the voice data of A1, is transmitted to the user terminal device 10 via the NW communication unit 21, and is output as the answer output sound data from the speaker 15 of the user terminal device 10.

When the user U1 is a foreigner, the translation process is performed by using the sound signal conversion unit 331 and the translation processing unit 332 of the server device 30 as in the case of the automatic response of the response processing unit 191 described above. Run.

Next, the operation of the question answering system 1 according to the present embodiment will be described with reference to the drawings.
FIG. 5 is a diagram showing an example of the operation of question answering processing by automatic answering in the embodiment.

As shown in FIG. 5, the user terminal device 10 of the question answering system 1 first determines whether or not a user has arrived (step S101). The response processing unit 191 of the user terminal device 10 determines whether or not the user U1 has come to the vicinity of the user terminal device 10 by detecting, for example, the motion sensor 17 or the camera 16. The response processing unit 191 advances the processing to step S102 when the user arrives (user U1 is detected) (step S101: YES). Further, the response processing unit 191 returns the processing to step S101 when the user has not arrived (user U1 has not been detected) (step S101: NO).

In step S102, the response processing unit 191 accepts operations and questions. The response processing unit 191 starts, for example, accepting an operation by the input unit 12 and accepting a question from the user U1 by the microphone 14.

Next, the response processing unit 191 transmits the received input sound data (question) to the server device 30 (step S103).
Next, the server device 30 converts the input sound data (question) into the input text data (step S104). The sound signal conversion unit 331 of the server device 30 converts the input sound data of the question into the text data of the question.

Next, the server device 30 transmits the input text data (question) converted by the sound signal conversion unit 331 to the user terminal device 10 (step S105).
Next, the response processing unit 191 of the user terminal device 10 transmits the text-converted input text data to the server device 30 as question data (step S106).

Next, the server device 30 generates an automatic answer (step S107). The answer generation unit 333 of the server device 30 generates automatic answer data for the question data received from the user terminal device 10. The response generation unit 333 generates the AI determination similarity of the automatic response when generating the automatic response data.

Next, the server device 30 transmits the automatic response data generated by the response generation unit 333 to the user terminal device 10 (step S108). The server device 30 transmits the AI determination similarity to the user terminal device 10 together with the automatic response data.

Next, the response processing unit 191 of the user terminal device 10 transmits the automatic response data as output text data (answer) to the server device 30 (step S109).
Next, the sound signal conversion unit 331 of the server device 30 voice-synthesizes the output text data (answer) and converts it into output sound data (step S110).

Next, the server device 30 transmits the output sound data (answer) converted by the sound signal conversion unit 331 to the user terminal device 10 (step S111).
Next, the response processing unit 191 of the user terminal device 10 outputs the received output sound data (step S112). The response processing unit 191 causes the speaker 15 to output the output sound data as the voice of the answer.

Next, the history processing unit 192 of the user terminal device 10 transmits the history information to the server device 30 (step S113). The history processing unit 192 transmits and stores a series of operation history information and conversation history information to the server device 30.
Next, the server device 30 stores the received history information in the history information storage unit 322 (step S114).

The question answering system 1 repeats the processes from step S102 to step S114 in the case of question answering processing by automatic answering.
If translation processing is required, translation processing by the translation processing unit 332 is added in the processes of steps S107 and S110.

Next, with reference to FIG. 6, the switching process from the automatic answering of the question answering system 1 according to the present embodiment to the answering by the respondent A1 will be described.
FIG. 6 is a diagram showing an example of the operation of the switching process from the automatic answer to the answer by the respondent A1 in the present embodiment.

As shown in FIG. 6, the response processing unit 191 of the user terminal device 10 first determines whether or not the automatic response has reached an appropriate response (step S201). The response processing unit 191 determines whether or not the automatic response has reached an appropriate answer based on an evaluation index such as the AI determination similarity. Here, as an example, an example of the case shown in (2) above will be described, and the response processing unit 191 will perform a predetermined number of automatic responses (for example, 80% or less) in which the AI determination similarity is equal to or less than the threshold value (for example, 80% or less). If it continues (twice), it is determined that the automatic answer has not reached the appropriate answer. When the response processing unit 191 determines that the automatic response has reached an appropriate answer (step S201: YES), the response processing unit 191 returns the processing to step S201. Further, when the response processing unit 191 determines that the automatic response has not reached an appropriate response (step S201: NO), the response processing unit 191 advances the processing to step S202.

In step S202, the response processing unit 191 displays a display recommending that the respondent A1 be connected to the display unit 13. For example, the response processing unit 191 "provides a staff member as shown in FIG. 3A. The button BT1 of "call" is blinked, and a message prompting the connection to the respondent terminal device 20 is displayed on the display unit 13.

Next, when the "call staff" button BT1 is pressed by the user U1, the response processing unit 191 displays a connection confirmation screen on the display unit 13 (step S203). The response processing unit 191 causes the display unit 13 to display a display screen as shown in FIG. 3B, for example. As a result, since the user terminal device 10 confirms the connection with the user U1, it is possible to reduce the possibility of erroneously connecting to the respondent terminal device 20 due to an erroneous operation.

Next, when the "call" button BT2 is pressed by the user U1, the response processing unit 191 transmits a connection request to the respondent terminal device 20 (step S204). The response processing unit 191 may transmit the operation history information and the conversation history information to the respondent terminal device 20 together with the connection request.

Next, when the connection request is received, the response processing unit 291 of the respondent terminal device 20 turns on the warning light 27 and displays on the display unit 23 that there is a connection request from the user terminal device 10 (step S205). .. The answer processing unit 291 causes the display unit 23 to display a display screen as shown in FIG. 4A, for example.

Next, when the "OK" button BT3 is pressed by the respondent A1, the response processing unit 291 displays the operation screen for the staff (for the respondent A1) on the display unit 23 (step S206). The answer processing unit 291 causes the display unit 23 to display an operation screen as shown in FIG. 4B, for example. The response processing unit 291 causes the display unit 23 to display the operation history and the conversation history as the inherited information, for example, as shown in FIG.

Next, the response processing unit 291 transmits the connection permission to the user terminal device 10 (step S207).
Next, the response processing unit 191 displays the clerk connection screen on the display unit 13 according to the connection permission (step S208). The response processing unit 191 causes the display unit 13 to display, for example, a clerk connection screen as shown in FIG. 3C.

Next, the response processing unit 191 and the response processing unit 291 execute a conversational question / answer process between the user terminal device 10 and the respondent terminal device 20 (step S209). That is, the response processing unit 191 transmits the question by the user U1 to the respondent terminal device 20, the response processing unit 291 transmits the answer by the respondent A1 to the user terminal device 10, and the response processing unit 191 receives. The answer by the respondent A1 is output to the user U1.

In the example shown in FIG. 6 described above, an example in which the user terminal device 10 executes the processing of step S202 and step S203 has been described, but the processing of step S202 and step S203 is not executed and the user terminal device 10 automatically executes the processing. , May be connected to the respondent terminal device 20.

Next, an example of the process of step S209 of FIG. 6 described above will be described with reference to FIG. 7.
FIG. 7 is a diagram showing an example of the operation of the response processing by the respondent A1 in the present embodiment. Here, an example will be described in which the user U1 is a foreigner and the mother tongue (home language) of the respondent A1 is Japanese.

As shown in FIG. 7, first, the user terminal device 10 receives the input sound data of the user U1 (step S301). The response processing unit 191 of the user terminal device 10 acquires the question of the user U1 as input sound data in a foreign language by the microphone 14.

Next, the response processing unit 191 transmits the acquired question input sound data (foreign language) to the respondent terminal device 20 via the NW communication unit 11 (step S302).
Next, the answer processing unit 291 of the respondent terminal device 20 transmits the input sound data (foreign language) of the question received from the user terminal device 10 to the server device 30 via the NW communication unit 21 (step). S303).

Next, the server device 30 executes text conversion and translation processing on the input sound data (foreign language) of the received question (step S304). That is, the sound signal conversion unit 331 of the server device 30 converts the input sound data (foreign language) of the question into the input text data (foreign language), and the translation processing unit 332 of the server device 30 converts the input text data (foreign language). ) Is converted to input text data (Japanese).

Next, the server device 30 transmits the input text data (Japanese) to the respondent terminal device 20 via the NW communication unit 31 (step S305).
Next, the response processing unit 291 transmits the input text data (Japanese) to the server device 30 via the NW communication unit 21 (step S306).

Next, the server device 30 voice-synthesizes the received input text data (Japanese) (step S307). The sound signal conversion unit 331 of the server device 30 converts the input text data (Japanese) into the input sound data (Japanese).

Next, the server device 30 transmits the converted input sound data (Japanese) to the respondent terminal device 20 via the NW communication unit 31 (step S308).
Next, the respondent terminal device 20 outputs the received input sound data (Japanese) (step S309). That is, the answer processing unit 291 of the respondent terminal device 20 causes the speaker 25 to output the received input sound data (Japanese) as the voice of the question.

On the other hand, the answer generation unit 333 of the server device 30 generates an automatic answer from the input text data (Japanese) (step S310). The answer generation unit 333 generates a plurality of answer candidates and also generates an AI determination similarity degree for each answer candidate.
Next, the server device 30 transmits the answer candidate (text) to the respondent terminal device 20 via the NW communication unit 31 (step S311). The server device 30 transmits a plurality of answer candidates and AI determination similarity generated by the answer generation unit 333 to the respondent terminal device 20.

The respondent terminal device 20 displays the received answer candidates on the display unit 23 (step S312). The response processing unit 291 of the respondent terminal device 20 causes the display unit 23 to display a plurality of answer candidates and the AI determination similarity degree with priority given to those having a high AI determination similarity degree.

Next, the respondent terminal device 20 receives the response sound data of the respondent A1 (step S313). That is, the answer processing unit 291 acquires the answer sound data (Japanese) of the respondent A1 by the microphone 24.
Next, the response processing unit 291 transmits the response sound data (Japanese) to the server device 30 via the NW communication unit 21 (step S314).

Next, the server device 30 executes text conversion and translation processing on the received response sound data (Japanese) (step S315). That is, the sound signal conversion unit 331 of the server device 30 converts the answer sound data (Japanese) into the answer text data (Japanese), and the translation processing unit 332 of the server device 30 converts the answer text data (Japanese). Convert to answer text data (foreign language).

Next, the server device 30 transmits the answer text data (foreign language) to the respondent terminal device 20 via the NW communication unit 31 (step S316).
Next, the response processing unit 291 transmits the response text data (foreign language) to the server device 30 via the NW communication unit 21 (step S317).

Next, the server device 30 voice-synthesizes the received answer text data (foreign language) (step S318). The sound signal conversion unit 331 of the server device 30 converts the answer text data (foreign language) into the answer sound data (foreign language).

Next, the server device 30 transmits the converted response sound data (foreign language) to the respondent terminal device 20 via the NW communication unit 31 (step S319).
Next, the response processing unit 291 of the respondent terminal device 20 transmits the received response sound data (foreign language) to the user terminal device 10 via the NW communication unit 21 (step S320).

Next, the user terminal device 10 outputs the received response sound data (foreign language) (step S321). That is, the response processing unit 191 of the user terminal device 10 outputs the received response sound data (foreign language) to the speaker 15 as the response voice.

Further, the server device 30 stores the history information (step S322). The server control unit 33 of the server device 30 stores, for example, the input text data (Japanese) and the answer text data (Japanese) of the question in association with the date and time information in the history information storage unit 322.

In the example shown in FIG. 7, when the user U1 is Japanese, the answer processing unit 291 uses the input sound data of the question received from the user terminal device 10 as the voice of the question in the speaker 25. The response sound data to be output and acquired by the microphone 24 is transmitted to the user terminal device 10. Further, in this case, the answer processing unit 291 transmits the input sound data of the question to the server device 30, converts the input sound data of the question into text, and acquires the answer candidate from the server device 30.

As described above, the question answering system 1 according to the present embodiment transmits the answer to the question input by the user U1 to the user terminal device 10 and the answer from the respondent A1 to the user terminal device 10. The respondent terminal device 20 is provided. The user terminal device 10 includes a response processing unit 191. The response processing unit 191 outputs an automatic answer, which is an answer automatically generated based on the learning result by machine learning, as an answer to the question, and the automatic answer is appropriate based on the evaluation index related to the answer to the question. When it is determined that the answer has not been reached, the process of connecting to the respondent terminal device 20 is executed.

As a result, the question answering system 1 according to the present embodiment can appropriately and quickly determine that the automatic answer has not reached an appropriate answer and can connect to the respondent terminal device 20, so that the usability of the user U1 can be achieved. Can be improved.

Further, in the present embodiment, the evaluation index includes a certainty degree (for example, AI determination similarity degree) indicating the certainty of the automatic answer in the process of automatic generation based on the learning result. The response processing unit 191 determines that the automatic answer has not reached an appropriate answer when the automatic answer having a certainty degree of less than or equal to the threshold value continues a predetermined number of times.

As a result, the question answering system 1 according to the present embodiment uses the certainty (for example, AI determination similarity) to more appropriately and quickly determine that the automatic answer has not reached an appropriate answer. Can be done.

Further, in the present embodiment, the evaluation index includes the number of questions input by the user U1. The response processing unit 191 determines that the automatic answer has not reached an appropriate answer when the number of questions reaches the threshold number.

As a result, in the question answering system 1 according to the present embodiment, when the number of questions reaches the threshold number, it is considered that an appropriate automatic answer has not been obtained, and the automatic answer has reached an appropriate answer. It can be more appropriately determined that there is no such thing.

Further, in the present embodiment, the evaluation index includes a non-input period in which the user U1 does not input the user terminal device 10. The response processing unit 191 determines that the automatic response has not reached an appropriate answer when the non-input period reaches the threshold period.

When the non-input period reaches the threshold period, it is considered that the user U1 is uncertain about how to use the guidance system in response to the cases (5) and (6) described above. Therefore, the question answering system 1 according to the present embodiment can be appropriately connected to the respondent terminal device 20 in such a case, and usability can be further improved.

Further, in the present embodiment, the evaluation index includes the evaluation result of the automatic response input by the user U1. The response processing unit 191 determines that the automatic response has not reached an appropriate answer based on the evaluation result.

As a result, the question answering system 1 according to the present embodiment can more appropriately and quickly determine that the automatic answer has not reached an appropriate answer by using the evaluation result by the user U1.

Further, in the present embodiment, the user terminal device 10 includes a display unit 13 that displays information including an automatic answer. The response processing unit 191 tells the user U1 to start the connection with the respondent terminal device 20 as a process of connecting to the respondent terminal device 20 when it is determined that the automatic answer has not reached an appropriate answer. The display information to be prompted is displayed on the display unit 13.

As a result, the question answering system 1 according to the present embodiment can appropriately connect to the respondent terminal device 20 by prompting the user U1 to start the connection with the respondent terminal device 20.

Further, the question answering system 1 according to the present embodiment includes an answer generation unit 333 that generates an automatic answer from the question based on the learning result by machine learning. The response processing unit 191 acquires the automatic answer generated by the answer generation unit 333 for the question. The answer generation unit 333 is provided in, for example, the server device 30, and is provided outside the user terminal device 10 and the respondent terminal device 20.

As a result, since the question answering system 1 according to the present embodiment automatically generates an answer by the server device 30, the processing load of the user terminal device 10 and the respondent terminal device 20 can be reduced.

Further, in the present embodiment, the user terminal device 10 has a microphone 14 (sound collecting unit) that collects the voice of the user U1 as input sound data, and a speaker 15 (sound collecting unit) that outputs the output sound data based on the answer as voice. It is equipped with a sound emitting part). Further, the question and answer system 1 includes a sound signal conversion unit 331 that converts the input sound data collected by the microphone 14 into the input text data and the output text data including the automatic answer into the output sound data. Then, the response processing unit 191 acquires the input text data of the question from the sound signal conversion unit 331, and causes the sound signal conversion unit 331 to convert the automatic answer into the output sound data.

As a result, the question answering system 1 according to the present embodiment asks a question by voice input and obtains an answer by voice output, so that the usability of the user U1 can be further improved.

In addition, the question-and-answer system 1 according to the present embodiment has at least a native language (first language, for example, Japanese) used by the respondent A1 and a foreign language (second language, for example, English) used by the user U1. It is provided with a translation processing unit 332 that mutually translates and. The response processing unit 191 causes the translation processing unit 332 to translate the input foreign language (English) question into the native language (Japanese) question, and to translate the answer in the native language (Japanese) into the foreign language (Japanese). Translate to the answer in English) and output the answer in the foreign language (English).

Thereby, the question answering system 1 according to the present embodiment can cope with the case where the user U1 is a foreigner, and the usability of the user U1 can be further improved.

Further, the question answering system 1 according to the present embodiment includes a machine learning unit 334 that relearns by machine learning and reconstructs an automatic answer generation model.
As a result, the question answering system 1 according to the present embodiment can improve the accuracy (certainty) of the automatic answer by re-learning, and can further improve the usability.

In the present embodiment, the respondent terminal device 20 acquires a response from the respondent A1 when the display unit 23 (respondent display unit) and the respondent terminal device 20 are connected to the user terminal device 10. It is provided with a response processing unit 291 that transmits the acquired response to the user terminal device 10. Further, the user terminal device 10 includes a display unit 13 (user display unit) which is a display unit that can be visually recognized by the user U1. The response processing unit 291 displays the image data used for explaining the answer on the display unit 23 (respondent display unit), which is the image data to which the respondent A1 can add information, and displays the image data on the user. It is transmitted to the terminal device 10. The response processing unit 191 causes the display unit 13 to display the image data.

As a result, the question answering system 1 according to the present embodiment explains the answer to the question while displaying an image such as a map or a facility map on both the user terminal device 10 and the respondent terminal device 20. Therefore, usability can be further improved.

In the present embodiment, the response processing unit 191 causes the display unit 13 to display a two-dimensional code that enables the image data to be acquired and displayed on a mobile terminal (for example, a smartphone or tablet terminal) owned by the user U1. ..
As a result, the question-and-answer system 1 according to the present embodiment can capture and display the image data used for explaining the answer on the mobile terminal (for example, a smartphone or tablet terminal) owned by the user U1. Even after the user U1 leaves the installation location of the user terminal device 10, the image used for explaining the answer can be confirmed.

Further, in the question answering method according to the present embodiment, the user terminal device 10 that outputs the answer to the question input by the user U1 and the respondent terminal device 20 that transmits the answer by the respondent A1 to the user terminal device 10. It is a question answering method of the question answering system 1 including the above, and includes a response processing step. In the response processing step, the user terminal device 10 outputs an automatic answer, which is an answer automatically generated based on the learning result by machine learning, as an answer to the question, and based on the evaluation index related to the answer to the question. , When it is determined that the automatic answer has not reached an appropriate answer, a process of connecting to the respondent terminal device 20 is executed.
Thereby, the question answering method according to the present embodiment has the same effect as the question answering system 1 described above, and the usability of the user U1 can be improved.

The present invention is not limited to the above embodiment, and can be modified without departing from the spirit of the present invention.
For example, in the above embodiment, the question answering system 1 has described an example including one user terminal device 10 and one respondent terminal device 20, but the present invention is not limited to this. For example, a plurality of user terminal devices 10 or a plurality of respondent terminal devices 20 may be provided.

Further, in the above embodiment, the response processing unit 191 of the user terminal device 10 has described an example of connecting to the respondent terminal device 20 after exchanging a question and an automatic answer with the user U1. It is not limited to this. For example, in the following cases (A) to (C), the response processing unit 191 may be connected to the respondent terminal device 20 before exchanging the question and the automatic answer.

(A) When the user U1 has a high priority for guidance by manned people such as the elderly and the physically handicapped (B) When the user U1 presses the "call staff" button BT1 (C) Exchange of money and When there is a question (inquiry) from user U1 regarding risk / importance / urgent content such as personal information, content related to crime, etc.

Here, with reference to FIG. 8, a modified example of the operation of the switching process from the automatic answering to the answering by the respondent in the question answering system 1 according to the present embodiment will be described.
FIG. 8 is a diagram showing a modified example of the operation of the switching process from the automatic answer to the answer by the respondent in the present embodiment.

As shown in FIG. 8, the response processing unit 191 of the user terminal device 10 first determines whether or not it is determined that the automatic answer cannot reach an appropriate answer (step S401). The response processing unit 191 determines that the automatic response cannot reach an appropriate answer based on the information of the user U1 (for example, the image of the user U1) acquired by the camera 16 (an example of the user information acquisition unit). It is determined whether or not it has been determined. For example, when the response processing unit 191 determines from the image of the user U1 that the user U1 is an elderly person or a physically handicapped person (when he / she is in a wheelchair, when he / she has a white cane, etc.). (In the case of (A) described above), it is determined that the automatic answer cannot reach an appropriate answer.

When it is determined that the automatic answer cannot reach an appropriate answer (step S401: YES), the response processing unit 191 advances the processing to step S402. Further, the response processing unit 191 returns the processing to step S401 when it is not determined that the automatic response cannot reach an appropriate answer (step S401: NO).
Since the processing from step S402 to step S409 is the same as the processing from step S202 to step S209 shown in FIG. 6 described above, the description thereof will be omitted here.

As described above, the question-and-answer system 1 according to the present embodiment is the user terminal device 10 that outputs the answer to the question input by the user U1 and the respondent who transmits the answer by the respondent A1 to the user terminal device 10. A terminal device 20 is provided, and the user terminal device 10 includes a user information acquisition unit (for example, a camera 16 or the like) for acquiring information of the user U1 and a response processing unit 191. The response processing unit 191 outputs an automatic answer, which is an answer automatically generated based on the learning result by machine learning, as an answer to the question, and is acquired by the user information acquisition unit (for example, the camera 16). When it is determined that the automatic answer cannot reach an appropriate answer based on the information of the user U1, the process of connecting to the respondent terminal device 20 may be executed.

As a result, the question answering system 1 according to the present embodiment can quickly connect to the respondent terminal device 20 when the user U1 is an elderly person or a physically handicapped person, and the usability of the user U1 can be improved. Can be improved.

Further, the question answering method according to the present embodiment is the question answering method of the question answering system 1 described above, and includes a user information acquisition step and a response processing step. In the user information acquisition step, the user terminal device 10 acquires the information of the user U1. In the response processing step, the user terminal device 10 outputs an automatic answer, which is an answer automatically generated based on the learning result by machine learning, as an answer to the question, and is acquired by the user information acquisition step. When it is determined that the automatic answer cannot reach an appropriate answer based on the information of the user U1, a process of connecting to the respondent terminal device 20 is executed.
Thereby, the question answering method according to the present embodiment has the same effect as the question answering system 1 described above, and the usability of the user U1 can be improved.

In the above-mentioned modification, an example of using the camera 16 as an example of the user information acquisition unit has been described, but the present invention is not limited to this, and for example, when the user U1 is confused by the operation, the operation is not limited to this. The operation interval by the input unit 12 may be used.

Further, in the above embodiment, the user terminal device 10 or the respondent terminal device 20 may include a part of the configuration included in the server device 30. For example, the user terminal device 10 or the respondent terminal device 20 may include a sound signal conversion unit 331 and a translation processing unit 332, or may include a response generation unit 333.

Further, in the above embodiment, an example of displaying the image of the respondent A1 on the user terminal device 10 has been described, but the present invention is not limited to this, and a character or an avatar is used instead of the image of the respondent A1. It may be displayed. In this case, the user terminal device 10 may convert the video and audio of the guidance for the user according to the avatar.

Further, in the above embodiment, the example in which the user terminal device 10 is installed in a facility or the like has been described, but the present invention is not limited to this, and for example, a see-through type head-mounted display terminal device may be used. ..

Further, in the above embodiment, the example in which the respondent terminal device 20 includes the speaker 25 as a sound emitting unit has been described, but the present invention is not limited to this, and the sound emitting unit may be a headphone or the like.
Further, in the above embodiment, the user terminal device 10 and the respondent terminal device 20 have described an example in which questions and answers are input / output by voice, but the present invention is not limited to this, and the input unit 12 and the respondent terminal device 20 and the respondent terminal device 20 are described. Text may be input using the input unit 22 and displayed as text on the display unit 13 and the display unit 23.

Each configuration of the question answering system 1 described above has a computer system inside. Then, a program for realizing the functions of each configuration included in the above-mentioned question answering system 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed. Therefore, the processing in each configuration provided in the question answering system 1 described above may be performed. Here, "loading and executing a program recorded on a recording medium into a computer system" includes installing the program in the computer system. The term "computer system" as used herein includes hardware such as an OS and peripheral devices.
Further, the "computer system" may include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, and a dedicated line. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. As described above, the recording medium in which the program is stored may be a non-transient recording medium such as a CD-ROM.

The recording medium also includes an internal or external recording medium accessible from the distribution server for distributing the program. It should be noted that the program may be divided into a plurality of parts, downloaded at different timings, and then combined with each configuration provided in the question answering system 1, or the distribution server for distributing each of the divided programs may be different. Furthermore, a "computer-readable recording medium" is a volatile memory (RAM) inside a computer system that serves as a server or client when a program is transmitted via a network, and holds the program for a certain period of time. It shall include things. Further, the above program may be for realizing a part of the above-mentioned functions. Further, a so-called difference file (difference program) may be used, which can realize the above-mentioned function in combination with a program already recorded in the computer system.

Further, a part or all of the above-mentioned functions may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each of the above-mentioned functions may be made into a processor individually, or a part or all of them may be integrated into a processor. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

1 Question and answer system 10 User terminal device 11, 21, 31 NW communication unit 12, 22 Input unit 13, 23 Display unit 14, 24 Microphone 15, 25 Speaker 16, 26 Camera 17 Human sensor 18, 28 Terminal storage unit 19 , 29 Terminal control unit 20 Respondent terminal device 27 Warning light 30 Server device 32 Server storage unit 33 Server control unit 40 External linkage server 181 Operation history storage unit 182 Conversation history storage unit 183, 283 Image information storage unit 191 Response processing unit 192 History processing unit 281 Succession information storage unit 282 Answer candidate storage unit 291 Answer processing unit 321 Learning result storage unit 322 History information storage unit 331 Sound signal conversion unit 332 Translation processing unit 333 Answer generation unit 334 Machine learning unit A1 Respondent NW1 Network U1 user

In order to solve the above problem, one aspect of the present invention is a user terminal device that outputs an answer to a question input by a user, and a respondent terminal that transmits the answer by the respondent to the user terminal device. The user terminal device includes an apparatus, and the user terminal device outputs an automatic answer, which is an answer automatically generated based on a learning result by machine learning, as the answer to the question, and evaluates the answer to the question. A response processing unit that executes a process of connecting to the respondent terminal device when it is determined that the automatic answer has not reached an appropriate answer based on the index is provided , and the evaluation index includes the use. The response processing unit has been able to reach an appropriate answer when the non-input period reaches the threshold period. It is a question-and-answer system characterized by determining that there is no such thing.

Further, one aspect of the present invention includes a user terminal device that outputs an answer to a question input by the user, and a respondent terminal device that transmits the answer by the respondent to the user terminal device. The user terminal device outputs the user information acquisition unit that acquires the user's information and the automatic answer, which is an answer automatically generated based on the learning result by machine learning, as the answer to the question. At the same time, a response that executes a process of connecting to the respondent terminal device when it is determined that the automatic answer cannot reach an appropriate answer based on the user information acquired by the user information acquisition unit. The response processing unit is provided with a processing unit, and when the user is determined to be an elderly person or a physically handicapped person with a high priority of manned guidance based on the user's information, the above-mentioned response processing unit is provided. It is a question-and-answer system characterized in that it is determined that an automatic answer cannot reach an appropriate answer .

Further, one aspect of the present invention includes a user terminal device that outputs an answer to a question input by the user, and a respondent terminal device that transmits the answer by the respondent to the user terminal device. In the question answering method of the system, the user terminal device outputs an automatic answer, which is an answer automatically generated based on a learning result by machine learning, as the answer to the question, and also responds to the question. The evaluation index includes a response processing step for executing a process of connecting to the respondent terminal device when it is determined that the automatic response has not reached an appropriate answer based on the evaluation index related to the response. Includes a non-input period in which the user does not input the user terminal device, and in the response processing step, when the user terminal device reaches the threshold period for the non-input period. , The question-and-answer method is characterized in that it is determined that the automatic answer has not reached an appropriate answer .

Further, one aspect of the present invention includes a question answering device including a user terminal device that outputs an answer to a question input by the user and a respondent terminal device that transmits the answer by the respondent to the user terminal device. In the question answering method of the system, the user information acquisition step in which the user terminal device acquires the user information, and the user terminal device responds to the question by learning results by machine learning. An automatic answer, which is an answer automatically generated based on the above, is output as the answer, and it is determined that the automatic answer cannot reach an appropriate answer based on the user information acquired by the user information acquisition step. When this is done, the user terminal device includes the response processing step of executing the process of connecting to the respondent terminal device, and in the response processing step, the user terminal device is based on the user's information. However, it is a question answering method characterized by determining that the automatic answer cannot reach an appropriate answer when it is determined that the person is an elderly person or a physically handicapped person who has a high priority of manned guidance .

In order to solve the above problem, one aspect of the present invention is a user terminal device that outputs an answer to a question input by a user, and a respondent terminal that transmits the answer by the respondent to the user terminal device. The user terminal device includes an apparatus, and the user terminal device outputs an automatic answer, which is an answer automatically generated based on a learning result by machine learning, as the answer to the question, and evaluates the answer to the question. A response processing unit that executes a process of connecting to the respondent terminal device when it is determined that the automatic answer has not reached an appropriate answer based on the index is provided, and the evaluation index includes the use. The automatic answer is appropriate when the non-input period in which the question is not input by the user terminal device is included and the response processing unit reaches the threshold period for the non-input period in the question. It is a question-and-answer system characterized by determining that an answer has not been reached.

Further, one aspect of the present invention includes a user terminal device that outputs an answer to a question input by the user, and a respondent terminal device that transmits the answer by the respondent to the user terminal device. The user terminal device outputs the user information acquisition unit that acquires the user's information and the automatic answer, which is an answer automatically generated based on the learning result by machine learning, as the answer to the question. At the same time, a response that executes a process of connecting to the respondent terminal device when it is determined that the automatic answer cannot reach an appropriate answer based on the user information acquired by the user information acquisition unit. The user information acquisition unit includes a processing unit, and the response processing unit is an image pickup unit that captures an image of the user, and the response processing unit is the user imaged by the image pickup unit, which is information on the user. Based on the image of, when it is determined that the user is an elderly person or a physically handicapped person who has a high priority of manned guidance, it is determined that the automatic answer cannot reach an appropriate answer. It is a question and answer system.

Further, one aspect of the present invention includes a user terminal device that outputs an answer to a question input by the user, and a respondent terminal device that transmits the answer by the respondent to the user terminal device. In the question answering method of the system, the user terminal device outputs an automatic answer, which is an answer automatically generated based on a learning result by machine learning, as the answer to the question, and also responds to the question. The evaluation index includes a response processing step for executing a process of connecting to the respondent terminal device when it is determined that the automatic response has not reached an appropriate answer based on the evaluation index related to the response. Includes a non-input period in which the user does not input the question in the user terminal device, and in the response processing step, the user terminal device has a threshold period in which the question is not input. It is a question answering method characterized by determining that the automatic answer has not reached an appropriate answer when the answer reaches.

Further, one aspect of the present invention includes a question answering device including a user terminal device that outputs an answer to a question input by the user and a respondent terminal device that transmits the answer by the respondent to the user terminal device. In the question answering method of the system, the user information acquisition step in which the user terminal device acquires the user information, and the user terminal device responds to the question by learning results by machine learning. The automatic answer, which is an answer automatically generated based on the above, is output as the answer, and it is determined that the automatic answer cannot reach an appropriate answer based on the information of the user acquired by the user information acquisition step. In the case of the above, the user information acquisition step includes an image of the user by the image pickup unit of the user terminal device, including a response processing step of executing a process of connecting to the respondent terminal device. In the response processing step, the user has a high priority of manned guidance based on the user's image captured by the image pickup unit, which is the user's information. It is a question answering method characterized by determining that the automatic answer cannot reach an appropriate answer when it is determined that the person or a person has a physical disability.

Claims (14)

A user terminal device that outputs an answer to a question input by a user and a respondent terminal device that transmits the answer by the respondent to the user terminal device are provided.
The user terminal device is
For the question, an automatic answer that is an answer automatically generated based on the learning result by machine learning is output as the answer, and the automatic answer is an appropriate answer based on the evaluation index for the answer to the question. A question answering system comprising a response processing unit that executes a process of connecting to the respondent terminal device when it is determined that the answering has not been reached. The evaluation index includes a certainty degree indicating the certainty of the automatic answer in the automatic generation process based on the learning result.
The first aspect of the present invention is characterized in that the response processing unit determines that the automatic answer has not reached an appropriate answer when the automatic answer having the certainty degree of the threshold value or less continues a predetermined number of times. The question answering system described. The evaluation index includes the number of times of the question entered by the user.
The first or second aspect of the present invention, wherein the response processing unit determines that the automatic answer has not reached an appropriate answer when the number of times of the question reaches the threshold number. Question answering system. The evaluation index includes a period of no input by the user without input of the user terminal device.
Any of claims 1 to 3, wherein the response processing unit determines that the automatic answer has not reached an appropriate answer when the non-input period reaches the threshold period. The question answering system described in paragraph 1. The evaluation index includes the evaluation result of the automatic response input by the user.
The question according to any one of claims 1 to 4, wherein the response processing unit determines based on the evaluation result that the automatic answer has not reached an appropriate answer. Response system. A user terminal device that outputs an answer to a question input by a user and a respondent terminal device that transmits the answer by the respondent to the user terminal device are provided.
The user terminal device is
The user information acquisition unit that acquires the user information and
In response to the question, an automatic answer, which is an answer automatically generated based on the learning result by machine learning, is output as the answer, and the user's information acquired by the user information acquisition unit is used as the answer. A question answering system including a response processing unit that executes a process of connecting to the respondent terminal device when it is determined that an appropriate answer cannot be reached. The user terminal device includes a display unit that displays information including the automatic answer.
When it is determined that the automatic answer has not reached an appropriate answer, the response processing unit starts the connection with the respondent terminal device as a process of connecting with the respondent terminal device. The question answering system according to any one of claims 1 to 6, wherein the display information prompting the user is displayed on the display unit. It is provided with an answer generation unit that generates the automatic answer from the question based on the learning result by the machine learning.
The question answering system according to any one of claims 1 to 7, wherein the response processing unit acquires the automatic answer generated by the answer generation unit for the question. The user terminal device includes a sound collecting unit that collects the user's voice as input sound data and a sound emitting unit that outputs output sound data based on the answer as voice.
It is provided with a sound signal conversion unit that converts the input sound data collected by the sound collecting unit into input text data and also converts the output text data including the automatic answer into output sound data.
Claim 1 to claim 1, wherein the response processing unit acquires input text data of the question from the sound signal conversion unit, and converts the automatic answer into the output sound data by the sound signal conversion unit. The question and answer system according to any one of items 8. It is provided with a translation processing unit that mutually translates at least the first language used by the respondent and the second language used by the user.
The response processing unit causes the translation processing unit to translate the input question in the second language into the question in the first language, and translates the answer in the first language into the answer in the second language. The question-and-answer system according to any one of claims 1 to 9, wherein the answer in the second language is output. The respondent terminal device is
Respondent display and
When the respondent terminal device is connected to the user terminal device, it is provided with a response processing unit that acquires an answer from the respondent and transmits the acquired answer to the user terminal device.
The user terminal device includes a user display unit which is a display unit that can be visually recognized by the user.
The answer processing unit is image data to which the respondent can add information, and the image data used for explaining the answer is displayed on the respondent display unit, and the image data is displayed on the user terminal device. Send to
The question answering system according to any one of claims 1 to 10, wherein the response processing unit displays the image data on the user display unit. The question according to claim 11, wherein the response processing unit displays on the user display unit a two-dimensional code that enables the user to acquire and display the image data on a mobile terminal owned by the user. Response system. It is a question answering method of a question answering system including a user terminal device that outputs an answer to a question input by a user and a respondent terminal device that transmits the answer by the respondent to the user terminal device.
The user terminal device outputs an automatic answer, which is an answer automatically generated based on a learning result by machine learning, to the question as the answer, and also based on an evaluation index related to the answer to the question. A question response method comprising a response processing step for executing a process of connecting to the respondent terminal device when it is determined that the automatic answer has not reached an appropriate answer. It is a question answering method of a question answering system including a user terminal device that outputs an answer to a question input by a user and a respondent terminal device that transmits the answer by the respondent to the user terminal device.
The user information acquisition step in which the user terminal device acquires the user information, and
The user terminal device outputs an automatic answer, which is an answer automatically generated based on a learning result by machine learning, to the question as the answer, and the use acquired by the user information acquisition step. A question response method comprising a response processing step for executing a process of connecting to the respondent terminal device when it is determined that the automatic answer cannot reach an appropriate answer based on the information of the respondent. ..

Images