JP7488617B1 - Program, method, information processing device, and system - Google Patents

Abstract

[Problem] To provide a program, method, information processing device, and system for outputting an appropriate answer to a question. [Solution] The method receives a question from a user, inputs the received question as input data into a large-scale language model, and obtains a judgment result as to whether the question is appropriate as input data for a question answering system capable of outputting an answer in response to the input of the question, and if a judgment result that the question is appropriate is obtained, an answer to the question that is output by inputting input data based on the question into the question answering system is presented to the user, and if a judgment result that the question is not appropriate is obtained, a reply to delve deeper into the question, rather than an answer to the question, is presented to the user, which is output by inputting input data based on the question into a large-scale language model. [Selected Figure] Figure 11

Description

This disclosure relates to a program, a method, an information processing device, and a system.

2. Description of the Related Art Systems (chatbot systems) that respond to online inquiries such as questions from users and output answers corresponding to those questions are known.
Patent Literature 1 discloses a technology that enables a chatbot to automatically respond to inquiries from users, and when the time comes to hand over to a human responder, to seamlessly hand over the response to the respondent.

JP 2022-189237 A

There is a problem in that it is not possible to output an appropriate answer to a question.
Therefore, the present disclosure has been made to solve the above problem, and has an object to provide a technology for outputting an appropriate answer to a question.

A program to be executed by a computer having a processor and a storage unit, the program executing: a question receiving step in which the processor receives a question from a user; a judgment step in which the question received in the question receiving step is input as input data into a large-scale language model to obtain a judgment result as to whether the question is appropriate as input data for a question answering system capable of outputting an answer in response to the input of the question; a first answering step in which, if the judgment step obtains a judgment result that the question is appropriate, an answer to the question is output by inputting input data based on the question into the question answering system, to the user; and a second answering step in which, if the judgment step does not obtain a judgment result that the question is appropriate, an answer to the question is output by inputting input data based on the question into the large-scale language model, to the user, instead of an answer to the question, but a response to delve deeper into the question.

According to this disclosure, it is possible to output an appropriate answer to a question.

FIG. 2 is a block diagram showing the functional configuration of the system 1. FIG. 2 is a block diagram showing the functional configuration of the server 10. FIG. 2 is a block diagram showing the functional configuration of a user terminal 20. FIG. 13 is a diagram showing the data structure of a user table 1012. FIG. 11 is a diagram showing the data structure of a question table 1013. FIG. 10 is a diagram showing the data structure of a keyword table 1014. FIG. 11 is a diagram showing the data structure of a candidate table 1015. FIG. 10 is a diagram showing the data structure of a response table 1016. 13 is a flowchart showing the operation of a search process. 13 is a flowchart showing an operation of an interaction process. 13 is a flowchart showing the operation of a proposal process. 1 is an example of the first screen of a dialogue screen. 13 is a second example of a dialogue screen. FIG. 2 is a block diagram showing the basic hardware configuration of a computer 90.

Embodiments of the present disclosure will be described below with reference to the drawings. In all figures describing the embodiments, common components are given the same reference numerals, and repeated explanations will be omitted. Note that the following embodiments do not unduly limit the contents of the present disclosure described in the claims. Furthermore, not all components shown in the embodiments are necessarily essential components of the present disclosure. Furthermore, each figure is a schematic diagram, and is not necessarily a precise illustration.

<Configuration of System 1>
The system 1 in the present disclosure is an information processing system that provides an information processing service (FAQ service) that outputs suitable answers to questions input by a user. For example, in a bank or the like, the system provides an information processing service that receives questions from users regarding account opening, domestic and international remittances, loan applications, ATMs, how to use cash cards, how to use mobile banking, etc., and provides the users with answers to solve their doubts and problems related to the questions.
The application of the information processing service according to the present disclosure is not limited to banks, but can be applied to any business entity, such as public transportation such as railways, airplanes, and taxis, accommodation facilities such as hotels and guesthouses, software services, hospitals, etc.
The system 1 comprises information processing devices including a server 10, a user terminal 20, a generation AI 50, and a question and answer system 60, all connected via a network N.
FIG. 1 is a block diagram showing the functional configuration of the system 1.
FIG. 2 is a block diagram showing the functional configuration of the server 10. As shown in FIG.
FIG. 3 is a block diagram showing the functional configuration of the user terminal 20. As shown in FIG.

Each information processing device is composed of a computer equipped with an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the server 10, the user terminal 20, the generation AI 50, and the question answering system 60, explanations that overlap with the basic hardware configuration of the computer and the basic functional configuration of the computer described later will be omitted.

<Configuration of Server 10>
The server 10 is an information processing device that provides an information processing service (FAQ service) that outputs appropriate answers to questions input by a user by communicating with the generation AI 50 and question answering system 60 described below.
The server 10 includes a memory unit 101 and a control unit 104 .

<Configuration of storage unit 101 of server 10>
The storage unit 101 of the server 10 includes an application program 1011 , a user table 1012 , a question table 1013 , a keyword table 1014 , a candidate table 1015 , and an answer table 1016 .

The application program 1011 is a program for causing the control unit 104 of the server 10 to function as each functional unit.
The application programs 1011 include applications such as a web browser application.

The user table 1012 is a table that stores and manages information about member users (hereinafter, users) who use the service. When a user registers to use the service, the user's information is stored in a new record in the user table 1012. This allows the user to use the service according to the present disclosure.
The user table 1012 is a table having columns of user ID and user name, with the user ID as the primary key.
FIG. 4 is a diagram showing the data structure of the user table 1012. As shown in FIG.

The user ID is an item for storing user identification information for identifying a user. The user identification information is an item for which a unique value is set for each user.
The user name is an item for storing the name of the user. The user name may be set to any character string such as a nickname instead of a name.

The question table 1013 is a table for storing and managing information relating to questions (question information).
The question table 1013 is a table having a question ID as a primary key, and columns of a question ID, a parent question ID, a user ID, a question text, and a date and time.
FIG. 5 is a diagram showing the data structure of the question table 1013. As shown in FIG.

The question ID is an item for storing question identification information for identifying a question. The question identification information is an item for which a unique value is set for each piece of question information.
The parent question ID is an item for storing parent question identification information for identifying a parent question.
The user ID is an item for storing user identification information for identifying a user.
The question sentence is an item for storing character string information (text information) related to the question sentence.
The date and time is an item for storing the date and time when a record is newly stored or updated in the question table 1013.

The keyword table 1014 is a table for storing and managing information relating to search keywords (search keyword information).
The keyword table 1014 is a table having columns for question ID, input sentence, keyword, and date and time.
FIG. 6 is a diagram showing the data structure of the keyword table 1014. As shown in FIG.

The question ID is an item for storing question identification information for identifying a question.
The input sentence is an item that stores character string information (text information) related to the input sentence to the generation AI 50.
The keyword is an item for storing character string information (text information) related to a search keyword. The keyword stores character string information related to one or more search keywords.
The date and time is an item for storing the date and time when a record is newly stored or updated in the keyword table 1014.

The candidate table 1015 is a table for storing and managing information relating to candidates (candidate information).
The candidate table 1015 is a table having columns for question ID, input sentence, question candidate, and date and time.
FIG. 7 is a diagram showing the data structure of the candidate table 1015. As shown in FIG.

The question ID is an item for storing question identification information for identifying a question.
The input sentence is an item that stores character string information (text information) related to the input sentence to the generation AI 50.
The question candidate is an item that stores character string information (text information) related to the question candidate. The question candidate stores character string information related to one or more question candidates.
The date and time is an item for storing the date and time when a record is newly stored or updated in the candidate table 1015.

The answer table 1016 is a table for storing and managing information related to answers (answer information).
The answer table 1016 is a table having columns for question ID, process type, input text, answer text, and date and time.
FIG. 8 is a diagram showing the data structure of the answer table 1016. As shown in FIG.

The question ID is an item for storing question identification information for identifying a question.
The processing type is an item for storing the type of processing executed to obtain the answer sentence.
Specifically, the process type stores information indicating any one of an interactive process, a search process, and a proposal process.
The input sentence is an item that stores string information (text information) related to the input sentence to the generation AI 50 or the question and answering system 60.
The answer sentence is an item that stores string information (text information) related to the output sentence output from the generation AI 50 or the question answering system 60.
The date and time is an item for storing the date and time when a record is newly stored or updated in the answer table 1016.

<Configuration of control unit 104 of server 10>
The control unit 104 of the server 10 includes a user registration control unit 1041, an AI control unit 1042, and a question answering unit 1043. The control unit 104 executes an application program 1011 stored in the storage unit 101, thereby realizing each functional unit.

The user registration control unit 1041 performs processing to store information about users who wish to use the service according to the present disclosure in the user table 1012 .
The information stored in the user table 1012 is generated when a user opens a web page operated by a service provider from any information processing terminal, inputs information into a specific input form, and transmits the information to the server 10. The user registration control unit 1041 stores the received information in a new record in the user table 1012, completing the user registration. This allows the user stored in the user table 1012 to use the service.
Before the user registration control unit 1041 registers user information in the user table 1012, the service provider may carry out a predetermined examination to restrict whether or not the user is permitted to use the service.
The user ID may be any character string or number that can identify the user, any character string or number desired by the user, or an arbitrary character string or number may be automatically set by the user registration control unit 1041.

The AI control unit 1042 is a control unit that controls input and output to the generated AI 50. Details will be described later.

The question answering unit 1043 is a control unit that controls input and output to the question answering system 60. Details will be described later.

<Configuration of User Terminal 20>
The user terminal 20 is an information processing device operated by a user who uses the service. The user terminal 20 may be, for example, a mobile terminal such as a smartphone or a tablet, a stationary personal computer (PC) or a laptop PC, or a wearable terminal such as a head mounted display (HMD) or a wristwatch terminal.
The user terminal 20 includes a memory unit 201 , a control unit 204 , an input device 206 , and an output device 208 .

<Configuration of storage unit 201 of user terminal 20>
The storage unit 201 of the user terminal 20 includes a user ID 2011 and an application program 2012 .

The user ID 2011 is the user's account ID. The user transmits the user ID 2011 from the user terminal 20 to the server 10. The server 10 identifies the user based on the user ID 2011 and provides the user with the services disclosed herein. The user ID 2011 includes information such as a session ID that is temporarily assigned by the server 10 when identifying the user who is using the user terminal 20.

The application program 2012 may be pre-stored in the storage unit 201, or may be configured to be downloaded from a web server operated by a service provider via a communication IF.
The application programs 2012 include applications such as a web browser application.
The application programs 2012 include an interpreted programming language such as JavaScript (registered trademark) that runs on a web browser application stored on the user terminal 20 .

<Configuration of the control unit 204 of the user terminal 20>
The control unit 204 of the user terminal 20 includes an input control unit 2041 and an output control unit 2042. The control unit 204 executes an application program 2012 stored in the storage unit 201, thereby realizing each functional unit.

<Configuration of the input device 206 of the user terminal 20>
The input device 206 of the user terminal 20 includes a camera 2061 , a microphone 2062 , a position information sensor 2063 , a motion sensor 2064 , and a touch device 2065 .

<Configuration of the output device 208 of the user terminal 20>
The output device 208 of the user terminal 20 includes a display 2081 and a speaker 2082 .

<Configuration of Generation AI 50>
Generative AI50 refers to large-scale artificial intelligence models used in the field of natural language processing (NLP). These models learn from large amounts of text data (web pages, books, articles, etc.) to understand patterns in human language and perform natural language generation (NLG) tasks effectively.
The generative AI 50 is used in many NLP tasks, such as generating responses to specific questions, automatically generating sentences, summarizing text, translation, and sentiment analysis. It can also be used for a variety of purposes, such as education, entertainment, customer service, and product development. The generative AI 50 includes the following: In this disclosure, a large-scale language model that mainly outputs text information as output information will be described as a type of generative AI 50.
・OpenAI ChatGPT
・Google Bard
・Stable Diffusion
・midjourney
In addition, the generation AI 50 may be configured to be realized as part of the function of the server 10.

<Configuration of question and answer system 60>
The question and answering system 60 is an information processing system that outputs a character string related to an answer in response to an input of a character string related to a question. For example, the question and answering system 60 includes a FAQ system.
Specifically, the question and answering system 60 is an information processing system that stores pairs of frequently asked questions from users and their answers (question and answer information). When the question and answering system 60 receives an input of an input character string, it searches for at least one of the questions or answers included in the question and answer information based on the input character string by full-text search or the like, and outputs the question and answer information related to the search result in association with the relevance (score) to the input character string.
By inputting a character string such as a search keyword or a question sentence into question answering system 60, it is possible to obtain an output of question answering information related to the character string. There does not need to be only one question answering system 60, and for example, a configuration may be adopted in which a question answering system A storing question answering information for Bank A, a plurality of question answering systems 60 are provided for each business operator providing a service related to questions, such as Bank B, Bank C, Accommodation Facility A, Accommodation Facility B, etc.
The question and answer system 60 may be configured to be realized as a part of the functions of the server 10 .

<Operation of System 1>
Each process of the system 1 will be described below.
FIG. 9 is a flowchart showing the operation of the search process.
FIG. 10 is a flowchart showing the operation of the dialogue processing.
FIG. 11 is a flowchart showing the operation of the proposal process.
FIG. 12 is an example of the first screen of the dialogue screen.
FIG. 13 is an example of a second screen of the dialogue screen.

The search process, dialogue process, and suggestion process in this disclosure will be described below.
In the information processing system disclosed herein, a search process, a dialogue process, and a proposal process are each executed for one question received from a user. Note that it is not necessary to execute all of the search process, the dialogue process, and the proposal process for one question received from a user. A configuration may be adopted in which at least one of the search process, the dialogue process, and the proposal process is executed for one question received from a user.
In addition, the search process, the dialogue process, and the proposal process may be executed in parallel or in sequence. When executed in sequence, a configuration may be adopted in which, depending on the execution result of a specific process, some of the processes subsequent to the specific process are not executed (skiped). The start of the search process, the dialogue process, and the proposal process may be executed in parallel, and after obtaining the respective process results, the presentation of the process results may be executed in sequence. In this case, too, a configuration may be adopted in which, depending on the execution result of a specific process, some of the process results subsequent to the specific process are not presented (skiped).
For example, in a case where a search process, a proposal process, and an interactive process are executed in this order, the execution of the proposal process and the interactive process may be omitted depending on the processing result of the search process. Similarly, the execution of the interactive process may be omitted depending on the processing result of the proposal process.

The information processing system disclosed herein ultimately presents to the user the output results from the question answering system 60 resulting from the search process or suggestion process. However, the user is not necessarily able to input an appropriate question. Therefore, until it is determined that the question answering system 60 can provide an appropriate answer to the question input by the user, a response text that is generated by the generation AI 50 based on the question text and that delves more specifically into the question text is presented to the user, thereby encouraging the user to input an additional question text. When it is determined that the question answering system 60 can provide an appropriate answer in response to a sufficient additional question text, the output results from the question answering system 60 are presented to the user.

In the information processing system of the present disclosure, in response to a question from a user, an answer sentence stored in advance in the question answering system 60 is presented, so that an inappropriate answer sentence is not presented to the user. On the other hand, if a similar process is performed in the conventional generation AI 50, problems such as hallucination (answering content that is different from the facts or content that is unrelated to the context) and generating an answer sentence containing an incorrect content will occur.
In addition, although the generation AI 50 does not provide an answer to the question received from the user, by presenting the user with a sentence (response sentence) that includes suggestions for naturally digging deeper into the question received from the user, it is possible to make abstract questions from the user more concrete.
The information processing system of the present disclosure utilizes a generation AI 50 to delve into the content of a question that the user himself has not been able to verbalize, and provides the user with an appropriate answer that does not contain erroneous content and is pre-stored in a question answering system 60 based on the specified question.

<Search process>
The search process is a process of presenting to the user an output statement from the question and answer system 60 based on a search keyword corresponding to a question received from the user.

<Search process overview>
The search process is a process of accepting a question from a user, inputting the input question into generation AI 50, obtaining search keywords output from generation AI 50, inputting the search keywords into question answering system 60, obtaining the output content output from question answering system 60, and presenting the output content as an answer.

<Search process details>
The search process will be described in detail below.

The user operates the input device 206 of the user terminal 20 to execute a browser application or the like, and opens the interactive screen D1 by inputting the URL or the like of a web page (interactive screen) for performing a search process. The control unit 204 of the user terminal 20 transmits a request including the user ID 2011 to open the interactive screen D1 to the server 10.

When the server 10 receives the request, it generates an interactive screen D1 and transmits it to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the interactive screen D1 on the display 2081 of the user terminal 20.

12 shows an example of the first screen of the interactive screen. An interactive screen D1 is displayed on the display 2081 of the user terminal 20.
The dialogue screen D1 includes a question input field D11, questions D121 and D122, and answers D131, D132, D133, D134, and D135.

13 is an example of a second screen of the interactive screen. An interactive screen D2 is displayed on the display 2081 of the user terminal 20.
The dialogue screen D2 includes a question input field D21, questions D221, D222, D223, and D224, answers D231, D232, D233, D234, and D235, and question candidates D251 and D252.

The first and second example screens of the dialogue screen are both examples of screens that present the results of the search process, dialogue process, and suggestion process as answers. Specifically, the first example screen is an example of a screen that appears when the user first inputs "overseas" as a question, and the second example screen is an example of a screen that appears when the user first inputs "account" as a question. Different answers are displayed depending on the content of the question received from the user.
In the second screen example, question candidates D251 and D252 are displayed. The question candidates will be described later in the proposal process.

In step S101, the AI control unit 1042 of the server 10 executes a question receiving step of receiving a question from a user.
The user inputs a question into the input field D11 by operating the input device 206 of the user terminal 20. The control unit 204 of the user terminal 20 transmits the question input into the input field D11 to the server 10. For example, the user can transmit the question input into the input field D11 to the server 10 by pressing an enter key or the like on the user terminal 20. The request transmitted to the server 10 includes a user ID 2011.
The control unit 104 of the server 10 receives and accepts the question and the user ID 2011. The control unit 104 of the server 10 stores the accepted question and user ID 2011 in the question and user ID fields of the new record in the question table 1013, respectively.

In step S101, the question receiving step includes a step of receiving a first question sentence, and a step of receiving a second question sentence.
Specifically, in the search process of the present disclosure, the user can check the answer sentence D132 outputted in response to the question sentence D121 (first question sentence), and can further send a new additional question sentence D122 (second question sentence) based on the answer sentence D132.
In this case, the control unit 104 of the server 10 stores the question ID, question D122, and user ID 2011 in the question table 1013 of the first question, which is the first question of the second question, in the parent question ID, question, and user ID fields of the new record in the question table 1013, respectively.

In step S101, the question receiving step executes a step capable of receiving an additional question sentence if one or more search keywords cannot be acquired in the keyword acquiring step.
Specifically, the control unit 204 of the user terminal 20 may control the question input field D11 to be capable of accepting input operations from the input device 206 of the user terminal 20 when the search keyword cannot be obtained in step S103 described below, and may control the question input field D11 to be unable to accept input operations from the input device 206 of the user terminal 20 when the search keyword can be obtained.
In other words, if the search process does not obtain an answer from question and answer system 60, additional questions may be accepted, and if the search process obtains an answer from question and answer system 60, additional questions may not be accepted.
When an answer sentence is obtained from the question and answering system 60 by the search process according to the present disclosure, the system may be configured not to accept any further question sentences input from the user. This makes it possible to end the search process, dialogue process, and proposal process by presenting the answer sentence from the question and answering system 60 to the user. In other words, when an answer to the search process is obtained, the dialogue with the user may be ended.

In step S102, the AI control unit 1042 of the server 10 transmits the question received in the question receiving step to the generation AI 50.
Specifically, the AI control unit 50 of the server 10 generates a prompt by inserting the question received in step S101 into the [Question] section of the following prompt template as an input sentence (prompt) for the generation AI 50, which is a large-scale language model, and transmits the generated prompt to the generation AI 50. The prompt template in the present disclosure includes a command statement that instructs an answer to delve deeper into the question without providing an answer to the question from the user. In other words, in the information processing system according to the present disclosure, the generation AI 50 does not provide an answer to the question from the user. In the information processing system according to the present disclosure, an answer to a question from a user is provided based on the answer result (search result) by the question answering system 60.
In the search process, the prompt includes an instruction to restrict the output data format of the generation AI 50 to output search keywords instead of answer sentences. Specifically, in OpenAI's GPT, an instruction to extract search keywords to be input to the question answering system 60 is included using a mechanism called FunctionCalling.
[Prompt template]
You are a very kind and capable AI assistant.
It resides on the FAQ site and supports users in searching for FAQs.
If the user's question is unclear, ask questions to delve deeper into the issue.
Since you are an AI assistant, you should never give users specific solutions to their problems.
Once the problem has been clarified, we will suggest what search keywords to use on the FAQ site.
#User question [question text]

In step S102, the AI control unit 1042 of the server 10 inputs the first question, the answer to the second question, and the second question into the large-scale language model.
Specifically, when the AI control unit 50 of the server 10 receives an additional question D122 in the question receiving step, it generates a prompt by inserting a string combining the question D121, answer D132, and question D122 into the [Question] section of the prompt template, and sends the prompt to the generation AI 50.
In other words, if one or more question sentences have been received in the past in step S101, and one or more answer sentences have been obtained for those one or more question sentences, a string formed by combining those one or more question sentences and one or more answer sentences with the newly received question sentence is sent to generation AI 50 as a question sentence.
The control unit 104 of the server 10 can acquire one or more question sentences included in one or more pieces of question information previously received by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information. The control unit 104 of the server 10 can acquire any parent generation question information and question sentence by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information.
The control unit 104 of the server 10 can obtain one or more answer sentences included in one or more pieces of answer information by searching the items in the answer table 1016 based on one or more question IDs included in one or more pieces of question information previously received. The question information and the answer information can be determined in correspondence (order relationship, which answer sentence corresponds to which question sentence) by the question ID. The order of the one or more answer sentences may also be the order of the date and time included in the answer information. The control unit 104 of the server 10 combines one or more question sentences and one or more answer sentences in order to generate a character string.
The one or more answer sentences to be combined may be answer sentences obtained by a search process, or may include answer sentences obtained by a dialogue process or a suggestion process.
In addition, it is not necessary to combine all of the one or more question sentences and one or more answer sentences. It is also possible to configure the generation AI 50 to combine a certain number of question sentences and answer sentences in order, starting with the most recent, up to the number of characters that it can accept.

In step S103, the AI control unit 1042 of the server 10 executes a judgment step of inputting the question sentence received in the question receiving step into a large-scale language model as input data, and obtaining a judgment result as to whether the question sentence is suitable as input data for a question answering system capable of outputting an answer sentence in response to the input of a question sentence.
Specifically, the judgment step includes a step of acquiring one or more keywords contained in the question sentence by inputting the question sentence accepted in the question receiving step into a large-scale language model as input data, a step of acquiring a judgment result indicating that the question sentence is suitable as input data if the one or more keywords are acquired, and a step of acquiring a judgment result indicating that the question sentence is not suitable as input data if the one or more keywords are not acquired.
The determining step includes a keyword obtaining step.

In step S103, the AI control unit 1042 of the server 10 executes a keyword acquisition step of acquiring one or more keywords that are output by inputting the question sentence received in the question receiving step into a large-scale language model as input data.
Specifically, the AI control unit 1042 of the server 10 receives and accepts the search keywords output from the large-scale language model.

In step S103, a keyword acquisition step executes a step of acquiring one or more keywords that are output by inputting the question sentence and the additional question sentence as input data into a large-scale language model.
Specifically, in step S102, when the AI control unit 50 of the server 10 receives an additional question D122 in the question receiving step, it receives and accepts the search keywords output from the large-scale language model in response to inputting the string combining the question D121, answer D132, and question 122 into the generation AI 50 as input data, which is a prompt inserted into the [question] section of the prompt template.

For example, the keyword acquisition step includes a step of acquiring a first keyword that is output by inputting the first question sentence as input data into the large-scale language model. The keyword acquisition step includes a step of acquiring a second keyword that is output by inputting the first question sentence, the second question sentence, and the first answer sentence as input data into the large-scale language model.

The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S101, the prompt generated in step S102, and the search keyword in the question ID, input sentence, and keyword fields of the new record in the keyword table 1014, respectively.

In addition, the large-scale language model may not output search keywords in step S103. For example, if the user's question included in the prompt does not contain enough search keywords, the generation AI 50 does not output search keywords. In such a case, steps S104 and after are skipped and the search process is terminated.
In this case, the control unit 104 of the server 10 may be configured to execute step S302 and subsequent steps of the dialogue process.
In this case, the control unit 104 of the server 10 may be configured to execute step S502 and subsequent steps of the proposal process.

In step S104, a question input step executes a step of inputting one or more keywords acquired in the keyword acquisition step into the question and answer system.
Specifically, the question answering unit 1043 of the server 10 transmits the search keywords output from the generation AI 50 in step S103 to the question answering system 60.
For example, the question answering unit 1043 of the server 10 transmits the first search keyword and the second search keyword output from the generation AI 50 in step S103 to the question answering system 60.

In step S105, the question and answering unit 1043 of the server 10 executes an answer acquisition step of acquiring a predetermined answer sentence in response to the input of one or more keywords to the question and answering system in the question input step.
Specifically, the question answering unit 1043 of the server 10 receives and accepts the output contents output from the question answering system 60. For example, when the question answering system 60 receives an input character string related to a search keyword, it searches for at least one of the questions or answers included in the question answering information stored in the question answering system 60 based on the input character string by full-text search or the like, and outputs the question answering information related to the search result in association with the relevance (score) to the input character string.
The question answering unit 1043 of the server 10 receives and acquires the answer sentence included in the question answer information.

For example, the answer acquisition step includes a step of acquiring a first answer sentence in response to input of a first keyword to the question answering system in the question input step. The answer acquisition step includes a step of acquiring a second answer sentence in response to input of a second keyword to the question answering system in the question input step.

In step S105, the question answering unit 1043 of the server 10 receives and accepts the answer sentence output from the question answering system 60. The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S101, the prompt generated in step S102, and the answer sentence in the question ID, input sentence, and answer sentence fields of the new record in the answer table 1016, respectively.

In step S106, the control unit 104 of the server 10 presents the answer sentence output from the question and answer system 60.
Specifically, the control unit 104 of the server 10 transmits the answer sentence received in step S105 to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the received answer sentence on the display 2081 of the user terminal 20.
For example, when the control unit 204 of the user terminal 20 receives the question D121 in step S101, a search keyword cannot be obtained from the question D121, so that the answer to the question D121 is not stored in the answer table 1016 and is not displayed on the dialogue screen D1. When the control unit 204 of the user terminal 20 receives the question D122 in step S101, the answer D135 is displayed on the dialogue screen D1.
Furthermore, a sentence including a keyword stored in the keyword table 1014 may be displayed on the dialogue screen D1 as an answer sentence D134. By referring to the answer sentence D134, the user can visually confirm what search keyword has been acquired from the question sentence D122 and what answer sentence D135 has been obtained in response to the search keyword.
In addition, in the interactive screen D2 in the second screen example of the interactive screen, since the search keyword could not be obtained, the answer sentence from the question and answer system 60 to the question sentence is not stored in the answer table 1016 and is not displayed on the interactive screen D2.

In step S106, if the control unit 104 of the server 10 obtains a judgment result in the judgment step that the question is appropriate, it executes a first answering step of presenting to the user an answer to the question that is output by inputting input data based on the question into the question answering system.
The first answering step includes a step of presenting to the user an answer to the question, which is output by inputting the one or more keywords acquired in the determining step as input data to the question and answering system.
The first answering step includes a step of presenting to the user an answer to the question sentence that is output by inputting the second keyword as input data to the question and answering system.

The control unit 104 of the server 10 executes an answering step of presenting a first answer sentence based on at least one of a first answering step and a second answering step, based on the first question sentence.
The AI control unit 1042 of the server 10 executes an answering step of presenting a second answer sentence based on at least one of the first answering step, the second answering step, and the third answering step, based on the first question sentence.
Specifically, in step S101, if one or more question sentences have been received in the past and one or more answer sentences have been obtained for the one or more question sentences, the one or more question sentences and one or more answer sentences are combined with a newly received question sentence to generate a character string, which is then transmitted as a question sentence to the generation AI 50. The one or more answer sentences to be combined may be answer sentences obtained by a search process, or may include answer sentences obtained by a dialogue process or a suggestion process.
The keywords obtained from the generation AI 50 are input as input data to the question answering system 60, and the answer to the question obtained thereby is presented to the user.

In step S106, the control unit 104 of the server 10 acquires the question information stored in the question table 1013. The control unit 104 of the server 10 searches the question ID item in the answer table 1016 based on the question ID included in the acquired question information, and acquires the answer information. The control unit 104 of the server 10 transmits the acquired question information and answer information to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the received question information and answer information on the display 2081 of the user terminal 20 in the order of the date and time included therein. As a result, a first screen example and a second screen example including the processing results of all the processes of the search process, the dialogue process, and the proposal process are displayed and presented on the display 2081 of the user terminal 20.
In other words, the steps related to presenting answer sentences, namely step S106 of the search process, step S304 of the dialogue process, and step S508 of the proposal process, may be a common process of displaying a list of the question information stored in the question table 1013 and the answer information stored in the answer table 1016.

<Dialogue processing>
Dialogue processing is a process in which output sentences from the generation AI 50 in response to questions received from the user are presented to the user.

<Overview of dialogue processing>
The dialogue processing is a process in which a question is input from a user, the input question is input to the generation AI 50, the output content output from the generation AI 50 is obtained, and the output content is presented as an answer.

<Details of dialogue processing>
The details of the interaction process will be described below.

In step S301, the AI control unit 1042 of the server 10 executes a question receiving step of receiving a question from a user.
The question reception step is the same as step S101 of the search process, and therefore will not be described here. In the present disclosure, a dialogue process is executed in response to the question received in step S101 of the search process. That is, step S301 of the dialogue process is common to step S101 of the search process.
In addition, the search process does not necessarily have to be executed, and the dialogue process may be executed. In addition, in the present disclosure, the dialogue process may be executed when the search keyword cannot be acquired in step S103 of the search process (when the question sentence is not determined to be appropriate).

In step S302, the AI control unit 1042 of the server 10 transmits the question received in the question receiving step to the generation AI 50.
Specifically, the AI control unit 50 of the server 10 generates a prompt by inserting the question sentence received in step S301 into the [Question sentence] section of the prompt template as an input sentence (prompt) for the generation AI 50, which is a large-scale language model, and sends the prompt to the generation AI 50.

In step S302, the AI control unit 1042 of the server 10 inputs the first question, the answer to the second question, and the second question into the large-scale language model.
Specifically, when the AI control unit 50 of the server 10 receives an additional question D122 in the question receiving step, it generates a prompt by inserting a string combining the question D121, the answer D132, and the question D122 into the [Question] section of the prompt template, and sends the prompt to the generation AI 50.
In other words, if one or more question sentences have been received in the past in step S301, and one or more answer sentences have been obtained for those one or more question sentences, a string formed by combining those one or more question sentences and one or more answer sentences with the newly received question sentence is sent to generation AI 50 as a question sentence.
The control unit 104 of the server 10 can acquire one or more question sentences included in one or more pieces of question information previously received by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information. The control unit 104 of the server 10 can acquire any parent generation question information and question sentence by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information.
The control unit 104 of the server 10 can obtain one or more answer sentences included in one or more pieces of answer information by searching the items in the answer table 1016 based on one or more question IDs included in one or more pieces of question information previously received. The question information and the answer information can be determined in correspondence (order relationship, which answer sentence corresponds to which question sentence) by the question ID. The order of the one or more answer sentences may also be the order of the date and time included in the answer information. The control unit 104 of the server 10 combines one or more question sentences and one or more answer sentences in order to generate a character string.
The one or more answer sentences to be combined may be answer sentences obtained by dialogue processing, or may include answer sentences obtained by search processing and suggestion processing, which will be described later.
Furthermore, it is not necessary to combine all of the one or more question sentences and one or more answer sentences. It is also possible to combine a predetermined number of question sentences and answer sentences in order of most recent, with the number of characters that the generation AI 50 can accept as the upper limit.

In this way, in the information processing system disclosed herein, the user can check the content of the answer to the question they entered and then enter a further question. This allows the user to take into account the answer from the information processing system and gradually enter a question in an interactive format to obtain the information they want.

In step S303, the AI control unit 1042 of the server 10 receives and accepts the answer sentence (not an answer sentence to the question sentence but a response sentence for digging deeper into the question sentence) output from the large-scale language model. The response sentence output from the generation AI 50 is a sentence that does not include the answer content to the question sentence accepted by the user in step S301, and is a sentence (response sentence) that includes a suggestion for digging deeper into the question sentence accepted from the user in more detail.
The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S301, the prompt generated in step S302, and the answer sentence in the question ID, input sentence, and answer sentence fields of the new record in the answer table 1016, respectively.

In step S304, if the control unit 104 of the server 10 does not obtain a judgment result that the question sentence is appropriate in the judgment step, it executes a second answering step in which it presents to the user a response sentence for digging deeper into the question sentence, rather than an answer sentence to the question sentence, which is output by inputting input data based on the question sentence into a large-scale language model.
Specifically, the control unit 104 of the server 10 transmits the answer sentence received in step S303 to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the received answer sentence on the display 2081 of the user terminal 20. The control unit 104 of the server 10 may be configured to transmit the answer sentence received in step S303 to the user terminal 20 when a search keyword cannot be acquired in step S103 of the search process.
For example, when the control unit 204 of the user terminal 20 receives the question sentence D121 in step S301, the control unit 204 displays the answer sentence D132 on the interactive screen D1. When the control unit 204 of the user terminal 20 receives the question sentence D122 in step S301, the control unit 204 displays the answer sentence D133 on the interactive screen D1.
For example, when the control unit 204 of the user terminal 20 receives the question sentence D221 in step S301, the control unit 204 displays the answer sentence D232 on the dialogue screen D2. When the control unit 204 of the user terminal 20 receives the question sentence D222 in step S301, the control unit 204 displays the answer sentence D233 on the dialogue screen D2.

<Proposal Processing>
The search process is a process of presenting to the user output sentences from the question and answering system 60 based on question candidates corresponding to the question sentence received from the user.

<Outline of proposal process>
The proposal process is a process in which a question sentence is input from a user, the input question sentence is input to the generation AI 50, question candidates are obtained from the generation AI 50, the question candidates are presented to the user for selection, one or more question candidates selected by the user are input to the question answering system 60, the output contents output from the question answering system 60 are obtained, and the output contents are presented as answer sentences.

<Details of the proposal process>
The details of the proposal process will be described below.

In step S501, the AI control unit 1042 of the server 10 executes a question receiving step of receiving a question from a user.
The question reception step is the same as step S301 of the dialogue processing, and therefore the process is omitted. Note that in the present disclosure, a proposal process is executed in response to the question received in step S301 of the dialogue processing. In other words, step S501 of the search process is common to step S301 of the dialogue processing.
It should be noted that the dialogue process does not necessarily have to be executed, and the configuration may be such that a proposal process is executed.

In step S502, the AI control unit 1042 of the server 10 transmits the question received in the question receiving step to the generation AI 50.
Specifically, the AI control unit 50 of the server 10 generates a prompt by inserting the question received in step S501 into the [Question] section of the prompt template as an input sentence (prompt) for the generation AI 50, which is a large-scale language model, and sends the prompt to the generation AI 50.
In the search process, the prompt includes an instruction to restrict the output data format of the generation AI 50 to output one or more search keyword candidates instead of an answer sentence. Specifically, in OpenAI's GPT, an instruction to extract one or more search keyword candidates to be input to the question answering system 60 is included using a mechanism called FunctionCalling.

In step S502, the AI control unit 1042 of the server 10 inputs the first question, the answer to the second question, and the second question into the large-scale language model.
Specifically, when the AI control unit 50 of the server 10 receives an additional question D122 in the question receiving step, it generates a prompt by inserting a string combining the question D121, answer D132, and question D122 into the [Question] section of the prompt template, and sends the prompt to the generation AI 50.
In other words, if one or more question sentences have been received in the past in step S501, and one or more answer sentences have been obtained for those one or more question sentences, a string formed by combining those one or more question sentences and one or more answer sentences with the newly received question sentence is sent to generation AI 50 as a question sentence.
The control unit 104 of the server 10 can acquire one or more question sentences included in one or more pieces of question information previously received by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information. The control unit 104 of the server 10 can acquire any parent generation question information and question sentence by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information.
The control unit 104 of the server 10 can obtain one or more answer sentences included in one or more pieces of answer information by searching the items in the answer table 1016 based on one or more question IDs included in one or more pieces of question information previously received. The question information and the answer information can be determined in correspondence (order relationship, which answer sentence corresponds to which question sentence) by the question ID. The order of the one or more answer sentences may also be the order of the date and time included in the answer information. The control unit 104 of the server 10 combines one or more question sentences and one or more answer sentences in order to generate a character string.
The one or more answer sentences to be combined may be answer sentences obtained by the suggestion process, or may include answer sentences obtained by the search process or the dialogue process.
Furthermore, it is not necessary to combine all of the one or more question sentences and one or more answer sentences. It is also possible to combine a predetermined number of question sentences and answer sentences in order of most recent, with the number of characters that the generation AI 50 can accept as the upper limit.

In step S503, a step of acquiring one or more candidate questions that are output by inputting the question sentence and the additional question sentence as input data into a large-scale language model is executed.
Specifically, the AI control unit 1042 of the server 10 receives and accepts search keyword candidates output from the large-scale language model.

Note that the large-scale language model may not output search keyword candidates in step S103. For example, if the user's question included in the prompt does not include sufficient search keyword candidates, the generation AI 50 does not output search keyword candidates. In such a case, steps S504 and onward are skipped and the suggestion process is terminated.

In step S504, the AI control unit 1042 of the server 10 executes a candidate presentation step of presenting to the user one or more question candidates output by inputting the question sentence received in the question reception step into a large-scale language model as input data.
Specifically, the AI control unit 1042 of the server 10 receives and accepts one or more search keyword candidates output from the large-scale language model.
The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S501, the prompt generated in step S502, and one or more search keyword candidates in the question ID, input sentence, and question candidate fields of the new record in the candidate table 1015, respectively.
The question and answer unit 1043 of the server 10 transmits one or more search keyword candidates to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the received one or more search keyword candidates on the display 2081 of the user terminal 20.
For example, when the control unit 204 of the user terminal 20 receives the question D224 in step S501, the control unit 204 displays and presents the question D224 in the question candidates D251 and D252 of the dialogue screen D2.

For example, the candidate presentation step includes a step of presenting a first question candidate based on the first question sentence. The candidate presentation step includes a step of presenting a second question candidate based on the first question sentence, the second question sentence, and the first answer sentence.

In step S504, a candidate presenting step is executed to present one or more question candidates to the user when one or more search keywords cannot be acquired in the keyword acquiring step.
Specifically, in a configuration in which a proposal process is executed after a search process, the proposal process may be executed if a search keyword cannot be obtained in step S103 of the search process, and the proposal process may be controlled not to be executed if a search keyword can be obtained in step S103 of the search process.
In other words, when an answer sentence is obtained from the question answering system 60 by the search process according to the present disclosure, there is no need to present question candidates to the user, so it is possible to control the system so that a suggestion process is executed when an answer sentence is not obtained from the question answering system 60 by the search process, and that the suggestion process is not executed when an answer sentence is obtained from the question answering system 60 by the search process.
This makes it possible to configure the system so that the suggestion process is not executed when the answer sentence from question and answering system 60 is presented to the user. In other words, when an answer to the search process is obtained, the dialogue with the user may be terminated.

In step S505, the AI control unit 1042 of the server 10 executes a candidate selection step of accepting, from the user, a selection of a predetermined question candidate, which is at least one of the one or more question candidates presented in the candidate presentation step.
Specifically, the user operates the input device 206 of the user terminal 20 to select at least one of one or more question candidates D251, D252 presented on the interactive screen D2. Note that a configuration may be adopted in which multiple question candidates D251, D252 can be selected. The control unit 204 of the user terminal 20 transmits information related to the selected predetermined one or more question candidates D251, D252 to the server 10.

For example, the candidate selection step includes a step of accepting a selection of a predetermined first question candidate, which is at least one of the first question candidates. The candidate selection step includes a step of accepting a selection of a predetermined second question candidate, which is at least one of the second question candidates.

In step S506, the question answering unit 1043 of the server 10 executes a question input step of inputting the predetermined question candidate selected in the candidate selection step to a question answering system capable of outputting an answer sentence in response to an input of a question sentence.
Specifically, the question and answering section 1043 of the server 10 transmits to the question and answering system 60 the predetermined one or more candidate questions D251, D252 selected in step S505.
For example, the question input step includes a step of inputting a predetermined first question candidate to the question and answering system, and a step of inputting a predetermined second question candidate to the question and answering system.

In step S507, the question and answering unit 1043 of the server 10 executes an answer obtaining step of obtaining a predetermined answer sentence to be output in response to input of a predetermined question candidate to the question and answering system in the question input step.
Specifically, the question answering unit 1043 of the server 10 receives and accepts the output contents output from the question answering system 60. For example, when the question answering system 60 receives an input character string related to a candidate question, it searches for at least one of the questions or answers included in the question answering information stored in the question answering system 60 by full-text search or the like based on the character string, and outputs the question answering information related to the search result in association with the degree of relevance (score) with the input character string.
The question answering unit 1043 of the server 10 receives and acquires the answer sentence included in the question answer information.

For example, the answer acquisition step includes a step of acquiring a predetermined first answer sentence in response to an output of a predetermined first question candidate to the question answering system. The answer acquisition step includes a step of acquiring a predetermined second answer sentence in response to an output of a predetermined second question candidate to the question answering system.

In step S507, the question answering unit 1043 of the server 10 receives and accepts the answer sentence output from the question answering system 60. The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S101, the prompt generated in step S102, and the answer sentence in the question ID, input sentence, and answer sentence fields of the new record in the answer table 1016, respectively.

In step S<b>508 , the control unit 104 of the server 10 presents the answer sentence output from the question and answer system 60 .
Specifically, the control unit 104 of the server 10 transmits the answer sentence received in step S507 to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the received answer sentence on the display 2081 of the user terminal 20.

<Basic computer hardware configuration>
14 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 901, a main storage device 902, an auxiliary storage device 903, and a communication IF 991 (interface). These are electrically connected to each other by a communication bus 921.

The processor 901 is hardware for executing a set of instructions written in a program. The processor 901 is composed of an arithmetic unit, registers, peripheral circuits, etc.

The main memory device 902 is used to temporarily store programs and data processed by the programs. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

The auxiliary storage device 903 is a storage device for saving data and programs. For example, it is a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc.

The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using a wired or wireless communication standard.
The network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, the network includes 3G, 4G, 5G mobile communication systems, LTE (Long Term Evolution), wireless networks that can connect to the Internet through a predetermined access point (e.g., Wi-Fi (registered trademark)), etc. In the case of wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), Bluetooth (registered trademark), etc. In the case of wired connection, the network also includes a network directly connected by a USB (Universal Serial Bus) cable, etc.

The computer 90 can be realized virtually by distributing all or part of each hardware configuration across multiple computers 90 and connecting them together via a network. In this way, the concept of computer 90 includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

<Basic Functional Configuration of Computer 90>
A description will now be given of the functional configuration of a computer realized by the basic hardware configuration (FIG. 14) of a computer 90. The computer includes at least the functional units of a control unit, a storage unit, and a communication unit.

The functional units of the computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 connected to each other via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

The control unit is realized by the processor 901 reading various programs stored in the auxiliary storage device 903, expanding them in the main storage device 902, and executing processing according to the programs. The control unit can realize functional units that perform various information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. Furthermore, the processor 901 can secure a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 in accordance with a program. Furthermore, the control unit can cause the processor 901 to execute processes for adding, updating, and deleting data stored in the storage unit in accordance with various programs.

The term database refers to a relational database, which is used to manage sets of data called masters and tables in a tabular format structurally defined by rows and columns, by associating them with each other. In a database, a table is called a table or master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be set and associated.
Usually, a column that serves as a primary key for uniquely identifying a record is set in each table and each master, but setting a primary key to a column is not essential. The control unit can cause the processor 901 to add, delete, or update records in a specific table or master stored in the storage unit according to various programs.
Furthermore, by storing data, various programs, and various databases in the storage unit, it can be considered that the information processing device and information processing system according to the present disclosure have been manufactured.

In addition, the databases and masters in this disclosure may include any data structure (such as a list, dictionary, associative array, or object) in which information is structurally defined. Data structures also include data that can be considered as data structures by combining data with functions, classes, methods, etc. written in any programming language.

The communication unit is realized by the communication IF 991. The communication unit realizes the function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 901 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 90.

<Additional Notes>
The matters described in the above embodiments will be supplemented below.

(Appendix 1)
A program to be executed by a computer having a processor and a memory unit, the program executing: a question receiving step (S101, S301, S501) in which the processor receives a question from a user; a judgment step (S103) in which the question received in the question receiving step is input as input data into a large-scale language model, thereby obtaining a judgment result as to whether the question is appropriate as input data for a question answering system capable of outputting an answer sentence in response to the input of the question sentence; a first answering step (S106) in which, if the judgment result that the question sentence is appropriate is obtained in the judgment step, an answer sentence to the question sentence that is output by inputting input data based on the question sentence into the question answering system is presented to the user; and a second answering step (S304) in which, if the judgment result that the question sentence is appropriate is not obtained in the judgment step, a reply sentence for digging deeper into the question sentence, rather than an answer sentence to the question sentence, is output by inputting the input data based on the question sentence into the large-scale language model.
This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to delve deeper into the question if the question is not appropriate.

(Appendix 2)
2. The program of claim 1, wherein the question answering system is a system different from the large-scale language model.
This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to delve deeper into the question if the question is not appropriate.

(Appendix 3)
The question answering system is a system that searches for answer sentences based on keywords and outputs a predetermined answer sentence related to the search result, and the judgment step (S103) includes a step of acquiring one or more keywords contained in the question sentence by inputting the question sentence accepted in the question receiving step into a large-scale language model as input data, a step of acquiring a judgment result that the question sentence is suitable as input data if the one or more keywords are acquired, and a step of acquiring a judgment result that the question sentence is not suitable as input data if the one or more keywords are not acquired, and the first answering step (S106) is a step of presenting to a user an answer sentence to the question sentence output by inputting the one or more keywords acquired in the judgment step into the question answering system as input data.
As a result, if a question received from a user contains keywords that are appropriate as input data for the question answering system, the user can be presented with an answer from the question answering system based on those keywords, and if the keyword is not included, the user can be presented with a response that delves deeper into the question.

(Appendix 4)
A program as described in Appendix 3, wherein the question receiving step (S101) is a step in which an additional question text can be received if one or more search keywords cannot be obtained in the judgment step, and the judgment step (S103) includes a step of obtaining one or more keywords that are output by inputting the question text and the additional question text as input data into a large-scale language model.
This makes it possible to accept an additional question if a search keyword cannot be obtained based on the question entered by the user, and to input a search keyword output based on the additional question into the question answering system.

(Appendix 5)
The program described in Appendix 1, wherein a processor executes a candidate presentation step (S504) of presenting to a user one or more question candidates that are output by inputting the question sentence received in the question receiving step into a large-scale language model as input data, a candidate selection step (S505) of accepting from the user a selection of a predetermined question candidate that is at least one of the one or more question candidates presented in the candidate presentation step, and a third answering step (S508) of presenting to the user an answer sentence to the question sentence that is output by inputting the predetermined question candidate whose selection was accepted in the candidate selection step into a question answering system.
This makes it possible to input a question to the question answering system in order to output a more appropriate answer based on question candidates output from the large-scale language model, even if the question received from the user is insufficient.

(Appendix 6)
The program according to claim 5, wherein the candidate presenting step (S504) is a step of presenting one or more question candidates to the user when one or more keywords included in the question sentence cannot be obtained in the determining step.
This allows the system to present candidate questions to the user when a search keyword cannot be obtained based on the question sentence entered by the user. Based on the candidate questions, a question sentence for outputting a more suitable answer sentence can be input to the question answering system.

(Appendix 7)
The program described in Appendix 3, wherein the question receiving step (S101, S301, S501) includes a step of receiving a first question sentence, and an answering step (S106, S304, S508) of presenting a first answer sentence based on at least one of a first answering step and a second answering step based on the first question sentence, the question receiving step (S101) includes a step of accepting a second question sentence, the judgment step (S103) includes a step of acquiring second keywords output by inputting the first question sentence, the second question sentence, and the first answer sentence as input data into a large-scale language model, and the first answering step (S106) includes a step of presenting to a user an answer sentence to the question sentence output by inputting the second keywords as input data into the question answering system.
This allows the user to ask multiple questions and input search keywords for the content including the answers obtained in response to the questions to the question and answering system, and the second answer based on the search keywords can be obtained from the question and answering system and presented.

(Appendix 8)
the question receiving step (S101, S301, S501) includes a step of receiving a first question sentence, and an answering step (S106, S304, S508) of presenting a second answer sentence based on at least one of a first answering step, a second answering step, and a third answering step based on the first question sentence; the question receiving step (S501) includes a step of accepting a second question sentence, the candidate presenting step (S504) includes a step of presenting to a user second question candidates output by inputting the first question sentence, the second question sentence, and the second answer sentence as input data into a large-scale language model, the candidate selecting step (S505) includes a step of accepting a selection of a predetermined second question candidate which is at least one of the second question candidates, and the third answering step (S508) includes a step of presenting to the user an answer sentence to the question sentence output by inputting the predetermined second question candidate into the question answering system.
This allows the user to ask multiple questions, and a candidate question for the content including the answer sentence obtained in response to the question sentence can be obtained from the large-scale language model. A second answer sentence based on the candidate question can be obtained from the question answering system and presented.

(Appendix 9)
A method implemented on a computer having a processor and a memory, the processor performing all of the steps performed in the invention according to any one of claims 1 to 8.
This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to dig deeper into the question if the question is not appropriate.

(Appendix 10)
An information processing device comprising a control unit and a memory unit, wherein the control unit executes all of the steps executed in the invention according to any one of Supplementary Note 1 to Supplementary Note 8.
This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to delve deeper into the question if the question is not appropriate.

(Appendix 11)
A system comprising means for performing all the steps performed in any of the inventions according to any one of appendixes 1 to 8.
This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to delve deeper into the question if the question is not appropriate.

1 System, 10 Server, 101 Memory unit, 104 Control unit, 106 Input device, 108 Output device, 20 User terminal, 201 Memory unit, 204 Control unit, 206 Input device, 208 Output device, 50 Generating AI, 501 Memory unit, 504 Control unit, 506 Input device, 508 Output device, 60 Question answering system, 601 Memory unit, 604 Control unit, 606 Input device, 608 Output device

Claims (11)

A program to be executed by a computer having a processor and a storage unit,
The processor,
a question receiving step of receiving a question from a user;
a determination step of inputting the question sentence received in the question receiving step into a large-scale language model as input data, thereby obtaining a determination result as to whether or not the question sentence is suitable as input data for a question answering system capable of outputting an answer sentence in response to an input of a question sentence;
a first answering step of presenting to the user an answer to the question, which is output by inputting input data based on the question into the question answering system, when a judgment result that the question is appropriate is obtained in the judging step;
a second answering step of generating a prompt into which the question sentence is inserted and inputting the prompt as input data into the large-scale language model when the question sentence is not judged to be appropriate in the judging step, and presenting to the user a response sentence for delving into the question sentence, rather than an answer sentence to the question sentence;
Run
the question receiving step is a step of receiving an additional question when a determination result is obtained in the determining step that the question is not appropriate as input data,
the determining step includes a step of generating a prompt into which the question sentence and the additional question sentence are inserted, and inputting the prompt as input data into the large-scale language model, thereby obtaining a determination result as to whether the question sentence and the additional question sentence are appropriate as input data for the question answering system;
program. The question answering system is a system different from the large-scale language model.
The program according to claim 1. the question and answering system is a system for searching for an answer sentence based on a keyword, and outputting a predetermined answer sentence related to the search result,
The determining step includes:
a step of inputting the question sentence received in the question receiving step as input data into the large-scale language model to acquire one or more keywords included in the question sentence;
acquiring a determination result that the question sentence is appropriate as input data when the one or more keywords are acquired;
obtaining a determination result that the question sentence is not suitable as input data when the one or more keywords cannot be obtained;
Including,
the first answering step is a step of presenting to the user an answer to the question sentence, the answer being output by inputting the one or more keywords acquired in the determining step as input data to the question answering system;
The program according to claim 1. the question receiving step is a step of receiving an additional question sentence when the one or more search keywords cannot be acquired in the determining step,
the determining step includes a step of acquiring one or more keywords output by inputting the question sentence and the additional question sentence as input data into the large-scale language model;
The program according to claim 3. The processor,
a candidate presenting step of presenting to the user one or more question candidates output by inputting the question sentence accepted in the question accepting step as input data into the large-scale language model;
a candidate selection step of accepting, from the user, a selection of a predetermined question candidate which is at least one of the one or more question candidates presented in the candidate presentation step;
a third answering step of presenting to the user an answer sentence to the question sentence that is output by inputting the predetermined question candidate selected in the candidate selecting step into the question and answering system;
Execute
The program according to claim 1. the candidate presentation step is a step of presenting the one or more question candidates to the user when the one or more keywords included in the question sentence cannot be acquired in the determination step;
The program according to claim 5. A program to be executed by a computer having a processor and a storage unit,
The processor,
a question receiving step of receiving a question from a user;
a determination step of inputting the question sentence received in the question receiving step into a large-scale language model as input data, thereby obtaining a determination result as to whether or not the question sentence is suitable as input data for a question answering system capable of outputting an answer sentence in response to an input of a question sentence;
a first answering step of presenting to the user an answer to the question, which is output by inputting input data based on the question into the question answering system, when a judgment result that the question is appropriate is obtained in the judging step;
a second answering step of presenting to the user, when a judgment result that the question sentence is appropriate is not obtained in the judgment step, a response sentence for digging deeper into the question sentence, which is output by inputting input data based on the question sentence into the large-scale language model, instead of an answer sentence to the question sentence;
Run
the question and answering system is a system for searching for an answer sentence based on a keyword, and outputting a predetermined answer sentence related to the search result,
The determining step includes:
a step of inputting the question sentence received in the question receiving step as input data into the large-scale language model to acquire one or more keywords included in the question sentence;
acquiring a determination result that the question sentence is appropriate as input data when the one or more keywords are acquired;
acquiring a determination result that the question sentence is not suitable as input data when the one or more keywords cannot be acquired;
Including,
the first answering step is a step of presenting to the user an answer to the question, the answer being output by inputting the one or more keywords acquired in the determining step as input data to the question answering system;
The question receiving step includes a step of receiving a first question sentence,
an answering step of presenting a first answer sentence based on at least one of the first answering step and the second answering step based on the first question sentence;
The question receiving step includes a step of receiving a second question sentence,
the determining step includes a step of acquiring second keywords output by inputting the first question sentence, the second question sentence, and the first answer sentence as input data into the large-scale language model;
the first answering step includes a step of presenting to the user an answer sentence to the question sentence, the answer sentence being output by inputting the second keyword as input data to the question and answering system;
program. A program to be executed by a computer having a processor and a storage unit,
The processor,
a question receiving step of receiving a question from a user;
a determination step of inputting the question sentence received in the question receiving step into a large-scale language model as input data, thereby obtaining a determination result as to whether or not the question sentence is suitable as input data for a question answering system capable of outputting an answer sentence in response to an input of a question sentence;
a first answering step of presenting to the user an answer to the question, which is output by inputting input data based on the question into the question answering system, when a judgment result that the question is appropriate is obtained in the judging step;
a second answering step of presenting to the user, when a judgment result that the question sentence is appropriate is not obtained in the judgment step, a response sentence for digging deeper into the question sentence, which is output by inputting input data based on the question sentence into the large-scale language model, instead of an answer sentence to the question sentence;
a candidate presenting step of presenting to the user one or more question candidates output by inputting the question sentence accepted in the question accepting step as input data into the large-scale language model;
a candidate selection step of accepting, from the user, a selection of a predetermined question candidate which is at least one of the one or more question candidates presented in the candidate presentation step;
a third answering step of presenting to the user an answer sentence to the question sentence that is output by inputting the predetermined question candidate selected in the candidate selecting step into the question and answering system;
Run
The question receiving step includes a step of receiving a first question sentence,
an answering step of presenting a second answer sentence based on at least one of the first answering step, the second answering step, and the third answering step based on the first question sentence;
The question receiving step includes a step of receiving a second question sentence,
the candidate presenting step includes a step of presenting to the user second question candidates output by inputting the first question sentence, the second question sentence, and the second answer sentence as input data into the large-scale language model;
the candidate selection step includes a step of accepting a selection of a predetermined second question candidate, which is at least one of the second question candidates;
the third answering step includes a step of presenting to the user an answer sentence to the question sentence, the answer sentence being output by inputting the predetermined second question candidate into the question and answering system;
program. A method executed by a computer having a processor and a memory, the processor executing all of the steps executed in the invention according to any one of claims 1 to 8. An information processing device comprising a control unit and a storage unit, the control unit executing all steps executed in the invention according to any one of claims 1 to 8. A system having means for executing all steps performed in any one of claims 1 to 8.