JP7009911B2 - Answer output program, answer output method and information processing device - Google Patents

Description

The present invention relates to an answer output program, an answer output method, and an information processing apparatus.

In recent years, an increasing number of call centers have introduced the FAQ (freak Asked Question) system in order to standardize operator skills and shorten response lead times. In addition, the FAQ is open to the outside to reduce the number of inquiries to the call center.

As the prior art, for example, there is a method in which the integrated classifier is trained by using the discrimination result of each of the plurality of classifiers and the true class as the secondary learning data. In addition, the accuracy of answer candidate extraction of new learning model information to which additional elemental information candidates have been added is compared with the accuracy of answer candidate extraction of the original learning model information, and the accuracy of answer candidate extraction is improved by a predetermined value or more. If so, there is a technique for keeping the new learning model information in the DB. In addition, the first answer obtained by the first processing unit for the first question was obtained, and the first answer was obtained by referring to the answer to the first question and the evaluation of the answer by the second processing unit. There is a technique for acquiring an evaluation of an answer and presenting the evaluation of the first answer and the evaluation of the first answer together. In addition, the range for searching the behavior history information is determined using the target of the inquiry, and the possibility is determined for each candidate for answering the inquiry from the behavior history indicated by the behavior history information within the range using the narrowing model. However, there is a technology to output candidate answers according to this possibility.

Japanese Unexamined Patent Publication No. 2015-11686 Japanese Unexamined Patent Publication No. 2007-219955 Japanese Unexamined Patent Publication No. 2017-97561 Japanese Unexamined Patent Publication No. 2012-128525

However, in the prior art, it is difficult to optimize the answer candidates to be output for the input question. For example, simply sorting the FAQs in descending order of the number of access to the FAQs or in descending order of similarity with the entered keywords or sentences may not be the order expected by the user.

In one aspect, the present invention aims to optimize the answer candidates to be output for a question.

In one embodiment, an input of a question is accepted, a plurality of answer candidates corresponding to the received question are specified, and one or a plurality of first answer candidates that have received a selection operation among the answer candidates for the question . , Each of the plurality of answer candidates identified by performing machine learning based on the stored contents of the storage unit that stores one or a plurality of second answer candidates that did not accept the selection operation among the answer candidates for the question. , The first probability calculated when displayed for the accepted question and the second probability not selected when displayed for the accepted question. Based on the probability of Multiple answer candidates are determined as one or more answer candidates for the received question, the determined one or more answer candidates are output , and log information indicating the answer candidate for which the selection operation for the question is accepted is output. Based on the accumulated log information, the probability that any of the upper predetermined number of answer candidates displayed for the question will be selected is calculated, and the calculated probability is based on the time-series change. , The process of adjusting the threshold is provided with a response output program that lowers the threshold by a predetermined value when it is determined that the probability is on a downward trend based on the calculated time-series change of the probability. To.

According to one aspect of the present invention, it is possible to optimize the answer candidates to be output to the question.

FIG. 1 is an explanatory diagram showing an embodiment of a response output method according to an embodiment. FIG. 2 is an explanatory diagram showing a system configuration example of the response output system 200. FIG. 3 is a block diagram showing a hardware configuration example of the information processing apparatus 101. FIG. 4 is an explanatory diagram showing an example of the stored contents of the FAQ master 220. FIG. 5 is an explanatory diagram showing an example of the stored contents of the access log DB 230. FIG. 6 is an explanatory diagram showing an example of the stored contents of the teacher data (Yoiko) DB 240. FIG. 7 is an explanatory diagram showing an example of the stored contents of the teacher data (wariko) DB 250. FIG. 8 is an explanatory diagram showing an example of the stored contents of the teacher data (maigo) DB 260. FIG. 9 is an explanatory diagram showing a screen example of the FAQ screen 900. FIG. 10 is a block diagram showing a functional configuration example of the information processing apparatus 101. FIG. 11 is an explanatory diagram (No. 1) showing a specific example of the access log. FIG. 12 is an explanatory diagram (No. 1) showing a specific example of teacher data. FIG. 13 is an explanatory diagram (No. 2) showing a specific example of the access log. FIG. 14 is an explanatory diagram (No. 2) showing a specific example of teacher data. FIG. 15 is an explanatory diagram showing an example of the stored contents of the analysis DB 1500. FIG. 16 is an explanatory diagram showing an example of determining FAQ. FIG. 17 is an explanatory diagram (No. 3) showing a specific example of the access log. FIG. 18 is an explanatory diagram showing an example of time-series changes in the hit rate on the first page. FIG. 19 is an explanatory diagram showing an embodiment of a method for estimating a morpheme. FIG. 20 is a flowchart (No. 1) showing an example of the teacher data generation processing procedure of the information processing apparatus 101. FIG. 21 is a flowchart (No. 2) showing an example of the teacher data generation processing procedure of the information processing apparatus 101. FIG. 22 is a flowchart (No. 1) showing an example of the response output processing procedure of the information processing apparatus 101. FIG. 23 is a flowchart (No. 2) showing an example of the response output processing procedure of the information processing apparatus 101.

Hereinafter, embodiments of the response output program, the response output method, and the information processing apparatus according to the present invention will be described in detail with reference to the drawings.

(Embodiment)
FIG. 1 is an explanatory diagram showing an embodiment of a response output method according to an embodiment. In FIG. 1, the information processing apparatus 101 is a computer that outputs answer candidates corresponding to a question. The question is to ask how to solve some problem. The question is, for example, a question about a product or service. The question may be represented by a word or a combination of words, or by one or more sentences.

In addition, the answer candidate is information for solving the problem. That is, the answer candidate corresponding to the question is the answer candidate indicating the solution method of the questioned problem. The answer candidate is, for example, FAQ. FAQ is an abbreviation for "Frequently Asked Questions" and is a collection of questions and answers to expected questions in advance.

Here, in call centers and support centers, FAQ systems are often introduced in order to level the skills of operators and shorten the response lead time. Proper operation of the FAQ system leads to improvements in the primary response rate and customer satisfaction, so the FAQ system is in an important position in call centers and the like.

In addition, some companies publish their FAQs on the Internet. By disclosing the FAQ site outside the company, customers can browse the FAQ themselves, which can be expected to improve customer convenience and reduce inquiries to call centers and the like.

On the other hand, since the contents of the FAQ tend to become obsolete, it is desirable to review it regularly, but the operational load required for reviewing the FAQ is heavy. For example, the number of FAQs registered in the FAQ master may be several thousand to tens of thousands, and it is extremely burdensome to manually review the contents of those FAQs in a short period of time.

Therefore, it is important to carefully select and present a more appropriate FAQ from the current FAQs for questions. However, simply sorting the FAQs in descending order of the number of access to the FAQs or in descending order of similarity to the entered keywords or sentences does not meet the user's expectations, that is, the user wants to know. FAQs are often not displayed at the top.

Further, in a service such as a chatbot, the number of search results (FAQs) displayed at one time for an input question is small, and it is required to narrow down and sort FAQs with higher search accuracy. For example, in the case of an FAQ system used by a call center operator or an FAQ site published outside the company, the number of search results displayed at one time is about 30. On the other hand, in the case of a chatbot, the number of search results displayed at one time is about five.

Therefore, in the present embodiment, an answer output method for optimizing the answer candidates to be output for the question will be described. Hereinafter, a processing example of the information processing apparatus 101 will be described.

(1) The information processing apparatus 101 accepts the input of a question. The question entered may be a word or a combination of words, or may be one or more sentences. The user who inputs the question is, for example, an operator such as a call center or a user who uses the FAQ site. In the example of FIG. 1, it is assumed that the input of the question "XXX" is accepted.

(2) The information processing apparatus 101 identifies a plurality of answer candidates corresponding to the received questions. The answer candidate is, for example, FAQ. The answer candidate corresponding to the question is, for example, a search result (FAQ) for the question, which is searched by using an existing search algorithm adopted in the FAQ system or the FAQ site.

Specifically, for example, the information processing apparatus 101 analyzes the received question into morphological elements and decomposes them into morphemes. Next, the information processing apparatus 101 searches the FAQ master (for example, the FAQ master 220 shown in FIG. 2 described later) for the FAQ corresponding to the decomposed morpheme according to a predetermined search condition. Then, the information processing apparatus 101 identifies the searched search result (FAQ) as a plurality of answer candidates corresponding to the received question.

In the example of FIG. 1, it is assumed that answer candidates 1 to 10 are specified as a plurality of answer candidates corresponding to the received question "XXX". The information processing apparatus 101 may acquire a plurality of answer candidates corresponding to the received questions from another computer.

(3) The information processing apparatus 101 refers to the storage unit 110 and determines one or a plurality of answer candidates for the received question from the specified plurality of answer candidates. Specifically, for example, the information processing apparatus 101 performs machine learning based on the stored contents of the storage unit 110, and determines one or a plurality of answer candidates for the question from the plurality of answer candidates.

Here, machine learning means that a computer learns from data to derive patterns and trends, and enables future prediction and decision making. The storage unit 110 stores one or a plurality of first answer candidates that have accepted the selection operation for the question, and one or a plurality of second answer candidates that have not accepted the selection operation for the question. That is, the storage unit 110 stores the teacher data used for machine learning.

More specifically, the storage unit 110 stores teacher data (yoiko) in which the first answer candidate that has received the selection operation for the question is associated with the question. In addition, the storage unit 110 stores teacher data (wariko) in which the second answer candidate for which the selection operation for the question is not accepted is associated with the question.

The first answer candidate may be, for example, an answer candidate that has accepted a selection operation from among the top N answer candidates for a question. Further, the second answer candidate may be, for example, an answer candidate that does not accept the selection operation among the top N answer candidates for the question.

The predetermined number N can be arbitrarily set. The predetermined number N is set to, for example, the upper limit number of answer candidates displayed on the first page of the screen for displaying the answer candidates for the question. In the FAQ system and FAQ site, the predetermined number N is set to a value of about 30. In the chatbot, the predetermined number N is set to a value of about 5.

That is, the first answer candidate is an answer candidate displayed on the first page of the search result for the question from the user and inquired (selected) by the user. On the other hand, the second answer candidate is an answer candidate that is displayed on the first page of the search result for the question from the user but is not inquired (selected) by the user.

By machine learning the first answer candidate using the teacher data associated with the question, the characteristics of the answer candidate that are likely to be selected when displayed on the first page are derived. Further, by machine learning the second answer candidate using the teacher data associated with the question, the characteristics of the answer candidate that are likely not to be selected even if they are displayed on the first page are derived.

In the example of FIG. 1, among the answer candidates 1 to 10 corresponding to the question "XXX", the answer candidates 1 to 5 are identified as the answer candidates that are likely to be selected when displayed on the first page. do. Further, it is assumed that the answer candidates 2 and 5 are specified as answer candidates that are unlikely to be selected even if they are displayed on the first page. In this case, the information processing apparatus 101 determines, for example, the answer candidates 1, 3 and 4 excluding the answer candidates 2 and 5 from the answer candidates 1 to 5 as the answer candidates for the question "XXX".

(4) The information processing apparatus 101 outputs one or a plurality of determined answer candidates. In the example of FIG. 1, the information processing apparatus 101 outputs the determined answer candidates 1, 3 and 4 as answer candidates for the received question "XXX". At this time, the information processing apparatus 101 may sort and output, for example, the answer candidates 1, 3 and 4 in the order in which they are likely to be selected when they are displayed on the first page.

In this way, according to the information processing apparatus 101, based on the operation history of the past user, there is a high possibility that the question will be selected (inquired) when it is displayed, and the question will be displayed. It is possible to identify an answer candidate that is strongly related to the question and an answer candidate that is weakly related to the question from two different viewpoints that the possibility of not being selected (inquired) at the time is high. For example, if there is a high possibility that a question will be selected (inquired) when it is displayed, it can be said that it is a candidate for an answer that is highly relevant to the question and is likely to lead to problem solving. On the other hand, if there is a high possibility that the question will not be selected (inquired) when it is displayed, it can be said that the answer candidate has a weak relevance to the question and is unlikely to lead to problem solving. As a result, it is possible to accurately narrow down the answer candidates that are inquired when displayed for the question and lead to the solution of the problem, and to optimize the answer candidates to be output to the question.

(System configuration example of answer output system 200)
Next, a system configuration example of the response output system 200 including the information processing apparatus 101 shown in FIG. 1 will be described. The answer output system 200 may be applied to, for example, a FAQ system such as a call center, or may be applied to a FAQ site published on the Internet.

FIG. 2 is an explanatory diagram showing a system configuration example of the response output system 200. In FIG. 2, the response output system 200 includes an information processing device 101 and a plurality of terminals 201 (three in the example of FIG. 2). In the answer output system 200, the information processing apparatus 101 and the plurality of terminals 201 are connected via a wired or wireless network 210. The network 210 is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or the like.

Here, the information processing apparatus 101 outputs answer candidates corresponding to the question. In the following explanation, "FAQ" may be taken as an example for explanation as an answer candidate. Further, the information processing apparatus 101 has an FAQ master 220, an access log DB (Data) 230, a teacher data (Yoiko) DB 240, a teacher data (wariko) DB 250, and a teacher data (Maigo) DB 260. The information processing device 101 is, for example, a server.

The stored contents of various DBs 220, 230, 240, 250, 260 and the like will be described later with reference to FIGS. 4 to 8.

The terminal 201 is a computer used by the user of the answer output system 200. The user of the answer output system 200 is, for example, a call center operator or a FAQ site user. The terminal 201 is, for example, a PC (Personal Computer), a smartphone, a tablet-type PC, or the like.

(Hardware configuration example of information processing device 101)
FIG. 3 is a block diagram showing a hardware configuration example of the information processing apparatus 101. In FIG. 3, the information processing apparatus 101 includes a CPU (Central Processing Unit) 301, a memory 302, an I / F (Interface) 303, a disk drive 304, and a disk 305. Further, each component is connected by a bus 300.

Here, the CPU 301 controls the entire information processing apparatus 101. The memory 302 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash ROM, and the like. Specifically, for example, the flash ROM stores the OS (Operating System) program, the ROM stores the application program, and the RAM is used as the work area of the CPU 301. The program stored in the memory 302 is loaded into the CPU 301 to cause the CPU 301 to execute the coded process.

The I / F 303 is connected to the network 210 through a communication line, and is connected to an external computer (for example, the terminal 201 shown in FIG. 2) via the network 210. The I / F 303 controls the interface between the network 210 and the inside of the device, and controls the input / output of data from an external computer. For the I / F 303, for example, a modem, a LAN adapter, or the like can be adopted.

The disk drive 304 controls data read / write to the disk 305 according to the control of the CPU 301. The disk 305 stores the data written under the control of the disk drive 304. Examples of the disk 305 include a magnetic disk and an optical disk.

In addition to the above-mentioned components, the information processing device 101 may include, for example, an SSD (Solid State Drive), an input device, a display, and the like. Further, the terminal 201 shown in FIG. 2 can also be realized by the same hardware configuration as the information processing apparatus 101. However, the terminal 201 has an input device, a display, and the like in addition to the above-mentioned components.

(Memory contents of various DB 220, 230, 240, 250, 260, etc.)
Next, the stored contents of various DBs 220, 230, 240, 250, 260 and the like included in the information processing apparatus 101 will be described with reference to FIGS. 4 to 8.

FIG. 4 is an explanatory diagram showing an example of the stored contents of the FAQ master 220. In FIG. 4, the FAQ master 220 has fields for FAQ number, question, and answer, and stores FAQ (for example, FAQ400-1,400-2) as a record by setting information in each field.

Here, the FAQ number is an identifier that uniquely identifies the FAQ. "#" In FAQ400- # corresponds to the FAQ number. The question is a question that is expected in advance. The answer is an answer to a question that is expected in advance. For example, FAQ400-1 asks the question "Please tell me how to return the PC to the initial state." And the answer "If a problem occurs and the PC does not operate normally, the recovery method is as follows. ... ", is a summary.

FIG. 5 is an explanatory diagram showing an example of the stored contents of the access log DB 230. In FIG. 5, the access log DB 230 has fields for session number, date and time, type, search word, inquiry FAQ, rank, and search list. By setting information in each field, access logs, for example, access logs 500-1 to 500-3 are stored as records.

Here, the session number is an identifier that uniquely identifies the session between the information processing apparatus 101 and the terminal 201. A session is a logical connection relationship set for exchanging information in communication between two points (devices). The date and time is the date and time when the operation was performed on the terminal 201.

The type is the type of operation performed at the terminal 201. Types include, for example, search, query, and the like. The type "search" represents an operation to search for the FAQ corresponding to the search word. The type "query" represents an operation of selecting an FAQ for a search word, that is, an operation of querying the contents of the FAQ.

The search word corresponds to a question input in the terminal 201. The search word may be a word or a combination of a plurality of words, or may be one or a plurality of sentences. The inquiry FAQ is a FAQ that has received a selection operation for a search word, that is, the FAQ number of the inquired FAQ.

The ranking is a ranking indicating the order in which the FAQ is displayed from the top when the FAQ is displayed for the search word. The search list is a list of FAQ numbers of FAQs searched from the FAQ master 220 (see, for example, FIG. 4) based on the search word.

FIG. 6 is an explanatory diagram showing an example of the stored contents of the teacher data (Yoiko) DB 240. In FIG. 6, the teacher data (Yoiko) DB 240 has fields for date and time, inquiry FAQ, and search word after morphological analysis. By setting information in each field, teacher data (Yoiko), for example, teacher data (Yoiko) 600-1,600-2 is stored as a record.

Here, the date and time is the date and time when the inquiry FAQ is inquired. The inquiry FAQ is a FAQ that has received a selection operation when it is displayed on the first page of the search word, that is, the FAQ number of the inquired FAQ. The FAQ displayed on the first page is the top N FAQs among the FAQs for the search word. The search word after morphological analysis is one or more morphemes detected by morphological analysis of the search word when the inquiry FAQ is queried.

FIG. 7 is an explanatory diagram showing an example of the stored contents of the teacher data (wariko) DB 250. In FIG. 7, the teacher data (wariko) DB 250 has fields for date and time, non-query FAQ, and search word after morphological analysis. By setting information in each field, teacher data (wariko), for example, teacher data (wariko) 700-1 to 700-3 are stored as records.

Here, the date and time is the date and time when the non-inquiry FAQ was not inquired, for example, the date and time when the session between the information processing apparatus 101 and the terminal 201 was disconnected without inquiring the non-inquiry FAQ. The non-inquiry FAQ is a FAQ that did not accept the selection operation when it was displayed on the first page for the search word, that is, the FAQ number of the FAQ that was not inquired. The post-morphological analysis search word is one or more morphemes detected by morphological analysis of the search word when the non-query FAQ is not queried.

FIG. 8 is an explanatory diagram showing an example of the stored contents of the teacher data (maigo) DB 260. In FIG. 8, the teacher data (maigo) DB 260 has fields for date and time, alternative words, and search words. By setting information in each field, teacher data (maigo), for example, teacher data (maigo) 800-1 is stored as a record.

Here, the date and time is the date and time when no FAQ was searched for the search word from the FAQ master 220, for example, the date and time when the operation for searching the FAQ corresponding to the search word was performed. The alternative word is another search word that accepts input after the search word for which no FAQ is displayed, and any FAQ displayed on the first page for the other search word. Is the selected search word. The search word is a question for which no FAQ has been searched from the FAQ master 220. The search word may be represented by, for example, one or a plurality of morphemes detected by morphological analysis of the search word.

(Screen example of FAQ screen 900)
Next, a screen example of the FAQ screen 900 displayed on the terminal 201 will be described. In the following description, a case where a click operation using an input device (not shown) is performed as an operation for the user to select a box, a button, or the like on the operation screen displayed on the terminal 201 will be described as an example.

FIG. 9 is an explanatory diagram showing a screen example of the FAQ screen 900. In FIG. 9, the FAQ screen 900 is an example of an operation screen for searching the FAQ or inquiring about the FAQ. If you click the box 901 on the FAQ screen 900, you can enter a search word. In the example of FIG. 9, the search word "FJ10 initialization" is input.

Further, by clicking the button 902 on the FAQ screen 900, the FAQ corresponding to the search word input in the box 901 can be searched. Specifically, when the button 902 is clicked, the search word entered in the box 901 is transmitted from the terminal 201 to the information processing apparatus 101.

As a result, the search result for the search word is transmitted from the information processing apparatus 101 to the terminal 201, and the searched FAQs are listed and displayed in the display area 910. At this time, the top N FAQs are displayed on the first page. Other FAQs are displayed on the second and subsequent pages. N can be arbitrarily set and is set to a value of about 5 to 30. Here, N is "N = 5".

In the example of FIG. 9, as a result of searching the FAQ corresponding to the search word "FJ10 initialization", the top five FAQs 911 to 915 are displayed in the display area 910 as the search result of the first page. If you click the button 903 on the FAQ screen 900, you can display the search results on the second page.

Further, by clicking any FAQ displayed in the display area 910 on the FAQ screen 900, the contents of the FAQ can be inquired. For example, by clicking FAQ911, the contents (question and answer) of FAQ911 can be inquired. Further, when any of the FAQs displayed in the display area 910 is clicked, the selection result indicating that the FAQ is selected is transmitted from the terminal 201 to the information processing apparatus 101.

(Example of functional configuration of information processing device 101)
FIG. 10 is a block diagram showing a functional configuration example of the information processing apparatus 101. In FIG. 10, the information processing apparatus 101 includes a reception unit 1001, a search unit 1002, a determination unit 1003, an output control unit 1004, a generation unit 1005, a determination unit 1006, a specific unit 1007, and a storage unit 1010. ,including. The reception unit 1001 to the specific unit 1007 are functions that serve as control units. Specifically, for example, by causing the CPU 301 to execute a program stored in a storage device such as the memory 302 and the disk 305 shown in FIG. Alternatively, the function is realized by the I / F 303. The processing result of each functional unit is stored in a storage device such as a memory 302 or a disk 305.

Further, the storage unit 1010 is realized by a storage device such as a memory 302 and a disk 305. For example, the storage unit 1010 stores various DBs 220, 230, 240, 250, 260, 1900 (see FIG. 19 described later) and the like. The storage unit 1010 may be provided by another computer different from the information processing device 101. In this case, the information processing apparatus 101 can access another computer and refer to the stored contents of the storage unit 1010.

The reception unit 1001 accepts the input of a question. The question is a search word for asking how to solve some problem, and may be a word or a combination of a plurality of words, or may be a sentence or a sentence. The question is input, for example, on the FAQ screen 900 shown in FIG. Specifically, for example, the reception unit 1001 receives the input of the search word from the terminal 201 by receiving the search word input to the FAQ screen 900. In the following explanation, the question may be referred to as a "search word".

The search unit 1002 searches for the FAQ corresponding to the search word. Specifically, for example, the search unit 1002 analyzes the search word into morphemes and decomposes them into morphemes. Next, the search unit 1002 searches the FAQ master 220 shown in FIG. 4 for the FAQ corresponding to the decomposed morpheme according to a predetermined search condition.

More specifically, for example, the search unit 1002 may search the FAQ from the FAQ master 220 by setting an AND condition or an OR condition for the morpheme detected by morphological analysis of the search word. At this time, the search unit 1002 decides to search for FAQs with a large number of accesses and FAQs with a high degree of similarity, for example, in consideration of the number of past accesses (number of inquiries) of each FAQ and the degree of similarity with the search word. May be good.

Thereby, the FAQ corresponding to the search word can be specified. The search result is stored in the access log DB 230 shown in FIG. 5, for example, as an access log of type “search”. Specifically, the session number of the session between the terminal 201 that received the search word and the information processing apparatus 101 is set in the session number of the access log. The date and time when the search word is accepted is set in the date and time of the access log. The accepted search word is set as the search word of the access log. The access log inquiry FAQ and rank are "blank". The FAQ number of the searched FAQ is set in the search list of the access log. However, if no FAQ corresponding to the search word is searched, "-(null)" is set in the search list of the access log.

The determination unit 1003 performs machine learning based on the stored contents of the storage unit 1010 that stores the teacher data (yoko) and the teacher data (wariko), and one or one of the searched FAQs for the search word. Determine multiple FAQs. Here, the teacher data (Yoiko) is information in which the FAQ that accepts the selection operation for the question is associated with the question. The teacher data (wariko) is information in which the FAQ for which the selection operation for the question is not accepted is associated with the question.

Specifically, for example, the determination unit 1003 performs machine learning with a naive Bayes classifier with reference to the teacher data (yoko) DB 240 shown in FIG. 6, and first for each of the plurality of searched FAQs. Calculate the probability of. The first probability is the probability that the FAQ will be selected when the FAQ is displayed for the search word that has received the input.

As a result, each of the plurality of searched FAQs indicates an index value for determining the high possibility of being inquired when displayed on the first page, in other words, the strength of relevance to the search word. The index value can be obtained.

Further, the determination unit 1003 performs machine learning by the naive Bayes classifier with reference to the teacher data (wariko) DB 250 shown in FIG. 7, and calculates a second probability for each of the plurality of searched FAQs. do. The second probability is the probability that the FAQ is not selected when the FAQ is displayed for the search word for which the input is accepted.

As a result, each of the plurality of searched FAQs is an index value that determines the high possibility that the FAQ will not be inquired when displayed on the first page, in other words, an index that indicates the weakness of the relevance to the search word. You can get the value.

Then, the determination unit 1003 determines one or a plurality of FAQs for the search word from the searched plurality of FAQs based on the first probability and the second probability calculated for each of the searched plurality of FAQs. decide. Thereby, the FAQ displayed for the search word can be narrowed down in consideration of both the first probability and the second probability.

More specifically, for example, the determination unit 1003 sorts one or a plurality of FAQs obtained by excluding the FAQs having a second probability of α or more from the searched FAQs in descending order of the first probability. The FAQ is determined to be one or more FAQs for the search word. The threshold value α can be set arbitrarily. The threshold value α is set to a value of about 80, for example, when the unit is [%].

Further, for example, the determination unit 1003 selects the upper N FAQs having a high first probability, and among the selected FAQs, the FAQs having a second probability of the threshold value α or more are set as one or a plurality of FAQs for the search word. You may decide. That is, the determination unit 1003 narrows down the FAQs of the top N cases having a high first probability, and then excludes the FAQs having a second probability of the threshold value α or more.

The specific processing contents for calculating the first and second probabilities will be described later with reference to FIG. Further, an example of determining the FAQ for the search word will be described later with reference to FIG.

The output control unit 1004 outputs one or more FAQs for the search word. Specifically, for example, the output control unit 1004 outputs one or a plurality of determined FAQs for the search word. Information that can specify the order (display order) of each FAQ is given to each FAQ. More specifically, for example, the output control unit 1004 transmits one or a plurality of determined FAQs to the terminal 201 to display one or a plurality of received search words in the display area 910 of the FAQ screen 900. Controls to display multiple FAQs.

Further, the reception unit 1001 accepts an FAQ selection operation for the search word. The FAQ selection operation is an operation for inquiring about the contents of the FAQ, and is performed, for example, on the FAQ screen 900 shown in FIG. Specifically, for example, the reception unit 1001 receives a selection operation of the FAQ by receiving a selection result indicating the FAQ selected (clicked) on the FAQ screen 900 from the terminal 201.

When the FAQ selection operation is accepted, the access log of type "inquiry" is stored in the access log DB 230. Specifically, the session number of the session between the terminal 201 that received the search word and the information processing apparatus 101 is set in the session number of the access log. The date and time when the FAQ selection operation is accepted is set as the date and time of the access log. The accepted search word is set as the search word of the access log. In the access log inquiry FAQ, the FAQ number of the FAQ that accepted the selection operation is set. In the order of the access log, the order of the FAQ that accepted the selection operation is set. The access log search list is "blank". As a result, it is possible to accumulate an access log indicating the FAQ that accepted the selection operation when displayed for the search word.

The generation unit 1005 generates teacher data (yoiko). In addition, the generation unit 1005 generates teacher data (wariko). In addition, the generation unit 1005 generates teacher data (maigo). Here, the teacher data (maigo) is information in which another search word for which input is subsequently accepted when there is no FAQ corresponding to the search word is associated with the search word. However, the other search word is, for example, a search word in which any FAQ displayed on the first page is selected with respect to the other search word.

Specifically, for example, the generation unit 1005 acquires an access log having the same session number from the access log DB 230. Next, the generation unit 1005 sorts the acquired access logs in chronological order based on the date and time of the acquired access logs. Then, the generation unit 1005 determines whether or not there is an access log of type "inquiry" in the acquired access log.

Here, if there is an access log of type "inquiry", the generation unit 1005 determines that the inquiry FAQ is the FAQ displayed on the first page, that is, the higher rank, based on the order of the inquiry FAQ specified from the access log. Judge whether or not there are N FAQs. Here, in the case of the FAQ displayed on the first page, teacher data (Yoiko) representing the inquiry FAQ specified from the access log of the type "inquiry" and the search word after morphological analysis are generated in association with each other. The post-morphological analysis search word is one or more morphemes detected by morphological analysis of the search word identified from the access log of type "query".

Further, the generation unit 1005 refers to the search list of the access log of the type "search", and the FAQ that was not queried when displayed for the search word, that is, the FAQ that did not accept the selection operation for the search word. Determine if there is. Here, if there is an FAQ that has not been inquired, the generation unit 1005 generates teacher data (wariko) in which the FAQ that has not been inquired is used as a non-inquiry FAQ. The post-morphological analysis search word for teacher data (wariko) is obtained, for example, from the search word identified from the access log of type "query".

Further, the generation unit 1005 refers to the search list of the access log of the type "search" and determines whether or not there is a search word for which no FAQ has been searched. Here, if there is a search word for which no FAQ has been searched, the generation unit 1005 has another search word for which input is accepted following the search word based on the date and time of the acquired access log. Judge whether or not. However, the other search word is a search word in which any FAQ displayed on the first page is queried (selected) for the other search word. Here, if there is another search word, the generation unit 1005 generates teacher data (maigo) using the other search word as an alternative word. The search word after morphological analysis of the teacher data (maigo) is, for example, one or a plurality of morphemes detected by morphological analysis of another search word.

The generated teacher data (Yoiko) is stored in, for example, the teacher data (Yoiko) DB 240 shown in FIG. The generated teacher data (wariko) is stored in, for example, the teacher data (wariko) DB 250 shown in FIG. 7. The generated teacher data (maigo) is stored in, for example, the teacher data (maigo) DB 260 shown in FIG.

Examples of generation of teacher data (yoiko), teacher data (wariko), and teacher data (maigo) will be described later with reference to FIGS. 11 to 14.

If the teacher data (yoiko) and the teacher data (wariko) are not accumulated, the information processing apparatus 101 cannot narrow down the FAQ using the first and second probabilities described above. Therefore, the information processing apparatus 101 may output the searched FAQ to the search word for a certain period from the start of operation. At this time, the information processing apparatus 101 may sort and output, for example, a plurality of searched FAQs in descending order of the number of past accesses (number of inquiries). More specifically, for example, the output control unit 1004 controls to display the FAQ for the accepted search word in the display area 910 of the FAQ screen 900 by transmitting the searched FAQ to the terminal 201. conduct. As a result, teacher data (yoiko) and teacher data (wariko) can be accumulated.

Further, the determination unit 1003 may adjust the threshold value α based on the access log indicating the FAQ that has received the selection operation for the search word. Specifically, for example, the determination unit 1003 is one of the top N FAQs for the search word, that is, the FAQ displayed on the first page, based on the access log of the type "inquiry" in the access log DB 230. Calculate the probability that is selected.

In the following description, the probability that any of the FAQs displayed on the first page will be selected may be referred to as "first page hit rate".

The hit rate for the first page is calculated, for example, at predetermined intervals. The predetermined period can be arbitrarily set, and is set to, for example, a period of several weeks to several months. At this time, the determination unit 1003 calculates the first page hit rate based on, for example, the access log for the latest predetermined period. This makes it possible to calculate the first page hit rate that reflects the latest access tendency.

Then, the determination unit 1003 adjusts the threshold value α based on the calculated time-series change in the hit rate of the first page. More specifically, for example, the determination unit 1003 lowers the threshold value α when the hit rate of the first page tends to decrease. The amount of reduction of the threshold value α can be arbitrarily set. For example, when the unit is [%], the value is set to about 1.

An example of adjusting the threshold value α will be described later with reference to FIGS. 17 and 18.

The determination unit 1006 determines whether or not there is an FAQ corresponding to the search word. Specifically, for example, the determination unit 1006 has no FAQ corresponding to the search word when no FAQ corresponding to the search word is searched from the FAQ master 220, so-called 0 hits. judge.

Further, the determination unit 1006 may determine that there is no FAQ corresponding to the search word when the determination unit 1003 does not determine any FAQ for the search word. More specifically, for example, the determination unit 1006 narrows down the FAQ using the first and second probabilities described above, and when no FAQ remains, the FAQ corresponding to the search word. It may be determined that there is no FAQ.

When it is determined that there is no FAQ corresponding to the search word, the specific unit 1007 performs machine learning based on the stored contents of the storage unit 1010 that stores the teacher data (maigo), and is subsequently input to the search word. Identify other search words. In the following description, the search word determined by the determination unit 1006 to have no FAQ may be referred to as "0 hit search word".

Specifically, for example, the specific unit 1007 refers to the teacher data (maigo) DB260, performs machine learning with a naive Bayes classifier, and performs 0 hit search words for each of the other search words (alternative words). Calculate the third probability that is entered following. Then, the specifying unit 1007 specifies an alternative word to be input after the 0 hit search word based on the third probability calculated for each of the other search words.

More specifically, for example, the specific unit 1007 may specify another search word having the maximum third probability as an alternative word. Further, the specifying unit 1007 may specify another search word having a third probability of the threshold value β or more as an alternative word. The threshold value β can be set arbitrarily. Further, the specifying unit 1007 may specify another search word having a third probability of the threshold value β or more and having the maximum value as an alternative word.

The specific processing content for calculating the third probability will be described later with reference to FIG.

The output control unit 1004 outputs the FAQ corresponding to the specified alternative word. Specifically, for example, first, the search unit 1002 searches for the FAQ corresponding to the alternative word. When a plurality of alternative words are specified, the search unit 1002 may search the FAQ for each of the plurality of alternative words.

Next, the determination unit 1003 performs machine learning based on the stored contents of the storage unit 1010, and determines one or a plurality of FAQs for the alternative word from the plurality of searched FAQs. Then, the output control unit 1004 outputs one or a plurality of determined FAQs to the alternative word. The output control unit 1004 may output the FAQ corresponding to the searched alternative word.

As a result, even if there is no FAQ corresponding to the input search word, the search word, that is, the alternative word to be replaced with the 0 hit search word is estimated based on the operation history (access log) of the past user. FAQ can be output.

(Example of generating teacher data)
Next, an example of generating teacher data (wariko) and teacher data (maigo) will be described with reference to FIGS. 11 to 14. Here, it is assumed that the top five FAQs are the FAQs displayed on the first page (N = 5).

FIG. 11 is an explanatory diagram (No. 1) showing a specific example of the access log. FIG. 12 is an explanatory diagram (No. 1) showing a specific example of teacher data. In FIG. 11, the access logs 1101 to 1103 are a set of access logs having the same session number sorted in chronological order.

In this case, the generation unit 1005 determines whether or not there is an access log of type "inquiry" in the access logs 1101 to 1103. Here, access logs 1102 and 1103 are access logs of type "query". Therefore, the generation unit 1005 determines that there is an access log of type "inquiry".

Next, the generation unit 1005 determines whether or not the inquiry FAQ is the FAQ displayed on the first page, based on the order of the access log 1102 of the type "inquiry". Here, the order of the access log 1102 is "1", which is 5 or less. Therefore, the generation unit 1005 determines that the inquiry FAQ of the access log 1102 is the FAQ displayed on the first page.

In this case, as shown in FIG. 12, the generation unit 1005 associates the inquiry FAQ “011” specified from the access log 1102 with the search word “enlarge, print” after morphological analysis, and represents the teacher data (Yoiko) 1201. To generate.

Similarly, the generation unit 1005 determines whether or not the inquiry FAQ is the FAQ displayed on the first page, based on the order of the access log 1103 of the type "inquiry". Here, the order of the access log 1103 is "2", which is 5 or less. Therefore, the generation unit 1005 determines that the inquiry FAQ of the access log 1103 is the FAQ displayed on the first page.

In this case, as shown in FIG. 12, the generation unit 1005 associates the inquiry FAQ “012” specified from the access log 1103 with the search word “enlarge, print” after morphological analysis, and represents the teacher data (Yoiko) 1202. To generate.

Further, the generation unit 1005 referred to the search list of the access log 1101 of the type "search" and did not accept the FAQ that was not queried when displayed for the search word, that is, the selection operation for the search word. Determine if there is an FAQ. Here, the FAQ of the FAQ number "013" is not inquired.

Therefore, the generation unit 1005 determines that there is an FAQ that has not been inquired. In this case, as shown in FIG. 12, the generation unit 1005 generates teacher data (wariko) 1203 in which the FAQ of the FAQ number “013” that has not been inquired is used as the non-inquiry FAQ.

Further, the generation unit 1005 refers to the search list of the access log of the type "search" and determines whether or not there is a search word for which no FAQ has been searched. Here, there is no search word for which no FAQ was searched. In this case, the generation unit 1005 does not generate the teacher data (maigo).

FIG. 13 is an explanatory diagram (No. 2) showing a specific example of the access log. FIG. 14 is an explanatory diagram (No. 2) showing a specific example of teacher data. In FIG. 13, access logs 1301 to 1304 are a set of access logs having the same session number sorted in chronological order.

In this case, the generation unit 1005 determines whether or not there is an access log of type "inquiry" in the access logs 1301 to 1304. Here, access logs 1303 and 1304 are access logs of type "query". Therefore, the generation unit 1005 determines that there is an access log of type "inquiry".

Next, the generation unit 1005 determines whether or not the inquiry FAQ is the FAQ displayed on the first page, based on the order of the access log 1303 of the type "inquiry". Here, the order of the access log 1303 is "1", which is 5 or less. Therefore, the generation unit 1005 determines that the inquiry FAQ of the access log 1303 is the FAQ displayed on the first page.

In this case, as shown in FIG. 14, the generation unit 1005 associates the inquiry FAQ “011” specified from the access log 1303 with the search word “enlarge, print” after morphological analysis, and represents the teacher data (Yoiko) 1401. To generate.

Similarly, the generation unit 1005 determines whether or not the inquiry FAQ is the FAQ displayed on the first page, based on the order of the access log 1304 of the type "inquiry". Here, the order of the access log 1304 is "2", which is 5 or less. Therefore, the generation unit 1005 determines that the inquiry FAQ of the access log 1304 is the FAQ displayed on the first page.

In this case, as shown in FIG. 14, the generation unit 1005 associates the inquiry FAQ “012” specified from the access log 1304 with the search word “enlarge, print” after morphological analysis, and represents the teacher data (Yoiko) 1402. To generate.

Further, the generation unit 1005 refers to the search list of the access logs 1301 and 1302 of the type "search", and accepts the FAQ that was not queried when displayed for the search word, that is, the selection operation for the search word. Determine if there is a FAQ that was not available. Here, the FAQ of the FAQ number "013" is not inquired.

Therefore, the generation unit 1005 determines that there is an FAQ that has not been inquired. In this case, as shown in FIG. 14, the generation unit 1005 generates teacher data (wariko) 1403 in which the FAQ of the FAQ number “013” that has not been inquired is used as the non-inquiry FAQ.

Further, the generation unit 1005 refers to the search list of the access logs 1301 and 1302 of the type "search", and determines whether or not there is a search word for which no FAQ has been searched. Here, the search list of the access log 1301 is "-", and the search word "large print" is a search word in which no FAQ is searched.

In this case, the generation unit 1005 determines whether or not there is another search word for which input is accepted following the search word "large print". However, the other search word is a search word in which any FAQ displayed on the first page is queried (selected) for the other search word.

Here, the search word "enlarge, print" is another search word that accepts input following the search word "large print". Further, the search word "enlarge, print" is a search word in which the FAQ of the FAQ number "001,002" displayed on the first page is inquired (selected) for the other search words. Therefore, the generation unit 1005 determines that there is another search word.

In this case, as shown in FIG. 14, the generation unit 1005 uses the other search word “enlargement, print” as an alternative word, and the teacher data in which the alternative word “enlargement, print” and the search word “large print” are associated with each other. (Maigo) Generates 1404.

(Specific processing contents for calculating the first, second, and third probabilities)
Next, with reference to FIG. 15, specific processing contents for calculating the first, second, and third probabilities by performing machine learning with a naive Bayes classifier will be described.

FIG. 15 is an explanatory diagram showing an example of the stored contents of the analysis DB 1500. Here, a calculation example will be described by taking the first probability as an example. First, the determination unit 1003 creates the analysis DB 1500 with reference to the teacher data (Yoiko) DB 240.

In the analysis DB 1500, "FAQ number" is set on the vertical axis and "keyword" is set on the horizontal axis, and the number of inquiries for each FAQ is totaled for each keyword. However, the FAQ number is the FAQ number of each of the plurality of FAQs searched by the search unit 1002. The keyword is a morpheme included in the search word after the morphological analysis of the teacher data (Yoiko) in the teacher data (Yoiko) DB 240.

Here, the search word is "keyword 1, keyword 2". Further, the FAQ numbers of the plurality of FAQs corresponding to the search words "keyword 1, keyword 2" searched by the search unit 1002 are set to "1, 2, 3, 4, 5". Further, the morpheme included in the search word after the morphological analysis of the teacher data (Yoiko) in the teacher data (Yoiko) DB 240 is defined as "keyword 1, keyword 2, keyword 3, keyword 4, keyword 5, keyword 6".

Specifically, for example, the determination unit 1003 selects unselected teacher data (Yoiko) from the teacher data (Yoiko) DB 240. Here, it is assumed that the inquiry FAQ of the selected teacher data (Yoiko) is set to "1", and the search word after the morphological analysis includes the keyword 1 and the keyword 2. In this case, the determination unit 1003 increments the number of inquiries for the keyword 1 of the FAQ number "1". Further, the determination unit 1003 increments the number of inquiries for the keyword 2 of the FAQ number “1”. The determination unit 1003 repeats the above-mentioned series of processes until there is no unselected teacher data (Yoiko) selected from the teacher data (Yoiko) DB 240.

Next, the determination unit 1003 refers to the analysis DB 1500 and calculates the first probability for each of the plurality of FAQs searched. Specifically, for example, the determination unit 1003 can calculate the first probability for each of the plurality of FAQs searched by using the following equation (1), for example.

P (FAQ | Search word) = P (FAQ) P (Search word | FAQ) ... (1)

The posterior probability P (FAQ | search word) indicates the probability that the corresponding FAQ is selected (inquired) when the FAQ is searched by designating the search word. That is, the posterior probability P (FAQ | search word) corresponds to the first probability. For example, P (FAQ1 | Accident) indicates the probability that FAQ1 will be selected (inquired) when the search word "accident" is specified and the FAQ is searched.

If the search word is composed of a plurality of keywords, the posterior probabilities are indicated by the product of each probability. For example, for P (FAQ1 | accident, car), when the search words "accident" and "car" are specified and the FAQ is searched, the probability that FAQ1 is selected is P (FAQ1) P (accident | FAQ1) ×. P (FAQ1) Indicates that it is P (car | FAQ1).

The prior probability P (FAQ) indicates the probability (ratio) that the corresponding FAQ is selected. For example, when the total number of FAQs is 5000 and the search word "accident" is used for searching, P (FAQ1) is "1/5000". The prior probability P (FAQ) is replaced with the total number of FAQs included in the search word in order to improve the accuracy in estimation. Assuming that the total number of FAQs is 5000 and the total number of FAQs that have been hit by the search word "accident" is 100, P (FAQ1) is "1/100" when searching with the search word "accident".

Likelihood P (search word | FAQ) indicates the probability that a search word will be created when the FAQ is determined. For example, P (accident | FAQ1) indicates the probability that the search word "accident" will be created from FAQ1. If the number of accesses of the keyword that accessed FAQ1 is 1000 and the search word "accident" is 10, P (accident | FAQ1) is "10/1000".

The first probabilities (P (FAQ | keywords 1, 2)) of each FAQ 1 to 5 calculated based on the analysis DB 1500 shown in FIG. 15 are as follows. The part that becomes "0" is replaced with "1" (Laplace smoothing).

P (FAQ1 | Keywords 1 and 2)
= P (keyword 1 | FAQ1) x P (FAQ1) x P (keyword 2 | FAQ1) x P (FAQ1) = (1/21) (1/23) x (12/21) (12/96)
= 0.00015

P (FAQ2 | Keywords 1 and 2)
= P (keyword 1 | FAQ2) x P (FAQ2) x P (keyword 2 | FAQ2) x P (FAQ2) = (10/85) (10/23) x (1/85) (1/96)
= 0.00001

P (FAQ3 | Keywords 1 and 2)
= P (keyword 1 | FAQ3) x P (FAQ3) x P (keyword 2 | FAQ3) x P (FAQ3) = (2/7) (2/23) x (1/7) (1/96)
= 0.00018

P (FAQ4 | Keywords 1 and 2)
= P (keyword 1 | FAQ4) x P (FAQ4) x P (keyword 2 | FAQ4) x P (FAQ4) = (6/124) (6/23) x (77/124) (77/96)
= 0.00629

P (FAQ5 | Keywords 1 and 2)
= P (keyword 1 | FAQ5) x P (FAQ5) x P (keyword 2 | FAQ5) x P (FAQ5) = (4/44) (4/23) x (6/44) (6/96)
= 0.00013

However, on a computer, if the number of digits becomes too small, it may not be possible to calculate (overflow). Therefore, the determination unit 1003 converts each calculated value into a logarithm to calculate the first probability. The first probability converted into a logarithm is as follows. The closer it is to 0, the higher the probability.

P (FAQ1 | Keywords 1 and 2)
= Log (1/21) (1/23) + Log (12/21) (12/96)
=-3.83

P (FAQ2 | Keywords 1 and 2)
= Log (10/85) (10/23) + Log (1/85) (1/96)
=-5.20

P (FAQ3 | Keywords 1 and 2)
= Log (2/7) (2/23) + Log (1/7) (1/96)
=-4.43

P (FAQ4 | Keywords 1 and 2)
= Log (6/124) (6/23) + Log (77/124) (77/96)
= -2.2

P (FAQ5 | Keywords 1 and 2)
= Log (4/44) (4/23) + Log (6/44) (6/96)
=-3.87

Next, the determination unit 1003 uses the softmax function to convert the first probabilities of each of the FAQs 1 to 5 in logarithmic notation into the notation of 0 to 100 [%]. Specifically, for example, the determination unit 1003 acquires the maximum value C "-5.20" of the absolute value among the first probabilities of each FAQ 1 to 5 expressed in logarithm. Then, the determination unit 1003 calculates an index of the value of (maximum value C-1st probability) for each FAQ 1 to 5 as follows.

P (FAQ1) = exp (-3.83- (-5.20)) = 3.93
P (FAQ2) = exp (-5.20- (-5.20)) = 1.0
P (FAQ3) = exp (-4.43- (-5.20)) = 2.16
P (FAQ4) = exp (-2.20- (-5.20)) = 20.09
P (FAQ5) = exp (-3.87- (-5.20)) = 3.42

Next, the determination unit 1003 calculates the total value S of the exponents of the values of (maximum value C-first probability) of each FAQ 1 to 5. Here, the total value S of the exponent is "30.06".

Next, the determination unit 1003 calculates a value obtained by dividing the index of the value (maximum value C-1st probability) by the total value S of the exponents for each FAQ 1 to 5 as follows.

P (FAQ1) = 3.93 / 30.06 = 0.13
P (FAQ2) = 1.0 / 30.06 = 0.03
P (FAQ3) = 2.16 / 30.06 = 0.07
P (FAQ4) = 20.09 / 30.06 = 0.67
P (FAQ5) = 3.42 / 30.06 = 0.11

Next, the determination unit 1003 converts the first probability into the notation of 0 to 100 [%] by multiplying the calculated value for each FAQ 1 to 5 by 100 as follows.

P (FAQ1) = 13 [%]
P (FAQ2) = 3 [%]
P (FAQ3) = 7 [%]
P (FAQ4) = 67 [%]
P (FAQ5) = 11 [%]

In this way, for each of the FAQs 1 to 5, when each of the FAQs 1 to 5 is displayed for the search word "keyword 1 and keyword 2", the first probability that each of the FAQs 1 to 5 is selected (inquired) is determined. Can be calculated. For FAQs that have never been selected (inquired), the first probability is "0 [%]".

Further, although the first probability has been described here as an example, the second and third probabilities can be obtained in the same manner.

For example, in the case of the second probability, the determination unit 1003 creates the analysis DB 1500 with reference to the teacher data (wariko) DB 250. In this case, in the analysis DB 1500, "FAQ number" is set on the vertical axis and "keyword" is set on the horizontal axis, and the number of non-inquiries in which each FAQ is not inquired is totaled for each keyword. However, the FAQ number is the FAQ number of each of the plurality of FAQs searched by the search unit 1002. The keyword is a morpheme included in the search word after morphological analysis of the teacher data (wariko) in the teacher data (wariko) DB250.

Specifically, for example, the determination unit 1003 selects unselected teacher data (wariko) from the teacher data (wariko) DB 250. Here, it is assumed that the non-inquiry FAQ of the selected teacher data (wariko) is set to "1", and the search word after the morphological analysis includes the keyword 1 and the keyword 2. In this case, the determination unit 1003 increments the number of non-inquiries of the keyword 1 of the FAQ number "1". Further, the determination unit 1003 increments the number of non-inquiries of the keyword 2 of the FAQ number “1”. The determination unit 1003 repeats the above-mentioned series of processes until there is no unselected teacher data (Yoiko) selected from the teacher data (Yoiko) DB 240.

As a result, it is possible to create an analysis DB 1500 in which the number of non-inquiries of each FAQ is stored for each keyword. Then, the determination unit 1003 calculates a second probability for each of the plurality of FAQs searched by using the above equation (1) with reference to, for example, the analysis DB 1500. However, the posterior probability P (FAQ | search word) indicates the probability that the corresponding FAQ is not selected (inquired) when the FAQ is searched by designating the search word. The prior probability P (FAQ) indicates the probability (ratio) that the corresponding FAQ is not selected. Likelihood P (search word | FAQ) indicates the probability that a search word will be created when the FAQ is determined. Since the specific calculation procedure of the second probability thereafter is the same as that of the first probability, detailed description thereof will be omitted.

Further, in the case of the third probability, the specific unit 1007 creates the analysis DB 1500 with reference to the teacher data (maigo) DB 260. In this case, the analysis DB 1500 sets "alternative words" on the vertical axis and "morphemes" on the horizontal axis, and totals the number of times each alternative word is input for each morpheme. However, the alternative word is an alternative word for the teacher data (maigo) in the teacher data (maigo) DB 260. That is, the alternative word corresponds to another search word input after the 0 hit search word. The morpheme is a morpheme detected by morphological analysis of 0 hit search words. When the morpheme is not detected from the 0-hit search word, for example, the 0-hit search word itself may be used as the morpheme on the horizontal axis.

Specifically, for example, the specific unit 1007 selects unselected teacher data (maigo) from the teacher data (maigo) DB 260. Then, when the search word of the selected teacher data (maigo) includes any morpheme in the analysis DB 1500, the specific unit 1007 inputs the substitute word of the selected teacher data (maigo) for the morpheme. Increment. The specific unit 1007 repeats the above-mentioned series of processes until there is no unselected teacher data (maigo) from the teacher data (maigo) DB 260.

This makes it possible to create an analysis DB 1500 in which the number of times each alternative word is input for each morpheme is stored. Then, the specific unit 1007 calculates a third probability for each of the alternative words in the analysis DB 1500 by using the above equation (1) with reference to, for example, the analysis DB 1500.

However, the posterior probability is P (alternative word | search word). The prior probability is P (alternative word). The likelihood is P (alternative word). The posterior probability P (alternative word | search word) indicates the probability that an alternative word is input after the search word. The prior probability P (alternative word) indicates the probability (ratio) of the corresponding alternative word. Likelihood P (search word | alternative word) indicates the probability that a search word will be created when the alternative word is determined. Since the specific calculation procedure of the third probability thereafter is the same as that of the first probability, detailed description thereof will be omitted.

(FAQ decision example)
Next, an example of determining the FAQ for the search word will be described with reference to FIG.

FIG. 16 is an explanatory diagram showing an example of determining FAQ. In FIG. 16, the first table 1601 stores the first probabilities calculated for each of the plurality of FAQs searched by the search unit 1002. The second table 1602 stores the second probabilities calculated for each of the plurality of FAQs searched by the search unit 1002.

Here, it is assumed that 10 FAQs with FAQ numbers "001 to 010" are searched as FAQs corresponding to the search words. Further, the threshold value α is set to “α = 80”. Further, the younger the FAQ number, the higher the priority when the search is performed by the existing search algorithm. For example, in an existing search algorithm, a FAQ with a large number of past accesses (inquiries) is set with a higher priority.

In this case, the determination unit 1003 identifies the FAQ whose second probability is equal to or greater than the threshold value α with reference to, for example, the second table 1602. Here, the FAQ of the FAQ number "003,005,006,008" is specified. Next, the determination unit 1003 deletes the FAQ record of the specified FAQ number "003,005,006,008" from the first table 1601.

Then, the determination unit 1003 refers to the first table 1601 and obtains the FAQ of the remainder sorted in descending order of the first probability, that is, the FAQ of the FAQ number "001,002,007,004,009,010". , Determine one or more FAQs for the search word.

In this case, the output control unit 1004 transmits the FAQ of the determined FAQ number "001,002,007,004,009,010" to the terminal 201. Here, in the FAQ screen 900, when only the top five items are displayed on the first page (N = 5), the FAQ of the FAQ number "001,002,007,004,009" is displayed on the first page in this order. The FAQ with the FAQ number "010" is displayed on the second page.

For example, as a result of narrowing down the FAQ using the first and second probabilities, the FAQ number "003,005,006,008" is compared with the case where the FAQ is displayed in descending order of the number of past accesses (inquiries). The FAQ of is excluded, and the FAQ of the FAQ number "007,009,010" is moved up and displayed on the first page.

(Example of adjusting threshold α)
Next, an example of adjusting the threshold value α will be described with reference to FIGS. 17 and 18. Here, it is assumed that the top five FAQs are the FAQs displayed on the first page (N = 5).

FIG. 17 is an explanatory diagram (No. 3) showing a specific example of the access log. In FIG. 17, access logs 1701 to 1705 are a set of access logs having the same session number sorted in chronological order. Hereinafter, a case where the hit rate of the first page is calculated from the access logs 1701 to 1705 will be described.

Specifically, for example, the determination unit 1003 calculates the hit rate of the first page from the access logs 1701 to 1705 using the following formula (2). However, X is the number of times that the inquired FAQ exists on the first page of the search result. Y is the number of FAQ inquiries. The unit of the hit rate on the first page is [%].

1st page hit rate = X / Y × 100 ・ ・ ・ (2)

More specifically, the determination unit 1003 calculates the number of times X that the inquired FAQ exists on the first page of the search result with reference to the access logs 1702 to 1705 of the type "inquiry". Here, the order of the access logs 1702, 1703 is "1, 2", which is 5 or less. On the other hand, the order of the access logs 1704 and 1705 is "40,45", which is larger than 5. Therefore, the number of times X is "X = 2".

Further, the determination unit 1003 calculates the number of inquiry times Y of the FAQ with reference to the access logs 1702 to 1705 of the type "inquiry". Here, the number of inquiries Y is the number of access logs 1702 to 1705 of the type "inquiry", and "Y = 4". Therefore, the hit rate on the first page is "50 [%] (= 2/4 x 100)".

The determination unit 1003 calculates the hit rate for the first page every predetermined period, for example, every few weeks to several months. Next, the determination unit 1003 performs regression analysis on the transition of the first page hit rate calculated for each predetermined period, and calculates the slope of a straight line representing the time-series change of the first page hit rate. Then, the determination unit 1003 adjusts the threshold value α according to the calculated inclination.

FIG. 18 is an explanatory diagram showing an example of time-series changes in the hit rate on the first page. In FIG. 18, the straight lines 1801 and 1802 are straight lines representing the time-series changes in the hit rate of the first page. The straight line 1801 indicates that the slope is larger than 0 and the hit rate on the first page tends to increase. On the other hand, the straight line 1802 indicates that the slope is smaller than 0 and the hit rate on the first page tends to decrease.

For example, when the slope of the straight line representing the time-series change of the first page hit rate is larger than 0, the determination unit 1003 does not adjust the threshold value α because the first page hit rate tends to increase. Further, when the slope of the straight line representing the time-series change of the hit rate of the first page is 0, the determination unit 1003 does not adjust the threshold value α because there is no change in the hit rate of the first page.

On the other hand, when the slope of the straight line representing the time-series change of the hit rate of the first page is smaller than 0, the determination unit 1003 adjusts the threshold value α because the hit rate of the first page tends to decrease. Specifically, for example, when the slope of the straight line representing the time-series change of the hit rate of the first page is smaller than 0, the determination unit 1003 lowers the threshold value α by 1 [%].

For example, when the threshold value α before adjustment is “80 [%]”, the determination unit 1003 changes the threshold value α to “79 [%]”. When the hit rate on the first page is on a downward trend, it is assumed that the FAQ on the first page is less likely to be inquired and the learning of teacher data (wariko) progresses. Therefore, in order to improve the reaction of the teacher data (wariko), the threshold value α is lowered and the FAQ is adjusted in the direction of cutting more.

The determination unit 1003 may change the amount of decrease in the threshold value α according to the magnitude of the slope of the straight line representing the time-series change of the hit rate on the first page. For example, if the slope of the straight line is small enough to determine that the hit rate on the first page has dropped significantly, the determination unit 1003 may change the decrease in the threshold value α to about 5 [%]. good.

(Estimation of morpheme)
Next, the process of estimating the morpheme will be described with reference to FIG.

FIG. 19 is an explanatory diagram showing an embodiment of a method for estimating a morpheme. When the determination unit 1007 determines that there is no FAQ corresponding to the search word, the specific unit 1007 performs machine learning based on the stored contents of the teacher data (maigo) DB 260, and inputs it after the 0 hit search word. Identify alternative words to be used. However, the alternative word cannot be specified until the teacher data (maigo) is accumulated.

For example, when a random search word is input due to a punch mistake or the like, the morpheme may not be detected from the search word and 0 hits may be obtained. Further, in this case, it is highly possible that the teacher data (maigo) corresponding to the search word is not accumulated. That is, when a random search word is input due to a punch mistake or the like, the alternative word cannot be specified, and 0 hits may be obtained.

Therefore, when the alternative word is not specified, the information processing apparatus 101 may estimate the morpheme corresponding to the search word and search for the FAQ corresponding to the estimated morpheme.

Specifically, for example, first, the generation unit 1005 generates teacher data (morpheme). More specifically, for example, the generation unit 1005 extracts a morpheme from the search word after the morphological analysis of the teacher data (Yoiko) in the teacher data (Yoiko) DB 240. Then, the generation unit 1005 generates teacher data (morpheme) in which the extracted morpheme and each character included in the morpheme are associated with each other.

As an example, assuming that the morpheme is "FILE", the morpheme "FILE" and the respective elements (characters) "" F "," I "," L "," E "" included in the morpheme "FILE" correspond to each other. The attached teacher data (morpheme) 1901 is generated. The generated teacher data (morpheme) is stored in the teacher data (morpheme) DB1900.

Further, the generation unit 1005 extracts, for example, a morpheme from the search word after morphological analysis of the teacher data (wariko) in the teacher data (wariko) DB 250, and the extracted morpheme and each character included in the morpheme. The teacher data (morpheme) associated with may be generated. Further, the generation unit 1005 extracts a morpheme from a substitute word or a search word of the teacher data (maigo) in the teacher data (maigo) DB260, and corresponds the extracted morpheme with each character included in the morpheme. The attached teacher data (morpheme) may be generated.

Next, when the morpheme is not detected from the search word, the specifying unit 1007 performs machine learning based on the stored contents of the teacher data (morpheme) DB1900 to specify the morpheme corresponding to the search word. However, even if the morpheme is not detected from the search word, the FAQ may be searched from the FAQ master 220.

That is, even when the morpheme is not detected from the search word, the FAQ corresponding to the search word may be output to the terminal 201. Therefore, the specific unit 1007 may specify the morpheme corresponding to the search word, for example, when the morpheme is not detected from the search word and it is determined that there is no FAQ corresponding to the search word. ..

More specifically, for example, the specific unit 1007 refers to the teacher data (morpheme) DB1900, performs machine learning by a naive Bayes classifier, and calculates a fourth probability for each of the plurality of morphemes. The fourth probability indicates a high degree of similarity to the search word in which the morpheme was not detected.

The fourth probability can also be obtained in the same manner as the first probability. First, the specific unit 1007 creates an analysis DB 1500 with reference to the teacher data (morpheme) DB 1900. In this case, in the analysis DB 1500, "morpheme" is set on the vertical axis and "character" is set on the horizontal axis, and the number of appearances for each character is totaled. However, the morpheme is a morpheme of the teacher data (morpheme) in the teacher data (morpheme) DB1900. The characters are characters included in the morpheme of the teacher data (morpheme) in the teacher data (morpheme) DB1900.

Specifically, for example, the specific unit 1007 selects unselected teacher data (morpheme) from the teacher data (morpheme) DB 1900. Here, the morpheme of the selected teacher data (morpheme) is "FILE", and the elements are "" F "," I "," L "," E "". In this case, the specific unit 1007 increments the number of appearances of each element "" F "," I "," L "," E "" of the morpheme "FILE" in the analysis DB 1500. The specific unit 1007 repeats the above-mentioned series of processes until, for example, there is no unselected teacher data (morpheme) selected from the teacher data (morpheme) DB 1900.

This makes it possible to create an analysis DB 1500 in which the number of appearances of each element is stored for each morpheme. Then, the determination unit 1003 calculates a fourth probability for each of the plurality of morphemes by using the above equation (1) with reference to, for example, the analysis DB 1500. However, the posterior probability is P (morpheme | character). The prior probability is P (morpheme). The likelihood is P (morpheme). The posterior probability P (morpheme | character) indicates the probability that a morpheme including the corresponding character is input. The prior probability P (morpheme) indicates the ratio of the corresponding character in all the elements included in the morpheme (vertical axis). The likelihood P (morpheme) indicates the probability that the corresponding character is included when the morpheme is determined. Since the specific calculation procedure of the fourth probability thereafter is the same as that of the first probability, detailed description thereof will be omitted.

Then, the specifying unit 1007 specifies the morpheme corresponding to the search word based on the fourth probability calculated for each of the plurality of morphemes in the analysis DB 1500. Specifically, for example, the specific unit 1007 specifies the morpheme having the maximum fourth probability as the morpheme corresponding to the morpheme word. This makes it possible to estimate the morpheme corresponding to the search word for which the morpheme was not detected by the morphological analysis.

In this case, the specific unit 1007 performs machine learning based on the stored contents of the storage unit 1010 that stores the teacher data (maigo), and specifies an alternative word that is subsequently input to the specified morpheme. Then, the search unit 1002 searches for the FAQ corresponding to the specified alternative word.

As a result, even if a random search word is input due to a punch mistake or the like, it is possible to prevent 0 hits. Further, in the above description, the specific unit 1007 identifies the alternative word to be input after the specified morpheme, and the search unit 1002 searches for the FAQ corresponding to the specified alternative word. Not limited to this. For example, the search unit 1002 may search for the FAQ corresponding to the specified morpheme.

Since the data structures of teacher data (yoiko) and teacher data (wariko) are FAQ numbers and search words after morphological analysis, if morphemes are not detected from the search words, they cannot be learned, and the first and first morphemes cannot be learned. The probability of 2 cannot be calculated. That is, it is not possible to narrow down the FAQ using the first and second probabilities. Therefore, even in such a case, the information processing apparatus 101 estimates the morpheme corresponding to the search word and searches the FAQ again so that the FAQ can be narrowed down using the first and second probabilities. May be.

(Procedure for generating teacher data of information processing device 101)
Next, the teacher data generation processing procedure of the information processing apparatus 101 will be described with reference to FIGS. 20 and 21. The teacher data generation process of the information processing apparatus 101 may be executed periodically, for example, every period of about 24 hours, or may be executed at a predetermined timing specified by the administrator of the response output system 200 or the like. good.

20 and 21 are flowcharts showing an example of the teacher data generation processing procedure of the information processing apparatus 101. In the flowchart of FIG. 20, first, the information processing apparatus 101 selects an unselected session number that has not been selected from the access log DB 230 (step S2001).

Next, the information processing apparatus 101 acquires the access log of the selected session number from the access log DB 230 (step S2002). Then, the information processing apparatus 101 sorts the acquired access logs in chronological order based on the date and time of the acquired access logs (step S2003).

Next, the information processing apparatus 101 determines whether or not there is an access log of type "inquiry" in the acquired access log (step S2004). Here, if there is no access log of type "inquiry" (step S2004: No), the information processing apparatus 101 shifts to step S2101 shown in FIG.

On the other hand, when there is an access log of type "inquiry" (step S2004: Yes), the information processing apparatus 101 determines whether or not the inquiry FAQ specified from the access log is the FAQ displayed on the first page. Determine (step S2005). Here, if the FAQ is not displayed on the first page (step S2005: No), the information processing apparatus 101 shifts to step S2101 shown in FIG.

On the other hand, in the case of the FAQ displayed on the first page (step S2005: Yes), the information processing apparatus 101 represents the inquiry FAQ specified from the access log of the type "inquiry" in association with the search word after morphological analysis. Generate teacher data (Yoiko) (step S2006). Then, the information processing apparatus 101 registers the generated teacher data (Yoiko) in the teacher data (Yoiko) DB 240 (step S2007), and shifts to step S2101 shown in FIG.

In the flowchart of FIG. 21, first, the information processing apparatus 101 refers to the search list of the access log of the type "search" and determines whether or not there is a search word with 0 hits (step S2101). Here, when there is no search word for 0 hits (step S2101: No), the information processing apparatus 101 shifts to step S2105.

On the other hand, when there is a search word with 0 hits (step S2101: Yes), the information processing apparatus 101 accepts input following the search word with 0 hits based on the acquired date and time of the access log. It is determined whether or not there is a search word (step S2102). However, the other search word is a search word in which any FAQ displayed on the first page is queried (selected) for the other search word.

Here, when there is no other search word (step S2102: No), the information processing apparatus 101 shifts to step S2105. On the other hand, when there is another search word (step S2102: Yes), the information processing apparatus 101 generates teacher data (maigo) using the other search word as an alternative word (step S2103).

Then, the information processing apparatus 101 registers the generated teacher data (maigo) in the teacher data (maigo) DB 260 (step S2104). Next, the information processing apparatus 101 refers to the search list of the access log of the type "search" and determines whether or not there is an FAQ that was not queried when displayed for the search word (step S2105). ).

Here, if there is no FAQ that has not been inquired (step S2105: No), the information processing apparatus 101 shifts to step S2108. On the other hand, when there is an FAQ that has not been inquired (step S2105: Yes), the information processing apparatus 101 generates teacher data (wariko) in which the FAQ that has not been inquired is a non-inquiry FAQ (step S2106).

Then, the information processing apparatus 101 registers the generated teacher data (wariko) in the teacher data (wariko) DB 250 (step S2107). Next, the information processing apparatus 101 determines whether or not there is an unselected session number that has not been selected from the access log DB 230 (step S2108).

Here, if there is an unselected session number (step S2108: Yes), the information processing apparatus 101 returns to step S2001 shown in FIG. On the other hand, when there is no unselected session number (step S2108: No), the information processing apparatus 101 ends a series of processes according to this flowchart.

As a result, teacher data (yoiko), teacher data (wariko), and teacher data (maigo) can be generated from the past user operation history (access log). The stored contents of the access log DB 230 are reset each time, for example, when the execution of the teacher data generation process of the information processing apparatus 101 is completed.

(Procedure for processing the answer output of the information processing device 101)
Next, the response output processing procedure of the information processing apparatus 101 will be described with reference to FIGS. 22 and 23.

22 and 23 are flowcharts showing an example of the response output processing procedure of the information processing apparatus 101. In the flowchart of FIG. 22, first, the information processing apparatus 101 determines whether or not the input of the search word is accepted (step S2201). Here, the information processing apparatus 101 waits for the input of the search word to be accepted (step S2201: No).

Then, when the information processing apparatus 101 accepts the input of the search word (step S2201: Yes), the information processing apparatus 101 morphologically analyzes the search word (step S2202). Next, the information processing apparatus 101 searches for FAQ from the FAQ master 220 based on the result of morphological analysis of the search word (step S2203).

Next, the information processing apparatus 101 refers to the teacher data (Yoko) DB 240, performs machine learning by the naive Bayes classifier, and calculates the first probability for each of the plurality of searched FAQs (step S2204). ). Next, the information processing apparatus 101 refers to the teacher data (wariko) DB 250, performs machine learning by the naive Bayes classifier, and calculates a second probability for each of the plurality of searched FAQs (step). S2205).

Then, the information processing apparatus 101 has one or a plurality of FAQs for the search word from the searched plurality of FAQs based on the first probability and the second probability calculated for each of the searched plurality of FAQs. Is determined (step S2206).

Next, the information processing apparatus 101 determines whether or not there is an FAQ corresponding to the search word (step S2207). Specifically, for example, the information processing apparatus 101 has a FAQ corresponding to a search word when no FAQ is searched in step S2203 or when no FAQ is determined in step S2206. Judge that there is no.

Here, when there is an FAQ corresponding to the search word (step S2207: Yes), the information processing apparatus 101 outputs one or a plurality of FAQs for the determined search word (step S2208), and a series of processes according to this flowchart. To finish. On the other hand, when there is no FAQ corresponding to the search word (step S2207: No), the information processing apparatus 101 shifts to step S2301 shown in FIG.

In the flowchart of FIG. 23, first, the information processing apparatus 101 refers to the teacher data (maigo) DB 260, performs machine learning by a naive Bayes classifier, and inputs each of the plurality of alternative words after the search word. The third probability is calculated (step S2301).

Then, the information processing apparatus 101 identifies the alternative word to be input after the search word based on the third probability calculated for each of the plurality of alternative words (step S2302). Next, the information processing apparatus 101 determines whether or not the alternative word has been specified (step S2303). For example, in step S2202 shown in FIG. 22, when the morpheme is not detected from the search word, the alternative word may not be specified.

Here, when the alternative word is specified (step S2303: Yes), the information processing apparatus 101 returns to step S2202 shown in FIG. In this case, the information processing apparatus 101 performs morphological analysis of the specified alternative word.

On the other hand, when the alternative word is not specified (step S2303: No), the information processing apparatus 101 refers to the teacher data (morpheme) DB1900 and performs machine learning by the naive Bayes classifier for each of the plurality of morphemes. , The fourth probability is calculated (step S2304). The fourth probability indicates a high degree of similarity to the search word.

Then, the information processing apparatus 101 identifies the morpheme corresponding to the search word based on the fourth probability calculated for each of the plurality of morphemes (step S2305), and returns to step S2203 shown in FIG. 22. In this case, the information processing apparatus 101 searches the FAQ from the FAQ master 220 based on the specified morpheme.

If no FAQ is searched in step S2203, the information processing apparatus 101 may shift to step S2304. Further, the transition from step S2305 to step S2203 shall not be performed more than once. Therefore, in step S2207 after the transition, if there is no FAQ corresponding to the search word (step S2207: No), the information processing apparatus 101 outputs, for example, information that the search result is 0.

This makes it possible to optimize the FAQ displayed for the search word based on the operation history (access log) of the past user. Further, even if there is no FAQ corresponding to the search word, it is possible to estimate the search word, that is, the alternative word to be replaced with the 0-hit search word, and output the FAQ based on the operation history of the past user. ..

As described above, according to the information processing apparatus 101 according to the embodiment, it is possible to accept the input of the search word and search the FAQ corresponding to the accepted search word. Further, according to the information processing apparatus 101, machine learning is performed based on the stored contents of the storage unit 1010 that stores the teacher data (yoiko) and the teacher data (wariko), and the search is performed from among the plurality of searched FAQs. One or more FAQs for a word can be determined. The teacher data (Yoiko) is, for example, information in which the FAQ for which the selection operation is accepted when the search word (question) is displayed on the first page is associated with the search word. The teacher data (wariko) is, for example, information in which the FAQ that did not accept the selection operation when displayed on the first page of the search word is associated with the search word. Then, according to the information processing apparatus 101, one or a plurality of determined FAQs can be output.

As a result, based on the operation history (access log) of the past user, the FAQ that is strongly related to the search word and the FAQ that is weakly related to the search word are identified from two different viewpoints, and the problem is solved. The FAQs to be connected can be narrowed down accurately.

Further, according to the information processing apparatus 101, the storage unit 1010 is referred to, machine learning is performed by the naive Bayes classifier, and the first probability and the second probability are calculated for each of the plurality of searched FAQs. be able to. The first probability is, for example, the probability of being selected when the search word is displayed on the first page. The second probability is, for example, the probability that the search word is not selected when it is displayed on the first page. Then, according to the information processing apparatus 101, one or a plurality of FAQs for the accepted search word from the plurality of FAQs based on the first probability and the second probability calculated for each of the searched plurality of FAQs. Can be determined.

This makes it possible to improve the accuracy of identifying FAQs that are strongly related to the search word and FAQs that are weakly related to the search word. In addition, by using machine learning with a naive Bayes classifier, it is possible to control the specific accuracy by the quality and quantity of teacher data (Yoiko, Wariko).

Further, according to the information processing apparatus 101, one or a plurality of FAQs obtained by sorting the FAQs excluding the FAQs having the second probability of the threshold α or more among the searched FAQs in descending order of the first probability are searched. You can decide on one or more FAQs for a word. As a result, FAQs that are strongly related to the search word can be displayed at the top of the search, and FAQs that are weakly related to the search word can be excluded from the display target. Specifically, it is possible to select FAQs with high accuracy that are inquired when displayed on the first page, and exclude FAQs with high accuracy that are not inquired even if they are displayed on the first page. Therefore, for example, it is possible to suppress the display of the FAQ that is frequently displayed on the first page but does not lead to the solution of the problem. Further, for example, it is possible to suppress the display of FAQ, which has been frequently inquired as an answer in the past, but has become obsolete and has not been inquired recently.

Further, according to the information processing apparatus 101, one of the FAQs displayed on the first page for the search word is selected (inquired) based on the access log indicating the FAQ that accepted the selection operation for the search word. The probability can be calculated. Then, according to the information processing apparatus 101, the threshold value α can be adjusted based on the time-series change of the calculated probability. As a result, the size of the threshold value α can be optimized based on the inquiry results during operation, such as how much the FAQ displayed on the first page (upper N items) is inquired.

From these facts, according to the information processing apparatus 101, it is possible to improve the primary response rate and customer satisfaction in the FAQ system and the FAQ site. In addition, since the search results can be optimized without reviewing the FAQ, the operating cost of the FAQ system and the FAQ site can be reduced.

The answer output method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This answer output program is a computer-readable recording medium such as a hard disk, flexible disk, CD (Compact Disk) -ROM, MO (Magnet-Optical disk), DVD (Digital Versaille Disk), and USB (Universal Serial Bus) memory. It is recorded on an optical disc and executed by being read from a recording medium by a computer. Further, this answer output program may be distributed via a network such as the Internet.

The following additional notes are further disclosed with respect to the above-described embodiment.

(Appendix 1) Accepting the input of questions,
Identify multiple answer candidates corresponding to the received question,
Machine learning is performed based on the stored contents of the storage unit that stores one or more first answer candidates that have accepted the selection operation for the question and one or more second answer candidates that have not accepted the selection operation for the question. Then, from the specified plurality of answer candidates, one or more answer candidates for the received question are determined.
Output the determined one or more answer candidates,
An answer output program characterized by having a computer perform processing.

(Appendix 2) The process to be determined is
With reference to the storage unit, machine learning by the naive Bayes classifier is performed, and for each of the plurality of identified answer candidates, the first probability selected when displayed for the received question and the first probability. Calculate the second probability that the question will not be selected when displayed for the accepted question.
Based on the first probability and the second probability calculated for each of the plurality of answer candidates, one or a plurality of answer candidates for the received question are determined from the plurality of answer candidates. The answer output program according to Appendix 1, which comprises the above.

(Appendix 3) The process to be determined is
One or a plurality of answer candidates obtained by sorting the answer candidates excluding the answer candidates whose second probability is equal to or higher than the threshold value among the plurality of answer candidates in descending order of the first probability is one or a plurality of answers. The answer output program described in Appendix 2, which is characterized in that a plurality of answer candidates are determined.

(Appendix 4) Accumulate log information indicating answer candidates that have accepted the selection operation for the question.
Based on the accumulated log information, the probability that one of the top predetermined number of answer candidates displayed for the question will be selected is calculated.
Adjusting the threshold value based on the calculated time-series change of the probability.
The answer output program according to Appendix 3, wherein the processing is executed by the computer.

(Appendix 5) The adjustment process is described in Appendix 4, characterized in that the threshold value is lowered by a predetermined value when it is determined that the probability is on a downward trend based on the calculated time-series change of the probability. Answer output program.

(Appendix 6) The first answer candidate is an answer candidate that has received a selection operation from among the top predetermined number of answer candidates for the question.
The second answer candidate is an answer candidate that does not accept the selection operation among the top predetermined number of answer candidates for the question.
The answer output program described in any one of the appendices 1 to 5, characterized in that.

(Appendix 7) The answer output program according to any one of the appendices 1 to 6, wherein the answer candidate is a FAQ (Frequently Asked Question).

(Appendix 8) Accepting the input of questions,
Identify multiple answer candidates corresponding to the received question,
Machine learning is performed based on the stored contents of the storage unit that stores one or more first answer candidates that have accepted the selection operation for the question and one or more second answer candidates that have not accepted the selection operation for the question. Then, from the specified plurality of answer candidates, one or more answer candidates for the received question are determined.
Output the determined one or more answer candidates,
An answer output method characterized by a computer performing processing.

(Appendix 9) Accepting the input of questions,
Identify multiple answer candidates corresponding to the received question,
Machine learning is performed based on the stored contents of the storage unit that stores one or more first answer candidates that have accepted the selection operation for the question and one or more second answer candidates that have not accepted the selection operation for the question. Then, from the specified plurality of answer candidates, one or more answer candidates for the received question are determined.
Output the determined one or more answer candidates,
An information processing device characterized by having a control unit.

101 Information processing device 110, 1010 Storage unit 200 Answer output system 201 Terminal 210 Network 220 FAQ master 230 Access log DB
240 Teacher data (Yoiko) DB
250 Teacher data (wariko) DB
260 Teacher Data (Maigo) DB
300 bus 301 CPU
302 Memory 303 I / F
304 Disk drive 305 Disk 900 FAQ screen 1001 Reception unit 1002 Search unit 1003 Determination unit 1004 Output control unit 1005 Generation unit 1006 Judgment unit 1007 Specific unit 1101, 1102, 1103, 1301, 1302, 1303, 1304, 1701, 1702, 1703 1704, 1705 Access log 1500 Analysis DB
1900 Teacher data (morpheme) DB

Claims (4)

Accepts questions and accepts questions
Identify multiple answer candidates corresponding to the received question,
One or more first answer candidates that received the selection operation among the answer candidates for the question and one or more second answer candidates that did not accept the selection operation among the answer candidates for the question are stored. For each of the plurality of identified answer candidates by performing machine learning based on the stored contents of the storage unit, the first probability selected when displayed for the received question and the received question On the other hand, calculate the second probability that it will not be selected when it is displayed,
Based on the calculated first probability and the second probability, the answer candidate excluding the answer candidate whose second probability is equal to or greater than the threshold value among the plurality of answer candidates has a high first probability. One or more answer candidates sorted in order are determined as one or more answer candidates for the received question.
Output the determined one or more answer candidates ,
The log information indicating the answer candidate that accepted the selection operation for the question is accumulated, and the log information is accumulated.
Based on the accumulated log information, the probability that any of the top predetermined number of answer candidates displayed for the question will be selected is calculated.
The threshold is adjusted based on the calculated time-series change of the probability.
The adjustment process lowers the threshold value by a predetermined value when it is determined that the probability is on a downward trend based on the calculated time-series change of the probability.
An answer output program characterized by having a computer perform processing. The first answer candidate is an answer candidate that has received a selection operation from the top predetermined number of answer candidates for the question.
The second answer candidate is an answer candidate that does not accept the selection operation from the top predetermined number of answer candidates for the question.
The answer output program according to claim 1. Accepts questions and accepts questions
Identify multiple answer candidates corresponding to the received question,
One or more first answer candidates that received the selection operation among the answer candidates for the question and one or more second answer candidates that did not accept the selection operation among the answer candidates for the question are stored. For each of the plurality of identified answer candidates by performing machine learning based on the stored contents of the storage unit, the first probability selected when displayed for the received question and the received question On the other hand, calculate the second probability that it will not be selected when it is displayed,
Based on the calculated first probability and the second probability, the answer candidate excluding the answer candidate whose second probability is equal to or higher than the threshold value among the plurality of answer candidates has a high first probability. One or more answer candidates sorted in order are determined as one or more answer candidates for the received question.
Output the determined one or more answer candidates,
The log information indicating the answer candidate that accepted the selection operation for the question is accumulated, and the log information is accumulated.
Based on the accumulated log information, the probability that any of the top predetermined number of answer candidates displayed for the question will be selected is calculated.
The threshold is adjusted based on the calculated time-series change of the probability.
The adjustment process lowers the threshold value by a predetermined value when it is determined that the probability is on a downward trend based on the calculated time-series change of the probability.
An answer output method characterized by a computer performing processing. Accepts questions and accepts questions
Identify multiple answer candidates corresponding to the received question,
One or more first answer candidates that received the selection operation among the answer candidates for the question and one or more second answer candidates that did not accept the selection operation among the answer candidates for the question are stored. For each of the plurality of identified answer candidates by performing machine learning based on the stored contents of the storage unit, the first probability selected when displayed for the received question and the received question On the other hand, calculate the second probability that it will not be selected when it is displayed,
Based on the calculated first probability and the second probability, the answer candidate excluding the answer candidate whose second probability is equal to or higher than the threshold value among the plurality of answer candidates has a high first probability. One or more answer candidates sorted in order are determined as one or more answer candidates for the received question.
Output the determined one or more answer candidates,
The log information indicating the answer candidate that accepted the selection operation for the question is accumulated, and the log information is accumulated.
Based on the accumulated log information, the probability that any of the top predetermined number of answer candidates displayed for the question will be selected is calculated.
The threshold is adjusted based on the calculated time-series change of the probability.
The adjustment process lowers the threshold value by a predetermined value when it is determined that the probability is on a downward trend based on the calculated time-series change of the probability.
An information processing device characterized by having a control unit.

Images