JP7475977B2 - Knowledge collection support system - Google Patents

Description

The present invention relates to a knowledge collection support system that supports the collection of knowledge required in an information system development project.

In projects to develop information systems, various information about the development project is collected. Examples include information about the client company that will own the information system, the characteristics of the data and processing that the information system will handle, and the technologies used in development. One way to collect this information is to use materials from similar past projects or information provided by experts. However, this work generally places a high burden on personnel and is an obstacle to the smooth progress of the development project. In particular, knowledge that is personal is difficult to pass on to other workers, and knowledge may be lost when an expert leaves the development project.

To pass on such knowledge, it is conceivable to build a knowledge search system and aggregate the knowledge within that system. However, unless searches are performed on the system using appropriate keywords, it is difficult to obtain appropriate answers. For example, it is conceivable to use machine learning to learn the correspondence between questions written in natural language strings and search keywords for the knowledge search system, and then use the learning results to convert the questions into appropriate search keywords, etc.

The following Patent Document 1 addresses the issue that "in the prior art, it is necessary to 'intentionally prepare specific information in order to identify the data you want to obtain,' and a dedicated procedure is also required for this," and discloses a technology that "provides master data, pattern data, and aggregation tables as a means for reducing the burden on both the inquirer and the answerer when extracting an answer, providing advice information to the inquirer, and searching for just-fit answers when searching for information. It also includes a knowledge database with a temporary area for storing useful questions even when an answer is not possible" (see abstract).

The following Patent Document 2 aims to "relate to a learning target extraction device, method, and program for extracting learning target data for a search system that enables natural language search." It discloses a technology that "includes a natural language storage unit that stores natural language including known questions that can be searched, a natural language classification unit that compares question data for which a search request has been made with the natural language and classifies the question data, a question and answer storage unit that stores known questions and answers, a natural language search unit that executes the natural language classification unit based on the question data and outputs search results, an answer extraction unit that extracts answers from the search results, an answer output unit that searches the question and answer storage unit based on keyword data and outputs answers, a learning target determination unit that determines whether the extraction result in which an answer corresponding to the question data is extracted and the question data are learning targets from the output results output by the answer extraction unit and the answer output unit, and a learning target extraction unit that extracts the extraction result and the question data as data to be learned." (See abstract).

JP 2009-042950 A JP 2019-175203 A

In conventional knowledge search systems, when processing questions using machine learning, it is necessary to collect learning data and conduct learning in advance. However, the workload of collecting the learning data required to generate a machine learning model is high, which can be an obstacle to the introduction of the system.

The present invention was made in consideration of the above-mentioned problems, and aims to provide a knowledge gathering support system that makes it easy to contact experts who have the knowledge required for information system development projects.

The knowledge collection support system of the present invention searches past conversation logs based on question data submitted via a chat tool, and during the search, also searches for expert IDs corresponding to the classification results of the question data, and displays the search results via a GUI. The questioner can select an expert via the search results on the GUI to start a conversation with that expert on the chat tool.

The knowledge collection support system of the present invention makes it easy to contact experts who have the knowledge required for information system development projects.

1 is a configuration diagram of a knowledge collection support system 10 according to a first embodiment. 11 is a sequence diagram illustrating the operation of each part when a user of the chat tool 300 sends a question. 11 is a sequence diagram illustrating the operation of each part when a user of the chat tool 300 sends a question. 1 is an example of a graph showing the association between development projects and experts. 11 is a sequence diagram illustrating the operation of each part after a questioner starts a conversation with an expert on the chat tool 300. FIG. FIG. 13 is a schematic diagram illustrating a search operation when emphasis is placed on the recall rate of the search. FIG. 13 is a schematic diagram illustrating a search operation when emphasis is placed on the search matching rate.

<First embodiment>
1 is a configuration diagram of a knowledge collection support system 10 according to a first embodiment of the present invention. The knowledge collection support system 10 is a system that supports the collection of knowledge required in an information system development project. The knowledge collection support system 10 includes a server 100, a database 200, and a chat tool 300.

The chat tool 300 is a tool used for two or more users to converse with each other. The chat tool 300 can be configured as, for example, a desktop application or a web application. A user uses the chat tool 300 to ask about knowledge required for a development project. The question is transmitted to the server 100 via a network as question data written in the form of a natural language string.

The server 100 is a computer that provides answers to question data. The server 100 includes an utterance data collection unit 110, a classifier 120, a question judger 130, a full-text search unit 140, and a UI generation unit 150.

The speech data collection unit 110 collects question data from the chat tool 300. The question determiner 130 distinguishes between parts of the character string contained in the question data that are questions and other parts. It may be possible to determine whether each message sent by the chat tool is a question or not, or it may be possible to determine each sentence. For the sake of convenience in the following description, it will be assumed that the determination is made for each message. Similarly, for the sake of convenience in the description, one message sent by the chat tool 300 will be described as one piece of question data.

The classifier 120 classifies the question data by machine learning. Any known technology can be used as the machine learning algorithm. Examples of items (classification labels) that serve as classification attributes include: (a) the type of information system to which the question data relates (e.g., the business content targeted by the system), (b) the characteristics of the development project of the information system to which the question data relates (e.g., a distinction between a new development project/maintenance project in the same system and the target of that project), and (c) the technology used in the development project to which the question data relates. For example, when using a classifier with these three classification labels, the classifier 120 performs machine learning in advance to classify the question data into one of the classes configured in the classification space.

The full-text search unit 140 uses the question data itself and the results of classification of the question data by the classifier 120 to perform a full-text search on the conversation log 210 (described below). If the search returns a hit, the UI generation unit 150 generates a UI (typically a GUI: Graphical User Interface) that presents the search results and sends it to the user of the chat tool 300. For example, it generates a web page that presents the search results and returns it to the chat tool 300. The chat tool 300 displays the web page on the user's terminal. The case where the search returns no hits will be described later.

The database 200 holds data tables and the like used by the server 100. Specifically, the database 200 holds a conversation log 210, an expert table 220, a development project table 230, an expert-project related table 240, and a learning data table 250.

The conversation log 210 is a record of past conversations on the chat tool 300. In principle, all conversations between all users are recorded, but it may also be possible to record only conversations between users and experts, as described below.

The expert table 220 is a data table that records the IDs of experts who participated in development projects that have been implemented in the past. All participants in a development project are recorded as experts. Therefore, experts here can also be considered as candidates who can be treated as experts.

The development project table 230 is a data table that records the IDs of development projects that have been implemented in the past and the classification attributes of those development projects. The classification attributes referred to here are configured to match the classification attributes used by the classifier 120 when classifying question data. The reason for this will be explained later.

The expert/project association table 240 is a data table for associating development projects that have been implemented in the past with the experts who participated in those development projects. For example, the expert IDs in the expert table 220 and the development project IDs in the development project table 230 are associated and recorded.

The learning data table 250 is a data table for temporarily storing conversations between users and experts on the chat tool 300 as learning data for the classifier 120.

<First embodiment: When a search for a conversation log returns a hit>
2 is a sequence diagram explaining the operation of each part when a user of the chat tool 300 sends a question. The knowledge collection assistance system 10 first searches for knowledge necessary for the questioner from the conversation log 210. Here, the operation when a search by the full-text search unit 140 in the conversation log 210 returns a hit will be explained.

The questioner uses natural language on the chat tool 300 to ask a question about past development projects. The chat tool 300 sends a message (question data) including the question to the server 100. The speech data collection unit 110 acquires the question data.

The classifier 120 classifies the question data according to the results of machine learning performed in advance, and outputs the classification results to the full-text search unit 140. The classification results here are the results of determining which classification in each classification attribute the question data belongs to. For example, when the above-mentioned (a) to (c) are used as classification labels, (a) information system type = insurance system, (b) development project characteristics = new development project for insurance product system, (c) technology used = in-house cloud, etc. The classification labels are not limited to the above-mentioned (a) to (c), and other classification labels may be used as long as they can appropriately describe the attributes of experts who have the knowledge required for the development project. The following assumes the above-mentioned (a) to (c).

The full-text search unit 140 uses the question data and the classification results to perform a full-text search on the conversation log 210. Details of how the question data and the classification results are used will be explained in the second embodiment described below, but in the simplest case, a search can be performed using a simple string combination of the question data string and the classification result string.

If the search returns a hit, the UI generation unit 150 generates a GUI that presents the search results and returns it to the chat tool 300. The chat tool 300 (or the user terminal running the chat tool 300) displays the GUI on a screen. Specifically, past conversation logs (conversational parties and conversation contents) obtained by the search can be displayed on the screen in chronological order of the conversation. If multiple conversations return a hit, they can be arranged by conversation.

By performing the above steps, the questioner can refer to question and answer conversations from past development projects that correspond to the content of the question, and gather the necessary knowledge related to the content of the question.

<First embodiment: When no hits are found in the conversation log search>
3 is a sequence diagram explaining the operation of each part when a user of the chat tool 300 sends a question. Here, the operation when the full-text search unit 140 does not find a hit in the search of the conversation log 210 will be explained. When no useful question-and-answer conversation is obtained from the conversation log 210, the knowledge collection support system 10 presents experts who are assumed to have knowledge on the content of the question to the questioner in the following procedure.

The process is the same as in FIG. 2 up to the step where the full-text search unit 140 searches the conversation log 210. The full-text search unit 140 further uses the question data and the classification results to perform a full-text search on the classification attributes of the development projects held in the development project table 230. How the question data and the classification results are used is the same as when searching the conversation log 210, and will be described in detail in the second embodiment. If the search of the conversation log does not return any hits, the development projects may be further automatically searched for, or a message indicating that no hits were found may be displayed on the chat tool 300, and a button or the like may be displayed to encourage the questioner to search further for development projects.

The full-text search unit 140 searches the expert/project association table 240 using the development project found in the search as a search key to obtain the ID of the participant who participated in the found development project. The full-text search unit 140 further searches the expert table 220 using the participant ID to obtain details about the participant (such as name, affiliation, and area of expertise). At this time, the full-text search unit 140 also obtains the relationship between the development project and the expert (such as which development project the expert participated in).

The UI generation unit 150 generates a GUI including a graph showing the relevance between the development projects and experts found by the search, and returns the GUI to the chat tool 300. The chat tool 300 (or the user terminal running the chat tool 300) displays the GUI on its screen. An example of the graph will be described later.

Figure 4 is an example of a graph showing the relationship between development projects and experts. By searching the development project table 230, it is possible to extract development projects that are assumed to be related to the question data and its classification results. Participants who have taken part in that development project can be considered candidates for experts who have knowledge related to the content of the question. This GUI presents those candidate experts.

Each expert ID is configured so that a conversation can be started with that expert via the chat tool 300. When the questioner selects (clicks) an expert ID, a conversation with that expert begins. Through the conversation with the expert, the questioner aims to gather knowledge related to the content of the question. If the selected expert does not have the necessary knowledge, the questioner may end the conversation with that questioner and select another expert ID.

Figure 4 shows an example of a graph represented by a tree view. The root of the tree corresponds to the information system type. The second level of the tree corresponds to the development project characteristics in that information system. The links connecting experts and development projects in the graph can be strength- or weakness-determined. Expert 3 is highlighted in Figure 4. Possible criteria for strength include, for example, the following:

(Example of link strength 1) In the expert/project related table 240, the role of each participant in the development project and the period of participation in the development project can be defined. The strength of the link between the expert and the development project can be determined according to the importance of the role and the length of participation. For example, a participant with an important role, such as a project manager, can have a thicker link than other participants, and a participant with a shorter participation period can have a thinner link.

(Example of link strength 2) In the expert table 220 or expert/project related table 240, the contribution of an expert to the question from the questioner can be recorded. For example, the number of times an expert is selected on a graph can be recorded in association with the ID of a development project in which the expert participated. Experts who have contributed greatly to questions related to development projects can have a thicker link than other participants. Parameters indicating the contribution other than the number of times selected can also be used. For example, the number of times the content of a past conversation has been found in a search could be considered.

(Example 3 of link strength) When the full-text search unit 140 searches the development project table 230, it can obtain a score that indicates the degree to which the search criteria match. It is conceivable to make the link thicker for experts who belong to development projects with high scores than for experts who belong to other development projects.

Figure 5 is a sequence diagram explaining the operation of each part after the questioner starts a conversation with an expert on the chat tool 300. When the questioner starts a conversation with an expert, it means that the questioner judges that the expert has the necessary knowledge about the content of the question. In that case, the content of the conversation between the questioner and the expert thereafter will be related to the classification result of the initial question content. In other words, the content of the conversation can be used as learning data when the classifier 120 classifies question data. Therefore, in this embodiment 1, the learning data for the classifier 120 is collected according to the procedure in Figure 5.

The questioner converses with the expert on the chat tool 300. The chat tool 300 sends a message (utterance data) including the conversation to the server 100. The utterance data collection unit 110 acquires the utterance data. As with question data, one message is considered to be one piece of utterance data.

The conversation between the questioner and the expert is a two-way conversation, unlike the scenes described in Figures 2 and 3. Therefore, the content of the conversation is not necessarily limited to questions from the expert, and may include general conversation. Therefore, the question determiner 130 determines for each message whether the speech data is a question. Whether a message is a question may be determined according to grammatical rules, or by a learning device that has previously performed machine learning to determine whether a message is a question. Any other appropriate method may also be used to determine the determination.

When the question determiner 130 determines that the speech data is a question, it stores the speech data in the learning data table 250. At this point, by selecting an expert, the questioner indirectly selects a development project in which the expert participated. In other words, there is a certain correlation between the question and the development project. By learning this correlation, it is believed that the search accuracy can be improved when searching for development projects in the future. Therefore, the question determiner 130 associates the classification attributes of the development project in which the selected expert participated with the speech data and stores the classification attributes in the learning data table 250. For example, the classification result of each message is also stored.

If the question determiner 130 determines that the utterance data is not a question, it does not process the utterance data. This allows the utterance data to be collected as learning data only when the questioner is requesting knowledge about the development project, making it possible to narrow down to only utterance data that is useful as learning data.

When the conversation between the questioner and the expert ends, the classifier 120 performs further learning using the messages and classification results stored in the learning data table 250. This can further improve the classification accuracy. In addition, since no work is required to collect learning data, the workload for doing so can be reduced.

<Embodiment 1: Summary>
When searching the conversation log 210 based on question data, the knowledge collection support system 10 according to the first embodiment also searches for expert IDs corresponding to the classification results of the question data and displays the search results on the GUI. The questioner can select an expert via the search results on the GUI and start a conversation with the expert on the chat tool 300. This allows the questioner to smoothly acquire knowledge related to past development projects even if the knowledge is personalized and not recorded in the conversation log 210 or the like.

In the knowledge collection support system 10 according to the first embodiment, the development project table 230 stores the classification attributes of past development projects, and the development project table 230 is searched for development projects having classification attributes that match the classification results of the question data, and participants who took part in the development projects are presented on the GUI as candidates for experts. This makes it possible to efficiently search for experts who are assumed to have knowledge related to the question data.

The knowledge collection support system 10 according to the first embodiment presents a graph showing the relevance between the development projects and experts obtained by the search as a search result. This allows the questioner to efficiently select an expert who is assumed to have the necessary knowledge. Even if the selected expert does not have the necessary knowledge, another expert can be quickly selected.

The knowledge collection support system 10 according to the first embodiment stores the conversation between the questioner and the expert on the chat tool 300 in the learning data table 250 in association with the classification results of the development project in which the expert participated, and the classifier 120 uses the data stored in the learning data table 250 as learning data. This reduces the workload of the classifier 120 in collecting learning data for learning.

The knowledge collection support system 10 according to the first embodiment stores conversations between the questioner and the expert on the chat tool 300 in the learning data table 250 in association with classification attributes of past development projects in which the expert participated, and the classifier 120 uses the data stored in the learning data table 250 as learning data. This makes it possible to learn the relationship between the speech data and the development projects, making it possible to search for appropriate experts in future expert searches.

In the knowledge collection support system 10 according to the first embodiment, the question determiner 130 stores only speech data that is determined to be a question in a conversation between a questioner and an expert on the chat tool 300 in the learning data table 250. This makes it possible to narrow down learning to only speech data that is useful as learning data.

The knowledge collection support system 10 according to the first embodiment searches the conversation log 210 using question data, and if a search result is obtained, presents the search result to the questioner. If no search result is obtained, the system searches again for development projects and experts that match the question data and presents the results to the questioner. This allows for efficient knowledge collection by presenting the necessary information as is when it exists in the conversation log 210, and also makes it possible to collect the necessary knowledge from experts even if the knowledge is personalized.

<Embodiment 2>
In the second embodiment of the present invention, a detailed operation will be described when the full-text search unit 140 uses question data and classification results to search the conversation log 210. The configuration of the knowledge collection support system 10 is the same as that of the first embodiment.

Figure 6 is a schematic diagram explaining the search operation when emphasis is placed on the recall rate of the search. The recall rate is an index that indicates the comprehensiveness of the search, and is expressed as (True Positive/(True Positive+False Negative)). In other words, the recall rate indicates the proportion of search results that actually hit the search among those that truly match the search keywords.

To increase recall, it is desirable to collect as many search results as possible that are assumed to be related to the question data. In this case, the full-text search unit 140 uses a simple string combination of the question data string and the classification result string as the search keyword. This makes it possible to collect a wide range of conversation logs that match either the question data or the classification results. This is thought to be due to the reduction in false negatives in the above formula.

Figure 7 is a schematic diagram explaining the search operation when the search precision is important. Precision is an index that indicates the accuracy of the search, and is expressed as (True Positive/(True Positive+False Positive)). In other words, precision indicates the proportion of search results that truly match the search keyword.

In order to increase the matching rate, it is useful to narrow down the conversation logs to be searched to those with a high probability of being a hit before conducting a search. In this case, the full-text search unit 140 narrows down the conversation logs to be searched to in advance by searching for conversation logs 210 that match the classification results, and then performs a full-text search on the narrowed-down conversation logs using the question data. This makes it possible to obtain conversation logs that are highly likely to match the question data. This is thought to be due to the reduction in false positives in the above calculation formula.

The search operations described in Figures 6 and 7 can also be applied when searching the development project table 230. That is, (a) to increase recall, the question data and the classification results are combined into a string and a full-text search is performed on the development project table 230, and (b) to increase precision, the records in the development project table 230 are first narrowed down using the classification results, and then the development project table 230 is again full-text searched using the question data.

<Modifications of the present invention>
In the above embodiments, each functional unit of server 100 can be configured by hardware such as a circuit device that implements that function, or can be configured by a computing device executing software that implements that function.

In the above embodiments, the classifier 120 can be configured with software that uses any machine learning algorithm (e.g., a neural network) and data that records the learning results.

In the above embodiment, if the full-text search unit 140 searches the development project table 230 but does not find any hits, it may reply to the chat tool 300 that there were no search results, and may also reply with a message encouraging the user to change the content of the question.

In Figure 4, an example is described in which the relationships between development projects and their participants are displayed in a tree-type graph, but the method of presenting the relationships is not limited to this, and other display formats may be used as long as they can present similar relationships.

10: Knowledge collection support system 100: Server 110: Speech data collection unit 120: Classifier 130: Question judgement unit 140: Full text search unit 150: UI generation unit 200: Database 210: Conversation log 220: Expert table 230: Development project table 240: Expert-project related table 250: Learning data table 300: Chat tool

Claims (13)

A knowledge collection support system that supports collection of knowledge required in an information system development project, comprising:
a speech data collection unit that receives question data describing natural language character strings including questions about development projects implemented in the past as messages transmitted by a chat tool;
a classifier configured to classify the query data using a machine learning model;
a full-text search unit that searches past conversation logs in the chat tool using a classification result by the classifier and the question data;
a user interface for displaying the search results on a screen;
Equipped with
The knowledge collection support system further includes a database for recording the conversation log and for recording IDs of experts who are treated in the conversation log as having knowledge required for the development project related to the conversation content,
When searching the conversation log, the full-text search unit also searches for experts in the conversation log that correspond to the classification result,
The user interface displays, on a screen, an ID of the expert as a result of the search;
the user interface is configured to enable a conversation with an expert in the chat tool to be started by selecting the ID of the expert obtained as a result of the search. The database records classification attributes of development projects implemented in the past in association with IDs of the experts who participated in the development projects,
the full-text search unit searches the database for the development projects having classification attributes matching the classification attributes obtained by the classifier classifying the question data, and searches for IDs of the experts who participated in the matching development projects;
2. The knowledge gathering support system according to claim 1, wherein the user interface displays on a screen ID of the expert who participated in the matching development project. The knowledge collection support system according to claim 2, characterized in that the user interface displays on a screen, as a search result by the full-text search unit, a graph showing the relevance between the development projects that match the classification attributes obtained by the classifier classifying the question data and the experts who participated in the development projects. The database records at least one of the roles of the experts who participated in development projects implemented in the past in the development projects or the period during which the experts participated in the development projects;
The knowledge collection support system according to claim 3, characterized in that the user interface is configured to represent the strength of the connection between the development project and the expert on the graph according to the importance of the role or the length of the participation period. the database records data representing the degree of contribution of the expert to questions related to development projects implemented in the past for each ID of the expert;
4. The knowledge collection support system according to claim 3, wherein the user interface is configured to represent, on the graph, the strength of the connection between the development project and the expert according to the degree of contribution. when searching the database for the development project, the full-text search unit obtains a score indicating a degree of match with a search condition together with the search result;
4. The knowledge collection support system according to claim 3, wherein the user interface is configured to represent, on the graph, the strength of the connection between the development project and the expert according to the score. the database records, in association with each other, classification attributes of the development projects in which the expert selected on the user interface participated, and speech data in conversations with the expert;
The knowledge collection support system according to claim 2, characterized in that the classifier classifies the speech data according to the classification attributes of the development project in which the selected expert participated by learning the classification attributes of the development project in which the selected expert participated and the speech data as learning data. the knowledge collection support system further includes a question determiner that determines whether or not a natural language character string included in the question data includes a question for each message sent through the chat tool;
when the question determiner determines that the natural language character string included in the question data includes a question, the classifier learns the classification result of the question data by the classifier and the question data as learning data;
2. The knowledge collection support system according to claim 1, wherein the classifier does not use the question data as learning data when the question determiner determines that the natural language character string contained in the question data does not contain a question. The classifier classifies the question data as follows:
A type of information system to which the query data pertains;
characteristics of the development project to which the inquiry data relates;
the technology used in the development project to which the questionnaire data relates;
The classification is configured to classify the data based on at least one of the classification attributes,
2. The knowledge collection support system according to claim 1, wherein the full-text search unit searches the conversation log using a result of classifying the question data according to the classification attribute. The database includes the following classification attributes for past development projects:
Type of information system,
the characteristics of the development project;
The technologies used in the development project,
At least one of the following has been recorded:
The classifier classifies the query data according to the classification attributes recorded in the database;
The knowledge collection support system according to claim 2, characterized in that the full-text search unit searches the conversation log using the result of classifying the question data according to the classification attribute, thereby searching for the ID of the expert related to the development project to be asked a question by the question data. When the conversation log that matches the classification result by the classifier and the question data is found, the full-text search unit outputs the found conversation log as a search result;
The knowledge collection support system according to claim 1, characterized in that if the full-text search unit does not find the conversation log that matches the classification result by the classifier and the question data, it outputs the ID of the expert as a search result. The knowledge collection support system described in claim 1, characterized in that the full-text search unit searches the conversation log using a keyword that combines the classification result by the classifier and the question data as a string, thereby improving the search recall rate more than when the conversation log is searched using the classification result by the classifier and the question data individually. The knowledge collection support system described in claim 1, characterized in that the full-text search unit narrows down the conversation log using the classification results by the classifier and performs a search on the narrowed-down conversation log using the question data, thereby improving the search precision compared to a case in which the classification results by the classifier and the question data are used in parallel in combination.