WO2020263182A1

WO2020263182A1 - Method and system for conducting a brainstorming session with a virtual expert

Info

Publication number: WO2020263182A1
Application number: PCT/SG2020/050354
Authority: WO
Inventors: Serhad SARICA; Jianxi LUO
Original assignee: Singapore University Of Technology And Design
Priority date: 2019-06-24
Filing date: 2020-06-24
Publication date: 2020-12-30

Abstract

A computer-implemented method of conducting a brainstorming session with a virtual expert includes obtaining a first knowledge graph including: vertices that are semantic entities in an expert vocabulary, qualitative relations between the entities, and edges that are weighted according to semantic similarity between the entities, said expert vocabulary being generated from a plurality of documents associated with the expert. The method further includes receiving input text from a user; processing the input text to determine one or more user-provided semantic entities; searching the first knowledge graph to determine a position or positions of the one or more user-provided semantic entities in the first knowledge graph; determining one or more suggested first semantic entities from the first knowledge graph based on their semantic similarity to the user-provided semantic entities; and displaying, to said user, a response including the one or more suggested first semantic entities.

Description

METHOD AND SYSTEM FOR CONDUCTING A BRAINSTORMING SESSION WITH A VIRTUAL EXPERT

Technical Field

The present invention relates, in general terms, to a computer-implemented method of conducting a brainstorming session with a virtual expert, and to a system that implements the method.

Background

Experts often have extensive and valuable knowledge, which can be inspiring and useful for many other people. However, such personal knowledge is intangible and difficult to be utilized by other people. Others can possibly brainstorm with an expert or read his/her documents to tap into his/her knowledge base for learning, inspiration and problem solving. However, the in- person accessibility of individual experts is naturally limited, and the documents that contain their knowledge are also generally unstructured.

It would be desirable to overcome or alleviate at least one of the above- described problems, or at least to provide a useful alternative.

Summary

A computer-implemented method of conducting a brainstorming session with a virtual expert, including :

(i) obtaining a first knowledge graph including: nodes that are semantic entities in an expert vocabulary, and edges that are weighted according to semantic similarity between the entities and that are labelled with one or more qualitative relations between the entities, said expert vocabulary being generated from a plurality of documents associated with, and/or authored by, one or more human experts; (ii) receiving input text from a user;

(iii) processing the input text to determine one or more user-provided semantic entities;

(iv) searching the first knowledge graph to determine a position or positions of the one or more user-provided semantic entities in the first knowledge graph;

(v) determining one or more suggested first semantic entities from the first knowledge graph based on their semantic relatedness to the user-provided semantic entities; and

(vi) displaying, to said user, a response including the one or more suggested first semantic entities.

In some embodiments, the plurality of documents is associated with, and/or authored by, a single human expert .

In some embodiments, the response is a natural language response constructed from the one or more suggested first semantic entities.

In some embodiments, the one or more suggested first semantic entities are determined based on a high semantic similarity to the one or more user- provided semantic entities.

In some embodiments, operations (ii)-(vi) are performed iteratively. In some embodiments, the method includes, after a predetermined number of iterations:

setting a distance threshold;

determining a first location corresponding to a centroid or medoid of vector representations of the user-provided semantic entities and the one or more suggested first semantic entities in a previous iteration;

determining a second location corresponding to a centroid or medoid of vector representations of the user-provided semantic entities and the one or more suggested first semantic entities in a current iteration; and

if the second location is less than the distance threshold from the first location, increasing the distance threshold. In some embodiments, the method includes:

determining, in operation (iv), that one or more of the one or more user- provided semantic entities do not exist in the first knowledge graph; and

for each said non-located user-provided semantic entity, searching a second knowledge graph to suggest one or more second semantic entities based on semantic relatedness to the non-located user-provided semantic entity, wherein the second knowledge graph is a general knowledge graph that is at least partly based on documents other than the documents associated with, and/or authored by, the one or more human experts. The present disclosure also relates to a computer-implemented method of generating a knowledge graph for one or more human experts, including:

obtaining a plurality of documents associated with, and/or authored by, the one or more human experts;

executing a part-of-speech tagging process and a syntactic dependency determination process on the plurality of documents to generate an expert vocabulary;

training a language model from the expert vocabulary; and

generating, using the language model and the expert vocabulary, a knowledge graph, said knowledge graph including nodes that are semantic entities in the expert vocabulary, and edges that are weighted according to semantic similarity between the semantic entities, the edges being labelled with qualitative relations between the entities.

In some embodiments, the plurality of documents are associated with, and/or authored by, a single human expert.

The present disclosure further relates to a system for conducting a brainstorming session with a virtual expert, comprising:

a knowledge retrieval module that is configured to:

(i) obtain a first knowledge graph including: nodes that represent semantic entities in an expert vocabulary, and edges that are weighted according to semantic similarity between the entities and that are labelled with one or more qualitative relations between the entities, said expert vocabulary being generated from a plurality of documents associated, and/or authored by, one or more human experts;

(ii) receive input text from a user;

(iii) process the input text to determine one or more user-provided semantic entities;

(iv) search the first knowledge graph to determine a position or positions of the one or more user-provided semantic entities in the first knowledge graph ;

(v) determine one or more suggested first semantic entities from the first knowledge graph based on their semantic relatedness to the user-provided semantic entities; and

(vi) display, to said user, a response including the one or more suggested first semantic entities. In some embodiments of the system, the plurality of documents is associated with, and/or authored by, a single human expert.

In some embodiments of the system, the response is a natural language response constructed from the one or more suggested first semantic entities.

In some embodiments of the system, the knowledge retrieval module is configured to determine the one or more suggested first semantic entities based on a high semantic similarity to the one or more user-provided semantic entities.

In some embodiments of the system, the knowledge retrieval module is configured to perform operations (ii)-(vi) iteratively. In some embodiments of the system, the knowledge retrieval module is configured to:

set a distance threshold;

determine a first location corresponding to a centroid or medoid of vector representations of the user-provided semantic entities and the one or more suggested first semantic entities in a previous iteration;

determine a second location corresponding to a centroid or medoid of vector representations of the user-provided semantic entities and the one or more suggested first semantic entities in a current iteration; and

if the second location is less than the distance threshold from the first location, increase the distance threshold.

Some embodiments include a knowledge extension module that is configured to:

determine, in operation (iv), that one or more of the one or more user- provided semantic entities do not exist in the first knowledge graph; and

for each said non-located user-provided semantic entity, search a second knowledge graph to suggest one or more second semantic entities based on semantic relatedness to the non-located user-provided semantic entity, wherein the second knowledge graph is a general knowledge graph that is at least partly based on documents other than the documents associated with, and/or authored by, the one or more human experts. Some embodiments include a knowledge representation module that is configured to generate the first knowledge graph by:

obtaining a plurality of documents associated with the one or more human experts;

constructing a language model from the expert vocabulary; and generating, using the language model and the expert vocabulary, the first knowledge graph.

Brief description of the drawings

Embodiments of the present invention will now be described, by way of non limiting example, with reference to the drawings in which :

Figure 1 is a system architecture diagram of an example system for conducting a brainstorming session;

Figure 2 is a flow diagram of an example process for generating a knowledge graph for a knowledge agent in certain embodiments of the present invention; Figure 3 is a flow diagram of an example process for conducting a brainstorming session with a virtual expert;

Figure 4 is an example architecture of a computing apparatus for conducting a brainstorming session;

Figure 5 is a schematic depiction of a user interacting with a knowledge agent in certain embodiments; and

Figure 6 is a further schematic depiction of a user interacting with a knowledge agent in certain embodiments.

Detailed description

Embodiments relate to a computerized agent that captures, represents and mimics the professional knowledge of an individual (or the knowledge related to a topic or field mastered by multiple individuals) as a knowledge graph, based on the documents (such as paper publications, patent documents, written reports, speeches, interviews, biographies) generated by or about that particular individual (and/or documents and data related to that particular topic or field from multiple individuals). That is, the computerized virtual agent may master the knowledge of a specific real human individual, multiple or many real human individuals, depending on the specific applications. That is, the computerized agent can mimic a person in terms of his/her knowledge systematically, or in terms of the knowledge of a topic or field of multiple persons, so as to emulate a "virtual expert" that can converse with a user in relation to a given topic or field of expertise. This agent is also capable of iterative brainstorming with a user considering the user's interests, based on the agent's trained knowledge graph. Thus, users who want to brainstorm with a specific person, e.g., an expert on certain topics, can now brainstorm with a computer agent that mimics the knowledge of the person. For example, in certain embodiments, students who want to consult a specific professor can brainstorm with a computer agent that has been trained on data indicative of the professor's knowledge, instead of with the professor him/herself.

In certain embodiments, this invention uses the documents and data of an individual (or a topic or field) to extract and build a knowledge graph that represents the knowledge of the individual (or the topic or field), and also uses the graph as the back end basis of a computerized agent to retrieve and present the knowledge elements of the individual (or a topic or field) in response to iterative inputs from a user in a computer-human interactive brainstorming process. The virtual agent may represent one or multiple experts in terms of their knowledge. Embodiments will be useful for a mass of users to tap into the knowledge and intelligence of the experts of the users' interests for targeted learning, inspiration and problem solving.

Embodiments relate to a data-driven intelligent computer agent which captures and represents an individual's knowledge (or the knowledge related to a topic or field that is mastered by more than one individual) for interactive and iterative human-computer brainstorming sessions. During such interaction sessions, in certain embodiments the agent is also capable of sensing a convergence around a specific topic and taking actions to either explore far areas of the agent knowledge graph or end the brainstorming based on the knowledge content generated in prior iterations. An example of a system for conducting a brainstorming session with a virtual expert will now be described, with reference to Figure 1. In Figure 1, a system 100 comprises a knowledge representation module 110 that is configured to ingest a store 108 of documents that are relevant to an individual expert (or a group of experts, or the knowledge related to a topic or field that is only mastered by many experts together), and to apply a series of natural language processing (NLP) algorithms to generate an agent knowledge graph 112.

The agent knowledge graph 112 is a graph representation of the knowledge structure of the virtual expert, based on the concepts extracted from the source documents 108. Typically, the agent knowledge graph 112 comprises a plurality of nodes or vertices that are elements of a vocabulary (such as words or phrases), connected by edges that may be labelled according to qualitative relationships such as "part of", "IsA", "UsedFor", "Causes", and the like. Accordingly, the agent knowledge graph 112 is a graph of semantic units that are retrieved from the data documents 108 related to the virtual expert and connected according to their semantic similarities and qualitative relationships.

Two nodes may be connected by an edge only if they have a semantic similarity that is above a certain threshold, for example. The semantic similarity may be computed based on respective vector representations, according to a language model that is trained on the data documents 108. In this regard, any suitable method may be used to compute the semantic similarity, such as a cosine similarity between the vector representations. In other embodiments, different representations of the entities may be used, in which case alternative measures of semantic similarity may be appropriate. It will be appreciated that a semantic similarity may be based on a distance metric (such as Euclidean distance in a suitably defined vector space).

The system 100 also comprises a knowledge retrieval module 130. Knowledge retrieval module 130 receives input from a user 102, for example via a web browser interface 104, parses the input to identify one or more semantic units, and searches the agent knowledge graph 112 to locate nodes corresponding to the one or more semantic units in the agent knowledge graph 112. The knowledge retrieval module 130 may then identify one or more nodes that are adjacent to the located nodes in the agent knowledge graph 112, and use these to generate a response to the user 102 for display in browser interface 104. In some embodiments, the adjacent nodes may be one or more nodes that are directly adjacent (i.e., directly connected) to the located nodes, while in other embodiments, the adjacent nodes may be one or more degrees removed from the located nodes.

Where there is more than one node connected to a located node, the adjacent nodes may be identified on a deterministic basis (e.g., according to the weights of the edges joining the located nodes and the adjacent nodes), or on a stochastic basis (e.g., by uniform sampling, or importance-based sampling according to the weights of respective edges joining the nodes).

In some embodiments, the knowledge retrieval module 130 may use not only the adjacent nodes identified during a current round of interaction (which may be termed a conversation turn) of the user 102 with the system 100, but may also make use of records from previous conversation turns to generate the next response of the knowledge retrieval module 130 to the user 102. For example, the knowledge retrieval module 130 may determine that previous conversation turns have covered a limited set of conversational topics, e.g. only spanning a limited subset of the nodes of the agent knowledge graph 112, and may randomly jump to another part of the agent knowledge graph 112 to retrieve one or more nodes for building the next response.

The system 100 further comprises a knowledge extension module 120. The knowledge extension module 120 is used by the knowledge retrieval module 130 to supplement the knowledge structure in agent knowledge graph 112 when one or more semantic units identified in the input from user 102 cannot be located in the agent knowledge graph 112. That is, given that the agent knowledge graph 112 is built from the expert knowledge of a virtual expert with a boundary of knowledge defined by either a real human individual, a group of individuals or the individuals related to a topic or field, there will be occasions on which a topic of interest to the user 102 is not encompassed by that virtual expert's knowledge. In such cases, the information in agent knowledge graph 112 can be supplemented by information from additional, more general, knowledge bases.

The knowledge extension module 120 creates an interface between the agent knowledge graph (AKG) 112 and one or more general knowledge graphs (GKG) 122, in order to retrieve semantic units in the agent knowledge graph (AKG) 112 when user inputs cannot be found in the agent knowledge graph (AKG) 112, according to their semantic relationships in a bigger general knowledge graph (GKG) 122. That is, in general, each node in the AKG is beatable in the GKG, such that the nodes in the AKG form a subset of the nodes of the GKG. Examples of GKGs suitable for this purpose include ConceptNet or WordNet, or more specialised (but still general) GKGs such as GKGs related to technical concepts, such as TechNet [Sarica et al., Expert Systems With Applications, vol. 142, 112995, 2020], and the like. If a semantic unit in the user's input cannot be located in the AKG, the knowledge extension module 120 searches for it in the GKG. If that specific semantic unit is found in GKG, then the system searches for the most relevant node(s) in the AKG to the corresponding semantic unit found in the GKG. This may be done by using the GKG's own semantic similarity or knowledge base relations to the nodes in AKG (supposing that there is a considerable amount of intersection between AKG and GKG).

Where the GKG needs to be consulted to locate a semantic entity in the user's input, one or more different strategies may be employed to select the relevant node in the AKG. For example, the selection strategy may require that the selected node in the GKG not be one of the semantic units in the current user input, though it may or may not be a node in a set of nodes that has been retrieved from the AKG in previous rounds of brainstorming, for example according to whichever mode in the system is in (divergence, or convergence).

Knowledge extension module 120 may access one or more previously generated general knowledge graphs 122, and determine the semantic relationships of semantic units in general knowledge graph(s) 122 to those in agent knowledge graph 112. Accordingly, when a semantic unit in the user input is not found in the agent knowledge graph 112, the general knowledge graph 122 can be searched to identify a node corresponding to that semantic unit, and one or more nodes in the agent knowledge graph 112 that are related to the node of the general knowledge graph 122 can be identified for construction of the next response to user 102.

A computer-implemented method 200 for generating a knowledge graph for an expert will now be described, with reference to Figure 2. The method may be carried out by knowledge representation module 110, for example, though this is not essential. The knowledge representation module 110 may mine data and documents related to a person to create a knowledge graph 112 that represents the structure of the knowledge of that person by extracting semantic units and their qualitative relations and quantifying their semantic similarities. At step 202, the process 200 begins by the knowledge representation module 110 obtaining a plurality of documents associated with an expert, for the purpose of extracting textual or natural language data related to the virtual expert of interest. The virtual expert may represent the knowledge of a real human individual, multiple human individuals, or a topic or field that is from many human individuals. The documents may be stored in a local data store 108. In some embodiments, documents may instead (or additionally) be retrieved "on the fly" from one or more distributed data sources.

Next, at step 204, the data are extracted from the collection 108 of documents. The documents may be ones generated by one or multiple human individuals or about the individuals (or a collection of documents which are related to the topic or field, which are generated by many human individuals). The documents may include technical documents, webpages, newspaper articles, Wikipedia pages, speeches, interviews, academic papers, patent documents, etc. Audio and visual data that can be transformed to natural language descriptions in textual form can also be used. Step 204 may comprise tokenisation of the source documents into words, phrases and sentences by any of a number of techniques, such as by use of tokenisation functions in the Natural Language Toolkit (NLTK - www.nltk.org), the spaCy API, Keras, Gensim, or custom functions based on regular expressions. In some embodiments, step 204 may also comprise one or more denoising steps, such as lemmatization to remove multiple inflection of words, and stopwords removal.

At step 206, a Part-of-Speech (POS) tagging process is run on tokenised words of the extracted text from step 204. For example, methods from standard libraries such as NLTK, Spacy or StanfordCoreNLP may be used in the POS tagging step 206. Next, at step 208, an expert vocabulary is generated from tokenised sentences of the extracted text. A syntactic parsing process, also referred to herein as a syntactic dependency determination process, may be executed on the tokenised sentences. For example, syntactic parsing functions of the NLTK or spaCy libraries may be used for this purpose. Using known dependency parsers, the syntactic dependencies between the words in a sentence can be derived. Using POS tags of words and dependency types between words, one or more syntactic rules can be defined to derive the syntactic dependencies. For example, from the sentence "the fan is a component of a jet engine", we can derive qualitative relations such as (fan, partOf, jet engine), (jet engine, has, component). A (NOUN, OF, NOUN) rule derives (jet engine, has, component), (SUBJ->AUX->X, OF->OBJ) derives (fan, part of, jet engine) relation for some X, while (NOUN, IS, DET, NOUN) rule derives (fan, isA, component) relation.

At step 210, a language model is constructed using the expert vocabulary and sentences obtained at steps 204 and 208. The language model may be chosen from among readily-available algorithms used in the art, such as word2vec, glove, fasttext, BERT, or ELMo. As will be appreciated, depending on the chosen algorithm, the training procedure for the language model, as well as post training procedures to convert the trained model to an AKG may vary. The language model is constructed (trained) by first preprocessing the expert data (the documents) and running the algorithm for the language model on the preprocessed data. The language model may be trained using context windows; that is, the training process may consider, for each word, one or more adjacent words that are likely to have a semantic relationship with that word. A parameter search procedure may be conducted to locate optimum parameters for the selected algorithm using a suitable downstream task for the selected algorithm such as semantic similarity tasks, text classification, and the like.

Following construction of the language model, every word or phrase in the extracted text is represented by a unique vector or a few vectors considering its context. For example, during pre-processing, words and phrases are detected in the text. Then, based on the chosen language model, these words or phrases are either given a single vector representation (when algorithms such as word2vec, glove, fasttext and the like are used), or multiple vector representations, with each vector representation corresponding to a different sense of the word or phrase affected by its context (when algorithms such as BERT, ELMo, and the like are used). Finally, at step 212, a knowledge graph of semantic entities (words or phrases) contained in the vocabulary generated at step 210 is constructed by the knowledge representation module 110. This is done by considering entities' contexts. The edges between these entities are constructed by considering qualitative relationships such as "part of", "IsA", "UsedFor", "Causes", etc, among them and weighted by their semantic relationships calculated as the cosine similarity between the vector representations of these entities. As noted above, a language model may be trained using context windows, thus enabling derivation of quantitative relationships between the semantic entities, these being the nodes in the knowledge graph. The resultant graph 112 is termed an Agent Knowledge Graph (AKG).

An example of a computer-implemented method for conducting a brainstorming session with a virtual expert will now be described with reference to Figure 3. In this example, the process 300 may be executed (wholly or in part) by the knowledge retrieval module 130, though it will be appreciated that other implementations are possible. The knowledge retrieval module 130 may run during the brainstorming session between the computer agent 100 and a real person 102 (referred to herein as the user).

The process 300 begins at step 302 by the knowledge retrieval module 130 receiving input from the user 102, for example via browser interface 104. For example, the input may be entered as text in a field presented by the browser interface 104. Other embodiments may employ alternative input methods, such as speech recognition. Advantageously, by accepting voice input, the process 300 may feel more "natural" to the user 102, so as to provide a greater sense that they are interacting with a real person (the expert whose knowledge is represented in the agent knowledge graph 112), rather than a computerised agent.

Next, at step 304, the knowledge retrieval module 130 processes the user's input by extracting words and phrases, and detecting their POS tags and syntactic dependencies, for example in similar fashion to steps 204, 206 and 208 of the process 200.

The processed semantic inputs are then positioned in the agent knowledge graph 112. This may be done in iterative fashion as follows.

First, at step 306, a next (or the first) semantic entities of the processed user input are retrieved, and the agent knowledge graph 112 is then searched at step 308 (for example, by a simple lookup process) to determine if the semantic entities correspond to nodes in the agent knowledge graph 112.

If the current semantic entities are located in the AKG 112, processing goes to step 310, where one or more related nodes to the located nodes for the semantic entity may be retrieved. For example, the one or more related nodes may be directly adjacent nodes, that is, nodes that are directly connected to the node(s) corresponding to the user input. The knowledge retrieval module 130 may select the adjacent node that is connected by the most highly weighted edge, for example, and corresponds to the same part of speech as the node from the user input. In some embodiments, the system 100 (e.g., via knowledge retrieval module 130) may use the vector representations of the semantic entities, generated using the language model, to determine one or more related nodes by traversing the vector space defined by the AKG 112. For example, during an iteration of the brainstorming process at time t, the knowledge retrieval module 130 may calculate a centroid (or alternatively, a medoid), cw(t), using the vector representations of the semantic entities of the user input and the given response, and store cw(t). In the next iteration, a change rate, cr(t), (i.e. the rate of change of conversation in a specific direction in the knowledge graph) can be calculated by using cw(t) - cw(t-l) and stored by the knowledge retrieval module 130.

Based on cr(t) and the entities from user input in the iteration at time t, the system 100 may choose related entities (E) in the direction of cr(t) in the vector space defined by the AKG 112. These entities may be selected such that they can be used to create a sentence using basic rule-based sentence forming methods based on POS tags. There may be more than one way of forming responses, such as 1) rule-based techniques that entities (E) and their POS tags are used to form a response; 2) extractive techniques where sentence(s) are selected from expert documents considering the quantitative relatedness of (E) to the sentence(s); (3) the selected nodes are preferred to the adjacent nodes in the direction cr(t), but in the case that these nodes have already been visited in the conversation and an early convergence is detected, the knowledge retrieval module 130 may go further in the cr(t) direction to find indirectly connected nodes or nodes that are weakly but directly connected to the input entities.

If the current semantic entity is not located in the AKG 112, processing goes to step 312. Here, the knowledge retrieval module 130 may invoke knowledge extension module 120 to make use of one or more general knowledge graphs (GKGs) 122. The knowledge extension module 120 may interface the AKG 112 with an external general knowledge graph (GKG) 122 to retrieve nodes of the AKG 112 that are relevant to the current semantic entity of the user input, despite the fact that the current semantic entity cannot be positioned in the AKG 112. For example, a simple lookup operation may be performed to locate the semantic unit in the GKG 122. Next, nearby nodes (as defined by the degree of semantic relatedness) of the GKG 122 may be searched until more or more nodes that are also present in the AKG 112 are found, for example in accordance with a relatedness criterion (e.g. the nearest available AKG node(s), or all nodes that are within a threshold degree of relatedness). The one or more AKG nodes may then be used to generate the next response to the user 102.

The qualitative and quantitative relations constructed in the AKG 112 enable connecting and integrating it into a wide variety of GKGs 122. To this end, the knowledge extension module 120 may take the nodes of AKG 112 as input, and locate them within GKG 122.

Returning to Figure 3, whenever the current semantic entity of the user input cannot be located in the agent knowledge graph 112, the knowledge extension module 120 may search, at step 312, an external general knowledge graph 122 to locate the user input. If the user input cannot be found in the GKG 122, it may be skipped, with processing returning to 306.

Next, at step 314, the GKG 122 is searched to identify a node that is common to both the AKG 112 and the GKG 122 and that is qualitatively relevant to and quantitatively most proximate to the current semantic entity, according to their qualitative relations and semantic similarities (equivalently, distances) in the greater GKG 122. Qualitative relevance to the current semantic entity may be determined by checking whether a quantitatively proximate node has a relation to the current node such as "isA", "hasA", "synonymOf", or "relatedTo". An example is illustrated in Figure 6, where a user input corresponding to "w" is not found in the AKG 112, but is found in the GKG 122. By searching the GKG 122, a node corresponding to user input "w" is located, and is found to be closest to a node "d" that is common to the AKG 112 and the GKG 122. Accordingly, the knowledge extension module 120 may return "d" as the related node.

If it is determined at step 316 that there is further user input to process, processing returns to step 306.

Otherwise, if it is determined at step 316 that all semantic entities of the user input have been processed, the knowledge retrieval module 130 may generate, at step 318, a response for the current conversation turn, by combining information from all of the related nodes located by searching the AKG 112 and (if applicable) the GKG 122. After outputting the response, the process 300 may return to step 302 to receive further user input for the next conversation turn (iteration).

For example, the knowledge retrieval module 130 may simply output, via browser interface 104 or other suitable means, the semantic entities that correspond to all of the located related nodes. Alternatively, the knowledge retrieval module 130 may generate a natural language response from the information of the located related nodes. For example, as mentioned above, rule-based techniques may use entities (E) of the related nodes, and their POS tags, to form a response; or extractive techniques may be used to select one or more sentences selected from expert documents considering the quantitative relatedness of (E) to the sentence(s).

In some embodiments, the natural language response may be output by passing it to a text-to-speech engine, to give the user 102 the impression of conducting a real conversation with the expert.

Accordingly, it can be seen that the process 300 enables a system 100 implementing a conversational agent to estimate a new direction for a conversation in the AKG 112, starting from the semantic entities identified in the user's input.

Based on the AKG positions of the current inputs and the inputs and responses in prior conversation iterations to estimate a new direction for the conversation in the knowledge graph. Based on the new conversation direction, a response is created using the semantic entities in the direction pointed in the agent knowledge graph 112. For instance, the agent may respond with the semantic elements that are most similar or proximate to the user inputs according to their positions in the knowledge graph. These semantic elements may be used to create natural language responses that can be abstracted from the knowledge sources used to create the knowledge graph.

In some embodiments, the knowledge retrieval module 130 may track the brainstorming session implemented in process 300 in terms of the positions of covered semantic entities in the agent knowledge graph 112. If the module 130 detects that the conversation sticks to a limited subspace in the agent knowledge graph 112 more than a predefined number of conversation-turns, the module may break the convergence by offering a distant response comparing to the ones given in a predefined number of previous conversation- turns.

For example, depending on the size (or radius) of the vector space created by the vector representations of the entities in AKG 112, a threshold (T) may be defined. In each turn of conversation, we check if cr(t) > T where cr(t) is the change rate variable (defined above). If cr(t) < T for N turns, then the system multiplies cr(t) with a predefined variable M, with M > 1, to break the convergence and allow the selection of more loosely related (i.e., more distant) entities to be used in constructing the response.

Figure 4 shows an example computing device 400 that is capable of implementing a system 100 for conducting a brainstorming session with a virtual expert. In some embodiments, multiple computing devices 400 may be considered to be a single such system.

The components of the computing device 400 can be configured in a variety of ways. The components can be implemented entirely by software to be executed on standard computer server hardware, which may comprise one hardware unit or different computer hardware units distributed over various locations, which may communicate over a network. Some of the components or parts thereof may also be implemented by application specific integrated circuits (ASICs) or field programmable gate arrays.

In the example shown in Figure 4, the computing device 400 is a commercially available server computer system based on a 32 bit or a 64 bit Intel architecture, and the processes and/or methods executed or performed by the computing device 400 are implemented in the form of programming instructions of one or more software components or modules 420 stored on non-volatile (e.g., hard disk) computer-readable storage 424 associated with the computing device 400. At least parts of the software modules 420 could alternatively be implemented as one or more dedicated hardware components, such as application-specific integrated circuits (ASICs) and/or field programmable gate arrays (FPGAs).

The computing device 400 includes at least one or more of the following standard, commercially available, computer components, all interconnected by a bus 435:

(a) random access memory (RAM) 426;

(b) at least one computer processor 428, and

(c) a network interface connector (NIC) 430 which connects the computer device 400 to a data communications network and/or to external devices.

The computing device 400 includes a plurality of standard software modules, including:

(a) an operating system (OS) 436 (e.g., Linux or Microsoft Windows);

(b) web server software 438 such as Apache, available at http://www.apache.org;

(c) scripting language support 440 such as PHP, available at http://www.php.net, or Microsoft ASP; and

(d) a graph database management module, such as GraphDB or neo4j 442.

Together, the web server 438 and scripting language module 440 provide the computing device 400 with the general ability to allow users of the Internet with standard computing devices equipped with standard web browser software (such as browser 104 indicated in Figure 1) to access the system 100 to provide input to, and receive responses from, the various modules of the system 100, including knowledge retrieval module 130. However, it will be understood by those skilled in the art that the specific functionality provided by the computing device 400 to such users is provided by scripts accessible by the web server 438, including the one or more software modules 400 implementing the processes, and also any other supporting scripts and data (not shown), including markup language (e.g., HTML, XML) scripts, PHP (or ASP), and/or CGI scripts, image files, style sheets, and the like.

Data retrieved and/or processed during processes 200 and 300 may be stored locally on storage medium 424. For example, source document database 108, agent knowledge graph(s) 112, and general knowledge graph(s) 122 may be stored locally. The graph database management module 442 may be used to retrieve data from, and write data to, one or more graph databases that store the agent knowledge graph(s) 112 and general knowledge graph(s) 122.

The boundaries between the modules and components in the software modules 420 are exemplary, and alternative embodiments may merge modules or impose an alternative decomposition of functionality of modules. For example, the modules discussed herein may be decomposed into submodules to be executed as multiple computer processes, and, optionally, on multiple computers. Moreover, alternative embodiments may combine multiple instances of a particular module or submodule. Furthermore, the operations may be combined or the functionality of the operations may be distributed in additional operations in accordance with the invention. Alternatively, such actions may be embodied in the structure of circuitry that implements such functionality, such as the micro-code of a complex instruction set computer (CISC), firmware programmed into programmable or erasable/programmable devices, the configuration of a field-programmable gate array (FPGA), the design of a gate array or full-custom application-specific integrated circuit (ASIC), or the like. Each of the blocks of the flow diagrams of processes 200 and 300 performed by the system 100 may be executed by a module (of software modules 420) or a portion of a module. The processes may be embodied in a non-transient machine-readable and/or computer-readable medium for configuring a computer system to execute the method. The software modules may be stored within and/or transmitted to a computer system memory to configure the computer system to perform the functions of the module.

For example, as shown in Figure 4, the modules 420 may include the knowledge representation module 110, knowledge extension module 120, and knowledge retrieval module 130, having functionality as described above.

The computing device 400 normally processes information according to a program (a list of internally stored instructions such as a particular application program and/or an operating system) and produces resultant output information via input/output (I/O) devices 430. A computer process typically includes an executing (running) program or portion of a program, current program values and state information, and the resources used by the operating system to manage the execution of the process. A parent process may spawn other, child processes to help perform the overall functionality of the parent process. Because the parent process specifically spawns the child processes to perform a portion of the overall functionality of the parent process, the functions performed by child processes (and grandchild processes, etc.) may sometimes be described as being performed by the parent process.

Figures 5 and 6 illustrate the interaction of a user in certain embodiments of the invention. Figure 5 is an illustration of an example use case. Documents of an individual, multiple individuals or a topic or field contributed by many individuals are mined to create an agent knowledge graph (AKG) 112 as described above. Figure 6 is an example in which the proposed technology needs to retrieve knowledge from a general knowledge graph (GKG) 122. Using the external knowledge in the GKG 122 and their relations to the knowledge of the expert in the AKG 112, a suitable response is drawn within the AKG 112. The input of a user is positioned in the AKG to retrieve elements of the AKG according to qualitative relations and semantic similarities to respond to the user. When a user provides questions or general semantic inputs, the computer will respond with new but relevant semantic elements or natural language text abstracted from a plurality of documents associated with the expert (or the topic or field) using these new semantic elements to the user, according to their qualitative relations and their semantic similarities with the semantic inputs in the knowledge graph.

Figure 5 demonstrates the case that the inputs "x", "y" and "z" of a user can be fully positioned in the AKG 112 to retrieve elements "a", "b" and "c" of the AKG 112, according to qualitative relations and semantic similarities of nearby nodes in the AKG 112, to respond to the user.

Figure 6 demonstrates the case that some inputs, such as input "w", of a user fall outside the AKG 112 and thus a GKG 122 is used together with the AKG 112 to retrieve the elements of the AKG 112 according to their qualitative and quantitative relations to the inputs in the GKG 122 to generate responses to the user.

In turn, the computer-generated semantic responses can inspire the user to come up with new questions and new semantic inputs. The computer will then retrieve the elements of the knowledge graphs again for responses to the user. With such iterations going on, the user will be able to expansively probe the elements through the semantic structure of the agent knowledge graph built on the data of a specific individual, multiple individuals, or the data of a specific topic or field contributed from many individuals, and thus tap into the knowledge of the computer-represented knowledge expert.

Embodiments of the invention provide a data-driven knowledge graph to represent the knowledge of an individual person, multiple individual persons (or the knowledge of a topic or field from many individual persons), and use the agent knowledge graph for facilitating expansive retrieval of the knowledge of the represented persons (or the knowledge of the topic or field) via brainstorming-like iterative interactions between the knowledge graph and a real person. Advantageously, embodiments may create AI agents that mimic knowledgeable experts to brainstorm and share knowledge with many human users who may need or benefit from the knowledge of the experts for learning, inspiration and problem solving. An expert often has limited time and accessibility to communicate with other individuals to inspire them, despite the value of his/her knowledge for others. With embodiments of the invention, a computer agent that has the same knowledge of the expert can do so, on behalf of the expert, and tirelessly. It will be appreciated that many further modifications and permutations of various aspects of the described embodiments are possible. Accordingly, the described aspects are intended to embrace all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims.

Throughout this specification and the claims which follow, unless the context requires otherwise, the word "comprise", and variations such as "comprises" and "comprising", will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps.

The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that that prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates.

Claims

1. A computer-implemented method of conducting a brainstorming session with a virtual expert, including :

(i) obtaining a first knowledge graph including : nodes that are semantic entities in an expert vocabulary, and edges that are weighted according to semantic similarity between the entities and that are labelled with one or more qualitative relations between the entities, said expert vocabulary being generated from a plurality of documents associated with, and/or authored by, one or more human experts;

(ii) receiving input text from a user;

(v) determining one or more suggested first semantic entities from the first knowledge graph based on their semantic relatedness to the user- provided semantic entities; and

2. A computer-implemented method according to claim 1, wherein the plurality of documents is associated with, and/or authored by, a single human expert.

3. A computer-implemented method according to claim 1 or claim 2, wherein the response is a natural language response constructed from the one or more suggested first semantic entities.

4. A computer-implemented method according to any one of claims 1 to 3, wherein the one or more suggested first semantic entities are determined based on a high semantic similarity to the one or more user-provided semantic entities.

5. A computer-implemented method according to any one of claims 1 to 4, wherein operations (ii)-(vi) are performed iteratively.

6. A computer-implemented method according to claim 5, including, after a predetermined number of iterations:

setting a distance threshold;

if the second location is less than the distance threshold from the first location, increasing the distance threshold.

7. A computer-implemented method according to any one of claims 1 to 6, including:

determining, in operation (iv), that one or more of the one or more user- provided semantic entities do not exist in the first knowledge graph; and for each said non-located user-provided semantic entity, searching a second knowledge graph to suggest one or more second semantic entities based on semantic relatedness to the non-located user-provided semantic entity, wherein the second knowledge graph is a general knowledge graph that is at least partly based on documents other than the documents associated with, and/or authored by, the one or more human experts.

8. A computer-implemented method of generating a knowledge graph for one or more human experts, including :

training a language model from the expert vocabulary; and

9. A method according to claim 8, wherein the plurality of documents are associated with, and/or authored by, a single human expert.

10. A system for conducting a brainstorming session with a virtual expert, comprising:

a knowledge retrieval module that is configured to:

(i) obtain a first knowledge graph including : nodes that represent semantic entities in an expert vocabulary, and edges that are weighted according to semantic similarity between the entities and that are labelled with one or more qualitative relations between the entities, said expert vocabulary being generated from a plurality of documents associated, and/or authored by, one or more human experts;

(ii) receive input text from a user;

(iv) search the first knowledge graph to determine a position or positions of the one or more user-provided semantic entities in the first knowledge graph;

(v) determine one or more suggested first semantic entities from the first knowledge graph based on their semantic relatedness to the user- provided semantic entities; and (vi) display, to said user, a response including the one or more suggested first semantic entities.

11. A system according to claim 10, wherein the plurality of documents is associated with, and/or authored by, a single human expert.

12. A system according to claim 10 or claim 11, wherein the response is a natural language response constructed from the one or more suggested first semantic entities.

13. A system according to any one of claims 10 to 12, wherein the knowledge retrieval module is configured to determine the one or more suggested first semantic entities based on a high semantic similarity to the one or more user-provided semantic entities.

14. A system according to any one of claims 10 to 13, wherein the knowledge retrieval module is configured to perform operations (ii)-(vi) iteratively.

15. A system according to claim 14, wherein the knowledge retrieval module is configured to, after a predetermined number of iterations:

set a distance threshold;

determine a second location corresponding to a centroid or medoid of vector representations of the user-provided semantic entities and the one or more suggested first semantic entities in a current iteration; and if the second location is less than the distance threshold from the first location, increase the distance threshold.

16. A system according to any one of claims 10 to 15, including a knowledge extension module that is configured to: determine, in operation (iv), that one or more of the one or more user- provided semantic entities do not exist in the first knowledge graph; and for each said non-located user-provided semantic entity, search a second knowledge graph to suggest one or more second semantic entities based on semantic relatedness to the non-located user-provided semantic entity, wherein the second knowledge graph is a general knowledge graph that is at least partly based on documents other than the documents associated with, and/or authored by, the one or more human experts.

17. A system according to any one of claims 10 to 16, including a knowledge representation module that is configured to generate the first knowledge graph by: