US20220207067A1 - User-customized question-answering system based on knowledge graph - Google Patents
User-customized question-answering system based on knowledge graph Download PDFInfo
- Publication number
- US20220207067A1 US20220207067A1 US17/697,614 US202217697614A US2022207067A1 US 20220207067 A1 US20220207067 A1 US 20220207067A1 US 202217697614 A US202217697614 A US 202217697614A US 2022207067 A1 US2022207067 A1 US 2022207067A1
- Authority
- US
- United States
- Prior art keywords
- question
- knowledge graph
- entity
- user
- natural language
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004458 analytical method Methods 0.000 claims abstract description 21
- 238000007726 management method Methods 0.000 description 22
- 238000010586 diagram Methods 0.000 description 8
- 230000014509 gene expression Effects 0.000 description 4
- 241000270666 Testudines Species 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000000034 method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013499 data model Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/335—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9032—Query formulation
- G06F16/90332—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
Definitions
- a question answering system analyzes a natural language question of a user, analyzes a correct answer type based on a result of the analysis, and generates numerous correct answer candidates through document searches based on core keywords in the question and knowledge base.
- Such an existing question-answering system answers the question by ranking correct answer candidates that are closest to correct answer type and limitation information demanded by the question and the context of the question and inferring one with the highest ranking as a correct answer.
- the present disclosure provides a customized question-answering system based on a knowledge graph capable of answering questions accurately.
- the knowledge graph management unit may include a context knowledge graph generating unit configured to extract the entity knowledge graphs including first-order relationship triples using entities of the question as subjects obtained from the natural language analysis result for the question and generate a context knowledge graph including the entity knowledge graphs; and a knowledge graph inquiring unit configured to select at least one entity knowledge graph from among entity knowledge graphs included in the context knowledge graph by using a property to be obtained from the question, which is a question classification result for the question.
- the question-answering system may further include a user knowledge graph configured to store a user entity knowledge graph, which is an entity knowledge graph used to generate the answer to be transmitted to the user from among the at least one entity knowledge graph selected above.
- the question target ranking unit may generate the answer from the one entity knowledge graph.
- the question target ranking unit may determine existence of the two or more entity knowledge graphs in the user entity knowledge graph stored in the selected user knowledge graph.
- the question target ranking unit may select the two or more entity knowledge graphs as entity knowledge graphs to be ranked and generate the answer or the reverse question for the question.
- the question target ranking unit may select the at least two entity knowledge graphs as entity knowledge graphs to be ranked and generate the answer or the reverse question for the question.
- the question target ranking unit may generate the answer for the question from the one user entity knowledge graph.
- the question target ranking unit may compare the selected ranking target entity knowledge graphs and determine whether there is an exclusive triple existing in only one entity knowledge graph. When an entity knowledge graph having the exclusive triple has one exclusive triple, the question target ranking unit may generate the reverse question from the one exclusive triple. When the entity knowledge graph having the exclusive triple has two or more exclusive triples, the question target ranking unit may generate the reverse question from an exclusive triple having a high frequency of existence in the integrated knowledge graph or having more objects having triple relationships. When there is no entity knowledge graph having the exclusive triple, the question target ranking unit may generate the reverse question from an overlapping triple having a high frequency of existence in the integrated knowledge graph or having more objects having triple relationships.
- the question target ranking unit may generate the object, which is the same answer, as the answer.
- Each of the entity knowledge graphs may use each of entities obtained from the natural language analysis result for the question as a subject to indicate a first-order relationship with respect to another entity, which is an object, as a triple having the subject-property-object structure.
- the question-answering system may inquire the user 10 by generating a reverse question by using a context knowledge graph before providing an answer for the corresponding question to the user, thereby improving reliability of an answer to be provided to the user, that is, a correct answer for the question.
- FIG. 1 is a block diagram showing the main configuration of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- FIG. 2 is a block diagram showing the main configuration of a natural language understanding unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- FIG. 3 is a block diagram showing the main configuration of a knowledge graph management unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- FIG. 4 is a schematic diagram showing an example of a context knowledge graph generated by a user-customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- FIG. 5 is a flowchart showing a process that question-answering is performed based on a knowledge graph in a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- FIG. 6 is a flowchart for describing an operation of a question target ranking unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- Keywords or information in a question are often insufficient and a small number of correct answer candidates against inference of a correct answer are frequently inferred as correct answers, the reliability of the correct answer cannot be guaranteed.
- a component indicated or described as one block may be a hardware block or a software block.
- the components may be independent hardware blocks that transmits and receives signals with each other or may be software blocks executed on one processor.
- FIG. 1 is a block diagram showing the main configuration of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- a customized question-answering system 1 based on a knowledge graph includes a user interface (UI) 100 that receives a question in the form of a natural language from a user 10 and transmits an answer, through a network 20 .
- UI user interface
- the network 20 may include all types of networks through which data may be transmitted and received, e.g., a wired Internet service, a local area network (LAN), a wide area network (WAN), an intranet, a wireless Internet service, a mobile computing service, a wireless data communication service, a wireless Internet access service, a satellite communication service, a wireless LAN, Bluetooth, etc.
- a wireless data communication service such as 3G, 4G, or 5G, a wireless LAN like Wi-Fi, or Bluetooth.
- An UI 100 may provide an interface for accessing the question-answering system 1 through a terminal used by the user 10 .
- the user 10 may transmit a reply for a question and a reverse question to the question-answering system 1 through the UI 100 and receive a reverse question and a reply for a question provided by the question-answering system 1 through the UI 100 .
- a ‘question’ refers to a question transmitted by the user 10 to the question-answering system 1 to obtain a correct answer from the question-answering system 1
- a ‘reverse question’ refers to a question that the question-answering system 1 transmits to the user 10 to accurately understand the ‘question’ of the user 10 .
- an ‘answer’ refers to an answer that is inferred by the question-answering system 1 as a correct answer for a question of the user 10 and transmitted to the user 10 by the question-answering system 1
- a ‘reply’ i.e., a reply to a reverse question
- a ‘reply’ refers to an answer transmitted by the user 10 to the question-answering system 1 for a reverse question transmitted from the question-answering system 1 .
- the question answering system 1 further includes a natural language understanding unit 200 capable of analyzing a question in the form of a natural language received by the UI 100 and a module server 300 that manages questions analyzed by the natural language understanding unit 200 and generates an answer or a reverse question for a question by using the data server 400 .
- a natural language understanding unit 200 capable of analyzing a question in the form of a natural language received by the UI 100
- a module server 300 that manages questions analyzed by the natural language understanding unit 200 and generates an answer or a reverse question for a question by using the data server 400 .
- the natural language understanding unit 200 may include a natural language analyzing unit 210 and a question classifying unit 220 .
- the natural language analyzing unit 210 may provide a natural language analysis result for a question of the user 10 as structured data having a table structure or a tree structure.
- Structured data that may be provided by the natural language analyzing unit 210 may be, for example, RDB (Relation DateBase), CSV (Comma-Seperated Variables), XML (eXtensible Markup Language), or JSON (JavaScript Object Notation), but the present disclosure is not limited thereto.
- the natural language analyzing unit 210 may separate a sentence constituting a question of the user 10 in the form of a natural language into morphemes and perform syntax analysis and entity name analysis on the sentence constituting the question separated into morphemes. Also, the natural language analyzing unit 210 may generate a simplified sentence by removing unnecessary features from the sentence constituting the question, classify intention and analyze domain of the question to which a semantic role is labeled based on a generated simplified sentence, and label a semantic role for the sentence constituting the question. The natural language analyzing unit 210 will be described below in detail with reference to FIG. 2 .
- the question classifying unit 220 classifies the question of the user 10 with reference to a natural language analysis result regarding the question of the user 10 in the form of a natural language performed by the natural language analyzing unit 210 .
- the question classifying unit 220 may classify the question of the user 10 , thereby obtaining a property to be obtained from the question of the user 10 .
- the property may also be referred to as a predicate.
- the question classifying unit 220 may classify, for example, that a question ‘what is the age of Yi Sun-shin?’ is to obtain an age property of a person and classify that a question ‘what is the height of Mt. Baekdu?’ is to obtain a height property of an object. In other words, the question classifying unit 220 may classify which property to search for in a triple of knowledge graphs in the form of subject-property-object.
- the module server 300 may include a knowledge graph management unit 310 , a question target ranking unit 320 , and a natural language generating unit 330 .
- the data server 400 may include an integrated knowledge graph 410 and a user knowledge graph 430 .
- the data server 400 may store a context knowledge graph 420 generated by the knowledge graph management unit 310 with reference to the integrated knowledge graph 410 .
- the knowledge graph management unit 310 may manage and update the integrated knowledge graph 410 .
- the integrated knowledge graph 410 may have a space for storing a knowledge graph built within the question-answering system 1 .
- the integrated knowledge graph 410 may have a space for storing data in any form, such as NoSQL, a relational database, or a file system.
- the integrated knowledge graph 410 may have one logically divided storage device, a division unit that logically separates one or a plurality of storage devices, or a portion of one physically divided storage device or one logically divided storage device.
- the integrated knowledge graph 410 may be a management module that is connected to at least one knowledge graph built outside the question-answering system 1 through the network 20 and manage the at least one knowledge graph built outside the question-answering system 1 .
- the integrated knowledge graph 410 may have a management module that integrates and manages a space for storing knowledge graphs built inside the question-answering system 1 with the at least one knowledge graph built outside the question-answering system 1 .
- a knowledge graph refers to a semantic graph, that is, a graph representing a semantic graph between entities in order to accumulate semantic information from various sources to improve search results.
- Semantic information refers to a relationship between resources expressed into information in a certain form.
- a knowledge graph indicates that semantic information is collected and data thereof is stored in the form of a graph.
- a knowledge graph may be built by using, for example, a resource description framework (RDF).
- RDF includes a model, a language, and a grammar for describing properties, characteristics, and relationships of all resources (web pages, images, videos, etc.) having uniform resource identifiers (URIs) and is a graph-type data model.
- the RDF expresses a relationship between two resources, that is, a subject and an object, and a property or a predicate describes features and characteristics of the relationship.
- a sentence having a subject-property-object structure is referred to as a triple, and a relationship therebetween may have directionality.
- a knowledge graph may include a multi-order relationship triple in which indication of a relationship between one entity and another entity as a triple having a subject-property-object structure by using the one entity as a subject and the other entity as an object and indication of a relationship between the other entity and yet another entity as a triple having the subject-property-object structure by using the other entity as the subject and the yet other entity as the object are repeated.
- the knowledge graph management unit 310 may generate the context knowledge graph 420 from the integrated knowledge graph 410 by using entities of a question of the user 10 extracted and analyzed by the natural language understanding unit 200 .
- the context knowledge graph 420 may be generated by cutting out only first-order relationship triples from the integrated knowledge graph 410 based on the entities of an extracted question of the user 10 .
- the context knowledge graph 420 may include first-order relationship triples that use each of entities of an extracted question of the user 10 as a subject to indicate a relationship with respect to another entity, which is an object, as a triple having the subject-property-object structure.
- a first-order relationship triple generated based on one entity is referred to as an entity knowledge graph
- all of entity knowledge graphs generated from entities of the question of the user 10 or a space in which entity knowledge graphs are stored is referred to as the context knowledge graph 420 .
- the context knowledge graph 420 for one question of the user 10 may include entity knowledge graphs respectively generated from entities extracted from the one question of the user 10 .
- an entity knowledge graph used in response to the question of the user 10 may be stored.
- the user knowledge graph 430 may store a plurality of entity knowledge graphs used for answers as questions of the user 10 are repeated. Entity knowledge graphs stored in the user knowledge graph 430 may be referred to as user entity knowledge graphs.
- the knowledge graph management unit 310 may generate a query regarding a question of the user 10 by using a result of classification for questions of the user 10 by the question classifying unit 220 , that is, a property to be obtained from the question of the user 10 .
- a ‘query’ refers to a computer language used to request information retrieval.
- the context knowledge graph 420 may receive a query for a question of the user 10 generated by the knowledge graph management unit 310 and transmit at least one entity knowledge graph corresponding to the query for the question of the user 10 from among a plurality of entity knowledge graphs included in the context knowledge graph 420 and a plurality of user entity knowledge graphs included in the user knowledge graph 430 to the knowledge graph management unit 310 .
- the question target ranking unit 320 may generate an answer or a reverse question for the question of the user 10 by using the at least one entity knowledge graph corresponding to the question of the user 10 .
- the natural language generating unit 330 may generate an answer or a reverse question for the question of the user 10 generated by the question target ranking unit 320 in the form of a natural language. In some embodiments, the natural language generating unit 330 may generate an answer or a reverse question for the question of the user 10 as a plurality of candidates in various expressions in the form of a natural language.
- the question-answering system 1 may further include an answer ranking unit 500 .
- the answer ranking unit 500 determines a ranking for a plurality of candidates, in the form of a natural language, of an answer or a reverse question for the question of the user 10 generated by the natural language generating unit 330 , the answer ranking unit 500 may select a candidate with a high ranking and provide the selected candidate as an answer or a reverse question to the user 10 through the UI 100 .
- the user 10 may transmit a reply for a reverse question to the question-answering system 1 through the UI 100 , and the question-answering system 1 may synthesize the question of the user 10 with the reply for the reverse question and provide an answer to the user 10 or provide another reverse question by repeating the above-stated steps.
- An entity knowledge graph used to generate an answer provided to the user 10 may be stored in the user knowledge graph 430 .
- the question-answering system 1 may inquire the user 10 by generating a reverse question before providing an answer for the corresponding question to the user 10 , thereby improving reliability of an answer to be provided to the user 10 , that is, a correct answer for the question.
- FIG. 2 is a block diagram showing the main configuration of a natural language understanding unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- the natural language understanding unit 200 may include a natural language understanding unit 210 that interprets a question in the form of a natural language received by the UI 100 .
- the natural language understanding unit 210 may include a morpheme analyzing unit 211 , a syntax analyzing unit 212 , an entity name analyzing unit 213 , a filtering analyzing unit 214 , an intention classifying unit 215 , a domain analyzing unit 216 , and a semantic role labeling unit (SRL) 217 .
- SRL semantic role labeling unit
- the morpheme analyzing unit 211 may separate a sentence constituting a question into morphemes.
- the syntax analyzing unit 212 and the entity name analyzing unit 213 may perform syntax analysis and entity name analysis on the sentence that constitutes the question and is separated into morphemes.
- the filtering analyzing unit 214 may generate a simplified sentence by removing unnecessary features from the sentence constituting the question.
- the intention classifying unit 215 and the domain analyzing unit 216 may perform intention classification and domain analysis for the question to which a semantic role is assigned based on the simplified sentence generated by the filtering analyzing unit 214 .
- the SRL 217 may assign a semantic role to the sentence constituting the question.
- FIG. 3 is a block diagram showing the main configuration of a knowledge graph management unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- the natural language understanding unit 200 may include the natural language analyzing unit 210 and the question classifying unit 220 .
- the module server 300 may include the knowledge graph management unit 310 , the question target ranking unit 320 , and the natural language generating unit 330 .
- the data server 400 may include the integrated knowledge graph 410 , the context knowledge graph 420 , and the user knowledge graph 430 .
- the knowledge graph management unit 310 may include an integrated knowledge graph management unit 312 , a context knowledge graph generating unit 314 , and a knowledge graph inquiring unit 316 .
- the integrated knowledge graph management unit 312 may manage and update the integrated knowledge graph 410 .
- the integrated knowledge graph 410 may have a space for storing a knowledge graph built within a question-answering system ( 1 of FIG. 1 ).
- the integrated knowledge graph 410 may have a space for storing data in any form, such as NoSQL, a relational database, or a file system.
- the integrated knowledge graph 410 may have one logically divided storage device, a division unit that logically separates one or a plurality of storage devices, or a portion of one physically divided storage device or one logically divided storage device.
- the integrated knowledge graph 410 may be a management module that is connected to at least one knowledge graph built outside a question-answering system ( 1 of FIG. 1 ) through a network ( 20 of FIG. 1 ) and manage the at least one knowledge graph built outside the question-answering system ( 1 of FIG. 1 ).
- the integrated knowledge graph 410 may have a management module that integrates and manages a space for storing knowledge graphs built inside a question-answering system ( 1 of FIG. 1 ) with the at least one knowledge graph built outside the question-answering system ( 1 of FIG. 1 ).
- the knowledge graph inquiring unit 316 may generate a query regarding a question of the user 10 by using a result of classification for questions of the user 10 by the question classifying unit 220 , that is, a property to be obtained from the question of the user 10 .
- the context knowledge graph 420 may receive a query for the question of the user 10 generated by the knowledge graph question unit 316 and transmit at least one entity knowledge graph corresponding to the query for the question of the user 10 from among a plurality of entity knowledge graphs included in the context knowledge graph 420 to the knowledge graph management unit 310 .
- the question target ranking unit 320 may generate an answer or a reverse question for the question of the user 10 by using the at least one entity knowledge graph corresponding to the question of the user 10 .
- the natural language generating unit 330 may generate an answer or a reverse question for the question of the user 10 generated by the question target ranking unit 320 in the form of a natural language.
- FIG. 4 is a schematic diagram showing an example of a context knowledge graph generated by a user-customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- FIG. 4 shows an example of a part of a context knowledge graph generated when a question of as user ( 10 of FIG. 1 ) is ‘Where is the birthplace of Yi Sun-Sin?’.
- the natural language analyzing unit 210 may extract ‘Yi Sun-Sin’ and ‘birthplace’ as entities and transmit them to the context knowledge graph generating unit 314 .
- the context knowledge graph generating unit 314 may extract a plurality of entity knowledge graphs EKG 1 and EKG 2 , which are first-order relationship triples, by using each of ‘Yi Sun-Sin’ and ‘birthplace’ as a subject and at least one other entity having a first-order relationship (i.e., at least one entity having a property) as an object, thereby generating a context knowledge graph CKG.
- the context knowledge graph CKG may further include first-order relationship triples using ‘birthplace’ as the subject.
- the context knowledge graph CKG may correspond to the context knowledge graph 420 shown in FIG. 1 .
- the context knowledge graph CKG may include a first entity knowledge graph EKG 1 , which is a first-order relationship triple using Yi Sun-Sin the Chungmugong E 1 as the subject and a second entity knowledge graph EKG 2 , which is a first-order relationship triple using Yi Sun-Sin the Muwuigong E 2 as the subject.
- the first entity knowledge graph EKG 1 may use the Yi Sun-Sin the Chungmugong E 1 as the subject and may have objects respectively having first-order triple relationships T 11 , T 12 , T 13 , T 14 , and T 15 with respect to properties like nationality (Nationality), birthplace (BornIn), job (Job), creation (Create), and spouse (Spouse).
- the second entity knowledge graph EKG 2 may use the Yi Sun-Sin the Muwuigong E 2 as the subject and may have objects respectively having first-order triple relationships T 21 , T 22 , and T 23 with respect to properties like nationality (Nationality), birthplace (BornIn), and job (Job).
- FIG. 5 is a flowchart showing a process that question-answering is performed based on a knowledge graph in a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- the question classifying unit 220 may classify ‘birthplace’, which is a property to be obtained by a question of a user ( 10 of FIG. 1 ) and transmit the same to the knowledge graph inquiring unit 316 .
- the knowledge graph inquiring unit 316 may generate a query regarding the question of the user 10 by using ‘birthplace’ (BornIn), which is a result of classification, inquire the context knowledge graph CKG (operation S 10 ), and select and receive entity knowledge graphs having the property ‘bornin’ with respect to ‘Yi Sun-Sin’, which is the subject, that is, the first entity knowledge graph EKG 1 and the second entity knowledge graph EKG 2 (operation S 110 ).
- the inquiring the context knowledge graph CKG (operation S 10 ) means that a query, which is a computer language used for a request for information collection, is transmitted to the context knowledge graph CKG.
- the context knowledge graph CKG has only one entity knowledge graph (e.g., the first entity knowledge graph EKG 1 ) having ‘birthplace’ (BornIn) as the property with respect to ‘Yi Sun-Sin’, which is the subject, Geoncheon-dong of Hansung T 12 , which is the ‘birthplace’ (BornIn) of Yi Sun-Sin the Chungmugong E 1 , may be inferred as a correct answer and provided to the user 10 (operation S 20 ).
- entity knowledge graph EKG 1 has only one entity knowledge graph (e.g., the first entity knowledge graph EKG 1 ) having ‘birthplace’ (BornIn) as the property with respect to ‘Yi Sun-Sin’, which is the subject, Geoncheon-dong of Hansung T 12 , which is the ‘birthplace’ (BornIn) of Yi Sun-Sin the Chungmugong E 1 , may be inferred as a correct answer and provided to the user 10 (operation S 20 ).
- the question target ranking unit 320 determines the existence of the first entity knowledge graph EKG 1 and the second entity knowledge graph EKG 2 from among user entity knowledge graphs, which are included in the user knowledge graph 430 and used to answer previous questions of the user 10 (operation S 120 ).
- the first entity knowledge graph EKG 1 may be selected from the user knowledge graph 430 (operation S 150 ) and Geoncheon-dong of Hansung T 12 , which is the birthplace of Yi Sun-Sin the Chungmugong E 1 , may be inferred as a correct answer and provided to the user 10 as an answer (operation S 20 ).
- the question target ranking unit 320 may determine a ranking for a question target with reference to the integrated knowledge graph 410 (operation S 130 ), provide a reverse question to the user 10 , and receive a result thereof (operation S 140 ).
- FIG. 6 is a flowchart for describing an operation of a question target ranking unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure.
- the question target ranking unit 320 compares the first entity knowledge graph EKG 1 with the second entity knowledge graph EKG 2 having the same property and determines whether there is an exclusive triple included in only one of the first entity knowledge graph EKG 1 and the second entity knowledge graph EKG 2 (operation S 310 ).
- first entity knowledge graph EKG 1 and the second entity knowledge graph EKG 2 have overlapping triples having objects having first-order triple relationships T 11 , T 12 , T 13 and first-order triple relationships T 21 , T 22 , and T 23 for the properties nationality, bornin, and job, only the first entity knowledge graph EKG 1 has an exclusive triple having the object having first-order triple relationships T 14 and T 15 for the properties create and spouse.
- an exclusive triple for a reverse question may be selected from the two or more exclusive triples T 14 and T 15 (operation S 320 ).
- the exclusive triple for a reverse question from among objects of the exclusive triples T 14 and T 15 , one with a high frequency of existence in the integrated knowledge graph 410 may be selected. In some other embodiments, one having more objects having triple relationships for the objects of the exclusive triples T 14 and T 15 as the subject may be selected as an exclusive triple for a reverse question.
- an exclusive triple having turtle ship T 14 as the property creation (Create) for the subject of Yi Sun-Sin the Chungmugong E 1 may be selected as an exclusive triple for a reverse question (operation S 320 ) and a reverse question may be provided to a user ( 10 of FIG. 1 ) (operation S 40 ).
- the first entity knowledge graph EKG 1 and the second entity knowledge graph EKG 2 may have overlapping triples having objects having first-order triple relationships T 11 , T 12 , T 13 , and ‘Chungmugong’ and first-order triple relationships T 21 , T 22 , T 23 , and ‘Muwuigong’ for properties of nationality (Nationality), birthplace (BornIn), job (Job), and artname (Artname). From among them, Joseon T 11 and Joseon T 2 and general T 13 and general T 23 have the same object, they are excluded from overlapping triple candidates for a reverse question, and Geoncheon-dong of Hansung T 12 and Siheung of Gyeonggido T 22 may be overlapping triple candidates for a reverse question.
- an overlapping triple corresponding to ‘Chungmugong’ may be selected as an overlapping triple for a reverse question (operation S 330 ) and a reverse question may be provided to a user ( 10 of FIG. 1 ) (operation S 40 ).
- an answer provided to the user 10 (operation S 20 ) or a reverse question provided to the user 10 (operation S 40 ) may be generated in the form of a natural language by the natural language generating unit 330 .
- the natural language generating unit 330 may generate an answer or a reverse question for the question of the user 10 as a plurality of candidates in various expressions in the form of a natural language.
- the answer ranking unit 500 may select a candidate with a high ranking and provide the selected candidate as an answer or a reverse question to the user 10 through the UI 100 .
- the question-answering system 1 may inquire the user 10 by generating a reverse question by using a context knowledge graph before providing an answer for the corresponding question to the user 10 , thereby improving reliability of an answer to be provided to the user 10 , that is, a correct answer for the question.
Abstract
Description
- The present application a continuation application of International Patent Application No. PCT/KR2019/013713, filed on Oct. 18, 2019, which claims priority to Korean patent application No. 10-2019-0114970 filed on Sep. 18, 2019, contents of each of which are incorporated herein by reference in their entireties.
- The present disclosure relates to a question-answering system, and more particularly, to a question-answering system capable of performing customized question-answering (QA) based on a knowledge graph.
- A question answering system analyzes a natural language question of a user, analyzes a correct answer type based on a result of the analysis, and generates numerous correct answer candidates through document searches based on core keywords in the question and knowledge base. Such an existing question-answering system answers the question by ranking correct answer candidates that are closest to correct answer type and limitation information demanded by the question and the context of the question and inferring one with the highest ranking as a correct answer.
- The present disclosure provides a customized question-answering system based on a knowledge graph capable of answering questions accurately.
- According to an aspect of the present disclosure, question-answering system includes a user interface configured to receive a question or a reply in the form of a natural language from a user and transmit an answer or reverse question to the user, through a network; a natural language understanding unit configured to perform natural language analysis and question classification for the question; an integrated knowledge graph configured to store or manage a knowledge graph including multi-order relationship triples; a knowledge graph management unit configured to generate a context knowledge graph having entity knowledge graphs including first-order relationship triples extracted from the integrated knowledge graph by using a result of the natural language analysis for the question and select at least one entity knowledge graph from among the entity knowledge graphs included in the context knowledge graph by using a result of the question classification; and a question target ranking unit configured to generate the answer or the reverse question for the question from the at least one entity knowledge graph selected above.
- The natural language understanding unit may include a natural language analyzing unit configured to provide a natural language analysis result for the question; and a question classifying unit configured to classify the question by obtaining a property to be obtained from the question by referring to the natural language analysis result for the question.
- The knowledge graph management unit may include a context knowledge graph generating unit configured to extract the entity knowledge graphs including first-order relationship triples using entities of the question as subjects obtained from the natural language analysis result for the question and generate a context knowledge graph including the entity knowledge graphs; and a knowledge graph inquiring unit configured to select at least one entity knowledge graph from among entity knowledge graphs included in the context knowledge graph by using a property to be obtained from the question, which is a question classification result for the question.
- The question-answering system may further include a user knowledge graph configured to store a user entity knowledge graph, which is an entity knowledge graph used to generate the answer to be transmitted to the user from among the at least one entity knowledge graph selected above.
- When one entity knowledge graph is selected by the knowledge graph management unit, the question target ranking unit may generate the answer from the one entity knowledge graph.
- When two or more entity knowledge graphs are selected by the knowledge graph management unit, the question target ranking unit may determine existence of the two or more entity knowledge graphs in the user entity knowledge graph stored in the selected user knowledge graph.
- When none of the two or more entity knowledge graphs exists in the user entity knowledge graph stored in the selected user knowledge graph, the question target ranking unit may select the two or more entity knowledge graphs as entity knowledge graphs to be ranked and generate the answer or the reverse question for the question. When at least two of the two or more entity knowledge graphs exist in the user entity knowledge graph stored in the selected user knowledge graph, the question target ranking unit may select the at least two entity knowledge graphs as entity knowledge graphs to be ranked and generate the answer or the reverse question for the question. When only one of the two or more entity knowledge graphs exists in the user entity knowledge graph stored in the selected user knowledge graph, the question target ranking unit may generate the answer for the question from the one user entity knowledge graph.
- The question target ranking unit may compare the selected ranking target entity knowledge graphs and determine whether there is an exclusive triple existing in only one entity knowledge graph. When an entity knowledge graph having the exclusive triple has one exclusive triple, the question target ranking unit may generate the reverse question from the one exclusive triple. When the entity knowledge graph having the exclusive triple has two or more exclusive triples, the question target ranking unit may generate the reverse question from an exclusive triple having a high frequency of existence in the integrated knowledge graph or having more objects having triple relationships. When there is no entity knowledge graph having the exclusive triple, the question target ranking unit may generate the reverse question from an overlapping triple having a high frequency of existence in the integrated knowledge graph or having more objects having triple relationships.
- When there are overlapping triples having an object that is the same answer for properties to be obtained from the question, which is a result of the question classification performed by the natural language understanding unit, from among overlapping triples shared by the selected entity knowledge graphs to be ranked, the question target ranking unit may generate the object, which is the same answer, as the answer.
- Each of the entity knowledge graphs may use each of entities obtained from the natural language analysis result for the question as a subject to indicate a first-order relationship with respect to another entity, which is an object, as a triple having the subject-property-object structure.
- When a question-answering system according to the present invention fails to derive an answer, that is, a correct answer with high reliability through the initial question of a user, the question-answering system may inquire the
user 10 by generating a reverse question by using a context knowledge graph before providing an answer for the corresponding question to the user, thereby improving reliability of an answer to be provided to the user, that is, a correct answer for the question. -
FIG. 1 is a block diagram showing the main configuration of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. -
FIG. 2 is a block diagram showing the main configuration of a natural language understanding unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. -
FIG. 3 is a block diagram showing the main configuration of a knowledge graph management unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. -
FIG. 4 is a schematic diagram showing an example of a context knowledge graph generated by a user-customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. -
FIG. 5 is a flowchart showing a process that question-answering is performed based on a knowledge graph in a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. -
FIG. 6 is a flowchart for describing an operation of a question target ranking unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. - Keywords or information in a question are often insufficient and a small number of correct answer candidates against inference of a correct answer are frequently inferred as correct answers, the reliability of the correct answer cannot be guaranteed.
- Hereinafter, example embodiments will be described in detail with reference to the accompanying drawings. Example embodiments are provided to more fully explain the disclosure to one of ordinary skill in the art. The disclosure may include various embodiments and modifications, and embodiments thereof will be illustrated in the drawings and will be described herein in detail. However, this is not intended to limit the disclosure to particular modes of practice, and it is to be appreciated that all changes, equivalents, and substitutes that do not depart from the spirit and technical scope of the disclosure are encompassed in the disclosure. Like reference numerals are used for similar elements in describing each drawing.
- The terms used in the present specification are merely used to describe particular embodiments, and are not intended to limit the disclosure. An expression used in the singular encompasses the expression of the plural, unless it has a clearly different meaning in the context. In the present specification, it is to be understood that the terms such as “including” or “having,” etc., are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof disclosed in the specification, and are not intended to preclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof may exist or may be added.
- Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the disclosure belongs. Terms identical to those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art and are not to be interpreted as ideal or overly formal in meaning unless explicitly defined in the present application.
- In the drawings and descriptions below, a component indicated or described as one block may be a hardware block or a software block. For example, the components may be independent hardware blocks that transmits and receives signals with each other or may be software blocks executed on one processor.
- To fully understand the configuration and effect of the present disclosure, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings.
-
FIG. 1 is a block diagram showing the main configuration of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. - Referring to
FIG. 1 , a customized question-answering system 1 based on a knowledge graph (hereinafter referred to as a question-answering system) includes a user interface (UI) 100 that receives a question in the form of a natural language from auser 10 and transmits an answer, through anetwork 20. - The
network 20 may include all types of networks through which data may be transmitted and received, e.g., a wired Internet service, a local area network (LAN), a wide area network (WAN), an intranet, a wireless Internet service, a mobile computing service, a wireless data communication service, a wireless Internet access service, a satellite communication service, a wireless LAN, Bluetooth, etc. When thenetwork 20 is connected to a smartphone or a tablet PC, thenetwork 20 may be a wireless data communication service, such as 3G, 4G, or 5G, a wireless LAN like Wi-Fi, or Bluetooth. - An
UI 100 may provide an interface for accessing the question-answering system 1 through a terminal used by theuser 10. Theuser 10 may transmit a reply for a question and a reverse question to the question-answeringsystem 1 through theUI 100 and receive a reverse question and a reply for a question provided by the question-answeringsystem 1 through theUI 100. - In this specification, unless otherwise specified, a ‘question’ refers to a question transmitted by the
user 10 to the question-answeringsystem 1 to obtain a correct answer from the question-answering system 1, and a ‘reverse question’ refers to a question that the question-answeringsystem 1 transmits to theuser 10 to accurately understand the ‘question’ of theuser 10. - Also, in this specification, unless otherwise specified, an ‘answer’ refers to an answer that is inferred by the question-answering
system 1 as a correct answer for a question of theuser 10 and transmitted to theuser 10 by the question-answering system 1, and a ‘reply’ (i.e., a reply to a reverse question) refers to an answer transmitted by theuser 10 to the question-answering system 1 for a reverse question transmitted from the question-answering system 1. - The
question answering system 1 further includes a natural language understandingunit 200 capable of analyzing a question in the form of a natural language received by theUI 100 and amodule server 300 that manages questions analyzed by the natural language understandingunit 200 and generates an answer or a reverse question for a question by using thedata server 400. - The natural language understanding
unit 200 may include a naturallanguage analyzing unit 210 and aquestion classifying unit 220. - The natural
language analyzing unit 210 may provide a natural language analysis result for a question of theuser 10 as structured data having a table structure or a tree structure. Structured data that may be provided by the naturallanguage analyzing unit 210 may be, for example, RDB (Relation DateBase), CSV (Comma-Seperated Variables), XML (eXtensible Markup Language), or JSON (JavaScript Object Notation), but the present disclosure is not limited thereto. - The natural
language analyzing unit 210 may separate a sentence constituting a question of theuser 10 in the form of a natural language into morphemes and perform syntax analysis and entity name analysis on the sentence constituting the question separated into morphemes. Also, the naturallanguage analyzing unit 210 may generate a simplified sentence by removing unnecessary features from the sentence constituting the question, classify intention and analyze domain of the question to which a semantic role is labeled based on a generated simplified sentence, and label a semantic role for the sentence constituting the question. The naturallanguage analyzing unit 210 will be described below in detail with reference toFIG. 2 . - The
question classifying unit 220 classifies the question of theuser 10 with reference to a natural language analysis result regarding the question of theuser 10 in the form of a natural language performed by the naturallanguage analyzing unit 210. For example, thequestion classifying unit 220 may classify the question of theuser 10, thereby obtaining a property to be obtained from the question of theuser 10. The property may also be referred to as a predicate. - The
question classifying unit 220 may classify, for example, that a question ‘what is the age of Yi Sun-shin?’ is to obtain an age property of a person and classify that a question ‘what is the height of Mt. Baekdu?’ is to obtain a height property of an object. In other words, thequestion classifying unit 220 may classify which property to search for in a triple of knowledge graphs in the form of subject-property-object. - The
module server 300 may include a knowledgegraph management unit 310, a questiontarget ranking unit 320, and a naturallanguage generating unit 330. Thedata server 400 may include anintegrated knowledge graph 410 and auser knowledge graph 430. Thedata server 400 may store acontext knowledge graph 420 generated by the knowledgegraph management unit 310 with reference to theintegrated knowledge graph 410. - The knowledge
graph management unit 310 may manage and update theintegrated knowledge graph 410. In some embodiments, theintegrated knowledge graph 410 may have a space for storing a knowledge graph built within the question-answeringsystem 1. Theintegrated knowledge graph 410 may have a space for storing data in any form, such as NoSQL, a relational database, or a file system. Theintegrated knowledge graph 410 may have one logically divided storage device, a division unit that logically separates one or a plurality of storage devices, or a portion of one physically divided storage device or one logically divided storage device. - In some other embodiments, the
integrated knowledge graph 410 may be a management module that is connected to at least one knowledge graph built outside the question-answeringsystem 1 through thenetwork 20 and manage the at least one knowledge graph built outside the question-answeringsystem 1. In some other embodiments, theintegrated knowledge graph 410 may have a management module that integrates and manages a space for storing knowledge graphs built inside the question-answeringsystem 1 with the at least one knowledge graph built outside the question-answeringsystem 1. - A knowledge graph refers to a semantic graph, that is, a graph representing a semantic graph between entities in order to accumulate semantic information from various sources to improve search results. Semantic information refers to a relationship between resources expressed into information in a certain form. In other words, a knowledge graph indicates that semantic information is collected and data thereof is stored in the form of a graph.
- A knowledge graph may be built by using, for example, a resource description framework (RDF). The RDF includes a model, a language, and a grammar for describing properties, characteristics, and relationships of all resources (web pages, images, videos, etc.) having uniform resource identifiers (URIs) and is a graph-type data model. The RDF expresses a relationship between two resources, that is, a subject and an object, and a property or a predicate describes features and characteristics of the relationship. A sentence having a subject-property-object structure is referred to as a triple, and a relationship therebetween may have directionality.
- For example, a knowledge graph may include a multi-order relationship triple in which indication of a relationship between one entity and another entity as a triple having a subject-property-object structure by using the one entity as a subject and the other entity as an object and indication of a relationship between the other entity and yet another entity as a triple having the subject-property-object structure by using the other entity as the subject and the yet other entity as the object are repeated.
- The knowledge
graph management unit 310 may generate thecontext knowledge graph 420 from theintegrated knowledge graph 410 by using entities of a question of theuser 10 extracted and analyzed by the naturallanguage understanding unit 200. Thecontext knowledge graph 420 may be generated by cutting out only first-order relationship triples from theintegrated knowledge graph 410 based on the entities of an extracted question of theuser 10. In other words, thecontext knowledge graph 420 may include first-order relationship triples that use each of entities of an extracted question of theuser 10 as a subject to indicate a relationship with respect to another entity, which is an object, as a triple having the subject-property-object structure. - In the present specification, for convenience of explanation, a first-order relationship triple generated based on one entity is referred to as an entity knowledge graph, and all of entity knowledge graphs generated from entities of the question of the
user 10 or a space in which entity knowledge graphs are stored is referred to as thecontext knowledge graph 420. In other words, thecontext knowledge graph 420 for one question of theuser 10 may include entity knowledge graphs respectively generated from entities extracted from the one question of theuser 10. - In the
user knowledge graph 430, from amongcontext knowledge graphs 420, an entity knowledge graph used in response to the question of theuser 10 may be stored. Theuser knowledge graph 430 may store a plurality of entity knowledge graphs used for answers as questions of theuser 10 are repeated. Entity knowledge graphs stored in theuser knowledge graph 430 may be referred to as user entity knowledge graphs. - Separately, the knowledge
graph management unit 310 may generate a query regarding a question of theuser 10 by using a result of classification for questions of theuser 10 by thequestion classifying unit 220, that is, a property to be obtained from the question of theuser 10. A ‘query’ refers to a computer language used to request information retrieval. - The
context knowledge graph 420 may receive a query for a question of theuser 10 generated by the knowledgegraph management unit 310 and transmit at least one entity knowledge graph corresponding to the query for the question of theuser 10 from among a plurality of entity knowledge graphs included in thecontext knowledge graph 420 and a plurality of user entity knowledge graphs included in theuser knowledge graph 430 to the knowledgegraph management unit 310. - The question
target ranking unit 320 may generate an answer or a reverse question for the question of theuser 10 by using the at least one entity knowledge graph corresponding to the question of theuser 10. - The natural
language generating unit 330 may generate an answer or a reverse question for the question of theuser 10 generated by the questiontarget ranking unit 320 in the form of a natural language. In some embodiments, the naturallanguage generating unit 330 may generate an answer or a reverse question for the question of theuser 10 as a plurality of candidates in various expressions in the form of a natural language. - In some embodiments, the question-answering
system 1 may further include ananswer ranking unit 500. After theanswer ranking unit 500 determines a ranking for a plurality of candidates, in the form of a natural language, of an answer or a reverse question for the question of theuser 10 generated by the naturallanguage generating unit 330, theanswer ranking unit 500 may select a candidate with a high ranking and provide the selected candidate as an answer or a reverse question to theuser 10 through theUI 100. - The
user 10 may transmit a reply for a reverse question to the question-answeringsystem 1 through theUI 100, and the question-answeringsystem 1 may synthesize the question of theuser 10 with the reply for the reverse question and provide an answer to theuser 10 or provide another reverse question by repeating the above-stated steps. - An entity knowledge graph used to generate an answer provided to the
user 10 may be stored in theuser knowledge graph 430. - When the question-answering
system 1 according to the present invention fails to derive an answer, that is, a correct answer with high reliability through the initial question of theuser 10, the question-answeringsystem 1 may inquire theuser 10 by generating a reverse question before providing an answer for the corresponding question to theuser 10, thereby improving reliability of an answer to be provided to theuser 10, that is, a correct answer for the question. -
FIG. 2 is a block diagram showing the main configuration of a natural language understanding unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. - Referring to
FIG. 2 , the naturallanguage understanding unit 200 may include a naturallanguage understanding unit 210 that interprets a question in the form of a natural language received by theUI 100. The naturallanguage understanding unit 210 may include amorpheme analyzing unit 211, asyntax analyzing unit 212, an entityname analyzing unit 213, afiltering analyzing unit 214, anintention classifying unit 215, adomain analyzing unit 216, and a semantic role labeling unit (SRL) 217. - The
morpheme analyzing unit 211 may separate a sentence constituting a question into morphemes. Thesyntax analyzing unit 212 and the entityname analyzing unit 213 may perform syntax analysis and entity name analysis on the sentence that constitutes the question and is separated into morphemes. Thefiltering analyzing unit 214 may generate a simplified sentence by removing unnecessary features from the sentence constituting the question. Theintention classifying unit 215 and thedomain analyzing unit 216 may perform intention classification and domain analysis for the question to which a semantic role is assigned based on the simplified sentence generated by thefiltering analyzing unit 214. TheSRL 217 may assign a semantic role to the sentence constituting the question. -
FIG. 3 is a block diagram showing the main configuration of a knowledge graph management unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. - Referring to
FIG. 3 , the naturallanguage understanding unit 200 may include the naturallanguage analyzing unit 210 and thequestion classifying unit 220. Themodule server 300 may include the knowledgegraph management unit 310, the questiontarget ranking unit 320, and the naturallanguage generating unit 330. Thedata server 400 may include theintegrated knowledge graph 410, thecontext knowledge graph 420, and theuser knowledge graph 430. - The knowledge
graph management unit 310 may include an integrated knowledgegraph management unit 312, a context knowledgegraph generating unit 314, and a knowledgegraph inquiring unit 316. - The integrated knowledge
graph management unit 312 may manage and update theintegrated knowledge graph 410. In some embodiments, theintegrated knowledge graph 410 may have a space for storing a knowledge graph built within a question-answering system (1 ofFIG. 1 ). Theintegrated knowledge graph 410 may have a space for storing data in any form, such as NoSQL, a relational database, or a file system. Theintegrated knowledge graph 410 may have one logically divided storage device, a division unit that logically separates one or a plurality of storage devices, or a portion of one physically divided storage device or one logically divided storage device. - In some other embodiments, the
integrated knowledge graph 410 may be a management module that is connected to at least one knowledge graph built outside a question-answering system (1 ofFIG. 1 ) through a network (20 ofFIG. 1 ) and manage the at least one knowledge graph built outside the question-answering system (1 ofFIG. 1 ). In some other embodiments, theintegrated knowledge graph 410 may have a management module that integrates and manages a space for storing knowledge graphs built inside a question-answering system (1 ofFIG. 1 ) with the at least one knowledge graph built outside the question-answering system (1 ofFIG. 1 ). - The context knowledge
graph generating unit 314 may generate thecontext knowledge graph 420 from theintegrated knowledge graph 410 by using entities of a question of a user (10 ofFIG. 1 ) extracted and analyzed by the naturallanguage analyzing unit 210. Thecontext knowledge graph 420 may be generated by cutting out only first-order relationship triples from theintegrated knowledge graph 410 based on the entities of an extracted question of theuser 10. In other words, thecontext knowledge graph 420 may include entity knowledge graphs, which are first-order relationship triples that use each of entities of an extracted question of theuser 10 as a subject to indicate a relationship with respect to another entity, which is an object, as a triple having the subject-property-object structure. - The knowledge
graph inquiring unit 316 may generate a query regarding a question of theuser 10 by using a result of classification for questions of theuser 10 by thequestion classifying unit 220, that is, a property to be obtained from the question of theuser 10. - The
context knowledge graph 420 may receive a query for the question of theuser 10 generated by the knowledgegraph question unit 316 and transmit at least one entity knowledge graph corresponding to the query for the question of theuser 10 from among a plurality of entity knowledge graphs included in thecontext knowledge graph 420 to the knowledgegraph management unit 310. - The question
target ranking unit 320 may generate an answer or a reverse question for the question of theuser 10 by using the at least one entity knowledge graph corresponding to the question of theuser 10. - The natural
language generating unit 330 may generate an answer or a reverse question for the question of theuser 10 generated by the questiontarget ranking unit 320 in the form of a natural language. -
FIG. 4 is a schematic diagram showing an example of a context knowledge graph generated by a user-customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. In detail,FIG. 4 shows an example of a part of a context knowledge graph generated when a question of as user (10 ofFIG. 1 ) is ‘Where is the birthplace of Yi Sun-Sin?’. - Referring to
FIGS. 3 and 4 together, when the question of the user (10 ofFIG. 1 ) is ‘Where is the birthplace of Yi Sun-Sin?’, the naturallanguage analyzing unit 210 may extract ‘Yi Sun-Sin’ and ‘birthplace’ as entities and transmit them to the context knowledgegraph generating unit 314. The context knowledgegraph generating unit 314 may extract a plurality of entity knowledge graphs EKG1 and EKG2, which are first-order relationship triples, by using each of ‘Yi Sun-Sin’ and ‘birthplace’ as a subject and at least one other entity having a first-order relationship (i.e., at least one entity having a property) as an object, thereby generating a context knowledge graph CKG. AlthoughFIG. 4 shows only the context knowledge graph CKG in which first-order relationship triples use ‘Yi Sun-Sin’ as the subject, the context knowledge graph CKG may further include first-order relationship triples using ‘birthplace’ as the subject. The context knowledge graph CKG may correspond to thecontext knowledge graph 420 shown inFIG. 1 . - When there is ‘Yi Sun-Sin the Muwuigong’ other than ‘Yi Sun-Sin the Chungmugong’, the context knowledge graph CKG may include a first entity knowledge graph EKG1, which is a first-order relationship triple using Yi Sun-Sin the Chungmugong E1 as the subject and a second entity knowledge graph EKG2, which is a first-order relationship triple using Yi Sun-Sin the Muwuigong E2 as the subject.
- The first entity knowledge graph EKG1 may use the Yi Sun-Sin the Chungmugong E1 as the subject and may have objects respectively having first-order triple relationships T11, T12, T13, T14, and T15 with respect to properties like nationality (Nationality), birthplace (BornIn), job (Job), creation (Create), and spouse (Spouse). The second entity knowledge graph EKG2 may use the Yi Sun-Sin the Muwuigong E2 as the subject and may have objects respectively having first-order triple relationships T21, T22, and T23 with respect to properties like nationality (Nationality), birthplace (BornIn), and job (Job).
-
FIG. 5 is a flowchart showing a process that question-answering is performed based on a knowledge graph in a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. - Referring to
FIGS. 3 to 5 together, thequestion classifying unit 220 may classify ‘birthplace’, which is a property to be obtained by a question of a user (10 ofFIG. 1 ) and transmit the same to the knowledgegraph inquiring unit 316. The knowledgegraph inquiring unit 316 may generate a query regarding the question of theuser 10 by using ‘birthplace’ (BornIn), which is a result of classification, inquire the context knowledge graph CKG (operation S10), and select and receive entity knowledge graphs having the property ‘bornin’ with respect to ‘Yi Sun-Sin’, which is the subject, that is, the first entity knowledge graph EKG1 and the second entity knowledge graph EKG2 (operation S110). Here, the inquiring the context knowledge graph CKG (operation S10) means that a query, which is a computer language used for a request for information collection, is transmitted to the context knowledge graph CKG. - When the context knowledge graph CKG has only one entity knowledge graph (e.g., the first entity knowledge graph EKG1) having ‘birthplace’ (BornIn) as the property with respect to ‘Yi Sun-Sin’, which is the subject, Geoncheon-dong of Hansung T12, which is the ‘birthplace’ (BornIn) of Yi Sun-Sin the Chungmugong E1, may be inferred as a correct answer and provided to the user 10 (operation S20).
- The question
target ranking unit 320 determines the existence of the first entity knowledge graph EKG1 and the second entity knowledge graph EKG2 from among user entity knowledge graphs, which are included in theuser knowledge graph 430 and used to answer previous questions of the user 10 (operation S120). When only the first entity knowledge graph EKG1 exists in theuser knowledge graph 430, the first entity knowledge graph EKG1 may be selected from the user knowledge graph 430 (operation S150) and Geoncheon-dong of Hansung T12, which is the birthplace of Yi Sun-Sin the Chungmugong E1, may be inferred as a correct answer and provided to theuser 10 as an answer (operation S20). - When neither the first entity knowledge graph EKG1 nor the second entity knowledge graph EKG2 exists in the
user knowledge graph 430, the questiontarget ranking unit 320 may determine a ranking for a question target with reference to the integrated knowledge graph 410 (operation S130), provide a reverse question to theuser 10, and receive a result thereof (operation S140). -
FIG. 6 is a flowchart for describing an operation of a question target ranking unit of a customized question-answering system based on a knowledge graph according to an example embodiment of the present disclosure. - Referring to
FIGS. 3 to 6 together, when the first entity knowledge graph EKG1 and the second entity knowledge graph EKG2 are selected as entity knowledge graphs to be ranked (operation S30), the questiontarget ranking unit 320 compares the first entity knowledge graph EKG1 with the second entity knowledge graph EKG2 having the same property and determines whether there is an exclusive triple included in only one of the first entity knowledge graph EKG1 and the second entity knowledge graph EKG2 (operation S310). - While the first entity knowledge graph EKG1 and the second entity knowledge graph EKG2 have overlapping triples having objects having first-order triple relationships T11, T12, T13 and first-order triple relationships T21, T22, and T23 for the properties nationality, bornin, and job, only the first entity knowledge graph EKG1 has an exclusive triple having the object having first-order triple relationships T14 and T15 for the properties create and spouse. When there are two or more exclusive triples T14 and T15, an exclusive triple for a reverse question may be selected from the two or more exclusive triples T14 and T15 (operation S320). In some embodiments, as the exclusive triple for a reverse question, from among objects of the exclusive triples T14 and T15, one with a high frequency of existence in the
integrated knowledge graph 410 may be selected. In some other embodiments, one having more objects having triple relationships for the objects of the exclusive triples T14 and T15 as the subject may be selected as an exclusive triple for a reverse question. - For example, when turtle ship T14 exists more frequently than Bang Su-Jin T15 in the
integrated knowledge graph 410 or there are more objects having triple relationships using ‘turtle ship’ as the subject, an exclusive triple having turtle ship T14 as the property creation (Create) for the subject of Yi Sun-Sin the Chungmugong E1 may be selected as an exclusive triple for a reverse question (operation S320) and a reverse question may be provided to a user (10 ofFIG. 1 ) (operation S40). - When there is no exclusive triple, although not shown as a separate triple in
FIG. 4 , it is assumed that there are separate triples for ‘Chungmugong’ and ‘Muwuigong’, which are artnames (Artname). - The first entity knowledge graph EKG1 and the second entity knowledge graph EKG2 may have overlapping triples having objects having first-order triple relationships T11, T12, T13, and ‘Chungmugong’ and first-order triple relationships T21, T22, T23, and ‘Muwuigong’ for properties of nationality (Nationality), birthplace (BornIn), job (Job), and artname (Artname). From among them, Joseon T11 and Joseon T2 and general T13 and general T23 have the same object, they are excluded from overlapping triple candidates for a reverse question, and Geoncheon-dong of Hansung T12 and Siheung of Gyeonggido T22 may be overlapping triple candidates for a reverse question. From between them, one with a high frequency of existence in the
integrated knowledge graph 410 or one having more objects having triple relationships may be selected as an overlapping triple for a reverse question. For example, with respect to the property of artname (Artname) for Yi Sun-Sin the Chungmugong E1 as the subject, an overlapping triple corresponding to ‘Chungmugong’ may be selected as an overlapping triple for a reverse question (operation S330) and a reverse question may be provided to a user (10 ofFIG. 1 ) (operation S40). - When a question of the
user 10 is ‘What is the job of Yi Sun-Sin?’, there are overlapping triples having general T13 and general T23, which are the same answer for the property of job (Job) for subjects of Yi Sun-Sin the Chungmugong E1 and Yi Sun-Sin the Muwuigong E2. Therefore, in this case, ‘general’ may be inferred and provided to theuser 10 as an answer without generating an overlapping triple for a reverse question. - Referring to
FIGS. 1, 5, and 6 together, an answer provided to the user 10 (operation S20) or a reverse question provided to the user 10 (operation S40) may be generated in the form of a natural language by the naturallanguage generating unit 330. At this time, the naturallanguage generating unit 330 may generate an answer or a reverse question for the question of theuser 10 as a plurality of candidates in various expressions in the form of a natural language. After theanswer ranking unit 500 determines a ranking for a plurality of candidates, in the form of a natural language, of an answer or a reverse question for the question of theuser 10 generated by the naturallanguage generating unit 330, theanswer ranking unit 500 may select a candidate with a high ranking and provide the selected candidate as an answer or a reverse question to theuser 10 through theUI 100. - When the question-answering
system 1 according to the present invention fails to derive an answer, that is, a correct answer with high reliability through the initial question of theuser 10, the question-answeringsystem 1 may inquire theuser 10 by generating a reverse question by using a context knowledge graph before providing an answer for the corresponding question to theuser 10, thereby improving reliability of an answer to be provided to theuser 10, that is, a correct answer for the question. - The present disclosure has been described above in detail with reference to preferred embodiments. However, the present disclosure is not limited to the above embodiments, and various modifications and changes may be made therein by one of ordinary skill in the art within the technical spirit and scope of the present disclosure.
Claims (10)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020190114970A KR102252459B1 (en) | 2019-09-18 | 2019-09-18 | Customized Question-Answering system based on knowledge graph |
KR10-2019-0114970 | 2019-09-18 | ||
PCT/KR2019/013713 WO2021054514A1 (en) | 2019-09-18 | 2019-10-18 | User-customized question-answering system based on knowledge graph |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/KR2019/013713 Continuation WO2021054514A1 (en) | 2019-09-18 | 2019-10-18 | User-customized question-answering system based on knowledge graph |
Publications (2)
Publication Number | Publication Date |
---|---|
US20220207067A1 true US20220207067A1 (en) | 2022-06-30 |
US11960513B2 US11960513B2 (en) | 2024-04-16 |
Family
ID=74883207
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/697,614 Active 2039-12-28 US11960513B2 (en) | 2019-09-18 | 2022-03-17 | User-customized question-answering system based on knowledge graph |
Country Status (3)
Country | Link |
---|---|
US (1) | US11960513B2 (en) |
KR (1) | KR102252459B1 (en) |
WO (1) | WO2021054514A1 (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210241050A1 (en) * | 2020-02-03 | 2021-08-05 | Samsung Electronics Co., Ltd. | System and method for efficient multi-relational entity understanding and retreival |
US20220036001A1 (en) * | 2020-04-21 | 2022-02-03 | Microsoft Technology Licensing, Llc | Labeled Knowledge Graph Based Priming Of A Natural Language Model Providing User Access To Programmatic Functionality Through Natural Language Input |
US20220351716A1 (en) * | 2021-04-28 | 2022-11-03 | Robert Bosch Gmbh | System and method for a personalized dialogue system using knowledge-based entity services |
US20230009946A1 (en) * | 2021-07-12 | 2023-01-12 | International Business Machines Corporation | Generative relation linking for question answering |
CN117235220A (en) * | 2023-09-15 | 2023-12-15 | 之江实验室 | Extensible large language model calling method and device based on graph database knowledge enhancement |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112989005B (en) * | 2021-04-16 | 2022-07-12 | 重庆中国三峡博物馆 | Knowledge graph common sense question-answering method and system based on staged query |
CN113672693B (en) * | 2021-08-23 | 2022-05-10 | 东北林业大学 | Label recommendation method of online question-answering platform based on knowledge graph and label association |
CN114153961B (en) * | 2022-02-07 | 2022-05-06 | 杭州远传新业科技有限公司 | Knowledge graph-based question and answer method and system |
CN114625843B (en) * | 2022-04-06 | 2022-11-29 | 山东亿方锦泽信息科技有限公司 | Inquiry method and device based on knowledge graph |
CN114996419A (en) * | 2022-05-09 | 2022-09-02 | 成都数之联科技股份有限公司 | Intelligent question answering method and device for weapon equipment, electronic equipment and storage medium |
CN117591657B (en) * | 2023-12-22 | 2024-05-07 | 宿迁乐享知途网络科技有限公司 | Intelligent dialogue management system and method based on AI |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190034780A1 (en) * | 2017-07-31 | 2019-01-31 | Microsoft Technology Licensing, Llc | Knowledge Graph For Conversational Semantic Search |
US20200242444A1 (en) * | 2019-01-30 | 2020-07-30 | Baidu Usa Llc | Knowledge-graph-embedding-based question answering |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101686068B1 (en) * | 2015-02-24 | 2016-12-14 | 한국과학기술원 | Method and system for answer extraction using conceptual graph matching |
US10606846B2 (en) | 2015-10-16 | 2020-03-31 | Baidu Usa Llc | Systems and methods for human inspired simple question answering (HISQA) |
KR101835345B1 (en) * | 2016-02-29 | 2018-03-07 | 경기대학교 산학협력단 | Concept graph expansion system based on knowledge base |
US10380169B2 (en) | 2016-07-29 | 2019-08-13 | Rovi Guides, Inc. | Systems and methods for determining an execution path for a natural language query |
KR101987915B1 (en) * | 2017-12-22 | 2019-06-12 | 주식회사 솔트룩스 | System for generating template used to generate query to knowledge base from natural language question and question answering system including the same |
-
2019
- 2019-09-18 KR KR1020190114970A patent/KR102252459B1/en active IP Right Grant
- 2019-10-18 WO PCT/KR2019/013713 patent/WO2021054514A1/en active Application Filing
-
2022
- 2022-03-17 US US17/697,614 patent/US11960513B2/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190034780A1 (en) * | 2017-07-31 | 2019-01-31 | Microsoft Technology Licensing, Llc | Knowledge Graph For Conversational Semantic Search |
US20200242444A1 (en) * | 2019-01-30 | 2020-07-30 | Baidu Usa Llc | Knowledge-graph-embedding-based question answering |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210241050A1 (en) * | 2020-02-03 | 2021-08-05 | Samsung Electronics Co., Ltd. | System and method for efficient multi-relational entity understanding and retreival |
US11687570B2 (en) * | 2020-02-03 | 2023-06-27 | Samsung Electronics Co., Ltd. | System and method for efficient multi-relational entity understanding and retrieval |
US20220036001A1 (en) * | 2020-04-21 | 2022-02-03 | Microsoft Technology Licensing, Llc | Labeled Knowledge Graph Based Priming Of A Natural Language Model Providing User Access To Programmatic Functionality Through Natural Language Input |
US11580302B2 (en) * | 2020-04-21 | 2023-02-14 | Microsoft Technology Licensing, Llc | Labeled knowledge graph based priming of a natural language model providing user access to programmatic functionality through natural language input |
US20220351716A1 (en) * | 2021-04-28 | 2022-11-03 | Robert Bosch Gmbh | System and method for a personalized dialogue system using knowledge-based entity services |
US20230009946A1 (en) * | 2021-07-12 | 2023-01-12 | International Business Machines Corporation | Generative relation linking for question answering |
CN117235220A (en) * | 2023-09-15 | 2023-12-15 | 之江实验室 | Extensible large language model calling method and device based on graph database knowledge enhancement |
Also Published As
Publication number | Publication date |
---|---|
KR102252459B1 (en) | 2021-05-14 |
US11960513B2 (en) | 2024-04-16 |
KR20210033348A (en) | 2021-03-26 |
WO2021054514A1 (en) | 2021-03-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11960513B2 (en) | User-customized question-answering system based on knowledge graph | |
US10943072B1 (en) | Contextual and intent based natural language processing system and method | |
US11151982B2 (en) | Cross-context natural language model generation | |
EP3671526B1 (en) | Dependency graph based natural language processing | |
US11568057B2 (en) | Systems and methods for triaging software vulnerabilities | |
US20200034375A1 (en) | Data virtualization using leveraged semantic knowledge in a knowledge graph | |
US20200356726A1 (en) | Dependency graph based natural language processing | |
KR101739540B1 (en) | System and method for building integration knowledge base based | |
JP2023545625A (en) | Systems and methods for software vulnerability triage | |
US20170103125A1 (en) | Apparatus and method of exploring and accessing relevant data from big data repository | |
US20200334251A1 (en) | Automatic transformation of complex tables in documents into computer understandable structured format and managing dependencies | |
KR20080049428A (en) | Method and apparatus for providing similarity searching services by semantic web | |
US11281980B2 (en) | Systems and methods for extending reasoning capability for data analytics in internet-of-things (IoT) platform | |
US10817554B2 (en) | Observation-based query interpretation model modification | |
KR102600752B1 (en) | Apparatus, system, and method for text semantic analysis using source code analysis, and method for providing chatbot question and answer service using the same | |
US20240070178A1 (en) | Disambiguating unrecognized abbreviations in search queries using machine learning | |
Gan et al. | Knowledge base question answering based on regularization and feature fusion | |
WO2023181064A1 (en) | A mechanized intellect orchestration | |
Sonawane et al. | Knowledge Graph | |
CN116304001A (en) | News abstract generation method and device, electronic equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
AS | Assignment |
Owner name: SALTLUX INC., KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LEE, KYUNG IL;REEL/FRAME:059429/0968 Effective date: 20220314 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |