CN116705338B - Traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on rules and paths - Google Patents
Traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on rules and paths Download PDFInfo
- Publication number
- CN116705338B CN116705338B CN202310990444.3A CN202310990444A CN116705338B CN 116705338 B CN116705338 B CN 116705338B CN 202310990444 A CN202310990444 A CN 202310990444A CN 116705338 B CN116705338 B CN 116705338B
- Authority
- CN
- China
- Prior art keywords
- chinese medicine
- traditional chinese
- knowledge graph
- mode
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000003814 drug Substances 0.000 title claims abstract description 356
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000004590 computer program Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 3
- 238000001228 spectrum Methods 0.000 claims 3
- 230000008569 process Effects 0.000 description 8
- 241000208340 Araliaceae Species 0.000 description 7
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 7
- 235000003140 Panax quinquefolius Nutrition 0.000 description 7
- 235000008434 ginseng Nutrition 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 6
- 208000024891 symptom Diseases 0.000 description 6
- 238000003745 diagnosis Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 239000008280 blood Substances 0.000 description 3
- 210000004369 blood Anatomy 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000006187 pill Substances 0.000 description 3
- 230000001737 promoting effect Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 210000003056 antler Anatomy 0.000 description 2
- 210000001124 body fluid Anatomy 0.000 description 2
- 239000010839 body fluid Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 108010010803 Gelatin Proteins 0.000 description 1
- 206010039424 Salivary hypersecretion Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 229920000159 gelatin Polymers 0.000 description 1
- 239000008273 gelatin Substances 0.000 description 1
- 235000019322 gelatine Nutrition 0.000 description 1
- 235000011852 gelatine desserts Nutrition 0.000 description 1
- 239000004615 ingredient Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 208000026451 salivation Diseases 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
- G06F18/256—Fusion techniques of classification results, e.g. of results related to same input data of results relating to different input data, e.g. multimodal recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/027—Frames
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Public Health (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Primary Health Care (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Epidemiology (AREA)
- Pathology (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
The invention provides a traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on rules and paths, and relates to the technical field of knowledge graph reasoning, wherein the method comprises the following steps: combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triples of the traditional Chinese medicine multi-mode knowledge graph; generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph; embedding the representation for creating a vector representation; and carrying out reasoning on the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge graph reasoning model of the traditional Chinese medicine. The invention can effectively improve the accuracy of reasoning and prediction based on the traditional Chinese medicine multi-mode knowledge graph reasoning model.
Description
Technical Field
The invention relates to the technical field of knowledge graph reasoning, in particular to a traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on rules and paths.
Background
The knowledge graph (knowledgegraph) can effectively manage and organize massive structured information, plays an important role in semantic analysis and the like, is composed of mutually related entities and relations thereof, has a large number of triples (tags), and is also abbreviated as。
In the related technology, along with the continuous increase of the scale of the knowledge graph, the relationship between the entities is increasingly complex, the structure sparsity and other problems are more obvious, and thus the reasoning performance is lower. Therefore, how to perform knowledge graph reasoning more effectively is a technical problem that needs to be solved by the person skilled in the art.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on rules and paths.
Specifically, the embodiment of the invention provides the following technical scheme:
in a first aspect, an embodiment of the present invention provides a method for reasoning a multi-modal knowledge graph of a traditional Chinese medicine based on rules and paths, including:
Establishing a traditional Chinese medicine multi-mode knowledge graph;
combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triples of the traditional Chinese medicine multi-mode knowledge graph; the target path represents a path between different entities; the target path is determined based on the relation between entities in the multi-mode knowledge graph; the target rule is used for indicating a combination rule between paths;
generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph; the embedded representation is used to create a vector representation;
and carrying out reasoning on the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge graph reasoning model of the traditional Chinese medicine.
Further, the combining the target paths in the multi-modal knowledge-graph according to the target rules in the multi-modal knowledge-graph to generate the target triplets of the multi-modal knowledge-graph includes:
And under the condition that the target paths are matched with a plurality of target rules, combining the target paths based on the target rule with the highest confidence in the plurality of target rules to generate a target triplet of the traditional Chinese medicine multi-mode knowledge graph.
Further, the generating a rule and path based traditional Chinese medicine multi-mode knowledge graph inference model according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph, and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph includes:
embedding the target triplet of the traditional Chinese medicine multi-mode knowledge graph into a preset N-dimensional vector space, and generating an embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph;
embedding the entity image in the traditional Chinese medicine multi-mode knowledge graph into a preset N-dimensional vector space to generate an entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph;
projecting entity description information in the traditional Chinese medicine multi-mode knowledge graph to a hyperplane to generate entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph; the hyperplane is determined based on structural knowledge of a multi-mode knowledge graph of the traditional Chinese medicine;
Projecting entity categories in the traditional Chinese medicine multi-mode knowledge graph to a hyperplane to generate entity category embedded representations in the traditional Chinese medicine multi-mode knowledge graph;
and generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity image in the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity description information in the traditional Chinese medicine multi-mode knowledge graph and the embedded representation of the entity category in the traditional Chinese medicine multi-mode knowledge graph.
Further, the generating a rule and path based traditional Chinese medicine multi-mode knowledge graph inference model according to the embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity image in the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity description information in the traditional Chinese medicine multi-mode knowledge graph, and the embedded representation of the entity category in the traditional Chinese medicine multi-mode knowledge graph includes:
and performing joint training on the embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity image in the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity description information in the traditional Chinese medicine multi-mode knowledge graph and the embedded representation of the entity category in the traditional Chinese medicine multi-mode knowledge graph to generate a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths.
Further, the embedding the target triplet of the multi-modal knowledge graph of the traditional Chinese medicine into a preset N-dimensional vector space, generating an embedded representation of the target triplet of the multi-modal knowledge graph of the traditional Chinese medicine includes:
the embedded representation of the target triplet of the multi-mode knowledge graph of the traditional Chinese medicine is represented by the following formula:
；
wherein,an embedded representation of a target triplet representing a multi-modal knowledge-graph of a traditional Chinese medicine; />Representing an energy function between target triples; />Representing the target Path->And entity relationship->Energy function of similarity.
Further, the reasoning of the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge graph reasoning model comprises:
and carrying out data set link prediction and triplet classification of the traditional Chinese medicine multi-mode knowledge graph according to the traditional Chinese medicine multi-mode knowledge graph reasoning model.
In a second aspect, an embodiment of the present invention further provides a multi-modal knowledge graph inference device for traditional Chinese medicine based on rules and paths, including:
the building module is used for building a traditional Chinese medicine multi-mode knowledge graph;
the generation module is used for combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triples of the traditional Chinese medicine multi-mode knowledge graph; the target path represents a path between different entities; the target path is determined based on the relation between entities in the multi-mode knowledge graph; the target rule is used for indicating a combination rule between paths;
The processing module is used for generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph; the embedded representation is used to create a vector representation;
and the reasoning module is used for reasoning the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge map reasoning model of the traditional Chinese medicine.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the rule and path based multi-modal knowledge graph inference method according to the first aspect when the processor executes the program.
In a fourth aspect, an embodiment of the present invention further provides a non-transitory computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the method for reasoning a multi-modal knowledge-graph of a traditional Chinese medicine based on rules and paths according to the first aspect.
In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, where the computer program when executed by a processor implements the method for reasoning a multi-modal knowledge graph of a traditional Chinese medicine based on rules and paths according to the first aspect.
According to the traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on the rules and paths, the traditional Chinese medicine multi-mode knowledge graph is established, and the target paths in the traditional Chinese medicine multi-mode knowledge graph are combined according to the target rules in the traditional Chinese medicine multi-mode knowledge graph to generate the target triples of the traditional Chinese medicine multi-mode knowledge graph; therefore, in the process of generating the traditional Chinese medicine multi-mode knowledge graph reasoning model and the knowledge graph reasoning, not only are the multi-mode information such as category, description and image added, but also the target triplet information determined based on the target rule and the target path added, the reasoning basis of the model is improved, and the interpretable and more convincing reasoning result is provided; based on abundant semantic information, the accuracy of reasoning and predicting can be improved, traditional Chinese medicine resources are effectively integrated, reasoning under the scene with high requirements on the safety performance of the model in the fields of traditional Chinese medicine and the like is achieved, information understanding in the traditional Chinese medicine field is assisted, the accuracy of knowledge provision in searching, recommending and asking and answering is improved, the laws of entities such as traditional Chinese medicine symptoms, prescriptions and treatment are deduced, doctors are helped to formulate diagnosis and treatment schemes, and life safety of people is guaranteed.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a multi-modal knowledge graph reasoning method of traditional Chinese medicine based on rules and paths provided by the embodiment of the invention;
fig. 2 is a schematic structural diagram of a multi-modal knowledge graph inference device for traditional Chinese medicine based on rules and paths according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The method provided by the embodiment of the invention can be applied to a knowledge graph reasoning scene, and the accuracy of reasoning prediction can be effectively improved based on a traditional Chinese medicine multi-mode knowledge graph reasoning model.
In the related technology, along with the continuous increase of the scale of the knowledge graph, the relationship between the entities is increasingly complex, the structure sparsity and other problems are more obvious, and thus the reasoning performance is lower. Therefore, how to perform knowledge graph reasoning more effectively is a technical problem that needs to be solved by the person skilled in the art.
According to the traditional Chinese medicine multi-mode knowledge graph reasoning method based on the rules and paths, the traditional Chinese medicine multi-mode knowledge graph is established, and the target paths in the traditional Chinese medicine multi-mode knowledge graph are combined according to the target rules in the traditional Chinese medicine multi-mode knowledge graph to generate the target triples of the traditional Chinese medicine multi-mode knowledge graph; therefore, in the process of generating the traditional Chinese medicine multi-mode knowledge graph reasoning model and the knowledge graph reasoning, not only are the multi-mode information such as category, description and image added, but also the target triplet information determined based on the target rule and the target path added, so that the reasoning basis of the model is improved, and the interpretable and more convincing reasoning result is provided; based on abundant semantic information, the accuracy of reasoning and predicting can be improved, traditional Chinese medicine resources are effectively integrated, reasoning under the scene with high requirements on the safety performance of the model in the fields of traditional Chinese medicine and the like is achieved, information understanding in the traditional Chinese medicine field is assisted, the accuracy of knowledge provision in searching, recommending and asking and answering is improved, the laws of entities such as traditional Chinese medicine symptoms, prescriptions and treatment are deduced, doctors are helped to formulate diagnosis and treatment schemes, and life safety of people is guaranteed.
The following describes the technical scheme of the present invention in detail with reference to fig. 1 to 3. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.
Fig. 1 is a schematic flow chart of an embodiment of a multi-modal knowledge graph inference method for traditional Chinese medicine based on rules and paths according to an embodiment of the present invention. As shown in fig. 1, the method provided in this embodiment includes:
step 101, establishing a multi-mode knowledge graph of the traditional Chinese medicine;
specifically, as the scale of the knowledge graph is continuously increased, the relationship between the entities is increasingly complex, the problems of structural sparsity and the like are more obvious, and therefore the reasoning performance is lower.
In order to solve the problems, the multi-modal information in the field of traditional Chinese medicine is fully utilized and the reasoning performance is improved, and in the embodiment of the invention, a traditional Chinese medicine multi-modal knowledge graph is firstly established; optionally, the multi-modal knowledge graph of the traditional Chinese medicine comprises 30656 entities, 13 relations and 204070 triples, including category, description and image information.
102, combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triples of the traditional Chinese medicine multi-mode knowledge graph; the target path represents a path between different entities; the target path is determined based on the relation between the entities in the multi-mode knowledge graph; the target rule is used for indicating a combination rule between paths;
Specifically, after a traditional Chinese medicine multi-mode knowledge graph is established, a target rule and a target path are extracted from the traditional Chinese medicine multi-mode knowledge graph in the embodiment of the invention; wherein the target path represents a path between different entities; the target path is determined based on the relation between the entities in the multi-mode knowledge graph; for example, (Ginseng cornu Cervi Pantotrichum pill, component, ginseng) represents component path; (Ginseng radix, functional indications, promoting salivation and nourishing blood) indicates the path of the functional indications; optionally, the target rule is used to indicate a combination rule between paths, indicating which paths can be combined and how to combine between paths; for example, the target paths in the multi-mode knowledge graph of the traditional Chinese medicine are combined according to the target rules in the multi-mode knowledge graph of the traditional Chinese medicine to generate new triplets, namely (ginseng pilose antler pills, ingredients, ginseng) 'Λ' (ginseng, functional indications, promoting the production of body fluid and nourishing blood) → (ginseng pilose antler pills, functional indications, promoting the production of body fluid and nourishing blood).
Step 103, generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph; embedding the representation for creating a vector representation;
Specifically, after combining target paths in a traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triplets of the traditional Chinese medicine multi-mode knowledge graph, the embodiment of the invention generates a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to target triplets of the traditional Chinese medicine multi-mode knowledge graph, entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph; wherein the embedded representation is used to create a vector representation; optionally, vector representation can be created for the entity image in the multi-mode traditional Chinese medicine knowledge graph to obtain entity image embedded representation in the multi-mode traditional Chinese medicine knowledge graph, and then fusion and joint training are carried out on the target triplet of the multi-mode traditional Chinese medicine knowledge graph, the entity image embedded representation in the multi-mode traditional Chinese medicine knowledge graph, the entity description information embedded representation in the multi-mode traditional Chinese medicine knowledge graph and the entity category embedded representation in the multi-mode traditional Chinese medicine knowledge graph, so that the multi-mode traditional Chinese medicine knowledge graph reasoning model can be obtained. In the process of establishing the traditional Chinese medicine multi-mode knowledge graph reasoning model, not only is multi-mode information such as category, description and image added, but also target triplet information determined based on target rules and target paths is added, so that the reasoning basis of the model is improved, and interpretable and more convincing reasoning results are provided; based on abundant semantic information, the accuracy of reasoning and predicting can be improved, traditional Chinese medicine resources are effectively integrated, reasoning under the scene with high requirements on the safety performance of the model in the fields of traditional Chinese medicine and the like is achieved, information understanding in the traditional Chinese medicine field is assisted, the accuracy of knowledge provision in searching, recommending and asking and answering is improved, the laws of entities such as traditional Chinese medicine symptoms, prescriptions and treatment are deduced, doctors are helped to formulate diagnosis and treatment schemes, and life safety of people is guaranteed.
And 104, reasoning the multi-mode knowledge of the traditional Chinese medicine according to the multi-mode knowledge map reasoning model of the traditional Chinese medicine.
Specifically, according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph, after the traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths is generated, namely, in the process of establishing the traditional Chinese medicine multi-mode knowledge graph reasoning model, not only the multi-mode information such as categories, descriptions and images is added, but also the target triplet information determined based on target rules and target paths is added, and the target triplet information is integrated into an embedded frame, the definition of triple energy is expanded to consider new multi-mode representation, and the target rules and the paths embedded by the traditional Chinese medicine multi-mode knowledge graph are integrated, so that the interpretability of the semantic level and the generalization of the data level of the model are endowed, and the accuracy of reasoning prediction can be improved based on abundant semantic information.
According to the method, the target triplet of the traditional Chinese medicine multi-mode knowledge graph is generated by establishing the traditional Chinese medicine multi-mode knowledge graph and combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph; therefore, in the process of generating the traditional Chinese medicine multi-mode knowledge graph reasoning model and the knowledge graph reasoning, not only are the multi-mode information such as category, description and image added, but also the target triplet information determined based on the target rule and the target path added, so that the reasoning basis of the model is improved, and the interpretable and more convincing reasoning result is provided; based on abundant semantic information, the accuracy of reasoning and predicting can be improved, traditional Chinese medicine resources are effectively integrated, reasoning under the scene with high requirements on the safety performance of the model in the fields of traditional Chinese medicine and the like is achieved, information understanding in the traditional Chinese medicine field is assisted, the accuracy of knowledge provision in searching, recommending and asking and answering is improved, the laws of entities such as traditional Chinese medicine symptoms, prescriptions and treatment are deduced, doctors are helped to formulate diagnosis and treatment schemes, and life safety of people is guaranteed.
In an embodiment, combining target paths in the traditional Chinese medicine multi-mode knowledge-graph according to target rules in the traditional Chinese medicine multi-mode knowledge-graph to generate a target triplet of the traditional Chinese medicine multi-mode knowledge-graph comprises:
and under the condition that the target paths are matched with a plurality of target rules, combining the target paths based on the target rule with the highest confidence in the plurality of target rules to generate a target triplet of the traditional Chinese medicine multi-mode knowledge graph.
Specifically, in the embodiment of the invention, the target rule and the target path are extracted from the traditional Chinese medicine multi-mode knowledge graph, and then the target paths in the traditional Chinese medicine multi-mode knowledge graph can be combined according to the target rule in the traditional Chinese medicine multi-mode knowledge graph to generate a new triplet, so that the accuracy of reasoning and prediction can be improved based on rich semantic information. Alternatively, the process may be carried out in a single-stage,
each rule will have a confidence level; optionally, under the condition that the target paths are matched with a plurality of target rules, the combination of the target paths is performed based on the target rule with the highest confidence coefficient in the plurality of target rules, and a new triplet of the traditional Chinese medicine multi-mode knowledge graph is generated, so that the new triplet of the traditional Chinese medicine multi-mode knowledge graph is generated, rich and comprehensive semantic information can be provided, the accuracy of the semantic information is effectively improved, and the accuracy of reasoning prediction can be effectively improved by the traditional Chinese medicine multi-mode knowledge graph reasoning model.
For example, to better describe the multi-modal knowledge graph inference model of traditional Chinese medicine presented herein, a relevant symbolic definition is given first. Representing the multi-modal knowledge graph asWherein->Expressed as a collection of entities>Expressed as a set of relationships>Expressed as a triplet set->Representing a set of entity categories, entity descriptions, and image information, respectively. Entities and relationships are embedded in->，/>Is the dimension of the embedding space.
Definition 1. Representation based on rules and paths: entity pairA certain path between them is denoted +.>，/>Representing entity pair->A set of paths between.
Definition 2. Image-based representation: each entityAre all corresponding to a plurality of pictures, n images of each entity are expressed as +.>。
Definition 3. Representation based on description and category: for each entityThe description and category embedded representation of (1) are +.>And->。
Alternatively, first, a target rule of length 2 with confidence is mined from the knowledge-graph and its confidence. Then, extracting paths on the knowledge graph, wherein each path is guaranteed by a path constraint resource allocation mechanism>Is defined as the reliability of the entity pair +.>Between->When between entity pairs->When the path reliability of (2) is greater than 0.01, adding it to the entity pair +. >Path set of->. Finally, the paths are combined by a target rule of length 2.
In particular, the relationship combining operation is performed iteratively as the path is traversed until the relationships can no longer be combined. Because each combination combines two relationships, the next step may be to re-combine the resulting combination results. Two main scenarios are considered in the actual combining of paths:
(1) PathThe relations in (a) can be combined by the target rule with length of 2 to finally obtain an entity pair +.>Is the optimal scenario.
(2) Some of the relationships in the paths cannot be combined according to the target rule of length 2, numerical operations are used to add the embedded representations of the relationships, and some of the relationships may be matched to multiple rules, such asAnd->. In this way, the rule with the highest confidence coefficient can be selected to combine paths, so that the accuracy of semantic information is effectively improved, and the accuracy of reasoning prediction can be effectively improved by the traditional Chinese medicine multi-mode knowledge graph reasoning model.
According to the method, the combination of the target paths is carried out based on the target rule with the highest confidence coefficient in the target rules, and the new triplet of the traditional Chinese medicine multi-mode knowledge graph is generated, so that the new triplet of the traditional Chinese medicine multi-mode knowledge graph can be generated to provide abundant and comprehensive semantic information, the accuracy of the semantic information is effectively improved, and the accuracy of reasoning prediction can be effectively improved by the traditional Chinese medicine multi-mode knowledge graph reasoning model.
In an embodiment, generating a rule and path based traditional Chinese medicine multi-mode knowledge graph inference model according to a target triplet of the traditional Chinese medicine multi-mode knowledge graph, an entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, an entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph, and an entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph, includes:
embedding the target triplet of the traditional Chinese medicine multi-mode knowledge graph into a preset N-dimensional vector space to generate an embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph;
embedding the entity image in the traditional Chinese medicine multi-mode knowledge graph into a preset N-dimensional vector space to generate an entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph;
projecting entity description information in the traditional Chinese medicine multi-mode knowledge graph to a hyperplane to generate entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph; the hyperplane is determined based on structural knowledge of the multi-modal knowledge graph of the traditional Chinese medicine;
projecting entity categories in the traditional Chinese medicine multi-mode knowledge graph to a hyperplane to generate entity category embedded representations in the traditional Chinese medicine multi-mode knowledge graph;
And generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity image in the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity description information in the traditional Chinese medicine multi-mode knowledge graph and the embedded representation of the entity category in the traditional Chinese medicine multi-mode knowledge graph.
Specifically, in the embodiment of the invention, the target triplet of the traditional Chinese medicine multi-mode knowledge graph is embedded into a preset N-dimensional vector space, and the embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph is generated; embedding the entity image in the traditional Chinese medicine multi-mode knowledge graph into a preset N-dimensional vector space to generate an entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph; alternatively, the solid images in the multi-modal knowledge-graph of traditional Chinese medicine may be encoded into the solid space such that the triad-structured based representation and the image-based representation are in the same vector space.
Optionally, in the embodiment of the invention, entity description information in the multi-mode knowledge graph of the traditional Chinese medicine is projected to a hyperplane to generate an embedded representation of the entity description information in the multi-mode knowledge graph of the traditional Chinese medicine; projecting entity categories in the traditional Chinese medicine multi-mode knowledge graph to a hyperplane to generate entity category embedded representations in the traditional Chinese medicine multi-mode knowledge graph; optionally, the hyperplane is determined based on structural knowledge of a multi-modal knowledge graph of the traditional Chinese medicine; alternatively, the structured vector representation may be regarded as a hyperplane, such that the category and description vector representations are projected to the hyperplane; it should be noted that, in the embodiment of the present invention, the entity description information in the multi-mode knowledge graph of the traditional Chinese medicine and the entity category in the multi-mode knowledge graph of the traditional Chinese medicine are projected to the hyperplane, instead of being embedded into the preset N-dimensional vector space, so that the structural knowledge is taken as the hyperplane, and the category knowledge and the description knowledge are projected to the hyperplane to be fused, so that the embedding of the category knowledge and the description knowledge can be maximized, and the emphasis is placed on the core of the structural knowledge, so that the accuracy of reasoning prediction can be effectively improved.
For example, for the embedded representation of target triples of a multi-modal knowledge graph of traditional Chinese medicine, paths are provided in embodiments of the inventionIs expressed as +.>，/>Also defined as the result of path combining. In the embodiment of the invention, the triplet embedded representation accords with: />. For each triplet->Two energy functions are defined:
；
；
wherein if triplesThen->The score of (2) is smaller. />Expressed as path->Relationship of->Energy function of similarity. />、/>And->Embedded representation of head entity, relation entity and tail entity, respectively,/->Representing the path->From entity pairs->Reliability between them. />Representing a confidence set of rules.
Thus, the rule and path based structure embedded representation is defined as:
；
for entity image embedded representation in a traditional Chinese medicine multi-mode knowledge graph, a plurality of images of each entity are used as input of a neural image encoder to obtain information characteristics of the imagesThe symptoms are represented and are represented in a structured vector space. In addition, for better learning the feature representation of multiple images per entity, a attention mechanism is used to calculate the attention of each entity to the different images for each entity Is expressed as n images of，/>。
The neural image encoder consists of a representation module and a projection module, wherein the representation module is used for extracting characteristic representations in images, and the projection module is used for mapping the image characteristic representations obtained by the representation module into a structured vector space. In the representation module, a 4096-dimensional vector output by the second full connection layer (also referred to as fc 7) is used as an image feature vector. Then, a connection of the image with the entity is established by the projection module. Specifically, it is converted by mapping the matrix into a matrix of the same dimension as the structure embedding. The ith image is defined in the structured space as:
；
wherein the method comprises the steps ofIs a projection matrix +.>Dimension of embedded representation for image, +.>Embedding a dimension of the representation for the entity,>the ith image feature representation representing the entity.
Most entities have different graphsThe invention can automatically calculate the attention degree of each entity to different images through an attention mechanism, and the ith image of the mth entity is expressed asThe mechanism of attention is defined as:
；
wherein the method comprises the steps ofFor the structured representation of the mth entity, the image representation is similar to the corresponding structured representation, and the aggregate representation of the mth entity based on the image is defined as:
；
The image embedded representation is defined as:
；
wherein the method comprises the steps ofThe same energy function as TransE, +.>Wherein->And->Is based on the head, tail entity representation of the image, < >>And->The representation structure-based representation and the image-based representation are trained in the same vector space.
For entity description information in a multi-mode knowledge graph of traditional Chinese medicine, in the embodiment of the invention, the language characterization model is utilized to encode the entity description information, firstly, the description information is converted into three different embedded representations, namely word embedded representation, segmentation embedded representation and position embedded representation, and then the three embedded representations are spliced; finally, the embedding of entity description information is represented as an average of all sentence vector embedding.
Optionally, an entityThe description embedded representation of (1) is defined as:
；
wherein,is->Descriptive comments of->Is a weight matrix.
Optionally, the description embedded representation is defined as:
；
optionally, for entity categories in the multi-modal knowledge-graph of traditional Chinese medicine, the category embedded representation and the structure embedded representation have the same dimension, and their values are randomly initialized to train with the structure embedded representation. Category embedding is represented as an entityAverage of all class embedded representations:
；
Wherein the method comprises the steps ofIs entity->Is defined in the specification.
Optionally, the category embedding representation is defined as:
；
in the embodiment of the invention, the entity description information in the traditional Chinese medicine multi-mode knowledge graph and the entity category in the traditional Chinese medicine multi-mode knowledge graph are projected to the hyperplane, instead of being embedded into the preset N-dimensional vector space, so that the structural knowledge is taken as the hyperplane, the category knowledge and the description knowledge are projected to the hyperplane to be fused, the embedding of the category knowledge and the description knowledge can be maximized, and the emphasis is placed on the core of the structural knowledge, thereby effectively improving the accuracy of reasoning prediction.
Optionally, after the description information vector representation and the category information vector representation are obtained, the structural embedded representation is used as a hyperplane, and the description information embedded representation and the category information embedded representation are projected into a structural vector space for joint training, so that the traditional Chinese medicine multi-mode knowledge graph inference model based on rules and paths can be obtained:
；
wherein the method comprises the steps of=/>，/>，/>，/>And->Represented as weight parameters.
Optionally, the loss function of the joint training is:
；
wherein the method comprises the steps ofIs margin superparameter, is->And->Representing a positive triplet set and a negative triplet set, respectively.
According to the method, the entity description information in the traditional Chinese medicine multi-mode knowledge graph and the entity category in the traditional Chinese medicine multi-mode knowledge graph are projected to the hyperplane instead of being embedded into the preset N-dimensional vector space, so that structural knowledge is used as the hyperplane, category knowledge and description knowledge are projected to the hyperplane to be fused, embedding of the category knowledge and the description knowledge can be maximized, and emphasis is placed on the core of the structural knowledge, so that accuracy of reasoning prediction can be effectively improved.
In an embodiment, according to a multi-modal knowledge graph inference model of traditional Chinese medicine, the multi-modal knowledge inference of traditional Chinese medicine is performed, including:
and carrying out data set link prediction and triplet classification of the traditional Chinese medicine multi-mode knowledge graph according to the traditional Chinese medicine multi-mode knowledge graph reasoning model.
Specifically, after the embodiment of the invention generates the traditional Chinese medicine multi-mode knowledge graph inference model, the data set link prediction and the triplet classification of the traditional Chinese medicine multi-mode knowledge graph can be performed according to the traditional Chinese medicine multi-mode knowledge graph inference model, so that the inference in the fields of traditional Chinese medicine and the like under the scene with higher requirements on the safety performance of the model is realized, the information understanding in the traditional Chinese medicine fields is assisted, the accuracy of knowledge provision in searching, recommending and asking is improved, the laws of entities such as traditional Chinese medicine symptoms, prescriptions and treatment are inferred, doctors are helped to make diagnosis and treatment schemes, and the life safety of people is ensured.
For example, triad information is extracted from a pharmacology database of a traditional Chinese medicine system and is manually checked, and the triad information comprises 30656 entities and 13 relations in total. It was divided into training dataset (80%), test dataset (10%) and validation dataset (10%) in the experiment.
Optionally, the loss function on the training set is minimized, the optimal superparameter is found on the validation set, and the model is evaluated on the test set. The super parameters are set as follows: embedding dimensionsE {50, 100, 150, 200, 250}, boundary->E {0.2,0.6,1.0}, number of filters ∈>E {10, 20, 30, 40}, learning rate +.>∈{5×/>，/>，5×/>}、/>E {0.0,0.1,0.2,0.3,0.4,0.5}. In addition, the maximum training epoch is 1000 and the batch size is fixed at 1/50 of the training set size.
In the link prediction process, the link prediction is a subtask of knowledge graph completion, and aims to predict entities with the missing triples in the knowledge graph, namely, predict the missing triplesTail entity->Or predict missing triplesHead entity->. In the test, +.>And randomly replacing the head entity or the tail entity in the triplet by all the entities in the knowledge-graph entity set, and then sorting according to the descending of the score function.
In the evaluation task, two evaluation criteria of the translation model are selected: (1) MR (MeanRank): the smaller the value of the index, the better the performance of the model is explained; (2) MRR (Mean Reciprocal Rank): the larger the value of the index, the better the performance of the model.
Comparative experiments were performed on TransE, transH, convKB and RP-MMKRL. The predicted evaluation results of the physical links are shown in table 1, and the best two results under each index are shown in bold.
TABLE 1
The results show that the traditional Chinese medicine multi-mode knowledge graph inference model can well express entities and relations in the knowledge graph, promote the inference performance of entity prediction and improve the accuracy of link prediction to a certain extent.
Optionally, the triplet classification is a judgment tripletWhether the task is correctly classified. The data set of the experiment is a knowledge graph data set, and the experiment setting in the experiment is the same as the link prediction task. The policy of triad classification is to set a different relationship threshold for each different relationship +.>For a triplet- >If its distance score is smaller than this threshold +.>This triplet is considered correct, otherwise it is wrong. Accuracy is one of the metrics that evaluate the performance of classification models. It represents the ratio between the number of samples that the model correctly classifies and the total number of samples. The calculation formula of the accuracy is accuracy= (number of correctly classified samples)/(total number of samples). The ternary categorization evaluation results are shown in Table 2, with the best two results for each index shown bolded. />
TABLE 2
As can be seen from Table 2, the multi-modal knowledge graph inference model of the traditional Chinese medicine achieves better experimental results in the ternary classification experiment compared with the comparison model.
The traditional Chinese medicine multi-mode knowledge graph inference model is used for predicting the existing efficacy and the new efficacy which are not mentioned in the data set for the traditional Chinese medicine in the knowledge graph data set, and most of the effects which are not mentioned can be verified in the literature. Table 3 lists the knowledge of the first 11 inferences (exemplified by the efficacy of donkey-hide gelatin).
TABLE 3 Table 3
Note that: # is the existing knowledge and labeled is the knowledge that was validated from the literature.
Therefore, the multi-mode knowledge graph inference model of the traditional Chinese medicine obtains the best experimental effect in classification and prediction, and obtains the reliable new efficacy of the traditional Chinese medicine.
The traditional Chinese medicine multi-mode knowledge graph reasoning device based on the rules and the paths, which is provided by the invention, is described below, and the traditional Chinese medicine multi-mode knowledge graph reasoning device based on the rules and the paths, which are described below, and the traditional Chinese medicine multi-mode knowledge graph reasoning method based on the rules and the paths, which are described above, can be correspondingly referred to each other.
Fig. 2 is a schematic structural diagram of a multi-modal knowledge graph inference device for traditional Chinese medicine based on rules and paths. The traditional Chinese medicine multi-mode knowledge graph reasoning device based on rules and paths provided by the embodiment comprises:
the establishing module 210 is configured to establish a multi-modal knowledge graph of the traditional Chinese medicine;
the generating module 220 is configured to combine the target paths in the multi-modal knowledge-graph of the traditional Chinese medicine according to the target rules in the multi-modal knowledge-graph of the traditional Chinese medicine, and generate a target triplet of the multi-modal knowledge-graph of the traditional Chinese medicine; the target path represents a path between different entities; the target path is determined based on the relation between the entities in the multi-mode knowledge graph; the target rule is used for indicating a combination rule between paths;
the processing module 230 is configured to generate a rule and path-based multi-modal knowledge graph inference model according to the target triplet of the multi-modal knowledge graph, the entity image embedded representation in the multi-modal knowledge graph, the entity description information embedded representation in the multi-modal knowledge graph, and the entity category embedded representation in the multi-modal knowledge graph; embedding the representation for creating a vector representation;
The reasoning module 240 is configured to perform reasoning of the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge map reasoning model of the traditional Chinese medicine.
The device of the embodiment of the present invention is configured to perform the method of any of the foregoing method embodiments, and its implementation principle and technical effects are similar, and are not described in detail herein.
Fig. 3 illustrates a physical schematic diagram of an electronic device, which may include: processor 310, communication interface (Communications Interface) 320, memory 330 and communication bus 340, wherein processor 310, communication interface 320, memory 330 accomplish communication with each other through communication bus 340. The processor 310 may invoke logic instructions in the memory 330 to perform a rule and path based traditional Chinese medicine multi-modal knowledge-graph inference method comprising: establishing a traditional Chinese medicine multi-mode knowledge graph; combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triples of the traditional Chinese medicine multi-mode knowledge graph; the target path represents a path between different entities; the target path is determined based on the relation between the entities in the multi-mode knowledge graph; the target rule is used for indicating a combination rule between paths; generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph; embedding the representation for creating a vector representation; and carrying out reasoning on the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge graph reasoning model of the traditional Chinese medicine.
Further, the logic instructions in the memory 330 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method of reasoning a multi-modal knowledge graph of a traditional Chinese medicine based on rules and paths provided by the above methods, the method comprising: establishing a traditional Chinese medicine multi-mode knowledge graph; combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triples of the traditional Chinese medicine multi-mode knowledge graph; the target path represents a path between different entities; the target path is determined based on the relation between the entities in the multi-mode knowledge graph; the target rule is used for indicating a combination rule between paths; generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph; embedding the representation for creating a vector representation; and carrying out reasoning on the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge graph reasoning model of the traditional Chinese medicine.
In yet another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the rule and path based multi-modal knowledge graph inference method of traditional Chinese medicine provided above, the method comprising: establishing a traditional Chinese medicine multi-mode knowledge graph; combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triples of the traditional Chinese medicine multi-mode knowledge graph; the target path represents a path between different entities; the target path is determined based on the relation between the entities in the multi-mode knowledge graph; the target rule is used for indicating a combination rule between paths; generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph; embedding the representation for creating a vector representation; and carrying out reasoning on the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge graph reasoning model of the traditional Chinese medicine.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. A traditional Chinese medicine multi-mode knowledge graph reasoning method based on rules and paths is characterized by comprising the following steps:
establishing a traditional Chinese medicine multi-mode knowledge graph;
combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triples of the traditional Chinese medicine multi-mode knowledge graph; the target path represents a path between different entities; the target path is determined based on the relation between entities in the multi-mode knowledge graph; the target rule is used for indicating a combination rule between paths;
generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph; embedding the representation for creating a vector representation; the generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph, the entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph and the entity category embedded representation in the traditional Chinese medicine multi-mode knowledge graph comprises the following steps:
Embedding the target triplet of the traditional Chinese medicine multi-mode knowledge graph into a preset N-dimensional vector space, and generating an embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph;
embedding the entity image in the traditional Chinese medicine multi-mode knowledge graph into a preset N-dimensional vector space to generate an entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph;
projecting entity description information in the traditional Chinese medicine multi-mode knowledge graph to a hyperplane to generate entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph; the hyperplane is determined based on structural knowledge of a multi-mode knowledge graph of the traditional Chinese medicine;
projecting entity categories in the traditional Chinese medicine multi-mode knowledge graph to a hyperplane to generate entity category embedded representations in the traditional Chinese medicine multi-mode knowledge graph;
generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity image in the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity description information in the traditional Chinese medicine multi-mode knowledge graph and the embedded representation of the entity category in the traditional Chinese medicine multi-mode knowledge graph;
And carrying out reasoning on the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge graph reasoning model of the traditional Chinese medicine.
2. The method for reasoning the multi-modal knowledge graph of traditional Chinese medicine based on rules and paths according to claim 1, wherein the steps of combining target paths in the multi-modal knowledge graph of traditional Chinese medicine according to target rules in the multi-modal knowledge graph of traditional Chinese medicine to generate target triplets of the multi-modal knowledge graph of traditional Chinese medicine comprise:
and under the condition that the target paths are matched with a plurality of target rules, combining the target paths based on the target rule with the highest confidence in the plurality of target rules to generate a target triplet of the traditional Chinese medicine multi-mode knowledge graph.
3. The method for reasoning the multi-modal knowledge graph of traditional Chinese medicine based on rules and paths according to claim 1, wherein the generating the reasoning model of the multi-modal knowledge graph of traditional Chinese medicine based on the embedded representation of the target triplet of the multi-modal knowledge graph of traditional Chinese medicine, the embedded representation of the entity image in the multi-modal knowledge graph of traditional Chinese medicine, the embedded representation of the entity description information in the multi-modal knowledge graph of traditional Chinese medicine and the embedded representation of the entity category in the multi-modal knowledge graph of traditional Chinese medicine comprises:
And performing joint training on the embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity image in the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity description information in the traditional Chinese medicine multi-mode knowledge graph and the embedded representation of the entity category in the traditional Chinese medicine multi-mode knowledge graph to generate a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths.
4. The method for reasoning the multi-modal knowledge spectrum of traditional Chinese medicine based on rules and paths according to claim 3, wherein the step of embedding the target triplet of the multi-modal knowledge spectrum of the traditional Chinese medicine into a preset N-dimensional vector space to generate an embedded representation of the target triplet of the multi-modal knowledge spectrum of the traditional Chinese medicine comprises the following steps:
the embedded representation of the target triplet of the multi-mode knowledge graph of the traditional Chinese medicine is represented by the following formula:
；
wherein,an embedded representation of a target triplet representing a multi-modal knowledge-graph of a traditional Chinese medicine; />Representing a target tripletAn energy function between groups; />Representing the target Path->And entity relationship->Energy function of similarity.
5. The method for reasoning the multi-modal knowledge graph of the traditional Chinese medicine based on rules and paths according to claim 4, wherein the reasoning of the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge graph reasoning model of the traditional Chinese medicine comprises the following steps:
And carrying out data set link prediction and triplet classification of the traditional Chinese medicine multi-mode knowledge graph according to the traditional Chinese medicine multi-mode knowledge graph reasoning model.
6. A traditional Chinese medicine multi-mode knowledge graph reasoning device based on rules and paths is characterized by comprising:
the building module is used for building a traditional Chinese medicine multi-mode knowledge graph;
the generation module is used for combining target paths in the traditional Chinese medicine multi-mode knowledge graph according to target rules in the traditional Chinese medicine multi-mode knowledge graph to generate target triples of the traditional Chinese medicine multi-mode knowledge graph; the target path represents a path between different entities; the target path is determined based on the relation between entities in the multi-mode knowledge graph; the target rule is used for indicating a combination rule between paths;
a processing module for
Embedding the target triplet of the traditional Chinese medicine multi-mode knowledge graph into a preset N-dimensional vector space, and generating an embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph;
embedding the entity image in the traditional Chinese medicine multi-mode knowledge graph into a preset N-dimensional vector space to generate an entity image embedded representation in the traditional Chinese medicine multi-mode knowledge graph;
Projecting entity description information in the traditional Chinese medicine multi-mode knowledge graph to a hyperplane to generate entity description information embedded representation in the traditional Chinese medicine multi-mode knowledge graph; the hyperplane is determined based on structural knowledge of a multi-mode knowledge graph of the traditional Chinese medicine;
projecting entity categories in the traditional Chinese medicine multi-mode knowledge graph to a hyperplane to generate entity category embedded representations in the traditional Chinese medicine multi-mode knowledge graph;
generating a traditional Chinese medicine multi-mode knowledge graph reasoning model based on rules and paths according to the embedded representation of the target triplet of the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity image in the traditional Chinese medicine multi-mode knowledge graph, the embedded representation of the entity description information in the traditional Chinese medicine multi-mode knowledge graph and the embedded representation of the entity category in the traditional Chinese medicine multi-mode knowledge graph; the embedded representation is used to create a vector representation;
and the reasoning module is used for reasoning the multi-modal knowledge of the traditional Chinese medicine according to the multi-modal knowledge map reasoning model of the traditional Chinese medicine.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the rule and path based multi-modal knowledge-graph inference method of traditional Chinese medicine as claimed in any one of claims 1 to 5 when the program is executed by the processor.
8. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the rule and path based multi-modal knowledge graph inference method of traditional Chinese medicine as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310990444.3A CN116705338B (en) | 2023-08-08 | 2023-08-08 | Traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on rules and paths |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310990444.3A CN116705338B (en) | 2023-08-08 | 2023-08-08 | Traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on rules and paths |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116705338A CN116705338A (en) | 2023-09-05 |
| CN116705338B true CN116705338B (en) | 2023-12-08 |
Family
ID=87841980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310990444.3A Active CN116705338B (en) | 2023-08-08 | 2023-08-08 | Traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on rules and paths |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116705338B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105824802A (en) * | 2016-03-31 | 2016-08-03 | 清华大学 | Method and device for acquiring knowledge graph vectoring expression |
| CN106934042A (en) * | 2017-03-16 | 2017-07-07 | 中国人民解放军国防科学技术大学 | A kind of knowledge mapping represents model and its method |
| CN110069638A (en) * | 2019-03-12 | 2019-07-30 | 北京航空航天大学 | A kind of knowledge mapping combination table dendrography learning method of binding rule and path |
| CN110895561A (en) * | 2019-11-13 | 2020-03-20 | 中国科学院自动化研究所 | Medical question and answer retrieval method, system and device based on multi-mode knowledge perception |
| CN112561064A (en) * | 2020-12-21 | 2021-03-26 | 福州大学 | Knowledge base completion method based on OWKBC model |
| CN115526322A (en) * | 2022-09-24 | 2022-12-27 | 中国人民解放军战略支援部队信息工程大学 | Sequence generating type knowledge inference method and system based on precision transform |
| CN115687642A (en) * | 2022-10-18 | 2023-02-03 | 广东省中医院（广州中医药大学第二附属医院、广州中医药大学第二临床医学院、广东省中医药科学院） | Traditional Chinese medicine diagnosis and treatment knowledge discovery method based on clinical knowledge graph representation learning |
| CN116186278A (en) * | 2022-12-20 | 2023-05-30 | 广西师范大学 | Knowledge graph completion method based on hyperplane projection and relational path neighborhood |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10877979B2 (en) * | 2018-01-16 | 2020-12-29 | Accenture Global Solutions Limited | Determining explanations for predicted links in knowledge graphs |
-
2023
- 2023-08-08 CN CN202310990444.3A patent/CN116705338B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105824802A (en) * | 2016-03-31 | 2016-08-03 | 清华大学 | Method and device for acquiring knowledge graph vectoring expression |
| CN106934042A (en) * | 2017-03-16 | 2017-07-07 | 中国人民解放军国防科学技术大学 | A kind of knowledge mapping represents model and its method |
| CN110069638A (en) * | 2019-03-12 | 2019-07-30 | 北京航空航天大学 | A kind of knowledge mapping combination table dendrography learning method of binding rule and path |
| CN110895561A (en) * | 2019-11-13 | 2020-03-20 | 中国科学院自动化研究所 | Medical question and answer retrieval method, system and device based on multi-mode knowledge perception |
| CN112561064A (en) * | 2020-12-21 | 2021-03-26 | 福州大学 | Knowledge base completion method based on OWKBC model |
| CN115526322A (en) * | 2022-09-24 | 2022-12-27 | 中国人民解放军战略支援部队信息工程大学 | Sequence generating type knowledge inference method and system based on precision transform |
| CN115687642A (en) * | 2022-10-18 | 2023-02-03 | 广东省中医院（广州中医药大学第二附属医院、广州中医药大学第二临床医学院、广东省中医药科学院） | Traditional Chinese medicine diagnosis and treatment knowledge discovery method based on clinical knowledge graph representation learning |
| CN116186278A (en) * | 2022-12-20 | 2023-05-30 | 广西师范大学 | Knowledge graph completion method based on hyperplane projection and relational path neighborhood |
Non-Patent Citations (1)
| Title |
|---|
| 医学知识推理研究现状与发展;董文波 等;计算机科学与探索;第16卷(第06期);1193-1213 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116705338A (en) | 2023-09-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2023546829A (en) | System and method for relationship extraction using self-adaptive thresholds and local context pooling | |
| CN111753865A (en) | Recognition of realistic synthetic images generated using generative confrontation networks based on deep neural networks | |
| US20220277858A1 (en) | Medical Prediction Method and System Based on Semantic Graph Network | |
| CN112765370B (en) | Entity alignment method and device of knowledge graph, computer equipment and storage medium | |
| Malekzadeh et al. | Honest-but-curious nets: Sensitive attributes of private inputs can be secretly coded into the classifiers' outputs | |
| US20220138185A1 (en) | Scene graph modification based on natural language commands | |
| CN112201359B (en) | Method and device for identifying severe inquiry data based on artificial intelligence | |
| EP4361843A1 (en) | Neural network searching method and related device | |
| CN113707299A (en) | Auxiliary diagnosis method and device based on inquiry session and computer equipment | |
| CN113988013A (en) | ICD coding method and device based on multitask learning and graph attention network | |
| CN110808095B (en) | Diagnostic result recognition method, model training method, computer equipment and storage medium | |
| Wu et al. | Expert-defined keywords improve interpretability of retinal image captioning | |
| CN113096756A (en) | Disease evolution classification method and device, electronic equipment and storage medium | |
| CN116705338B (en) | Traditional Chinese medicine multi-mode knowledge graph reasoning method and device based on rules and paths | |
| CN113609233A (en) | Entity object coding method and device, electronic equipment and storage medium | |
| Sekiyama et al. | Automated proof synthesis for propositional logic with deep neural networks | |
| CN115858799A (en) | Knowledge representation learning method integrating ordered relationship path and entity description information | |
| Theodorou et al. | Synthesize extremely high-dimensional longitudinal electronic health records via hierarchical autoregressive language model | |
| Arabian et al. | Network Architecture Influence on Facial Emotion Recognition | |
| WO2024007119A1 (en) | Training method for text processing model, and text processing method and device | |
| Selitskiy et al. | Loss Function with Memory for Trustworthiness Threshold Learning: Case of Face and Facial Expression Recognition | |
| CN117688974B (en) | Knowledge graph-based generation type large model modeling method, system and equipment | |
| CN114372458B (en) | Emergency detection method based on government work order | |
| Refat et al. | SentiNet: A nonverbal facial sentiment analysis using convolutional neural network | |
| Bogaards | Comparing Case-based Classification Models with Interpretable Machine Learning Classification Models-a study based on human moral judgement in the bio-ethical domain |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |