CN117829287A - Method, device, equipment and storage medium for evaluating performance of knowledge graph embedded model - Google Patents
Method, device, equipment and storage medium for evaluating performance of knowledge graph embedded model Download PDFInfo
- Publication number
- CN117829287A CN117829287A CN202410127010.5A CN202410127010A CN117829287A CN 117829287 A CN117829287 A CN 117829287A CN 202410127010 A CN202410127010 A CN 202410127010A CN 117829287 A CN117829287 A CN 117829287A
- Authority
- CN
- China
- Prior art keywords
- index value
- complement
- triplet
- knowledge graph
- candidate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000000295 complement effect Effects 0.000 claims abstract description 140
- 238000011156 evaluation Methods 0.000 claims abstract description 64
- 238000004590 computer program Methods 0.000 claims abstract description 26
- 230000006870 function Effects 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 5
- 230000004927 fusion Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a method, a device, computer equipment, a storage medium and a computer program product for evaluating the performance of a knowledge graph embedded model, and relates to the technical field of knowledge graphs. The method comprises the following steps: obtaining a complement triplet of a pre-constructed power knowledge graph by utilizing an embedding model of the knowledge graph to be evaluated; acquiring the complement rationality index value of the complement triplet, acquiring the average interval index value of the knowledge graph embedded model to be evaluated based on the complement rationality index value, and acquiring the performance accuracy index value of the knowledge graph embedded model to be evaluated; and acquiring a performance evaluation result of the to-be-evaluated knowledge graph embedded model based on the average interval index value and the performance accuracy index value. The method can improve accuracy of model performance evaluation.
Description
Technical Field
The present invention relates to the technical field of knowledge maps, and in particular, to a method, an apparatus, a computer device, a storage medium, and a computer program product for evaluating performance of a knowledge map embedding model.
Background
In general, the power system has the characteristics of complex knowledge system, diversified data sources, large knowledge scale and the like. A large amount of power information may be hidden in huge data, such as power grid state, abnormal condition, energy consumption mode, etc., and in the same way, in building the power knowledge graph of the power system, there may be a relationship between entities that is not clear enough, for this case, generally, a knowledge graph embedding model may be introduced to mine the entity relationship in the power knowledge graph, so as to implement completion of the power knowledge graph, and improve the comprehensiveness and usability of the data for building the power knowledge graph. In order to improve the accuracy of the power knowledge graph completion, the performance of the corresponding knowledge graph embedded model can be evaluated, and the existing model evaluation method has the problem of low performance evaluation accuracy.
Disclosure of Invention
Based on this, it is necessary to provide a knowledge graph embedded model performance evaluation method, apparatus, computer device, storage medium and computer program product in view of the above technical problems.
In a first aspect, the present application provides a method for evaluating performance of a knowledge-graph embedded model. The method comprises the following steps:
Obtaining a complement triplet of a pre-constructed power knowledge graph by utilizing an embedding model of the knowledge graph to be evaluated;
acquiring the complement rationality index value of the complement triplet, acquiring the average interval index value of the knowledge graph embedded model to be evaluated based on the complement rationality index value, and acquiring the performance accuracy index value of the knowledge graph embedded model to be evaluated;
and acquiring a performance evaluation result of the knowledge graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value.
In one embodiment, the obtaining, based on the reasonability index value, an average interval index value of the knowledge-graph embedding model to be evaluated includes: determining a candidate triplet set corresponding to the complement triplet; a plurality of candidate triples in the candidate triplet set are obtained by replacing any one of a completion head entity, a completion relation and a completion tail entity in the completion triples; acquiring the average value of candidate rationality index values of a plurality of candidate triples in the candidate triplet set; and determining a difference value of the average value of the complement rationality index value and the candidate rationality index value as the average interval index value.
In one embodiment, the determining the candidate triplet set corresponding to the complement triplet includes: acquiring a plurality of power head entities of the pre-constructed power knowledge graph, and determining other power head entities except for the complement head entity in the complement triplet in the plurality of power head entities as the rest power head entities; sequentially replacing the completion head entities in the completion triples with the rest power head entities to obtain candidate triples corresponding to the completion triples; or acquiring a plurality of power tail entities of the pre-constructed power knowledge graph, and determining other power tail entities except for the complement tail entity in the complement triplet in the plurality of power tail entities as the rest power tail entities; sequentially replacing the complement tail entities in the complement triples with the rest power tail entities to obtain candidate triples corresponding to the complement triples; or sequentially replacing the complement relation in the complement triples with other power relations to obtain a candidate triplet set corresponding to the complement triples.
In one embodiment, the obtaining the performance accuracy index value of the knowledge graph embedding model to be evaluated includes: determining a candidate triplet set corresponding to the complement triplet, acquiring a candidate rationality index value of each candidate triplet in the candidate triplet set, acquiring a first number of the candidate rationality index values exceeding a preset rationality threshold value, and acquiring a second number of the candidate triples in the candidate triplet set; and determining the ratio of the first number to the second number as the performance accuracy index value.
In one embodiment, the obtaining the performance evaluation result of the knowledge graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value includes: and under the condition that the average interval index value exceeds a preset interval threshold value and the performance accuracy index value exceeds a preset accuracy threshold value, determining the performance evaluation result as that the to-be-evaluated knowledge graph embedded model performance evaluation passes.
In one embodiment, the obtaining the complement rationality index value of the complement triplet includes: acquiring a three-dimensional tensor of the complement triplet; and obtaining the complement rationality index value based on the three-dimensional tensor of the complement triplet by utilizing a pre-constructed entity bilinear function.
In a second aspect, the present application provides a knowledge graph embedding model performance evaluation apparatus. The device comprises:
the embedding module is used for utilizing the knowledge graph embedding model to be evaluated to acquire a complement triplet of the pre-constructed power knowledge graph;
the computing module is used for acquiring the complement rationality index value of the complement triplet, acquiring the average interval index value of the knowledge graph embedded model to be evaluated based on the complement rationality index value, and acquiring the performance accuracy index value of the knowledge graph embedded model to be evaluated;
And the evaluation module is used for acquiring a performance evaluation result of the knowledge graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:
obtaining a complement triplet of a pre-constructed power knowledge graph by utilizing an embedding model of the knowledge graph to be evaluated;
acquiring the complement rationality index value of the complement triplet, acquiring the average interval index value of the knowledge graph embedded model to be evaluated based on the complement rationality index value, and acquiring the performance accuracy index value of the knowledge graph embedded model to be evaluated;
and acquiring a performance evaluation result of the knowledge graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
Obtaining a complement triplet of a pre-constructed power knowledge graph by utilizing an embedding model of the knowledge graph to be evaluated;
acquiring the complement rationality index value of the complement triplet, acquiring the average interval index value of the knowledge graph embedded model to be evaluated based on the complement rationality index value, and acquiring the performance accuracy index value of the knowledge graph embedded model to be evaluated;
and acquiring a performance evaluation result of the knowledge graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:
obtaining a complement triplet of a pre-constructed power knowledge graph by utilizing an embedding model of the knowledge graph to be evaluated;
acquiring the complement rationality index value of the complement triplet, acquiring the average interval index value of the knowledge graph embedded model to be evaluated based on the complement rationality index value, and acquiring the performance accuracy index value of the knowledge graph embedded model to be evaluated;
And acquiring a performance evaluation result of the knowledge graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value.
In the method, the device, the computer equipment, the storage medium and the computer program product for evaluating the performance of the knowledge graph embedded model, the knowledge graph embedded model to be evaluated can be utilized to obtain the complement triples of the pre-constructed power knowledge graph; further, a completion rationality index value of the completion triplet can be obtained, an average interval index value of the knowledge graph embedded model to be evaluated is obtained based on the completion rationality index value, and a performance accuracy index value of the knowledge graph embedded model to be evaluated is obtained; finally, a performance evaluation result of the knowledge graph embedding model to be evaluated can be obtained based on the average interval index value and the performance accuracy index value. According to the method provided by the embodiment of the application, the average interval index value and the performance accuracy index value are introduced to evaluate the knowledge graph embedded model to be evaluated, so that the accuracy of the performance evaluation of the model can be improved.
Drawings
Fig. 1 is a schematic flow chart of a method for evaluating performance of a knowledge graph embedding model according to an embodiment of the present application;
Fig. 2 is a flowchart of obtaining an average interval index value according to an embodiment of the present application;
FIG. 3 is a flowchart of obtaining a performance accuracy index value according to an embodiment of the present disclosure;
fig. 4 is a block diagram of a knowledge graph embedded model performance evaluation device according to an embodiment of the present application;
fig. 5 is an internal structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In one embodiment, as shown in fig. 1, a method for evaluating performance of a knowledge graph embedded model is provided, and the embodiment is illustrated by applying the method to a server, it can be understood that the method can also be applied to a terminal, and can also be applied to a system including the terminal and the server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the steps of:
and step S101, acquiring a complement triplet of the pre-constructed power knowledge graph by utilizing the knowledge graph embedding model to be evaluated.
The power knowledge graph of the power system can be constructed, and further, model performance evaluation can be performed on the knowledge graph embedding model to be evaluated based on the pre-constructed power knowledge graph. The power knowledge graph may include a plurality of power head entities, a plurality of power tail entities, and a plurality of power relationships. The construction method of the power knowledge graph can comprise the following steps:
the first step: a plurality of power information data, e.g., power equipment, plant information, and power network topology, etc., of the power system is obtained from a plurality of power data sources associated with the power system.
The power knowledge graph of the power system is logically divided into 2 layers of a mode layer and a data layer, the mode layer is constructed in a top-down construction mode, and the data layer is constructed in a bottom-up mode under the guidance of the mode layer. The model layer is above the data layer, is a conceptual model and a logic foundation of a stable situation map, stores refined situation knowledge, is generally managed by an entity library, and performs standard constraint on the data layer. The data layer is an example of an entity, stores real data, consists of a series of facts describing the power system, and is formed by the processes of knowledge extraction, knowledge fusion, knowledge processing and the like.
And a second step of: and carrying out data conversion and information fusion on the plurality of electric power information data to obtain electric power fusion information of the electric power system.
And a third step of: and carrying out power entity identification on the power fusion information by using the named entity extraction model to realize power entity extraction, and marking to obtain marked power entities, such as power equipment, power stations and the like, wherein the marked power entities can comprise a power head entity and a power tail entity.
Fourth step: and based on the marked electric entity, extracting the electric relation of the electric fusion information through an entity relation extraction model so as to extract an electric triplet, and constructing an electric knowledge graph of the electric system by the electric triplet. The entity relation extraction model comprises a bidirectional circulation network, an expansion gate convolution neural network and a self-attention model.
The power knowledge graph may be a structured knowledge representation describing entities, concepts and relationships between power systems in the real world. The power entities in the power knowledge graph may be anything in the real world, including people, places, events, concepts, objects, etc. These electrical entities exist in the form of nodes in the electrical knowledge graph and are connected by relationships between them. It should be appreciated that the power knowledge graph may contain a plurality of power entities, which may include a plurality of power head entities and a plurality of power tail entities. In the power knowledge graph, a relationship between a power head entity and a power tail entity, i.e., a power relationship, is generally described as an edge, which is an association connecting two entities, and which describes a semantic relationship or association between the power head entity and the power tail entity, the power head entity is generally a starting point of the power relationship, and the power tail entity is an ending point of the power relationship. The power relationship may be any attribute describing an association between a power head entity and a power tail entity, such as "owned" and "located" and the like. In this way. In general, the power head entity, the power tail entity, and the power relationships may be stored in the power knowledge graph in the form of power triplets, which may be used to describe the relationships between the entities, one of the most basic elements in the knowledge graph, typically expressed in the form of: (Power head entity, power relation, power Tail entity). The power head entity represents a starting point of a relation, the power relation represents a semantic relation between the power head entity and the power tail entity, and the power tail entity represents an ending point of the power relation. In this way, the power knowledge graph may describe complex relationships and semantic connections between power entities.
The power knowledge-graph obtained above is an original, thousand-level knowledge set, and in one possible implementation, the deeper knowledge hidden in the power knowledge-graph can be further mined by a knowledge-graph embedding model that can represent entities and relationships in the power knowledge-graph as vectors in a continuous vector space in order to capture semantic relationships between them, these models typically using neural networks to learn embedded representations of entities and relationships. In this embodiment of the present application, information mining may be performed on a pre-constructed power knowledge graph based on the knowledge graph embedding model, to obtain a complement triplet of the power knowledge graph, and to complement the power knowledge graph, where the knowledge graph embedding model may be a RESCAL model, and a core idea of the RESCAL model is to encode the whole power knowledge graph into a three-dimensional tensor, and decompose the tensor into a core tensor and a factor matrix, where each two-dimensional matrix slice in the core tensor represents a power relationship, and each line in the factor matrix represents a power entity. The result restored by the core tensor and the factor matrix is regarded as the probability that the corresponding complement triplet is established, and if the probability is larger than a preset probability value, the complement triplet is indicated to be correct; and if the probability is smaller than or equal to the preset probability value, indicating that the complement triplet is incorrect. In one possible implementation manner, the probability that the complete triplet is established may be represented by obtaining a reasonability index value of the complete triplet, where the reasonability index value of the complete triplet may be calculated by a scoring function set by the RESCAL model.
Step S102, obtaining the completion rationality index value of the completion triplet, obtaining the average interval index value of the to-be-evaluated knowledge graph embedded model based on the completion rationality index value, and obtaining the performance accuracy index value of the to-be-evaluated knowledge graph embedded model.
Wherein, the completion rationality index value of the completion triplet can be obtained by calculation through a scoring function set by the RESCAL model, and the scoring function can be referred to the following formula (1):
wherein,may be a three-dimensional tensor of a complement triplet (h, r, t); m is M r Core tensor slices representing relationship r; f (f) r (h, t) may be a make-up rationality index value for the make-up triplet (h, r, t).
Furthermore, an average interval index value of the to-be-evaluated knowledge-graph embedded model may be obtained based on the complement rationality index value, where the average interval index value may represent an interval distance between the complement rationality index value of the complement triplet and the candidate rationality index value of the corresponding candidate triplet, and the larger the average interval index value is, the better the model performance of the to-be-evaluated knowledge-graph embedded model is, and the smaller the average interval index value is, the worse the model performance of the to-be-evaluated knowledge-graph embedded model is. Any one of a completion head entity, a completion relation and a completion tail entity in the completion triplet can be replaced to obtain a candidate triplet corresponding to the completion triplet, specifically, in one possible implementation manner, the completion head entity in the completion triplet can be replaced by other power head entities except the completion head entity in a plurality of power head entities in the power knowledge graph in sequence to obtain a candidate triplet set corresponding to the completion triplet; in another possible implementation manner, the complement tail entities in the complement triples may be sequentially replaced with the rest of the power tail entities except for the complement tail entities in the plurality of power tail entities in the power knowledge graph, so as to obtain a candidate triplet set corresponding to the complement triples; in yet another possible implementation manner, the complement relationships in the complement triples may be sequentially replaced by the rest of the power relationships except for the complement relationships in the plurality of power relationships in the power knowledge graph, so as to obtain the candidate triplet set corresponding to the complement triples. It should be understood that, based on the average value of the completion rationality index value of the completion triplet and the candidate rationality index value of the corresponding candidate triplet set, an average interval index value of the knowledge graph embedding model to be evaluated may be determined, and the calculation formula may be referred to as the following formula (2):
Wherein N may be the number of candidate triples contained in the candidate triplet set; f (f) r (h i T) may be a candidate rationality index value for an i-th candidate triplet in the set of candidate triples; f (f) r (h, t) may be a make-up rationality index value for a make-up triplet (h, r, t); "MeanMargin" may be the average interval index value of the knowledge-graph embedding model to be evaluated.
Further, a performance accuracy index value of the to-be-evaluated knowledge-graph embedded model can be obtained based on the complement rationality index value, the performance accuracy index value can be used for measuring the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet, the larger the performance accuracy index value is, the higher the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet is, the smaller the performance accuracy index value is, and the lower the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet is. Specifically, a first number of candidate triples, in the candidate triplet set, of which the candidate rationality index value exceeds a preset rationality threshold value, and a second number of candidate triples, contained in the candidate triplet set, may be obtained, and based on the first number and the second number, a performance accuracy index value of the knowledge graph embedding model to be evaluated is determined. The calculation formula can be seen in the following formula (3):
Wherein n may be a first number; n may be a second number; the ACC may be a performance accuracy index value of the knowledge graph embedding model to be evaluated.
And step S103, acquiring a performance evaluation result of the to-be-evaluated knowledge graph embedded model based on the average interval index value and the performance accuracy index value.
The average interval index value may represent an interval distance between the completion rationality index value of the completion triplet and the candidate rationality index value of the corresponding candidate triplet, where the larger the average interval index value is, the better the model performance of the to-be-evaluated knowledge-graph embedded model is, and the smaller the average interval index value is, the worse the model performance of the to-be-evaluated knowledge-graph embedded model is. The performance accuracy index value can be used for measuring the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet, and the larger the performance accuracy index value is, the higher the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet is, and the smaller the performance accuracy index value is, the lower the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet is. A preset interval threshold value and a preset accuracy threshold value for evaluating the performance of the knowledge-graph embedded model to be evaluated can be preset, in one possible implementation manner, the performance evaluation result of the knowledge-graph embedded model to be evaluated can be the minimum average interval index value and the minimum preset accuracy threshold value which are required to be met by the performance evaluation of the model, and the performance evaluation result is determined to be the performance evaluation of the knowledge-graph embedded model to be evaluated under the condition that the average interval index value exceeds the preset interval threshold value and the performance accuracy index value exceeds the preset accuracy threshold value; under the condition that the average interval index value exceeds a preset interval threshold value and the performance accuracy index value does not exceed a preset accuracy threshold value, determining a performance evaluation result as that the performance evaluation of the to-be-evaluated knowledge graph embedded model is not passed; and under the condition that the average interval index value does not exceed the preset interval threshold value and the performance accuracy index value exceeds the preset accuracy threshold value, determining the performance evaluation result as that the performance evaluation of the to-be-evaluated knowledge graph embedded model is not passed.
According to the method, the to-be-evaluated knowledge graph embedding model can be utilized to obtain the complement triples of the pre-constructed electric power knowledge graph; further, a completion rationality index value of the completion triplet can be obtained, an average interval index value of the knowledge graph embedded model to be evaluated is obtained based on the completion rationality index value, and a performance accuracy index value of the knowledge graph embedded model to be evaluated is obtained; finally, a performance evaluation result of the knowledge graph embedding model to be evaluated can be obtained based on the average interval index value and the performance accuracy index value. According to the method provided by the embodiment of the application, the average interval index value and the performance accuracy index value are introduced to evaluate the knowledge graph embedded model to be evaluated, so that the accuracy of the performance evaluation of the model can be improved.
In some embodiments, as shown in fig. 2, the obtaining, based on the complement rationality index value, the average interval index value of the knowledge-graph embedding model to be evaluated in step S102 may include:
step S201, determining a candidate triplet set corresponding to the complement triplet.
In some embodiments, step S201 may include: acquiring a plurality of power head entities of a pre-constructed power knowledge graph, and determining other power head entities except for the complement head entity in the complement triplet in the plurality of power head entities as the rest power head entities; sequentially replacing the completion head entities in the completion triples with the rest power head entities to obtain candidate triples corresponding to the completion triples; or acquiring a plurality of power tail entities of a pre-constructed power knowledge graph, and determining other power tail entities except the complement tail entity in the complement triplet in the plurality of power tail entities as the rest power tail entities; sequentially replacing the complement tail entities in the complement triples with the rest power tail entities to obtain candidate triples corresponding to the complement triples; or sequentially replacing the complement relation in the complement triples with the rest power relation to obtain candidate triples corresponding to the complement triples.
Step S202, obtaining the average value of candidate rationality index values of a plurality of candidate triples in the candidate triplet set.
Step S203, a difference between the average value of the complement rationality index value and the candidate rationality index value is determined as an average interval index value.
Any one of a completion head entity, a completion relation and a completion tail entity in the completion triplet can be replaced to obtain a candidate triplet corresponding to the completion triplet, specifically, in one possible implementation manner, the completion head entity in the completion triplet can be replaced by other power head entities except the completion head entity in a plurality of power head entities in the power knowledge graph in sequence to obtain a candidate triplet set corresponding to the completion triplet; in another possible implementation manner, the complement tail entities in the complement triples may be sequentially replaced with the rest of the power tail entities except for the complement tail entities in the plurality of power tail entities in the power knowledge graph, so as to obtain a candidate triplet set corresponding to the complement triples; in yet another possible implementation manner, the complement relationships in the complement triples may be sequentially replaced by the rest of the power relationships except for the complement relationships in the plurality of power relationships in the power knowledge graph, so as to obtain the candidate triplet set corresponding to the complement triples. It should be understood that, based on the average value of the completion rationality index value of the completion triplet and the candidate rationality index value of the corresponding candidate triplet set, an average interval index value of the knowledge graph embedding model to be evaluated may be determined, and the calculation formula may be referred to as the following formula (2):
Wherein N may be the number of candidate triples contained in the candidate triplet set; f (f) r (h i T) may be a candidate rationality index value for an i-th candidate triplet in the set of candidate triples; f (f) r (h, t) may be a make-up rationality index value for a make-up triplet (h, r, t); "MeanMargin" may be the average interval index value of the knowledge-graph embedding model to be evaluated. The average interval index value can represent the interval distance between the completion rationality index value of the completion triplet and the candidate rationality index value of the corresponding candidate triplet, and the larger the average interval index value is, the better the model performance of the knowledge graph embedding model to be evaluated is, the average intervalThe smaller the index value is, the worse the model performance of the knowledge graph embedding model to be evaluated is.
According to the method, the average interval index value of the knowledge graph embedded model to be evaluated can be determined based on the complement rationality index value of the complement triplet and the average value of the candidate rationality index values of the corresponding candidate triplet set, so that the knowledge graph embedded model to be evaluated can be evaluated conveniently based on the average interval index value, and the accuracy of model performance evaluation is improved.
In some embodiments, as shown in fig. 3, the obtaining a performance accuracy index value of the knowledge-graph embedding model to be evaluated in step S102 may include:
step S301, a candidate triplet set corresponding to the complement triplet is determined, and a candidate rationality index value of each candidate triplet in the candidate triplet set is obtained.
Step S302, a first number of candidate rationality index values exceeding a preset rationality threshold is obtained, and a second number of candidate triples of the candidate triplet set is obtained.
Step S303, determining the ratio of the first number to the second number as a performance accuracy index value.
Further, a performance accuracy index value of the to-be-evaluated knowledge-graph embedded model can be obtained based on the complement rationality index value, the performance accuracy index value can be used for measuring the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet, the larger the performance accuracy index value is, the higher the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet is, the smaller the performance accuracy index value is, and the lower the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet is. Specifically, a first number of candidate triples, in the candidate triplet set, of which the candidate rationality index value exceeds a preset rationality threshold value, and a second number of candidate triples, contained in the candidate triplet set, may be obtained, and based on the second number and the first number, a performance accuracy index value of the knowledge graph embedding model to be evaluated is determined. The calculation formula can be seen in the following formula (3):
Wherein n may be a first number; n may be a second number; the ACC may be a performance accuracy index value of the knowledge graph embedding model to be evaluated.
According to the method, the performance accuracy index value of the knowledge graph embedded model to be evaluated can be determined, the subsequent evaluation of the knowledge graph embedded model to be evaluated based on the performance accuracy index value is facilitated, and the accuracy of the performance evaluation of the model is improved.
In some embodiments, step S103 may include:
and under the condition that the average interval index value exceeds a preset interval threshold value and the performance accuracy index value exceeds a preset accuracy threshold value, determining a performance evaluation result as that the to-be-evaluated knowledge graph embedded model performance evaluation passes.
The average interval index value may represent an interval distance between the completion rationality index value of the completion triplet and the candidate rationality index value of the corresponding candidate triplet, where the larger the average interval index value is, the better the model performance of the to-be-evaluated knowledge-graph embedded model is, and the smaller the average interval index value is, the worse the model performance of the to-be-evaluated knowledge-graph embedded model is. The performance accuracy index value can be used for measuring the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet, and the larger the performance accuracy index value is, the higher the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet is, and the smaller the performance accuracy index value is, the lower the accuracy of the to-be-evaluated knowledge-graph embedded model prediction triplet is. A preset interval threshold value and a preset accuracy threshold value for evaluating the performance of the knowledge-graph embedded model to be evaluated can be preset, in one possible implementation manner, the performance evaluation result of the knowledge-graph embedded model to be evaluated can be the minimum average interval index value and the minimum preset accuracy threshold value which are required to be met by the performance evaluation of the model, and the performance evaluation result is determined to be the performance evaluation of the knowledge-graph embedded model to be evaluated under the condition that the average interval index value exceeds the preset interval threshold value and the performance accuracy index value exceeds the preset accuracy threshold value; under the condition that the average interval index value exceeds a preset interval threshold value and the performance accuracy index value does not exceed a preset accuracy threshold value, determining a performance evaluation result as that the performance evaluation of the to-be-evaluated knowledge graph embedded model is not passed; and under the condition that the average interval index value does not exceed the preset interval threshold value and the performance accuracy index value exceeds the preset accuracy threshold value, determining the performance evaluation result as that the performance evaluation of the to-be-evaluated knowledge graph embedded model is not passed.
According to the method, the average interval index value and the performance accuracy index value are introduced to evaluate the knowledge graph embedded model to be evaluated, so that the accuracy of the performance evaluation of the model can be improved.
In some embodiments, step S101 may include:
acquiring a three-dimensional tensor of the complement triplet; and obtaining the complement rationality index value based on the three-dimensional tensor of the complement triplet by utilizing a pre-constructed entity bilinear function.
In this embodiment of the present application, information mining may be performed on a pre-constructed power knowledge graph based on the knowledge graph embedding model, to obtain a complement triplet of the power knowledge graph, and to complement the power knowledge graph, where the knowledge graph embedding model may be a RESCAL model, and a core idea of the RESCAL model is to encode the whole power knowledge graph into a three-dimensional tensor, and decompose the tensor into a core tensor and a factor matrix, where each two-dimensional matrix slice in the core tensor represents a power relationship, and each line in the factor matrix represents a power entity. The result restored by the core tensor and the factor matrix is regarded as the probability that the corresponding complement triplet is established, and if the probability is larger than a preset probability value, the complement triplet is indicated to be correct; and if the probability is smaller than or equal to the preset probability value, indicating that the complement triplet is incorrect. In one possible implementation manner, the probability that the complete triplet is established may be represented by obtaining a reasonability index value of the complete triplet, where the reasonability index value of the complete triplet may be calculated by a scoring function set by the RESCAL model. The scoring function can be found in the following equation (1):
Wherein,may be a three-dimensional tensor of a complement triplet (h, r, t); m is M r Core tensor slices representing relationship r; f (f) r (h, t) may be a make-up rationality index value for the make-up triplet (h, r, t).
According to the method, the completion rationality index value is introduced, whether the completion triplet is correct or not can be judged through the completion rationality index value, and the accuracy of triplet judgment is improved.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a knowledge graph embedding model performance evaluation device for realizing the knowledge graph embedding model performance evaluation method. The implementation scheme of the solution provided by the device is similar to the implementation scheme recorded in the method, so the specific limitation of the embodiment of the performance evaluation device for embedding one or more knowledge maps provided below can be referred to the limitation of the performance evaluation method for embedding the knowledge maps in the model, which is not described herein.
In one embodiment, as shown in fig. 4, there is provided a knowledge-graph embedding model performance evaluation apparatus, including: an embedding module 401, a computing module 402, and an evaluation module 403, wherein:
the embedding module 401 is configured to obtain a complement triplet of a pre-constructed power knowledge graph by using the knowledge graph embedding model to be evaluated;
a calculation module 402, configured to obtain a completion rationality index value of the completion triplet, obtain an average interval index value of the to-be-evaluated knowledge-graph embedded model based on the completion rationality index value, and obtain a performance accuracy index value of the to-be-evaluated knowledge-graph embedded model;
And the evaluation module 403 is configured to obtain a performance evaluation result of the knowledge graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value.
In addition, the computing module 402 is further configured to: determining a candidate triplet set corresponding to the complement triplet; a plurality of candidate triples in the candidate triplet set are obtained by replacing any one of a completion head entity, a completion relation and a completion tail entity in the completion triples; acquiring the average value of candidate rationality index values of a plurality of candidate triples in the candidate triplet set; and determining a difference value of the average value of the complement rationality index value and the candidate rationality index value as the average interval index value.
A computing module 402, further configured to: acquiring a plurality of power head entities of the pre-constructed power knowledge graph, and determining other power head entities except for the complement head entity in the complement triplet in the plurality of power head entities as the rest power head entities; sequentially replacing the completion head entities in the completion triples with the rest power head entities to obtain candidate triples corresponding to the completion triples; or acquiring a plurality of power tail entities of the pre-constructed power knowledge graph, and determining other power tail entities except for the complement tail entity in the complement triplet in the plurality of power tail entities as the rest power tail entities; sequentially replacing the complement tail entities in the complement triples with the rest power tail entities to obtain candidate triples corresponding to the complement triples; or sequentially replacing the complement relation in the complement triples with other power relations to obtain a candidate triplet set corresponding to the complement triples.
Further, the computing module 402 is further configured to: determining a candidate triplet set corresponding to the complement triplet, and acquiring a candidate rationality index value of each candidate triplet in the candidate triplet set; acquiring a first number of candidate rationality index values exceeding a preset rationality threshold value, and acquiring a second number of candidate triples of the candidate triplet set; and determining the ratio of the first number to the second number as the performance accuracy index value.
The evaluation module 403 is further configured to: and under the condition that the average interval index value exceeds a preset interval threshold value and the performance accuracy index value exceeds a preset accuracy threshold value, determining the performance evaluation result as that the to-be-evaluated knowledge graph embedded model performance evaluation passes.
The embedding module 401 is further configured to: acquiring a three-dimensional tensor of the complement triplet; and obtaining the complement rationality index value based on the three-dimensional tensor of the complement triplet by utilizing a pre-constructed entity bilinear function.
The above-mentioned knowledge graph embedding model performance evaluation device may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing knowledge graph embedding model performance evaluation related data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program when executed by a processor implements a knowledge-graph embedded model performance assessment method.
It will be appreciated by those skilled in the art that the structure shown in fig. 5 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.
In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
It should be noted that, user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (10)
1. A method for evaluating performance of a knowledge graph embedding model, the method comprising:
obtaining a complement triplet of a pre-constructed power knowledge graph by utilizing an embedding model of the knowledge graph to be evaluated;
acquiring the complement rationality index value of the complement triplet, acquiring the average interval index value of the knowledge graph embedded model to be evaluated based on the complement rationality index value, and acquiring the performance accuracy index value of the knowledge graph embedded model to be evaluated;
And acquiring a performance evaluation result of the knowledge graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value.
2. The method according to claim 1, wherein the obtaining average interval index values of the knowledge-graph embedding model to be evaluated based on the rationality index values includes:
determining a candidate triplet set corresponding to the complement triplet; a plurality of candidate triples in the candidate triplet set are obtained by replacing any one of a completion head entity, a completion relation and a completion tail entity in the completion triples;
acquiring the average value of candidate rationality index values of a plurality of candidate triples in the candidate triplet set;
and determining a difference value of the average value of the complement rationality index value and the candidate rationality index value as the average interval index value.
3. The method of claim 2, wherein the determining the candidate triplet set for the complement triplet comprises:
acquiring a plurality of power head entities of the pre-constructed power knowledge graph, and determining other power head entities except for the complement head entity in the complement triplet in the plurality of power head entities as the rest power head entities;
Sequentially replacing the completion head entities in the completion triples with the rest power head entities to obtain candidate triples corresponding to the completion triples;
or (b)
Acquiring a plurality of power tail entities of the pre-constructed power knowledge graph, and determining other power tail entities except for the complement tail entity in the complement triplet in the plurality of power tail entities as the rest power tail entities;
sequentially replacing the complement tail entities in the complement triples with the rest power tail entities to obtain candidate triples corresponding to the complement triples;
or (b)
And sequentially replacing the complement relation in the complement triples with other power relations to obtain a candidate triplet set corresponding to the complement triples.
4. The method according to claim 1, wherein the obtaining the performance accuracy index value of the knowledge-graph embedding model to be evaluated includes:
determining a candidate triplet set corresponding to the complement triplet, and acquiring a candidate rationality index value of each candidate triplet in the candidate triplet set;
acquiring a first number of candidate rationality index values exceeding a preset rationality threshold value, and acquiring a second number of candidate triples of the candidate triplet set;
And determining the ratio of the first number to the second number as the performance accuracy index value.
5. The method according to claim 1, wherein the obtaining the performance evaluation result of the knowledge-graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value includes:
and under the condition that the average interval index value exceeds a preset interval threshold value and the performance accuracy index value exceeds a preset accuracy threshold value, determining the performance evaluation result as that the to-be-evaluated knowledge graph embedded model performance evaluation passes.
6. The method of claim 1, wherein the obtaining the make-up rationality index value for the make-up triplet comprises:
acquiring a three-dimensional tensor of the complement triplet;
and obtaining the complement rationality index value based on the three-dimensional tensor of the complement triplet by utilizing a pre-constructed entity bilinear function.
7. A knowledge-graph embedded model performance evaluation device, characterized in that the device comprises:
the embedding module is used for utilizing the knowledge graph embedding model to be evaluated to acquire a complement triplet of the pre-constructed power knowledge graph;
The computing module is used for acquiring the complement rationality index value of the complement triplet, acquiring the average interval index value of the knowledge graph embedded model to be evaluated based on the complement rationality index value, and acquiring the performance accuracy index value of the knowledge graph embedded model to be evaluated;
and the evaluation module is used for acquiring a performance evaluation result of the knowledge graph embedding model to be evaluated based on the average interval index value and the performance accuracy index value.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1-6 when the computer program is executed.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1-6.
10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410127010.5A CN117829287A (en) | 2024-01-30 | 2024-01-30 | Method, device, equipment and storage medium for evaluating performance of knowledge graph embedded model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410127010.5A CN117829287A (en) | 2024-01-30 | 2024-01-30 | Method, device, equipment and storage medium for evaluating performance of knowledge graph embedded model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117829287A true CN117829287A (en) | 2024-04-05 |
Family
ID=90517225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410127010.5A Pending CN117829287A (en) | 2024-01-30 | 2024-01-30 | Method, device, equipment and storage medium for evaluating performance of knowledge graph embedded model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117829287A (en) |
-
2024
- 2024-01-30 CN CN202410127010.5A patent/CN117829287A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112214499B (en) | Graph data processing method and device, computer equipment and storage medium | |
| CN114298417A (en) | Anti-fraud risk assessment method, anti-fraud risk training method, anti-fraud risk assessment device, anti-fraud risk training device and readable storage medium | |
| CN114579584A (en) | Data table processing method and device, computer equipment and storage medium | |
| CN116167289A (en) | Power grid operation scene generation method and device, computer equipment and storage medium | |
| CN116130029A (en) | Transformer fault early warning method, device, equipment, storage medium and program product | |
| CN113326343A (en) | Road network data storage method and system based on multi-level grids and file indexes | |
| CN116191398A (en) | Load prediction method, load prediction device, computer equipment and storage medium | |
| CN117829287A (en) | Method, device, equipment and storage medium for evaluating performance of knowledge graph embedded model | |
| CN115169718A (en) | Cellular automaton-based regional landslide risk dynamic prediction method and device | |
| CN114168581A (en) | Data cleaning method and device, computer equipment and storage medium | |
| CN117909517A (en) | Knowledge graph completion method, apparatus, device, storage medium, and program product | |
| CN117192373B (en) | Power battery fault analysis method, device, computer equipment and storage medium | |
| CN116227585B (en) | Parallel execution method and device for cluster tasks, computer equipment and storage medium | |
| CN117078441B (en) | Method, apparatus, computer device and storage medium for identifying claims fraud | |
| CN112445838B (en) | Efficient space k-kernel mining method for space data | |
| CN116485586B (en) | Intelligent building management method and system based on comprehensive digital platform | |
| CN116613754A (en) | Power distribution system reliability assessment method, model training method, device and equipment | |
| CN116911416A (en) | Technical trend prediction method, device and storage medium | |
| CN116227535A (en) | Optimization method and device for graph characterization learning model and computer equipment | |
| CN117633531A (en) | Photovoltaic output scene reduction method, model training method, device and storage medium | |
| CN116643961A (en) | Performance data complement method, device, equipment and storage medium | |
| CN117526289A (en) | Power distribution network partitioning method, device, computer equipment and storage medium | |
| CN117389780A (en) | Converter station fault analysis method, device, computer equipment and storage medium | |
| CN116501836A (en) | Method and device for determining electric public opinion risk level and computer equipment | |
| CN116957799A (en) | Method, device, computer equipment and storage medium for determining net value of product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |