DE202021102099U1 - Apparatus for processing a knowledge graph embedding - Google Patents

Abstract

Device (100) for processing a knowledge graph embedding (102) of a knowledge graph (200), characterized by at least one processor (104) which is designed for the following processes: determining a first embedding, the first embedding representing a subject in a first instance, wherein the knowledge graph (200) comprises an entity for the subject, retrieving a second embedding from the knowledge graph embedding (102), the second embedding representing an object in the knowledge graph (200), the knowledge graph (200) comprising an entity for the object, Retrieving at least one third embedding from the knowledge graph embedding (102), the at least one third embedding representing context, the knowledge graph (200) comprising at least one entity for the context, determining a fourth embedding, the fourth embedding being a relation in the knowledge graph (200 ) represents between the subject and the object, depicting the first embedding processing, the second embedding, the at least one third embedding and the fourth embedding with an encoder (106) on a fifth embedding, the fifth embedding representing the subject in a second instance, determining a score depending on the second embedding, the fourth Embedding and the fifth embedding, the score representing a plausibility for the relation of the subject with the object in the context of the second instance.

Description

background

Processes for knowledge graph embedding ("Knowledge Graph Embedding", KGE) generate a single vector representation for entity instances and relation types in the corresponding knowledge graph ("Knowledge Graph", KG). Embedding captures global distribution semantics of the KG in relation to an entity and is optimized for the prediction of generally applicable facts. This is a knowledge graph task (KGT) known as link prediction. However, this assumption of general validity only rarely applies in real inference tasks, since the situational context is decisive in order to make nuanced predictions.

A single static KGE per entity and relation is not sufficient for many KGTs. Instead, entities and relations have to be brought into context through situation-related factors and their subjective history. This requires a different entity embedding for each situation, not just one that tries to be general.

As a result, there is a need to adapt static KGEs to situational and subjective contexts.

Description of the invention

The description discloses a model that generates relational embeddings that capture a situation-specific relational context, and entity embeddings that contain a subject's history of related observations.

The particularly computer-implemented method for processing a knowledge graph embedding of a knowledge graph comprises determining a first embedding, the first embedding representing a subject at a first instance, the knowledge graph comprising an entity for the subject, retrieving a second embedding from the knowledge graph embedding, the second embedding represents an object in the knowledge graph, wherein the knowledge graph comprises an entity for the object, retrieving at least one third embedding from the knowledge graph embedding, wherein the at least one third embedding represents context, wherein the knowledge graph comprises at least one entity for the context, determining a fourth embedding, the fourth embedding representing a relation in the knowledge graph between the subject and the object, the mapping of the first embedding, the second embedding, the at least one third embedding and the fourth embedding embedding with an encoder to a fifth embedding, whereby the fifth embedding represents the subject in a second instance, determining a score depending on the second embedding, the fourth embedding and the fifth embedding, the score being a plausibility for the relation of the subject represented with the object in context at the second instance.

The method may include operating a device as a function of the relation of the subject to the object in context at the second instance if the score meets a condition, and otherwise not operating the device as a function of the relation of the subject to the object in context in the second instance, the device being a computer-controlled machine, in particular a robot, a vehicle, a household appliance, an electric tool, a manufacturing machine, a personal assistant or an access control system. The second through fifth embeddings represent a temporal contextualized knowledge graph fact for the subject at the second instance in a sequence of contextualized knowledge graph facts. The score indicates the plausibility of the existence of the temporally contextualized knowledge graph fact. The score therefore indicates the plausibility that the object or the relation exists in the context. The score fulfills the condition, for example, if it exceeds a threshold value. The threshold may be a reference score determined for another temporally contextualized knowledge graph fact for the subject.

Operating the device can include: receiving data, in particular sensor data, including audio data, image data, video data, radar data, lidar data and / or metadata, the data including information about the subject, the object and the context in the first instance, and determining a classification of the data, in particular for location recommendation, event prediction and / or a driving scene classification, depending on the relation of the subject to the object in the context of the second instance, if the score meets the condition, or otherwise non-classification of the data depending on the Relation of the subject to the object in the context at the second instance, and then operating the device as a function of the classification.

The method may include retrieving the first embedding from the knowledge graph embedding. The embeddings retrieved from the knowledge graph embedding in the first instance are used for prediction for the second instance or for training the encoder for this prediction.

Determining the first embedding may include determining an embedding that the subject represents in an instance before the first instance, and the mapping of an embedding from the knowledge graph embedding, which represents an object in the knowledge graph, at least one embedding from the knowledge graph embedding, which represents the context, and an embedding, which a relation in the knowledge graph between the subject and the Object represented, with an encoder to encompass the first embedding. The first embedding is determined from an embedding that represents the subject. The first embedding therefore includes the history for the subject.

The method can either retrieve the embedding that the subject represents in the instance before the first instance from the knowledge graph embedding or determine the embedding that the subject represents in an instance before the first instance, depending on an embedding that the Subject represented and which is retrieved from the knowledge graph embedding. In a first step, the first embedding, which represents the subject, is retrieved from the knowledge graph embedding.

The method can provide training data comprising the subject, a reference object and a reference relation, the reference relation relating the subject and the reference object in a reference context, and training the encoder with a backpropagation of a deviation between the object and the reference object and / or a discrepancy between the relation and the reference relation. This enables the encoder to transform a global knowledge graph embedding into a user-defined embedding, taking into account situation-specific factors of the relation and the subjective history of the entity that the subject represents.

The method can map the first embedding, the second embedding, the at least one third embedding and the fourth embedding in a first part of the encoder to a representation of the first embedding, a representation of the second embedding, a representation of the at least one third embedding and a Representation of the fourth embedding and the mapping of the representation of the first embedding, the representation of the second embedding, the representation of the at least one third embedding and the representation of the fourth embedding in a second part of the encoder on the fifth embedding.

The method can include mapping the first embedding, the second embedding, the at least one third embedding and the fourth embedding with at least one self-attention layer in the first part of the encoder.

The method can include mapping the representation of the first embedding, the representation of the second embedding, the representation of the at least one third embedding and the representation of the fourth embedding with at least one attention layer in the second part of the encoder.

The device for processing a knowledge graph embedding of a knowledge graph comprises at least one processor which is designed for the following processes: determining a first embedding, the first embedding representing a subject in a first instance, the knowledge graph comprising an entity for the subject, retrieving a second Embedding from the knowledge graph embedding, the second embedding representing an object in the knowledge graph, the knowledge graph comprising an entity for the object, retrieving at least one third embedding from the knowledge graph embedding, the at least one third embedding representing context, the knowledge graph including at least one entity for comprises the context, determining a fourth embedding, the fourth embedding representing a relation in the knowledge graph between the subject and the object, mapping the first embedding, the second embedding, the at least one third embedding and de r fourth embedding with an encoder on a fifth embedding, whereby the fifth embedding represents the subject in a second instance, determining a score depending on the second embedding, the fourth embedding and the fifth embedding, the score being a plausibility for the relation of the Subject represented with the object in context at the second instance.

The at least one processor is advantageously designed to operate a device as a function of the relation of the subject to the object in the context of the second instance if the score fulfills a condition, and otherwise the device as a function of the relation of the subject to the object in the Context not to operate at the second instance, wherein the device is a computer-controlled machine, in particular a robot, a vehicle, a household appliance, an electric tool, a manufacturing machine, a personal assistant or an access control system.

The device provides temporally contextualized knowledge graph facts as a modeling template for situation-specific information in a knowledge graph. This adds a time sequence of hyper-edges to an existing static knowledge graph, i.e. time-stamped subject-relation-object triples, the relation being n-ary in order to capture n contextualizing factors. The device provides a deep learning framework that static global entity and relation embeddings are transformed into temporally contextualized embeddings, if corresponding temporally contextualized knowledge graph facts are available.

The device can comprise at least one sensor and / or an interface for sensor data, the at least one processor being designed to operate the device to supply data, in particular sensor data, which include audio data, image data, video data, radar data, lidar data and / or metadata received, wherein the data includes information about the subject, the object and the context in the first instance, and depending on the relation of the subject to the object in the context in the second instance a classification of the data in particular for a location recommendation, an event prediction and / or to determine a driving scene classification if the score meets the condition, or otherwise not to classify the data depending on the relation of the subject to the object in the context at the second instance, and then to operate the device depending on the classification.

The at least one processor can be designed to retrieve the first embedding from the knowledge graph embedding.

To determine the first embedding, the at least one processor can be designed to determine an embedding that represents the subject in an instance before the first instance, and an embedding from the knowledge graph embedding that represents an object in the knowledge graph, at least one embedding from the Knowledge graph embedding, which represents the context, and an embedding, which represents a relation in the knowledge graph between the subject and the object, to be mapped onto the first embedding with an encoder.

The at least one processor can be designed to either retrieve the embedding that the subject represents in the instance before the first instance from the knowledge graph embedding or by retrieving the embedding that the subject represents in an instance before the first instance, depending on an embedding determined which represents the subject and which is retrieved from the knowledge graph embedding.

The at least one processor can be designed to provide training data that include the subject, a reference object and a reference relation, the reference relation relating the subject and the reference object in a reference context, and the encoder with a backpropagation of a deviation between the object and to train the reference object and / or a deviation between the relation and the reference relation.

The at least one processor can be designed to set the first embedding, the second embedding, the at least one third embedding and the fourth embedding in a first part of the encoder to a representation of the first embedding, a representation of the second embedding, a representation of the at least one to map the third embedding and a representation of the fourth embedding and to map the representation of the first embedding, the representation of the second embedding, the representation of the at least one third embedding and the representation of the fourth embedding in a second part of the encoder on the fifth embedding.

The at least one processor can be designed to map the first embedding, the second embedding, the at least one third embedding and the fourth embedding with at least one self-attention layer in the first part of the encoder.

The at least one processor can be designed to map the representation of the first embedding, the representation of the second embedding, the representation of the at least one third embedding and the representation of the fourth embedding with at least one attention layer in the second part of the encoder.

• 1 stellt zumindest einen Teil einer Vorrichtung zum Verarbeiten einer Wissensgrapheneinbettung schematisch dar,
• 2 stellt schematisch einen Wissensgraphen dar,
• 3 stellt schematisch eine Abfolge von Encodern dar,
• 4 zeigt Schritte in einem Verfahren zum Verarbeiten der Wissensgrapheneinbettung.

Further advantageous embodiments emerge from the following description and the drawings. Drawings:

• 1 represents at least part of a device for processing a knowledge graph embedding schematically,
• 2 shows schematically a knowledge graph,
• 3 shows schematically a sequence of encoders,
• 4th Figure 12 shows steps in a method for processing knowledge graph embedding.

1 shows schematically at least part of a device 100 for processing a knowledge graph embedding 102 . The device 100 includes at least one processor 104 and at least one encoder 106 . The at least one processor 104 and the at least one encoder 106 are designed to perform steps in a method described below. The device 100 is for the operation of a device 108 designed.

The device 108 the example described below is a computer controlled Machine. In an example described below, the device is 108 a vehicle.

The device 108 can be a robot, a household appliance, a power tool, a production machine, a personal assistant or an access control system.

The device 108 in this example comprises at least one sensor 110 . The device 100 in the example includes an interface 112 for sensor data received from at least one sensor 110 be received. The device 108 in this example comprises at least one actuator 114 . the interface 112 in the example is for receiving the sensor data and for sending instructions for operating the at least one actuator 114 designed.

The device 100 includes a data connection 116 that at least temporarily use the at least one processor 104 with the memory of the knowledge graph embedding 102 , the at least one encoder 106 and the interface 112 connects. the interface 112 in the example is via a data connection 118 with the sensor 110 and via a data connection 120 with the actuator 114 tied together.

A knowledge graph 200 , which is embedded in the knowledge graph 102 is coded is with reference to 2 described.

The knowledge graph 200 includes an entity 202 for a subject. The subject in the example is the vehicle. The knowledge graph 200 includes an entity 204 for an object. The object in the example is a pedestrian. The knowledge graph 200 includes at least one entity 206 that represents context. The context in the example includes one or more traffic signs. In a first instance 208 that are on the left of 2 is illustrated includes the knowledge graph 200 a first relation 210 between the subject and the object. In a second instance 212 that are to the right of 2 is illustrated includes the knowledge graph 200 a second relation 214 between the subject and the object.

The first relation 210 in the knowledge graph 200 is a temporal instance of a contextualized knowledge graph fact that captures a first situation in which the subject is at a point in time t = 1. The second relation 214 in the knowledge graph 200 is a temporal instance of a contextualized knowledge graph fact that captures a second situation in which the subject is at a different point in time t = 2.

The second situation at time t = 2 is a result of operating the device 108 in the first situation at time t = 1. In the example represents the first relation 210 the first situation of the vehicle and the pedestrian in context in the first instance at time t = 1. The second relation 214 represents the second situation of the vehicle and the pedestrian in the context of the second instance at time t = 2.

The first situation and the second situation can be used in a training to use the encoder 106 to train to predict the second situation depending on the first situation. The first situation can be given as input to an already trained encoder for predicting the second situation.

The device 108 can be operated to collect training data comprising the subject, a reference object and a reference relation, the reference relation relating the subject and the reference object in a reference context.

The device 108 can be operated in the first situation according to a prediction of the second situation, which includes the relation between the subject and the object in the context at the second point in time t = 2.

In the example, the prediction can be an object in the knowledge graph 200 provide that indicates the second situation in which the vehicle is to move, provided that the vehicle is in the first situation.

In the example, the prediction can be an object in the knowledge graph 200 provide that indicates whether the second situation is a dangerous situation or not, provided that the vehicle is in the first situation.

In the example, the prediction can be an object in the knowledge graph 200 that indicates whether the vehicle should give priority or drive to arrive in the second situation, provided that the vehicle is in the first situation.

The knowledge graph embedding 102 includes a first embedding e ^ s, which represents the subject. The knowledge graph embedding 102 includes a second embedding e ^ o, which represents the object. The knowledge graph embedding 102 includes at least a third embedding e ^ c1, ..., e ^ cnc, which represents context. The knowledge graph embedding 102 includes a fourth embedding r, which is a relation in the knowledge graph 200 represented between the subject and the object.

In the example the subject is the same, in a sequence of time instances ti. The object that Relation and context can change from one time instance ti to another time instance ti + 1.

The at least one processor 104 is designed for the first embedding e ^ s, the second embedding e ^ o, the at least one third embedding e ^ c1, ..., e ^ cnc and the fourth embedding r with the at least one encoder 106 to be mapped onto a fifth embedding that represents the subject. The fifth embedding includes history.

3 represents at least part of a sequence of encoders 106 schematically. The encoder 106 may include a recurrent architecture and a restricted multi-person self-attention layer. These can be used to enforce a relational structure of temporally contextualized knowledge graph facts during the training period. The encoder 106 comprises a neural encoder stack with a feedback loop and limited multi-headed self-attention layers. The encoder 106 is designed to transform global knowledge graph embeddings into user-defined embeddings, taking into account situation-specific factors of the relation and the subjective history of the entity that the subject represents.

The encoder 106 is designed for contextualized embedding for downstream knowledge graph tasks such as link prediction for location recommendations, event prediction or driving scene classification.

The at least one processor 104 is designed for the first embedding e ^ s, the second embedding e ^ o, the at least one third embedding e ^ c1, ..., e ^ cnc and the fourth embedding r in a first part of the encoder on a representation of the first Embedding, a representation of the second embedding, a representation of the at least one third embedding and a representation of the fourth embedding to be mapped.

The at least one processor 104 is designed to map the representation of the first embedding, the representation of the second embedding, the representation of the at least one third embedding and the representation of the fourth embedding in a second part of the encoder on the fifth embedding.

The at least one processor 104 is designed for an instance ti + 1 with at least one self-attention layer 302 in the first part of the encoder 106 a first entrance 302-1 of the encoder 106 , which includes the first embedding e ^ s_ti of a previous instance ti, a second input 302-2 of the encoder 106 , which comprises the second embedding e ^ o_ti + 1, a third input 302-3 of the encoder 106 , which contains the at least one third embedding e ^ c1, ..., e ^ cnc; e ^ c1_ti + 1, ..., e ^ cnc_ti + 1, and a fourth input 302-4 of the encoder 106 , which includes the fourth embedding r ^ c_ti + 1 of this instance ti + 1. In the example the first input 302-1 , the second entrance 302-2 , the third entrance 302-3 and the fourth entrance 302-4 to a first exit 302-5 , a second exit 302-6 , the third exit 302-7 and a fourth exit 302-8 pictured.

The at least one processor 104 is designed for the instance ti + 1 with at least one attention layer 304 in the second part of the encoder 106 the representation of the first embedding at a first input 304-1 the at least one attention layer 304 , the representation of the second embedding at a second input 304-2 the at least one attention layer 304 , the representation of the at least one third embedding at a third input 304-3 the at least one attention layer 304 and the representation of the fourth embedding at a fourth input 304-4 the at least one attention layer 304 to a first exit 304-5 of the encoder 106 , which includes the fifth embedding.

The at least one processor 104 can be designed to have a score f depending on the first outcome 304-5 and the second entrance 302-2 and the fourth entrance 302-4 of the encoder 106 to determine. That is, the score f is derived from the representation of the subject e ^ s_ti + 1 of the instance ti + 1 and the embedding of the object e ^ o_ti + 1, which is entered for the instance ti + 1, and the relation r of the subject e ^ s and the object e ^ o_ti + 1 in the knowledge graph 200 certainly.

The at least one processor is optional 104 designed for the instance ti + 1 with the at least one attention layer 304 in the second part of the encoder 106 the representation of the first embedding at the first entrance 304-1 the at least one attention layer 304 , the representation of the second embedding at the second input 304-2 the at least one attention layer 304 , the representation of the at least one third embedding at the third input 304-3 the at least one attention layer 304 and the representation of the fourth embedding at the fourth input 304-4 the at least one attention layer 304 to a second exit 304-6 of the encoder 106 , which includes a contextualized relation.

The at least one processor 104 can be designed to determine the score f as a function of the second outcome 304-6 of the encoder 106 instead of the fourth entrance 302-4 of the encoder 106 to determine. This means that the score f from the contextualized relation r ^ c_ti + 1 between the Subject e ^ s and the object e ^ o_ti + 1 of the instance ti + 1 is determined.

A sequence of encoders 106 can be used for forecasting.

The first entrance 302-1 of the encoder 106 , which is used for mapping in the instance ti + 1, can be obtained from the first output 304-5 of the encoder used for mapping at the previous instance ti. The first input 302-1 of the encoder 106 , which is used for mapping in the instance ti, can be the embedding of the subject that is derived from the knowledge graph embedding 102 or can be the first output of another encoder 106 be. In the example, the first entry is 302-1 for the first encoder 106 that is in the sequence of the encoder 106 is used, the coding of the subject obtained from the knowledge graph embedding.

The at least one processor 104 can be designed to get the score for the final encoder 106 to be determined in the sequence. For training the encoder 106 the sequence can be the score f for the individual encoder 106 to be determined.

The knowledge graph 200 includes KG facts that are defined as a triple (e ^ s, r, e ^ o) in the example, where e ^ s, e ^ o ∈ {e ^ 1, ..., e ^ ne} embeddings for a Set of ne entity instances and r ∈ {r1, ..., m} embeddings for a set of m relation types.

KG facts represent subject-predicate-object statements that are assumed as static and stable background knowledge.

In the example, tKG facts are defined as quadruples (e ^ s, r, e ^ o, ti), where ti ∈ N indicates a point in a sequence t1, ..., tn at which the fact occurred. In an exemplary scenario, ti is obtained from the discretization of time stamps and thus generates a globally ordered set of facts, where n is the total number of times.

KG facts without a time dimension are considered true for any t.

In the example, cKG facts are defined as (n + 1) tuples (e ^ s, r ^ c, e ^ o, e ^ c1, ..., e ^ c_nc), which allow context to be nc-are To model relation, where e ^ c1, ..., e ^ c_nc are embeddings of nc context entities which influence a relation between a subject represented by the embedding e ^ s and an object represented by the embedding e ^ o. The relation is represented by an embedding r ^ c.

As with KG facts, cKG facts are viewed as non-dynamic background knowledge of a current situation.

In the example, tcKG facts are (n + 2) tuples (e ^ s, r ^ c, e ^ o, ti, e ^ c_1, ..., e ^ c_nc) that represent sequentially contextualized KG facts by they combine the features of tKGs and cKGs. tcKG facts capture a specific situation in which the subject represented by the embedding e ^ s is at the time ti. The embeddings e ^ c_1, ..., e ^ c_nc represent influencing factors, i.e. context, on the relation of the subject represented by the embedding e ^ s to the object represented by the embedding e ^ o.

4th Figure 12 shows steps in a method for processing knowledge graph embedding. In the example is the device 100 designed to carry out the method.

The method is repeated, for example, in instances ti that occur during the operation of the device 108 are executed. The method includes receiving data. In the example, sensor data is received that includes audio data, image data, video data, radar data, lidar data and / or metadata. The data includes information about the subject, the object and the context in instances ti.

The method comprises one step 402 . In step 402 becomes the first embedding e ^ s_t1 in the first instance t1 from the knowledge graph embedding 102 retrieved.

The embeddings that result from the knowledge graph embedding 102 retrieved in the first instance can be used for prediction for the second instance or to train the encoder 106 can be used for this prediction. In the example, an already trained sequence of encoders 106 described. The training is described below.

In the case of an nc-ary relation r ^ c, an entity involved in r ^ c is defined in the example as the subject represented by the embedding e ^ s, whose perspective is represented in relation to an object represented by the embedding e ^ o.

The contexts are given by the other entities involved in the nc-ary relation r ^ c and are represented by embeddings c ^ 1, ..., c ^ nc.

The context defines the influencing factors in a specific situation.

Thereafter, the method includes processing the knowledge graph embedding 102 with the Sequence of encoders 106 in n-1 instances t2, ..., tn. One instance is described below, the other instances are processed in the same way using the output of the previous instance.

A step 404 comprises determining the first embedding e ^ s_ti, the first embedding e ^ s_ti representing the subject at a first instance ti.

Determining the first embedding e ^ s_ti can include determining an embedding e ^ s_ti-1 which represents the subject in an instance ti-1 before the first instance ti. The determination of the first embedding e ^ s_ti can be the mapping of an embedding from the knowledge graph embedding 102 that is an object in the knowledge graph 200 represents, at least one embedding from the knowledge graph embedding 102 , which represents context, and an embedding, which is a relation in the knowledge graph 200 represented between the subject and the object, with the encoder 106 on the first embedding include e ^ s_ti.

A step 406 includes retrieving a second embedding from the knowledge graph embedding 102 , the second embedding an object in the knowledge graph 200 represents.

Depending on the application, the same embedding e ^ o can be used in the instances t1, ..., tn or different embeddings e ^ o_ti can be called up in several or any of the instances t1, ..., tn. In the example, the embedding e ^ s for the subject represents the vehicle and the embedding e ^ o for the object represents the pedestrian. In the example, the same embedding e ^ o is used for the same object in the instances t1, ..., tn.

A step 408 comprises retrieving at least a third embedding from the knowledge graph embedding 102 , wherein the at least one third embedding represents context.

Depending on the application, the same embeddings e ^ c1, ..., e ^ cnc can be used in the instances t1, ..., tn or different embeddings e ^ c1_ti, ..., e ^ cnc_ti can be used in several or one any of the instances t1, ..., tn can be called up. In the example, the context changes in the instances.

If present, the context can be expanded to include entities that depend deterministically on the object represented by the embedding e ^ o. Non-deterministically dependent context of the object represented by the embedding e ^ o or context or facts that are not only specific for a certain point in time ti must not be explicitly modeled.

A step 410 comprises determining a fourth embedding, the fourth embedding being a relation in the knowledge graph 200 represented between the subject and the object. In the example, the same embedding r is used in the instances t1, ..., tn.

The time context is in the encoder 106 taken into account by nc relation instances of relations, which are represented by different embeddings r ^ c and concern the subject represented by the embedding e ^ s.

Sorted by time stamp t, the relations represented by different embeddings r ^ c define a sequential context in the encoder 106 from the perspective of the subject represented by the embedding e ^ s to its relation to the object represented by the embedding e ^ o.

After that is a step 412 executed.

The step 412 comprises the mapping of the first embedding e ^ s_ti, the second embedding e ^ o, the at least one third embedding e ^ c1_ti + 1, ..., e ^ cnc_ti and the fourth embedding r with the encoder 106 on the fifth embedding e ^ s_ti + 1.

The fifth embedding e ^ s_ti + 1 represents the subject in the second instance ti + 1.

In the example, the first embedding e ^ s_ti, the second embedding e ^ o, the at least one third embedding e ^ c1_ti + 1, ..., e ^ cnc_ti + 1 and the fourth embedding r in a first part of the encoder ( 106 ) mapped onto a representation of the first embedding, a representation of the second embedding, a representation of the at least one third embedding and a representation of the fourth embedding.

In the example, the representation of the first embedding, the representation of the second embedding, the representation of the at least one third embedding and the representation of the fourth embedding are in the second part of the encoder 106 mapped to the fifth embedding e ^ s_ti + 1.

In the example, the first embedding e ^ s_ti, the second embedding e ^ o_ti + 1, the at least one third embedding e ^ c1_ti + 1, ..., e ^ cnc_ti + 1 and the fourth embedding r with at least one self-attention -Layer 302 in the first part of the encoder 106 pictured.

In the example, the representation of the first embedding, the representation of the second Embedding, the representation of the at least one third embedding and the representation of the fourth embedding with at least one attention layer 304 in the second part of the encoder 106 pictured.

The encoder 106 receives pre-trained static embeddings e ^ s, r, e ^ o, e ^ c1_ti, ..., e ^ cnc_ti and outputs a contextualized embedding r ^ c. In addition, the previous subjective memory of the subject, the first embedding e ^ s_ti, is passed on in order to generate the temporally contextualized embedding e ^ s_ti + 1 for the next instance ti + 1. Thus the only untrained embedding that is passed on to the next instance ti + 1 is e ^ s_ti + 1.

Inside the encoder 106 the embedding of the relation r is transformed into a contextualized relation r ^ c. Inside the encoder 106 the first embedding e ^ s_ti is transformed into the fifth embedding e ^ s_ti + 1. In the example, the encoder includes 106 a stack of encoder layers, each consisting of a self-attention layer followed by a feed-forward network.

The attention heads in each self-attention layer are designed to resemble the structure of the relations, which are defined by a temporarily contextualized knowledge graph with tcKG facts. Instead of pairwise attention for all inputs of the encoder 106 To calculate, the method can include assigning the contextualized embedding r ^ c to the input embeddings {e ^ s, r, e ^ o, e ^ c1_ti, ..., e ^ cnc_ti}.

The attention layers 302 and 304 can limit the attention of the embedding e ^ s_ti + 1 to input embeddings {e ^ s_ti, r ^ c_ti, e ^ o_ti}.

After that is a step 414 executed. The step 414 comprises determining the score f as a function of the second embedding e ^ o, the fourth embedding r and the fifth embedding e ^ s_ti + 1.

The score f represents the plausibility for the relation of the subject with the object in the context for the second instance ti + 1.

The encoder 106 may include stacked encoder layers, each with limited multi-headed self-attention, followed by a scoring function that returns a scalar score f for an (e ^ s, r, e ^ o) triple.

After that is a step 416 executed. The step 416 comprises determining a classification of the data as a function of the relation of the subject with the object in the context at the second instance ti + 1. In the example, the classification is a driving scene classification, e.g. B. the classification of the current situation as a situation in which the vehicle should give priority to the pedestrian or drive. In another example, the classification can be a location recommendation, e.g. B. for a route of the vehicle, or an event prediction, e.g. B. whether a situation is dangerous or not, or with regard to a behavior of the pedestrian. The location recommendation can relate to a navigation destination or to a trajectory for avoiding a dangerous situation.

In the example, the step includes 416 determining the classification if the score f meets the condition, or otherwise not classifying the data as a function of the relation of the subject with the object in the context in the case of the second instance ti + 1. In the example, the score provides plausibility. The classification is determined, for example, if the score meets the condition. The condition is fulfilled, for example, if the score indicates that the plausibility exceeds a threshold value.

The threshold value may be a reference score determined for another temporally contextualized knowledge graph fact for the subject. The steps 402 until 414 can be performed for several different objects or different contexts in order to determine individual scores. In this case, the temporally contextualized knowledge graph fact for the subject with the highest score can be selected to determine the classification.

After that is a step 418 executed. The step 418 includes operating the device 108 depending on the relation of the subject with the object in the context in the case of the second instance ti + 1, if the score f meets the condition, and otherwise not operating depending on the relation of the subject with the object in the context in the case of the second instance ti + 1.

In the example, the vehicle is being operated. The operation of the vehicle in the example depends on the classification if the plausibility of the classification exceeds the threshold value. Otherwise, the operation of the vehicle can remain unchanged or unaffected by the result of the processing of the knowledge graph embedding.

The encoder 106 can be trained with training data comprising the subject, a reference object and a reference relation.

The reference relation places the subject and the reference object in a reference context.

The training of the encoder 106 may include back propagation of a discrepancy between the object and the reference object and / or a discrepancy between the relation and the reference relation.

The score is determined with a scoring function. The deviation is determined with a loss function. Backpropagation aims to achieve a training goal.

The encoder 106 may comprise an artificial neural network. To optimize the weight matrices in the feed-forward and self-attention layers of the network, the backpropagation uses the scoring function as a measure of the plausibility of the predicted KG facts and a loss function that shows a deviation from the predicted KG facts from known KG facts -Measures facts from the training data. The encoder 106 is independent of the choice of the scoring function and the loss function. Any scoring function, loss function, and training goal can be used as long as it allows a gradient to be calculated.

In the example link prediction is used as the training goal. In one example, the training goal is the correct prediction of e ^ o_t + 1 for a given e ^ s_t and r ^ c_t, where e ^ o_t + 1 is masked out.

The weights adapt during training in such a way that a scoring function f (e ^ s_t, r ^ c_t, e ^ o_t) has a high score for the correct e ^ o_t + 1 from the training data and a low score for all other entities. The use of a soft max function on the predicted scores for the e ^ o used enables the calculation of the cross-entropy loss against a correct triple and backpropagation of the error.

Claims (10)

Device (100) for processing a knowledge graph embedding (102) of a knowledge graph (200), characterized by at least one processor (104) which is designed for the following processes: determining a first embedding, the first embedding representing a subject in a first instance, wherein the knowledge graph (200) comprises an entity for the subject, retrieving a second embedding from the knowledge graph embedding (102), the second embedding representing an object in the knowledge graph (200), the knowledge graph (200) comprising an entity for the object, Retrieving at least one third embedding from the knowledge graph embedding (102), the at least one third embedding representing context, the knowledge graph (200) comprising at least one entity for the context, determining a fourth embedding, the fourth embedding being a relation in the knowledge graph (200 ) is represented between the subject and the object, depicting the first embedding processing, the second embedding, the at least one third embedding and the fourth embedding with an encoder (106) on a fifth embedding, the fifth embedding representing the subject in a second instance, determining a score depending on the second embedding, the fourth Embedding and the fifth embedding, the score representing a plausibility for the relation of the subject with the object in the context of the second instance. Device according to Claim 1 characterized in that the at least one processor (104) is designed to operate a device (108) as a function of the relation of the subject to the object in the context at the second instance, if the score meets a condition, and otherwise the device (108) not to be operated in the context of the second instance depending on the relation of the subject to the object, the device (108) being a computer-controlled machine, in particular a robot, a vehicle, a household appliance, an electric tool, a manufacturing machine personal assistant or an access control system. Device (100) according to Claim 1 or 2 , characterized in that the device (100) comprises at least one sensor (110) and / or an interface (112) for sensor data, wherein for operating the device (108) the at least one processor (104) is designed to transmit data, in particular To receive sensor data comprising audio data, image data, video data, radar data, lidar data and / or metadata, the data comprising information about the subject, the object and the context in the first instance, and depending on the relation of the subject to the object to determine a classification of the data in the context at the second instance, in particular for location recommendation, event prediction and / or driving scene classification, if the score meets the condition, or otherwise the data depending on the relation of the subject to the object in the context at the second instance not to be classified, and then to operate the device depending on the classification. Device (100) according to Claim 1 , 2 or 3 , characterized in that the at least one processor (104) is designed to retrieve the first embedding from the knowledge graph embedding (102). Device (100) according to Claim 1 , 2 or 3 , characterized in that for determining the first embedding, the at least one processor (104) is designed to determine an embedding that represents the subject in an instance before the first instance, and an embedding from the knowledge graph embedding (102) that represents an object in the knowledge graph (200) to map at least one embedding from the knowledge graph embedding (102), which represents context, and an embedding, which represents a relation in the knowledge graph (200) between the subject and the object, onto the first embedding with an encoder (106). Device (100) according to Claim 5 , characterized in that the at least one processor (104) is designed to either retrieve the embedding that represents the subject in the instance before the first instance from the knowledge graph embedding (102) or by retrieving the embedding that the subject in a Instance represented before the first instance, determined as a function of an embedding which represents the subject and which is retrieved from the knowledge graph embedding (102). Device (100) according to one of the preceding claims, characterized in that the at least one processor (104) is designed to provide training data that include the subject, a reference object and a reference relation, the reference relation being the subject and the reference object in a reference context in relation, and to train the encoder with a backpropagation of a deviation between the object and the reference object and / or a deviation between the relation and the reference relation. Device (100) according to one of the preceding claims, characterized in that the at least one processor (104) is designed to process the first embedding, the second embedding, the at least one third embedding and the fourth embedding in a first part of the encoder (106 ) to a representation of the first embedding, a representation of the second embedding, a representation of the at least one third embedding and a representation of the fourth embedding and the representation of the first embedding, the representation of the second embedding, the representation of the at least one third embedding and the Representation of the fourth embedding in a second part of the encoder (106) to be mapped onto the fifth embedding. Device (100) according to Claim 8 , characterized in that the at least one processor (104) is designed to process the first embedding, the second embedding, the at least one third embedding and the fourth embedding with at least one self-attention layer (302) in the first part of the encoder ( 106). Device (100) according to Claim 8 or 9 , characterized in that the at least one processor (104) is designed for the representation of the first embedding, the representation of the second embedding, the representation of the at least one third embedding and the representation of the fourth embedding with at least one attention layer (304) to be mapped in the second part of the encoder (106).

Images