KR20230155943A - Method for embedding graph and system thereof - Google Patents

Abstract

A graph embedding method and system are provided. A graph embedding method according to some embodiments of the present disclosure includes obtaining a first embedding representation and a second embedding representation for a target graph, and reflecting a specific value in the second embedding representation to form the second embedding representation. It may include changing the value of and generating an integrated embedding expression for the target graph by aggregating the first embedding expression and the changed second embedding expression. According to this method, various embedding representations can be integrated without loss of information (or expressive power), resulting in a more powerful embedding representation for the target graph.

Description

Graph embedding method and system {METHOD FOR EMBEDDING GRAPH AND SYSTEM THEREOF}

This disclosure relates to a graph embedding method and system.

Graph embedding means converting a given graph into a vector or matrix representation in the embedding space. Recently, research on methods for embedding graphs using neural networks has been actively conducted, and neural networks that deal with graphs are called GNNs (Graph Neural Networks).

Meanwhile, different GNNs (e.g., GNNs with different structures, GNNs operating in different ways) generate different embedding vectors for the same graph. Therefore, if these embedding vectors can be integrated without loss of information (or expressive power), a more powerful embedding representation can be created.

To prevent information loss during the integration process, you can consider creating an integrated embedding vector by concatenating the embedding vectors as is. However, this method has an obvious problem in that the number of dimensions of the integrated embedding vector and the complexity of the associated task (e.g., downstream tasks such as classification and regression) increase in proportion to the number of connected embedding vectors.

Korean Patent Publication No. 10-2022-0032730 (published on March 15, 2022)

The technical problem to be solved through some embodiments of the present disclosure is to provide a method that can integrate various embedding representations of a graph without loss of information (or expressive power) and a system that performs the method.

Another technical problem to be solved through some embodiments of the present disclosure is to provide a method that can integrate various embedding representations for a graph without increasing the complexity of related tasks and a system that performs the method.

Another technical problem to be solved through some embodiments of the present disclosure is to provide a method for embedding node information and topology information of a graph together and a system for performing the method.

The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below.

A graph embedding method according to some embodiments of the present disclosure for solving the above-described technical problem is a method performed by at least one computing device, including a first embedding representation and a second embedding representation for the target graph. Obtaining an embedding expression, changing a value of the second embedding expression by reflecting a specific value in the second embedding expression, and aggregating the first embedding expression and the changed second embedding expression. It may include generating a unified embedding representation for the target graph.

In some embodiments, one of the first embedding expression and the second embedding expression is generated by an embedding method that aggregates information of neighboring nodes constituting the target graph, and the first embedding expression and the second embedding expression are generated by an embedding method that aggregates information of neighboring nodes constituting the target graph. Another one of the second embedding expressions may be generated by an embedding method that reflects topology information of the target graph.

In some embodiments, the first embedding expression and the second embedding expression may be generated by embedding the target graph through different graph neural networks (GNNs).

In some embodiments, the specific value may be an irrational number.

In some embodiments, the specific value is a value based on a learnable parameter, and predicting a label for a predetermined task based on the generated integrated embedding representation and the predicted label and the target graph. It may further include updating the value of the learnable parameter based on the difference between the correct answer labels.

In some embodiments, reflecting the specific value in the second embedding representation is performed based on a multiplication operation, and aggregating the first embedding representation and the modified second embedding representation is performed based on an addition operation. It can be.

In some embodiments, obtaining the first embedding representation and the second embedding representation may include obtaining a first embedding matrix and a second embedding matrix for the target graph, wherein the size of the first embedding matrix is different from the second embedding matrix - and obtaining the first embedding representation and the second embedding representation by performing a resizing operation on at least one of the first embedding matrix and the second embedding matrix. It can be included.

In some embodiments, obtaining the first embedding representation and the second embedding representation includes obtaining the first embedding representation through a graph neural network (GNN) of a neighboring node information aggregating method, and the first embedding representation It may include generating the second embedding representation by extracting topology information of the target graph using the embedding representation.

In some embodiments, generating the unified embedding representation may include performing a pooling operation on each of the first embedding representation and the changed second embedding representation, and aggregating the results of the pooling operation to generate the unified embedding. It may include the step of generating an expression.

In some embodiments, generating the integrated embedding representation includes obtaining a third embedding representation for the target graph, and reflecting a value different from the specific value in the third embedding representation to express the third embedding representation. It may include changing the value of and generating the integrated embedding representation by aggregating the first embedding expression, the changed second embedding expression, and the changed third embedding expression.

A graph embedding system according to some embodiments of the present disclosure for solving the above-described technical problem includes one or more processors and a memory that stores one or more instructions, wherein the one or more processors store the one or more instructions. An operation of obtaining a first embedding representation and a second embedding representation for a target graph by executing an instruction, an operation of changing a value of the second embedding representation by reflecting a specific value in the second embedding representation, and An operation of generating an integrated embedding representation for the target graph may be performed by aggregating the first embedding expression and the changed second embedding expression.

A computer program according to some embodiments of the present disclosure for solving the above-described technical problem includes the steps of combining with a computing device to obtain a first embedding representation and a second embedding representation for a target graph, changing the value of the second embedding expression by reflecting a specific value in the second embedding expression, and aggregating the first embedding expression and the changed second embedding expression to create a unified embedding expression for the target graph. It may be stored in a computer-readable recording medium in order to execute the generating step.

According to some embodiments of the present disclosure, an integrated embedding representation for the target graph can be generated by reflecting and aggregating a specific value in at least some of the various embedding representations for the target graph. In this case, since the values of different embedding expressions can be prevented from being offset by a specific value during the aggregating process, various embedding expressions can be integrated without loss of information (or expressiveness). For example, by multiplying a specific embedding expression by an irrational number, the values of different embedding expressions can be effectively prevented from being canceled out during the aggregating (e.g., addition) process.

Additionally, a specific value may be derived based on a learnable parameter. In this case, as learning to generate an integrated embedding expression progresses, the optimal value that can prevent loss of information (or expressive power) of the embedding expressions can be naturally and accurately derived.

Additionally, various embedding expressions (e.g., embedding vectors) for the target graph can be aggregated through addition operations. In this case, even if the number of embedding expressions increases, the size of the integrated embedding expression (e.g., the number of dimensions of the integrated embedding vector) does not increase, so there is a problem of increasing complexity of related tasks (e.g., downstream tasks such as classification and regression). It can be easily solved.

In addition, embedding representations (e.g., embedding matrices) of different sizes can be easily integrated through a resizing operation implemented based on a multi-layer perceptron, etc.

In addition, by integrating the node information-centered embedding expression and the topology-centered embedding expression, a powerful unified embedding expression for the target graph can be created, and by using the unified embedding expression, various tasks related to the graph (e.g., classification, regression, etc.) can be created. The accuracy of various downstream tasks) can also be greatly improved.

Additionally, a second embedding expression may be generated by extracting topology information of the graph from the first embedding expression output through the neighbor node information aggregating module. In this case, an integrated GNN that generates (outputs) an integrated embedding representation can be easily constructed by having the neighboring node information aggregating module function as a kind of shared neural network.

The effects according to the technical idea of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below.

1 and 2 are exemplary diagrams for schematically explaining the operation of a graph embedding system according to some embodiments of the present disclosure.
3 and 4 are exemplary diagrams for explaining a graph neural network (GNN) of a neighboring node information aggregating method that may be referred to in some embodiments of the present disclosure.
5 and 6 are exemplary diagrams for explaining a graph neural network (GNN) of a topology information extraction method that may be referred to in some embodiments of the present disclosure.
7 is an example flowchart illustrating a graph embedding method according to some embodiments of the present disclosure.
FIG. 8 is an exemplary diagram for further explaining a graph embedding method according to some embodiments of the present disclosure.
FIG. 9 is an exemplary diagram illustrating a learning process for graph embedding according to some embodiments of the present disclosure.
FIG. 10 is an exemplary diagram for explaining a graph embedding method according to some other embodiments of the present disclosure.
FIG. 11 is an exemplary diagram for explaining a graph embedding method according to some other embodiments of the present disclosure.
12 and 13 are exemplary diagrams for explaining a graph embedding method according to some other embodiments of the present disclosure.
14 illustrates an example computing device that can implement a graph embedding system according to some embodiments of the present disclosure.

Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings. The advantages and features of the present disclosure and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the technical idea of the present disclosure is not limited to the following embodiments and may be implemented in various different forms. The following examples are merely intended to complete the technical idea of the present disclosure and to cover the technical field to which the present disclosure belongs. is provided to fully inform those skilled in the art of the scope of the present disclosure, and the technical idea of the present disclosure is only defined by the scope of the claims.

In describing various embodiments of the present disclosure, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description will be omitted.

Unless otherwise defined, terms (including technical and scientific terms) used in the following embodiments may be used in a meaning that can be commonly understood by those skilled in the art in the technical field to which this disclosure pertains. It may vary depending on the intentions of engineers working in the related field, precedents, the emergence of new technologies, etc. The terminology used in this disclosure is for describing embodiments and is not intended to limit the scope of this disclosure.

The singular expressions used in the following embodiments include plural concepts, unless the context clearly specifies singularity. Additionally, plural expressions include singular concepts, unless the context clearly specifies plurality.

In addition, terms such as first, second, A, B, (a), (b) used in the following embodiments are only used to distinguish one component from another component, and the terms The nature, sequence, or order of the relevant components are not limited.

Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

FIG. 1 is an exemplary diagram schematically illustrating the operation of the graph embedding system 10 according to some embodiments of the present disclosure.

As shown in FIG. 1 , graph embedding system 10 may be a computing device/system capable of generating an embedding representation (e.g., 16 ) for a graph 13 . Specifically, the graph embedding system 10 may integrate various embedding representations 14 and 15 for the graph 13 to generate a unified embedding representation 16 for the graph 13. Figure 1 shows as an example a case where the graph embedding system 10 uses (or integrates) two GNNs 11 and 12 and/or embedding representations 14 and 15. The number of GNNs (Graph Neural Networks) and/or embedding representations may be three or more (see FIG. 11). Hereinafter, for convenience of explanation, the graph embedding system 10 will be abbreviated as 'embedding system 10'.

Each of the embedding expressions 14 and 15 may represent a vector or matrix representation of the graph 13. At this time, the matrix (e.g., matrix of three dimensions or more) may encompass the concept of a tensor. Embedding expression is sometimes used interchangeably with terms such as ‘embedding vector’, ‘embedding matrix’, ‘embedding code’, and ‘latent representation’. It can also be used.

As shown, the first embedding representation 14 may include graph information that is at least partially different from the second embedding representation 15 or may be generated through other GNNs 11 and 12. For example, the first embedding representation 14 is generated through a first GNN 11 and the second embedding representation 15 is generated through a second GNN 12 having a different representation power than the first GNN 11. It may have happened. In this case, as shown in Figure 2, the embedding system 10 integrates the embedding representations 14 and 15 to produce a unified embedding representation 16 that has stronger expressive power (or contains richer information). can do. As a more specific example, the first embedding representation 14 is generated through the first GNN 11 of the neighbor node information aggregating method, and the second embedding representation 15 is generated through the second GNN 12 of the topology information extraction method. Let's assume that it was created through . In this case, the embedding system 10 may integrate the two embedding expressions 14 and 15 to generate an integrated embedding expression 16 in which neighboring node information and topology information are reflected together. In relation to this, please refer further to the description of FIGS. 3 to 6, 12, and 13.

For reference, the embedding expressions 14 and 15 may correspond to the final output of the GNN or may be derived during the internal processing of the GNN. In addition, the first GNN 11 and the second GNN 12 may refer to separate models or different parts of the overall model (e.g., the integrated model illustrated in FIG. 12 or FIG. 13). It may be possible. Additionally, GNNs 11 and 12 may or may not be learned models.

In addition, differences in the expressive power of GNNs (or embedding representations) mean that, for example, different embedding representations (e.g., embedding representations with different embedded information) are generated (extracted) for the same graph, or a set of graphs distinguished by a GNN is generated (extracted). It can mean that they are different. For a more specific example, assume that a graph classification task is performed using the embedding representations of the first GNN and the second GNN. And, using the embedding representation of the first GNN (e.g., GNN of the neighboring node information aggregating method), the first graph and the second graph are distinguished (i.e., classified into different classes), but the third graph and the fourth graph are distinguished from each other (i.e., classified into different classes). Assume that the graph is indistinguishable. In contrast, when using the embedding representation of the second GNN (e.g., GNN using the topology information extraction method), assume that the first graph and the second graph are not distinguished, but the third graph and the fourth graph are distinguished. In this case, it can be explained that the expressive power of the first GNN is different from that of the second GNN.

In addition, the expressive power of the first GNN (or the first embedding representation) is stronger than that of the second GNN (or the second embedding representation). For example, the embedding representation of the first GNN (or the first embedding representation) is stronger than that of the second GNN. It may mean that it contains richer information than the embedding representation (or the second embedding representation), or that the set of graphs distinguished by the first GNN is larger than that of the second GNN. For a more specific example, assume that a graph classification task is performed using the embedding representations of the first GNN and the second GNN. Also, let us assume that all 10 graphs are distinguished when using the embedding expression of the first GNN, but some of the same 10 graphs are not distinguished when using the embedding expression of the second GNN. In this case, the expressive power of the first GNN can be explained as being stronger than that of the second GNN, and the first GNN can be understood as an excellent model that generates an embedding representation by better reflecting changes in the input graph than the second GNN. there is.

The embedding system 10 learns the modules (e.g., GNNs, resizing module, etc.) and parameters necessary for generating the integrated embedding representation 16 by performing a predetermined task (e.g., classification task, regression task, etc.). You can do it. If the target task of the integrated embedding expression 16 is predetermined, the embedding system 10 can perform learning using the target task.

In addition, the embedding system 10 may directly perform the target task using learned modules and parameters, or may provide an integrated embedding representation (e.g., 16) for the input graph to the device performing the target task. there is.

The specific method by which the embedding system 10 generates the integrated embedding representation will be described in detail later with reference to the drawings of FIG. 7 and below.

Embedding system 10 may be implemented with at least one computing device. For example, all functionality of embedding system 10 may be implemented in a single computing device, wherein a first functionality of embedding system 10 is implemented in a first computing device and a second functionality is implemented in a second computing device. It could be. Alternatively, specific functionality of embedding system 10 may be implemented in multiple computing devices.

The computing device may include any device equipped with a computing function. Refer to FIG. 14 for an example of such a device. Since a computing device is a collection of interacting various components (e.g. memory, processor, etc.), it may be called a 'computing system' in some cases. Additionally, a computing system may refer to a collection of interacting computing devices.

So far, the operation of the embedding system 10 according to some embodiments of the present disclosure has been schematically described with reference to FIGS. 1 and 2 . Hereinafter, for convenience of understanding, the GNN of the neighbor node information aggregation method and the topology information extraction method will be briefly described with reference to FIGS. 3 to 6.

First, with reference to FIGS. 3 and 4, the structure and operation of the GNN using the neighboring node information aggregation method will be briefly described.

FIG. 3 illustrates a GNN 30 of a neighboring node information aggregating scheme that may be referenced in some embodiments of the present disclosure.

As shown in FIG. 3, the illustrated GNN 30 receives a graph 33 (more precisely, graph data) and repeatedly aggregates information (e.g., features) of neighboring nodes to generate the input graph 33. ) may operate to generate an embedding representation 34 for ). To this end, the GNN 30 may be configured to include a plurality of blocks 31-1 to 31-n and a pooling module 32, and may further include other modules depending on the case.

A plurality of blocks 31-1 to 31-n may repeatedly aggregate information of neighboring nodes constituting the graph 33. Each block (e.g., 31-1) may be composed of, for example, multiple multi-layer perceptrons (i.e., fully connected layers), but the scope of the present disclosure is not limited thereto.

Next, the pooling module 32 may perform a pooling operation to generate (output) an embedding representation 34 of an appropriate size. Anyone working in the relevant technical field will already be familiar with the pooling operation used in GNN, so description thereof will be omitted. In some cases, the pooling module 32 may be used interchangeably with terms such as 'pooling layer', 'readout layer', and 'readout module'.

The embedding representation (34) generated through the example GNN (30) usually well reflects the node information of the graph (33), but also reflects the topology information (e.g., overall shape/shape of the graph, etc.) of the graph (33) well. It is not reflected. Therefore, if the embedding expression 34 is used, performance on tasks in which topology information is important will inevitably deteriorate.

A more detailed example of the GNN 30 described above is shown in FIG. 4.

The GNN 40 illustrated in Figure 4 is a feature matrix for a node tuple (e.g., when the number of nodes is 'v' and the number of dimensions of features is 'd', a three-dimensional matrix of size v*v*d Matrix) can be input and operated to generate (output) an embedding vector (45, e.g., 'p' dimension embedding vector).

Specifically, the GNN 40 creates an embedding matrix (e.g., with a size of v*v*p) for the node tuple by repeatedly aggregating the feature matrix input through a plurality of blocks 41-1 to 41-n. A 3D matrix) can be created, and finally, an embedding vector 45 for the input graph can be generated (output) through the pooling module 42.

For reference, GNNs such as PPGN (Provably Powerful Graph Network) operate in a similar manner to Figure 4. For this, please refer to the paper titled 'Provably Powerful Graph Network'.

Next, the structure and operation of the GNN using the topology information extraction method will be briefly described with reference to FIGS. 5 and 6.

FIG. 5 illustrates a GNN 50 of a topology information extraction method that may be referenced in some embodiments of the present disclosure.

As shown in FIG. 5 , the illustrated GNN 50 may operate to generate an embedding representation 55 of the input graph 54 by reflecting topology information extracted from the input graph 54 . In the drawings of FIG. 5 and below, the embedding expression with added shading (e.g., 55) means an embedding expression in which topology information is reflected.

The GNN 50 may be configured to include a plurality of blocks 51-1 to 51-n, a topology information extraction module 52, and a pooling module 53, and may further include other modules in some cases. there is.

Similar to the GNN 30 illustrated in FIG. 3, multiple blocks 51-1 to 51-n may repeatedly aggregate information of neighboring nodes constituting the graph 55. Each block (e.g., 51-1) may be composed of, for example, multiple multi-layer perceptrons (i.e., fully connected layers), but the scope of the present disclosure is not limited thereto.

Next, the topology information extraction module 52 can extract topology information of the input graph 54. For example, the topology information extraction module 52 may extract topology information by calculating a persistence diagram. Anyone working in the relevant technical field will already be familiar with the concept and calculation method of a persistence diagram, so description thereof will be omitted.

Next, the pooling module 53 may perform a pooling operation to generate (output) an embedding representation 55 of an appropriate size.

The embedded representation 55 generated through the illustrated GNN 50 usually well reflects the topology information of the graph 54, but does not well reflect the node information of the graph 54. Therefore, if the embedding representation 55 is used, performance on tasks in which graph node information is important will inevitably deteriorate.

A more detailed example of the GNN 50 described above is shown in FIG. 6.

The GNN 60 illustrated in FIG. 6 receives and embeds a feature matrix for a node (e.g., when the number of nodes is 'v' and the dimensionality of features is 'd', a two-dimensional matrix of size v*d). It can be operated to output a vector (65, e.g., 't' dimension embedding vector).

Specifically, the GNN 60 may generate an embedding matrix for a node by repeatedly aggregating a feature matrix input through a plurality of blocks 61-1 to 61-n. Then, the GNN (60) extracts topology information from the node embedding matrix through the persistence diagram calculation module (62) and reflects the extracted information to generate an embedding matrix (65, e.g., a two-dimensional matrix with size v*t). This can be done, and finally, an embedding vector 65 for the input graph can be generated (output) through the pooling module 63.

For reference, GNNs such as GFL (Graph Filtration Learning) operate in a similar manner to Figure 6. For this, please refer to the paper titled 'Graph Filtration Learning'.

So far, GNNs 30 to 60 that can be referenced in some embodiments of the present disclosure have been described with reference to FIGS. 3 to 6. Hereinafter, various methods that can be performed in the above-described embedding system 10 will be described with reference to the drawings of FIG. 7 and below. However, to provide convenience of understanding, the description will be continued assuming that all steps/operations of the methods to be described later are performed in the embedding system 10. Accordingly, when the subject of a specific step/action is omitted, it can be understood as being performed in the embedding system 10. However, in a real environment, some steps/operations of the methods described below may be performed on other computing devices.

First, a graph embedding method according to some embodiments of the present disclosure will be described in detail with reference to FIGS. 7 to 9.

This embodiment relates to a method for integrating two embedding representations for a target graph. A method of integrating three or more embedding expressions will be described later with reference to FIG. 11.

7 is an example flowchart illustrating a graph embedding method according to some embodiments of the present disclosure. However, this is only a preferred embodiment for achieving the purpose of the present disclosure, and of course, some steps may be added or deleted as needed.

As shown in Figure 7, this embodiment may begin at step S71 of obtaining a first embedding representation and a second embedding representation for the target graph. For example, as shown in FIG. 8, the embedding system 10 obtains the first embedding representation 83 through the first GNN 81 and uses a second GNN 82, e.g., which has different representation power from the first GNN. The second embedding representation 84 can be obtained through GNN). At this time, the first GNN 81 may be a neighboring node information aggregating GNN (see FIGS. 3 and 4), and the second GNN 82 may be a topology information extraction GNN (see FIGS. 5 and 6). , the scope of the present disclosure is not limited thereto. Alternatively, the first embedding expression 83 is generated by an embedding method that aggregates information (e.g., feature information) of neighboring nodes of the target graph (i.e., node information-centered embedding expression), and the second embedding expression is (84) may be generated by an embedding method that reflects the topology information of the target graph (i.e., a topology information-centered embedding expression). However, the scope of the present disclosure is not limited thereto.

As described above, the first embedding expression and the second embedding expression may correspond to the final output of the GNN or may be derived during the internal processing of the GNN. For example, the first embedding expression and the second embedding expression may be an embedding vector obtained as a result of a pooling operation, or may be an embedding matrix (e.g., a 2-dimensional/3-dimensional matrix, etc.) derived before the pooling operation. For reference, if the first embedding expression and the second embedding expression are embedding vectors obtained as a result of a pooling operation, unlike the example in FIG. 8, the pooling operation may not be performed.

In step S72, a specific value (e.g., a scalar value) may be reflected in the second embedding expression, and as a result, the value of the second embedding expression may be changed. For example, as shown in Figure 8, the embedding system 10 reflects a specific value (e.g., '+ε') in the second embedding expression 84 through a multiplication operation (e.g., specific to each element of the embedding matrix (multiply the value). Here, the reason for reflecting a specific value in the second embedding expression 84 is to prevent the values of the first embedding expression 83 and the second embedding expression 84 from being offset during the aggregating (e.g., addition operation) process. It can be understood that it is for. This is because the fact that the values are offset means that the information (or expressive power) inherent in the two embedding expressions 84 and 85 is lost.

Figure 8 shows as an example a case where a specific value is reflected only in the second embedding expression 84, but the scope of the present disclosure is not limited thereto. For example, the embedding system 10 may reflect a specific value in the first embedding expression 83 instead of the second embedding expression 84. Alternatively, the embedding system 10 may reflect the first specific value in the first embedding representation 83 and a second specific value (i.e., a value different from the first specific value) in the second embedding representation 84. there is.

Meanwhile, the specific method of deriving/generating a specific value may vary depending on the embodiment.

In some embodiments, the specific value may be a preset value. For example, a specific value is a value based on a type of hyperparameter (e.g., 'ε') and can be set in advance by the user. As a more specific example, the specific value may be set to an irrational number. This is because if one (or both) of the two embedding expressions is multiplied by an irrational number, the values of the two embedding expressions can be effectively prevented from being offset during the aggregating (e.g., addition operation) process.

In some other embodiments, the specific value may be a value derived based on a learnable parameter (e.g., when the learnable parameter is 'ε', the specific value is the value of ε itself, ε + an irrational number, etc. could be). For example, the embedding system 10 can predict the label for a predetermined task (i.e., a learning task) using an integrated embedding expression and learn based on the difference between the predicted label and the correct label (i.e., the correct label of the target graph). The value of the parameter can be updated. In this case, as learning to generate an integrated embedding expression progresses, the optimal value that can prevent loss of information (or expressive power) of the embedding expressions can be naturally and accurately derived.

In step S73, a unified embedding representation for the target graph may be generated by aggregating the first embedding representation and the changed second embedding representation. For example, as illustrated in FIG. 8, the embedding system 10 performs a pooling operation on each of the first embedding representation 83 and the changed second embedding representation (not shown), and produces the results 85 and 86 of the pooling operation. ) can be aggregated to create a unified embedding representation (87, e.g., embedding vector). However, as described above, if the first embedding expression 83 and the second embedding expression 84 are obtained as a result of a pooling operation (e.g., if it is an embedding vector), the pooling operation is omitted among the operations illustrated in FIG. 8. It could be.

Figure 8 shows as an example a case where the pooled embedding expressions 85 and 86 are aggregated through an addition operation, but the scope of the present disclosure is not limited thereto. However, as illustrated in FIG. 8, if the embedding vectors 83 and 84 are integrated (aggregated) using a multiplication operation and an addition operation, the information (or expressive power) inherent in the embedding expressions 83 and 84 It has been mathematically proven by the inventor of the present disclosure that is preserved. In addition, when the addition operation is used, even if the number of integrated embedding expressions (e.g., 83, 84) increases, the size of the integrated embedding expression (87) (e.g., the number of dimensions of the integrated embedding vector) does not increase, so the correlation There is also the advantage that the problem of increasing complexity of tasks (e.g., downstream tasks such as classification, regression, etc.) can be easily solved.

Meanwhile, although not shown in FIG. 7, the embedding system 10 can learn the modules and parameters necessary to generate an integrated embedding representation by performing a predetermined task (i.e., a learning task). Specifically, as shown in Figure 9, the embedding system 10 inputs the integrated embedding representation 87 into a task-specific prediction module 91, or 'prediction layer', to obtain a label 92, e.g., for the target graph. , class label) can be predicted. And, the embedding system 10 determines related modules/parameters (e.g., prediction module 91, specific value) based on the difference (e.g., loss 94) between the prediction label 92 and the correct answer label 93 for the target graph. Parameters for derivation, resizing module illustrated in FIG. 10, first GNN 81, second GNN 82, etc.) can be learned. This learning process can, of course, be performed repeatedly on multiple graphs (i.e., training set) given the correct answer label (e.g., 93).

For reference, if the predetermined task is a classification task, the prediction module 91 may be implemented, for example, as a neural network layer (e.g., fully connected layer) configured to predict class labels. However, the scope of the present disclosure is not limited thereto, and the detailed structure of the prediction module 91 may be modified depending on the task.

So far, a graph embedding method according to some embodiments of the present disclosure has been described with reference to FIGS. 7 to 9. According to the above, an integrated embedding expression for the target graph can be created by reflecting and aggregating specific values in at least some of the various embedding expressions for the target graph. In this case, since the values of different embedding expressions can be prevented from being offset by a specific value during the aggregating process, various embedding expressions can be integrated without loss of information (or expressiveness). For example, by multiplying a specific embedding expression by an irrational number, the values of different embedding expressions can be effectively prevented from being canceled out during the aggregating (e.g., addition) process. Additionally, various embedding expressions (e.g., embedding vectors) for the target graph can be aggregated through addition operations. In this case, even if the number of embedding expressions increases, the size of the integrated embedding expression (e.g., the number of dimensions of the integrated embedding vector) does not increase, so there is a problem of increasing complexity of related tasks (e.g., downstream tasks such as classification and regression). It can be easily solved.

Hereinafter, a graph embedding method according to some other embodiments of the present disclosure will be described with reference to FIG. 10. However, for clarity of the present disclosure, description of content that overlaps with the previous embodiments will be omitted.

As shown in FIG. 10, this embodiment performs resizing when the sizes (e.g., size of the embedding matrix, number of dimensions of the embedding vector) of the first embedding expression 103 and the second embedding expression 104 are different. ) operation (or module) is further used to generate a unified embedding representation (107). Figure 10 assumes that the embedding representations 103 and 104 are embedding matrices generated through different GNNs 101 and 102.

Specifically, the embedding system 10 may perform a resizing operation on the first embedding representation 103 and the second embedding representation 104. For example, when the two embedding expressions 103 and 104 are matrix-type expressions, the embedding system 10 can adjust the size of the two embedding expressions 103 and 104 through an operation that changes the size of the matrix. Figure 10 shows that the resizing operation is performed on the length of the last dimension (eg, 'p ₁ ', 'p ₂ ') of the two embedding matrices 103 and 104 (or the number of dimensions of the embedding vectors 105 and 106 obtained as a result of the pooling operation). ) is exemplified for the purpose of matching. However, the scope of the present disclosure is not limited thereto. In addition, Figure 10 shows as an example a case where the resizing operation is implemented based on a multi-layer perceptron. The multi-layer perceptron can naturally change the size of the input embedding matrix through multiplication of the input embedding matrix and the weight matrix (i.e., its weight parameters). However, the scope of the present disclosure is not limited thereto, and the resizing operation may be implemented in other ways.

Next, the embedding system 10 may perform operations such as reflecting a specific value and pooling, and generate an integrated embedding expression 107 by aggregating the results 105 and 106 of the pooling operation. For example, the embedding system 10 may perform an addition operation on the embedding vectors 105 and 106 obtained as a result of the pooling operation to generate an integrated embedding representation 107 in the form of a vector.

Meanwhile, Figure 10 shows as an example a case where a resizing operation is performed on both of the two embedding representations 103 and 104, but of course, in some cases, a resizing operation may be performed on only one of them.

In addition, Figure 10 shows as an example a case where the resizing operation is performed before the pooling operation is performed, but in some cases, the resizing operation may be performed after the pooling operation. For example, a resizing operation may be performed on the embedding vectors (e.g., 105, 106) obtained as a result of the pooling operation.

In addition, Figure 10 shows as an example a case where the resizing operation is performed before a specific value (e.g., '1+ε') is reflected, but in some cases, after a specific value (e.g., '1+ε') is reflected. A resizing operation may be performed on .

Meanwhile, in some embodiments, a multi-layer perceptron may be applied even when the first and second embedding representations 103 and 104 have the same size. In this case, the multi-layer perceptron can play the role of converting a given embedding representation (e.g., 103, 104) into a representation of an appropriate embedding space (e.g., space of other embedding representations, joint embedding space, etc.). Accordingly, in this embodiment, the multi-layer perceptron may be named by terms such as 'transformation module', 'transformation layer', 'projection module', and 'projection layer'.

So far, the graph embedding method according to several other embodiments of the present disclosure has been described with reference to FIG. 10. According to the above, embedding representations (e.g., embedding matrices) of different sizes can be easily integrated through a resizing operation implemented based on a multi-layer perceptron, etc.

Hereinafter, a graph embedding method according to some other embodiments of the present disclosure will be described with reference to FIG. 11. However, for clarity of the present disclosure, description of content that overlaps with the previous embodiments will be omitted.

As shown in Figure 11, this embodiment aggregates (integrates) K (e.g., 3 or more) embedding expressions 112-1 to 112-k to create an integrated embedding expression 116 for the target graph. ) is about how to create. Figure 11 assumes the case where K embedding representations (112-1 to 112-k) are embedding matrices generated through different GNNs (111-1 to 111-k), and the sizes are also different. I am assuming.

Specifically, the embedding system 10 may perform a resizing operation on the K embedding expressions 112-1 to 112-k and reflect the specific values 113 and 114. At this time, the specific values (e.g., 113, 114) reflected in each embedding expression (e.g., 112-2, 112-k) may be different values (e.g., irrational numbers) or values based on different learnable parameters. . For example, the specific value 113 reflected in the second embedding expression 112-2 is the first irrational number, and the specific value 114 reflected in the Kth embedding expression 112-k is the second irrational number (i.e. , a value different from the first irrational number). In such a case, the values of different embedding expressions 112-1 to 112-k can be effectively prevented from being offset during the aggregation (e.g., addition) process.

Next, the embedding system 10 may perform a pooling operation and aggregate the results 115-1 to 115-k of the pooling operation to generate a unified embedding representation 116. For example, the embedding system 10 may perform an addition operation on the embedding vectors 115-1 to 115-k obtained as a result of the pooling operation to generate an integrated embedding representation 116 in the form of a vector.

So far, a graph embedding method according to some other embodiments of the present disclosure has been described with reference to FIG. 11. Hereinafter, a graph embedding method according to some other embodiments of the present disclosure will be described with reference to FIGS. 12 and 13. However, for clarity of the present disclosure, description of content that overlaps with the previous embodiments will be omitted.

As shown in FIG. 12, this embodiment relates to a method of generating an integrated embedding representation 125 in which the neighboring node information and topology information of the target graph 121 are reflected together using an integrated GNN. The structure of the integrated GNN illustrated in FIG. 12 aggregates neighboring node information between GNNs (see GNNs 30 and 50 illustrated in FIGS. 3 and 5) of two methods (i.e., neighboring node information aggregating method and topology information extraction method). It can be understood that it was derived by focusing on the fact that the module for gating (i.e., neural network) is common. The integrated GNN illustrated in FIG. 12 will be described as a first GNN (e.g., GNN 30 in FIG. 3) and a second GNN (e.g., GNN 50 in FIG. 5) sharing the node information aggregating module 122. It may be possible.

Specifically, the embedding system 10 may generate a first embedding representation 123 or 124 for the target graph 121 through the neighboring node information aggregating module 122. The first embedding representation (123 or 124) may be in the form of a three-dimensional matrix (see 123) or a two-dimensional matrix (see 124).

Next, the embedding system 10 may analyze the first embedding expression 123 or 124 to extract topology information and generate a second embedding expression (not shown) by reflecting the extracted topology information.

Next, the embedding system 10 performs operations such as resizing, reflecting specific values, and pooling on the first embedding expression (123 or 124) and the second embedding expression (not shown), and integrates the performance results by aggregating them. An embedding representation 125 can be created. For example, the embedding system 10 can generate a unified embedding representation 125 in vector form.

In order to provide easier understanding, further explanation will be made with reference to FIG. 13.

Figure 13 shows a more detailed example of the integrated GNN of Figure 12. The integrated GNN shown in FIG. 13 may be understood as, for example, an integration of the GNNs 40 and 60 illustrated in FIGS. 4 and 6.

As shown in FIG. 13 , the embedding system 10 may generate a first embedding representation 133 for the target graph through the neighbor node information aggregating module 132. Figure 13 shows that the neighboring node information aggregating module 132 displays a feature matrix 131 for a node tuple (e.g., when the number of nodes is 'v' and the dimensionality of features is 'd', the size is v*v*d. This example illustrates a case where a 3D matrix) is input and an embedding matrix (133, e.g., 3D matrix of size v*v*p) for a node tuple is output (generated).

Next, the embedding system 10 can generate a second embedding expression 135 that reflects the topology information of the target graph by calculating a persistence diagram using the first embedding expression 133. For example, when the first embedding representation 133 is a three-dimensional embedding matrix, the embedding system 10 extracts the diagonal elements of the embedding matrix 133 to generate a two-dimensional embedding matrix 134, and The second embedding representation (135) can be generated by computing the persistence diagram for (134). Here, the reason for extracting the diagonal elements is that the diagonal elements of the 3D embedding matrix 133 represent a part where the information of each node is concentrated. However, in some cases, the two-dimensional embedding matrix 134 may be generated in a different way.

Next, the embedding system 10 performs operations such as resizing, reflecting specific values, and pooling on the first embedding expression 133 and the second embedding expression 135 (136-1, 136-2, 137- 1 and 137-2), the performance results 137-1 and 137-2 can be aggregated to generate an integrated embedding expression 138. For example, the embedding system 10 can generate a unified embedding representation 138 in vector form.

So far, a graph embedding method according to some other embodiments of the present disclosure has been described with reference to FIGS. 12 and 13. According to the above, an integrated embedding expression that reflects the node information and topology information of the target graph can be easily created through the integrated GNN, and the overall performance of various tasks related to the graph can be improved by using this integrated embedding expression. You can.

Below, we will briefly introduce the performance test results for the above-described graph embedding method (hereinafter referred to as the ‘proposed method’).

The present inventors conducted an experiment to evaluate the accuracy of the graph classification task using datasets in the bioinformatics field (MUTAG, PTC, PROTEINS, NCI1). This is because high task accuracy means that the performance of the graph embedding method is excellent. Specifically, the present inventors performed an experiment to generate a unified embedding representation through an integrated GNN as illustrated in FIG. 13 and predict the class of the input graph based on the generated integrated embedding representation. Additionally, the present inventors performed the same experiment on 'PPGN', a representative GNN of the neighboring node information aggregating method. The experimental results are listed in Table 1 below.

division MUTAG PTC PROTEINS NCI1 PPGN 90.55 66.17 77.2 83.19 proposed method 92.78 70.88 78.11 84.67

Referring to Table 1, it can be seen that the performance of the proposed method is better than PPGN regardless of the dataset. This is believed to be because the integrated embedding representation generated by the proposed method includes not only node-centered information but also graph topology information, and through this, the proposed method provides a more powerful embedding representation than the GNN of the neighboring node information aggregating method. You can see that it can be created. In addition, it can be seen that almost no loss of information (or expressive power) occurs during the process of generating the integrated embedding representation.

In addition, the present inventors also conducted an experiment to evaluate the accuracy of the regression task using the QM9 (Quantum Machines 9) dataset in the bioinformatics field. Specifically, the present inventors generate a unified embedding representation through an integrated GNN as illustrated in Figure 13, predict the values of the targets listed in Table 2 below based on the generated integrated embedding expression, and average absolute values for the predicted values. An experiment was performed to measure the error. Additionally, the same experiment was performed for 'PPGN'. The experimental results are listed in Table 2 below.

target PPGN proposed method Dipole moment 0.231 0.0844 Isotropic polarizability 0.382 0.0181 HOMO(Highest occupied molecular orbital energy) 0.00287 0.00169 Lowest unoccupied molecular orbital energy (LUMO) 0.00309 0.0018

Referring to Table 2, it can be seen that the performance of the proposed method is better than PPGN even for regression tasks. This shows that the overall performance of various tasks related to graphs can be improved by using the proposed method.

So far, we have briefly introduced the performance test results for the graph embedding method according to some embodiments of the present disclosure. Hereinafter, an exemplary computing device 140 capable of implementing the embedding system 10 described above with reference to FIG. 14 will be described.

14 is an exemplary hardware configuration diagram showing the computing device 140.

As shown in FIG. 14, the computing device 140 includes one or more processors 141, a bus 143, a communication interface 144, and a memory (loading) a computer program executed by the processor 141. 142) and a storage 145 for storing a computer program 146. However, only components related to the embodiment of the present disclosure are shown in FIG. 14. Accordingly, a person skilled in the art to which this disclosure pertains can recognize that other general-purpose components may be included in addition to the components shown in FIG. 14 . That is, the computing device 140 may further include various components in addition to those shown in FIG. 14 . Additionally, in some cases, the computing device 140 may be configured with some of the components shown in FIG. 14 omitted. Hereinafter, each component of the computing device 140 will be described.

The processor 141 may control the overall operation of each component of the computing device 140. The processor 141 is at least one of a Central Processing Unit (CPU), Micro Processor Unit (MPU), Micro Controller Unit (MCU), Graphic Processing Unit (GPU), or any type of processor well known in the art of the present disclosure. It can be configured to include. Additionally, the processor 141 may perform operations on at least one application or program to execute operations/methods according to various embodiments of the present disclosure. Computing device 140 may include one or more processors.

Next, memory 142 may store various data, commands and/or information. Memory 142 may load a computer program 146 from storage 145 to execute operations/methods according to various embodiments of the present disclosure. The memory 142 may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

Next, bus 143 may provide communication functionality between components of computing device 140. The bus 143 may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

Next, the communication interface 144 may support wired and wireless Internet communication of the computing device 140. Additionally, the communication interface 144 may support various communication methods other than Internet communication. To this end, the communication interface 144 may be configured to include a communication module well known in the technical field of the present disclosure.

Next, storage 145 may non-transitory store one or more computer programs 146. The storage 145 may be a non-volatile memory such as Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), flash memory, a hard disk, a removable disk, or a device well known in the art to which this disclosure pertains. It may be configured to include any known type of computer-readable recording medium.

Next, the computer program 146 may include one or more instructions that, when loaded into the memory 142, cause the processor 141 to perform operations/methods according to various embodiments of the present disclosure. there is. That is, the processor 141 may perform operations/methods according to various embodiments of the present disclosure by executing loaded instructions.

For example, the computer program 146 may include an operation of obtaining a first embedding expression and a second embedding expression for a target graph, an operation of changing the value of the second embedding expression by reflecting a specific value in the second embedding expression, and a second embedding expression. It may include one or more instructions for performing an operation of generating a unified embedding expression for the target graph by aggregating the first embedding expression and the changed second embedding expression. In this case, the embedding system 10 according to some embodiments of the present disclosure may be implemented through the computing device 140.

Meanwhile, in some embodiments, the computing device 140 shown in FIG. 14 may mean a virtual machine implemented based on cloud technology. For example, the computing device 140 may be a virtual machine running on one or more physical servers included in a server farm. In this case, at least some of the processor 141, memory 142, and storage 145 shown in FIG. 14 may be virtual hardware, and the communication interface 144 may also be a virtual switch, etc. It may be implemented as a virtualized networking element.

So far, an exemplary computing device 140 capable of implementing the embedding system 10 according to some embodiments of the present disclosure has been described with reference to FIG. 14 .

So far, various embodiments of the present disclosure and effects according to the embodiments have been mentioned with reference to FIGS. 1 to 14 . The effects according to the technical idea of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below.

In addition, although it has been described in the above embodiments that a plurality of components are combined or operated in combination, the technical idea of the present disclosure is not necessarily limited to these embodiments. That is, as long as it is within the scope of the technical idea of the present disclosure, all of the components may be operated by selectively combining one or more of them.

The technical ideas of the present disclosure described so far can be implemented as computer-readable code on a computer-readable medium. A computer program recorded on a computer-readable recording medium can be transmitted to another computing device through a network such as the Internet, installed on the other computing device, and thus used on the other computing device.

Although operations are shown in the drawings in a specific order, it should not be understood that the operations must be performed in the specific order shown or sequential order or that all illustrated operations must be performed to obtain the desired results. In certain situations, multitasking and parallel processing may be advantageous. Although various embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the technical idea of the present disclosure can be translated into another specific form without changing the technical idea or essential features. It is understandable that it can also be implemented. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. The scope of protection of this disclosure should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of the technical ideas defined by this disclosure.

Claims (17)

In a method performed by at least one computing device,
Obtaining a first embedding representation and a second embedding representation for the target graph;
changing the value of the second embedding expression by reflecting a specific value in the second embedding expression; and
Generating a unified embedding representation for the target graph by aggregating the first embedding representation and the changed second embedding representation.
Graph embedding method. According to paragraph 1,
One of the first embedding expression and the second embedding expression is generated by an embedding method that aggregates information of neighboring nodes constituting the target graph,
The other one of the first embedding expression and the second embedding expression is generated by an embedding method that reflects topology information of the target graph,
Graph embedding method. According to paragraph 1,
The first embedding expression and the second embedding expression are generated by embedding the target graph through different GNN (Graph Neural Network),
Graph embedding method. According to paragraph 1,
The specific value is an irrational number,
Graph embedding method. According to paragraph 1,
The specific value is a value based on a learnable parameter,
predicting a label for a predetermined task based on the generated integrated embedding representation; and
Further comprising updating the value of the learnable parameter based on the difference between the predicted label and the correct label for the target graph,
Graph embedding method. According to paragraph 1,
Reflecting the specific value in the second embedding representation is performed based on a multiplication operation,
Aggregating the first embedding representation and the modified second embedding representation is performed based on an addition operation,
Graph embedding method. According to paragraph 1,
Obtaining the first and second embedding representations includes:
Obtaining a first embedding matrix and a second embedding matrix for the target graph, wherein the size of the first embedding matrix is different from the second embedding matrix; and
Obtaining the first embedding representation and the second embedding representation by performing a resizing operation on at least one of the first embedding matrix and the second embedding matrix,
Graph embedding method. In clause 7,
The resizing operation is implemented through a multi-layer perceptron,
Graph embedding method. In clause 7,
The step of generating the integrated embedding representation is,
generating a first embedding vector by performing a pooling operation on the first embedding representation;
generating a second embedding vector by performing a pooling operation on the changed second embedding representation, wherein the second embedding vector has the same dimensionality as the first embedding vector; and
Aggregating the first and second embedding vectors to generate the unified embedding representation in vector form,
Graph embedding method. According to paragraph 1,
The step of obtaining the first and second embedding representations includes:
Obtaining the first embedding representation through a Graph Neural Network (GNN) using a neighboring node information aggregation method; and
Generating the second embedding representation by extracting topology information of the target graph using the first embedding representation,
Graph embedding method. According to clause 10,
The first embedding representation is a three-dimensional embedding matrix generated by aggregating feature matrices for node tuples,
The step of generating the second embedding representation is,
generating a two-dimensional embedding matrix by extracting diagonal elements of the three-dimensional embedding matrix; and
Comprising the step of extracting topology information of the target graph by analyzing the two-dimensional embedding matrix,
Graph embedding method. According to clause 10,
Extracting the topology information of the target graph is performed by calculating a persistence diagram,
Graph embedding method. According to paragraph 1,
The step of generating the integrated embedding representation is,
performing a pooling operation on each of the first embedding representation and the changed embedding representation; and
Aggregating results of the pooling operation to generate the unified embedding representation,
Graph embedding method. According to paragraph 1,
The step of generating the integrated embedding representation is,
Obtaining a third embedding representation for the target graph;
changing the value of the third embedding expression by reflecting a value different from the specific value in the third embedding expression; and
Aggregating the first embedding representation, the modified second embedding representation, and the modified third embedding representation to generate the unified embedding representation,
Graph embedding method. According to paragraph 1,
The step of generating the integrated embedding representation is,
Obtaining third to Kth embedding representations for the target graph (where K is a natural number of 3 or more);
changing the value of each embedding expression by reflecting a value different from the specific value in the third to Kth embedding expressions; and
Generating the integrated embedding representation by aggregating the first embedding representation and the changed second to Kth embedding representations,
Graph embedding method. One or more processors; and
Includes memory for storing one or more instructions,
The one or more processors:
By executing one or more instructions stored above,
Obtaining a first embedding representation and a second embedding representation for the target graph,
An operation of changing the value of the second embedding expression by reflecting a specific value in the second embedding expression, and
Performing an operation of generating a unified embedding expression for the target graph by aggregating the first embedding expression and the changed second embedding expression.
Graph embedding system. Combined with a computing device,
Obtaining a first embedding representation and a second embedding representation for the target graph;
changing the value of the second embedding expression by reflecting a specific value in the second embedding expression; and
Stored in a computer-readable recording medium to execute the step of aggregating the first embedding representation and the changed second embedding representation to generate a unified embedding representation for the target graph,
computer program.

Images