CN117808084A - Pre-selection method based on graph reduction brief representation and graph neural network - Google Patents

Abstract

The invention relates to the technical field of artificial intelligence, and relates to a premise selection method based on graph reduction representation and graph neural network, which includes the following steps: Step 1: Obtain a simplified first-order logic formula graph by judging and deleting continuously repeated quantifiers; 2: Based on the simplified logical formula diagram, a term-walk graph neural network model with an attention mechanism is proposed. The model aggregates the node information located in the upper, middle and lower parts of the term-walk triplet according to the term-walk mode, and introduces attention. The force mechanism calculates the item wandering feature weight of the node, and combines the weight with the node information to generate a new node embedding vector, and then obtains the final formula graph feature vector through global average pooling; Step 3: Combine the candidate premises with the given conjecture The graph feature vector is input to the binary classifier to classify the candidate premises. The present invention can better perform premise selection.

Description

Premise selection method based on graph reduction representation and graph neural network

Technical field

The present invention relates to the field of artificial intelligence technology, and specifically to a premise selection method based on graph reduction representation and graph neural network.

Background technique

Automated Theorem Provers (ATPs) is a core and cutting-edge direction in the field of artificial intelligence. As an important part of artificial intelligence systems, it is widely used in expert systems, circuit design, compilers and software verification and other fields. ATPs first formalizes conjectures and premises into logical formulas, and then inputs the logical formulas into the automatic theorem prover, thereby automatically deducing conjectures from premises. ATPs achieves proof of new problems by continuously iteratively searching all pending clause sets in the problem library, which will lead to an exponential explosion of the search space in a larger problem library. Premise selection provides a new approach to solving this problem, that is, identifying and selecting formulas that help prove the conclusion of a given problem before entering logical formulas into ATPs.

Effective premise selection methods can greatly improve the capabilities of ATPs. The early premise selection method was mainly a hand-designed heuristic method based on symbolic comparative analysis. It extracted symbolic and structural features such as clause depth and symbol count from the input formula through comparative analysis to screen the premise set that is more relevant to the conclusion. Related to the premise, this method is limited to hand-designed features. With the development of computer capabilities, some machine learning methods have become an effective alternative method for premise selection. They naturally transform the problem into a classification or sorting problem, and can capture the deep characteristics of logical formulas better than the original method, such as Convolutional neural network, long short-term memory neural network, gated recurrent neural network, etc.

On this basis, because logical formulas can be naturally represented as graphs, and the characteristics of fused graph topology information can fully reflect the characteristics of logical formulas, the combination of graph neural networks and automatic theorem proving has become a popular research topic at present. . Although the existing premise selection method based on graph neural network can improve the ability of premise selection classification to a certain extent, it still has some shortcomings:

(1) Logic formula graphs contain rich grammatical and semantic properties. Most premise selection methods ignore the impact of different graph representations of logical formulas on the graph neural network model. This will result in the graph neural network not being able to capture the internal and external aspects of the logic formula well. information;

(2) Existing graph neural network models often generate features that retain more logical formula information by aggregating information from neighbor nodes or other nodes. These features often contain a large amount of node information, which may cause the generated formula features to be The influence of unimportant information on the graph makes it impossible to fully represent the graph features of logical formula information.

Contents of the invention

The content of the present invention is to provide a premise selection method based on graph reduction representation and graph neural network, which can assign different weights to nodes on the graph, thereby better encoding first-order logic formula graph features.

The premise selection method based on graph reduction representation and graph neural network according to the present invention includes the following steps:

Step 1: Get a simplified first-order logic formula diagram by judging and deleting consecutive repeated quantifiers;

Step 2: Based on the simplified logical formula diagram, a term-walk graph neural network model with an attention mechanism is proposed. The model aggregates the node information located in the upper, middle and lower parts of the term-walk triplet according to the term-walk mode, and introduces The attention mechanism calculates the item walking feature weight of the node, combines the weight with the node information to generate a new node embedding vector, and then obtains the final formula graph feature vector through global average pooling;

Step 3: Input the candidate premise and the graph feature vector of the given conjecture into the binary classifier to classify the candidate premise.

As a preferred method, in step one, quantifiers that meet the same and continuous conditions are combined on the basis of directed acyclic graphs DAGs to obtain a simplified first-order logic formula graph.

As a preferred option, in step two, an item-walking graph neural network model with an attention mechanism is specifically:

(1) Input graph G = (V, E), where V is all the nodes on the graph and E is all the edges on the graph; first assign an initial embedding x _v to each node v∈V, and then go through k rounds of iterations Generated node status vector Implement the message passing process, where k∈{1,…,K};

in, is a fixed-size initial state vector,/> is the output dimension of the node embedding vector, and F _V is a lookup table;

(2) Gather node information according to the term walking mode, input each node v of the graph G = (V, E), let T _u (v), T _m (v) and T _l (v) respectively represent that node v is located The item wandering feature sets in the upper, middle and lower parts are:

T _u (v)={(v,u,w)|(v,u), (u,w)∈E},

T _m (v)={(u, v, w)|(u, v), (v, w)∈E},

T _l (v)={(u,w,v)|(u,w),(w,v)∈E}

Among them, u and w are any nodes on the graph;

In order to distinguish the nodes v from different positions of the item wandering feature, the vectors in the triples T _u (v), T _m (v) and T _l (v) are spliced respectively, and then the model is based on _Tu (v), T _m (v) and T _l (v) and the aggregation functions F _u , F _m and F _l aggregate node v information from different locations;

Among them, the semicolon in the formula represents the splicing of different node state vectors; |T _u (v)|, |T _m (v)| and |T _m (v)| respectively represent the set T _u (v), T _m ( v) and the sum of the numbers of all triples in T _l (v);

The model introduces an attention mechanism to balance node status information Item travel feature information with nodes and/> To reduce the complexity of the model structure, use/> and/> As the attention score of item wandering feature information to node information, the attention score is normalized by the softmax function to obtain the attention weights α _vu , α _vm and α _vl , and finally obtained by using the aggregation functions F _U , F _M and F _L Balanced node information/> and/>

Among them, α _vu , α _vm and α _vl respectively represent the node item wandering characteristic information. and/> For node information,/> The attention weights, vu, vm and vl respectively represent the nodes located in the upper, middle and lower parts of the item wandering triplet;

Finally, node v aggregates the balanced node information from the sets T _u (v), T _m (v) and T _l (v) and/> is summarized as/>

(3) Using the adjusted total node information from the sets _Tu (v), _Tm (v) and _Tl (v) and the node v state vector of the previous step/> For node vector/> to deliver and update;

Among them, F _sum is the node information transfer function;

(4) Perform average pooling operation AvgPool on all nodes on the logical formula graph. The final embedding vector h _G of the formula graph is:

in, AvgPool is the average pool.

As a preferred method, in step three, input the graph embedding vector pair (h _p , h _c ) of the candidate premise and the given conjecture into the classification function F _class to obtain the usefulness score of the candidate premise under the conjecture;

z＝F _class ([h _p ；h _c ])

Among them, z∈R ² represents the score of the candidate premise being useful and useless for the guess;

The model uses the softmax function to normalize the usefulness and useless scores of the candidate premises for the guess, and divides the candidate premises according to the score size:

in, Represents the normalized usefulness and useless scores of the premise, z _i is the i-th element in z,/> for/> In the l-th element, the useful score and useless score of the candidate premise correspond to different label attributes. Based on the division results, the label attribute of the candidate premise is determined and compared with the existing labels to achieve classification.

The beneficial effects of the present invention are as follows:

1) The invention proposes a simplified first-order logical formula graphical representation method that deletes repeated quantifiers, which can prevent different graphical representations of logical formulas from affecting the graph neural network model, so that the graph neural network can well capture the internal and external information of the logical formula ;

2) The present invention proposes an item-walking graph neural network model with an attention mechanism, and applies it to the premise selection problem. This model can prevent the formula features in the graph neural network from being affected by unimportant information on the graph, assign different weights to the nodes on the graph, and thus better encode the first-order logic formula graph features.

Description of drawings

FIG1 is a flow chart of a premise selection method based on graph reduction representation and graph neural network in an embodiment.

Detailed ways

In order to further understand the content of the present invention, the present invention will be described in detail with reference to the accompanying drawings and embodiments. It should be understood that the embodiments are only for explanation of the present invention but not for limitation.

Example

As shown in Figure 1, this embodiment provides a premise selection method based on graph reduction representation and graph neural network, which includes the following steps:

Step 1: Obtain a simplified first-order logic formula diagram by judging and deleting continuously repeated quantifiers; the first-order logic formula diagram includes a first-order logic premise formula diagram and a first-order logic conjecture formula diagram;

Commonly used representations of logical formulas into graphs are mostly extended directed acyclic graphs (DAGs). The general steps are: 1) Convert logical formulas into a syntax parse tree similar to a programming language; 2) Merge the same parsing trees subexpressions and leaf nodes; 3) Rename the variables in the logical formula.

In order to reduce the size of graph data and allow DAGs to contain more logical properties, the present invention proposes directed acyclic graphs (Simplified-DAGs) based on deletion of repeated quantifiers to represent first-order logic formulas. The operation of Simplified-DAGs is equivalent to merging quantifiers that meet the same and continuous conditions on the basis of the original DAGs.

Step 2: Based on the simplified logical formula diagram, an item-walking graph neural network model with attention mechanism (Attention-TW-GNN) is proposed. The model is aggregated in the upper and middle parts of the item-walking triplet according to the item-walking mode. and the node information in the lower part, introduce the attention mechanism to calculate the item walking feature weight of the node, combine the weight with the node information to generate a new node embedding vector, and then obtain the final formula graph feature vector through global average pooling;

In step two, an item-walking graph neural network model with an attention mechanism follows the workflow of the graph neural network model and goes through the graph node vector initialization phase, the graph node information aggregation phase, the graph node information transfer phase and the graph feature readout The stage (graph pooling) iteratively updates node embedding information and obtains the final first-order logic formula graph vector. Specifically:

(1) Input graph G = (V, E), where V is all the nodes on the graph and E is all the edges on the graph; first assign an initial embedding x _v to each node v∈V, and then go through k rounds of iterations Generated node status vector Implement the message passing process, where k∈{1,…,K};

in, is a fixed-size initial state vector,/> is the output dimension of the node embedding vector, and F _V is a lookup table;

(2) Aggregate node information according to the term walking mode, input each node v of the graph G = (V, E), let T _u (v), T _m (v) and T _l (v) respectively represent that node v is located The item wandering feature sets in the upper, middle and lower parts are:

T _u (v)={(v,u,w)|(v,u), (u,w)∈E},

T _m (v) = {(u, v, w) | (u, v), (v, w) ∈ E},

T _l (v)={(u,w,v)|(u,w),(w,v)∈E}

Among them, u and w are any nodes on the graph;

In order to distinguish nodes v from different positions of the item walk feature, the vectors in the triplets of _Tu (v), _Tm (v) and _Tl (v) are concatenated respectively. Then the model aggregates the information of nodes v from different positions for _Tu (v), _Tm (v) and _Tl (v) and the aggregation functions _Fu , _Fm and _Fl .

Among them, the semicolon in the formula represents the splicing of different node state vectors; |T _u (v)|, |T _m (v)| and |T _m (v)| respectively represent the set T _u (v), T _m ( v) and the sum of the numbers of all triples in T _l (v);

The model introduces an attention mechanism to balance node status information Item travel feature information with nodes and/> To reduce the complexity of the model structure, use/> and/> As the attention score (contribution) of item wandering feature information to node information, the attention score is normalized through the softmax function to obtain the attention weights α _vu , α _vm and α _vl , and finally the aggregation functions F _U , F _M and F _L gets the balanced node information/> and/>

Among them, α _vu , α _vm and α _vl respectively represent the node item wandering characteristic information. and/> Node information The attention weights, vu, vm and vl respectively represent the nodes located in the upper, middle and lower parts of the item wandering triplet;

Finally, node v aggregates the balanced node information from the sets T _u (v), T _m (v) and T _l (v) and/> is summarized as/>

(3) Using the adjusted total node information from the sets _Tu (v), _Tm (v) and _Tl (v) and the node v state vector of the previous step/> For node vector/> to deliver and update;

Among them, F _sum is the node information transfer function; F _sum is a simple single-layer MLPs.

(4) Perform average pooling operation (AvgPool) on all nodes on the logical formula graph. The final formula graph embedding vector h _G is:

in, AvgPool is the average pool.

Step 3: Input the candidate premise and the graph feature vector of the given conjecture into the binary classifier to classify the candidate premise.

Input the graph embedding vector pair (h _p , h _c ) of the candidate premise and the given conjecture into the classification function F _class to obtain the usefulness score of the candidate premise under the conjecture;

z＝F _class ([h _p ; h _c ])

Among them, z∈R ² represents the score of the candidate premise being useful and useless for the guess;

The model uses the softmax function to normalize the usefulness and useless scores of the candidate premises for the guess, and divides the candidate premises according to the score size:

in, Represents the normalized usefulness and useless scores of the premise, z _i is the i-th element in z,/> for/> In the l-th element, the useful score and useless score of the candidate premise correspond to different label attributes (1 or 0). According to the division result (maximum score), the label attribute of the candidate premise is determined, and compared with the existing labels, and then Implement classification.

experiment

(1)Data set

In this embodiment, two data sets are established based on the MPTP2078 question bank, namely the original (MPTP) data set and the conjunction normal form (CNF) data set, to test the prediction and classification effect of the model. Among them, the MPTP data set is the logical formula in the MPTP2078 question bank, and the CNF data set is the conjunction normal form corresponding to the logical formula in the MPTP2078 question bank. The question bank contains 1,469 conjectures and 24,087 premises used to prove these conjectures.

This embodiment constructs MPTP data sets for training, verification and testing (40996, 13990 and 14068 samples), where each data set is shaped like a triplet (premise, conjecture, label), the premise is a given conjecture The candidate premise of , the label is 0 or 1 in binary classification (1 indicates that the premise is useful, 0 indicates that the premise is useless); at the same time, the distribution of the CNF data set constructed in this embodiment is the same as the MPTP data set.

(2)Model settings

This embodiment converts the logic formula into a simplified logic formula graph based on deleting repeated quantifiers according to the data set, and obtains the graph information (node id, node name, parent node id, child node id) of each simplified logic formula. Subsequently, the logic formula graph is passed into the graph neural network, and the formula graph feature vector is obtained. The specific model settings of the graph neural network are as follows:

In the graph neural network model of this embodiment, the initial heat vector of each node has d _v dimensions. F _V is an embedding network, which embeds an initial heat vector of d _v dimensions into dimensional node initial state vector. The configurations of F _u , F _m and F _l are the same. They have input dimensions/> and output dimensions/> Fully Connected Layer (FC). The configuration of F _sum , F _U , F _M and F _L is similar to F _u in that it only changes the input dimension/> F _class has two fully connected layers: the first is of dimension/> FC and batch normalization (BN); the second is FC of dimension 2 by including softmax. It is worth noting that d _v is 793, which represents 793 node tags in which we uniformly represent variables as "Var".

(3) Experimental settings

The model parameters of this embodiment are set as follows:

(a) Use the default settings of the adaptive moment estimation Adam optimizer to train the model;

(b) The batch size is set to 32;

(c) The regularization parameter is set to 0.0001;

(d) The initial learning rate is set to 0.01;

(e) The model automatically adjusts the learning rate using the ReduceLROnPlateau strategy in the Pytorch library.

(f) The model uses the cross-entropy loss function to train the model

(4)Experimental results and analysis

This embodiment evaluates the premise selection model based on Attention-TW-GNN on the MPTP data set and the CNF data set, and compares the model with some mainstream methods. It can be seen from this:

The premise selection methods based on the graph neural network model can achieve good classification accuracy, which is better than some mainstream graph neural network models. For example, the Accuracy indicators of GCN, GAT, SGC and other models under the MPTP dataset are 86.25%, 85.38%, and 85.67% respectively. This shows that the mainstream graph neural network can only simply capture the topological structure of the logic formula graph, but cannot capture the deep information of the logic formula.

Among other baseline methods, the graph neural networks PC-GCN and TW-GNN based on hand-designed features are significantly better than mainstream graph neural networks. For example, the F1 index of PC-GCN and TW-GNN models in the CNF dataset is 83.98% and 83.72% respectively. A reasonable explanation is that these graph neural network models aggregate information from more distant nodes in addition to neighboring node information.

The classification accuracy of the premise selection model based on Attention-TW-GNN in this embodiment can surpass other existing graph neural network-based premise selection models in most cases. For example: Under the MPTP data set, Attention-TW-GNN has improved by at least 2% compared to mainstream graph neural networks and 0.5% compared to other baseline methods; under the CNF data set, Attention-TW-GNN has improved compared to mainstream graph neural networks The neural network model improved by 3%. This shows that the graph neural network adjusted by adding the attention mechanism can better represent the syntax and semantic information of the first-order logic formula. At the same time, the graph representation of the first-order logic formula also affects the classification effect of the model to a certain extent.

The present invention and its embodiments have been schematically described above. This description is not limiting. What is shown in the drawings is only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person of ordinary skill in the art is inspired by the invention and without departing from the spirit of the invention, can devise structural methods and embodiments similar to the technical solution without inventiveness, they shall all fall within the protection scope of the invention. .

Claims (4)

1. The pre-selection method based on the graph reduction brief representation and the graph neural network is characterized by comprising the following steps of: the method comprises the following steps:

step one: obtaining a simplified first-order logic formula diagram by judging and deleting continuous repeated graduated words;

step two: based on a simplified logic formula diagram, a term walk diagram neural network model with an attention mechanism is provided, the model aggregates node information positioned at the upper part, the middle part and the lower part of a term walk triplet according to a term walk mode, the attention mechanism is introduced to calculate term walk characteristic weights of nodes, the weights are combined with the node information to generate a new node embedded vector, and the final formula diagram characteristic vector is obtained through global average pooling;

and thirdly, inputting the candidate preconditions and the given guess graph feature vectors into a binary classifier, and further classifying the candidate preconditions.

2. The graph reduction profile and graph neural network based precursor selection method of claim 1, wherein: in the first step, the adjectives satisfying the same and continuous condition are combined on the basis of the directed acyclic graph DAGs, so that a simplified first-order logic formula graph is obtained.

3. The graph reduction profile and graph neural network based precursor selection method of claim 2, wherein: in the second step, a term walk graph neural network model with an attention mechanism specifically comprises:

(1) Inputting a graph g= (V, E), wherein V is all nodes on the graph and E is all edges on the graph; first, each node V e V is assigned an initial embedding x _v Node state vector generated through k rounds of iterationImplementing a messaging process, where K e {1, …, K };

wherein,is a fixed-size initial state vector, +.>Embedding the output dimension of the vector for the node, F _V Is a lookup table;

(2) Aggregating node information according to the term walk pattern, setting T for each node V of the input graph g= (V, E) _u (v)，T _m (v) And T _l (v) Item walk feature sets representing that the node v is located at the upper, middle, and lower portions, respectively, have:

T _u (v)＝{(v，u，w)|(v，u)，(u，w)∈E}，

T _m (v)＝{(u，v，w)|(u，v)，(v，w)∈E}，

T _l (v)＝{(u，w，v)|(u，w)，(w，v)∈E}

wherein u and w are any nodes on the graph;

to distinguish nodes v from different positions of the item walk feature, T is respectively calculated _u (v)，T _m (v) And T _l (v) The vectors in the triples are stitched and then the model is directed to T _u (v)，T _m (v) And T _l (v) Aggregation function F _u ，F _m And F _l Aggregating node v information from different locations;

wherein, the semicolons in the formula represent the concatenation of different node state vectors; i T _u (v)|，|T _m (v) I and T _m (v) I respectively represent the set T _u (v)，T _m (v) And T _l (v) The sum of all triples in the database;

model-induced attention mechanism to balance node state informationItem wander characteristic information of node->And->To reduce the complexity of the model structure +.>And->Attention score of term walk characteristic information to node information, and attention score is normalized through softmax function to obtain attention weight alpha _vu ，α _vm And alpha _vl Finally use the aggregation function F _U ，F _M And F _L Obtain balanced node information->And->

Wherein alpha is _vu ，α _vm And alpha _vl Respectively representing node item wander characteristic informationAnd->Information about node->Is that the concentration weights of (v), vu, vm and vl represent nodes located at the upper, middle and lower parts of the item walk triplet, respectively;

finally, node v aggregates from set T _u (v)，T _m (v) And T _l (v) Balance node information of (a)And->Is summarized as->

(3) Using data from set T _u (v)，T _m (v) And T _l (v) Is adjusted node total informationAnd node v state vector of the last step +.>For node vector->Carrying out transfer and updating;

wherein F is _sum A node information transfer function;

(4) Performing average pooling operation AvgPool on all nodes on the logic formula diagram, and embedding a vector h into the final formula diagram _G The method comprises the following steps:

wherein,AvgPool is the average pool.

4. The graph reduction profile and graph neural network based precursor selection method of claim 3, wherein: in step three, the candidate preconditions and the map of the given guess are embedded into vector pairs (h _p ，h _c ) Input to classification function F _class Obtaining usefulness scores of candidate preconditions under guesses;

z＝F _class ([h _p ；h _c ])

wherein z is E R ² A score representing the candidate preconditions as useful and useless for guessing;

the model normalizes the candidate preconditions using a softmax function to guess the useful and useless scores and divides the candidate preconditions by score size:

wherein,indicating normalized precondition useful and useless scores, zi being the i-th element in z, ++>Is->If the candidate precondition useful score and the useless score correspond to different tag attributes, determining the tag attribute of the candidate precondition according to the dividing result, and comparing the tag attribute with the existing tag, thereby realizing classification.