CN113327652A - Crystal property prediction and classification method based on attention mechanism and crystal atlas neural network - Google Patents

Crystal property prediction and classification method based on attention mechanism and crystal atlas neural network Download PDF

Info

Publication number
CN113327652A
CN113327652A CN202110509660.2A CN202110509660A CN113327652A CN 113327652 A CN113327652 A CN 113327652A CN 202110509660 A CN202110509660 A CN 202110509660A CN 113327652 A CN113327652 A CN 113327652A
Authority
CN
China
Prior art keywords
crystal
neural network
vector
node
classification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110509660.2A
Other languages
Chinese (zh)
Other versions
CN113327652B (en
Inventor
王步维
范谦
邵宇
乐云亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yangzhou University
Original Assignee
Yangzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yangzhou University filed Critical Yangzhou University
Priority to CN202110509660.2A priority Critical patent/CN113327652B/en
Publication of CN113327652A publication Critical patent/CN113327652A/en
Application granted granted Critical
Publication of CN113327652B publication Critical patent/CN113327652B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
    • G16C20/00—Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
    • G16C20/70—Machine learning, data mining or chemometrics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
    • G16C60/00—Computational materials science, i.e. ICT specially adapted for investigating the physical or chemical properties of materials or phenomena associated with their design, synthesis, processing, characterisation or utilisation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Crystallography & Structural Chemistry (AREA)
  • Databases & Information Systems (AREA)
  • Chemical & Material Sciences (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a crystal property prediction and classification method based on an attention mechanism and a crystal atlas neural network, which comprises the following steps: acquiring a crystallography information file and DFT calculation data of the crystal, and dividing the crystallography information file and the DFT calculation data into a training set, a verification set and a test set; extracting crystal characteristics from the crystallography information file, inputting the crystal characteristics into a neural network, and acquiring neural network output; training and verifying the constructed neural network model by adopting a training set and a verification set respectively to obtain a prediction model and a classification model; the prediction of the crystal properties is done by a prediction model and the classification of the crystal properties is done by a classification model. The method can effectively improve the prediction and classification precision of the crystal properties, consumes less time, has engineering practical value, is beneficial to realizing accurate large-scale crystal research simulation, and provides method guarantee for the development and research of new crystal materials.

Description

Crystal property prediction and classification method based on attention mechanism and crystal atlas neural network
Technical Field
The invention relates to a crystal property prediction and classification technology, in particular to a crystal property prediction and classification method based on an attention mechanism and a crystal volume neural network.
Background
The simulation of the crystal properties is usually realized by a first-principle calculation method based on DFT (density functional theory), but the screening of crystal materials with ideal properties by using the first-principle is very time-consuming and is not low in calculation cost. Therefore, how to realize large-scale screening of crystalline materials becomes a difficult problem. With the development of computers, machine learning is becoming an important topic in the field of academic, and people try to adopt a machine learning method to perform large-scale crystal property simulation. With the continuous optimization of the machine learning algorithm, the simulation accuracy gradually approaches the result of the first principle calculation. The combination of machine learning and crystal simulation is helpful for realizing large-scale crystal research simulation, and accelerates the development and research of new crystal materials, thus receiving wide attention of people.
The difficulty in using machine learning methods for the simulation of crystal properties is: how to correctly encode chemical information (such as atomic information, crystal topology, etc.) in crystals of any size and to be compatible with machine learning models, and how to train models with sufficient accuracy from the limited available data.
The crystal map convolutional neural network is a machine learning algorithm for crystal property research, directly learns the crystal property from the connection of atoms in the crystal, and provides a universal and interpretable crystal chemical information coding mode. Various physical properties of crystals can be predicted based on a crystal map convolution neural network (abbreviated CGCNN) of a Graph Convolution (GCN). The crystal structure diagram is an omnidirectional multi-graph in which atoms are represented by nodes and the sides represent atomic bonds between the atoms. In CGCNN, node i uses a feature vector viIs to represent viThe characteristics of the encoding property of the atom i are contained in the method. Non-directional edge (i, j)kRepresents the k-th bond between atoms i and j, u (i, j)kThen the eigenvector representing the kth atomic bond between atoms i and j is represented. In order to solve the problem of the difference of interaction strength between neighbors, CGCNN designs a new convolution function,
Figure BDA0003059804380000011
wherein
Figure BDA0003059804380000012
Representing the connection of atoms and the eigenvectors of the atomic bonds.
Figure BDA0003059804380000013
b(t)The convolution weight matrix, the self weight matrix and the bias of the t-th layer respectively, and g (-) represents the softplus activation function between the layers.
However, the CGCNN method has limited prediction accuracy as a machine learning method that is fast and capable of large-scale screening of crystalline materials. This is because CGCNN reduces the complexity of the network in order to improve the efficiency of the machine learning algorithm, and although the running rate is increased, it may cause a reduction in prediction accuracy. And the number of the default operation cycles (epochs) of the CGCNN method is 30, which can reduce the time consumption of the model establishing process, can also influence the fitting of the network, and can also cause the accuracy of the prediction model to be reduced.
Disclosure of Invention
The purpose of the invention is as follows: in order to overcome the defects in the prior art, a crystal property prediction and classification method based on an attention mechanism and a crystal graph volume neural network is provided, a new convolution function is designed for the crystal graph convolution neural network, the capability of fusing the topological structure and the node characteristics in graph convolution can be improved, the calculation accuracy is improved, a new normalization method is introduced to regularize a depth graph volume network, the fitting of the network is improved, a better model is established, and the improved new network has the characteristics of rapid and large-scale crystal material screening.
The technical scheme is as follows: in order to achieve the above object, the present invention provides a crystal property prediction and classification method based on attention mechanism and crystal volume neural network, comprising the following steps:
s1: acquiring a crystallography information file (crystal structure data) and DFT calculation data, and dividing the crystallography information file and the DFT calculation data into a training set, a verification set and a test set;
s2: extracting crystal characteristics from the crystallography information file, inputting the crystal characteristics into a neural network, and acquiring neural network output;
s3: training and verifying the constructed neural network model by adopting a training set and a verification set respectively to obtain a prediction model and a classification model, and completing prediction of crystal properties through the prediction model; classification of the crystal properties is accomplished by a classification model.
Further, the method for acquiring the structure data of the crystal and the DFT calculation data in step S1 includes:
a1: connecting a Materials Project database through a pymatgen program package in python software, and exporting the id number of the crystal and DFT calculation data forming physical properties such as energy, absolute energy, band gap, Fermi energy and the like to a csv file;
a2: connecting a Materials Project database through a pymatgen program package in python software, reading a crystal id number in the exported csv file, and exporting a corresponding cif file (crystallography information file);
a3: prepare one atom _ init.json file: and the JSON file is used for storing the initialization vector of each element.
Further, the obtaining process of the neural network output in step S2 is as follows:
b1: extracting cif the atom characteristics in the document, the bond characteristics between each atom and its neighbor atoms, the index of each atom's neighbor atoms and the index of crystal mapping to atoms, and using them as the input of the neural network;
b2: generating a new vector by inputting the atomic characteristics of the network through the embedding layer, and then inputting the new atomic characteristic vector, the key characteristic vector and the index vector of the neighbor atom into the convolution layer;
b3: in the convolutional layer, an atom is regarded as a node, an atomic bond is regarded as an edge, a node vector, a neighbor node vector and an edge vector are connected through an index vector to form a new embedded vector, and the new vector passes through a full connection layer 1 and then node normalization processing is carried out on output;
b4: node vector h after node normalization and softplus function activation(t)Is M hidden vectors Z merged into neighbor featuresT∈R1×FThe combined arrays are transformed by a non-linear transformation, and then a shared attention vector q ∈ R is usedF′×1Get the attention value ωT；
B5: attention to the value ω using the softmax function1,ω2,…,ωMStandardizing to obtain final weight;
b6: combining the M hidden vectors merged into the neighbor features and the attention values of the M hidden vectors to obtain a final node embedding H(t)Carrying out batch normalization, adding normalized original node feature vectors of the input convolutional layers, activating by a softplus function and outputting;
b7: after 3 layers of convolution layers, a new vector fused with a local chemical environment is generated, the new vector generates a vector representing the whole crystal through the pooling layer, is activated through a softplus function, is connected to the full-link layer 2, is activated through the same function, and is input into the full-link layer 3 for output.
Further, the formula of the node normalization processing in step B3 is as follows:
Figure BDA0003059804380000031
wherein h is(t)Embedding vectors, μ, for newly generated nodes(t)Is node h(t)Average value of (a) ("sigma(t)Is the deviation of the node;
the node vector h in the step B4(t)Is expressed as:
Figure BDA0003059804380000032
wherein T is h(t)The T line of (1), T belongs to M, M represents the maximum number of neighbor atoms, and F is the number of hidden features of atoms;
the value ω of interest in said step B4TThe expression of (a) is:
ωT＝qT·tanh(W·(ZT)T+b) (3)
wherein W ∈ RF′×FIs a weight matrix, b ∈ RF′×1Is a bias vector;
the final weight expression in step B5 is:
Figure BDA0003059804380000033
embedding H into the node in the step B6(t)The expression of (a) is:
H(t)＝a1Z1+a2Z2+…+aMZM。 (5)
further, the convolution formula of the neural network in step S2 is:
Figure BDA0003059804380000041
wherein, Nodnorm (cndot.) represents node normalization, g (cndot.) represents a softplus activation function, Attention (cndot.) represents an Attention mechanism, and BN (cndot.) represents batch normalization.
Further, the training method of the prediction model in step S3 includes:
using mean square loss and random gradient descent as a loss function and optimizer; the mean square loss is shown in equation (7),
loss(xi,yi)＝(xi-yi)2 (7)
in the formula, xiIs an input value, yiIs a target attribute value, namely a DFT calculated value; the prediction model uses the Mean Absolute Error (MAE) as an index for evaluating the performance of the model.
Further, the Mean Absolute Error (MAE) in the step S3: MAE represents the average value of absolute errors between the predicted value and the tested value, and is an evaluation index of the prediction model. As shown in the formula (8),
Figure BDA0003059804380000042
wherein x isiIndicates the predicted value, yiThe test values are indicated.
Further, the classification process of the classification model in the step S3 for the crystal property is as follows:
under the framework of the same neural network, the activation function of the output layer is changed into a logsoftmax activation function and matched with a negative log-likelihood loss function to realize the classification of crystal properties.
Further, the logsoftmax activation function in the step S3 is shown as formula (9), the negative log likelihood loss function is shown as formula (10),
Figure BDA0003059804380000043
Figure BDA0003059804380000044
the classification model takes accuracy (accuracuracy) and area under ROC curve (AUC) as indexes for evaluating model performance.
Further, the area under the ROC curve (AUC) is obtained by summing the areas of the portions under the ROC curve. The ROC curve abscissa is False Positive Rate (FPR), i.e., the probability that a positive case is determined but not a true case. The ordinate is the true rate (TPR), i.e., the probability that the positive case is also the true case. An AUC size of approximately 1 indicates a better classification model.
The attention mechanism is a special structure embedded in a machine learning model by people and used for automatically learning and calculating the contribution of input data to output data. The method can improve the capability of fusing the topological structure and the node characteristics in graph convolution by learning the importance weight of self-adaptive embedding.
The invention provides a crystal property prediction and classification method based on an attention mechanism and a crystal graph volume neural network. In addition, in order to reduce the risk of overfitting, node normalization is further introduced. The depth map convolution network is regularized by suppressing the characteristic correlation of hidden embedding and improving the smoothness of the model relative to the input node characteristics, and the overfitting risk of the network is reduced.
Has the advantages that: compared with the prior art, the method has the advantages that the crystal data collection, the crystal property prediction and the crystal property classification are taken as a complete system, the crystal graph convolution neural network and the attention mechanism are fully combined, the prediction and classification precision of the crystal property can be effectively improved, the consumed time is low, the engineering practical value is realized, the accurate large-scale crystal research simulation is facilitated, and the method guarantee is provided for the development and research of new crystal materials.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a diagram of the convolution structure of a convolutional layer in the present invention;
fig. 3 is a structural diagram of a neural network in the present invention.
Detailed Description
The present invention is further illustrated by the following figures and specific examples, which are to be understood as illustrative only and not as limiting the scope of the invention, which is to be given the full breadth of the appended claims and any and all equivalent modifications thereof which may occur to those skilled in the art upon reading the present specification.
The invention provides a crystal property prediction and classification method based on an attention mechanism and a crystal atlas neural network, which mainly comprises two stages of crystal property prediction and crystal property classification, wherein in the first stage, the mean square loss is used as a loss function, the random gradient descent is used as an optimizer, and the crystal Formation energy (Formation energy) and the Absolute energy (Absolute energy) are respectively used for the crystal) Band gap (Bandgap) and Fermi energy (Fermi energy) were predicted and compared to DFT calculation data. In the second stage, the activation function of the output layer is changed into logsoftmax activation function, the loss function is changed into negative log-likelihood loss function, and the threshold value of the total magnetic moment is 0.5 muBThe crystal of (2) is classified, and a wide bandgap semiconductor crystal having a bandgap threshold of 2.3eV is also classified.
As shown in fig. 1, the method for predicting and classifying crystal properties based on attention mechanism and crystal volume neural network provided by the present invention specifically includes the following steps:
s1: acquiring structure data and DFT calculation data of the crystal, and dividing the structure data and the DFT calculation data into a training set, a verification set and a test set;
the method for acquiring the structure data and DFT calculation data of the crystal comprises the following steps:
a1: connecting a Materials Project database through a pymatgen program package in python software, and exporting the id number of the crystal and DFT calculation data forming physical properties such as energy, absolute energy, band gap, Fermi energy and the like to a csv file;
a2: connecting a Materials Project database through a pymatgen program package in python software, reading a crystal id number in the exported csv file, and exporting a corresponding cif file (crystallography information file);
a3: prepare one atom _ init.json file: and the JSON file is used for storing the initialization vector of each element.
S2: collecting a crystallography information file, extracting crystal characteristics, inputting the crystal characteristics into a neural network, and acquiring neural network output;
the acquisition process of the neural network output is as follows:
b1: extracting cif the atom characteristics in the document, the bond characteristics between each atom and its neighbor atoms, the index of each atom's neighbor atoms and the index of crystal mapping to atoms, and using them as the input of the neural network;
b2: generating a new vector by inputting the atomic characteristics of the network through the embedding layer, and then inputting the new atomic characteristic vector, the key characteristic vector and the index vector of the neighbor atom into the convolution layer;
b3: in the convolutional layer, an atom is regarded as a node, an atomic bond is regarded as an edge, as shown in a convolutional structure diagram of fig. 2, a node vector, a neighbor node vector and an edge vector are connected through an index vector to form a new embedded vector, and the new vector passes through a full connection layer 1 and then node normalization processing is performed on output;
the formula of the node normalization processing is as follows:
Figure BDA0003059804380000061
wherein h is(t)Embedding vectors, μ, for newly generated nodes(t)Is node h(t)Average value of (a) ("sigma(t)Is the deviation of the node;
b4: node vector h after node normalization and softplus function activation(t)Is M hidden vectors Z merged into neighbor featuresT∈R1×FThe combined arrays are transformed by a non-linear transformation, and then a shared attention vector q ∈ R is usedF′×1Get the attention value ωT；
Node vector h(t)Is expressed as:
Figure BDA0003059804380000062
wherein T is h(t)The T line of (1), T belongs to M, M represents the maximum number of neighbor atoms, and F is the number of hidden features of atoms;
note the value ωTThe expression of (a) is:
ωT＝qT·tanh(W·(ZT)T+b) (3)
wherein W ∈ RF′×FIs a weight matrix, b ∈ RF′×1Is a bias vector;
b5: attention to the value ω using the softmax function1,ω2,…,ωMStandardizing to obtain the mostA final weight;
the expression for the weights is:
Figure BDA0003059804380000071
b6: combining the M hidden vectors merged into the neighbor features and the attention values of the M hidden vectors to obtain a final node embedding H(t)Carrying out batch normalization, adding normalized original node feature vectors of the input convolutional layers, activating by a softplus function and outputting;
node embedding H(t)The expression of (a) is:
H(t)＝a1Z1+a2Z2+…+aMZM (5)
b7: referring to fig. 3, after 3 convolutional layers, a new vector fused with the local chemical environment is generated, and then a vector representing the whole crystal is generated through the pooling layer, and is connected to the fully-connected layer 2 after being activated by the softplus function, and then is activated by the same function and then is input into the fully-connected layer 3 for output.
Based on the above process, the convolution formula of the neural network is:
Figure BDA0003059804380000072
wherein, Nodnorm (cndot.) represents node normalization, g (cndot.) represents a softplus activation function, Attention (cndot.) represents an Attention mechanism, and BN (cndot.) represents batch normalization.
S3: training and verifying the constructed neural network model by adopting a training set and a verification set respectively to obtain a prediction model and a classification model, and completing prediction of crystal properties through the prediction model according to the neural network output;
here the prediction model uses mean-square loss and random gradient descent as loss functions and optimizers; the mean square loss is shown in equation (7),
loss(xi,yi)＝(xi-yi)2 (7)
in the formula, xiIs an input value, yiIs a target attribute value, namely a DFT calculated value; the prediction model takes the average absolute error (MAE) as an index for evaluating the performance of the model;
mean Absolute Error (MAE): MAE represents the average value of absolute errors between the predicted value and the tested value, and is an evaluation index of the prediction model. As shown in the formula (8),
Figure BDA0003059804380000073
wherein x isiIndicates the predicted value, yiThe test values are indicated.
S4: classification of the crystal properties is accomplished by a classification model.
The classification process of the classification model for the crystal properties here is:
under the framework of the same neural network, the activation function of the output layer is changed into a logsoftmax activation function and matched with a negative log-likelihood loss function to realize the classification of crystal properties.
The logsoftmax activation function is shown in equation (9), the negative log-likelihood loss function is shown in equation (10),
Figure BDA0003059804380000081
Figure BDA0003059804380000082
the classification model takes accuracy (accuracuracy) and area under ROC curve (AUC) as indexes for evaluating model performance.
The area under the ROC curve (AUC) is the sum of the areas of the sections under the ROC curve. The ROC curve abscissa is False Positive Rate (FPR), i.e., the probability that a positive case is determined but not a true case. The ordinate is the true rate (TPR), i.e., the probability that the positive case is also the true case. An AUC size of approximately 1 indicates a better classification model.
In the step, positive and negative samples are prepared as a classification basis, and the method specifically comprises the following steps: the total magnetic moment is greater than 0.5 muBIs set to 1, and the total magnetic moment is less than 0.5 muBThe crystal (b) is set to 0, the value of the test results is between 0 and 1, and crystals with a value greater than 0.5 are considered to have a total magnetic moment greater than 0.5 muBCrystals with a value less than 0.5 are considered to have a total moment less than 0.5 muB. Similarly, a crystal having a band gap greater than 2.3eV is set to 1, a crystal having a band gap less than 2.3eV is set to 0, and the results of the test are found to be a value between 0 and 1, a crystal having a value greater than 0.5 is considered to have a band gap greater than 2.3eV, and a crystal having a value less than 0.5 is considered to have a band gap less than 2.3 eV.
In the embodiment, the above scheme is applied to an example experiment, and data of 3 tens of thousands of crystals are collected in the experiment, wherein 80% of the data is training data, 10% is verification data, and 10% is detection data. In the prediction experiment, the errors of the predicted values of the two physical quantities of the absolute energy and the formation energy and the calculated DFT value are minimum, and the MAE is 0.103eV/atom and 0.060 eV/atom; the band gap and fermi energy predictions are most erroneous from the DFT calculations, with MAEs of 0.312eV and 0.343 eV. Greater than 0.5 mu for total magnetic momentBIn the crystal classification experiment, the accuracy of the model classification reaches 87.9 percent, and the AUC is 0.919. In a wide-bandgap semiconductor crystal classification experiment with a band gap larger than 0.23eV, the model classification accuracy is more 93.9%, and the AUC is 0.981. The running environment is Win10, the CPU is i7-10700k, and the GPU is RTX 3080.
Secondly, in order to better embody the effect of the method of the present invention, the present example performs a comparative experiment on the method of the present invention and the CGCNN method, and under the same data and over-parameter conditions, the most obvious improvement of the method of the present invention is the band gap, and the MAE thereof is reduced by 8.8%. In addition, the MAE formation energy and absolute energy were also reduced by 4.8% and 3.7%, respectively. Although the prediction error of fermi energy is the largest, the MAE still decreases by 1.4% by comparison. The research results further show that compared with the CGCNN method, the method of the invention introduced with an attention mechanism and node normalization has obvious improvement on the aspect of prediction precision.
When classifying the total magnetic moment, CGCNN uses 80% of the collected data as training data, which can be 86.9% accurate. However, the method of the present invention requires only 60% of the data to be used as training data to achieve the same accuracy. And, when 80% of the data is also used as the training set, the accuracy of the method of the present invention is improved by 1%. In classifying wide bandgap semiconductor crystals, CGCNN can achieve 92.1% accuracy using 80% of the data as training data, while the new method can achieve almost the same accuracy using only 40% of the data as training data. The accuracy of the method of the invention can be improved even by 1.8% when training is also performed using 80% of the data. Therefore, the accuracy level of the CGCNN can be achieved by using less training data, and the accuracy can be improved by using the training set with the same data quantity.

Claims (10)

1. A crystal property prediction and classification method based on an attention mechanism and a crystal volume neural network is characterized by comprising the following steps:
s1: acquiring a crystallography information file and DFT calculation data of the crystal, and dividing the crystallography information file and the DFT calculation data into a training set, a verification set and a test set;
s2: extracting crystal characteristics from the crystallography information file, inputting the crystal characteristics into a neural network, and acquiring neural network output;
s3: training and verifying the constructed neural network model by adopting a training set and a verification set respectively to obtain a prediction model and a classification model, and completing prediction of crystal properties through the prediction model; classification of the crystal properties is accomplished by a classification model.
2. The method for predicting and classifying the crystal property based on the attention mechanism and the crystal volume neural network as claimed in claim 1, wherein the method for acquiring the structure data and the DFT calculation data of the crystal in the step S1 is as follows:
a1: connecting a Materials Project database through a pymatgen program package in python software, and exporting the id number of the crystal and DFT calculation data of physical properties to a csv file;
a2: connecting a Materials Project database through a pymatgen program package in python software, reading a crystal id number in the exported csv file, and exporting a corresponding crystallography information file (cif file);
a3: an initialization vector for each element is stored.
3. The method for predicting and classifying crystal properties based on attention mechanism and crystal volume neural network as claimed in claim 2, wherein the obtaining procedure of the neural network output in step S2 is:
b1: extracting cif the atom characteristics in the document, the bond characteristics between each atom and its neighbor atoms, the index of each atom's neighbor atoms and the index of crystal mapping to atoms, and using them as the input of the neural network;
b2: generating a new vector by inputting the atomic characteristics of the network through the embedding layer, and then inputting the new atomic characteristic vector, the key characteristic vector and the index vector of the neighbor atom into the convolution layer;
b3: in the convolutional layer, an atom is regarded as a node, an atomic bond is regarded as an edge, a node vector, a neighbor node vector and an edge vector are connected through an index vector to form a new embedded vector, and the new vector passes through a full connection layer 1 and then node normalization processing is carried out on output;
b4: node vector h after node normalization and softplus function activation(t)Is M hidden vectors Z merged into neighbor featuresT∈R1×FThe combined arrays are transformed by a non-linear transformation, and then a shared attention vector q ∈ R is usedF′×1Get the attention value ωT；
B5: attention to the value ω using the softmax function1,ω2,…,ωMStandardizing to obtain final weight;
b6: combining the M hidden vectors merged into the neighbor features and their attention valuesTo obtain the final node embedding H(t)Carrying out batch normalization, adding normalized original node feature vectors of the input convolutional layers, activating by a softplus function and outputting;
b7: after 3 layers of convolution layers, a new vector fused with a local chemical environment is generated, the new vector generates a vector representing the whole crystal through the pooling layer, is activated through a softplus function, is connected to the full-link layer 2, is activated through the same function, and is input into the full-link layer 3 for output.
4. The method for predicting and classifying crystal properties based on attention mechanism and crystal volume neural network as claimed in claim 3, wherein the formula of the node normalization process in step B3 is:
Figure FDA0003059804370000021
wherein h is(t)Embedding vectors, μ, for newly generated nodes(t)Is node h(t)Average value of (a) ("sigma(t)Is the deviation of the node;
the node vector h in the step B4(t)Is expressed as:
Figure FDA0003059804370000022
wherein T is h(t)The T line of (1), T belongs to M, M represents the maximum number of neighbor atoms, and F is the number of hidden features of atoms;
the value ω of interest in said step B4TThe expression of (a) is:
ωT＝qT·tanh(W·(ZT)T+b) (3)
wherein W ∈ RF′×FIs a weight matrix, b ∈ RF′×1Is a bias vector;
the final weight expression in step B5 is:
Figure FDA0003059804370000023
embedding H into the node in the step B6(t)The expression of (a) is:
H(t)＝a1Z1+a2Z2+…+aMZM。 (5)
5. the method for predicting and classifying crystal properties based on attention mechanism and crystal volume neural network as claimed in claim 4, wherein the convolution formula of the neural network in step S2 is:
Figure FDA0003059804370000024
wherein, Nodnorm (cndot.) represents node normalization, g (cndot.) represents a softplus activation function, Attention (cndot.) represents an Attention mechanism module, and BN (cndot.) represents batch normalization.
6. The method for predicting and classifying crystal properties based on attention mechanism and crystal volume neural network as claimed in claim 1, wherein the training method of the prediction model in step S3 is:
using mean square loss and random gradient descent as a loss function and optimizer; the mean square loss is shown in equation (7),
loss(xi,yi)＝(xi-yi)2 (7)
in the formula, xiIs an input value, yiIs a target attribute value, namely a DFT calculated value; the prediction model uses the Mean Absolute Error (MAE) as an index for evaluating the performance of the model.
7. The method for predicting and classifying crystal properties based on attention mechanism and crystal volume neural network as claimed in claim 6, wherein the Mean Absolute Error (MAE) in step S3 is: MAE represents the average of absolute errors between predicted values and test values, as shown in equation (8),
Figure FDA0003059804370000031
wherein x isiIndicates the predicted value, yiThe test values are indicated.
8. The method for predicting and classifying crystal properties based on attention mechanism and crystal volume neural network as claimed in claim 1, wherein the classification process of the classification model in step S3 for the crystal properties is:
under the framework of the same neural network, the activation function of the output layer is changed into a logsoftmax activation function and matched with a negative log-likelihood loss function to realize the classification of crystal properties.
9. The method for predicting and classifying crystal properties based on attention mechanism and crystal volume neural network as claimed in claim 8, wherein the logsoftmax activation function in step S3 is shown as formula (9), the negative log-likelihood loss function is shown as formula (10),
Figure FDA0003059804370000032
Figure FDA0003059804370000033
the classification model uses accuracy and area under the ROC curve (AUC) as indexes for evaluating the performance of the model.
10. The method of claim 9, wherein the area under the ROC curve (AUC) is a sum of areas of sections under the ROC curve.
CN202110509660.2A 2021-05-11 2021-05-11 Crystal property prediction method based on attention mechanism and graph convolution neural network Active CN113327652B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110509660.2A CN113327652B (en) 2021-05-11 2021-05-11 Crystal property prediction method based on attention mechanism and graph convolution neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110509660.2A CN113327652B (en) 2021-05-11 2021-05-11 Crystal property prediction method based on attention mechanism and graph convolution neural network

Publications (2)

Publication Number Publication Date
CN113327652A true CN113327652A (en) 2021-08-31
CN113327652B CN113327652B (en) 2023-07-25

Family

ID=77415260

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110509660.2A Active CN113327652B (en) 2021-05-11 2021-05-11 Crystal property prediction method based on attention mechanism and graph convolution neural network

Country Status (1)

Country Link
CN (1) CN113327652B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114334028A (en) * 2022-01-05 2022-04-12 哈尔滨工业大学（深圳） Method, system and storage medium for generating indefinite 3D structure based on graph neural network
CN114818948A (en) * 2022-05-05 2022-07-29 北京科技大学 Data-mechanism driven material attribute prediction method of graph neural network
CN115101149A (en) * 2022-06-30 2022-09-23 曲阜师范大学 Total energy prediction method for microstructure of material
CN115713986A (en) * 2022-11-11 2023-02-24 中南大学 Attention mechanism-based material crystal property prediction method
CN117476117A (en) * 2023-12-27 2024-01-30 宁德时代新能源科技股份有限公司 Method for predicting initial magnetic moment in crystal structure
CN117747012A (en) * 2024-02-04 2024-03-22 吉林大学 Crystal property prediction method and device, computer equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190272468A1 (en) * 2018-03-05 2019-09-05 The Board Of Trustees Of The Leland Stanford Junior University Systems and Methods for Spatial Graph Convolutions with Applications to Drug Discovery and Molecular Simulation
CN111028893A (en) * 2019-10-28 2020-04-17 山东天岳先进材料科技有限公司 Crystal growth prediction method and device
CN111882044A (en) * 2020-08-05 2020-11-03 四川大学 Eutectic prediction method and deep learning framework based on graph neural network
CN112489732A (en) * 2019-09-12 2021-03-12 罗伯特·博世有限公司 Graphic transducer neural network force field for predicting atomic force and energy in MD simulation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190272468A1 (en) * 2018-03-05 2019-09-05 The Board Of Trustees Of The Leland Stanford Junior University Systems and Methods for Spatial Graph Convolutions with Applications to Drug Discovery and Molecular Simulation
CN112489732A (en) * 2019-09-12 2021-03-12 罗伯特·博世有限公司 Graphic transducer neural network force field for predicting atomic force and energy in MD simulation
DE102020210965A1 (en) * 2019-09-12 2021-03-18 Robert Bosch Gesellschaft mit beschränkter Haftung NEURAL GRAPH TRANSFORMER NETWORK FORCE FIELD FOR THE PREDICTION OF ATOMIC FORCES AND ENERGIES IN MOLECULAR DYNAMICS SIMULATIONS
CN111028893A (en) * 2019-10-28 2020-04-17 山东天岳先进材料科技有限公司 Crystal growth prediction method and device
CN111882044A (en) * 2020-08-05 2020-11-03 四川大学 Eutectic prediction method and deep learning framework based on graph neural network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
STEPH-YVES LOUIS ET AL.: ""GLOBAL ATTENTION BASED GRAPH CONVOLUTIONAL NEURAL NETWORKS FOR IMPROVED MATERIALS PROPERTY PREDICTION"", 《ARXIV》 *
TIAN XIE ET AL.: ""Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties"", 《ARXIV》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114334028A (en) * 2022-01-05 2022-04-12 哈尔滨工业大学（深圳） Method, system and storage medium for generating indefinite 3D structure based on graph neural network
CN114818948A (en) * 2022-05-05 2022-07-29 北京科技大学 Data-mechanism driven material attribute prediction method of graph neural network
CN114818948B (en) * 2022-05-05 2023-02-03 北京科技大学 Data-mechanism driven material attribute prediction method of graph neural network
CN115101149A (en) * 2022-06-30 2022-09-23 曲阜师范大学 Total energy prediction method for microstructure of material
CN115101149B (en) * 2022-06-30 2024-05-24 曲阜师范大学 Method for predicting total energy of microstructure of material
CN115713986A (en) * 2022-11-11 2023-02-24 中南大学 Attention mechanism-based material crystal property prediction method
CN117476117A (en) * 2023-12-27 2024-01-30 宁德时代新能源科技股份有限公司 Method for predicting initial magnetic moment in crystal structure
CN117476117B (en) * 2023-12-27 2024-04-23 宁德时代新能源科技股份有限公司 Method for predicting initial magnetic moment in crystal structure
CN117747012A (en) * 2024-02-04 2024-03-22 吉林大学 Crystal property prediction method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN113327652B (en) 2023-07-25

Similar Documents

Publication Publication Date Title
CN113327652A (en) Crystal property prediction and classification method based on attention mechanism and crystal atlas neural network
CN109492822B (en) Air pollutant concentration time-space domain correlation prediction method
CN108805188B (en) Image classification method for generating countermeasure network based on feature recalibration
US11385292B2 (en) Battery materials screening
CN110363230B (en) Stacking integrated sewage treatment fault diagnosis method based on weighted base classifier
TR201514432T1 (en) Method for pseudo-recurrent processing of data using a feedforward neural network architecture
CN114019370B (en) Motor fault detection method based on gray level image and lightweight CNN-SVM model
CN110851654A (en) Industrial equipment fault detection and classification method based on tensor data dimension reduction
CN117524353B (en) Molecular large model based on multidimensional molecular information, construction method and application
Blot et al. Shade: Information-based regularization for deep learning
Zhou et al. Sequential data-driven cross-domain lithology identification under logging data distribution discrepancy
Ma et al. VNAS: Variational Neural Architecture Search
Zhang et al. PM-ARNN: 2D-TO-3D reconstruction paradigm for microstructure of porous media via adversarial recurrent neural network
Pamir et al. Electricity theft detection for energy optimization using deep learning models
CN109711485A (en) A kind of multi-tag classification method based on community's detection
CN109034497A (en) Prediction technique, system, medium and the equipment of polycrystalline reduction process energy consumption value
CN117236374A (en) Layering interpretation method based on fully developed material graph neural network
CN117172376A (en) Graph structure link prediction method and system of generated graph neural network based on network reconstruction
Alam et al. Deepqmlp: A scalable quantum-classical hybrid deep neural network architecture for classification
Ren et al. Classification method for imbalanced LiDAR point cloud based on stack autoencoder
Zhu et al. Structural landmarking and interaction modelling: a “slim” network for graph classification
Zhang et al. Research and application of electric power curve inversing dynamometer diagram technology using big data approach
Lee et al. Methodological framework for materials discovery using machine learning
CN113361476B (en) Zhang Heng one-number pre-earthquake abnormal signal identification method based on artificial intelligence technology
Sulaiman et al. Credit Card Fraud Detection Using Improved Deep Learning Models.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant