CN113327652A - Crystal property prediction and classification method based on attention mechanism and crystal atlas neural network - Google Patents

Crystal property prediction and classification method based on attention mechanism and crystal atlas neural network Download PDF

Info

Publication number
CN113327652A
CN113327652A CN202110509660.2A CN202110509660A CN113327652A CN 113327652 A CN113327652 A CN 113327652A CN 202110509660 A CN202110509660 A CN 202110509660A CN 113327652 A CN113327652 A CN 113327652A
Authority
CN
China
Prior art keywords
crystal
neural network
vector
node
classification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110509660.2A
Other languages
Chinese (zh)
Other versions
CN113327652B (en
Inventor
王步维
范谦
邵宇
乐云亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yangzhou University
Original Assignee
Yangzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yangzhou University filed Critical Yangzhou University
Priority to CN202110509660.2A priority Critical patent/CN113327652B/en
Publication of CN113327652A publication Critical patent/CN113327652A/en
Application granted granted Critical
Publication of CN113327652B publication Critical patent/CN113327652B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
    • G16C20/00—Chemoinformatics, i.e. ICT specially adapted for the handling of physicochemical or structural data of chemical particles, elements, compounds or mixtures
    • G16C20/70—Machine learning, data mining or chemometrics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16C—COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE
    • G16C60/00—Computational materials science, i.e. ICT specially adapted for investigating the physical or chemical properties of materials or phenomena associated with their design, synthesis, processing, characterisation or utilisation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Crystallography & Structural Chemistry (AREA)
  • Databases & Information Systems (AREA)
  • Chemical & Material Sciences (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本发明公开了一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，包括：获取晶体的晶体学信息文件和DFT计算数据，并将其分为训练集、验证集和测试集；从晶体学信息文件中，提取晶体特征，将晶体特征输入神经网络，获取到神经网络输出；采用训练集和验证集分别对构建好的神经网络模型进行训练和验证，获取到预测模型和分类模型；通过预测模型完成对于晶体性质的预测，通过分类模型完成对于晶体性质的分类。本发明能够有效提高对于晶体性质的预测和分类精度，并且耗时少，具有工程实用价值，有助于实现精确的大规模晶体研究模拟，为新晶体材料的开发和研究提供了方法保障。

Figure 202110509660

The invention discloses a method for predicting and classifying crystal properties based on an attention mechanism and a crystal graph volume neural network. Extract the crystal features from the crystallographic information file, input the crystal features into the neural network, and obtain the output of the neural network; use the training set and the verification set to train and verify the constructed neural network model respectively, and obtain the prediction model and Classification model; the prediction of crystal properties is completed through the prediction model, and the classification of crystal properties is completed through the classification model. The invention can effectively improve the prediction and classification accuracy of crystal properties, and is less time-consuming, has engineering practical value, is helpful for realizing accurate large-scale crystal research simulation, and provides a method guarantee for the development and research of new crystal materials.

Figure 202110509660

Description

Crystal property prediction and classification method based on attention mechanism and crystal atlas neural network
Technical Field
The invention relates to a crystal property prediction and classification technology, in particular to a crystal property prediction and classification method based on an attention mechanism and a crystal volume neural network.
Background
The simulation of the crystal properties is usually realized by a first-principle calculation method based on DFT (density functional theory), but the screening of crystal materials with ideal properties by using the first-principle is very time-consuming and is not low in calculation cost. Therefore, how to realize large-scale screening of crystalline materials becomes a difficult problem. With the development of computers, machine learning is becoming an important topic in the field of academic, and people try to adopt a machine learning method to perform large-scale crystal property simulation. With the continuous optimization of the machine learning algorithm, the simulation accuracy gradually approaches the result of the first principle calculation. The combination of machine learning and crystal simulation is helpful for realizing large-scale crystal research simulation, and accelerates the development and research of new crystal materials, thus receiving wide attention of people.
The difficulty in using machine learning methods for the simulation of crystal properties is: how to correctly encode chemical information (such as atomic information, crystal topology, etc.) in crystals of any size and to be compatible with machine learning models, and how to train models with sufficient accuracy from the limited available data.
The crystal map convolutional neural network is a machine learning algorithm for crystal property research, directly learns the crystal property from the connection of atoms in the crystal, and provides a universal and interpretable crystal chemical information coding mode. Various physical properties of crystals can be predicted based on a crystal map convolution neural network (abbreviated CGCNN) of a Graph Convolution (GCN). The crystal structure diagram is an omnidirectional multi-graph in which atoms are represented by nodes and the sides represent atomic bonds between the atoms. In CGCNN, node i uses a feature vector viIs to represent viThe characteristics of the encoding property of the atom i are contained in the method. Non-directional edge (i, j)kRepresents the k-th bond between atoms i and j, u (i, j)kThen the eigenvector representing the kth atomic bond between atoms i and j is represented. In order to solve the problem of the difference of interaction strength between neighbors, CGCNN designs a new convolution function,
Figure BDA0003059804380000011
wherein
Figure BDA0003059804380000012
Representing the connection of atoms and the eigenvectors of the atomic bonds.
Figure BDA0003059804380000013
b(t)The convolution weight matrix, the self weight matrix and the bias of the t-th layer respectively, and g (-) represents the softplus activation function between the layers.
However, the CGCNN method has limited prediction accuracy as a machine learning method that is fast and capable of large-scale screening of crystalline materials. This is because CGCNN reduces the complexity of the network in order to improve the efficiency of the machine learning algorithm, and although the running rate is increased, it may cause a reduction in prediction accuracy. And the number of the default operation cycles (epochs) of the CGCNN method is 30, which can reduce the time consumption of the model establishing process, can also influence the fitting of the network, and can also cause the accuracy of the prediction model to be reduced.
Disclosure of Invention
The purpose of the invention is as follows: in order to overcome the defects in the prior art, a crystal property prediction and classification method based on an attention mechanism and a crystal graph volume neural network is provided, a new convolution function is designed for the crystal graph convolution neural network, the capability of fusing the topological structure and the node characteristics in graph convolution can be improved, the calculation accuracy is improved, a new normalization method is introduced to regularize a depth graph volume network, the fitting of the network is improved, a better model is established, and the improved new network has the characteristics of rapid and large-scale crystal material screening.
The technical scheme is as follows: in order to achieve the above object, the present invention provides a crystal property prediction and classification method based on attention mechanism and crystal volume neural network, comprising the following steps:
s1: acquiring a crystallography information file (crystal structure data) and DFT calculation data, and dividing the crystallography information file and the DFT calculation data into a training set, a verification set and a test set;
s2: extracting crystal characteristics from the crystallography information file, inputting the crystal characteristics into a neural network, and acquiring neural network output;
s3: training and verifying the constructed neural network model by adopting a training set and a verification set respectively to obtain a prediction model and a classification model, and completing prediction of crystal properties through the prediction model; classification of the crystal properties is accomplished by a classification model.
Further, the method for acquiring the structure data of the crystal and the DFT calculation data in step S1 includes:
a1: connecting a Materials Project database through a pymatgen program package in python software, and exporting the id number of the crystal and DFT calculation data forming physical properties such as energy, absolute energy, band gap, Fermi energy and the like to a csv file;
a2: connecting a Materials Project database through a pymatgen program package in python software, reading a crystal id number in the exported csv file, and exporting a corresponding cif file (crystallography information file);
a3: prepare one atom _ init.json file: and the JSON file is used for storing the initialization vector of each element.
Further, the obtaining process of the neural network output in step S2 is as follows:
b1: extracting cif the atom characteristics in the document, the bond characteristics between each atom and its neighbor atoms, the index of each atom's neighbor atoms and the index of crystal mapping to atoms, and using them as the input of the neural network;
b2: generating a new vector by inputting the atomic characteristics of the network through the embedding layer, and then inputting the new atomic characteristic vector, the key characteristic vector and the index vector of the neighbor atom into the convolution layer;
b3: in the convolutional layer, an atom is regarded as a node, an atomic bond is regarded as an edge, a node vector, a neighbor node vector and an edge vector are connected through an index vector to form a new embedded vector, and the new vector passes through a full connection layer 1 and then node normalization processing is carried out on output;
b4: node vector h after node normalization and softplus function activation(t)Is M hidden vectors Z merged into neighbor featuresT∈R1×FThe combined arrays are transformed by a non-linear transformation, and then a shared attention vector q ∈ R is usedF′×1Get the attention value ωT；
B5: attention to the value ω using the softmax function1,ω2,…,ωMStandardizing to obtain final weight;
b6: combining the M hidden vectors merged into the neighbor features and the attention values of the M hidden vectors to obtain a final node embedding H(t)Carrying out batch normalization, adding normalized original node feature vectors of the input convolutional layers, activating by a softplus function and outputting;
b7: after 3 layers of convolution layers, a new vector fused with a local chemical environment is generated, the new vector generates a vector representing the whole crystal through the pooling layer, is activated through a softplus function, is connected to the full-link layer 2, is activated through the same function, and is input into the full-link layer 3 for output.
Further, the formula of the node normalization processing in step B3 is as follows:
Figure BDA0003059804380000031
wherein h is(t)Embedding vectors, μ, for newly generated nodes(t)Is node h(t)Average value of (a) ("sigma(t)Is the deviation of the node;
the node vector h in the step B4(t)Is expressed as:
Figure BDA0003059804380000032
wherein T is h(t)The T line of (1), T belongs to M, M represents the maximum number of neighbor atoms, and F is the number of hidden features of atoms;
the value ω of interest in said step B4TThe expression of (a) is:
ωT＝qT·tanh(W·(ZT)T+b) (3)
wherein W ∈ RF′×FIs a weight matrix, b ∈ RF′×1Is a bias vector;
the final weight expression in step B5 is:
Figure BDA0003059804380000033
embedding H into the node in the step B6(t)The expression of (a) is:
H(t)＝a1Z1+a2Z2+…+aMZM。 (5)
further, the convolution formula of the neural network in step S2 is:
Figure BDA0003059804380000041
wherein, Nodnorm (cndot.) represents node normalization, g (cndot.) represents a softplus activation function, Attention (cndot.) represents an Attention mechanism, and BN (cndot.) represents batch normalization.
Further, the training method of the prediction model in step S3 includes:
using mean square loss and random gradient descent as a loss function and optimizer; the mean square loss is shown in equation (7),
loss(xi,yi)＝(xi-yi)2 (7)
in the formula, xiIs an input value, yiIs a target attribute value, namely a DFT calculated value; the prediction model uses the Mean Absolute Error (MAE) as an index for evaluating the performance of the model.
Further, the Mean Absolute Error (MAE) in the step S3: MAE represents the average value of absolute errors between the predicted value and the tested value, and is an evaluation index of the prediction model. As shown in the formula (8),
Figure BDA0003059804380000042
wherein x isiIndicates the predicted value, yiThe test values are indicated.
Further, the classification process of the classification model in the step S3 for the crystal property is as follows:
under the framework of the same neural network, the activation function of the output layer is changed into a logsoftmax activation function and matched with a negative log-likelihood loss function to realize the classification of crystal properties.
Further, the logsoftmax activation function in the step S3 is shown as formula (9), the negative log likelihood loss function is shown as formula (10),
Figure BDA0003059804380000043
Figure BDA0003059804380000044
the classification model takes accuracy (accuracuracy) and area under ROC curve (AUC) as indexes for evaluating model performance.
Further, the area under the ROC curve (AUC) is obtained by summing the areas of the portions under the ROC curve. The ROC curve abscissa is False Positive Rate (FPR), i.e., the probability that a positive case is determined but not a true case. The ordinate is the true rate (TPR), i.e., the probability that the positive case is also the true case. An AUC size of approximately 1 indicates a better classification model.
The attention mechanism is a special structure embedded in a machine learning model by people and used for automatically learning and calculating the contribution of input data to output data. The method can improve the capability of fusing the topological structure and the node characteristics in graph convolution by learning the importance weight of self-adaptive embedding.
The invention provides a crystal property prediction and classification method based on an attention mechanism and a crystal graph volume neural network. In addition, in order to reduce the risk of overfitting, node normalization is further introduced. The depth map convolution network is regularized by suppressing the characteristic correlation of hidden embedding and improving the smoothness of the model relative to the input node characteristics, and the overfitting risk of the network is reduced.
Has the advantages that: compared with the prior art, the method has the advantages that the crystal data collection, the crystal property prediction and the crystal property classification are taken as a complete system, the crystal graph convolution neural network and the attention mechanism are fully combined, the prediction and classification precision of the crystal property can be effectively improved, the consumed time is low, the engineering practical value is realized, the accurate large-scale crystal research simulation is facilitated, and the method guarantee is provided for the development and research of new crystal materials.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a diagram of the convolution structure of a convolutional layer in the present invention;
fig. 3 is a structural diagram of a neural network in the present invention.
Detailed Description
The present invention is further illustrated by the following figures and specific examples, which are to be understood as illustrative only and not as limiting the scope of the invention, which is to be given the full breadth of the appended claims and any and all equivalent modifications thereof which may occur to those skilled in the art upon reading the present specification.
The invention provides a crystal property prediction and classification method based on an attention mechanism and a crystal atlas neural network, which mainly comprises two stages of crystal property prediction and crystal property classification, wherein in the first stage, the mean square loss is used as a loss function, the random gradient descent is used as an optimizer, and the crystal Formation energy (Formation energy) and the Absolute energy (Absolute energy) are respectively used for the crystal) Band gap (Bandgap) and Fermi energy (Fermi energy) were predicted and compared to DFT calculation data. In the second stage, the activation function of the output layer is changed into logsoftmax activation function, the loss function is changed into negative log-likelihood loss function, and the threshold value of the total magnetic moment is 0.5 muBThe crystal of (2) is classified, and a wide bandgap semiconductor crystal having a bandgap threshold of 2.3eV is also classified.
As shown in fig. 1, the method for predicting and classifying crystal properties based on attention mechanism and crystal volume neural network provided by the present invention specifically includes the following steps:
s1: acquiring structure data and DFT calculation data of the crystal, and dividing the structure data and the DFT calculation data into a training set, a verification set and a test set;
the method for acquiring the structure data and DFT calculation data of the crystal comprises the following steps:
a1: connecting a Materials Project database through a pymatgen program package in python software, and exporting the id number of the crystal and DFT calculation data forming physical properties such as energy, absolute energy, band gap, Fermi energy and the like to a csv file;
a2: connecting a Materials Project database through a pymatgen program package in python software, reading a crystal id number in the exported csv file, and exporting a corresponding cif file (crystallography information file);
a3: prepare one atom _ init.json file: and the JSON file is used for storing the initialization vector of each element.
S2: collecting a crystallography information file, extracting crystal characteristics, inputting the crystal characteristics into a neural network, and acquiring neural network output;
the acquisition process of the neural network output is as follows:
b1: extracting cif the atom characteristics in the document, the bond characteristics between each atom and its neighbor atoms, the index of each atom's neighbor atoms and the index of crystal mapping to atoms, and using them as the input of the neural network;
b2: generating a new vector by inputting the atomic characteristics of the network through the embedding layer, and then inputting the new atomic characteristic vector, the key characteristic vector and the index vector of the neighbor atom into the convolution layer;
b3: in the convolutional layer, an atom is regarded as a node, an atomic bond is regarded as an edge, as shown in a convolutional structure diagram of fig. 2, a node vector, a neighbor node vector and an edge vector are connected through an index vector to form a new embedded vector, and the new vector passes through a full connection layer 1 and then node normalization processing is performed on output;
the formula of the node normalization processing is as follows:
Figure BDA0003059804380000061
wherein h is(t)Embedding vectors, μ, for newly generated nodes(t)Is node h(t)Average value of (a) ("sigma(t)Is the deviation of the node;
b4: node vector h after node normalization and softplus function activation(t)Is M hidden vectors Z merged into neighbor featuresT∈R1×FThe combined arrays are transformed by a non-linear transformation, and then a shared attention vector q ∈ R is usedF′×1Get the attention value ωT；
Node vector h(t)Is expressed as:
Figure BDA0003059804380000062
wherein T is h(t)The T line of (1), T belongs to M, M represents the maximum number of neighbor atoms, and F is the number of hidden features of atoms;
note the value ωTThe expression of (a) is:
ωT＝qT·tanh(W·(ZT)T+b) (3)
wherein W ∈ RF′×FIs a weight matrix, b ∈ RF′×1Is a bias vector;
b5: attention to the value ω using the softmax function1,ω2,…,ωMStandardizing to obtain the mostA final weight;
the expression for the weights is:
Figure BDA0003059804380000071
b6: combining the M hidden vectors merged into the neighbor features and the attention values of the M hidden vectors to obtain a final node embedding H(t)Carrying out batch normalization, adding normalized original node feature vectors of the input convolutional layers, activating by a softplus function and outputting;
node embedding H(t)The expression of (a) is:
H(t)＝a1Z1+a2Z2+…+aMZM (5)
b7: referring to fig. 3, after 3 convolutional layers, a new vector fused with the local chemical environment is generated, and then a vector representing the whole crystal is generated through the pooling layer, and is connected to the fully-connected layer 2 after being activated by the softplus function, and then is activated by the same function and then is input into the fully-connected layer 3 for output.
Based on the above process, the convolution formula of the neural network is:
Figure BDA0003059804380000072
wherein, Nodnorm (cndot.) represents node normalization, g (cndot.) represents a softplus activation function, Attention (cndot.) represents an Attention mechanism, and BN (cndot.) represents batch normalization.
S3: training and verifying the constructed neural network model by adopting a training set and a verification set respectively to obtain a prediction model and a classification model, and completing prediction of crystal properties through the prediction model according to the neural network output;
here the prediction model uses mean-square loss and random gradient descent as loss functions and optimizers; the mean square loss is shown in equation (7),
loss(xi,yi)＝(xi-yi)2 (7)
in the formula, xiIs an input value, yiIs a target attribute value, namely a DFT calculated value; the prediction model takes the average absolute error (MAE) as an index for evaluating the performance of the model;
mean Absolute Error (MAE): MAE represents the average value of absolute errors between the predicted value and the tested value, and is an evaluation index of the prediction model. As shown in the formula (8),
Figure BDA0003059804380000073
wherein x isiIndicates the predicted value, yiThe test values are indicated.
S4: classification of the crystal properties is accomplished by a classification model.
The classification process of the classification model for the crystal properties here is:
under the framework of the same neural network, the activation function of the output layer is changed into a logsoftmax activation function and matched with a negative log-likelihood loss function to realize the classification of crystal properties.
The logsoftmax activation function is shown in equation (9), the negative log-likelihood loss function is shown in equation (10),
Figure BDA0003059804380000081
Figure BDA0003059804380000082
the classification model takes accuracy (accuracuracy) and area under ROC curve (AUC) as indexes for evaluating model performance.
The area under the ROC curve (AUC) is the sum of the areas of the sections under the ROC curve. The ROC curve abscissa is False Positive Rate (FPR), i.e., the probability that a positive case is determined but not a true case. The ordinate is the true rate (TPR), i.e., the probability that the positive case is also the true case. An AUC size of approximately 1 indicates a better classification model.
In the step, positive and negative samples are prepared as a classification basis, and the method specifically comprises the following steps: the total magnetic moment is greater than 0.5 muBIs set to 1, and the total magnetic moment is less than 0.5 muBThe crystal (b) is set to 0, the value of the test results is between 0 and 1, and crystals with a value greater than 0.5 are considered to have a total magnetic moment greater than 0.5 muBCrystals with a value less than 0.5 are considered to have a total moment less than 0.5 muB. Similarly, a crystal having a band gap greater than 2.3eV is set to 1, a crystal having a band gap less than 2.3eV is set to 0, and the results of the test are found to be a value between 0 and 1, a crystal having a value greater than 0.5 is considered to have a band gap greater than 2.3eV, and a crystal having a value less than 0.5 is considered to have a band gap less than 2.3 eV.
In the embodiment, the above scheme is applied to an example experiment, and data of 3 tens of thousands of crystals are collected in the experiment, wherein 80% of the data is training data, 10% is verification data, and 10% is detection data. In the prediction experiment, the errors of the predicted values of the two physical quantities of the absolute energy and the formation energy and the calculated DFT value are minimum, and the MAE is 0.103eV/atom and 0.060 eV/atom; the band gap and fermi energy predictions are most erroneous from the DFT calculations, with MAEs of 0.312eV and 0.343 eV. Greater than 0.5 mu for total magnetic momentBIn the crystal classification experiment, the accuracy of the model classification reaches 87.9 percent, and the AUC is 0.919. In a wide-bandgap semiconductor crystal classification experiment with a band gap larger than 0.23eV, the model classification accuracy is more 93.9%, and the AUC is 0.981. The running environment is Win10, the CPU is i7-10700k, and the GPU is RTX 3080.
Secondly, in order to better embody the effect of the method of the present invention, the present example performs a comparative experiment on the method of the present invention and the CGCNN method, and under the same data and over-parameter conditions, the most obvious improvement of the method of the present invention is the band gap, and the MAE thereof is reduced by 8.8%. In addition, the MAE formation energy and absolute energy were also reduced by 4.8% and 3.7%, respectively. Although the prediction error of fermi energy is the largest, the MAE still decreases by 1.4% by comparison. The research results further show that compared with the CGCNN method, the method of the invention introduced with an attention mechanism and node normalization has obvious improvement on the aspect of prediction precision.
When classifying the total magnetic moment, CGCNN uses 80% of the collected data as training data, which can be 86.9% accurate. However, the method of the present invention requires only 60% of the data to be used as training data to achieve the same accuracy. And, when 80% of the data is also used as the training set, the accuracy of the method of the present invention is improved by 1%. In classifying wide bandgap semiconductor crystals, CGCNN can achieve 92.1% accuracy using 80% of the data as training data, while the new method can achieve almost the same accuracy using only 40% of the data as training data. The accuracy of the method of the invention can be improved even by 1.8% when training is also performed using 80% of the data. Therefore, the accuracy level of the CGCNN can be achieved by using less training data, and the accuracy can be improved by using the training set with the same data quantity.

Claims (10)

1.一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，包括如下步骤：1. a crystal property prediction and classification method based on attention mechanism and crystal graph volume neural network, is characterized in that, comprises the steps: S1：获取晶体的晶体学信息文件和DFT计算数据，并将其分为训练集、验证集和测试集；S1: Obtain the crystallographic information file and DFT calculation data of the crystal, and divide it into a training set, a validation set and a test set; S2：从晶体学信息文件中，提取晶体特征，将晶体特征输入神经网络，获取到神经网络输出；S2: Extract the crystal features from the crystallographic information file, input the crystal features into the neural network, and obtain the output of the neural network; S3：采用训练集和验证集分别对构建好的神经网络模型进行训练和验证，获取到预测模型和分类模型，通过预测模型完成对于晶体性质的预测；通过分类模型完成对于晶体性质的分类。S3: Use the training set and the validation set to train and verify the constructed neural network model respectively, obtain the prediction model and the classification model, and complete the prediction of the crystal properties through the prediction model; complete the classification of the crystal properties through the classification model. 2.根据权利要求1所述的一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，所述步骤S1中晶体的结构数据和DFT计算数据的获取方法为：2. A method for predicting and classifying crystal properties based on an attention mechanism and a crystal graph volume neural network according to claim 1, wherein the method for obtaining the structural data of the crystal and the DFT calculation data in the step S1 is as follows: : A1：通过python软件中的pymatgen程序包连接Materials Project数据库，从中将晶体的id号码以及物理性质的DFT计算数据导出到.csv文件中；A1: Connect the Materials Project database through the pymatgen package in the python software, and export the id number of the crystal and the DFT calculation data of the physical properties to a .csv file; A2：通过python软件中的pymatgen程序包连接Materials Project数据库，并读取导出的.csv文件中的晶体id号码，将对应的晶体学信息文件(cif文件)导出；A2: Connect to the Materials Project database through the pymatgen package in the python software, read the crystal id number in the exported .csv file, and export the corresponding crystallographic information file (cif file); A3：存储每个元素的初始化向量。A3: Stores the initialization vector for each element. 3.根据权利要求2所述的一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，所述步骤S2中神经网络输出的获取过程为：3. a kind of crystal property prediction and classification method based on attention mechanism and crystal graph volume neural network according to claim 2, is characterized in that, the acquisition process of neural network output in described step S2 is: B1：提取cif文件中的原子特征，每个原子与其邻居原子之间的键特征，每个原子的邻居原子的索引以及晶体映射到原子的索引，将它们作为神经网络的输入；B1: Extract the atom features in the cif file, the bond features between each atom and its neighbor atoms, the index of each atom's neighbor atoms, and the index of the crystal mapped to the atom, and use them as the input of the neural network; B2：输入网络的原子特征经过嵌入层生成新的向量，然后将新的原子特征向量和键特征向量以及邻居原子的索引向量一起输入到卷积层中；B2: The atomic features of the input network are passed through the embedding layer to generate a new vector, and then the new atomic feature vector, the bond feature vector and the index vector of neighbor atoms are input into the convolutional layer together; B3：在卷积层中，将原子看成节点，原子键看作边，先通过索引向量连接节点向量、邻居节点向量和边向量组成新的嵌入向量，新向量经过一个全连接层1后先对输出进行节点归一化处理；B3: In the convolutional layer, the atoms are regarded as nodes, and the atomic bonds are regarded as edges. First, the node vector, the neighbor node vector and the edge vector are connected by the index vector to form a new embedding vector. The new vector passes through a fully connected layer 1. Perform node normalization processing on the output; B4：经过节点归一化和softplus函数激活后的节点向量h(t)是M个融入邻居特征的隐藏向量ZT∈R1×F组合成的数组，通过非线性变换对向量进行变换，然后用一个共享的注意向量q∈RF′×1得到注意值ωT；B4: The node vector h (t) after node normalization and softplus function activation is an array composed of M hidden vectors Z T ∈ R 1×F which are integrated into the neighbor features. The vectors are transformed by nonlinear transformation, and then Use a shared attention vector q∈R F′×1 to get the attention value ω T ; B5：用softmax函数将注意值ω1,ω2,…,ωM标准化，得到最终的权重；B5: Use the softmax function to standardize the attention values ω 1 , ω 2 ,...,ω M to obtain the final weight; B6：将这M个融入邻居特征的隐藏向量和它们的注意值组合起来得到最终的节点嵌入H(t)，进行批量归一化,归一化后与输入卷积层的原节点特征向量相加后经softplus函数激活并输出；B6: Combine the M hidden vectors integrated into the neighbor features and their attention values to obtain the final node embedding H (t) , perform batch normalization, and after normalization, it is consistent with the original node feature vector of the input convolution layer. After adding, it is activated and output by the softplus function; B7：经过3层卷积层后，生成融合了局部化学环境的新向量，新向量再通过池化层生成代表整个晶体的向量，通过softplus函数激活后连接到全连接层2接着再通过同样的函数激活然后输入全连接层3输出。B7: After 3 convolution layers, a new vector that integrates the local chemical environment is generated. The new vector is then passed through the pooling layer to generate a vector representing the entire crystal, activated by the softplus function, and then connected to the fully connected layer 2 and then through the same The function activation is then fed into the fully connected layer 3 output. 4.根据权利要求3所述的一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，所述步骤B3中节点归一化处理的公式为：4. A kind of crystal property prediction and classification method based on attention mechanism and crystal graph volume neural network according to claim 3, is characterized in that, the formula of node normalization processing in described step B3 is:
Figure FDA0003059804370000021
Figure FDA0003059804370000021
 其中，h(t)为新生成的节点嵌入向量，μ(t)是节点h(t)的平均值，σ(t)是节点的偏差；where h (t) is the newly generated node embedding vector, μ (t) is the average value of node h (t) , and σ (t) is the deviation of the node; 所述步骤B4中的节点向量h(t)的表达为：The expression of the node vector h (t) in the step B4 is:
Figure FDA0003059804370000022
Figure FDA0003059804370000022
 其中，T就是h(t)的第T行，T∈M，M代表邻居原子的最大数量，F是原子隐藏特征的数量；Among them, T is the T-th row of h (t) , T∈M, M represents the maximum number of neighbor atoms, and F is the number of hidden features of atoms; 所述步骤B4中注意值ωT的表达式为：The expression of the attention value ω T in the step B4 is: ωT＝qT·tanh(W·(ZT)T+b) (3)ω T =q T ·tanh(W·(Z T ) T +b) (3) 其中W∈RF′×F是权值矩阵，b∈RF′×1是偏置向量；where W∈R F′×F is the weight matrix, and b∈R F′×1 is the bias vector; 所述步骤B5中最终的权重的表达式为：The expression of the final weight in the step B5 is:
Figure FDA0003059804370000023
Figure FDA0003059804370000023
 所述步骤B6中节点嵌入H(t)的表达式为：The expression of node embedding H (t) in the step B6 is: H(t)＝a1Z1+a2Z2+…+aMZM。 (5)H (t) = a 1 Z 1 +a 2 Z 2 +...+a M Z M . (5) 5.根据权利要求4所述的一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，所述步骤S2中神经网络的卷积公式为：5. a kind of crystal property prediction and classification method based on attention mechanism and crystal graph volume neural network according to claim 4, is characterized in that, the convolution formula of neural network in described step S2 is:
Figure FDA0003059804370000024
Figure FDA0003059804370000024
 其中，Nodenorm(·)代表节点归一化，g(·)代表softplus激活函数，Attention(·)代表注意力机制模块，BN(·)代表批量归一化。Among them, Nodenorm( ) represents node normalization, g( ) represents softplus activation function, Attention( ) represents attention mechanism module, and BN( ) represents batch normalization. 6.根据权利要求1所述的一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，所述步骤S3中预测模型的训练方法为：6. a kind of crystal property prediction and classification method based on attention mechanism and crystal graph volume neural network according to claim 1, is characterized in that, the training method of prediction model in described step S3 is: 使用均方损失和随机梯度下降作为损失函数和优化器；均方损失如公式(7)所示，The mean squared loss and stochastic gradient descent are used as the loss function and optimizer; the mean squared loss is shown in Equation (7), loss(xi,yi)＝(xi-yi)2 (7)loss(x i ,y i )=(x i -y i ) 2 (7) 式中，xi是输入数值，yi是目标属性数值，即DFT计算数值；预测模型以平均绝对误差(MAE)作为评价模型性能的指标。In the formula, x i is the input value, y i is the target attribute value, that is, the DFT calculation value; the prediction model uses the mean absolute error (MAE) as an index to evaluate the performance of the model. 7.根据权利要求6所述的一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，所述步骤S3中平均绝对误差(MAE)：MAE表示预测值和测试值之间绝对误差的平均值，如公式(8)所示，7. A method for predicting and classifying crystal properties based on an attention mechanism and a crystal graph volume neural network according to claim 6, wherein in the step S3, mean absolute error (MAE): MAE represents the predicted value and The mean value of the absolute error between the test values, as shown in Equation (8),
Figure FDA0003059804370000031
Figure FDA0003059804370000031
 其中，xi表示预测值，yi表示测试值。Among them, x i represents the predicted value, and y i represents the test value. 8.根据权利要求1所述的一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，所述步骤S3中分类模型对于晶体性质的分类过程为：8. The method for predicting and classifying crystal properties based on an attention mechanism and a crystal graph volume neural network according to claim 1, wherein in the step S3, the classification process of the classification model for crystal properties is: 在相同的神经网络的框架下，将输出层的激活函数变为logsoftmax激活函数并配合负对数似然损失函数实现对晶体性质的分类。Under the framework of the same neural network, the activation function of the output layer is changed to a logsoftmax activation function and a negative log-likelihood loss function is used to achieve the classification of crystal properties. 9.根据权利要求8所述的一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，所述步骤S3中logsoftmax激活函数如公式(9)所示，负对数似然损失函数如公式(10)所示，9. A method for predicting and classifying crystal properties based on an attention mechanism and a crystal graph volume neural network according to claim 8, wherein the logsoftmax activation function in the step S3 is as shown in formula (9), and the negative The log-likelihood loss function is shown in formula (10),
Figure FDA0003059804370000032
Figure FDA0003059804370000032
Figure FDA0003059804370000033
Figure FDA0003059804370000033
 分类模型以准确度和ROC曲线下面积(AUC)作为评价模型性能的指标。The classification model uses accuracy and area under the ROC curve (AUC) as indicators to evaluate the performance of the model. 10.根据权利要求9所述的一种基于注意力机制和晶体图卷神经网络的晶体性质预测和分类方法，其特征在于，所述ROC曲线下面积(AUC)是对ROC曲线下各部分的面积求和而得。10. A method for predicting and classifying crystal properties based on an attention mechanism and a crystal graph volume neural network according to claim 9, wherein the area under the ROC curve (AUC) is the ratio of each part under the ROC curve. The area is summed up.
CN202110509660.2A 2021-05-11 2021-05-11 Crystal property prediction method based on attention mechanism and graph convolution neural network Active CN113327652B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110509660.2A CN113327652B (en) 2021-05-11 2021-05-11 Crystal property prediction method based on attention mechanism and graph convolution neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110509660.2A CN113327652B (en) 2021-05-11 2021-05-11 Crystal property prediction method based on attention mechanism and graph convolution neural network

Publications (2)

Publication Number Publication Date
CN113327652A true CN113327652A (en) 2021-08-31
CN113327652B CN113327652B (en) 2023-07-25

Family

ID=77415260

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110509660.2A Active CN113327652B (en) 2021-05-11 2021-05-11 Crystal property prediction method based on attention mechanism and graph convolution neural network

Country Status (1)

Country Link
CN (1) CN113327652B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114334028A (en) * 2022-01-05 2022-04-12 哈尔滨工业大学（深圳） Method, system and storage medium for generating indeterminate 3D structure based on graph neural network
CN114818948A (en) * 2022-05-05 2022-07-29 北京科技大学 Data-mechanism driven material attribute prediction method of graph neural network
CN115101149A (en) * 2022-06-30 2022-09-23 曲阜师范大学 Total energy prediction method for microstructure of material
CN115713986A (en) * 2022-11-11 2023-02-24 中南大学 Attention mechanism-based material crystal property prediction method
CN117476117A (en) * 2023-12-27 2024-01-30 宁德时代新能源科技股份有限公司 Method for predicting initial magnetic moment in crystal structure
CN117747012A (en) * 2024-02-04 2024-03-22 吉林大学 A method, device, computer equipment and storage medium for predicting crystal properties

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190272468A1 (en) * 2018-03-05 2019-09-05 The Board Of Trustees Of The Leland Stanford Junior University Systems and Methods for Spatial Graph Convolutions with Applications to Drug Discovery and Molecular Simulation
CN111028893A (en) * 2019-10-28 2020-04-17 山东天岳先进材料科技有限公司 Crystal growth prediction method and device
CN111882044A (en) * 2020-08-05 2020-11-03 四川大学 A eutectic prediction method and deep learning framework based on graph neural network
CN112489732A (en) * 2019-09-12 2021-03-12 罗伯特·博世有限公司 Graphic transducer neural network force field for predicting atomic force and energy in MD simulation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190272468A1 (en) * 2018-03-05 2019-09-05 The Board Of Trustees Of The Leland Stanford Junior University Systems and Methods for Spatial Graph Convolutions with Applications to Drug Discovery and Molecular Simulation
CN112489732A (en) * 2019-09-12 2021-03-12 罗伯特·博世有限公司 Graphic transducer neural network force field for predicting atomic force and energy in MD simulation
DE102020210965A1 (en) * 2019-09-12 2021-03-18 Robert Bosch Gesellschaft mit beschränkter Haftung NEURAL GRAPH TRANSFORMER NETWORK FORCE FIELD FOR THE PREDICTION OF ATOMIC FORCES AND ENERGIES IN MOLECULAR DYNAMICS SIMULATIONS
CN111028893A (en) * 2019-10-28 2020-04-17 山东天岳先进材料科技有限公司 Crystal growth prediction method and device
CN111882044A (en) * 2020-08-05 2020-11-03 四川大学 A eutectic prediction method and deep learning framework based on graph neural network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
STEPH-YVES LOUIS ET AL.: ""GLOBAL ATTENTION BASED GRAPH CONVOLUTIONAL NEURAL NETWORKS FOR IMPROVED MATERIALS PROPERTY PREDICTION"", 《ARXIV》 *
TIAN XIE ET AL.: ""Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties"", 《ARXIV》 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114334028A (en) * 2022-01-05 2022-04-12 哈尔滨工业大学（深圳） Method, system and storage medium for generating indeterminate 3D structure based on graph neural network
CN114334028B (en) * 2022-01-05 2024-12-13 哈尔滨工业大学（深圳） Method, system and storage medium for generating indefinite 3D structure based on graph neural network
CN114818948A (en) * 2022-05-05 2022-07-29 北京科技大学 Data-mechanism driven material attribute prediction method of graph neural network
CN114818948B (en) * 2022-05-05 2023-02-03 北京科技大学 Data-mechanism driven material attribute prediction method of graph neural network
CN115101149A (en) * 2022-06-30 2022-09-23 曲阜师范大学 Total energy prediction method for microstructure of material
CN115101149B (en) * 2022-06-30 2024-05-24 曲阜师范大学 Method for predicting total energy of microstructure of material
CN115713986A (en) * 2022-11-11 2023-02-24 中南大学 Attention mechanism-based material crystal property prediction method
CN117476117A (en) * 2023-12-27 2024-01-30 宁德时代新能源科技股份有限公司 Method for predicting initial magnetic moment in crystal structure
CN117476117B (en) * 2023-12-27 2024-04-23 宁德时代新能源科技股份有限公司 Method for predicting initial magnetic moment in crystal structure
CN117747012A (en) * 2024-02-04 2024-03-22 吉林大学 A method, device, computer equipment and storage medium for predicting crystal properties

Also Published As

Publication number Publication date
CN113327652B (en) 2023-07-25

Similar Documents

Publication Publication Date Title
CN113327652A (en) Crystal property prediction and classification method based on attention mechanism and crystal atlas neural network
CN108805188B (en) An Image Classification Method Based on Feature Recalibration Generative Adversarial Networks
CN109492822B (en) Temporal and spatial correlation prediction method of air pollutant concentration
CN110363230B (en) Fault diagnosis method of stacking integrated sewage treatment based on weighted base classifier
CN109165743A (en) A kind of semi-supervised network representation learning algorithm based on depth-compression self-encoding encoder
CN110895705B (en) Abnormal sample detection device, training device and training method thereof
CN114707011B (en) A feature fusion method for multi-source heterogeneous data based on tensor decomposition
CN103544392A (en) Deep learning based medical gas identifying method
CN105913450A (en) Tire rubber carbon black dispersity evaluation method and system based on neural network image processing
CN101699514B (en) SAR Image Segmentation Method Based on Immune Cloning Quantum Clustering
CN114120041A (en) A Few-Sample Classification Method Based on Dual Adversarial Variational Autoencoders
CN116842463A (en) A fault diagnosis method for electric vehicle charging pile equipment
CN110516733A (en) A logging lithology identification method based on improved multi-classification twin support vector machine
CN115510975A (en) Multivariate Time Series Anomaly Detection Method and System Based on Parallel Transofmer-GRU
CN115561005A (en) Fault Diagnosis Method of Chemical Process Based on EEMD Decomposition and Lightweight Neural Network
CN113837266B (en) Software defect prediction method based on feature extraction and Stacking ensemble learning
Kalpana et al. Multiple parametric fault diagnosis using computational intelligence techniques in linear filter circuit
CN119270085A (en) Method and system for monitoring safety state of lead-acid battery of backup power supply of transformer substation
CN111723010B (en) Software BUG classification method based on sparse cost matrix
CN113962265A (en) Underdetermined blind source separation method and equipment based on structured sparse subspace clustering
Alexandrou et al. Unsupervised identification of the phase transition on the 2D-Ising model
CN117972546A (en) A mixed gas identification and concentration detection method
CN117574274A (en) PSO-XGBoost system construction method adopting mixed feature screening and super-parameter optimization
CN116595468A (en) Latent space optimized autoencoder anomaly detection method and system
CN116304849A (en) A Two-Dimensional Piston Pump Fault Diagnosis Method Based on Local Tangent Space Arrangement and Gated Loop Network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant