CN111695260B - Material performance prediction method and system - Google Patents

Material performance prediction method and system Download PDF

Info

Publication number
CN111695260B
CN111695260B CN202010535731.1A CN202010535731A CN111695260B CN 111695260 B CN111695260 B CN 111695260B CN 202010535731 A CN202010535731 A CN 202010535731A CN 111695260 B CN111695260 B CN 111695260B
Authority
CN
China
Prior art keywords
network
predicted
attention network
matrix
loss function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010535731.1A
Other languages
Chinese (zh)
Other versions
CN111695260A (en
Inventor
钱权
曾毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Shanghai for Science and Technology
Original Assignee
University of Shanghai for Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Shanghai for Science and Technology filed Critical University of Shanghai for Science and Technology
Priority to CN202010535731.1A priority Critical patent/CN111695260B/en
Publication of CN111695260A publication Critical patent/CN111695260A/en
Application granted granted Critical
Publication of CN111695260B publication Critical patent/CN111695260B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a material performance prediction method and a material performance prediction system. The method provided by the invention utilizes an attention mechanism to realize the prediction of the material performance based on a deep learning mode, and overcomes the technical defect that the specific material performance can be predicted only by combining professional field knowledge and historical experience in the traditional material research method. In addition, the multi-head attention network is constructed, and the multi-head attention network is used for training each attention network of performance indexes needing to be predicted in a dynamic weight mode, so that the technical defect that in multi-task learning, loss function weights among tasks are easily and improperly distributed, and therefore training cannot be rapidly converged is overcome, and the training speed is increased.

Description

Material performance prediction method and system
Technical Field
The invention relates to the technical field of new material design, in particular to a material performance prediction method and system.
Background
In material science research, performance prediction (such as corrosion resistance, fatigue strength and the like) is often required to be carried out on a specific material so as to evaluate the practical application value of the material. Traditional material research methods require a combination of professional domain knowledge and historical experience to predict specific material properties. Moreover, many of the internal mechanisms of materials are not fully understood, subject to the development of current materials and related disciplines. The method of data science, especially the deep learning method, is a favorable complement for the development of new materials. How to predict the material performance based on a deep learning mode becomes a technical problem to be solved urgently.
Disclosure of Invention
The invention aims to provide a material performance prediction method and a material performance prediction system, which are used for predicting material performance based on a deep learning mode and overcoming the technical defect that the specific material performance can be predicted only by combining professional field knowledge and historical experience in the traditional material research method.
In order to achieve the purpose, the invention provides the following scheme:
a method of predicting material properties, the method comprising the steps of:
acquiring material data of a molded material sample to form a sample data set; the material data includes performance parameters and design parameters;
performing element embedding processing on the component parameters of the design parameters by using an element embedding model to obtain component characteristics; performing cross processing on non-component parameters except for the component parameters in the design parameters by using a cross processing network to obtain non-component characteristics and obtain a sample characteristic data set;
constructing an attention network for each performance index of the material to be predicted; designing attention networks of all performance indexes in the parameters to form a multi-head attention network; weighting and summing the loss function of the attention network of each performance index by using a weight matrix group to obtain the loss function of the multi-head attention network;
training and testing an embedding vector of the element embedding model, the weight and the bias of a cross processing network and each attention network in a multi-head attention network in a dynamic weight mode based on the sample characteristic data set until a loss function of the multi-head attention network converges and the predicted accuracy reaches an accuracy threshold, and obtaining the trained element embedding model, the trained cross processing network and the trained attention network of each performance index;
performing feature extraction on the component parameters of the design parameters of the material to be predicted by using the trained element embedded model, and performing feature extraction on the non-component parameters of the design parameters of the material to be predicted by using the trained cross processing network to obtain input feature data;
and respectively inputting the input characteristic data into the trained attention network corresponding to the performance index of the material to be predicted, and performing attention mechanism processing to obtain the prediction result of the performance index of each material to be predicted.
Optionally, the training and testing an embedded vector of an element embedded model, a weight and bias of a cross processing network, and each attention network in a multi-head attention network in a dynamic weight manner based on the sample feature data set until a loss function of the multi-head attention network converges and a prediction accuracy reaches an accuracy threshold, to obtain the trained element embedded model, the trained cross processing network, and the trained attention network for each performance index specifically includes:
inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention network, and using the formula loss-sigma2L (W) calculating the loss function value after each subtask is weighted; wherein, σ represents the weight of the subtask, L (W) represents the loss function of the subtask, and loss represents the loss function after weighting;
summing the weighted loss function values of all the subtasks to serve as the loss function value of the multi-head attention network;
judging whether the loss function value of the multi-head attention network is converged to obtain a second judgment result;
if the second judgment result shows that the first judgment result shows that the second judgment result does not show that the second judgment result shows that the second judgment result does not show the second judgment result, the embedding vector of the element embedding model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network are updated, the weight of each subtask in the weight matrix array is corrected by using a gradient updating method, the step of returning is that the sample feature data in the sample feature data set are input into each attention network in the multi-head attention network, and the formula loss sigma is used for inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention network2L (w) calculating a loss function value weighted by each subtask ";
and if the second judgment result shows that the first judgment result shows that the second judgment result shows that the first judgment result shows that the second embedding vector of the updated element embedding model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network are output.
Optionally, the feature extraction of the component parameters of the design parameters of the material to be predicted by using the trained element embedding model specifically includes:
multiplying the content of each component of the component parameters of the design parameters needing to be predicted by the embedding vector to obtain an embedding expression vector of each component;
and splicing the embedded expression vectors of each component to obtain the component characteristics of the component parameters of the design parameters of the predicted material.
Optionally, the performing feature extraction on the non-component parameters of the design parameters of the material to be predicted by using the trained cross processing network specifically includes:
using the formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing on the i-layer network; wherein x is0Representing the original input, xi-1And xiRespectively showing the cross processing results of the i-1 layer network and the i-layer network, wherein the cross processing result of the 0-layer network is a non-component parameter; wi-1Weight of the i-1 th network, bi-1Indicating the bias of the i-1 layer network;
judging whether the numerical value of i is smaller than the layer number of the cross-processing network or not to obtain a first judgment result;
if the first judgment result shows yes, increasing the value of i by 1, and returning to the step of utilizing the formula xi＝xi-1Wi- 1x0+bi-1+xi-1Performing cross processing of an i-layer network;
and if the first judgment result shows no, outputting the cross processing result of the ith layer.
Optionally, the inputting the feature data into the trained attention network corresponding to the performance index of the material to be predicted respectively, and performing attention mechanism processing to obtain the prediction result of the performance index of each material to be predicted specifically includes:
inputting the input characteristic data into a trained attention network of the performance index of the material to be predicted, and generating a Query matrix, a Key matrix and a Value matrix of the input characteristic data;
multiplying the Query matrix and the Key matrix to obtain a product matrix;
dividing the product matrix by the square root of the input feature data dimension to obtain a scaled product matrix;
performing Softmax operation on the scaled product matrix to obtain a Softmax operation result matrix;
and multiplying the Softmax operation result matrix by the Value matrix to obtain an attention mechanism processing result which is used as a prediction result of the performance index of the material needing to be predicted.
A material property prediction system, the prediction system comprising:
the sample data set acquisition module is used for acquiring material data of the molded material sample to form a sample data set; the material data includes performance parameters and design parameters;
the sample data set feature extraction module is used for carrying out element embedding processing on the component parameters of the design parameters by using an element embedding model to obtain component features; performing cross processing on non-component parameters except for the component parameters in the design parameters by using a cross processing network to obtain non-component characteristics and obtain a sample characteristic data set;
the multi-head attention network construction module is used for constructing an attention network of each performance index of the material to be predicted; designing attention networks of all performance indexes in the parameters to form a multi-head attention network; weighting and summing the loss function of the attention network of each performance index by using a weight matrix group to obtain the loss function of the multi-head attention network;
the dynamic weight training module is used for training and testing the embedded vector of the element embedded model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network in a dynamic weight mode based on the sample characteristic data set until the loss function of the multi-head attention network converges and the predicted accuracy reaches an accuracy threshold, so as to obtain the trained element embedded model, the trained cross processing network and the trained attention network of each performance index;
the feature extraction module is used for extracting features of the component parameters of the design parameters of the materials to be predicted by using the trained element embedded model, and extracting features of the non-component parameters of the design parameters of the materials to be predicted by using the trained cross processing network to obtain input feature data;
and the performance prediction module is used for respectively inputting the input characteristic data into the trained attention network corresponding to the performance index of the material to be predicted, and performing attention mechanism processing to obtain the prediction result of the performance index of each material to be predicted.
Optionally, the dynamic weight training module specifically includes: a loss function value weighting submodule for inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention network, and using the formula loss as sigma2L (W) calculating the loss function value after each subtask is weighted; wherein, σ represents the weight of the subtask, L (W) represents the loss function of the subtask, and loss represents the loss function after weighting;
the loss function value summation submodule is used for summing the weighted loss function values of all the subtasks to serve as the loss function value of the multi-head attention network;
the second judgment sub-module is used for judging whether the loss function value of the multi-head attention network is converged to obtain a second judgment result;
a parameter output submodule, configured to update the embedding vector of the element embedding model, the weight and bias of the cross processing network, and each attention network in the multi-head attention network if the second determination result indicates no, modify the weight of each subtask in the weight matrix array by using a gradient update method, return to the step "input the sample feature data in the sample feature data set into each attention network in the multi-head attention network, and use the formula loss ═ σ ═ σ -2L (w) calculating a loss function value weighted by each subtask ";
and the training result output submodule is used for outputting the embedding vector of the updated element embedding model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network if the second judgment result indicates yes, so as to obtain the trained element embedding model, the trained cross processing network and the trained attention network of each performance index.
Optionally, the feature extraction module for the design parameter of the material to be predicted specifically includes:
the embedded expression vector acquisition submodule is used for multiplying the content of each component of the component parameters of the design parameters needing to be predicted by the embedded vector to obtain the embedded expression vector of each component;
and the splicing submodule is used for splicing the embedded expression vectors of each component to obtain the component characteristics of the component parameters of the predicted design parameters of the material.
Optionally, the feature extraction module for the design parameter of the material to be predicted specifically includes:
a cross-processing submodule for utilizing the formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing on the i-layer network; wherein x is0Representing the original input, xi-1And xiRespectively showing the cross processing results of the i-1 layer network and the i-layer network, wherein the cross processing result of the 0-layer network is a non-component parameter; wi-1Weight of the i-1 th layer network, bi-1Indicating the bias of the i-1 layer network;
the layer number judgment submodule is used for judging whether the numerical value of the i is smaller than the layer number of the network subjected to cross processing or not to obtain a first judgment result;
a returning submodule, configured to increase the value of i by 1 if the first determination result indicates yes, and return to the step "use formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing of an i-layer network;
and the result output submodule is used for outputting the cross processing result of the ith layer if the first judgment result shows that the first judgment result is not the first judgment result.
Optionally, the performance prediction module specifically includes:
the matrix generation submodule is used for inputting the input characteristic data into the trained attention network of the performance index of the material to be predicted and generating a Query matrix, a Key matrix and a Value matrix of the input characteristic data;
the product matrix acquisition submodule is used for multiplying the Query matrix and the Key matrix to obtain a product matrix;
a scaling submodule, configured to divide the product matrix by a square root of the input feature data dimension to obtain a scaled product matrix;
the Softmax operation submodule is used for carrying out Softmax operation on the zoomed product matrix to obtain a Softmax operation result matrix;
and the prediction result output submodule is used for multiplying the Softmax operation result matrix and the Value matrix to obtain an attention mechanism processing result which is used as a prediction result of the performance index of the material needing to be predicted.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention discloses a material performance prediction method, which comprises the following steps:
acquiring material data of a molded material sample to form a sample data set; performing element embedding processing on the component parameters of the design parameters by using an element embedding model to obtain component characteristics; performing cross processing on non-component parameters except for the component parameters in the design parameters by using a cross processing network to obtain non-component characteristics and obtain a sample characteristic data set; constructing an attention network for each performance index of the material to be predicted; designing attention networks of all performance indexes in the parameters to form a multi-head attention network; weighting and summing the loss function of the attention network of each performance index by using a weight matrix group to obtain the loss function of the multi-head attention network; training and testing an embedded vector of the element embedded model, the weight and the offset of a cross processing network and each attention network in a multi-head attention network in a dynamic weight mode based on the sample characteristic data set until a loss function of the multi-head attention network converges and the prediction accuracy reaches an accuracy threshold, and obtaining the trained element embedded model, the trained cross processing network and the trained attention network of each performance index; performing feature extraction on the component parameters of the design parameters of the material to be predicted by using the trained element embedded model, and performing feature extraction on the non-component parameters of the design parameters of the material to be predicted by using the trained cross processing network to obtain input feature data; and respectively inputting the input characteristic data into the trained attention network corresponding to the performance index of the material to be predicted, and performing attention mechanism processing to obtain the prediction result of the performance index of each material to be predicted. The invention utilizes the attention network to realize the prediction of the material performance based on the deep learning mode, and overcomes the technical defect that the traditional material research method can predict the specific material performance only by combining professional field knowledge and historical experience. In addition, the multi-head attention network is constructed, and the multi-head attention network is used for training each attention network of performance indexes needing to be predicted in a dynamic weight mode, so that the technical defect that in multi-task learning, loss function weights among tasks are easily and improperly distributed, and therefore training cannot be rapidly converged is overcome, and the training speed is increased.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flow chart of a method for predicting material properties according to the present invention;
FIG. 2 is a schematic diagram of the loss function weighting process provided by the present invention;
FIG. 3 is a schematic diagram of an attention mechanism process provided by the present invention;
FIG. 4 is a schematic diagram illustrating a principle of a neural network model-based material performance prediction method according to the present invention.
Detailed Description
The invention aims to provide a material performance prediction method and a material performance prediction system, which are used for predicting material performance based on a deep learning mode and overcoming the technical defect that the specific material performance can be predicted only by combining professional field knowledge and historical experience in the traditional material research method.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Deep learning is good at capturing the relation directly from the data, and a specific network structure and a training method can implicitly capture the mapping relation between given input and output and store the information in a model through hyper-parameters. Through deep learning, material data can be modeled, the limitation that trial and error or knowledge in complex fields is relied on in traditional material research and development is made up, and the model is automatically learned and predicted.
Embedded representation is a concept in natural language processing that expresses words as vectors of a certain length so that they can be put into deep learning networks for training. The cross transformation refers to multiplying the input features item by item and outputting. The attention mechanism is a method of calculating the correlation between samples, and the higher the correlation, the greater their attention value. The attention mechanism can be used for well grasping the relation in the training data, so that the final output of similar samples is similar, and the fitting capability of the model is enhanced. Multitask learning is an extension of traditional single task learning, emphasizing that two similar tasks train together, promoting each other to achieve better results than training alone.
The invention aims to solve the problem of attribute prediction of a given new material in a material research and development process, and provides a material performance prediction method combining an attention mechanism and multi-task deep learning. The method is based on deep learning, and the material characteristic data is processed by using an element embedding representation and characteristic cross transformation method, so that the expression capability of the original data is greatly improved by the two processing modes. The processed data is then transformed using a multi-head self-attention mechanism. On the basis of the single task model, in order to better predict, the invention selects a second similar task, the two tasks have the same input and different outputs, and are jointly trained by using multi-task learning, and the two tasks are mutually promoted, thereby achieving better effect. In order to solve the problem of how to integrate the loss functions among different tasks in the multi-task learning, the invention uses the dynamic weight based on uncertainty to set the weight of the loss function among the tasks in the multi-task learning, so that the model can be better converged.
The element-embedded representation is a data processing method, and is generally used in a deep learning model. The basic idea is to represent an element as a vector with length n, and the value of n has different selection modes according to different models. This vector is initialized randomly at model creation, then changes as the model is trained, and finally, when the model converges, the vector represents the best vector representation of the element under the current input and output conditions. The method of using the vector to replace the original single value to represent the elements enables each element to have a larger feature representation space, enhances the feature representation capability of the elements, can fit different representation contents along with different tasks, and greatly enhances the fitting capability of the model.
Feature cross-transformation is also a data processing method. In the present invention, the processing logic of each layer of cross transformation is to multiply the input of the layer by a parameter matrix, which also changes with the training of the network, and then multiply the result by the most original input, and finally obtain the output. The benefit of this design is that each level of cross-transformation represents the multiplication of features once and the transformation of first order features to second order features. If high-order characteristics are required to be obtained, only the cross conversion layer needs to be continuously superposed, and the operation is very concise and clear.
The attention mechanism is a method of calculating the correlation between samples, and for each sample, the attention mechanism provides three vectors: the method comprises the following steps that a Query vector, a Key vector and a Value vector are initialized randomly when network training starts, and values of the three vectors are corrected along with the training process. When calculating the correlation between a certain sample and other samples, firstly multiplying the Query vector corresponding to the sample with the Key vectors of other samples, after normalizing the obtained result, multiplying the result with the Value vectors of all samples, and finally adding the calculation results of all samples to obtain the attention Value output of the sample. The attention mechanism has the outstanding advantage that the output of the sample can be calculated according to the context information, the calculation mode is similar to the attention distribution when a human being views the picture or the text information, the focus can be focused on a place needing attention, and the efficiency of the model is improved.
Multi-task learning is a method to improve the learning effect of related tasks. Because model learning lacks a reference in the single-task case, the present invention uses a multi-task learning model to model and predict target attributes. Each subtask has an independent subtask loss function, the model assigns a loss function weight to each subtask, and the weight is initialized randomly and modified along with the network training process, so that the model is converged finally.
As shown in fig. 1, the present invention provides a material property prediction method, which includes the following steps:
step 101, obtaining material data of a molded material sample to form a sample data set; the material data includes performance parameters and design parameters. The design parameters include, but are not limited to, material composition, and the material performance parameters include, but are not limited to, material yield strength, material modulus of elasticity, material poisson's ratio, material density, material coefficient of expansion, and the like.
102, performing element embedding processing on the component parameters of the design parameters by using an element embedding model to obtain component characteristics; and performing cross processing on the non-component parameters except the component parameters in the design parameters by using a cross processing network to obtain non-component characteristics and obtain a sample characteristic data set.
Element embedding processing: the input is element feature vector, and the output is spliced element embedded expression vector. Setting the length of the element embedding expression vector to be N, randomly initializing the embedding expression vector of each element, multiplying the content c of the element in the original data by the embedding vector to obtain the result of the embedding expression of each element, and splicing the embedding expressions of all the elements together to finish the element embedding processing.
Characteristic cross processing: there are two cases where if the cross-processing layer is the first layer, the input is the original feature x0(ii) a If not the first layer, then the input feature is the original feature x0Plus the output characteristic x of the previous layer. After randomly initializing the weighting matrix w in the layer, multiplying the weighting matrix w with the output x of the previous layer, and then multiplying the result with the original input x0Multiplying, performing feature crossing operation, and finally returning the sum of the crossed features and the bias matrix b and the output of the previous layer. The specific calculation formula is as follows: x is the number ofi＝xi-1Wi-1x0+bi-1+xi-1。
103, constructing an attention network of each performance index of the material to be predicted; designing attention networks of all performance indexes in the parameters to form a multi-head attention network; and carrying out weighted summation on the loss function of the attention network of each performance index by utilizing a weight matrix array to obtain the loss function of the multi-head attention network.
Attention mechanism processing: attention mechanism processing is carried out on input features, and for the same input, the input features are expanded into three matrixes: a Query matrix, a Key matrix, and a Value matrix. Firstly, calculating the dot multiplication between Query and Key, and dividing the dot multiplication result by the square root of the matrix dimension in order to prevent the calculation error caused by overlarge result
Figure BDA0002536926100000101
The result is then normalized to a range of 0,1 using a Softmax operation]And multiplying the probability Value by the Value matrix to obtain the final attention Value output. For a certain sample, after the attention value outputs corresponding to all other samples are calculated, the attention value outputs are summed, and the output of the sample under the attention mechanism can be obtained.
Given Query, Key and Value matrixes are operated, each operation unit is called a head, and a plurality of operation units can form multi-head attention. And transforming the input Key, Value and Query matrixes through a linear layer, and splitting the input Key, Value and Query matrixes into k sub-matrixes with the same size. Each submatrix corresponds to an attention mechanism (arithmetic unit). After multi-head design, the model can more comprehensively capture the association of different levels of data in a sample.
Multi-head attention networks using multitask combining: and constructing a multi-task learning network according to the material performance to be predicted. The final model is a multi-path model, and each path represents a task for predicting the performance of the material;
combine inter-subtask loss functions: the loss function for each subtask is computed independently and combined together by the combining operation in this step to form the overall loss function for the final model. In the combining operation, a weight matrix array is first randomly initialized according to the number of tasks, and each element in the array represents the ratio of the subtask loss function value to the total loss function value. This weight matrix is automatically modified as the network is trained, eventually enabling the model to converge.
Network training and parameter tuning: the original data set is divided into a training set and a testing set, the model is trained by using the data on the training set, and the model is subjected to effect verification by using the data of the testing set. The evaluation index is an R square value. And when the square value of the test set R of the subtasks meets the expected requirement, finishing the network training. At this point, given a set of inputs, the output of the model is the prediction. When the model training effect is not good, the parameters can be tried to be adjusted, and the model effects corresponding to different parameter combinations are different.
And 104, training and testing an embedding vector of the element embedding model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network in a dynamic weight mode based on the sample characteristic data set until a loss function of the multi-head attention network converges and the prediction accuracy reaches an accuracy threshold, and obtaining the trained element embedding model, the trained cross processing network and the trained attention network of each performance index.
Step 104 specifically includes: inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention network, and using a formula loss＝σ2L (W) calculating a loss function value after each subtask is weighted; where σ represents the weight of the subtask, L (W) represents the loss function of the subtask, and loss represents the weighted loss function.
And summing the weighted loss function values of all the subtasks to obtain the loss function value of the multi-head attention network.
And judging whether the loss function value of the multi-head attention network is converged to obtain a second judgment result.
If the second judgment result shows that the first judgment result shows that the second judgment result does not show that the second judgment result shows that the second judgment result does not show the second judgment result, the embedding vector of the element embedding model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network are updated, the weight of each subtask in the weight matrix array is corrected by using a gradient updating method, the step of returning is that the sample feature data in the sample feature data set are input into each attention network in the multi-head attention network, and the formula loss sigma is used for inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention network2L (w) calculate the weighted loss function value for each subtask ". The gradient updating method comprises the following steps: in each training process, a gradient formula of the loss function to the network parameters is obtained, and the obtained loss function value is substituted into the gradient formula, so that the update value of each parameter is obtained. Specifically, for a certain parameter, assuming that the gradient of the loss function is ρ and the learning rate is γ, the update formula of the weight W for each subtask is: w- γ ρ.
If the second judgment result shows that the first judgment result shows that the second embedding vector of the updated element embedding model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network are output.
Specifically, as shown in fig. 2, a weight array with a length of n is randomly initialized according to the number n of tasks, and each element represents the proportion of the loss function of the corresponding task to the total loss function;
and calculating the loss function value after each subtask is weighted. The calculation formula is as follows, where σ represents the initialized weight, L (W) represents the original loss function of the subtask, and loss represents the new loss function:
loss＝σ2L(W)
the weighted loss function values for all subtasks are summed as the final output. The calculation formula is as follows:
Figure BDA0002536926100000121
in the training process, the weight value of each subtask is corrected along with the gradient of back propagation, and finally the network is converged.
And 105, performing feature extraction on the component parameters of the design parameters of the materials to be predicted by using the trained element embedded model, and performing feature extraction on the non-component parameters of the design parameters of the materials to be predicted by using the trained cross processing network to obtain input feature data.
The feature extraction of the component parameters of the design parameters of the material to be predicted by using the trained element embedding model specifically comprises the following steps: multiplying the content of each component of the component parameters of the design parameters needing to be predicted by the embedding vector to obtain an embedding expression vector of each component; and splicing the embedded expression vectors of each component to obtain the component characteristics of the component parameters of the design parameters of the predicted material.
The method for extracting the characteristics of the non-component parameters of the design parameters of the material to be predicted by using the trained cross processing network specifically comprises the following steps: using the formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing on the i-layer network; wherein x is0Representing the original input, xi-1And xiRespectively showing the cross processing results of the i-1 layer network and the i-layer network, wherein the cross processing result of the 0-layer network is a non-component parameter; wi-1Weight of the i-1 th layer network, bi-1Indicating the bias of the i-1 layer network; judging whether the numerical value of i is smaller than the layer number of the cross-processing network or not to obtain a first judgment result; if the first judgment result shows yes, increasing the value of i1, return to step "Using formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing of an i-layer network; if the first judgment result shows no, outputting the cross processing result of the ith layer
And 106, respectively inputting the input characteristic data into the trained attention network corresponding to the performance index of the material to be predicted, and performing attention mechanism processing to obtain a prediction result of the performance index of each material to be predicted.
As shown in step 106 of fig. 3, the inputting the input feature data into the trained attention network corresponding to the performance index of the material to be predicted, and performing attention mechanism processing to obtain the prediction result of the performance index of each material to be predicted specifically includes: inputting the input characteristic data into a trained attention network of the performance index of the material to be predicted, and generating a Query matrix, a Key matrix and a Value matrix of the input characteristic data; multiplying the Query matrix and the Key matrix to obtain a product matrix; dividing the product matrix by the square root of the input feature data dimension to obtain a scaled product matrix; performing Softmax operation on the scaled product matrix to obtain a Softmax operation result matrix; and multiplying the Softmax operation result matrix by the Value matrix to obtain an attention mechanism processing result which is used as a prediction result of the performance index of the material needing to be predicted. The formula is as follows:
Figure BDA0002536926100000131
q, K, V represents a Query matrix, a Key matrix and a Value matrix respectively, and d represents the dimension of input characteristic data.
A material property prediction system, the prediction system comprising:
the sample data set acquisition module is used for acquiring material data of the molded material sample to form a sample data set; the material data includes performance parameters and design parameters.
The sample data set feature extraction module is used for carrying out element embedding processing on the component parameters of the design parameters by using an element embedding model to obtain component features; and performing cross processing on the non-component parameters except the component parameters in the design parameters by using a cross processing network to obtain non-component characteristics and obtain a sample characteristic data set.
The multi-head attention network construction module is used for constructing an attention network of each performance index of the material to be predicted; designing attention networks of all performance indexes in the parameters to form a multi-head attention network; and carrying out weighted summation on the loss function of the attention network of each performance index by utilizing a weight matrix array to obtain the loss function of the multi-head attention network.
And the dynamic weight training module is used for training and testing the embedded vector of the element embedded model, the weight and the offset of the cross processing network and each attention network in the multi-head attention network in a dynamic weight mode based on the sample characteristic data set until the loss function of the multi-head attention network converges and the predicted accuracy reaches an accuracy threshold, so as to obtain the trained element embedded model, the trained cross processing network and the trained attention network of each performance index.
The dynamic weight training module specifically comprises: a loss function value weighting submodule for inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention network, and using the formula loss as sigma2L (W) calculating the loss function value after each subtask is weighted; wherein, σ represents the weight of the subtask, L (W) represents the loss function of the subtask, and loss represents the loss function after weighting; the loss function value summation sub-module is used for summing the weighted loss function values of all the subtasks to serve as the loss function value of the multi-head attention network; the second judgment submodule is used for judging whether the loss function value of the multi-head attention network is converged or not to obtain a second judgment result; a parameter output submodule, configured to update the embedding vector of the element embedding model, the weight and bias of the cross processing network, and each attention network in the multi-head attention network if the second determination result indicates no, and correct the weight of each subtask in the weight matrix array by using a gradient update methodReturning to the step of inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention network, and using the formula loss as sigma2L (w) calculating a loss function value after weighting each subtask; and the training result output sub-module is used for outputting the embedding vector of the updated element embedding model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network if the second judgment result indicates yes, so as to obtain the trained element embedding model, the trained cross processing network and the trained attention network of each performance index.
And the feature extraction module is used for extracting the features of the component parameters of the design parameters of the materials to be predicted by using the trained element embedded model and extracting the features of the non-component parameters of the design parameters of the materials to be predicted by using the trained cross processing network to obtain input feature data.
The feature extraction module for the design parameters of the material to be predicted specifically comprises: the embedded expression vector acquisition submodule is used for multiplying the content of each component of the component parameters of the design parameters needing to be predicted by the embedded vector to obtain the embedded expression vector of each component; and the splicing submodule is used for splicing the embedded expression vectors of each component to obtain the component characteristics of the component parameters of the predicted design parameters of the material. A cross-processing submodule for utilizing the formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing on the i-layer network; wherein x is0Representing the original input, xi-1And xiRespectively showing the cross processing results of the i-1 layer network and the i-layer network, wherein the cross processing result of the 0-layer network is a non-component parameter; wi-1Weight of the i-1 th network, bi-1Indicating the bias of the i-1 layer network; the layer number judgment submodule is used for judging whether the numerical value of the i is smaller than the layer number of the network subjected to cross processing or not to obtain a first judgment result; a returning submodule for increasing the value of i by 1 if the first judgment result shows yes, and returning to the step of' utilizing publicFormula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing of an i-layer network; and the result output submodule is used for outputting the cross processing result of the ith layer if the first judgment result shows that the first judgment result is not the first judgment result.
And the performance prediction module is used for respectively inputting the input characteristic data into the trained attention network corresponding to the performance index of the material to be predicted, and performing attention mechanism processing to obtain the prediction result of the performance index of each material to be predicted.
The performance prediction module specifically includes: the matrix generation submodule is used for inputting the input characteristic data into the trained attention network of the performance index of the material to be predicted and generating a Query matrix, a Key matrix and a Value matrix of the input characteristic data; the product matrix acquisition submodule is used for multiplying the Query matrix and the Key matrix to obtain a product matrix; a scaling submodule, configured to divide the product matrix by a square root of the input feature data dimension to obtain a scaled product matrix; the Softmax operation submodule is used for carrying out Softmax operation on the zoomed product matrix to obtain a Softmax operation result matrix; and the prediction result output submodule is used for multiplying the Softmax operation result matrix and the Value matrix to obtain an attention mechanism processing result which is used as a prediction result of the performance index of the material needing to be predicted.
To verify the effect of the present invention, the method of the present invention was compared with the method of the neural network.
As shown in fig. 4, the material data deep learning method uses a neural network to model material data, and the process includes the following steps: step one, material data cleaning: clearing data samples suspected of being error abnormal, and clearing outlier samples deviating from the overall distribution of the samples; performing correlation analysis such as chi-square test, correlation coefficient calculation, covariance analysis and the like on the data, and removing redundant or low-correlation useless characteristics through data cleaning; step two, material data transformation: according to the subsequent modeling requirements, carrying out transformation such as smoothing, normalization, standardization, discretization and the like on the data; step three, selecting a base line: fitting and modeling material data by using a plurality of machine learning models such as random forests, support vector regression and the like, and selecting a machine learning model with the best performance as a baseline model by using indexes such as R-square value, mean square error, classification accuracy and the like so as to compare the effects of subsequent deep learning models; step four, constructing a network: building a neural network according to the network design; step five, training a network: and selecting the model in the step four, performing multiple rounds of training on the model, and adjusting the hyper-parameters to make the model converge.
The corresponding steps of the method of the invention are step one, inputting cleaned material data: the material contains 49 elements and 95-dimensional atomic features;
step two, element embedding processing and feature cross processing: for the element features, the length of an embedding vector is set to be 3, each element feature is multiplied by the corresponding embedding vector, and finally the element features are spliced together to obtain a 147-dimensional feature; setting 3 layers of cross processing layers corresponding to the atomic features, and performing cross processing operation on the atomic features to finally obtain the crossed 95-dimensional atomic features; and step three, referring to fig. 3, performing attention mechanism operation. At the moment, the Query matrix, the Key matrix and the Value matrix are all initialized to be the same input; step four, referring to fig. 2, performing loss function weighting processing to obtain a model overall loss function; step five, setting the number of network iteration rounds as 10000 rounds, setting an optimizer as an Adam optimizer, setting the learning rate as 0.0001, and starting to train the network; and step six, obtaining model output after the network training is finished.
The invention has the advantages by comparison: the method can use any neural network to fit the material data, and can model and predict any characteristic (attribute) in the given data by virtue of the strong fitting capability of deep learning. Firstly, the element embedding expression and feature crossing method expands the original data to a high-dimensional space, greatly improves the expression capability of the original data and is convenient for network modeling; secondly, through the attention network, the network can automatically capture samples with similar characteristics and give higher weight to the samples, so that the data has hierarchy, and the network can learn the characteristics of the samples strongly related to the target value more conveniently; finally, the multi-task learning technology enables the network not to be limited to one prediction task, and through combination of multiple prediction tasks, the invention can enable the subtasks to mutually promote and learn together.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The principle and the implementation manner of the present invention are explained by applying specific examples, the above description of the embodiments is only used to help understanding the method of the present invention and the core idea thereof, the described embodiments are only a part of the embodiments of the present invention, not all embodiments, and all other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the protection scope of the present invention.

Claims (8)

1. A material performance prediction method is characterized by comprising the following steps:
acquiring material data of a molded material sample to form a sample data set; the material data includes performance parameters and design parameters;
performing element embedding processing on the component parameters of the design parameters by using an element embedding model to obtain component characteristics; performing cross processing on non-component parameters except for the component parameters in the design parameters by using a cross processing network to obtain non-component characteristics and obtain a sample characteristic data set;
constructing an attention network for each performance index of the material to be predicted; the attention networks of all the performance indexes in the performance parameters form a multi-head attention network; weighting and summing the loss function of the attention network of each performance index by using a weight matrix group to obtain the loss function of the multi-head attention network;
training and testing an embedding vector of the element embedding model, the weight and the bias of a cross processing network and each attention network in a multi-head attention network in a dynamic weight mode based on the sample characteristic data set until a loss function of the multi-head attention network converges and the predicted accuracy reaches an accuracy threshold, and obtaining the trained element embedding model, the trained cross processing network and the trained attention network of each performance index;
performing feature extraction on the component parameters of the design parameters of the material to be predicted by using the trained element embedded model, and performing feature extraction on the non-component parameters of the design parameters of the material to be predicted by using the trained cross processing network to obtain input feature data;
inputting the input characteristic data into trained attention networks corresponding to the performance indexes of the materials to be predicted respectively, and performing attention mechanism processing to obtain the prediction result of the performance index of each material to be predicted;
the training and testing of the embedded vector of the element embedded model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network based on the sample feature data set by adopting a dynamic weight mode until the loss function of the multi-head attention network converges and the prediction accuracy reaches an accuracy threshold value to obtain the trained element embedded model, the trained cross processing network and the trained attention network of each performance index specifically comprises the following steps:
inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention network, and using the formula loss-sigma2L (W) calculating the loss function value after each subtask is weighted; wherein, σ represents the weight of the subtask, L (W) represents the loss function of the subtask, and loss represents the loss function after weighting;
summing the weighted loss function values of all the subtasks to serve as the loss function value of the multi-head attention network;
judging whether the loss function value of the multi-head attention network is converged to obtain a second judgment result;
if the second judgment result shows no, updating the embedding vector of the element embedding model, the weight and the bias of the cross processing network and the multi-head noteAnd each attention network in the attention networks, correcting the weight of each subtask in the weight matrix array by using a gradient updating method, returning to the step of inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention network, and using a formula loss-sigma2L (w) calculating a loss function value weighted by each subtask ";
and if the second judgment result shows that the first judgment result shows that the second judgment result shows that the first judgment result shows that the second embedding vector of the updated element embedding model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network are output.
2. The method for predicting material performance according to claim 1, wherein the feature extraction of the component parameters of the design parameters of the material to be predicted by using the trained element embedding model specifically comprises:
multiplying the content of each component of the component parameters of the design parameters needing to be predicted by the embedding vector to obtain an embedding expression vector of each component;
and splicing the embedded expression vectors of each component to obtain the component characteristics of the component parameters of the design parameters of the predicted material.
3. The method for predicting material performance according to claim 1, wherein the feature extraction of the non-component parameters of the design parameters of the material to be predicted by using the trained cross processing network specifically comprises:
using the formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing on the i-layer network; wherein x is0Representing the original input, xi-1And xiRespectively showing the cross processing results of the i-1 layer network and the i-layer network, wherein the cross processing result of the 0-layer network is a non-component parameter; wi-1Weight of the i-1 th network, bi-1Indicating i-1 layer networksBias of (3);
judging whether the numerical value of i is smaller than the layer number of the cross-processing network or not to obtain a first judgment result;
if the first judgment result shows yes, increasing the value of i by 1, and returning to the step of utilizing the formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing of an i-layer network;
and if the first judgment result shows no, outputting the cross processing result of the ith layer.
4. The method for predicting material performance according to claim 1, wherein the step of inputting the input feature data into the trained attention network corresponding to the performance index of the material to be predicted respectively to perform attention mechanism processing to obtain the prediction result of the performance index of each material to be predicted specifically comprises:
inputting the input characteristic data into a trained attention network of the performance index of the material to be predicted, and generating a Query matrix, a Key matrix and a Value matrix of the input characteristic data;
multiplying a Query matrix and a Key matrix to obtain a product matrix;
dividing the product matrix by the square root of the input feature data dimension to obtain a scaled product matrix;
performing Softmax operation on the scaled product matrix to obtain a Softmax operation result matrix;
and multiplying the Softmax operation result matrix by the Value matrix to obtain an attention mechanism processing result which is used as a prediction result of the performance index of the material needing to be predicted.
5. A material property prediction system, the prediction system comprising:
the sample data set acquisition module is used for acquiring material data of the molded material sample to form a sample data set; the material data includes performance parameters and design parameters;
the sample data set feature extraction module is used for carrying out element embedding processing on the component parameters of the design parameters by using an element embedding model to obtain component features; performing cross processing on non-component parameters except for the component parameters in the design parameters by using a cross processing network to obtain non-component characteristics and obtain a sample characteristic data set;
the multi-head attention network construction module is used for constructing an attention network of each performance index of the material to be predicted; the attention networks of all the performance indexes in the performance parameters form a multi-head attention network; weighting and summing the loss function of the attention network of each performance index by using a weight matrix group to obtain the loss function of the multi-head attention network;
the dynamic weight training module is used for training and testing the embedded vector of the element embedded model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network in a dynamic weight mode based on the sample characteristic data set until the loss function of the multi-head attention network converges and the predicted accuracy reaches an accuracy threshold, so as to obtain the trained element embedded model, the trained cross processing network and the trained attention network of each performance index;
the feature extraction module is used for extracting features of the component parameters of the design parameters of the materials to be predicted by using the trained element embedded model, and extracting features of the non-component parameters of the design parameters of the materials to be predicted by using the trained cross processing network to obtain input feature data;
the performance prediction module is used for respectively inputting the input characteristic data into the trained attention network corresponding to the performance index of the material to be predicted, and performing attention mechanism processing to obtain the prediction result of the performance index of each material to be predicted;
the dynamic weight training module specifically comprises:
a loss function value weighting submodule for inputting the sample feature data in the sample feature data set into each attention network in the multi-head attention networkUsing the formula loss ═ σ2L (W) calculating the loss function value after each subtask is weighted; wherein, σ represents the weight of the subtask, L (W) represents the loss function of the subtask, and loss represents the loss function after weighting;
the loss function value summation submodule is used for summing the weighted loss function values of all the subtasks to serve as the loss function value of the multi-head attention network;
the second judgment submodule is used for judging whether the loss function value of the multi-head attention network is converged or not to obtain a second judgment result;
a parameter output submodule, configured to update the embedding vector of the element embedding model, the weight and bias of the cross processing network, and each attention network in the multi-head attention network if the second determination result indicates no, modify the weight of each subtask in the weight matrix array by using a gradient update method, return to the step "input the sample feature data in the sample feature data set into each attention network in the multi-head attention network, and use the formula loss ═ σ ═ σ -2L (w) calculating a loss function value weighted by each subtask ";
and the training result output submodule is used for outputting the updated embedding vector of the element embedding model, the weight and the bias of the cross processing network and each attention network in the multi-head attention network if the second judgment result indicates yes, so as to obtain the trained element embedding model, the trained cross processing network and the trained attention network of each performance index.
6. The material property prediction system of claim 5, wherein the feature extraction module for the design parameters of the material to be predicted specifically comprises:
the embedded expression vector acquisition submodule is used for multiplying the content of each component of the component parameters of the design parameters to be predicted by the embedded vector to obtain an embedded expression vector of each component;
and the splicing submodule is used for splicing the embedded expression vectors of each component to obtain the component characteristics of the component parameters of the predicted design parameters of the material.
7. The material property prediction system of claim 5, wherein the feature extraction module for the design parameters of the material to be predicted specifically comprises:
a cross-processing submodule for utilizing the formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing on the i-layer network; wherein x is0Representing the original input, xi-1And xiRespectively representing the cross processing results of an i-1 th network and an i-th network, wherein the cross processing result of a 0 th network is a non-component parameter; wi-1Weight of the i-1 th network, bi-1Indicating the bias of the i-1 layer network;
the layer number judgment submodule is used for judging whether the numerical value of the i is smaller than the layer number of the network subjected to cross processing or not to obtain a first judgment result;
a returning submodule, configured to increase the value of i by 1 if the first determination result indicates yes, and return to the step "use formula xi＝xi-1Wi-1x0+bi-1+xi-1Performing cross processing of an i-layer network;
and the result output submodule is used for outputting the cross processing result of the ith layer if the first judgment result shows that the first judgment result is not the first judgment result.
8. The material property prediction system of claim 5, wherein the property prediction module specifically comprises:
the matrix generation submodule is used for inputting the input characteristic data into the trained attention network of the performance index of the material to be predicted and generating a Query matrix, a Key matrix and a Value matrix of the input characteristic data;
the product matrix acquisition submodule is used for multiplying the Query matrix and the Key matrix to obtain a product matrix;
a scaling submodule, configured to divide the product matrix by a square root of the input feature data dimension to obtain a scaled product matrix;
the Softmax operation submodule is used for carrying out Softmax operation on the zoomed product matrix to obtain a Softmax operation result matrix;
and the prediction result output submodule is used for multiplying the Softmax operation result matrix and the Value matrix to obtain an attention mechanism processing result which is used as a prediction result of the performance index of the material needing to be predicted.
CN202010535731.1A 2020-06-12 2020-06-12 Material performance prediction method and system Active CN111695260B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010535731.1A CN111695260B (en) 2020-06-12 2020-06-12 Material performance prediction method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010535731.1A CN111695260B (en) 2020-06-12 2020-06-12 Material performance prediction method and system

Publications (2)

Publication Number Publication Date
CN111695260A CN111695260A (en) 2020-09-22
CN111695260B true CN111695260B (en) 2022-06-21

Family

ID=72480636

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010535731.1A Active CN111695260B (en) 2020-06-12 2020-06-12 Material performance prediction method and system

Country Status (1)

Country Link
CN (1) CN111695260B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112214933B (en) * 2020-10-20 2024-04-19 集萃新材料研发有限公司 Fatigue performance prediction method based on machine learning
CN113362915B (en) * 2021-07-16 2022-08-02 上海大学 Material performance prediction method and system based on multi-modal learning

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109948165A (en) * 2019-04-24 2019-06-28 吉林大学 Fine granularity feeling polarities prediction technique based on mixing attention network
CN110489567A (en) * 2019-08-26 2019-11-22 重庆邮电大学 A kind of node information acquisition method and its device based on across a network Feature Mapping
CN111046907A (en) * 2019-11-02 2020-04-21 国网天津市电力公司 Semi-supervised convolutional network embedding method based on multi-head attention mechanism
CN111062775A (en) * 2019-12-03 2020-04-24 中山大学 Recommendation system recall method based on attention mechanism

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109948165A (en) * 2019-04-24 2019-06-28 吉林大学 Fine granularity feeling polarities prediction technique based on mixing attention network
CN110489567A (en) * 2019-08-26 2019-11-22 重庆邮电大学 A kind of node information acquisition method and its device based on across a network Feature Mapping
CN111046907A (en) * 2019-11-02 2020-04-21 国网天津市电力公司 Semi-supervised convolutional network embedding method based on multi-head attention mechanism
CN111062775A (en) * 2019-12-03 2020-04-24 中山大学 Recommendation system recall method based on attention mechanism

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"基于多头注意力机制和残差神经网络的肽谱匹配打分算法";闵鑫等;《计算机应用》;20200109;第1-8页 *

Also Published As

Publication number Publication date
CN111695260A (en) 2020-09-22

Similar Documents

Publication Publication Date Title
Zhang et al. Incorporating rich features into deep knowledge tracing
JP7312906B2 (en) Resource Constrained Neural Network Architecture Search
Liu et al. A rolling grey model optimized by particle swarm optimization in economic prediction
CN110807154A (en) Recommendation method and system based on hybrid deep learning model
Bayram et al. Comparison of multi layer perceptron (MLP) and radial basis function (RBF) for construction cost estimation: the case of Turkey
Srdjevic et al. Synthesis of individual best local priority vectors in AHP-group decision making
CN108985335B (en) Integrated learning prediction method for irradiation swelling of nuclear reactor cladding material
CN107463993B (en) Medium-and-long-term runoff forecasting method based on mutual information-kernel principal component analysis-Elman network
CN111695260B (en) Material performance prediction method and system
CN110766060B (en) Time series similarity calculation method, system and medium based on deep learning
CN113743675B (en) Construction method and system of cloud service QoS deep learning prediction model
CN109472346B (en) Emergency material demand prediction method considering fuzzy and missing of partial data
JP2014059754A (en) Information processing system, and information processing method
Abbas et al. Curve fitting predication with artificial neural networks: A comparative analysis
Lim et al. Inverse multi-objective robust evolutionary design optimization in the presence of uncertainty
CN110147284B (en) Supercomputer working load prediction method based on two-dimensional long-short term memory neural network
Bukhtoyarov et al. A comprehensive evolutionary approach for neural network ensembles automatic design
Taffese Case-based reasoning and neural networks for real estate valuation.
CN114781503A (en) Click rate estimation method based on depth feature fusion
Hart et al. An advanced cost estimation methodology for engineering systems
Rao et al. Fuzzy logic-based expert system to predict the results of finite element analysis
Martínez et al. Time series forecasting by generalized regression neural networks trained with multiple series
Lola et al. Improving the performance of ann-arima models for predicting water quality in the offshore area of kuala terengganu, terengganu, malaysia
Ureña et al. On incomplete fuzzy and multiplicative preference relations in multi-person decision making
EP2357603A1 (en) Evolutionary design optimization of real world structures with adaptive manipulation of nonlinear constraints

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant